Over the past few years, the RK3588 has become a frequent sight in embedded vision. It appears everywhere—from onboard processing for drones and edge AI boxes to a wide range of robotic platforms. The reason is straightforward: four Cortex-A76 cores plus four Cortex-A55 cores, a 6-TOPS NPU, and a high-spec video codec engine that supports 8K decoding, 4K encoding, and multiple concurrent streams. With its high level of domestic integration, the RK3588 is firmly in the top tier of ARM-based boards.

However, many vision developers discover something unexpected after connecting a camera. Pulling video from a USB camera is easy, but when they receive a 1080p H.265 RTSP stream, build a GStreamer pipeline, decode the stream, and then use OpenCV for image processing or send the frames to the NPU for inference, the actual frame rate often drops into the single digits—even though the pipeline itself is not complicated.

This raises a question: the RK3588 has the necessary hardware, but its capabilities are divided among different modules and must be used for the right jobs. While debugging a GX40 gimbal camera on SpireCV-Pro, we built a GStreamer pipeline for its H.265 main stream and initially achieved only 5 FPS. Fully using the RK3588's dedicated hardware was the key to restoring high-speed streaming.

01

MPP: A Complete Hardware Codec Framework

MPP (Media Process Platform) is Rockchip's general-purpose media processing framework for its own chips. It exposes low-level capabilities such as video encoding and decoding to upper-layer applications, connecting the application, user-space libraries, drivers, and video hardware. In the SpireCV-Pro system image used in this test, GStreamer calls Rockchip's hardware decoder through mppvideodec. Compared with a software decoder such as libde265dec, it hands the main decoding workload to the video hardware and significantly reduces CPU pressure. Plugin names and package layouts may vary between system images, so checking the actual environment first is the most reliable approach:

gst-inspect-1.0 mppvideodec

After confirming that the plugin is available, the key H.265 portion of an RTSP pipeline will typically include rtph265depay, h265parse, and mppvideodec. If hardware decoding followed by fakesink already reaches the full input frame rate, there is little value in continuing to focus on changing the decoder. The investigation should move downstream.

Hardware decoding can greatly reduce the CPU's decoding workload, but it does not automatically make the entire application pipeline zero-copy. Whether subsequent buffers remain in hardware-shareable memory depends on the plugins, caps negotiation, memory type, and how the downstream application retrieves frames.

02

RGA: A Dedicated 2D Acceleration Engine

Once the decoding problem is solved, another hidden bottleneck remains.

MPP outputs decoded frames in a YUV format such as NV12, while downstream components such as OpenCV and deep-learning inference frameworks usually require three-channel BGR or RGB images. A color-format conversion is therefore unavoidable.

GStreamer's general-purpose solution is videoconvert. It performs well on x86, but ARM is a different story. For a 1080p NV12-to-BGR conversion, roughly 6 MB of data per frame must undergo per-pixel color-space calculations and memory reordering on the CPU. In our test, adding this stage dropped the frame rate directly from 30 FPS to 5 FPS.

A common response is to lower the resolution, but the RK3588 has another hardware module that can solve the real problem.

RGA (Raster Graphics Acceleration) is a dedicated 2D image acceleration engine built into Rockchip SoCs. Working through DMA, it handles image scaling, cropping, rotation, and color-format conversion. Its supported inputs include YUV formats such as NV12, NV21, and YUV420, along with multiple RGB variants; its outputs include BGR, RGB, BGRx, and RGBx. Like MPP, RGA supports DMA-BUF sharing. It can connect directly to MPP's decoded output so the data path remains in hardware and the CPU barely touches the image data.

The corresponding GStreamer plugin is rgaconvert. Check for it with:

gst-inspect-1.0 | grep rgaconvert

If the system image does not include it, install the relevant Rockchip RGA GStreamer plugin package. Once available, replace videoconvert with rgaconvert in the GStreamer pipeline.

There is one easy-to-miss detail: rgaconvert requires the output caps to specify both width and height explicitly. Setting only format=BGR is not enough. RGA is a hardware module, so it needs exact dimensions when allocating internal buffers. Without them, caps negotiation may fall back to an abnormal default such as 1×1, causing RGA to report an Error dstRect. This is the most important usage difference from videoconvert.

The correct form is:

... mppvideodec ! rgaconvert ! video/x-raw,format=BGR,width=1920,height=1080 ! appsink

After adding the complete caps, the pipeline ran successfully and the terminal reported 31.6 FPS. The three configurations compare as follows:

The jump from 5 FPS to 31.6 FPS came simply from moving work off the CPU and onto the correct hardware. RGA can do far more than color conversion. If a 1080p frame must be resized to 640×384 before being passed downstream, set the target resolution directly in the caps after rgaconvert. RGA can complete scaling and conversion in a single stage with almost no additional processing delay, which is much more efficient than calling cv2.resize() separately in OpenCV. For the best memory-alignment efficiency, four-channel formats such as BGRx are also generally more stable on RGA than three-channel BGR.

03

A Transferable Development Habit

The roles of these two RK3588 modules are now clear. More valuable than the modules themselves, however, is the underlying mindset: discover the hardware and use it well.

Embedded development differs fundamentally from pure software development. Slow code does not always mean the algorithm is inadequate; it may mean that available hardware acceleration has not been used. The RK3588 is only one example. The same logic applies to Jetson, Raspberry Pi CM4, and even x86 platforms. Our work on the SpireCV-Pro BOX suggests three practical habits.

First, identify what is unique to a new board. For a vision task, do not rush into business logic. Run gst-inspect-1.0 and look for plugins that are not part of upstream GStreamer but instead carry vendor-specific names—such as mpp, rga, nvv4l2, or vaapi. These are the hardware acceleration entry points provided by the manufacturer. Reviewing the relevant manuals is another fast way to identify platform-specific capabilities.

Second, make performance comparisons a habit. When a bottleneck appears, do not change parameters based only on intuition. Run direct A/B tests: hardware decoding versus software decoding, RGA versus CPU processing, pipeline scaling versus OpenCV scaling. Each comparison eliminates one possible cause and keeps the diagnosis grounded in data.

Third, keep data moving between hardware modules and minimize trips through the CPU. If MPP-decoded frames go to RGA through DMA-BUF and then continue to the NPU, the full path remains in hardware and the CPU barely touches the data. Insert a CPU-based conversion in the middle, however, and the path becomes “hardware → CPU → hardware,” combining copy overhead with conversion overhead. The same principle applies beyond the RK3588: NVIDIA's nvvidconv and Intel's VAAPI follow similar logic.

04

Final Thoughts

Returning to the opening question: why does an RK3588 RTSP stream sometimes run at such a low frame rate? In most cases, the problem is not the chip's computing power. The available hardware resources are simply not being used in the right places.

Two modules play distinct roles in the RK3588 video pipeline: MPP handles encoding and decoding, while RGA handles image processing. Using both produces the optimal pipeline. Once this path is working, downstream tasks—object detection, visual tracking, or video transmission—will no longer be bottlenecked by basic video processing. This is easy for developers focused purely on vision algorithms to overlook, but it is extremely important. The following demonstration shows QR-code recognition after the streaming bottleneck was removed: recognition now runs in sync, and video acquisition is no longer the limiting factor.

If your RK3588 project is still struggling with RTSP streaming, this case may offer a useful direction. More importantly, the next time you receive a new hardware carrier board, ask three questions first: What dedicated hardware does it provide? Can I run a comparison test? Does the data path stay in the hardware pipeline? Often the chip is fast enough; its acceleration units are simply not being fully used. Mastering this development logic leads to broader, more systematic diagnosis and faster solutions.