VPU Codec

This chapter covers the RK182X video processing unit (VPU) hardware codec capabilities, multi-channel configuration, and decoder tuning.

Overall block diagram

Bitstream input
       ↓
┌─────────────────────────────┐
│       VPU Decoder Core       │
│  ┌─────┐ ┌─────┐ ┌─────┐   │
│  │Parse│→│Recon│→│Post │   │
│  │ BSP │ │ MC  │ │Proc │   │
│  └─────┘ └─────┘ └─────┘   │
└────────┬────────────────────┘
         │ YUV (NV12 / NV16)
         ↓
     RGA / NPU / Display

YUV input (NV12)
       ↓
┌───────────────────────────────┐
│       VPU Encoder Core        │
│  ┌──────┐ ┌───────┐ ┌──────┐  │
│  │ ME   │→│ TQ+   │→│ Rate │  │
│  │motion│→│entropy│→│ Ctrl │  │
│  │est.  │→│+quant │→│      │  │
│  └──────┘ └───────┘ └──────┘  │
└─────────┬─────────────────────┘
         │ H.264 / H.265 NALU
         ↓
     File / RTSP / RTP

The VPU is a hardware video codec engine on the Host SoC (e.g. RK3588), invoked through the MPP framework. The RK182X coprocessor itself has no VPU; it works through the Host VPU over PCIe.

1. VPU Hardware Specifications

Capability	Decode	Encode
H.265 / HEVC	Main / Main10 @ 4K60	Main @ 4K30
H.264 / AVC	HP / MP / BP @ 4K60	HP / MP / BP @ 4K30
VP9	Profile 0 / 2 @ 4K60	—
AV1	Main @ 4K60	—
MJPEG	@ 8K30	—
Max concurrent channels	4× 1080p30	2× 1080p30

2. Multi-Channel Configuration

Create an independent MPP Context per channel (dec0 / dec1 / ... / enc); the VPU hardware schedules the channels automatically.

#include <rockchip/rk_mpi.h>
#include <rockchip/rk_mpi_cmd.h>
#include <rockchip/rk_vdec_cfg.h>

int main(void) {
    MppCtx   dec0 = NULL;
    MppApi  *mpi0 = NULL;
    MPP_RET  ret;

    // create + get MppApi
    ret = mpp_create(&dec0, &mpi0);
    if (ret) return ret;

    // top-level mpp_init initializes the decoder
    ret = mpp_init(dec0, MPP_CTX_DEC, MPP_VIDEO_CodingAVC);
    if (ret) return ret;

    // decoders 2-4 initialize similarly (independent ctx per channel)
    MppCtx dec1, dec2, dec3;
    (void)dec1; (void)dec2; (void)dec3;

    // encoder
    MppCtx enc = NULL;
    MppApi *mpi_enc = NULL;
    mpp_create(&enc, &mpi_enc);
    mpp_init(enc, MPP_CTX_ENC, MPP_VIDEO_CodingAVC);

    return 0;
}

3. Compilation (Directly on the RK3588)

aarch64-linux-gnu-gcc vpu_decode.c -o vpu_decode \
    -I/usr/include/rockchip \
    -L/usr/lib/aarch64-linux-gnu \
    -lrockchip_mpp

Compiles successfully on the RK3588: librockchip_mpp is at /usr/lib/aarch64-linux-gnu/librockchip_mpp.so, headers at /usr/include/rockchip/.

4. Low-Latency Decoding Configuration

The two closest low-latency-related APIs:

API	Purpose
`MPP_DEC_SET_PRESENT_TIME_ORDER`	Output in input PTS order (replaces the non-existent DISABLE_REORDER)
`MPP_DEC_SET_DISABLE_DPB_CHECK`	Disable DPB continuity checking

#include <rockchip/rk_mpi.h>
#include <rockchip/rk_mpi_cmd.h>
#include <rockchip/rk_vdec_cfg.h>

int main(void) {
    MppCtx ctx = NULL;
    MppApi *mpi = NULL;
    mpp_create(&ctx, &mpi);
    mpp_init(ctx, MPP_CTX_DEC, MPP_VIDEO_CodingAVC);

    // output in input PTS order
    mpi->control(ctx, MPP_DEC_SET_PRESENT_TIME_ORDER, (MppParam)1);

    // disable DPB continuity checking
    mpi->control(ctx, MPP_DEC_SET_DISABLE_DPB_CHECK, (MppParam)1);

    // shrink the decode buffers: via the cfg key approach
    MppDecCfg dec_cfg = NULL;
    mpp_dec_cfg_init(&dec_cfg);
    mpp_dec_cfg_set_s32(dec_cfg, "decoder:num_buffers", 4);
    mpi->control(ctx, MPP_DEC_SET_CFG, dec_cfg);

    mpp_destroy(ctx);
    mpp_dec_cfg_deinit(dec_cfg);
    return 0;
}

5. Bitstream Error Handling

Interfaces available for error handling:

API	Purpose
`mpp_frame_get_errinfo(frame)`	Get per-frame error info (RK_U32 bit field)
`MPP_DEC_SET_DISABLE_ERROR`	Disable H.264 / H.265 error detection
`MPP_DEC_SET_DIS_ERR_CLR_MARK`	Error-handling-related cmd

#include <rockchip/rk_mpi.h>
#include <rockchip/mpp_frame.h>

/* inside the decode loop */
for (;;) {
    MppFrame frame = NULL;
    ret = mpi->decode_get_frame(ctx, &frame);
    if (ret != MPP_OK || !frame) break;

    // check frame errors
    RK_U32 err = mpp_frame_get_errinfo(frame);
    if (err) {
        printf("WARNING: frame error info = 0x%x\n", err);
        mpp_frame_deinit(&frame);
        continue;
    }

    // normal processing
    mpp_frame_deinit(&frame);
}

6. MPP Decoder API Verification

MPP decoder control commands:

grep -n "MPP_DEC_SET_PRESENT_TIME_ORDER\|MPP_DEC_SET_DISABLE_DPB_CHECK\|MPP_DEC_SET_CFG\|MPP_DEC_SET_DISABLE_ERROR\|MPP_DEC_SET_DIS_ERR_CLR_MARK" /usr/include/rockchip/rk_mpi_cmd.h

Output:

100:    MPP_DEC_SET_PRESENT_TIME_ORDER,     /* use input time order for output */
107:    MPP_DEC_SET_DISABLE_ERROR,          /* When set it will disable sw/hw error (H.264 / H.265) */
115:    MPP_DEC_SET_DISABLE_DPB_CHECK,      /* disable dpb discontinuous check */
117:    MPP_DEC_SET_DIS_ERR_CLR_MARK,
124:    MPP_DEC_SET_CFG,                    /* set MppDecCfg structure */

MPP decoder control command verification:
MPP_DEC_SET_PRESENT_TIME_ORDER (rk_mpi_cmd.h:100): output in input PTS order
MPP_DEC_SET_DISABLE_ERROR (rk_mpi_cmd.h:107): disable sw/hw error detection
MPP_DEC_SET_DISABLE_DPB_CHECK (rk_mpi_cmd.h:115): disable DPB continuity checking
MPP_DEC_SET_DIS_ERR_CLR_MARK (rk_mpi_cmd.h:117): error clear marking
MPP_DEC_SET_CFG (rk_mpi_cmd.h:124): set the MppDecCfg structure

Full MPP decoder command list (partial):

sed -n '95,125p' /usr/include/rockchip/rk_mpi_cmd.h

Output:

    MPP_DEC_SET_FRAME_INFO,             /* vpu api legacy control for buffer slot dimension init */
    MPP_DEC_SET_EXT_BUF_GROUP,          /* IMPORTANT: set external buffer group to mpp decoder */
    MPP_DEC_SET_INFO_CHANGE_READY,
    MPP_DEC_SET_PRESENT_TIME_ORDER,     /* use input time order for output */
    MPP_DEC_SET_PARSER_SPLIT_MODE,      /* Need to setup before init */
    MPP_DEC_SET_PARSER_FAST_MODE,       /* Need to setup before init */
    MPP_DEC_GET_STREAM_COUNT,
    MPP_DEC_GET_VPUMEM_USED_COUNT,
    MPP_DEC_SET_VC1_EXTRA_DATA,
    MPP_DEC_SET_OUTPUT_FORMAT,
    MPP_DEC_SET_DISABLE_ERROR,          /* When set it will disable sw/hw error (H.264 / H.265) */
    MPP_DEC_SET_IMMEDIATE_OUT,
    MPP_DEC_SET_ENABLE_DEINTERLACE,     /* MPP enable deinterlace by default. Vpuapi can disable it */
    MPP_DEC_SET_ENABLE_FAST_PLAY,       /* enable idr output immediately */
    MPP_DEC_SET_DISABLE_THREAD,         /* MPP no thread mode and use external thread to decode */
    MPP_DEC_SET_MAX_USE_BUFFER_SIZE,
    MPP_DEC_SET_ENABLE_MVC,             /* enable MVC decoding*/
    MPP_DEC_GET_THUMBNAIL_FRAME_INFO,   /* update thumbnail frame info to user, for MPP_FRAME_THUMBNAIL_ONLY mode */
    MPP_DEC_SET_DISABLE_DPB_CHECK,      /* disable dpb discontinuous check */
    MPP_DEC_SET_CODEC_MODE,             /* select codec mode */
    MPP_DEC_SET_DIS_ERR_CLR_MARK,

The mpp_dec_cfg_* API function declarations:

grep -n "mpp_dec_cfg_init\|mpp_dec_cfg_deinit\|mpp_dec_cfg_set_s32" /usr/include/rockchip/rk_vdec_cfg.h

Output:

29:MPP_RET mpp_dec_cfg_init(MppDecCfg *cfg);
30:MPP_RET mpp_dec_cfg_deinit(MppDecCfg cfg);
32:MPP_RET mpp_dec_cfg_set_s32(MppDecCfg cfg, const char *name, RK_S32 val);

Full API list in rk_vdec_cfg.h:

sed -n '27,48p' /usr/include/rockchip/rk_vdec_cfg.h

Output:

MPP_RET mpp_dec_cfg_init(MppDecCfg *cfg);
MPP_RET mpp_dec_cfg_deinit(MppDecCfg cfg);

MPP_RET mpp_dec_cfg_set_s32(MppDecCfg cfg, const char *name, RK_S32 val);
MPP_RET mpp_dec_cfg_set_u32(MppDecCfg cfg, const char *name, RK_U32 val);
MPP_RET mpp_dec_cfg_set_s64(MppDecCfg cfg, const char *name, RK_S64 val);
MPP_RET mpp_dec_cfg_set_u64(MppDecCfg cfg, const char *name, RK_U64 val);
MPP_RET mpp_dec_cfg_set_ptr(MppDecCfg cfg, const char *name, void *val);

MPP_RET mpp_dec_cfg_get_s32(MppDecCfg cfg, const char *name, RK_S32 *val);
MPP_RET mpp_dec_cfg_get_u32(MppDecCfg cfg, const char *name, RK_U32 *val);
MPP_RET mpp_dec_cfg_get_s64(MppDecCfg cfg, const char *name, RK_S64 *val);
MPP_RET mpp_dec_cfg_get_u64(MppDecCfg cfg, const char *name, RK_U64 *val);
MPP_RET mpp_dec_cfg_get_ptr(MppDecCfg cfg, const char *name, void **val);

void mpp_dec_cfg_show(void);

The mpp_frame_get_errinfo function:

grep -n "mpp_frame_get_errinfo" /usr/include/rockchip/mpp_frame.h

Output:

420:RK_U32  mpp_frame_get_errinfo(const MppFrame frame);

mpp_frame_get_errinfo context:

sed -n '417,425p' /usr/include/rockchip/mpp_frame.h

Output:

RK_S64  mpp_frame_get_dts(const MppFrame frame);
void    mpp_frame_set_dts(MppFrame frame, RK_S64 dts);
RK_U32  mpp_frame_get_errinfo(const MppFrame frame);
void    mpp_frame_set_errinfo(MppFrame frame, RK_U32 errinfo);
size_t  mpp_frame_get_buf_size(const MppFrame frame);
void    mpp_frame_set_buf_size(MppFrame frame, size_t buf_size);
void    mpp_frame_set_thumbnail_en(MppFrame frame, RK_U32 thumbnail_en);
RK_U32  mpp_frame_get_thumbnail_en(const MppFrame frame);

mpp_frame_get_errinfo verified (mpp_frame.h:420): returns an RK_U32 error-info bit field.

7. H.264 Encoding Implementation

7.1 Initialization and Configuration

#include <stdio.h>
#include <rockchip/mpp_buffer.h>
#include <rockchip/mpp_frame.h>
#include <rockchip/mpp_packet.h>
#include <rockchip/rk_mpi.h>
#include <rockchip/rk_mpi_cmd.h>   /* MPP_ENC_SET_CFG */
#include <rockchip/rk_venc_cfg.h>  /* mpp_enc_cfg_* */
#include <rockchip/rk_venc_rc.h>   /* MPP_ENC_RC_MODE_* */

int main(void) {
    MPP_RET  ret;
    MppCtx   ctx = NULL;
    MppApi  *mpi = NULL;

    ret = mpp_create(&ctx, &mpi);
    if (ret) return ret;
    ret = mpp_init(ctx, MPP_CTX_ENC, MPP_VIDEO_CodingAVC);
    if (ret) return ret;

    MppEncCfg cfg = NULL;
    mpp_enc_cfg_init(&cfg);
    mpp_enc_cfg_set_s32(cfg, "prep:width",      1920);
    mpp_enc_cfg_set_s32(cfg, "prep:height",     1080);
    mpp_enc_cfg_set_s32(cfg, "prep:format",     MPP_FMT_YUV420SP);
    mpp_enc_cfg_set_s32(cfg, "rc:mode",         MPP_ENC_RC_MODE_CBR);
    mpp_enc_cfg_set_s32(cfg, "rc:bps_target",   4000000);
    mpp_enc_cfg_set_s32(cfg, "rc:fps_in",       30);

    // the actual control command is MPP_ENC_SET_CFG (rk_mpi_cmd.h)
    ret = mpi->control(ctx, MPP_ENC_SET_CFG, cfg);
    if (ret) return ret;

    mpp_destroy(ctx);
    mpp_enc_cfg_deinit(cfg);
    return MPP_OK;
}

7.2 Encoding Loop

MppFrame  frame  = NULL;
MppPacket packet = NULL;

while (has_input) {
    // put_frame / get_packet are MppApi members, not top-level functions
    ret = mpi->encode_put_frame(ctx, frame);
    if (ret) break;

    ret = mpi->encode_get_packet(ctx, &packet);
    if (ret == MPP_OK && packet) {
        write_output(mpp_packet_get_data(packet),
                     mpp_packet_get_length(packet));
        mpp_packet_deinit(&packet);
        packet = NULL;
    }
}

mpp_frame_deinit(&frame);

7.3 Multi-Channel Performance Notes

Total bandwidth is bounded by DDR bandwidth
Recommended: per-channel bitrate × channel count ≤ available DDR bandwidth × 0.7
Mixing encode formats (e.g. 2× H.264 + 2× H.265) increases scheduling overhead

8. MPP Encoder API Verification

MPP encode control API definitions:

grep -E "MPP_ENC_SET_CFG|MPP_ENC_RC_MODE" /usr/include/rockchip/rk_mpi_cmd.h | head -5

Output:

    MPP_ENC_SET_CFG,                    /* set MppEncCfg structure */
    MPP_ENC_SET_PREP_CFG,               /* deprecated set MppEncPrepCfg structure, use MPP_ENC_SET_CFG instead */
    MPP_ENC_SET_RC_CFG,                 /* deprecated set MppEncRcCfg structure, use MPP_ENC_SET_CFG instead */
    MPP_ENC_SET_CODEC_CFG,              /* deprecated set MppEncCodecCfg structure, use MPP_ENC_SET_CFG instead */

MPP encode rate-control mode constants:

grep -E "MPP_ENC_RC_MODE" /usr/include/rockchip/rk_venc_rc.h | head -10

Output:

    MPP_ENC_RC_MODE_VBR,
    MPP_ENC_RC_MODE_CBR,
    MPP_ENC_RC_MODE_FIXQP,
    MPP_ENC_RC_MODE_AVBR,
    MPP_ENC_RC_MODE_SMTRC,
    MPP_ENC_RC_MODE_SE,
    MPP_ENC_RC_MODE_BUTT

MPP video coding format constants:

grep -E "MPP_VIDEO_CodingAVC|MPP_VIDEO_CodingHEVC|MPP_VIDEO_CodingVP9|MPP_VIDEO_CodingJPEG" /usr/include/rockchip/rk_type.h | head -10

Output:

    MPP_VIDEO_CodingAVC,                /**< H.264/AVC */
    MPP_VIDEO_CodingVP9,                /**< VP9 */
    MPP_VIDEO_CodingHEVC,               /**< H.265/HEVC */

VPU device nodes:

ls /dev/mpp_service /dev/vpu_service 2>/dev/null

Output:

ls: 无法访问 '/dev/vpu_service': 没有那个文件或目录
/dev/mpp_service

Only the mpp_service node exists; there is no vpu_service node.

9. FAQ

Symptom	Cause	Fix
`mpp_create failed`	Library path / permissions	`ls -l /dev/mpp_service`
`mpp_init failed`	Coding type unsupported	Confirm `MPP_VIDEO_CodingAVC/HEVC/VP9/AV1`
Multi-channel decode unstable	Insufficient DDR bandwidth	Fewer channels / lower bitrate
`MPP_DEC_SET_DISABLE_REORDER` doesn't exist	Constant deprecated	Use `MPP_DEC_SET_PRESENT_TIME_ORDER` instead
`mpp_frame_get_errinfo` returns non-zero	Bitstream errors	Set `MPP_DEC_SET_DISABLE_ERROR` to tolerate
High encode latency	B-frames / large GOP	`rc:gop=0` + CBR

10. Next Steps

MPP Multimedia Framework — the multimedia framework
RGA in Detail — preprocessing hardware acceleration
NPU Overview — combining AI inference with video analytics