09 Intelligent Video Engine Application

This chapter describes the GK7206 Intelligent Video Engine (IVE) application example — sample_ive. The application demonstrates various IVE image-processing algorithms, including Sobel edge detection, Canny edge detection, GMM background modeling, Motion detect, Od (occlusion detection), St Lk (optical flow), and more.

The application source code is located in the SDK directory sample/ive/, and serves as a reference template for developing intelligent video analysis applications.

1 Application Overview

1.1 Features

Multiple algorithms: Sobel, Edge, Gmm2, MemoryTest, Motion detect, Od, St Lk, dma
Two data-flow modes: FILE→FILE (requires input images) and VI→VENC (requires a sensor)
Flexible input: supports reading input from image files or a video stream

1.2 Technical Parameters

Parameter	Value
`index`	0~8 (see table below)
`complete`	Complete mode (1) or quick mode (0)
`encode`	Encoded output (1) or raw output (0)
`vo`	VO output (1) or VENC output (0)

1.3 Test Case List

index	Name	Data flow	index	Name	Data flow
0	Sobel	FILE→FILE	5	Canny	FILE→FILE
1	Edge	FILE→FILE	6	Od (occlusion detection)	VI→VENC
2	Gmm2	FILE→FILE	7	St Lk (optical flow)	FILE→FILE
3	MemoryTest	FILE→FILE	8	dma	FILE→FILE
4	Motion detect	VI→VENC

1.4 Directory Structure

sample/ive/
├── Makefile              # Build script
└── sample_ive.c          # Main program

2 Build and Deployment

2.1 Prerequisites

Before building this application, make sure the following preparations are complete:

SDK environment ready: set up the cross-compilation toolchain and SDK configuration by following SDK Build
FILE-type cases require input images to be uploaded to /sd_card
VI-type cases require a sensor

2.2 Build the Application

# Enter the sample directory
cd <SDK_PATH>/sample

# Build the ive sample
make -C ive clean && make -C ive

2.3 Deploy to the Board

# Mount the SD card (recommended)
mkdir -p /sd_card
mount -t vfat /dev/mmcblk1p1 /sd_card

# Transfer to the development board via SCP
scp sample/ive/sample_ive root@<board IP>:/sd_card/

# For FILE-type cases, transfer the input image
scp input.jpg root@<board IP>:/sd_card/

2.4 Run the Application

# Add execute permission
chmod +x /sd_card/sample_ive

# Enter the SD card directory
cd /sd_card

# FILE-type case (Sobel edge detection)
./sample_ive 0

# VI-type case (motion detection)
./sample_ive 4 1 1 0

2.5 Expected Output

After a successful run (FILE-type case), the terminal prints output similar to the following:

sample_ive running
ive test index: 0
...
ive Sobel test success!
output file: /sd_card/output_sobel.yuv

3 Internal Execution Logic

3.1 Application Architecture

The application adopts the following architecture:

FILE type: input image → IVE processing → output image
VI type:  Sensor → VI → VPSS → IVE → VENC → RTSP streaming

3.2 IVE Initialization Flow

// Step 1: IVE initialization
xmedia_ive_init();

// Step 2: create an IVE task
xmedia_ive_handle_t ive_handle = xmedia_ive_create();

// Step 3: configure the IVE task
switch(index) {
    case 0:  // Sobel
        xmedia_ve_sobel(ive_handle, &src_img, &dst_img);
        break;
    case 5:  // Canny
        xmedia_ive_canny(ive_handle, &src_img, &dst_img);
        break;
    // ... other cases
}

// Step 4: destroy the IVE task
xmedia_ive_destroy(ive_handle);

// Step 5: IVE exit
xmedia_ive_exit();

4 Key Programming Points

4.1 Sobel Edge Detection

xmedia_ve_sobel_attr_t sobel_attr;

sobel_attr.edge_type = XMEDIA_IVE_SOBEL_EDGE_TYPE_BOTH;  // horizontal + vertical
sobel_attr.bit_width = 8;

xmedia_ve_sobel(ive_handle, &src_img, &dst_img, &sobel_attr);

4.2 Canny Edge Detection

xmedia_ive_canny_attr_t canny_attr;

canny_attr.low_threshold = 50;
canny_attr.high_threshold = 150;

xmedia_ive_canny(ive_handle, &src_img, &dst_img, &canny_attr);

4.3 GMM Background Modeling

xmedia_ive_gmm_attr_t gmm_attr;

gmm_attr.max_bg_num = 3;
gmm_attr.variance_threshold = 2500;

xmedia_ive_gmm(ive_handle, &src_img, &bg_model, &fg_mask, &gmm_attr);

4.4 Motion Detection

xmedia_ive_motion_detect_attr_t md_attr;

md_attr.width = 640;
md_attr.height = 480;
md_attr.threshold = 100;

xmedia_ive_motion_detect(ive_handle, &curr_img, &prev_img, &motion_result, &md_attr);

5 Troubleshooting

Problem	Possible Cause	Solution
FILE case cannot find input file	Image not uploaded to /sd_card	Upload the input image to /sd_card
VI case initialization fails	Sensor not connected	Check the sensor connection and configuration
IVE processing fails	Unsupported input image format	Check that the input image format is YUV420SP