RKNN Model Conversion

This chapter explains how to convert PyTorch / ONNX / TensorFlow models into .rknn models executable on the RK182X NPU.

Overall block diagram

Training-framework model (PyTorch / TensorFlow / ...)
       ↓
Export to intermediate format (.onnx / .pb / .tflite)
       ↓
RKNN Toolkit conversion
   ├─ Graph optimization
   ├─ Operator fusion
   ├─ Quantization calibration
   └─ Target-platform compilation
       ↓
.rknn model file
       ↓
Deploy to the RK182X board for execution

1. Installing RKNN Toolkit

Actual wheel filenames (from /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit/packages/):

ls -la /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit/packages/

Output:

总计 367580
drwxr-xr-x 2 linaro linaro      4096 2026年 1月28日 .
drwxr-xr-x 5 linaro linaro      4096 2026年 1月28日 ..
-rw-r--r-- 1 linaro linaro       226 2026年 1月28日 md5sum.txt
-rw-r--r-- 1 linaro linaro       421 2026年 1月28日 requirements_cp310-1.0.0.txt
-rw-r--r-- 1 linaro linaro       413 2026年 1月28日 requirements_cp312-1.0.0.txt
-rw-r--r-- 1 linaro linaro 188832423 2026年 1月28日 rknn3_toolkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
-rw-r--r-- 1 linaro linaro 187542742 2026年 1月28日 rknn3_toolkit-1.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

MD5:

cat /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit/packages/md5sum.txt

Output:

7c7c7366ad7ae483142dbd4ea9b6b1f0  rknn3_toolkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
54554b352e8b7fdf5efd09227871123b  rknn3_toolkit-1.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Architecture restriction: RKNN3 Toolkit only ships x86_64 wheels and can only run on a PC (x86_64). The development board (aarch64) can only install rknn3-toolkit-lite.

Installation (only on an x86_64 PC):

python3 -m venv ~/rknn-env
source ~/rknn-env/bin/activate

# The actual wheel is named rknn3_toolkit-1.0.0-...x86_64.whl (not rknn_toolkit2)
pip install /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit/packages/rknn3_toolkit-1.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

# Verify
python3 -c "from rknn.api import RKNN; print('RKNN Toolkit OK')"

Verified on the board (aarch64):

python3 -c "from rknn.api import RKNN; print('RKNN Toolkit OK')"

Output:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'rknn'

The x86_64 wheel cannot be pip-installed on the aarch64 board. The board can only install rknn3-toolkit-lite (inference only, no conversion).

2. PyTorch → ONNX → RKNN Complete Walkthrough

With a ResNet50 end-to-end example.

2.1 Export ONNX from PyTorch

import torch
import torchvision.models as models

model = models.resnet50(pretrained=False)
model.load_state_dict(torch.load('resnet50_best.pth'))
model.eval()

dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(
    model, dummy_input, 'resnet50.onnx',
    input_names=['input'],
    output_names=['output'],
    dynamic_axes={'input': {0: 'batch'}, 'output': {0: 'batch'}},
    opset_version=11,
)
print('ONNX exported: resnet50.onnx')

2.2 Conversion with RKNN Toolkit

from rknn.api import RKNN

rknn = RKNN(verbose=True)

# Configure conversion parameters
# Actual legal values (rknn.py:177-178):
#   target_platform = rv1103 / rv1103b / rv1106 / rv1106b / rk2118 /
#                     rk3562 / rk3566 / rk3568 / rk3576 / rk3588 / rk1820
# Actual legal values (rknn.py:173):
#   quantized_dtype = w8a8 / w4a16 (default w16a16)
rknn.config(
    mean_values=[[123.675, 116.28, 103.53]],
    std_values=[[58.395, 57.12, 57.375]],
    target_platform='rk1820',
    quantized_dtype='w8a8',
)

# Load ONNX
rknn.load_onnx(model='resnet50.onnx')

# Build (including quantization calibration)
# Actual signature (rknn.py:291): build(do_quantization, dataset, rknn_batch_size, auto_hybrid)
rknn.build(
    do_quantization=True,
    dataset='./calib_images.txt',
)

# Accuracy analysis (optional)
# Actual signature (rknn.py:417): accuracy_analysis(inputs, output_dir, core_mask, target, device_id)
rknn.accuracy_analysis(
    inputs=['./test_images/'],
    output_dir='./accuracy_report/',
)

# Export
rknn.export_rknn('resnet50.rknn')

# Release
rknn.release()
print('RKNN model exported: resnet50.rknn')

2.3 Preparing the Calibration Dataset

# calib_images.txt format: one image path per line
# ./calib/IMG_001.jpg
# ./calib/IMG_002.jpg
# ./calib/IMG_003.jpg

import os, random

image_dir = '/path/to/dataset/train/images'
all_images = [os.path.join(image_dir, f) for f in os.listdir(image_dir)
              if f.endswith(('.jpg', '.png'))]

random.seed(42)
calib = random.sample(all_images, 100)

with open('calib_images.txt', 'w') as f:
    for img in calib:
        f.write(img + '\n')
print(f'Calibration set: {len(calib)} images')

3. YOLOv8 Object Detection Conversion

Set custom_string='yolov8' to enable the YOLOv8 custom post-processing, and set mean / std to 0 / 255 (pixel-value normalization); the rest of the flow is identical to standard conversion.

from rknn.api import RKNN

rknn = RKNN(verbose=True)

rknn.config(
    mean_values=[[0, 0, 0]],
    std_values=[[255, 255, 255]],
    target_platform='rk1820',
    quantized_dtype='w8a8',
    custom_string='yolov8',
)

rknn.load_onnx(model='yolov8n.onnx')
rknn.build(do_quantization=True, dataset='./coco_calib.txt')
rknn.export_rknn('yolov8n_rk1820.rknn')
rknn.release()

4. Quantization Accuracy Tuning

Symptom	Direction to check
Top-1 drops >1%	Insufficient calibration coverage
Detection mAP collapses	Sensitive layers got quantized
Poor accuracy for a specific class	Imbalanced class samples

Available knobs:

• config(quantized_method='layer' / 'channel' / 'group32') — quantization granularity
• build(auto_hybrid=True) — automatic mixed precision

5. Key Parameters

Parameter	Description	Values
target_platform	Target platform	'rk1820'
mean_values	Input mean subtraction	ImageNet [255*0.485, 255*0.456, 255*0.406]; pre-normalized [0, 0, 0]
std_values	Input std division	ImageNet [255*0.229, 255*0.224, 255*0.225]
input_attrs	Input tensor type	{'input': {'dtype': 'uint8', 'layout': 'NHWC'}}
quantized_dtype	Quantization type	'w8a8' / 'w4a16' / 'w4a8'
do_quantization	Whether to do INT8 quantization	True
dataset	Calibration dataset	.txt (CNN) / .json (LLM)

6. Quantization Calibration Datasets

Model type	Dataset path	Count
CNN (MobileNet V2)	datasets/imagenet/.../dataset_20.txt	20 images
LLM (Qwen3)	datasets/CMMLU/dataset.json	Dozens of entries (Chinese Q&A)
Object detection (yolov5)	dataset.txt	Usually 20-50 images

# 1. Collect representative images
mkdir -p calib_images
cp /path/to/representative_*.jpg calib_images/

# 2. Generate the path list
ls calib_images/*.jpg > dataset.txt

# 3. Reference it from the conversion script
rknn.build(do_quantization=True, dataset='./dataset.txt')

7. Accuracy Verification

RKNN3 has no built-in accuracy comparison tool; you need to write a verification script:

python3 -c "from rknn3lite.api import RKNN3Lite; print('RKNN3Lite available')"

Output:

RKNN3Lite available

from rknn3lite.api import RKNN3Lite  # use lite on the board

rknn_lite = RKNN3Lite()
rknn_lite.load_rknn('model.rknn', 'model.weight')
rknn_lite.init_runtime()

# Run the same input and compare RKNN3 output vs PyTorch output
outputs = rknn_lite.inference(inputs=[img])

# Compute the difference
import numpy as np
mse = np.mean((outputs[0] - pytorch_output) ** 2)

Task	Metric
Classification	Top-1 / Top-5 accuracy
Detection	mAP / IoU
Super-resolution / restoration	MSE / PSNR
LLM	Perplexity

8. Examples in the SDK

Model Zoo examples:

ls /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-model-zoo/examples/

Output:

FastVLM         glm_edge        GME-Qwen2-VL    HY_MT_1_5
InternVLM       Janus_Pro       MiniCPM_V_4     mobilenet_v1
mobilenet_v2    Qwen2_5         Qwen2_5_Omni    Qwen2_5_VL
Qwen3           Qwen3_Embedding Qwen3_VL        resnet
SmolVLM         yolov5          yolov6          yolov8

MobileNet V2 conversion scripts:

ls /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-model-zoo/examples/mobilenet_v2/python/

Output:

convert.py
dataset_eval.py

• examples/mobilenet_v2/python/convert.py — Convert MobileNet V2 to RKNN3
• examples/mobilenet_v2/python/dataset_eval.py — MobileNet V2 dataset evaluation

Qwen3 conversion scripts:

ls /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-model-zoo/examples/Qwen3/python/

Output:

export_llm.py
export_rknn.py

• examples/Qwen3/python/export_llm.py — Export Qwen3 to ONNX
• examples/Qwen3/python/export_rknn.py — Convert ONNX to RKNN

Prepare the datasets (ImageNet / CMMLU) before running the examples.

9. FAQ

Symptom	Cause	Fix
ModuleNotFoundError: No module named 'rknn'	Installed on the board (aarch64)	Move to a PC (x86_64) to install
target_platform 'rk1828' not supported	Typo	Change it to 'rk1820'
Toolkit won't install on Python 3.11	Only 3.10 / 3.12 supported	Install Python 3.10 or 3.12
do_quantization=True complains about missing dataset	Dataset not downloaded	Prepare 20+ samples
Accuracy drops after quantization	Insufficient calibration coverage	Expand the calibration set + change quantized_method

10. Next Steps

• INT8 Quantized Inference — quantization + performance tuning
• NPU Overview — the AI inference engine
• Model Conversion — hands-on steps

11. References

• RK1820_RK1828_AI_Release-Note_CN.md (in the SDK) — full RKNN3 V1.0.0 release notes
• Rockchip_RK1820_RK1828_AI_SDK_RELEASE_CN.pdf (in the SDK) — release note PDF
• Rockchip_RK1820_RK1828_AI_SDK_Quick_Start_CN.pdf (in the SDK) — quick start guide