RKNN3 SDK Overview

RKNN3 SDK is the AI deployment software stack provided by Rockchip for the RK1820 / RK182X / RK3572 platforms.

Overall block diagram

PC side (RKNN3-Toolkit)               Device side (RKNN3-Lite)
┌──────────────────────┐             ┌──────────────────────┐
│ 1. Load ONNX/PyTorch │             │ 4. load_rknn()       │
│ 2. config + build    │ ──────────► │ 5. inference()       │
│ 3. export_rknn()     │             │ 6. Parse output      │
└──────────────────────┘             └──────────────────────┘

RKNN3 Toolkit is not on PyPI; the .whl file must be downloaded from Rockchip officially
The GitHub repository is hard to access from mainland China; a mirror site is recommended
The full PC-side toolkit requires Python 3.10 or 3.12 (3.11 is not supported)

1. SDK Components

Component	Description	Status
RKNN3-Toolkit (PC)	Model conversion, quantization, profiling	Not installed, download needed
RKNN3-Toolkit-Lite (board)	Inference runtime	Installed, 1.0.0
RKNN3 Model Zoo	Examples and pre-converted models	Included in the SDK
librknnrt.so	C/C++ inference library	`/usr/lib/librknnrt.so`
rknn3_transfer_proxy	PCIe EP proxy	Running on the board

2. Version Reference Table

Component	Current version	Source	Description
RKNN3-Toolkit (PC whl)	1.0.0	Bundled in SDK at `rknn/rknn3-toolkit/rknn3-toolkit/packages/`	Initial version released with the official SDK
RKNN3-Toolkit-Lite (board pip)	1.0.0	Installed via pip (whl bundled in SDK)	Matches the SDK version
RKNN3 Release Note	V1.0.0	`docs/RK1820_RK1828_AI_Release-Note_CN.md` in the SDK	Baseline release version of the whole SDK
rkllm3-server	1.0.0	`/usr/bin/rkllm3-server` (`b4bd144@2026-01-22T21:27:27`)	Same version as the Toolkit
RKNN3 Runtime	1.0.0	`librknn3_api.so` (board side)	Major version of the board-side inference library

Version relationships:
The whl file = 1.0.0 is the version bundled with the SDK, corresponding to RKNN3 Release Note V1.0.0
The board-side pip-installed rknn3-toolkit-lite is 1.0.0, consistent with the SDK release version
The board-side rkllm3-server binary is 1.0.0, same version as the Toolkit
A .rknn file produced by conversion should be loaded with a runtime of the same version or newer than the one used for conversion

3. Package Name / Module Name Reference

pip package	Python module	Usage
`rknn3-toolkit-lite`	`rknn3lite`	Board-side inference
`rknn-toolkit3` (PC)	`rknn`	PC-side conversion

Installed packages:

pip3 list | grep rknn

Output:

rknn3-toolkit-lite    1.0.0

Module path:

python3 -c "import rknn3lite; print(rknn3lite.__file__)"

Output:

/usr/local/lib/python3.11/dist-packages/rknn3lite/__init__.py

Inference library:

ls /usr/lib/librknnrt.so

Output:

/usr/lib/librknnrt.so

PCIe proxy processes:

ps aux | grep rknn3_transfer_proxy | grep -v grep

Output:

root        1396  0.0  178176   2028 ?        Sl   11:24   0:08 /bin/rknn3_transfer_proxy
root       58359 23.0  578496    228 ?        Sl   13:43  37:06 rknn3_transfer_proxy_c36211b1 -s 0004:41:00.0

The ps aux output varies with PID / memory / CPU time, and the hash suffix is randomized on every restart; the key fields are the command name and the BDF after -s (0004:41:00.0 on this machine).

4. Python Version Requirements

Tool	Supported versions
RKNN3-Toolkit (PC)	3.10 / 3.12 (3.11 not supported)
RKNN3-Toolkit-Lite (board)	3.9 / 3.10 / 3.11 / 3.12 (a cp311 whl exists)
RKNN-Toolkit2 (PC)	3.6 ~ 3.12

The board currently runs Python 3.11.2 — only Lite can be used; the PC Toolkit requires reinstalling Python.

5. Supported Platforms

Toolchain	Target chips
RKNN3	RK1820 / RK182X (PCIe NPU accelerator cards), RK3572 (SoC)
RKNN-Toolkit2	RK3588 / RK3576 / RK3568 / RK3562, etc.

.rknn files generated by RKNN3 and RKNN-Toolkit2 are not mutually compatible. Do not use both toolchains to generate models within the same project.

6. Obtaining the RKNN3 Toolkit

Access to GitHub is restricted in mainland China; recommended approaches:

Official Rockchip download: contact Rockchip sales or technical support
Domestic mirrors: obtain via mirror sites such as Gitee

Verifying that PyPI is not available:

curl -s https://pypi.org/pypi/rknn-toolkit3/json

Output:

{"message": "Not Found"}

7. Supported Models

Type	Model	Status in SDK
LLM	Qwen3-1.7B / 4B	`Qwen3-1.7B.rknn`, `Qwen3-4B.rknn`
VLM	Qwen3-VL-2B	Tokenizer only, no model file
CNN	MobileNet V2 / YOLOv8s / YOLOv5s	Available under `examples/gstreamer/`
Embedding	bge-small-zh-v1.5	Not deployed
ASR / TTS	Whisper / VITS / Qwen3-TTS	Not deployed

Deployed models:

ls /userdata/models/

Output:

Qwen3-1.7B
Qwen3-4B
Qwen3-VL-2B
run.sh
run.sh.bak.1.7B
run.sh.bak.4b
run.sh.bak.test

CNN model examples:

ls /userdata/RK1820_RK1828_AI_SDK/examples/gstreamer/

Output:

MobilenetV1
MobilenetV2
README_CN.md
README_EN.md
resnet50v2
run_model.sh
yolov5s
yolov6s
yolov8s

Locating CNN model files:

find /userdata/RK1820_RK1828_AI_SDK/examples/gstreamer/ -name "*.rknn"

Output:

.../gstreamer/yolov8s/yolov8s.rknn
.../gstreamer/MobilenetV1/mobilenet_v1.rknn
.../gstreamer/resnet50v2/resnet50v2.rknn
.../gstreamer/yolov5s/yolov5s.rknn
.../gstreamer/yolov6s/yolov6s.rknn
.../gstreamer/MobilenetV2/mobilenetv2-12.rknn

8. SDK Release Note

cat /userdata/RK1820_RK1828_AI_SDK/docs/RK1820_RK1828_AI_Release-Note_CN.md | head -30

Output:

# 更新日志

## Release V1.0.0

1. Examples 应用示例
 - 新增 Docker 环境部署说明及示例，提升示例运行与复现效率
 - rockit_yolo_detect 示例兼容 RKNN2 与 RKNN3 接口
 - 新增工业异常检测、vi_yolo_venc 示例，并补充对应应用文档说明

2. 工具集更新
- RKNN-Console 新增多卡与 USB 设备支持，修复异常场景下的稳定性问题
- RKNN-SMI 新增 NPU 负载、电流、电压、功耗等查询接口，并优化 CLI 与日志机制
- 新增 rknn-test 自动化测试脚本，提升功能验证与稳定性验证效率

3. RKNN3更新
- 版本更新至 V1.0.0，覆盖 RKNN3 Toolkit、RKNN3 Runtime、RKNN3 Model Zoo
- 大幅提升 LLM/ViT 性能，LLM Decode 性能整体提升超过15%
- 扩展模型支持范围，新增适配 Qwen3-VL / Qwen2.5-Omni(Thinker) / GLM Edge / SmolVLM 等模型
- 支持连板精度分析
- 支持数据传输与推理并行
- 支持 mRoPE / Function Call / YUV 格式输入
- rkllm3 server 新增支持 embedding 模型，并兼容音频输入
- 支持多核多模型同时推理
- 支持用户在协处理器上自定义模型后处理
- 优化 exSDPA、exMatMul、Resize、Transpose 等算子实现

(The above is the literal content of the Chinese release note file shipped with the SDK.)

9. Differences from RKNN-Toolkit2

Feature	RKNN3 Toolkit	RKNN-Toolkit2
Target chips	RK1820 / RK182X / RK3572	RK3588 / RK3576 / RK3568
LLM support	Core capability	Only the legacy RKNN-LLM toolchain
VLM support	Qwen3-VL series	A few models such as MiniCPM-V
Python version	3.10 / 3.12	3.6 ~ 3.12
Compatibility	`.rknn` not interchangeable	`.rknn` not interchangeable

10. FAQ

Symptom	Cause	Resolution
`rknn-toolkit3` not found on PyPI	Distributed only via SDK / GitHub	Contact Rockchip / use a mirror
Toolkit won't install on Python 3.11	Only 3.10 / 3.12 supported	Install Python 3.10 or 3.12
`rknn3lite` module import fails	Lite package not installed	`pip install rknn3-toolkit-lite`
RKNN3 and RKNN2 `.rknn` incompatible	Toolchains not interchangeable	Use one toolchain per project

11. Next Steps

RKNN3 Toolkit Installation — PC-side installation
NPU Overview — NPU architecture
Model Conversion — hands-on PC-side guide

12. References

RKNN3 Model Zoo (with conversion examples)
RKNN-Toolkit2
Full documentation: docs/RK1820_RK1828_AI_Release-Note_CN.md in the SDK