RKNN3 Toolkit Installation and Usage

RKNN3 Toolkit is the PC-side model conversion tool of the RKNN3 SDK.

Overall block diagram

PC side (x86_64 + Python 3.10/3.12)       Board side (aarch64 + Python 3.11)
┌──────────────────────────┐             ┌──────────────────────────┐
│ rknn3_toolkit-*.whl      │             │ rknn3_toolkit_lite-*.whl │
│ from rknn.api import RKNN│  ────→      │ from rknn3lite.api       │
│ config / build / export  │  scp/adb    │ load_rknn + inference    │
└──────────────────────────┘             └──────────────────────────┘

If you only need to run the official pre-converted models, skip this chapter and refer directly to the application development chapters.

The Python module name is rknn; the board-side Lite module is rknn3lite
The whl bundled in the SDK is version 1.0.0
All board-side devices are aarch64 (including RK3588 host devices) and cannot install the PC-side x86_64 whl

1. whl Files Bundled with the SDK

find /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit -name "*.whl"

Output:

.../rknn3-toolkit/packages/rknn3_toolkit-1.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.../rknn3-toolkit/packages/rknn3_toolkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.../rknn3-toolkit-lite/packages/rknn3_toolkit_lite-1.0.0-cp39-cp39-linux_aarch64.whl
.../rknn3-toolkit-lite/packages/rknn3_toolkit_lite-1.0.0-cp311-cp311-linux_aarch64.whl
.../rknn3-toolkit-lite/packages/rknn3_toolkit_lite-1.0.0-cp310-cp310-linux_aarch64.whl
.../rknn3-toolkit-lite/packages/rknn3_toolkit_lite-1.0.0-cp312-cp312-linux_aarch64.whl
.../rknn3-toolkit/docker/docker_file/ubuntu_22_04_cp310/rknn3_toolkit-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

The last line is a legacy 0.5.0 whl under docker/docker_file/ubuntu_22_04_cp310/ (used by the docker image shipped with an older SDK); do not install it by mistake. For a normal installation use the 1.0.0 whl under packages/.

Directory structure:

ls -la /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/

Output:

CHANGELOG.md
doc
.git
LICENSE
README_EN.md
README.md
res
rknn3-toolkit
rknn3-toolkit-lite

Package name reference:

Usage	Package name	Version	Module name	Architecture
PC-side conversion	`rknn3-toolkit`	1.0.0 (whl)	`rknn`	x86_64
Board-side inference	`rknn3-toolkit-lite`	1.0.0	`rknn3lite`	aarch64

PC-side toolkit size:

ls -lh /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit/packages/*.whl

Output:

-rw-r--r-- 1 linaro linaro 181M  2026年 1月28日 rknn3_toolkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
-rw-r--r-- 1 linaro linaro 179M  2026年 1月28日 rknn3_toolkit-1.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Board-side lite size:

ls -lh /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit-lite/packages/*.whl | head -4

Output:

-rw-r--r-- 1 linaro linaro 303K  2026年 1月28日 rknn3_toolkit_lite-1.0.0-cp310-cp310-linux_aarch64.whl
-rw-r--r-- 1 linaro linaro 302K  2026年 1月28日 rknn3_toolkit_lite-1.0.0-cp311-cp311-linux_aarch64.whl
-rw-r--r-- 1 linaro linaro 274K  2026年 1月28日 rknn3_toolkit_lite-1.0.0-cp312-cp312-linux_aarch64.whl
-rw-r--r-- 1 linaro linaro 303K  2026年 1月28日 rknn3_toolkit_lite-1.0.0-cp39-cp39-linux_aarch64.whl

2. Installation

The following is a PC-side operation; the current device is aarch64 and cannot execute it.

Current environment:

python3 --version

Output:

Python 3.11.2

uname -m

Output:

aarch64

2.1 PC-Side Installation (x86_64 + Python 3.10/3.12)

# PC side (from the whl bundled in the SDK)
pip install /userdata/RK1820_RK1828_AI_SDK/rknn/rknn3-toolkit/rknn3-toolkit/packages/rknn3_toolkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

# Verify
python -c "from rknn.api import RKNN; print('OK')"

2.2 Board Side (already installed)

pip3 list | grep rknn

Output:

rknn3-toolkit-lite           1.0.0

3. LLM Model Conversion

The following is a PC-side operation and requires an NVIDIA GPU with CUDA (for GRQ quantization).

from rknn.api import RKNN
import os

os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com/'

rknn = RKNN(verbose=True)

# Step 1: ONNX export + GRQ quantization (still requires the CMMLU dataset)
# Done via export_llm.py from the model zoo
os.system('python examples/Qwen3/python/export_llm.py --quant --model_path Qwen/Qwen3-1.7B')

# Step 2: Convert to RKNN
rknn.config(target_platform='rk1820',
            quantized_dtype='w4a16',
            quantized_algorithm='grq',
            quantized_method='group32')

rknn.load_llm(model='Qwen3-1.7B.onnx', config='Qwen3-1.7B.config.pkl')
rknn.build(do_quantization=True, dataset='dataset.txt')
rknn.export_rknn('./Qwen3-1.7B.rknn')

4. CNN Model Conversion

from rknn.api import RKNN

rknn = RKNN(verbose=True)

rknn.config(target_platform='rk1820',
            mean_values=[[0, 0, 0]],
            std_values=[[255, 255, 255]])

rknn.load_onnx(model='./mobilenet_v2.onnx')
rknn.build(do_quantization=True, dataset='./dataset.txt')
rknn.export_rknn('./mobilenet_v2.rknn')

5. Key Parameters

Parameter	Value	Description
`target_platform`	`'rk1820'`	Target platform (not `'rk1828'`)
`quantized_dtype`	`'w4a16'`	Quantized data type
`quantized_algorithm`	`'grq'` / `'mmse'`	Quantization algorithm
`quantized_method`	`'group32'` / `'channel'`	Quantization granularity
`max_context_len`	Set in step 1 of `export_llm.py`	LLM context length

6. Environment Requirements

Item	PC requirement	Board requirement
Python	3.10 / 3.12 (3.11 not supported)	3.9 / 3.10 / 3.11 / 3.12
OS	Linux x86_64 (the whl is Linux-only)	Linux aarch64
GPU	NVIDIA (CUDA 11.8+) — only needed for GRQ quantization	Not required
Plain load / build	CPU suffices	CPU suffices

7. LLM Conversion Outputs

Qwen3-1.7B actually deployed on the board:

ls /userdata/models/Qwen3-1.7B/ | grep -E "\.(onnx|pkl|rknn|weight|gguf|bin)$"

Output:

Qwen3-1.7B.embed.bin
Qwen3-1.7B.rknn
Qwen3-1.7B.tokenizer.gguf
Qwen3-1.7B.weight

Total size:

du -sh /userdata/models/Qwen3-1.7B/

Output:

1.7G	/userdata/models/Qwen3-1.7B/

Description of the converted files:

Qwen3-1.7B.onnx            # ❌ Step 1: ONNX (intermediate file, deleted after conversion)
Qwen3-1.7B.config.pkl      # ❌ Conversion config (intermediate file, deleted after conversion)
Qwen3-1.7B.rknn            # ✅ RKNN model structure (24 MB)
Qwen3-1.7B.weight          # ✅ RKNN model weights (1.1 GB)
Qwen3-1.7B.tokenizer.gguf  # ✅ Tokenizer (5.7 MB)
Qwen3-1.7B.embed.bin       # ✅ Embedding table (594 MB)

The intermediate files (onnx, pkl) only exist during the PC-side conversion; final deployment needs only the final 4 files.

8. FAQ

Symptom	Cause	Resolution
Toolkit won't install on Python 3.11	Only 3.10 / 3.12 supported	Install Python 3.10 or 3.12
`rknn-toolkit3` not found on PyPI	Distributed only via SDK / GitHub	Use the whl bundled in the SDK
GRQ quantization complains about missing dataset	Dataset not downloaded	Prepare `CMMLU/dataset.json`
Accidentally installed the docker 0.5.0 whl	Wrong path chosen	Use the 1.0.0 under `packages/`
`target_platform` error	Written as `'rk1828'`	Change to `'rk1820'`
Board-side Lite won't install	pip / whl mismatch	Use the cp311 version

9. Next Steps

Model Conversion — hands-on PC-side guide
NPU Overview — NPU architecture
RKNN3 SDK Overview — SDK overview

10. References

RKNN3 Model Zoo (conversion examples)
RKNN-Toolkit2
Full documentation: docs/RK1820_RK1828_AI_Release-Note_CN.md in the SDK