PaddleOCR-VL Text Recognition

This example adapts Baidu's PaddleOCR-VL model from GPU to the RKNN NPU platform, managed by systemd ocr.service, exposing a single-image recognition endpoint via curl /ocr.

Overall block diagram:

Client curl → :7880/ocr
    │
    ▼
ocr_server.py (systemd ocr.service)
    ├─→ vision encoder → RK1828 NPU (507ms)
    ├─→ vision-mlp_AR   → RK1828 NPU
    ├─→ LLM decoder     → RK1828 NPU (168ms)
    └─→ response JSON  {text, vision_ms, llm_ms, total_ms, tokens}

Two processing domains:

RK3588: HTTP service process, file IO, JSON encode/decode
RK1828 (PCIe co-processor): three-stage inference — vision encoder + mlp_AR + LLM decoder — connected to the Host via /dev/dri/renderD128 (PCIe address 0004:41:00.0)

1. Install dependencies

# 1. NPU node
ls -l /dev/dri/renderD128

# 2. rknn3 systemd service
systemctl is-active rknn3

# 3. Python dependencies (rknn3lite / cv2 / gguf)
python3 -c "import rknn3lite, cv2; from gguf import GGUFReader; print('OK')"

# 4. Model + service unit
ls /userdata/models/Qwen3-VL-2B/PaddleOCR-VL/
ls -l /etc/systemd/system/ocr.service

Measured output (excerpt):

crw-rw---- 1 root video 226, 128  8月27日 11:11 /dev/dri/renderD128
active
OK
PaddleOCR-llm      PaddleOCR-vision
-rw-r--r-- 1 root root 477  8月27日 11:11 /etc/systemd/system/ocr.service

The NPU node / rknn3.service / model / service unit are all in place.
The gguf Python package is installed under /home/linaro/.local/lib/python3.11/site-packages/ (user-local install for linaro); root's default PYTHONPATH cannot find it — see section 2.

2. Required service-file modification

The SDK's ocr.service runs as root, but the gguf package is in the linaro home directory, invisible to root → startup fails with ModuleNotFoundError: No module named 'gguf'.

Fix: add User=linaro + PYTHONPATH in the [Service] section:

[Service]
Type=simple
User=linaro
Group=linaro
Environment=PYTHONUSERBASE=/home/linaro/.local
Environment=PYTHONPATH=/home/linaro/.local/lib/python3.11/site-packages
ExecStart=/usr/bin/python3 /userdata/models/Qwen3-VL-2B/ocr_server.py
Environment=OCR_HOST=127.0.0.1
Environment=OCR_PORT=7880
Restart=on-failure
RestartSec=10

Automatic modification command (run once manually on first deployment):

sudo cp /etc/systemd/system/ocr.service /etc/systemd/system/ocr.service.bak.$(date +%Y%m%d)
sudo sed -i '/^\[Service\]$/a User=linaro\nGroup=linaro\nEnvironment=PYTHONUSERBASE=/home/linaro/.local\nEnvironment=PYTHONPATH=/home/linaro/.local/lib/python3.11/site-packages' /etc/systemd/system/ocr.service
sudo systemctl daemon-reload
sudo systemctl restart ocr

4. One-click deployment

cd /home/linaro/paddleocr-vl
bash install.sh           # interactive (precheck → y/n → verify)
bash install.sh --yes     # skip confirmation

install.sh prechecks 3 things: the 8 model files / the 3 Python packages / the ocr.service unit. No files are installed — pure reuse + verification.

5. Daily usage

5.1 `make status` — systemd status

make status

Measured output (excerpt):

● ocr.service - PaddleOCR-VL OCR HTTP Service
     Loaded: loaded (/etc/systemd/system/ocr.service; disabled; preset: disabled)
     Active: active (running) since ...
   Main PID: 12345 (python3)
      Tasks: 8 (limit: 7653)
     Memory: 1.2G
        CPU: 5.132s

5.2 `make health` — health check

make health

Measured output:

{
  "status": "ok",
  "model": "PaddleOCR-VL"
}

5.3 `make ocr` — run one recognition

make ocr    # uses the SDK-bundled test.jpg

Input image (SDK-bundled test.jpg):

PaddleOCR-VL test input image — three lines of Chinese text

Measured output:

PaddleOCR-VL measured output — recognition result for the SDK-bundled test.jpg Chinese text + timing stats

PaddleOCR-VL verified end to end: vision encoder + LLM decoder inference is correct and Chinese text is recognized.

<fcel> / <lcel> / <nl> are PaddleOCR-VL's special markers (similar to HTML table tags) indicating recognition-box positions. The actual text content sits between the tags (in this example 群号:1025468710).

To clean it up, use a client-side regex:

import re
text = re.sub(r'<[^>]+>', '', raw_text)

Output:

群号:1025468710

5.5 `make restart` / `make logs`

make restart    # systemctl restart ocr (model load takes ~60s)
make logs       # journalctl -u ocr -n 20
make stop       # systemctl stop ocr (temporary stop)

6. HTTP API

6.1 `GET /health`

curl -s http://127.0.0.1:7880/health

Response:

{ "status": "ok", "model": "PaddleOCR-VL" }

6.2 `POST /ocr`

Request:

{ "image": "<base64 of image>" }