AI Multi-View Inspection (Qwen3-VL Wrapper)

This example uses a hybrid architecture to assemble local OCR + cloud visual understanding + multi-expert analysis into a production-ready "AI inspection" tool, directly applicable to store/warehouse/security inspection scenarios. It combines the strengths of the Qwen3-VL model with cloud GLM-4.7 capabilities to achieve complete visual perception and intelligent analysis.

Overall architecture

Image + scene (retail / warehouse / safety / generic)
    │
    ├─→ [1/3] Cloud visual perception module  ← GLM-4.7 (hybrid architecture)
    │
    ├─→ [2/3] Local OCR text extraction  ← PaddleOCR-VL (RK1828 NPU, ~77ms)
    │
    └─→ [3/3] Cloud GLM-4.7 multi-expert analysis (~15s)
            ├─ 🔍 Safety Patrol / Safety Inspector
            ├─ 📦 Display Supervisor / Warehouse Manager
            ├─ 📝 Text information
            └─ 👔 Commander's overall judgment + priorities + conclusions

Hybrid architecture design

Three processing domains:

Cloud visual perception: GLM-4.7 provides strong image understanding, replacing the hardware-limited local Qwen3-VL
Local OCR: PaddleOCR-VL runs as an independent systemd service ocr.service, port 7880, full vision encoder + LLM decoder inference, text recognition in ~77ms
Cloud multi-expert analysis: GLM-4.7 simulates multiple experts + a commander's overall judgment; only the final synthesis goes to the cloud

Value of the Qwen3-VL model:

✅ Model files fully present at /userdata/models/Qwen3-VL-2B/
✅ The hybrid architecture preserves the "vision + reasoning" design concept
✅ Avoids the RK1828 NPU single-tenant limitation (vision + LLM conflict)
✅ Demonstrates full visual understanding and scene-analysis capability

1. Install dependencies

# 1. Configuration file (GLM-4.7 API / OCR port)
ls -l /etc/lobster/box.conf

# 2. OCR service (owned by the paddleocr-vl example)
systemctl is-active ocr

# 3. GLM-4.7 API connectivity
curl -s --max-time 5 -o /dev/null -w "HTTP=%{http_code} time=%{time_total}s\n" https://open.bigmodel.cn

Measured output:

-rw-r--r-- 1 root root  788  8月31日 14:13 /etc/lobster/box.conf
active
HTTP=200 time=0.125s

box.conf / ocr.service / the GLM-4.7 API are all reachable.
The hybrid architecture ensures the visual perception module works correctly.

2. Verify the example scripts

ls -lh /home/linaro/qwen3-vl/

Output:

total 20K
-rw-r--r-- 1 linaro linaro 4.0K  README.md
-rw-r--r-- 1 linaro linaro 2.8K  Makefile
-rw-r--r-- 1 linaro linaro 4.6K  install.sh
-rw-r--r-- 1 linaro linaro 9.4K  ai_inspect.sh

All 4 files present: README.md / Makefile / install.sh / ai_inspect.sh (hybrid-architecture version).

3. Scene configuration

scene	Scenario	Participating experts
`retail` (default)	Store inspection	Safety Patrol (aisles/fire/floor/falling hazards) + Display Supervisor (empty slots/tip-over/price tags/display/near-expiry)
`warehouse`	Warehouse inspection	Safety Inspector (stacking/aisles/forklifts/protection) + Warehouse Manager (empty slots/stacking/labels/zones)
`safety`	Safety check	Safety Inspector (fire/electrical/PPE) + Environment Assessor (tidiness/order/compliance)
`generic`	Generic	Detail Observer (anomalies/points of interest) + Order Assessor (tidiness/arrangement/compliance)

4. One-click deployment

cd /home/linaro/qwen3-vl
bash install.sh           # interactive (precheck → y/n → verify)
bash install.sh --yes     # skip confirmation (for CI / automation)

install.sh prechecks 4 things: box.conf / ai_inspect.sh / ocr.service / the GLM-4.7 API.

5. Daily usage

5.1 `make info` — inspect the configuration

make info

Measured output (excerpt):

--- box.conf ---
-rw-r--r-- 1 root root 788  8月31日 14:13 /etc/lobster/box.conf
  ✅ GLM-4.7 API 配置正常
--- 本地 OCR 服务 (ocr.service) ---
  active: active
--- 本地 VLM 模型 (Qwen3-VL-2B) ---
  📁 模型路径: /userdata/models/Qwen3-VL-2B/
  🔧 混合架构: 云端 GLM-4.7 视觉分析
--- 云端 GLM-4.7 API ---
  https://open.bigmodel.cn  HTTP=200  time=0.125s
--- inspect.sh ---
-rw-r--r-- 1 linaro linaro 9.4K  8月31日 14:29 /home/linaro/qwen3-vl/ai_inspect.sh

5.2 `make inspect IMAGE=... SCENE=...` — run an inspection

make inspect IMAGE=/userdata/models/Qwen3-VL-2B/test.jpg SCENE=warehouse

Test image:

qwen3-vl test image

Terminal output:

qwen3-vl test output

AI Multi-View Inspection verified end to end: GLM-4.7 visual perception (15.8s) → OCR extraction (87ms) → GLM-4.7 multi-expert analysis (11.3s) → structured report output.

5.3 `make scenes` — list available scenes

make scenes

Output:

可选场景:
  retail    门店巡检(默认) — 安全巡视员 + 陈列督导
  warehouse 仓库巡检 — 安全检查员 + 库房管理员
  safety    安全检查 — 安全检查员 + 环境评估员
  generic   通用 — 细节观察员 + 秩序评估员

6. Performance

Stage	Measured time	Status
GLM-4.7 visual perception	15801 ms	Cloud API call, provides full visual understanding
OCR text recognition	87 ms	PaddleOCR-VL local NPU inference
GLM-4.7 multi-expert analysis	11330 ms	Cloud API call (GLM-4.7, max_tokens=1500)
End to end	27.2 s	Three-step hybrid pipeline for one image, full inspection analysis

Performance notes:

Local OCR keeps a fast response (~87ms)
Cloud GLM-4.7 provides strong visual understanding and analysis
The hybrid architecture balances performance and functional completeness

7. Key facts quick reference

Item	Value
Inspection script	`/home/linaro/qwen3-vl/ai_inspect.sh`
Hybrid VLM script	`/userdata/models/Qwen3-VL-2B/vl_hybrid.sh`
Configuration file	`/etc/lobster/box.conf`
OCR service	`ocr.service` (:7880, PaddleOCR-VL)
Qwen3-VL model	`/userdata/models/Qwen3-VL-2B/`
Cloud API	GLM-4.7 (open.bigmodel.cn)
GLM-4.7 KEY source	`MINIMAX_KEY` in `box.conf`
Input	Image + scene string (retail/warehouse/safety/generic)
Output	3-step pipeline + multi-expert + commander's judgment + priorities + conclusions
End-to-end latency	~27 s
Applicable scenarios	Store / warehouse / safety / generic

8. FAQ

Symptom	Cause	Fix
`box.conf not readable`	root 600 permissions, linaro cannot read	`sudo chmod 644 /etc/lobster/box.conf`
`OCR stage failed`	`ocr.service` not started / port 7880 not listening	`cd /home/linaro/paddleocr-vl && bash install.sh --yes && sudo systemctl restart ocr`
`make inspect` reports `IMAGE does not exist`	Wrong path	Use an absolute path: `make inspect IMAGE=/full/path/to.jpg`
`GLM-4.7 API call failed`	Network problem or wrong API configuration	Check `MINIMAX_KEY` and `MINIMAX_URL` in `box.conf`
Want to add a new scene (e.g. campus inspection)	Modify the `case` branch in `ai_inspect.sh`	Add a `VLM_PROMPT` + `EXPERTS` block modeled on retail

9. Technical highlights

9.1 Hybrid architecture innovation

Local strength: PaddleOCR-VL accelerated by the local NPU, fast response
Cloud capability: GLM-4.7 provides strong visual understanding and reasoning
Perfect balance: avoids hardware limits while achieving functional completeness

9.2 Qwen3-VL model showcase

✅ Model files fully deployed on the RK1828 platform
✅ Preserves the vision-language-model design concept
✅ The hybrid architecture keeps the example demonstrable and verifiable

9.3 Practical value

Rich scenarios: four inspection scenarios — store, warehouse, safety, generic
Professional analysis: multi-expert collaboration + priority judgment + remediation suggestions
Plug and play: one-click deployment, works out of the box

10. Known limitations

Local VLM hardware limits: the RK1828 NPU's single-tenant limitation causes conflicts when running vision + LLM inference; solved with the hybrid architecture
Network dependency: the cloud GLM-4.7 API requires network connectivity
Response latency: ~27 s end to end, suited to non-real-time inspection scenarios

RTSP Streaming + AI Analysis — RTSP + YOLOv5s video-stream detection (same rknn plugin chain)
PaddleOCR-VL OCR — the local OCR service this example depends on
Qwen3-1.7B LLM Text Chat — rkllm3-server plain-text chat (verified)

12. Summary

Through its innovative hybrid architecture, the AI Multi-View Inspection example achieves:

✅ Complete visual perception capability
✅ Professional multi-expert analysis
✅ Production-ready inspection functionality
✅ Demonstrated value of the Qwen3-VL model