PP-OCR (Optical Character Recognition)

1. Introduction

PP-OCR is a practical optical character recognition (OCR) tool library open-sourced by Baidu PaddlePaddle. It aims to provide a high-accuracy, easy-to-use, and flexibly deployable text recognition solution. It integrates PaddlePaddle's technical accumulation in computer vision and supports multi-language, multi-scenario text detection and recognition. It is widely used in scenarios such as document digitization, license plate recognition, industrial quality inspection, and smart office. Its core features include a balance between accuracy and practicality, optimized for real business scenarios; while ensuring recognition accuracy, it balances speed and deployment cost through lightweight model design (such as the mobile model PP-OCRv3-mobile), and supports recognition of Chinese and English, multiple languages (Japanese, Korean, French, etc.), and special scenarios (such as curved text and blurred text).

Project Directory

PP-OCR
├─cpp
│  ├─dependencies		##C++例程依赖
│  │
│  └─ppocr_bmcv
│      │  CMakeLists.txt	##交叉编译所需文件
│      │  ppocr_bmcv.soc	##提供的交叉编译好的可执行文件
│      │
│      ├─include			##交叉编译的依赖项
│      │      clipper.h
│      │      postprocess.hpp
│      │      ppocr_cls.hpp
│      │      ppocr_det.hpp
│      │      ppocr_rec.hpp
│      │
│      ├─src				##交叉编译源码
│      │      clipper.cpp
│      │      main.cpp
│      │      postprocess.cpp
│      │      ppocr_cls.cpp
│      │      ppocr_det.cpp
│      │      ppocr_rec.cpp
│      │
│      └─thirdparty			##交叉编译第三方库
│              cnpy.cpp
│              cnpy.h
│
├─docs		##帮助文档
│  │  PP-OCR.md
│  │
│  └─images
├─python	##python例程所需文件
│      ppocr_cls_opencv.py
│      ppocr_det_opencv.py
│      ppocr_rec_opencv.py
│      ppocr_system_opencv.py
│      requirements.txt
│
├─scripts
│      download.sh		##下载数据集和模型所需的脚本文件
│
└─tools					##比较和评估的文件
        compare_statis.py
        eval_icdar.py

2. Running Steps

Before running the test examples, you need to download the required dataset and model.

#安装下载工具
pip3 install dfss --upgrade
#执行下载脚本
bash scripts/download.sh

1. Python Example

1.1 Text Detection Inference Test

The parameters of ppocr_det_opencv.py are:

usage: ppocr_det_opencv.py [-h] [--dev_id DEV_ID] [--input INPUT] [--bmodel_det BMODEL_DET]

optional arguments:
  -h, --help            show this help message and exit
  --dev_id DEV_ID       tpu card id
  --input INPUT         input image directory path
  --bmodel_det BMODEL_DET
                        bmodel path

A text detection test example:

# 程序会自动根据文件夹中的图片数量来选择1batch或者4batch，优先选择4batch推理。
python3 python/ppocr_det_opencv.py --input datasets/cali_set_det --bmodel_det models/BM1684X/ch_PP-OCRv4_det_fp32.bmodel --dev_id 0

After execution, predicted images are saved under the results/det_results folder.

seg

1.2 Text Recognition Inference Test

The parameters of ppocr_rec_opencv.py are:

usage: ppocr_rec_opencv.py [-h] [--dev_id DEV_ID] [--input INPUT] [--bmodel_rec BMODEL_REC] [--img_size IMG_SIZE] [--char_dict_path CHAR_DICT_PATH] [--use_space_char USE_SPACE_CHAR] [--use_beam_search]
                           [--beam_size {1~40}]

optional arguments:
  -h, --help            show this help message and exit
  --dev_id DEV_ID       tpu card id
  --input INPUT         input image directory path
  --bmodel_rec BMODEL_REC
                        recognizer bmodel path
  --img_size IMG_SIZE   You should set inference size [width,height] manually if using multi-stage bmodel.
  --char_dict_path CHAR_DICT_PATH
  --use_space_char USE_SPACE_CHAR
  --use_beam_search     Enable beam search
  --beam_size {1~40}    Only valid when using beam search, valid range 1~40

A text recognition test example:

# 程序会自动根据文件夹中的图片数量来选择1batch或者4batch，优先选择4batch推理。
python3 python/ppocr_rec_opencv.py --input datasets/cali_set_rec --bmodel_rec models/BM1684X/ch_PP-OCRv4_rec_fp32.bmodel --dev_id 0 --img_size [[640,48],[320,48]] --char_dict_path datasets/ppocr_keys_v1.txt

reg

1.3 Full-Pipeline Inference Test

The parameters of ppocr_system_opencv.py are:

usage: ppocr_system_opencv.py [-h] [--input INPUT] [--dev_id DEV_ID] [--batch_size BATCH_SIZE] [--bmodel_det BMODEL_DET] [--det_limit_side_len DET_LIMIT_SIDE_LEN] [--bmodel_rec BMODEL_REC] [--img_size IMG_SIZE]
                              [--char_dict_path CHAR_DICT_PATH] [--use_space_char USE_SPACE_CHAR] [--use_beam_search]
                              [--beam_size {1~40}] [--rec_thresh REC_THRESH] [--use_angle_cls]
                              [--bmodel_cls BMODEL_CLS] [--label_list LABEL_LIST] [--cls_thresh CLS_THRESH]

optional arguments:
  -h, --help            show this help message and exit
  --input INPUT         input image directory path
  --dev_id DEV_ID       tpu card id
  --batch_size BATCH_SIZE
                        img num for a ppocr system process launch.
  --bmodel_det BMODEL_DET
                        detector bmodel path
  --det_limit_side_len DET_LIMIT_SIDE_LEN
  --bmodel_rec BMODEL_REC
                        recognizer bmodel path
  --img_size IMG_SIZE   You should set inference size [width,height] manually if using multi-stage bmodel.
  --char_dict_path CHAR_DICT_PATH
  --use_space_char USE_SPACE_CHAR
  --use_beam_search     Enable beam search
  --beam_size {1~40}    Only valid when using beam search, valid range 1~40
  --rec_thresh REC_THRESH
  --use_angle_cls
  --bmodel_cls BMODEL_CLS
                        classifier bmodel path
  --label_list LABEL_LIST
  --cls_thresh CLS_THRESH

A test example:

python3 python/ppocr_system_opencv.py --input datasets/train_full_images_0 \
                           --batch_size 4 \
                           --bmodel_det models/BM1684X/ch_PP-OCRv4_det_fp32.bmodel \
                           --bmodel_rec models/BM1684X/ch_PP-OCRv4_rec_fp32.bmodel \
                           --dev_id 0 \
                           --img_size [[640,48],[320,48]] \
                           --char_dict_path datasets/ppocr_keys_v1.txt

After execution, the predicted fields are printed; the predicted visualization results are saved under the results/inference_results folder, and the inference results are saved under results/ppocr_system_results_b4.json.

omni

regseg

2. C++ Example

1. Cross-Compilation Environment Setup

1.1 Build Environment

C++ programs need their dependencies compiled to run on the board. To reduce load on the edge device, we use an x86 Linux environment for cross-compilation.

Two ways to set up the cross-compilation environment are provided:

(1) Install the cross-compilation toolchain via apt:

If your system's libc version matches the target SoC platform's libc version (check with ldd --version), you can install with:

sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu

To uninstall:

sudo apt remove cpp-*-aarch64-linux-gnu

If your environment does not meet the above requirements, method (2) is recommended.

(2) Set up the cross-compilation environment via Docker:

You can use the Docker image we provide — stream_dev.tar — as the cross-compilation environment.

If you are using Docker for the first time, run the following commands to install and configure it (only needed once):

sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker

Load the image from the downloaded image directory:

docker load -i stream_dev.tar

You can view the loaded image with docker images; it is named stream_dev:latest by default.

Create a container:

docker run --privileged --name stream_dev -v $PWD:/workspace  -it stream_dev:latest
# stream_dev只是举个名字的例子, 请指定成自己想要的容器的名字

The workspace directory inside the container is mounted to the host directory where you ran docker run; you can compile the project inside this container. The workspace directory is under the root directory, and changes in it are mapped to the corresponding files in the local directory.

Note: When creating the container, you must be in the parent directory of soc-sdk (the dependency build environment) or above.

1.2 Package Dependency Files

  1. Package libsophon

    For libsophon_soc_x.y.z_aarch64.tar.gz (where x.y.z is the version number), decompress it.

    # 创建依赖文件的根目录
mkdir -p soc-sdk
# 解压libsophon_soc_x.y.z_aarch64.tar.gz
tar -zxf libsophon_soc_${x.y.z}_aarch64.tar.gz
# 将相关的库目录和头文件目录拷贝到依赖文件根目录下
cp -rf libsophon_soc_${x.y.z}_aarch64/opt/sophon/libsophon-${x.y.z}/lib soc-sdk
cp -rf libsophon_soc_${x.y.z}_aarch64/opt/sophon/libsophon-${x.y.z}/include soc-sdk

  2. Package sophon-ffmpeg and sophon-opencv

    For sophon-mw-soc_x.y.z_aarch64.tar.gz (where x.y.z is the version number), decompress it.

    # 解压sophon-mw-soc_x.y.z_aarch64.tar.gz
tar -zxf sophon-mw-soc_${x.y.z}_aarch64.tar.gz
# 将ffmpeg和opencv的库目录和头文件目录拷贝到soc-sdk目录下
cp -rf sophon-mw-soc_${x.y.z}_aarch64/opt/sophon/sophon-ffmpeg_${x.y.z}/lib soc-sdk
cp -rf sophon-mw-soc_${x.y.z}_aarch64/opt/sophon/sophon-ffmpeg_${x.y.z}/include soc-sdk
cp -rf sophon-mw-soc_${x.y.z}_aarch64/opt/sophon/sophon-opencv_${x.y.z}/lib soc-sdk
cp -rf sophon-mw-soc_${x.y.z}_aarch64/opt/sophon/sophon-opencv_${x.y.z}/include soc-sdk
1.3 Perform Cross Compilation

After the cross-compilation environment is set up, use the cross-compilation toolchain to build the executable:

cd cpp/ppocr_bmcv
mkdir build && cd build
#请根据实际情况修改-DSDK的路径，需使用绝对路径。
cmake -DTARGET_ARCH=soc -DSDK=/workspace/soc-sdk/ ..
make

After the build, a .soc file is generated in the corresponding directory, e.g. cpp/ppocr_bmcv/ppocr_bmcv.soc; this file is also provided and can be used directly.

2. Inference Test

Copy the cross-compiled executable and the required model and test data to the SoC platform (i.e. the BM1684X development board) for testing.

Parameter Description

The executable has a default set of parameters; pass parameters according to your actual situation. The parameters of ppocr_bmcv.soc are:

Usage: ppocr_bmcv.soc [params]

        --batch_size (value:4)
                ppocr system batchsize
        --beam_size (value:3)
                beam size, default 3, available 1-40, only valid when using beam search
        --bmodel_cls (value:../../models/BM1684X/ch_PP-OCRv3_cls_fp32.bmodel)
                cls bmodel file path, unsupport now.
        --bmodel_det (value:../../models/BM1684X/ch_PP-OCRv4_det_fp32.bmodel)
                det bmodel file path
        --bmodel_rec (value:../../models/BM1684X/ch_PP-OCRv4_rec_fp32.bmodel)
                rec bmodel file path
        --dev_id (value:0)
                TPU device id
        --help (value:true)
                print help information.
        --input (value:../../datasets/cali_set_det)
                input path, images directory
        --labelnames (value:../../datasets/ppocr_keys_v1.txt)
                class names file path
        --rec_thresh (value:0.5)
                recognize threshold
        --use_beam_search (value:false)
                beam search trigger
Image Test

The following is an image test example; testing an entire image folder is supported.

#文件加上可执行权限
chmod 755 cpp/ppocr_bmcv/ppocr_bmcv.soc
#执行文件
./cpp/ppocr_bmcv/ppocr_bmcv.soc --input=datasets/train_full_images_0 \
                  --batch_size=4 \
                  --bmodel_det=models/BM1684X/ch_PP-OCRv4_det_fp32.bmodel \
                  --bmodel_rec=models/BM1684X/ch_PP-OCRv4_rec_fp32.bmodel \
                  --labelnames=datasets/ppocr_keys_v1.txt

After the test, predicted images are saved under results/images, and the prediction results are saved under results/. The prediction results, inference time, and other information are also printed.

CPPreg