Sophgo SDK Development

1. SDK Introduction

SOPHON SDK is a deep-learning SDK customized by Sophgo for BM1684 and BM1684X. It covers model optimization and efficient runtime support required for the neural-network inference phase, providing an easy-to-use, efficient, full-stack solution for deep-learning application development and deployment.

1. Basic Toolkit

The basic toolkit includes:

tpu-nntc: performs offline compilation and optimization of neural-network models trained under third-party deep-learning frameworks, producing the BModel required at runtime. It currently supports Caffe, Darknet, MXNet, ONNX, PyTorch, PaddlePaddle, TensorFlow, etc.
libsophon: provides BMCV, BMRuntime, BMLib, and other libraries that drive the VPP and the intelligent-vision deep-learning processor module to perform image processing, tensor operations, model inference, and other operations for deep-learning application development.
sophon-mw: wraps SOPHON-OpenCV, SOPHON-FFmpeg, and other libraries that drive the VPU, JPU, and other hardware to support RTSP stream and GB28181 stream parsing and video/image codec acceleration for deep-learning application development.
sophon-sail: provides high-level Python/C++ interfaces that wrap the lower-level BMRuntime, BMCV, sophon-mw, and BMLib interfaces for deep-learning application development.

2. Advanced Toolkit

The advanced toolkit includes:

tpu-mlir: provides a complete toolchain for the Tensor Processing Unit compiler project, capable of converting pretrained neural networks from different frameworks into BModel binary files that run efficiently on the Sophgo intelligent-vision deep-learning processor. Currently directly supports tflite, onnx, and Caffe.
tpu-perf: provides a complete toolkit for model performance and accuracy validation.
tpu-kernel: the low-level development interface of the Tensor Processing Unit; it can invoke dedicated instructions to accelerate deep-learning business logic, or general instructions to accelerate custom algorithms.

For details, refer to the Sophgo documentation.

Note

sophon-demo provides comprehensive examples for single-model or scenario-based use under x86 and SoC environments, for reference during deep-learning application development.
sophon-pipeline provides a simple pipeline-based high-performance inference framework that runs preprocessing, inference, and post-processing on three separate threads to maximize parallelism, for reference during deep-learning application development.

3. SOPHON SDK Folder Directory

Folder	Notes
libsophon	Libraries for image processing, tensor operations, and model inference accelerated by SOPHON device hardware
sophon-mw	Multimedia libraries accelerated by SOPHON device hardware
tpu-kernel	Low-level development interface
tpu-mlir	Tensor Processing Unit compiler toolchain
tpu-nntc	Tensor Processing Unit compiler toolchain
tpu-perf	Model performance and accuracy validation toolkit
sophon-pipeline	Pipeline-based high-performance inference framework
sophon-img	SoC mode installation package
sophon-demo	Comprehensive examples for single models or scenarios
sophon-sail	Interface library that wraps low-level interfaces with C++/Python APIs
sophon-rpc	On PCIe card products, offloads some compute tasks to the ARM on the card
docker-image	Docker image

4. SOPHON SDK Folder Directory After Decompression

As follows:

    SOPHONSDK
    ├── docker-image
    │   ├── md5sum.txt
    │   ├── sophgo-tpuc_dev-v2.1-82d75f5c633d.tar.bz2
    │   └── sophgo-tpuc_dev-v2.2-f72913f3a83d.tar.bz2
    ├── libsophon_20240108_210425
    │   ├── BMCV开发参考手册.pdf
    │   ├── BMCV_Technical_Reference_Manual.pdf
    │   ├── BMLIB开发参考手册.pdf
    │   ├── BMLib_Technical_Reference_Manual.pdf
    │   ├── BMRUNTIME开发参考手册.pdf
    │   ├── BMRuntime Technical Reference Manual.pdf
    │   ├── centos
    │   ├── libsophon_0.4.9-LTS_aarch64.tar.gz
    │   ├── libsophon_0.4.9-LTS_loongarch64.tar.gz
    │   ├── libsophon_0.4.9-LTS_x86_64.tar.gz
    │   ├── LIBSOPHON使用手册.pdf
    │   ├── libsophon_dockerfile
    │   ├── libsophon.MD5
    │   ├── LIBSOPHON_User_Guide.pdf
    │   ├── release_version.txt
    │   ├── sophon-driver_0.4.9-LTS_amd64.deb
    │   ├── sophon-driver_0.4.9-LTS_arm64.deb
    │   ├── sophon-libsophon_0.4.9-LTS_amd64.deb
    │   ├── sophon-libsophon_0.4.9-LTS_arm64.deb
    │   ├── sophon-libsophon-dev_0.4.9-LTS_amd64.deb
    │   └── sophon-libsophon-dev_0.4.9-LTS_arm64.deb
    ├── sophon-demo_20231116_085900
    │   ├── release_version.txt
    │   ├── sophon-demo.MD5
    │   ├── sophon-demo_v0.1.8_dbb4632_20231116
    │   └── sophon-demo_v0.1.8_dbb4632_20231116.tar.gz
    ├── sophon-img_20240116_212937
    │   ├── bsp-debs
    │   ├── bsp_update.tgz
    │   ├── libsophon_soc_0.4.9-LTS_aarch64.tar.gz
    │   ├── release_version.txt
    │   ├── sdcard.tgz
    │   ├── SOPHON BSP开发参考手册.pdf
    │   ├── SOPHON_BSP_Technical_Reference_Manual.pdf
    │   ├── sophon-img.MD5
    │   ├── system.tgz
    │   └── tftp.tgz
    ├── sophon-mw_20240116_152830
    │   ├── MULTIMEDIA常见问题手册.pdf
    │   ├── MULTIMEDIA开发参考手册.pdf
    │   ├── MULTIMEDIA使用手册.pdf
    │   ├── Multimedia FAQ.pdf
    │   ├── Multimedia Technical Reference Manual.pdf
    │   ├── Multimedia User Guide.pdf
    │   ├── release_version.txt
    │   ├── sophon-mw_0.8.0_aarch64.tar.gz
    │   ├── sophon-mw_0.8.0_loongarch64.tar.gz
    │   ├── sophon-mw_0.8.0_x86_64.tar.gz
    │   ├── sophon-mw.MD5
    │   ├── sophon-mw-soc_0.8.0_aarch64.tar.gz
    │   ├── sophon-mw-soc-sophon-ffmpeg_0.8.0_arm64.deb
    │   ├── sophon-mw-soc-sophon-ffmpeg-dev_0.8.0_arm64.deb
    │   ├── sophon-mw-soc-sophon-opencv_0.8.0_arm64.deb
    │   ├── sophon-mw-soc-sophon-opencv-dev_0.8.0_arm64.deb
    │   ├── sophon-mw-soc-sophon-sample_0.8.0_arm64.deb
    │   ├── sophon-mw-sophon-ffmpeg_0.8.0_amd64.deb
    │   ├── sophon-mw-sophon-ffmpeg_0.8.0_amd64.rpm
    │   ├── sophon-mw-sophon-ffmpeg_0.8.0_arm64.deb
    │   ├── sophon-mw-sophon-ffmpeg_0.8.0_arm64.rpm
    │   ├── sophon-mw-sophon-ffmpeg_0.8.0_loongarch64.deb
    │   ├── sophon-mw-sophon-ffmpeg-dev_0.8.0_amd64.deb
    │   ├── sophon-mw-sophon-ffmpeg-dev_0.8.0_amd64.rpm
    │   ├── sophon-mw-sophon-ffmpeg-dev_0.8.0_arm64.deb
    │   ├── sophon-mw-sophon-ffmpeg-dev_0.8.0_arm64.rpm
    │   ├── sophon-mw-sophon-ffmpeg-dev_0.8.0_loongarch64.deb
    │   ├── sophon-mw-sophon-opencv_0.8.0_amd64.deb
    │   ├── sophon-mw-sophon-opencv_0.8.0_arm64.deb
    │   ├── sophon-mw-sophon-opencv_0.8.0_loongarch64.deb
    │   ├── sophon-mw-sophon-opencv-abi0_0.8.0_amd64.rpm
    │   ├── sophon-mw-sophon-opencv-abi0_0.8.0_arm64.rpm
    │   ├── sophon-mw-sophon-opencv-abi0-dev_0.8.0_amd64.rpm
    │   ├── sophon-mw-sophon-opencv-abi0-dev_0.8.0_arm64.rpm
    │   ├── sophon-mw-sophon-opencv-dev_0.8.0_amd64.deb
    │   ├── sophon-mw-sophon-opencv-dev_0.8.0_arm64.deb
    │   ├── sophon-mw-sophon-opencv-dev_0.8.0_loongarch64.deb
    │   ├── sophon-mw-sophon-sample_0.8.0_amd64.deb
    │   ├── sophon-mw-sophon-sample_0.8.0_amd64.rpm
    │   ├── sophon-mw-sophon-sample_0.8.0_arm64.deb
    │   ├── sophon-mw-sophon-sample_0.8.0_arm64.rpm
    │   └── sophon-mw-sophon-sample_0.8.0_loongarch64.deb
    ├── sophon-rpc_20231208_174527
    │   ├── release_version.txt
    │   ├── sophon-rpc_3.2.0-LTS_amd64.deb
    │   ├── sophon-rpc_3.2.0-LTS_amd64.rpm
    │   ├── sophon-rpc_3.2.0-LTS_arm64.deb
    │   ├── sophon-rpc_3.2.0-LTS_arm64.rpm
    │   ├── sophon-rpc_3.2.0-LTS.tar.gz
    │   ├── sophon-rpc使用指南.pdf
    │   └── sophon-rpc.MD5
    ├── sophon-sail_20231116_085400
    │   ├── release_version.txt
    │   ├── sophon-sail_3.7.0.tar.gz
    │   ├── sophon-sail_en.pdf
    │   ├── sophon-sail.MD5
    │   └── sophon-sail_zh.pdf
    ├── sophon-stream_20231116_011200
    │   ├── release_version.txt
    │   ├── sophon-stream.MD5
    │   └── sophon-stream_v0.0.4-rc4_10a8ed8_20231115.tar.gz
    ├── tpu-kernel_20231130_055600
    │   ├── release_version.txt
    │   ├── tpu-kernel-1684x_v3.1.7-520261d8-231130.tar.gz
    │   └── tpu-kernel.MD5
    ├── tpu-mlir_20231116_054500
    │   ├── release_version.txt
    │   ├── tpu-mlir.MD5
    │   └── tpu-mlir_v1.3.140-g3180ff37-20231116.tar.gz
    ├── tpu-nntc_20231130_054100
    │   ├── release_version.txt
    │   ├── tpu-nntc.MD5
    │   └── tpu-nntc_v3.1.9-29fa956b-231130.tar.gz
    └── tpu-perf_v1.2.37
        ├── tpu_perf-1.2.37-py3-none-manylinux2014_aarch64.whl
        ├── tpu_perf-1.2.37-py3-none-manylinux2014_x86_64.whl
        ├── tpu_perf-v1.2.37.tar.gz
        └── tpu-perf-v1.2.37.zip

5. Main SDK Modules

Hardware driver and runtime library LIBSOPHON: contains BMCV, BMRuntime, BMLib, and other libraries that drive the VPP and the intelligent-vision deep-learning processor to perform image processing, tensor operations, and model inference.
Multimedia library SOPHON-MW: SOPHON-OpenCV and SOPHON-FFmpeg accelerated by SOPHON device hardware; supports RTSP stream and GB28181 stream parsing and video/image codec.
Model compilation and quantization toolchain TPU-MLIR: supports model conversion from Caffe, TFLite, ONNX, and other frameworks; supports model quantization: original model -> MLIR Model -> FP32 BModel, or original model -> MLIR Model -> cali_table -> INT8 BModel; also provides the model_deploy.py quantization script.
Model compilation and quantization toolchain TPU-NNTC: supports model conversion from Caffe, TensorFlow, Pytorch, MXNet, Darknet, PaddlePaddle, ONNX, and other frameworks; supports model quantization: original model -> FP32 UModel -> INT8 UModel -> INT8 BModel; also provides the auto-cali automatic quantization tool.
Tensor operations and image processing library BMCV: color space conversion, scaling, affine transformation, projection transformation, linear transformation, drawing boxes, JPEG encoding, BASE64 encoding, NMS, sorting, feature matching.
Device management BMLib: base interface: device Handle management, memory management, data transfer, API send and synchronization, A53 enable, etc.
Sophgo deep-learning acceleration library SAIL: high-level Python/C++ interfaces that wrap the lower-level BMRuntime, BMCV, sophon-mw, and other libraries.
Custom operator high-level programming library BMLang: a C++-based high-level programming library for the SOPHON intelligent-vision deep-learning processor; decoupled from hardware details, requires no knowledge of hardware architecture; write code using tensor data (bmlang::Tensor) and compute operations (bmlang::Operator), then use bmlang::compile or bmlang::compile_with_check to generate a BModel that runs on the intelligent-vision deep-learning processor; also supports using the ARM processor in BM168X to implement operators not yet supported by the intelligent-vision deep-learning processor.
Algorithm parallel acceleration programming library TPUKernel: a low-level programming interface based on the atomic operation interface of SOPHON BM1684 and BM1684X; requires familiarity with device hardware architecture and the instruction set.
Model performance and accuracy validation tool TPUPerf: performs performance analysis and accuracy validation on models.

2. SDK Acquisition and Installation

See

Obtain v24.04.01 SDK

Install v24.04.01 SDK

3. More Materials

See

Sophgo materials