Algorithm deployment
1. Introduction to SOPHON-D
SOPHON-DEMO is developed based on the SOPHONSDK interface and provides a series of migration routines for mainstream algorithms, including model compilation and quantization based on TPU-NNTC and TPU-MLIR, inference engine migration based on BMRuntime, and pre- and post-processing algorithm migration based on BMCV/OpenCV.
SOPHONSDK is a deep learning SDK customized by Suanneng Technology based on its self-developed deep learning processor. It covers the model optimization and efficient runtime support required in the neural network inference stage, and provides an easy-to-use and efficient full-stack solution for deep learning application development and deployment. It is currently compatible with BM1684/BM1684X/BM1688 (CV186X).
2. Directory structure and description
The examples provided by SOPHON-DEMO are divided into three modules from easy to difficult:tutorial,sample and application.
Warning
tutorialThe module stores some usage examples of basic interfacessampleThe module stores serial examples of some classic algorithms on SOPHONSDK;applicationThe module stores typical applications for some typical scenarios.
| Modules | Link |
|---|---|
| tutorial | LINK1 |
| sample | LINK2 |
| application | LINK3 |
3.版本说明
| 版本 | 说明 |
|---|---|
| 0.2.1 | Improve and fix documentation and code issues, add CV186X support to some routines, adapt YOLOv5 to SG2042, add GroundingDINO and Qwen1_5 to the sample module, StableDiffusionV1_5 supports multiple resolutions, Qwen, Llama2, and ChatGLM3 add web and multi-session modes. Add blend and stitch routines to the tutorial module |
| 0.2.0 | Improve and fix documentation and code issues, add application and tutorial modules, add ChatGLM3 and Qwen routines, add web ui to SAM, adapt BERT, ByteTrack, and C3D to BM1688, change the original YOLOv8 to YOLOv8_det and add cpp post-processing acceleration method, optimize auto_test of common routines, and update TPU-MLIR installation method to pip |
| 0.1.10 | Fixed document and code issues, added ppYoloe, YOLOv8_seg, StableDiffusionV1.5, SAM, refactored yolact, adapted CenterNet, YOLOX, and YOLOv8 to BM1688, added BM1688 performance data for YOLOv5, ResNet, PP-OCR, and DeepSORT, and provided C++ cross-compilation methods for WeNet |
| 0.1.9 | Fix document and code issues, add segformer, YOLOv7, Llama2 routines, refactor YOLOv34, adapt YOLOv5, ResNet, PP-OCR, DeepSORT, LPRNet, RetinaFace, YOLOv34, WeNet to BM1688, accelerate OpenPose post-processing, add compilation methods and int8/int4 quantization to chatglm2. |
| 0.1.8 | Improve and fix documentation and code issues, add BERT, ppYOLOv3, ChatGLM2, refactor YOLOX, add beam search to PP-OCR, add tpu-kernel post-processing acceleration to OpenPose, and update the SFTP download method. |
| 0.1.7 | Fix document and other issues, some routines support BM1684 mlir, refactor PP-OCR, CenterNet routines, add sail support for YOLOv5 |
| 0.1.6 | Fixed document issues and added ByteTrack, YOLOv5_opt, WeNet routines |
| 0.1.5 | Fixed document and other issues, added DeepSORT routines, refactored ResNet and LPRNet routines |
| 0.1.4 | Fixed document issues and added C3D and YOLOv8 routines |
| 0.1.3 | Added OpenPose routines, refactored YOLOv5 routines (including adapting to arm PCIe, supporting TPU-MLIR to compile BM1684X models, using ffmpeg components to replace opencv decoding, etc.) |
| 0.1.2 | Fix document and other issues, refactor SSD related routines, use ffmpeg components to replace opencv decoding in LPRNet/cpp/lprnet_bmcv |
| 0.1.1 | Fix documentation and other issues, refactor LPRNet/cpp/lprnet_bmcv using BMNN related classes |
| 0.1.0 | Provides 10 routines including LPRNet, adapted to BM1684X (x86 PCIe, SoC), BM1684 (x86 PCIe, SoC) |
4. Environmental Dependence
SOPHON-DEMO mainly relies on TPU-MLIR, TPU-NNTC, LIBSOPHON, SOPHON-FFMPEG, SOPHON-OPENCV, and SOPHON-SAIL. The version requirements are as follows:
| SOPHON-DEMO | TPU-MLIR | TPU-NNTC | LIBSOPHON | SOPHON-FFMPEG | SOPHON-OPENCV | SOPHON-SAIL | release date |
|---|---|---|---|---|---|---|---|
| 0.2.0 | >=1.6 | >=3.1.7 | >=0.5.0 | >=0.7.3 | >=0.7.3 | >=3.7.0 | >=23.10.01 |
| 0.1.10 | >=1.2.2 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.7.0 | >=23.07.01 |
| 0.1.9 | >=1.2.2 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.7.0 | >=23.07.01 |
| 0.1.8 | >=1.2.2 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.6.0 | >=23.07.01 |
| 0.1.7 | >=1.2.2 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.6.0 | >=23.07.01 |
| 0.1.6 | >=0.9.9 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.4.0 | >=23.05.01 |
| 0.1.5 | >=0.9.9 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.4.0 | >=23.03.01 |
| 0.1.4 | >=0.7.1 | >=3.1.5 | >=0.4.4 | >=0.5.1 | >=0.5.1 | >=3.3.0 | >=22.12.01 |
| 0.1.3 | >=0.7.1 | >=3.1.5 | >=0.4.4 | >=0.5.1 | >=0.5.1 | >=3.3.0 | - |
| 0.1.2 | Not support | >=3.1.4 | >=0.4.3 | >=0.5.0 | >=0.5.0 | >=3.2.0 | - |
| 0.1.1 | Not support | >=3.1.3 | >=0.4.2 | >=0.4.0 | >=0.4.0 | >=3.1.0 | - |
| 0.1.0 | Not support | >=3.1.3 | >=0.3.0 | >=0.2.4 | >=0.2.4 | >=3.1.0 | - |
Warning
- The requirements for versions may vary among different routines. Specifically, please refer to the README of each routine. It may be necessary to install other third-party libraries.
- The SDKs corresponding to BM1688/CV186X and BM1684X/BM1684 are not the same set. They have not been released on the official website yet. Please contact the technical staff to obtain them.
5. Technical information
Tips
Please go to Suanneng official website for technical information to obtain relevant documents, materials and video tutorials.
6. Community
Tips
The Suanneng Community encourages developers to communicate and learn together. Developers can communicate and learn through the following channels.
Suanneng Community Website: https://www.sophgo.com/
Sophgo Developer Forum: https://developer.sophgo.com/forum/index.html