Hybrid Vision Algo

Key features

• 🚀 High performance: optimized C++17 implementation, supports real-time event-stream processing
• 🧠 Multiple denoise algorithms: includes both traditional and deep-learning methods
• 🔧 Easy integration: provides CMake config and pkg-config support
• 📦 Modular design: clean module separation, easy to extend and maintain
• 🎯 Practical samples: rich sample code and documentation

Algorithm modules

Denoise algorithms

1. Double Window Filter

   • Classifies events using two circular buffers
   • Suited to real-time processing in low-noise environments

2. Event Flow Filter

   • Noise suppression based on event-flow density and flow-velocity features
   • Suited to denoising motion scenes

3. Khodamoradi Denoiser

   • A classic denoise algorithm based on spatiotemporal neighborhoods
   • A general-purpose solution balancing performance and effect

4. Reclusive Event Denoiser

   • A recursive event-processing algorithm
   • Suited to complex noise environments

5. Time Surface Denoiser

   • A denoise method based on time surfaces
   • Preserves event temporal characteristics

6. Yang Noise Filter

   • A noise-filter algorithm proposed by Yang et al.
   • Efficient real-time processing

7. MLP (Multi-Layer Perceptron) Filter (optional)

   • Smart denoising based on deep learning
   • Requires PyTorch support
   • Suited to high-precision denoising in complex scenes

Computer Vision (CV)

• Basic image-processing algorithms
• Feature extraction and matching

3D Vision (CV3D)

• Stereo-vision algorithms
• Depth estimation
• 3D reconstruction

Image Restoration

• Image-enhancement algorithms
• Deblurring and super-resolution

System requirements

Required dependencies

• Ubuntu == 22.04
• CMake >= 3.16
• C++17-compatible compiler
• OpenEB SDK
• Eigen3

Optional dependencies

• PyTorch (for the MLP filter)
• CUDA (GPU acceleration support)

Installation guide

1. Install dependencies

Ubuntu/Debian

# 安装基础依赖
sudo apt update
sudo apt install cmake build-essential libeigen3-dev

#安装HV Toolkits
# 拉取子仓库
git submodule update --init
# 进入子仓库目录
cd external/shimetapi_hybrid_vision_toolkit/
sudo ./install_libs.sh

# 安装 Openeb
curl -L https://propheseeai.jfrog.io/artifactory/api/security/keypair/prophesee-gpg/public >/tmp/propheseeai.jfrog.op.asc
sudo cp /tmp/propheseeai.jfrog.op.asc /etc/apt/trusted.gpg.d
sudo add-apt-repository 'https://propheseeai.jfrog.io/artifactory/openeb-debian/'
sudo apt update
sudo apt -y install metavision-openeb

Optional: PyTorch support

# 下载 LibTorch
wget https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.7.1%2Bcpu.zip
unzip libtorch-cxx11-abi-shared-with-deps-2.7.1+cpu.zip
export CMAKE_PREFIX_PATH=/path/to/libtorch:$CMAKE_PREFIX_PATH

2. Build and install

mkdir build && cd build

# 基础编译
cmake ..
make -j$(nproc)

# 启用 PyTorch 支持（可选）
cmake -DENABLE_TORCH=ON ..
make -j$(nproc)

# 安装到系统
sudo make install

3. Build options

Option	Default	Description
ENABLE_TORCH	OFF	Enable PyTorch support
BUILD_SAMPLES	OFF	Build the sample programs
BUILD_TESTING	OFF	Build the test programs
CMAKE_BUILD_TYPE	Release	Build type (Debug/Release)

Usage

CMake integration

find_package(HVAlgo REQUIRED)

add_executable(my_app main.cpp)
target_link_libraries(my_app HVAlgo::hv_algo)

pkg-config integration

# 编译
g++ -o my_app main.cpp `pkg-config --cflags --libs hv_algo`

Basic usage example

#include <denoise/double_window_filter.h>
#include <metavision/sdk/base/events/event_cd.h>

int main() {
    // 创建双窗口滤波器
    Shimeta::Algorithm::Denoise::DoubleWindowFilter filter(36, 9, 1);

    // 处理事件
    Metavision::EventCD event;
    bool is_signal = filter.evaluate(event);

    if (is_signal) {
        // 处理有效事件
    }

    return 0;
}

Sample programs

The project provides rich sample programs showing how to use each algorithm:

# 编译示例
cd samples/with_hv_toolkit
mkdir build && cd build
cmake ..
make
# 运行示例
./dwf_denoising ../../../data/events.raw

Program screenshot

Available samples

• dwf_denoising: double-window filter denoising sample
• event_flow_denoising: event-flow filter sample
• khodamoradi_denoising: Khodamoradi denoiser sample
• mlpf_denoising: MLP filter sample (requires PyTorch)
• re_denoising: reclusive event denoiser sample
• ts_denoising: time-surface denoiser sample
• y_denoising: Yang filter sample

Project structure

hv_algo/
├── include/                 # 头文件
│   ├── denoise/            # 去噪算法
│   ├── cv/                 # 计算机视觉
│   ├── cv3d/               # 三维视觉
│   └── restoration/        # 图像恢复
├── src/                    # 源代码
├── samples/                # 示例程序
│   ├── with_metavision/    # Openeb SDK 示例
│   └── with_hv_toolkit/    # HV Toolkit 示例
├── cmake/                  # CMake 配置文件
├── data/                   # 测试数据
├── models/                 # 预训练模型
└── external/               # 外部依赖

Acknowledgements

• OpenEB SDK - event-camera development framework
• Eigen - linear-algebra library
• PyTorch - deep-learning framework

Contact

Open-source hardware website: https://www.shimetapi.cn (domestic) / https://www.shimetapi.com (overseas) Online technical docs: https://forum.shimetapi.cn/wiki/zh/ Online technical community: https://forum.shimetapi.cn

Note: this project is still under active development and the API may change. Fully test it before using it in production.