Hybrid Vision Algo API

Overview

Hybrid vision algo is a high-performance algorithm library designed for event cameras, providing a range of advanced event-denoise algorithms. This document details the usage of all public APIs in the library.

For terminology and data formats see Glossary and Data Formats; for algorithm selection and samples see Samples Overview.

Namespaces

All algorithms live under the Shimeta::Algorithm namespace, further divided by functional module:

Shimeta::Algorithm::Denoise - denoise algorithm module
Shimeta::Algorithm::CV - computer-vision module
Shimeta::Algorithm::CV3D - 3D-vision module
Shimeta::Algorithm::Restoration - image-restoration module

Denoise algorithm module (Denoise)

1. DoubleWindowFilter

The double-window filter classifies CD events using two circular buffers.

Class definition

class DoubleWindowFilter {
public:
    explicit DoubleWindowFilter(
        const size_t bufferSize = 36,
        const size_t searchRadius = 9,
        const size_t intThreshold = 1
    );

    void initialize();
    size_t countNearbyEvents(const Metavision::EventCD &event);
    bool evaluate(const Metavision::EventCD &event);
    bool retain(const Metavision::EventCD &event) noexcept;
    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
};

Constructor parameters

bufferSize: circular-buffer size (default: 36)
searchRadius: search radius, the maximum L1 distance for considering nearby events (default: 9)
intThreshold: the minimum number of nearby events to classify an event as real (default: 1)

Main methods

initialize(): initializes the filter
countNearbyEvents(): counts nearby events in the two windows
evaluate(): evaluates whether an event is signal or noise
retain(): inline method that processes a single event
process_events(): processes a batch event vector

Usage example

#include <denoise/double_window_filter.h>

// 创建滤波器
Shimeta::Algorithm::Denoise::DoubleWindowFilter filter(36, 9, 1);
filter.initialize();

// 处理单个事件
Metavision::EventCD event;
bool isSignal = filter.evaluate(event);

// 批量处理
std::vector<Metavision::EventCD> events;
auto filteredEvents = filter.process_events(events);

2. EventFlowFilter

A noise-suppression filter based on event-flow density and flow-velocity features.

Class definition

class EventFlowFilter {
public:
    explicit EventFlowFilter(
        const size_t bufferSize = 100,
        const size_t searchRadius = 1,
        const double floatThreshold = 20.0,
        const int64_t duration = 2000
    );

    void initialize();
    double fitEventFlow(const Metavision::EventCD &event);
    bool evaluate(const Metavision::EventCD &event);
    bool retain(const Metavision::EventCD &event) noexcept;
    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
};

Constructor parameters

bufferSize: buffer size (default: 100)
searchRadius: spatial-neighborhood radius (default: 1)
floatThreshold: flow-velocity threshold (default: 20.0)
duration: time window, in microseconds (default: 2000)

Main methods

initialize(): initializes the filter
fitEventFlow(): computes the event flow velocity
evaluate(): judges whether an event is signal
retain(): inline method for the retain interface
process_events(): processes a batch of events

3. KhodamoradiDenoiser

A classic denoise algorithm based on spatiotemporal neighborhoods, suited to Metavision CD events.

Class definition

class KhodamoradiDenoiser {
public:
    explicit KhodamoradiDenoiser(
        uint16_t width,
        uint16_t height,
        Metavision::timestamp duration = 2000,
        size_t int_threshold = 2
    );

    void initialize();
    bool filter(const Metavision::EventCD &event);
    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
};

Constructor parameters

width: sensor width
height: sensor height
duration: time-window length, in microseconds (default: 2000)
int_threshold: supporting-pixel threshold (default: 2)

Main methods

initialize(): initializes the filter
filter(): filters a single event
process_events(): processes a batch of events

4. MultiLayerPerceptronFilter

A deep-learning filter that classifies events using a pretrained neural network.

Class definition

class MultiLayerPerceptronFilter {
public:
    explicit MultiLayerPerceptronFilter(
        const std::pair<int, int> &resolution,
        const fs::path &modelPath = fs::path(),
        const size_t batchSize = 5000,
        const int64_t duration = 100000,
        const double floatThreshold = 0.8,
        const std::string &device = "cuda:0"
    );

    void initialize();
    bool evaluate(const Metavision::EventCD &event);
    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
};

Constructor parameters

resolution: sensor resolution (width, height)
modelPath: path to the pretrained PyTorch model
batchSize: number of events processed per batch (default: 5000)
duration: time-feature duration, in microseconds (default: 100000)
floatThreshold: neural-network output threshold (default: 0.8)
device: device name ("cpu", "cuda:0", etc.; default: "cuda:0")

Main methods

initialize(): initializes the filter
evaluate(): evaluates whether an event is signal or noise
process_events(): processes a batch of events

Dependency requirements

Requires the PyTorch C++ library
ENABLE_TORCH=ON must be set at compile time

5. ReclusiveEventDenoisor

Reclusive Event Denoisor (RED), supporting batch processing of Metavision event format.

Class definition

class ReclusiveEventDenoisor {
public:
    ReclusiveEventDenoisor(int width, int height, int tau, int n);

    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
    void reset();
};

Constructor parameters

width: sensor width
height: sensor height
tau: time constant, in microseconds
n: spatial-neighborhood radius

Main methods

process_events(): processes a batch of events and returns the denoised events
reset(): resets the internal state

6. TimeSurfaceDenoisor

Denoises events based on the time-surface features of their spatiotemporal neighborhood.

Class definition

class TimeSurfaceDenoisor {
public:
    TimeSurfaceDenoisor(
        int width,
        int height,
        double decay = 20000,
        size_t searchRadius = 1,
        double floatThreshold = 0.2
    );

    void initialize();
    bool evaluate(const Metavision::EventCD &event);
    bool retain(const Metavision::EventCD &event) noexcept;
    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
};

Constructor parameters

width: image width
height: image height
decay: time-decay constant, in microseconds (default: 20000)
searchRadius: search radius (default: 1)
floatThreshold: decision threshold (default: 0.2)

Main methods

initialize(): initializes the surface
evaluate(): judges whether a single event is signal
retain(): inline method that processes a single event
process_events(): processes a batch of events

7. YangNoiseFilter

A noise filter that classifies events using a spatiotemporal-density method.

Class definition

class YangNoiseFilter {
public:
    explicit YangNoiseFilter(
        const int16_t width,
        const int16_t height,
        const int64_t duration = 10000,
        const size_t searchRadius = 1,
        const size_t intThreshold = 2
    );

    void initialize();
    size_t calculateDensity(const Metavision::EventCD &event);
    bool evaluate(const Metavision::EventCD &event);
    bool retain(const Metavision::EventCD &event) noexcept;
    std::vector<Metavision::EventCD> process_events(const std::vector<Metavision::EventCD> &events);
};

Constructor parameters

width: sensor width
height: sensor height
duration: time-window duration, in microseconds (default: 10000)
searchRadius: maximum L1 distance for the spatiotemporal search (default: 1)
intThreshold: minimum number of nearby events to classify an event as real (default: 2)

Main methods

initialize(): initializes the filter
calculateDensity(): computes the spatiotemporal density around an event
evaluate(): evaluates whether an event is signal or noise
retain(): inline method that processes a single event
process_events(): processes a batch event vector

Common interface design

Event types

All algorithms use the Metavision SDK standard event types:

Metavision::EventCD: change-detection event
std::vector<Metavision::EventCD>: event vector

Common method pattern

Most filters follow this interface pattern:

Constructor: accepts algorithm-specific parameters
initialize(): initializes internal state
evaluate(): evaluates a single event
retain(): inline version of single-event processing
process_events(): processes a batch event vector

Performance recommendations

Batch processing: prefer process_events() for batch processing for better performance
Inline methods: for real-time processing, use the retain() inline method
Parameter tuning: adjust algorithm parameters to your specific application scenario
GPU acceleration: for the MLP filter, using a CUDA device significantly improves performance

Build requirements

Base dependencies

C++17-compatible compiler
CMake >= 3.16
Metavision SDK (base, core components)
Eigen3 linear-algebra library

Optional dependencies

PyTorch C++ library (for the MLP filter)
CUDA (GPU acceleration support)

Build options

# 基础编译
cmake ..
make -j$(nproc)

# 启用 PyTorch 支持
cmake -DENABLE_TORCH=ON ..
make -j$(nproc)

Usage examples

Basic usage

#include <denoise/double_window_filter.h>
#include <metavision/sdk/base/events/event_cd.h>

int main() {
    // 创建滤波器
    Shimeta::Algorithm::Denoise::DoubleWindowFilter filter(36, 9, 1);
    filter.initialize();

    // 处理事件
    std::vector<Metavision::EventCD> events;
    // ... 填充事件数据

    auto filteredEvents = filter.process_events(events);

    return 0;
}

Combining multiple algorithms

#include <denoise/double_window_filter.h>
#include <denoise/yang_noise_filter.h>

int main() {
    // 创建多个滤波器
    Shimeta::Algorithm::Denoise::DoubleWindowFilter dwf(36, 9, 1);
    Shimeta::Algorithm::Denoise::YangNoiseFilter ynf(640, 480, 10000, 1, 2);

    dwf.initialize();
    ynf.initialize();

    std::vector<Metavision::EventCD> events;
    // ... 填充事件数据

    // 串联处理
    auto step1 = dwf.process_events(events);
    auto step2 = ynf.process_events(step1);

    return 0;
}

Error handling

Common errors

Model file does not exist: the MLP filter requires a valid model-file path
Device unavailable: falls back to CPU when the CUDA device is unavailable
Out of memory: may occur when processing very large batches of events
Invalid parameters: sensor size, thresholds, and other parameters must be within a reasonable range

Debugging recommendations

Check the validity of the event data
Verify the sensor parameter settings
Monitor memory usage
Use an appropriate batch size

Version information

The current API version is based on HVAlgo v0.1.0; the API may change in subsequent versions. Fully test before using in production.