Event Processing (Denoising)

Denoise algorithms run at the application layer. Event capture itself supports x86_64 USB and S100/X5 MIPI; whether a specific algorithm can run on an ARM board depends on the cross-compilation and dependencies of the Algo component — it is not automatically guaranteed by the HV Toolkit.

Task goal

An event camera produces a small amount of background activity noise even in a static scene, mixed in with real motion events. Denoising judges whether each event is signal or noise and discards the noise; it is the first step of almost every event-camera algorithm pipeline. This page covers the common interface pattern and algorithm selection.

Classes and concepts involved

Denoise algorithms all live under the Shimeta::Algorithm::Denoise namespace (see Algo API), with a unified interface pattern:

Construct: pass in the algorithm parameters;
initialize(): initialize the internal state;
process_events(events): batch-process and return the denoised event vector;
(some algorithms) evaluate(event): evaluate a single event.

Choosing among the 7 denoise algorithms

Algorithm	Core idea	Use case	Compute
`DoubleWindowFilter`	Two time windows count nearby events	General starter, low compute	Low
`EventFlowFilter`	Event flow-velocity features	Scenes with clear motion	Medium
`KhodamoradiDenoiser`	Classic spatiotemporal neighborhood	Offline, comparison baseline	Low
`YangNoiseFilter`	Spatiotemporal density	General motion scenes	Medium
`TimeSurfaceDenoisor`	Time-surface features	Texture/edge-rich scenes	Medium
`ReclusiveEventDenoisor`	Recursive spatiotemporal	When state continuity is needed	Medium
`MultiLayerPerceptronFilter`	Neural-network classification	High precision (needs PyTorch / optional GPU)	High

The runnable sample for each algorithm is listed in Samples Overview; selection guidance is at Hybrid Vision Algo.

Single-algorithm processing

Taking the double-window filter as an example. The current public Algo API signature is still Metavision::EventCD, so it is not a native HV Toolkit v2.0 type; the application needs to provide its own conversion layer between Shimeta::EventCD and the Algo input type.

#include <shimetapi/hv/camera.h>
#include <shimetapi/codec/mipi_raw8_codec.h>
#include <denoise/double_window_filter.h>
#include <metavision/sdk/base/events/event_cd.h>
#include <vector>

Shimeta::Algorithm::Denoise::DoubleWindowFilter dwf(36, 9, 1);
dwf.initialize();

// events is the Algo input events output by the conversion layer
std::vector<Metavision::EventCD> events;

auto filtered = dwf.process_events(events);
// filtered 即去噪后的事件

When wired into a v2.0 camera, first get the raw bytes and decode them with the corresponding codec, then denoise each batch of events. USB uses Evt2Decoder/Evt3Decoder; MipiHvs on S100/X5 uses MipiRaw8Decoder.

// Wiring sketch: convert_to_algo_events must be implemented by the application per the Algo SDK definition
Shimeta::codec::MipiRaw8Decoder dec; // S100/X5; use Evt2Decoder for USB
cam.SetFrameCallback([&dwf, &dec](const Shimeta::Frame& frame) {
    std::vector<Shimeta::EventCD> shimeta_events;
    dec.Decode(frame.evs.data, frame.evs.size, shimeta_events);
    // Convert shimeta_events into the Metavision::EventCD required by the Algo API
    // convert_to_algo_events is an application-defined adapter function (sketch only)
    std::vector<Metavision::EventCD> events = convert_to_algo_events(shimeta_events);
    auto filtered = dwf.process_events(events);
    // 用 filtered 做后续处理
});

Constructor parameter meanings: bufferSize (buffer length), searchRadius (neighborhood radius), intThreshold (minimum nearby events to judge as signal). Each algorithm's parameters are detailed in Algo API.

Chaining multiple algorithms

The strengths of different algorithms can stack — feed the previous stage's output as the next stage's input:

#include <denoise/double_window_filter.h>
#include <denoise/yang_noise_filter.h>

Shimeta::Algorithm::Denoise::DoubleWindowFilter dwf(36, 9, 1);
Shimeta::Algorithm::Denoise::YangNoiseFilter ynf(768, 608, 10000, 1, 2);
dwf.initialize();
ynf.initialize();

std::vector<Metavision::EventCD> events;

auto step1 = dwf.process_events(events);   // 第一级
auto step2 = ynf.process_events(step1);    // 第二级

Chaining drops events stage by stage

Each stage discards some events by its own criterion. The more stages, the lower the retention — balance "clean" against "informationally complete", and tune per scene.

Build requirements

C++17, CMake ≥ 3.16, Metavision SDK, Eigen3; enabling MultiLayerPerceptronFilter additionally requires the PyTorch C++ library (cmake -DENABLE_TORCH=ON ..). See Algo API — Build requirements.