Display & Visualization

Task goal

Events are sparse asynchronous points (x, y, polarity, t) that the eye cannot directly "see" as motion. The most common visualization method is to accumulate events over a time window into one frame and display it. This page covers accumulation display implemented with OpenCV, plus no-code visualization options.

Concepts involved

  • Accumulate into a frame (accumulation): draw the events within a time window onto one image by their coordinates, and the motion trajectory emerges. The window length is called accumulation time — see Fundamentals.
  • Polarity coloring: EventCD.polarity of true (brighten) and false (darken) are shown in different colors.
  • Sensor size comes from Frame.width / Frame.height (768×608 on both USB and MIPI).

Accumulate events into a frame + OpenCV display

Pick up frames → decode events → draw them onto a cv::Mat by coordinates; the main loop refreshes and clears at a fixed period (the clearing period is the accumulation time):

#include <shimetapi/hv/camera.h>
#include <shimetapi/hv/device_config.h>
#include <shimetapi/codec/evt2_codec.h>
#include <opencv2/opencv.hpp>

int main() {
    Shimeta::hv::Camera cam;
    Shimeta::hv::DeviceConfig cfg;
    cfg.backend   = Shimeta::hv::Backend::Usb;
    cfg.vendor_id = 0x1d6b;
    cfg.product_id = 0x0105;
    cfg.event_fmt = Shimeta::hv::EventFormat::Evt2;
    cam.Init(cfg);
    if (!cam.StartStream()) return 1;

    cv::Mat frame(608, 768, CV_8UC3, cv::Scalar(0, 0, 0));
    Shimeta::codec::Evt2Decoder dec;   // Stateful; construct outside the loop
    Shimeta::Frame f;

    while (true) {
        if (cam.GetFrame(f, 33)) {
            std::vector<Shimeta::EventCD> events;
            dec.Decode(f.evs.data, f.evs.size, events);
            for (const auto& e : events) {
                if (e.polarity) frame.at<cv::Vec3b>(e.y, e.x) = {0, 0, 255};   // 变亮：红
                else            frame.at<cv::Vec3b>(e.y, e.x) = {0, 255, 0};   // 变暗：绿
            }
        }
        cv::imshow("events", frame);
        if (cv::waitKey(33) == 27) break;   // ESC 退出
        frame = cv::Scalar(0, 0, 0);        // 清空 = 固定 accumulation time
    }
    cam.StopStream();
    cam.Destroy();
    return 0;
}

Two ways to tune accumulation time

  • Clear period (the code above): clear every frame at a fixed interval; the refresh interval is the accumulation time. Simple and intuitive.
  • Time decay: do not clear; decay old pixels by the event timestamp e.t, producing a smear. Smoother but slightly more complex to implement. If the picture is "smeared", shorten the accumulation; if it is "empty", lengthen the accumulation — see Guide 06 — Tuning.

MIPI HVS

Swap the EVT2 decoder for MipiRaw8Decoder (Frame.evs is an apx003 RAW8 sub-frame stream); the rest of the accumulation / display logic is unchanged:

Shimeta::codec::MipiRaw8Decoder dec;   // Stateless
// dec.Decode(f.evs.data, f.evs.size, events);  → same events, accumulate and draw as above

The EVS visualization logic on S100 and X5 is identical — both use MipiRaw8Decoder; only the APS display format differs (NV12 on S100, Gray8 on X5). See Guide 07.

Optical-flow visualization

For EVS optical-flow visualization (per-pixel motion-direction vectors), refer to the OpticalFlow sample's showOf in the Windows Algo SDK — see Windows Algo SDK and Samples Overview.

No-code options

You can visualize without writing code: use our host software MultiVision Studio for direct real-time preview, or use the Toolkit's built-in viewer sample (samples/cpp/viewer in the repo) to replay raw files (see Toolkit).

Further reading