Record & Replay

Task goal

Save the live event stream to a file and replay it offline later. This is the foundation of dataset collection, offline algorithm debugging, and issue reproduction. The Toolkit uses EVT2/EVT3 as the standard event file formats (see Data Formats).

Classes involved

  • Shimeta::io::EventWriter: write events to a RAW file. open(file, w, h, fmt, start_ts) → writeRaw(...) / writeEvents(...) → flush() / close().
  • Shimeta::io::EventReader: read back for replay (readAllEvents); see Guide 02.
  • Shimeta::io::HybridWriter: combined EVS + APS recording (EVS as RAW, APS as AVI).

Platform note: Frame.evs over USB is usually EVT2; MipiHvs on S100/X5 is apx003 RAW8. APS is NV12 on S100 and Gray8 on X5 — when recording both streams, handle the image according to Frame.format.

Recording: camera → file

Over the USB backend, Frame.evs is already EVT2 bytes; write it directly with writeRaw (no re-encoding, cheapest):

#include <shimetapi/hv/camera.h>
#include <shimetapi/hv/device_config.h>
#include <shimetapi/io/event_writer.h>
#include <chrono>
#include <iostream>
#include <thread>

int main() {
    Shimeta::hv::Camera cam;
    Shimeta::hv::DeviceConfig cfg;
    cfg.backend   = Shimeta::hv::Backend::Usb;
    cfg.vendor_id = 0x1d6b;
    cfg.product_id = 0x0105;
    cfg.event_fmt = Shimeta::hv::EventFormat::Evt2;
    cam.Init(cfg);
    if (!cam.StartStream()) return 1;

    Shimeta::io::EventWriter writer;
    writer.open("events.raw", 768, 608, Shimeta::io::RawFormat::Evt2, 0);

    auto end = std::chrono::steady_clock::now() + std::chrono::seconds(5);
    Shimeta::Frame f;
    while (std::chrono::steady_clock::now() < end && cam.GetFrame(f, 1000)) {
        writer.writeRaw(f.evs.data, f.evs.size);   // Write Frame.evs directly (no encoding)
    }

    writer.flush();
    writer.close();
    cam.StopStream();
    cam.Destroy();
    std::cout << "录制完成" << std::endl;
    return 0;
}

writeRaw vs writeEvents

  • writeRaw(data, len): passes raw bytes through (a direct write of Frame.evs) without re-encoding — fastest; requires the bytes to already be in the target format.
  • writeEvents(events): encodes a vector<EventCD> with Evt2Encoder and writes it — for when you hold decoded events and want to record them as EVT2; writtenEventCount() reflects the number of events written via this path.

Always flush() when capture ends; close() also flushes automatically.

Replay: file → processing

Read the RAW file back with EventReader and process it (details in Guide 02):

Shimeta::io::EventReader reader;
reader.open("events.raw");
std::vector<Shimeta::EventCD> events;
reader.readAllEvents(events);   // Same EventCD as live capture

The v2.0 EventReader currently only provides readAllEvents (reads everything in); streaming/batched reads are not yet available; watch memory with very large files.

For visual replay (pause, variable speed, frame generation), use the Toolkit's built-in viewer sample (samples/cpp/viewer in the repo); see Toolkit — Samples.

EVS + APS hybrid recording: HybridWriter

To save EVS as RAW and APS as AVI at the same time, use HybridWriter:

#include <shimetapi/io/hybrid_writer.h>

Shimeta::io::HybridWriter hw;
hw.open("events.raw", "aps.avi", 768, 608, Shimeta::io::RawFormat::Evt2, 30.0);
Shimeta::Frame f;
while (cam.GetFrame(f, 1000)) {
    hw.writeFrame(f);   // EVS goes through writeRaw; APS is written to the AVI according to Frame.format
}
hw.close();

Hybrid replay: HybridReader

Read back the .raw + .avi recorded by HybridWriter with HybridReader (like Camera, it returns raw bytes; your application decodes them):

#include <shimetapi/io/hybrid_reader.h>
#include <shimetapi/codec/evt2_codec.h>

Shimeta::io::HybridReader hr;
hr.open("events.raw", "aps.avi");   // Leave either path empty to skip that side

Shimeta::codec::Evt2Decoder dec;
Shimeta::Frame f;
while (hr.readApsFrame(f)) {
    // f.aps = raw APS bytes: usually NV12 on S100/USB, Gray8 on X5
    // f.ts.aps_ts_ns = nanosecond timestamp after this frame's processing
}
// Read the EVS side packet by packet (skipping the EVT3 text header):
while (hr.readEvsPacket(f)) {
    std::vector<Shimeta::EventCD> events;
    dec.Decode(f.evs.data, f.evs.size, events);
}
hr.close();

MIPI notes

Frame.evs from Backend::MipiHvs is an apx003 RAW8 sub-frame stream (not EVT2). A direct writeRaw produces a RAW8 file that EventReader cannot replay as EVT2. Two options:

  • Want an EVT2 file: decode to EventCD with MipiRaw8Decoder first, then writer.writeEvents(events) (encodes to EVT2);
  • Keep raw RAW8: save with writeRaw and convert offline with a tool.

RAW8 event recording and replay follow the same path on S100 and X5; just switch the build target and deploy directory (out/s100/build or out/x5/build), and don't mislabel RAW8 files as EVT2.

Further reading