Hybrid Vision Toolkit Python API

This document describes the v2.0 Python binding — a single hv_toolkit module (generated with pybind11, source at src/python/hv_toolkit_module.cpp). The same API supports both USB (x86_64) and MIPI HVS (S100 / X5).

Source of truth: this page follows the repository's src/python/hv_toolkit_module.cpp; fields/methods map one-to-one to the C++ public API (API.md → Python binding appendix). When the binding changes, sync against that file.

v2.0 has removed the four legacy standalone modules (hv_evt2_codec_python / hv_event_reader_python / hv_event_writer_python / hv_camera_python), unified into one hv_toolkit module.

Prebuilt distribution: the release repo ships lib/x86_64/python/hv_toolkit.cpython-310-x86_64-linux-gnu.so (Python 3.10); on boards you must cross-compile it yourself. RK3588 support is pending.

Module overview

import hv_toolkit as hv

hv.__version__   # "2.0.0"

Exported surface at a glance:

Symbol	Kind	Purpose
EventCD	class	A single event (x/y/t/polarity, all writable).
Frame	class	Frame: width/height/frame_id/format + read-only evs/aps (numpy views)
Backend	enum	Capture backend: Auto/Usb/Mipi/MipiHvs/Ethernet.
EventFormat	enum	Event format: Evt2/Evt3.
PixelFormat	enum	APS pixel format: BayerRG8/RGB888/Gray8/RAW8/RAW10/NV12.
QueuePolicy	enum	Queue policy: DropOldest/Block.
RawFormat	enum	RAW file format: Evt2/Evt3.
DeviceConfig	class	Capture configuration (all USB + MIPI + Ethernet fields).
Camera	class	Unified capture camera.
Evt2Decoder / Evt2Encoder	class	EVT2 decoder / encoder.
Evt3Decoder / Evt3Encoder	class	EVT3 decoder / encoder.
MipiRaw8Decoder	class	MIPI HVS apx003 RAW8 subframe-stream decoder.
MipiRaw8Layout	class	RAW8 subframe layout constants (kSubframeBytes etc.).
extract_evs_timestamp(data)	func	Extracts the sensor timestamp from a RAW8 subframe header.

Still not exported (use the C++ API when needed — see the C++ API): async callbacks (set_frame_callback / set_event_callback / set_image_callback), EventReader / EventWriter, HybridReader / HybridWriter.

EventCD

A single CD event; all fields are writable.

e = hv.EventCD()
e.x = 100
e.y = 50
e.t = 12345        # microseconds
e.polarity = True  # True = CD_ON, False = CD_OFF

Attribute	Type	Meaning
x / y	int	Pixel coordinates.
t	int	Timestamp (microseconds).
polarity	bool	True = CD_ON, False = CD_OFF.

Frame

The frame object. evs/aps are zero-copy numpy uint8 views: the underlying data lives in pool slabs, the views hold the owner's reference count, and the views stay valid after the Frame is released (until the views themselves are garbage-collected).

Attribute	Type	Meaning
width / height	int	Frame width / height (writable).
frame_id	int	Frame sequence number (writable).
format	PixelFormat	APS pixel format (usually NV12 for MIPI HVS).
evs	numpy.ndarray(uint8)	Read-only, raw undecoded event bytes from the HAL (format depends on backend and event_fmt).
aps	numpy.ndarray(uint8)	Read-only, raw APS bytes (NV12 for MIPI HVS).

f = hv.Frame()
if cam.get_frame(f, 1000):
    raw = bytes(f.evs)          # convert to bytes to feed a decoder
    print(f.aps.nbytes, "bytes APS", f.format)

Enums

hv.Backend.Usb           # USB backend (x86_64)
hv.Backend.MipiHvs       # MIPI HVS dual-VC backend (S100 / X5; RK3588 pending)
hv.EventFormat.Evt2      # / Evt3
hv.PixelFormat.NV12      # MIPI HVS APS output format
hv.QueuePolicy.DropOldest
hv.RawFormat.Evt3        # RAW file format

DeviceConfig

Capture configuration, passed to Camera.init(). USB / MIPI / Ethernet fields all live on the same class — just fill in the fields for the selected backend.

Group	Attribute	Type	Meaning
Common	backend	Backend	Capture backend.
Common	event_fmt	EventFormat	Event format (USB/Ethernet).
Common	buffer_count	int	Number of frame buffers (default 8).
Common	queue_policy	QueuePolicy	Frame-queue policy (default DropOldest).
Common	evs_fps	int	MIPI frame-rate tier (0 = default 240; 120/240/300/500/750/1000; non-tier values raise an error; applied at init).
USB	vendor_id	int	USB vendor ID.
USB	product_id	int	USB product ID.
USB	event_urbs	int	In-flight URBs on the USB event endpoint (default 4).
MIPI	device_node	str	MIPI device node (e.g. /dev/video0).
MIPI	sensor_index	int	MIPI sensor index (VC0 EVS configuration index, default 0).
MIPI	i2c_bus	int	Secure-chip authentication I2C bus (default 1).
Ethernet	ip	str	Peer IP.
Ethernet	data_port / ctrl_port	int	Data / control port (default 8000/8001).
Ethernet	listen_port	int	TCP listen port (default 8888).
Ethernet	bind_ip	str	Local bind IP (empty = INADDR_ANY).

# USB (x86_64)
cfg = hv.DeviceConfig()
cfg.backend    = hv.Backend.Usb
cfg.vendor_id  = 0x1d6b
cfg.product_id = 0x0105

# MIPI HVS (S100 / X5)
cfg = hv.DeviceConfig()
cfg.backend      = hv.Backend.MipiHvs
cfg.device_node  = "/dev/video0"
cfg.sensor_index = 0     # S100 defaults to 9; X5 defaults to 49 (determined by the SDK configuration)
cfg.i2c_bus      = 1
cfg.evs_fps      = 0     # optional: 120/240/300/500/750/1000; 0 = default 240

Camera

The unified capture camera, corresponding to the C++ Shimeta::hv::Camera. Method names are Python-style (snake_case) and shared by all backends.

cam = hv.Camera()
cam.init(cfg)                       # cfg: DeviceConfig; returns bool
cam.start_stream()                  # returns bool (whether the device connected)
f = hv.Frame()
ok = cam.get_frame(f, 1000)         # timeout_ms defaults to 1000; returns whether a frame was obtained
cam.stop_stream()
cam.destroy()

Method	Purpose	Parameters / return
init(cfg)	Initializes the backend per DeviceConfig	cfg (in) DeviceConfig; returns bool
start_stream()	Starts the capture thread.	Returns bool (whether the device connected).
stop_stream()	Stops capture and joins the thread.	—
destroy()	Releases backend resources.	—
get_frame(frame, timeout_ms=1000)	Synchronously pulls one frame (events + APS)	frame (in/out) Frame¹; returns bool
set_exposure(value)	Sets APS exposure.	Returns bool.
set_frame_rate(fps)	Sets the EVS event frame rate (USB/Ethernet).	Returns bool.
get_frame_rate()	Reads the current EVS frame rate.	Returns (ok: bool, fps: int).
sync_clock()	Clock sync (e.g. Ethernet PTP).	Returns bool.

¹ timeout_ms defaults to 1000.

Codecs

Decoder.decode() takes bytes and returns a numpy structured array (dtype fields x/y/t/polarity); Encoder.encode() takes a list[EventCD] and returns bytes. The EVT2/EVT3 codecs are stateful (they maintain the time base across packets) — reuse one instance for a continuous stream and call reset() before a new stream; MipiRaw8Decoder is stateless.

ev = dec.decode(b"\x00\x01...")   # → ndarray with fields x/y/t/polarity
len(ev)                            # event count
ev['x']                            # x coordinates of all events (ndarray)
ev[:100]                           # slicing

EVT2

32-bit word stream, the default event format of the USB backend.

• Evt2Decoder: decodes an EVT2 32-bit word byte stream into a CD event array.
• Evt2Encoder: encodes events into an EVT2 32-bit word byte stream (with the necessary TimeHigh redundancy words).

dec = hv.Evt2Decoder()
events = dec.decode(bytes(f.evs))   # f.evs → ndarray
dec.reset()

enc = hv.Evt2Encoder()
raw = enc.encode([e1, e2])          # list[EventCD] → bytes

Method	Purpose	Parameters / return
Evt2Decoder.decode(data)	Decodes an EVT2 byte stream into an event array	data: bytes → numpy.ndarray¹
Evt2Encoder.encode(events)	Encodes events into an EVT2 byte stream	events: list[EventCD] → bytes
*.reset()	Clears time state; call before a new stream	—

¹ Fields: x / y / t / polarity.

EVT3

16-bit word stream; used by Frame.evs when event_fmt=Evt3 (decode input length must be a multiple of 2).

• Evt3Decoder: decodes an EVT3 16-bit word byte stream into a CD event array.
• Evt3Encoder: encodes events into an EVT3 16-bit word byte stream.

dec = hv.Evt3Decoder(); events = dec.decode(raw_bytes)
enc = hv.Evt3Encoder(); raw = enc.encode([e1, e2])

Method signatures and semantics match EVT2: decode(data: bytes)→ndarray, encode(list[EventCD])→bytes, and reset() to clear state.

MIPI RAW8

The apx003 subframe stream, produced only by the MIPI HVS backend; Frame.evs must be decoded with MipiRaw8Decoder — not with EVT2/EVT3.

• MipiRaw8Decoder: decodes the apx003 RAW8 subframe stream into a CD event array; stateless.

dec = hv.MipiRaw8Decoder()
events = dec.decode(bytes(f.evs))                    # default auto mode: decode all subframes by data length
events = dec.decode(bytes(f.evs), subframe_count=4)  # decode only the first 4 subframes

Method	Purpose	Parameters / return
MipiRaw8Decoder.decode(data, subframe_count=0)	Decodes a RAW8 subframe stream	data: bytes¹ → numpy.ndarray

¹ subframe_count: int; ≤0 = auto mode decodes all by length; >0 = only the first N.

• MipiRaw8Layout: RAW8 subframe layout constants (class attributes), for reference when decoding subframe by subframe.

Constant	Value	Meaning
kSubWidth / kSubHeight	384 / 304	Single-subframe resolution
kEvsWidth / kEvsHeight	768 / 608	Full-frame EVS resolution
kSubframeBytes	32768	Bytes per subframe
kTotalSubframes	32	Subframes per package (4 spatial × 8 merged)

• extract_evs_timestamp(data): extracts the sensor timestamp from an apx003 RAW8 subframe header (45-bit / 200 → microseconds). Pairs with MipiRaw8Decoder — take the timestamp first, then decode the events.

raw_ts, processed_us, valid = hv.extract_evs_timestamp(bytes(f.evs))
# raw_ts: 45-bit raw timestamp; processed_us: raw_ts/200 (microseconds); valid: whether valid

USB vs. MIPI HVS

	USB (x86_64)	MIPI HVS (S100 / X5)
backend	Backend.Usb	Backend.MipiHvs
Key DeviceConfig fields	vendor_id/product_id	device_node/sensor_index/i2c_bus/evs_fps
Frame.evs decoder	Evt2Decoder / Evt3Decoder	MipiRaw8Decoder
Frame.format	NV12	NV12 (S100) / Gray8 (X5)

Camera / Frame / get_frame / stop_stream etc. are all identical — the backend is just a runtime value of DeviceConfig.backend; there is no second API to learn.

Build and deploy

The release repo ships a prebuilt Python module in lib/x86_64/python/ (cpython-310-x86_64) — no build needed:

sudo ./run.sh install x86_64        # install libraries into the system path
python3 -c "import hv_toolkit; print(hv_toolkit.__version__)"   # smoke test
python3 samples/python/get_started.py

For a source checkout, or to run on S100 / X5 boards:

./run.sh --python build x86_64     # USB; artifact build/hv_toolkit.<abi>.so
./run.sh --python build s100       # MIPI HVS (aarch64); artifact out/s100/build/hv_toolkit.cpython-310-aarch64-linux-gnu.so
./run.sh --python build x5         # X5 (aarch64); artifact out/x5/build/hv_toolkit.cpython-310-aarch64-linux-gnu.so

Board deployment (set the library and module paths, then run):

export LD_LIBRARY_PATH=/app/build:$LD_LIBRARY_PATH
export PYTHONPATH=/app/build:$PYTHONPATH
python3 /app/build/samples/python/get_started_mipi.py

See also HV Toolkit Quick Start → Python samples and Your First C++ Program → S100 board deployment.

License

Apache License 2.0.