.xmm Model Loading

.xmm is the proprietary neural network model format for the GK7206 NPU. It is a binary file that mainly contains the following:

Content	Description
Model Weights	Quantized network parameters
Computation graph structure	Operator layout and inter-layer connections
Private Data	Input/output tensor descriptions, quantization parameters, etc.
File header	Begins with the "ZZZZ" magic number

Model source: typically exported from a training framework (such as PyTorch/ONNX), then quantized and compiled into a .xmm file using the XMTVM model conversion tool provided by Goke.

Note

A .bin model file is the model format used by the higher-level SVP interface (which internally also loads the .xmm via the NPU).

GK7206 provides two sets of APIs for different scenarios:

Method	API level	Header	Applicable scenario
Low-level CL interface	xmedia_cl_*	xmedia_cl.h	Directly load .xmm, fully control the inference flow
High-level SVP interface	xmedia_svp_*	xmedia_svp.h	Use .bin models with built-in pre/post-processing (NMS, etc.), out of the box

Tip

For the high-level SVP interface, see SVP Video Processing.

Low-Level CL Interface: Complete Steps to Load a .xmm Model

Stage	Key operation	Description
Stage 1	System initialization	Initialize the XMedia system module and the Compute Library (NPU runtime)
Stage 2	Get the NPU device and create a context	First get the device count, then the device IDs, and finally create a context as the basis for resource management
Stage 3	Query the memory required by the model	Get the workspace and weight sizes via xmedia_cl_graph_querysize_from_file()
Stage 4	Allocate NPU-private memory (MMZ)	Allocate physically contiguous memory for the workspace and weight; ordinary malloc() cannot be used
Stage 5	Load the model	Load the .xmm model using xmedia_cl_graph_loadmodel_from_file_withmem() or the simplified interface
Stage 6	Get the input/output tensor information	Query the number of inputs/outputs and obtain the complete tensor description information
Stage 7	Prepare data and run inference	Allocate input/output buffers, set tensor addresses, flush the cache, and run inference
Stage 8	Release resources	Unload the model, release the context, release the device, de-initialize, and free the MMZ memory

Stage 1: System Initialization

#include "xmedia_cl.h"
#include "xmedia_mmz.h"
#include "xmedia_sys.h"

// Initialize the XMedia system module
ret = xmedia_sys_init(XMEDIA_NULL);

// Initialize the Compute Library (NPU runtime)
ret = xmedia_cl_init();

Stage 2: Get the NPU Device & Create a Context

xmedia_cl_u32 num_devices = 0;
xmedia_cl_device_id *devices = NULL;
xmedia_cl_context context = NULL;

// First call: get the device count
ret = xmedia_cl_get_device_ids(XMEDIA_CL_DEVICE_NPU, NULL, &num_devices);

// Allocate the device array
devices = calloc(num_devices, sizeof(xmedia_cl_device_id));

// Second call: get the device IDs
ret = xmedia_cl_get_device_ids(XMEDIA_CL_DEVICE_NPU, devices, &num_devices);

// Create the context (the basis for all resource management)
xmedia_cl_s32 err_code = 0;
context = xmedia_cl_create_context(num_devices, devices, &err_code);

Stage 3: Query the Memory Required by the Model

xmedia_cl_u32 worksize, weightsize;

// Query the workspace and weight sizes required by the model
ret = xmedia_cl_graph_querysize_from_file("data/neuron_network.xmm",
                                           &worksize, &weightsize);

• worksize: temporary buffer for intermediate inference results (workspace)
• weightsize: space occupied by the model weights

Stage 4: Allocate NPU-Private Memory (MMZ)

xmedia_u64 phy_addr[4] = {0};
void *virt_addr[4] = {0};

// Allocate the workspace
if (worksize) {
    XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[0], &virt_addr[0],
        "npu_workspace", NULL, worksize);
}

// Allocate the weight
if (weightsize) {
    XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[1], &virt_addr[1],
        "npu_weight", NULL, weightsize);
}

Note

The NPU uses physically contiguous memory (MMZ); ordinary malloc() cannot be used. You must use xmedia_mmz_alloc() / xmedia_mmz_map().

Stage 5: Load the Model

xmedia_cl_graph graph = NULL;

// Load from a file, using user-provided memory
ret = xmedia_cl_graph_loadmodel_from_file_withmem(
    &context,
    "data/neuron_network.xmm",   // .xmm file path
    virt_addr[0],                 // workspace address
    worksize,                     // workspace size
    virt_addr[1],                 // weight address
    weightsize,                   // weight size
    &graph                        // output: graph handle
);

There is also a simplified version (the SDK allocates memory automatically):

ret = xmedia_cl_graph_loadmodel_from_file(&context, "data/neuron_network.xmm", &graph);

Stage 6: Get Input/Output Tensor Information

xmedia_cl_tensor_info_inout input = {0}, output = {0};
xmedia_cl_u32 input_num = 0, output_num = 0;

// First time: get the input count
ret = xmedia_cl_graph_get_input(graph, input_num, &input);
input_num = input.num;

// Allocate the tensor array memory
malloc_inout_tensor_mem(&input);

// Second time: get the full input tensor description (shape, size, type, quant)
ret = xmedia_cl_graph_get_input(graph, input_num, &input);

// Get the output the same way
ret = xmedia_cl_graph_get_output(graph, output_num, &output);
output_num = output.num;
malloc_inout_tensor_mem(&output);
ret = xmedia_cl_graph_get_output(graph, output_num, &output);

Each tensor contains:

typedef struct {
    xmedia_cl_u32 tensor_id;
    void *addr;              // Data address (the user must set this)
    xmedia_cl_tensor_shape shape;  // Dimension information (N,C,H,W)
    xmedia_cl_tensor_quant quant;  // Quantization parameters (scale, zero_point)
    xmedia_cl_u32 size;      // Data size in bytes
    xmedia_cl_s8 *name;      // Tensor name
} xmedia_cl_tensor;

Stage 7: Prepare Data & Run Inference

// Allocate input/output buffers
XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[2], &virt_addr[2],
    "npu_input", NULL, inputsize);
XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[3], &virt_addr[3],
    "npu_output", NULL, outputsize);

// Set the input data address (write the preprocessed data here)
input.tensor[0].addr = virt_addr[2];
// ... copy actual image data to input.tensor[i].addr ...

// Set the output data address
output.tensor[0].addr = virt_addr[3];

// Flush the cache (ensure the NPU sees the latest data)
XMEDIA_API_SYS_MmzFlushCache(phy_addr[2], virt_addr[2], inputsize);

// Bind input/output to the graph
ret = xmedia_cl_graph_set_inout(graph, &input, &output);

// Run inference (synchronous)
ret = xmedia_cl_graph_process(graph);

After inference completes, output.tensor[i].addr contains the inference result.

Stage 8: Release Resources

// Unload the model
xmedia_cl_graph_unload(graph);

// Release the context
xmedia_cl_release_context(context);

// Release the device
xmedia_cl_release_device_ids(devices, &num_devices);
free(devices);

// De-initialize
xmedia_cl_uninit();
xmedia_sys_exit();

// Free MMZ memory
for (i = 0; i < 4; i++) {
    if (virt_addr[i]) XMEDIA_API_SYS_MmzFree(phy_addr[i], virt_addr[i]);
}

Tip

1. Before using the NPU, you must first load the kernel driver:
   • Load the ko driver (under out/xm7206xxx/ko): ./load xm7206v11a -i
   • Or manually load the NPU module: insmod xm_npu.ko
2. For detailed .xmm model usage, see sample/npu/xmm.