Skip to Content
DEEPCRAFT™ Studio 5.14 has arrived. Read more →

Deployment API

This section provides information on the two deployment APIs available in DEEPCRAFT™ Model Converter: Imagimob API and ml-middleware API. You can choose one of the two APIs to deploy your model on the target device.

Choosing the Firmware API for your application

When generating code in DEEPCRAFT™ Model Converter, you can select which API your firmware will use to run the model. Both the Imagimob API and the ml-middleware API convert and quantize the model identically, and both run on Infineon’s ModusToolbox™ ML middleware at runtime. The difference is how much integration code the Mode Converter generates for your application.

  • ml-middleware: When this option is selected, the Model Converter generates only the model artifacts, so your application calls the Infineon’s mtb_ml_* API directly. You perform the model initialization, arena setup, and inference in your firmware. By default, this option is selected as the target API when you generate code.

  • Imagimob API: When this option is selected, the Model Converter generates the model artifacts in addition to a small, self-contained model.c and model.h. These files expose the IMAI_* functions and wrap initialization, arena setup, quantization, and inference behind IMAI_init() and IMAI_compute().

Both options generate a code_generation_report.md. Depending on the API you select, the output includes either the raw middleware artifacts or the IMAI_* model files. You can select the API in the GUI, in the project file, or with the corresponding CLI flag.

Ml-middleware API

The Model Converter generates model artifacts that the ModusToolbox™ ML middleware uses directly. The middleware abstracts the underlying TensorFlow Lite Micro runtime and provides a C API for model setup, input handling, and inference, such as mtb_ml_model_init(), mtb_ml_model_inputs(), and mtb_ml_model_invoke().

Include the generated model files

Include the generated model files using the helper macros. In the example below, test_model is the output prefix generated by the Model Converter. Include the ml-middleware header first, because it defines both the API and the required macros.

#include "mtb_ml.h"                                     // ml-middleware API + macros
#include MTB_ML_INCLUDE_MODEL_FILE(test_model)         // model weights/params
#include MTB_ML_INCLUDE_MODEL_X_DATA_FILE(test_model)   // regression input (optional)
#include MTB_ML_INCLUDE_MODEL_Y_DATA_FILE(test_model)   // regression output (optional)
Initialize the model

To initialize the model, call mtb_ml_model_init() exactly once before running inference. You can let the middleware allocate memory internally, or you can provide your own tensor arena.

mtb_ml_model_t *model_object;
mtb_ml_model_bin_t model_bin = {MTB_ML_MODEL_BIN_DATA(test_model)};
 
// Internal allocation
mtb_ml_model_init(&model_bin, NULL, &model_object);
 
// External allocation
uint8_t tensor_arena[MTB_ML_MODEL_ARENA_SIZE(test_model)];
mtb_ml_model_buffer_t mem_buf = {tensor_arena, MTB_ML_MODEL_ARENA_SIZE(test_model)};
mtb_ml_model_init(&model_bin, &mem_buf, &model_object);

Call mtb_ml_model_init() exactly once before running inference. On NPU targets, such as Ethos-U55, you need to call mtb_ml_init() to initialize the NPU drivers. Add the middleware to your project with the ModusToolbox™ COMPONENTS/DEFINES (ML_TFLM, precision ML_FLOAT32 /ML_INT8x8, CPU/NPU ML_CPU_ONLY / U55, optional CMSIS_DSP)

Choose this option when you want full control over model integration inside a ModusToolbox™ application. For a complete configuration example, see the Readme and makefiles in the deployment Code Example.

Using the TFLM API Directly

The ml-middleware option generates the model artifacts directly, you are not required to use the mtb_ml_* abstraction. You can integrate the generated files with the underlying runtime yourself by using the native TensorFlow Lite Micro (TFLM) C++ interpreter , depending on the target selected during model generation. This approach is useful if your application already has an inference stack, requires runtime behavior that the middleware does not expose, or needs to bypass the middleware layer. However, on the NPU targets, you must still call mtb_ml_init() before inference to initialize the NPU drivers. This requirement applies even when you use the TFLM runtime directly.

Running Inference

To run inference with the ml-middleware API, set each input tensor with mtb_ml_model_inputs(), then run inference withmtb_ml_model_invoke(). Read each output with mtb_ml_model_get_output_tensor(). The output layer is capped at 64 nodes.

You can feed input data to the model in either of two formats:

  • Use data that matches the model’s native tensor type.
  • Use floating-point data and let the middleware quantize the data before inference.

For floating-point inputs, call mtb_ml_utils_model_quantize_tensor() to quantize the data using the model’s scale and zero-point. After inference, call mtb_ml_utils_model_dequantize_tensor() to convert native output tensors back to floating-point values.

// Inputs — already native:        mtb_ml_model_inputs(model_object, native_input[i], i);
// Inputs — float, quantize first:
for (int i = 0; i < input_count; i++) {
    mtb_ml_utils_model_quantize_tensor(model_object, i, src_f32[i], native_input[i], count_in[i]);
    mtb_ml_model_inputs(model_object, native_input[i], i);
}
 
mtb_ml_model_invoke(model_object);
 
for (int i = 0; i < output_count; i++) {
    MTB_ML_DATA_T *out; int size;
    mtb_ml_model_get_output_tensor(model_object, &out, &size, i);
    mtb_ml_utils_model_dequantize_tensor(model_object, i, dst_f32[i], size); // native → float
}

Imagimob API: Auto-Generated Wrapper

The Imagimob API option generates model.c and model.h, which expose a compact, portable C API for running the model. The generated functions wrap the middleware used by the ml-middleware option, while handling model initialization, tensor arena setup, and input quantization.

Core API

The generated API includes three main functions:

int  IMAI_init(void);                           // Initialize the model
void IMAI_compute(in0, in1, ..., out0, ...);   // Run inference
void IMAI_finalize(void);                       // Free resources

IMAI_compute() runs inference in a single call. Its arguments list all model inputs, followed by all model outputs.

The API returns the following status codes: IMAI_RET_SUCCESS (0), IMAI_RET_NODATA (-1), and IMAI_RET_ERROR (-2). The default function prefix is IMAI_, but you can configure prefix during code generation.

Choose the Imagimob API option when you want minimal, drop-in integration. The generated wrapper still links against the ModusToolbox™ ML middleware, so your project must include the required ModusToolbox™ COMPONENTS, such as ML_TFLM, the selected CPU or NPU target, and optionally CMSIS_DSP. The generated code already contains the selected quantization type, so you do not need to add separate precision components.

Auto-quantization is optional. By default, IMAI_compute() accepts floating-point input and quantizes it before inference. If your application already provides quantized input data, you can disable auto-quantization and use the generated IMAI_quantize() and IMAI_dequantize() helpers manually. These helpers use each tensor’s scale and zero-point. See the multi-IO example below.

Arena and Weights Placement

The Imagimob API lets you choose where the tensor arena and model weights are stored. IMAI_init() applies the selected configuration automatically.

  • Arena allocation: Use Static allocation, the default, to place a fixed tensor arena buffer in the binary. Use Dynamic (Heap) allocation to allocate the arena with malloc() at runtime, so it consumes RAM only while the model is loaded.

  • Weights placement: Keep weights in flash for read-only access without a RAM copy, or choose Dynamic placement to store the weights in flash and copy them to RAM automatically during IMAI_init(). Dynamic placement can improve performance when running from RAM is faster than execute-in-place from flash.

With the ml-middleware option, your application manages the tensor arena and weights placement directly.

Multiple inputs and outputs models

For multi-input and multi-output models, IMAI_compute() lists all input tensors first, followed by all output tensors.

IMAI_compute(in0, in1, ..., out0, out1, ...);

For a known model, call the function directly with the explicit arguments. For example, a model with two inputs and three outputs would use:

IMAI_compute(in0, in1, out0, out1, out2);   // IMAI_COMPUTE_INPUTS=2, IMAI_COMPUTE_OUTPUTS=3

The generated header also advertises the counts, so a generic loop can fill inputs and read outputs and dispatch through IMAI_COMPUTE_PTR:

void *args[IMAI_COMPUTE_INPUTS + IMAI_COMPUTE_OUTPUTS];           // inputs first, then outputs
for (int i = 0; i < IMAI_COMPUTE_INPUTS; i++)  args[i] = input_ptr[i];
for (int i = 0; i < IMAI_COMPUTE_OUTPUTS; i++) args[IMAI_COMPUTE_INPUTS + i] = output_ptr[i];
IMAI_COMPUTE_PTR(args);                                            // calls IMAI_compute(in0, …, out0, …)

The model metadata accessors, IMAI_INPUT_META(i) and IMAI_OUTPUT_META(i), provide each tensor’s size, scale, and offset. Use these values to perform manual quantization for each tensor individually.

// Inputs: float → native, quantized per tensor before compute
for (int i = 0; i < IMAI_COMPUTE_INPUTS; i++)
    IMAI_quantize(in_f32[i], in_q[i], IMAI_INPUT_META(i)->count,
                  IMAI_INPUT_META(i)->scale, IMAI_INPUT_META(i)->offset);
 
IMAI_compute(in_q[0], in_q[1], out_q[0], out_q[1], out_q[2]);
 
// Outputs: native → float, dequantized per tensor
for (int i = 0; i < IMAI_COMPUTE_OUTPUTS; i++)
    IMAI_dequantize(out_q[i], out_f32[i], IMAI_OUTPUT_META(i)->count,
                    IMAI_OUTPUT_META(i)->scale, IMAI_OUTPUT_META(i)->offset);
Last updated on