ML Inference

Portable inference API for running TFLite models via LiteRT (formerly TensorFlow Lite Micro). More...

Data Structures
struct	ove_tensor_info
	Tensor descriptor returned by ove_model_input() / ove_model_output(). More...
struct	ove_model_config
	Configuration for an ML inference session. More...

Enumerations
enum	ove_tensor_type {   OVE_TENSOR_FLOAT32 = 0 , OVE_TENSOR_INT8 = 1 , OVE_TENSOR_UINT8 = 2 , OVE_TENSOR_INT16 = 3 ,   OVE_TENSOR_INT32 = 4 }
	Tensor element types. More...

Functions
int	ove_model_init (ove_model_t *model, ove_model_storage_t *storage, void *arena, const struct ove_model_config *cfg)
	Initialise a model using caller-supplied storage and arena.
void	ove_model_deinit (ove_model_t model)
	Release resources held by a model initialised with ove_model_init().
int	ove_model_create (ove_model_t *model, const struct ove_model_config *cfg)
	Allocate and initialise a model from the heap.
void	ove_model_destroy (ove_model_t model)
	Destroy and free a model allocated with ove_model_create().
int	ove_model_invoke (ove_model_t model)
	Run inference on the currently populated input tensor(s).
int	ove_model_input (ove_model_t model, unsigned int index, struct ove_tensor_info *info)
	Get a descriptor for an input tensor.
int	ove_model_output (ove_model_t model, unsigned int index, struct ove_tensor_info *info)
	Get a descriptor for an output tensor.
uint64_t	ove_model_last_inference_us (ove_model_t model)
	Return inference time of the last ove_model_invoke() in microseconds.

Detailed Description

Portable inference API for running TFLite models via LiteRT (formerly TensorFlow Lite Micro).

Provides a C API for loading pre-trained .tflite FlatBuffer models and running inference on them. The same model binary runs unchanged across all four oveRTOS backends (FreeRTOS, Zephyr, NuttX, POSIX).

Two allocation strategies are available:

• _create() / _destroy() — heap-allocated, including the tensor arena. Available only when OVE_HEAP_INFER is defined (i.e. CONFIG_OVE_ZERO_HEAP is not set).
• _init() / _deinit() — caller-supplied storage and arena buffer. Available in both modes. See OVE_MODEL_DEFINE_STATIC for a one-step static helper.

Note

Requires CONFIG_OVE_INFER.

Enumeration Type Documentation

ove_tensor_type

enum ove_tensor_type

Tensor element types.

Subset of TFLite tensor types that are relevant for microcontroller inference (quantised int8/int16 and float32).

Function Documentation

ove_model_init()

int ove_model_init	(	ove_model_t *	model,
		ove_model_storage_t *	storage,
		void *	arena,
		const struct ove_model_config *	cfg
	)

Initialise a model using caller-supplied storage and arena.

No heap allocation is performed. The arena must be at least cfg->arena_size bytes and remain valid for the lifetime of the model. It should be 16-byte aligned for optimal CMSIS-NN performance.

Parameters

[out]	model	Receives the opaque model handle on success.
[in]	storage	Pointer to caller-allocated backend storage.
[in]	arena	Caller-allocated tensor arena buffer.
[in]	cfg	Model configuration.

Returns

OVE_OK on success, OVE_ERR_INVALID_PARAM for bad arguments, OVE_ERR_ML_FAILED if model parsing or tensor allocation fails.

See also

ove_model_deinit, ove_model_create

ove_model_deinit()

void ove_model_deinit

(

ove_model_t

model

)

Release resources held by a model initialised with ove_model_init().

The static storage and arena buffer supplied at init time are not freed.

Parameters

[in]

model

Handle returned by ove_model_init().

See also

ove_model_init

ove_model_create()

int ove_model_create	(	ove_model_t *	model,
		const struct ove_model_config *	cfg
	)

Allocate and initialise a model from the heap.

Both the backend storage and the tensor arena are allocated from the heap. The arena size is taken from cfg->arena_size.

Parameters

[out]	model	Receives the opaque model handle on success.
[in]	cfg	Model configuration.

Returns

OVE_OK on success, OVE_ERR_NO_MEMORY if allocation fails, OVE_ERR_ML_FAILED if model parsing or tensor allocation fails.

See also

ove_model_destroy, ove_model_init

ove_model_destroy()

void ove_model_destroy

(

ove_model_t

model

)

Destroy and free a model allocated with ove_model_create().

Parameters

[in]

model

Handle returned by ove_model_create().

See also

ove_model_create

ove_model_invoke()

int ove_model_invoke

(

ove_model_t

model

)

Run inference on the currently populated input tensor(s).

Before calling this function, populate the input tensor data via the pointer returned by ove_model_input().

Parameters

[in]

model

Model handle.

Returns

OVE_OK on success, OVE_ERR_ML_FAILED on inference error.

See also

ove_model_input, ove_model_output

ove_model_input()

int ove_model_input	(	ove_model_t	model,
		unsigned int	index,
		struct ove_tensor_info *	info
	)

Get a descriptor for an input tensor.

Populates info with the data pointer, shape, type, and size of the input tensor at index. Write input data to info->data before calling ove_model_invoke().

Parameters

[in]	model	Model handle.
[in]	index	Zero-based input tensor index.
[out]	info	Receives the tensor descriptor.

Returns

OVE_OK on success, OVE_ERR_INVALID_PARAM if index is out of range.

See also

ove_model_output, ove_model_invoke

ove_model_output()

int ove_model_output	(	ove_model_t	model,
		unsigned int	index,
		struct ove_tensor_info *	info
	)

Get a descriptor for an output tensor.

Populates info with the data pointer, shape, type, and size of the output tensor at index. Call after ove_model_invoke() to read results.

Parameters

[in]	model	Model handle.
[in]	index	Zero-based output tensor index.
[out]	info	Receives the tensor descriptor.

Returns

OVE_OK on success, OVE_ERR_INVALID_PARAM if index is out of range.

See also

ove_model_input, ove_model_invoke

ove_model_last_inference_us()

uint64_t ove_model_last_inference_us

(

ove_model_t

model

)

Return inference time of the last ove_model_invoke() in microseconds.

The timing is measured using ove_time_get_us() around the interpreter invocation. Returns 0 if no inference has been run yet.

Parameters

[in]

model

Model handle.

Returns

Inference duration in microseconds.