Registering sensors or development boards using Protocol version 2
Before we begin, let’s look into some commonly used terms to explain Protocol version 2 and a session example that describes how the client and a board communicate using the protocol.
Refer to the Tensor Streaming Protocol (opens in a new tab) Bitbucket repository to know more about the protocol.
Understanding Terminology and Protocol Session
The following terms are commonly used:
- Board: hardware platform containing one or more devices.
- Device: component on a board that can produce or consume data. For instance: sensors, playback devices, models.
- Client: application that connects to a board via TCP, UDP, serial port, or Bluetooth to interact with devices.
- Stream: data channel that can either send data (Output) or receive data (Input). Streams are associated with devices.
- Frame: unit of data sent periodically over a stream, in the form of a tensor.
- Tensor: multidimensional array used to represent data. The number of dimensions are referred to as rank.
- Shape: size of each dimension in a tensor. For example, [640, 480, 3] represents the shape of a video frame.
Protocol Session Example
The following steps outline a typical session between a client and a board using Protocol version 2:
- BoardCapabilitiesRequest: client sends a
BoardCapabilitiesRequestmessage to inquire about the device capabilities on the board. - BoardCapabilitiesResponse: board responds with a
BoardCapabilitiesResponsemessage detailing the available devices and their capabilities. If the response includes awatchdog_timeout, the client must periodically sendWatchdogResetRequestmessages to prevent the board from resetting. - WatchdogResetRequest: resets the watchdog timer on the board. This request should be sent at intervals specified by
watchdog_timeout. - DeviceConfigurationRequest: configures the specified device with the desired options. This request prompts a
DeviceConfigurationResponsefrom the board. - DeviceConfigurationResponse: board responds with a
DeviceConfigurationResponsemessage describing the configured data streams and their properties. - StartRequest: client sends a
StartRequestto initiate data streaming for the specified device. - DataChunk: board streams data in
DataChunkmessages for each active subscription.
Registering sensor or development board
Below is an example of how to register a sensor or development board using the Protocol API, configure the streams and handle data using essential callback functions: configure_streams, start, stop, and poll. We will begin by demonstrating how to register a microphone sensor. However, the same logic can be applied to add other sensors to the protocol.
Step 1: Implement the Device Registration Function
This function initializes the device-specific data structure, sets up the device manager, registers the device with the protocol, add integer option for gain and and oneof option for frequency. To make it easy, combine all steps into a single function that registers your device.
Example Device Registration Function
#include "protocol.h"
typedef struct {
int16_t audio_buffer0[FRAME_SIZE];
int16_t audio_buffer1[FRAME_SIZE];
int16_t* active_rx_buffer;
int16_t* full_rx_buffer;
bool pdm_pcm_initialized;
cyhal_pdm_pcm_t pdm_pcm;
bool have_data;
bool stereo;
int skipped_frames;
} mic_device_t;
static bool mic_configure_streams(protocol_t* protocol, int device, void* arg);
static void mic_start(protocol_t* protocol, int device, void* arg);
static void mic_stop(protocol_t* protocol, int device, void* arg);
static void mic_poll(protocol_t* protocol, int device, pb_ostream_t* ostream, void* arg);
static bool mic_write_payload(protocol_t* protocol, int device_id, int stream_id, int frame_count, int total_bytes, pb_ostream_t* ostream, void* arg);
#define MIC_OPTION_KEY_GAIN 1
#define MIC_OPTION_KEY_STEREO 2
#define MIC_OPTION_KEY_FREQUENCY 3
bool mic_register(protocol_t* protocol) {
int status;
// Allocate memory for the device-specific data structure
// Static memory may be used if a no malloc policy exists
mic_device_t* mic = (mic_device_t*)malloc(sizeof(mic_device_t));
if (mic == NULL)
return false;
memset(mic, 0, sizeof(mic_device_t));
// Set up the device manager with the callbacks
device_manager_t manager = {
.arg = mic,
.configure_streams = mic_configure_streams,
.start = mic_start,
.stop = mic_stop,
.poll = mic_poll,
.data_received = NULL // has no input streams
};
// Register the device with the protocol
int device = protocol_add_device(
protocol,
protocol_DeviceType_DEVICE_TYPE_SENSOR,
"Microphone",
"PCM Microphone",
manager);
if (device < 0)
return false;
// Add integer option for gain
status = protocol_add_option_int(
protocol,
device,
MIC_OPTION_KEY_GAIN,
"Gain",
"Microphone gain in dB",
0, -20, 20);
if (status != PROTOCOL_STATUS_SUCCESS)
return false;
// Add boolean option for stereo mode
status = protocol_add_option_bool(
protocol,
device,
MIC_OPTION_KEY_STEREO,
"Stereo",
"Stereo or Mono",
false);
if (status != PROTOCOL_STATUS_SUCCESS)
return false;
// Add oneof option for frequency
status = protocol_add_option_oneof(
protocol,
device,
MIC_OPTION_KEY_FREQUENCY,
"Frequency",
"Sample frequency (Hz)",
1,
(const char*[]) { "8 kHz", "16 kHz" },
2);
if (status != PROTOCOL_STATUS_SUCCESS)
return false;
if (!mic_configure_streams(protocol, device, manager.arg))
return false;
return true;
}Step 2: Define the configure_streams Callback Function
The configure_streams callback is invoked every time a DeviceConfigurationRequest is received. This callback is responsible for configuring the data streams for the device, including setting the frequency, direction, data type, and tensor shape of the streaming data . After the callback is executed, a DeviceConfigurationResponse is sent.
Example configure_streams Callback
static bool mic_configure_streams(protocol_t* protocol, int device, void* arg) {
UNUSED(arg);
int frequency_index;
int frequency;
// Get the frequency option value
if (protocol_get_option_oneof(protocol, device, MIC_OPTION_KEY_FREQUENCY, &frequency_index) != PROTOCOL_STATUS_SUCCESS) {
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Failed to get option frequency.");
return true; // Keep the connection open to allow for reconfiguration
}
// Determine the frequency based on the option index
switch (frequency_index) {
case 0:
frequency = SAMPLE_RATE_8_KHZ;
break;
case 1:
frequency = SAMPLE_RATE_16_KHZ;
break;
default:
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Invalid frequency index.");
return true;
}
// Get the stereo option value
bool stereo;
if (protocol_get_option_bool(protocol, device, MIC_OPTION_KEY_STEREO, &stereo) != PROTOCOL_STATUS_SUCCESS) {
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Failed to get option stereo.");
return true;
}
// Clear any existing streams for the device
if (protocol_clear_streams(protocol, device) != PROTOCOL_STATUS_SUCCESS) {
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Failed to clear streams.");
return true;
}
// Determine the maximum number of frames in a chunk based on stereo option
int max_number_of_frames_in_chunk = stereo ? FRAME_SIZE / 2 : FRAME_SIZE;
// Add a new stream for audio data
int stream = protocol_add_stream(
protocol,
device,
"Audio",
protocol_StreamDirection_STREAM_DIRECTION_OUTPUT,
protocol_DataType_DATA_TYPE_S16,
frequency,
max_number_of_frames_in_chunk,
NULL);
if (stream < 0) {
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Failed to add streams.");
return true;
}
// Add tensor rank based on stereo option
if (stereo) {
protocol_add_stream_rank(protocol, device, stream, "Channel", 2, (const char*[]) {"Left", "Right"});
} else {
protocol_add_stream_rank(protocol, device, stream, "Channel", 1, (const char*[]) {"Mono (Left)"});
}
// Set device status to ready after successful configuration
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_READY, "Microphone is ready.");
return true;
}Step 3: Define start and stop Callbacks Function
The start callback is invoked when a StartRequest is received, and this initializes and starts the device. The stop callback is invoked when a StopRequest is received, and this stops and de-initializes the device. These functions manage the device's active state.
Example Start Callback
static void mic_start(protocol_t* protocol, int device, void* arg) {
cy_rslt_t result;
int gain_db;
int frequency_index;
int sample_rate;
bool stereo;
mic_device_t* mic = (mic_device_t*)arg;
// Get the configured options for frequency, stereo, and gain
protocol_get_option_oneof(protocol, device, MIC_OPTION_KEY_FREQUENCY, &frequency_index);
protocol_get_option_bool(protocol, device, MIC_OPTION_KEY_STEREO, &stereo);
protocol_get_option_int(protocol, device, MIC_OPTION_KEY_GAIN, &gain_db);
// Determine the sample rate based on the frequency index
switch (frequency_index) {
case 0:
sample_rate = SAMPLE_RATE_8_KHZ;
break;
case 1:
default:
sample_rate = SAMPLE_RATE_16_KHZ;
break;
}
printf("MIC START STREAMING sample_rate=%d stereo=%d gain_db=%d\r\n", sample_rate, stereo, gain_db);
// Initialize and start the hardware
result = pcm_start(mic, sample_rate, gain_db, stereo ? CYHAL_PDM_PCM_MODE_STEREO : CYHAL_PDM_PCM_MODE_LEFT);
if (CY_RSLT_SUCCESS != result) {
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Failed to initialize hardware.");
return;
}
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ACTIVE, "Device is streaming");
}Example Stop Callback
static void mic_stop(protocol_t* protocol, int device, void* arg) {
cy_rslt_t result;
mic_device_t* mic = (mic_device_t*)arg;
printf("MIC STOP STREAMING\r\n");
// Stop and de-initialize the hardware
result = pcm_stop(mic);
if (CY_RSLT_SUCCESS != result) {
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_ERROR, "Mic: Failed to stop the PDM/PCM operation.");
return;
}
protocol_set_device_status(protocol, device, protocol_DeviceStatus_DEVICE_STATUS_READY, "Device stopped");
}Step 4: Define Poll Callback Function
When the device is in the Active state, the poll callback is called periodically to check if there is data available and send it using the protocol_send_data_chunk function. The protocol_send_data_chunk function will send a DataChunk message.
Example Poll Callback
static void mic_poll(protocol_t* protocol, int device, pb_ostream_t* ostream, void* arg) {
mic_device_t* mic = (mic_device_t*)arg;
if (mic->have_data) {
int frames_in_chunk = mic->stereo ? FRAME_SIZE / 2 : FRAME_SIZE;
// Send a DataChunk message with the available data
protocol_send_data_chunk(protocol, device, 0, frames_in_chunk, mic->skipped_frames, ostream, mic_write_payload);
mic->skipped_frames = 0;
}
}Step 5: Define the mic_write_payload Helper Function
The mic_write_payload function is used by protocol_send_data_chunk to write the actual data to the output stream.
Example mic_write_payload Function
static bool mic_write_payload(protocol_t* protocol, int device_id, int stream_id, int frame_count, int total_bytes, pb_ostream_t* ostream, void* arg) {
UNUSED(protocol);
UNUSED(stream_id);
UNUSED(frame_count);
mic_device_t* mic = (mic_device_t*)arg;
// Write the payload data to the output stream
if (!pb_write(ostream, (const pb_byte_t*)mic->active_rx_buffer, total_bytes))
return false;
mic->have_data = false;
return true;
}After implementing your own firmware, you can use the DotNetCli test client to evaluate your implementation. For instructions on how to use the test client, refer to tensor-streaming-protocol (opens in a new tab).
Refer to Flashing your custom firmware using ModusToolbox to know how to flash the custom firmware on your board using ModusToolbox™.
For flashing information on boards other than Infineon boards, refer to your specific board documentation.