DEEPCRAFT™ Studio 5.2 has arrived. Read more →

Deploy siren detection model on AURIX™ TC375 Lite Kit

This tutorial is based on TC375 Lite Kit with the combination of Automotive MEMS Microphones on the Audio Shield Board to demonstrate AI based Application for Siren Detection.

  • This project is based on an Infineon Aurix_AudioAppKit (opens in a new tab) project .
  • This project is extended with Ethernet and LwIp to stream PCM data via UDP.
  • This project is extended with additional task to process the audio data in AI model "Siren Detection".
  • The AI model for siren detection is based on the DEEPCRAFT™ Studio starter project "Siren Detection" and is integrated in this project with permission of Imagimob.

Prerequisites

Device

  • 1 x Aurix™ 2G TC375TP - AA Step
  • 2 x MEMS Microphones IM67D120A

Hardware Board

AURIX™ TC375 Lite Kit Board


KITA2G Audio Shield Board

Scope of work

The scope of this Project is to showcase the following features:

  • receiving PDM stream from the Microphones (IM67D120A) via QSPI
  • process the PDM stream and convert the PDM to PCM stream
  • process the PCM stream in the AI model to recognize siren sounds.
  • print the results of recognition via serial interface (UART) to a terminal
  • additionally the 16kHz PCM mono stream is streamed via UDP to a host PC

Third-Party Packages

  • This software package includes the FreeRTOS V10.4.6
  • This software package includes the lwIP - A Lightweight TCP/IP stack(2.1.2)
  • This software package includes the OpenPDMFilter v1.0.0

Hardware setup

⚠️

Make sure 3.3V Supply Jumper is set to the left (covering LK and 3V3) and AURIX Audio-RX Source Jumper is set to the right (enabling the MIC).


Compiling

The example projects can be compiled and debugged with the Aurix™ Development Studio (opens in a new tab) (v1.9.12 upwards).

Implementation

The Project contains the following folders and content:

  • Configurations folder contains user configurations for FreeRTOS, BootModeHeader and LwIp.
  • Libraries folder contains all Driver and Middleware Software used in the project:
    • AudioShield folder: contains the configuration for the KITA2G Audio Shield Board and the OpenPDM library.
    • Ethernet folder: contains the LwIp Ethernet Stack and the Ethernet Transceiver Driver.
    • FreeRTOS folder: contains the FreeRTOS and abstraction layer for AURIX™
    • iLLD folder: contains the Infineon Low Level Driver for all the AURIX™ modules, peripherals.
    • Infra folder: contains the Startup Code and special function register definition.
    • SirenDetection folder: contains AI Model generated by DEEPCRAFT™ Studio. To integrate a new DEEPCRAFT™ Studio Model (siren detection) the model.c and model.h files has to be replaced in this folder
    • UART folder: contains the serial communication abstraction.
  • TriCore Debug folder contains the build artifacts (*.map, .elf, .hex) after a build of the project.
  • The Top folder contains the FreeRTOS Task implementation and the start code for each CPU.

The complete application is running on Cpu0. The entry point for the application is the core0_main function. In this function the LwIp, Interrupts get initialized also the Tasks for AUDIO and Uart get created. The last step in this Function is the start of the FreeRTOS task scheduling.

Following AUDIO Tasks are running in the application:

  • OsTask_ExternalAudioControl: reads the ADC values from an Potentiometer to setup the volume for the AUDIO output.
  • OsTask_AudioProcess: read the PDM values (transferred via DMA from the SPI). Separate the 2 channel PDM data and transform each of the PDM to PCM channels. One of the channels is written into a circular buffer to be processed in the DEEPCRAFT™ Studio Model. Additionally the 2 PCM channels get merged to an PCM Stream and output via AUDIO Amplifier.
  • OsTask_DataProcessing: This Task is getting the PCM 16 kHZ mono channel data from circular buffer and que this data to Siren Detection.
  • OsTask_UART: This Task prints the result buffer to Serial Interface.

Run and Test

The software can be flashed with Aurix™ Development Studio, see the chapter Compiling. The functionality of siren detection can be tested with the following test video (opens in a new tab) with different siren sounds. To see the results of the siren detection a terminal window need to be opened. Aurix™ Development Studio Terminal or another Terminal software like PuTTY (opens in a new tab) can be used for this purpose.



The Results can be observed in the Terminal Window.
For the UDP PCM raw data stream the IP Address on the HOST side has to be configured as "192.168.2.10" with the MAC "255.255.255.0". In Wireshark (opens in a new tab) the UDP steram looks like below: