Building Capacitive Sensing machine learning models
This tutorial shows how to build neural network–based algorithms for touch detection, gesture recognition, and slider position estimation in the DEEPCRAFT™ Studio for the PSOC4™ 4100S Max pioneer kit, and how these models match the kit’s classical algorithms in performance. The tutorial concludes with a demo that runs the classic and neural network–based algorithms in parallel on the board demonstrating comparable behavior and accuracy.
This tutorial covers the building of the following neural network models:
- Touch Button Detection: detects a touch event whenever a button on the kit is pressed
- Slider Gesture Recognition: detects left and right swipe gestures on the slider
- Slider Position Calculation: determines the slider position at the point where the slider is touched
What is Capacitive Sensing?
Capacitive sensing is a versatile and efficient way to measure capacitance and detect finger touches in user interface panel applications such as capacitive touch buttons, sliders, touchpads, touch screens, and proximity sensors. Refer to Infineon CAPSENSE™ Controllers to know more.
Prerequisites
Before starting this tutorial, ensure you meet the following requirements:
- Register an account and Install DEEPCRAFT™ Studio
- PSOC4™ 4100S Max pioneer kit
- Install ModusToolbox™
- Download the CapSense Project
CapSense Project Structure
The CapSense project is organized into two main folders: data_streaming and data_processing.
data_streaming | Folder Description |
|---|---|
mtb_signals_streaming.hex | Use the .hex file to flash the kit with the Streaming firmware using ModusToolbox™ Programmer. |
mtb_signals_streaming-firmware | We are also providing the firmware source files used to build the hex file. You can customize the firmware in ModusToolbox™, if needed. The firmware uses Streaming Protocol Version 2. For instructions on modifying the existing firmware or implementing new streaming firmware for the board, refer to Registering sensors and boards for data streaming using Protocol version 2. |
data_processing | Folder Description |
|---|---|
| mtb_nn_demo_project | Runs classic signal-processing algorithms and neural network–based algorithms in parallel on the board. |
| imagimob_nn_projects | consists of neural network based projects build in Studio: position_nn: project to create neural network for slider position calculation model gesture_nn: project to create neural network for gesture detection model button_nn: project to create neural network for touch button detection model Each project consists of Data folder that contains Datasets for training, testing, and validation, including scripts to format data into the required structure and Models folder that trained model artifacts produced for each project/configuration. |
The Touch Button Detection and Slider Gesture Recognition neural network models are build using Classification projects and Slider Position Calculation neural network model using a Regression project. Navigate to CapSense Project> data_preprocessing and copy the imagimob_nn_projects folder into the Studio Workspace. The project contains separate folders for button, gesture, and position, each with its own project file, data, model, and related files.
Tutorial Flow
- Flashing the Streaming Firmware
- Data Preparation
- Preprocessing
- Model Training and Evaluation
- Code Generation
Flashing the Streaming Firmware for PSoC4™ 4100S Max pioneer kit
After downloading the CapSense Project, navigate to data-streaming> mtb_signals_streaming.hex. To collect data with the kit, flash the board with the HEX file, which contains the firmware required to stream data from from the CAPSENSE™ Buttons and Slider into the Studio. Refer to Streaming Firmware for PSoC™ 4100S Max Pioneer Kit for detailed flashing instructions.
Data Preparation: Collecting and Labeling, Importing, and Distributing Data
The first step is to collect and label data from button, Slider position and Slider gesture present on the kit using Graph UX. After collecting and labeling data, we will import data into the respective projects followed by distributing the data into the different datasets. Refer to Data Preparation for more information.
Preprocessing
The second main step is preprocessing where the data signal processing techniques are applied to the sensor data, such as windowing and filtering techniques to reduce the noise and extract the features of the sensors data. Refer to Preprocessing for more information.
Model Training
After preprocessing, the data is passed to the model for training. In this step, we will generate, train and evaluate the models to find the best fitting model. Refer to Model Training for more information.
Code Generation
The last step is to generate the code after evaluating the model. We will generate the C code for the selected model in DEEPCRAFT™ Studio. Refer to Code Generation for more information.
Conclusion
After completing this tutorial, you will be able to build a wide range of home, consumer, industrial, and automotive applications. You can use our streaming protocol to develop firmware that streams data from the sensors on your board, tailored to your specific use case, and then use DEEPCRAFT™ Studio to build and train the corresponding model.