In Machine Learning, quantization generally refers to the process of converting from floating point to fixed point integer. Imagimob Studio now has the quantization functionality available in the Edge tab, which allows you to quantize your model with a simple click. The quantization in Studio is a hybrid quantized method, meaning that only the network is quantized but not the preprocessor.
In this section, we are going to guide you through how to quantize a model in Imagimob Studio. A list of all supported Tensorflow layers that can be quantized in Imagimob Studio is available in the Quantization Layers. Notice that we only focus on the quantization here. For other functionalities in the Edge tab, please refer to Edge Optimization.
For you who are interested in delving into the maths of quantization, the theory behind it, and its implementation in Imagimob AI, please refer to quantization white paper (opens new window).
# Edge Build with quantization
Firstly, in Imagimob Studio, open the model .h5 file that you want to quantize and go to the Edge tab. Next, select Preprocessor and Quantized Network as the Build method and click on the Quantization Options button.
# Quantization Options
# Representative Data Set
For model quantization, you need to have the training data and/or validation data as the representative data, which should be a set of data that is large enough to represent typical values. Here we recommend that you use all of the training data.
As shown in the picture above, we provide two options to get the representative dataset. You can select
- Use Project file(.improj or .imdb) to choose the Imagimob project file that you used for training the model that you want to quantize.
- Recursive Directory Search if you know the training data location. Then you need to choose the directory where you saved the data. Furthermore, remember to use a correct File filter to get the right data files. And check the two boxes if the data files are in CSV format and they have a header row as well as timestamps as the first column.
# Check Introduced Quantization Error
With the quantization, the model might behave differently from the original model. Hence, to make sure that the quantized model predicts similarly to the original model, you need to check the quantization error before deploying it on the device. You can check the abs error, the mean error, and the argmax error of the quantized model's prediction compared to the original prediction. You can use the default values for them or customize them. In practice, the choice of the error metric depends on the application. For example, a regression model is more sensitive to the abs error or mean error. But for a classifier, the argmax error is more important.