Updates

• Adds support for Python3.11 and Python 3.12
• Removes support for Python3.7 and Python3.8
• Adds support for Tensorflow 2.16
• Adds support for GCC 13
• Adds support for BRD2705 platform
• Adds support for GSDK 4.3.3
• Adds support for latest Tensorflow-Lite Micro as of March 2024

Updates

• Added support for Tensorflow-2.13
• Added support for new embedded platform: BRD4401

New Datasets

• On/Off - Synthetically generated keywords "on" and "off"
• Yes/No - Synthetically generated keywords "yes" and "no"
• Ten Digits - Synthetically generated keywords "zero" through "ninie"

New Tutorials

• Model Quantization Tips - Provides tips on how to gain better quantization for your model
• Quantized LSTM - Describes how to create a quantized keyword spotting model with an LSTM layer

New Models

• Keyword Spotting - Numbers - CNN+LSTM keyword spotting model used to detect the keywords "zero" through "nine"

New Examples

• Audio Feature Generator Example - Shows how to manually process an audio sample in the Audio Feature Generator and run inference using TF-Lite and TF-Lite Micro

New Feature

• Added support for generate_quantization_report to TfliteConverter settings. This allows for generating a quantization report

Bug Fixes

• Fixed issue with loading models with float32 inputs into audio_classifier app
• Fixed issue with profiling Conv1D models

Updates

• Updated to Gecko SDK 4.3.0
• Updated to latest Tensorflow-Lite Micro
• Added new dataset: on_off - Contains synthetically generated samples of the keywords: "On" and "Off"
• Added new dataset: mit_ir_survey - Used for applying an "impulse response" to training samples
• Added new model: keyword_spotting_on_off_v3 - A more robust model for detecting the keywords: "On" and "Off"
• Updated tutorial: Keyword Spotting On/Off - Uses v3 model

Fixes/Improvements

• Fixed issue with model profiler not returning latency estimate when using the simulator
• Updated TfliteModel.predict() API to support models with multiple inputs and outputs
• Fixed issue with max_samples_per_class argument not properly updating class_counts dictionary in list_dataset_directory

Tutorials

• Cloud Logging with Weights & Biases - New tutorial for logging model development information to the cloud
• Keyword Spotting Pac-Man - Updated tutorial for usage with keyword_spotting_pacman_v3 model

Models

• Keyword Spotting Pac-Man v3 - New model that is more robust against background noise

Other Updates

• Updated the Gecko SDK to version 4.2.1
• Updated Tensorflow-Lite Micro to latest version as of February 2023
• Added more reference datasets

New Tutorials

• Synthetic Audio Dataset Generation - describes how to use Text-to-Speech (TTS) of various clouds to generate synthetic keyword audio datasets
• Keyword Spotting - Alexa - demonstrates how to use an embedded development board as the audio source/sink for the Alexa Voice Services backend

New Model

• Keyword Spotting - Alexa - this model is designed to detect the keyword: “Alexa”

New Python APIs

• Audio Dataset Generator - allows for generating a synthetic keyword audio datasets
• UART Stream - allows for streaming binary data between a Python script and embedded device via UART

General Updates / Improvements

• Updated to latest Tensorflowflow-Lite Micro as of November 2022
• Updated to Gecko SDK 4.1.3
• Removed dynamic installation of optional Python packages. Optional packages may now be installed with: pip install silabs-mltk[full]
• Updated TfliteModel APIs to provide additional calculated parameters used by Tensorflowflow-Lite Micro
• Added progress bar to evaluation CLI

Model / Dataset Updates

• Updated keyword_spotting_on_off_v2 to be more robust
• Added keyword_spotting_pacman_v2 to be more robust
• Updated Google Speech Commands to optionally remove invalid samples

New Features/Improvements

• Added support for Python 3.10
• Added new command: mltk tensorboard which invokes Tensorboard utility
• Added better support for Tensorflow Dataset API
• Updated ParallelImageDataGenerator and ParallelAudioDataGenerator to more efficiently process data which improves training latency
• Add new data preprocessing utilities
• Updated reference models to support running directly from python, e.g.: python basic_example.py

New Tutorials

• Model Debugging - This tutorial demonstrates how to debug a model specification Python script during model training. This allows for single-step debugging while the model is being actively trained.
• Add an Existing Script to the MLTK - Shows how to modify an existing model training script to work with the MLTK

New Examples

• TF-Lite Model API Example - Demonstrates how to use the TF-Lite Model package
• TF-Lite Micro Model API Example - Demonstrates how to use the TF-Lite Micro Model package.

New Models

• basic_example - This provides a basic example of how to create a model specification. It is based off the Simple MNIST convnet Keras example.
• basic_tf_dataset_example - This provides a basic example of how to create a model specification using the Tensorflow Dataset API for dataset management.
• image_tf_dataset - This provides an example of how to use the Tensorflow Dataset API with the various Tensorflow image augmentations to augment images during model training.
• audio_tf_dataset - This provides an example of how to use the Tensorflow Dataset API with the third-party Python library audiomentations to augment audio during model training.

Bug Fixes / Improvements

• Fixed install error by removing direct dependency on the netron Python package
  Netron is now only installed when using the view_model API
• Only automatically run .tflite model evaluation after training if the model specification supports .tflite model evaluation
• Add better support for generating stack tracing during assertions triggered in Windows/Linux apps

Bug Fixes/Improvements

• Set tflite-support<0.4.2 in setup.py. This works around dependency issue where tflite-support>=0.4.2 requires flatbuffer>=2.0 while Tensorflow requires < 2.0
• Fixed bug in AudioFeatureGenerator Python wrapper causing dynamic quantization during training to not work correctly
• Re-trained keyword_spotting_on_off_v2 using fixed AudioFeatureGenerator Python wrapper
• After training completes, do not automatically run evaluation if the model does not inherit EvaluateMixin

New Features/Improvements

• Added support for training on a remote SSH server: mltk ssh train
• Added tutorial for model training in the "cloud": Cloud Training Tutorial
• Updated ParallelAudioDataGenerator and ParallelImageDataGenerator to use joblib

New Features/Improvements

• Added Pac-Man demo: https://mltk-pacman.web.app
• Added Pac-Man tutorial
• Added BLE Audio Classifier C++ example application
• Add keyword_spotting_pacman reference model
• Added CLI option to disable GPU
• Optimized MVP-accelerated Conv2D kernels. Improved latency by 2x at expense of additional RAM
• Added Supported Hardware documentation page

Bug Fixes

• Fixed issue with reporting required tensor memory during profiling
• Fixed issue with reporting unsupported layers when profiling on device
• Fixed issue with recording audio when using activity detection block
• Fixed issue with building MVP Python wrapper on Winodws

General Updates

• Updated to Gecko SDK 4.1.0
• Updated to Tensorflow-Lite Micro June 8th, 2022
• Updates to support Tensorflow-2.9

New Tutorials

• Fingerprint Reader
• Image Classification

See all tutorials here

New C++ Examples

• fingerprint_authenticator
• image_classifier
• Updated audio_classifier to support new Audio Feature Generator settings

See all C++ examples here

New Reference Models

• keyword_spotting_on_off_v2
• rock_paper_scissors
• fingerprint_signature_generator

See all reference models here

Other Changes

• Added new settings Audio Feature Generator:
  • fe.activity_detection_enable - Enable the activity detection block. This indicates when activity, such as a speech command, is detected in the audio stream
  • fe.activity_detection_alpha_a - The activity detection “fast filter” coefficient. The filter is a 1-real pole IIR filter: computes out = (1-k)in + kout
  • fe.activity_detection_alpha_b - The activity detection “slow filter” coefficient. The filter is a 1-real pole IIR filter: computes out = (1-k)in + kout
  • fe.activity_detection_arm_threshold - The threshold for when there should be considered possible activity in the audio stream
  • fe.activity_detection_trip_threshold - The threshold for when activity is considered detected in the audio stream
  • fe.dc_notch_filter_enable - Enable the DC notch filter. This will help negate any DC components in the audio signal
  • fe.dc_notch_filter_coefficient - The coefficient used by the DC notch filter, DC notch filter coefficient k in Q(16,15) format, H(z) = (1 - z^-1)/(1 - k*z^-1)
  • fe.quantize_dynamic_scale_enable - Enable dynamic quantization of the generated audio spectrogram. With this, the max spectrogram value is mapped to +127, and the max spectrogram minus fe.quantize_dynamic_scale_range_db is mapped to -128. Anything below max spectrogram minus fe.quantize_dynamic_scale_range_db is mapped to -128.
  • fe.quantize_dynamic_scale_range_db - The dynamic range in dB used by the dynamic quantization
  • samplewise_norm.rescale - Value to scale each element of the sample: norm_sample = sample * . The model input dtype should be a float32
  • samplewise_norm.mean_and_std - Normalize the sample by the mean and standard deviation: norm_sample = (sample - mean(sample)) / std(sample). The model input dtype should be a float32