Bring Your Own Model

The SyNAP AI Toolkit offers exceptional flexibility by enabling execution of diverse neural networks, efficiently utilizing the available hardware accelerators. The SyNAP toolkit allows you to convert a model from its original format to an internal representation optimized for the target hardware. In this tutorial, you will learn how to bring your own model, optimize it for the Synaptics Astra SL16xx processors, and achieve high-performance inference speeds.

For this tutorial, you will optimize the model by converting a tflite model to synap model format. You can import the mobilenet_v2 TensorFlow Lite model from Kaggle and compile it specifically for the NPU/GPU of the Synaptics Astra board.

Compiling the model allows you to optimize it for the exact NPU or GPU resources on-chip, which can significantly improve inference speed. There are also advanced operations, such as quantization, preprocessing, and even secure signing and encryption available in the SyNAP AI toolkit manual, you'll be using in this guide.

info

Models from TensorFlow, Caffe, or ONNX frameworks can be optimized for any Astra chip in a single CLI command.

synap convert --target {$CHIP_MODEL} --model example.{$MODEL_FORMAT}

For example:

synap convert --target SL1680 --model example.tflite

Compiling the model

If you want to set up the environment and optimize a model on your local machine, you can follow this tutorial.

To save time on environment setup, you can do your model compilation and optimization on a Google Colab Notebook which you can run right now:

Open Compiling a model to Synaptics Astra Google Colab

important

Once you complete the Colab, you will have compiled MobileNetv2 for the NPU on a Synaptics Astra™ Machina™ SL1680 / SL1640 and for GPU on Machina SL1620. Once it's finished, you need to download converted-model/model.synap from Colab to run on your board in the next section.

Upload model to Astra board

You should now have compiled and downloaded model.synap to your host machine from Colab.

You can now upload it to your Machina board in a terminal with:

adb push model.synap /home/weston/model.synap

Time inference on your Machina Board

On your dev machine, open an ADB shell to Astra:

adb shell

Then run the synap_cli command using random input:

cd /home/weston/
synap_cli -r 5 random

You should see the following output for running inference using our optimized mobilenet_v2-1-0-224-quantized.tflite:

Flush/invalidate: yes
Loop period (ms): 0
Network inputs: 1
Network outputs: 1
Input buffer: input size: 150528 : random
Output buffer: output size: 1001
Predict #0: 9.01 ms
Predict #1: 1.90 ms
Predict #2: 1.89 ms
Predict #3: 1.89 ms
Predict #4: 1.89 ms
Inference timings (ms):  load: 29.18  init: 15.56  min: 1.89  median: 1.89  max: 9.01  stddev: 2.85  mean: 3.31

info

The above results are from the SL1680 Board which runs the model on NPU.

Run image classification using the model

There are some preinstalled sample images on the Astra board, such as goldfish_224x224.jpg:

To run image classification on this image using the model you just optimized for NPU/GPU, you can run:

MODELS=/usr/share/synap/models/
synap_cli_ic -m model.synap $MODELS/image_classification/imagenet/sample/goldfish_224x224.jpg

You should see the highest confidence result is class 2. You can look this up on the imagenet labels list.

Congratulations

You've just imported a model from Kaggle, optimized it for NPU, and ran it on Synaptics Astra!

This was a quick one, the Optimize Ultralytics YOLO for the edge using SyNAP tutorial will guide you on model optimization in detail.