In this guest post, Ramona Rayner from our partner Sony shows you how to quickly explore different models and AI capabilities, and how you can easily build applications on top of the Raspberry Pi AI Camera.

The recently launched Raspberry Pi AI Camera is an extremely capable piece of hardware, enabling you to build powerful AI applications on your Raspberry Pi. By offloading the AI inference to the IMX500 accelerator chip, more computational resources are available to handle application logic right on the edge! We are very curious to see what you will be creating and we are keen to give you more tools to do so. This post will cover how to quickly explore different models and AI capabilities, and how to easily build applications on top of the Raspberry Pi AI Camera.

If you didn’t have the chance to go through the Getting Started guide, make sure to check that out first to verify that your AI Camera is set up correctly.

Getting Started : https://www.raspberrypi.com/documentation/accessories/ai-camera.html#getting-started

Explore pre-trained models

A great way to start exploring the possibilities of the Raspberry Pi AI Camera is to try out some of the pre-trained models that are available in the IMX500 Model Zoo. To simplify the exploration process, consider using a GUI Tool, designed to quickly upload different models and see the real-time inference results on the AI Camera.

IMX500 Model Zoo : https://github.com/raspberrypi/imx500-models

GUI Tool : https://github.com/SonySemiconductorSolutions/aitrios-rpi-sample-app-gui-tool

In order to start the GUI Tool, make sure to have Node.js installed. (Verify Node.js is installed by running in the terminal.) And build and run the tool by running the following commands in the root of the repository:node --version

https://github.com/SonySemiconductorSolutions/aitrios-rpi-sample-app-gui-tool

make build

./dist/run.sh

The GUI Tool will be accessible on http://127.0.0.1:3001. To see a model in action:

• Add a custom model by clicking the button located at the top right corner of the interface.ADD
• Provide the necessary details to add a custom network and upload the file, and the (optional) file.network.rpklabels.txt
• Select the model and navigate to to see the model in action!Camera Preview

Here are just a few of the models available in the IMX500 Model Zoo:

Download address:

mobilenet_v2, network.rpk: https://raw.githubusercontent.com/raspberrypi/imx500-models/main/imx500_network_mobilenet_v2.rpk

mobilenet_v2, imagenet_labels.txt: https://github.com/pytorch/hub/blob/master/imagenet_classes.txt

efficientdet_lite0_pp, network.rpk: https://raw.githubusercontent.com/raspberrypi/imx500-models/main/imx500_network_efficientdet_lite0_pp.rpk

efficientdet_lite0_pp, imagenet_labels.txt: https://github.com/amikelive/coco-labels/blob/master/coco-labels-paper.txt

deeplabv3plus, network.rpk: https://raw.githubusercontent.com/raspberrypi/imx500-models/main/imx500_network_deeplabv3plus.rpk

posenet, network.rpk: https://raw.githubusercontent.com/raspberrypi/imx500-models/main/imx500_network_posenet.rpk

Exploring the different models gives you insight into the camera’s capabilities and enables you to identify the model that best suits your requirements. When you think you’ve found it, it’s time to build an application.

Building applications

Plenty of CPU is available to run applications on the Raspberry Pi while model inference is taking place on the IMX500. To demonstrate this we’ll run a Workout Monitoring sample application.

Workout Monitoring: https://github.com/SonySemiconductorSolutions/aitrios-rpi-sample-apps/tree/main/examples/workout-monitor

The goal is to count real-time exercise repetitions by detecting and tracking people performing common exercises like pull-ups, push-ups, ab workouts and squats. The app will count repetitions for each person in the frame, making sure multiple people can work out simultaneously and compete while getting automated rep counting.

To run the example, clone the sample apps repository and make sure to download the HigherHRNet model from the Raspberry Pi IMX500 Model Zoo.

sample apps repository : https://github.com/SonySemiconductorSolutions/aitrios-rpi-sample-apps

HigherHRNet model : https://raw.githubusercontent.com/raspberrypi/imx500-models/main/imx500_network_higherhrnet_coco.rpk

Make sure you have OpenCV with Qt available:

sudo apt install python3-opencv

And from the root of the repository run:

python3 -m venv venv --system-site-packages
source venv/bin/activate
cd examples/workout-monitor/
pip install -e .

Switching between exercises is straightforward; simply provide the appropriate argument as one of , , or .--exercisepulluppushupabworkoutsquat

workout-monitor --model 
/path/to/imx500_network_higherhrnet_coco.rpk
 --exercise pullup

Note that this application is running:

• Model post-processing to interpret the model’s output tensor into bounding boxes and skeleton keypoints
• A tracker module (ByteTrack) to give the detected people a unique ID so that you can count individual people’s exercise reps
• A matcher module to increase the accuracy of the tracker results, by matching people over frames so as not to lose their IDs
• CV2 visualisation to visualise the results of the detections and see the results of the application

And all of this in real time, on the edge, while the IMX500 is taking care of the AI inference!

Now both you and the AI Camera are testing out each other’s limits. How many pull-ups can you do?

Original link： https://www.raspberrypi.com/news/bringing-real-time-edge-ai-applications-to-developers/