Skip to content

Commit

Permalink
Merge pull request #3966 from naushir/develop
Browse files Browse the repository at this point in the history
Some changes for AI Kit/HAT+
  • Loading branch information
nathan-contino authored Dec 10, 2024
2 parents 659eba6 + d50cc58 commit 5553383
Show file tree
Hide file tree
Showing 4 changed files with 56 additions and 15 deletions.
2 changes: 1 addition & 1 deletion documentation/asciidoc/accessories/ai-hat-plus/about.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -72,4 +72,4 @@ image::images/ai-hat-plus-installation-02.png[width="60%"]

== Get started with AI on your Raspberry Pi

To start running AI accelerated applications on your Raspberry Pi, check out our xref:../computers/ai.adoc[Getting Started with AI] guide.
To start running AI accelerated applications on your Raspberry Pi, check out our xref:../computers/ai.adoc[Getting Started with the AI Kit and AI HAT+] guide.
2 changes: 1 addition & 1 deletion documentation/asciidoc/accessories/ai-kit/about.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -90,4 +90,4 @@ WARNING: Always disconnect your Raspberry Pi from power before connecting or dis

== Get started with AI on your Raspberry Pi

To start running AI accelerated applications on your Raspberry Pi, check out our xref:../computers/ai.adoc[Getting Started with AI] guide.
To start running AI accelerated applications on your Raspberry Pi, check out our xref:../computers/ai.adoc[Getting Started with the AI Kit and AI HAT+] guide.
65 changes: 53 additions & 12 deletions documentation/asciidoc/computers/ai/getting-started.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ Serial Number: HLDDLBB234500054
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP
----

NOTE: AI HAT+ devices may show `<N/A>` for `Serial Number`, `Part Number` and `Product Name`. This is expected, and does not impact functionality.
+
Additionally, you can run `dmesg | grep -i hailo` to check the kernel logs, which should yield output similar to the following:
+
Expand Down Expand Up @@ -96,24 +98,20 @@ The `rpicam-apps` suite of camera applications implements a xref:camera_software

The following demos use xref:camera_software.adoc#rpicam-hello[`rpicam-hello`], which by default displays a preview window. However, you can use other `rpicam-apps` instead, including xref:camera_software.adoc#rpicam-vid[`rpicam-vid`] and xref:camera_software.adoc#rpicam-still[`rpicam-still`]. You may need to add or modify some command line options to make the demo commands compatible with alternative applications.

To begin, download the post-processing JSON files required for the demos. These files determine which post-processing stages to run and configure the behaviour of each stage. For example, you can enable, disable, strengthen, or weaken the strength of the temporal filtering in the object detection demos. Or you could enable or disable the output mask drawing in the segmentation demo.

To download the entire collection of post-processing JSON files, clone the `rpicam-apps` repo. Run the following command to clone only the most recent commit from the repo, saving space:
To begin, run the following command to install the latest `rpicam-apps` software package:

[source,console]
----
$ git clone --depth 1 https://github.com/raspberrypi/rpicam-apps.git ~/rpicam-apps
$ sudo apt update && sudo apt install rpicam-apps
----

TIP: The commands provided in subsequent sections use the JSON files in this repository. To make it easy to reference these files, this command creates the cloned `rpicam-apps` directory in your home folder. If you modify the location of this directory, you must also alter the demo commands below to reference the new location of the JSON files.

==== Object Detection

This demo displays bounding boxes around objects detected by a neural network. To disable the viewfinder, use the xref:camera_software.adoc#nopreview[`-n`] flag. To return purely textual output describing the objects detected, add the `-v 2` option. Run the following command to try the demo on your Raspberry Pi:

[source,console]
----
$ rpicam-hello -t 0 --post-process-file ~/rpicam-apps/assets/hailo_yolov6_inference.json --lores-width 640 --lores-height 640
$ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov6_inference.json
----

Alternatively, you can try another model with different trade-offs in performance and efficiency.
Expand All @@ -122,21 +120,21 @@ To run the demo with the Yolov8 model, run the following command:

[source,console]
----
$ rpicam-hello -t 0 --post-process-file ~/rpicam-apps/assets/hailo_yolov8_inference.json --lores-width 640 --lores-height 640
$ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov8_inference.json
----

To run the demo with the YoloX model, run the following command:

[source,console]
----
$ rpicam-hello -t 0 --post-process-file ~/rpicam-apps/assets/hailo_yolox_inference.json --lores-width 640 --lores-height 640
$ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolox_inference.json
----

To run the demo with the Yolov5 Person and Face model, run the following command:

[source,console]
----
$ rpicam-hello -t 0 --post-process-file ~/rpicam-apps/assets/hailo_yolov5_personface.json --lores-width 640 --lores-height 640
$ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov5_personface.json
----

==== Image Segmentation
Expand All @@ -145,7 +143,7 @@ This demo performs object detection and segments the object by drawing a colour

[source,console]
----
$ rpicam-hello -t 0 --post-process-file ~/rpicam-apps/assets/hailo_yolov5_segmentation.json --lores-width 640 --lores-height 640 --framerate 20
$ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov5_segmentation.json --framerate 20
----

==== Pose Estimation
Expand All @@ -154,8 +152,51 @@ This demo performs 17-point human pose estimation, drawing lines connecting the

[source,console]
----
$ rpicam-hello -t 0 --post-process-file ~/rpicam-apps/assets/hailo_yolov8_pose.json --lores-width 640 --lores-height 640
$ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov8_pose.json
----

=== Alternative Package Versions

The AI Kit and AI HAT+ do not function if there is a version mismatch between the Hailo software packages and device drivers. In addition, Hailo's neural network tooling may require a particular version for generated model files. If you require an specific version, complete the following steps to install the proper versions of all of the dependencies:

. If you have previously used `apt-mark` to hold any of the relevant packages, you may need to unhold them:
+
[source,console]
----
$ sudo apt-mark unhold hailo-tappas-core hailort hailo-dkms
----

. Install the required version of the software packages:
+
[tabs%sync]
======
v4.18::
To install version 4.18 of Hailo's neural network tooling:
+
+
[source,console]
----
$ sudo apt install hailo-tappas-core=3.29.1 hailort=4.18.0 hailo-dkms=4.18.0-2
----
+
[source,console]
----
$ sudo apt-mark hold hailo-tappas-core hailort hailo-dkms
----
v4.17::
To install version 4.17 of Hailo's neural network tooling:
+
[source,console]
----
$ sudo apt install hailo-tappas-core-3.28.2 hailort=4.17.0 hailo-dkms=4.17.0-1
----
+
[source,console]
----
$ sudo apt-mark hold hailo-tappas-core hailort hailo-dkms
----
======

=== Further Resources

Expand Down
2 changes: 1 addition & 1 deletion documentation/index.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
"subpath": "camera_software.adoc"
},
{
"title": "AI software",
"title": "AI Kit and AI HAT+ software",
"description": "Software and libraries for artificial intelligence on a Raspberry Pi hardware",
"image": "full-sized/ai.png",
"subpath": "ai.adoc"
Expand Down

0 comments on commit 5553383

Please sign in to comment.