Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update stop sign detector to run without EdgeTPU #953

Open
Ezward opened this issue Nov 19, 2021 · 5 comments
Open

Update stop sign detector to run without EdgeTPU #953

Ezward opened this issue Nov 19, 2021 · 5 comments

Comments

@Ezward
Copy link
Contributor

Ezward commented Nov 19, 2021

The StopSignDetector runs a canned version of mobilenet for the EdgeTPU. This means that anyone that wants to use the stop sign detector must have an EdgeTPU. We should generalize the stop sign detector so it can also run the model on an RPi or a Nano.

Current code is using this model:
https://github.com/google-coral/edgetpu/raw/master/test_data/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
That model will recognize a stop sign without any additional training.

So we could use that same ssd mobilenet v2 coco in a more generic tflite variation, then modify the code run it on the RPi; see https://github.com/google-coral/edgetpu/blob/master/test_data/ssd_mobilenet_v2_coco_quant_postprocess.tflite.

See https://medium.datadriveninvestor.com/mobile-object-detector-with-tensorflow-lite-9e2c278922d0
and https://www.tensorflow.org/lite/examples/object_detection/overview for some related info.

Further, we could then compile the .tf file using the Nvidia compiler to get a file that will run fast on the Nano's GPU.

  1. Generalize the code so it can run either the EdgeTPU version or the CPU (tflite) version.
  2. Add configuration to choose either EdgeTPU or CPU
  3. Add support for JetsonNano GPU;
    • compile the model to an optimized version for the Jetson;
    • modify the code to enable it if on the Nano.
    • Add configuration to select GPU version
@Ezward
Copy link
Contributor Author

Ezward commented Oct 14, 2022

It looks like Tensorflow has changed their approach to object detection; https://www.tensorflow.org/lite/examples/object_detection/overview So we might want to use the Tensorflow Object Detection API https://github.com/tensorflow/models/tree/master/research/object_detection

There are now better models that mobilenet; a little research on state of the art my yield higher accuracy than mobilenet can provid.

@Ezward
Copy link
Contributor Author

Ezward commented Feb 26, 2023

Note on performance; remember that we are also running the normal autopilot model, so performance described above will be low because we are trying to run multiple models. No one has done this as yet, so we don't know what the performance would be.

  • RaspberryPi: The original stop sign detector used a USB connected Coral Edge TPU; that is still an option and relatively cheap at about $60 https://coral.ai/products/accelerator/.
  • Jetson Nano: Nano users generally run the autopilot using a TensorRT model on the GPU. If you also want to do object/person detection, then you are better off running a tflite autopilot model on the cpu and a TensorRT object detection model on the GPU; in that way you can run the heavy model on the GPU.

@Ezward
Copy link
Contributor Author

Ezward commented Feb 26, 2023

Ok, now Google has deprecated https://github.com/tensorflow/models/tree/master/research/object_detection, and now recommends https://github.com/tensorflow/models/tree/master/official/vision Google sucks. This thing looks harder to use, perhaps just go back to the original mobilenet suggestion in the initial issue description.

@Ezward
Copy link
Contributor Author

Ezward commented Feb 27, 2023

A user on discord also has used this tensorflow-lite example for people detection on small images (160x120) on the RaspberryPi

@cfox570
Copy link
Contributor

cfox570 commented May 19, 2023

See Pull Request StopSign and Cone Detection #1131
I used TensorFlow Lite Object Detection with a custom cone model to implement traffic cone avoidance. It was tested with Raspberry Pi 4. I am adding a pull request for the docs Create object_detection.md #51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants