I spent my 2022-2023 Winter Break coding. Of course I did. I always do.
This time I spent my time learning about and creating a tool for FIRST Tech Challenge (FTC) teams to train their robots to recognize objects using a camera. This type of task has been apart of FTC for about 4 or 5 years now. In the past we have used Vuforia and their VuMarks technology. We have also used OpenCV for object detection in the past. But this had a very steep learning curve that required students to use Android Studio when making their code. Most of my students prefer the OnBot Java setup for its ease of use while still allowing them to use all of their programming skills from AP CSA.
FTC uses TensorFlowLite Object Detection and has created FIRST Machine Learning Toolchain (FTC-ML) so that teams can train their own machine learning models. To access this, teams need to have a login with the FIRST Inspires website. This is limiting for two reasons: 1) some students across the world may not be on an official team and want to learn about machine learning. And 2) my school district does not allow students to have a FIRST Inspires account. (This might have to do with the management of private student information)
So, I set out to learn how to make my own TensorFlowLite (TFLite) machine learning models for Object Detection. After about two weeks, lots of reading, lots of coding, lots of mistakes, lots of issues, etc., I finally have a working tool that all teams of FTC can use to create their models. I have created a tutorial video for the tool below.
The tool itself it embedded into a Google Colab Notebook. I had never used this technology before and I have become very impressed with it. It is essentially a Google Doc for Jupyter Notebooks. It is a web-based interface for Python programming. Using Google Colab allows us to use Google run servers and GPUs to make the machine learning model training pretty quick. I have been able to successfully make TFLite models in a matter of hours. This includes collecting visual data, annotating the data, and training the model.
Below is my tutorial video with the Google Colab Notebook linked.