TensorFlow Lite Model Maker (Updated for CENTERSTAGE)

This year I have updated my tool for creating an object detection model for the FIRST Tech Challenge CENTERSTAGE. See my post from last year to read about why I created this tool.

The biggest update to the tool was due to Google updating Colab to use Python version 3.10. Several of the packages that are used in the tool have not yet been updated to use Python 3.10 and still require 3.9 to be used. This means last year’s tool is broken. To fix this issue, I now use miniconda to create a Python 3.9 virtual environment that can run inside of the Colab Notebook.

The second biggest update was the tool used to create the annotations. Last year we used CVAT.ai. However, they no longer allow for export with free accounts. So now the tool uses Kili-Technology.com. This service works very similar to CVAT and allows for export of the annotations. As a plus it also exports the data in a format that the tflite model maker is already compatible with. This allowed me to get rid of a bunch of code that had to reformat the exports from CVAT.

If your FTC team wants to make an object detection model for your robot, check out the link to the Google Colab Notebook below and watch the tutorial video to walk you through how to use it.

Link to Google Colab Notebook

https://colab.research.google.com/drive/1U4g6Lf_PZ9DSS43DDHdqTWQ3WGQyTAUn

Video Tutorial

Making a TensorFlow Lite Model for FIRST Tech Challenge

I spent my 2022-2023 Winter Break coding. Of course I did. I always do.

This time I spent my time learning about and creating a tool for FIRST Tech Challenge (FTC) teams to train their robots to recognize objects using a camera. This type of task has been apart of FTC for about 4 or 5 years now. In the past we have used Vuforia and their VuMarks technology. We have also used OpenCV for object detection in the past. But this had a very steep learning curve that required students to use Android Studio when making their code. Most of my students prefer the OnBot Java setup for its ease of use while still allowing them to use all of their programming skills from AP CSA.

FTC uses TensorFlowLite Object Detection and has created FIRST Machine Learning Toolchain (FTC-ML) so that teams can train their own machine learning models. To access this, teams need to have a login with the FIRST Inspires website. This is limiting for two reasons: 1) some students across the world may not be on an official team and want to learn about machine learning. And 2) my school district does not allow students to have a FIRST Inspires account. (This might have to do with the management of private student information)

So, I set out to learn how to make my own TensorFlowLite (TFLite) machine learning models for Object Detection. After about two weeks, lots of reading, lots of coding, lots of mistakes, lots of issues, etc., I finally have a working tool that all teams of FTC can use to create their models. I have created a tutorial video for the tool below.

The tool itself it embedded into a Google Colab Notebook. I had never used this technology before and I have become very impressed with it. It is essentially a Google Doc for Jupyter Notebooks. It is a web-based interface for Python programming. Using Google Colab allows us to use Google run servers and GPUs to make the machine learning model training pretty quick. I have been able to successfully make TFLite models in a matter of hours. This includes collecting visual data, annotating the data, and training the model.

Below is my tutorial video with the Google Colab Notebook linked.

FTC TFLite Model Maker - Google Colab Notebook

Garage-pi

In the summer of 2019 I realized that I needed a smart garage door opener because I kept leaving the garage open when leaving the house in the morning. I needed to know if I left it open and have the ability to close it without having to drive home.

I found some instructions and code on howchoo website. However, I didn’t like it that much. I needed it to work securely from outside of my home network. So I spent A LOT of time updating and revising their code and system. It now has a login system (the crypto is industry standard), and SSL/TLS built in with Let’s Encrypt.

Scratch

“With Scratch, you can program your own interactive stories, games, and animations — and share your creations with others in the online community.

Scratch helps young people learn to think creatively, reason systematically, and work collaboratively — essential skills for life in the 21st century.

Scratch is a project of the Lifelong Kindergarten Group at the MIT Media Lab. It is provided free of charge.”
— https://scratch.mit.edu/about

Here are some of the few examples I have made in preparation for teaching Exploring Computer Science:

I have used the Pythagorean Puzzle to help my students discover the Pythagorean theorem themselves! Not only can this tool be used to teach students how to code, but it can be used by educators to create their own interactive lessons/applets.

My App Inventor Example

“MIT App Inventor is an innovative beginner’s introduction to programming and app creation that transforms the complex language of text-based coding into visual, drag-and-drop building blocks. The simple graphical interface grants even an inexperienced novice the ability to create a basic, fully functional app within an hour or less. ”
— http://appinventor.mit.edu/explore/about-us.html

Click here to see my App Inventor example. Although I don't teach history/geography and I can code without it, this was my first attempt at using this software.

MIT's App Inventor provides an easy way for everyone (that means you and your students) to create apps for the Android operating system. In my Exploring Computer Science course, App Inventor allows my students to think computationally and explore how technology can empower individuals.