Machine Learning Maze Runner

The first thing that had to be done was that a camera needed to be mounted to the front of the robot. I created a model to fit hanging off the front to get a level view of each obstacle but found that objects could not be identified up close due to the shallow field of view on the Raspberry Pi 3 Camera. To solve this, another small mount was made, placed further back, and raised up to get a more encompassing view of each object. This ended up in our final design.

A large part of this project was software-based. The Create3 uses ROS to interface with its motors and other drivers, and Python was used to write the controlling software. We used Google Teachable Machines to create a model from object samples. To increase accuracy and repeatability, the onboard camera was used to take our training data images. Different angles, orientations, and lighting conditions were used to improve each model.

Once the image model was created, it could be used dynamically in the robot. A ROS subscriber was created that allowed the robot to use its integrated IR sensor array. Once averaged together, we could use the front three sensors as a distance gauge to tell when to stop and "read" the next object. A ROS publisher was created to allow information to be sent to the Create's motors, telling them how or when to turn or move forward. Some basic logic was then used to allow for the robot to move forward until it found an object, it would then take a picture, "decide" which direction to go based on a truth table, turn, and continue onwards until the number of objects was met. As a fun challenge, I created a Midi interpreter for the robot that could take a Midi file and convert it into a format that the Create3 could play using another publisher. I intend to use this in a future project to allow multiple robots to "sing" in harmony. All code can be found here.