The Robomow 622 Haar Cascade Object Detector Project
The following outlines the steps in this project.
Step 1 - To build the Garage for the Robomow.
The article Constructing a Garage for Robomow 622 walks you through how I built my garage. There are many ways to accomplish this task. You can also buy pre-built garages now a days as well. Here is a link. If you are handy and you want to tackle this yourself, you can do a search on youtube and find plenty of examples of how others have done it. I had a lot of fun building this garage.
Step 2 - The mechanism for opening and closing of the garage door
There are many creative examples of how to accomplish this on youtube. For my garage, the first solution was to leverage gravity. When the robomow started a lawn mowing session it would and the door was weighted top heavy so it would stay open while cutting the lawn. After it completed its run and needed to recharge, it would come back and after successfully connecting to the charging station would push a leaver connected to the garage door that would push the garage door just far enough that it would .
The second solution is to leverage computer vision and the machine learning Cascade Object Detection algorithm to detect the Robomow and then send the command to open the garage door.
WARNING DISCLAIMER: Please know that the first solution works just fine. The second solution is just an excuse for us to get to learn a little bit about machine learning.
Step 3 - Understanding the components of the machine trained Robomow Garage Door Opener
There are 2 components to this garage door opener. One is the hardware, which accepts the instructions to physically open the garage and close it. The other is the software, we need to write, that uses a surveillance video camera to watch and detect when the Robomow comes into view and trigger the appropriate action.
The hardware we will use is listed below
- Video Camera -Video Camera
- 3 - A really cool low cost credit size computer called that is the brains of this system. We will attach surveillance video camera andto. The price for this amazing device is between $10-$30. Yes! I am not kidding! Includes built in WIFI and a pretty darn powerful processor.
- - Ato open andwhen we detect the Robomow.
- - To notify us when the Robomow has successfully docked with the charging station.s
The computer runs a flavor of a powerful modern operating system called Linux. We will use it to write the software that will turn the video camera on at the right time, then run the machine learning Robomow Cascade Object detector and if the Robomow is detect send the command to theto open or at the appropriate time. In addition the computer has wifi built in which we will use to stream this to the web so we can watch it in action from this web site. Sound like fun?
STEP 4 - SETUP
The software we need listed below.
Operating System - a flavor of linux that has been optimized for the operating system.
Python 3 - a really nice programming language that is object oriented as of late and is very simple and interfaces really nicely with all components attached to the computer. All the software we write will use this programming language.
OpenCV - This is an amazing Computer Vision library that has the capabilities to train a Cascade plus much much more. We will be using it to train of Robomow 622 Object Detector.
Camera API - we will be using this library to capture the video of the Robomow so we can detect it.
Our Own Custom Software that will
- Capture each Video Frame from Rasbperry Camera
and run our trained Robomow Cascade Model on each video frame.
- Communicate with the Servo Motors to tell it how far to rotate in order to get the door open or closed.
- Receive notifications from theto tell us when the garage door should be closed.
- Stream the video feed of all of this in action to this web site for our viewing pleasure.
Getting your Computer up and running
The best place to get started with is at www.rasbperrypi.org. You can purchase this computer, the, and theat just about any major electronics store online or offline. I purchased the computer and its components from MicroCenter and Frys.
Getting the's operating system installed is straight forward. You can follow the instructions with the purchase of your pi, but getting OpenCV and all of its dependancies loaded properly I found very challenging. Also getting started with OpenCV I also felt was challenging.
I found this Practical Python and OpenCV + Case Studies book / course by Adrian Rosebrock to be very effective at tying all of these concepts together. What I really like about Adrian teaching style is he explains his concepts usually in code and more importantly line by line in a very structured consistent way that makes it easy to follow along and understand. Check it out when you get a chance https://www.pyimagesearch.com. His site has many additional great AI/Deep Learning/Computer Vision articles.
One of the nice things when you purchase this book is he gives you a completely configured version of the Rasbperry PI operating system with OpenCV version 3.1 installed and ready to go. This will save you many many hours fighting trying to get your environment setup properly.
Creating the Robomow Haar Cascade Object Detector
The overall goal is to feed the OpenCV Cascade training function a bunch of negative and positive images of your Robomow which it uses to create the different stages/cascades. These end up as collection of stage/cascade xml files and upon successful training creates one final xml file which OpenCV will use to detect the Robomow on each frame we feed it from the video camera. Upon successful detection of the Robomow our python program will send a message to the to turn 45 degrees which will either open or close the garage. Make sense? If not don't worry, just continue following along and you will see it all in action in just a moment.
Video below shows theCascade machine trained Robomow Object Detector in action.
First thing we will want to do is get a nice clean well organized directory structure that will allow us to easily manage training multiple models so that in the end we can choose the best trained model and understand what elements where needed to build the model (xml files generated).
To help facilitate this I created a separate repo that stores all the videos, positive, negative images, scripts to help create these images, scripts to train the videos and build the xml files. Lets take a look at it.
Check out the article below to see what the actual process of training cascade actually looks like. In this article I will describe this repo in detail. How the directory structure is setup. How to easily generate images for training. Setting yourself up for success to get the best trained model.