Reef mapping using a drone
The goal with this effort is to create an automated system to map reefs and other shallow-sea ecosystems. Usually, reef mapping is either done at a large scale using satellite imagery, or manually by divers. The manual methods are time consuming and expensive, so it's hard to do them frequently. With this project we're combining robotics and artificial intelligence to collect and analyze underwater images to create these maps more easily and hopefully more regularly.
The design for our DIY glider is based on that of Alex Williams, which is well documented on his project page on the Hackaday website. You can also check out his 3D model on OnShape and all of his code on Github.
It's basically a long cylinder with wings, that works by sucking water in to increase its density, which moves it down and with the wings also moves it forward. It then pushes water back out, which makes it go up and continue moving forward. It also has a weight that it can shift front to back and left to right, which tilts the glider and allows it to aim down and up and make turns.
Our design, adapted by our intrepid intern Jeremy Granadillo, has a couple of significant changes and a lot of small changes from the original design. The main difference is that it has a separated head module with a camera for collecting images of the sea floor. It's brain is a Raspberry Pi that connects to an Arduino that operates several stepper motors to control the motion of the glider. The 3D model of that design is also on OnShape. And that code is also on Github.
Currently, our glider is not yet operational. The hardware has been assembled, and it has been partly programmed, but there is still a lot of testing to do before we can set it free in the ocean.
In the meantime, we are starting to collect images of the seafloor using a Trident underwater drone from Sofar Ocean, formerly known as Open ROV. The Trident has a great camera integrated into it, but it points forward, so to photograph the sea floor it needs to have its nose pointing down. We've 3D printed this adapter we found on Thingiverse, and mounted it on the bottom of the drone. When we attach the GoPro to it and set it to record on timelapse mode with 0.5s intervals, it collects all the pictures we could ever want of the seafloor, as long as the water is clear enough and the drone stays upright.
Image Classification using Zooniverse
Zooniverse is an online image classification crowdsourcing tool, in their words, "a platform for people-powered research". Basically, we upload all of our pictures of the seafloor to the platform, and people everywhere can log in and look at the pictures and tell us what's in them. At the end we can download the image data and make a map of what is in each picture.
After collecting the images using the methods above, we can add them to our Zooniverse project page, and even update it with new images as we collect them.
We need to reduce the size of the images before we upload them, as it accepts images at a maximum size of 1000kb, and the gopro takes pictures around 4-5mb. To do that in Windows we can download a program called, wait for it, Image Resizer for Windows, and it lets us select all of the images we want to resize, then right click and resize them as a batch.
To upload them, in the project you need to put them into a "Subject Set", which can be different for different sets of pictures. So we can separate our image collections into different dates or locations. Once you click on a subject set or create a new one, you can just drag and drop your images at the bottom.
Image Classification using AI
Once the images are collected, they need to be processed to create a map. Using Tensorflow, we can program a machine learning algorithm to look at the images and classify them for us. Initially, this can be done to distinguish between sand, seagrass, and coral. At a later stage further distinction can be done to assess the health of the reef. Training of the AI model is also important. Once we have some sample images, we plan to upload images to Zooniverse and open it up to the public, so people in Aruba and around the world can help train the program to map our reefs.
We had an initial workshop to learn the basics of working with Tensorflow where we were able to build a machine learning model and train it using 50 images of sand and 50 images of seagrass, and it was able to correctly identify another sample of 50 unclassified images correctly. In that workshop, we did our programming using Python, and followed this tutorial by Francium Tech to get us started. All related links for that are below.
Installing TensorFlow on Windows
We had a few issues setting up Windows to follow the tutorials mentioned above, this is what we did in order to get Tensorflow running using Sublime Text.
Note that setting it up on Linux and Mac went fine according to the other tutorials, linked here at the bottom.
- install sublime
- install miniconda
- conda create -n tensorflow_env tensorflow
- (tensorflow_env can be any name)
- type y for yes and hit enter
- conda activate tensorflow_env
- pip install pandas
- pip install scikit-learn
- pip install opencv-python
- pip install matplotlib
- pip install tqdm
- pip install keras
- conda deactivate
- in Sublime
- ctrl + shift + P
- Start typing and select: Install Package Control in the command palette
- Type + select: Package Control: Install Package
- Type + select: Conda
- In the menu go to: Preferences > Package Settings > Conda > Settings - Default
- select all and copy (ctrl + A then ctrl + C)
- In the menu go to: Preferences > Package Settings > Conda > Settings - User
- paste (ctrl + V) and edit as follows:
- line 5:
- "executable": "C:\\Users\\username\\Miniconda3\\python",
- line 9:
- "environment_directory": "C:\\Users\\username\\Miniconda3\\envs\\",
- line 12:
- "configuration": "C:\\Users\\username\\Miniconda3\\.condarc",
- ctrl + S to save
- In the menu go to: Tools > Build System > Conda
- ctrl + shift + P
- Type + select: Conda: Activate Environment
- Select: tensorflow_env (or whatever you called the env you made in step 3)
- In the menu go to: File > Open Folder and select whatever python file you want to edit, and press ctrl + B to build / run. The console should pop up at the bottom with the outputs.
- Alex Williams open source glider design
- Jeremy Granadillo reef mapping glider revision
- Scoutbots robotic boat makers
- Tensorflow workshop links
- Adding Python and Conda to PATH (go to step 4) - https://medium.com/@GalarnykMichael/install-python-on-windows-anaconda-c63c7c3d1444
- Machine Learning crash course - https://developers.google.com/machine-learning/crash-course/
- TensorFlow - https://www.tensorflow.org/
- Python - https://www.python.org/
- Anaconda - https://www.anaconda.com/download/
- MiniConda (only ~50mb) - https://docs.conda.io/en/latest/miniconda.html
- PyCharm - https://www.jetbrains.com/pycharm/
- LinkedIn Learning course homepage - https://www.linkedin.com/learning/building-and-deploying-deep-learning-applications-with-tensorflow/
- Brenchie’s Lab Github - https://github.com/brenchies
- Supervised, unsupervised, and reinforcement machine learning - https://medium.com/@machadogj/ml-basics-supervised-unsupervised-and-reinforcement-learning-b18108487c5a
- scikit-learn python machine learning tools - https://scikit-learn.org/stable/index.html
- Zooniverse crowdsourced data training - https://www.zooniverse.org/
- Image recognition tutorial / article - https://blog.francium.tech/build-your-own-image-classifier-with-tensorflow-and-keras-dc147a15e38e
- Handy file renaming software - https://www.advancedrenamer.com/download
- Saving / loading models - https://machinelearningmastery.com/save-load-keras-deep-learning-models/