Using fastai to Detect Solar Panels From Orthophotos (2/3)
Welcome to the second post in this three-part ML project on using
fastai to detect solar panels. Throughout the miniseries, I’ll be covering the following ML project steps:
- Data collection and preprocessing
- Training neural network model using
- Deploying your app using
In this blog, we’ll begin building a model to automatically detect solar panels in orthophotos. To do this, we’ll focus on creating an image segmentation model using the
fastai library. Although we’ll be using satellite imagery, more specifically – orthophotos, the methods presented here apply to any tasks working with images in
- What is fastai?
- Image segmentation as a machine learning task
- Preparing fastai dataloaders for modeling
- Segmentation model training using fastai
- fastai model serialization
- Prediction results using trained fastai model
- Next Step
What is fastai?
fastai is a deep learning library created by Jeremy Howard et al. that gives practitioners both high- and low-level components to quickly create advanced PoC models. It’s an innovative tool that gives us ease of use without sacrificing flexibility or performance. Since we are working on a vision task, we’ll use
fastai.vision, a part of the greater
fastai ecosystem. But it is important to note that
fastai provides convenient interfaces for:
- image classification, single and multi-label
- image segmentation
- estimating points in the image
- Text data
- creating of language models
- Tabular data
- convenient creation of embeddings
- Collaborative filtering
- recommender systems
Image segmentation as a machine learning task
A quick reminder from where we finished in the previous post. After preprocessing data, we have two folders
masks. The folders contain our preprocessed patches of images (500 x 500px) as well as a
fill_ratio.csv file containing the percentage of pixels occupied by solar panels in every image.
So let’s load the necessary libraries and start modeling! Note that
from fastai.vision.all import * already imports
Path and more.
As you might recall, a lot of images (nearly 85%) don’t contain any solar panels. This is a lot of “useless” data to process. So if we can reduce the target data down to only those of value we can save a lot of time. But how do we do this?
Since we’re using
fastai in this project, we’ll use the
L class as well. This is
fastai‘s answer to a container that’s most similar to Python list –
numpy one-dimensional ndarray.
L class supports a lot of useful functionalities like map and filter. You can read about the
L class in the documentation.
Looking for effective real-time object recognition? Explore the possibilities of image classification with the YOLO object detection algorithm.
Path on the other hand, comes directly from the Python standard library. This feature introduced in Python 3.4 allows easier working with paths on various systems. Check out this great tutorial on pathlib paths. What’s important for us is that you can create paths by using slash, e.g., Path(‘.’) / ‘images’.
And the paths will be valid on every system Python is running on! Finally, paths in the csv file describe the mask files while we need paths to image files. Function map applies our lambda function to every element of
nzero_fill and the second
Path objects from strings.
Preparing fastai dataloaders for modeling
Since we have nicely organized data, we would like to use the
SegmentationDataLoaders.from_label_func function to create dataloaders. To do so, we need a function that will say which image file corresponds to which mask. All that’s left to do is to create dataloaders and preview an example batch.
Segmentation model training using fastai
In general, image segmentation might be a difficult machine learning task to train a model from scratch. Fortunately, we can leverage pre-trained image classification models. With
fastai all we have to do is create a
Learner with the
Now we can run the
fine_tune method and leave all training-related issues to
fastai! Using Nvidia T4 single GPU, each epoch took around 20 minutes. In all, the entire training should last about an hour.
Since we are using large images, we have to create small training batches to fit them into our memory. To compensate for the small batches, we use gradient accumulation. This is very easy in
fastai with its callback system.
To quickly preview the model’s performance let’s run
show_results. The parameter
figsize allows us to make images larger, which is particularly useful if we want to view the results.
fastai model serialization
Usually, after we spend our precious resources training the model, we want to save it. It’s not unheard of that a Jupyter kernel dies unexpectedly and we have to train from scratch. If you’ve been lucky enough to escape this fate, congratulations. But trust me when I say, it’s no fun. We also plan to use this model in part 3 of the miniseries, so we really ought to save it.
Prediction results using trained fastai model
It’s always nice to see the decreasing loss values and some predictions on the validation set. For me, what’s truly impressive about machine learning is its ability for generalization. Let’s take some orthophoto images of Warsaw suburbs and see whether our model will be able to find solar panels in it!
Explore our guide to GPU-accelerated ship recognition using Keras and R.
Here we’re simulating usage of the model after its deserialization. Our model returns the predicted mask as a PyTorch tensor. To visualize found solar panels, let’s create an alpha mask that will fade out parts of the image that are not panels.
In just a few lines of code, we’ve trained a PoC model for solar panel detection using
fastai and performed predictions on our files. We’ve also serialized the model for future use.
Need assistance with object detection, image classification, or satellite image analysis? Contact our Computer Vision team. We build models that identify objects, defects, and patterns that human auditors miss. We can provide fast development, going from concept to a working prototype in as little as one week.
Closing notes on fastai
When working with libraries like
fastai sometimes you have to do odd things like juggling filenames back and forth to prepare appropriate input for provided functions. But this is only a minor inconvenience when compared to accessing the impressive features of
Another thing to note, while we trained the model on images 500 x 500px, we’ve made predictions on images of different sizes and it just worked. But this wasn’t magic.
fastai took care of everything in the background. There are many beneficial reasons to utilize fastai in your project – the best of which you won’t actively notice. We recommend you begin with fastai’s quick start and explore their tutorials to get a better grasp of what’s possible.
fastai version 2.4.1.