Using AI to Detect Solar Panels From Orthophotos (1/3)
This is the first blog of a three-part miniseries on developing an AI to detect solar panels from orthophotos. The articles will be broken down into three project steps:
- Data collection and pre-processing
- Training a neural network model using
- Deploying your app to the world wide web using
But before we jump into the project, we need to understand some of the basics.
- Solar panels
- What are orthophotos
- Orthophotos for training
- Image segmentation
- Import libraries
- Creating segmentation masks
- Next steps
The solar panel industry is booming. In 2020, the solar industry generated roughly $25 billion in private funding in the U.S. alone. With more than 10,000 solar companies across the U.S., a nearly 70% decline in installation costs, and competitive utility prices it’s no wonder 43% of new electric capacity additions to the American electrical grid have come from solar. Although it only makes up a meager 4% of all U.S. electricity production, the solar industry is seeing clear skies.
So how do economists and policymakers track this growth? Where should solar business owners target for new markets? Is it even possible to monitor this adoption at such a massive scale? The answer to these questions and more lies in AI.
Because in an age when fossil fuels are beginning to darken our days, renewable sources of energy are starting to shine. From geothermal and wind turbines to hydropower and solar, our options are steadily improving. And with declining costs in technologies, adoption has been skyrocketing. But there are still improvements that can be made. Using AI and machine learning we can speed up adoption, improve sales, and track large-scale implementation.
So how do we hope to achieve this? By using AI to detect solar panels from satellite and aerial images!
AI is being used to increase response times to natural disasters and improve humitarian aid. Test Appsilon’s AI model for assessing building damage.
What are orthophotos
Open your google maps, zoom to a city, and turn on the satellite view. Chances are, you’re looking at one or two orthophotos. These are typically satellite images but can be aerial photographs from something like a plane or UAV. However, when you take an unprocessed image its features are likely distorted unless viewing from directly above the target area (aka “nadir”). Tall buildings, mountains, basically anything with elevation, will look like they’re leaning and straight lines like pipelines will “bend” with the topography.
In order to correct the parallax displacement from camera tilt and relief, we need to orthorectify the image, i.e. create a mathematical interpolation of the image using ground control points. This allows us to minimize the scale warping effect as we move away from nadir. In doing so, we’ve created a photo with a uniform scale that can overlap other maps with minimal spatial errors.
Are you using AI for a social good project? Appsilon’s data science team can help through our Data4Good initiative.
But why is this important to identifying Solar Panels? Well, because regular satellite images don’t have a uniform scale, any attempts to quantify precise distances and areas would be inaccurate. For example, if you’re training a model to identify 1 x 1-meter solar panels on a hillside and calculate the total area, the farther away from nadir it goes, the more likely the panels are to distort in size. Your image segmentation and classification will probably be incorrect and your area measurements inaccurate.
Orthophotos for training AI to detect solar panels
You can access the data we’ll be using to train our model from Distributed Solar Photovoltaic Array Location and Extent Data Set for Remote Sensing Object Identification. The dataset consists of 526 images of 5000 x 5000 px and 75 images at 6000 x 4000 covering four areas of California. The dataset also contains CSV files describing the locations of roughly 20,000 solar panels.
Image segmentation as a machine learning task
Once you have your images, you can think about the panel identification task as an image segmentation problem. To solve the problem, you’ll need to predict a class for each pixel of an image. In our case, we’ll keep it simple with two classes, “solar panel” and “not a solar panel.” So, we’ll try and predict whether a specific pixel belongs either to a solar panel or not.
To do this, we will first prepare training data, i.e. create an image segmentation mask. A segmentation mask is an image with the same bounds as the original image, but with information on the contents at the pixel level with a unique color for each class. This helps parse the image into important areas to focus on rather than processing the image as a whole.
There are two types of segmentation: semantic and instance. Semantic segmentation groups pixels of a similar class and assigns the same value to all in that group (e.g., people vs background seen in the image below). In our case, we’ll be creating semantic segmentation. If you’re looking to perform additional tasks like assigning subclasses or counting classes, you’ll want to create an instance segmentation mask.
To build the model yourself, you’ll need to import a few libraries and load the necessary data. I’ll show you how, below:
As we will discover later,
polygon_idis a unique column that serves as an ID. We should immediately set it as a pandas index. We can also explore a few columns from the
The first file details the folder structure and geospatial data related to the polygons of individual solar panel islands.
Each row in the second file describes a single polygon with coordinates of its vertices.
Creating segmentation masks for the solar panel detection algorithm
To create segmentation masks we will process every image separately. Then for each image, we will locate and draw all polygons. We will use
ImageDraw the PIL package to easily draw polygons on the images.
Image.new("L", size) allows us to create a new image with one channel – black/white as this is what we need for the segmentation mask. We fill polygons with value 1 as the
fastai library expects each class to have a sequential class number.
Since there are over 40GB of image files to process, it will take some time. Because every image can be processed simultaneously, we can easily parallelize calculations. Parallel map functionality with a progress bar is provided by the
tqdm package, that’s what
process_map function does. The
process_map function requires a single argument so we have to wrap
process_image as the function
wrap_process_image does. Line 26 in
process_all_images converts a list of lists into a unified list. Since
image_data is list of lists of tuples of the same length, it is easily convertible to pandas dataframe.
The last missing piece is the
split_save_image function. Original images of size 5000x5000px or 6000x4000px are far to big to process them in neural network. We have to split them into smaller pieces. That what
split_save_image is for.
Here we decided to split images into 500 x 500 px patches. Because every image is split into 100 or 96 patches, many won’t contain solar panel fragments. But we won’t toss these out. At least not yet, because the images are that are empty might prove to be useful later. If we pass the argument
meta=True then after saving the image, the percentage of pixels occupied by solar panels will be calculated and added to the list. The example patch has been presented above.
Through Data4Good, Appsilon helps build support systems for disaster risk management in Madagascar.
The script will run for a while and process every image in the
raw_images directory. It might take about 20 minutes. The progress bar will help to track the remaining time. The last thing to do is to save the data frame with a pixel fill ratio.
Now we end up with 59800, 500 x 500 px patches, and corresponding segmentation masks. Finally, we can visualize a sample patch and corresponding mask:
Next Steps in building an AI to detect solar panels
To recap, in the first of this series we’ve downloaded orthophotos of solar panels in California and processed them by creating corresponding segmentation masks. Later we split images into smaller pieces to make them consumable for a neural network.
In part two we will use the
fastai library to train a PoC model for solar panel detection.
Use Shiny to build elegant, engaging apps to better serve at-risk communities. Explore the possibilities of Shiny with Appsilon’s VisuaRISK application.