1. Insight to crop harvesting dates in 2020
During the HaTRY (Harvesting Time Recommendation for maximum crop Yield) project we have collected and analyzed exact sowing and harvesting times for more than 700 crop fields in Estonia.
Season 2020 sowing and harvesting events for those 3 crops were distributed over time like this (numbers and colors indicate, how many fields were sowed or harvested that specific day):
Next, we are going to match those events with signatures from Sentinel-1, Sentinel-2 and weather parameters and try to predict the best harvesting windows during season 2021 already in live prototype. It’s going to be a challenge, for sure.
Mihkel Järveoja, HaTRY project leader
2. Sentinel-2 labeled datasets
It was impressive to find out that there are at least 2 high-quality Sentinel-2 labeled datasets which can have potential use inside our free AI-based cloud mask processor for Sentinel-2.
One of the datasets, “Sentinel-2 reference cloud masks generated by an active learning method” by Louis Baetens and Olivier Hagolle, consists of 7 scenes labeled by human and 31 scenes generated by active learning methods. There are classes for the ground-truth mask: no data, not used, low clouds, high clouds, cloud shadows, land, water and snow. However, the dataset is generated at 60 m resolution. While we are building a detector for images with a ground resolution of 10 m, we still believe that we can incorporate this data into our processor.
The second dataset we were excited to find, “Sentinel-2 Cloud Mask Catalogue” by Francis, Alistair et al., was released recently in November. The dataset comprises of 513 labeled subscenes of 1022×1022 pixels at 20 m resolution. The labels represent 3 classes: clear, cloud and cloud shadow. Additionally, the subscenes have been categorized by surface type, cloud type, cloud height, thickness and extent. The data was annotated semi-automatically, using the IRIS toolkit, which makes use of a random forest model for pre-labeling.
We believe that with more variety of data and sources we can build the best cloud mask processor so far. Thus, let us know if you know any other S2 labeled dataset! 🙂
Marharyta Domnich, Machine learning engineer
3. Generating synthetical Sentinel-2-like images from Sentinel-1 images
The Sentinel-2 satellites are known for providing us beautiful optical imagery at high spatial resolution. More than just pretty pictures, the Sentinel-2 data can be utilized to calculate a variety of vegetation indices, among others the normalized difference vegetation index (NDVI). There is just one problem: when the sky is covered with clouds, we don’t have access to any of this. This can be a frustrating problem especially in the autumn season.
The images from the Sentinel-1 satellites are not so easy to interpret. Even so, they have a huge advantage on their side: as radar satellites they can see through clouds and offer an uninterrupted time series of images. And, after all, Sentinel-1 images depict the same places and objects as Sentinel-2 images.
In KappaZeta, we have started to work on modelling synthetical Sentinel-2-like images, based on Sentinel-1 input. For that we use a specific generative adversarial network called “pixel2pixel”, an image-to-image translation architecture which learns a function to map from an input image to an output image. The model is trained on co-located pairs of Sentinel 1 and cloud-free Sentinel-2 images, and eventually we plan to use the history of Sentinel-2 images as well. It means, we can produce cloud-free Sentinel-2-like images in case the actual Sentinel-2 images are disturbed by clouds. This way we can combine the strengths of both sensors.
In the image: an example of a preliminary training result. Left: Sentinel-1 image, middle: synthetical output from the model, right: the actual Sentinel-2 image.
Heido Trofimov, Software developer
