Five new satellite analytics tools for agriculture

January 31

Written by Tanel Kobrusepp, product manager

Introduction to KappaZeta's new AI-based Agricultural Solutions

KappaZeta's latest contribution to agricultural technology involves a set of five innovative products based on AI and machine-learning, each uniquely leveraging Sentinel-1 and Sentinel-2 satellite data. These services are designed to address key aspects of agricultural management and analysis, catering to the needs of scientists, government officials, and professionals in farming, farm management and insurance. The suite includes tools for Crop Type Detection, enabling precise identification of various crop types; Parcel Delineation for accurate land mapping; Seedling Emergence Detection to monitor early crop growth; Farmland Damaged Area Delineation for assessing areas affected by adverse events and Ploughing and Harvesting Events Detection to track critical farming activities. These tools collectively aim to enhance agricultural practices through data-driven insights, fostering more informed decision-making in the field of agriculture.

Crop Type Detection

The Crop Type Detection service developed by KappaZeta effectively leverages Sentinel satellite data to accurately identify various crop types across extensive agricultural regions. For crop insurance companies, this tool aids in risk assessment by providing detailed information about the crops insured, allowing for more accurate risk profiling and premium calculation. Government agencies can utilize this data for agricultural policy planning and monitoring, ensuring resources are appropriately allocated. Additionally, the tool assists in environmental monitoring, as understanding crop distribution is key in assessing ecological impacts and land use planning. For the agricultural market at large, this information helps in forecasting supply and demand trends, crucial for market stability and pricing strategies. Thus, the Crop Type Detection service offers practical benefits across multiple facets of the agricultural industry.

Parcel Delineation

The Parcel Delineation, a part of KappaZeta's array of tools, focuses on the critical task of accurately mapping agricultural land parcels. Utilizing images from Sentinel satellites, the tool provides detailed and precise outlines of farm plots. This product is particularly valuable for Earth observation data analysis companies, as it enhances their ability to develop downstream applications, especially in scenarios where existing parcel boundaries are not readily available. By providing detailed and accurate delineations, the product enables these companies to generate more refined insights into land use, crop health, and environmental monitoring. Government entities benefit from this in land use planning and policy implementation, ensuring fair and efficient allocation of resources and compliance with agricultural regulations. Beyond its operational value, accurate parcel delineation is also a step towards more responsible and sustainable agricultural practices, as it helps in better understanding and managing land resources.

Seedling Emergence Detection

The Seedling Emergence Detection service addresses a critical early stage in the agricultural cycle. This service effectively identifies the emergence of seedlings across various crop types. This early detection capability is invaluable for farmers and agronomists, enabling them to swiftly assess germination success and take timely actions if needed, such as re-sowing or adjusting crop management practices.

By knowing the exact emergence dates, insurance providers can better evaluate the vulnerability of winter crops to early-season adversities such as frost, or pest attacks, leading to more accurate premium calculations, portfolio management and efficient claim management.

For Earth observation companies, this service adds a crucial layer of data, enhancing their ability to provide comprehensive agricultural analyses and insights. In contexts where early growth stages are critical for yield prediction and risk assessment, this tool provides a significant advantage. Furthermore, the Seedling Emergence Detection assists in fine-tuning irrigation and fertilization plans, contributing to more efficient and sustainable farming methods. Its utility extends to research and policy planning, offering data that can inform studies on crop development and agricultural strategies.

Farmland Damage Assessment

The Farmland Damaged Area Delineation specializes in mapping areas within agricultural lands that have sustained damage by adverse events such as extreme weather, pests, or disease outbreaks. For farmers, this tool is invaluable for quickly pinpointing affected areas, enabling them to implement targeted responses such as reallocating resources, adjusting irrigation, or applying specific treatments to the damaged zones. In the realm of crop insurance, this service provides crucial data for the prompt and accurate processing of claims, offering objective, verifiable evidence of the extent and location of damage, without even going to the field. This service is additionally vital in disaster response and management, assisting government agencies in efficiently directing resources and aid to the most affected regions. Furthermore, the data gathered by this product can contribute to long-term agricultural planning and environmental monitoring, helping to understand patterns of damage and inform future mitigation strategies.

Detecting Ploughing and Harvesting Events

The Ploughing and Harvesting Events’ date Detection service is a critical tool for monitoring key agricultural activities. For crop insurance companies, this information is essential in assessing the timing and methods of farming practices, which are integral factors in risk assessment and claim verification. This tool also plays a significant role in compliance with agricultural policies. Specifically, for government agricultural paying agencies, the detection of ploughing events is mandatory under the CAP2020 policy. The product’s ability to provide accurate and timely data ensures that these agencies can effectively monitor and enforce compliance with agricultural policies. Additionally, cultivation data offers valuable insights into farming patterns and their environmental impacts, assisting in the development of more sustainable farming practices which is a key component of carbon farming project monitoring.

Empowering the Future of Agriculture with AI and Satellite Data

KappaZeta's innovative suite of AI and machine learning-based tools, utilizing Sentinel-1 and Sentinel-2 satellite data, represents a significant advancement in agricultural technology. These tools address key areas of agricultural management and analysis, catering to a diverse range of users including scientists, government officials, and professionals in farming, farm management, and insurance.

Overall, these tools collectively enhance data-driven decision-making in agriculture, leading to more efficient and sustainable practices. They demonstrate the pivotal role of advanced satellite analytics in transforming modern agriculture.

The prototypes for all five services were developed during the project “Satellite monitoring-based services for the insurance sector – CropCop”, supported by the European Regional Development Fund and Enterprise Estonia.

SNDVI: Synthesized NDVI (from SAR)

February 08, 2023

The Normalized Difference Vegetation Index (NDVI) is a widely used index for monitoring the health and productivity of vegetation. It is derived from the red and near-infrared (NIR) bands of passive optical sensors, such as satellites like Landsat or Sentinel-2. However, cloud cover and other atmospheric interferences can often obscure optical satellite imagery, making it difficult to accurately measure NDVI. This hinders vegetation monitoring and downstream applications, such as detecting crop damage, yield map synthesis, and others that rely on clear imagery or a sufficiently dense series of NDVI images, given constrained satellite revisit times. Cloud cover is especially problematic in the autumn in Northern Europe, where cover persists for long periods of time.

Figure 1. Cloudy NDVI image and SNDVI alternative (See more examples on our demo map)

An alternative to using passive optical sensors for vegetation monitoring is radar satellite data, which can penetrate clouds and vegetation, providing a more consistent view of the Earth's surface. Although differing in their sensing modalities, the backscatter features of Synthetic Aperture Radar (SAR) data are useful for detecting crop cover over agricultural areas whilst the coherence feature has been observed to be inversely correlated to NDVI, making them complementary data sources for interpolating missing NDVI data [1] [2].

Figure 2. Comparing co-registered NDVI & SAR images, 17 days apart. False-color SAR image with VV-backscatter (Red), VH-backscatter (Green), VV-6 day-Coherence (Red). Note: Image values are scaled (-1, 1) represented in order of increasing pixel intensity.

Modelling Approach

To explore this relation between SAR and NDVI imagery, we rely on a multi-temporal deep generative model (based on pix2pix [3]), to train 512 × 512 px sub-tiles of aligned SAR and optical imagery (as shown in Figure 3), covering mostly agricultural parcels. Each optical input is collocated with a SAR image +- 2 days from each other. For the baseline model (data architecture in Figure 3), model inputs are a recent SAR image +- 2 days from the NDVI image to be synthesized, and historical S1-S2 image sub-tiles (max 30 days from target). A recent variation to this approach, adds recent optical inputs (RGB, NIR, NDVI) to the SAR image, applying our in-house cloud mask (KappaMask) to predict NDVI for occluded areas only. This yields better performance for larger heterogenous areas.

Figure 3. MTCGAN Model Architecture describing input sources

Evaluating Synthetic NDVI Imagery

Full-reference image metrics such as SSIM, PSNR, and MAE are utilized to evaluate the accuracy of synthesized images, at the sub-tile level, with results in Table 1.

Sub-tiles	MAE	SSIM	MS-SSIM	PSNR
2547	0.05	0.88	0.93	24.88

Table `1. Sub-tile image quality assessment for unseen test area (NW Estonia)

However, owing to the parcel heterogeneity in a single sub-tile, more efforts were placed on evaluating predictions at the parcel-level. These measures include evaluating absolute NDVI changes (MAE), the correlation between real and predicted NDVI changes (pixel histogram correlation) and exploring subfield variance classes. For vegetation monitoring, NDVI is useful for monitoring crop phenology, and we began exploring this use-case with a broad categorization of observed changes w.r.t NDVI increase (crop growth) or decrease (crop decline/maturity).

Figure 4. Comparing Historical Changes (hmae = abs(hndvi - ndvi)) and prediction errors (mae = abs(sndvi-ndvi))

Figure 4 describes the overall change in NDVI (between historical and target NDVI), compared with the MAE prediction error between SNDVI and target NDVI for observed parcels. In summary, NDVI synthesis is more accurate for crop growth events, and less accurate for crop decline and the sudden changes (including, plowing, harvesting) that influences these events.

Figure 5. Comparing heterogenous parcel changes between HNDVI, SNDVI, NDVI subtiles. (Historical inputs lag: 17 days, NDVI date: June 8, 2018)

Figure 6. Comparing within-parcel changes (from sub-tile in Figure 5) between HNDVI, SNDVI, NDVI. (Crop: Spring barley, Historical inputs lag: 17 days, NDVI date: June 8, 2018)

Conclusion, limitations and future work

In this article, we introduced a generative approach to synthesize NDVI images from multi-temporal SAR and historical optical data. We highlighted some evaluation metrics and demonstrated that our MTCGAN model is more effective at predicting NDVI increases compared to negative changes. Another point worth highlighting is that the performance (MAE) does not degrade significantly by using older historical S2 inputs (within the 30-day limit).

However, the results shared here are for 6-day coherence SAR inputs. Performance worsens for the current Sentinel-1 alternative (12-day coherence, due to Sentinel 1B malfunction), moreso for predicting crop decline. For crop decline, a factor is the contrast between interferometric coherence in historical and target SAR images. After a crop removal event, at least 2 SAR images are required for the coherence to reflect the change, compared to crop growth with less drastic changes in coherence. Accounting for this SAR limitation will be key in improving results for this case.

To further understand the usefulness of synthesized NDVI images, we plan to examine their ability to indicate specific crop phenological stages. This will help us understand the limitations and potential applications of these images for individual crop monitoring. We intend to expand our prediction of NDVI increase/decrease to include more specific stages such as crop emergence, flowering, crop maturity, and senescence, and we will test this on different types of crops.

Lastly, another useful application for monitoring agencies and/or farmers may be zoning or detecting fields and subfields which demonstrate homogenous or heterogenous growth. Initial analyses of SNDVI images show F1-score accuracies of 71% for classifying low NDVI variance parcels and 42% for high variance classes. Continuing work will evaluate crop-specific cases before concluding analyses.

For some examples of AI-generated images, visit our SNDVI demo map here. In addition to SNDVI created for 6-days coherence, we will be adding other images created from 12-days coherence, and in the future, other AI-derived vegetation indices for crop monitoring.

Acronyms
Synthetic Aperture Radar (SAR), Normalized Difference Vegetation Index (NDVI), Generative Adversarial Network (GAN), Conditional GAN (CGAN), Multi-Temporal CGAN (MTCGAN), Synthetic NDVI (SNDVI), Historical NDVI (HNDVI), Mean Absolute Error (MAE), Historical MAE (HMAE), Structure Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR)

References

[1] Voormansik, K. et al. (2020) “Separability of mowing and ploughing events on short temporal baseline sentinel-1 coherence time series,” Remote Sensing, 12(22), p. 3784. Available at: https://doi.org/10.3390/rs12223784.

[2] Harfenmeister, K., Spengler, D. and Weltzien, C. (2019) “Analyzing temporal and spatial characteristics of crop parameters using sentinel-1 backscatter data,” Remote Sensing, 11(13), p. 1569. Available at: https://doi.org/10.3390/rs11131569.

[3] Isola, P. et al. (2017) “Image-to-image translation with conditional adversarial networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [Preprint]. Available at: https://doi.org/10.1109/cvpr.2017.632.

Towards the operational service of crop classification

October 12, 2020

We are about to finish a R&D project, where we developed and tested crop classification methodology specifically suited for Estonian agricultural, ecological and climatic conditions. We relied mostly on Sentinel-1 and -2 data and used neural network machine learning approach to distinguish 28 different crop types. Results are promising and the methodology is ready for operational service to automate another part of agricultural monitoring.

Using machine learning in crop type classification is not new, and definitely not a revolutionary breakthrough - already for decades different classifiers (Support Vector Machine, Decision Trees, Random Forest and many more) have been used in land cover classification. Recently also neural networks, the wunderkind of machine learning and image recognition, are widely used in crop discrimination. Satellite data, as the main input to classification models, has no serious alternatives, since our aim is to implement it on worldwide scale and in applications, which run near real time. So, why even get excited about another crop type classification study, which exploits same methods and datasets as tens of previous studies?

I can give you one reason. Estonia has been very successful in following European Commission (EC) guidelines and rules in modernizing the EU Common Agricultural Policy. In 2018 EC adopted new rules that allow to completely replace physical checks on farms with a system of automated checks based on analysis of Earth observation data. The same year Estonian Agricultural Registers And Information Board (ARIB) launched the first nation-wide fully automated mowing detection system, which uses Sentinel-1 and Sentinel-2 data and where the prediction model inside the system is developed by KappaZeta. The system has been running for 3 years, it has significantly reduced the amount of on-site checks and increased the detection of non-compliances. In short – saved Estonian and EU taxpayers’ money. Automated crop discrimination is the next step in pursuing the above-mentioned vision and will probably become the foundation of all agricultural monitoring. With proved and tested methodology, it’s highly likely that Estonia will take this next step in the very near future and launch it again on the nationwide level. This is definitely a perspective to be excited about.

Now, let’s see how we tackled “the good old” crop classification task.

Input data

Although algorithms and methods are important to make a difference in prediction model performance, the training data is the most valuable player in this game. In Estonia all farmers who want to be eligible for subsidies need to declare crops online (field geometry + crop type label). This open dataset is freely accessible to everyone and has the permission to re-use and redistribute both commercial and non-commercial purposes. Since the crop type labels are defined by farmers and most of them are not double-checked by ARIB, there can be mistakes (according to ARIB’s estimations, less than 5%). Therefore, for additional validation we ran our own cluster analysis on time-series to filter out obvious outliers in each class.

After we had the parcels and labels, we calculated time-series of different satellite-based, plus some ground-based features (precipitation, average temperatures, soil type). When extracting features from satellite images there are two ways to go: pixel- or parcel-based extraction. We selected the latter and averaged pixel values over the parcel to obtain one numerical feature value per statistic for each point in time (see Figure 1).

Figure 1. An example time-series of one Sentinel-1 feature (cohvh - 6-day coherence in the VH polarization) for one parcel.

For Sentinel-1 images preprocessing we have developed our own processing chain to produce reliable time-series for several features. From the previous studies it’s known that features (channel values and indexes) from Sentinel-2 images combined with features from Sentinel-1 images (coherence, backscatter) give better classification results than any of these features separately.

Figure 2. The whole dataset can be imagined as a three-dimensional tensor with the feature parameters on one axis, parameter statistics on another, and date-time on the third axis.

We used data from 2018 and 2019 seasons (altogether more than 200 000 parcels) and aggregated all crop type labels into 28 classes which were defined by the need of the ARIB.

Model architecture

Due to the very unbalanced dataset we had to under-sample some classes and over-sample others for the training data. In small classes we used the existing time-series and added noise for data augmentation.

Model architecture was rather simple – input layer, flatten layer, three fully connected dense layers (two of them followed by batch normalization layer) and output (Figure 3). Our experiment with adding 1D CNN layer after input didn’t improve results significantly. More complicated ResNet (residual neural network) architecture increased training time by approx. 30%, but results were similar to a linear neural network.

Classification results

F1 score on validation set (9% of all dataset) was 0.85 and on test set (2% of all dataset) 0.84. In 10 classes the recall was more than 0.9 and in 16 classes more than 0.8. See more from Figure 4 and 5.

Figure 5. Normalized confusion matrix of the crop classification results (recall values).

Some features are more important than others

In a near-real time operative system our model and feature extraction would have to be as efficient as possible. For an R&D project we could easily calculate 20+ features from satellite images, feed them all to the model and let the machines compute. But what if not all features are equally important?

They are not. We found that the 5 most important features are Sentinel-1 backscatter (s1_s0vh, s1_s0vv), NDVI, TC Vegetation and PSRI from Sentinel-2. To our surprise, soil type and precipitation sum before satellite image acquisition had low relevance.

The 5 most important features played different role during the season – Sentinel-2 features were more important in the beginning and in the end of the season, while Sentinel-1 features had more effect during mid-season.

Figure 6. Importance of different features in crop classification, estimated using Random Forest.

What next?

This project was part of a much larger initiative called “National Program for Addressing Socio-Economic Challenges through R&D. Using remote sensing data in favor of the public sector services.” Several research groups all over Estonia worked on prototypes to use remote sensing in fields like detecting forest fire hazard, mapping floods and monitoring urban construction. Now its up to Estonia’s public sector institutions to take the initiative and turn prototypes into operational services. With this work we have proved, that satellite-based crop classification in Estonia is possible, accurate enough and ready to be implemented as the next monitoring service for ARIB.

If you are more interested about this study, our Sentinel-1 processing pipeline or machine learning expertise, then feel free to get in touch. We have the mentality to share not hide our experience and learn together on this exciting journey.

Open access to ALOS-2 radar satellite data?

April 28, 2020

End of previous year (2019) Japan announced that the Japan Aerospace Exploration Agency (JAXA) will be providing open access to information and data from a suite of their radar satellites (original statement here). To be more specific, free and open access to the wide-swath observation data from the L-band radar satellites, ALOS (ALOS/AVINIR-2, PALSAR) and ALOS-2 (ALOS-2/ScanSAR) will be made available. The price of ScanSAR images is at the moment around 700 euros.

ALOS-2 spacecraft in orbit (image credit: JAXA)

The Japanese space and satellite program consist of two series of satellites – those used mainly for Earth observation and others for communication and positioning. There are 3 Earth Observation satellites in nominal phase, 3 in latter phase in operation and 3 more under development.

Greenhouse gases Observing SATellite-2 "IBUKI-2" (GOSAT-2) is measuring global CO₂ and CH₄distribution of lower and upper atmosphere. Climate "SHIKISAI" (GCOM-C) satellite carries an optical sensor capable of multi-channel observation at wavelengths from near-UV to thermal infrared wavelengths (380nm to 12µm) to execute global, long-term observation of the Earth’s environment. Advanced Land Observing Satellite-2 "DAICHI-2" (ALOS-2) aims are to monitor disaster areas, cultivated areas and contribute to cartography.

ALOS-2, which is specifically interesting for radar enthusiasts, is a follow-on mission from the ALOS “DAICHI”. Launched in 2006, ALOS was one of the largest Earth observation satellites ever developed and had 3 different sensors aboard: PRISM (Panchromatic Remote-sensing Instrument for Stereo Mapping) for digital elevation mapping, AVNIR-2 (Advanced Visible and Near Infrared Radiometer type 2) for precise land coverage observation and PALSAR (Phased Array type L-band Synthetic Aperture Radar) for day-and-night and all-weather land observation. ALOS operations were completed in 2011, after it had been operated for over 5 years.

ALOS-2 was launched in 2014 and carries only radar instrument aboard. New optical satellite, ALOS-3, which will improve ground resolution by approx. three times from that of ALOS (2.5 to 0.8 m at nadir, wide-swath of 70 km at nadir), is already under development together with ALOS-4, which will take over from ALOS-2 to improve the functionality and performance.

Let’s come back to present day. The state-of-the-art L-band Synthetic Aperture Radar (PALSAR-2) aboard ALOS-2 have enhanced performance compared to its predecessor. It has a right-and-left looking function and can acquire data in three different observation modes:

Spotlight – spatial resolution 1x3 m, NESZ -24, swath 25 km.
Stripmap – spatial resolution 3-10 m, swath 30–70 km. Consist of Ultrafine (3 m), High sensitive (6 m) and Fine (10 m) modes.
ScanSAR – spatial resolution 60-100 m, swath 350–490 km.

PALSAR-2 specifications (images credit: JAXA)

Emergency observations have highest priority for ALOS-2, but for systematic observations Basic Observation Scenario (BOS) has been developed. This ensures spatial and temporal consistency at global scales and adequate revisit frequency. ALOS-2 BOS has separate plans for Japan and for the rest of the world, success rate for these acquisitions is 70–80 %.

PALSAR-2 observation modes (images credit: JAXA)

Basic observations over Japan are mostly undertaken in Stripmap Ultrafine mode and sea ice observations during winter in ScanSAR mode.

Stripmap Fine and ScanSAR modes are used for global BOS. There are several areas of interest, where ALOS-2 is putting more focus, for example:

Wetlands and rapid deforestation regions in ScanSAR mode
Crustal deformation regions both in Stripmap Fine and ScanSAR mode
Polar regions both in Stripmap Fine and ScanSAR mode

In addition to those special regions global land areas are observed in Stripmap Fine mode at least once per year.

We made a little experiment to test, how many acquisitions we get over city of Tartu per year. Here are the results (platform for viewing and ordering data is here):

Screenshot from Earth Observation Data Utilization Promotion Platform.

Year	Number of images per year
2015	6
2016	7
2017	9
2018	8
2019	7

So, compared to Sentinel-1 radar-satellite, ALOS-2 acquisitions frequency is much lower over Europe, and its difficult to develop agriculture monitoring services only on this platform. For forestry and other environmental monitoring, where changes are not happing that often as in agriculture, ALOS-2 can be very useful due to its better spatial resolution than Sentinel-1. Being an L-band satellite it can also penetrate deeper into vegetation and provide information about the lower layers of the canopy. JAXA is already developing ALOS-4 with PALSAR-3 aboard, which will aim broader observation swath compared to the predecessor.

Overview of new RADARSAT Constellation Mission

April 20, 2020

Exciting remote sensing news from last year. Canadian Space Agency has launched new generation of Earth observation satellites called The RADARSAT Constellation Mission (RCM) on June 12, 2019 aboard a SpaceX Falcon 9 rocket. It became operational in December 2019 and provides data continuity to RADARSAT-1 (not operational anymore) and RADARSAT-2 (still operational) users.

Illustration of the three RCM satellites on the same orbital plane. Image credit: Canadian Space Agency

RCM is a combination of three identical and equally spaced satellites, flying in the same orbit plane 32 minutes apart at an altitude of 600 km. Each of the spacecraft carries Synthetic Aperture Radar (SAR) aboard, plus a secondary sensor for Automatic Identification System (AIS) for ships. When RADARSAT-2 has left- and right-looking operation, then RCM is only right-looking, because multiple satellites increase revisit times and eliminate the need to look both ways. The SAR device aboard RCM satellites is quite similar to RADARSAT-2 – C-Band antenna, 100 MHz bandwidth, 4 regular polarization modes (HH, VV, HV, VH) plus compact polarimetry. Polarization isolation is slightly better: >30 dB. See detailed comparison of RADARSAT satellites here.

The constellation system provides better coverage with smaller and less expensive satellites. This configuration allows for daily revisits of Canada’s territory, as well as daily access to 90% of the world’s surface. RCM can provide a four-day exact revisit (3 satellites equally phased in a 12 day repeat cycle orbit), allowing coherent change detection with InSAR. For specific applications (ship detection, maritime surveillance) data latency from acquisition to delivery can be only 10-30 minutes, but in general it will be from hours to 1 day.

RCM has several observation modes, but the mission is primarily designed for medium-resolution monitoring:

Low resolution (100 m), swath 500 km, NESZ -22 dB
Medium resolution (16, 30, 50 m), swath 30-350 km, NESZ -25…-22 dB
High and very high resolution (3-5 m), swath 20-30 km, NESZ -17..-19 dB
Spotlight (1x3 m), swath 20 km, NESZ -17 dB