Prediction of Soil Moisture Content Based On Satellite Data and Sequence-to-Sequence Networks

Pr ediction of Soil Moistur e Content Based On Satellite Data and Sequence-to-Sequence Networks Natalia Efremo va ∗ Univ ersity of Oxford natalia.efremova2@sbs.ox.ac.uk Dmitry Zausaev Deep Planet dmitry@deepplanet.ai Gleb Antipov gantipov@gmail.com Abstract The main objectiv e of this study is to combine remote sensing and machine learning to detect soil moisture content. Growing population and food consumption has led to the need to impro ve agricultural yield and to reduce wastage of natural resources. In this paper , we propose a neural network architecture, based on recent work by the research community , that can make a strong social impact and aid United Nations’ Sustainable De velopment Goal of Zero Hunger . The main aims here are to: impro ve ef ficienc y of water usage; reduce dependence on irrigation; increase ov erall crop yield; minimise risk of crop loss due to drought and extreme weather conditions. W e achie ve this by applying satellite imagery , crop segmentation, soil classification and ND VI and soil moisture prediction on satellite data, ground truth and climate data records. By applying machine learning to sensor data and ground data, farm management systems can ev olve into a real time AI enabled platform that can provide actionable recommendations and decision support tools to the farmers. 1 Introduction The world’ s population is expected to rise from se ven billion to ten billion in years to come. Fresh water scarcity and increased food consumption ha v e lead to the need to increase agricultural yields up to 70 % in the upcoming years [1]. Machine learning (ML) has emer ged with big data technologies and high performance computing to create ne w opportunities in the agri-tech domain. ML tools can help to optimise the farming practice and enable sustainable use of en vironmental resources, e.g. without de grading the land but at the same time obtaining the most out of it. Consequentially , this way of f arming can also lead to restoration of en vironmental resources. One way to help farmers is to pro vide specialised recommendations about their land, optimum type of crop, observed soil moisture, salinity , pH, and nitrogen content for fertilisation. If this information is aggregated and analysed automatically at a region-le vel, it can help to proactiv ely take actions that can increase agricultural yields, and, at the same time, preserve scarce natural resources, such as fresh water , and minimise the usage of fertilisers and pesticides. The advantage of using ML models versus manual labour is significant, since automated methods work in real time and cost significantly cheaper i.e. provide analysis of large territories much f aster than con ventional geographic information systems (GIS). Applying tools, which are flexible and adaptable, on large scale in precision agriculture gi ves an opportunity to hav e an integrated system on ∗ Centre for Corporate Reputation and Future of Marketing Initiati ve Preprint. W ork in progress. div erse le vels of aggre gation for the en vironmental resources. In this w ork, we describe an ongoing project, that combines state-of-the-art neural network approaches to build an automated system for providing practical recommendations for farmers. In the rest of the paper , we describe the model that we dev eloped for soil moisture prediction on the farms and the requires data structure (section 2). W e discuss the potential of the proposed system and outline the directions of future work in section 3. 2 Methods Soil moisture detection is one of the most important components of agricultural models, since it allo ws monitoring the state of the soil and water the crops. This paper offers a high-le vel architecture for vineyard management using free a vailable satellite and in-situ sensing technologies. Specifically , the improv ement of the ef ficiency in water use for irrigation to achiev e a sustainable intensification of irrigated vineyards is now adays a fundamental need. T o plan the proper use of water and prev ent ov er-irrigation and to generally control irrigated areas, Earth Oberse vation (EO) data and in-situ sensing can be used to deriv e actual and forecasted crop water requirement maps, as well as soil moisture estimates. W e propose to use satellite imagery and radar data from the European Space Agency’ s Sentinel-1 and Sentinel-2 satellite constellations for automatic feature extraction for soil moisture content (SMC) prediction. W e compare the performance of 2 model, one of which includes ground measurements and the other one does not. In w ater , molecular dipole moment oscillation, induced by f alling electromagnetic radiation, produces polarization in reflected radiation. Therefore, satellite radar’ s backscattering in polarized microw av e bands depends on the SMC, incidence angle and landscape details (ground roughness and v egetation). Sentinel-1 is a constellation of two imaging Synthetic Aperture Radar (SAR) missions at C-band [2]. W e use dual (VV+VH or HH+HV) and single (HH or VV) polarisation for SM modes. T o increase accuracy and add automatic crop segmentation to the process, we use Sentinel-2 imagery (R,G,B, NIR bands) [3,4]. W e propose automatic feature extraction based on the ground data observ ations and historical satellite imagery from Sentinel-1. W e use additional satellite imagery and av ailable ground measurements, such as weather station data and data from soil moisture sensors, to verify the accuracy of predictions. Usage of VV alone or the combination of VV and VH giv e similar accuracy on SMC estimates. W e use ND VI values from Sentinel-2 imagery to increase the accuracy of our predictions (Fig.1). Figure 1: Input data.W e used 14 features from multiple channels (EO and ground data). W e use dual (VV+VH or HH+HV) and single (HH or VV) polarisation from Sentinel-1, R,G,B and NIR bands from Sentinel-2 (top panel) and observations of soil moisture sensors from 20 sites from 3 wine regions in Australia; 7 years of daily data. 2 (a) (b) Figure 2: T est location for our model application (Hunter V alley , Australia): (a) location of the soil moisture sensors for each sort of the crop on the field: 1 - Glenesk Shardonnay R80, 2-Glenesk Shardonnnay R110, 3-Glenesk Shardonnay R132, 4 - Glenesk Shardonnay R192 and Mendoza, 5-Glenesk Merlot R165, 6-Glenesk Semillon R245, 7-Glenesk Shiraz R50, 8-Glenesk V erdelho R211; (b) heatmap for crop stress analysis (Normalized Difference V e getation Index, ND VI). W e use next image prediction with con v olutional sequence-to-sequence autoencoder for prediction of soil moisture content from historical radar data (satellite aperture radar , SAR). Deep recurrent neural networks prov ed to be very useful in predicting sequences because they learn temporal dependencies in sequential data [5]. There are many examples of successful applications of seq2seq architectures to sequences of images [6], [7]. The type of sequence-to-sequence architecture, proposed in this work, was first applied to satellite imagery in [8]. T o access satellite imagery , we used Sentinel Hub for bulk do wnload of Sentinel-1 and Sentinel-2 data. As input features, we used a combination of bands (Sentinel 1 : C and X bands, Sentinel 2: combinations of R, G, B and near-infrared (NIR) bands). As a source of ground-truth data, we use soil moisture sensor readings and historical weather sensor data from a vineyard in Hunter V alley , Australia. The map of the vineyard we were working with with ov erlaid location of soil moisture sensors is depicted on Fig.1. W e also use historical data, collected ov er 20 years, together with ground data from weather station and soil moisture sensors to provide ground-truth measurements and v alidation to our model. The proposed architecture consists of two recurrent neural networks, combined in an encoder-decoder framework (Fig.3. The input image was flattened into the 1D array and processed with an encoder network. Conv olutional long-short term (LSTM) cells are used in both encoder and decoder networks. The output of the model is the sequence of predicted images, which contain visual information about the amount of water in the soil. W e compare two architectures for soil moisture content prediction AE and LSTM models. Our research suggests that we can obtain similar prediction accuracy with AE and LSTM architectures (Fig.4. LSTM architecture performs better with smaller amount of observ ations but requires soil moisture sensor (SMS) data, while AE works only with EO data, but requires significant amounts of historical data. 3 Results and Discussion In this paper, we aim to demonstrate an application of AI that can help underpin the work needed to support the sustainability goal of zero hunger . Increasing agricultural yield and reduction of wastage of natural resources, such as water , can dri v e positiv e economical outcomes for the f armer e.g. reduced costs and in vestment on irrigation, soil moisture sensors and manual labour . On a macro scale, increasing agricultural yield and total f actor producti vity (TFP) of a country will assure that the needs of the growing population don’ t outstrip the ability to supply food. According to [10], growth in yield and labour productivity are highly associated with pov erty reduction, but the e xtent to which they af fect pov erty sharply v aries across regions. W orldwide precision agriculture market is expected to reach approximately US$ 7.9 billion by 2022 gro wing at a compound annual gro wth rate (CA GR) of 16% [11]. While organisations and companies 3 Figure 3: The architecture of a model for prediction of soil moisture content consists of tw o recurrent neural networks, combined in an encoder -decoder frame work. The input image w as flattened into the 1D array and processed with an encoder network. Conv olutional LSTM cells are used in both encoder and decoder networks. The output of the model is the sequence of predicted images. Here, colour represents the amount of water in the soil from wet (blue) to dry (red). Figure 4: The architecture of the LSTM model.The inputs to this model are radar reflectance data and ground SMC measurements. W e use the data from tw o satellite constellations (Sentinel 1 and Sentinel 2) as input EO data and seven years of daily ground SMC measurements from fourteen sites for training. Additionally , we use publicly av ailable weather data and rainfall data. may be interested in the lar ge financial v alue of this mark et US$240B [11], there is also a growing trend of organisations working on global sustainability goals that not only aids social good on a large scale but also makes a positi ve impact on the firm’ s reputation. According to McKinsey [12], one top reason why organisations address sustainability is to "b uild, maintain or improve reputation". Therefore, application of AI to earth observat ion (EO) data has a significant potential on large scale as well on the small scale. On the large scale, it will benefit lar ge organisation and country economy as a whole. At the same time, on the small scale it will help farmers to manage their land more efficiently and to g ain more profits from their land. The usage of earth observ ation (EO) data and ML-tools for analytics and forecasting thus significantly impacts the country and provides agricultural sustainability and productivity on global scale. Current method provides better recommendations for farmers. Such recommendations are particularly important in the re gions where usage of measuring equipment for obtaining the information about soil or using drones for field monitoring is difficult due economic situation in the re gion. W e hav e tried our predictions on the medium-sized vine yard in Australia. Our experiments on the pilot farm hav e sho wn that the current state-of-the-art ML tools can effecti vely replace costly soil moisture sensors and are four times cheaper in implementation. It has been shown, that the usage of EO data and ML for analytics and forecasting can be valuable tools for agricultural yield prediction [13] and for pov erty prediction [14]. W e propose adding more to this functionality by soil moisture prediction, automatic crop detection and crop stress analysis. The combination of the proposed tools pro vide visual aid and recommendations for farmers ho w to 4 use their land in a the best way . W e propose an intuitive interf ace that includes: detecting the current crops on the patch of land the farmer is located in; providing recommendations on actions, required for particular crops recommendations for using other crops for current state of the land (including soil analysis, water content analysis etc.), which will produce better yield while consuming less resources, scarce in particular region. References [1] Food and Agricultural Organisation of the United Nations (2009) How to F eed the W orld in 2050. Expert r eport . [2] Snoeij, P . & Attema, E. & T orres, R. & Le vrini, G. & Croci, R. & L ’Abbate, M. & Pietropaolo, A. & Rostan, F . & Huchler, M. (2009). Sentinel 1 - the future GMES C-band SAR mission. [3] Paloscia, S. & Pettinato, S. & Santi, E. & Notarnicola, C. & Pasolli, L. & Reppucci, A. (2013) Soil moisture mapping using Sentinel-1 images: Algorithm and preliminary validation. Remote Sensing of Envir onment 134 , pp. 234 – 248. [4] P asolli, L. & Notarnicola, C. & Bruzzone, L. & Bertoldi, G. & Della Chiesa, S. & Hell, V . & V aglio Laurin, G. (2011) Estimation of Soil Moisture in an Alpine Catchment with RAD ARSA T2 Images. Applied and En vir onmental Soil Science, 2011 , pp. 1 – 12. [5] Sutskev er , I. & V in yals, O. & Le, Q. V . (2014) Sequence to sequence learning with neural networks. Advances in Neural Information Pr ocessing Systems , pp. 3104 – 3112. [6] Smilevsk y , M. & Lalkovsk y , L. & Madjaro v , G. (2018) Stories for images-in-sequences by using V isual and Narrativ e Components. ArXiv preprint: 1805.05622v3. [7] Noroozi, M. & Fav aro, P . (2016) Unsupervised Learning of V isual Representations by Solving Jigsaw Puzzles. Eur opean confer ence on computer vision, ECCV 2016. [8] Hong,S. & Kim, S. & Joh, M. & Song,S-K. (2017) PSIque: Next Sequence Prediction of Satellite Images using a Conv olutional Sequence-to-Sequence Network. W orkshop on Deep Learning for Physical Sciences, NIPS 2017 . [9] Ronneberger , O. & Fischer,P . & Brox, T . (2017) U-Net: Con volutional Networks for Biomedical Image Segmentation. MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention pp. 234 – 241. [10] Goldman Sachs (2016) Precision Agriculture: Cheating Malthus with Digital Agriculture. Profiles in Innovation. [11] Beige Market Intelligence (2017) Global Precision Agriculture Market - Strategic Assessment and Forecast 2017-2022. 141 p. [12] Bové, A. & D’Herde, D. & Steven S. (2017) Sustainability’ s deepening imprint. McKinsey & Company , Insights on Sustainability & Resource Pr oductivity . [13] Y ou, J. & Li, X. & Low , M. & Lobell M. & Ermon S. (2017) Deep Gaussian Process for Crop Y ield Prediction Based on Remote Sensing Data. Pr oceedings of the Thirty-Fir st AAAI Confer ence on Artificial Intelligence (AAAI-17) . [14] Jean, N. & Burke, M. & Xie,M. & Davis, W .M. & Lobell M. & Ermon S. (2016) Combining satellite imagery and machine learning to predict pov erty Science . 5