Deep learning has revolutionized computer vision and natural language processing in the past decade. Neural networks that approximate functions are the workhorse of deep learning, and have been tailored to a wide variety of applications. Often, deep learning relies on large amounts of labeled data for training.
Deep learning applications in science are beginning to open up new insights into earth and environmental data, especially in the remote sensing domain. High availability of large amounts of remote sensing data and “off-the-shelf” models and code now makes deep learning practical for remote sensing tasks such as image classification, object detection, scene segmentation, data fusion, transfer learning, and time series analysis.
An example of how a deep learning based image segmentation model can be applied to remote sensing imagery.
Earth Lab has used very high resolution multispectral imagery from DigitalGlobe to map impervious surfaces in urban areas. Traditionally a product generated from a series of supervised image classification tasks, image segmentation models from the deep learning community have been applied to train a model to segment impervious surfaces using any subset of spectral bands available from DigitalGlobe’s WorldView-2 multispectral imaging system. A collaboration with the Department of Environmental Design at CU Boulder enabled the training of the image segmentation model [BLOG].
A UNet image segmentation model was trained on a DigitalGlobe WorldView-2 image from 2016, and also applied to an image acquired in 2015, using a GIS polygon dataset as shown in top left as training data. The resulting segmentation mask is for all surfaces marked as ‘impervious’ in the training dataset (bottom), with the continuous prediction surface available before thresholding (top).
Neural networks are increasingly being used in science to infer hidden dynamics of natural systems from noisy observations. Earth Lab is at the forefront of science-based deep learning in environmental science. While typical deep learning applications are focused on image classification, sequence prediction, or regression tasks without reference to any particular scientific model or dynamical system, we are particularly interested in ways to embed science knowledge in deep learning models. As an example, we are building neural hierarchical models of ecological populations that combine the function approximation power of deep learning with well-known ecological models for occupancy, capture-recapture, an animal movement data (Joseph 2020 https://doi.org/10.1111/ele.13462).
Earth Lab draws upon the rich set of methods provided by deep learning to gain new insights from spatiotemporal data streams, and new applications in earth and environmental science are emerging at an increasing pace. As these applications develop, we believe that building strong links back to science is critical.
Dr. E. Natasha Stavros is the Director of Earth Lab Analytics Hub. She specializes in complex systems science, data science, image processing, and information technologies. She developed these skills as a fire ecologist, but has applied them in other complex systems including NASA Flight Projects, biodiversity science, and urban ecology.