Title: Accuracy assessment of binary classifiers across gradients in feature abundance
Speaker: Matthew Rossi, Earth Lab

Abstract:
With increasing access to high-resolution topography (< 1m spatial resolution), new opportunities are emerging to better map fine-scale features on Earth’s surface. As such, binary classifiers built from such data can be used to interrogate how the location and abundance of features are related to process. Whether mapping tree throw pits, mima mounds, landslide scars, or individual bedrock outcrops, it is an exciting time to test how well our geomorphic transport laws reproduce spatial patterns in surface morphology. To this end, there are many metrics in the data sciences that are well suited to quantifying pixel-level accuracy of binary classifiers, though they are potentially vulnerable to methods where classification is attempted across gradients in feature abundance. In this talk, I present a suite of synthetic examples that illustrate how two commonly used metrics, F1-score and Matthews Correlation Coefficient, perform under situations where the classifier is expected to perform well on unbalanced data. I critically examine how the type of error (random versus systematic) and the scale of features (size and shape) impact conventional approaches toward accuracy assessment. While Matthews Correlation Coefficient strongly outperforms F1-score in these scenarios, it retains sensitivity to feature abundance when the error is systematic with a bias that may encode information about the shape and scale of features on the landscape.

Speaker Bio:
Matthew Rossi is a geomorphologist at Earth Lab who is interested in developing a quantitative understanding of the processes and process interactions that sculpt the Earth’s surface. His research largely focuses on the relative roles of climate, tectonics, material properties, vegetation, and hydrology on setting the form and function of rivers and hillslopes. He received his B.S. in Geology at the College of William and Mary (2003) and his Ph.D. in Geological Sciences from Arizona State University (2014).