70 years of machine learning

Project Details

Book

Writer

highlight coding machine learning research writing

Screengrab

Description

I was invited to write a review about the history of machine learning for Advances in Geophysics. I reviewed the historical co-development of machine learning and geoscientific techniques. A particular focus is laid on Gaussian Processes (Kriging) and Neural Networks.

I read and analyzed over 300 papers for this work, featuring a selection throughout the manuscript.

Modern applications are illustrated with reproducible code examples.

This discussion of modern deep learning also ventures into specific components including:

• dense neural networks
• convolutional neural networks
• recurrent neural networks

Including significant architectures that developed and were applied in this time. The review does not yet include the transformer architecture as it was barely used in geoscience (and the chapter got long enough as is).

There is a preprint version and the typeset version available.

This review gives an overview of the development of machine learning in geoscience. A thorough analysis of the co-developments of machine learning applications throughout the last 70 years relates the recent enthusiasm for machine learning to developments in geoscience. I explore the shift of kriging towards a mainstream machine learning method and the historic application of neural networks in geoscience, following the general trend of machine learning enthusiasm through the decades. Furthermore, this chapter explores the shift from mathematical fundamentals and knowledge in software development towards skills in model validation, applied statistics, and integrated subject matter expertise. The review is interspersed with code examples to complement the theoretical foundations and illustrate model validation and machine learning explainability for science. The scope of this review includes various shallow machine learning methods, e.g. Decision Trees, Random Forests, Support-Vector Machines, and Gaussian Processes, as well as, deep neural networks, including feed-forward neural networks, convolutional neural networks, recurrent neural networks and generative adversarial networks. Regarding geoscience, the review has a bias towards geophysics but aims to strike a balance with geochemistry, geostatistics, and geology, however excludes remote sensing, as this would exceed the scope. In general, I aim to provide context for the recent enthusiasm surrounding deep learning with respect to research, hardware, and software developments that enable successful application of shallow and deep machine learning in all disciplines of Earth science.