You can immediately tell how hard it is to build a machine learning solution.

There's a way to estimate the complexity of the model and the amount of research necessary. This approach is inspired by Nassim Taleb's idea of Known Knowns and Unknown Unknowns.

The four types of machine learning problems:

Solved and Computable Problems

These are problem sets we have a good grasp of and a good solution for. In computery terms, this would probably be a p-hard problem. They're deterministic and often abundant.

Sure, we could teach a machine learning system to learn addition. Alternatively, and I know this is blasphemy in ML circles, but we could simply use a calculator and work on interesting problems.

Solved and not Computable Problems

Now it gets interesting. What are problems that are solved but cannot be computed easily?

We get into NP-hard things. The things you have to use heavy compute for like integer optimization, crypto mining. These are the problems with nested loops, the one's where you consider digging up trusty ol' Fortran instead of Python.

Physical inversion comes to mind, where the solution is clear, but it takes optimization to get there. Here the mechanisms are clear, but machine learning can speed up the computation significantly.

Machine learning can learn some good heuristics to reduce the problem space.

Unsolved and Computable Problems

This one is a bit of an oxymoron. How can we compute something that in itself is unsolved?

Fluid dynamics comes to mind. Interestingly there are fluids with wildly varying properties, like viscosity, that behave the same. This can be described by elementary calculations like the Reynolds or the Prandtl number.

There are multiple approaches for machine learning here. Machine learning can generate candidates from these abstractions for experiments. Alternatively, clustering algorithms can give insight into new computable abstractions.

Unsolved and not Computable Problems

This one is my favourite.

These are usually active topics of research, and creative machine learning solutions can work wonders.

These are the topics where experts, for decades, have been saying it's impossible. Not because the solution or a deterministic connection is non-existent, but because it is so complex that no theoretical, empirical, or computational solution could be built.

Machine learning techniques are uniquely positioned to work out these problems. The heuristics machine learning models learn from the training data can abstract the complexity and build systems that finally make this problem computable.

This is the Herculean trial of problems.

This atomic essay was part of the October 2021 #Ship30for30 cohort. A 30-day daily writing challenge by Dickie Bush and Nicolas Cole. Want to join the challenge?