How to Expect the Unexpected: Leveraging the Science of Predictions

Author and mathematician Kit Yates provides a deep dive into the science of making predictions and understanding randomness.

He explores how our inherent biases can lead to misconceptions about randomness, nonlinearity, and value for money.

Drawing from his book ‘How to Expect the Unexpected’, he shares insights on how we can leverage this knowledge in everyday decision-making.

‘Exponential growth bias’ is a common misunderstanding where people underestimate exponential growth, which is growth in proportion to its current size.

This bias can significantly impact financial behavior and pandemic responses.

We are so familiar with linear relationships that we never, or we often don’t, stop to step back and think, what happens if the relationship between these two variables is not linear? – Kit Yates

Our perception of randomness is often skewed due to our tendency to look for patterns where there may be none.

This can lead us into making causative inferences based on perceived patterns—a phenomenon known as apophenia.

‘Middle bias’ is a phenomenon where people tend to choose middle-most options when uncertain about an answer.

This extends from multiple-choice tests and games like Battleship to choosing toilet cubicles or insurance packages.

Humans struggle with generating randomness ourselves.

In an experiment involving pressing two keys randomly on a keyboard, a computer could predict the next key press with accuracy greater than 50%, highlighting our inability to truly be random.

The fact that there’s this nonlinear relationship, that the area scales as the square of the diameter while the price scales linearly with the diameter, means you get better value for money with your larger pizzas. – Kit Yates

Biases and misconceptions about randomness can affect decision-making processes in various aspects of life—from trivial choices like picking toilet stalls or music playlists—to more significant decisions like choosing insurance packages.

By understanding these biases and misconceptions about randomness and nonlinearity, we can make better-informed decisions and avoid cognitive traps.

Awareness of our ‘middle bias’ or tendencies towards pattern recognition helps us make choices that are more aligned with reality.

‘Value for money’ often exhibits a nonlinear relationship.

For instance, larger pizzas offer better value for money than smaller ones if the price scales linearly with diameter but the area (which determines how much pizza you get) scales as the square of the diameter.

Our perception of randomness often leads to counterintuitive solutions.

For instance, Apple had to alter their iPod shuffle algorithm so that songs from the same artist wouldn’t play back-to-back, making it ‘less random to appear more random’.