How learning from 1 data point is possible:

From a scientific or statistical standpoint, it seems ridiculous to think 1 data point can teach us much. Even a study with 10 data points is laughably small.

Yet, 1 data point sometimes teaches us a LOT. But how?

5 ways:

[thread]

1. By Bayes Factor: suppose hypothesis “A” says a data point is nearly impossible, and hypothesis “B” says the data point is quite likely.

Then the existence of that one data point (by Bayes’ rule) should move you substantially toward believing hypothesis B (relative to A).

Example: you have had a rash on your arm for 10 years (with no variability). You buy some “rash cream” off of a shady website, and within 2 hours of applying it, the rash is gone. You can be confident the cream works because it’s otherwise highly unlikely for the rash to vanish.

2. By Alerting Us to a New Possibility: sometimes a single data point shows us that a possibility or phenomena exists that we had never before seen or considered.

Example: Roentgen had a cathode tube covered in heavy black paper, and was surprised when an incandescent green light escaped and projected onto a nearby fluorescent screen. He had discovered a new phenomenon. It eventually led to the discovery of x-rays!

3. By Illustrating a Causal Mechanism: studying a single data point or example can allow us to see HOW a mechanism works, enabling us to build up a causal understanding or model. We can then apply this understanding to other examples.

Example: you take apart one mechanical clock, and pay close attention to how it works. From this experience you build up a causal model of how such clocks function. Later, when a different mechanical clock stops working this causal model helps you quickly diagnose the problem.

4. By Providing the Mean When There is Very Low Variance: normally a single data point doesn’t allow an accurate estimate of any statistics. But in situations of very low variability, a single data point can be an accurate approximation of the mean!

Example: there is very little variability in how long it takes to walk to the store from your home. Hence, walking that route just once allows you to estimate quite accurately how long it will take in the future (e.g., it takes about 40 minutes to walk to the store).

5. By Causing Us to Think of a Hypothesis: if we see something work for a single example, or see that things went a certain (surprising) way, it can give us a hypothesis or approach that may apply to other cases - especially if it coheres with other justified beliefs we have.

Example: suppose that, while bartering over a price at a food stand, you see a friend use a negotiation tactic you have never seen before. Upon seeing this tactic used, it immediately makes sense to you that the tactic works, yet the idea had simply never occurred to you before.

Unfortunately, we humans often err on the side of OVER reacting to a single data point. We take one example, anecdote, or life experience and generalize it inappropriately.

But, if we are very careful, there are sometimes valid ways to learn a LOT from just 1 data point!