Jason Dea

These aren't the droids you're looking for

A lot of people think "accuracy" is all that matters in AI. They're half right.

Accuracy in AI is actually a combination of two competing goals: precision and recall. It's the tension between them that explains why your spam filter sometimes feels too aggressive (or too lax), why medical tests work the way they do, and why there's no such thing as a "perfect" AI model.

Precision vs. recall Two competing goals behind "accuracy" in AI Your use case sets the balance PRECISION Being right when you act When it's better to be sure Medical diagnoses, fraud alerts RECALL Seeing everything worth acting on When it's better to catch everything Disease screening, customer support Which costs you more: being wrong, or missing out?

Here's the difference:

  • Precision = being right when you act.
  • Recall = seeing everything worth acting on.

Take a spam filter:

If it flags only a few emails and they're always spam it may be because it's tuned for high precision, but low coverage. If it flags every possible spam email but catches a few real ones too the opposite is true, it has high coverage, but low precision.

Different use cases will have different goals, which in turn require different tradeoffs:

  • High precision use cases could be things like medical diagnoses, or fraud alerts where it's better to be sure.
  • High recall use case examples could be disease screening, customer support where it's better to catch everything, just in case.

There's no universal "best." It depends whether the bigger risk in your use case is being wrong or missing out.

Related