These aren't the droids you're looking for
A lot of people think "accuracy" is all that matters in AI. They're half right.
Accuracy in AI is actually a combination of two competing goals: precision and recall. It's the tension between them that explains why your spam filter sometimes feels too aggressive (or too lax), why medical tests work the way they do, and why there's no such thing as a "perfect" AI model.
Here's the difference:
- Precision = being right when you act.
- Recall = seeing everything worth acting on.
Take a spam filter:
If it flags only a few emails and they're always spam it may be because it's tuned for high precision, but low coverage. If it flags every possible spam email but catches a few real ones too the opposite is true, it has high coverage, but low precision.
Different use cases will have different goals, which in turn require different tradeoffs:
- High precision use cases could be things like medical diagnoses, or fraud alerts where it's better to be sure.
- High recall use case examples could be disease screening, customer support where it's better to catch everything, just in case.
There's no universal "best." It depends whether the bigger risk in your use case is being wrong or missing out.
Related
- Hallucinations are a feature not a bugWhy AI hallucinations are built into how LLMs work rather than a bug to fix, and practical ways to reduce them: context, grounding, and cross-checks.
- How to Tell If Your Problem Is Actually a Good Fit for AITen signs a problem is a good fit for AI, from building an AI legal bill review product: unstructured inputs, high volume, inconsistent judgment, and more.
- Most AI Product Strategies Fail Before the First PromptAI product strategies fail at product 101, not model choice. Four tests for a real AI strategy: work replacement, default use, user confidence, and frequency.