Precision vs Recall: Your Spam Filter's Two Scores

Your spam filter gets two scores, and raising one often lowers the other. So does any AI that sorts things, like fraud alerts or search results. This 500x Explained Short draws it as a pile of mail on a table and a ring that marks what gets flagged, with two gauges that score the result.

The example

Say thirty emails arrive, twelve of them spam. The filter flags ten emails, and nine of those are spam. These numbers (and the 16- and 6-email nets later) are an illustration made for this video; the math on them is exact.

Precision: when it says spam, is it right?

Precision asks: of everything the filter flagged, how much was really spam? Nine of the ten flagged emails were spam, so precision is 9 / 10 = 90%. In Google's Machine Learning Crash Course terms, precision is true positives divided by everything the model called positive: TP / (TP + FP).

Recall: did it catch all the spam?

Recall asks: of all the real spam, how much did the filter catch? It caught nine of the twelve, so recall is 9 / 12 = 75%. The formula is TP / (TP + FN): the spam you caught divided by all the spam there was.

Let it flag more

Now let the filter flag sixteen emails: all twelve spam, plus four real ones. Recall goes up to 12 / 12 = 100%. Precision drops to 12 / 16 = 75%, because a quarter of what you flagged is now real mail.

That's the trade. How much the filter flags is set by its classification threshold. A lower threshold tends to catch more of the spam and more of the real mail with it. Google's course puts it this way: precision and recall often show an inverse relationship, where improving one worsens the other. The word "often" matters: scikit-learn's docs note that lowering a threshold can leave recall unchanged, and precision can even go up if the threshold was set too high.

Where AI makes the same trade

AI moderation filters score content and use a confidence threshold. AWS describes its moderation threshold as a way to balance detection of content (recall) against the accuracy of detection (precision). "Chat with your docs" tools make the same trade when they decide how many passages to retrieve: retrieving more passages can raise recall@K, and every extra passage that isn't relevant lowers precision@K. RAG evaluation tools such as Ragas score retrievers on context recall and context precision.

Try this

Decide which mistake costs you more, then tune for that.

• When false alarms cost more, like losing a real email or blocking a real customer, favor precision: flag less (a higher threshold).
• When misses cost more, like fraud getting through, favor recall: flag more (a lower threshold).

Every filter makes mistakes. You pick which kind.