When Metrics Lie

3 decision traps that make smart teams misread dashboards

5-min interactive
Outcome Bias

Judging decisions by results, not process quality.

Sample size
50High noise

Small samples amplify luck. Success tells you almost nothing about decision quality.

Availability Bias

Overweighting data that's easy to recall or access.

Recency-weighted

Recent support tickets dominate over silent churn signals.

Metric Illusion

Optimizing the metric instead of the goal.

Optimization pressure30
Proxy metric
Actual outcome

The proxy still reflects the real outcome.

Playbook

  • 01Separate decision quality reviews from outcome reviews—run both, but on different schedules.
  • 02Build a 'silent metrics' dashboard: churn predictors, cohort curves, and leading indicators you rarely check.
  • 03For every KPI, document the outcome it proxies and the gap between them.

Olga Tatarkina · Product / Data Analyst · Decision-grade analytics

GitHub · LinkedIn