The Tyranny of Metrics

An excellent look at how metrics can be used poorly to justify bad policies. Either we measure the wrong thing, measure it the wrong way, or – more often than not – game the measurement system so that we look better than we are.

Want it to look like your city’s crime rate is going down? Stop taking reports. Say that a burglary was just trespassing, or an assault was just disorderly conduct.

Want it to keep your surgery success rate high? Stop operating on marginal cases. Cherry pick your patients so you only get good outcomes.

If you mandate statistics, people will work to the stat, not the benefit. Teachers will “teach to the test,” rather than some larger, unmeasurable goal that actually benefits society.

In the end, it’s about trust. Do we trust that people are doing things to benefit the world? If we don’t trust, then we want to measure. And when we measure, we sometimes destroy the original goal. We can’t operationalize everything. And we hate that.