Goodhart-Aware Metric Selection

Thinking Tools5.1 Defining the Evaluation CriteriaAlso known as Goodhart's law

From the law of Charles Goodhart (1975), in the wording Marilyn Strathern gave it (1997), with the failure types of David Manheim and Scott Garrabrant (2018).

Before a number is tied to pay, promotion, or budget, predicts how people will game it and adds counterweights so the metric keeps tracking the result that matters.

When to reach for it

Whenever a measure is about to become a target, or an existing scorecard has started producing odd behavior.

How it runs

  1. Name the result that matters, apart from any metric.
  2. List candidate metrics and how well each tracks that result today.
  3. Ask how people could hit each metric without moving the result.
  4. Classify the likely failure: noise, extremes, false cause, or deliberate gaming.
  5. Add counterweights: paired metrics, guardrails, and audits.
  6. Recheck the metric against the result once incentives apply.

Where it sits in SOLVE eX

Go to the source

  • Problems of Monetary Management: The U.K. Experience, Papers in Monetary Economics, Reserve Bank of Australia, 1975
  • Improving Ratings: Audit in the British University System, European Review, 1997
  • Categorizing Variants of Goodhart's Law, 2018

Back to the catalog