Success metrics get redefined so the numbers show improvement or a rival looks worse, with each change presented as fixing a measurement bias or aligning to strategy, whatever happened to actual results.
An example
At a regional airline, the on-time measure the operations group reported to the executive team changed twice in one year. The first change, introduced the quarter after a summer of cancellations, stopped counting delays coded to air traffic control and weather, described as fixing a known bias. The second moved delays where an aircraft waited for a part, which had been counted under the ramp, to maintenance. The maintenance director noticed when the monthly dashboard showed operations at 88 percent on time, up from 79, and maintenance-caused delays rose threefold. No restatement of prior months was published, so the old series simply stopped. The director rebuilt six months of data under both definitions from the raw flight logs, which took three weekends and required access the director had to request from the analytics team that wrote the new formula.
Why it works
Presented as more rigorous or better aligned to strategy, the new definition makes the reported gain look better, and accepting it takes no audit work, which makes it easy. Insisting on the old measure gets cast as nostalgia for an obsolete number, so stable comparison looks worse, and rebuilding the original series costs the challenger data access, technical fluency, and the nerve to accuse a peer in public. It depends on one party controlling the definitions, an audience with little time for methodology, and no independent function that locks baselines.
The thinking behind it
Herbert A. Simon Herbert Simon's Administrative Behavior (1947) analyzes organizations as systems of decisions shaped by premises the organization supplies, such as goals, information, and procedures, so whoever controls the premises influences the decisions. A metric definition is one of those premises. Change the formula and every later judgment about the operation changes with it, with no argument about the operation itself. Separating the people who define metrics from those judged by them puts the premise out of the judged party's reach.
George A. Akerlof In The Market for "Lemons": Quality Uncertainty and the Market Mechanism (1970), George Akerlof shows how a gap in information between those who know quality and those who cannot check it distorts judgment, and how people who cannot verify claims end up discounting all of them. The team that writes the formula knows what changed; the executive reading the dashboard does not. Restating prior periods and keeping a parallel measure close the gap, and they also protect honest improvements from being discounted along with doubtful ones.
Warning signs
Metric definitions or baselines change in the same cycle that reported results improve, or in the quarter the old series would have turned down. The explanation sits in a methodology footnote and calls the new formula more representative. No restatement of prior periods is published, so historical comparison breaks. A rival's numbers worsen under the new definition while the sponsor's improve.
How to defend yourself
Keep a parallel measure the sponsor does not control: audited financials, a third-party benchmark, or an outcome someone outside the function validates. When a definition changes, ask for the old and new formulas to be run on the same data for the last four quarters, and do not accept a reported gain until you have seen both. Request a written change log and a restatement of prior periods. If the new definition hurts you, get raw data access early, before you need it, and bring the comparison to internal audit or a board analytics committee with the dates the changes were made.
How a leader shuts it down
Separate the people who design a metric from the people judged by it, and give custody of definitions to an independent function. Require a change log and a restatement of prior periods with every new definition. Ask, each time a number improves, whether its definition also changed that cycle.