Sponsors set a program's success metrics before results exist, choosing measures they expect the program to move, and publish them so anyone can argue for others.
Borderline: whether this move is justified depends on the situation, and only the person in it can judge.
An example
Staff trained, documents uploaded, monthly logins, and hours per audit engagement: those were the four measures the chief knowledge officer of a national accounting firm wrote into a new knowledge-management program's scorecard in its first month. The officer expected the first three to move fast and the fourth slowly. The scorecard went to the firm's leadership team with a memo explaining why each measure was chosen and inviting other proposals. One leader added proposal win rates, and the officer accepted it. Two years later, the activity measures read green and hours per audit had fallen two percent. When a new chief operating officer asked whether the program was working, the officer pointed to the published scorecard, the measures everyone had agreed at the start, and the outcome still moving slowly, and the program's budget was renewed.
Why it works
Measures chosen because the program will move them (activity counts, leading indicators, ratios with friendly denominators) make the scorecard read as success, so the program looks better. Supporters no longer have to argue the merits at each review, which makes continued endorsement easy. Anyone who later proposes different criteria pays for it by looking inconsistent or opportunistic, which makes the alternative costlier. Control of the criteria-setting meeting is the precondition, along with metrics that look technically defensible, reviews tied to the same scorecard, and no outside party with the standing to reopen it.
The thinking behind it
Robert B. Cialdini Robert Cialdini's Influence (1984) describes commitment and consistency: people who have taken a position, especially a public or written one, feel pressure to act in line with it. A scorecard the leadership team approved at launch is that kind of position. Two years later, proposing different measures means departing from what everyone agreed, which is why accepting reasonable additions at the start, and publishing the rationale, makes the scorecard both stronger and fairer.
Daniel Kahneman In Judgment under Uncertainty: Heuristics and Biases (1974), Tversky and Kahneman describe anchoring: people estimate by adjusting from a starting value, and the adjustment is usually too small. The first scorecard on the table becomes the starting value for every later discussion of whether the program works, and measures proposed later are judged as departures from it. That is why writing the scorecard first carries so much weight, and why skeptics should contest it at the start.
When it fits
Useful at the launch of a program you sponsor, when no results exist yet and the criteria-setting meeting is still open. It needs the standing to author the scorecard, real metrics with real pedigree, and a review cadence tied to the same measures.
How to use it well
Draft the scorecard before anyone else does, and make it defensible.
- Choose measures you expect the program to move, and include at least one outcome measure, even a slow one.
- Publish a short rationale for each measure and invite proposals for others before approval.
- Accept reasonable additions, since they make the scorecard harder to reopen later.
- Tie every review to the same scorecard.
When results come in, report all the measures, including the ones that disappoint, and say what you expected of each. A sponsor who reports a slow outcome honestly keeps the right to the scorecard.
What it costs and where it backfires
A scorecard of easy measures earns you short-term peace and long-term suspicion: the first outsider who asks about outcomes can make the whole program look like theater. The clean form needs the scorecard published and open to challenge. Hiding the rationale, refusing outcome data to people entitled to it, or changing the measures later to rescue the result moves into concealment or falsification.
When it is used on you
Treat the criteria-setting meeting as the decision. Ask for an outcome measure beside each activity measure, and ask what result would count as failure. If the scorecard is set, propose additions alongside the sponsor's indicators, and bring in a new board chair or an independent reviewer to test it against results that matter.