The first pilot sites are chosen and resourced to succeed, and the result is presented honestly as proof that the model can work under good conditions, before harder sites are tried.
Borderline: whether this move is justified depends on the situation, and only the person in it can judge.
An example
After two earlier tools had failed, the chief medical information officer of a twelve-hospital health system needed the executive committee to believe an ambient documentation tool could work at all. The officer and the vendor chose three flagship hospitals with strong IT teams and recently upgraded record systems, and the vendor's staff were on the units daily. Documentation time fell forty percent. The proposal to the committee said so, and also said that the three sites were the best-equipped in the system, that the support ratio was three times what a rollout would have, and that the next phase would test two rural hospitals with weak networks before any full rollout. At the rural sites the gain was nine percent until the network was upgraded, and the committee funded the upgrades as part of the rollout.
Why it works
A single vivid result makes the case for the model look better, because a concrete case is easier to believe than a forecast, and it lowers the perceived risk of trying further. Holding back the next phase becomes costly once people have seen it work. Presenting the result for what it is, proof under good conditions, keeps the vivid number from standing in for a rollout forecast. It works when the audience needs to see the model work at all before investing further and when the next phase tests harder sites. Undisclosed subsidies and site choice turn the same pilot into a misleading forecast.
The thinking behind it
Amos Tversky Tversky and Kahneman's Judgment under Uncertainty: Heuristics and Biases (1974) describes anchoring: estimates are made by adjusting from a starting value, and the adjustment is usually too small. A forty percent gain from the best sites becomes the starting value for every estimate of the rollout. Presenting the conditions with the number, and testing harder sites next, is how the clean form keeps the anchor from setting the forecast.
Robert B. Cialdini In Influence (1984), Cialdini describes social proof: people look to what others like them are doing, and the more people seem to be doing something, the more correct it looks. Three flagship hospitals reporting success make the tool look proven to the other nine. The pull is strongest from people like oneself, which is why a rural hospital's question about how comparable the pilot sites were deserves an answer.
When it fits
Past failures or deep skepticism, and a question of whether a model can work at all, call for it. You need sites with the conditions to succeed, the resources to support them, and a plan to test harder sites next.
How to use it well
Decide before the pilot starts what it is meant to prove, and say so in the proposal: that the model can work under good conditions.
- Choose sites that give the model its best chance, and say that is what you did.
- Resource them properly, and record what that support cost per unit of outcome.
- Present the result with its conditions: why the sites were chosen and how they differ from the rest.
- Propose the next phase at typical or difficult sites before any full rollout.
Report the harder sites' results with the same prominence as the first.
What it costs and where it backfires
A result from the best sites will be discounted, fairly, by people who run the worst ones. If later sites underperform, people may remember the headline number and feel misled. The move crosses the line when site choice and subsidy are hidden and the result is offered as a forecast for the whole rollout; that is concealing material facts from the people deciding, and it tends to produce a rollout that collapses at ordinary sites.
When it is used on you
Offered a pilot result as the case for rollout, ask what it cost per unit of outcome, which sites were considered and rejected, and how the chosen ones differ from the rollout population. Propose a replication at a typical site before full rollout.