The signals that matter before outcomes move

Why implementation outcomes, fidelity, and broader performance signals matter before intervention outcomes begin to move.

Transformation is often judged through outcomes that move late, slowly, or ambiguously. In many programmes, leaders look first to intervention outcomes such as productivity, waiting times, utilisation, financial performance, or quality indicators.

Those measures matter, but on their own they are often a blunt way of judging progress.

By the time a high-level outcome has failed to move, months of activity may already have passed without enough visibility of whether the intervention is actually taking hold. Equally, pressure for early results can encourage premature judgement before a new model has had time to stabilise in practice. That is why implementation outcomes matter as much as intervention outcomes: they provide earlier, more sensitive signals of whether change is being used, carried, and sustained as intended.[1][2]

Intervention outcomes and implementation outcomes are not the same

A useful starting point is to distinguish the two. Intervention outcomes are the service, clinical, operational, or financial effects a change is trying to produce. Implementation outcomes are the indicators that show whether the intervention is actually being adopted and delivered in practice. Proctor and colleagues made this distinction explicitly, arguing that implementation outcomes are conceptually separate from service system and clinical outcomes and function as important preconditions for them.[1] In practical terms, this means a programme may be working hard without yet producing visible service impact; or may be failing quietly while senior teams wait for late outcome measures to reveal it.

Earlier signals matter because they are more sensitive

This is why implementation outcomes are often more sensitive indicators of progress. Measures such as uptake, reach, consistency of use, adherence to the new process, drop-off, local variation, workaround behaviour, and early sustainment signals can show much earlier whether a change is gaining traction. They are not a substitute for intervention outcomes, but they are often the first useful signs that implementation is moving in the right direction – or drifting. The literature over the last decade has only strengthened this point, with adoption, fidelity, penetration, and sustainment continuing to sit at the centre of implementation outcomes research.[2]

Fidelity determines whether outcomes can be interpreted properly

Fidelity is especially important in making sense of results. Without some understanding of whether the intervention is being used as intended, outcome data becomes difficult to interpret. Weak results may reflect a poor intervention, poor implementation, or a distorted version of the intended model in practice. Carroll and colleagues’ fidelity framework remains useful because it separates the content, frequency, duration, and coverage of an intervention from the moderating factors that influence how faithfully it is delivered.[3] The practical implication is straightforward:

if leaders do not know whether the process is being followed as designed, they may either judge the intervention unfairly or overestimate success based on superficial compliance.

Learning, adaptation, and timing all shape progress

This matters because transformation in real settings is rarely a straight line from design to measurable impact. Implementation takes time. It often requires a period of testing, learning, and local tailoring before intervention outcomes become stable enough to judge. In that phase, a rigid demand for early interventional results can be counterproductive. It may suppress adaptation, encourage performative compliance, or drive teams toward shortcuts that produce short-term numbers without building sustainable practice. Improvement science has long argued for structured testing and learning in local context rather than immediate wholesale implementation. The Model for Improvement and PDSA cycles, for example, are explicitly designed to help teams learn how a change works in their local environment before assuming that early results are final.[4][5]

Quick results are not always the strongest signal of sustainability

A further complication is that quick results are not always the strongest signal of sustainability. Some early gains are achieved through exceptional effort, temporary funding, or delivery conditions that are difficult to maintain. Recent elective sprint-style recovery efforts illustrate the point: activity can rise and headline metrics can improve, while the underlying mechanics of delivery remain fragile. If the fundamental BAU constraints are unchanged, the system may simply be pushing the problem forward rather than resolving it.

In that sense, quick gains can amount to “kicking the can down the road” unless they are accompanied by changes that hold once the sprint conditions fall away. At the same time, the opposite risk is also real:

if progress is not evidenced clearly enough or quickly enough, executive trust can weaken, teams can disengage, and programmes may be decommissioned before the intervention has had a fair chance to mature.

When metrics stop functioning as useful indicators

This is also where performance metrics become more complicated than they first appear. Metrics are essential, but they can stop functioning as useful indicators when they are interpreted too narrowly. A metric that begins as a signal can become distorted when it becomes the target itself. Crawford’s BMJ discussion of Goodhart’s law captures the problem clearly: when waiting times became a target, they ceased to function as a good measure in the way originally intended.[7] The broader lesson for transformation is that a single metric, read in isolation, can mislead. Improvement in one number may conceal deterioration elsewhere. Equally, lack of movement in a high-level outcome may hide genuine implementation progress that has not yet had time to translate into service effect.

The issue is not whether to use metrics, but whether they are being interpreted intelligently, alongside balancing measures, implementation outcomes, and the wider pattern of operational signals.

Why implementation outcomes should sit at the centre of delivery oversight

In my view, this is one of the common weaknesses in transformation programmes: too much emphasis on eventual intervention outcomes, not enough on the earlier signals that tell you whether the model is actually taking hold. It is also why I think implementation outcomes should be seen not as secondary process measures, but as part of the core decision-making apparatus of change. They help answer practical questions that intervention outcomes cannot answer early enough.

• Are people using the model?

• Are they using it consistently?

• Has it reached the intended teams?

• Are they following the intended process?

• Are workarounds emerging?

• Is the intervention holding under pressure?

These are not minor operational details. They are often the earliest clues as to whether later results are likely to materialise.

Delivery structures make progress observable and steerable

Delivery structures matter because they make these signals observable and usable. PMO structures, governance, QI methods, milestone discipline, and review cadence are not administrative overheads. They are part of the mechanism by which implementation becomes steerable. PMO provides visibility of dependencies, slippage, risk, and escalation. QI methods provide disciplined testing and refinement. Governance provides decision-making and backing when adaptation is needed. Measurement for improvement, rather than measurement for assurance alone, allows leaders to use data as a guide to learning rather than only as a retrospective judgement.[4][8] In practice, this is how organisations maintain both pace and intelligence: enough structure to stay credible, enough learning to avoid rigidity.

Conclusion

Ultimately, successful transformation depends not just on whether outcomes move, but on whether organisations can see early enough what is happening, whether the model is being implemented as intended, and whether the signals being watched are actually meaningful. Intervention outcomes remain essential. But without implementation outcomes, fidelity, and a broader reading of performance, leaders risk judging progress too late, too narrowly, or on the wrong evidence. In practice, that is often the difference between a change that is learnable, steerable, and sustainable, and one that is either forced prematurely or abandoned too soon.[1][3][7][8]

References

[1] Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011.

[2] Lengnick-Hall R, Stadnick N, Dickson KS, et al. Ten years of implementation outcomes research: a scoping review. Implementation Science. 2023.

[3] Carroll C, Patterson M, Wood S, et al. A conceptual framework for implementation fidelity. Implementation Science. 2007.

[4] Institute for Healthcare Improvement. Model for Improvement. IHI.

[5] Institute for Healthcare Improvement. How to Improve: Model for Improvement – Testing Changes. IHI.

[6] NHS England. Leading Large Scale Change: A practical guide. 2018.

[7] Crawford SM, Thorlby R, Dixon A. Goodhart’s law: when waiting times became a target, they stopped being a good measure. BMJ. 2017.

[8] NHS Improvement. Measurement for improvement. NHS Improvement. 2017.