Design the intervention before you design the indicator

#Question

What should be specified before a public-health programme chooses its indicators?

#Answer in brief

An indicator should measure a claim the programme is actually making. That requires a prior model of the problem, its determinants, the population the programme intends to reach, the mechanism through which activities are expected to produce change, the conditions required for implementation, and the outcomes that should follow. The 2024 CDC Program Evaluation Framework places programme description—including need, inputs, activities, outcomes, contextual factors and a logic model—before the choice of evaluation questions and measures.^[1] Updated MRC guidance for complex interventions likewise emphasizes programme theory, context, stakeholder perspectives, uncertainty and refinement across development and evaluation.^[2]

If the indicator comes first, the programme can end up optimizing what is easy to count rather than what needs to change.

#Start with the epidemiological problem

A programme should begin with a bounded statement of the problem.

Not merely: vaccination coverage is too low.

A stronger problem statement might specify that timely receipt of a defined dose is lower in particular cohorts or areas, with delays concentrated after a known transition in service contact. That formulation exposes the population, outcome, time dimension, heterogeneity and a possible point in the service pathway where intervention could occur.

The problem statement should not prematurely commit to a solution. If the observed issue is delayed vaccination, a reminder campaign may be appropriate—or the true constraint may be appointment availability, incomplete records, mobility, stock interruptions, eligibility uncertainty, or repeated missed opportunities during health-service contacts.

#Move from problem to determinants

The programme needs a working explanation for why the problem exists.

A useful determinant map can separate individual and household constraints, service access, provider workflow, information quality, organizational capacity, policy conditions, and social or environmental context.

This is not a demand to prove every causal mechanism before acting. Public health often has to act under uncertainty. It is a demand to state which mechanism the intervention expects to change.

The updated MRC framework treats programme theory as central because complex interventions interact with systems and context rather than acting as isolated treatments.^[2]

#Define the target population before the activity

Target population can mean people expected to benefit, people who receive the intervention, people whose behaviour must change, organizations that implement it, or areas in which it operates. These groups may differ.

For a hypertension-detection programme, the population expected to benefit may be adults with undiagnosed hypertension. The direct intervention recipients may be people attending primary care. The implementers may be health professionals. The organization-level target may be the appointment and follow-up workflow.

If these roles are collapsed, reach becomes difficult to interpret.

#Specify an intervention mechanism

A public-health intervention should be able to answer: what changes immediately because we do this, and how is that expected to produce the desired outcome?

Consider a synthetic vaccination-recovery programme. The intervention sends reminders to people recorded as overdue.

Its proposed pathway is:

valid registry record → overdue person correctly identified → reachable contact information → reminder delivered → appointment made → access feasible → vaccination received → registry updated.

Every arrow is a potential failure point.

If contact details are wrong, message quality is not the limiting factor. If appointments are unavailable, improved awareness may increase frustration without increasing vaccination. The programme theory therefore determines what must be monitored.

#Implementation conditions are part of the intervention

CDC's 2024 framework asks programmes to describe what needs to happen for implementation to achieve intended outcomes, including training, dose, duration, setting and participant characteristics.^[1]

This matters because the intervention is not just the visible activity. A school-based health programme may depend on staffing, consent procedures, scheduling, referral capacity, data exchange, material supply, local adaptation and safeguarding.

If those conditions fail, the programme may not have delivered the intervention it intended to evaluate.

#Outputs are not outcomes

An output is a product of programme activity: invitations sent, appointments scheduled, training completed, people screened.

An outcome is a change in the state of people, organizations, systems or health: increased timely vaccination, improved treatment initiation, reduced exposure, improved detection, reduced morbidity.

CDC's updated framework distinguishes outputs from outcomes and notes that programmes can be tempted to focus on inputs and outputs because outcomes are harder to measure.^[1]

That temptation is strongest when indicators are selected too early.

#Indicators should interrogate the theory

Once the logic is explicit, indicators can be attached to specific links.

For the synthetic reminder programme, a target-list step can be tested by the positive predictive value of overdue classification; contactability by valid contact proportion; outreach by booking after contact; service delivery by vaccination after booking; and population effect by cohort-specific on-time coverage.

This shows why one headline indicator is rarely enough.

#Expected outcome is not causal attribution

A programme can define an outcome it hopes to change without claiming in advance that it will cause that change.

An objective such as increasing on-time vaccination by a defined age is a programme goal. Demonstrating that the programme caused an observed increase is a separate evaluation question requiring an appropriate design and counterfactual reasoning.

This distinction protects the programme from building causal conclusions into its KPI architecture.

#A synthetic failure: excellent indicators, weak programme

Imagine a programme with a polished dashboard: 98% of outreach sessions completed, 95% of staff trained, 92% of reports submitted on time, and 10,000 information leaflets distributed.

All indicators are green. Yet the intended health outcome does not change.

A programme-theory review finds that the main barrier was transport to service locations. None of the activities addressed it.

The monitoring system was not inaccurate. It was accurately measuring successful execution of activities with a weak relationship to the binding determinant.

This is a design failure before it is an evaluation failure.

#The programme roadmap should include uncertainty

The MRC framework argues against treating complex intervention development as a linear pipeline and emphasizes identifying key uncertainties, context and system interaction.^[2]

A practical roadmap can therefore label established mechanisms, plausible but uncertain mechanisms, contextual dependencies, implementation assumptions and known competing explanations.

If the programme fails, investigators can ask whether the theory was wrong, implementation was weak, the target population was poorly defined, context changed, the outcome measure was insensitive, or the intervention worked differently across groups.

#A design sequence

A defensible sequence is:

problem → determinants → target population → intervention mechanism → implementation conditions → outputs → short-term outcomes → longer-term outcomes → evaluation questions → indicators.

The sequence is iterative rather than rigid. But reversing it—indicator first, programme logic later—encourages performative measurement.

#Practical questions

Before approving a public-health programme, ask:

1. What problem, in which population, at what time?
2. Which determinants are considered modifiable?
3. Which mechanism does each activity target?
4. What has to be true for implementation to occur as intended?
5. Which outputs confirm delivery?
6. Which outcomes represent meaningful change?
7. Which contextual factors could modify the pathway?
8. Which results would weaken the programme theory?
9. What evaluation design is needed for the intended claim?
10. Which indicators test each stage without substituting for it?

If the programme cannot answer these questions, adding more indicators will not create a coherent intervention.

#Limitations and uncertainty

Logic models and programme theories are representations, not proofs. A plausible pathway can still be wrong, incomplete or altered by context. The CDC framework is intentionally nonprescriptive and should be adapted to the programme and users.^[1] The MRC framework is designed for complex interventions and emphasizes iterative development rather than one mandatory diagram or method.^[2] The synthetic vaccination example is illustrative and does not represent an evaluation of a real programme.