Indicators are the metrics that are used in MSD programmes to track progress towards programme objectives and to assess the impact of its interventions.
In MSD programmes indicators should capture both the immediate outputs of the programme's activities and the broader systemic changes that are expected.
Defining indicators of change
Indicators are used to track the extent to which an intervention is producing the changes anticipated in your results chain. At least one relevant indicator needs to be defined for each level of the chain, i.e. for outputs, intermediate outcomes, outcomes and impacts. Indicators can be either qualitative or quantitative, but need to correspond to the change they are trying to measure, and also fulfil SMART principles (i.e. they need to be specific, measurable, achievable, relevant and timebound).
The DCED Standard defines a set of common indicators that are recommended for use by all programmes applying the Standard. One of the virtues of these common indicators is that results from different initiatives and programmes can be compared more easily.
They include:
- Scale: for instance, the number of target enterprises which benefit financially as a result of programme activities.
- Net income: net additional income accrued by targeted enterprises as a result of the programme.
- Net additional jobs created: net additional full time equivalent jobs created in target enterprises as a result of the programme.
These indicators may not fit well with every intervention. In this case, the DCED Standard allows the use of alternative indicators, if their use is justified.
A toolkit for measuring SME performance helps programme teams decide what aspects of SME performance to measure, as well as how to measure them. It includes an indicator inventory to choose from.
Measuring changes in indicators
Once the indicators for the results chain have been defined, the following steps should be carried out and recorded in a results measurement plan:
- Establish a baseline position for each indicator.
- Project the result expected at a specific moment in time.
- Measure changes in each indicator through the course of implementing the intervention.
- Establishing a baseline
Information on the baseline position should be recorded for each indicator. Conducting baseline studies can be expensive, so it makes sense to use information gathered during the initial diagnostic process for the intervention or for the programme as a whole, where possible. As the diagnosis is often carried out before actual implementation begins, it is important to check that data is still accurate.
The need for market systems programmes to be flexible and adaptive means that baseline data may become redundant if a decision is made at programme level to abandon a particular intervention or sector. So it may make sense to establish small baselines for pilot interventions, rather than producing a comprehensive baseline for all interventions. It is advisable to project the result expected at a specific moment in time. - Projections
Projections are not targets but estimates of what is expected to be achieved by a given date. Change at the systems level can be slow at the beginning of an intervention, and it is important to reflect this so as to not raise expectations. Even so, while it is important not to be bound by these projections, it is essential to have a benchmark that allows the programme team to assess whether change is occurring as expected.
Projections are also useful for estimating Value for Money in advance of implementing an activity, and for understanding whether particular investments are worth their cost. - Measuring changes in each indicator through the intervention
The methods and sources of information that will be used to measure changes in the relevant indicators should be recorded in the measurement plan. It is also necessary to include details of who will measure what, and when.
Monitoring activities should be carried out at relevant intervals. The precise time period will depend on the intervention in question and how quickly it is expected to show results. This will also depend on the pace of change in the wider market system. Monitoring efforts will need to be stepped up in very dynamic environments for instance.
It is important to consider what indicator data suggests is happening across all levels of the results chain
- If indicator data shows that progress is being made towards achieving the desired outcome for the intervention, the fact that indicator data is available for preceding levels will help to illustrate whether the intervention has contributed to this or not.
- If the outcome is not achieved, results for indicators in the preceding levels of the results chain may help to identify where things went wrong.
Using proxy indicators to track progress shows how Ghana MADE uses output level information to measure progress towards outcome and impact level indicators.