Market systems programmes typically work with selected market actors to drive systemic change but often have no direct contact with the wider populations expected to benefit. Although challenging, measuring these broader indicators is important for assessing the scale of market systems change.
A pragmatic approach to assessing systemic change provides guidance on how to develop a system change strategy and intervention plans that lay the groundwork for system change assessment, including how to set system boundaries and how to identify the system changes a programme aims to catalyse.
Beyond performance monitoring
For a programme to be successful it is important to understand how the context is changing, as a result of both programme interventions and other factors in the market. Programme monitoring should go beyond individual interventions and actively consider what is happening in the wider market system. There are two key reasons for this.
- Interventions are likely to generate both positive and negative effects beyond those anticipated in advance or captured in a results chain. If monitoring activities focus exclusively on what is expected, other effects and results may be missed. Additional monitoring activities are needed which take a step back from the activity's intervention logic. Although a results chain may define a pathway to systemic change, outcomes may follow a different trajectory or look quite different from what was expected.
- There are a wide range of external factors that will influence the effectiveness of a programme. Continuous surveillance of what is happening in the market system provides broader and richer information allowing programme activities as a whole to adapt to changing circumstances where necessary.
Principles to consider in assessing wider system change
The following principles are useful when designing a monitoring framework that looks beyond intervention performance to assess change in the wider market system.
Observe without focusing on confirming a hypothesis
Focusing on proving a hypothesis right (such as one specified in a results chain) will shape perceptions of how trends are interpreted or analysed. Remember that programme interventions are not necessarily the dominant causal factor in a process of change, and outcomes may also take a different form from that predicted in a results chain.
Consider relationships, perspectives, and boundaries
An insight from systems theory, which can be usefully applied to monitoring in a market systems context, is to focus on relationships, perspectives and boundaries.
- Relationships. While relationships are central to monitoring and evaluation, tools that seek to capture them, such as results chains or theories of change, are typically static.
- Perspectives. A single-perspective approach is likely to miss how different market actors relate to one another and why they behave as they do. While donors and programme teams may view an intervention as economic development, partners may be motivated by quite different considerations.
- Boundaries. Staff need to be conscious of the judgements involved in defining what is included within the frame of reference and what is excluded.
Adopt-adapt-expand-respond
The Adopt-Adapt-Expand-Respond (AAER) framework is designed to manage and measure systemic change processes, particularly in market systems development. The framework breaks down change into four distinct components:
- Adopt. This occurs when a market player successfully adopts a behaviour or practice, recognises the value of continuing these changes, and plans to invest in upholding them independently.
- Adapt. A market player adapts the adopted behaviour or practice changes, making qualitative or quantitative investments to continue or augment these practices without programme support, demonstrating a commitment to sustainability.
- Expand. A number of market players similar to the initial adopters implement comparable changes, either as direct copies or variants of the original innovation, which are maintained without programme support, indicating broader uptake.
- Respond. Market players in supporting systems may respond to a new innovation by reorganising, taking on new roles, developing their own offers or capitalising on emerging opportunities, enabling further development and signalling new capabilities within the system.
The framework enables programme teams to assess progress and judge whether further facilitation is needed. It also includes “if you left now…” questions to prompt reflection on the sustainability and scale of results, supporting the development of intervention-specific indicators and assessment of overall system-level impact.
Adopt-Adapt-Expand-Respond introduces the Adopt-Adapt-Expand-Respond (AAER) Framework to assess systemic change.
Systems, sapiens, and systemic change in markets seeks to refine a practitioner-developed framework – Adopt-Adapt-Expand-Respond (AAER) – for conceptualising systemic change, and offers case studies to demonstrate its utility in planning for, and measuring, such change.
Monitoring the pace of change
Monitoring activities need to provide information on the pace of change to inform programme management. Defining indicators that capture change at different stages can help teams understand how quickly change is unfolding.
- Leading indicators provide information before the result takes place, and reveal early signs that the intended change is likely to happen, or not. In a market systems context such an indicator could be a decision by a market actor not associated with a programme to adopt an innovation trialled by the programme.
- Coincident indicators yield information at about the same time as the result. These indicators confirm that the intended change actually happened and are important both for programme management and reporting. In a market systems context information on sales trends for a particular product or service would provide a coincident indicator.
Lagging indicators provide data about what has happened as a result of programme activities. While these are important for understanding how an intervention has worked, the fact that information becomes available only after the event may make them less useful for understanding how change is unfolding in a rapidly moving environment. Examples of lagging indicators include the DCED common indicators which record changes in jobs, income or the number of enterprises benefiting financially from an intervention.