Reflections on the good, the bad and the ugly of results chains.
Results chains are a commonly used tool in the design and monitoring of private sector interventions – but are they still fit for purpose and is it time for a re-think?
Last year I was lucky enough to attend the DCED Conference in Nairobi. I helped organise a micro-clinic taking a fresh look at results chains, a key tool in the design and monitoring of programme interventions. Here I share some of the group’s reflections and what might come next.
What are results chains?
It’s essentially a theory of change, developed at the level of specific interventions or sectors (versus a programme’s overall Theory of Change). They consist of boxes and arrows, illustrating the hypothesised causal pathways from activities all the way to the desired impact.
They’re widely used in private sector development programming, thanks in large part to the influence of the DCED Standard for Results Measurement. As described in the Standard, every intervention should have its own results chain, which provide the foundation for Monitoring, Evaluation, and Learning (MEL).
Below is an example results chain, reproduced from the DCED guidance:
The good…
There was consensus in the clinic that results chains are an extremely useful planning and design tool. The process of developing them helps teams to think through the intervention logic and provides a frame for identifying assumptions and interlinkages. They are also very flexible and can be used for any type of intervention and impact pathway.
We also agreed that results chains provide a strong foundation for MEL. From a ‘prove’ perspective, measuring change in each box helps programmes to build a contribution narrative (showing the causal pathway from activities through to impact). From an ‘improve’ perspective, teams can test the intervention logic and pinpoint where it is breaking down should the results be less than expected.
Several people in our group also saw results chains as a vital tool in fostering collaboration between the MEL team and implementation teams – helping to build a common understanding of the intervention logic and aligning expectations. Results Chains also provide a useful visualisation of the intervention, helping to communicate the essence of the intervention strategy in a single graphic.
…the bad and the ugly
A major issue that came up was around practicality. Perhaps because they are so flexible, and because there’s not much guidance regarding the essential elements (the ‘must-haves’ versus the ‘nice-to-haves’), results chains can easily become too complex, with too many boxes. Given that each box will have one or more indictors (as per DCED guidance), if results chains become too complex, the monitoring system can become overwhelmed.
I remember visiting a programme that was struggling with its MEL system. The first results chain I looked at had 42 boxes, and this was just for one of seven interventions in the livestock sector, which was just one of six sectors. As they had developed indicators for every box, they ended up with over 1,000 indicators!
Programmes can also struggle at the portfolio-level, with too many results chains to manage. Here there’s a trade-off between practicality and specificity. For example, several people in our clinic said they generally develop results chains for each partner – this allows them to capture the nuances of each partnership, but at the cost of multiplying the number of results chains.
We also felt that there’s a lack of guidance around the ‘hierarchy’ of results chains - from the programme-level Theory of Change, to sector and intervention-level results chains – including around the detail needed at each level, how the different results chains fit together, and what to monitor at each level. For example, the DCED Standard jumps straight to intervention-level results chains.
Finally, because results chains are a blank canvas there are no ‘guardrails’ in-place, meaning their quality can vary significantly. For example, results chains often lack clarity about the specific practice changes the intervention aims to catalyse and the system actors involved. Similarly, they do not contain, in themselves, any general theory or conceptual model of how change happens. This means they are not as useful as they could be in helping programme teams systematically think through how change will happen, or to identify key assumptions - especially in relation to systemic change.
Rethinking results chains
We agreed that results chains are still a useful tool, but that more guidance and structure is needed. Our initial recommendations focused on two areas:
- Adding guidance on results chains at different levels of strategy-making, from the programme-level to the sector and intervention-level. Sector-level results chains can be particularly useful in showing how different interventions within a given market system fit together, and for providing a frame for ‘top-down’ monitoring.
- Developing improved guidance on results chain essentials, to help programmes better manage complexity. The necessary elements include:
- the practice / behaviour changes to be adopted by system actors
- how these changes are expected to happen, including changing capabilities and motivations of actors
- the improvements in the ‘performance’ of actors resulting from the practice / behaviour changes (critical for sustainability)
We also discussed how you might build-in a general theory of how (systemic) change happens, to improve the consistency and utility of results chains, but without them becoming too complex or difficult to monitor. There were several suggestions for existing conceptual models that could be used as a foundation, such as the Capabilities, Opportunities, Motivation, Behaviour change (COM-B) model and various models of systemic change such as the Adopt-Adapt-Expand-Respond framework.
We would love to hear from the wider community on your experiences of using results chains and any suggestions you have.
At Tandem we are also working on a new framework for developing results chains which we hope to share soon.
1 comment
Dear Gareth,
Many thanks for a insightful and balanced blog on results chains in the MSD community.
You spot an important gap that has persisted in the guidance: the need to link intervention based results chains with the overall project or programme. Much work was done on inter-locking or cascading logframes - with large programmes back in the 1990's. The practice then was a bit simplistic: to equate a programme's output with a project/intervention's purpose. Here is some guidance on the subject from Intrac that I find quite useful
https://www.intrac.org/app/uploads/2016/06/Monitoring-and-Evaluation-Series-Beyond-logframes-13.pdf
On MSD work, my experience with a trust fund in Tanzania was to start conversations as to how AA parts of AAER framework map onto the Outcome and the ER map onto the impact (in a geographical, not a socio-economic sense). This has obvious implications on the M&E approach and their associated responsibilities. Impact is a function of scale (of practice) and socio-economic change (of market actors and, in the case of agricultural programmes, among farmers themselves)
Five lessons we learnt in doing this were:
a) making sure we linked the overall ToC and/or Logframe to the outcome of the market system analysis/diagnostic which also threw up a useful depiction of the current situation so made developing the baseline a more efficient exercise;
b) in the context of your fingerling example, designing a results chains involved a diagnosis of the constraints and opportunities among farming households who produce the fingerlings - in my view a major oversight of MSD work - ie, farmers are a highly heterogenous group;
c) to ensure a results chain is more than a "mini-me" logframe without the assumptions: i) some boxes were labelled as assumptions thus escaping a proliferation of indicators; and ii) we applied the rigour a ToC asks for by developing statements of how the changes unfold typified among the boxes in your fingering example at outcome level and above. Invariably these were to do with how and why the company responds to the support, how the traders respond to and benefit and how farmers would respond to and benefit from the traders' offerings, and what this trading relationship would look like;
d) a Monitoring and/or Evaluation system should not base itself solely in reference to a results chain. There is more to M and/or E than this; and
e) look to other more experienced M&E approaches and writings beyond MSD and the DCED standards that presented equally/more valid practices and lessons - we were at times too parochial in our knowledge gathering.
Given you took your title from a film: "the good" of results chains - played by Clint Eastwood - is clear, yet I was left wondering what's the difference between "the bad" - Lee Van Cleef - and "the ugly" Eli Wallach of results chains. 😉
Hoping some of the above is helpful and good luck with the new framework
Cheers,
Daniel
Add your comment
Sign up or log in to comment and contribute.
Sign up