What is Outcome Harvesting?
Outcome harvesting is a monitoring and evaluation (M&E) method that allows us to retrospectively determine changes achieved and the project’s or initiative’s contribution to these. This method helps the users identify the relevant changes (“outcomes”) that took place, collect (“harvest”) evidence of the changes, and then, working backwards, determine whether and how the intervention has contributed to these changes. In this sense, it differs from traditional monitoring and evaluation methods, which measure progress toward predetermined objectives. [1]
When is Outcome Harvesting used?
Outcome Harvesting is especially useful and suited to monitoring and evaluating initiatives in complex programming contexts where paths to change are less predictable and the relation between cause and effect cannot be fully understood. [2] Such contexts often require adaptive management approaches, where continuous analysis and adaptation of the intervention to respond to the changing and dynamic environment is necessary. Traditional monitoring & evaluation methods tend not to suffice in such cases; instead, they require M&E systems that can cope with intentional project changes during implementation; that fit well into rapid cycles of planning, monitoring, and reflection; that encourage continuous experimentation and testing of approaches as situations evolve; and that “feed hunger for learning.” [3]
The Strengthening Rural Governance for the Right to Adequate Food multi-country program, implemented by six different partner organizations and supported by Welthungerhilfe, adopted Outcome Harvesting to monitor its social accountability and advocacy work.
The program found the method especially useful for the following reasons:
The method shifts the focus of discussions from activities to changes. For example, instead of discussing the number of community score card processes completed and issues discussed, it puts the focus on discussing what actions the duty bearers took as a result.
It gives a new perspective on the achievements and challenges, allowing the users to tell the story about these. The teams are not constrained by pre-defined indicators but instead describe changes that are actually meaningful to them.
It supports the team members in recapitulating their achievements and celebrating even small successes, which can be motivating for the implementation teams. For example, in Malawi, the Ministry Director’s endorsement of the coalition’s advocacy efforts was harvested as an outcome and celebrated as one of the coalition’s achievements.
It guides the users to explicate and analyze the achieved changes in depth, rather than just mentioning them, in order to learn from change processes and to revise strategies.
It encourages discussion and learning from unintended—as well as intended—changes that the initiative has influenced. For example, in Kenya, the method encouraged the advocacy coalition to discuss an unintended outcome that the duty bearers resisted the initiative’s social accountability action. This has helped the coalition to analyze the duty bearer allies, structures, and lessons learned to adjust the strategies for their advocacy entry points.
It is participatory in nature, and it can strengthen connections within the teams as well as with other stakeholders in the process. In all of the program’s four countries, the advocacy coalitions were involved in harvesting the outcomes, which promoted the coalition members’ ownership of achievements, challenges, as well as strategies.
How can we apply Outcome Harvesting to steer social accountability and advocacy initiatives?
Outcome Harvesting can be applied as a monitoring and/or evaluation tool, depending on the desired use of the findings. This may also define the frequency (for example, single evaluation, regular yearly, or bi-annual monitoring) and the depth of the harvest.
A four-step methodology to structure a workshop employing this method is outlined below.
Step 1: Establishing the scope & definitions: actor-focused Theory of Change and what is an outcome
Given that Outcome Harvesting recognizes that most social accountability and advocacy initiatives operate in a complex environment, and that the initiative often has little control over the final change, actors’ behavior change is the most useful result to monitor. Therefore, the first step’s aim is to reflect on the Theory of Change and the actors whom the initiative targets, which in effect defines the scope of the outcome harvest.
The initiative should reflect on its pathway to change using three spheres: sphere of interest, sphere of influence, and sphere of control.
The overall goal and the foreseen impact [4] of the initiative on the lives of the communities or the final target groups are located in the sphere of interest.
The initiative should reflect and identify key actors who have control or influence over the initiative’s overall goale. The initiative has no control over these actors, but it has direct contact with them, and can therefore try to influence them. They are located in the sphere of influence. As the power to make changes to the initiative’s goal stays with these key actors, any changes in their behavior will be important for the initiative. Therefore, outcome harvesting monitors the key actors’ behavior changes.
In effect, outcomes are found at the level of the sphere of influence and are defined as observable changes in the behavior (actions, activities, relationships, policies, or practices) of individuals, groups, organizations, or institutions that are influenced in a small or large way, directly or indirectly, intentionally or not, positively or negatively, by the activities of the initiative.
The strategies and activities of the initiative fall into the sphere of control.
An actor-focused Theory of Change may be, but does not have to be, established using the Outcome Mapping method presented in a separate chapter.
Example of an actor-focused Theory of Change from the Strengthening Rural Governance for the Right to Adequate Food program:
Step 2: Tracing and grouping outcomes over the reviewed time period
The objective in this step is to identify the outcomes, or changes in behavior, in the actors that the initiative aims to influence.
This involves asking the participants an open question:
“What were the outcomes, meaning changes in behavior, in the actors that the initiative aims to influence, over the reviewed time period?”
The participants then have an open discussion about relevant key outcomes that they observed, which they write on Post-it notes and stick onto a whiteboard or flipchart.
The outcomes should be phrased as “person/ organization/ group/ department changed behavior in that…,” to ensure that they focus on the sphere of influence rather than activities of the initiative or changes in the context.
Active words should be used to describe the outcomes. For example, instead of saying “actively participated” or “collaborated,” the participants should describe the specific behavior such as “brought up issues xyz” or “signed an agreement” or “published.” The outcomes should not be value statements, but objective and observable claims backed up by factual information.
The outcomes may then be grouped according to the actor group or change area for further analysis. If numerous groups of outcomes are generated, the most important two to three may be prioritized.
Step 3: Elaborating outcome description, contribution, and significance
The workshop participants may be split into breakout groups, each working on a specific group of outcomes. In these, the participants elaborate the outcomes identified in Step 2 and analyze the contribution of the initiative and the significance of the outcome in achieving the overall goal of the initiative.
The outcome description should describe WHO changed behavior, WHAT changed, WHEN, WHERE, and HOW. The information should be in a sufficient level of detail to enable teams to reflect on the specific elements of the intervention that are of interest. For example, a specific actor, local governance structure, or committee involved in facilitating change may be clearly named in order to later facilitate reflection on its role and functioning.
The outcome description should avoid abbreviations and jargon. Instead it should be understandable to an outsider who is not familiar with the initiative or the topic. It should be described in full sentences rather than bullet points.
The outcome description should include any quantitative or qualitative data expressing the scale of the change. Any information about evidence related to the outcome can be attached (e.g., internal reports, links to publications, newspaper articles, photographs, emails, etc.).
Analysis of the initiative’s contribution to the outcome should address the following questions: How did the initiative contribute to this outcome? What were the key actions or activities the initiative engaged in with respect to this outcome? What makes us think that these actions or activities have contributed to the change happening? What were the most important contextual factors?
Analysis of the outcome significance should address questions, such as: How is the change in behavior/the outcome effective in addressing the issue/problem/goal of the initiative? What is the importance of the outcome in view of the objective or vision of the initiative? How does this outcome contribute to the progress markers that the initiative has planned for? Have we left out important aspects, people, or changes? Are we doing enough? What else should we do to progress further through the change pathway? What about viability of the outcome/behavior change? Sustainability? Is it locally owned or is the change dependent on the initiative? Who benefits from this outcome? Where, how many, and is it sufficient? Does the outcome require any follow-up from the initiative?
A good and detailed note-taker and a skilled facilitator should be allocated to each breakout group.
The outcome description may be compiled as a narrative describing several smaller intermediate outcomes leading to the more significant outcome in one story, along with the initiative’s contribution at different points in the change process.
Where appropriate, the teams also note down contextual events relevant to the outcome and available evidence to support their outcomes.
Step 4: Generating insights for future direction
In the same breakout groups or in the plenary (if sufficient time can be allocated), the participants reflect on their analysis of the contribution and significance of outcomes to discuss the root causes of successes and failures related to their assigned group of outcomes and draw conclusions about the initiative’s strategies or activities that can be maintained or that need to be adjusted or changed. Reflection is done by asking the questions:
“What went well and what should we maintain going forward? What were the challenges and what should we change?”
Besides reflecting on the strategies and activities, the participants should consider the outcomes and their significance, and include reflections on the specific elements in the intervention as well as other contextual factors that may have contributed to or hindered the outcome. It can be useful to refer back to the overall Theory of Change of the initiative to check if the initial assumptions about the intended effect of certain strategies are still valid or if adjustments are needed.
For example: Are there indications that the outcomes represent results that will be sustained beyond the duration of the initiative? If not, what does this mean for the initiative’s strategies? If certain contextual factors limit the scale of the outcome, how should the initiative respond in the future? If the initiative has learned something new about how the actors operate, what does this mean for the current planning? What enabled the initiative to seize a specific window of opportunity? Are there any implications for future strategy?
These insights can be gathered during an open discussion between the participants or by inviting every participant to individually note down their points on a card, which they then present one by one in the plenary.
Success factors
The Strengthening Rural Governance for the Right to Adequate Food multi-country program has determined the following success factors for using outcome harvesting:
Time allocation has proved to be one of the key factors that defines whether outcome harvesting can be employed in a meaningful way. As a minimum, even with prior preparation of outcome descriptions ahead of the workshop, one full day of workshop is recommended to allow for sufficiently deep and meaningful discussions. Without prior preparation, at least two workshop days should be planned.
Preparation to save workshop time can be done by splitting the workshop steps into two phases: (1) Steps 1 and 2 in one meeting, followed by individual work, bilateral discussions, and document review to prepare and consolidate the outcome descriptions, before (2) holding the workshop focused mainly on Steps 3 and 4.
Facilitation by a skilled and experienced facilitator can help probe with the right questions and ensure meaningful discussions. The overall lead facilitator should be trained or well familiarized with the method, he/she should be supported by skilled facilitators for the breakout groups.
The participants in the workshop should be those most knowledgeable about the outcomes and motivated to describe them. This is often the whole team of the initiative, project, or program including the field staff.
Outcome harvesting principles can be used to stimulate participation and views of the target groups (e.g., by asking the communities about the changes they observed, about the view of the community with respect to the contributions to those changes and their significance), and to feed the initiative’s harvest or to verify the accuracy and credibility of already harvested outcomes. This should not be narrowed down to mere consultation. Instead, the harvest results should be fed back to the target groups along with information on which suggestions from the field were or were not considered in decisions about the initiative’s strategy.
In order to analyze how the outcomes represent progress against the initiative’s proposal or its specific objectives/areas of change, the harvested outcomes could be aggregated based on types of change, their main characteristics, or other categories.
The Outcome Harvesting method is flexible, and it can be merged and/or complemented with other monitoring, evaluation, and learning methods. The users may decide to take a more rigid approach to its implementation [5], or only use elements or some of its principles in reflection and learning platforms.
Case Study
Social accountability action from the Strengthening Rural Governance for the Right to Adequate Food project in Malawi that used the Outcome Harvesting method
Notes:
[1] Ricardo Wilson-Grau and Heather Britt. Outcome Harvesting. (Ford Foundation (MENA Office), 2013).
[2] Wilson-Grau and Britt. Outcome Harvesting.
[3] Boru Douthwaite, John Mayne, Cynthia Mcdougall, and Rodrigo Paz-Ybarnegaray. “Evaluating Complex Interventions: A Theory Driven Realist-informed Approach.” Evaluation 23, No. 3 (July 2017): 295-296, https://doi.org/10.1177/1356389017714382.
[4] In contrast to rigorous impact evaluation, Outcome Harvesting does not aim to establish causal connections. It recognizes that in a complex setting, cause and effect cannot be fully understood and an initiative can only contribute to a change.
[5] Wilson-Grau and Britt. Outcome Harvesting.
About the Author:
Tereza Kaplan works at the intersection of human rights and public health, endeavoring to improve outcomes for vulnerable populations. With a background in law and public health, she has specialized in monitoring and evaluation of public service delivery and health outcomes. She started her career in the UK's public service before joining the non-governmental sector to operate in countries affected by emergencies and poverty. She is currently affiliated with Welthungerhilfe.
Jan Van Ongevalle leads the ‘global development’ research team at HIVA-KU Leuven in Belgium and has more than 25 years of experience as a development practitioner and researcher. Jan’s experience and interest is particularly situated around exploring participatory planning, monitoring and evaluation approaches that can foster learning and adaptive programming. His work is strongly influenced by the principles of collaborative action research.