Operational guidelines for assessing the impact of agricultural
research on livelihoods
Good practices from CIMMYT
La Rovere and Dixon, CIMMYT, 2007
Back to Contents
Agricultural research for development is based on the hypothesis that knowledge can be used to raise poor people’s level of well being by producing more food at lower cost, reducing risks and providing options for them to choose from. Research is therefore part of a means-end hierarchy that usually comprises a chain of interlinked objectives. The complete chain of objectives that links inputs to activities, activities to outputs, outputs to outcomes, and outcomes to impact is a logic model. Reviewing the logic model for a research project or program is a crucial component of planning an impact study. Logic models, or program theory in the evaluation language, are used in IA and in the evaluation of research programs and projects. A well-articulated logic model is needed for IA, thus researchers need to review, update or refine logic models in the process of IA. Some generic logic models, their main strengths and weaknesses, are described below. There are many names, definitions, and uses of logic models, for example the logical framework (Baur 1998), impact pathways (Springer-Heinze et al., 2003; Douthwaite et al., 2003) or program theory (e.g., Patton 1995). The most important aspect of logic models is that they provide a systematic articulation of what a program or project intends to achieve and how. Logic models are chains of hypotheses. The elements of the chain are connected by assumed causal links. Evaluation and IA ask whether the assumed causal link between the elements in the chain does or does not exist. A logic model may be implicit. Especially in cases where the planning function is weak or under-resourced, the assumed causal links between research outputs and outcomes may not be well articulated. The hypotheses are thus only partially known and cannot be scrutinized and improved. Logic models are essential for building hypotheses and testing causalities. They encourage impact assessors to be more precise in developing a more scientific study design. An IA, however, is usually different in its level of ambition. While science is about proof, falsification and definitive conclusions, the aim of IA at the level of peoples’ livelihoods has to be more modest. Stakeholders usually expect from an IA some plausible estimates of the likelihood that particular research activities have contributed in concrete ways to improved livelihoods, which is different from definitive proof of impact. Different approaches have been suggested to address this trade-off between seeking proof and the practical need of investors for information that will help them decide what to do next.
Contribution analysis (Mayne 1999) has been suggested as a way to deal with accountability in the move towards results-based management. It usually relies on performance management data and addresses the question of what contribution a program has made to a development impact. It answers the question of how much success or failure can be attributed to a program. Contribution analysis of an agricultural research program would aim to reduce uncertainty about the contribution made, not provide proof.
Outcome Mapping (Smutylo and Carden, IDRC) is a process focusing on outcomes. Outcomes are defined as changes in the behavior and activities of people, groups, and organizations with whom a program works and allows for a realistic evaluation. Other examples of logic models for different types of research are applied models (applicable to crop breeding), adaptive models (for example participatory research, used in CIMMYT’s participatory work in Oaxaca, Mexico; Smale et al., 2003; La Rovere et al., forthcoming) and livelihood frameworks.
| The results-based monitoring (GTZ 2004) approach of GTZ focuses on impacts rather than outputs. It captures the social and environmental changes required for innovation that development cooperation seeks to bring. GTZ requires project managers to monitor changes in the context of a program and identify whether such changes are linked to the project. The mere occurrence of a change is not sufficient to merit its designation as a project or program result, there must be a causal or plausible link between observed change and outputs. The difficulty of attributing change in the wider context of project interventions (the attribution gap) is recognized. Attribution is addressed in the contract and cooperation agreement by stipulating monitoring requirements for objectives up to and beyond the outcomes. Up to the outcome, managers are required to monitor outputs, use of outputs, and external factors that facilitate or constrain the use of outputs, and to demonstrate causal links between the desired changes and project outputs. Beyond the outcomes, they need to monitor changes in the wider context of research and establish what contribution the project may have made to observed changes. The contribution beyond outcome must be plausibly argued, not causally demonstrated. |
| Back to Contents |  |
Previous Annex | |
Next Annex |
