Operational guidelines for assessing the impact of
agricultural research on livelihoods

Good practices from CIMMYT


Coordinated by: Roberto La Rovere, John Dixon

Impacts Targeting and Assessment (ITA) Unit
CIMMYT, Int., El Batan, Mexico

 

Click here to download entire manual in PDF (693kb)

Preface

This manual responds to the need of CIMMYT scientists and field partners for guidance on impact assessment (IA) . It has been developed through a two-year process involving colleagues at CIMMYT and various stakeholders of CIMMYT work.

The guidelines naturally draw on many sources of information on IA, and condense and enhance what is known about IA for hands-on users: researchers and managers of crop improvement projects and their NARS partners interested in IA, and social scientists who are not expert in IA.

This manual will initially be published on-line so that it can be easily upgraded and linked to other sources. Users will be able to provide feedback, upgrade and enrich the content, and add details to IA case studies.

The aim of this document is to help ensure quality in IA, institutionalize good IA practices, provide a resource list of approaches, tools, and suggestions, and give examples of how CIMMYT does good IA with partners in diverse places and conditions.

The manual often refers to complementary documents, such as the “Strategic Guidelines” that are forthcoming from the Standing Panel on IA (SPIA) of the CGIAR, and draws from many sources.

Please inform us of any instances where we have neglected to cite original sources, so we can revise the manual accordingly.

Contents
1. Introduction
  The need for guidelines for assessing impacts on livelihoods
  Applications of the guidelines
    Definition of impact and impact assessment
  Overview of key operational concepts for Impact Assessment
    Livelihoods
    Poverty
    Other key concepts in impact assessment
  Links between impact assessment, priority setting and targeting
  External demand for IA
    Investor demands
    Partner demands
  Internal demand from CIMMYT
  Capacity for IA
  IA approaches to date: strengths and weaknesses
  Impact assessment quality standards
2.Good practices in conducting impact assessment
  2.1 Good practices in designing an impact assessment
    (I) Clarify the purpose, context, scope, and limitations of the IA
      What is an IA?
      What types of impact need to be assessed?
      Why do I need an IA? How can we use the outputs of an IA?
      Is IA good for my work? What if the IA shows little or no impact?
      Who do I need to develop an IA?
      Who does IA in and outside the CGIAR?
      How do we get rapid and cost effective IA?
      What are the risks in doing an IA?
      How can the outputs of an IA study be (made more) credible?
    (II) Focus on the key questions and hypotheses
    (III) Towards implementation: ensuring partners’ involvement, and planning for learning and communicating the results
    Responsibilities and roles
  2.2 Good practices in implementing an impact assessment
    (IV) Select/develop the analytical instruments
    Livelihood IA indicators
    Good practice for choosing an assessment method
      A: Qualitative methods
      B: Quantitative methods
      C: Integrating Quantitative and Qualitative Methods
    (V) Describe the impact pathway of the program/project
    (VI) Acquire and manage the data and information
    (VII) Analyze and validate impacts, and interpret the findings
    (VIII) Report, disseminate, communicate externally and internally
    (IX) Evaluate the assessment, reflect and learn internally
3. Incorporating IA into projects and institutions
  Training in IA
  Writing IA into projects, and developing a budget
  Institutionalizing impact assessment
   
Annexes  
Annex 1 Glossary of key terms and concepts
Annex 2 Logic models
Annex 3 The extra need to learn and change
Annex 4 Modified standards of the Africa Evaluation Association
Annex 5 Selected IA resources
Annex 6 Selected impact Studies of CIMMYT
Annex 7 Key web resources on IA


Structure of the guidelines

This document contains step-by-step guidelines for IA and presents procedures, methods, and options to help users develop appropriate IA for projects or studies. The document provides:

  • A selection of IA good practices, tools, and methods with examples, in the form of Boxes, often with a brief description of their main strengths and weaknesses.
  • Suggestions on minimum datasets from which to select indicators of impact on livelihoods.
  • Key approaches for assessing impacts of agricultural technologies on farmers’ livelihoods.
  • A checklist for designing an IA project or identifying projects that would benefit from an IA component.
  • References from the international (Annex 5) and CIMMYT (Annex 6) literature; and web links to resources and organizations (Annex 7) from CIMMYT’s experience.

 

Top

January, 2008