CIMMYT. Operational guidelines for assessing the impact of agricultural research livelihoods. Good practices from CIMMYT. Annex 5

Annex 5. Selected IA resources

Adato, M., Meinzen-Dick, R., 2002. Assessing the Impact of Agricultural Research on Poverty Using the Sustainable Livelihoods Framework: Concepts and Methods. EPTD discussion paper 89. IFPRI, Washington, D.C., USA. www.ifpri.org/divs/eptd/dp/papers/eptdp89.pdf

Adato M., Meinzen-Dick R. (eds.) 2007. Agricultural Research, Livelihoods, and Poverty Studies of Economic and Social Impacts in Six Countries. Published for IFPRI by Johns Hopkins University Press., 416 pages (ISBN 978-0-8018-8596-9)

Alston, J.M., Pardey, P.G., 2001. Attribution and other problems in assessing the returns to agricultural R&D, Agricultural Economics 25 141-152

Alston, J. M., Norton, G.W., Pardey,P.G. 1995. Science under Scarcity, ISNAR, Cornell University Press

Alwang, J., Siegel, P.B., 2003. Measuring the impacts of agricultural research on poverty reduction, Agricultural Economics 29 1-14

Antle, J., Stoorvogel, J., Bowen, W., Crissman, C. Yanggen, D., Making an Impact with IA: The Tradeoff Analysis Approach and Lessons from the Tradeoff Analysis Projects in Ecuador and Peru, Paper, 2002, International Conference on the Impacts of Research and Development.

Antle, J.M., Stoorvogel, J.J. 2006. Incorporating Systems Dynamics and Spatial Heterogeneity in Integrated Assessment of Agricultural Production Systems. Environment and Development Economics 11(1):39-58

Ashley, C., Hussein, K., 2000. Developing Methodologies for Livelihood IA: Experience of the African Wildlife Foundation in East Africa, ODI Working Paper 129. www.livelihoods.org

Baker, J.L., 2000. Evaluating the Impact of Development Projects on Poverty: A Handbook for Practitioners, The World Bank. siteresources.worldbank.org/INTISPMA/Resources/handbook.pdf

Baur, H., Bosch, M., Krall, S., Kuby, T., Lobb-Rabe, A., Schutz, P-T, Springer-Heinze, A., 2003. Establishing Plausibility in Impact Assessment, The Program Evaluation Standards. www.tropentag.de/2003/abstracts/full/431 Planck Forum 5, Innovative Structures in Basic Decision Research. Ringberg Symposium, Munich, 173-179. www.iew.uzh.ch/wp/iewwp302.pdf

Brown, D., 2000. Indicators of Sustainable Livelihoods, Livelihoods Connect/DFID

Casley, D.J., Lury, A.A. 1982. Monitoring and Evaluation of Agriculture and Rural Development.

Bellamy, M., 2000. Approaches to impact evaluation (assessment) in agricultural information management: Selective review of the issues, the relevant literature and some illustrative case studies, Paper, 2000, CTA Working Document no. 8021. cta.trafika.co.uk/data/pdfs/8021.pdf

Brook, R. 2002. The Role of Evaluation as a Tool for Innovation in Research. Proc. 2000 Max Projects. The World Bank and the Johns Hopkins University Press, Baltimore & London

CGIAR Secretariat, 2003. Impact: everybody's business, Annual Report.

CGIAR, 2004. Nourishing the future through scientific excellence. Snapshot of CGIAR Impacts

CGIAR Science Council, Inaugural Meeting of Science Council, ICARDA, Aleppo, Syria, 12-15 May 2004. Report of Science Council’s Standing Panel on Impact Assessment (SPIA), April.

Cromwell, E., Kambewa, P., Mwanza, R., Chirwa R. KWERA Development Centre, 2001. Impact assessment using participatory approaches: ‘starter pack’ and sustainable agriculture in Malawi. AGREN Network Paper no. 112. www.odi.org.uk/agren/papers/agrenpaper_112.pdf

DCD/DAC, 1998. Methodological Note, DCD/DAC(98)6/ADD

DFID. Impact of Agricultural Research on Poverty (IARP) Mexico, Zimbabwe, Bangladesh, Kenya, Livelihoods Connect/DFID

Dixon, J., Gulliver, A. Gibbon, D. 2001. Farming Systems and Poverty: Improving farmers livelihoods in a changing world. FAO & The World Bank, Rome, Italy & Washington, DC, USA.

Douthwaite, B., Kuby, T., van de Fliert, E., Schulz, S. 2003. Impact pathway evaluation: an approach for achieving and attributing impact in complex systems. Agricultural Systems 78 (2) pp 243-265

ECART/ASARECA/CTA Workshop, 1999. On IA of Agricultural Research in Eastern and Central America, Executive Summary/Draft, ECART/ASARECA/CTA Workshop

Asian Development Bank. 2006. Enhancing maize productivity in drought-prone environments in East and Southeast Asia, Report, RETA no. 6208. Workshop: Greater impact through participatory approaches to variety development and delivery. www.adb.org/Documents/TARs/REG/38525-REG-TAR.pdf

Evenson, R.E., Gollin, D. 2003. Assessing the Impact of the Green Revolution, 1960 to 2000, Science (Vol 300)

Feldman, M.P. Impact Assessment Studies conducted in the CGIAR, 1970-1999: An Annotated Bibliography, CIMMYT

Fujisaka, S., White, D. 2004. Ex post methods to measure Natural Resource Management Research Impacts. www.icarda.org/INRMsite/documents/others/expost.pdf

GRDC, RIRDC, 1997. Guidelines for Economic Evaluation of R&D, Centre for International Economics, Canberra

GTZ, 2004. Results based Monitoring/Guidelines for Technical Cooperation Projects and Programmes, GTZ Unit 04 Corporate Development www.gtz.de

Guijt I., Woodhill, J. 2000. A guide to project M&E - Managing for Impact. IFAD, Rome, Italy. www.ifad.org/evaluation/guide

Guijt, I. Gaventa J. 1998. Participatory Monitoring and Evaluation. Institute of Development Studies, University of Sussex, Brighton, U.K. www.ids.ac.uk/ids/bookshop/briefs/PB12.pdf

Gupta Kappor, A. 2002. Review of Impact Evaluation methodologies used by the Operations Evaluation Department, The Work Bank Operations Evaluation Department

Hall, A., Sulaiman, V.R., Clark, N., Yoganand B. 2003. From measuring impact to learning institutional lessons: an innovations systems perspective on improving the management of international agricultural research. Agricultural Systems 78(2): 213-242.

Hazell, P.B.R. 1999. The impact of Agricultural Research on the Poor: A Review of the State of Knowledge.
ciat-library.ciat.cgiar.org/paper_pobreza/041.pdf

Henry, G.T., Mark M.M.. 2003. Beyond Use: Understanding Evaluation’s Influence on Attitudes and Actions. American Journal of Evaluation. 24(3): 293-314.

Horton, D., Mackay, R., 2003. Special Issue Learning for the Future: Innovative Approaches for Evaluating Agricultural Research and Development, Agricultural Systems 78:119-336

Horton, D., Mackay, R. 2003. Using evaluation to enhance institutional learning and change: recent experiences with agricultural research and development. Agricultural Systems 78, 127-142. evi.sagepub.com/cgi/content/abstract/8/1/13

IAEG, 1999. Impact Assessment of Agricultural Research: Context and State of the Art. Paper presented by the IAEG for the ASARECA/ECART/CTA Workshop on IA of Agricultural Research in Eastern and Central Africa (Uganda, November 1999).

IAEG, 2000. Impact assessment of agricultural research: Context and state of the art, FAO. impact.cgiar.org/methods/docs/sofart.pdf

IAEG, 1997. Status of impact assessment and evaluation in the CGIAR: Proceedings of CGIAR Impact Assessment and Evaluation Workshop held at ISNAR, The Hague 15-17 April 1996.

IFPRI, 2002. Impact Evaluation. Assessing the Impact of Policy-Oriented Social Science Research, Newsletter, IFPRI. www.ifpri.org/pubs/ib/ib5.pdf

Johnson, N., Lilja, N., Ashby, J.A. 2002. Measuring the Impact of User Participation in Natural Resource Management Research, Paper presented at the International Conference on the Impacts of Research and Development.

Kelley, T.,Gregersen, H. 2003. NRM impact assessment in the CGIAR: meeting the challenge and implications for ICRISAT. In Shiferaw, B., Freeman, H.A. (eds). Methods for assessing the impacts of natural resource management research. A summary of the proceedings of an International Workshop, 6-7 December 2002, International Crops Research Institute for the Semi-Arid Tropics (ICRISAT), Patancheru 502 324, India.

Kerr, J., Kolavalli, S. 1999. Impact of Agricultural Research on Poverty Alleviation: Conceptual Framework with Illustrations from the Literature. EPTD Discussion Paper No. 51. Washington D.C., IFPRI. www.ifpri.org/divs/eptd/dp/papers/eptdp56.pdf

Kristjanson, P.M., Thornton, P.K.. 2002. Herding cats: Is impact assessment the ultimate exercise in futility? Paper for the International Conference on the Impacts of Agricultural Research and Development; “Why Has Impact Assessment Research Not Made More of a Difference? San Jose, Costa Rica, 4-7, February 2002.

Kuby, T., 1999. Innovation is a social process: What does this mean for IA in Agricultural Research? p. 63 unesdoc.unesco.org/images/0013/001354/135470e.pdf

Lariviere, S., Martin, F., Mellali, S., Sanogo, D., 2003. Proposal of a Framework for Monitoring and Evaluating Progress Towards the Millennium Development Goals (MDGs). www.undp.org/surf-wa/mdgforum/documents/forumpres/idea.pdf

Lele, U. Ekboir, J., Technology Generation, Adaptation, Adoption and Impact: Towards a Framework for Understanding and Increasing Research Impact. siteresources.worldbank.org/INTARD/825826-1111405593654/20432115/TGAAI.pdf

LFP: Livelihoods and Forestry Program, 2004. Methodologies for Conducting a Livelihoods Baseline Study, Livelihoods and Forestry Programme, Nepal. www.livelihoods.org/lessons/project_summaries/docs/LFP%20Report_Methodology_%20baseline.pdf

Lipton, M, 2002. Impact of Agricultural Research on Poverty Reduction: Messages from the San José Workshop. Paper for the International Conference on the Impacts of Agricultural Research and Development; “Why Has Impact Assessment Research Not Made More of a Difference? San Jose, Costa Rica, 4-7, February 2002.

Low, A., S.R. Waddington, E.M. Shumba. 1991. "On-Farm Research in Southern Africa: The Prospects for Achieving Greater Impact.” In Planned Change in Farming Systems, R. Tripp (ed.). U.K.: Wiley-Sayce.

Low, A. 1988. Farm Household-Economics and the Design and Impact of Biological Research in Southern Africa. Agric. Admin. & Extension 29: 23-34.

Mackay, R., Horton, D. 2003. Expanding the use of impact assessment and evaluation in agricultural research and development. Agricultural Systems 78: 143-165

Maredia, M., Pingali, P. 2001. Environmental Impacts of Productivity-Enhancing Crop Research: A Critical Review, TAC Secretariat/FAO.

Maredia M., Byerlee D., Anderson J. 2000. Ex post evaluation of economic impacts of agricultural research programs: A tour of good practice. In workshop on “The future of impact assessment in CGIAR: Needs, constraints, and options,” Standing Panel on Impact Assessment (SPIA) of Technical Advisory Committee, May 3-5, Rome. impact.cgiar.org/methods/docs/maredia.pdf

Marsland, N., Wilson, I., Abeyasekera, S., Kleih, U. 2001. Combining Quantitative (Formal) and Qualitative (Informal) Survey Methods/Socio-Economic Methodologies for Natural Resources Research Best Practice Guidelines, Natural Resources Institute - UK/DFID

Masters, W., Coulibaly, B., Sanogo, D., Sidibé, M., Williams, A. 1996. The Economic Impact of Agricultural Research: A Practical Guide. West Lafayette, Indiana, USA: Department of Agricultural Economics Purdue University. p. 45. www.agecon.purdue.edu/staff/masters/ImpactCD/Manuel/EconSurplusManual-English.pdf

Mathur, S., Gaiha. 2003, R. Impact of agricultural research on rural poverty: a review of some analytical issues. In S. Mathur and D. Pachico ( eds) Agricultural research and poverty reduction: some issues and evidence, Economics and Impact Series 2, Cali, Colombia: International Center for Tropical Agriculture (CIAT).

Matlon, P.J., 2003. Editorial Foreword. Agricultural Systems 78, 123–125.

Mayne, J, 1999. Addressing Attribution Through Contribution Analysis: Using Performance Measures Sensibly, Discussion Paper Office of the Auditor General of Canada.

McQueen, D. 2001. Measurement malaise - is the SL approach inoperable?

Meizen-Dick, R., Adato, M., Haddad, L., Hazell, P. 2003. Impacts of Agricultural Research on Poverty: Findings of an Integrated Economic and Social Analysis, Paper, EPTD Discussion Paper No. 111. FCND Discussion Paper No. 164.

Miles, M.B., Huberman, A.M. 1994. Qualitative Data Analysis: An Expanded Sourcebook, 2nd Edition, Sage Publications A Regional Training Workshop on Monitoring, Evaluation, and IA of R&D Investments in Agriculture.

Mustafa, Y., Grando, S., Ceccarelli, S. 2006. Assessing the benefits and costs of participatory and conventional barley breeding programs in Syria. ICARDA, Aleppo, Syria. www.icarda.org/Participatory_Research/PDF/Cost-Benefit-of-PPB.pdf

Nairobi M&E Network, 2002. The African Evaluation Guidelines: 2002. A checklist to assist in planning evaluations, negotiating clear contracts, reviewing progress and ensuring adequate completion of an evaluation, 2002, Evaluation and Program Planning 25 (2002) 481-492
elsevier.lib.tsinghua.edu.cn/cgi-bin/sciserv.pl?collection=journals&journal=01497189&issue=v25i0004

Nelson, M., Maredia, M., 1999. Environmental Impacts of the CGIAR: An initial assessment. TAC Secretariat. FAO, 71 pp. wbln0018.worldbank.org/Apps/CGIAR/ic_cgiar.nsf/4F1B144AAA9938338525664C00017FDB/147803CAA97C8E1A8525690A0057B58B/$FILE/acenviro.pdf

Nutley, S., Davies, H., Walter, I., Wilkinson, J., 2004. Developing Projects to Assess Research Impact, Report of Seminar, University of St Andrews.

OECD, 2002. Glossary of Key Terms in Evaluation and Results Based Management. Evaluation and Aid Effectiveness 6, OECD www.oecd.org/dataoecd/29/21/2754804.pdf

Özgediz, S. 1995. Strengthening Evaluation in the CGIAR: Needs and Options. CGIAR Task Force on Impact Assessment. Washington, DC: CGIAR Secretariat. impact.cgiar.org/methods/docs/ozgediz95.pdf

Kristjanson P., Place F., Franzel, S., Thornton, P.K. 2002. Assessing research impact on poverty: the importance of farmers' perspectives. Agricultural Systems, Volume 72, Issue 1, pp. 73-92

Pannell, D.J., 1999. On the estimation of on-farm benefits of agricultural research, Agricultural Systems 61: 123-134

Pasteur, K., 2001. Tools for Sustainable Livelihoods: Livelihoods Monitoring and Evaluation, IDS www.livelihoods.org/info/tools/pas-PP01.rtf

Patton, M.Q. 1995. Utilization-Focused Evaluation: The New Century Text. Sage Publications

Perrin, B. 2002. How to - and how not to - evaluate research. Evaluation, Vol. 8, No. 1, 13-28

Perrin, B. 2002. ‘Towards a New View of Accountability’. Paper presented at the Conference of the European Evaluation Society ‘Three Movements in Contemporary Evaluation: Learning, Theory and Evidence’, 10–12 Oct., Seville. www.europeanevaluation.org/

Perrin, B., 2001. Commentary: Making Yourself - and Evaluation - Useful, American Journal of Evaluation, 22(2).
library.cimmyt.org/download/reprints/Commentary_makingyourself_Perrin.PDF

Place, F., Adato, M., Hebinck, P., Omosa, M., 2005. The Impact of agro forestry-based soil fertility replenishment practices on the poor in Western Kenya, Research Report 142. www.ifpri.org/pubs/abstract/142/rr142.pdf

Raitzer D., Winkel K. 2005. Donor demands and uses for evidence of research impact - the case of the Consultative Group on Agricultural Research (CGIAR). Rome: Science Council Secretariat, FAO. www.sciencecouncil.cgiar.org/activities/spia/pubs/Donor%20Demands.pdf

Raitzer, D.A., Lindner, R. 2005. Benefit-Cost Meta-Analysis of Bilateral Investment by the Autralian Centre for International Agricultural Research.

Raitzer, D.A., 2003 Benefit-Cost Meta-Analysis of Investment in the International Agricultural Research Centers of the CGIAR, Paper, Science Council Secretariat/FAO www.sciencecouncil.cgiar.org/activities/spia/pubs/0303-1.pdf

Raitzer, D.A., 2003. Benefit from the CGIAR Have Been Nine Times Greater Than Costs, CGIAR Impact Highlight no. 1

Ravallion, M. 2001. The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation, World Bank Economic Review, Vol. 15, no. I, 115-140.

Renkow, M. 1994. "Technology, Production Environment, and Household Income: Assessing the Regional Impacts of Technological Change." Agricultural Economics 10: 219-231.

Rosegrant, M.W., Meijer, S., Cline, S.A. 2002. International Model for Policy Analysis of Agricultural Commodities and Trade (IMPACT): Model Description, Paper, IFPRI. www.ifpri.org/themes/impact/impactwater.pdf

Ryan, J.G., Garrett, J.L. 2003. The Impact of Economic Policy Research: Lessons on Attribution and Evaluation from IFPRI, IA Discussion Paper no. 20/IFPRI. www.ifpri.org/impact/iadp20.pdf

Ryan, J.G. 2001. Synthesis Report of Workshop on Assessing the Impact of Policy-Oriented Social Science Research in Scheveningen, The Netherlands, IA Discussion Paper no. 15, IFPRI. www.ifpri.org/impact/iadp15.pdf

Salter, A., Martin, B.R., 2001. The economic benefits of publicly funded basic research: a critical review, Research Policy 30: 509-532

Science Council Secretariat FAO, 2000. An Evaluation of the Impact of Integrated Pest Management Research at International Agricultural Research Centers. www.sciencecouncil.cgiar.org/activities/spia/pubs/ipm.pdf

Science Council Secretariat FAO, 2005. Monitoring and Evaluation System for CGIAR Centers.

Science Council Secretariat FAO 2004. Report to the Annual General Meeting.

Scott, C. 1985. Sampling for monitoring and evaluation. Washington DC: The World Bank.

Shiferaw, B, Freeman, H.A., Swinton, S.M. (eds) 2004. Natural Resource Management (NRM) in Agriculture: Methods for assessing economic and environmental impacts. CABI

Springer-Heinze, A., Hartwich, F., Henderson, J.S., Horton, D., Minde, I. 2003. Impact pathway analysis: an approach to strengthening the impact orientation of agricultural research. Agricultural Systems 78: 267-285.

Sustainable Livelihoods Guidance Sheets.
www.livelihoods.org/info/info_guidancesheets.html; www.nssd.net/references/SustLiveli/DFIDapproach.htm#Guidance

TAC Secretariat FAO, 2001. The future of IA in the CGIAR: needs, constraints and options. TAC Secretariat FAO

Task Force on Impact Assessment and Evaluation, European Initiative for Agricultural Research for Development (EIARD). 2003. Impact assessment and evaluation in agricultural research for development. Agricultural Systems 78 (2003) 329–336.

Tassey, G., 1999. Lessons learned about the methodology of economic impact studies: the NIST experience, Evaluation and Program Planning 22 (1999) 113-119.

Thiele, G., Devaux, A., Velasco, C., Manrique, K. 2006. Horizontal evaluation: Stimulating social learning among peers, ILAC brief no. 13

Ticehurst, D. Cameron, C. 2000. Performance and IA: Emerging Systems among Development Agencies, Policy Series 8 Natural Resources Institute, University of Greenwich

Turton, C., 2001. Livelihood monitoring and evaluation: iImproving the impact and relevance of development interventions, IDL. www.livelihoods.org/post/Docs/tur-ME01.rtf

Virtanen, P., Uusikyla, P. 2004. Exploring the missing links between cause and effect: a conceptual framework for understanding micro-macro conversions in programme evaluation, Evaluation Vol. 10(1):77-91.

Waibel, H. 2000. Evaluation of the Impact of Integrated Pest Management Research at the International Agricultural Research Centres. 99 pp. TAC Secretariat. Rome, Italy: FAO. www.sciencecouncil.cgiar.org/activities/spia/pubs/ipm.pdf

Watson, D.J. 2003. International Conference on Impacts of Agricultural Research and Development: Why Has Impact Assessment Research Not Made More of a Difference? Proceedings of a conference organized by the SPIA and CIMMYT, 4-7 February 2002 San Jose, Costa Rica. 107 pp. Mexico: CIMMYT. www.cimmyt.org/english/docs/tech_pubs/impact_proceedings.pdf

Watts, J. 2005. Learning-oriented evaluation: a tool for promoting institutional learning and program improvement. ILAC Brief no. 3. www.cgiar-ilac.org/downloads/Briefs/Brief3Proof2.pdf

Watts, J., Horton, D., Douthwaite, B., La Rovere, R., Thiele, G., Prasad, S., Staver, C. 2008. Transforming Impact Assessment: beginning the quiet revolution of Institutional Learning And Change. Experimental Agriculture, Volume 44, Issue 01, January 2008, pp 21-35

Yanggen, D. Antle, J., Stoorvogel, J., Bowen, W., Crissman, C. 2002. Tradeoff Analysis as a Tool for Integrated Assessment of Economic and Environmental Impacts of Agricultural Research, Paper presented at the International Conference on the Impacts of Research and Development. www.tradeoffs.montana.edu/pdf/TOAYanggen.pdf

Top

Back to Contents

Previous Annex

Next Annex