I m p a c t
Studies: Room for Improvement?
Reaching the right
people: International research organizations have documented how their
work helps the poor, but are the results of these
impact studies making a difference?
The complex and costly nature of good impact assessment studies and the multiplicity of factors that determine their outcome were among the issues discussed during an international conference held in Costa Rica in 2002.
The conference, provocatively entitled, "Impacts of Agricultural Research and Development: Why Has Impact Assessment Research Not Made More of a Difference?", was hosted by the CGIAR's Standing Panel on Impact Assessment (SPIA) and the CIMMYT Economics Program. Leading experts reviewed impact assessment studies, communicated the pivotal role of research and development to policymakers, and shared best practices.
Do We Learn from
Our Mistakes?
"Experience suggests that one of the best ways to achieve food security, good environmental stewardship, and sustainable economic development is through the development and application of improved agricultural technologies," says Prabhu Pingali, director of the CIMMYT Economics Program at the time of conference. He notes that these improved technologies take a long time to develop, and their future availability depends largely on current investments in research.
"If current investments are to be effective, we have to understand the outcomes of past investments," says Pingali. "This is why our conference focused on ways to make impact assessment more understandable."
Participants developed principles and strategic guidelines for future impact studies. They examined the multiple purposes of impact assessment—accountability to donors, improving future research, resource mobilization, and public awareness. Mention was also made of the need for multidisciplinary studies that look at a range of impacts.
"People want information that guides them on design, program or project choices, on how to allocate resources across programs or projects, or to demonstrate at the completion of a project that resources were effectively used," says Alex McCalla, emeritus professor in the Department of Agricultural and Resource Economics at the University of California-Davis and Chair of CIMMYT's Board of Trustees.
McCalla points out that impact assessment has become more complicated because there are more players with more objectives, and they demand more sophisticated analysis. "Answering questions of impacts for these multiple players with multiple objectives, using complex conceptual models, has made impact assessment more costly," he says.
Communicating Impacts to
Funding Agencies and the Media
Representatives from funding agencies such as GTZ, the International Fund for Agricultural Development (IFAD), and the US Agency for International Development (USAID) outlined issues that concerned donors. They emphasized the need for more credible results, including a more balanced selection of case studies that examine research failures as well as successes.
A media panel included journalists from The Economist and The Hindu and Barbara Rose, executive director of Future Harvest at the time of conference. The panelists observed that journalists are interested in stories that are relevant to current problems such as environmental degradation, poverty, and global warming. Because myriad issues vie for journalists' attention, it is critical to target the right audience and media outlet for messages about research impacts.
Immediate Impacts
of the Conference
The conference exceeded expectations. "There appeared to be a real openness to rethinking how impact assessment is done at the Future Harvest Centers," remarks Rose. Archana Godbole from the Applied Environmental Research Foundation in India and Carmen Nieves Mortensen from the Institute of Seed Pathology in Denmark say that they learned a lot. "I have to admit I was surprised to see that even after four long days of meetings, the room was still packed," says SPIA chair Hans Gregersen.
Instead of issuing a traditional proceedings volume, conference organizers are assembling selected papers for publication in special issues of professional journals. "Special issues reach a much larger audience than proceedings," says Michael Morris, assistant director of the CIMMYT Economics Program. "They're externally reviewed and regarded as more substantial publications."
Special issues are currently being prepared for Agricultural Economics, Quarterly Journal of International Agriculture, and Agricultural Systems. Work has also begun on the development of a web site to promote best practices in impact assessment, disseminate results, foster dialogue between impact assessment practitioners, and demonstrate organizational learning.
For more information:
Michael Morris
(m.morris@cgiar.org)
August, 2004