CIMMYT. Operational guidelines for assessing the impact of agricultural research livelihoods. Good practices from CIMMYT. Annex 4

Annex 4. Modified standards of the Africa Evaluation Association

Utility: to ensure that IA will serve the needs of intended users and be owned by stakeholders

- U1 Stakeholder identification: persons or organizations involved in or affected by the IA (and the beneficiaries) should be included in the IA process, so that their needs can be addressed and IA findings can be usable and owned by the stakeholders.

- U2 Assessor’s credibility: the persons conducting the IA should be trustworthy and competent to perform the IA, so that the findings are credible and accepted.

- U3 Information scope and selection: information collected should address pertinent questions and be responsive to the needs and interests of clients and other stakeholders.

- U4 Values identification: describe the perspectives, procedures, and rationale used to interpret the findings so as to clarify the bases for judgments. Multiple interpretations should be preserved, if these respond to stakeholders’ concerns and needs for utilization.

- U5 Report clarity: IA reports should clearly describe what is being assessed, - the context, purposes, procedures, and findings of the IA - so that these are understood.

- U6 Report timeliness and dissemination: interim findings and IA reports should be shared with the intended users to the extent that this is useful, feasible and allowed. Feedback of intended users on interim findings should be taken into consideration before the final report.

- U7 IA (evaluation) impact: IAs should be planned, conducted, and reported in ways that encourage follow up by stakeholders, in order to increase the chance that the IA will be used.

Feasibility: intended to ensure that an IA is realistic and in the overall feasible.

- F1 Procedures: IA procedures should be practical, to reduce disruption while gathering data.

- F2 Political viability: the IA should be planned and done by anticipating the position of various interest groups, so that their cooperation may be obtained, and so that possible attempts by any of these groups to curtail the procedures, or bias or misapply the results can be averted or counteracted as much as possible in the given context and situation.

- F3 Cost effectiveness: the IA should be efficient and produce information of sufficient value, so that costs are justified. It should stay within its budget and account for its costs.

Propriety: an IA should be conducted legally, ethically, and with due regard for the welfare of those involved in it and affected by its results.

- P1 Service orientation: IA should be designed to assist organizations to address and effectively serve the needs of the full range of targeted participants.

- P2 Formal agreements: the obligations of formal parties in an IA (who does what, how and when) should be agreed through dialogue and in writing, as appropriate, so to have a common understanding of the agreement and be in a position to formally renegotiate it, if needed. Attention should be paid to informal and implicit expectations by all parties.

- P3 Rights of human participants: IA should be designed and conducted in such as manner as to respect and protect the rights and welfare of the subjects and communities of which they are part.

- P4 Human interaction: assessors should respect human dignity in dealing with those involved in the IA and not harm participants or compromise their culture or religion.

- P5 Complete and fair assessment: IA should be fair in examining strengths/weaknesses of what is assessed, so that strengths can be built upon and problems addressed.

- P6 Disclosure of findings: formal parties to IAs should ensure that all findings and limitations are made accessible to the subjects of the IA. The assessors will determine what can ensure that confidentiality is respected, and that the assessors are not exposed to potential harm.

- P7 Conflict of interest: conflict of interest should be dealt with openly and transparently so as not to compromise the reliability and credibility of the process and the IA results.

- P8 Fiscal responsibility: the assessor’s allocation and expenditure of resources should be appropriate, reflect accountability procedures, be prudent, frugal, and ethically responsible.

Accuracy: intended to ensure that an IA will reveal and convey technically adequate information about the features that determine the worth of merit of what is being assessed.

- A1 Program documentation: the objective of the IA should be described accurately, so that the program is clearly identified, with attention to all forms of communications.

- A2 Context analysis: the context in which the program exists should be examined in detail (including political, social, cultural, ecological aspects) so that its influence can be assessed.

- A3 Described purposes and procedures: purposes and procedures of the IA should be monitored and described in enough detail, so that they can be identified and assessed.

- A4 Defensible information sources: information sources should be described, so that their adequacy can be assessed without compromising anonymity or cultural sensitivity

- A5 Valid information: information gathering procedures should be chosen and implemented to assure that the implementation is valid for the intended use. Information likely to be susceptible to bias should be checked with various methods and sources.

- A6 Reliable information: the information gathering process should be developed so that the collectors will assure that the information obtained is sufficiently reliable for the intended use.

- A7 Systematic information: information collected, processed, and reported in an IA should be systematically reviewed and any errors found should be corrected.

- A8/9 Analysis of information: quantitative and qualitative IA information should be appropriately and systematically analyzed so that IA questions are effectively answered.

- A10 Conclusions: the conclusions of IA should be justified, for stakeholders to assess them.

- A11 Impartial reporting: reporting should guard against distortion caused by the personal feelings and biases of any party to the IA, so that IA reports fairly reflect the IA findings.

- A12 Meta-evaluation: the IA should be formatively and summatively evaluated so that, on completion, stakeholders can assess its validity, usefulness and relevance.

Top

Back to Contents

Previous Annex

Next Annex