This Horticulture Innovation Lab webinar outlines fundamental concepts and best practices for monitoring and evaluation. Topics include defining monitoring, evaluation, and learning and introduces monitoring evaluation tools such as theory of change, results framework, performance indicators, and different types of evaluations. This was the seventh webinar in the Horticulture for Development Professional Series, held live on September 5, 2019.
Molly Chen, with RTI International, was the speaker for this webinar. She provided an overview of what monitoring and evaluation entails and how to set projects up for success. This includes how to design indicators and how performance can be evaluated.
This summary includes the recorded webinar video presentation, the presenter's slides, and additional resources referenced in the presentation.
What is MEL and why it is important?
Monitoring, evaluating, and learning (MEL) is a systematic approach to measuring project performance, successes and challenges, to aid in decision making. International development projects can be complex and have numerous goals or objectives over a long period of time. MEL provides a methodical process enabling development professionals to track their performance and incorporate learning into their implementation as changes occur.
At its core, MEL is important in establishing good project management. The process can aid project managers and other development professionals to track performance against planned activities, establish fidelity of implementation across projects and locations, help to establish effectiveness in achieving results, and allow management to identify areas for adjustment. When working with various stakeholders, accountability is an important element to maintain throughout a project's lifecycle. Data gleaned from MEL can help contribute to effective communication and account for resources such as time and money used during a project. MEL can also promote program effectiveness by establishing evidence which can be used to test assumptions, make informed decisions, and apply learning across projects and locations.
MEL is beneficial for a variety of individuals involved in a project. This can range from those who fund projects to local organizations involved in project activities. MEL can benefit several stakeholders in the following ways:
- Donors: MEL helps a program measure its achievements and therefore provides accountability for funding.
- Beneficiaries: MEL helps a program track their activities, outputs, and results to provide transparency to intended end-users and/or recipients.
- Project staff: MEL helps to guide the implementation of activities in a standardized and coordinated way.
- Project leadership: MEL helps them understand the results of the program to inform decision-making using evidence.
Monitoring, evaluating, and learning tools to guide program design
Theory of change
Almost all projects can be tied back to a theory of change. A theory of change is an if-then statement that summarizes the expected or anticipated outcomes that will occur by implementing certain interventions. This helps project staff, donors, and other stakeholders understand the problem and makes all implicit assumptions more explicit. A theory of change will often illustrate the proposed actions and intended consequences. It is important to note that within the anticipated outcomes, there are both short-term and long-term outcomes that are described. Short-term outcomes follow initial actions or interventions soon after they occur. Long-term outcomes may result after the project has ended and take time to become apparent or measurable.
Results framework
A results framework builds off of the theory of change to express the goal and objective of a program. Strategic and program objectives may be used that convey strategic impacts for donors as well as direct objectives that may result in broader contributions. In order for program and strategic objectives to be met and measured along the way, specific results are established that are necessary to achieve these objectives, called intermediate results and sub-intermediate results, which are necessary to achieve the intermediate objectives. Creating a tiered structure enables very clear results to be established over time and allows various points for iteration and adjustment.
Performance indicators
In order for program objectives to be evaluated, indicators are established providing particular aspects of a program to be tested. Two characteristics of indicators are:
- Indicators are variables whose value changes from the beginning of the project to a later time when activities have made their impact.
- Indicators are measurements in which their values can be measured based on changes from one point in time to another.
Two types of indicators that Chen discussed are input indicators, which track the number and nature of material, financial, and human resources made available through a project, as well as output indicators, which represent the direct, tangible benefits produced by the project as a result of inputs. When designing indicators, ensure that they are specific, measurable, attainable, relevant, and time-bound (SMART).
Types of evaluations
When it comes to evaluating the progress and performance of a project, it is beneficial to conduct a baseline evaluation at the beginning of a project. Baselines help project managers determine progress in achieving outputs and outcomes and the extent to which change has occurred over time. Impact evaluations measure a defined intervention’s attributable change to a development outcome. A lack of baseline data can make it difficult for project managers to measure performance and poses challenges for potential decisions to conduct impact evaluations. Internal evaluations may be conducted by an organization to measure changes as a result of interventions that can inform internal decision making. Other evaluations include performance evaluations which encompass a broad mix of methods. They may incorporate before-after comparisons, but generally lack a rigorously defined counterfactual.
The importance of learning
Over the course of development projects, there are always variables and factors that change that were not originally anticipated. While learning has always been central to international development, particular emphasis has been made recently on incorporating learning to allow projects to evolve and adapt, in order to be more effective in their pursuits. Learning can occur in multiple ways through formal and informal settings, with internal or external stakeholders, as well as for programmatic or organizational contexts. Some examples include "pause and reflect" sessions, after-action reviews, stakeholder engagement sessions, and community of practice meetings. As environments become more complex and dynamic, the process of learning becomes increasingly important and valuable to achieve greater results.