Key Messages from the April FORCE Community Webinar: Evaluation and Adaptation of Health Programs Using the Coaching Approach

Background

The FORCE webinars for the first four months of 2021 have focused on building the capacity of the community on the coaching approach (CA) using the steps of the Country Engagement (CE) framework. Using country experiences from coaches and policymakers across Africa, each webinar has highlighted the technical and non-technical skills required by a coach to support a country through the different stages of its journey to achieve health system reform. So far, the webinars have discussed the following:

• January: Overview of the CA using the CE framework: Understanding the problem
• February: Introduction of evidence and solution selection
• March: The design and implementation of COVID-19 vaccine programs in Africa
• April: Evaluation and Adaptation of Health Programs Using the CA

April’s webinar was the last in the CE framework series and focused on Monitoring and Evaluation (M&E) of health programs using the CA.

Evaluation and Adaptation of Health Programs Using the Coaching Approach

M&E is an essential component of any health intervention, program, or policy. When done right, it can highlight the difference a program is making and for whom; it can also identify areas that are on target, what needs to be adjusted and can trigger policy reform. Determining the effectiveness and efficiency of a program requires a sustainable, comprehensive and strategic M&E plan. Systematically collecting data on the activities, characteristics and outcomes of a specific program can help stakeholders and implementers determine its merit and inform decisions on future resource allocation.

The April FORCE webinar focused on the final step in the CE framework, evaluation and adaptation. The objectives of the April session were to:

1. Highlight the basic principles of M&E.
2. Highlight the technical and soft skills needed to support a country to evaluate and adapt existing health programs.
3. Describe the design of an M&E process and how basic M&E tools can be used to monitor and evaluate country programs.

Our panellists for the webinar were M&E experts Ama Fenny from Ghana and Seyni Mbaye from Senegal. Ama provided information on the fundamentals of M&E and how coaches can use it to adapt and improve health programs. At the same time, Seyni gave an overview of his experience supporting this process in African countries. The webinar provided technical information and practical, real-world experiences to convey learnings from the field using the CA.

Key Takeaways from the Session

• Monitoring is the process of assessing the progress of a program using data collected on selected indicators that measure the effectiveness of implementation.
• Evaluation is usually the last step of the CE process. It assesses if the policy has achieved its broad goal and provides information on the impact of a program on its target population.
• A theory of change (TOC) is a critical aspect of program design and implementation. The TOC outlines the proposed outputs, outcomes, and impact of the program and will help ensure that the right indicators are selected to evaluate the program.
• Indicators are a critical part of the M&E process; without good indicators that reflect the project’s objectives, the effectiveness and impact cannot be measured.
• Stakeholder mapping should be conducted before program implementation to ensure that all key stakeholders are included in the planning process.
• Data collection often requires persistence. Traditional and modern data collection methods can be used during M&E, and these do not have to be expensive. These include using apps on mobile phones as a cheap and effective way of data collection in low resource settings, key informant interviews, Focus Group Discussions (FGDs) and one-on-one methods of data collection.
• Data confidentiality is key to building trust with stakeholders. Consent must be sought to collect and use data or quotes from stakeholders while reporting on evaluation responses.

Coaching skills required to evaluate and adapt health programs

While M&E seems primarily quantitative and requires hard skills, specific soft skills are necessary for a coach to support a country to evaluate a health program successfully.

• Coaches should work to ensure that indicators are program and context-specific. To achieve this, coaches must have a good understanding of the program objectives bearing in mind the different priorities of stakeholder groups involved in the program and ensuring that their priorities are captured within the selected indicators.
• Process facilitation is a crucial skill for a coach during M&E. A coach can serve as a neutral and competent broker to organize stakeholder engagements and ensure that expectations are managed during the planning and evaluation of programs.
• Coaches require strong negotiation skills to build a consensus among stakeholders with different or competing views.
• Pitching skills are necessary for coaches to communicate the value of the evaluation to stakeholders and encourage them to participate in the process.
• Coaches need to be culturally sensitive. This is critical to the successful conduction of M&E as interactions between stakeholders and coaches are often guided by the cultural context of the country/region.
• A coach must be able to get the information they need using a culturally appropriate method. Coaches need to be flexible while conducting M&E processes and use their unique knowledge of the context to achieve their results.
• The engagement of country program officers and policymakers throughout the project duration is vital to creating ownership of the project by key stakeholders. A coach must also be able to generate and convey evidence in a contextually appropriate manner that will improve the likelihood that results will be accepted and used for policymaking.

Successful implementation of M&E requires technical skills such as program design and management, and knowledge of data collection methods suited to any program being conducted. However, soft skills are also required to ensure M&E processes can be undertaken to achieve the desired outputs of the program. These include skills such as stakeholder engagement, process facilitation, negotiation, pitching skills and cultural sensitivity. Stakeholder identification, mapping, and engagement are critical in program planning and must be conducted at the beginning of every project. Coaches must be aware of cultural contexts when engaging key stakeholders at every step of the implementation process to ensure the right kind of data is collected and utilized.

To learn more about the coaching approach, please take the online module here. Join us for our next webinar in May on the tools and technology that can be deployed for the CA.