Mali Office, BP: E1171, Hamdallaye, ACI 2000

Rue 311, Bamako, Mali

Tel : +2 23 20 29 41 55

Website: www.blumont.org

 

 

Waati Yelema Labenw

Project Final Evaluations

Terms of Reference

 

November 2018



Terms of Reference

Waati Yelema Labenw Final Evaluation

BRACED extension

 

International Relief and Development/Blumont

Mali

 

  1. Project Background

The Waati Yelema Labenw project (WYL) is funded under the BRACED-X program by UKaid.

It draws on the rich experiences and satisfactory results of RIC4REC to reinforce its achievements through: Training, coaching and support for community and community leaders to facilitate analysis, planning and actions adaptation and resilience of the CCAM within their communes and communities.

It institutionalizes the management of climate change adaptation actions by the community. Local elected officials will acquire capacities to facilitate the analysis of adaptation, planning and development actions.

Municipalities will approve Community priority actions for funding through a project micro-grant mechanism. In the designated communes, WYL's activities will be integrated into the BRACED Decentralized Climate Financing (NEF) project.

The experience of the communes through RIC4REC and WYL will be shared with regional and national policy and programmatic decision-makers through study tours, documentation of the experience and a workshop of learning and influence on policies.

  1. Project Goal, objectives and implementing strategy

The project is implemented in ten (10) Communes and thirty (30) villages of the Regions of Mopti, Ségou and Koulikoro.

The Waati Yelema Labenw (WYL) project aims to improve the resilience of poor people in Mali to climate-related shocks and constraints and aims to reach 26,000 beneficiaries including 4,000 women through the creation of 90 sustainable, climate-friendly and market-oriented businesses and savings groups. It will improve incomes, increase savings and access to finance, while increasing the capacity for anticipation, adaptation and absorption.

Project outcome indicators, output indicators, and activities have been structured under the three (03) strategies to achieve project objectives which are:

Strategy 01: Making CCAM a priority with strong community basis for implementation

Facilitation in the 10 targeted communes of the Climate Change Adaption Management (CCAM) process in thirty (30) new villages with a focus on broad impact actions and funding of the Resilience Action Plan

Strategy 02: Empower communities to use climate information to make management decisions about their livelihood activities that reduce risk

Promote the adoption of agricultural practices and livestock, both adapted to the climate and environmentally friendly, while promoting access to targeted climate information.

Strategy 03: Increase resources and access to resources through creation of market oriented group membership micro enterprises owned by vulnerable persons

  • Creation of 10 market-oriented vegetable companies.
  • Creation / reinforcement of 30 farmer grain marketing groups of 50 members each.
  • Creation / reinforcement of 30 groups of breeding and marketing of small ruminants (majority of women 50 members)
  • Establishment or strengthening of 30 savings and credit groups around established microenterprise activities and building links with formal microfinance institutions
  1. Consortium Implementing Partners

Name of organization

Description of role

Added value

Core Partners: Substantial role in implementation and strategic management

BLUMONT (prime)

Coordination and integration of project activities and partners, strategic vision, and accountability for program results and reporting. BLUMONT leads in gender, women’s cereal processing enterprises, learning and enabling environment component, and coordination with national and project structures.

Large and complex project management experience; know-how in improving market- and policy-enabling environments; and sectoral expertise in value chain development, climate change, and DRR. In addition, strong regional presence.

AMASSA Afrique Verte

Climate-adapted livelihoods of the project, helping communities select and lead their own interventions to promote adapted agriculture, livestock, and women’s businesses. Lead program operations in Mopti.

Strong contextual knowledge of agriculture in Mali, MIS, and women’s livelihoods. 

ICRISAT/CCAFS

ICRISAT provides technical direction for improved seeds marketing, soil quality management, agricultural P&T demonstrations. CCAFS support the “farm of the future” approach and beneficiary use of climate information, integrate WYL in national and regional science-policy dialogues.

Sharing one administration in Mali, they draw on CGIAR global network of climate adaptation research and a strong set of learning and dissemination platforms to support WYL objectives.

Resource Partners: Organizations with discrete technical assistance

Orange Mali

BLUMONTis expanding their mobile phone on demand advice service Senekela to address climate adaptation and livestock and promoting use of Sandji 2-day weather forecast for farmer planning.

Offers the only mobile phone information dissemination platform of its kind in Mali. 

 

  1. BRACED Extension FE Aims and Objectives

The overarching aim of the FE is to gather information about WYL progress and lesson learning throughout the life of the project since the RIC4REC Final Evaluation. This should help us to continue to build our understanding about: how, where, when and why do BRACED interventions work, and what can be learned, and to understand:  What difference did the BRACED extension make? How, why, for whom and in what circumstances?

The extension FE should enable to undertake robust reflection and gathering of evidence on project success and failure in order to explore, test and revise assumptions about the project, linked to the theory of change.

The project extension final evaluation should examine the outcomes WYL has achieved, linking these to the “activity packages” implemented under your project (such as climate information, community planning, financial services/inclusion, agriculture, policy and advocacy work), and any synergies between these, as well as taking a broad view of the project as a whole. In order to understand what has worked well (or in need of improvement), and why, the BRACED X final evaluation needs to examine the mechanisms through which change happens. It also needs to investigate the way that WYL’ context shapes how and why change happens, leading to outcomes and processes that build and strengthen resilience (applying realist evaluation thinking).

 

  • Scope

The independent party will design, conduct and produce the written deliverables ( report) for a Final Evaluation for the DFID funded “Waati Yelema Labenw” (WYL) project implemented by Blumont International (replacing “International Relief and Development/IRD”) and partners in Mali.

BLUMONT will retain the services of an independent consultant or firm “FE Consultant” to conduct the FE. The FE Consultant will design and implement a rigorous survey and data collection plan that meets BLUMONT and donor data quality standards.

  • The Final Evaluation should focus on project progress and lesson learning since the BRACED Final Evaluation (December 2017)
  • The consultant should plan and deliver against a standard set of headline Evaluation Questions with sub-questions tailored to the WYL theory of change
  • It will be process-orientated and explanatory in nature
  • Should focus on outcomes, explaining pathways to change, where possible ‘testing’ ICMOs (Intervention-context–mechanism–outcome), and ‘mechanisms’ of change (applying realist evaluation thinking)
  • Consultant should also ensure that their questions address some of the OECD DAC evaluation criteria set out in Annex 1 of M&E Guidance note 7– however, the evaluation and sub-questions naturally ‘map’ onto the OECD-DAC criteria.

 

 

  1. Evaluation Questions

The following set of evaluation questions (from M&E Guidance Note 7) should be used as the basis for planning the FE of WYL:

  • Evaluation question 1 – To what extent have particular WYL’ interventions led to anticipated changes and results? This opening question, consistent across BRACED MTR and Final Evaluation and the extension FE, requires IPs and consultants to reflect on the evidence of results delivered against the results / changes anticipated in the project Theory of Change. How have interventions/ activities interacted to lead to change?
  • Evaluation question 2 - Specifically focusing on understanding ‘mechanisms’ (the causal forces or powers that explain why a change happens), and the contexts or conditions that affect or create mechanisms, how and why have particular intervention packages led to observed results and changes? Projects should focus on defining ‘mechanisms’ (e.g. changes in behaviour and reasoning) in order to understand what it is about the nature and design of an intervention that has enabled it to be effective or not. Sub-questions under this question should explore:
    • What has WYL learned about delivering these packages of interventions?
    • What evidence is there that the interventions and the mechanisms that support them have the potential to deliver ‘amplified results’ and/or ‘transformational impact’?
    • What are important contextual factors – conditions that cause the mechanisms to ‘spark’ that lead to outcomes?
  • Evaluation question 3 - Based on your accumulated knowledge and understanding, what key resilience strengthening lessons can be learned and replicated from “WYL”? This final evaluation question requires IPs to reflect on the intervention-level learning you have accumulated over the life of the project, reflecting specifically on what has happened throughout the BRACED extension phase, and to distil this learning into a set of evidence-based lessons, defining which of these can potentially be replicated elsewhere. Under this question you should also explore:
    • What difference has BRACED X made? How, why, for whom and in what circumstances? This should include reflection of both the difference it made for IRD/Blumont as well as for people potentially benefitting from the interventions.

 

The Evaluation Synthesis and Support team will support consultants and IRD/Blumont to develop a detailed and project-specific Evaluation Matrix for the extension FE and to prioritise your outcomes and assumptions to be tested at FE. These will guide FE. When refining evaluation questions and sub-questions, it may be helpful to review BRACED X and project Theories of Change, and The “3As Approach” to measuring resilience.

 

 

  1. Evaluation data collection and data analysis
    • Methods
      • Data Collection
    • The project evaluations will take a mixed methods approach, using and generating data through a range of qualitative and quantitative methods. Quantitative data may be drawn from logframe, annual reporting and baseline-endline comparison. Additional data collected for the extension FE is most likely to be qualitative to get at ‘how’ and ‘why’ things happened the way they did, applying a realist ‘way of thinking’.
    • The FE will need to test ‘assumptions’ based on the project theory of change about how the project worked, drawing on realist evaluation approaches, refining these in the light of your experience in implementing the project. You will build on what you learned in the MTR and FE and you/ the consultant will need to define/identify key processes and outcomes and their indicators to include in the inception report.
    • The FE is likely to combine the review of existing project routine results reporting data (including KPI4, the 3As, Evaluative Monitoring, MTR, BRACED FE) with specific primary data collection activities.
    • Primary data collection is likely to be primarily qualitative (through interviews, focus groups and participatory exercises and feedback mechanisms) but may also include quantitative data from fieldwork or web/email surveys with staff.
    • The reviews and evaluations will engage a broad range of project stakeholders, from project team members to project beneficiaries/participants as well as wider key informants, champions, and observers, including those with ‘external’ perspectives.

Consultant for the FE will collect quantitative data about the two (02) important measure in the project (impact and the outcome for resiliency). Below are the indicators concerned:

  • % of beneficiaries whose resiliency has improved as a result of WYL support
  • % of beneficiaries interviewed that state they have used climate information (early warning systems, weather forecasts, or advisory services) to make livelihood decisions with project support
  • % of beneficiaries interviewed that state they have applied climate smart practices with project assistance
  • % of beneficiaries interviewed that state they have increased revenue from project supported activities
    • Data Analysis

Consultants must clearly describe the way that they plan to arrive at a set of robust and evidence-based findings and conclusions.

Evidence should be consistently cross-referenced throughout the FE report, and claims made should be substantiated / validated through reference to the evidence to support these claims.

  • Evaluation Team Roles and Responsibilities

 

Position

Roles and Responsibilities during FE

1. Project Director (PD; Blumont)

David BENAFEL (male)

 

·    Overall supervision of FE team, consultant, process and partner/key stakeholder inclusion

·    Review and approval of FE Report

2. Applied Learning, Monitoring and Evaluation Manager (ALMEM/)

Aliou Badara SAMAKE (male)

·    FE Team Leader – Initial design of ToR (October 2018)

·    Update and revisions of FE ToR jointly with BRACED-KM

·    Update and revisions of FE Inception report and Evaluation Matrix

·    Draft SOW consultants  that may be hired and supervise work quality

·    Co-drafting of FE report with Facilitation consultant

3. Chief of Operations (D/PD; )

Mme MALLE Fatoumata Alidou GAREKA (female)

·    Coordination of team and stakeholder inputs, FE field work

·    Review of draft FE report

4.  Monitoring & Evaluation (M&E) Assistant

Losseni DIARRA (male)

·    Processing of data; support to design of data collection tools; support to FE field work, documentation of mini workshops outcomes, etc.

5. Facilitation Consultant (FC)

 To be hired;

·    Design Inception report and Evaluation Matrix

·    Design of questionnaires, KII questionnaires, focus groups guides and overall data collection plan

·    Facilitate training of staff and enumerators in focus group discussion (FGD) facilitation

·    Co-Facilitate stakeholder workshop

·    Analysis of primary data according to data analysis plan

·    Drafting of FE report

·    Produce a FE finale version

6. Field Team (FT; AMASSA and Blumont)

Coordinators (1M); Senior Advisors (02); Advisors (10);

·    Organizational Support

7. Communication Consultant

Dioumawoye Sangho (Female)

·    Photo and video documentation

8. KMEL:

Yaya BOUARE (male)

 

·    Support to initial design of ToR (October 2018)

·    Technical support to inception report and Evaluation matrix finalization, data analysis plan finalization, data collection tools design.

·    Reviewer for first and second drafts of FE report

9. Mali Finance & Administrative Team (DFA, PFM LA, HR, RC)

Aissata DEME, Director (F);  Ahmadou TRAORE/Project Finance Manager (M);  Mamadou Diarra/ Logistics assistant(M); Mme DIOP Djenebou DIAKITE/HR Officer (M); others

·    Hiring of consultant;

·    Procurement: rental vehicle as needed, workshop venue, material, etc.;

·    Itinerary security pre-checks and overall situation monitoring;

·    Financial analysis of costs and cost efficiency of activities

10. Project Associate/HQ (HQ/PM; )

Patrick KELLY (Male)

·    Communications with Fund Manager and Coordination with Knowledge Manager

 

  • Limitations

In terms of limitations we can mention a few points already. As for the duration of the 18 month extension for the implementation of the project will be a limit because the implementation of some activities will be completed only from March (end of the project) where the evaluation is supposed to unfold at the same time. So it will not be possible to measure the impact of certain activities such as cereal banks, groups of small ruminants’ breeders, market gardening groups and those of fattening. Because unlike RIC4REC for WYL we have chosen new villages for the implementation of the project. So for some indicators we risk having underestimated values such as:

  • Number of people whose resiliency has improved as a result of WYL support
  • % of beneficiaries interviewed that state they have increased revenue from project supported activities

However, the consultant should pay specific attention to the outcomes of those activities on the project beneficiaries.

 

  1. Key Cross Cutting Considerations
    • Gender and Social Difference

See BRACED Working Paper ‘Gender and Resilience’ (Le Masson, Norton and Wilkinson 2015).

BRACED interventions will make a better contribution to individual, household and community resilience to climate extremes and disasters if implementing agencies address existing social dynamics (including gender) and power relations.

The WYL FE methodology should account for influence of gender dynamics and social power relations on project implementation and impacts.  During the WYLFE, you should:

  • Assess how all activities (not just those aimed at empowering women and girls), processesand outcomes affect and benefit gender and other social groups (including through the use of disaggregated data)
  • Facilitate the participation of different gender and social groups in the extension FE, including building comparisons (e.g. between data gathered from male- and female-only focus groups, or interviews with different members of the same household).
  • Encourage reflection on how the implementation process is addressing gender and social relations. This could include a ‘reality check’ of your WYL project ambitions for changes in women’s empowerment and gender norms. This will ensure that these ambitions are realistic, and identify any changes that you need to make your programming more effective.
    • Quality, Rigour and Ethics:
  1. Expected Outputs
  2. Inception Report and Evaluation Matrix
  3. FE report (up to 50 pages long). The FE report must contain (but is not confined to) the following sections:
  • Description of WYL project and outcomes achieved, the ‘intervention packages’ – activities being undertaken – and the synergies between these;
  • Project theory of change for the extension activities;
  • Description of assumptions underlying the project (drawing on theory of change and other evidence).
  • Description of the extension FE approach (conceptual and analytical framework) and data collection methods (e.g. routine data collection / interviews etc.) including participant selection, and the reasons for your choice of methods;
  • Description of the BRACED extension FE team and data collection process;
  • Key evidence and analysis – how and why things happened the way they did, for whom and in what circumstances, focusing on both outcomes and processes important for building and strengthening resilience; how assumptions about the project have changed since the start;
  • Lessons learned from the beneficiaries and the implementation team that could be used in the future;
  • Limitations;
  • Conclusions;
  • Set of evidence-based lessons and discussion of what should/ could be replicated and/or done differently next time.

 

NOTE: The final evaluation report should be fully referenced back to the underlying data sources. You should ensure that you have adequate data storage protocols so that data can easily be shared between consultants and IPs (i.e. through Dropbox or Google Drive).

 

  1. Timeline and Budget Summary

You should include a timeline that includes planning, data collection, analysis, verification (including any workshops), report drafting and time for IPs to review i) consultants’ inception reports before the 31st December 2018 deadline and ii) FE reports before the 17th May 2019 deadline.

You should also include a summary/headline budget.

The consultant should take account the information in the table below when making timeline and budget summary.

 

September-October 2018

IPs recruit Consultants to undertake BRACED-XFEs. Evaluation Synthesis and Support (ESS) Team are available on request to review TORs.

October-November 2018

One-to-one discussions between IPs, Consultants and the ESS Team on the BRACED-X final evaluation inception report and matrix template. The team will contact IPs to arrange a suitable time.

November-December 2018

ESS Team are available on request to provide feedback on draft inception reports and evaluation matrix.

31st December 2018

Deadline: IP submission of Inception report and matrix to the Evaluation Synthesis and Support Team

15thJanuary 2019

ESS Team provide feedback on inception report and terms of reference, including any changes required for sign off.

February-April 2019

ESS team one-to-one support with IPs/consultants available on request.

31st March 2019

Data Collection to be completed

January-30th April 2019

Final evaluation preparation and data collection.

30thApril 2019

Review and revision of FE Reports: IPs to review and consultants to revise reports before submission to ESS team.

17th May 2019

Deadline: IP submission of Final Evaluation report to the Evaluation Synthesis and Support Team

1st June 2019

Evaluation Synthesis and Support Team to seek clarifications from IPs and consultants on FE contents on a case-by-case basis.

17th May-5th July 2019

BRACED-X FE programme–level synthesis.

 

How to apply:

 

  • Please send your proposal package by December 13th 2018 at malirecruitment@blumont.org or you can also deliver your application in closed hard copy at the IRD-Blumont Country Office at: Hamdallaye ACI 2000 Rue 209 Porte 420, Bamako-Mali.
  • For more questions please address to Fatoumata Alidou Gareka, Chief of Operations, fgareka@blumont.org
  • We thank all applicants for their interest.
  • Only those selected will be contacted and Applications are not returned.

 

NB: All incomplete offer will be eliminated or won’t be considered