Country: Uganda
Closing date: 18 Mar 2018
Terms of Reference for End of Project Evaluation
Project Title:Multi-sectoral Emergency Assistance to South Sudanese Refugees in West Nile - Uganda
Donor:Department of Foreign and International Development (DFID), UK
Project Location:Imvepi, Omugo, Palabek and Rhino Camp
Project duration:July 1st, 2017 to March 31st, 2018
*Evaluation to be conducted in: April 2018*
1. BACKGROUND:
In 2011 South Sudan became the newest independent country of the world; despite the high expectations and promises, peace did not prevail for long. Since the onset of the new civil war, more than one million South Sudanese refugees have sought shelter in neighbouring countries. In July 2017, further fighting broke out and led to more thousands of South Sudanese people being displaced. Serious abuses against civilians by both government and opposition forces continues to prevail and it is apparent at this point that South Sudan will continue to face high insecurity and violence, lack of governance and justice, collapsed economic situation, food insecurity in majority of the population, increase of displacement, and very limited returns of South Sudanese refugees back in country. Uganda has continued to open its doors for the refugees. The consortium has been instrumental in supporting the needs of the refugees settled in Imvepi, Omugo (Rhino Extension) and Lamwo with emergency water and sanitation, health and protection assistance.
The project’s aim has been to save lives and build the resilience of South Sudanese refugees and in some cases support the host communities. The consortium has focused on promoting and implementing activities that focus on longer-term solutions and support sustainability and greater self-reliance – including in terms of sustainable water supply systems, capacity building of local user committees, building the capacity of communities on protection issues, and preventing and responding to gender-based violence, while at the same time implementing emerging issues on health and protection which will support refugees while in displacement and also prepare them better for durable solutions – most likely through local integration.
The Multi-sectoral Emergency assistance to South Sudanese refugees in West Nile- Uganda is a nine-month program funded by DFID seeking to ensure refugees in West Nile have access to appropriate WASH, Protection Health and Nutrition. The project is being implemented across three settlements of Imvepi (zone 1, 2 and 3), Palabek (zone 1, 2, 5 and 7), Omugo (all villages) and Rhino Camp in consortium with Lutheran World Federation (LWF), Save the Children (SC), Action against Hunger- US, International Rescue Committee (IRC) and Danish Refugee Council as the lead partner.
2. TARGET GROUP
The project targeted 184,150 direct beneficiaries, including South Sudanese refugees and the host community. The implementing of this project is through the consortium modalities: through 5 international NGOs, which have been operational in the West Nile, and with whom DRC has partnered in past projects. This evaluation applies to the delivery of the project through consortium modalities.
3. EXPECTED RESULTS OF THE PROJECT:
Through WASH, Health/Nutrition and Protection intervention, the project aimed to achieve the following outcomes and results:
Objectives/ or Result area
Indicators
Outcome
Reduced likelihood of onward migration, conflict and tensions and revitalised approaches’ to development and humanitarian aid (characterised with improved quality, effectiveness and efficiency of humanitarian response and greater economic opportunities)
% of Households with safe access to clean water in settlements
% of reported protection cases referred to the relevant stakeholder through the referral pathway
Outpatient utilization rate (target 1-4)
Output 1:
Refugees and surrounding host communities have improved access to water and sanitation and hygiene facilities, which are user friendly and gender sensitive
No. of refugees with access to at least 15 litres of clean water/ per day for drinking, cooking and personal hygiene
No. of households with access to HH sanitation facilities
# of schools and health centres with basic WASH facilities
# of Individuals reached with hygiene messages on safe water use, personal and environmental hygiene
Output 2
SSD refugees and host communities living in the West Nile region have access to quality primary health care services including immunisation and IYCF services
Proportion of reproductive health staff and relevant clinical staff trained on Clinical management of rape and management of Survivors of SGBV
Number of Visits to MBAs by mother-baby pairs
Number of OPD consultations conducted
Number of children reached through static and outreach immunisation activities
Number of Health Facilities supported by SC
Output 3
The safety and self-protection capacity of vulnerable refugees is improved
# of men, women, boys, girls and persons with specific needs that receive protection services
# of women and adolescent girls supported by SGBV prevention and response activities
4. PURPOSE AND SCOPE OF EVALUATION
This DFID-funded project will end in March 2018 and an external summative evaluation is planned to take place after completion of all project activities. As this is a summative evaluation, its objective is to assess the extent to which the program has delivered against its expected results. The scope for examination is determined using OECD-DAC criteria for evaluating humanitarian action. Relevant criteria are associated with a number of key questions (under section 5) that are to be addressed and explored.
Specific Objectives:
▪ To assess the level/degree of quantitative and qualitative impact of the project against its expected results;
▪ To assess the design, planning, delivery and management of the project by DRC and its partners in accordance with DRC Programme Standards, Minimum Standards in Emergencies, IASC Gender Marker and DRC Minimum Standards for Gender in Emergency;
▪ To identify and assess key internal and external factors (positive and negative) that have contributed, affected, or impeded the achievements, and how DRC and the partner have managed these factors;
▪ To assess how the project has impacted upon the protection of the target and affected population and contributed to a reduction of factors of vulnerability;
▪ To draw key lessons and learning from the project and make recommendations that will help inform DRC’s formulation and design of future projects that will benefit the host community and refugees of the South Sudan Crisis.
Primary users of the evaluation findings are DRC and Consortium partners and staff, DFID, and other actors directly involved in the implementation of the project.
5. EVALUATION CRITERIA AND KEY QUESTIONS TO BE ADDRESSED IN THE EVALUATION:
The following provides a guide to the questions to be addressed by this evaluation, under each of the criteria below:
Relevance and Appropriateness:
▪ Have DRC and its consortium partner selected relevant operational areas for their work?
▪ Have DRC and its partner targeted the most vulnerable people, including vulnerable men, women, boys and girls and people of disability?
▪ To what degree did the intervention address the WASH, Health and protection/GBV needs of the targeted women, men, boys and girls and contribute to reduced vulnerability?
▪ What was the level and quality of participation of the beneficiaries in project design?
▪ How responsive were project activities to the targeted women, men, boys and girls in both communities (i.e. refugees and host communities), given their circumstances and priorities?
▪ Were the activities carried out in the most appropriate and relevant manner, given the circumstances, and in line with the priorities of the refugees and the host community?
▪ To what extent did the key contextual changes, threats and opportunities that arose during implementation influence and inform project implementation?
▪ How appropriate were the alternative solutions/changes proposed and/or implemented by the team to overcome the challenges faced during the project implementation?
▪ Were the activities in line with the outputs and results of the project, as stated in the logical framework?
▪ How and to what extent were the monitoring, evaluation findings used to inform decision-making and the improvement of project implementation?
Efficiency:
▪ Was the project implemented based on the best use of existing resources/capacity; e.g. the capacity of the DFID consortium partner and the internal capacity and expertise of DRC itself? What key limitations exist on this front? What could DRC’s future consortia do to increasingly develop and invest in existing resources?
▪ Did the project align with DFID’s ‘value for money principles’? How and what could DRC and partners improve on in its future interventions in terms of value for money?
▪ How cost effective was the intervention? What were the key cost drivers for each consortium partner? What cost-effective alternatives could have been used?
Effectiveness:
▪ Were the project’s planned results achieved for men, women, boys and girls?
▪ What were the main challenges of the project and how well were they addressed?
▪ How the beneficiaries were selected and were the beneficiaries informed of the selection criteria?
▪ How effective were the selection criteria in reaching out to the most vulnerable populations?
▪ How effective and gender, vulnerability responsive was the implementation approach employed and implemented by the WASH, Health and protection/GBV teams?
▪ How inclusive and culturally sensitive were the activities carried out in terms of the approach, quality of participation, information and its dissemination?
▪ How and how well gender and disability concerns and considerations integrated into the assistance components? What were the effects of this? If it did not happen, why not?
▪ What unintended consequences (if any), whether positive or negative, has the intervention had on women, men, boys and girls (refugees and the host community)?
▪ To what extent has gender been mainstreamed into the project?
▪ Generally, were the activities carried out in line with the original plans? If not, were the changes adequately discussed, documented, and justified?
▪ How well were the activities, outputs and outcomes documented and monitored? What kinds of systems and tools are developed and in practice and how well have they worked?
▪ Was project design consistent with the Do No Harm principle? and with a safe programme approach? Was project delivery consistent with these two core principles?
▪ What system and mechanism are in place to ensure accountability to the beneficiaries and how well did it work?
In addition to the criteria above, the evaluation would explore the theme of Partnership:
▪ What role has the quality of the partnership played in the project, and how might this be built upon in the future?
▪ To what extent were the consortium partners been involved in proposal development, project implementation, reporting, monitoring and learning?
▪ What was the level and quality of coordination led by DRC in project?
▪ To what extent did this partnership influences the project during implementation?
▪ Have the ways of working with the partner and the support to the partner been effective and did they contribute to the project’s achievements?
▪ Have the ways of working with the partner actually led to building the capacity of the partner?
▪ What kinds of systems are developed for mutual accountability between partners and DRC and how well did they work?
▪ What are the key issues related to the technical and managerial capacities of DRC and the partner’s team for the effective implementation of the project? How effectively have these issues been addressed and what are the recommendations to address these issues, if any?
6. METHODOLOGY:
It is expected that the evaluation will be carried out in conformity with the ‘Evaluating Humanitarian Action using the OECD-DAC Criteria’ and evaluation best practices. The Evaluating Humanitarian Action using the OECD-DAC Criteria can be accessed through this link: http://www.alnap.org/resource/5253.aspx
The evaluation will use a mix of quantitative and qualitative methods, and draw on both primary and secondary data collection techniques. The evaluator is expected to develop a detailed methodology (to be endorsed by DRC and DFID consortium Partners). Note that a baseline survey was conducted at the beginning of some project. Endline assessment may be done for some of the projects. The evaluation methodology should make use of this data already generated by DRC’s and DFID consortium partner monitoring and evaluation team.
We anticipate that this summative evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s) to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:
● Targeted beneficiaries
● Partners and actors directly involved in the project at different levels:
o The implementing partner;
o Community leaders (if applicable) and representative bodies of the affected population;
o Local authorities, regional Government, actors involved in the coordination of humanitarian interventions of the affected population;
● DRC and DFID Consortium partner staff involved the implementation of the project e.g. country and field teams;
7. TIME FRAME, KEY ACTIVITIES AND EXPECTED LEVEL OF EFFORT
Time-frame: The evaluation is to be carried out as early as possible in April 2018, with the final report submitted to DRC for approval no later than end of May 2018. The exact dates of the evaluation are to be confirmed with the selected consultant(s).
Total expected level of effort: 25 working days
Review essential documents of the project, including but not limited to the original project proposal, interim or on-going internal reports, and evaluations and lessons learned exercises undertaken thus far and review the key questions suggested and if necessary propose adjustment (5 days – work to be done from consultant’s home location, all documents will be shared by DRC via email); Review of evaluation concept to ensure appropriate representation of content by all DFID IPs
Develop a detailed Evaluation plan (to be endorsed by DRC) (1 day);
Primary data collection (7 days) including training of data collectors;
Data analysis and preparation of draft evaluation report (6 days);
Workshop to share and validate the findings from the evaluation (1 day). Selected DRC and partner staff will participate in this workshop;
Final report presentation to all DFID partners and key stakeholders
Finalize the final evaluation report and send it to DRC (3 days). DRC will then prepare a management response to be annexed to the evaluation report;
Additional time for travel (from the home location of the consultant to project location and back) will be included based on the location of the selected consultant(s). – (travel time is estimated at 2 days)
Reporting:The consultant will report directly to DRC’s Consortium Manager based in Kampala and to DRC’s Senior M&E Officer based in Kampala, and will closely work with other DRC and Consortium staff in West Nile and Lamwo.
8. EVALUATOR QUALIFICATIONS:
This summative project evaluation should be led by a person (or persons) with a minimum of 5 years experience in humanitarian interventions including demonstrated experience in the monitoring and evaluation of WASH programming (a must), with preferably some experience in cash transfer/voucher modalities, gender in emergencies programming, and protection, including the use of participatory quantitative and qualitative methods. Strong facilitation and English writing skills are also required and a familiarity or direct experience working in Northern Uganda, particularly West Nile region of Uganda, and the working modalities of DFID funded partnership, is preferred.
9. EXPECTED OUTPUTS:
a) Prepare an inception report detailing approach of delivering the evaluation to ensure attainment of key objectives
b) Develop a detailed evaluation plan (to be submitted after the document review but before the data collection - to be endorsed by DRC), outlining the proposed methodology;
c) Draft evaluation report in English to be presented to DRC (no more than 30 pages excluding annexes, including executive summary not exceeding 2 pages);
d) Workshop to present the draft report (including the findings of the evaluation and the lessons learned), and to give feedback to DFID Consortium partners;
e) Submit a draft electronic copy of the final evaluation report within one week (7 days) of the workshop. Feedback from DRC and Consortium partners will be provided within one (1) week after the submission of the draft report. The final report will be produced in one week (7 days) of submission of the comments. It will include changes/modifications, agreed between DRC in Kampala and the consultant.
o The report should systematically answer the key questions posed;
o It should fairly and clearly represent the views of the different actors/stakeholders;
o It should give the conclusions of the evaluator, in a way that is clear and substantiated by the available evidence.
10. Payment and instructions for interested consultants
Payment will be done in two instalments, 25% upon contract signature, and 75% upon DRC’s approval of the final evaluation report.
What costs to include in the offer:
Consultants should include the following costs in their offer’s budget: daily rate, accommodation in the West Nile in the towns of Arua, Adjumani and Kitgum (note that document review should be done from the consultant’s home/Office location, – the program can advise the selected consultant on the cost), and miscellaneous costs.
What costs not to include in the offer:
DRC will pay for and procure the following for the evaluation and therefore the following costs should NOT be included in the offer: pick-up/drop-off of consultant(s) from Kampala, travel inside West Nile for data collection and meetings, interpretation services during meetings with beneficiaries/the DFID partner, printing/photocopying costs, and hiring data collectors/enumerators (if needed).
Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation, so estimates of costs in the offer should be as accurate as possible, as any extra costs incurred by the consultant during the conduct of the evaluation cannot be reimbursed by DRC.
11. CODES OF BEHAVIOR:
The evaluation process will be directed by DRC’s guidelines for the ethical conduct of evaluations and research, guiding the evaluation team through careful consideration of the key ethical implications at every stage of the evaluation. These guidelines will be provided as an attachment to the contract of the successful firm/individual
12. SHARING AND USING FINDINGS:
The DRC International’s Policy on Program Evaluation requires a management response should always be developed as part of the follow up process in regards to the evaluation. The management response can be filled out as part of the de-briefing exercise. The DRC template for Management Response below should be used.
13. DISCLOSURE:
Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of DRC. All data collected as part of this consultancy belongs to DRC and public dissemination of the data and evaluation products can only be done with the written consent of the DRC.
14. EXPRESSIONS OF INTEREST (EOI):
DRC invites EOI from firms, or individuals, with the experience and skills described above. The EOI must include:
A cover letter of no more than 3 pages introducing the evaluator/organization and how the skills and competencies described above are met, with concrete examples. Please also use this cover letter to indicate the consultants’ availability for the proposed period. Note: As stated above (section 7), the evaluation should start as early as possible in April 2015 and the final evaluation report should be submitted to DRC before the end of May 2015;
An outline of no more than 2 pages of the proposed process and key considerations including:
a. Key considerations for this evaluation;
b. Proposed outline methodology for this evaluation;
A CV for the evaluator (s), including 2 referees (with phone number and email address);
A one-page budget of the offer, covering all major anticipated costs (see section 10 above on what costs should be included in the offer);
Two examples of reports from previous evaluations/reviews relevant to this consultancy.
How to apply: