external evaluation plan
SOLIDARITÉS INTERNATIONAL (SI) is a humanitarian NGO which is committed to meeting the vital needs of populations faced with a major crisis of human or natural origin and to strengthening their resilience, particularly with regard to WASH, FSL and S-NFI. Since early 2018, SI has been involved in several programs for rehabilitation of water supply infrastructures in NES. Since 2019, SI has been supported by ICSP to implement infrastructures rehabilitations and support to the Water Departments operating the rehabilitated systems.
Access to clean, safe and sufficient water supply has been one of the significant issues in the areas liberated from ISIS. Before the conflict, the water stations and infrastructure in Ar-Raqqa governorate were achieving a high coverage rate and were providing high quality services. The systems were state-owned and state-managed, requiring costly and technical maintenance and included a range of water infrastructure (irrigation stations, booster pumps, water network, raw water station, treatment stations).
Prior to the conflict, the responsibility of dealing with water resources management lied with a number of ministries, all represented on the Council of General Commission for Water Resource Management (Source: FAO). The Ministry of Irrigation (MOI) was the central institution for managing, developing and protecting the water resources, supervising the investments and the establishments in all water basins and drawing up strategic plans for executing the water policies to achieve sustainable development for water resources. The ministry was responsible for making available suitable water resources for all water using sectors, for controlling drilled wells and for licensing future wells. The Ministry of Housing and Construction (MHC) was responsible for supplying drinking water from surface and underground water resources by building, operating and investing in the water networks and water purification stations as well as building sewage-water networks and treatment plants and enhancing the efficiency of water and sewage networks. Finally, the Ministry of Local Administration and Environment (MLAE) was responsible for monitoring and controlling water quality through its laboratories and observatory networks, for issuing national standards for the protection of water resources and tracking the sources of pollution in order to implement local Environmental Law when existing.
Each Ministry had local bodies (local directorates or local institutions) related to the central body of each Ministry and distributed over the 14 administrative units (governorates). Water departments, previously linked to these local bodies, were in charge of relaying the priorities and implementing the various tasks mentioned above. That system of management was however disrupted with the conflict erupting in 2011 and lasting until this day.
As part of the “terre brûlée” policy of ISIS, these vital infrastructures and the whole water system were severely impacted by fighting to take back the area from ISIS by the coalition and SDF, leading to widespread damage to the physical infrastructure.
While some infrastructure was rehabilitated following the end of the crisis, many of these works focused on quick fixes to ensure basic water supply, and were not focused on ensuring that the stations were able to attain a sustainable level of operation. There continues to be a severe lack in capacity to implement the operation and maintenance (O&M) of assets that are rehabilitated and functional, in addition to the large number of assets still to be rehabilitated.
Practically, there is a lack of spare parts and financial means to pay key teams, particularly as many of the skilled workers in charge of O&M have left the area. Overall, very limited attempts tried to reinforce the capacities of the water departments to take charge of the O&M, which did not necessarily positively affect the functionality and quality of services provided by the rehabilitated stations, posing a threat to the sustainable use of these critical infrastructure.
SI’s proposed action enable water supply in the area to shift from emergency relief assistance to more a sustainable provision of services by the local authorities to communities. This shift also aims at returning water supply to the context of pre-war Syria. Increased service provision for access to water should be creating stability for returned communities, and reinforcing the general perception of political stability in the Ar-Raqqa region presently. This funding also allows a much more integrated and wider scope of assistance targeting the area of northern Ar-Raqqa governorate which has not been targeted by agencies focusing on Ar-Raqqa city and Deir Ez zor.
The overall goal of the project was to restore the production capacities of the water department through the rehabilitation of water infrastructures and support of water departments. The intervention more especially aims at improving water access and water quality for the population of through the rehabilitation of water infrastructure (Ar-Raqqa, Al Hassakeh and Aleppo governorates) and support to water departments (Ar-Raqqa, Deir ez Zor and Al Hassakeh governorates)
Two main outputs are expected for this project :
- Output 1: Needs in terms of drinking water are covered from rehabilitated water management infrastructures
- Output 2: Capacities of local water authorities are reinforced to enable a proper and durable water management linked to drinking water access
The intervention started on 01/09/2019 and will finish on 30/09/2021. The following activities are being conducted:
|O1A1: Rehabilitation of water infrastructures, North of Ar-Raqqa: Zaher WS||Rehabilitation of 1 station which will provide water to an area populated with an estimated 14,122 individuals gathered in 31 villages|
|O1A2: Rehabilitation of water infrastructures, North of Ar-Raqqa: Ain-Issa water station||Rehabilitation of 1 raw water and one treatment station will provide water to an area populated with an estimated 6,562 individuals gathered in 11 villages|
|O1A3: Rehabilitation of water infrastructures, East of Ar-Raqqa: S-Hamiat water station (2 booster stations).||Rehabilitation of 2 booster stations which will provide water to an area of 700 sq. km populated with an estimated 4,987 individuals gathered in 6 villages|
|O1A4: Rehabilitation of water infrastructures, Ar-Raqqa: Qalta water station||Rehabilitation of 1 treatment stations which will provide water to area populated with an estimated 14,755 individuals gathered in 9 villages|
|O1A5: Rehabilitation of water network in Ar Raqqa: 1.4km pipe DN400 in Qalta city||Rehabilitation of 1 treatment stations which will provide water to area populated with an estimated 14,755 individuals gathered in 9 villages|
|O1A6: Rehabilitation of water infrastructures, south Ar-Raqqa: Ekershi||Rehabilitation of 1 treatment stations which will provide water to area populated with an estimated 8,885 individuals gathered in 5 villages|
|O1A7: Rehabilitation of water infrastructure in Ar Raqqa: Ekershi summer pumping station||Rehabilitation of 1 treatment stations which will provide water to area populated with with an estimated 8,885 individuals gathered in 5 villages|
|O1A8: Rehabilitation of water infrastructures, South Ar-Raqqa: Mansoura||Rehabilitation of 1 treatment station which will provide water to area populated with an estimated 14,013 individuals gathered in 1 village|
|O1A9: Rehabilitation of water infrastructures, south Ar-Raqqa: Sefsafa||Rehabilitation of 1 treatment station which will provide water to area populated with an estimated 2,071 individuals gathered in 1 village|
|O1A10: Rehabilitation of water infrastructures, South Ar-Raqqa: Al Jarafat||Rehabilitation of 1 treatment station which will provide water to area populated with an estimated 55,520 individuals gathered in 1 city|
|O1A11: Rehabilitation of water infrastructures, South Ar-Raqqa: Network to Northern Tanks 1.4km pipe DN400 in Tabqa city||Rehabilitation of 1 treatment station which will provide water to area populated with an estimated 55,520 individuals gathered in 1 city|
|O1A12: Rehabilitation of water infrastructures, East Al Hasakeh: Belqis/Tal Hamis||Rehabilitation of 1 treatment station which will provide water to area populated with an estimated 5,954 individuals gathered in 9 villages|
|O1A13: Rehabilitation of water infrastructure in Aleppo, Kobane sub-district: Kobane Complex (Shuyukh raw station, Qaraqi booster station and Al Ethaa distribution station)||Rehabilitation of 3 water stations station which will provide water to area populated with an estimated 100,000 individuals gathered in 30 villages and Kobane city|
|O1A14: Rehabilitation of water infrastructure in Al Hassakeh, Al Himme sub-district: Al Himme Raw Water Station||Rehabilitation of 1 raw water station which will provide water to area populated with an estimated 233,334 individuals gathered in Hassakeh city|
|O1A15: Rehabilitation of water infrastructure in Al Hassakeh, Al Himme sub-district: Aziziyeh Booster Station||Rehabilitation of 1 water booster station which will provide water to area populated with an estimated 140,000 individuals gathered in Hassakeh city|
|O1A16: Rehabilitation of water infrastructure in Ar Raqqa, Raqqa sub-district: Installation of feeder pipes for Raqqa Main Water Station||Rehabilitation of 1 feeder pipe in Raqqa Main Water station which will provide water to area populated with an estimated 14,7435 individuals gathered in Raqqa city|
|O2A1: Provision of operational means to water departments for the adequate operation and maintenance of the water infrastructure||Provision of tools and spare parts for O&M to local water departments for them to be able to operate the fixes required on the water infrastructure|
|O2A2: Provision of equipment and rehabilitation of infrastructure for the Al Himme water analysis laboratory||Provision of equipment and repairs in Al Himme Treatment station laboratory to local water department for them being able to run water testings in the Governorate|
Purpose of evaluation:
After one and a half years of implementation of this program, this evaluation includes:
- Sustainable impact of the intervention (technical review and water department perspective) ;
- Relationship between SI and contractors implementing the works, from the purchase process to the final approval of the works
- Operation of the rehabilitated infrastructures, seen from the operators level
By choosing these topics, SI aims at having an outside perspective on the quality of the intervention conducted by SI and its relationship with water departments, contractors and technical operators of water stations. This evaluation also aims at gathering qualitative feedback on aspects of the intervention that were not the main focus of monitoring activities conducted by SI.
First, this evaluation will focus on the sustainability of the interventionThe objective of the project is to support the recovery of the water systems in North East Syria by rehabilitating the infrastructures and giving means to water departments to sustainably manage these systems. It is therefore expected for the intervention to have a long-term effect on the capacity of the water departments. Monitoring activities conducted by SI mainly focus on the outcome of the intervention by checking if the activities resulted in an improvement of the capacity of the water departments to produce safe water at a specific moment in time. This evaluation should focus on making sure that this increased capacity will lasts and to what extent it is resilient to shocks. Two main aspects should be considered by the evaluation for this point:
- Quality and sustainability of the works implemented by SI from the consultant perspective (technical review)
- Feedback of water departments on SI’s intervention focusing especially on the relationship between SI and water departments during the course of the project. The goal is to ensure that the way SI worked with the water departments ensured that the intervention had a long-term effect (timeliness and quality of the intervention, choice of infrastructures and modality of assistance, identification of needs, overall collaboration, etc.). This should be investigated through the water departments perspective.
Second, the evaluation will focus on the contractors appointed by SI to execute the works, and on their relationship with SI. The purpose will be to collect and synthetise the contractors feedback about they work with SI, from the tender process to the final payment in order to understand how these partnerships can be developed and improved to ensure compliance, transparency and accountability between SI and contractors. The added value of having this evaluation conducted by a third party instead of by SI MEAL team, is to have a neutral party outside of SI gathering this feedback. It is therefore crucial for the evaluator to be able to guarantee some confidentiality to the contractors when interviewing contractors.
Third, the evaluation will focus to the first users of the rehabilitated water infrastructures, meaning technical operators of the water stations. Thus, the evaluators should not only collect the feedbacks of the Water Departments head staffs, but also those of the people in the field.The objective would be to evaluate the improvement technical capacities and working conditions of the technical operators as a result of the intervention. The focus will very much be on the evaluation of the outcome of the activities relating to technical operators’ capacity building.
It is worth noting that as the stations targeted through this intervention are widespread across Ar Raqqa and Hassake governorates. As such, it is recommended to the evaluator to focus on stations located in Tabqa and Raqqa.
Findings and recommendations should take into account contextual constraints and the challenges of implementation to ensure that they can suggest realistic area of improvement and a pragmatic way forward for future programming. Consequently, the evaluation should systematically investigate barriers for activities being assessed as well as potential hindrances for the implementation of recommendations included in the evaluation report.
Evaluation type: ex-post performance evaluations
- Sustainable impact of the intervention :
- Have SI rehabilitation interventions met the technical needs of the WD to properly operate the supported infrastructures? If not, how can this be addressed?
- In which degree SI donations of tools, equipment, spares and chemicals to support WD met their needs and helped them operate their infrastructures? If not, how can this be addressed?
- Have the services provided met expectations of the WD in terms of quality and timeliness?
- How did SI implement relevant pre-evaluation and monitoring of the impact of its activities to the Water Departments? Is this process relevant and how could it be improved?
- Is the Water Department satisfied with the way SI approached and coordinated with the WD and did SI take the WD needs and concerns properly into consideration? Was it efficient? How could it be improved?
- Assessment of the relationship between SI and its contractors:
- How does Si implement the purchase process for rehabilitation works? Is it relevant? How could it be improved?
- Are the technical and non-technical documents provided by SI at the call for tender stage sufficient for the contractor to provide relevant offers to SI?
- How is the selection process understood and accepted by contractors, including rejected contractors? Is this process clear and transparent enough?
- How does SI implement follow up of the works with the contractors? Is it relevant? How could it be improved?
- How does SI can provide support to the contractors in case of need during the works (technical support, amendments, etc)? What is the contractors’ feedback on this point?
- How does SI implement the reception of the works, from the first acceptance to the final payment? What is the feedback of the contractors on this point?
- What is the feedback of the contractors on the invoicing and payment process with SI? How does SI support the contractors in this process?
- Are SI payment deadlines respected from works/contracts perspective?
- Have payment delays a real/huge negative impact on the company management manner or delays are on an acceptable timeline according to Syria situation?
3/ Assessment of the operators feedback
- How SI took into account operators working conditions (quality & security) in the design of the activities?
- How operators working conditions (quality & security) actually improved after SI rehab’ intervention?
- From operators’ side, what expectations remained in terms of working conditions (quality & security) and how could they have been addressed?
- Did the operators working conditions (quality & security) actually improved after SI donations of materials, tools and chemicals during handover period? What is still needed and how could it be addressed?
- Did specific technical trainings for operators was sufficient enough after a water station handover period? Were they accomplished in a constructive way (time/ relevance/ efficiency/ autonomy recovered)? What could be improved and addressed?
- Proposal of intervention and amendments,
- Project monitoring documents,
- Assessment reports, BoQs and technical specifications
- Water station reports, handovers
- Monitoring reports,
- Technical framework and tools,
- Activity internal reports, including pictures
- Purchase files,
- Supply SoPs
Quantitative data collected by SI monitoring unit through its planned activities throughout the project and made available to the consultant.
As such, the consultant will do a secondary review of the available quantitative data and only collect qualitative data through the following activities:
- Site visits, observation in the field, if possible
- Key Informant interviews,
It is expected that the evaluation will be performed by the end of the project (30/09/2021). Its overall duration will be of a maximum of 6 weeks, and the report delivered within a month after the end of the project (31/10/2021)
Proposed indicative schedule for the mission: the final detailed schedule will be proposed by the consultant.
Writing of a report in English including:
Before starting data collection
- 1 inception report (3-5 pages max) outlining planned methodology, key stakeholders to interview, secondary data to review, plan of tools required, detailed workplan, etc.
After data collection
- 1 presentation of the findings and recommendations to SI team
- 1 executive summary (1-2 pages max)
- 1 narrative report (max 20 pages, not including annexes)
- 1 table with the principle conclusions and recommendations (separate short, medium and long term) and the lessons learned.
Technical appendices : Containing the technical details of the evaluations as well as the terms of reference, the questionnaire models, check list and canvas interviews, potential tables or graphs, references and other sources, people and institutions contacted
Please note that both inception and final reports are subject to SI’s approval before they are considered as final deliverables. Time for SI’s review should be included in the evaluation’s timetable.
The evaluation team is expected to demonstrate the following experience and qualifications:
- A solid and diversified experience in monitoring and evaluating humanitarian assistance activities, including evaluation of Water Infrastructure projects is required;
- For the person who will interview water stations’ operators : technical awareness of water infrastructure and rehabilitation works is required;
- Proven technical background of water infrastructure works and rehabilitation works is an advantage;
- Knowledge of monitoring and accountability methodologies and best practices;
- Excellent spoken and written communication skills in English;
- Full working knowledge of Arabic ,
- A proven record of experience in conducting similar evaluations.
Interested candidates should submit:
- A technical offer including:
- An understanding of the issues at stake as explained in the ToR, and description of the proposed approach
- Methodology and proposed tools
- The timetable showing the details for the completion of each of the evaluation phases. The proposed schedule should include time for briefing and debriefing on the mission and as much as possible at SI headquarter (or remotely).
- A financial offer including a detailed budget in columns (fees, other costs)
- An up-to-date CV of all participants within the evaluation team
- Organisation chart of the evaluation team structure
- Samples of previous pieces of work (10 to 20 pages) from at least 2 different assignments
- List of past experiences including description of the project evaluated, area of intervention/location and total budget
Solidarités International will consider individual candidates or teams equally.
The deadline is 30/06/2021.
 Modality of data collection can be adjusted depending on the context relating to COVID-19