External Programme Evaluation - Jobs and Perspectives Programme in Iraq

Contracting authority: Stichting SPARK

Subject: Final Programme Evaluation

Location: Iraq (Erbil and Mosul)

Duration: 6-9 weeks, max 30 days

Application Method: Offers should be sent to (Email at Apply Button)

Expected Start Date: February 15, 2023

‍

Context

SPARK develops higher education and entrepreneurship to empower young, ambitious people to lead their fragile and conflict-affected societies into prosperity. SPARK is a dynamic and growing, international not-for-profit development organisation with 100+ staff members, in more than 14 offices around the world. SPARK supports refugees in the Middle-East by providing them with scholarships in universities and higher education institutions in the region. SPARK also supports young entrepreneurs in fragile states, to start or grow their own businesses.

‍

Programme Background

SPARK has implemented a Jobs and Perspectives project from August 2019 till December 2022. Jobs and Perspectives (J&P) Iraq is an innovative and sustainable programme funded by the Ministry of Foreign Affairs of the Netherlands. The main aim of the programme is to improve the economic inclusion of vulnerable youth, especially women, by preparing them for the job market and employment through providing market-relevant education and job placements. Our approach is based on creating a pathway for the youth journey from education to decent employment or starting and growing their own businesses.

‍

Programme Main Components

• Supporting structural reforms and innovations in the Iraqi Higher Education System to bridge the gap between education and labour market:
• Facilitating Bologna adaptation process in the Iraq Higher Education System with the approval and support of both Ministries of Higher Education (MOHESR of Federal Iraq and KR-I). - Improvement/development of 6 curricula of the selected main courses to match labour market needs (including teaching materials which were specifically adapted to the international academic standards, gender needs, local context), establishment of 6 laboratories and ToTs for 30 Iraqi professors for the new course materials - Enrolment of 2335 students to the improved main courses - Introduction of e-learning tools as well as labour-market oriented curricula improvement tools - Supporting gender units and Career Centres of the universities to develop their policies to advance gender equality and women’s advancement in Iraq universities as well as refugee/IDP inclusion.
• Ensuring market relevant internships/job placements to Increase employability of the students in the labour marker
• Development of 6 TVET courses to be delivered under the Career Centres as short courses
• Scholarship provision for 800 students for those TVET short courses
• Innovation Hub activities in the Career Centres where the three main stakeholders (universities, companies and students) can come together for information sharing, research and solutions.
• Remote and offline internship placements (168) including the introduction of the remote internship model to the Career Centres of the Iraqi universities to make their current and future students more competitive candidates for the future digital job market. .
• SME support for 20 SMEs to respond to the Covid-19 crisis

‍

Programme Partners

• Erbil Polytechnic University
• University of Mosul
• American University of Iraq, Sulaimani the Centre for Gender and Development Studies
• Hanze Universities of Applied Sciences Groningen
• University of Twente
• University of Antwerp
• Yasar University
• The Station Foundation

‍

Objectives of the Final Evaluation

SPARK seeks an independent expert/, TPM or research company for a final evaluation of its Jobs and Perspective Program (J&P) implemented in Iraq (Erbil and Mosul) to understand the extent to which the project has contributed to improve prospects for youth in Iraq in the field of education, entrepreneurship and access to decent jobs. The expert/s will assess the progress made towards the achievement of a program’s objectives covering the period from August 2019 till December 2022.

• Assess the overall impact of the project based on the evidence-based outcomes whether intended (according to the project’s logical framework) or unintended outcomes and determine SPARK’s contribution to identified changes.
• To evaluate the relevance, effectiveness, efficiency, sustainability and impact of each component of the programme as per the Organisation for Economic Co-operation and Development- Development Assistance Committee (OECD-DAC) evaluation criteria;
• To identify lessons learned from the programme’s implementation and
• To formulate recommendations for similar future programming.

‍

Key Questions of the Final Evaluation

The analysis will be focused on the following evaluation questions organised by criteria. The study is expected to prioritise collecting feedback from direct and indirect participants:

Relevance: Extent to which the intervention objectives and design respond to beneficiaries.

1. To what extent have target groups and possibly other relevant interest groups/stakeholders been involved in the planning/implementation process?
2. Was the project relevant to the needs of its target population and institutional beneficiaries?
3. Were the project activities and outputs consistent with the intended outcomes and objective?
4. What factors have contributed in achievements or hindrances of the output or outcome?
5. Were the strategies adopted, applied tools, and inputs identified realistic, appropriate, and adequate for achievements of results?
6. Was the project aligned with and supportive of SPARK Theory of Change?

Efficiency: Extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way.

1. Was the project implemented in the most cost-efficient way compared to alternative means of implementation? Specifically did the actual results (output/outcome) justify the cost incurred.
2. How well have resources (funds, expertise, time) been converted into results in the current context of the project without compromising the quality of the resultant output and outcome?
3. To what extent were activities implemented as scheduled?
4. Were there any specific enablers/ challenges that affect the successful implementation of the project? How was it managed?

Effectiveness: Extent to which the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups.

1. Have the project outputs and outcomes been achieved in accordance with the stated plans?
2. What observed changes in attitudes, capacities and institutions can be causally linked to the project’s interventions?
3. How effectively were the project results monitored?
4. To what extent has the project adapted to contextual challenges and/or changing external conditions (e.g. Covid-19 pandemic) in order to ensure project outcomes?
5. To what extent have there been any unplanned positive or negative effects in relation to anticipated results of the project interventions?
6. Have there been any factors and/or obstacles that prevented beneficiaries and project partners from accessing the results/services/products? If yes, has the project been successful in addressing them, and how did they affect the overall effectiveness of the project?
7. What could have been done differently (design and implementation approaches) to make sure short and long terms target results are reached?

Impact: Extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.

1. What impact did the project have on the beneficiaries and what are the key project intervention and delivery strategies that contributed to the observed impact if any?
2. Did the impact come from the project activities, from external factors or from both?
3. Which positive/negative and intended/unintended effects have been produced by the project?
4. Did the project take timely measures for mitigating any unplanned, negative and/or unintended impacts?

Sustainability: Extent to which the net benefits of the intervention continue, or are likely to continue.

1. Are structures, resources and processes in place to ensure that benefits generated by the project continue once external support ceases?
2. Was the project supported by local institutions and well-integrated into local social, cultural and political structures in the targeted country/ countries?
3. Do the project partners have the financial capacity and are they committed to maintaining the benefits of the project in the long run?
4. To what extent are the project results likely to be sustained in the long-term?
5. Are there signs that the project results and activities will be scaled up, replicated or continued by project partners/stakeholders?
6. What should/could have been done differently to better guarantee sustainability, if applicable?

Cross-Cutting: Cross-cutting themes include gender, non-discrimination, human rights, accountability, communication, visibility, and success stories.

1. To what extent have cross-cutting issues such as gender, non-discrimination and human rights been integrated in the project design and implementation?
2. To what extent were gender mainstreaming issues considered in design and implementation?
3. Were any barriers to equal gender participation identified in design or implementation, and was anything done to address these barriers?
4. Did the project have any feedback mechanisms and to what degree was the project accountable to beneficiaries?
5. Have the communication and visibility actions been implemented in an appropriate manner?
6. Are there any project success stories generated from the communication/ visibility actions?

Best Practices & Lessons Learnt:

1. What are the major achievements contributable to the project?
2. What best practices can be identified to inform future programming?
3. What lessons can be drawn from this project to inform future programming?

‍

Methodology and Data Collection Tools

The final evaluation will utilize a mixed method approach consisting of both qualitative and quantitative primary and secondary data collections. The evaluation should triangulate data from multiple sources and stakeholders in order to infer reliable findings.

Desk Review: The consultant(s) will review key project documents including the proposal, the mid-term evaluation report, logical framework, and set meetings with key personnel to better understand the project, its goal and activities. In addition, the consultant(s) will utilize primary available M&E data that was formerly collected by M&E staff including activity surveys, assessments, project annual reports, etc.

Data collection tools and approach: the consultant(s) will develop quantitative and qualitative data collection tools and collect data in the form of surveys, interviews, focus group discussions (FGDs), and possibly case studies using a solid sampling approach. To ensure a better understanding of the key issues that will be addressed in the project, the data will be disaggregated by age, gender, and residency status (host community member, IDP and refugee), persons with disabilities as well as by geographical location.

Sampling strategy and approach: The consultant(s) shall determine the appropriate sample size and approach in consultation with SPARK ME&R staff (country-based staff and regional staff). The sampling strategy will take into consideration the activities carried out, target population and the project areas.

‍

Time Schedule of the Final Programme Evaluation

The evaluation is expected to begin on the15th of February 2023 and end on the 15th of May 2023. This includes preparation, evaluation design, field assessments in the programme’s areas, report writing, presentation of findings, lessons learned and recommendations to SPARK and local partner organizations. .

‍‍

Main Tasks and Responsibilities

• Development of work plan(s) and timeline of evaluation activities
• Review of existing J&P Programme documentation including assessments, curriculums, deliverables, mid-rem evaluation reports and annual progress reports, etc.
• Meetings and/or interviews with SPARK Programme Staff and Monitoring & Evaluation staff, local partners and beneficiaries (Remote OK)
• Analysis of gathered data through reviews, meetings and/or interviews
• Development of draft evaluation report and submission to SPARK Programme and M&E teams
• Processing of feedback on draft report from SPARK Programme and M&E teams
• Finalisation of mid-term evaluation report and submission to SPARK Programme and M&E teams
• Presentation of the key findings

‍

Deliverables

Following deliverables for this evaluation will be produced:

1. A detailed work plan and timeline which will be approved by the Programme Management.
2. A short report based on the initial results along with sets of recommendations per component that can be implemented in the short run.
3. Draft and final report, which will have the following structure:

• Executive summary
• Introduction
• Methodology
• Analysis and findings per activity
• Conclusions of findings, lessons learned and recommendations per activity
• Annexes: Relevant maps and photographs of the evaluation areas, desktop review references, data collection tools, list of interviewees and anonymized participants in focus group discussions and raw data in an agreed-upon format

1. Presentation of key findings
2. Tools developed for this assignment
3. Raw data gathered from the evaluation exercise

The final Evaluation Report will be written in English, in PDF and Word format and will be published on the SPARK website.

‍

Requirements and Skills

The evaluator(s) should be (a team of) experienced and independent consultant(s), freelance or from a consultancy organisation, with at least the following expertise:

• University degree in International Development, Social Sciences, Higher Education or/and other related fields
• A minimum of 7 years of professional experience with conducting programme/project evaluations in INGO sector.
• Proven experience in developing external assessment reports within the MENA region
• Previous professional experience in the INGO domain in Iraq will be considered an advantage
• Advanced communication, analytical and reporting skills
• Fluent in English (both reading and writing)
• Fluency in Arabic and/or Kurdish will be considered an advantage

‍

SPARK Offers

The payments for the Mid-Term Programme Evaluation will be done by SPARK, according to the following scheme:

• %35 of total contracting sum upon signing of the agreement
• %65 of total contracting sum upon approval of final Programme Evaluation Report

‍

Selection Criteria

The tender evaluation committee will evaluate and award the contract on the basis of the following selection criteria:

• Price

Criteria weight is 50 %

Formula: (Best Tenderer Price/Evaluated Tenderer Price)

Price shall be quoted in USD

• Quality

Criteria weight is 50%

Point evaluation (weighted points): Criterion will obtain between 0 and 5-point, 5 point being the maximum, 0 points the minimum and the points obtained will be multiplied by criterion weight.

‍

Score Benchmark for Services/Works

PIN expert evaluation committee will assign up to 5 points for the best quality of the sample, based on the following evaluation grid.

• 5: Excellent response with no weaknesses shown and exceeds the requirement - also provides comprehensive, detailed, and convincing assurances that the services will be delivered to an excellent standard
• 4: A very response that demonstrates real understanding and fully meets the requirements - offers assurances that the service delivered will be of a high standard
• 3: A satisfactory response which demonstrates a reasonable understanding of the requirements and gives reasonable assurance of delivery of services to an adequate standard, but does not provide sufficiently convincing assurance to be able to award them a higher mark
• 2: A response where reservations exist - lacks full credibility/convincing detail, and there is a significant risk that the response will not deliver/be successful
• 1: A response where serious reservations exist - the may be because, for example, insufficient detail is provided and the response has fundamental flaws, or seriously lacks credibility with a high risk of non-delivery
• 0: Response completely fails to address the criterion under consideration

‍

Those bidders who get less than 3 for the capacity of the bidder shall not be accepted or considered and their offer will be rejected.

For each bid, points from all evaluation criteria will be added up and the winning bid will be the bid with the highest number of points. In case of equal number of points, the winning bid will be the one with the lowest price.

‍

How to Apply

The deadline for submission of applications is February 5, 2023, by 17:00 Iraq time.

All applications should include the following:

• Technical Proposal highlighting related work with similar projects
• CV’s of all evaluation members
• Financial Proposal which should include consultancy fees that include VAT and logistical costs, with a breakdown of days and fees

If you have any questions please contact SPARK, via email: (Email at Apply Button)

Please note that incomplete applications will not be considered.

‍

Offers should be sent to (Email at Apply Button)

‍