Terms of Reference

Evaluation of the work in the country projects Malawi and Mozambique

Project title:

Promotion of integrated adult education systems in Africa (in general)

Countries:

Malawi and Mozambique (sub – projects of the overall continental project)

Project number:

PN 2022.2607.4/DVV0005

Funding agency:

BMZ

1 Introduction

DVV International is the Institute for International Cooperation of the Deutscher Volkshochschul-Verband e.V. (DVV), the German Adult Education Association. DVV represents the interests of approximately 920 adult education centres (Volkshochschulen) and their state associations, the largest service provider of continuing education in Germany. As the leading professional organisation in the field of adult education and development cooperation, DVV International has committed itself to supporting lifelong learning for 50 years now. DVV International provides worldwide support for the establishment and development of sustainable structures for a lifelong Adult Learning and Education (ALE).

Funded in a large part by the BMZ for the creation/strengthening of social structures (Sozialstruktur-förderung - SSF), the Institute cooperates with more than 200 partners in more than 30 countries in Africa, Asia, Latin America and Europe. DVV International promotes, together with national, regional and global associations of adult education (AE), lobbying and advocacy activities for the human right to education and lifelong learning. In doing so, DVV International contributes to the United Nations Sustainable Development Goals (SDGs), the Global Agenda for Education (Education 2030) and the UNESCO International Conferences on Adult Education (CONFINTEA).

DVV International supports the exchange of information and interaction between professionals at European and global level through conferences, seminars and publications. DVV International's interventions cover three levels: first level – contribution to policies, strategies, standards and norms, as well as funding on national level (macro level), second level – institutional capacity building and capacity building of teaching staff (meso level), and, third level - the basic model interventions for the population with potential for replication at national, regional, and international level (micro level).

2 Background of the project

After a certain history of cooperation between DVV International and its partner countries including Mozambique (cooperation since 2008) and Malawi (where the cooperation took off only in 2017), the donor (BMZ) and DVV International agreed on the formulation of a new project with new timelines.

The projects to be evaluated are part of a continental project which comprises 11 country projects and a continental project (a digital platform for information, exchange and (blended) learning). They are embedded in a timeframe of three 3- year -long phases, with the first-one from 2020 to 2022.

3 Objectives of the programme

In general, the general objectives of the work of DVV International in the context of creation, respectively strengthening of social structures are as follows:

The technical and institutional capacities of the cooperation partners for carrying out practical activities and for the management of a system are reinforced (macro level);
The inclusion of adult education in policies, poverty reduction strategies and local and national budgets has increased (meso level).
Adult education offers are improved and multiplied (micro level);

Yet, more specifically, the projects to be evaluated work under one general overall objective on continental level:

Overall objective 9 years: An adequately resourced, professionally managed adult education system across different sectors provides good quality services to youth and adults in a demand-driven manner, thus contributing to the SDGs.

Moreover, each country project has an even more specific objective for each of the 3 – year phases:

Overall project objective phase 2 (2023 – 2025) for Malawi: The launch of the National Adult Education Strategy strengthens the coordination and implementation of integrated and cross-sectoral adult education in Malawi and contributes to the achievement of the SDGs

Overall project objective phase 2 (2023 – 2025) for Mozambique: Adult education is strengthened through improved funding, management, and coordination, as well as the expansion of innovative approaches to contribute to the achievement of the SDGs.

Project concept Southern Africa region

The continental project proposal was elaborated by DVV International in a participatory way with its regional and country offices and approved by BMZ. It had an introductory chapter for each region to contextualize approach and corresponding log-frames. Here the relevant part for the evaluation:

The continuation proposal comprises three sub-projects in Southern Africa within the framework of a differentiated planning arc per sub-project with a total duration of nine years until 2028. The projects are managed by country offices in Mozambique and South Africa as well as a regional office in Malawi, which steers the work in all three countries.

In the first phase (2020-2022), in line with the System Building Approach (ALESBA) developed by DVV International, the Adult Learning System (ALE) in all three countries was analysed in a participatory manner and system building was supported on a country-specific basis. Due to the Covid-19 pandemic, this could not be done in the required depth everywhere and will require follow-up work. However, the main parameters of the results to be achieved in phase 2 in terms of the medium-term overall objective were identified.

The common denominator in all three countries is the work on the following key issues:

Lobbying for the recognition of a multi-sectoral approach to ALE that includes literacy, including the development of a strategy to mobilize resources for ALE at the macro level , as ALE is critically underfunded in all countries;
Capacity development for strategic individuals and institutions in ALE at the meso level , with a key technical area being the respective management information systems for monitoring ALE;
Providing evidence-based examples of micro-level ALE offerings and approaches that respond to the respective strategic national ALE frameworks, particularly for vulnerable and marginalized groups and communities with demonstrated development ambitions and initiative.

National approaches will be discussed and further developed in terms of cross-cutting themes and methodologies both in regional exchanges and in collaboration with the continental sub-project, including through the use of the MOJA digital platform.

In addition, the country-specific focus will be on the following aspects:

In Malawi, the focus will be on supporting the ALE policy developed in the last phase. The project will support the national ALE thematic group. Another key element is the implementation of the first university course in Adult Education and Development at the Catholic University, which will start before the end of 2022.
In Mozambique, there is a focus on the integrated program for ALE - developed with DVV International and recognized in the Strategic Education Plan 2020-2029. The work focuses on its application in more and more regions. This involves other ministries at all levels in addition to the Ministry of Education.

4 Activities of the programme

With the objective to sustainably develop social structures for adult education, DVV International’s programs aim to have an impact on three levels (micro, meso and macro) in accordance with the above-mentioned objectives. The indicators to measure the achievement of the objectives are part of the log-frame (attached to these ToR). The project proposal is also accompanied by a catalogue of activities. The choice of action areas depends of prior project activities and the continuous dialogue between all stakeholders of the respective country projects.

5 Evaluation goal and objectives

DVV International understands itself as a learning organization. In order to improve further project activities, DVV International, as the implementing agency, would like to learn more about achievements, challenges and best practices of the above-mentioned project.

The evaluation’s main goal is to document project achievements towards the objectives set in the initial project proposal. Furthermore, the evaluation should review and assess the relevance of the project activities, the achievement of the expected results, and their perspectives for sustainability. It should include a reflection about lessons learnt and make recommendations for the project counterparts regarding methodologies, strategies and approaches for possible further activities. The evaluation is aimed at revealing problems and barriers in the project implementation, as well as outlining how they were overcome. The objectives and impacts will be evaluated on the basis of the information laid out in the Logical Framework Matrix.

The evaluation and its recommendations shall be a useful tool of information for the project implementing agency – DVV International and its partners [1] .

6 Main Evaluation questions

As a learning evaluation, the work should take into account

the overall project logic of the 9 – year project for the continent as proposed to BMZ and approved;
the observations and recommendations of the prior evaluations 2021;
The system diagnosis according to the ALESBA approach done in 2021 and 2024;
the impressions gathered during the investigations in 2024;
additional questions and suggested ideas for the further work (see table below in green).

In this context, Criteria for Evaluation Development Assistance of the DAC of the OECD serve as a guide. The evaluation should give clear statements regarding the following categories:

6.1 Relevance

The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor. In evaluating the relevance of a programme or a project, it is useful to consider the following questions (compare also chapter 6.6.):

To what extent are the objectives of the programme still valid?
Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
Are the activities and outputs of the programme consistent with the intended impacts and effects?

6.2 Effectiveness

A measure of the extent to which an aid activity attains its objectives. In evaluating the effectiveness of a programme or a project, it is useful to consider the following questions:

To what extent were the objectives achieved / are likely to be achieved?
What were the major factors influencing the achievement or non-achievement of the objectives?

6.3 Efficiency

Efficiency measures the outputs -- qualitative and quantitative -- in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted. When evaluating the efficiency of a programme or a project, it is useful to consider the following questions:

Were activities cost-efficient?
Were objectives achieved on time?
Was the programme or project implemented in the most efficient way compared to alternatives?

6.4 Impact

The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions. When evaluating the impact of a programme or a project, it is useful to consider the following questions:

What has happened as a result of the programme or project?
What real difference has the activity made to the beneficiaries?
How many people have been affected?

6.5 Sustainability

Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable. When evaluating the sustainability of a programme or a project, it is useful to consider the following questions (compare also chapter 6.6.):

To what extent did the benefits of a programme or project continue after donor funding ceased?
What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?

6.6 Additional evaluation questions

With regard to chapter 6.1. on „Relevance“ and specific reference to the macro level: To which extent do the governmental partners see the relevance of ALE as a pillar of their education and training systems in the light of the existing challenges in their formal education and training systems?

With regard to chapter 6.6. on „Sustainability“ and specific reference to the macro level: As a pre-requisite for sustainability: Which level of ownership is measurable in the responsible governmental partner-structures in terms of
Associating enough qualified staff to the respective units?
Mobilizing resources within the governmental channels, and
Supporting the process of resource mobilization from funding partners, technical partners (locally, internationally, govern-mental and non-governmental?

With regard to specific technical aspects in the country projects in Malawi respectively Mozambique:
Can further organizational and institutional development of NACLAE (Malawi) be a road towards sus-tainability and a tool for cross-sectoral pooling of resources towards an integrated system for ALE?
Comment on the quality and sustainability of the management structures of Community Literacy Fora (CLF) in Mozambique and Community Learning Centres (CLCs) in Malawi
To which extent can the users of more qualified ALE personnel (government agencies and NGOs) fund or co-fund the upgrading of their knowledge and skills? (exemplary case: CUNIMA / Malawi)

7 Evaluation approach and methods

The evaluation will consist of both desk and field study. A combination of primary data collection and secondary data review is expected during the evaluation.

Project proposal
Interim project reports
Mid-term and other kinds of (self)evaluations referring to the project
Partner documents of project implementation (as far as available)
Etc.

It should be checked if standardized quantitative approaches might be applicable.

8 Indicative Timetable and work input

The maximum number of working days foreseen is 29 days (3 days for preparation including inception report, 3 days for the workshop to present and discuss findings and 3 days for reporting plus 20 days for field work (10 for Malawi and 10 for Mozambique including travel days). The field phase could be covered by two persons, e.g. one for Malawi and one for Mozambique, in this case, up to 5 more days can be granted for the double - coverage of the workshop to discuss the findings and to coordinate planning and reporting.

The contractor to make alternative suggestions as long as the overall time - budget line is not exceeded.

The indicative timetable for the evaluation is as follows:

9 Target group of the evaluation results

The target group of the evaluation results will be the project management team of DVV International country and regional office as well as the head office in Bonn. The results of the evaluation will also be sent to the Contracting Authority (BMZ). Furthermore, the results of the evaluation will also be presented to the key partners of the two projects in the two countries.

10 Right of use

The principal has the exclusive and unlimited right to use the results fulfilled by the contract. The transfer of this Right of Use is complete with the payment made to the Contractor.

The Contractor has Right of Use permission strictly limited for personal non-commercial purposes. Transfer of the Right of Use to third parties is excluded.

11 Contractual Conditions

The legal basis for this contract is provided by the „General terms and conditions for the implementation of Services" (VOL/B). These general terms and conditions are not attached to this contract; however, an exemplary service contract is part of these Terms of Reference for perusal.

12 Tasks of the evaluator/ evaluation team

The evaluation team will be requested to fulfil the following tasks:

Methodological design and planning of the evaluation in communication with DVV International;
Desk Study of relevant project documentation (application and logframe, annual plans and reports, other monitoring documents and project reports, teaching and PR material);
Design of data collection instruments for the field phase;
Prepare an Inception Report;
Conduct the field phase with participation and involvement of all relevant stakeholders;
Compile preliminary evaluation report, integrate comments, and finalize the report.
De-briefing sessions, possibly in country of evaluation and/or Bonn, Germany

13 Deliverables

The following deliverables are expected:

Inception report : The expert/ team of experts will present a report giving detailed information about methods and specified questions (evaluation matrix); all data collection tools need to be presented beforehand. In case of applying quantitative data collection methods, a pre-test will be required. This report will be presented to DVV International; DVV International will provide feedback in due time and may request amendments. The Inception Report needs to be approved by DVV International before the data collection commences.
Debriefing in the country : By arrangement with the DVV International programme coordinator, the (team of) expert(s) will present the preliminary, country-specific results of the evaluation and will answer questions in this regard, documenting the discussion, which must form part of the final report.
Draft evaluation report : DVV International will receive a first draft of the final report, which will be examined on the basis of the DAC Evaluation Quality Standards; on this basis, comments will be made on the rough draft before it is returned to the expert(s).
Final evaluation report : The final report will take into account the comments on the rough draft about the entire evaluation process and all results and recommendations.

The evaluation report should be produced in English (part concerning Malawi) and Portuguese (part concer-ning Mozambique) with an executive summary in both languages and submitted to DVV International electronically. A template for the final report can be discussed in the inception phase with the contractor.

14 Responsible persons at DVV International and logistics

DVV International will be the contracting authority for this evaluation. The main contact persons will be

Maja Avramovska (Senior Manager on Monitoring and Evaluation at DVV International Headquarters in Bonn) e-mail address: avramovska@dvv-international.de
Dr. Thekla Kelbert (Senior Desk Officer for Southern Africa at DVV International Headquarters in Bonn) e-mail address: kelbert@dvv-international.de
Gerhard Quincke, Regional Director for Southern Africa, e-mail address: quincke@dvv-international.de
José Mucuapa, Country Director, DVV International Mozambique, e-mail address: jmucuapa@dvv-international.org.mz
Elisabeth Chiwewe, Senior Program Officer, DVV International Malawi, e-mail address: Chiwewe@dvv-international.mw

The DVV country office will provide all relevant documents and support the evaluator/ team of evaluators in the field with arranging interviews and field visits etc.

The final report will be accepted by the senior manager for monitoring and evaluation and the senior desk officer for southern Africa of DVV International in Bonn.

15 Evaluator eligibility criteria

The evaluation will be done by an evaluator (or a team) needing to demonstrate experience in evaluating development projects. The evaluation expert(s) will further need experience with organisational develop-ment and capacity building. Previous experience in Malawi and in Mozambique is indispensable and active command of Portuguese and of Chichewa will be considered as an asset. Further criteria include:

at least 5-year experience in the field of evaluation of development projects (please, provide proof of this experience);
proven sound knowledge of evaluation methods and instruments;
being (an) experienced professional(s) in the field of adult education,
experiences with stakeholder participation and transparent knowledge management;
analytical and writing skills;
sector and/or development-related exposure in the region of Southern Africa and or Eastern Africa will be considered as an asset.

16 Quality standards

The candidates should propose a suitable evaluation design and methodology, well adapted to the project objectives and TOR for the evaluation, in order to meet the evaluation’s goal and objectives.

The evaluation must comply with the following standards and guidelines:

OECD-DAC Principles for Evaluation of Development Assistance [2]
Standards of the DeGEval – Gesellschaft für Evaluation Standards für Evaluation [3]

17 Coordination/Responsibility

The Senior Desk Officer for Southern Africa as well as the Senior Desk Officer for Monitoring and Evaluation at DVV International Headquarters in Bonn will be in charge of the contracting. The evaluator should work in close cooperation with DVV International Regional Director for Southern Africa, the Country Director in Mozambique and the Senior Programme Officer in Malawi during the preparatory and implementation phase, who will also handle the cooperation regarding the exchange of relevant project documentation and data. Logistical support will be offered to the evaluator by the respective project teams (DVV International country office staff), including booking of accommodation, local transportation, translation and assistance in arranging interviews and project visits.

18 Estimated Budget

The estimated cost of the assignment is in function of the no of working days indicated in chapter no. 8 and shall include honorarium (with taxes), per diems, travel and transportation costs of the evaluator (team).

19 Application package

Experts participating in the tender for this evaluation should send the application package by latest 04.05.2024 by e-mail to:

Maja Avramovska (Senior Manager on Monitoring and Evaluation at DVV International Headquarters in Bonn) e-mail address: avramovska@dvv-international.de

Dr. Thekla Kelbert (Senior Desk Officer for Southern Africa at DVV International Headquarters in Bonn) e-mail address: kelbert@dvv-international.de

Gerhard Quincke, Regional Director for Southern Africa, e-mail address: quincke@dvv-international.de

José Mucuapa, Country Director, DVV International Mozambique, e-mail address: jmucuapa@dvv-international.org.mz

Elisabeth Chiwewe, Senior Program Officer, DVV International Malawi, e-mail address: Chiwewe@dvv-international.mw

Late submissions will not be accepted.

The application package should include the following documents. Please make sure to comply to the provided requirements below and use the annexed templates (or download them from () :

All interested candidates should submit the following documents:

CV(s) of person(s) to be involved in the assignment in English (Please use the template provided);

Technical proposal covering the following aspects ( maximum of 5 pages );
- Narrative overview displaying an understanding of the task
- Detailed methodology
- Work Plan, including detailed proposed steps of implementation
- Consideration of any specific requirements envisaged for the evaluation
- Consideration of the specific context (country, topic)
- Internal Quality Management

A detailed financial proposal with all expected costs (Please use the template provided and submit with a hand-written signature ).

20 Evaluation criteria of proposals

The proposals will be assessed according to the following criteria:

Qualifications of the evaluation team ( counts one third ) (based on criteria mentioned above);
Technical proposal (counts one third) (based on criteria mentioned above; all of the above mentioned criteria for technical proposals will be taken into account) ;
Financial proposal (counts one third) (best cost-service relation; please consider the envisaged total number of working days).

21 Request for clarification

If you should have questions concerning this tender, please send your questions to the 5 persons mentioned as contact person, above. The latest date for submitting questions is the 12 th April 2024. The answers to the questions will be published on our website () for all interested parties the following day.

[1] The work of the project during its first phase in the 3-phase cycle (meaning between 2020 and 2022) had been evalua- ted by Naturalive Consulting for Malawi and by RL CONSULTORES for Mozambique.