Teaching at the Right Level Africa (Catch Up) Zambia RCT 2024 Endline: Terms of Reference

Posted on Opportunities

 Summary

Title Data Collection for Catch Up (CU) and Catch Up Continuing Professional Development (CU CPD) Randomized Controlled Trial (RCT) Endline 2024
Purpose The selected research firm will conduct endline data collection in 273 schools to measure student learning levels and program outcomes for a three-arm RCT studying the impact of the Catch Up and Catch Up CPD programs.
Budget To be submitted as part of the application
Duration 6 months
Start and end date August 01, 2024 – January 20, 2025

 

Note: Data collection must end by October 11, 2024

Geographical boundaries Zambia: 182 zones across Western Province, Central Province, and Itezhi-Tezhi District (in Southern Province)
Report to The Teaching at the Right Level (TaRL) Africa and VVOB

●      Principal Investigator (PI),

●      Research and Learning Advisor, and

●      Senior Research and Learning Coordinator

 

Background

The purpose of this announcement is to solicit proposal requests for the engagement of a qualified research firm for the provision of professional research services to further advance the objectives of Teaching at the Right Level (TaRL) Africa and VVOB - education for development.

Teaching at the Right Level Africa’s objective is to help children develop basic reading and mathematics skills, opening doors to a brighter future. TaRL Africa is an independent organization formed through a partnership between Pratham, an education NGO, and the Abdul-Latif Jameel Poverty Action Lab (J-PAL), a research center based at the Massachusetts Institute of Technology (MIT). These two organizations have been working together to create cost-effective, scalable literacy and numeracy interventions for more than 15 years. The fruits of these efforts have been the development of what is now referred to as the TaRL approach.

VVOB - education for development is a non-profit organization and was founded in 1982. Commissioned by the Flemish and Belgian governments they contribute to the quality of education in developing countries. VVOB ensures the improvement of the quality of education through capacity building of government education actors and other education actors in the global South; by connecting these actors in the global South with each other and connecting actors in the North with actors in the South; and by inspiring education actors in Belgium and the global South on equity-related issues in education, based on worldwide research, experiences and expertise.

TaRL Africa and VVOB have partnered with the Ministry of Education in Zambia to deliver the TaRL program in the country. The TaRL approach works by dividing children in upper primary into groups based on learning levels rather than age or grade. In Zambia, the TaRL approach is locally known as “Catch Up” and was piloted by the Zambian Ministry of Education (MoE) in 2016-17. As of 2024, Catch Up is being implemented by MoE in 5,315 primary schools in 9 out of the 10 provinces in Zambia, with support from VVOB, TaRL Africa, and other partners.

 

Learning Objectives

Lessons from the evaluation will inform future Catch Up scale-up plans. The RCT has four main learning goals:

  1. Investigate and compare the effectiveness of the Catch Up and Catch Up CPD models in their ability to improve student learning, including for relevant student subgroups (e.g., girls vs. boys and students with different baseline learning levels).
  2. Assess Catch Up and Catch Up CPD functionality and scalability (including implementation fidelity and take-up).
  3. Investigate program impacts on potential mediators that enable student learning (e.g., instructional quality and student engagement).
  4. Diagnose contextual constraints and needs and identify potential solutions to these.

 

TaRL Africa Research Team

This study is designed by a TaRL Africa/VVOB-appointed Principal Investigator (PI) and will be overseen by the PI and supporting TaRL Africa/VVOB Research Team members. We refer to this group as the TaRL Africa/VVOB Research Team.

RCT Study Design

The TaRL Africa/VVOB Research Team has engaged with external academics to conduct a randomized controlled trial (RCT) that evaluates the impact of the Catch Up model, as well as of a new model that is referred to as “Catch Up Continuing Professional Development”. The latter consists of the normal Catch Up model, with an additional teacher continuous professional development (CPD) component. The RCT assigned zones and their schools into different experimental arms, with (1) a control arm of 91 zones that do not receive either program variant and continue with business-as-usual (control zones); (2) an arm of 91 zones that receive the normal Catch Up model (Catch Up treatment zones); and (3) within each of the 91 treatment zones, an arm of select 91 schools that receive Catch Up with additional CPD (Catch Up CPD treatment schools). All schools in treatment zones are receiving their respective intervention (and all schools in control zones are not), with 91 schools in each of the 3 arms (control, Catch Up treatment, or Catch Up CPD treatment) sampled for this study’s data collection activities. In total, the survey sample is 273 schools.

 

The Assignment

These Terms of Reference are for an endline survey data collection in the 273 sampled schools identified by TaRL Africa and VVOB. To avoid attrition, the research firm must conduct the endline data collection exercise in all of these 273 schools, with no room to replace schools, drop schools, or obtain a non-response.

VVOB and TaRL Africa seek a research firm to carry out the activities listed below. All data will need to account for unique student IDs that also allow for the individual tracking of students across the full study period. The data collection instruments must be administered in the official language of instruction of each given primary school, covering four Zambian languages overall (Icibemba, Silozi, Nyanja/Chewa, or Chitonga).

During this round of announced visits to all study schools, the research firm will be expected to:

    1. Contact all schools to inform them on the purpose of the visit, timelines and what support will be needed to ensure all learners are in school on the days of visit.
    2. Visit and conduct surveys in all the schools (100%); that is, all 273 schools surveyed at baseline.
    3. Conduct student interviews, consisting of two parts: a student assessment and survey questions, with all students who were interviewed at baseline in each of the 273 schools. This is an average of 30 students per school, with a minimum of 8 and a maximum of 40 students. The wide range of the number of students in a school is because some schools have only one stream and others have many streams. The overall sample size of students being tracked is 8,025.
  1. Note that the student interview, which includes a short assessment, will take approximately 45 minutes per student.
  2. The firm will be expected to track these students individually. This may require revisits and/or home visits for follow-up exercises to the school and community to complete the student interview.
    1. Collect student administrative records, including data on class attendance, dropout rates, and the national exam center data for each school.

The research firm will carry out the following tasks:

  • Apply/update clearance from the Zambia Ethical Institutional Review Board (IRB) and other research authorities.
  • Obtain a government-issued permission letter to enter schools and administer the surveys. This includes Province and District level government permissions where applicable.
  • Update and use an already-coded electronic survey program that ensures data security, is easy to use, and offers offline capabilities (using SurveyCTO).
  • Recruit, hire, and train a set of enumerators well-versed in primary school education activities (preferably current or former primary school teachers).
    • Enumerators will have to be native speakers of Icibemba, Silozi, Nyanja/Chewa, or Chitonga (all surveys and data collection must be conducted in the school’s official language of instruction.
  • Note that the students who speak these languages, as per the baseline, are: Chitonga (28.2%), Cinyanja (0.4%), Icibemba (40.3%), and Silozi (30.1%). Only 31 students speak Cinyanja, and just 1 school uses that language.
  • We expect the research firm to train 30% more enumerators than needed and use daily quizzes and performance at pilot to determine which enumerators to bring onto the team.
  • The training will be designed in collaboration with the TaRL Africa/VVOB team. We expect a 5-day training of the survey firm core team, a 5-day training of enumerators and a 4-day pilot (includes testing the survey tools in schools and debrief meetings with enumerators) for the endline survey.
  • The costs of these training workshops should be built into the firm’s budget.
  • Oversee the project management of the enumerators as they conduct the surveys including managing all transportation and team management.
  • Share details of the study with government officials and schools and follow up to ensure that school level management have consented to their learners taking part in the study.
  • Ensure the quality of the assessment data by:
    • Only allowing data collection in a school's official language, which has to match a surveyor's native language (Icibemba, Silozi, Nyanja/Chewa, or Chitonga).
    • Conducting spot-checks and analysis, whereby research staff observe surveyors conducting interviews, for at least 15% of all data points; and submitting the resulting data to the TaRL Africa Research Team for further analysis.
    • Conducting daily data high-frequency checks (HFCs), checking for data irregularities and enumerator fixed effects, and reporting the result of these checks (including interview counts, and an inventory of raw and cleaned data) to the TaRL Africa/VVOB Research Team on a daily basis review.

Kindly note, TaRL Africa/VVOB research teams will give input and approval of the final agreed upon HFCs tool and that Stata is the required tool for cleaning and analysis.

  • Collecting data anew, if the above-mentioned quality checks reveal strong irregularities (e.g., in the case of enumerator shirking).
  • Share raw and clean data daily with the TaRL Africa/VVOB Research Team.

Kindly note that Stata is the required tool for cleaning and analysis.

  • Ensuring strong team leadership including team leaders (specifically selected strong enumerators) for each group of enumerators that visit a school, and an overall project manager on the ground in Zambia who checks in with the various teams of enumerators.
  • Keeping personally identifiable information (PII) separated from the raw data in an encrypted file and sharing both the de-identified and identifiable data with the TaRL Africa/VVOB Research Team.
  • Ensuring all staff are trained in government protocol guidelines and human subjects research including deep familiarity with ethics and best practices of informed consent.
  • Ensuring staff are respectful of the school environment, school staff, and children.
  • Ensuring staff are trained in protecting children’s safety.
  • Track attrition to ensure it is below 5% in both control and the two treatment experimental arms, with no differential attrition rates across arms. An "attritor" is any student, teacher, or head teacher on the baseline roster without follow-up data. This may require revisits and/or home visits for follow-up exercises to further reduce attrition. For learners not available during the first school visits, the survey team will be required to make up to three additional visits to locate the learner.
  • The survey firm will also be required to contact the schools ahead of the visits to ensure that all learners are present during the survey days.

Kindly note that SurveyCTO is the required tool for data collection, and Stata is the required tool for matching and cleaning the data.

Timeline

The endline survey activities will be staggered across the following timeline:

1 Training of survey firm by TaRL Africa/VVOB August 2024
2 Survey Firm trains enumerators & tests tools August 2024
3. Pilot survey and updating of final tools September 2024
3 Data collection September - October 2024
4 Data cleaning and submission October 2024 -January 2025

A more specific timeline is shown in the next sub-section.

Tasks and Deliverables

The total time allocated for the endline survey tasks is 6 months. The research firm is expected to provide the following deliverables.

Tasks Expected Activities Deliverables Timeframe
1. Inception phase Apply/update and acquire an approval letter from the necessary Zambia Institutional Review Board (IRB).

Acquire a written government-issued permission letter, to enter schools and administer the assessments.

 

Translate and back translate all survey tools and consent documents

Review, update, and upload already coded survey tool(s) and consent scripts which will be used for the endline survey

 

Receive consent from the necessary people for the endline survey.

Approved IRB application

 

Government permission letter

 

Enumerator training plan

 

Consent report from government and school officials

 

Inception report

 

Translated survey instruments

 

Approved (by TaRL Africa/VVOB) coded surveys and consent scripts

 

Training plan

 

Field plan

August 2024
2. Training Train survey firm core team – 5 days

Train and certify enumerators (in collaboration with the TaRL Africa/VVOB Research Team) – 5 days

Pilot the endline survey with all enumerators and field teams over 2 days in schools, and Conduct debrief meetings with enumerators post the pilot over 2 more days.

Provide approved enumerator allocation per location/ zone

Note, there are 4 native languages (Icibemba, Silozi, Nyanja/Chewa, or Chitonga) in the study area and each enumerator assigned to a location must use the local native language as the first language of administering the tools.

Enumerator Training feedback report

 

Pilot data

 

Updated tools and processes based on pilot feedback, approved by TaRL Africa/VVOB before surveying beings

August -September 2024
3. Endline survey

 

 

Submit daily endline survey data to the TaRL Africa/VVOB Research Team Approved field plan

 

Provide daily dashboard reporting on progress (including quality assurance updates)

 

Conduct daily field team calls to discuss updates and challenges

 

Provide daily high frequency checks (HFC) and quality checks (QC) updates and access to codes and output for

endline survey data submitted daily to the Research Associate and TaRL Africa Senior Research and Learning Coordinator

September - October 2024
4. Final data submission

 

 

Submit final endline survey data to the TaRL Africa/VVOB Research Team Final clean deidentified endline survey data set (approved by TaRL Africa/VVOB)

 

Final clean endline survey data set with Personal Identifiable Information (PII) (approved by TaRL Africa/VVOB)

 

Final cleaning, matching and analysis STATA files (including .do files) (approved by TaRL Africa/VVOB)

 

All final data collection instruments (including document format questionnaires, SurveyCTO forms, and preloads) files (approved by TaRL Africa/VVOB)

 

Attrition report Note attrition should be balanced and not be above 5%

 

Invoices and financial report

November 2024 - January 20, 2025

 

Evaluation Management and Governance Arrangements

The research firm will be expected to work closely with the TaRL Africa/VVOB Research Team on the study design, implementation, and data quality assurance, and provide them with the reports and other deliverables detailed above in a timely manner.

Note, costs above the agreed upon final bid will be the responsibility of the research firm to bear as this is a deliverables-based agreement.

Ethical Considerations

Ethical aspects of the evaluation which include, among others, data collection from human subjects and their consent should be covered in detail in the technical proposal. The evaluation team will come into contact with human subjects, particularly children, and should take precautions to protect the rights and wellbeing of any children.

In addition, research instruments used as part of the evaluation need to undergo ethics approval processes within Zambia, and this will be the responsibility of the research firm in collaboration with TaRL Africa/VVOB Research Team.

The research firm must also adhere to the data confidentiality and privacy requirements of the TaRL /VVOB Research Team.

Location

The research firm will need to have a team (including the research manager) based in Zambia. The research firm should also have proven experience of collecting data in the Western and Central provinces of Zambia, particularly Western Province which has rough and sandy terrain, making driving difficult.

Qualification/Specialized Knowledge and Experience

The successful organization must offer the following demonstrated experience, knowledge, and competencies:

  • A competitive budget.
    • The budget should be itemized, in Excel, and clearly state its assumptions (e.g., for enumerator full-time equivalent [FTEs], productivity per school, and salaries).
    • Applicants may charge up to 10% in overhead or indirect costs, applied to total direct costs. We understand that the cap on overhead or indirect costs is low and that applicants may have reasonable project support costs included in budgets as direct costs. Such costs should be reasonable and explained in detail in the budget notes.
    • If the applicant plans to incur costs outside Zambia, these costs must not be included as direct costs; they must be included as overhead or indirect costs. For example, if applicable, costs for regional management or international grants managers are to be Included as overhead or indirect costs.
    • Any computer/equipment purchases or rentals should include a breakdown of what is being purchased or rented, (e.g., how many computer tablets/phones for data collection), as well as the project staff that will be assigned to the equipment.
    • Unallowable costs include those labeled as “incidental,” “miscellaneous,” “contingency”, or “other direct costs”. Any costs for rent should be explained in the budget notes.
  • Experience collecting data in Zambia.
  • Background in collecting high-quality data ethically. This will be measured by:
    • Proven record of getting approval through the Zambia Institutional Review Board (IRB).
    • Reference checks with past clients.
    • Protocols to protect children’s safety.
    • Strong research protocols within the company including in support of highest-quality data / data integrity.
  • Experience collecting education data and working with children will be viewed as advantageous.

Application

Proposals must contain:

  1. Technical proposal for the research work, not to exceed 10 pages, including:
    1. Detailed work plan that describes your suggested data collection plan, including human resources that would be involved (e.g., number of enumerators).
    2. The budget should be in Zambian Kwacha (ZMW). Provide an itemized budget for each of the activities described in the “Expected Deliverables” section (i.e., set up; training; secondary research; primary data collection; data cleaning and reporting). This should include all the costs of data collection, including but not limited to: IRB/government approval fees, enumerator training workshops, transportation, any required software, and printing. The budget should also include the costs for up to three rounds of revisits and/or home visits for follow-up exercises for children estimated to have not been interviewed during the initial visit. This is to reduce attrition to below 5%. In addition, the budget should be inclusive of a 20% Withholding Tax (WTH) deduction as per the laws of Kenya. Note, the budgeted costs will be met under a deliverables-based contract where the agreed upon final bid will be the responsibility of the research firm to bear.
    3. A brief description of the experience collecting data highlighting the following (please include the experience of enumerators you will likely work with on this project): experience working in Zambia, on education projects, and with school officials, as well as preferred software(s) for quantitative and qualitative data collection, data cleaning, and analysis. Please also highlight any learners survey data collection experiences in particular.
  2. A list of up to three references to whom you would feel comfortable with us reaching out.
  3. Curriculum vitae(s) of the survey firm’s core team members.

Interested research firms must submit their written proposals (and questions) in English to TaRL Africa via email at procurement@teachingattherightlevel.org by 30th June 2024, 23.59 CAT.

Job Information

Status: Open Job type: Full Time Salary: Negotiable Publish date: 10 Jun 2024 Expire in: 4 months

Apply for job

External website

Sorry! job link is not available right now. this might be temporary, please check back later.

Translate »
Scroll to Top