Teaching at the Right Level Africa Programme d’Enseignement Ciblé (PEC) – Côte d’Ivoire (CIV) RCT Baseline, Process Monitoring, and Endline Survey: Terms of Reference

Summary 

Background

The purpose of this announcement is to solicit proposal requests for the engagement of a qualified research firm for the provision of professional research services to advance further the objective of Teaching at the Right Level (TaRL), which is to help children develop basic reading and mathematics skills, opening doors to a brighter future.

TaRL Africa is an independent organization formed through a partnership between Pratham, an education NGO, and the Abdul-Latif Jameel Poverty Action Lab (J-PAL), a research center based at the Massachusetts Institute of Technology. These two organizations have worked together to create cost-effective, scalable literacy and numeracy interventions for over 15 years. The fruits of these efforts have been the development of what is now referred to as the TaRL approach.

The TaRL approach divides children in upper primary into groups based on learning levels rather than age or grade. In Côte d’Ivoire (CIV), the TaRL approach is referred to as Programme d’Enseignement Ciblé (PEC). It is already led and implemented by the government in 1000+ primary schools in the Southern part of the country. TaRL Africa is now working with the government to scale up PEC more widely in both the South and North parts of the country. 

There exists a plethora of evidence on the effectiveness of TaRL from India that supports the generalizable theory behind tailored instruction. For instance, six (6) randomized evaluations conducted by J-PAL-affiliated researchers prove that when instruction is tailored to the child’s level and the TaRL accelerated learning methodology is used, learning outcomes improve.. 

Study Overview

Over the last two years, TaRL Africa has conducted several learning exercises to unpack potential innovations to best improve program delivery for PEC in Côte d’Ivoire. The current innovation seeks to blend better data use and remote mentoring to improve mentoring provided to teachers. Our innovation contains two components, as highlighted below. 

1. Train mentors and statisticians on how to use data to target school visits
2. Train PAs on remote mentoring.  

The effectiveness of this innovation will be tested through an RCT. 

We will conduct a baseline at the inception of this RCT.  This is to: -

1. Assess understanding of the innovation concepts – A pre- and post-training assessment of the trainees (mentors and teachers) will be conducted to assess understanding of innovation concepts. The pre-training assessment will collect information on current mentoring practices and the motivation for those practices. 
2. Assess pre-existing facilitators’ practices and students’ learning outcomes. Interviews will be conducted with school directors to understand current PEC implementation, challenges, and support received from PAs. At the teacher level, we will assess knowledge and practices of the PEC approach. We will measure learning outcomes using ASER and other independently constructed tests for students.  

At the end of the baseline, results will be analyzed, presented, and discussed with various stakeholders (country team, government, content team) to inform items to be assessed during process monitoring (data collection by survey firm) and endline (data collection by survey firm) in the RCT. 

We will conduct a process monitoring survey at the midline of this RCT.  This is to: -

1. This is to support the measurement of program implementation fidelity, program take-up among mentors, teachers, and school directors, and intermediate outcomes for the RCT. 

We will conduct an endline survey at the end of this RCT.  This is to: -

1. Assess uptake of the innovation concepts – An assessment of the trainees (mentors and teachers) will be conducted to assess understanding and uptake of innovation concepts. The assessment will collect information on practices.
2. Assess facilitators’ practices and students’ learning outcomes. Interviews will be conducted with school directors to understand PEC implementation, challenges, and support received from PAs following the innovation. At the teacher level, we will assess knowledge and practices of the PEC approach. For students, we will measure learning outcomes using ASER and other independently constructed tests.  

At the end of the endline, results will be analyzed, presented, and discussed with various stakeholders (country team, government, content team).

The study’s overall learning objectives

1. How can data help PAs decide which schools to support in person? 
2. How can remote mentoring help reach teachers? 
3. How does the combined approach help increase and improve teacher support from PAs? 

What do we want to learn/ understand from the baseline study? 

Training of pedagogical advisors (CPPPs) evaluation 

1. How PAs mentor teachers before training on the innovation
2. What PAs learned from the training about the innovation?
   1. Can PAs better analyze data following the training? 
   2. Can they use the visit targeting template? 
   3. Do they know what to do to support schools remotely?
3. How did the PAs evaluate the current training? 
4. What behavioral changes do PAs expect after the training? 
5. What behavioral changes PAs got after the training? 

School directors survey

1. How is PEC implemented at the school level? 
2. What mentor support do school directors receive from CPPPs?
3. What challenges related to mentoring do school directors face?

Teacher/facilitator survey

1. What factors contribute to effective teaching of PEC?
2. What factors might influence facilitators' performance in literacy and numeracy?
3. What types of support are typically available to facilitators from administrative personnel and supervisors? 

Learner assessment

1. What are the children's literacy and numeracy learning levels based on ASER and additional assessment tools? 

Research Design

To effectively reach these learning goals, the research firm is expected to carry out 3 months of the following baseline activities and samples as detailed out below. Note, these are provisional dates and can change in the future:

Process monitoring

The dates for the process monitoring have not been set yet. Tentatively it will take place around 15 January 2024 - 15 February 2024. Note, unlike the baseline, there will be no pre- and post-training surveys at the process monitoring stage. It will also entail a smaller sample as described under the ‘Summary section’. This will focus on a sample of the treatment arm schools. That is, schools supported by mentors who are trained on how to use data guided by a template to target schools to visit in person and on how to mentor teachers in other schools remotely.

Endline:

The dates for the endline have not been set yet. Tentatively the endline will take place around April - mid May 2024. Note, unlike the baseline, there will be no pre- and post-training surveys at the endline.

The Assignment

The selected research firm will work closely with the TaRL Africa Research Team.  They will be required to: 

1. Suggest and contribute to the overall research design.
2. Apply and acquire clearance from Cote d’Ivoire Institutional Review Board and other relevant authorities.
3. Support in obtaining a government-issued permission letter to enter schools and survey children and school officials.
4. Review survey instruments, suggest revisions based on context, and translate surveys into English and French.
5. Code and program the surveys, if necessary, (English, French and local languages) into appropriate data collection tool(s) / programming platform(s) such as Survey CTO as agreed on with the TaRL Africa Research Team for the qualitative and quantitative tools. 
   1. The tool programme must ensure data security, be easy to use, and offer offline capabilities that maintain safe storage and data integrity.
6. Pilot survey in collaboration with TaRL Africa Research Team.
7. Design high-quality reporting protocols in collaboration with TaRL Africa Research Team for daily frequency checks and quality assurance checks for the quantitative and qualitative data.
8. Recruit, hire and train a set of enumerators. 
   1. We expect the data partner to train 30% more enumerators than needed and use daily quizzes to determine which enumerators to bring onto the team. Kindly note we would require an enumerator performance report at the end. 
   2. The training will be designed in collaboration with the TaRL Africa Research Team. 
   3. The costs of these training workshops should be built into the firm’s budget. 
9. Oversee the project management of the enumerators as they conduct the assessments and other surveys including managing all transportation and team management.
10. Keep personally-identifiable information separated from the raw data in an encrypted file.
11. Ensure the quality of the assessment and interview data by:
    1. Conducting spot-checks and analysis, whereby research staff observe surveyors conducting interviews, for at least 15% of all data points, and submitting the resulting data to the TaRL Africa Research Team for further analysis.
    2. Conducting back-checks and analysis of 10%-15% of quantitative survey activities by re-visiting schools where enumerators were supposed to have visited, re-conducting the respective interviews, and submitting the data to the TaRL Africa Research Team for further analysis. 
    3. Conducting daily high-frequency data checks, checking for data irregularities and enumerator fixed effects, and reporting the result of these checks (including interview counts, and inventory of raw and cleaned data) to the TaRL Africa Research Team for daily review, for quantitative survey activities.  
    4. Collecting data anew, if the above-mentioned quality checks reveal strong irregularities (e.g., in the case of enumerator malfeasance or poor quality).
    5. Strong team leadership including team leaders for each group of enumerators that visit a school, and an overall project manager who checks in with the various smaller teams of enumerators. 
    6. Ensuring staff are trained in human subjects including ensuring informed consent.
    7. Ensuring staff are respectful of the school environment, school staff, and children.
    8. Ensuring staff are trained in protecting children’s safety.
    9. Ensuring staff follow Covid-19 protection protocols.
12. Collaborate closely with the TaRL Africa Research Team, who will also be engaging in in-person monitoring and data checks to ensure quality. This likely includes daily phone calls during the data collection process and very close engagement in the planning phases.
13. Share the raw data with the TaRL Africa Research Team daily in French, and English.
14. Translate collected data into English based on agreed-upon indicators or surveys of interest.
15. Analyze collected data and write up reports and presentations in English and French.
16. Track overall progress on the project and ensure adherence to timelines, guidelines, and the project plan.

Expected Deliverables

Baseline

The total time allocated for the Baseline is roughly 3 months (August - October 2023). The work will be split as follows: 

*Timelines are subject to change based on the PEC implementation calendar.

Process monitoring

The total time allocated for the process monitoring is roughly 1 month (15 January 2024 - 15 February 2024). 

• Note the deliverables for the process monitoring will follow a similar structure in 15 January 2024 - 15 February 2024. However, the allocated time is shorter because it will be a smaller sample and there will be no trainings for mentors.

• The breakdown of activities will be provided before the process monitoring. 

Endline

The total time allocated for the Endline is roughly 2 months (April- mid May 2024). 

• Note the deliverables for the endline will follow a similar structure in April- mid May 2024. However, the allocated time is shorter because there will be no trainings for mentors.
• The breakdown of activities will be provided before the endline. 

Eligibility

The successful research firm must offer the following demonstrated experience, knowledge, and competencies: 

General requirements:

1. A competitive budget.
2. Experience collecting data in Côte d’Ivoire.
3. Experience collecting education data and working with teachers and governments will be viewed as advantageous.
4. Proficiency in qualitative and quantitative data collection and analysis softwares.
5. Native/fluent in French and English. Familiarity with the various Ivorian languages (local language) is preferred.

Research requirements:

1. Background in collecting high-quality data ethically. This will be measured by:
   1. Reference checks with past clients or published research work.
   2. Strong research protocols.
2. Proven track record of qualitative and quantitative data collection in the industry or academic settings.
3. Experience leading, designing, and implementing research with references available.
4. Strong communication, observational, and engagement skills.
5. Perceptive to engagement dynamics and relations.
6. Strong communication and rapport-building skills.

Evaluation Management and Governance Arrangements 

The data partner will be expected to work closely with the TaRL Africa Research Team on the study design, implementation, and data quality assurance, and provide them with the reports and other deliverables detailed above in a timely manner. 

Ethical Considerations 

Ethical aspects of the evaluation which include, among others, data collection from human subjects and their consent should be covered in detail in the technical proposal. The evaluation team will come into contact with human objects, particularly children, and should take precautions to protect the rights and wellbeing of any children.

In addition, research instruments used as part of the evaluation need to undergo ethics approval processes within Côte d’Ivoire, and this will be the responsibility of the data partner.

The data partner must also adhere to the data confidentiality and privacy requirements of the TaRL Africa Research Team.

Location 

The data partner will need to have a team based in Côte d’Ivoire. 

Qualification/Specialized Knowledge and Experience

The successful organization must offer the following demonstrated experience, knowledge, and competencies: 

• A competitive budget. 
  • The budget should be itemized, in Excel, and clearly state its assumptions (e.g., for enumerator FTEs, productivity per school, and salaries).
  • Applicants may charge up to 15% in overhead or indirect costs, applied to total direct costs. We understand that the cap on overhead or indirect costs is low and that applicants may have reasonable project support costs included in budgets as direct costs. Such costs should be reasonable and explained in detail in the budget notes. 
  • If the applicant plans to incur costs outside Côte d’Ivoire, these costs must not be Included as direct costs; they must be included as overhead or indirect costs. For example, if applicable, costs for regional management or international grants managers are to be Included as overhead or Indirect costs. 
  • Any computer/equipment purchases or rentals should include a breakdown of what is being purchased or rented, (e.g., how many laptops), as well as the project staff that will be assigned to the equipment. 
  • Unallowable costs include those labeled as “incidental,” “miscellaneous,” “contingency”, or “other direct costs”. Any costs for rent should be explained in the budget notes.
• Experience collecting data in Côte d’Ivoire. 
• Background in collecting high-quality data ethically. This will be measured by:
  • Proven record of getting approval through Côte d’Ivoire Institutional Review Board. 
  • Reference checks with past clients.
  • Protocols to protect children’s safety.
  • Strong research protocols within the company including in support of highest-quality data / data integrity,
• Experience collecting education data and working with children will be viewed as advantageous.

Application

Proposals must contain:

1. Technical proposal for the research work, not to exceed 6 pages, including:
   1. Detailed work plan that describes your suggested data collection plan, including human resources that would be involved (e.g., number of enumerators) for each of the three broad categories; baseline, process monitoring, and endline.
   2. Budget for each of the activities described in the “Expected Deliverables” section (i.e., set up; training; secondary research; primary data collection; and data analysis and reporting). This should include all the costs of data collection, including but not limited to: IRB/government approval fees, enumerator training workshops, transportation, any required software, and printing. Note, the budget must be split into three broad categories; baseline costs, process monitoring costs, endline costs, then have a grand total.
   3. A brief description of the experience collecting data highlighting the following (please include the experience of enumerators you will likely work with on this project): experience working in Côte d’Ivoire, on education projects, and with school officials as well as preferred softwares for quantitative and qualitative data collection and analysis.
2. A list of up to three references to whom you would feel comfortable with us reaching out.
3. Curriculum vitae.

Interested research firms must submit their written proposals (and questions) in English to TaRL Africa via email at procurement@teachingattherightlevel.org by 11th August 2023, 23.59 GMT.