Teaching with the test: Experimental evidence on diagnostic feedback and capacity-building for public schools in Argentina (with R. de Hoyos and P. A. Holland)

Despite the recent growth in the number of of large-scale student assessments, there is little evidence on their potential to inform improvements in school management and classroom instruction in developing countries. We conducted an experiment in the Province of La Rioja, Argentina in which we randomly assigned 105 public primary schools to: (a) a "diagnostic feedback" group in which we administered standardized tests in math and reading comprehension at baseline and two follow-ups and made their results available to the schools through user-friendly reports; (b) a "capacity-building" group in which we also provided schools with workshops and school visits for supervisors, principals, and teachers; or (c) a control group, in which we administered the tests only at the second follow-up. After two years, diagnostic feedback schools outperformed control schools by .34 and .36 standard deviations (SDs) in third grade math and reading, and by .28 and .38 SDs in fifth grade math and reading. Principals at these schools were more likely to report using assessment results for management decisions and students were more likely to report that their teachers engaged in more instructional activities and improved their interactions with them. Capacity-building schools saw more limited impacts due to lower achievement at baseline, low take up, and little value-added of workshops and visits. However, in most cases we cannot discard the possibility that both interventions had the same impact.

The untapped math skills of working children in India: Evidence, possible explanations, and implications (with A. V. Banerjee, S. Bhattacharjee & R. Chattopadhyay)

It has been widely documented that many children in India lack basic arithmetic skills, as measured by their capacity to solve subtraction and division problems. We surveyed children working in informal markets in Kolkata, West Bengal, and confirmed that most were unable to solve arithmetic problems as typically presented in school. However, we also found that they were able to perform similar operations when framed as market transactions. This discrepancy was not explained by children’s ability to memorize prices and quantities in market transactions, assistance from others at their shops, reliance on calculation aids, or reading and writing skills. In fact, many children could solve hypothetical transactions of goods that they did not sell. Our results suggest that these children have arithmetic skills that are untapped by the school system.

Disrupting education? Experimental evidence on technology-aided instruction in India (with K. Muralidharan & A. Singh)

We present experimental evidence on the impact of a personalized technology-aided after-school instruction program on learning outcomes. Our setting is middle-school grades in urban India, where a lottery provided winning students with a voucher to cover program costs. We find that lottery winners scored 0.36σ higher in math and 0.22σ higher in Hindi relative to lottery losers after just 4.5-months of access to the program. IV estimates suggest that attending the program for 90 days would increase math and Hindi test scores by 0.59σ and 0.36σ respectively. We find similar absolute test score gains for all students, but the relative gain was much greater for academically-weaker students because their rate of learning in the control group was close to zero. We show that the program was able to effectively cater to the very wide variation in student learning levels within a single grade by precisely targeting instruction to the level of student preparation. The program was cost effective, both in terms of productivity per dollar and unit of time. Our results suggest that well-designed technology-aided instruction programs can sharply improve productivity in delivering education.