Trials testing new educational methods in schools often fail to produce useful evidence, say researchers

Posted on Friday 15 March 2019

The new study, published in the journal Educational Researcher, found that 40% of large-scale randomised controlled trials (RCTs) in the UK and the US failed to produce any evidence as to whether an educational intervention helped to boost academic attainment or not. 

The researchers evaluated 141 trials involving more than one million students, which tested schemes ranging from whether providing free school breakfasts raises grades in Maths and English, to whether playing chess improves numeracy skills.

The trials, which were carried out by the charitable organisation the Education Endowment Foundation (EEF) in the UK and the National Center for Education Evaluation and Regional Assistance (NCEE) in the US, are expensive - with costs often exceeding £500,000.

Informed decisions

The authors of the study argue that more research is urgently needed to understand why RCTs in education are so often uninformative.

Lead author of the research, Dr Hugues Lortie-Forgues, from the Department of Education at the University of York, UK, said: “Just like in medicine, trials of educational interventions are an important way to allow policy makers and teachers to make informed decisions about how to improve education. However, many of these trials are currently not fulfilling their main aim of demonstrating which interventions are effective and which are not.”

“Further research to investigate the reasons for this should be a priority. These organisations are trying to achieve something positive and reform is urgently needed to help them to do so.”

Higher standards

In recent years there have been a growing number of RCTs conducted in education. For example, in the UK, the Education Endowment Foundation (EEF) has commissioned more than 191 trials since 2012.

The researchers cite possible reasons why current trials may be ineffective, including:

• The interventions being tested may not be suitable for trial in the first place.
• Interventions may not be being correctly implemented during trials – for example due to inadequate training of teachers in the methods being tested.
• The trials themselves may be poorly designed

The authors suggest a series of changes that could make the trials more informative, including higher-standards when considering which new initiatives are trialled.