Evidence for success of independent public schools is flawed

Education Minister Christopher Pyne has been committed to making public schools in Australia independent, saying the more control a principal has over a school the better that school performs. While more school autonomy has its merits, there is no evidence to suggest it will lift a school’s performance.

On Wednesday, The Age reported on a “landmark study” commissioned by the federal government that found school autonomy does improve school outcomes. But, on taking a closer look at the study and its methods, not all is as it seems.

What the study found and how

The study looked at four autonomous schools in Australia and found the greater level of independence enjoyed by the school had a positive effect on students' results. This sounds like a coup for the independent public schools movement, but there are serious flaws in the study’s methodology, including how the schools were selected.

There were two criteria for selecting schools to be a part of the study. The first was that the schools must have taken up a higher level of autonomy for at least two years. The second was that they achieved evidence of gains in student achievement that they can confidently claim were the effect of autonomy.

The author, school autonomy proponent Brian Caldwell, made a disclaimer that no attempt was made to generalise from the experience of these four schools. They were considered “demonstration schools” that illustrated how the links to student achievement have been made under conditions of autonomy.

What we have therefore is a small sample (four schools) selected because they support the policy position being advanced. This is far less evidence of impact than it is an illustration of practice. Any claims of school autonomy leading to improved student achievement are flawed.

The history of school autonomy in Australia

In many ways, Australia was a leading adopter of school autonomy – particularly in the ACT and Victoria. There have been many pushes towards greater autonomy for schools, including advisory roles to governments, publications and scholarly critique. Rarely, however, have these diverse sources come together to deliver a comprehensive statement on the merits of school autonomy for improving student outcomes.

Obscuring the debate on the merits of school autonomy is the slippery use of language. We have “decentralisation”, "devolution”, "self-managing”, "school autonomy”, "principal autonomy” and "independent public schools”, just to name a few.

Internationally, we can add charter schools (US), academies (England), free schools (Sweden) and the growth of for-profit providers running schools in India and Africa among others. All of these reforms are often used interchangeably to make arguments both for and against school autonomy.

Assessing the effect of school autonomy

Support for increased autonomy is frequently found in large-scale international tests. In particular, the OECD claims that in countries where schools have greater autonomy, students tend to perform better. However, the presence of autonomy at the school level does not mean it is the cause of improved or higher performance.

What is often omitted from this OECD evidence is that this autonomy is specific to what is taught and how students are assessed. Such a degree of autonomy is not part of the current reform agenda in Australia.

It is actually possible to claim the counter. The advent of national testing (such as NAPLAN),national curriculum, national professional standards, teacher education reforms and public accountabilities such as the MySchool website comparing schools create an environment that is far from autonomous.

In an era where prominent education researcher John Hattie’s work on quantifying the effect of various interventions in education has come to dominate education dialogue and debate, it is surprising that a reform with little – and mixed – empirical support has remained so popular.

As noted previously in The Conversation by Glenn Savage (here and here), there is a lack of rigorous and robust evidence that school autonomy leads to better student outcomes. Even a study from the University of Melbourne (where Brian Caldwell is affiliated) found:

… little evidence of changes to student outcomes [and] no substantive increase in student achievement.

Similar findings have been published from the United Kingdom, the United States, New Zealand and Australia. Even the OECD has presented mixed evidence. In short, the empirical evidence for school autonomy and improved student outcomes is, at best, inconclusive.

Absence of debate

What is most disappointing is the absence of serious dialogue and debate on the merits of school autonomy.

In a 2012 review of related literature for the evaluation of empowering local schools for the Department of Education, Employment and Work Relations, Brian Caldwell reduces any engagement with opposing view to a single sentence:

Robust criticisms were mounted.

Similarly, this week he has again dismissed counter claims as straw man arguments.

In the interest of improving our schools there is a need for serious dialogue and debate. We need clarity on what school autonomy really means, what evidence we have to support it (or not), and what we need to know.

There is little doubt that the argument for greater autonomy is persuasive. The logic of the claim seems like commonsense and impossible to refute. This, however, is the greatest danger of the debate.

The unquestioned support of a commonsense claim is not helpful for improving performance in the Australian education system. This is not to say that school autonomy is not a factor, or even the answer.

Rather, we simply don’t know. The evidence at this point is inconclusive. Dialogue and debate is minimal – and more likely to be people talking past one another. To that end, we cannot claim that school autonomy will improve student outcomes.

Scott Eacott is Director, Office of Educational Leadership at UNSW.

This opinion piece was first published in The Conversation.