The latest skirmish in the never-ending war against NAPLAN is being fought on the grounds of “comparability”. According to American consultants commissioned by the NSW Teachers’ Federation, this year’s NAPLAN results “should be discarded” because around 20 per cent of students completed NAPLAN online while the rest used paper and pencil. The consultants claim that “enormous” differences between the two test formats make any comparison between them misleading.
Curiously, the offshore consultants reached their conclusion without any reference to the 2018 NAPLAN results. Instead, they relied on a few studies of other tests, including some 30-year-old ones — what sort of computers were around then? — their own opinions, and some gratuitous comments about the incompetence of Australian statisticians.
The consultants’ report is riddled with errors. Despite the report’s claims to the contrary, students sitting the online test can in fact go back to review and change answers to previous questions. More importantly, there are numerous examples of large-scale assessments like NAPLAN that have been able to draw valid comparisons between online and paper results. These include the Program for International Assessment, or PISA, and the Trends in International and Science Study, or TIMMS.
ACARA has now released this year’s results and they clearly show that the online and paper and pencil tests are indeed comparable.
This does not mean that the paper and pencil and the online test are identical — they are different — but they are comparable because both measure the same underlying skills: numeracy and literacy. Comparing NAPLAN scores across testing modes — or across years, for that matter — is like comparing length using centimetres and inches. They are different, but they can be compared because they both measure the same thing (length).
As it turns out, there was a difference between the paper and pencil and the online version of NAPLAN, but it was not one that the union’s consultants predicted. Based on a 1992 study, the consultants claimed that typewritten essays receive lower marks than handwritten ones. The year 9 NAPLAN results showed just the opposite — students who completed their writing tests online scored higher on the average than those who wrote by hand. This result reflects older students’ experience with writing on computers and the ease with which computer writing can be reviewed and edited.
The ability to write clearly is a vital skill; it is essential to success in practically all lines of work, yet this year’s NAPLAN results show that writing scores are at their lowest level since NAPLAN testing began. Because students are more likely to review and edit their work when writing on a computer, online writing has the potential to improve both instruction and assessment. Instead of criticising word processing and online writing, we should be harnessing this technology to improve writing skills.
The online version of NAPLAN offers numerous benefits. Results will be available much earlier in the school year to facilitate earlier intervention, and they will also be more precise. In contrast to the present one-size-fits-all paper test, NAPLAN online is tailored to the abilities of each student. Teachers receive a more precise picture of each student’s strengths and weaknesses. Also, for the first time, NAPLAN will be able to be tailored to the individual needs of students with disabilities.
Like the legendary Rorschach inkblots used by psychiatrists, NAPLAN elicits radically divergent responses from different observers. Depending on whom you ask, the tests are too short, too long, too soft, too difficult, too narrow, too broad, too frequent or too rare. And now we are told that they cannot be compared. None of this is true.
NAPLAN exposes the truth. This year it exposed a persistent lack of improvement in writing in the 10 years since the assessment started, with one in five Year 9 students failing to achieve the minimum benchmark. Without NAPLAN we would be in the dark about these parlous education outcomes, which risks seeing our students continue to fail.
NAPLAN holds teachers, principals, schools and governments accountable. And it ensures the transparency of education results — allowing parents to be well informed. Many people find this uncomfortable, so they attack the assessment using every argument that they can mount.
It is time for parents, policymakers, and community leaders to make their voices heard. This battle will likely not be the last skirmish in the war on NAPLAN. But if the battalions that are attacking NAPLAN win, it is our students who will lose.
Emeritus Professor Steven Schwartz is the former chair of ACARA and a senior fellow at The Centre for Independent Studies, where Blaise Joseph is an education policy analyst.
09 February 2019 | The Australian
Outdated teaching methods based on disproved theories remain widespread despite the abundance of good and easily available information on effective, evidence-based instruction. The gap between research and practice…
01 February 2019 | Ideas@TheCentre
You have to wonder how many times something has to be tried before people stop calling it ‘innovative’ and ‘new’. Especially when it has fallen short of expectations…
24 January 2019 | The Australian
Universities have drastically dropped their entry requirements for teaching degrees, resulting in a hit on the taxpayer when many students inevitably fail to graduate. Jayden (not his real name) barely…