Of course teachers fear robo-marking

Computers are twice as accurate as doctors at diagnosing lung cancer. They can translate foreign languages, trounce grandmasters at chess, pilot planes, design buildings, compose music and they will soon drive cars. Nevertheless, the New South Wales Teachers Federation claims that computers are incapable of marking children’s writing.

Based on the advice of an American critic of computer marking, the union has conjured up an apocalyptic vision in which “robot markers” swarm through schools putting our “bewildered” children in “peril”.

The union’s ire has been incurred by a proposal from ACARA (the authority responsible for NAPLAN) to have computers mark NAPLAN writing tests. Despite the hype, ACARA is not proposing to alter the role of teachers — they will remain the ultimate authority on school marks and expert teachers will train the computer.

ACARA is simply proposing a trial for 2018, in which both humans and computers will mark writing. This will allow researchers to compare computers to humans and see how well the computers perform. If the trial is successful, computers may take on a greater role in marking in future. Because computer marking is faster than traditional methods, results would be available to teachers and schools earlier in the school year.

The teachers’ union knows all this, but they are adamant that computers should be shunned. They do not want to wait to see the outcome of the trial; they do not even want it to take place.  Why not?

One possibility is that it will cost their members money. Traditionally, teachers have done much of the marking of writing. If computers prove capable of doing the job, some teachers will still be needed to train them, but others may find themselves redundant. Like so many workers before them, they fear being replaced by machines.

This is a shame because ACARA’s research, using the same marking rules used by expert markers, have shown that computers perform as well as humans. In some cases, they perform better. The American state of Utah, which has been marking writing by computer for many years, has also found computers to be more consistent than traditional markers.

The teachers union says that students may ‘game’ computer markers by learning how they assign grades and responding accordingly. Utah officials have seen no evidence for this. Anyway, giving the markers what they expect is even easier when humans do the marking because the rules they use to assign marks are publicly available. (As an aside, it is not apparent why writing an essay that meets the marking standards should be considered wrong.)

Critics also claim that computers will never be able to measure creativity. This is not true. Computers learn to assign marks by mimicking those given by humans. If the marking rules require human markers to assign higher marks to creative essays, the computer will mimic the markers and reward creativity as well.

But what about the children; what do they think? ACARA’s research involving children right across Australia, from rural and remote areas and all school sectors, found that children report feeling happier taking NAPLAN online than on paper and report no disadvantage.

While we can all sympathise with the fear of disruptive change, we cannot simply rule out new testing and marking methods. If computerised marking fails, it will be dropped. If it works, it should be adopted. This is how evidence-based progress works. We cannot have a 21st-century education system by relying on 19th-century methods.

Emeritus Professor Steven Schwartz AM is Chairman of the Australian Curriculum, Assessment and Reporting Authority (ACARA) and Senior Fellow at the Centre for Independent Studies.