England v Scotland, the battle of the PISA tests

What can we learn from the comparison?

The mainstream media does not do nuance. So the results of international education tests called PISA have been reported as victory to England, because 15-year-olds there did significantly better in the core disciplines of maths, science and reading tests. But that is not a complete picture. Scots scored better in wellbeing, feeling supported and curiosity.

England’s Education Department doesn’t seem to have published their results on wellbeing, so I have used the UK stats below. I guess it depends what you think matters - English kids are less likely to feel they belong at school, but at least they can do the maths. We have something to learn here. We need both.

(The gap between Scotland and England in the academic subjects in percentage terms is a similar size: Scotland is 1.5% behind England in reading and 5% in maths and science)

Because education is completely devolved, there is a political edge to the debate. Unionists imply that if the Scottish gov can’t win in education, it is surely a sign that it can’t be trusted with more powers. And maybe they have a point - Scotland does need high expectations for its young people. They may be the first generation in three centuries to run this place as an independent country so let’s make sure they can add up.

Are the English PISA stats dodgy because the lowest cohort don’t show up in the test room?

Some commentators cast doubt on the English stats. Of pupils randomly selected to participate in the tests, about a quarter did not. The OECD assumes that these children are similar to the ones who did take part - but in fact almost half of the no-shows were entitled to free school meals compared to a quarter of the test takers. (In Scotland, the test takers had a 14% FSM rate and the no-shows 16% FSM.)

Prof of Education at UCL John Jerrim tweeted: “PISA results out today. Look at this for England. Sample of participating pupils = 24% FSM. Compared to 40% who did not participate! Yes, our results look good. But remember, our scores are likely upwards biased by around 10 points!”

In the National, Dr Joseph Smith, an education expert at Stirling University, opined that many English schools are no-go areas for the poorest cohort who effectively massage themselves out of the stats by ducking school.

But even if the English stats are a bit flattering, there would still probably be a gap in the core academic competencies.

Is my hen sick?

Scotland’s scores have not moved for a decade (taking into account the margin of error) and they remain above the OECD average - but that itself is falling. The OECD expressed concern about the drop, especially in reading.

To find out more, I went on the PISA site and took a sample reading test. It featured a message board where a woman named Ivanka was seeking advice on whether to give aspirin to her sick hen. We were asked to identify the most relevant response on a grid - there were people touting animal pharmaceuticals and one man asked plaintively: “how can you tell if a hen is sick?’ I quickly identified the correct reply. It was from Frank, a vet, who told Ivanka how to work out the correct dosage.

I tend to overthink these things, so I found myself asking: why is a busy vet like Frank spending time offering advice on a chicken-fanciers’ webchat? For free? How believable is that? And why is Ivanka using a message board in the first place, rather than asking Jenny AI like most people do today? Furthermore, have 15-year-olds today even seen a message board? That is the challenge when trying to make test content relevant - you can easily end up in a cul-de-sac full of technological tumbleweed.

It reminded me of a feature I once did about high-achieving students from low-performing schools. A teenage boy told me his favourite subject was Latin because he enjoyed translating Cicero. He said the content of modern languages courses tended to be about ordering food in restaurants. As one of eight children from a Catholic family in the East End of Glasgow, that wasn’t something he was much interested in. He preferred Cicero’s political gossip and philosophical debates. That interaction stuck with me as an example of how “relatable” context can miss the mark.

Talking of missing the mark, you will be wondering how I did in the test. Well, I did not do amazingly because I clicked through too fast and missed a bit. I taught myself speed-reading when young - there is a line in a Woody Allen movie where he says “I speed read War and Peace. It’s about Russia.”

I am not alone - the number of ‘hasty readers’ has almost doubled in recent years. But these online tests don’t affect the student in any way. What happens if you finish early? Do you get to go home? Is it possible that a lack of engagement with Ivanka’s quaint message board could be one factor in the decline. Just a thought - maybe try some Cicero?

Is digital distraction a factor?

At the PISA launch webinar, Education director Andreas Schleicher pointed the finger of blame for lower reading scores pretty squarely at technology. Many countries that were the most aggressive adopters of screens in the classroom - Sweden, Finland - are now rowing back hard.

Scotland was one of these - perhaps it was too quick to jump on the virtual bandwagon - Glasgow City Council rolled out the biggest Apple education programme in Europe with 80,000 iPads handed out in the name of digital equality.

I have some sympathy. My generation of parents wanted our children to stride forward confidently into the digital age so we let them lead - maybe too much. When one of my young associates was given an ipad he stopped reading overnight. He soon became absorbed by a game called Runescape. Trying to keep up and not be a Luddite, I bought a book called Don’t Bother Me Mom, I’m Learning. The thesis of the book was that game developers had perfected a thing called ‘flow’ where they kept the difficulty level at exactly the right level for the player - this new technique was going to be adopted by educators and make learning fun.

Maybe I should not have bought that. Schleicher talked instead of the ‘productive struggle’ that tech tools were displacing - to the detriment of pupils. We don’t become fit by watching sports - getting a robot to lift weights for you won’t give you strong arms.

China remains the leader in PISA. Schleicher said that in Asian countries, tech is not the individualised device in your schoolbag that we know, instead teachers help design and drive it in ways that help them to challenge students.

China does particularly well with the bottom cohort. According to PISA there is hardly a soul there who could not easily parse an electricity bill. And yet - researchers say that because rural provinces don’t participate, China too is manipulating the test.

Areas for improvement

PISA is just one metric and we should beware of how it can be used for political ends. England’s success is being used to attack Labour reforms and to defend the idea of a traditional top-down curriculum, which tells teachers exactly what to teach and when. But as James McEnaney pointed out in the Herald, Ireland does better than England without taking that approach.

Nevertheless, it is useful to get a peek over the neighbours’ fences. Instead of going straight to the curriculum, there are other issues that the test reveals - disruptive behaviour for one. Scotland’s pupils reported less disruption by misbehaving pupils than the OECD average - but it still seems incredibly prevalent. Getting to grips with this disengaged, underperforming and at times disruptive cohort - which seems to be growing across the developed world - is clearly a huge issue.

These are the pupils saying this NEVER happens so the rest of the cohort presumably said it sometimes does

We have heard a lot in the media about trouble in Scottish schools. There certainly does seem to be an issue with verbal abuse and intimidation but on many behavioural issues, Scotland does better than the average.

High levels of absenteeism and lateness are also common across developed countries. One sad statistic is that so many children in so many countries seem to say their families don’t take much interest in what they are learning at school. Why is that? Maybe the parents are on their phones too much as well?

Scotland’s PISA results are not terrible - in fact, they are above average. Second-generation immigrants as a group do particularly well - immigrants the world over tend to show a high degree of commitment to education. But there is surely room for improvement, and perhaps teachers, families and communities could do better too, to support all of our children.

Further reading: The Scottish government’s report on PISA 25

OECD PISA 25 launch video

Graphics created by AI