Anyone following my work will know that I’m not impressed with the research on phonics. But I am impressed by the motivated reasoning of leading researchers who claim that the science of reading strongly supports the importance of phonics and don’t see the gaping holes in the evidence. And even more impressed that most proponents of phonics ignore work that challenges the evidence. And more impressed still that I am yet to get a proponent of phonics to respond to a simple but direct (and polite!) request: “Can you provide me a reference to a study or meta-analysis that you think provides the best evidence for phonics?”. And I’ve asked quite a few experts.
I’m writing this post in response to the furore that has erupted in response to an article by Wyse and Bradbury and an associated Guardian article challenging the claim that systematic phonics has worked in England. Let me start with a blogpost by Greg Ashman and a twitter exchange that ensued.
And indeed, he did. But more importantly, his blogpost is full of errors. One is particularly egregious:
The graphs are hard to see on the screenshots from twitter, but here they are blown up.
First, here is Greg’s graph:
Second, here is a table that shows the results from England going back to 2000:
Do you see the problem? It is just poor science to selectively compare two out of multiple English-speaking countries, and even poorer science to exclude the UK results in 2000 and 2003 that mess up the story (England was doing much better in 2000, and still better in 2003 compared to 2018). I’ve asked Greg to update his blogpost to correct his error:
There are plenty of additional problems with his blogpost. For example, he challenges Wyse and Bradbury’s conclusion regarding the PIRLS results (an international test of reading comprehension). I tweeted the following with two screenshots attached:
Screen shot 1: What Ashman wrote in his blogpost:
That is a lot of qualifiers — what does Greg mean by writing that the most recent PIRLS results “arguably showed significant improvements”? I tweeted him this passage of my work that considers the PIRLS results in more detail:
Screen shot 2: A passage from Bowers (2020) that shows that the PIRLS results provide no evidence that reading comprehension has improved in England:
Despite these and other problems, multiple prominent researchers write to congratulate Greg on his response. For example:
Shortly after Jennifer Buckingham wrote a blogpost full of mistakes and selective citations, and again, many endorsements followed. For example:
Here are the three images I attached to the tweet. The first depicts a graph by Stainthorp (2020) – a paper that Buckingham strongly endorces and takes as evidence that phonics has improved SAT reading results in England, the second is a graph I reported (Bowers, 2020), and the third image explains the difference:
1) The Stainthorp (2020) graph. Note how the black bars start going up, consistent with improved reading outcomes (the test became harder in 2016, explaining the drop in performance, but the key conclusion is based on the black bars going up).
2) The Bowers (2020) graph. Note how the maths and science results go up at a similar rate as the reading and writing outcomes. And even worse for the story that phonics is responsible for the improved SAT reading scores, the timing does not work, as all scores start increasing the year before the introduction of the phonics screening check.
3) The explanation of key differences between graphs:
Regarding the Clackmannanshire study, Buckingham wrote in her blogpost: “Due to the very narrow (and, dare I say, not very systematic) method of selecting studies to review, one of the most important, and certainly most influential, studies of synthetic phonics instruction was left out – the ‘Clackmannanshire’ study in Scotland.” It has certainly been influential, but…. Here is a bit more detail I provided in an attachment to my next tweet.
It would be hard to find a more striking example of a confound that has been pointed out for years but that is just ignored. Indeed, I’ve directly pointed out this confound to Buckingham on twitter and she responded saying that she doesn’t think it is a problem (I can’t show this tweet exchange as I have been blocked). But it doesn’t stop Buckingham and many others from citing this work in support of systematic synthetic phonics.
A few days later Ashman comes up with another blogpost, again full of misrepresentations. This time, instead of calling Wyse and Bradbury liars, he suggests that they may be engaged in “deliberate misrepresentation”.
Ashman is not just engaged in hyperbole when characterizing the critics of phonics as thinking “nasty phonics zealots want kids to sit in rows all day…”; he fundamentally misrepresents the Guardian article and the published paper. Here are two tweets and attached images pointing out the problem:
Again, many endorsements from leading researchers. For example, Pamela Snow writes.
I continue to be struck by researchers who strongly endorse phonics and ignore straightforward mistakes that have been explicitly pointed out. No one has challenged any of the points I’ve made in these exchanges, and this is the norm, see: https://jeffbowers.blogs.bristol.ac.uk/blog/key-lesson/. Instead researchers just repeat the mistaken and misleading claims over and over again to their thousands of followers (and in published papers), and get annoyed when I have the gall to challenge them with evidence in public. I have to say I was disappointed in Dorothy Bishop who has done such great work on challenging poor scientific practice. See this excellent talk: https://www.youtube.com/watch?v=cQGD_Uw-Bj8&t=2s Indeed, Dorothy has written about a serious problem with the administration of the Phonics Screening Test: http://deevybee.blogspot.com/search?q=phonics+check But she will not engage with me regarding a more widespread problem with the science of reading, namely, the evidence for phonics itself.
There is a serious problem with the science of reading when it comes to reading instruction, but it is sure hard to get anyone to notice. For more fun absurdities, check out my other blogposts where you can find more of these twitter exchanges amongst other things: https://jeffbowers.blogs.bristol.ac.uk/blog/ You can download my freely available articles cited above at: https://jeffbowers.blogs.bristol.ac.uk/publications/education-literacy/
As always, it would be great to get a response, especially from the authors I’m criticizing. If I’ve made any mistakes, it would be good to know.
Ashman has responded to this blogpost (Response to Bowers – by Greg Ashman – Filling The Pail (substack.com)) and here is my response.
Ashman makes one reasonable point that was also made by James Theo on twitter:
This is with regards to the PISA graph that Ashman included in his blogpost that omitted the 2000 and 2003 data. I found it odd that these years were missing and quickly found a graph that included data for these years:
But as both Ashman and Theo rightly point out, there is some doubt about the accuracy of the 2000 and 2003 results given that England failed to meet the 85% participation rate in schools. This is from the Oates paper where this graph was taken:
In fact, I did include this qualification in the table above, but fair enough to point out, and happy to clarify this point. However, it does not address the problem of selectively comparing England to Canada, amongst other problems, as I previously wrote:
Indeed, re-reading this passage and looking over the table again made me realize there is an additional reason why the Ashman graph is misleading. It not only omitted other countries, but it also omitted other outcome measures. And as you can see from the table, the increase in the Maths scores from 2015 to 2018 (the 2018 cohort were the first to receive legally mandated systematic phonics) was greater than for Reading (indeed, as noted in the passage pasted above, only the mean increase in Maths was significant). That rather weakens the basis for concluding that phonics was responsible for the improved reading outcomes. This is a similar problem to the Stainthorp (2020) graph I pointed out above. In this case, the graph selectively reported the SAT reading outcomes, and the increasing scores were taken to provide evidence that phonics was responsible. But again, other outcome measures (both Maths and Science results) improved as well, undermining this conclusion.
So, yes, I should have highlighted more clearly that the 2000 and 2003 PISA results are problematic as England failed to meet the 85% participation rate in schools, and I accept that Ashman had a dataset that did not include these data, explaining why he did not include these years in his graph. But the point remains, the PISA results do not provide evidence that phonics is improving reading outcomes in England.
Greg also rejects my critique of the PIRLS results, writing:
But this omits the many problems I noted in the passage above, including the first point regarding the 2001 results repeated here:
So, lets recap: There is no evidence that phonics has improved SAT or PISA results if you track all outcome measures, and no evidence that PIRLS results have improved since 2001. And as I detail in the paper below, there is also little or no evidence from intervention studies as summarized by all the meta-analyses carried out since the National Reading Panel (2000). The claim that the science of reading strongly supports phonics and that reading has improved in England since the mandatory teaching of phonics in state schools is not even close to being supported.
Bowers, J.S. (2020) Reconsidering the Evidence that Systematic Phonics is more Effective than Alternative Methods of Reading Instruction. Educational Psychology Review, 32, 681-705.
Moving away from the science of reading, Ashman does not like the fact that I criticize other researchers for endorsing his work.
My view is that senior and influential researchers should be willing to defend their endorsements of research claims that they share with 1000s of their followers. I’ve not challenged junior researchers, but people who have prominent platforms aiming to influence researchers, policy makers, and teachers alike. I have left the comment section open on this blogpost for them to respond.
Finally, I suppose I should respond to the following:
Again, here is the context:
Ashman is now clarifying that he was only pointing out a general lie that is often told about phonics instruction, not attributing this lie to Wyse and Bradbury (which would make them liars). Easy to misunderstand, good to hear that he was not calling them liars, and happy to retract this claim. I would just note again that the “phonics only” characterisation of phonics critics is a strawman, and the following an absurdity.
For example, this is a quote from my original critique of phonics:
Take from: Bowers, J. S. (2020). Reconsidering the evidence that systematic phonics is more effective than alternative methods of reading instruction. Educational Psychology Review, 1-25.
Finally, a simple truth. Everyone has a point of view, from the researcher who has a long history claiming the science of reading strongly supports phonics to the person who has a brother who puts forward data and arguments in support of Structured Word Inquiry, an alternative to phonics. The way to assess whether someone is engaged in motivated reasoning is to consider whether they: a) ignore contrary data to their position, b) mischaracterize data contrary to their position, c) fail to correct errors when pointed out, and d) fail to engage in debates on the topic. I give Ashman credit for avoiding d).
I hope the researchers I’ve criticised do post a response below or elsewhere, and as I’ve written many times, I would be happy to engage in a debate about the evidence for phonics online or in some other context. The science of reading has been used to justify the legal requirement to teach synthetic phonics in all state schools and England, and there is a strong push to put in similar regulations in Australia. Scientists should always avoid motivated reading as best as possible, especially so here given the policy implications.