Fact-checking in science journalism

The Gordon and Betty Moore Foundation has helped produce a report on fact-checking in science journalism, and it is an eye-opening read. It was drafted by Deborah Blum and Brooke Borel; there is a nice summary here.

The standout findings for me, as a science editor working with journalists for a news publication in India, all had something to do with the fact that most people like to refer to the New Yorker model as the gold standard, and feed an implicit aspiration that that is the only way fact-checking must be done. But while the thoroughness and level of quality control exemplified by the New Yorker model are very high, the aspiration itself tends to be frequently unrealistic. The following lines from the report (paraphrased) support this view:

• About half of all outlets surveyed for the report (mostly American) delegated fact-checking to the reporters, the editors or a combination of them
• Fact-checkers in the US made anywhere from $15 to $75 an hour, with the average being $30; more importantly, fact-checkers cost money that publications may not always be able to afford. As one editor put it, “The difference made by incrementally-increased quality [due to fact-checking] is hard to quantify and hard to justify financially” – more so in India, where, for example, it has seemed increasingly evident that readers will not penalise a publication for working without a style guide.

(In the report, the newspaper model “does not employ fact-checkers, per se. Instead, the accuracy of the story lies mostly with the journalist. Many newspaper journalists have their own systems for double-checking facts in their stories… – for example, checking the piece line-by-line and cross-referencing to original sources. In the newspaper model all stories also go through editors, who push back on iffy claims and look for other holes in sourcing or logic. Rather than going line-by-line and checking all the facts, the editor is looking for potential problems. Finally, the story will go through the copy desk, where copy editors will check for style and grammar. At some publications, copy editors do an abbreviated fact check, confirming facts against written sources, although they don’t typically re-interview people who appear in the story.”)

• Editors use who use the newspaper model go by a ‘sniff test’, where they stay alert for facts or phrasing that sounds problematic, controversial, etc.
• The Wire Science uses the newspaper model (although I am the ‘editor’ and the ‘copy desk’ both) and, relative to publications in the West, I can’t help but wonder from time to time – even if irrationally so – if the work that we are doing is somehow poorer in quality. But reading of the names of publications that employ the same model has provided overwhelming validation: “Ars Technica, Sky at Night Magazine, Chemical & Engineering News, … Environmental Health News, Gizmodo, Nature Medicine, Newsweek (both print and online), NOVA Next, PBS NewsHour, Quartz, Retraction Watch, Science, Science News for Students, Vox (except features), and the Washington Post, as well as digital-only stories from Sierra, and Smithsonian.”

(At The Wire Science, once an article is submitted, one of three workflows kicks in. If I am not familiar with the article’s topic: I check it for clarity and flow, forward it to an independent topical expert who can comment on its technical details, and proceed to edit it together with the author. If I am familiar with the article’s topic: I check it for clarity and flow, perform a ‘sniff test’ fact-check, and proceed to edit it together with the author. If the article is in long-form: I check it for clarity and flow, forward one copy to an independent topical expert who can comment on its technical details, one copy to an independent fact-checker, and finally edit it together with the author.)

• Fact-checking has been on the decline but it is not as easy as attributing it to the rise of digital publishing. In fact, the divide between print and digital newsrooms  vis-a-vis fact-checking is much less strong than between news and long-form publishing.
• Some 61% of publications that had a fact-checker did not provide written guidelines and 57% did not provide training for the person in that position
• Most editors “don’t allow anyone to share unpublished materials – whether an entire story or a short excerpt – with sources during a fact check”; The Wire Science treads the same line for the most part
• It is easier to correct an article post-publication in digital form than in print form. But most people forget that the digital medium aids preservation and reproducibility, i.e. a captured screenshot can last for many years more than a piece of paper bearing some words.

A concluding note from my end: Facts are important, but in science journalism, we are also often in the business of uncertainty and exploration – realms of endeavour in which facts are, more often than not, contingent. So fact-checking itself must not fetishise precision, especially when there might be an advantage in dangling doubt from a well-constructed web of contingencies, as much as give facts room to move and breathe freely.

The Wire
September 14, 2018