five insights from historic reports

The UK Royal Society has more experience of peer review than most publishers, with the practice used by its journals for nearly 200 years. Last month, the society unsealed more than 1,600 historic reports, dating from 1949 to 1954, and added them to its archive. The reports include assessments of high-profile papers.

The Royal Society’s Philosophical Transactions was the first journal to establish peer review, says Louisiane Ferlier, the society’s digital resources manager in London. “This is really the biggest archive you can have for any scientific journal,” she says.

Researchers reveal the emotional and professional cost of drawn-out peer review

The initial process was much more informal than the one scientists know today, which became formalized in the 1970s, she adds. “Some early referees’ reports have news about their holidays or what else they are doing.”

What do these and other discussions show us about peer review? “When peer review goes well, it is a system that allows authors to improve the way they communicate their results. It is a unique moment of candid exchange between scientists where anonymity can neutralize the discourse,” Ferlier suggests.

“When it goes wrong, it can be a biased or inefficient quality-control that simply slows down the circulation of scientific knowledge.”

Nature rummaged through the archive for insights into the evolution of peer review.

Some reviews were brief — but not all

By the 1950s, the Royal Society was asking reviewers to respond to standardized questions, including whether a study contained “contributions to knowledge of sufficient scientific interest” and simply whether the society should publish it.

These questions could prompt brief responses even to significant pieces of work. Chemist Dorothy Hodgkin wrote barely 50 words when asked to review the full manuscript of the structure of DNA by Francis Crick and James Watson in 1953, which was published in Proceedings of the Royal Society in April 1954¹. (A shorter paper announcing the discovery had already appeared in Nature².)

In her sole comment, beyond a series of yes and no answers, Hodgkin suggests the duo should “touch up” photographs to eliminate distracting reflections of “chairs in the perspex rod” — a technical fix that modern cameras perform routinely. Crick and Watson seemed to follow the advice.

The archive is also littered with long reports, many in handwritten scrawl. In 1877, reviewer Robert Clifton finished a 24-page report on two related papers on optics, with an apology: “How you will hate me for bothering you with this tremendously long letter, but I hope before we meet time will have softened your anger.”

Ferlier says that the introduction of the standardized referee questions significantly reduced the amount of time and effort put in by reviewers. “There’s really this understanding in the nineteenth century and very early twentieth century that the peer review is a real discussion,” she says. “After that, it becomes a way of managing the influx of papers for the journal.”

Bias was rife

Supporters of double-blind peer review — in which the authors’ identity is withheld from the referee — will find much ammunition in the archive. Many early reviewers refer to the character of the authors and their relationship with them.

For a 1950 paper that discussed ‘anisotropic elastic continuum’ by mathematician James Oldroyd, geophysicist Harold Jeffreys wrote: “Knowing the author, I have confidence that the analysis is correct.”

By contrast, physicist Shelford Bidwell didn’t mince his words about the author of a 1900 article on ‘colour sense’. Bidwell had lent apparatus to author Frederick Edridge-Green, but writes: “I was prepared to find that his new paper was rubbish; and it turned out to be rubbish of so rank a character that no competent person could possibly take any other view of it.”

At least one referee, physicist George Burch, was so concerned that rejecting a paper might give the impression of bias that he recommended publication in 1901 even though he was not convinced by the results and couldn’t replicate them.

Early computing sparked disagreement

When the Royal Society first asked referees to give their opinions on a paper in 1831, the first two reviewers profoundly disagreed on the manuscript’s merits and whether it should be published (it was).

The archive shows a similar stalemate over mathematician Alan Turing’s 1951 paper³ on ‘the chemical basis of morphogenesis’ that proposed new methods of mathematical modelling for the biological field, including the use of computers.

Anonymizing peer review makes the process more just

A negative review by scientist J. B. S. Haldane states: “I consider that the whole non-mathematical part should be re-written.” A more positive opinion came from Charles Darwin’s grandson, physicist Charles Galton Darwin, who said: “This paper is well worth printing, because it will convey to the biologist the possibilities of mathematical morphology.”

Still, Darwin criticized the proposed use of a “digital computer” to simulate wave theory. “The machinery is far too heavy for such a simple purpose,” Darwin argued.

In a move no doubt encouraged by scientists who feel that the peer-review process has mangled their papers, Turing seems to have ignored both sets of comments.

Reviewers helped publishers to cut costs

The notion that peer review should preserve the quality of the scientific record is a modern viewpoint.

The Royal Society archive demonstrates another, less noble, reason to use referees: to help journals control costs. Starting in the late nineteenth century, reviewers were asked to weigh up the soaring price of printing and were asked “are portions of the paper, or any illustrations, redundant?”

In a 1943 review by Haldane of a paper on the fecundity of Drosophila flies, the reviewer suggested cutting figures and tables, some of the discussion and, bizarrely, a short eulogy of a fly biologist. The authors followed the suggestions in full.

Geopolitics influenced publication

Today, journals are cautious when it comes to publishing maps that feature contested territories. In the 1940s, the influence of geopolitics was more acute. Around the time of the Second World War, the Royal Society added a question for referees: could a scientific manuscript somehow help the Nazis?

The researchers using AI to analyse peer review

It was obvious that many papers weren’t of value to the enemy. But some studies prompted concerns and were often referred to military specialists. Selig Brodetsky, who was a fluid-dynamics researcher at the University of Leeds, UK, submitted a paper on aeroplane motion in 1939, but after hostilities began he was told its publication would be delayed until after the war.

Brodetsky accepted the decision, but asked for six copies of his paper — which had been marked ‘SECRET’ — to share with his colleagues for a government project. “I shall of course,” he promised, “take every precaution that no copy gets into unauthorised hands.”