PEER REVIEW DOUBTS

Is peer review really the Holy Grail of science?

For 300 years peer reviewed papers have served science and scholarly publishing without question, until recently. In the words of Wikipedia, peer review works like this:

Peer review is the evaluation of work by one or more people of similar competence to the producers of the work (peers). It constitutes a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are employed to maintain standards of quality, improve performance, and provide credibility in academia. Scholarly review is often used to determine an academic paper’s suitability for publication. Peer review can be categorized by the type of activity and by the field or profession in which the activity occurs.

In a nutshell, peer review helps validate research findings submitted to science journals for publication. The journal will invite other scientists and researchers to comment on the paper before deciding to publish it. The review may take one of several forms. The single blind review is one in which the name(s) of the reviewer(s) are hidden from the author. In a double-blind review the reviewer’s and author’s names are not disclosed. Finally, there is the open review in which the author and reviewer are known to each other. Each review type has its own advantages and disadvantages and there is no perfect system. Personal bias can, and often does, play a part, and the author, whether the name is revealed or not, can often be identified by the writing style or topic, and a reviewer may be influenced by the standing, or lack of standing, of the author.

Acceptance of established scientific principles can change over time and an author with a paper revealing new discoveries may often receive an adverse review from a reviewer who supports the status quo or simply goes with the consensus opinion. History is full of discoveries that were harshly criticized by the establishment but later became mainstream thinking.

In 2006, the Journal of the Royal Society of Medicine published this:

Peer review is at the heart of the processes of not just medical journals but of all of science. It is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won. Yet it is hard to define. It has until recently been unstudied. And its defects are easier to identify than its attributes. Yet it shows no sign of going away. Famously, it is compared with democracy: a system full of problems but the least worst we have.

Perhaps the most damning comment come from The Guardian:

Peer review is the process that decides whether your work gets published in an academic journal. It doesn't work very well any more, mainly as a result of the enormous number of papers that are being published (an estimated 1.3 million papers in 23,750 journals in 2006). There simply aren't enough competent people to do the job. The overwhelming effect of the huge (and unpaid) effort that is put into reviewing papers is to maintain a status hierarchy of journals. Any paper, however bad, can now get published in a journal that claims to be peer-reviewed.

Journals themselves can be biased and actively seeking research findings that suit the publisher’s bias and seeking reviews from reviewers known to also be biased. This makes a mockery of peer reviewing. 

See also:
Media Bias and Fake News
Climate Cyber Bullies
Australia and Climate Change
Elections and Social Media

During the lifetime of this writer, many changes to CPR (cardio-pulmonary resuscitation) have taken place. But each change found bitter resistance because the current method was developed by ‘experts’ and peer reviewed to give it extra authority. My introduction to CPR in the 1950’s, then known as artificial resperation, was to learn the Holger Neilsen technique for use in reviving workmates who had been ‘killed’ by contact with electric power transmission lines. We practised on ‘victims’ hanging in safety harness 30 feet above the ground. The rescuer would climb a ladder, secure his own harness, and perform the revival from behind the victim by moving his arms back and forth to fill the lungs and restart breathing. I never knew of a case where it worked, but it was a universally accepted technique. Present day first aid people would scoff at such an idea, but in its day the Holger Nielsen technique was only questioned by fools, sceptics and agitators. While peer review can help introduce revolutionary new scientific discoveries, it can also block the acceptance of new scientific discoveries.

Economist Professor George J. Borjas

Economics is a field where opinion and data are often disputed, always has been disputed and probably always will be disputed. It is a controversial area of learning. Professor George J. Borjas wrote in his blog:

I have a few pet peeves. One of them is how “peer review” is perceived by far too many people as the gold standard certification of scientific authority. Any academic who’s been through the peer review process many times (as I have) knows that the process is full of potholes and is sometimes subverted by unethical behaviour on the part of editors and reviewers.

Unethical behaviour? Some authors have even been caught peer reviewing their own work.

In recent years the peer review system has become such a shambles that some of the leading journals now knock back everything that doesn’t conform to their own pre-conceived idea of the world and the way it should be. This is the exact opposite of what peer review was supposed to achieve.

Holger Louis Nielsen, Danish Olympian
and creator of Holger Nielsen
artificial respiration

Mention has already been made of economics and CPR and how change has been opposed. Man-made climate change, and its dire consequences for the planet, has found widespread public acceptance. But, interestingly, it has found less acceptance in the academic community, particularly among meteorologists, climate scientists, geologists and historians. But they are largely shouted down by those citing peer reviewed papers.

Peer review is frequently used, by those claiming scientific backgrounds, to silence people who lack a PhD in some scientific discipline. On social media these people, and those citing their work, frequently lambast their critics as ignorant, or challenge them to list their own peer reviewed papers. The idea that anyone without a PhD and a peer reviewed paper is of no consequence, or is an ignorant meaningless individual, is repugnant.

I wonder, if John D. Rockefeller were alive today, would the ivory tower crackpots want to peer review his business plan and tell him it wouldn’t work because he hadn’t been to university. Astronaut John Glenn, author Mark Twain, and industrialist Henry Ford would have failed the PhD/peer review test too. To that list can be added William Shakespeare, Winston Churchill, Abraham Lincoln, Richard Branson, Ted Turner, George Eastman and John Major. All dropped out of school before obtaining a degree. Are we to believe that government funded researchers are smarter than the successful men listed above? I don’t think so.

It is easy to write-off people who are uneducated. When I was growing up in rural Northland, New Zealand, I was fortunate to live next door to one of the wisest and most respected men I have ever known. He was in his seventies and was the most successful farmer in the district. He had taught himself to read and write, had a house full of books on every subject under the sun, and could talk with, or debate with, the best on any subject. After spending many hours, days and years in his shadow, I knew that a man or woman does not need to be educated to be a genius.

Peer reviews have become a joke. However, if someone genuinely desires a peer review on the future of the climate, my suggestion would be to take a wander down to your nearest seaside pier and ask a crusty old fisherman what he thinks about the future climate. He will tell it the way it is, the way it could be tomorrow, and beyond that summer will follow winter and winter will follow summer the way it always has, and always will.

One pier review will be worth a thousand peer reviews every time.