Peer Review Alternatives?

[MSI]

August 24, 2010: In an article in the NY Times Scholars Test Web Alternative to Peer Review they include:

• For professors, publishing in elite journals is an unavoidable part of university life. The grueling process of subjecting work to the up-or-down judgment of credentialed scholarly peers has been a cornerstone of academic culture since at least the mid-20th century
  ...they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience.
  ...scientists and economists ... have created online repositories for unpublished working papers, like http://www.repec.org
  ...mathematicians used blogs and wikis to evaluate a supposed mathematical proof in the space of a week — the scholarly equivalent of warp speed

DISCUSSION: Will objective scientific review be replaced by “collective judgment” which may possibly devolve into some form of popularity contests for scientific discourse? Let’s hope not!

[brian]

Thanks to Doug Milliken for forwarding the following article:
Sept 9, 2010: Peer review highly sensitive to poor refereeing, claim researchers

• Just a small number of bad referees can significantly undermine the ability of the peer-review system to select the best scientific papers. That is according to a pair of complex systems researchers in Austria who have modelled an academic publishing system and showed that human foibles can have a dramatic effect on the quality of published science.
  The research shows that article quality can drop if referees do not behave 'correctly.' At high levels of self-serving or random behavior, 'the peer-review system will not perform much better than by accepting papers by throwing (an unbiased) coin.' The model also includes calculations for 'friendship networks' (nepotism) between authors and reviewers.

• • See the full article
    • Peer review highly sensitive to poor refereeing, claim researchers
  • The paper by Stefan Thurner and Rudolf Hane is
    • Peer-review in a world with rational scientists: Toward selection of the average
  • For our own issues with peer review see Peer Review Process Gone Wild:
    • SAE Accident Reconstruction Session Review Process: A caustic journey from 1997 to 2008

[MSI]

Jan 16, 2012: NY Times: Cracking Open the Scientific Process
which begins:

• "The New England Journal of Medicine marks its 200th anniversary this year with a timeline celebrating the scientific advances first described in its pages: the stethoscope (1816), the use of ether for anesthesia (1846), and disinfecting hands and instruments before surgery (1867), among others."

The article includes a discussion of the Peer review system and some modern day alternatives like:

• arXiv
  Public Library of Science (PLoS)
  GalaxyZoo
  MathOverflow
  ResearchGate

Also included is a discussion of the Research Works Act, introduced in Congress last month, which seeks to protect publishers’ rights by effectively restricting access to research papers and data. In an Op-Ed article in The New York Times last week by Michael B. Eisen, a molecular biologist at the University of California, Berkeley, and a founder of the Public Library of Science, wrote that if the bill passes, “taxpayers who already paid for the research would have to pay again to read the results.”

• See the full article in the NY Times: Cracking Open the Scientific Process