A new tool that selects peer reviewers by algorithm could make the peer review process more reliable, says Richard Price
Technology has changed some aspects of science tremendously. We have deep space telescopes that produce terabytes of data every day. We have tools to create synthetic living cells. However, the system of peer review – the mechanism by which scientific results are vetted – hasn’t materially changed since the 1600s when journal publishing was invented.
The reliability of the peer review process has recently come under the spotlight when the journal Science wrote a fake biology paper and submitted it to a number of open access journals. The fake paper made it through the peer review process of the majority of the journals.
Pharmaceutical companies have also drawn attention to the fact that the majority of peer reviewed papers in the biomedical space are not reproducible. Amgen reported that 89% of peer reviewed articles are not reproducible. Bayer’s estimation stands at64%. The Economist has also discussed studies that question the reliability of the peer review process.
Part of the issue is that in the traditional peer review process, only two or three scientists peer review a paper. The system places too much weight on what a small number of scientists think. There are more than 50,000 people working in a field like breast cancer. What two or three people think is too small a sample size: the signal is not statistically significant. We need to expand peer review so that there are vastly more scientists peer reviewing papers, sharing their thoughts and comments on each other’s papers.
In the past, it hasn’t been possible for journals to solicit peer reviews from a large number of people, because the process by which journal editors find peer reviewers for a paper is manual and labour intensive. That may change with large communities of scientists emerging online: Academia.edu has 5 million academics on its platform; ResearchGate lists over 3 million and Mendeley has 2.6 million.
If we combine these large communities with technology currently available, we can algorithmically determine the top scientists in a given field, and weight their peer reviews accordingly. The goal is to build a system that incentivises scientists to share their peer reviews and thoughts on the papers they are reading. Science is a conversation, and progress happens through the rapid discovery of errors, and quickly learning from those errors and moving on.
Cue the acquisition by Academia.edu of Plasmyd, a peer review and discussion platform for papers. The goal is to tackle the next building block in open science: building a better peer review system. The integration will combine Plasmyd’s experience in fostering peer review with Academia.edu’s research community and experience developing new ways of measuring reputation in research.
Humanity faces many problems that require science-based solutions. We need to find cures for Alzheimers, Parkinson’s, HIV, malaria and cancer. The planet is on a path to self-destruction with increasing carbon emissions and we need to develop a carbon-free source of energy that works at scale.
The goal of open science is to accelerate scientific discovery by making the process of science more open. Part of open science is getting every science PDF that has ever been written onto the internet and accessible for free. Another part is building a better peer review process, one that is more open and does a better job at separating the reproducible research from the non-reproducible research.
Richard Price is founder and CEO at Academia.edu –follow him on Twitter @richardprice100
Ada Lovelace Day: how much do you know about women in science? quiz
Blog,To mark Ada Lovelace Day, a celebration of women in science, technology, engineering ...
Open access is not enough on its own data must be free too
Blog,Academics have been encouraged to make their research freely available, but their dat...
If you want creativity, let artists and technologists work together
Blog,Universities should stop trying to turn out jacks-of-all-trades and give students the...
DIY labs are an exciting alternative to university science research
Blog,Free from bureaucracy, independent science labs offer a flexibility that can't be mat...
Will a degree made up of Moocs ever be worth the paper it's written on?
Blog,The University of the People can now hand out degrees to its online students but wil...
Big data can transform learning as long as lecturers take control
Blog,The collection of data on a large scale has already revolutionised our experience of ...
Ten reasons we should ditch university lectures
Blog,Students have just one chance to hear a lecture - and mostly it's just someone readin...
Collecting data on students: is it useful to know which books they've read?
Blog,Universities can gain valuable information by tracking students' behaviour online, bu...
Welcome to the age of Martini marketing – any time, any place, anywhere
Blog,University marketers have to develop digital campaigns that operate round-the-clock...
Have traditional student recruitment campaigns lost their bite?
Blog,Universities are finding more creative ways to recruit students – through soc...
How to organise an academic conference | 10 tips
Blog,Be picky about your speakers and location, says Brian Lucey, and make sure you run ...
Europe needs to map its research base
Blog,Could a Europe-wide database of facilities and equipment help research, funding and...
Leaving academia? How to sell yourself to new employers
Blog,If you fancy your chances at working in the private sector, here are some expert ti...
Unlocking chemistry: it's time to make the subject as open as bioscience
Blog,Now that millions of patented compounds are open information, chemistry has a chanc...
Universities quiz: 10 questions on 2013
Blog,Kept up with the twists and turns of higher education news and views this year? Put ...
Academic blogging – 10 top tips
Blog,Find your authentic blogging voice, harness the power of social media and, remember...
In praise of the university lecture and its place in academic scholarship
Blog,The lecture is not just for students, says Frank Furedi – it helps academics ...
Academic blogging: a risk worth taking?
Blog,After her blog was republished without credit, PhD student Lucy Williams says we mu...
University data can be a force for good
Blog,Data analytics shouldn't be seen as a dark art but a tool to aid student retention ...
Open data: we need to share research results, even when they are wrong
Blog,There are huge flaws in the way research data is uploaded, says Mark Hahnel, but ho...
Ada Lovelace day: share your stories of women in science and technology
Blog,Help us celebrate the achievements of women in science, technology, engineering and ...
Higher education in 2020: three key forecasts from new report
Blog,Findings show that people (not technology) will drive change, revenue is the ultima...
FutureLearn courses mark UK's entry into global online learning race
Blog,UK's first homegrown Mooc platform launches with 23 universities offering free onli...
FutureLearn and free courses: is this what students really want?
Blog,Moocs are an online product of current higher education offline, says Stephen Caddi...
Home Technology Specialists of America Launches New Marketing Tools for Members
Blog,Continuing to improve on collaborative marketing programs with members and vendors, ...
University libraries: 10 global portraits
Blog,University library chiefs worldwide reveal the challenges they face, plans for the ...
Library futures: Tama Art University, Japan
Blog,A university library designed by a world-leading architect inspires and challenges ...
Library futures: Nanyang Technological University, Singapore
Blog,Finding information is not the problem, says Choy Fatt Cheong – libraries are...
Home Technology Specialists of America Adds New Member, Modern Home Systems
Blog,Helping to further grow the HTSA network of home technology specialists, Modern Home...