Great piece as always James - I think if science is a process (which most agree with as a concept) then that process should, in a very meta sense, include questioning the process. Kudos on doing so or at least shedding light on it for others to understand
Thanks for your support (as always!). Indeed - science is a process, and we need to be vigilant in regularly checking in that we are on the right track.
Thanks! As university faculty who are regularly involved in peer-review (as authors and reviewers) and understand the system, it's easy to overlook that non-academics may not be familiar with just how "human" the process is. So, I always like to shed light on this issue. Thanks for sharing with your students!
I will be teaching an evidenced-based health course in the Fall; and I will have a lecture on the publication process. This will be a great read for my students.
Great piece James. I concur as my experience was equally patchy over my 80 peer reviewed contributions and the public servants I advised thinking that science papers came down from the mount. Might be the least of our worries mind you.
As far as all of the time spent reviewing, that will definitely be the topic of a future post! There's definitely a balance to strike between doing service to help the scientific process versus volunteering one’s time to help a billion-dollar publishing company make money.
Thank you, James. But the problem is much worse than what readers are likely to take away.
I am very familiar with a number of papers in top journals where the papers have very serious errors that the reviewers and editors should have caught. Why didn't they? Who knows.
Will the editors correct them later? No.
Will the editors accept and print letters that carefully and respectfully point out the problems. No, they often will not. Why? I have receipts.
Are editors afraid of looking bad for allowing such flawed work to be published in their journals? They give that impression.
Is there a strong publication bias towards papers that affirm what people want to believe, even when the papers are seriously flawed?
Granted, reviewers are busy people and not paid to do reviews. So how can we expect them to dig deeply enough into each paper to find the serious flaws? Well, some of the errors are obvious to readers who pay attention. Is there a real solution?
Great piece as always James - I think if science is a process (which most agree with as a concept) then that process should, in a very meta sense, include questioning the process. Kudos on doing so or at least shedding light on it for others to understand
Thanks for your support (as always!). Indeed - science is a process, and we need to be vigilant in regularly checking in that we are on the right track.
Great post. Very informative. I'm going to have my university students read it!
Thanks! As university faculty who are regularly involved in peer-review (as authors and reviewers) and understand the system, it's easy to overlook that non-academics may not be familiar with just how "human" the process is. So, I always like to shed light on this issue. Thanks for sharing with your students!
I will be teaching an evidenced-based health course in the Fall; and I will have a lecture on the publication process. This will be a great read for my students.
Peer review must be compensated. Until it is, it will continue to be substandard.
Great piece James. I concur as my experience was equally patchy over my 80 peer reviewed contributions and the public servants I advised thinking that science papers came down from the mount. Might be the least of our worries mind you.
Cudos on finding time for all that reviewing too.
Thanks, John!
As far as all of the time spent reviewing, that will definitely be the topic of a future post! There's definitely a balance to strike between doing service to help the scientific process versus volunteering one’s time to help a billion-dollar publishing company make money.
Oh my, don't tell them that, they will think all science is a scam🤫
Thank you, James. But the problem is much worse than what readers are likely to take away.
I am very familiar with a number of papers in top journals where the papers have very serious errors that the reviewers and editors should have caught. Why didn't they? Who knows.
Will the editors correct them later? No.
Will the editors accept and print letters that carefully and respectfully point out the problems. No, they often will not. Why? I have receipts.
Are editors afraid of looking bad for allowing such flawed work to be published in their journals? They give that impression.
Is there a strong publication bias towards papers that affirm what people want to believe, even when the papers are seriously flawed?
Granted, reviewers are busy people and not paid to do reviews. So how can we expect them to dig deeply enough into each paper to find the serious flaws? Well, some of the errors are obvious to readers who pay attention. Is there a real solution?