Scientific peer reviewed publications

phyllo · October 22, 2014, 1:35pm

Someone on ILP recently asked if a poster’s ‘100% scientific’ claims had been peer reviewed. I realize that this is just a tactic to dismiss claims without actually examining them. After all, who will be making a thread on a philosophy forum about published scientific research?

Anyways, back to the real world : Some people are not aware of the problems with peer review and trust it much more than is warranted. Here is something to think about:

Academic publishing was rocked by the news on July 8 that a company called Sage Publications is retracting 60 papers from its Journal of Vibration and Control, about the science of acoustics. The company said a researcher in Taiwan and others had exploited peer review so that certain papers were sure to get a positive review for placement in the journal. In one case, a paper’s author gave glowing reviews to his own work using phony names.

Acoustics is an important field. But in biomedicine faulty research and a dubious peer-review process can have life-or-death consequences. In June, Dr. Francis Collins, director of the National Institutes of Health and responsible for $30 billion in annual government-funded research, held a meeting to discuss ways to ensure that more published scientific studies and results are accurate. According to a 2011 report in the monthly journal Nature Reviews Drug Discovery, the results of two-thirds of 67 key studies analyzed by Bayer researchers from 2008-2010 couldn’t be reproduced.

That finding was a bombshell. Replication is a fundamental tenet of science, and the hallmark of peer review is that other researchers can look at data and methodology and determine the work’s validity. Dr. Collins and co-author Dr. Lawrence Tabak highlighted the problem in a January 2014 article in Nature. “What hope is there that other scientists will be able to build on such work to further biomedical progress,” if no one can check and replicate the research, they wrote.

The authors pointed to several reasons for flawed studies, including “poor training of researchers in experimental design,” an “emphasis on making provocative statements,” and publications that don’t “report basic elements of experimental design.” They also said that “some scientists reputedly use a ‘secret sauce’ to make their experiments work—and withhold details from publication or describe them only vaguely to retain a competitive edge.”

Papers with such problems or omissions would never see the light of day if sound peer-review practices were in place—and their absence at many journals is the root of the problem. Peer review involves an anonymous panel of objective experts critiquing a paper on its merits. Obviously, a panel should not contain anyone who agrees in advance to give the paper favorable attention and help it get published. Yet a variety of journals have allowed or overlooked such practices.

Absent rigorous peer review, we get the paper published in June in the Proceedings of the National Academy of Sciences. Titled “Female hurricanes are deadlier than male hurricanes,” it concluded that hurricanes with female names cause more deaths than male-named hurricanes—ostensibly because implicit sexism makes people take the storms with a woman’s name less seriously. The work was debunked once its methods were examined, but not before it got attention nationwide.

Such a dubious paper made its way into national media outlets because of the imprimatur of the prestigious National Academy of Sciences.

Yet a look at the organization’s own submission guidelines makes clear that if you are a National Academy member today, you can edit a research paper that you wrote yourself and only have to answer a few questions before an editorial board; you can even arrange to be the official reviewer for people you know. The result of such laxity isn’t just the publication of a dubious finding like the hurricane gender-bias claim. Some errors can have serious consequences if bad science leads to bad policy.

In 2002 and 2010, papers published in the Proceedings of the National Academy of Sciences claimed that a pesticide called atrazine was causing sex changes in frogs. As a result the Environmental Protection Agency set up special panels to re-examine the product’s safety. Both papers had the same editor, David Wake of the University of California, Berkeley, who is a colleague of the papers’ lead author, Tyrone Hayes, also of Berkeley.

In keeping with National Academy of Sciences policy, Prof. Hayes preselected Prof. Wake as his editor. Both studies were published without a review of the data used to reach the finding. No one has been able to reproduce the results of either paper, including the EPA, which did expensive, time-consuming reviews of the pesticide brought about by the published claims. As the agency investigated, it couldn’t even use those papers about atrazine’s alleged effects because the research they were based on didn’t meet the criteria for legitimate scientific work. The authors refused to hand over data that led them to their claimed results—which meant no one could run the same computer program and match their results.

Earlier this month, Nature retracted two studies it had published in January in which researchers from the Riken Center for Development Biology in Japan asserted that they had found a way to turn some cells into embryonic stem cells by a simple stress process. The studies had passed peer review, the magazine said, despite flaws that included misrepresented information.

Fixing peer review won’t be easy, although exposing its weaknesses is a good place to start. Michael Eisen, a biologist at UC Berkeley, is a co-founder of the Public Library of Science, one of the world’s largest nonprofit science publishers. He told me in an email that, “We need to get away from the notion, proven wrong on a daily basis, that peer review of any kind at any journal means that a work of science is correct. What it means is that a few (1-4) people read it over and didn’t see any major problems. That’s a very low bar in even the best of circumstances.”

But even the most rigorous peer review can be effective only if authors provide the data they used to reach their results, something that many still won’t do and that few journals require for publication. Some publishers have begun to mandate open data. In March the Public Library of Science began requiring that study data be publicly available. That means anyone with the ability to check should be able to reproduce, validate and understand the findings in a published paper. This should also ensure that there is much better scrutiny of flawed claims about sexist weather events and hermaphroditic frogs—before they appear on every news station in America.

online.wsj.com/articles/hank-cam … 1405290747

phyllo · October 22, 2014, 1:39pm

Mods , please don’t move this into the Science Forum. This is something for the layman.

Dan1 · October 22, 2014, 11:34pm

The only people fit to review something have to be both virtuous and smart.
A fool that is friendly still can’t understand or do what is most advanced.
A smart but corrupt man will produce little good for the world.

When I hear the words “Peer review”, I want to laugh, and I also get pissed off.

HatingMeIsEasier · October 23, 2014, 12:08am

Do you consider me smart and corrupt because of my controversial demeanor?

I’ll just say that I am no more harsh than the Universe is on its creations. a.k.a it takes a tremendous amount of time and heat for a diamond to emerge from elements of life yet to be distinguished in terms of design.

fuse · October 23, 2014, 12:41am

Thank goodness the peer review process is reviewed! The conclusion of the article isn’t that peer review is untrustworthy, but that peer review needs to be protected from abuse. Peer review is just a vetting process. The more stringent the vetting, the more reliable the work is likely to be. You still have to do your own research and verification when it really matters.

Mr_Reasonable · October 23, 2014, 2:51am

This reminds me of my job in college. We were funded by ORI, (office of research integrity), which is a division oh HHS. Basically, we worked on creating educational tools for scientists with the goal of ensuring that they didn’t “accidentally” falsify data and run the risk of losing their funding. It’s a tricky business.

HatingMeIsEasier · October 23, 2014, 3:02am

This goes back to who watches the watchmen?

Transposed into:

Who peer-reviews the peer-reviewers?

Mr_Reasonable · October 23, 2014, 3:08am

The best. Or the ones with the power. Or each other. You pick.

HatingMeIsEasier · October 23, 2014, 3:57am

The best, clearly. Hands down.

AutSider · October 23, 2014, 2:31pm

phyllo I assume your OP is directed at me. Whether peer reviewed scientific researches are reliable or not is irrelevant in the context of the point I was trying to make. He claimed that his theory is 100% scientific. I asked him to back up his claim. That’s all.

phyllo · October 23, 2014, 3:53pm

You wanted him to back it up with a peer reviewed paper. If he could not provide such a paper, then it was your grounds for dismissing him.
Science can be good, even if not peer reviewed. Peer reviews can support bad or even false science. Scientists have faked results and had peer reviewed papers published.

This is what was said:

James_S_Saint · October 23, 2014, 4:13pm

Atheris, can you give us even one link to a 100% scientifically peer reviewed paper (within the last century)?

AutSider · October 23, 2014, 8:20pm

I wanted him? He’s the one who claimed it’s 100% scientific. That’s all I wanted him to back up, his own goddamn assertion.

My time is to precious to be wasted arguing with trolls who can’t get even the simplest things straight.

James S Saint, remember when you said this?

What’s next, you’re going to tell me that food is consumed through ears and then expect me to take you seriously?

HatingMeIsEasier · October 23, 2014, 8:23pm

phyllo:

phyllo I assume your OP is directed at me. Whether peer reviewed scientific researches are reliable or not is irrelevant in the context of the point I was trying to make. He claimed that his theory is 100% scientific. I asked him to back up his claim. That’s all.

You wanted him to back it up with a peer reviewed paper. If he could not provide such a paper, then it was your grounds for dismissing him.
Science can be good, even if not peer reviewed. Peer reviews can support bad or even false science. Scientists have faked results and had peer reviewed papers published.

This is what was said:

It’s 100% Science.

Atheris wrote :
I’m sure that it has been 100% peer reviewed then! Give me the linky to the peer reviewed research paper!

Atheris wrote :
I asked for a link to the peer reviewed scientific paper, since you claim it is “100% scientific”. If you fail to provide it again I know just how seriously I should take you from now on.

Atheris wrote :
Still no link to the peer reviewed scientific research to your topic.

Don’t forget. He hasn’t provided anything!

I provided a formula: Fibonacci Sequence [“f” represents numbers from the Fibonacci Sequence]:

Practical Implication - 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55? and 89?

Planetary Implication - 0 (Moonless Mercury) 1 (Moonless Venus) 1 2 (Earth & Mars) 5 (Pluto) 14(?) (5+8f=13/Neptune) 27 (14+13f=27/Uranus) 62

(27+14+21f=62/Saturn) 70 (Jupiter)

The “Solar System Was Shuffled Between Earth, Mars and Pluto For A Reason” Theory: 70 - 62 = 8. 1 + 2 + 5 = 8. The Solar System was Shuffled, that is why the planet’s moons are in a weird order. When you put the planets in order of having least moons to having most moons, you will see that this theory is connecting the Planet System to the Fibonacci Sequence. Not only that, but the two are “intertwined”.

I provided a picture for reference to my genius work:

I provided an explanation as to why the Universe shuffled at Pluto (it has 5 moons): Space bends in the Universe, so the mathematical properties of space also bends in accordance to how it bends. Here on this planet, the Fibonacci Sequence is 100% chronological, but here on this planet, space does not bend. In fact, space bends in our heads hence why we have past, present and future tense. This world we live in is the physical model of a metaphysical system. Our brains are swimming in the metaphysical system and interacting with the physical model. Look at that picture - do you see where the numbers separate at 5? It’s an advanced version of the Fibonacci Sequence. This is an incredible formula that should be heeded regardless of your ignorance and impatience for true knowledge.

I have other theories that are profoundly plausible: When someone has more than one theory that makes you want to rip your hair out because you can’t destroy their argument as you could when it came to religious people and conspiracy theorists, that’s when you need to just stop talking and start listening because you found yourself a genius.

I solved the mystery of Uranus’ tilt, which no one on this planet could solve: I explained how the subatomic particles shaped the physical objects like biology shapes our physical appearance and physical apparatuses. I showed pictures of normal orbitals of methane that took the shape of Neptune and I showed pictures of abnormal orbitals of methane that took the shape of Uranus.

I took the time to explain sex on a cosmic level:

I explained why rainbows never change their colors: The colors are ROYGBIV. These colors are not just colors - they are the Universe’s stages of personality ranging from low frequencies designed for our survival to higher frequencies designed for our “soul”. Our planet from the core to outside of this planet is in this order of colors. Also the Zooid, a creature found in the ocean from the surface of the ocean to the bottom are found in this order of colors.

This isn’t even the end of what I have accomplished: Ignorant people attack my intelligence just like fat people attack people for being fit. It’s not news to me. I just wish people would of caught on by now, so they can save me from explaining myself by shaming these fools for attempting to bring people like me down to their level.

James_S_Saint · October 23, 2014, 8:43pm

Sooo… your are saying that you are a waste of your own time?
Is that like … nihilism of time or something?

Atheris:

James S Saint, remember when you said this?
9 out of 10 women who have gone into strip club dancing will have only female children.
What’s next, you’re going to tell me that food is consumed through ears and then expect me to take you seriously?

So you think that because someone states something that you take out of context and he hasn’t bothered to debate it with a moron, he is more ignorant as you? I tend to suspect the reverse.

And I take it that you said that (and probably all you say) merely to avoid answering the questions put to you;

AutSider · October 23, 2014, 8:53pm

Context? It’s in the Science section, so you can’t make up the excuse that you meant it figuratively or whatever. It’s a proposition with a truth value, it’s either false or true. Until you admit you’re wrong on obvious things I don’t see why I should take you or your challenges seriously.

AutSider · October 23, 2014, 8:55pm

It’s fun, all I did was ask for a person to back up their claim with evidence that should be easy to provide if the claim is true and people start bitching and moaning about something completely different pretending they are addressing my point. Pretty much sums up the mentality here.

James_S_Saint · October 23, 2014, 8:59pm

What is “obvious” to you, is also known as “common stupidity”.
You cannot support your objections to anything even as much as others support their assertions.

You simply do the childish, “I’ll rant to them how wrong that are!”
…although I’ll admit that you aren’t the worst of them here lately.

And of course, as always, you fear answering direct questions.

fuse · October 24, 2014, 1:59am

I don’t know what other thread you all are arguing about but this is true.

HatingMeIsEasier · October 24, 2014, 2:24am

My thread explaining why Uranus tilts because of degenerate orbitals.

When someone can’t make sense of your established theory, they either ignore you, attack you or reduce you. In this case, Arthritis is reducing me by lying to himself about the credentials of my theory that I have pictures, data, plausible explanations and so on - I mean for cosmic sake, I have the entire Universe’s Fibonacci Sequence figured out, Arthritis never touched that because he knows he can’t reduce something so incredibly assessed. There are some things that are simply too intelligent for the ignorant that they legitimately stay far from it. They know anything they say will defeat their facade right there and then.