http://fivethirtyeight.com/features/science-isnt-broken/

Christie Aschwanden writes that "the headline-grabbing cases of misconduct and fraud are mere distractions. The state of our science is strong, but it’s plagued by a universal problem: Science is hard — really fucking hard.

"If we’re going to rely on science as a means for reaching the truth... it’s important that we understand and respect just how difficult it is to get a rigorous result. I could pontificate about all the reasons why science is arduous, but instead I’m going to let you experience one of them for yourself. Welcome to the wild world of p-hacking."

Hit the link for an interactive demo that lets you easily find a statistically significant (hence "publishable") result that Democrats or Republicans have been good or bad for the economy.

"The p-value reveals almost nothing about the strength of the evidence.... 'researcher degrees of freedom'... include things like which observations to record... which factors to control for, or, in your case, whether to measure the economy using employment or inflation numbers (or both). Researchers often make these calls as they go, and often there’s no obviously correct way to proceed, which makes it tempting to try different things until you get the result you’re looking for.... 'You really believe your hypothesis and you get the data and there’s ambiguity about how to analyze it.'...

"...perhaps playing around with different methods... [should be] encouraged... Nosek’s team invited researchers to take part in a crowdsourcing data analysis project. The setup was simple. Participants were all given the same data set and prompt: Do soccer referees give more red cards to dark-skinned players than light-skinned ones?...

"Despite analyzing the same data, the researchers got a variety of results. Twenty teams concluded that soccer referees gave more red cards to dark-skinned players, and nine teams found no significant relationship between skin color and red cards... '...there’s a there there. It’s hard to look at that data and say there’s no bias against dark-skinned players.'"

The article goes on to discuss the rise in retractions of journal articles, coinciding with the growth of new journals; as well as predatory "pay-to-publish" journals with essentially no standards, and both honest mistakes and outright fraud in legitimate journals' peer reviewing.

On the question of whether science is broken: "Science is not a magic wand that turns everything it touches to truth. Instead, 'science operates as a procedure of uncertainty reduction. The goal is to get less wrong over time.'...

"The scientific method is the most rigorous path to knowledge, but it’s also messy and tough. Science deserves respect exactly because it is difficult — not because it gets everything correct on the first try. The uncertainty inherent in science doesn’t mean that we can’t use it to make important policies or decisions. It just means that we should remain cautious and adopt a mindset that’s open to changing course if new data arises. We should make the best decisions we can with the current evidence and take care not to lose sight of its strength and degree of certainty. It’s no accident that every good paper includes the phrase 'more study is needed' — there is always more to learn."

[Ellipses and emphases mine. These are snippets from a long yet worthwhile article -- read the whole thing!]

Views: 267

Replies to This Discussion

Speaking of weak but "publishable" effects, over and over we see expensive new drugs being approved whose clearest benefits aren't to the patients, but to the coffers of the pharmaceutical companies. They might be only marginally "better" than previous treatments -- or might treat an entirely manufactured "ailment"! -- and have significantly more dangerous side effects.

Yeah, GB, it's interesting and important that laws require drug advertisers to disclose the dangers of pharmaceutical company laboratories. Many such disclosures mention the dangers to life and health.

It's important to life and health to ask conservative which regulations they will repeal. 

I read it all, and liked it.

Science does face a risk: many projects cost so much that they require funding by taxpayers.

Getting such funding requires persuading politicians. American education got a needed boost in the 1960s after the Soviet Union sent a man  into earth orbit in the late 1950s. The Cold War was warm then and America's leaders realized that the USSR could use the same rocket to send weapons into earth orbit.

The fear persuaded Congress to provide funds to improve math and science education. Putting Americans on the Moon became a non-war-related goal and funding began. It was a coincidence then that a Catholic priest a few years earlier had interpret findings by Edwin Hubble to support the use of Genesis by Catholicism and other religions to explain the universe.

A few scientists had questioned that use of Hubble's finding but most wanted to do science, not politics.

The people who knew space became necessary to defend America from Soviet imperialism. The funding began and the "Big Bangers" soon realized that they had to persuade taxpayers to continue their support.

Among the early "proofs" was the claim that radio-frequency signals from space were "echoes from the Big Bang", the ex nihilo (everything from nothing) Genesis account.

Despite the use of vast amounts of taxpayer money, the claims of Big Bangers are all untested and untestable by scientific experiment.

The imaginative thinking required by the production of science fiction supports today's cosmology and taxpayer money pays the salaries of those who do that thinking.

Stopping a taxpayer-supported program is not easy.

I am currently reading the book Merchants of Doubt.  I'm only two chapters in, but the more I read, the more I'm convinced that it should be required reading for us and anyone who cares both about science and the abuse of science which some organizations with specific agendas may attempt.  Their approach is rather like The Big Lie - they figure if they lie long and loudly enough, their lie will be taken for the truth.  In the case of the problems associated with tobacco, this worked for a considerable while.  Similar efforts regarding issues such as President Reagan's Strategic Defense Initiative ("Star Wars"), depletion of the ozone layer and climate change have varying backgrounds, some of which go a lot longer than I originally thought.

The one commonality predictably appears to be that those who are lying do so because they want to continue doing things which are ultimately deleterious to us, the planet, or both ... and they have to be fought hard ... NOW.

Loren, I'm less certain than you that they want to do things "deleterious to us, the planet, or both".

My years in the retail business and later in hardball politics tells me a few people want stuff for themselves and like sociopaths don't care if it's deleterious to us, the planet, or both.

Their lies help them get what they want.

Tom, my point is that such people are more about short-term profits than they are about long-term consequences.  They want what they want, and being reminded about those consequences just gets in their way.

Bogus allegedly peer reviewed journals make real science harder. Lobbyists can submit crap and use it to attack real science.

Bogus Journal Accepts Profanity-Laced Anti-Spam Paper

The International Journal of Advanced Computer Technology has accepted for publication a manuscript that was first written in 2005 to protest spam conference invitations.

After receiving a spam email from the International Journal of Advanced Computer Technology, Dr. Peter Vamplew of Federation University Australia’s School of Engineering and Information Technology sent the anti-spam article as a reply to the spam email without any other message, expecting that they might open it and read it, but not that it would be considered for publication.

To his surprise, the journal accepted the paper and sent him an acceptance email that had two PDF attachments. One was a formal statement of acceptance and the second was the reviewer report.

The paper "Get me off Your Fucking Mailing List" has two figures that are good for a laugh. And it even has section headings, subheadings, and a couple of references! No scientific content needed, or even linguistic content beyond one repeated sentence.

The predatory (they're pay-to-publish) "journal"'s website is stunning in its sloppiness. (BTW, anyone here who hasn't read the discussion "Decline in Writing Accuracy" might enjoy it.)

So that's how climate deniers can get published! (In different "journals" with similarly nonexistent standards.)

I once saw a review form from a real journal, where the reviewer could rate a paper along a scale from "One of the most significant papers of the decade" through "Accepted", "Revision needed", and "Rejected" to "Wrap fish with this."

Climate deniers might get published in real journals very rarely (5 articles out of 24,210 analyzed by James Lawrence Powell), but as a rule their papers don't get cited by (other) scientists (one article did -- only once).

"Do 97% of Scientists Really Agree on Climate Change? Nope. It’s More Like 99.9%, Says Expert"

( http://www.patheos.com/blogs/friendlyatheist/2016/01/14/do-97-of-sc... )

Bogus allegedly peer reviewed journals make real science harder. Lobbyists can submit crap and use it to attack real science.

Such journals are an instance of America's free market.

That market has long made possible and profitable, and will continue to make possible and profitable, as many swindles as homo sapiens sapiens can devise and/or be victimized by.

It's natural selection at its most deliberate (Ma and Pa Nature's contribution) and CUNNING (our contribution).

What to do besides:

1. Finding a benevolent despot (which has been tried often), or

2. Acting on the reality that to the extent possible we are each others' "keepers"?

RSS

About

line

Update Your Membership :

Membership

line

line

Nexus on Social Media:

line

Latest Activity

© 2018   Atheist Nexus. All rights reserved. Admin: Richard Haynes.   Powered by

Badges  |  Report an Issue  |  Terms of Service