19. The Science Episode: P-Hacking, the Banality of Replication Studies, and other Perverse Incentives in the Scientific Community
I'M THE VILLAIN
English - October 30, 2019 12:01 - 40 minutes - 55.5 MB - ★★★★★ - 41 ratingsSociety & Culture Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed
If you are a scientist, you know that the frenzy for funding can be fierce, which is why, as with any system where people are competing for a pool of money, there can be perverse incentives to fudge the numbers, or select your variables to give you the most significant result. This practice, known as "p-hacking," is rampant, as argued by a mathematician known as Jordan Ellenberg, whose book Deondre' read recently, and whose talk on the subject is linked below. (There's a great explanatory interactive that explains p-hacking made by 538 linked below as well.)
In this episode, we talk with Tayeb Zaidi, a biomedical physicist at the FDA, about the various way in which science isn't always the well oiled machine of truth that some people think it is. Replication studies, which are the mechanism of 'checking' other scientist's work by doing the same experiment again to see if you got the same results, aren't sexy, especially when scientists usually got into the field to discover new things in the first place, and it's hard to build a reputation for yourself as a scientist by re-doing other people's work. On top of that, they often aren't well-funded, because funding organizations would also like to spend time and money discovering new things over replicating other studies.
Links:
The 538 Guide to P-Hacking: https://projects.fivethirtyeight.com/p-hacking/
How Not to Be Wrong Book: https://www.amazon.com/How-Not-Be-Wrong-Mathematical/dp/0143127535
Debunking the power pose study: https://www.sciencedaily.com/releases/2017/09/170911095932.htm
xkcd comic around jelly beans and p-values: https://xkcd.com/882/
Birth control drug trials: https://www.washingtonpost.com/news/retropolis/wp/2017/05/09/guinea-pigs-or-pioneers-how-puerto-rican-women-were-used-to-test-the-birth-control-pill/
Music is The Beauty of Maths by Meydän.
---
Support this podcast: https://podcasters.spotify.com/pod/show/im-the-villain/support