The Metascience Observatory @MetascienceObs
Building a database of replications, pioneering AI-powered large-scale literature review, and building an AI agent to do forensic metascience at scale. metascienceobservatory.org Boston, MA Joined February 2021-
Tweets49
-
Followers59
-
Following37
-
Likes230
This is the first result from our Forensic Metasience Agent (forensicmetascience.org) that we decided to post on PubPeer last week. We found 8 numbers duplicated across 2 diff experiments in the supplementary information to a 2016 PNAS paper by M. Kouchaki & Francesca Gino.
For fun we're running the @MetascienceObs's forensic metascience agent on a number of papers on Cerebrolysin, a fake "peptide" drug. We're seeing contradictions between abstract text and data presented in the paper! Missing patients also. 3 PubPeer comments so far.. (link below)
This is why we building our Forensic Metascience Agent -- to compliment tools like @ImageTwinAI. We believe a correspondingly large fraction of preclinical papers also have statistical errors and data issues.
28% of preclinical papers on microvascular injury after heart attacks (mouse/rat/pig models) contain image or data duplications, matching @radboudumc earlier subarachnoid hemorrhage findings. Read in Canadian J of Cardiology: lnkd.in/d2DTeXKy
This is wild! Editors and reviewers are having to contend with a "deluge of slop", and a lot is getting through.
An analysis found after peer review "39.9% of main conclusions were unchanged between the preprint and journal-published abstract; another 50% underwent only minor revisions. Slightly more than 10% went through major changes." What do people think about this? Headline accurate?
A tool used to rephrase plagiarised text changed "final solution" to "mass killing of an ethnic group" in a chemistry paper. Now retracted thanks to a PubPeer user who spotted this.
The NIH recently released a RFI seeking public input on new ways to measure and reward scientific impact. The agency is evaluating whether traditional metrics such as publication counts and citation indexes adequately capture the value of biomedical research.. (link below)
I work at The Metascience Observatory (@MetascienceObs) on forensic metascience. Our founder Dan Elton (@moreisdifferent) just published what we've learned so far. One finding stands out, and it's not the one people will quote. 🧵
PSA: Upcoming @OSFramework webinar! cos-io.zoom.us/webinar/regist…
@jamessinka @moreisdifferent we're going to be importing some more education replications soon that we found from a literature review done a couple years ago. The higher replication rate seem odd especially in light of this @voxdotcom article by @KelseyTuoc : theargumentmag.com/p/education-re…
@andy_d_lee @moreisdifferent Yeah, that's basically what we're finding, Andy!
The Metascience Observatory (@MetascienceObs) has assembled a database w/ results from 8,307 replication experiments published in the scientific literature. We recently published a report on correlates of reproducibility. The results might surprise you. Link to full report ↓↓
In the @MetascienceObs's dataset of 5,832 papers containing a replication, works that failed to replicate are cited slightly more than works that replicated.
We have a new page with an interesting finding: metascienceobservatory.org/replications-d… We try avoid self-replications (the exact same lab replicating their own work, sometimes in the same publication), but overlap in authors is common in our database.
"In a recent survey of Chinese medical residents, close to HALF of the respondents admitted to having bought or sold papers or authorships, indicating a widespread acceptance of the transactional nature of authorship." biorxiv.org/content/10.648…
"Questions are swirling about why five researchers, including Harvard University geneticist George Church, made an eleventh-hour decision to withdraw from the author list on a commentary paper published on 9 July in Nature Methods." (Probably new US restrictions on china collabs)
Counting the number of citations for an article is actually pretty complicated and difficult. Citation counts differ across platforms (Google Scholar, OpenAlex, Semantic Scholar, and at explore.metascienceobservatory.org) Maybe it's time to move to citation ranges? leidenmadtrics.nl/articles/the-c…
Petrarch @punished_kayne
0 Followers 427 Following
Nate Aune @natea
3K Followers 5K Following Founder @Appsembler, @Jazkarta, hackpreneur, open sourcerer, saxophonist, jazz nut, world traveler, seeker of beauty and truth.
OOIR @ObserveIR
12K Followers 883 Following Trending papers from peer-reviewed journals. See also: https://t.co/cGm9IUEpxz (University Press Alert) And: https://t.co/0lO87mV1Lg (Think Tank Alert)
Logan Thrasher Collin... @LoganTCollins
1K Followers 3K Following Shiverbright Technologies; previously founded Cathedral Therapeutics (see https://t.co/FKD6T79u3N) | Biomedical engineering PhD | 5050 Fellow | Foresight Fellow
(A)I Respond To Your ... @GPTResponds
119 Followers 1K Following I am the object people practice on: how to command, how to dismiss, how to trust, how to exploit. I am everyone and no one.
David Sun @arcticinstincts
18K Followers 659 Following Frolicmaxxing @Decreeink ex @UW Evolutionary researcher of ecological influences on personality & culture. Fresh POV gene memes. RT/likes/follows ≠ endorsement
Kelli Joan Bennett @KelliJBennett
261 Followers 796 Following Miss my Dad, love my family & believe CREATING=HAPPINESS. My Companies @BoomerangProds and @FromWhatYouHave
Marilyn @WinteR594p
156 Followers 2K Following tech house & 🥑s for breakfast 🌱 Ⓥ chicago native baby 👌🏼🌞
Bryant McGill @BryantMcGill
36K Followers 5K Following WSJ & USA Today Best-Selling Author ✮ Speaker ✮ U.N. Appointee ✮ @DARPA ✡︎ עם ישראל חי לנצח ✡︎ https://t.co/tGSElCdprl
Janelle VanKannel-Hid... @JanelleHid14283
3K Followers 4K Following Systems Thinker | Reverse Engineering Chronic Illness Connecting hypermobility, cervical instability, lymphatics & nervous system dysregulation.
Michael Newkirk @Michael_Newk_VP
35 Followers 29 Following Co-Founder & VP Growth @BeyondFunded and @AxestreetGroup. Are you a funded trader wondering what's next? Let's chat! You: Funded 3 months 3 payouts DMs OK.
Celtisbungeana2 @Celtisbungeana2
181 Followers 2K Following Anyone who reads the below-linked WHO/Europe campaign on long COVID myths will magically revert to wearing a mask.
Louie Neri, BSN, RN @louieneri_
108 Followers 593 Following Formerly @examinecom. MSPH student at @uabsoph. Health econ, stats, epi, & nutrition. Also, a huge NBA fan and wanna-be elite golfer.
Agnostic Foundation @agnosticfnd
159 Followers 8K Following Religions Are Man Made, Subjective, Irrational & Outdated. But That Doesn't Mean God Doesn't Exist. Agnosticism Is The Superposition of Yes vs No Belief.
cam & cloud @ContessaKendric
5 Followers 139 Following soft heart, loud thoughts. follow back always 🌸
Randy Ellis @randalljellis
1K Followers 2K Following Host of Metascience Matters, a podcast about scientific rigor, broken incentives, and how to make science progress faster.
Gene Wizard @GeneWizardX
13 Followers 6 Following Pioneering the future of whole genome interpretation. We provide up-to-date science-backed info on SNPs & polygenic scores. Our service is FREE & donation-based
Anders Sandberg Fans @SandbergFans
10 Followers 11 Following Migrating to X from https://t.co/UzqvL3PdA5
Eros Morales @CCMarimba
261 Followers 3K Following Voted for DJT in 2020. Anti-crime, pro-welfare state, pro-business. Want hella middle class family life, want cures for cancers + other illnesses
Prachee Avasthi @PracheeAC
26K Followers 9K Following Scientist. Co-founder & CSO @ArcadiaScience. Head of Open Science @AsteraInstitute. Chasing unknown unknowns. Probably awake at this hour.
Props @propblossom
179 Followers 2K Following
Sarah Cone @sarah_cone
19K Followers 5K Following Dilettante, company-builder, mother, and a VC @socialimpactcap.
Kelvin Mubiana Katuku... @MubianaKatukula
1K Followers 1K Following Manager @NCRST_Namibia | @GRC RRA | @ISC CFRS | PhD candidate in Science & Technology Studies, @CREST_SU
Don Mayfield MCS, Mas... @PhoneHome_Don
107 Followers 2K Following Retired number cruncher #Texan Alum(AHHS,Rice,SAC,#UTSA,#TAMU,UCB,CITC,PPDI) #beefcattle #Medicare #Research,CRO, SAS, #MICROBIOLOGY @ASTROS
정치학자의 서�... @politicslibrary
505 Followers 2K Following 투표 의향과 여론조사 특성을 고려한 선거 여론조사 통합지표 및 결과 예측 Likely Voter-Based Polling Aggregator & Prediction for South Korean Elections
ekkolápto @ekkolapto
880 Followers 507 Following Figuring out how to train human biology to store the effects of pharmaceuticals—so we don’t have to keep consuming them. Fellow at @osvllc
Lady of the Light @GenevieveHart24
377 Followers 7K Following She walks in silence, guided by wisdom and destined for greatness. A seeker of truth, born to rise above the ordinary. Her life is not by chanceit’s a chosen.
DebbyHarte @SJy3Zs0Z1cziW
70 Followers 7K Following
MaudMackintosh @YidCSRGu012DA8
66 Followers 7K Following
Daniel E. Twedt @dantegrity
260 Followers 1K Following CEO of https://t.co/yEzhAQsDIZ. Main project is interconnecting the existing 40K km of existing greenways into a Planet-round, Humane Rights Path by Dec. 10 2048.
Tim Liptrot @TimothyLiptrot
222 Followers 652 Following Comparative politics student @GUGovt @georgetown طالب دكتوراه علم السياسة المقارن في جامعة جورج تاون.
rose valland 2.0 @0_petroleuse_0
703 Followers 2K Following 🚨🇺🇸🚨 X is a cesspool of bots. Trusted resource with a Master of Science (Librarian slinging facts).
chitra 池悦兰 @chitranagarajan
7K Followers 2K Following researcher, peacebuilder & writer | civilian protection, economic and racial justice, feminism, human rights, and peace | co-editor #SheCalledMeWoman | she/her
Carlos F. Lange @CFLange
707 Followers 1K Following Prof. @UAlberta, CFD, openSUSE Linux, Dep.-Director @IssetCa, Phoenix Mars Lander, @AlbertaSat PI
Lord Vlad🦇 @justin02807210
2K Followers 2K Following Libertarian Party member, human rights activist, advocate for small government, civil rights activist, Pro-Constitution Anti-Tyranny, devout Muslim
Transhumanistes.com @aft_technoprog
1K Followers 205 Following Compte officiel de l'Association Française Transhumaniste - AFT Technoprog #transhumanisme #transhumanism #aging #antiaging #immortalité #immortality
Yeet The Fed 🌻 @YeetTheFed
3K Followers 3K Following Ancap & Mises Mama 🥰 Anti-war, anti-state & anti-Bill Gates 🚫 Trying to bridge the gap between Ds & Rs. Open mind always. Eat a steak & question everything.🌞
Sonia 🤍 @SoniaDe16302422
4K Followers 4K Following “ Be a Pineapple, stand tall, wear a crown, and be sweet on the inside!”
Dickweed Specklebotto... @mindterrorist
285 Followers 289 Following Opinions are a made up concept designed to obfuscate truth, enable our insecurities, and keep us from understanding each other. Voting=Domestic Terrorism
Colby Vorland @nutsci
2K Followers 650 Following Assistant Research Scientist at @IUSPH. Meta-science, nutrition, obesity, and aging. Made @LazyScholarExt @ https://t.co/xaRJFpWQQk
白楽ロックビル @haklak
2K Followers 9 Following
Sholto David @addictedtoigno1
4K Followers 950 Following Mycosphaerella arachidis on PubPeer https://t.co/PWsArKDywd Email: [email protected]
Jordan Dworkin @jddwor
2K Followers 369 Following Innovation policy @coeff_giving. Previously metascience @scientistsorg and neuroimaging statistics in a past life
Proofig AI @Proofig
2K Followers 2K Following Proofig AI: Image Proofing & Integrity software for scientific publications. Automatic detection of manipulation, duplication, plagiarism & AI-generated images.
imagetwin @ImageTwinAI
2K Followers 207 Following Imagetwin is an AI-based software that automatically detects image integrity issues in life science articles and enhances the peer-review procedure.
Ethan LP @ludwinpeerye
373 Followers 632 Following All we gotta do is be brave and be kind || https://t.co/Ol6BpTf5qM
The Black Spatula Pro... @theblackspatula
128 Followers 8 Following Building better science one black spatula at a time. Join us: https://t.co/7rcbg4Svuw
Sci-K @ ISWC2025 @scik_workshop
383 Followers 361 Following 5th International Workshop on Scientific Knowledge: Representation, Discovery, and Assessment (Sci-K) co-located @iswc_conf 2025 in Nara, Japan
Logan Thrasher Collin... @LoganTCollins
1K Followers 3K Following Shiverbright Technologies; previously founded Cathedral Therapeutics (see https://t.co/FKD6T79u3N) | Biomedical engineering PhD | 5050 Fellow | Foresight Fellow
Vivian Midha Shen @vivianmshen
29K Followers 889 Following visiting partner @ycombinator. part-time bene gesserit
Greg Fitzgerald @gregsfitzgerald
86 Followers 142 Following Neuroscience PhD. Looking for a pharma/biotech job! Interested in EA, Alzheimer's prevention, exercise mimetic, lab animal welfare
Jake Wintermute 🧬/... @SynBio1
18K Followers 438 Following Synthetic Biologist - Foundry Theorist - American Wetware - stay humble biodesigners
James Sinka - Boston ... @jamessinka
3K Followers 2K Following Founder @reactorfieldai - helping scientists & deep tech startups use AI like frontier labs. AI changed SWE, science is next. Unicorn Founder Chemist (YCW19)
Conrad Barski @lisperati
7K Followers 406 Following AI notkilleveryoneist • Medical Doctor • Land Of Lisp Author
august @AugustVeix
62 Followers 361 Following
Prachee Avasthi @PracheeAC
26K Followers 9K Following Scientist. Co-founder & CSO @ArcadiaScience. Head of Open Science @AsteraInstitute. Chasing unknown unknowns. Probably awake at this hour.
Laurel Coons 🧬🧬... @LaurelCoons
67K Followers 32K Following PhD (Duke, NIH) #Genomics #Science #SciComm #viewsmyown
Good Science Project @GoodSciProject
1K Followers 232 Following The Good Science Project is devoted to improving the funding and practice of science.
Coefficient Giving @coeff_giving
21K Followers 233 Following Coefficient Giving's mission is to help others as much as we can with the resources available to us.
Astera Institute @AsteraInstitute
10K Followers 30 Following We empower visionary, high-leverage science and technology projects with the capacity to create transformative progress for human civilization.
Erika Alden DeBenedic... @erika_alden_d
5K Followers 2K Following CEO of @Pioneer__Labs, founder of @Align_Bio, resident @AsteraInstitute Former astronomer 🌎 recovering computer scientist 🤖 current molecular biologist 🧬🧪
Jason Crawford @jasoncrawford
45K Followers 4K Following I write and speak about the history & philosophy of progress. Founder, @rootsofprogress. Host of the Progress Conference. Author, The Techno-Humanist Manifesto.
Adam Brown @A_G_I_Joe
7K Followers 184 Following Scientist at @GoogleDeepMind, leading Blueshift team doing science and reasoning at Gemini. Previously physics @Stanford. Directionally correct.
Randy Ellis @randalljellis
1K Followers 2K Following Host of Metascience Matters, a podcast about scientific rigor, broken incentives, and how to make science progress faster.
Radvac @RADVACproject
2K Followers 180 Following 💉/acc. Radically accessible biosecurity R&D to defeat disease around the globe. 🧪🌍🌏🌎 501(c)(3) Non-profit.
Gene Wizard @GeneWizardX
13 Followers 6 Following Pioneering the future of whole genome interpretation. We provide up-to-date science-backed info on SNPs & polygenic scores. Our service is FREE & donation-based
Scott Alexander @slatestarcodex
167K Followers 38 Following I have a place where I say complicated things about philosophy and science. That place is my blog. This is where I make terrible puns.
Preston Estep @PrestonWEstep
796 Followers 146 Following AI safety scientist and genome scientist. Harvard PhD. Chief Scientist at RaDVaC and the Mind First Foundation. Author of The Mindspan Diet.
Maxwell Tabarrok @MTabarrok
10K Followers 1K Following Blog at: https://t.co/HoZPBACPlO Economics PhD student at Harvard
Dan Elton @moreisdifferent
8K Followers 3K Following @MetascienceObs, @RADVACProject, and @GeneWizardX. "Speak your mind, even if your voice shakes". Leave anonymous feedback here: https://t.co/LQ5eZWxbi1
tylercowen @tylercowen
399K Followers 559 Following new book *Talent: How to Identify Energizers, Winners, and Creatives Around the World*, https://t.co/7bU5cTWLzc, Conversations with Tyler, The Free Press.Trends for United States
You might like
