Well, sometimes there’s another step missing just before the Bullshit: “Use the small, narrow findings to inform a greater narrative beyond the data’s scope”
I love how all the comments in this thread are like “yeah but it is bullshit tho!”
Well I’d like to think I’m not! I wanted to point to an actually dubious thing where we might call into question a study, so we could still respect the work being done while validating the importance of keeping standards in research.
You’re right though that it’s disappointing how many responses seem to address only the flaws in modern science and not acknowledge the strength of the scientific process. I think a big part of it does come down to how scientific findings are interpreted and reported to the public, and even further an all-too-human misunderstanding of epistemic limitations. Our cultures should spend more time educating people about the limits of knowledge and fact, how they are constrained by other flawed systems, etc. That would be a half-decent start, if we could only fix the entire reporting problem too.
Thank you for pointing this out.
guy on lemmy “this was already obvious, why don’t they try studying something actually useful”
This, but to some degree, unironically. If studies aren’t reproducible (or deemed worthy of reproduction) then there’s definitely a disconnect between the folks handing out research assignments and the folks engineering applicable solutions to scientific problems.
That goes two ways. You could be a guy who successfully formulates a mathematical model to support the existence of Neutrinos and face a funding board that has no interest in building a LHC. That’s arguably a problem of malinvestment within the scientific community. Or you could be a guy who successfully formulates a mathematical model for a new kind of mouse trap that’s 10% less efficient than traditional mouse traps. That’s more of a university research assignment problem. Or you could have a researcher who claims he’s the only one who can do a particular thing, because he’s got the magic touch. If the research is unfalsifiable by design, that’s an entirely new kind of problem.
i think you bring up valid instances where this is fair.
but i think i’m speaking to the very obvious and important ones that are worthy of reproduction. like i’ve seen articles be like “these corporations are responsible for 99% of climate change” or something
and the comments will be like “duh we knew that”
which true, but not empirically. being able to cite data from actual research from professionals is so valuable and far better than anecdotes or guesses.
that said, is there some way for a layperson like me to identify when research is not deemed worthy of reproduction? or is it a lost cause
which true, but not empirically. being able to cite data from actual research from professionals is so valuable and far better than anecdotes or guesses.
While its certainly helpful to get the raw numbers down on paper, you don’t need a filing cabinet full of documents to recognize that fossil fuel consuming electricity producers and airliners and manufacturing centers the but-for cause of climate change. Fossil Fuel goes in. Carbon emissions come out.
We can definitely use a more meticulous bit of R&D to find exactly where and when these emissions peak, in order to reduce total emissions without sacrificing an abundance of economic productivity. But “did you know burning the fuel makes the pollution?” isn’t a shocking conclusion.
Where things get annoying (and where in-depth research genuinely comes in handy) is in the functional policy that follows this recognition. Once you know a widget factory in China is 10x less efficient than its counterpart in the US, you can formulate a trade law to limit imports contingent on reform. But as soon as you start impacting some retailer’s bottom line, you get some screamer ad “Congressman Greenpeace Wants To Make Your Widgets 10x as Expensive to Save The Stupid Spotted Owl! In Truth it is the Spotted Owl that produces all the emissions! Kill the Spotted Owl!” financed by the worst people you know.
And that’s when you get some facebook troll group (or marketing team or bot army) spamming “Spotted Owl Farts Killed The Environment While Joe Brandon Clapped!!!” And then it becomes orthodoxy in the denialist community such that you’ve got Sunday Morning talk shows with people arguing over Spotted Owl emissions rather than trade law.
is there some way for a layperson like me to identify when research is not deemed worthy of reproduction?
Not practically, no. As soon as you’ve got that kind of info, you’re no longer a lay person.
At some level, you need a network of trust with someone who does know and does have a serious take on this. And that network is going to be informed by who you already trust and listen to. And that’s going to be informed by who they trust and listen to.
That’s the real terror of the modern mass media system. We’ve corrupted so much of our information stream that its genuinely hard to find a serious media venue that’s not been gobbled up by a for-profit marketing firm.
Hasn’t read the article methods but still decided to comment: cOrReLaTiOn dOeSn’t eQuAl cAuSaTiOn
Yeah but also just publishing correlation is a shitty practice. That’s supposed to be a hint to look deeper, not the end conclusion.
Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’.
All my literal this
this is the way doggo puppers, hecking upboats to the left
herp doesn’t imply derp
There’s a generation of internet debate guys who seem convinced that correlation disproves causation
The only bullshit I see is the 3 years for a PhD. How the hell did you pull that off;)
I find humour in the “clear peer review” part. I’ll take that with a grain of salt.
Its hard to trust self-validation
If after all that preparation, your pride can be pierced and wounded by one of myriad neckbeards or Karens on twatter, you might need let go a little bit.
Solution: don’t give a fck on the brain farts of random people.
Sometimes it is bullshit.
https://arstechnica.com/science/2024/02/scientists-aghast-at-bizarre-ai-rat-with-huge-genitals-in-peer-reviewed-article/Like callouscomic said, sometimes academics are incentivised to churn out bullshit
A better example is the Stanford prison experiment. Guy purposely put cruel bullies as the “guards” and more submissive participants as the “prisoners” to sway the study preemptively. Not to mention all the funky things people do with collected data. This isn’t to say that when somebody with no expertise in a field doesn’t understand a study that that study is bs tho, and I’ll admit this is a fine line to walk as many pseudoscientists and crackpot theorists are created this way.
Obligatory Stewart Lee clip: https://www.youtube.com/watch?v=FzOv14fA-BI
You know, the guy on the internet might have also been doing stuff all that time
Guy on internet: “this study is flawed in the following ways [proceeds to list shit they thought of in 25 seconds that may in no way matter, but since they thought of it, it totally disqualifies any and all science which may not agree with the armchair brain farts]”
They forgot:
Lab gets subsidized by company xyz
Alter perception of data in favor of xyz
Publish article praising xyz, and dissolve them from negativity.
Guy on the internet: bullshit
Dunning Kruger curve. The people who know the least about a topic speak the most confidently about it.
Don’t think it’s exactly Dunning Kruger. We all think about the curve of gathered knowledge and perceived knowledge.
But they didn’t even start to gather knowledge, they just respond with something that sounds truthful and fits their world view in order to feel better without doing anything.
But hey maybe that’s just my Dunning Kruger talking.
I see this name everywhere these days. I think… I’m having a Baader-Meinhof about Dunning-Kruger
Nice we’re keeping the Reddit tradition of just repeating “Dunning Kruger” every time we see disagreement
To be fair, journal articles and scientific research in general have gotten to be pretty bullshit. Haven’t they studied this and proven the vast majority of published journal papers probably shouldn’t have been?
A couple easily Google examples of discussion regarding scientific publications likely being bullshit.
Why Most Published Research Findings Are False
Surge in number of ‘extremely productive’ authors concerns scientists
Too much academic research is being published
More than 10,000 research papers were retracted in 2023 — a new record
Whistleblowers flagged 300 scientific papers for retraction. Many journals ghosted them
Top 10 most highly cited retracted papers
And on and on. Publish or perish and general shitty culture in academia is why I quit phd and took my masters and left.
I’m guessing not all hypotheses receive the same interest or funding to begin with. Definitely seems to be a selection bias on what actually gets funded/studied. Even worse, when they withhold results they don’t like from being published.
I saw a clip on how kids out of uni don’t believe anything not peer reviewed; even intuitive observations in nature that otherwise undocumented or site specific observations that went against the grain.
Science is a way of thinking and observing, rather than papers, but papers are a good way to refine your thinking
In theory, a paper gives you a methodology that you can use to reproduce the findings. And a refusal to use papers to repeat findings (because shit costs money and nobody wants to publish iterative studies) means you end up with a bunch of novel findings that are never confirmed through repetition.
But the fact that nobody is bothering to repeat these studies also raises a question of what exactly is being researched. Certainly, the more useful scientific research efforts are about formulating applicable techniques. So they would need to be reproducible to have any functional value.
The fact that we’re not seeking to replicate studies suggests that we’re investing a ton of time in niche under-utilized fields. And that may be a problem of investigative research (we’re so focused on publishing that we don’t care what we’re actually studying) or a problem of applied sciences (we’re so focused on scaling up older methods to industrial scale that we’re leaving better methods of production on the cutting room floor).
But its definitely some kind of problem.
TBF I’ve lost count of the number of times someone has cited some paper as a reference for the point they are trying to make and when I inspect the paper it has shitty “n”, the paper is written for an agenda (not sure what that’s called where I.e. a paper saying smoking is good for you/not harmful is paid for by the tobacco industry and written by tobacco industry scientists), or it might even just be straight up bullshit written to look like a legit paper.
Peer Review at least offers some barriers to the problems with papers, but it’s definitely not a panacea.
The problem is without peer reviewed papers it’s hard to credit that someone all the way around the world observed something.
In a perfect world nobody is lying and everyone has the scientific base education to understand how to report phenomena properly. But uhhh… Yeah.
