Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I see the author's point but I respectfully disagree. If anything, there is now much more pressure to make sure the research is error-free because so many people independently look at it. In the past if you found an error in published research, you had limited ways to share your findings. Most journals are reluctant to put their papers under scanner. Now we can just write a blog or tweet out the findings. The researchers today face much more scrutiny compared to the ones in the past. In economics, people are using AI to replicate older famous papers and reporting problems with them. Tools like refine.ink can help researchers identify the errors in their research before even submitting the paper anywhere for a peer review.by malshe
- What I am seeing is not people not being diligent in their research, but people not being diligent in their reviewing of research. But maybe there is a positive feedback loop here...by auggierose
- Sounds a lot like the incentives for making a software change / PR or whatever at BigCo...
Publish a PR that has the word "fix" in the description, get it approved, release it to prod... you are the hero, took action, will get promoted, etc.
When it turns out that the "fix" makes everything worse... well, that's everyone _else's_ problem, like whichever users run into that change blowing up, whoever is on support that day, whoever has to undo it.
The old... "concentrated benefits and diffuse costs" problem.
- I think everyone who enters academia goes through this phase of disillusionment. This thing you revered so much as this temple of knowledge turns out to involve the same kind of human business you see anywhere else, with petty fights, little cliques opposing each other, grudges, backscratching, quid pro quo etc etc.
I think once you come to terms with this, you come out on the other side realizing that this is nothing new. The idealized version never existed. What's new is the scale and all the quantitative metrics regarding publication.
> Academic research was meant to be about breaking new ground, keeping to honesty and good scientific conduct, being clear and upfront about uncertainties, errors, and mistakes, and improving our common understanding of science, all the while training the new generation to follow these goals and principles.
This is a naive idealism. Research takes effort and someone has to pay. Sometimes this someone's may profess beliefs that they only want to sponsor unrestricted objective inquiry, but everyone has biases and in an iterated game, soft preferences will be gamed too. In other words, there's is always an effect of "the one who pays orders the song".
The one who pays wants results. He wants prestige through the sponsored project. He doesn't want the researcher to just sit in a room and fail to generate results for years. He wants the researcher to have groundbreaking results. What they are matters less than that they be impressive and improve prestige of the funding source, among peers (in case of nobility funding it), or among the political classes (when government funds it), or to improve the PR and make the sponsor look good and charitable (if a company funds it as a sort of donation to academia).
Much of this "reward" that accrues to the funder does not really depend on the technical content of the work, only inasmuch as such quality was necessary to convince the research community to designate the work as impressive. However if this designation can be helped along via other paths, ie the quid pro quo and backscratching networks, that can be just as good for the funder as long as it doesn't rise to the level of fraud or something that may really damage the reputation.
Always,always follow the money and follow the chain of decisions.
Who will feel good at a cocktail party among their social peers when some impressive research result happens? Who can puff up their chest and say "We paid for that! Look how awesome we are!" and when they summarize the result in a soundbite will it impress their social peers?
This is not only applicable to the actual funding provider but also other orgs in the chain like university administration who provide facilities to the lab etc.
by bonoboTP - There is a great blog post [0] that sohl dickstein had a couple years ago, about what happens when the metric being optimized is only a proxy for the thing you want to maximize — what happens eventually is “overfitting”. The original metric (which we cannot directly optimize for) actually starts to get worse.
I believe publishing papers and citations are those proxy metrics.
[0] https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....
by smath - Hmm... The author starts with C.Munger's quote “Show me the incentive and I will show you the outcome”, then proceeds to express dissatisfaction with the current outcome of academic research. Regrettably, the author never returns back with:
a) analysis of current incentives; and
b) proposal to change those for a better outcome;
by senshan - Hasn't it always been that way? But it is sad, admittedly.
- I would add that the problem is not new, and not limited to junior researchers.
In my first faculty position about 20 years ago, I followed a line of research that a senior (MacArthur Fellowship-winning) colleague had pioneered. After three years and a few negative results, I came to suspect that several high-profile publications were p-hacked and likely false positives.
It was extremely demotivating, to say the least. I switched to a different field and fairly soon afterward left academia altogether.
A very senior scientist later confirmed my suspicions about their disregard for professional ethics.
by t0mpr1c3