Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- "A lot of the content of this blog was initially drafted by an agent of some sort"
What? I mean who does this. My voice is my voice and it's literally never occurred to me to have an LLM do a first draft. I thought that was college kid stuff.
by jsw97 - I can accept having a human first draft and let LLM proofreading and/or do some polish on language, but not the reversed order.by xiaoyu2006
- > Both papers were accepted for oral presentations with the condition that they simply fix the hallucinated references.
I do wonder what truthfully could be on ai verification, if even one paper with such an error is accepted it sets the precedent you hopefully get lucky to not get caught (then again verifying for basic tells isn't the same verifying is this genuinely a worthwhile publication, but that's a separate matter)
by tolugenius - It is in alpha (hoping to do Show HN in a few weeks), but for those interested I am working on an application to do this, https://veruscite-data.com/
Most of the folks on HN will be more familiar with genAI tools and can just use the skill Caleb and Isaac provided in this blog post. My tool is just likely more token efficient and has a GUI where you can review the extracted bib and edit it more easily.
by apwheele - I didn't read carefully enough at the beginning of the article, and at first I thought Caleb and Issac must be two different algorithms for detecting AI in published papers. Eventually I realized they are the names of the two authors who wrote this article (with AI assistance). Anyways, I was wondering how the two of them managed to disagree? Were the papers each human flagged as concerning the same between the two humans? I should keep reading carefully, just thought I would point it out to other humans too, maybe save them from the same mistaken thought.by skyberrys
- See Chavda's Paradox. 1. https://zencapital.substack.com/p/chavdas-paradoxby zenincognito
- “…because conferences have made it mandatory to review 4-5 papers if you submit to them.”
Is that really a thing? So if I submit a legit paper it is being reviewed forcefully by a bunch of random people from god knows where?
- Speaking for the *ACL conferences, they are not "random people" in the sense that they must have "(a) at least two papers in main ACL events or Findings, plus (b) at least one more paper in the ACL Anthology or a major ML/AI venue" [1]. So they are published authors.
As for it "being a thing": sadly yes. The number of papers has been steadily increasing over the years and the number of volunteer authors is simply not enough.
- That's correct. And 4–5 is low-balling.
I was forced to review 7 papers.
Now, how are these papers chosen for you?
You get to bid on which to review.
Bid on 30 papers out of 30,000.
I got none of those, and I had to review 7 papers I wasn't really competent enough to review.
by emil-lp - This is a side effect of academia never taking open accessibility to papers and journals seriously.
If all these papers were not gatekept by journals, it would be trivially easy to validate at least the existence of cited papers and quotes.
by dghlsakjg - At least in biomedical research most everything is either open access on day 1 or open access after 1 year.by asdff
- It is trivially easy to validate the existence of a cited paper.by nhinck2
- > This is a side effect of academia never taking open accessibility to papers and journals seriously.
From my perspective, it's a clear manifestation of humanity's most pervasive failing - the one that defines every group, eventually.
No One Anywhere Wants To Clean Their Own House.by WarOnPrivacy - Validating existence should already be trivial: virtually all journal already have publicly-available indices which contain at least author information and an abstract. Combine that with DOI citations and you're basically done - even with closed-access journals.
Checking the content is of course a lot more difficult, but that doesn't magically become trivial with open-access journals: you still need to read and interpret what is being said in the paper and compare it to the claims being made in the citation. Granted, these days you could use AI for a first pass, but it's still going to be incredibly tedious work.
by crote - I don't think it's academia not taking it seriously, more like these publishers are just coasting on brand name and trying to milk every cent out of it. Peer review is essentially being a Reddit mod or something, you get nothing out of it but karma or a sticker, and the platform/publisher gets all the benefit.
Imagine being an author and paying to publish your novel. Also btw the editor is another author but has to proofread your book for free.
Many academics will publish preprints or their more popular papers on their own websites etc. It's just common sense because to academics they don't get paid a single cent by the publisher (and instead have to pay the publishers instead) and more publicity for them is always better than less.
by podocarp - https://arxiv.org/stats/monthly_submissions
They should consider swapping this for a log plot.
I can imagine in 2027 academia looking like Moltbook.
by kingstnap - > I can imagine in 2027 academia looking like Moltbook.
This is certainly “directionally correct”, but keep in mind this is all highly uneven across fields and subfields. The people generating slop articles are mostly trying to publish big flashy things, and naturally ML research has it much worse than most other fields. There are many topics that are super important and interesting, but niche or obscure enough that no slop authors is trying to publish on them yet. So plenty of topics are still dominated by real earnest researchers doing their best, but they’re niche enough that you wouldn’t know about them unless you study that field.
- Since the log trend seemingly started before the age of LLMs papers - cant we just innocently hope that this reflects increasing popularity of prepublishing on arxivx?by voxelghost
- I feel like this should be treated as, and have consequences similar to, plagiarism.
Alas, it's probably just wishful thinking on my part.
by DarkUranium - So wrist slap?by azan_
- I thought that it was. I thought journals were starting to implement full bans upwards of a year+ for those who don't honestly disclose AI usage in their work? If I'm not mistaken, arXiv is doing this as well? granted, disclosure is different from overuse, but it seems like a small jump to just go ahead and just ban not checking ones work! ...disclosed or not disclosed. my own personal view on it, is if you can't be bothered to spot hallucinations and other such errors, its not ai that is the issue, it is incompetence and lazinessby wavewrangler
- The next logical step in "publish or perish". Get rid of p-o-p and this problem will also largely disappear.
I know managers and administrators want a simple metric, but there is no simple metric for research.
by bradley13 - That misses the fact professors have plenty of other requirements beyond publications already. Teaching requirements, service requirements, chairing committees, taking on and graduating grad students. All of these are considered when one goes up for tenure.by asdff
- But is there a not-simple metric?
The advantage of "publish or perish" is that it is based on something concrete. The system is gamed, for sure, and even more so with AI, but still, a paper which is cited a lot tend to be useful and the authors get rewarded.
Remove that metric, and for the lack of a better idea, it will just turn into a game of who has the best connections or who talks the most convincingly, I mean, even more than it is now.
by GuB-42 - "Publish or perish" in science is like the "lines of code produced" in software development.
Even with the technical debt. We have zillions of papers published, we know that most of them probably won't replicate, we don't know which ones.
by Viliam1234 - At this point, in the field of AI research:
- papers are written by AI (as pointed out in this article, and as obvious to anyone who spends a while actually reading recent AI research)
- papers are reviewed by AI (NeurIPS is doing an AI assisted review experiment - https://neurips.cc/Conferences/2026/ai-reviewing-experiment - and I feel the trend is moving towards AI reviewers whether we like it or not)
- papers are read, summarized and digested by AI, because there are just so many papers at leading AI conferences that nobody has time to eyeball them all
We are very rapidly automating humans out of the academic publication loop here.
by nneonneo - It’s going to go back to the old boys club where personal connections between research groups and institutions will matter more.by wombatpm
- We try to iterate regarding efficiency (saying that as a neutral observation). Not sure if that works out. As often the case, sciences that don't serve as a foundation for real world results (can this plane fly faster now?) are in much more danger.by bulbar
- > We are very rapidly automating humans out of the academic publication loop here.
It more looks like the breakdown of the current academic publication system, which was rotten to the core pre-AI and which internal contradictions are just accelerated by AI to the point of breakdown now.
- It’s important to notice that flagged papers were still accepted; anyone who spends any time around researchers knows the fact that most people are salivating over AI. That’s why they don’t want to punish it, they’re also using it. And they don’t want to create an environment where it is overly punished, at least not while they can take advantage of it. There is only a small amount of serious people complaining, but in my experience the opinion of the vast majority is “stop worrying and learn to love it”. Which brings to the surface a very harsh reality: almost nobody gave a damn about the science to begin with.