36 points torutofu 56 minutes ago 24 comments
patrickmay 41 minutes ago | parent
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."
p-e-w 35 minutes ago | parent
“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”
itomato 21 minutes ago | parent
- is optimized for machine input
- is convertible to human-native symbols, context
- includes metadata and proof-of-custody/origin
- leans into its role as an evolving language
- is necessarily terse and token-efficient
ramses0 2 minutes ago | parent
`zip` => `zAIp`
It gets around some of my general complaints about "checking generated artifacts into version control", really interesting idea!
bennettnate5 2 minutes ago | parent
[1] https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber...
Eddy_Viscosity2 41 minutes ago | parent
bessbd 24 minutes ago | parent
avaer 40 minutes ago | parent
Framed another way: AIs are arguing with AIs about the merits of AIs, while the Board of Governors of the Federal Reserve System is nodding.
sathish316 29 minutes ago | parent
So far we’ve been using volume and time as constraints to justify why reviewing so much AI-generated text and code is humanly impossible.
But if we solve time and cognitive constraints for subject matter experts, Can AI-generated text and code survive peer reviews?
astro1234 29 minutes ago | parent
Peer review will no longer matter in about 2 years and it will feel silly that we were ever asking ourselves these questions.
Despite being proven time and time again that performance trends are incredibly stable and if anything accelerating slightly, that pretraining scaling laws and their analogs for post training are reliable enough for this to be mathematically uncontroversial, it is just as reliable that these clearly accurate projections are so incredible that people still don’t seem to accept it. This sentence has been true and could have been said at any point in time over the last 4-5 years (and it has, many times).
The AI 2027 scenario while hilarious when posted is alarmingly not that far off from where we are today. As Dwarkesh reports from his conversations with Anthropic researchers and those at other labs, we are on track to solve RSI sometime in 2027 (again a relatively easy thing to project if you know a little bit about performance trends and how model performance is measured).
The world will not implode like a dying star, and I expect the experience to be more or less what we’re going through today, though some capabilities will experience a much higher rate of improvement, and since there are unavoidable bottlenecks that AI cannot get around ever and some that will take years to figure out how to break down. But “oh no we don’t have humans to peer review” is incredible to me: peer review is a quality filter, and humans are notoriously bad at it. Models are already better in many ways at peer review and will become so obviously better that having a human in the loop at all will be like hailing a horse and buggy ride from the airport.
marginalia_nu 28 minutes ago | parent
Reviewers don't get paid and have zero incentive to do a good job. You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through. The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
ModernMech 13 minutes ago | parent
First, reviewers do get compensated, just not with money but with reviews from other researchers when they submit their papers. Not every review is good, but I've gotten enough helpful reviews from highly competent researchers whose time is very valuable which I didn't have to pay a cent for, that I'm incentivized to do the same for others. Also I want the system to work, so that's also an incentive.
> You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through.
This is not how to build a career in research, only high impact journals and conferences matter.
> The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
Peer review isn't supposed to detect fraud, it's not designed to do that, and the people reviewing aren't trained in fraud detection, nor should they be.
The system is supposed to detect fraud through reproducible studies, but if you hop over to this thread (https://news.ycombinator.com/item?id=50538303) you'll see people complaining all over that the results are obvious so what was the point of the research. Then you might appreciate how hard it is to make a career off reproducing results. If we think fraud in research is such a problem, we have to actually allocate funds toward reproducing results.
But to your point, peer review does stop plenty of shoddy research, just not all of it. It's not reasonable to expect peer review to stop all shoddy research, because we might be working on different and conflicting definitions of shoddy.
marginalia_nu 7 minutes ago | parent
We know replication doesn't prevent fraud either. Like in many fields most studies simply don't replicate. It's a whole thing.
It's very difficult to get institutions to work at scale when they assume every participant is acting in good faith. That may have worked 150 years ago when scientists were wealthy and doing the work for the love of the game (and also ironically not subject to peer review), but it doesn't work in the human centipede of fraud era of society.
abroszka33 28 minutes ago | parent
tenzin3371 22 minutes ago | parent
jsw97 20 minutes ago | parent
There should be strong, public, and embarrassing consequences for reviewers who do this. And the same applies to authors who use AI-generated text.
I just can't believe that people feel that they can inflict AI text on other humans as though it were their own. We can't normalize this, it's incredibly offensive. I recently received an AI communication from a doctor. Did the doctor even think about it, or did they just press enter? How much credence do I give their decision on the question I asked? Certainly much less than if they had written it themselves.
That said, the marketplace for ideas is going to look a lot different in the future, and that's probably ok. There needs to be a role for agentic or hybrid communication and discovery, and we need to find new ways to ferret out the truth in the midst of all of that. The old gatekeepers will still play some role, but it will be different. But feeding colleagues AI writing as your own needs to be a hard no.
hackermailman 14 minutes ago | parent
Same professor was rejected twice by peer review for what later became landmark papers one of which even got some Intel industry award. He did not change the paper and merely resubmitted that's how broken peer review already is.
drewlanenga 13 minutes ago | parent
Is the same true for AI-generated text that helps a human contributor contribute more than they otherwise would be able to? (like ESL, or someone who's just not great at writing) As in, the thoughts are human but the text is AI.
How are we supposed to gauge human contribution?
giladd 11 minutes ago | parent