358 points pretext 16 hours ago 332 comments
crooked-v 15 hours ago | parent
layman51 15 hours ago | parent
AlexErrant 15 hours ago | parent
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
svachalek 14 hours ago | parent
steveklabnik 10 hours ago | parent
AlexErrant 8 hours ago | parent
https://github.com/anthropics/claude-code/issues/92769
The <command-args> I know because I just audited my raw Claude transcripts via ccglass. You can verify by just asking Haiku "hey, /grill-me does this skill autoload for you?" and then "/grill-me hey, how many times does this text appear in your context?"
You can trivially doublecheck me. The above prompts won't even eat 1% of a 5 hr on 20$ plan.
devin 15 hours ago | parent
lavezzi 9 hours ago | parent
their claude code changelog 'fixed' % over time.
https://i.imgur.com/KJcem6c.png
I believe their peak month was >13k issues opened, but worth bearing in mind: "This issue has been automatically locked since it was closed and has not had any activity for 7 days.".
Rapzid 4 hours ago | parent
Human contributions to issues, comments or submissions, were drowning in a sea of AI vomit. Tons of the issues had those AI issue submissions with the classic confidence of having identified the problem, but that smell I know a lot of us recognize of it not actually knowing what the problem is at all because it hadn't failed to fix it 5 times yet.
jstummbillig 15 hours ago | parent
t-writescode 15 hours ago | parent
oofbey 15 hours ago | parent
jstummbillig 15 hours ago | parent
sandeepkd 15 hours ago | parent
ActionHank 15 hours ago | parent
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
shadow28 12 hours ago | parent
mixtureoftakes 15 hours ago | parent
aacid 15 hours ago | parent
sp4cec0wb0y 15 hours ago | parent
_verandaguy 15 hours ago | parent
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
ActionHank 15 hours ago | parent
Have you seen the benefit of the churn of 1000 PRs a day?
leoqa 15 hours ago | parent
IneffablePigeon 13 hours ago | parent
Ah, so you've privatised the benefits and externalised the costs.
JackSlateur 15 hours ago | parent
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
for i in {1..10000000}
do
echo "I am working !" >> README.md
git add README.md
git commit -s -m "feat: working with AI"
done
(automerged PR, JOB IS DONE)devin 15 hours ago | parent
grey-area 14 hours ago | parent
JamesSwift 14 hours ago | parent
dfunckt 5 hours ago | parent
Rapzid 3 hours ago | parent
vavkamil 15 hours ago | parent
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
hyperpape 15 hours ago | parent
ChrisArchitect 15 hours ago | parent
dylanhouli 15 hours ago | parent
g-b-r 15 hours ago | parent
spiderice 14 hours ago | parent
tripleee 14 hours ago | parent
They've built an entire strategy around moving slow and doing it better
g-b-r 14 hours ago | parent
xyzsparetimexyz 13 hours ago | parent
benoau 15 hours ago | parent
dyauspitr 15 hours ago | parent
g-b-r 15 hours ago | parent
And if people are not looking it, those people don't care about their customers.
andrekandre 9 hours ago | parent
> those people don't care about their customers
if it was any other industry like medicine, food, or construction etc, could they get away with just shipping products where they have no idea whats in it?where things like "it tastes good to me" or "cement didn't crumble when i hit it with a hammer" is good enough to ship we'd be inundated with lawsuits and customer claims... its a total joke where the software industry is headed
g-b-r 5 hours ago | parent
tripleee 15 hours ago | parent
Code quality matters to AI just as much as it did to people
bbctr1 15 hours ago | parent
tripleee 15 hours ago | parent
SmirkingRevenge 14 hours ago | parent
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
tripleee 14 hours ago | parent
piker 14 hours ago | parent
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
Rudybega 14 hours ago | parent
bpeng2000 14 hours ago | parent
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
hn_acct-1029393 12 hours ago | parent
giveaccountpls 10 hours ago | parent
Funnily enough, the models learned from GGC and LLVM, yet, cannot produce something half-decent.
nerdo 15 hours ago | parent
copperx 15 hours ago | parent
gwynforthewyn 15 hours ago | parent
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
g-b-r 15 hours ago | parent
so long as his software isn't used for anything that can affect you
Quothling 5 hours ago | parent
I do think the capabilities of AI depends a lot on how you work. If you're doing very low abstraction, fail fast defensive programming in something like Rust then it's fairly good at writing code with high quality. If you're working in some OOP language with tons of freedom, then my experience is that it's usually not better (or worse) than most human programmers, but certainly much worse than some.
I've got decades of experience in enterprise development and I've also been an external examiner for CS students. We have people with no coding back ground vibe coding internal tools in Python which are terrible, but not really worse than what we've previously bought from some of the cheaper coding houses around here. At the same time we have AI writing internal SDK's and libraries in Rust which will frankly write better code than I do on a thursday afternoon after too many meetings and too little sleep because the children kept me awake all week.
Good code can mean a lot of things though. If you're deep into clean SOLID DRY OOP programming I suspect that an AI will produce unreadable horrible code, much worse than most humans. On the flip-side the AI probably knows the difference between a generator and looping over a List. It probably also knows more about JVM optimization than most of those banking programmers trying to replace COBOL with JAVA.'
I doubt Antropic is a good sample though. Bun is written in Rust and it's a single purpose software, which seem to be an optimal environment for LLM's.
drewg123 14 hours ago | parent
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
gwynforthewyn 14 hours ago | parent
This is, of course, tangential to your point.
troupo 14 hours ago | parent
weakfish 14 hours ago | parent
I do, my coworkers do, my friends do, etc etc.
swat535 13 hours ago | parent
Your end users certainly don't either, heck they'll gladly sit there and scroll through their phone until the 50GB React app loads.
xyzsparetimexyz 13 hours ago | parent
stillpointlab 15 hours ago | parent
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
troupo 14 hours ago | parent
BobbyJo 14 hours ago | parent
troupo 13 hours ago | parent
BobbyJo 13 hours ago | parent
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
troupo 4 hours ago | parent
Ah yes. "all the magic the people say work 100% of the time"
> Do you think it's better to experiment with high stakes projects?
This again assumes that people working on high stakes projects only work on one single project.
This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
This again assumes that people experiencing something different from the others are inevitably stupid.
etc. etc.
stillpointlab 14 hours ago | parent
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
g-b-r 14 hours ago | parent
It's those who don't do it who'd need a talk..!
9dev 13 hours ago | parent
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
stillpointlab 13 hours ago | parent
However, I suspect there are teams that are making different choices. In the most obvious example, I can do whatever I want on my personal projects (I hope you agree). And if I started a business and hire in programmers, I have some expectation on defining the culture, making hiring decisions based on my preferences, etc.
Whether or not such teams succeed will be a matter of fact, which I do not believe has been decided at this time.
troupo 13 hours ago | parent
How does "low-stakes environment" fix that?
jazzyjackson 13 hours ago | parent
Last time I chose to vibe code, I did it in a language I didn’t know so that I wasn’t tempted to try and comprehend the choices the robot made. I ended up with the application I wanted.
Caveat: it was code I would never have to touch again, I ran a script to generate the 140,000 static html pages that the rails app provided and then just uploaded them all to R2 buckets (million reads per month on free tier)
troupo 4 hours ago | parent
No, they are saying "who cares if all those issues with code are true, just don't review it"
> Caveat:
It's not really a caveat. I also have throw-away code. But we're not talking about throw--away code. We're talking about the systemic issues in LLM-produced code, which are completely ignored and the answer is always "who cares, close your eyes and don't review the code".
stillpointlab 13 hours ago | parent
The person is saying "I am still unable to stop" and that is most likely because they feel a sense of responsibility. If they choose a project where there is no responsibility, where it literally doesn't matter if the project fails, they may feel able to stop. And if they manage to make that project work, if they can succeed, then they will build up confidence.
It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code. In fact, keeping their job in the long term may (or may not) require having both skills.
daveguy 12 hours ago | parent
troupo 4 hours ago | parent
They are not overblown. LLMs do all the things the person described.
> Or they may learn that they cannot build any kind of software without reviewing every line of code.
We already had bad horrendous software before the onslaught of LLM-slop. Why contribute to that?
> And if they manage to make that project work, if they can succeed, then they will build up confidence.
Confidence in what? It doesn't fix the underlying issue. And the underlying issue isn't "I can't stop" but "redundant code, duplicate code, overly complex architecture, etc." that LLMs regularly produce.
> It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code.
"If I close my eyes maybe this goes away"
dang 14 hours ago | parent
(You're welcome to make your substantive points thoughtfully, of course.)
xyzsparetimexyz 13 hours ago | parent
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
stillpointlab 12 hours ago | parent
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
xyzsparetimexyz 12 hours ago | parent
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
chrisjj 2 hours ago | parent
First hit is free ;)
mattm 12 hours ago | parent
stillpointlab 12 hours ago | parent
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
hn_acct-1029393 12 hours ago | parent
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
stillpointlab 12 hours ago | parent
If you cannot tolerate failure then don't experiment. Don't mix the two.
72deluxe 12 hours ago | parent
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
stillpointlab 10 hours ago | parent
I mean, I've had useful software that I don't understand how it works where I am stuck hoping some small company continues to maintain it. And I've had some of those companies go out of business and leave me with useless software that I eventually have to replace once it no longer works on my machine.
Having the code that I can dump into an LLM to understand myself or have it alter it seems better than that.
arcanemachiner 9 hours ago | parent
chrisjj 2 hours ago | parent
I tried that. ChatGPT's result was laden with loud, clear and wrong fabications which I might have believed had I not examined the code.
Unreliable program be unreliable.
xboxnolifes 11 hours ago | parent
zeroonetwothree 8 hours ago | parent
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
chrisjj 2 hours ago | parent
I see the same in e.g. simple CSS. I ask ChatGPT to apply styling X to aĺl instances of say a UI control type. Then to another. Later I ask for a tweak of X. Bot updates some but not all instances. It didn't create a single class for all. No intelligence seen.
The most remarkable fails are simpler though. E.g. at version 303 I ask "Revert to version 299". I get 299 with 303's edits. Bot explains it interpreted "Revert" to mean something no intelligent programmer would.
And most importantly, these are just the ones I directly notice. To see the total, I have to read through and understand the most repulsive code I've seen in my life.
The only way to be satisfied with bot code is by having exceptionally low expectations. And never ever let yourself be kidded that your helper is more than a stochastic parrot gorged on stackoverflow etc.
fantasizr 9 hours ago | parent
thisisauserid 15 hours ago | parent
gonzalohm 15 hours ago | parent
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
enraged_camel 14 hours ago | parent
gonzalohm 13 hours ago | parent
enraged_camel 11 hours ago | parent
andrekandre 9 hours ago | parent
JackSlateur 15 hours ago | parent
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
stymaar 15 hours ago | parent
luisgvv 15 hours ago | parent
rileymichael 14 hours ago | parent
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
warkdarrior 14 hours ago | parent
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
rileymichael 14 hours ago | parent
walthamstow 14 hours ago | parent
a3nz3g 11 hours ago | parent
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
How about your pattern?
NorthSouthNorth 13 hours ago | parent
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
tills13 10 hours ago | parent
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
PJDK 12 hours ago | parent
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
koolba 14 hours ago | parent
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
brational 9 hours ago | parent
RGS1811 14 hours ago | parent
jazzyjackson 13 hours ago | parent
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
mahogany 9 hours ago | parent
Although, following Anthropic’s ethos and taking this to its conclusion, I’m not even sure why they need human reviewers at all. What, they bring in a human to look at a diff of an “important” section of code that was written by agents and has changed a dozen times since last week through 12 PRs of churn? Obviously they don’t know what’s going on. So what’s a human to do? Do what the other approvers (if any) did: fire up a Claude to command it “verify this change has no regression; check carefully for edge cases”, then come back after a half hour of High Thinking to see “Change is clean (with some non-blocking findings below, in order of significance)” and click Approve. Not sure what the purpose of the human is in such a process.
ativzzz 14 hours ago | parent
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
enraged_camel 14 hours ago | parent
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
lavezzi 9 hours ago | parent
UberMouse 7 hours ago | parent
UberMouse 7 hours ago | parent
enraged_camel 7 hours ago | parent
zeroonetwothree 8 hours ago | parent
jwpapi 14 hours ago | parent
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
YmFzZTY0ZGFkZHk 14 hours ago | parent
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
markbao 14 hours ago | parent
Compare that to the cost of human code review which can take up a third of an engineer’s time. You’re looking at $50-100K a year per engineer fully loaded in code review costs.
If AI assists in code review (not even replaces – we still do human review ourselves), you could spend thousands a month per engineer and still come out ahead.
Ironic that this comment talks about cost without weighing the numbers.
evolve-maz 5 hours ago | parent
I'm glad they're doing this, because it lets me see an extreme of LLM development.
grey-area 14 hours ago | parent
stillpointlab 14 hours ago | parent
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
grey-area 14 hours ago | parent
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
stillpointlab 13 hours ago | parent
First, I think we overestimate how different code is from other code. We even joke about CRUD apps and how cookie-cutter they are. A lot of changes in those apps are: add some new features to a data access layer (SQL, ORM, whatever), add new routes, add new UI surface, etc. There is an entire industry around "Design Patterns" and people live by the DRY axioms. Things like Ruby on Rails exists because rebuilding these systems over and over is wasteful. I feel we are just moving further down that same path.
Second, your question of "why would we want" is totally backwards. Progress doesn't follow your desires. Progress doesn't require justification. Sometimes we adapt the environment to our needs, other times we adapt to a changing environment. Which case we are currently in is still undecided and you are free to fight for the future you prefer. But demanding an industry justify itself before it is allowed to change fails in the face of evolution. If it works, we will justify it in hindsight.
g-b-r 12 hours ago | parent
stillpointlab.com , "Get more from your LLM"
jbvlkt 12 hours ago | parent
fipar 14 hours ago | parent
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
jeremyjh 13 hours ago | parent
0c3ca83 13 hours ago | parent
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
cadamsdotcom 10 hours ago | parent
That's new. All analogies to 2022 software engineering need to be thrown out.
Software construction is now more like an industrial process, with the artisanal work shifting towards both
1. aligning what's built with what's needed (talking to customers!)
as well as
2. implementing and tuning your software construction process (ie. hi - we're Anthropic. We're burying ourselves in a mountain of code but it's okay because the fix is a second mountain of code, try some tokens the first hit's free!)
Back to my original point...
Writing code is now on the production line; making quality output demands industrial grade/scale verification (some involving humans of course).. which is a fascinating engineering challenge, and it's brand new!
0c3ca83 10 hours ago | parent
Verification means nothing for industrial processes when the desired outcome is a moving target, so the first part of an industrial process is to develop a fixed target and aggressively remove variation.
Moving the target around is a research problem, before you feed the result into an industrial process.
Software solved industrialization so well we even forgot that we go from research to industrial with `npm` deployments or whatever your package manager of choice is.
cadamsdotcom 8 hours ago | parent
It absolutely can be industrialized: the agent writes tests that conform to your requirements; the harness makes sure those tests pass before work can stop.
Have a look at https://codeleash.dev/docs/tdd-guard for my (dated but still interesting) take on this. Claude Code hooks are really useful to do the enforcement.
It's open source so no need to read my crappy generated docs - you can point your agent at the repo and ask it questions.
grey-area 5 hours ago | parent
The process of anthropic, turning gold into shit software, is hardly something to emulate. What is the evidence software should now be produced in an industrial way? Because arthropic (the ones selling you the shovels) says so?
cush 11 hours ago | parent
mirekrusin 13 hours ago | parent
swaptr 13 hours ago | parent
foolfoolz 14 hours ago | parent
good model
corentin88 14 hours ago | parent
surgical_fire 14 hours ago | parent
Anyway, flagging as misinformation.
visarga 14 hours ago | parent
dominotw 14 hours ago | parent
Its like telling other ppl about your dreams. Not interesting.
acedTrex 13 hours ago | parent
kaashif 12 hours ago | parent
Eventually I did, but it was hours and hours of effectively random stuff happening that I had to write down or dictate with speech to text.
It was a waste of time and I'm now glad I forget almost all of my dreams. My point is even my own dreams are uninteresting.
I'm not that interested in my own agent setup either - usually the tools become better on their own after a few months and render it all obsolete.
nzjrs 8 hours ago | parent
zeroonetwothree 8 hours ago | parent
insin 4 hours ago | parent
kettlez 14 hours ago | parent
status_quo69 14 hours ago | parent
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
Terr_ 13 hours ago | parent
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
markbao 13 hours ago | parent
From the article, the 30,000 changes a month are in the monorepo, and Claude Code is in a separate repo. So it’s not the TUI that has 1,000 PRs a day.
I could easily see 1,000 PRs a day on the backend. Big tech was doing more than that pre-AI.
status_quo69 13 hours ago | parent
zeroonetwothree 8 hours ago | parent
frereubu 13 hours ago | parent
jonahx 13 hours ago | parent
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
peab 13 hours ago | parent
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
jonahx 12 hours ago | parent
It is true that this gamble has existed in the form you mention for many years. Still, I sense something new here, both in scale and in the attitude of trust. Also, I find those massive, incomprehensible human-built systems frightening too.
dasil003 12 hours ago | parent
So far I kind of feel like it nets out to the same bottleneck being good judgment from decision makers. And you might say, no, leadership will just put more and more pressure to add more complexity just because we can and it makes it appear that we are moving fast. But when it comes to real-world apps, there is a backstop which is that things that are too complex and illegible end up not being fit-for-purpose for human problems.
Claude Code and other general purpose AI tools are not like this, because they are literally trying to optimize it for every single kind of knowledge work with zero constraints on use case or target audience other than chasing the next juiciest one they can with essentially unlimited resource. 99% of software is not like this, and when the dust settles I suspect software won't be anywhere near as different as the doomers or the maxxers are predicting right now.
skippyboxedhero 5 hours ago | parent
In most orgs, there is also a gap between leadership, even tech leadership and people actually using LLMs every day. And it is challenging for actual devs who are being told that they should be going much faster and the reality which is more mixed. Pushing back on people who are convinced that Claude is God is going to be tough if you are a junior.
I don't feel the job has really changed, I don't write code as much but that time goes into working out the structure...which is what I was doing anyway. But it has changed the general environment, how work is planned, expectations, etc.
tripleee 7 hours ago | parent
If you have a large human-written system with a little thought put into it you should be able to find the specific pieces that handle the behavior you want and work on that, and if it's good then changing it should have a small blast radius and be covered with regression tests
In contrast, I've fully vibe coded a 15k line tool for myself with Opus and I have no idea how on earth it's architected after reading through it. It has "tests" but I don't trust them. It's just all over the place with no coherent design to it. It works, but it's making me more and more nervous to change it
skippyboxedhero 6 hours ago | parent
It is not completely useless as a tool but there is still a huge gap in most real-world applications (I work in an org with multiple thousands of devs and I am top 5 AI user consistently, I have used Claude Code since pre-release). If you are building something like an SQL database which has decades of logic tests built by humans then it can work. If you do not have this (i.e. 99.99% of the time) then you still need to be able to issue prompts that give context. And if you start without that context, it will produce shit code in extremely basic scenarios. In particular, humans are far better at producing code that is easy to change (in practice, you need to create unintuitive, non-human rules to govern this i.e. creating many artificial boundaries which would be completely unnecessary for humans).
I am not sure whether the models just capped out in terms of coding or whether they are just trained by people who have never worked on enterprise software but the gap isn't closing. It has just been filled by people who lack the knowledge to identify when they are being bullshitted (the prevalence of consultants suggests that this markets is large).
I have worked on legacy stuff before. In most cases, there is a kernel of reason and logic that can be recovered. I have seen pure AI projects and they are the worst grade of TCS-authored abandonware. They can often do one specific thing but are basically unmaintainable outside of that.
bpye 4 hours ago | parent
I frequently find this in LLM generated comments, both when using an LLM myself or when reviewing code that someone else used an LLM for. I find it is often the most obvious indication that nobody really read through what was generated.
tored 2 hours ago | parent
Then you investigate it and it is often something relatively simple that could have been explained with short sentence.
rwmj 1 hour ago | parent
ithkuil 5 hours ago | parent
This is not very different from what already happens to architects that delegate implementation to teams.
The challenge Is to find a way to encode the architecture in a way that both you and the agents can understand and agree on.
I'm still learning how to do this effectively, but my initial experience is quite positive. Not because agents are great but because humans are bad; the degree of coherence I've seen agents following is bigger than many human teams I've led in my career.
My benchmark is a quite big project for which the architecture is already well known, namely the reimplementation of the OpenVMS OS (kernel and user-land). There are many traps and many opportunities for the agents to just go ahead and diverge from the goals, so I found it a good playground toearn how reign them in but let them be effective at "drawing the rest of the owl"
bhaak 4 hours ago | parent
If you give them no constraints, they’ll produce code that just works, and it’ll look exactly like that.
We already know the solution to this problem. Software engineering. Put the LLM through the same grinder as any software dev and out constraints into place. Tell the LLM that the code design is bad and that it should improve it.
In my case, I had to go through several rounds of refactoring before the code at least resembled a good codebase.
A fun fact is that the refactorings didn’t break any functionality. Only adding new features broke preexisting behavior.
majormajor 7 hours ago | parent
And you can ask them to address this. And you can ask them to look for that in all your PRs. Etc etc. And that helps a lot.
But it's not something that can be brute-forced in the same way "this failed a test, let me fix it" is. So without a long-term-memory brain holding all the pain of past incidents and knowing and applying ahead-of-time judgement about what to build, it's a challenge to turn the general guidance into the specifics for any unique piece of software.
They're trying to get there of course. We shall see what's next.
("Human code turns into this too" is also a curious rejoinder from the "don't read the code anymore" crowd because there are entire disciplines and areas of study that have emerged and evolved from the problems of large ad-hoc multi-faceted systems being difficult and risky to deal with.)
patrickdavey 13 hours ago | parent
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
daveguy 12 hours ago | parent
csallen 12 hours ago | parent
I also don't even know if it's really true. Sure, maybe it's not coherent enough that a person understands it, but if an LLM wrote it, you can be ~99% certain (in my experience) that that LLM will understand it, and usually in less than the time it took to write it. And unlike humans, the LLMs don't die and take their knowledge with them, or change companies, or get ornery when you ask them to explain.
Also, in my experience, Claude doesn't really do what the parent is saying and dive head-first into complex problems and bork your code. It is fairly humble in the face of complexity. It constantly recommends breaking down larger problems into smaller ones that are more easy to grok.
But most importantly, anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful. Especially when they're writing code that's important. Unless you're an AI researcher pushing the bounds of what LLMs can do, which let's face it, the vast majority of programmers are not, it's almost always the case that you'll have your LLM writing intelligible code that you can understand and have it explain easily enough.
dieortin 11 hours ago | parent
Sure, you can do that, but irreparable harm might have been already caused. Sure, that’s unlikely when your product is an LLM wrapper. But in something like a database, it could be catastrophic.
csallen 9 hours ago | parent
(Yes, there will be individual exceptions to this, just like with everything else in life.)
what 6 hours ago | parent
lol, I assume you’re not a parent then. Or at best a new parent.
jdlshore 10 hours ago | parent
Is this a definitional argument? As in, if you’re not thoughtful, you must not be a decently skilled engineer? Because I see no reason to believe that people are going to be thoughtful about LLM-generated code. Just the opposite. It’s incredibly tempting to LGTM it.
csallen 9 hours ago | parent
I think that's true for low value code. But the more important and crucial and high-stakes code is, the more effort people put into it, generally. Even with LLMs.
pofferfish 9 hours ago | parent
customguy 29 minutes ago | parent
So the real question would be, how many people designing / implementing the boot process of any given system don't understand what what's going on.
evolve-maz 6 hours ago | parent
Those sorts of changes are really rough to manage. Maybe still manageable by LLMs but we'll find out.
myegorov 4 hours ago | parent
antihipocrat 12 hours ago | parent
SR2Z 12 hours ago | parent
dmd 10 hours ago | parent
Large company codebases are very much not like that. I've worked on systems (e.g., banking) that have accreted still-running code for as much as 60 years. Nobody understands how it works. Nobody ever understood how the whole worked, and many of the parts were never actually intended to work together.
hxugufjfjf 5 hours ago | parent
leonidasrup 5 hours ago | parent
But I have see quite terrible finance software, patches over patches hold together by a duct tape. Also finance software has to implement large complexity of tax law, which is very large and always changing. It also doesn't help if the company developing finance software sees software development as a cost center and wants always to push the costs down (especially short term costs).
hnedeotes 2 hours ago | parent
At some point I was tasked with improving the queries/retrieval times and started converting parts of the code and not only was it possible to write tests that would convey the actual meaning of the code and crystallise it but I started finding bugs in the implementation that no-one realized were even bugs because they no longer knew what it should do when queries grew to certain complexity and the search space was in the hundreds of millions of rows that thinking through them just with napkin calcs wasn't cutting it (specially when the bugs were subtle and the results still allowed the program to work "mostly" correctly).
pofferfish 9 hours ago | parent
What you are saying is leaning on 2 premises that seem to not hold water 1. Code written by people is always more readable and maintainable than LLM code 2. Code written by an LLM is always less readable and maintainable
Can you really make those statements in all intellectual honesty?
marcindulak 3 hours ago | parent
The comment https://github.com/anthropics/claude-code/issues/30407#issue..., written by Claude Code, and posted by the human GitHub account says:
> "Issues here are auto-closed by a bot when they've had no activity for a while and are missing information needed to reproduce (usually a version and steps, or a /feedback ID). Nothing is triaged solely by counting"
In reality all issues in this repo are subject to time-based auto-closing, unless they collect a number of upvotes. See details at https://github.com/anthropics/claude-code/issues/87647.
Therefore, I wonder whether the internal review process described in the Anthropic's blog for internal repos also results in such issues, and whether Anthropic has a internal way to measure and mitigate that. The blog post says:
> "When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today."
The Claude Code public repo problem shows that LLMs don't always "reflect the code as it is today", and can be a source of additional misunderstanding.
aturek 6 hours ago | parent
I love this phrase, and I think it captures the difference between the "60 years of legacy code" mentioned elsewhere here, and the vibe-coded monstrosity that we all know we're going to be encountering in the next 5 years. The fact that there were human brains with intent making all the little (short-sighted, myopic, ignorant, etc) changes in the legacy code means that you DO have at least some shot of gaining enough context to make safer changes.
There is no real intent in LLMs and particularly in blind vibe coding, so you'll end up reading stuff that works, or appears to work, but there was no human brain behind the change for you to model and thus predict.
TeMPOraL 1 hour ago | parent
gieksosz 4 hours ago | parent
curt15 1 hour ago | parent
At this point, why even bother with source code? Source code is designed for humans to comprehend. If humans are no longer the intended audience, why not have machines output and maintain some more efficient intermediate representation of computer programs?
hgh 13 hours ago | parent
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
xyzsparetimexyz 13 hours ago | parent
30,000 changes a day is not my idea of trivial.
californical 12 hours ago | parent
With a few thousand lines of code over the course of a week or so of my limited evening energy, I have agentic loops, ide integration, and a limited selection of custom tools which are all very safe (no unintended changes), along with session management and awareness of git.
It works great, is customizable, and allows integration with any provider.
If they need 30k changes per day, it’s not due to the nature of the problem!
xyzsparetimexyz 12 hours ago | parent
californical 12 hours ago | parent
Haha, I don’t doubt it! Pretty damning overall for long-term maintainability of vibe-coded projects, even as the models are great at a lot of individual tasks
chicagobuss 5 hours ago | parent
You have to remember all of the surrounding features it works with now, like the infra behind the cloud agents, or how remote control works, the mobile apps, their connectors, team skills and mcp servers, on and on....
xyzsparetimexyz 3 hours ago | parent
status_quo69 13 hours ago | parent
dimbletimbers 11 hours ago | parent
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
There are no shortcuts to true understanding.
tdhz77 10 hours ago | parent
giancarlostoro 9 hours ago | parent
j45 8 hours ago | parent
fouc 4 hours ago | parent
I guess it wouldn't be surprising for people prone to AI psychosis to be attracted to working at the frontier labs, resulting companies that are heavily in the throes of mass AI psychosis.
Being tongue-in-cheek here.
mk12 14 hours ago | parent
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
jazzyjackson 13 hours ago | parent
levkk 12 hours ago | parent
cjkhowes 7 hours ago | parent
nickspacek 2 hours ago | parent
TeMPOraL 1 hour ago | parent
paimapi 13 hours ago | parent
jansport123 13 hours ago | parent
lavezzi 11 hours ago | parent
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
atty 10 hours ago | parent
myth2018 10 hours ago | parent
wannabe44 7 hours ago | parent
mk12 7 hours ago | parent
> Most of the path is automated; a person's attention goes where the risk is
--> We automate as much as possible, focusing human attention on risky parts
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
--> We run reviewer evals against every incident
But that's just my interpretation and my voice. I can't know for sure what the authors would have said if they took the time to write it themselves.
versteegen 1 hour ago | parent
damontal 7 hours ago | parent
brof 4 hours ago | parent
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves.
userbinator 2 hours ago | parent
MisterMunchkin 1 hour ago | parent
mstaoru 13 hours ago | parent
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
vatsachak 13 hours ago | parent
Opus 5.5 is obviously stellar but come on what are you doing lol
pokstad 8 hours ago | parent
Rapzid 4 hours ago | parent
If it detects they aren't paying attention, fall asleep, or haven't approved enough PRs per hour; off the work at the factory.
compounding-err 13 hours ago | parent
KasianFranks 12 hours ago | parent
lbrito 13 hours ago | parent
This is not HN-worthy
nathanqueme 12 hours ago | parent
alasano 12 hours ago | parent
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
vektormemory 12 hours ago | parent
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
Very impressive now, compared to 6 months ago.
You can view it if interested: www.vekmem.com
throwaway_1016 8 hours ago | parent
vb-8448 12 hours ago | parent
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
cush 11 hours ago | parent
vb-8448 1 hour ago | parent
Agree that 30k PR alone doesn't mean much, but there aren't other details on what's inside. Based on I saw on how LLM programming works, it's hard to believe that there are single-line fixes. If I have to bet I'd bet on 200 LOC on average, that is 6M/day. Even if only a small portion reach production, it's still insane.
miroljub 12 hours ago | parent
I vibe coded my harness that is superior to CC in functionality, and it still doesn't flicker.
I don't need to hear their bullshit.
bdcravens 12 hours ago | parent
pianopatrick 12 hours ago | parent
maxgashkov 11 hours ago | parent
lifeisloving 11 hours ago | parent
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
maxgashkov 10 hours ago | parent
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
lifeisloving 9 hours ago | parent
While Im rooting for Anthropic to collapse under the weight of slop and their own contradictions. There's a leaked interview for Anthropic from earlier this year (I think) for their inference team. The interviewer seemed very competent and the quality of the questions were good.
This article is more likely a marketing gimmick in service of shelling shovels (for devs to dig their own grave) "hey look guys we automate everything with claude, you can too, just give us all your money!"
0c3ca83 11 hours ago | parent
smetannik 11 hours ago | parent
dieortin 11 hours ago | parent
That’s not how rates work
dozerly 9 hours ago | parent
Rapzid 5 hours ago | parent
Regressions are usually the absolute worst.
Up to now we've just been eating shit on the DX side as far as regressions in Codex and Claude Code because everything is all "go-go-go OMG I don't want to fall behind" and also.. Access to new models is tied to harness updates.
But a ton of industries have users that are not so willing to eat the regressions in the name of more features and changes than they can keep up with.
Edit: Also moving too fast.. You may have been less likely to "get it right" than ever before leading to design churn. Again, depending on industry and users, they may not be very tolerant to releasing the 5th complete overhaul of some feature in so many weeks..
senderista 11 hours ago | parent
sigotirandolas 1 hour ago | parent
* They can afford frequent CC incidents since their value is having the best models, and 95%+ uptime is still OK-ish.
* The core functionality of CC (the harness "agentic loop") is simple, an if that works then for most purposes CC works.
* CC just stores a bunch of config and history files, there isn't any financial or irreversible safety loss if they mess it up once in a while.
Once you work on something where a minor slipup can result in a huge real world mess, then I don't see how their approach can work.
gaal_dornick 10 hours ago | parent
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
llama052 10 hours ago | parent
c0rruptbytes 8 hours ago | parent
rddbs 10 hours ago | parent
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
pokstad 8 hours ago | parent
btbuildem 10 hours ago | parent
Personally, I find Claude's degenerate urge to just spew uncontrollable amounts of code quite counterproductive. Yes I get it, you get paid by the token. I'd like to see more reasoning, more understanding, and less generating.
Truth is, we are still very very far from actual AI software engineering. Some people are working on useful tooling, and (predictably, like anything good and useful) it is taking a long time. LLMs are great, and hopefully they will help us accelerate building the tools that bring this profession to the next level.
30k PRs in a month (or whatever) is not the brag they make it out to be.
TNWin 10 hours ago | parent
pokstad 8 hours ago | parent
cheema33 4 hours ago | parent
I disagree. Many of us have an emotional attachment to the way we do things and have difficulty letting go. Anthropic is showing a different way of doing things. And I am not going to judge them too harshly if the final output is good. I use Claude Code every day. And it is good. Ahead of the pack.
dimator 3 hours ago | parent
Yes, this is a different way of doing things, and it's clearly working for them, but there is no direct line between that and a "ahead of the pack" product.
qnleigh 3 hours ago | parent
beardbandit 9 hours ago | parent
On the other hand, I would have expected Anthropic to have a drastically more elegant SDLC.
avinoth 9 hours ago | parent
Similarly, Anthropic can afford to do this because they have virtually unlimited tokens and labs to train models for their specific usecase. The rest of us need code comprehension lest we become dependent on continuous supply of tokens (which is a state these labs badly want to achieve)
So, unless you work at a lab, or have unlimited tokens guaranteed through the life of your company, it would do both the company and you good to continue to read the code and not let these 30k PRs per engineer be the norm.
An excerpt from the article makes my point
> Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to. When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today.
thatsabadlook 9 hours ago | parent
Think I'm good on taking advice from companies that aren't making money.
avinoth 9 hours ago | parent
samuell 1 hour ago | parent
Rapzid 6 hours ago | parent
tayo42 5 hours ago | parent
scottyeager 4 hours ago | parent
jbjbjbjb 1 hour ago | parent
unddoch 1 hour ago | parent
thatsabadlook 31 minutes ago | parent
cheema33 5 hours ago | parent
For sufficiently large codebases in large organization that is how it has been with humans, who come and go. In many respects, What Anthropic is doing is a massive upgrade from the status quo.
ares623 3 hours ago | parent
Jesus. Give me 1 billion dollars and I'll keep your entire codebase in my head + personal notes for you until I die. Hell, I'll even force my descendants to do the same. Now _that's_ an improvement.
matltc 9 hours ago | parent
drdexebtjl 7 hours ago | parent
mcbishop 5 hours ago | parent
> ...Architecture that sets a pattern others will copy...
I'm happy that even Anthropic agrees this is best left to humans.
aryehof 4 hours ago | parent
So code review has become newspeak for “testing” and “education”?
Well not really testing. We the users are the testers and it falls to lucky us to really discover and find the errors, edge cases and security issues. The consequence of the rush to continually release…
Is the AI advance in the field of software testing nothing more than “now an AI agent can do a code review?”.
bob1029 3 hours ago | parent
I have some bad news about that "monorepo" designation.
stkdump 3 hours ago | parent
petersumskas 3 hours ago | parent
crtk 3 hours ago | parent
chrisjj 3 hours ago | parent
The less you understand code, the more you're impressed by coding chatbots - in part because you're less likely to read the bot's code, and if you do, less likely to realise how poor it is. Code gets judged good simply because it exists and costed little to create.
julesrms 3 hours ago | parent
I've written a harness myself and fair enough, it's more complicated than you might expect.. but surely at this point the claude CLI is pretty stable, and this must mainly be a ridiculous amount of test churn.
Or maybe their team of clankers is just creating busy-work to make the humans think they're essential. That's not unheard of in the human behaviours that they're trained on.
K0balt 3 hours ago | parent
Not only that, but AI is categorically terrible at ux review. I don’t mean aesthetics, I mean attention flow, workflow in general and what the user anticipates as a logical path to feature discovery and exploration. Increasingly, features are hidden behind workflows that meet a single use case but fail to be universally logical or in many cases, even accessible at all outside of the test case.
So, people… use AI in code review. It’s actually pretty great at it if you do it right, and you can ship better software with fewer bugs than ever before. But please, don’t pressure your ux guy to “10x” with AI. In fact, now that you are “10x”ing your development cycle, you are probably going to need to hire -more- ux developers if you don’t want your product to feel like slop even if it’s gold under the hood.
willtemperley 2 hours ago | parent
Vibe-coding complex software can work, but it's going to take a really disciplined team to manage this properly, like in this article.
movedx01 1 hour ago | parent
Nobody on the team keeps the mental model of Claude Code as it is today in their head, so nobody on the team is able to reason and decide about Claude Code in the future, is that right? So no long term architecture strategy can exist as no current architecture understanding exists. While it seemingly does work for a system which is a TUI, would the same be true for a complex distributed system today?
stuaxo 1 hour ago | parent
baq 1 hour ago | parent
misschresser 42 minutes ago | parent
Some of the most capable engineers in the world high on their own supply… I have no doubt they think this is aspirational but even as I know LLMs will forever change how we code, I’m not entirely sure 30,000 PRs for a coding agent is a volume that makes any sense.