Yet another Hacker News

Postgres 19 is late, and it's AI's fault

53 points soheilpro 1 day ago 41 comments

droppedasbaby 1 day ago | parent

I guess the real challenge is figuring out how to scale verification, cause discovery isn't the bottleneck anymore. Generating more bug reports doesn't necessarily translate into faster fixes if the same small group of maintainers still have to validate the reports, review fixes, etc.

c0_0p_ 14 minutes ago | parent

Probably true if development accelerates because of AI, but if it's reasonably stable then hopefully new bug reports will plateau.

hopa 35 minutes ago | parent

I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify

simonw 33 minutes ago | parent

I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included?

For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.

27183 24 minutes ago | parent

> The most common error for a search task is omission, and that's really hard to spot

NP-Hard, even[1]!

[1] https://www.i-programmer.info/news/112-theory/3830-physics-i...

whattheheckheck 7 minutes ago | parent

We're never solving nphard. Just like we'll never extract all of the coal/oil/natural gas from earth

ipdashc 33 minutes ago | parent

I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc

mihaelm 10 minutes ago | parent

The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it.

MBCook 28 minutes ago | parent

But it DIDNT “write that feature”. They made it extremely clear on the delay announcement the code was written without AI.

AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.

Postgres did not have a slop issue and it’s wrong for people to insinuate it.

bheztio 26 minutes ago | parent

"Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem.

Forgeties79 10 minutes ago | parent

This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.”

vatsachak 18 minutes ago | parent

What? AI can't easily implement features. Just write a spec, ask if to expand it and then ask it to implement it

r_lee 14 minutes ago | parent

I mean the transformer model came out of Google, and it's literally a query thing so yeah.

any kind of pattern matching / transformation is going to be an ideal task for LLMs

jeremyjh 6 minutes ago | parent

It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search.

ltbarcly3 33 minutes ago | parent

I have found major bugs in late beta's of postgres long before AI came along. I found a bug where malformed json would segfault the entire server (a segfault in a server process forces the entire cluster to shut down by design) when using jsonb in a release candidate once. They fixed it rather than ripping the entire feature out.

https://www.postgresql.org/message-id/20140818195634.2543.56...

It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.

StilesCrisis 29 minutes ago | parent

If you are at the RC stage and still uncovering segfaults, "rip the feature out" is probably the only choice.

benjiro29 22 minutes ago | parent

Maybe its more a issue that X amount of people have more work to fix discovered issues. So the new features have not undergone the same level of testing as before.

And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.

LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.

When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.

ltbarcly3 15 minutes ago | parent

Are the features bad by design? Like are they bad even if they work perfectly because they add bad abstractions to pg's SQL syntax, or are just broken in how they are supposed to work? If so the process is dysfunctional to let them get this far before noticing.

Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.

Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.

27183 32 minutes ago | parent

I'm not very positive on the social, technical, or intellectual impact of AI on software development. Actually, that's understating it. I think AI is basically cancer for the mind. That said, this framing gave me pause:

> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.

Two thoughts:

1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.

2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.

From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.

sverhagen 8 minutes ago | parent

Signal quality... what is that even? The bugs may be valid, but they may also be obscure, irrelevant, low priority. A sibling thread mentioned that they had found CVE-level bugs in Postgres, and those require attention in some version of "immediately". But beyond that... not every bug needs fixing; that's a trap.

meow9 20 minutes ago | parent

I’m still amazed that with all the vibe coding, no one has created a Postgres alternative

nfriedly 19 minutes ago | parent

Why would they? Postgres is free and does an amazing job.

spolitry 16 minutes ago | parent

Also Postgres has enjoyed decades of debugging.

scrollaway 15 minutes ago | parent

To be honest I would love for someone to work on the structuring of pg_dump for example, which is entirely driven by regexes and binary structures. It’s awful and absolutely not programmable - a real shame.

sa46 18 minutes ago | parent

Many recent relational databases speak the Postgres wire protocol. For wholesale replacement, pgrust is a recent, contentious contender.

gnabgib 14 minutes ago | parent

I hope you don't believe this, PGRust has fallen far from grace. Wire protocol support is useful (other than said recommendation)

Remove pgrust: the entry is cheating (per-query kernels and result caching) (197 points, 10 days ago, 166 comments) https://news.ycombinator.com/item?id=50205816

TomatoCo 16 minutes ago | parent

Someone tried. https://jepsen.io/analyses/particledb-1.1.4

esalman 11 minutes ago | parent

You can create a project on vibes, but you cannot maintain one on vibes.

CharlesW 20 minutes ago | parent

> This year is going to be different. PostgreSQL 19 most likely won’t be released until November… […] So, what happened? The short answer is that LLMs happened. […] There’s a huge amount of code, and that code contains a huge number of bugs. […] What’s different now is that it’s become vastly easier to find those bugs.

This seems…good?

That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.

27183 14 minutes ago | parent

Yes, exactly. The only question I have is whether the delay is attributable to high-quality bug reports or the burden of triaging low-quality ones. It's not completely clear which situation they're experiencing.

jeremyjh 11 minutes ago | parent

It’s pretty clear there are a lot of valid reports based on the large increase in backport commits cited.

micromacrofoot 10 minutes ago | parent

likely both

27183 8 minutes ago | parent

That's kind of what I was afraid of. Both is really bad news. If you think of it in terms of average speed over ground, working on low quality reports is like spending a bunch of time at v=0.

InsidiousWombat 7 minutes ago | parent

I'd imagine just as every other project, both.

tibbar 19 minutes ago | parent

I did a little AI-assisted bug-hunting in Postgres this summer in between jobs, and the haul was staggering. Multiple serialization anomalies, foreign key integrity issues, planner correctness bugs. Various CVE-worthy vulnerabilities. An unfixable timezone bug. The list got so long that I had to triage what I even had time to report, and settled for a dozen or two out of hundreds of candidates. Now imagine the entire world doing this.

rogerrogerr 11 minutes ago | parent

> unfixable timezone bug

Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?

tibbar 9 minutes ago | parent

I will add more example links in the top-level comment, but here is the thread for that bug. [0]

[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...

flossly 4 minutes ago | parent

“A wizard is never late, nor is he early; he arrives precisely when he means to” -- Gandalf