YHK - yet another hacker news clone

Jepsen: ParticleDB 1.1.4

72 points aphyr 5 hours ago 35 comments

https://github.com/jepsen-io/particledb

Powdering7082 2 hours ago | parent

> ParticleDB’s home page advertised that ParticleDB was “formally proven” with a TLA+ proof.

Perhaps they meant formerly, probably just a speech to text problem

lumost 1 hour ago | parent

TLA+ doesnt prove the code. if they never read the proof, then how do we k ow that it proved the right things?

hibikir 57 minutes ago | parent

And you still have to make sure all the real constraints are in. I had to fix a proven correct distributed algorithm that just happened to believe updates were done atomically and with no latency.

wavemode 1 hour ago | parent

lol, reminds me of the common misreading "provably safe" / "probably safe"

Powdering7082 2 hours ago | parent

-https://github.com/jepsen-io/particledb is currently 404 (probably still private?)-

EDIT: It's up now

aphyr 2 hours ago | parent

Sorry about that--yes, it's still private. Hoping to have that resolved shortly.

dang 1 hour ago | parent

Looks like it's up now. I'll put that link in the toptext as well. Thanks!

edude03 1 hour ago | parent

> We present 96 issues in ParticleDB, ranging from the pedestrian (crashes, deadlocks, data loss, etc.) to the piquant (partially-invisible rows which could not be deleted, WHERE 4 = 5, dropped tables re-emerging in distorted forms, confused columns, strings turning into integers, and so on).

Definitely seems worse than usual (looking back most databases seem to have <10 issues total, and the majority seem "mild") - since it's a database for AI was it also likely vibe-coded or just new? Then again even if new, I wouldn't expect so many issues in a stable release

cwillu 54 minutes ago | parent

The discussion section of the report is very damning.

Thaxll 1 hour ago | parent

Slop DB written in Rust. They probably changed the landing page after the bad results, you can still see the old one: https://web.archive.org/web/20260518203012/https://particled...

"Written in Rust · Formally Verified · PostgreSQL Compatible"

the_duke 1 hour ago | parent

Brutal!

> ParticleDB 1.1.4 claimed to be a Postgres-compatible database ... Instead, we found a library of confusion

> exhibited frequent, profound transactional anomalies in essentially every way we could measure. We routinely observed the loss of committed writes, internal consistency violations, G1a (Aborted Read), G1b (Intermediate Read), and P4 (Lost Update). In the append workload, successfully appended values would routinely disappear from later reads of a given key. Our test suite inferred every kind of cyclic anomaly including G0, G1c, G-Single, and G2-item, but many of these can be attributed to more basic issues, like write loss. These issues occurred with a single server and without faults, and affected every transaction mode, interface, and WAL mode tested

d5wiz 40 minutes ago | parent

these were tests done on pre release builds and some experimental concurrency modes as well.... The engine has since been redesigned and all of these issues have been fixed..

Its also faster than Redis for KV, also numbers show this should be top on clickbench, official submission is in progress

fabian2k 35 minutes ago | parent

Why should anyone believe that? I mean it might be true, I don't know. But if such a report turns out many fundamental issues it doesn't quite sound plausible that a redesign would suddenly not have any issues anymore. It might have fewer issues, it likely will have different issues as well though.

rat9988 31 minutes ago | parent

Because this is why companies pay for jepsen tests. Not because they enjoy to be destroyed in jepsen report, but to fix the issues.

fabian2k 30 minutes ago | parent

Most Jepsen tests leave me with the impression "databases are damn complicated and difficult to get right". My thoughts with this particular report are "I'll never ever touch this DB if I can help it".

cwillu 27 minutes ago | parent

This is like paying for a pentest of a treehouse, and claiming that this implies the updated treehouse with a working padlock instead of a novelty lock is in the same league as a bank or a data center.

It makes mongodb from the report a decade ago look good.

the_duke 33 minutes ago | parent

I look forward to the second report from Jepsen for the new implementation then.

kubelsmieci 32 minutes ago | parent

> and all of these issues have been fixed..

You can't be sure until they test it again

d1l 25 minutes ago | parent

Bro you've got to be kidding. pre-release builds? Aphyr directly addresses this in this analysis. Furthermore, given the absolutely ridiculous nature of some of these bugs, and the low confidence exhibited by the authors in the fixes, how can anyone assume that suddenly things are correct? The database was so janky that it sounds like most of the time was spent on basic bugs that a decent set of unit-tests would've caught. Who knows what wasn't uncovered because the db was so broken to begin with?

I mean did you see the "primary key equality" thing? Holy shit.

random3 4 minutes ago | parent

I wonder if it’s vibe coded by amateurs

pluto_modadic 58 minutes ago | parent

Aphyr does great work, Jepsen always kicks it out of the park :)

_bohm 57 minutes ago | parent

Is/was ParticleDB open source? There is a Github link on their website, however the only public repo in their org seems to be the Golang client library.

d5wiz 19 minutes ago | parent

https://github.com/orgs/particledb-ai/repositories

SDKs are MIT license CLI tools and Studio will be also be MIT License

Server is not open source for now but free for developers

foltik 4 minutes ago | parent

“No.”

CodesInChaos 51 minutes ago | parent

Since the library claims to be postgres compatible, I wonder if its developers ran the existing postgres jepsen suite against it (possibly with some adjustments). Or why not.

Especially now, where you could probably just ask AI to adapt the existing suites to the cosnistency claims this database makes, and get something useful out of it. While certainly not as good as a professional test, I'd expect it to at least find some low hanging fruits before going for the expensive and public official Jepsen test.

yla92 45 minutes ago | parent

The company says they did. From the post

"Orkes said they ran Jepsen’s existing Postgres test harness to find and resolve issues in ParticleDB prior to this work. However, that same test harness failed immediately when run against ParticleDB 1.1.4. Orkes also stated they tested ParticleDB with SQLancer, which generates far more complex queries than the ones we tested. It is therefore perplexing that nearly every feature we tested exhibited glaring errors, and that our findings seemed to come as a surprise."

mxey 40 minutes ago | parent

Seems to me that “we ran the Jepsen test” is just something an LLM thought would sound good on a DB’s website.

stacktraceyo 42 minutes ago | parent

Fatality

mxey 41 minutes ago | parent

It’s wild how bad it is. Not surprising for a vibecoded database, but why did they hire Kyle for a test, instead of first doing ANY testing themselves?

yipinwong 33 minutes ago | parent

Great to implement a DB just like human memory, unstable, and untrustworthy.

semiquaver 27 minutes ago | parent

Good on them for being good sports, but after reading this I would never touch a piece of software built by anyone involved with this company. The sheer hubris of their claims combined with the specific reality of their broken garbage software (with the contradicted marketing claims and hallucinated documentation) is literally breathtaking.

It must have been amusing to see all these defect tickets coming in, only for them to frantically shovel each one to increasingly overworked LLM agents to apply pig-lipstick while the humans in charge oh-so-slowly start to realize their software quality is not quite what their glazing agents had been telling them and that what they have on their hands is basically mongo-quality slop. What a world we live in, huh?

tensegrist 24 minutes ago | parent

> ParticleDB’s home page advertised that ParticleDB was “formally proven” with a TLA+ proof. However, during our collaboration no TLA+ specification was available. It is unclear what properties might be proven, or whether they described ParticleDB as built.

t3ntxcles 9 minutes ago | parent

Shipping a bug like "WHERE 4 = 5" returning rows says it all to me