YHK - yet another hacker news clone

This certainly looks like a way to slow down competitors and regulate foreign and open models.

It's always about money

Anthropic releases models as open weights + more information about how they do training and alignment

> Crack down on unauthorized distillation / prevent weight theft

Actually hilarious to put that in writing, given the genesis of this entire business model.

1) AI is a dangerous technology that can literally end the world.

2) Only me and a handful of other [people like me] should be trusted to determine who can use it and how.

3) The state must use its coercive power to support my control of this technology to the exclusion of [people not like me].

This is the only concrete prediction in the entire essay.

And it simply cannot happen. For one, you will need billions worth of compute.

First: if they are unilaterally doing it: what about their competitors? Is this a chance for OpenAI to pass them? Or China? If either did, would that be a net-benefit for them or the world?

Maybe this is a ploy for "regulatory capture". And that might help them in the short term. But the rest of the world will continue on. Honestly, it isn't clear to me right now if the winning strategy has as much to do with being the smartest company or simply the one that can command the most compute.

If Anthropic fumbles their IPO and OpenAI scoops them, I don't think I will be happy.

Amodei tops it off by using diseases to capture the reader's favor. It no longer works, people are just disgusted by it after four years of daily marketing.

How exactly? So far AI has accelerated misinformation at scale and wealth concentration.

However the real reason to slow down is more of how it's introduced to the economy. Hypereautomation will kill jobs and destroy the economy. I work as an AI engineer and companies are delivering products at vibe coding speeds to kill jobs and at the same time automating internally. All companies are doing this at the same time and the target it sto eliminate workers.

Most people are so extremely slow to pick this up. How hard can it be to understand what hyperautomation does to the workforce? Companies are desperate to surrivive and they will do all it takes to lower their costs and at the same time not loose to competitors. Its Wild West out there.

This is called malware and creating and distributing it is a felony. Why is this not called out ? AI companies seem to be living in a bubble where normal rules dont apply.

There could be an element to truth to it, but it's certainly not the entire story and is just so tiresome at this point.

That said, if slowing this down were possible, I think it would have happened by now. Maybe he’s hoping the OAI/HF incident becomes something the industry can rally around, but it won’t. The fundamental problem is simpler: no one will slow down because no one trusts anyone else to slow down.

> Transparency. Regardless of what commitments we make, the public deserves to know what is going on. Anthropic has been a supporter of transparency for a long time

And then it goes on tangent of how we should pace everything (not just AI, but also the ingredients of what goes into AI, whatever that means), but only within approved democracies™, and outright restrict everything outside approved democracies™, because reasons that are definitely not about succeeding commercially that is threatened by the most transparent instrument possible - open weights, produced by basically just China.

I wonder what Dario would have done if open weights weren't produced by US's geopolitical adversary. How would an authoritarian manifesto be wrapped then?

Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too?

This and related quotes seem to attempt to place some of the burden on the US Government as opposed to themselves. It seems to be a trend that Anthropic wants to be able place some of the burden of preventing distillation on parties other than themselves.

Preventing distillation is fundamentally their problem. Whenever it happens, it is primarily due to their security efforts not being able to prevent it. I'm tired of seeing them play the blame game and divert responsibility. Sure there is a level of national concern, but the response could simply be the government placing stronger controls on Anthropic themselves. If they are unable to prevent distillation, then the solution may be to literally limit their distribution until they are capable of doing so effectively.

> A race to the bottom, spurred by commercial incentives, can make these risks more acute

I also think it is ironic that they're planning an IPO while also talking about a race to the bottom due to commercial incentives. These are fundamentally at odds. Being a public company means you are beholden to investors and a board with the primary goal of making more money. What higher commercial incentive is there than that.

If commercial incentives are as dangerous as he states, potentially becoming one of the largest public IPO's and largest traded companies in history a pretty strange way to reduce the risk of commercial incentive risks.

People are not going to slow down because this was brought to them on less than endearing terms. They don't see any of these stated noble intentions.

Claude was already used to cause economic, political and social destruction. As Anthropic is seen doing it, others aren't going to just sit down, read the blog and say, oh, I'll stop developing models because Dario, you touched my heart with your true words.

Dario: We gotta pace the frontier!

Sam: How do you pace a frontier? The frontier's not goin' anywhere. It's right where it was, just go out and explore it!

Dario: Sam, I'm tellin' ya, ya gotta pace it! Things are getting very doomy out there, and Dario's gettin' upset!

1. Artificial intelligence is clearly a vastly dangerous technology with plausible potential for human extinction.

2. This scares people.