51 points sshah03 1 hour ago 27 comments

sshah03 1 hour ago | parent

Full title had to be shortened: “Anthropic is Not a Safe AI Company” - Broad Coalition of Scientists, Legal Scholars, and Tech Leaders Challenge Anthropic’s Safety Branding Ahead of IPO

This is what they're all signing: https://misanthropicstatement.com/

I have not

gensym 47 minutes ago | parent

It's interesting to note that the list includes notable doomers and notable anti-doomers. One thing they can agree on is that Anthropic is not being honest or responsible. Hard to argue with that even just taking Ant's own statements at face value.

sshah03 35 minutes ago | parent

I do believe there are people at Anthropic (possibly including Dario himself to some degree) that genuinely care about AI safety. I also think that they realized it made for good marketing/hype, but realized to do what they want to do they'd have to throw some caution to the wind for the time being (as a collective, I'm sure there are some individuals that are purists about it).

I'm interested in seeing which way that behavior & honesty skews post-IPO.

argee 29 minutes ago | parent

"Caution to the wind" is a strange way to describe blatant plagiarism for publicity as in the Bonsai/Scion case. What possible "honest" reason can there be to do such a thing?

mpalmer 27 minutes ago | parent

Are you sure you're not completely changing the subject?

sshah03 24 minutes ago | parent

The Bonsai/Scion stuff is a different conversation I think.

They did say the system was inspired by Sapling etc... & That discussion is really around 1) What responsibility do companies have to make sure Agents haven't gone off and reverse engineered existing things, and 2) If there were ex-meta, ex-google engineers that produced code, or had AI produce code based on their memory of architecture at Google/Meta, at what point does that become an issue.

As a whole it's a different conversation than honesty around AI safety though.

platinumrad 21 minutes ago | parent

> What responsibility do companies have to make sure Agents haven't gone off and reverse engineered existing things

Low if it's only an internal tool. Very high if you're going to make a blog (marketing) post bragging about it.

pigpag 27 minutes ago | parent

They appear to have cared about safety much more when they were behind OpenAI.

onemoresoop 11 minutes ago | parent

I think they still care about safety. But the profits come before that.

platinumrad 24 minutes ago | parent

The money quote: "People in this coalition disagree on almost everything, yet we all recognize the danger of a company that convinces itself its own dominance is the greater good"

sleno 12 minutes ago | parent

The natural reaction to this quote is to sneer I guess...but isn't this an inescapable reality of anyone that seeks dominance in anything? What is the alternative we should expect? Dominance that acknowledges itself as evil? Is fighting for dominance automatically 'not good'?

I don't have a strong opinion on Anthropic but the situation is essentially a prisoners dilemma with no good option; either dominate and risk messing it up or relinquish dominance and hot potato that responsibility to someone else.

throwa356262 31 minutes ago | parent

No need for a coalition their own sec-1 filling spends tens of pages describing how they will end the humanity.

That's why I won't buy any stocks in the IPO. The Terminator is coming, I rather cash out and party hard my remaining days :)

luxuryballs 27 minutes ago | parent

they challenging all of them or was this coalition organized by OpenAI lobsters on a rack

neya 20 minutes ago | parent

Anthropic heavily lobbies and invests millions into lobbying. The only company to do so. And their agenda has always been to stand against open source models because they know they can't win that war. It's no co-incidence that Jensen's leaks were timed perfectly right before Anthropic's IPO. Jensen was a huge advocate for open weight models.

They're also far from caring about anyone's privacy and were caught spying on their users so..

strictnein 14 minutes ago | parent

> "invests billions into lobbying"

Off by a factor of ~1000:

"The Financial Times reported that Anthropic nearly tripled its lobbying expenditure to $3.53 million."

https://finance.yahoo.com/technology/ai/articles/anthropic-n...

neya 10 minutes ago | parent

Sorry, I meant millions. Fixed.

mastax 7 minutes ago | parent

> The only company to do so.

What do you mean by this?

moralestapia 16 minutes ago | parent

They should leave the unsafe AI company and raise money to build a truly true this-time-we-are-not-a-bunch-of-crooks-lying-for-money-guys-for-real safe AI company.

measurablefunc 14 minutes ago | parent

There is no such thing as a "safe AI company" because there is no actual safety algorithm. This follows from basic computability theory

caidan 13 minutes ago | parent

Does anyone have any credibility any more? What a pointless statement. We all think anthropic is naughty! And we co-sign this tweet. Me and the other wankers from Harvard et al. Have you guys thought about it or investigated or done anything besides sign a tweet like a bunch of tweens hearting a get ready with me post? Can’t say I see that anywhere. So what, we are just supposed to trust that your word means something special? Oh piss off.

The whole world can see the writing on the wall, Anthropic has said a million times that they are on a quest to get rich or kill everyone including themselves. And what did the world do? Gave them a trillion bucks. So what does your word do vs that? Help you sleep at night? Pointless twaddle.

I can only hope you don’t escalate to posting a kick me sign on Dario’s back!

jrflo 12 minutes ago | parent

I feel like their logic is internally consistent, even if you don't agree with the key assumptions.

1) RSI is very dangerous, and such self improving models should not fall into the wrong hands.

2) Releasing an open model is like opening Pandora's box, you can't recall it once it's released. If that model is capable of RSI, there's nothing you can do to stop bad actors from getting superintelligence.

3) Therefore, the safest thing to do is to ban open weights above a certain capability level. The problem is we won't know what that capability level is until it happens.

I don't know what the "weakening safety pledges" part means, would be curious to learn what they're referring to.

darepublic 4 minutes ago | parent

4) AI is capable of having a soul. With all the new souls that will be released into the world, Dario will be their pope and his captains the high priests

AshotSanamyan 8 minutes ago | parent

Do they need to be safe or everyone else needs to be safe from Anthropic? I think theres is a huge difference on which side needs to be safer, the AI to be safe or everyone using AI should be safe.

darepublic 5 minutes ago | parent

anthropic at least comes across as being the most up their own ass in regards to their view of AI and what is right and how they will be the judge of that. I still can't get over their lobbying to the vatican asking them to declare AI capable of having a soul. They are dangerous despite the liberal chalk drawings outside their HQ.