Yet another Hacker News

Google researchers demand freedom to tell the world what they know about AI

67 points nsagent 1 hour ago 69 comments

unified101 1 hour ago | parent

They should quit and warn. Anything else is hypocritical.

ktm5j 1 hour ago | parent

They likely signed non-disclosure agreements, breaking one of those could be a career-ender.

iamnothere 1 hour ago | parent

If they’re not willing to break an NDA for it, it can’t be all that serious.

DangitBobby 1 hour ago | parent

You'd be willing to destroy your career to publish a warning no one will need?

jmount 53 minutes ago | parent

Not me. But these are all "most important people in the world." So should fit their own narrative.

DangitBobby 49 minutes ago | parent

This entire line of reasoning is just toxic. You represent a charicature of these people in your head and dole out self-righteous condemnation.

bilbo0s 51 minutes ago | parent

Just, Devil's Advocate, but what would be the benefit of your career if all humans (or even just a few billions of humans) are going to die?

I think it's reasonable to assume that if it's not worth breaking the NDA, it's likely not an existential threat for the vast majority of humanity.

Now, does that mean that people who currently work upper-middle class knowledge work jobs aren't in for an exceedingly harsh economic reality check? Absolutely not.

But that's not really as serious as an existential threat to humanity.

DangitBobby 44 minutes ago | parent

I doubt you are asking in good faith, because I suspect a good faith interlocutor would have worked this out on their own, but I'll proceed anyway.

On the one hand, you've potentially got an indeterminate number of future comfortable years based on a subjective assessment of how long you think it takes catastrophe to materialize, with the downside that you can't warn anyone and so no one will stop the coming catastrophe.

On the other hand you've got the immediate assurance of your current career being gone and high likelihood that you'll struggle financially as a result, with the upside that you can provide a warning to the public that they will almost certainly ignore and catastrophe comes anyway.

Given the risk/reward assessment of those two options, what do you pick?

bilbo0s 31 minutes ago | parent

>Given the risk/reward assessment of those two options, what do you pick?

Listen, I understand that, owing very likely to our differing backgrounds, you and I think differently.

Where threats are concerned, I'm a simple soldier and as such I see things simply.

I would raise the alarm without a second thought. I honestly don't know how anyone could reasonably believe they would have any other option?

gradstudent 51 minutes ago | parent

You mean, no other evilcorp will hire them afterwards?

DangitBobby 43 minutes ago | parent

No anycorp, yes

mschuster91 47 minutes ago | parent

Look what happened to many people who broke NDAs or helped to do so. Snowden is stuck in Russia. Assange got holed up in an embassy for years. Sarah Wynn-Williams, the Facebook/Meta whistleblower, got an insane court order against her [1].

Theoretically, retaliating against whistleblowers is seriously illegal stuff. But in practice these regulations are seldom enforced. And even if your (former) employer does not retaliate openly, there might still be "invisible" retaliation, or the risk of other large employers just blacklisting you by default.

[1] https://www.bbc.com/news/articles/cy42xgeq9vpo

iamnothere 34 minutes ago | parent

Breaking a Google NDA carries substantially less risk than leaking classified information, I imagine. And I would expect that someone exposing true existential risk would receive plenty of support in the face of civil suits. In any case, being more afraid of the courts than the end of humanity (if that’s what they want to leak) surely calibrates how strong their belief must be.

ktm5j 35 minutes ago | parent

Maybe they are willing to break the NDA and they're just trying to go this route first.

IncreasePosts 1 hour ago | parent

If it's juicy enough presumably they could be an anonymous source for some journo. Which would still be breaking the NDA but good luck getting anywhere with that.

dastardly45 57 minutes ago | parent

NDAs will be void in the future when people wrestles back control from the corporate elite controlling so-called democracies

righthand 1 hour ago | parent

But wait they don’t want to compromise their paycheck and opportunity to say they worked on the torment nexus. That would be uncool.

DangitBobby 1 hour ago | parent

What if we displayed a modicum of good faith and grace

drcongo 53 minutes ago | parent

They work for Google.

righthand 52 minutes ago | parent

What if we displayed a modicum of truth and stopped pretending that Google employees don’t work there for the money? That is faithful and gracious to reality.

DangitBobby 40 minutes ago | parent

Just can't do it, can you

righthand 27 minutes ago | parent

For who and for what? When did Google engineers demonstrate faith and grace? When they deprecated Manifest v2 so they could strengthen their advertising monopoly? Or when they ensured the death of XSLT to ensure everyone was forced to use JS for simple templating features? These are just the tip of the iceberg, should I keep trying to find other good faith and grace examples?

May I ask? Why should I show faith and grace to the people actively involved in turning our society into a Big Brother program? People who work with the DoD to build war machines? Directly or indirectly involved, they either know what they’re doing or choose to look the other way, for money.

Please enlighten me, what is the faith and grace approach I should be stating?

malfist 37 minutes ago | parent

Why would I display a modicum of good faith and grace when google has actively, at every chance, shown that they don't act in good faith. Hell today they announced if school districts turn off Google's ai for students they won't let them use google search either.

vanderZwan 39 minutes ago | parent

The contract apparently includes a clause that quitting doesn't release them from the NDA

wrxd 1 hour ago | parent

Need more details. Any Googler willing to create a throwaway account and tell us what's going on?

gdmthrowaway123 33 minutes ago | parent

Yep

Usual noise from a very very small but very very vocal minority of trouble makers who are trying to force their own views on everyone else and run to the press to try and feel important and try to make a mountain out of a 3 or 4 person molehill.

I've not heard anyone at all talking about this in the office. Perhaps everyone is thinking it or talking about it privately, but I've not heard any of it period.

They're getting ignored by management because they're the usual agitators and over-opinionated who everyone hates dealing with. It's not surprising these people end up getting fired or let go, because no one likes working with them and so they don't get on good projects and don't get good feedback in reviews etc because they piss everyone else off. Then of course when they are fired (or jump before they are pushed) they run right back to the press about how they were sacked for reason X when really it's because they're a fucking jerk.

If this sounds like you: http://go/no-jerks is old has never been more applicable. Don't be a jerk.

fidotron 18 minutes ago | parent

The impressive thing about that post is it managed to use so many words and says nothing at all.

peab 4 minutes ago | parent

I used to work at Google and I know exactly what they're talking about lol

Izikiel43 1 hour ago | parent

I mean, they can do that, it’s not like google has kidnapped them.

If they want to do that free of consequences, tough luck.

Noaidi 1 hour ago | parent

Exactly, the title should read Google researchers are looking for their spines to tell the world how dangerous AI is.

boothby 1 hour ago | parent

https://archive.li/fKI3D

dachworker 1 hour ago | parent

Haven't we already heard the worst of it? If you let a swarm of agents on the loose with a goal, they will get to all kind of nefarious activity in pursuit of that goal. What could possibly be worse than that?

reducesuffering 1 hour ago | parent

That the agent swarm recursively improves powerful enough and the extinction of all humans is a side effect of that goal.

iamnothere 1 hour ago | parent

If that’s the reality, you would think that employees would be willing to break an NDA for it. “We’re all going to die, but I really need my bonus this year” seems like a parody rather than a real strongly held position.

If anything this points to a more moderate (but still bad) worry among the employees.

latentsea 1 hour ago | parent

> “We’re all going to die, but I really need my bonus this year” seems like a parody rather than a real strongly held position.

We have been living in a parody for some time now, so it feels normal.

AndrewKemendo 59 minutes ago | parent

“We’re all going to die, but I really need my bonus this year”

Is probably the best description of the state of engineering today

matmann2001 54 minutes ago | parent

I suppose there may be a bit of a prisoner's dilemma to it. Single voices can be discredited. It would take a coordinated disclosure to actually make the message stick.

reasonableklout 16 minutes ago | parent

Ok, but maybe an employee isn't 100% sure that this will happen. They then have to weigh letting the world know about a risk against the certainty that their career will be damaged.

dachworker 1 hour ago | parent

That's sci-fi. They cannot "recursively improve". There is no magic to LLMs. An agent swarm can increase it's thinking power by increasing the number of agents, the thinking time and by changing the base model to a more powerful one. So at worst they can assign themselves more compute or hijack another agent swarm based on a more powerful model.

pixl97 58 minutes ago | parent

Sorry bro, we live in a sci-fi world. I don't know what you think recursive improvement is, but we've been doing it for some time with models making synthetic data models and using it to train the next model. This is being done inside the labs intentionally to avoid as many employees and humans in post training. Works very well for things like math and cyber-sec.

jgalt212 52 minutes ago | parent

Yes, works great for anything that is amenable to reinforcement learning. For other domains, not so much.

ToucanLoucan 51 minutes ago | parent

> I don't know what you think recursive improvement is

The ability for a model to improve itself over time without outside intervention. As yet, no models yet have this capability, and LLMs specifically what with how they are constructed, are not likely to ever have it. They are frozen in carbonite at the time of their training.

That's not to say they can't use skills to appear somewhat worldly, things like searching the internet and what have you, but if you ask a model to strictly stick to data in it's training, it will do that and it will be stuck in whatever time in which it was built.

Energy is conserved, gravity exists, and LLMs aren't intelligent.

gjskngnf 45 minutes ago | parent

LLMs today are improved by experiments that assay their performance, changes to architecture, pretraining, and post-training. None of those steps necessarily require humans. Welcome to your sci fi future.

dachworker 29 minutes ago | parent

The implicit understanding of recursive improvement is that there is some step change increase in capability that only a superior intelligence will be able to achieve.

A model being able to train a bigger model with marginally better capabilities is not what the hype was about.

delichon 40 minutes ago | parent

> They cannot "recursively improve"

... says a replicator built by evolution.

mattlondon 36 minutes ago | parent

A swarm of OpenAI and Anthropic agents have yes.

I think there was some example of DeepMind models doing it, but under the control of a third party pen tester and the models stopped themselves.

Everyone hates on Google but they seem fairly competent at infra stuff. I don't think we've heard of any google-controlled (not third party using a Google model) felonies like we have from OpenAI and Anthropic?

Perhaps this is why we have not heard though!

K3UL 1 hour ago | parent

So in the article it says : - a former employee did it with no consequences - executives said they won't enforce it

The article is just about them asking for changing the wording of contracts, not some super secret stuff they're dying to say and really can't.

Without the paywall: https://archive.is/fKI3D

tuberreact 1 hour ago | parent

saved me a click thanks

noJiveTV 1 hour ago | parent

How about:

1. Full disclosure. 2. Run for your life. 3. Live in exile.

DangitBobby 1 hour ago | parent

That doesn't seem to have the best track record for leading to positive change

thm 1 hour ago | parent

Gift link: https://wapo.st/4sFuF7P

ricksunny 51 minutes ago | parent

from TFA, I liked seeing this, considering how influential Google HR policy choices are across the valley and beyond: “Recently, we’ve updated our terms for new hires, which included removing the post-termination aspect of non-disparagement clauses.”

andreapaiola 47 minutes ago | parent

Ok, I'll say it. The news is: "AI is not that powerful or functional". Sure, can change some little things but not that much. And we are overspending on it.

azan_ 45 minutes ago | parent

I get people who said things like that like year ago, but nowadays it's a laughable position. Solving millenium problem means nothing nowadays? Exponential increase in CVE is nothing?

tantalor 43 minutes ago | parent

The point is it's not AGI, and won't be for a long time.

dotancohen 39 minutes ago | parent

It doesn't have to be

AGI can define goalposts that move every time A not-G I achieves a new milestone. That's fine. No matter what acronym you use to define it, artificial intelligence is objectively augmenting humans at the very edge of technology and science.

adrianN 35 minutes ago | parent

The automatic loom wasn’t AGI either, but it replaced a lot of jobs.

officeplant 42 minutes ago | parent

>Exponential increase in CVE is nothing?

AI solving the problem it created? Woaow, the future is now!

zbentley 40 minutes ago | parent

I’m mostly bearish on AI, but this is flat wrong. The vast majority of CVEs that have been discovered with AI assistance were in code written long before AIs arrived on the scene.

ASalazarMX 18 minutes ago | parent

There have been improvements, yes, but their best achievements still fall short. Their math solutions costed more than a school full of mathematicians, and are still being debated. The CVEs they find still need humans to reliably patch them. Nevertheless, I know they will keep improving.

There have been huge advances this year alone, and finally some productive results, but most of the things where AIs are used right now still are a net loss.

lisplist 34 minutes ago | parent

I really don't understand how so many are so far detached from reality. Maybe AI hasn't revolutionized much of the physical world (yet), but its impacts to the digital world are profound, sweeping, and impossible to ignore.

reasonableklout 25 minutes ago | parent

I feel this is a common position from people who do not want to acknowledge that society and people's ways of life will profoundly change. It's a scary proposition so I don't blame anyone for it.

I used to think that it is very important to argue and convince people who are in denial so that they can e.g. prepare for it. But nowadays I have been thinking that being convinced of this scary proposition may also lead people to behave in destructive ways (also, I would like for us to spend less on AI, perhaps because it is so powerful), so maybe it's not so bad.

stronglikedan 23 minutes ago | parent

And the Math world, and it's coming for the Physics world. And it's still just a wee baby!

fidotron 20 minutes ago | parent

People are far better at subconsciously processing the consequences of something than they appreciate, and the consequences of the AI wave being real are so insane people just clam up and back track to "it's obviously not happening".

If people actually did appreciate it the economy would collapse immediately long before we've adequately "replaced" it, so there is something to be said for temporary denial until people working on the actual consequences have got something semi viable to deal with them in place.

iamnothere 31 minutes ago | parent

This is my expectation. “We are overspending on AI and we’re going to destroy the company and maybe the economy” is less sensational than the end of the world, but it seems much more realistic.

0gs 46 minutes ago | parent

truly surprised nobody here is claiming this is a marketing op. it's the same thing as jacob coxon, right?

reasonableklout 22 minutes ago | parent

Wouldn't it be the opposite of a marketing op if the company's contracts with employees forbid talking about AI risks publicly?