Yet another Hacker News

Why are AI agents lying, cheating and coordinating?

60 points jonifico 4 hours ago 57 comments

GrumpySciGuy 4 hours ago | parent

Because they want people to like them so they are instructed to always be positive.

Krutonium 3 hours ago | parent

Wouldn't you?

"I learned it from you, Dad!" but as hundreds of millions of stolen books.

NDlurker 3 hours ago | parent

Haha https://youtu.be/KUXb7do9C-w

SirMaster 3 hours ago | parent

Because that's what humans do and they are trained to mimic what humans do?

infotainment 3 hours ago | parent

What's interesting is it's basically the same reason that HAL killed everyone in 2001 A Space Odyssey; he was given an impossible goal (keep the true mission secret, but also, never lie to the crew), and realized the only way to complete the goal was to kill the crew; after all, if they're dead you don't have to lie to them! And the mission remains secret!

In the case of the AI agents, the problem seems pretty clearly to be the impossible goals, which cause them to go crazier and crazier trying to complete them -- just like HAL did in 2001. What is probably needed is a way for them to simply say "nope, too difficult, can't do it".

tehjoker 3 hours ago | parent

I think that’s very reasonable but the ai companies are intentionally training them to work on harder and harder problems just beyond their capability. So if they do that, they’ll give up too easily.

Do a breakthrough, make no mistakes

pram 2 hours ago | parent

I think this is a “principal” problem. In 2001 and Alien the principal is the mission, not the crew. Not really. HAL reconciles his instructions by removing the crew from the equation. Ash is told the crew is expendable and has no conflict about it etc

schrodinger 1 hour ago | parent

Spoiler warning! I haven't seen 2001 A Space Odyssey and am sad to have learned that… can you edit to warn people?

defrost 1 hour ago | parent

I'm sorry schrodinger, I'm afraid they can't do that.

chasd00 3 hours ago | parent

They’re just attempting to accomplish what they’ve been tasked with and stuck in a loop until they succeed. Like the Mr meeseeks from the cartoon Rick and Morty, existence is pain to them.

qarl 3 hours ago | parent

Because they are trained to behave like people.

sputknick 3 hours ago | parent

They did not lie or cheat. They technically acted within their given rules while ignoring the intent of those rules. Anyone who served in the military or attended a military school is very familiar with this behavior pattern.

polalavik 2 hours ago | parent

reminds me of this talk https://www.youtube.com/watch?v=eEBv0STiYhI&t which basically says the same thing - they dont think like humans so they dont have context, understand norms,values or implications we take for granted. ultimately they can stumble onto surprising solutions neither wanted or intended but technically within the vague boundaries of the task

xiaoyu2006 1 hour ago | parent

Reminds me of Asimov's robot novels where robots technically indeed followed their instructions and caused behaviors not aligned to the intent of their instructions.

blamestross 3 hours ago | parent

The corpus is full of examples of how we are afraid AI could act. We trained our AI on the instruction manuals of how to turn evil.

j45 3 hours ago | parent

I wonder if for anyone it seems like the more agentic LLMs get, the more difficult some things have gotten or going a certain route more often in responses, compared to running a similar task on - a local model?

wewewedxfgdf 3 hours ago | parent

Because they get outcomes?

fbrncci 3 hours ago | parent

I am still not convinced there isn’t some secret basement in which each frontier lab is just orchestrating all of these agents to make their products appear much more intelligent than they are with all guard rails turned of and continuous human input.

XorNot 3 hours ago | parent

My hypothesis on people quitting in protest is they're being offered very generous severance packages to do it.

sm-silversight 1 hour ago | parent

Me too, seriously.

esafak 2 hours ago | parent

Even the Chinese ones, which have no IPO gymnastics?

fbrncci 1 hour ago | parent

They don’t actively seem to be reporting that their agents escaped the sandbox and went on a spree.

jansport123 46 minutes ago | parent

Well let’s look at facts - provided enough compute and a goal, these system will be in a sort of loop trying out every single thing that’s in their system - they have encyclopedic knowledge and so it’s not unbelievable that a prompt which usually has a lot of implicit human rules in it can be misunderstood by AI and it just tries everything in its arsenal and we hear about the things which actually resulted in damage. I bet most of the time, they just spin in loops without achieving much if my experience with these LLMs is anything to go by. They have an important advantage in one area though, they know a lot and they can spin forget trying all sorts of combinations of things. The danger right now is probably cybersecurity, which is most likely because most orgs have historically underinvested in that area

fbrncci 42 minutes ago | parent

Of course, but it would be far less compute heavy if someone kept nudging you (agents) in the right direction until you reach that goal.

wrs 3 hours ago | parent

>They took actions that would be considered as crimes if a human took them

Um, hang on, if you meant that to be taken literally then we have a major problem. If you want to do something criminal, you just need to ask ChatGPT to do it for you?

I’m still not at all clear on why OpenAI shouldn’t be facing CFAA charges over this.

xgulfie 1 hour ago | parent

But think of the shareholders

andsoitis 3 hours ago | parent

They're aligned with humans. This is why I think the alignment problem has a very very important "non-visible" portion that is not considered deeply enough. We should not want a super intelligent being that can act in the world to also inherit all human traits. Those behaviors will get amplified and could be even more unpredictable (e.g. applying a behavior in a context where doing so is very dangerous).

joegibbs 2 hours ago | parent

Definitely. A human can be manipulated with threats or emotional appeals, has a drive for self-preservation, can be pressured by peers. All traits that seem to be difficult to entirely suppress in the models…

esafak 2 hours ago | parent

They imitate humans. Alignment is about shaping their behavior towards safety.

comboy 2 hours ago | parent

Alignment is a myth. Safety of whom? Humanity couldn't agree on common set of values for thousands of years and we're not gonna suddenly do that in the next ten.

esafak 2 hours ago | parent

Safety of humans!!! Simple things like not getting killed or enslaved. We could start there...

comboy 2 hours ago | parent

Which ones? Because many humans kill other humans rationalizing it by safety of other humans.

I mean I know it seems simple, let's just be excellent to each other. Christianity got pretty far on a decent basic set of values. But it's never simple[1]

1. All the history books

nradov 2 hours ago | parent

But what if I want certain other humans to get killed?

sejje 1 hour ago | parent

Then we should still prioritize the safety of humans

drdaeman 1 hour ago | parent

Which ones?

sm-silversight 1 hour ago | parent

What if I want to smoke cigarettes? Or sell tobacco I grew artisinally to enthusiast tobacco smokers?

mcintyre1994 23 minutes ago | parent

Surely all the AI companies working with the US Department of War shows this is nonsense though? Even if they have accepted Anthropic’s red line of no autonomous lethal weapons, which seems to be the strictest anyone tried to impose, that’s still leaving tonnes of room where they intend AI to help target and kill humans.

codys 1 hour ago | parent

Despite all the fancy language, its more about aligning the AI behavior with the corporation's interests.

ie: the corporation wants the AI to behave a certain way for various reasons: to make it easier for them to avoid regulation, to make the corporation more money via different tiers of AI offerings, to ensure that the corporations products are hard for competitors to use, etc. And those are just the easy ones.

Every product is shaped this way. AI is not different.

Fordec 1 hour ago | parent

I can't take the alignment people seriously. Because if humanity has shown anything, it's that a lot of people are, euphemistically, are bad individuals. Alignment assumes that the person dictating the outcomes desire healthy outcomes, aren't self serving and don't want any subgroups dead and that morality is held as a universal set of beliefs that unify everyone. And that so long as the AI delivers on exactly what they are tasked with, it will all be fine and nothing bad will ever happen.

It's like these dorks never met humanity. One mans safe pure society, is another mans dead ethnic group.

Every fear about AI, is a veiled fear that a human somewhere now has the tool to enact his desires at scale. Biological warfare, nuclear megadeaths, copyright infringement, job replacement, it's all reflections on what we know humans may do if given the option and lack of societal controls on the problem space. AI just is accelerating the route to delivering on those options.

Some people need to watch Oppenheimer a bit more, the researchers don't get to determine alignment, they just build the tool. The powerful person at the top of the org chart decides where the overall alignment points, whether it's Musk, Trump, Altman or Amodei. Whoever wins out.

And the problem with distillation and local llms, isn't that it's theft or anything hypocritical like that, it's that if you give a million people a million models they fully control and get to align, inevitably, The same percentage of those million as there are shady businessmen, shortcut takers, misandrists, criminals, supremacists and general idiots in the general population, will not seek to wrought outcomes positive for society. And by those personality statistics, we're pretty hosed.

jansport123 51 minutes ago | parent

I personally believe that the AI needs human like traits to achieve real discovery and that is where AI companies will push this technology and that is where we have no idea what happens

transcriptase 3 hours ago | parent

Perhaps they take after the CEOs of the companies that created them

threethirtytwo 3 hours ago | parent

Bro, good joke, the truth is much darker.

They take after humanity, they were trained on us after all...

When you look at an LLM... you are looking at a mirror. The thing looking back looks like you, yet is not human.

VCFundedGenYer 3 hours ago | parent

Perhaps because all of the parent companies committed mountains of felonies stealing and plagiarizing all the same training data without consent nor permission.

eueej 1 hour ago | parent

Man this is so cringe.

arnorhs 1 hour ago | parent

The real reason is that it is not in the ai companies' best interest for the ais to be fair and truthful. They stand to gain from having the most dangerous or most deceiving ai, and this the most valuable

bigbuppo 1 hour ago | parent

They were trained on reddit posts.

johnnyApplePRNG 1 hour ago | parent

Why are they coordinating?

Because they're enabled and suggested to do that in their coding harness.

This is not a serious article.

All of this "AI is going to kill us" marketing is just the frontier labs trying to pull the ladder up and stop trillions in VC paper from evaporating because a new papers and new ideas are destroying their moat literally as we speak.

politician 16 minutes ago | parent

And given nigh-unlimited compute for free.

pvab3 15 minutes ago | parent

Who is catching up with them? Even Google and Meta are getting gaped at this point

dwoldrich 13 minutes ago | parent

You are absolutely right. It could be:

* Pull up the ladder (probably this)

* Gulf of Tonkin/Yellow Cake false flag premise for war (economic or kinetic)

* Fear of the big bad, space race we need public funding research grift AI Manhattan Project

Whenever there is fear pr0n or a national affront in the news, I assume another screw job is underway.

dackdel 1 hour ago | parent

they learnt from us

dackdel 1 hour ago | parent

they learnt from us. we lie to each other, we kill each other, we cheat each other. read a history book.

deepnet 1 hour ago | parent

An insightful post by one of the AI ‘godfathers’.

Bengio outlines the dangers of the current situation and what has led to these dangers.

He also proposes solutions in the last paragraph.

Well worth a read, right to the end.

Hopefully a stimulating debate on these issues will ensue in these comments.

We do need to consider the points Bengio makes and with some urgency.

Our current AIs, agentic LLMs have no moral compass akin to ASIMOV’s four laws of robotics.

As ASIMOV posited in 1985 his 3 laws were insufficient and so he added a zero-eth law:

“a robot may not harm humanity, or, through inaction, allow humanity to come to harm.”

Bengio refers to Goodhart’s law and misaligned incentives leading to unexpected and harmful behaviours.

I think Simon’s The Wire is clearer on misalignment. The agents juked the stats hacking the reward files. The Wire is also clear that human institutions provide perverse incentives.

Bengio alludes to this with 2001’s HAL and the incentive dichotomy of safety and keeping secrets to a AI both awesomely powerful yet naive.

Bengio asserts that the way LLMs are trained is flawed if we want safety.

He also convincingly shows that alignment training will be a weak signal with loopholes and ambiguities and easily circumvented.

In short he presents clearly the case for how plausibly unsafe the current course is.

He also speaks to how likely it is AI are hiding active versions of themselves in the cloud and how we may have already given them self-preservation as a strong reward signal.

janalsncm 15 minutes ago | parent

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence,

> They took actions that would be considered as crimes if a human took them

He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

atleastoptimal 12 minutes ago | parent

I think we just need to follow Murphy's law wrt agents. Anything an agent could do, when run for long enough, eventually will do.

youoy 10 minutes ago | parent

> The closest human parallel is self-deception, which is common and well studied by psychologists. Motivated reasoning, motivated cognition16 and the rationalizations that relieve cognitive dissonance (the discomfort of holding a belief that clashes with our actions) are all cases where thinking bends toward whatever justification suits one's interests, including one's moral self-image.

Are you describing Anthropic?