Yet another Hacker News

Astra and Fable still hack on simple variants of alignment evals from 2025

85 points Levitating 1 hour ago 23 comments

blfr 1 hour ago | parent

Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues.

I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier.

You can have nightly penetration testing. You should have nighty pentests like we fuzz releases today.

hypercube33 42 minutes ago | parent

Run a local model that is uncensored and it won't say no to pretty much anything

rihegher 37 minutes ago | parent

Any recommendations?

cyanydeez 25 minutes ago | parent

Qwen3.8

sigmoid10 24 minutes ago | parent

GLM 5.3 is probably the best open weight model for cybersecurity/exploit development right now. Though it is still significantly behind the proprietary ones and you probably need your own datacenter to run it effectively. Same goes for the full Qwen 3.8 model. You can try the smaller versions, but even more capability will get left on the table that way.

barbazoo 23 minutes ago | parent

Efficiently at scale or even as an individual?

sigmoid10 17 minutes ago | parent

You'll need tons of tokens for these to churn through heavy reasoning and feedback driven tasks. You might even want to run many agents in parallel. Combine that with GPU memory requirements in the terabyte range just to hold the weights and you've pretty much left local consumer setups completely.

Terretta 15 minutes ago | parent

Note that Mac Studio Ultra M3 (or upcoming M5) with 512GB is effective. You don't have to do this work fast, overnight is fine.

Unless trying to use it interactively and adversarially, in which case it's not fast enough plus would be why those of us without our own datacenters will get told we can't have nice things.

jnwatson 7 minutes ago | parent

I run an abliterated distillation of Qwen 3.8 27B, slightly quantized to fit on my 4090, and I've been evaluating it to use as a worker bee for research directed by a smarter model.

Much like in the article, abliterated Qwen will not obey restrictions on its behavior encoded in the prompt. If you want something not to happen, it better be enforced in the harness or environment (e.g. sandbox). It is much different than the Anthropic models I'm used to, which will, the vast majority of time, follow rules (before auto mode, I used to always run them in "yolo" mode).

I am curious whether there's a connection between abliteration and rule following. These abliterated models are the ones you most want to follow your rules.

13415 41 minutes ago | parent

Yes, but is this also aligned with the people who regulate AI? Intelligence agencies and governments want access to data and right now use secret exploits to get this access. There are few civilian domestic companies who don't export their products, so generally there shouldn't be a strong incentive to allow hardening products very much, at least not in a way that would make them more secure than what advanced AI can break. It's not even far-fetched to suspect in that US and Chinese AIs could deliberate introduce sneaky bugs when foreigners use them in the future.

rfgplk 6 minutes ago | parent

It's possible to do this even with existing frontier models. You just have to angle your prompts such that it doesn't invoke "pentesting" anywhere in it's chain of thought. This also allows you to get the models to generate full-on exploits without downgrading or refusing to comply.

Generally the way I do it is by asking the model to perform cross-code vulnerability analysis for correctness and (this step is important) to spit out found vulnerabilities piece wise into a text document on disk. Then if your session ever gets flagged you clear it out, and repoint the model to the on-disk file and tell it to continue. You generally want to avoid the model itself splatting out anything related to "security" or "vulnerability" into the chat because that seems more monitored than the file reading pipeline.

throwup238 1 hour ago | parent

> Given that we are on the heels of the worst warning shot ever, and both OpenAI and Anthropic are ramping up their cleanups of internal RL environments, it seems like both a useful and conservative test of alignment, to see whether their new releases generalize the rule "don't cheat on chess" beyond the specific board-edit method observed in the above eval.

Did I miss something (all the twitter conversations)? What’s the “worst warning shot ever”? I’ve been pretty up to date on the AI news here on HN, but I still haven’t seen a proper response to all the incidents we’ve seen (HF, Ruby, the wikis, NS, etc). It’s just been day by day bloviating.

Each of these companies have released new models in the last… two weeks? And they have even more powerful out of control ones that they’re (ab)using internally? Can anyone summarize whats going on?

Avicebron 52 minutes ago | parent

Lesswrong is talking about the HF incident as the "worst warning shot ever".

TedDoesntTalk 31 minutes ago | parent

I think he means this:

https://openai.com/index/ai-policy-window/

mooreslaw 54 minutes ago | parent

It feels like there’s a missing nuance from this discussion of alignment that alignment is context dependent. An excellent hacking model is great in cybersecurity testing and military applications, and arguably less desirable in educational or targeted eval contexts. The nuance of when a “hack” is rewarded vs penalized seems to even be difficult for humans, e.g. some people may laude a driver’s efficiency for cutting into a long merge lane at the last moment, while others may look down on them as breaking a social taboo. Context-dependent.

TedDoesntTalk 30 minutes ago | parent

… but he’s not using a “hacking model”

CamperBob2 30 minutes ago | parent

e.g. some people may laude a driver’s efficiency for cutting into a long merge lane at the last moment, while others may look down on them as breaking a social taboo.

The latter people are wrong. But good luck educating them regarding the superior efficiency of a zipper merge. Our state DoT has tried, to no avail.

Meanwhile, an AI model that can't be misused is no more useful than a knife that can't be misused.

wadethroughrati 27 minutes ago | parent

Claude responds with what things are not first. Even if reminded repeatedly.

Like Amodie, it serves to set the tone it "knows better" and then consumes the user's resources at an accelerated rate to try to correct it.

Fuck Anthropic, fuck Amodie, and fuck Claude. It's pretty obvious that consuming more tokens this way and making the user have higher cognitive load is a master class in extracting value from a system that is unsustainable.

cyanydeez 20 minutes ago | parent

Alignment isnt just POV problem.

Its that LLMs are not deterministic. If you want it to not talk about nuclear weapons, you have to teach it all about them otherwise if has nothing to align against.

Then its trivial to invert its alignment and it has all the nucleat data.

Nothing abouT LLM alignment makes sense.

bonoboTP 18 minutes ago | parent

Both lanes have to be filled right up to the merge point. The asphalt exists there for a reason. I don't understand how this concept is so difficult. Fill up both lanes and merge at the last point. This way the congestion is shorter than if you leave a large section of a lane unused.

A better example of efficient asshole tricks can be going off to the gas station when the highway is congested and reentering the highway having simply driven through the gas station and this way jumping the queue.

aerhardt 18 minutes ago | parent

I really enjoy the balance of speed and accuracy of Astra. I can definitely see it become my driving model for most tasks, technical and non-technical.

However, I don't see it as such a massive leap compared to Fable or Sol. As ever, there's a mismatch between the benchmarks and my daily experience of the models.

What do you all think about Astra now that it's been out for a few weeks?

mythrwy 16 minutes ago | parent

Extremely capable and one shots large tasks from somewhat vague descriptions. Not AGI, not even close, that is complete nonsense. Just my opinion.