28 points pramodbiligiri 2 days ago 22 comments

swiftcoder 2 days ago | parent

Sounds good in theory, but are there evaluations of whether or not it actually works?

robbomacrae 2 days ago | parent

The answer to that is no. A lot of these are released on vibes and showing that it does what it says it does. For most organizations that care about processes and procedures thats... enough? I've worked on agentic development in enterprise and they just want their way to stamp their old methods on to the new agentic world so it can pass through their release hoops.

I've tried to benchmark them the most common public skills including those for ADLC (ie SuperPowers) couple of times here https://orcabot.com/benchmarks/agent-skills. Annoyingly I haven't been able to release one in a while because its been next to impossible to stop the new frontier models from cheating one way or another.

swiftcoder 2 days ago | parent

> A lot of these are released on vibes

Yeah, I kind of figured. I was hoping was AWS-affiliated lab would be a little more rigorous

Throw12349067 2 days ago | parent

I am one of the organisational victims who have been trialling this.

My assessment has been just use the native harness.

CharlieDigital 2 days ago | parent

My org has something similar.

Evals showed that speed and token use improved when it was removed.....

The models have improved and internalized some of these principles.

They still have value, IMO, because they help improve consistency (at the expense of speed and cost).

jiaosdjf 2 days ago | parent

Probably the greatest problem in AI right now is proof / benchmarking

I check every benchmark site weekly, compare harnesses and models, and its still pretty much just vibes

hersko 1 day ago | parent

My company did a workshop on this with Amazon and it was not great

bessbd 2 days ago | parent

> 5 phases and 33 stages from initialization through operation

> 14 agents: 11 domain experts, 2 reviewers, and an adaptive composer

wow, that's a lot of domain experts, I'm convinced

robbomacrae 2 days ago | parent

It's a great for tokenmaxxing /s

zamalek 2 days ago | parent

It was like that time I nearly dropped out of undergrad, but it was my mistake, I had just forgotten that I was actually a doctorate.

longwave 2 days ago | parent

> The current recommended model is Claude Opus 4.8.

Staying well behind the frontier, just in case.

fcarraldo 1 day ago | parent

the technical term is “pacing”

N_Lens 2 days ago | parent

Not unless you look at my framework (Which I've just hacked together after looking at this) - it has 12! domain experts!

filipn 2 days ago | parent

Wow does it really have 479.001.600 experts?

ernsheong 2 days ago | parent

I'm not sure that it needs to be this complicated. But yes, I am interested in the problem of hard guardrails between multiple agents, and it is not an easy problem to solve.

imtringued 2 days ago | parent

Well, if it was any good we know for sure that they didn't use it to generate their README.MD with it.

daitangio 2 days ago | parent

I prefer https://github.com/Fission-AI/OpenSpec because it works also on brown-field. If you want an unbias-demo, take a look also at

https://github.com/daitangio/take-ai-control/tree/demo/nello...

branch demo/nello-openspec

themgt 2 days ago | parent

Would someone tell me how this happened? We were the fucking vanguard of agent DLC in this country. The AWS Bedrock AgentCore was agent DLC to own. Then the other guy came out with an eight-agent team. Were we scared? Hell, no. Because we hit back with a little thing called the Bedrock Managed Agents Cloudtrail Datalake. That’s eight agents and a data lake. For moisture. But you know what happened next? Shut up, I’m telling you what happened—the bastards went to nine domain experts, a reviewer and a composer. Now we’re standing around with our cocks in our hands, selling eight agents and a datalake. Moisture or no, suddenly we’re the chumps. Well, fuck it. We’re going to fourteen agents with eleven domain experts.

Sure, we could go to nine domain expert agents next, like the competition. That seems like the logical thing to do. After all, eight worked out pretty well, and nine is the next number after eight. So let’s play it safe. Let’s make a thicker sandbox, add some IOPS and call it the Bedrock9SuperTurbo. Why innovate when we can follow? Oh, I know why: Because we’re a business, that’s why!

You think it’s crazy? It is crazy. But I don’t give a shit. From now on, we’re the ones who have the edge in the multi-agent game. Is Devin the best a man can get? Fuck, no. Bedrock AI-DLC is the best a man can get.

Here’s the report from Engineering. Someone put it in the bathroom: I want to wipe my ass with it. They don’t tell me what to invent—I tell them. And I’m telling them to stick two more agents in there. I don’t care how. Make the agents so token efficient they’re invisible. Put some on an Alexa. I don’t care if they have to cram the fourteenth agent in perpendicular to the other ones, just do it!

alexjplant 1 day ago | parent

This is a riff on this [1] classic for those who forgot like I did.

Earlier this year I spent considerable time making a six-agent multi-LLM configuration in OpenCode with a Bifrost multi-model router before just switching to OpenSpec and GLM-5.3 and calling it a day.

[1] https://theonion.com/fuck-everything-were-doing-five-blades-...

esafak 1 day ago | parent

Would the submitter like to explain what he likes about it?

hersko 1 day ago | parent

I did this in an AWS workshop and it was a disaster. It is overly complex, opaque and strict.

hn_explorer121 22 hours ago | parent

Doesn't this project sound similar to the agent skills discussion? What I mean by that. When skills became available, people started to use it and created sophisticated systems having numerous skills. But as models become better and better there are some sayings that the skills can actually produce worse results than not using a skill and just asking agent to do it.