383 points super_linear 2 hours ago 119 comments
super_linear 2 hours ago | parent
singron 6 minutes ago | parent
I'm curious how many versions they made and the token spend on each one. There are a lot of failure modes to projects like these (e.g. gas town endlessly developing itself). Is this more of a sly marketing stunt or a real indication that this methodology is practical and economical?
FYI I checked out the repo. `du --apparent-size -h` output for some dirs:
2.2G .
1.3G .git
58M issues/things-that-do-not-work
74M issues/things-that-now-work
311 MiB of rust code in /tests/ directories. 357 MiB of rust code outside of /tests/ directories. 117 MiB of json. 1 MiB of toml.CharlieDigital 2 hours ago | parent
Floci[0] has been great for me. Use the Testcontainers[1], too.
Bonus: works for GCP and Azure, too (so far have only used in limited capacity, though).
ke4qqq 2 hours ago | parent
dogleash 2 hours ago | parent
Official statements made by a company are still poor representations of their current business plans, and don't mean anything 3 months later (no matter how much they pinky promise). But a HN post isn't even that.
Cthulhu_ 1 hour ago | parent
forgotpwd16 1 hour ago | parent
CharlieDigital 41 minutes ago | parent
This feels like table stakes for an emulator.
ke4qqq 1 hour ago | parent
That said, it’s been announced publicly here: https://www.aboutamazon.com/news/aws/how-autonomous-ai-agent...
Additionally Swami, our SVP of AI blogged about it separately and our CTO is tweeting about it - which is more than a number of products get in terms of coverage.
dogleash 1 hour ago | parent
ke4qqq 1 hour ago | parent
Eridrus 48 minutes ago | parent
I think this is a super interesting direction, but how can we trust the accuracy of this simulator? Agentic loops are great and all, but they can still miss plenty of things.
Or maybe a better question: where is the accuracy of this simulator going to be guaranteed enough that we should trust it?
NewsaHackO 39 minutes ago | parent
Eridrus 26 minutes ago | parent
Like, what am I actually getting here? Is it just happy path operations being correct here?
iceboundrock 26 minutes ago | parent
toomuchtodo 2 hours ago | parent
polishdude20 1 hour ago | parent
xandrius 1 hour ago | parent
CharlieDigital 16 minutes ago | parent
The Copilot CLI https://aws.github.io/copilot-cli/ you can feel polish and pride of the team that built this, as an example.
dotancohen 1 hour ago | parent
jghn 1 hour ago | parent
Google builds up a product over time and abandons it. This is annoying for long term adopters.
AWS has a habit of throwing a bunch of products at the wall, but only bothering to maintain & build up a few of them. This is annoying for early adopters.
llama052 1 hour ago | parent
drob518 1 hour ago | parent
gwbas1c 1 hour ago | parent
jasonseattle 57 minutes ago | parent
CharlieDigital 53 minutes ago | parent
QGQBGdeZREunxLe 1 hour ago | parent
switchbak 1 hour ago | parent
aseipp 1 hour ago | parent
sequence7 1 hour ago | parent
jasonseattle 1 hour ago | parent
jbmchuck 55 minutes ago | parent
OptionOfT 14 minutes ago | parent
They'll have something like this:
MySchema:
type: object
properties:
enum_member: { enum: [foo, bar] }
enum_member_2: { enum: [quz, qux] }
if: { properties: { enum_member: { const: bar } } }
then: { required: [enum_member_2] }
Except, this is not defined in schemas, this is merely defined in prose around the documentation.So your Rust API is full of Option, and it gets really annoying when you're testing things like say Bedrock, which takes 10 minutes, and then fails because what you submitted 10 minutes before doesn't match their schema.
I understand this can be fixed by reading the documentation in more detail, but it has gotten so verbose, and sometimes you just want to start something.
jcims 52 minutes ago | parent
frenchtoast8 19 minutes ago | parent
icedchai 45 minutes ago | parent
tyingq 42 minutes ago | parent
Edit: The current one: https://github.com/aws/aws-emulator/releases/download/v1.0.3...
You can get some idea if the product name is unique enough with <ctrl-f> and looking at each of the related test scores.
tyingq 8 minutes ago | parent
And output something meaningful for some of the more popular services:
Service Tests All Happy Reject Ctrl Data Solo Integ Wire# Wire%
----------------- ----- --- ----- ------ ---- ---- ---- ----- ----- -----
Data Firehose 288 77% 85% 91% 92% 72% 89% 59% 24 62%
SES 132 72% 65% 79% 79% 42% 79% 0% 21 52%
EventBridge 889 71% 74% 64% 72% 68% 83% 67% 75 84%
KMS 432 70% 82% 69% 81% 68% 97% 36% 69 84%
Step Functions 907 70% 79% 53% 69% 67% 73% 65% 68 76%
DynamoDB 1437 69% 70% 61% 62% 65% 82% 53% 243 62%
SNS 594 69% 72% 55% 62% 68% 75% 65% 39 69%
SQS 1480 68% 70% 67% 61% 68% 94% 62% 29 45%
Secrets Manager 414 68% 78% 65% 66% 72% 79% 57% 26 77%
Cognito UserPools 781 65% 67% 66% 82% 36% 75% 33% 193 64%
S3 2488 62% 62% 57% 61% 57% 83% 51% 296 68%
CloudFront 627 62% 62% 63% 70% 46% 71% 41% 190 22%
API Gateway 1009 61% 64% 59% 77% 41% 75% 43% 200 73%
SES v2 210 61% 63% 59% 75% 48% 77% 40% 44 30%
Route 53 130 61% 63% 59% 62% 53% 69% 0% 49 67%
CloudWatch 1186 60% 66% 51% 66% 53% 77% 36% 152 55%
ECR 392 60% 55% 75% 67% 49% 87% 27% 74 65%
Kinesis 380 59% 69% 44% 63% 50% 62% 54% 25 44%
IAM 991 58% 63% 50% 60% 34% 78% 41% 228 57%
Batch 274 58% 62% 59% 78% 20% 65% 17% 19 47%
CloudWatch Logs 1016 58% 61% 54% 71% 47% 78% 41% 185 59%
OpenSearch 342 57% 62% 52% 70% 29% 59% 50% 98 48%
API Gateway v2 292 57% 64% 52% 74% 38% 73% 33% 107 27%
EKS 349 57% 60% 52% 65% 10% 59% 38% 55 62%
Glue 646 54% 51% 55% 66% 32% 69% 40% 155 43%
Lambda 2808 53% 53% 42% 58% 48% 48% 54% 171 42%
ECS 808 51% 45% 52% 62% 23% 62% 34% 118 44%
Redshift 241 49% 57% 47% 53% 52% 47% 57% 50 46%
STS 218 49% 58% 46% 56% 30% 57% 34% 22 50%
ELBv2 596 45% 43% 47% 55% 19% 58% 30% 111 63%
RDS 659 43% 54% 45% 54% 35% 51% 31% 99 28%
ElastiCache 186 42% 43% 49% 54% 0% 47% 17% 27 41%
EC2 2043 40% 35% 36% 39% 5% 44% 32% 429 33%
CloudFormation 2019 36% 33% 31% 30% 40% 78% 24% 2161 3%
AppSync 419 34% 35% 35% 54% 1% 44% 0% 91 13%
SSM 566 28% 28% 27% 27% 25% 26% 30% 175 11%
Athena 420 22% 20% 15% 7% 28% 16% 37% 37 0%
EC2 Auto Scaling 440 21% 19% 23% 19% 18% 21% 20% 89 16%
Pass rates from the release's scores.json (commit 7753983), joined with the eval suite's manifest.jsonl. Tests = behavioral checks touching the service. All = overall pass rate. Happy/Reject = valid requests vs. requests AWS should refuse. Ctrl/Data = control plane vs. data plane. Solo/Integ = single-service vs. multi-service checks. Wire# / Wire% = count and pass rate of recorded request/response replay tests (a test passes only if every call matches).dataviz1000 36 minutes ago | parent
switchbak 1 hour ago | parent
I didn't have a good experience with test containers, but perhaps it'd work better with fast starting containers. We run our test fixture with compose, and have the tests manage their data in isolation - so we don't pay the startup penalty in our test suites.
I think I'd trust this project at least as much as Floci. Embedded Rust mode also looks very interesting for us.
CharlieDigital 45 minutes ago | parent
iLoveOncall 1 hour ago | parent
CharlieDigital 51 minutes ago | parent
> How we built the agents that built AWS Emulator
> Thousands of autonomous AI agents wrote every line of code in AWS Emulator, a new local emulator for AWS now available for download.
Feels more like a science experiment than a product from that announcement.Doesn't feel like a real product. But I also didn't look very hard for docs; I kind of expect it to be linked on the GH in the site and have a nice `llms.txt` for me to give to my agents.
iLoveOncall 42 minutes ago | parent
??? It's in bold just before the "Quick start" section
CharlieDigital 39 minutes ago | parent
The team didn't even take the time to review these small details of the repo, you know, by editing the repository details to place the docs URL there.
I mean, I don't know about you, but I feel like I still like to see people care about the stuff they produce and missing something as basic as the repo website URL in GH settings shows a lack of polish and love.
simgoh 38 minutes ago | parent
> Doesn't feel like a real product. But I also didn't look very hard for docs
You should try, they're there.
> I kind of expect it to be linked on the GH in the site and have a nice `llms.txt` for me to give to my agents.
There's an official Amazon Documentation site that's linked. There's also an `AGENTS.md` file in the repository.
CharlieDigital 34 minutes ago | parent
This post https://www.aboutamazon.com/news/aws/how-autonomous-ai-agent... doesn't set the right tone for me at all and it has nothing to do with agents and everything to do with the lens of looking at this as a science experiment instead of a real enterprise tool.
simgoh 30 minutes ago | parent
This feels like such a weird nit-pick when AWS is criticized for not moving quickly enough. Why should I care if there's a link in the side bar, when there's plenty of links in the `README`? If anything, I'm happy that there's alternatives to Localstack that are supported by AWS themselves that I can begin incorporating. They have a well written `README` that I can start working with and a User Guide to get me started. What else could I want?
> Did you look at the blog post as well?
I did.
> Does it look like a product that some team there loved and will care for? Or the output of some exercise for some enterprise content?
Do you even have the grounds to cast judgement like this when you admittedly didn't even look hard enough?
CharlieDigital 20 minutes ago | parent
This lacks polish.
Small details like getting your GH repo set up for a public audience. Lacking. Compatibility and support matrix clearly advertised? None.
If this public facing artifact exhibits a lack of care and love, what does it say about the code? I am all for vibe coding and agent software factories, but then the humans should show some love with all of that extra time saved.
simgoh 17 minutes ago | parent
_msw_ 18 minutes ago | parent
The product documentation is here: https://docs.aws.amazon.com/emulator/latest/userguide/what-i...
While the GitHub project is young, the internal effort to build this has been going for quite some time now.
sebmellen 2 hours ago | parent
tonymet 2 hours ago | parent
Cthulhu_ 1 hour ago | parent
https://www.aboutamazon.com/news/aws/how-autonomous-ai-agent...
tonymet 1 hour ago | parent
I've been prompting the agents to build everything using stdlib
jjice 54 minutes ago | parent
I'll continue to choose both though, depending on the thing being tested.
gateauxd 2 hours ago | parent
aabhay 1 hour ago | parent
sequence7 1 hour ago | parent
https://blog.localstack.cloud/2026-upcoming-pricing-changes/
schonfinkel 18 minutes ago | parent
swiftcoder 1 hour ago | parent
forgotpwd16 1 hour ago | parent
cxr 1 hour ago | parent
mh- 1 hour ago | parent
paavohtl 1 hour ago | parent
https://github.com/search?q=repo%3Aaws%2Faws-emulator+load-b...
https://github.com/search?q=repo%3Aaws%2Faws-emulator+%2Fno+...
https://github.com/search?q=repo%3Aaws%2Faws-emulator+golden...
mystifyingpoi 1 hour ago | parent
switchbak 1 hour ago | parent
whalesalad 1 hour ago | parent
morelandjs 1 hour ago | parent
Agentic development is all about closing off a system as much possible, then mocking up or simulating any inputs or dependencies that cannot be closed off. At that point, you let Claude crawl over the system and do its thing.
This inevitably runs into problems with infrastructure and any tools that help mock up those interfaces will be tremendously valuable. Thank you!
gosen 1 hour ago | parent
ggcr 1 hour ago | parent
[0] https://www.aboutamazon.com/news/aws/how-autonomous-ai-agent...
varispeed 1 hour ago | parent
ChrisArchitect 1 hour ago | parent
Waterluvian 1 hour ago | parent
I'm curious just how realistic the "internal" behaviour is emulated. It's been a while (since I got my stack working and I just don't like touching it now...), but I recall it being a real pain that sometimes odd, sometimes inconsistent things happened with setup and teardown (even through Terraform), which led to long timeouts if you did things in the wrong (but seemingly valid) order.
billyp-rva 48 minutes ago | parent
It seems like some reasonable shortcuts are taken, like services reading from each other's states directly instead of using APIs [0]. But then it will go so far as to run an in-process NodeJS emulator for lambda functions, and give it a shim so the code it runs can call other emulated AWS services using the main router [1].
[0] https://app.ilograph.com/demo.ilograph.AWS%2520Emulator/Serv... [1] https://app.ilograph.com/demo.ilograph.AWS%2520Emulator/Invo...
andreadito 1 hour ago | parent
__natty__ 1 hour ago | parent
microflash 1 hour ago | parent
They can't help themselves, can they?
I'll stick with Floci and Testcontainers.
toenail 1 hour ago | parent
jbmchuck 59 minutes ago | parent
schonfinkel 57 minutes ago | parent
https://github.com/faiscadev/fakecloud
I had a good time using its SDK inside my integration tests. I'll give a couple months before even moving to this because it's weird how similar both projects look like.
EDIT: After checking AWS's project I'd say stay away from it, looks heavily vibe slopped.
Bnjoroge 7 minutes ago | parent
InsidiousWombat 55 minutes ago | parent
fschuett 47 minutes ago | parent
dboreham 43 minutes ago | parent
hollowturtle 28 minutes ago | parent
nunez 24 minutes ago | parent
ironically localstack is solid enough to probably not need this for most things
OJFord 24 minutes ago | parent
> AI agents keep the emulator up to date with AWS.
Is pretty interesting! No more detail than that tease though.
https://docs.aws.amazon.com/emulator/latest/userguide/what-i...
asdfman123 24 minutes ago | parent
I was able to create a TV emulator at my work with a mostly independent agentic loop, because we have a lot of tests to examine a smart TVs APIs. The agents mostly just need to make the tests pass.