YHK - yet another hacker news clone

RTK reports token savings, but our cost benchmarks disagree

30 points michalwarda 1 hour ago 12 comments

vrighter 56 minutes ago | parent

well yeah.... now you're giving it output it wasn't trained on.

nextaccountic 13 minutes ago | parent

What if the next-gen models are trained on RTK output as well? Then you will actually have less tokens in the context window, and the model won't become confused (which would require more turns, wasting tokens)

aeneas_ory 55 minutes ago | parent

All of these "hacks" are snakeoil and I think deep down we all know. Whether it's caveman, RTK, or whatever other vibe-coded productivity/token cost saving hacks/skills/claude.md.

What I had success with (although benchmarks are older) is to index the codebase with a dedicated local code embedding model. It's a bit expensive on the CPU side but in my benchmarks it reduced token use and wall clock time significantly. Of course, it's always dependent on statistical noise + host system load, and running sufficiently large benchmarks is simply too expensive, so take em with a grain of salt.

Why does it work you may ask? Well, LLMs basically brute force words/phrases and pipe that into find/grep/pgrep/whatever (or as recently discussed here write a python script for it - https://news.ycombinator.com/item?id=49654229). Semantic search looks for similarities so you have to do less brute forcing. Comes of course at the cost of indexing everything first.

You can find the project here: https://github.com/ory/lumen

sreekanth850 52 minutes ago | parent

i don't know if such hacks works, but in C# if you use roslyn mcp, you save a lot.

VulgarExigency 22 minutes ago | parent

I don't think they're comparable. RTK just modifies the output of CLI tools to reduce the number of tokens, a Roslyn MCP gives the agent a fundamentally superior way of interacting with a C# codebase.

sreekanth850 20 minutes ago | parent

Yes. and i find model makes less errors and reasoning the codebase well, especially when you do a large refactor.

antupis 13 minutes ago | parent

My main issue with rtk is that rtk somehow messes modification and agent start polling same tool continuously.

fwlr 45 minutes ago | parent

This makes sense. “Don’t try to penny-pinch your employees” is a lesson most managers learn eventually, and I guess agent-orchestrators will have to learn it too.

gillesjacobs 30 minutes ago | parent

Main takeaway:

``` Average cost per attempt, without → with RTK:

Claude/Fable: $1.72 → $1.64 (~5% cheaper) DeepSeek: $0.115 → $0.121 (~5% more expensive)

Almost all Claude savings came from a single task. Excluding it, savings were under 1%. ```

It took me a few rereads to parse out the top-line. This article really buries the lede.

fleetfox 14 minutes ago | parent

I don't understand how this or all these magic skill bundles and methodologies get traction and why they are so popular. It's either plain worse or has serious trade offs.

semiquaver 7 minutes ago | parent

Just another instance of the bitter lesson. The model itself knows how to be clever and conserve tokens in command output by using shell primitives and as the models get smarter they get better at anticipating large output and defensively adapting the input commands.

hokkos 3 minutes ago | parent

If you are using maven you should tell your agent to use its quiet mode or rtk, because mvn love to write a lot of useless output.