32 points ankitvgupta 1 hour ago 7 comments

datadrivenangel 41 minutes ago | parent

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

datadrivenangel 41 minutes ago | parent

it only cost them $10,000 in Claude API usage. :)

hmate9 36 minutes ago | parent

You get that much usage in a monthly $200 subscription.

dofm 14 minutes ago | parent

And a kitten.

risyachka 25 minutes ago | parent

Fpga is software-driven, model can verify its output easily.

Zero surprise it did it

codekansas 19 minutes ago | parent

This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

chalkycrimp 6 minutes ago | parent

Good chance I'm not the only one thinking this: share the code