49 points pythonic_hell 2 days ago 5 comments
iLoveOncall 1 hour ago | parent
> We propose intelligence per watt (IPW), task accuracy per unit of power
Stupid metric. It's not because a model is better performing that it necessarily requires more energy or compute.
utopiah 1 hour ago | parent
I don't think that's what they are saying. In fact if they did the metric would be pointless. Rather they are saying by estimating that value on different architectures, one can find more efficient ones. They use open model to be able to remove unknowns. They aren't advocating for one model or another, only more efficient architectures.
_diyar 1 hour ago | parent
> We propose miles per hour (MPH), distance travelled per unit of time
Stupid metric. It‘s not because you spend more time that you travel farther.
s/
api 28 minutes ago | parent
Unless I misread it, are they saying local GPUs use less energy?
That’s surprising, almost unbelievable, due to batching. Local is usually not batched.
stymaar 8 minutes ago | parent
Small models are much smaller than frontier models though, which is how they end up consuming less energy despite low batch count. (Though with local models growing strong agentic capabilities, batching becomes a reality with local models as well).