21 points eiln 2 days ago 2 comments
eiln 2 days ago | parent
RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s.
Neywiny 23 minutes ago | parent
Just checking here- this systemverilog is a hypothetical telling of what you think is going on? Or do you have the actual source of the RTL?