Skip to content

perf(linear_attention): halve the prefill chunk and split the scan over dv (-25% op, -1.07 s TTFT, 129 -> 49 MiB scratch) - #627

Draft
BoarQing wants to merge 6 commits into
mainfrom
perf/la-prefill-chunk16-dvsplit
Draft

perf(linear_attention): halve the prefill chunk and split the scan over dv (-25% op, -1.07 s TTFT, 129 -> 49 MiB scratch)#627
BoarQing wants to merge 6 commits into
mainfrom
perf/la-prefill-chunk16-dvsplit

perf(linear_attention): halve the prefill chunk and split the scan ov…

072ce17
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar