Real bench on your Mac: llama-bench -m ~/.xariv/relay/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -p 128 -n 64 vs GGML_METAL_FUSION_DISABLE=1. Relay deploy
XARIV Fusion Studio
One decode step. Hundreds of kernels vs a dozen fused blocks.
Pick a cached Llama 3.2 1B GGUF and replay how llama.cpp Metal fusion collapses micro-ops into fewer GPU launches — the same idea as our kernel fusion study, scaled down for edge inference.
Illustrative replay (not a live Metal trace). Timings are scaled from local llama-bench on Apple Silicon with fusion on vs GGML_METAL_FUSION_DISABLE=1. Real weights live in ~/.xariv/relay/models/.
Left · unfused
Fusion disabled
GGML_METAL_FUSION_DISABLE=1 — every micro-op launches separately
Right · fused
Metal fusion ON
Default llama.cpp — fused RMSNorm·RoPE·QKV, attn epilogues
Fusion disabled
432 launches · target 17.4 ms / step
Launched
— / 432
Launch overhead
—
Step time
—
tok/s
57.6
Kernel timeline
Metal fusion ON
176 launches · target 15.5 ms / step
Launched
— / 176
Launch overhead
—
Step time
—
tok/s
64.6
Kernel timeline