XARIV Relay · Compare
Llama 3.2 1B — F16 vs Q4. This page replays a typical Mac Metal timing profile so you can show the speed gap without downloading weights on Vercel.
Illustrative replay (not live GPU inference in this browser). Edit the prompt below — both panes still stream the same canned answer so the speed gap stays clear. Real models cache in ~/.xariv/relay/models/.
Left · baseline
Llama 3.2 1B Instruct · F16 (non-quantized)
localhost:8090
Right · quantized
Llama 3.2 1B Instruct · Q4_K_M (quantized)
localhost:8091
Llama 3.2 1B Instruct · F16 (non-quantized)
localhost:8090
TTFT
—
tok/s
—
est. tokens
—
Llama 3.2 1B Instruct · Q4_K_M (quantized)
localhost:8091
TTFT
—
tok/s
—
est. tokens
—