KV Cache Economics at Scale

How context length, batch size, and GQA interact to determine whether your cluster is weight-bound or cache-bound.

Full article coming soon.