Blog
Shorter technical notes on inference serving, GPU optimization, and platform engineering.
AI InfrastructureDistributed InferenceGPU OptimizationTensorRTvLLMMoENetworkingBenchmarkingCapacity PlanningAgentic AI
Distributed Inference
Prefill, Decode, and the Two Regimes of LLM Serving
Why autoregressive serving has two distinct performance regimes — and why conflating them leads to wrong capacity plans.
·8 min
Capacity PlanningKV Cache Economics at Scale
How context length, batch size, and GQA interact to determine whether your cluster is weight-bound or cache-bound.
·10 min
TensorRTTensorRT-LLM vs vLLM: When Each Runtime Wins
A practical comparison for teams choosing a production inference stack — latency, throughput, and operational trade-offs.
·12 min
Agentic AIInfrastructure Patterns for Agentic AI Workloads
Multi-step tool use changes the request distribution. What that means for GPU sizing, caching, and tail latency.
·9 min