#ai

Articles tagged with #ai

When Does Heterogeneous Inference Pay Off?
How hardware fit, communication costs, and production traffic shape prefill/decode, attention/FFN, and target/draft disaggregation.
Oct 8, 202634 min read37
Building an LLM Inference Engine with AI: The Code Was the Easy Part
What continuous batching, KV-cache ownership, speculative decoding, and performance regressions taught me about AI-assisted systems engineering.
Oct 1, 202619 min read66