Building an LLM Inference Engine with AI: The Code Was the Easy Part
What continuous batching, KV-cache ownership, speculative decoding, and performance regressions taught me about AI-assisted systems engineering.
Oct 1, 202619 min read48
Command Palette
Search for a command to run...