System design · Free PDF
AI System Design
A framework for the design round and worked cases: support bot, document search, coding agent and more.
System Design for LLM Inference
Serving models at scale: batching, caching, GPU capacity, streaming and cost per request.
Free PDF · sent by email the day it’s ready
Who it’s for
For infrastructure and ML platform engineers asked how they would serve a model to many users.
Expect numbers: tokens per second, memory per request and what a request costs.
What it covers
- Batching
- Caching
- GPU capacity
- Cost per request
What the guide will include
A framework for the round
Requirements, architecture, tradeoffs, evaluation and failure modes, in the order you say them.
Worked designs
Prompts worked end to end the way you would answer them at the whiteboard, with the diagram.
Tradeoffs to name
The choices interviewers push on, and what each option costs in latency, money and risk.
Failure modes and evals
How the system breaks in production, how you would notice, and what you would measure.
A PDF you can print
Read it on your phone before the call, or print the sections you want to drill.
Planned contents
The guide isn’t out yet, so chapters and their order may still change.
- 1. How the design round runs
- 2. Batching
- 3. Caching
- 4. GPU capacity
- 5. Cost per request
- 6. Practice prompts
Free PDF
Get the LLM Inference PDF free
Serving models at scale: batching, caching, GPUs, streaming and cost per request.
Questions about the guide
Is the PDF really free?
Yes. Leave your email and we send the PDF the day it is ready. No payment details, and every email has a one-click unsubscribe link.
Is it based on a specific company’s interviews?
No. It is written from the concepts the role or system depends on. We don’t name companies or claim to know what any one company asks.
When will it be ready?
It isn’t out yet. Leave your email and we send it the day it is; nothing else arrives before then.
Related guides
Role guide · Free PDF
MLOps Engineer
Deploying and serving models, pipelines, monitoring, drift and rollbacks.
Role guide · Free PDF
Generative AI Engineer
How LLMs work, prompting, fine-tuning vs RAG, structured outputs, cost and latency.