Index

# 3. ML Inference & Serving

Why This Matters¶

Training a model is half the battle. Serving it at low latency and high throughput to millions of users is the other half. This section covers the systems that make LLM inference fast and cost-effective.

Topics¶

• vLLM & Serving Engines
• KV Cache & Memory Management
• Quantization
• Batching Strategies