# Inference Engineering — Chapter 02: Inference (book Chapter 0)
- **Date Created**: 2026-10-06
- **Status**: In Progress
- **Reading Span**: PDF pages 17–24

---

## 1. Pre-Reading Briefing
- **Core Question**: Why is serving generative AI models so much harder than serving classic ML, and what layers does a complete inference system need?
- **Key Points to Watch For**:
  - **Two phases, two eras**: how the author splits a model's lifecycle, and why he says inference for classic ML (e.g., XGBoost on CPUs) was easy while generative AI is not.
  - **Three layers**: names the three layers of a complete inference system and the single job of each. Notice the claim that one layer alone is "not enough," and why.
  - **The runtime toolkit**: a list of six model-performance techniques, each defined in one line. Don't master them yet (Chapter 7 / book Ch 5 covers them). For each, note just *what it trades or reuses*. Two terms will be new: **KV cache** and **prefill vs. decode**.
  - **Scale changes the problem**: infrastructure problems shift as you grow. Note the stages and the rough GPU count where the shift happens.
  - **Developer-experience spectrum**: two extremes and where the author says the right answer sits. Ask how this connects to the Ch1 thread "when is self-hosting wrong?"
- **Context & Thread from Prior Chapters**: The Preface claimed inference engineering makes models faster, cheaper and more reliable. This chapter is the first look at *how*. Map each layer to the Preface's three dimensions (latency, availability, cost): which layer serves which? Also note the book's stated knowledge cutoff (Jan 2026) and what the author says will last despite it.

---

## 2. Reading Review & Reflections
- **Prompt Questions**: Pending reading.
- **User Key Takeaways**: Pending.
- **Scaffolding & Feedback**: Pending.

---

## 3. Chapter Synthesis
- **Core Thesis**: Pending.
- **Key Concepts / Mental Models**: Pending.
- **Notable Arguments & Evidence**: Pending.
- **Updates to Prior Understanding**: Pending.
- **Weekly Action Item**: Pending.