Your agent pipeline is 6 frontier LLM calls. Most of them don't need to be.

The pipeline replacement story

Here's a typical customer-support pipeline. Every box is a separate call to Claude / GPT / Gemini.

Before — every step is a frontier call

user msg ──→ ┌────────────────────┐
             │  intent classifier │ ─→ $0.005 / 400 ms   (Claude/GPT)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │  PII redactor      │ ─→ $0.005 / 400 ms   (Claude/GPT)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │  urgency scorer    │ ─→ $0.005 / 400 ms   (Claude/GPT)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │  agent router      │ ─→ $0.005 / 400 ms   (Claude/GPT)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │ response generator │ ─→ $0.020 / 1500 ms  (Claude/GPT)  ← actually hard
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │ brand/tone check   │ ─→ $0.005 / 400 ms   (Claude/GPT)
             └──────────┬─────────┘
                        ↓
                   response
            ━━━━━━━━━━━━━━━━━━━━━━━━━━
            $0.045  /  3,500 ms  /  6 vendor dependencies

After — distill the narrow tasks, keep frontier for the hard one

user msg ──→ ┌────────────────────┐
             │ 🍾 Spirit: intent   │ ─→ $0 / 30 ms        (8M, CPU, local)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │ 🍾 Spirit: PII      │ ─→ $0 / 30 ms        (14M, CPU, local)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │ 🍾 Spirit: urgency  │ ─→ $0 / 30 ms        (8M, CPU, local)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │ 🍾 Spirit: router   │ ─→ $0 / 50 ms        (20M, CPU, local)
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │  Frontier LLM      │ ─→ $0.020 / 1500 ms   ← keep ONE call for the hard step
             └──────────┬─────────┘
                        ↓
             ┌────────────────────┐
             │ 🍾 Spirit: tone     │ ─→ $0 / 30 ms        (10M, CPU, local)
             └──────────┬─────────┘
                        ↓
                   response
            ━━━━━━━━━━━━━━━━━━━━━━━━━━
            $0.020  /  1,670 ms  /  1 vendor dependency

The deltas, at scale

Cost numbers use Claude Sonnet 4.6 pricing as of 2026-05. Latency is end-to-end including network. Numbers shift with model choice — but the shape of the win holds.

Why this matters (the three walls)

Every team building with LLMs eventually hits the same three walls. The pipeline replacement above is what knocks each one down.

The three options, compared honestly

Distillation isn't always the answer. Here's where each one wins:

The honest tradeoff: distillation gives up generality for size, speed, and ownership. If your task is narrow and you do it a lot — that's a trade you'll happily make.

How is 1.5T → 20M even possible?

A frontier LLM is a Swiss Army knife with a thousand blades. Your tool-calling endpoint uses one of them.

Distillation works because your task doesn't need the other 999 blades. The teacher generates synthetic examples that pin down exactly the surface area you care about, and the student learns just that surface.

TEACHER (1.5T params)                  STUDENT (20M params)
┌──────────────────────────┐           ┌──────────────┐
│  reasoning · code · poem │           │              │
│  translation · math      │           │  tool-call   │
│  rag · summarize · chat  │  ──→──→   │  parsing     │
│  ▶ tool-call parsing ◀   │           │              │
│  audio · vision · …      │           │              │
└──────────────────────────┘           └──────────────┘
   "knows everything"                    "knows the part you need"
                                          ~72,000× smaller
                                          ~10,000× cheaper at inference

The compression ratio is a function of how narrow your task is. Wide tasks (general assistant): hard to distill, large student needed. Narrow tasks (intent classifier, NER, function-call parser): drop 4-5 orders of magnitude with minor quality loss.

When to distill — and when not to

The bet

The trajectory of frontier models is bigger, slower, more expensive, more political. The trajectory of efficient inference is smaller, faster, edge-runnable, owned.

Most production AI workloads are narrow tasks dressed up as general intelligence. Once you have a teacher that can do the task at all, the engineering problem is to compress it to where it costs nothing and runs everywhere.

That's the bet behind The Distillery. Not that frontier models stop mattering — they remain the source. But that the delivery vehicle for production AI is increasingly a tiny student trained on a tiny pile of teacher-generated examples — and that production pipelines stop being "frontier LLM all the way down" and start being "a constellation of cheap Spirits with one frontier call where the reasoning actually lives."