An open-source toolkit that turns a frontier teacher LLM into a 20–50M parameter model — a Spirit — that runs on CPU, edge, or browser. No API at inference. Pure model.
Production AI pipelines often chain six frontier LLM calls when only one of them needs to be frontier. Distillarium is a toolkit, a vocabulary, and a working reference Spirit for replacing the other five with tiny task-specific models you actually own.
Distillarium is three things in one repo.
Three commands — distill, taste, bottle — that
take a YAML recipe, a teacher API key, and 30 minutes, and hand you back
a deployable model.
Distillation has needed names for these concepts for years. We borrowed them from the people who've been doing this for centuries. Mash is your seed corpus. The Still is the training run. Cuts are your data splits. Proof is held-out accuracy. Tasting Notes are the eval report — strengths, weaknesses, and failure cases, auto-generated, no hiding.
Needle is a 20.7M-parameter attention-only transformer that does function calling. It was distilled from Gemini 2.5 Flash in 27 minutes for $0.30 on a single RTX 5090. It runs on CPU at ~45ms median latency. It is an alpha. The Tasting Notes say so honestly.
Write a recipe. Pour a mash. Run the still. Taste the cuts. Bottle what's good.
A YAML file naming your teacher, your task, and your student architecture.
Distillarium prompts the teacher to generate training data from a seed corpus.
The student trains on the teacher's outputs. Heads and tails are discarded; hearts are kept.
Honest, auto-generated eval on held-out cuts. You see exactly what your Spirit can and can't do.
Export to ONNX, GGUF, or browser-WASM. Run anywhere.
# Distill — uses GOOGLE_API_KEY from env distillery distill recipes/needle.tool-calling.yaml # Taste — auto-generates Tasting Notes against held-out cuts distillery taste spirits/needle.pt # Bottle — export to ONNX, GGUF, or WASM for deployment distillery bottle spirits/needle.pt --format onnx
Most ML libraries report a single accuracy number and call it done. We auto-generate a full Tasting Notes report for every Spirit: tool-name accuracy, argument-key F1, exact-call accuracy, failure modes, retention vs. teacher (v0.2), and BFCL alignment for function-calling Spirits (v0.2).
The honesty is the brand. If a Spirit is 78% on tool names and 3% on exact calls, the Tasting Notes say so on the first line. You shouldn't have to read the source to find the weakness.
A library of Spirits — what's bottled, what's aging, what's in the recipe book.
Four exhibits in the gallery — concrete scenarios where the technique earns its keep.
A customer chat front-end needs to classify "are you trying to buy, complain, or ask a question?" before routing to the right pipeline. A frontier LLM round-trip adds 800ms and $0.002 per call. An 8M Spirit (see Routor, currently aging) does it in 12ms for free, locally.
Healthcare and legal apps need to strip PHI/PII from user input before anything reaches a hosted model. A 14M Spirit runs server-side at the trust boundary and never lets raw data leave — your compliance team gets an audit trail of a model you can actually inspect, not an API call to somebody else's box.
Your agent has 47 tools available. You don't need GPT-4 to pick which one to call — you need a 20M model that's seen the schemas and knows the patterns. Needle is the reference for this exact case: distilled from Gemini Flash on 15 categories, 78% tool-name accuracy, sub-50ms on CPU.
The Recipe is sealed — same teacher, same seed, same student arch equals the same Spirit. You can fork a public recipe, swap the tool catalog, and re-distill on your own data without ever calling a hosted API again at inference time.
Distillarium sits in a specific gap in the existing distillation ecosystem. For breadth and SOTA function-calling, the alternatives below are excellent. For replacing a narrow LLM step with a CPU-runnable, audit-able artifact — that's our lane.
.onnx / .gguf / .wasm as the output, not an afterthought).Every bottled Spirit is a single self-contained artifact. Hover the parts to see what each one is in ML terms.
Recipe. Sealed, signed config that produced this Spirit. You can re-distill from it byte-for-byte.inference shape. Tokenizer, vocab size, max sequence length.bottling format. PyTorch, ONNX, or GGUF wrapper around the weights.trained weights. The actual model parameters. Darker = higher proof.Tasting Notes. Name, vintage, batch, headline proof. Auto-generated, honest.Every term means something concrete. Each maps to an ML concept — and that's deliberate.
Distillarium is an exhibit from Automate Capture — a creative-technical research studio. Built by Andrew Young. Distilled from candidates surfaced by Automate Capture's own Research Radar pipeline.
The CLI is free, runs locally, uses your own teacher API key, and produces models you own. Nothing reports back to us. The "Cellar" is a public showcase you can opt into. MIT licensed.