Kentucky AI Shelbyville · Kentucky
← Read the full field note
Visual field map · educational

Distillation & Fine-Tuning, drawn

Eleven frames on putting a frontier model's judgment inside a small model you run on your own Mac — distillation, LoRA, on-policy training, quantization, and evaluation — each mechanism drawn against a real Division 9 takeoff.

Kentucky AI · companion to Forging Local Models · method after Song & Zheng, 2026 · arXiv:2604.00626

  1. Slide 1 of 11. Distillation & fine-tuning: a frontier teacher network in the cloud is distilled, with a tune-config panel (temperature, KD weight, rank, learning rate), into a 4-bit student that reads a floor plan locally.
    01Distillation & fine-tuning — teach a small model you own to read finish plans like a frontier model would.
  2. Slide 2 of 11. Frontier judgment, run locally: a rented cloud API beside a local model stack — the drawings never leave the office.
    02Frontier judgment, run locally — the drawings never leave the office.
  3. Slide 3 of 11. Distill, then adapt: teacher and student networks plus the LoRA decomposition W prime equals W plus B times A, with dimension labels.
    03Distill, then adapt — a LoRA adapter (W′ = W + B·A) learns your callout shorthand.
  4. Slide 4 of 11. Match the softened posterior: a temperature-softened softmax compares the teacher and student distributions over finish types.
    04Match the softened posterior — temperature spreads confidence across look-alike finishes.
  5. Slide 5 of 11. Train on the student's own rollouts: an on-policy loop — sample, score, reverse-KL — with an off-policy inset; a misread is caught on the student's own output.
    05Train on the student's own rollouts — on-policy: sample → score → reverse-KL.
  6. Slide 6 of 11. Weights for pixels, retrieval for rules: a vision-language model reads a floor plan beside a typed knowledge graph — CPT-1 read off the sheet, its meaning from the schedule.
    06Weights for pixels, retrieval for rules — read CPT-1 off the sheet; get its meaning from the graph.
  7. Slide 7 of 11. Capacity is a precondition: an emergence curve of accuracy versus parameter count, with a task threshold; a small base drops a cross-sheet callout.
    07Capacity is a precondition — below the threshold, a small base silently drops a callout.
  8. Slide 8 of 11. Freeze the backbone, inject low-rank: a transformer stack with LoRA on q_proj and v_proj and swappable adapters — one adapter per trade.
    08Freeze the backbone, inject low-rank — LoRA on q_proj / v_proj; one adapter per trade.
  9. Slide 9 of 11. Quantize to 4-bit, fit unified memory: an FP16 to INT4 bit layout beside a to-scale 128 GB memory bar holding a 30B reasoner next to the estimating workstation.
    09Quantize to 4-bit, fit unified memory — FP16 → INT4 puts a 30B reasoner in the 128 GB envelope.
  10. Slide 10 of 11. Evaluate on the as-built distribution: covariate-shift curves and an estimated-versus-installed metrics table — closeout quantities against the estimate.
    10Evaluate on the as-built distribution — score estimated-vs-installed against closeout, not a held-out split.
  11. Slide 11 of 11. Kentucky AI sign-off: the wordmark and wreath.
    11Kentucky AI — we ship weights, not decks.
01 / 11

Swipe, use ← →, or tap a dot