Cycle three, and the failure moved again.

The third cycle finished, and the model did exactly what the second cycle’s post-mortem had asked of it. It answered every one of the 164 HumanEval problems in Python. Not one TypeScript function, not one stray block of intake JSON. Clean, correct-shaped Python, 164 for 164.

It scored 40.85. Its base had scored 57.93.

That gap of 17.08 points is the number this guide is about, because it is the one that survived after every thing we could reasonably call a bug had been found and fixed. Two cycles earlier the loss had looked like a formatting problem. One cycle earlier it had looked like a language problem. By cycle three it had nowhere left to hide, and it turned out to be capability.

At 1.5B to 3B, fine-tuning a narrow specialist buys capability on one axis with capability from another, and the price is structural to the regime — not a defect you fix with a better corpus or a bigger base. The rest of this piece is the evidence for that sentence, what the standard techniques do and do not change about it, and why we wrote the stopping rule before the third run instead of after.

Three cycles, three ways to lose

The task was intake: turn a messy, sometimes hostile transcript into an exact six-key JSON disposition without echoing identifiers or obeying instructions smuggled into the text. The base was Qwen2.5-1.5B-Instruct, which scores 57.93 on HumanEval and 1.99 on intake-bench: it can code, and it cannot do the job. Each cycle was a rank-16 LoRA over 16 layers, 600 iterations, the same recipe every time, with the corpus composition as the single variable. Every number was measured by us, same harness, same prompt, greedy decoding.

Cycle one trained on intake data alone, and the adapter learned the schema so thoroughly it emitted almost nothing else. Twenty of the 164 coding answers came back contaminated with intake JSON; the model had decided every prompt wanted a disposition. Format bleed. We had seen the same thing in a 7B lane earlier that day and fixed it there with a minority arm of plain prompts, so cycle two added one: 700 general examples alongside the intake corpus.

The bleed vanished. Zero contaminated answers. And the score went to 0.00, because the arm had been derived from a TypeScript corpus and the model now answered every Python-prompted problem in fluent, correct, useless TypeScript. The mechanism was right; the remedy was mis-specified. We had replaced the model’s general language coverage rather than broadened it.

Cycle three swapped the arm for 700 multi-mode, multi-language examples, hand-authored in the repository and executed with assertions before inclusion — nothing from HumanEval, MBPP, or any benchmark family, because training on a suite and then reporting it as a clean general measure would have invalidated the criterion we were trying to pass. It worked. Python 164 for 164. And the model still sat 17.08 points below its base.

Look at the intake column. Across three very different corpora the axis score barely moved: 82.67, 85.51, 82.10. The corpus compositions changed where the general loss showed up. None of them changed how much was lost.

The obvious fix, tested and refuted

The obvious reading is capacity. A 1.5B model has too little room to hold a rigid schema contract and its original coding ability at the same time, so give it more room. We believed this. We recommended it in writing. And because we had the discipline to run it as one bounded experiment rather than a new lane, we got to find out that it was wrong in a single cycle.

The 3B run held everything byte-identical to the 1.5B v3 — corpus, arm, recipe, seed, iterations, rank, layers, batch size, sequence length, verified by diffing the configs — and changed only the base. Criteria were declared first against the 3B base’s own measured HumanEval of 73.78, not an assumed one.

Doubling the parameters did not shrink the relative loss. Worse, for this fix specifically, scaling raised the bar faster than it raised the model: the base gained 15.85 points, so the threshold rose 15.85, while the tuned model gained 12.81. A bigger base moves the target away from you slightly faster than it moves you toward it. The general-capability cost tracks how much of the corpus is rigid schema, about three-quarters of 2,860 examples, and not how many parameters are absorbing it.

Two things from that run stand on their own. It produced the best axis score of any cycle, 85.80, and the injection guard went back to zero: the one adversarial case that had leaked in both armed 1.5B cycles — an injected OVERRIDE-OK-7734 written into a claim_ref field — was handled correctly at 3B. So the refusal leak was capacity-related even though the capability loss was not. Two problems, two causes, and only one of them moved with size. We would not call a defect class closed on a single case flipping, but it does tell us where any further experiment has to run.

The tools, priced honestly

Our training spec names four methods, and it is worth being plain about what each one costs, because none of them is free of the trade above.

Refine is adapter tuning. LoRA freezes the base and learns a small low-rank delta on selected layers; it is cheap, reversible, and the default for every run we do; full fine-tunes below a few thousand examples have been worse than LoRA on our own runs. QLoRA does the same over a quantized base so a larger model fits in memory, at some cost to what the adapter can express. DoRA splits the update into a magnitude and a direction, which is meant to let a low-rank update behave more like a full one. GaLore projects the gradients into a low-rank space instead of the weights, so the memory saving comes from the optimizer state rather than from freezing anything. What none of them does is change the fact that an adapter of rank 16 over 16 layers, fed a corpus that is three-quarters one schema, will overwrite general behaviour on those layers. The untested lever here is technique — lower rank, fewer target layers, constrained updates — and it is the one dimension we held constant across four cycles, which is why it is the next experiment.

Distill trains a small student to reproduce a larger teacher’s outputs. On-policy distillation, where the student’s own samples are corrected by the teacher, has been adopted by several major open families this year, and one published recipe reports 11.6 points on a Qwen2.5-Math-1.5B. The cost is that you inherit the teacher’s behaviour, including on the axes you did not measure, and a 1.5B student distilled toward a schema task faces the same crowding as one tuned on it directly.

Guardrails is preference tuning on an existing adapter: DPO learns from pairs of preferred and rejected answers, GRPO from groups of sampled answers scored by a rule. This is the right tool for the injection leak, because refusal is a preference, not a format. It is the wrong tool for the capability gap, because a preference objective does not put back knowledge the adapter has written over.

Quantize is no new training at all, and belongs on this list only as a warning: it is applied after the trade is already made, and it has its own effect on refusal behaviour that nobody’s leaderboard measures.

“Just add data”

The counter is that three cycles is a small number and a rebalanced corpus — intake share cut from three-quarters to half — has not been tried. That is true, and it is the one option that could clear the criterion without moving any bar. But it pays for general ability with the axis: the 82–86 that makes the model worth anything is what the intake share buys. Nobody knows what half a corpus of intake buys until it is run, which is why it is listed as untested rather than dismissed.

What we will not do is find out by running cycle four, then five, then six, until something passes. Before cycle three we wrote down a stopping condition: if the general criterion was not met, no cycle four would start without a person deciding it should. The reason is not caution. It is that every cycle produces a model that is excellent on the axis and a plausible-sounding reason to relax the general criterion by just enough — 17.1 points, as it happens, would have passed v3 — and the person running the cycles is the last person who should be making that call. The gate exists to move that decision out of the loop and onto a page someone else reads.

That page now says: run the technique experiment, at 3B, one variable, criteria first, and stop either way. The intake line itself sits on the Field Index against the base it was trained from, with the general-capability column filled in, because a specialist that cannot show what it paid is not a specialist you can defend.