Platform · Quantization evidence
A 13 GB model and a 4 GB model that behaves the same
Quantization is retired and is not available as a Studio job. This page is a design and evidence reference for a future governed integration; it does not start a quantization job.
Why slim at all
The cheapest model is the one you can actually run
Hardware-fit planning often considers edge devices, air-gapped sites, and cost-sensitive batch work. The claims below describe target tradeoffs; LeanLogix has not produced or admitted a live slim build through the current Studio.
Run where the data is
On-device and CPU inference can support a private deployment design. A real deployment would still need measured hardware evidence and boundary attestation; the format alone does not prove either.
Cut cost and latency
4-bit weight-only formats can roughly quarter memory footprint and improve throughput on compatible hardware. Those are planning characteristics until the exact artifact is built and measured.
Keep the evidence intact
A future governed slim build must inherit exact parent lineage and re-run the required eval suite. No current Studio action produces that artifact.
The four formats
Pick the format for the hardware, not the hype
There is no universal best quantization. This reference matrix explains hardware and accuracy tradeoffs; listing a format does not mean LeanLogix currently executes or validates it.
Retention figures are representative ranges, not LeanLogix measurements. A future supported build must attach the measured number for its exact artifact.
Worked example · illustrative
SprintLoop-7B · Edge — a Q4_K_M build, end to end
An illustrative walkthrough of slimming the signed SprintLoop-7B v3 adapter (the prior production release, since superseded by v6) for on-device and CPU serving. The numbers below are representative targets — the signed retention report attaches to the artifact when the build is produced and the eval suite re-runs against it.
# attaches when sprintloop-7b-edge is built
parent: sprintloop-7b "v3"
method: "GGUF Q4_K_M"
size_gb: "~4 (target)"
retention: "~0.97 (target)" # vs fp16 parent
eval_rerun: "18-probe behavioral (on build)"
leakage: "target 0"
signed: # set true when the report is produced
Every build ships with one
The quality-retention report
A slim build with no retention number is a liability. This section defines the report a future authoritative integration must produce; the current Studio does not run quantization or mint this report.
Same probes, recompressed
The quantized model faces the identical behavioral, safety, and leakage probes the parent passed. Retention is measured against that baseline, not a generic benchmark.
Delta, not a vibe
The report records the per-category change. A 3-point drop on task accuracy with leakage held at zero is a different decision than the reverse — and the report makes that decision legible.
Travels with the artifact
A supported implementation must attach the report to the exact registry artifact and sign it before any release decision can rely on it.
Beyond quantization
Pruning — drop the weights that don't earn their place
Quantization shrinks precision while pruning removes weights. Wanda and SparseGPT are reference techniques only; LeanLogix does not currently expose pruning execution.
Wanda
Prunes by the product of weight magnitude and input activation norm — no gradient updates, no retraining. A fast first pass to thin a model before quantizing it further.
SparseGPT
Solves a layer-wise reconstruction problem to remove weights while keeping the layer's output close to the original. Higher sparsity at lower quality cost than naive magnitude pruning.
Beyond quantization
Distillation — a leaner student from a stronger teacher
When a slim build can't hold quality at the size you need, distillation trains a smaller student to imitate a larger teacher's behavior — often recovering accuracy a raw quantize would lose.
On-policy distillation
The student generates, the teacher scores, and the student learns from the teacher's judgment on its own outputs — closing the gap between a small model and a large one on the tasks that matter.
When to reach for it
Distillation costs a training run, so it is the move when quantization and pruning leave too much on the table — a regulated edge target that still needs the larger model's refusal discipline, for instance.
The whole tradeoff in one place · illustrative
Before and after — SprintLoop-7B v3 → Edge
The shape of the accounting. Roughly a quarter of the size, faster on CPU, a few points of quality spent, leakage held at zero — the actual figures fill in when the build is produced and the eval re-runs.
Note
Parent fp16 footprint is the standard 7B half-precision size; the on-disk edge figure is the representative Q4_K_M target, not a measured artifact yet. The point of the report is to make the quality you spend and the things you keep — leakage, refusal behavior — explicit before anyone deploys, with the signed numbers attached to the artifact once the build runs.
Review the quantization evidence contract
Use the reference to plan hardware and retention evidence. Execution remains unavailable until an authoritative source, runner, artifact, and measured-report path exists.