solo-model-shrink

# model-shrink — ship a trained model to phones and browsers A PyTorch checkpoint becomes: an eval, an ONNX int8 file (web/Android), a Core ML package (Apple), an on-device number, and a download-on-demand delivery. Every step below was paid for once; the order is the method. ## Workflow 1. **Eval before anything.** Ground truth + one scalar metric (F-measure, accuracy — whatever the task has) + a script that runs any model variant against it. Every later decision — quantize? fp16? distill? — is this script's output, never a guess. Keep 5–10 held-out samples as a smoke set. 2. **Export ONNX at a fixed shape** (`torch.onnx.export`, one export per batch size — see Gotchas). Verify against the original: max abs diff and the eval score. `onnxruntime` CPU is the portable baseline: web (`onnxruntime-web` WASM), Android (ORT mobile), desktop. 3. **Quantize weights to int8** (`onnxruntime.quantization` / `ct.optimize.coreml.linear_quantize_weights`). Weights-only int8 is usually free — measured twice on a 20M-param transformer: F 0.872→0.871 (ONNX) and 0.864→0.863 (Core ML), 4x smaller. Re-run the eval anyway; "usually" is not "always". 4. **Apple: convert to native Core ML** (`coremltools`), never the ORT Core ML execution provider (see Don't). Conversion of transformer-ish models fails on Python-int shape math; the fixes that work: - Replace einops layers/`rearrange` with explicit `permute`/`reshape`. - Bake shapes as constants: capture ...