← ClaudeAtlas

onnx-quantize-tflmlisted

Take a trained neural network to a microcontroller or edge target. Covers post-training INT8 quantisation of ONNX models with real calibration data, TFLite full-integer conversion, choosing calibration sets, diagnosing accuracy loss, flash and tensor-arena budgeting, the TFLite Micro C++ setup with MicroMutableOpResolver, and CMSIS-NN acceleration. Use when quantising or converting a model, when deploying to a Cortex-M, STM32, ESP32, Arduino or other MCU, when a model is too large or too slow for a device, when accuracy drops after quantisation, when sizing a tensor arena, or when a TFLite Micro interpreter fails to allocate or invoke. Not for training the model, and not for server-side inference optimisation.
Premchand006/AI-assistant-skills · ★ 0 · AI & Automation · score 64
Install: claude install-skill Premchand006/AI-assistant-skills
# Model to microcontroller Two things go wrong on this path. The model does not fit, which is a budget problem you can answer in ten minutes. Or it fits and the accuracy quietly drops, which is a measurement problem — and usually a calibration-data problem wearing a disguise. ## Budget first Write down four numbers before converting anything: flash available, RAM available for the tensor arena, latency allowed per inference, and the accuracy you are willing to lose. A model that cannot meet these does not get better with quantisation tuning, and finding that out now costs ten minutes instead of a week. The arena has to hold the largest set of simultaneously live tensors, which is usually driven by the widest early activation rather than by the weights. A model that fits flash comfortably can still miss RAM by a factor of three. ## Quantising ONNX ```bash python3 scripts/quantize_onnx.py model.onnx -o model.int8.onnx \ --calib data/calibration --compare ``` The script preprocesses (shape inference and folding, skipping which is the usual cause of "quantisation ran and the model got slower"), calibrates on real samples, quantises to QDQ format with per-channel weights, and then reports max absolute difference, cosine similarity and argmax agreement against the float model. That last part is the point: a quantised model without a measured accuracy delta is a guess. For a model that loses too much at int8, `--activation-type uint16 --weight-type int8` recovers most of