119а. Model optimization for a microcontroller: pruning, weight clustering, distillation, choosing the precision — size, speed and quality in the real environment

In section 119 the network was compressed by quantization: the same weights, but in int8. That is one step out of several. A model can be made smaller in other ways too: zero out the weak weights (pruning), remove whole neurons, replace the weights with indices into a short table (clustering), train a small network on the answers of a big one (distillation), or simply take a narrower network. Here all the methods are run on one task and measured the same way: correct answers on held-out recordings, bytes of flash, instructions per answer in QEMU and energy per answer from the F407 datasheet. At the end — what happens to the model in the field: a different sensor, noise, a confidence threshold and what to watch on the device.

Sources: numpy 2.3.3 and scikit-learn 1.7.2 (KMeans only) in the stm-ml:1 image, the train119b.py script. The tensorflow_model_optimization package is not in the images — the pruning masks, the shared cluster weights and the teacher's soft answers are written in the script itself; a rerun produces the same files. The C inference (infer119b.c) is built with gcc 12.2.0 (stm-sys:1) for the cross-check, with armclang 6.24 for sizes, and with arm-none-eabi-gcc 12.2.1 and QEMU 7.2 (stm-arm:1, the netduinoplus2 board with an STM32F405 and mps2-an500 with a Cortex-M7) for instruction counts. The current comes from datasheet DS8626 rev. 9 in the Keil pack. Quantization in detail is section 119, training is 118, features are 117.

Terms used in the documentation

Term	Meaning
pruning	zeroing out part of the weights
magnitude pruning	zero the weights with the smallest absolute value
unstructured / structured pruning	unstructured — single weights; structured — whole neurons or channels
sparsity	the share of zeros
gradual pruning, pruning schedule	pruning that grows step by step on a schedule
fine-tuning	further training after a change
sparse format, CSR (compressed sparse row)	the non-zero values plus the indices of their inputs
weight clustering, weight sharing	many weights share one common value
codebook, centroid	table of centres, centre of a cluster
knowledge distillation	a small model learns from a big one
teacher, student	the big model and the small one
soft targets, temperature	the teacher's probabilities, softened by dividing logits by T
half precision (float16)	2 bytes per number
latency, energy per inference	time and energy per answer
distribution shift, sensor drift	data unlike the training data, the sensor slowly changing
confidence threshold	below it the answer is "not sure"

119а.1 The task, the measure and the spread

An analog accelerometer on a motor housing, the STM32 ADC — 12 bits over ±4 g, 1024 Hz, a window of 256 samples (250 ms). The features are 64 spectrum bins with a Hann window, from 4 to 256 Hz, in dB relative to 1 g:

    q = np.clip(np.round(x / 8.0 * 4096 + 2048), 0, 4095)
    x = (q - 2048) * 8.0 / 4096
    x = x - x.mean(axis=1, keepdims=True)
    m = np.abs(np.fft.rfft(x * hann, axis=1))[:, 1:NB + 1] * (2.0 / N)
    return (20 * np.log10(m + 1e-4)).astype(np.float32)

Four classes: normal, imbalance (the harmonic at the rotation frequency of about 25 Hz grows), shaft misalignment (the second and third harmonics grow) and bearing (a carrier at 120…200 Hz modulated by the rotation). Sensor noise is 0.01 g, and every recording has its own mounting, ±10 %. In total 160 recordings of 10 windows; recordings are split whole (section 118): training — 96 recordings and 960 windows, validation — 32 and 320, test — 32 and 320.

The working network is 64 → 32 → 4 in float32, 2212 parameters: it gets 278 of 320 test windows right. The teacher is 64 → 128 → 64 → 4, 16 836 parameters: 283.

Before comparing, you need to know the measurement noise. The same network, trained with five other initial weights, gave from 271 to 282 on the test set. A second test set — 32 more new recordings from the same generator — gives the working network 301 of 320. So a difference of a few windows is not a gain but chance; choose a variant on the validation set, and touch the test set once.

119а.2 Magnitude pruning: zeros without speed

Unstructured pruning zeroes the share of weights with the smallest absolute value in a layer. The hidden layer is pruned — 2048 weights; the output layer (128 weights) is left whole:

    k = int(round(sparsity * W.size))
    thr = np.sort(np.abs(W).ravel())[k - 1]
    return (np.abs(W) > thr).astype(W.dtype)

Three ways. One-shot — zero out and evaluate at once. One-shot with fine-tuning — 40 epochs, the gradient of the zeroed weights is masked out. Gradual — the same 40 epochs, but the share of zeros grows on a cubic schedule over the first 30:

    sched = lambda ep, sp=sp: sp * (1 - (1 - min(ep, 30) / 30.0) ** 3)

Correct out of 320, validation / test:

Without fine-tuning, 90 % zeros cost 28 test windows and 95 % cost 45. Fine-tuning recovers the loss, and gradual pruning beats one-shot on validation at 75, 90 and 95 %: the network has time to move the weight onto the remaining connections.

Why zeros do not speed things up. A dense kernel multiplies every weight in turn, zeros included. In QEMU the network with 90 % zeros in dense format takes the same 14 125 instructions per answer as without zeros. To skip zeros you need a sparse format: the non-zero values of a row and a byte with the input index for each.

        for (int k = L->rowp[j]; k < L->rowp[j + 1]; k++)
            s += v[k] * x[L->col[k]];

The input index is an extra load for every multiplication: in CSR one non-zero weight takes 11.6 instructions against 6.5 in the dense loop. Here the format pays off already at half zeros, but not twofold: 50 % zeros cut the instructions by 26 %, not by 50.

119а.3 Structured pruning and weight clustering

Structured pruning removes a neuron entirely — a row of hidden-layer weights and a column of the output layer. A neuron's importance is the product of the norms of its incoming and outgoing weights:

imp = np.linalg.norm(BASE[0][0], axis=1) * np.linalg.norm(BASE[1][0], axis=0)

Validation / test, correct out of 320. Cut down to 8 neurons, the network immediately loses 69 windows, and after fine-tuning none; it is simply a 64 → 8 → 4 network: 544 multiplications instead of 2176, 4021 instructions instead of 14 125, 2224 bytes. A plain kernel, no special format. With 4 neurons the fine-tuned pruned network turned out better than one trained from scratch — 273 against 264: the big network gave it a good start.

Clustering replaces a layer's weights with k shared values: k-means over the weights themselves; memory holds a cluster index for every weight and a table of centres.

        km = KMeans(n_clusters=k - (1 if keep_zero else 0), n_init=10, random_state=SEED).fit(v[nz].reshape(-1, 1))

Fine-tuning the centres (a centre's gradient is the sum of the gradients of its weights) added nothing here: the network barely noticed it had 4 distinct weights per layer left. On the device a weight is fetched from the table:

            s += L->tab[(p[k >> 1] >> ((k & 1) * 4)) & 15] * x[i];

16 centres: 1360 bytes against 8848 — 6.5 times smaller. But 31 607 instructions per answer — 2.2 times slower than dense float: unpacking the index and reading the table on every weight. Clusters are for flash and for sending the model over the air, not for speed. Together with gradual pruning to 90 % (zero is its own centre, it is not fine-tuned) — 277 on the test set; in this format an index is stored for every weight, and zeros save no space.

119а.4 Knowledge distillation

The teacher gets 283 of 320, the student 64 → 4 → 4 trained from scratch gets 264. Distillation teaches the student not only the labels but also the teacher's soft answers: the logits of both are divided by the temperature T, and the student pulls its probabilities towards the teacher's. The loss is 0.3 of the usual cross-entropy and 0.7 · T² · KL; the gradient with respect to the student's logits:

                pt = softmax(teacher[bi], T)
                    d += alpha * T * (softmax(acts[-1], T) - pt)

At T = 1 there is no benefit: on its own training windows the teacher is right in 960 of 960, and its mean top probability is 0.9998; the soft answer equals the label. At T = 8 it is 0.7614 — the teacher tells which classes are similar. T = 8 was chosen on validation. With five other initial weights the student without a teacher got from 263 to 279 (268.0 on average), with the teacher from 274 to 279 (275.4 on average): a gain of about 7 windows and a smaller spread.

The student takes 2337 instructions and 1120 bytes, the teacher 103 025 instructions and 67 344 bytes: 44 times faster and 60 times smaller for a difference of 5 windows.

119а.5 Width and precision: the summary table

64 → H → 4 networks from scratch, correct out of 320 (int8 is our own: one scale per tensor for the input and the hidden layer, one per output for the weights; in detail — section 119):

The cost of an answer. Flash is weights and biases from the armclang object file (the normalization, 512 bytes, is shared by all); instructions are from QEMU, on a Cortex-M4F (netduinoplus2) and a Cortex-M7 (mps2-an500); time and energy are from the instruction counts at 168 MHz:

RAM is the stack: model_run takes 1088 bytes for any float variant (two buffers of 128 numbers, sized for the widest network — the teacher), i8_run takes 184 bytes on the M4 and 176 on the M7 (gcc, -fstack-usage).

What it shows. From 8 neurons on, quality no longer grows, while the cost grows linearly. float16 gave the same answers as float32 at all five widths and half the flash, but 31 % more instructions: every weight is converted to float by a vcvtb instruction. int8 is within ±2 windows and 3.6 times less flash at H = 32, but on the M4 it is no faster than float: rounding the input and the hidden layer with floorf is a library call on the M4, while on the M7 it is the vrintm instruction (per the gcc disassembly), and there int8 is 9 % faster than float. The plain loop does one multiplication per instruction; SMLAD, two multiplications per instruction, is in the CMSIS-NN kernels from section 119. For the float instruction counts QEMU does not distinguish the M4 from the M7: it is the same code. Cycles of the M7 with dual issue and a cache are not modelled by QEMU; they were not measured.

119а.6 In the real environment: latency, energy, sensor, confidence

Latency. The time in the table is instructions divided by 168 MHz: the M4 needs at least as many cycles as instructions, and at 168 MHz the flash has 5 wait states (DS8626, section 5.3.6). A window arrives every 250 ms; even the teacher needs at least 613.2 µs.

Energy. DS8626 rev. 9, table 20: 168 MHz, ART on, all peripherals off — 40 mA typical at 25 °C and 3.3 V (with the peripherals on — 87 mA). Energy per answer is 3.3 V · 40 mA · time: the working network takes at least 11.1 µJ, CSR with 90 % zeros 3.0, the student 1.8, the teacher 80.9. That is the core doing the computation; the ADC, the sensor and sleep between windows are a separate item.

A different sensor, drift, noise. The same 32 test recordings, but a different sensor: sensitivity −20 % and +25 % (calibration drift or another unit), noise three times higher (worse mounting, interference) and a zero offset of 0.1 g. Correct out of 320:

In dB, sensitivity is a shift of 1.9 dB, and the network barely noticed it. A zero offset is removed by subtracting the mean. Noise ×3 is a collapse for all: the spectrum floor has risen, and the weak signs of misalignment and bearing drowned. Pruned and clustered variants hold up like the original network; small networks fall further. Training with sensor variation — 6 copies of the training recordings, sensitivity 0.7…1.4 and noise 1…3 times higher — raised noise ×3 to 232, but cost 22 windows on clean data.

Confidence threshold. The answer is "not sure" if the top softmax probability is below the threshold. The threshold was chosen on validation — the largest of 0.5…0.95 for which "not sure" is at most 5 %: 0.7 (14 windows of 320).

        sum += expf(s[c] - mx);
    return 1.0f / sum >= TAU;

For the working network on the test set "not sure" is 11 windows, and of 309 confident answers 275 are right. With noise ×3 "not sure" is only 19, while of 301 confident answers only 159 are right. The network is confidently wrong: the "not sure" share barely sees a data shift.

Monitoring in the field. So watch the input as well. The spectrum floor is the median of the 64 bins in dB; in training it is no higher than −57.7 dB (median −61.6). With noise ×3 all 320 windows are above this threshold; on clean data, with a different sensitivity and with an offset — none. On the device both shares — "not sure" and "input outside training" — are accumulated over a day and sent upstream: their growth is a reason to check the mounting and recalibrate the sensor rather than trust the class.

119а.7 Pitfalls

After pruning the accuracy dropped by 28 windows. Zeroed without fine-tuning: 90 % zeros — 250 instead of 278. Fine-tuning with a mask gives 277, gradual pruning 280.

A model that is 90 % zeros computes just as long. A dense kernel multiplies zeros: 14 125 instructions with them and without them. You need a sparse format: CSR — 3863 instructions and 1885 bytes.

The clustered model is slower than the original. Unpacking a 4-bit index and a table lookup on every weight: 31 607 instructions against 14 125. Clusters save flash, not time.

Distillation gave nothing. T = 1: on its own training windows the teacher is 0.9998 sure, the soft answers equal the labels. Use a temperature of 4…8 and choose it on validation.

The student became unsure. The temperature is applied only to the teacher, the student's logits are not divided: the student learns to answer in a smeared way. Correct answers are the same (278), but the mean top probability is 0.896 instead of 0.969 and "not sure" is 55 windows of 320 instead of 11. Divide both logits by T and multiply the gradient by T.

On the test set, 95 % pruning is "better" than the original network. 286 against 278 — but on the second test set of new recordings it is 300 against 301. The spread from initial weights is 271…282, and the two test sets differ by 23 windows. Choose on validation, touch the test set once, and settle disputes on new recordings.

int8 on the F407 is no faster than float. Our own integer kernel without SIMD and floorf from the library: 15 240 instructions against 14 125. You need the CMSIS-NN kernels with SMLAD (section 119).

In the field the model is confidently wrong. Noise ×3: 159 of 301 confident answers are right, "not sure" is 19. Watch the input: the spectrum floor is above the training one in all 320 windows.

119а.8 Summary

• Measure on held-out recordings and know the noise: from initial weights — 271…282 of 320, and two test sets differ by 23 windows.
• Unstructured 90 % pruning with fine-tuning costs no quality (280), but speed comes only from a sparse format: CSR — 3863 instructions and 1885 bytes against 14 125 and 8848.
• Structured pruning down to 8 neurons — 278, 4021 instructions, a plain kernel.
• Clusters: 16 centres — 1360 bytes (6.5 times smaller), but 31 607 instructions.
• Distillation with T = 8: the student 64 → 4 → 4 gets 278 instead of 264; with T = 1 — nothing; without the temperature on the student — 55 "not sure" instead of 11.
• float16 — the same answers, half the flash, +31 % instructions; int8 — 3.6 times smaller, on the M4 without SIMD no faster.
• Energy from DS8626 (40 mA at 168 MHz): the working network takes at least 11.1 µJ, the student 1.8.
• Noise ×3 drops every variant (278 → 161), and "not sure" barely grows; watch the input too — the spectrum floor is above the training one in all 320 windows.
• All 24 variants in C matched Python bit for bit — on the PC, in QEMU on the Cortex-M4F and on the Cortex-M7.

Full sources

train119b.py — data, training in numpy, pruning, clustering, distillation, field checks and export of all variants to model119b.h.

# -*- coding: utf-8 -*-
"""Раздел 119а: оптимизация модели для микроконтроллера — прунинг, кластеризация весов,
дистилляция, ширина сети и разрядность.

Запуск — в образе stm-ml:1:  python train119b.py <каталог_вывода>

Всё на numpy (и KMeans из scikit-learn): пакета tensorflow_model_optimization нет, маски
весов, общие веса кластеров и мягкие ответы учителя написаны здесь же. Данные — вибрация
мотора с аналогового акселерометра на АЦП STM32 (12 бит, ±4 g, 1024 Гц, окно 256 отсчётов),
признаки — 64 бина спектра в дБ (4…256 Гц). Четыре класса: норма, дисбаланс, перекос валов,
подшипник. Записи делятся на обучение, подбор и проверку целиком.

Выход: model119b.h (веса всех вариантов для infer119b.c, проверочные окна, контрольные суммы
логитов, которые обязан повторить C) и summary119b.txt (все числа раздела)."""
import os
import sys

os.environ["OPENBLAS_NUM_THREADS"] = "1"
os.environ["OMP_NUM_THREADS"] = "1"
import numpy as np
import sklearn
from sklearn.cluster import KMeans

OUT = sys.argv[1] if len(sys.argv) > 1 else "."
SEED = 1192
N, FS, NB, WPR = 256, 1024.0, 64, 10           # отсчётов в окне, Гц, бинов, окон на запись
REC_TR, REC_VA, REC_TE = 24, 8, 8              # записей на класс: обучение, подбор, проверка
t = np.arange(N) / FS
hann = 0.5 * (1 - np.cos(2 * np.pi * np.arange(N) / N))
lines = ["versions=%s,%s" % (np.__version__, sklearn.__version__)]
S = {}                                          # сводка: ключ → строка


def put(k, *v):
    S[k] = ",".join(str(x) for x in v)
    lines.append("%s=%s" % (k, S[k]))


# ══ 1. Данные ═════════════════════════════════════════════════════════════
drng = np.random.RandomState(SEED)


def recording(c):
    """Одна запись: чистая вибрация в g (без шума датчика) и шум отдельно — датчик применяется потом."""
    u = lambda lo, hi: drng.uniform(lo, hi)
    f1 = 25.0 + u(-1.5, 1.5)                    # частота вращения
    mount = u(0.9, 1.1)                         # крепление этого датчика на этом моторе
    a1 = 0.06 * u(0.5, 1.5)
    a2, a3, ab, fb = 0.01 * u(0.5, 1.5), 0.0, 0.0, 0.0
    if c == 1:                                  # дисбаланс: сильная 1×
        a1 = 0.15 * u(0.6, 1.4)
    elif c == 2:                                # перекос: растут 2× и 3×
        a2, a3 = 0.05 * u(0.4, 1.6), 0.02 * u(0.5, 1.5)
    elif c == 3:                                # подшипник: несущая 120…200 Гц, модулированная вращением
        ab, fb = 0.02 * u(0.3, 1.5), u(120.0, 200.0)
    sig_n = 0.01 * u(0.8, 1.2)
    X, Nz = [], []
    for _ in range(WPR):
        p = drng.uniform(0, 2 * np.pi, 4)
        k = drng.uniform(0.95, 1.05, 4)
        x = (k[0] * a1 * np.sin(2 * np.pi * f1 * t + p[0]) + k[1] * a2 * np.sin(4 * np.pi * f1 * t + p[1])
             + k[2] * a3 * np.sin(6 * np.pi * f1 * t + p[2])
             + k[3] * ab * (1 + 0.6 * np.cos(2 * np.pi * f1 * t)) * np.sin(2 * np.pi * fb * t + p[3]))
        X.append(mount * x)
        Nz.append(sig_n * drng.randn(N))
    return X, Nz


CLEAN, NOISE, y, g = [], [], [], []
for c in range(4):
    for r in range(REC_TR + REC_VA + REC_TE):
        X_, N_ = recording(c)
        CLEAN += X_
        NOISE += N_
        y += [c] * WPR
        g += [r] * WPR
for c in range(4):                              # вторая проверка: ещё по 8 новых записей на класс
    for r in range(REC_TE):
        X_, N_ = recording(c)
        CLEAN += X_
        NOISE += N_
        y += [c] * WPR
        g += [100 + r] * WPR
CLEAN, NOISE, y, g = np.array(CLEAN), np.array(NOISE), np.array(y), np.array(g)
mtr, mva = g < REC_TR, (g >= REC_TR) & (g < REC_TR + REC_VA)
mte, mte2 = (g >= REC_TR + REC_VA) & (g < 100), g >= 100


def sensor(gain=1.0, noise=1.0, offset=0.0):
    """Акселерометр + АЦП 12 бит на ±4 g → 64 бина спектра в дБ относительно 1 g."""
    x = gain * CLEAN + noise * NOISE + offset
    q = np.clip(np.round(x / 8.0 * 4096 + 2048), 0, 4095)
    x = (q - 2048) * 8.0 / 4096
    x = x - x.mean(axis=1, keepdims=True)
    m = np.abs(np.fft.rfft(x * hann, axis=1))[:, 1:NB + 1] * (2.0 / N)
    return (20 * np.log10(m + 1e-4)).astype(np.float32)


X = sensor()
put("windows", int((g < 100).sum()), int(mtr.sum()), int(mva.sum()), int(mte.sum()), int(mte2.sum()))
put("recordings", 4 * (REC_TR + REC_VA + REC_TE), 4 * REC_TR, 4 * REC_VA, 4 * REC_TE, 4 * REC_TE)
MU = X[mtr].mean(axis=0).astype(np.float32)
SD = X[mtr].std(axis=0).astype(np.float32)
norm = lambda A: ((A - MU) / SD).astype(np.float32)
Z = norm(X)
Y1 = np.eye(4)[y]

# ══ 2. Обучение: numpy, Adam, маски и общие веса ═══════════════════════════


def init(sizes, rng):
    return [[rng.randn(o, i) * np.sqrt(2.0 / i), np.zeros(o)] for i, o in zip(sizes[:-1], sizes[1:])]


def forward(P, A):
    acts = [A]
    for k, (W, b) in enumerate(P):
        A = A @ W.T + b
        if k < len(P) - 1:
            A = np.maximum(A, 0.0)
        acts.append(A)
    return acts


def softmax(Zl, T=1.0):
    e = np.exp((Zl - Zl.max(axis=1, keepdims=True)) / T)
    return e / e.sum(axis=1, keepdims=True)


def magnitude_mask(W, sparsity):
    """Маска прунинга по величине: обнулить долю sparsity самых малых по модулю весов слоя."""
    k = int(round(sparsity * W.size))
    if k == 0:
        return np.ones_like(W)
    thr = np.sort(np.abs(W).ravel())[k - 1]
    return (np.abs(W) > thr).astype(W.dtype)


def prune_masks(P, sparsity):
    """Прунинг скрытых слоёв; выходной слой (32 × 4 = 128 весов) не трогаем — в нём каждый вес на счету."""
    return [magnitude_mask(W, sparsity) if k < len(P) - 1 else np.ones_like(W) for k, (W, _) in enumerate(P)]


def train(P, idx, epochs, lr, seed, masks=None, schedule=None, teacher=None, T=1.0, alpha=0.0, bug=None,
          clusters=None, l2=1e-4):
    """Обучение (или дообучение) сети P на окнах idx. masks — нули прунинга; schedule(эпоха) → доля нулей
    для постепенного прунинга; teacher — логиты учителя на тех же окнах (дистилляция); clusters — для
    каждого слоя (номера кластеров, центры): учатся только центры."""
    rng = np.random.RandomState(seed)
    P = [[W.copy(), b.copy()] for W, b in P]
    m1 = [[np.zeros_like(W), np.zeros_like(b)] for W, b in P]
    m2 = [[np.zeros_like(W), np.zeros_like(b)] for W, b in P]
    if clusters is not None:
        C = [c.copy() for _, c, _ in clusters]
        cm1, cm2 = [np.zeros_like(c) for c in C], [np.zeros_like(c) for c in C]
    step = 0
    for ep in range(epochs):
        if schedule is not None:
            masks = prune_masks(P, schedule(ep))
            for k in range(len(P)):
                P[k][0] *= masks[k]
        order = rng.permutation(idx)
        for s0 in range(0, len(order), 32):
            bi = order[s0:s0 + 32]
            acts = forward(P, Z[bi])
            p = softmax(acts[-1])
            d = (p - Y1[bi]) * (1.0 - alpha)
            if teacher is not None:
                pt = softmax(teacher[bi], T)
                if bug == "student_T1":                       # ученик без температуры
                    d += alpha * (softmax(acts[-1]) - pt)
                elif bug == "no_T2":                          # забыт множитель T²
                    d += alpha * (softmax(acts[-1], T) - pt) / T
                else:                                         # T² · KL → градиент T · (ps − pt)
                    d += alpha * T * (softmax(acts[-1], T) - pt)
            d /= len(bi)
            step += 1
            grads = []
            for k in range(len(P) - 1, -1, -1):
                gW = d.T @ acts[k] + l2 * P[k][0]
                gb = d.sum(axis=0)
                grads.append((k, gW, gb))
                if k:
                    d = (d @ P[k][0]) * (acts[k] > 0)
            for k, gW, gb in grads:
                if masks is not None:
                    gW = gW * masks[k]
                if clusters is not None:                      # общий вес: градиент центра — сумма по кластеру
                    a = clusters[k][0]
                    gC = np.bincount(a.ravel(), weights=gW.ravel(), minlength=len(C[k]))
                    if clusters[k][2]:
                        gC[0] = 0.0                           # кластер «ноль» остаётся нулём
                    cm1[k] = 0.9 * cm1[k] + 0.1 * gC
                    cm2[k] = 0.999 * cm2[k] + 0.001 * gC * gC
                    C[k] -= lr * (cm1[k] / (1 - 0.9 ** step)) / (np.sqrt(cm2[k] / (1 - 0.999 ** step)) + 1e-8)
                    P[k][0] = C[k][a]
                    gW = None
                for j, gr in ((0, gW), (1, gb)):
                    if gr is None:
                        continue
                    m1[k][j] = 0.9 * m1[k][j] + 0.1 * gr
                    m2[k][j] = 0.999 * m2[k][j] + 0.001 * gr * gr
                    P[k][j] -= lr * (m1[k][j] / (1 - 0.9 ** step)) / (np.sqrt(m2[k][j] / (1 - 0.999 ** step)) + 1e-8)
                if masks is not None:
                    P[k][0] *= masks[k]
    return P


def acc(P, mask, Zs=None):
    Zs = Z if Zs is None else Zs
    return int((forward(P, Zs[mask])[-1].argmax(axis=1) == y[mask]).sum())


TR = np.where(mtr)[0]
EPOCHS, LR = 150, 0.003

# ══ 3. Ширина сети и учитель ═══════════════════════════════════════════════
WIDTHS = (4, 8, 16, 32, 64)
NETS = {}
for h in WIDTHS:
    NETS["w%d" % h] = train(init((NB, h, 4), np.random.RandomState(SEED + h)), TR, EPOCHS, LR, SEED + h)
    put("width_%d" % h, acc(NETS["w%d" % h], mva), acc(NETS["w%d" % h], mte))
REP = [acc(train(init((NB, 32, 4), np.random.RandomState(SEED + 500 + r)), TR, EPOCHS, LR, SEED + 500 + r), mte)
       for r in range(5)]
put("w32_seeds", *REP)
TEACH = train(init((NB, 128, 64, 4), np.random.RandomState(SEED + 128)), TR, EPOCHS, LR, SEED + 128)
put("teacher", acc(TEACH, mva), acc(TEACH, mte), sum(W.size + b.size for W, b in TEACH))
BASE = NETS["w32"]

# ══ 4. Прунинг по величине: разовый, разовый + дообучение, постепенный ═══════
FT = 40
SPARS = (0.5, 0.75, 0.9, 0.95)
PRUNED = {}
for sp in SPARS:
    mk = prune_masks(BASE, sp)
    once = [[W * m, b.copy()] for (W, b), m in zip(BASE, mk)]
    ft = train(once, TR, FT, 0.001, SEED + 7, masks=mk)
    sched = lambda ep, sp=sp: sp * (1 - (1 - min(ep, 30) / 30.0) ** 3)
    grad = train(BASE, TR, FT, 0.001, SEED + 7, schedule=sched)
    grad_mask = prune_masks(grad, sp)
    grad = [[W * m, b] for (W, b), m in zip(grad, grad_mask)]
    PRUNED[sp] = grad
    zeros = sum(int((W == 0).sum()) for W, _ in grad)
    put("prune_%d" % round(sp * 100), acc(once, mva), acc(once, mte), acc(ft, mva), acc(ft, mte),
        acc(grad, mva), acc(grad, mte), zeros)

# выбор по проверке: лучшая из 12 настроек прунинга по mte — и она же на новых записях mte2
cand = []
for sp in SPARS:
    v = [int(x) for x in S["prune_%d" % round(sp * 100)].split(",")]
    cand += [(v[1], v[0], sp, "once"), (v[3], v[2], sp, "ft"), (v[5], v[4], sp, "grad")]
bt = max(cand, key=lambda c: (c[0], c[1]))
bv = max(cand, key=lambda c: (c[1], c[0]))
put("pick_by_test", bt[3], round(bt[2] * 100), bt[0], acc(PRUNED[bt[2]], mte2) if bt[3] == "grad" else -1)
put("pick_by_val", bv[3], round(bv[2] * 100), bv[1], bv[0], acc(PRUNED[bv[2]], mte2) if bv[3] == "grad" else -1)
put("base_test2", acc(BASE, mte2))

# ══ 5. Структурный прунинг: целые нейроны скрытого слоя ════════════════════
imp = np.linalg.norm(BASE[0][0], axis=1) * np.linalg.norm(BASE[1][0], axis=0)
STRUCT = {}
for keep in (16, 8, 4):
    kk = np.sort(np.argsort(-imp)[:keep])
    cut = [[BASE[0][0][kk], BASE[0][1][kk]], [BASE[1][0][:, kk], BASE[1][1].copy()]]
    ft = train(cut, TR, FT, 0.001, SEED + 9)
    STRUCT[keep] = ft
    put("struct_%d" % keep, acc(cut, mva), acc(cut, mte), acc(ft, mva), acc(ft, mte))

# ══ 6. Кластеризация весов: k-средних, номера + таблица центров ═════════════


def cluster(P, k, keep_zero=False):
    out = []
    for n, (W, _) in enumerate(P):
        v = W.ravel()
        nz = v != 0 if keep_zero and n < len(P) - 1 else np.ones(v.size, bool)
        km = KMeans(n_clusters=k - (1 if keep_zero else 0), n_init=10, random_state=SEED).fit(v[nz].reshape(-1, 1))
        cen = np.sort(km.cluster_centers_.ravel())
        lab = np.zeros(v.size, np.int64)
        idx = np.abs(v[nz][:, None] - cen[None, :]).argmin(axis=1)
        if keep_zero:
            cen = np.concatenate([[0.0], cen])
            lab[nz] = idx + 1
        else:
            lab[:] = idx
        out.append((lab.reshape(W.shape), cen, keep_zero))
    return out


CLUST = {}
for k in (4, 8, 16):
    cl = cluster(BASE, k)
    raw = [[c[a], b] for (a, c, _), (_, b) in zip(cl, BASE)]
    ft = train(raw, TR, 20, 0.001, SEED + 11, clusters=cl)
    CLUST[k] = (ft, cl)
    put("cluster_%d" % k, acc(raw, mva), acc(raw, mte), acc(ft, mva), acc(ft, mte),
        sum(len(np.unique(W)) for W, _ in ft))
cl = cluster(PRUNED[0.9], 16, keep_zero=True)
raw = [[c[a], b] for (a, c, _), (_, b) in zip(cl, PRUNED[0.9])]
DEEP = train(raw, TR, 20, 0.001, SEED + 11, clusters=cl)
DEEP_CL = cl
put("deep", acc(raw, mva), acc(raw, mte), acc(DEEP, mva), acc(DEEP, mte), sum(int((W == 0).sum()) for W, _ in DEEP))

# ══ 7. Дистилляция: ученик 64 → 4 → 4 на мягких ответах учителя ═══════════
TL = forward(TEACH, Z)[-1]
SW = 4
st0 = init((NB, SW, 4), np.random.RandomState(SEED + SW))
put("teacher_train", acc(TEACH, mtr), int(mtr.sum()),
    *["%.4f" % float(softmax(TL[mtr], T_).max(axis=1).mean()) for T_ in (1.0, 2.0, 4.0, 8.0)])
DIST = {}
for name, T, bug in (("T1", 1.0, None), ("T2", 2.0, None), ("T4", 4.0, None), ("T8", 8.0, None),
                     ("T4_noT2", 4.0, "no_T2"), ("T4_studentT1", 4.0, "student_T1")):
    st = train(st0, TR, EPOCHS, LR, SEED + SW, teacher=TL, T=T, alpha=0.7, bug=bug)
    DIST[name] = st
    pm = softmax(forward(st, Z[mte])[-1]).max(axis=1)
    put("dist_%s" % name, acc(st, mva), acc(st, mte), "%.3f" % float(pm.mean()))
pm = softmax(forward(NETS["w%d" % SW], Z[mte])[-1]).max(axis=1)
put("dist_scratch", acc(NETS["w%d" % SW], mva), acc(NETS["w%d" % SW], mte), "%.3f" % float(pm.mean()))
names = ("T1", "T2", "T4", "T8")
best_val = max(names, key=lambda n: (int(S["dist_" + n].split(",")[0]), -names.index(n)))
best_test = max(names, key=lambda n: (int(S["dist_" + n].split(",")[1]), -names.index(n)))
put("dist_pick", best_val, best_test)
STUDENT = DIST[best_val]
Tb = float(best_val[1:])
sc, di = [], []
for r in range(5):                              # то же с пятью другими начальными весами
    s0 = init((NB, SW, 4), np.random.RandomState(SEED + 700 + r))
    sc.append(acc(train(s0, TR, EPOCHS, LR, SEED + 700 + r), mte))
    di.append(acc(train(s0, TR, EPOCHS, LR, SEED + 700 + r, teacher=TL, T=Tb, alpha=0.7), mte))
put("dist_seeds_scratch", *sc)
put("dist_seeds_teacher", *di)

# ══ 8. Устойчивость: другой датчик, дрейф, шум ═════════════════════════════
DRIFTS = (("clean", 1.0, 1.0, 0.0), ("gain08", 0.8, 1.0, 0.0), ("gain125", 1.25, 1.0, 0.0),
          ("noise3", 1.0, 3.0, 0.0), ("offset", 1.0, 1.0, 0.1))
ZD = {n: norm(sensor(gn, nz, of)) for n, gn, nz, of in DRIFTS}
# обучение с разбросом датчика: те же записи, усиление 0,7…1,4 и шум в 1…3 раза больше
arng = np.random.RandomState(SEED + 3)
gains = np.concatenate([[1.0], arng.uniform(0.7, 1.4, 5)])          # первая копия — сам датчик
noises = np.concatenate([[1.0], arng.uniform(1.0, 3.0, 5)])
ZA = np.concatenate([norm(sensor(gn, nz))[mtr] for gn, nz in zip(gains, noises)])
YA = np.concatenate([y[mtr]] * len(gains))
_Z, _Y1 = Z, Y1
Z, Y1 = ZA, np.eye(4)[YA]
AUG = train(init((NB, 32, 4), np.random.RandomState(SEED + 32)), np.arange(len(YA)), EPOCHS, LR, SEED + 32)
Z, Y1 = _Z, _Y1
ROBUST = (("teacher", TEACH), ("w32", BASE), ("prune90", PRUNED[0.9]), ("deep", DEEP), ("struct8", STRUCT[8]),
          ("student", STUDENT), ("w4", NETS["w4"]), ("aug", AUG))
for n, P in ROBUST:
    put("robust_%s" % n, *[acc(P, mte, ZD[d]) for d, *_ in DRIFTS])

# ══ 9. Порог уверенности и доля «не уверен» ═══════════════════════════════
pv = softmax(forward(BASE, Z[mva])[-1]).max(axis=1)
TAUS = (0.5, 0.6, 0.7, 0.8, 0.9, 0.95)
tau = max(tt for tt in TAUS if (pv < tt).mean() <= 0.05)
put("tau", tau, int((pv < tau).sum()), int(mva.sum()))
for n, P in (("w32", BASE), ("aug", AUG), ("student", STUDENT), ("studentT4", DIST["T4"]), ("studentT1", DIST["T4_studentT1"])):
    row = []
    for d, *_ in DRIFTS:
        lo = forward(P, ZD[d][mte])[-1]
        pm = softmax(lo).max(axis=1)
        sure = pm >= tau
        row += [int((~sure).sum()), int(((lo.argmax(axis=1) == y[mte]) & sure).sum()), int(sure.sum())]
    put("conf_%s" % n, *row)

# монитор входа: «пол» спектра — медиана 64 бинов, дБ; порог — наибольший на обучении
floor = lambda A: np.median(A, axis=1)
fl_thr = float(floor(X[mtr]).max())
put("floor_thr", "%.1f" % fl_thr, "%.1f" % float(np.median(floor(X[mtr]))))
put("floor_flag", *[int((floor(sensor(gn, nz, of)[mte]) > fl_thr).sum()) for _, gn, nz, of in DRIFTS])

# ══ 10. Варианты для C: float32, float16, int8, CSR, кластеры ══════════════
ZT = Z[mte]
f32 = np.float32


def seq_dense(Wd, b, A, relu):
    """Как на C: s = b; s += w·x по входам по порядку, каждое действие во float32."""
    s = np.tile(b.astype(f32), (A.shape[0], 1))
    for i in range(Wd.shape[1]):
        s = (s + (Wd[:, i][None, :] * A[:, i][:, None]).astype(f32)).astype(f32)
    return np.where(s > f32(0), s, f32(0)) if relu else s


def run_float(layers, A):
    for k, (Wd, b) in enumerate(layers):
        A = seq_dense(Wd.astype(f32), b.astype(f32), A, k < len(layers) - 1)
    return A


def quant_i8(P):
    """int8: вход и скрытый слой — свой масштаб на тензор (по обучению), веса — на каждый выход."""
    (W1, b1), (W2, b2) = P
    sx = f32(np.abs(Z[mtr]).max() / 127.0)
    h = np.maximum(Z[mtr] @ W1.T + b1, 0)
    sh = f32(h.max() / 127.0)
    sw1 = (np.abs(W1).max(axis=1) / 127.0).astype(f32)
    sw2 = (np.abs(W2).max(axis=1) / 127.0).astype(f32)
    q1 = np.round(W1 / sw1[:, None]).astype(np.int8)
    q2 = np.round(W2 / sw2[:, None]).astype(np.int8)
    bq1 = np.round(b1 / (sx * sw1)).astype(np.int32)
    bq2 = np.round(b2 / (sh * sw2)).astype(np.int32)
    m1 = (sx * sw1 / sh).astype(f32)
    m2 = (sh * sw2).astype(f32)
    return dict(sx=sx, q1=q1, bq1=bq1, m1=m1, q2=q2, bq2=bq2, m2=m2)


def run_i8(Q, A):
    xq = np.clip(np.floor((A / Q["sx"]).astype(f32) + f32(0.5)), -128, 127).astype(np.int64)
    a1 = xq @ Q["q1"].T.astype(np.int64) + Q["bq1"]
    hq = np.clip(np.floor((a1.astype(f32) * Q["m1"]).astype(f32) + f32(0.5)), 0, 127).astype(np.int64)
    a2 = hq @ Q["q2"].T.astype(np.int64) + Q["bq2"]
    return (a2.astype(f32) * Q["m2"]).astype(f32)


def fnv(L):
    h = 2166136261
    for v in np.ascontiguousarray(L, f32).view(np.uint32).ravel():
        h = ((h ^ int(v)) * 16777619) & 0xFFFFFFFF
    return h


VARIANTS = []                                   # (имя, вид, данные для C, логиты на проверке)
for h in WIDTHS:
    P = NETS["w%d" % h]
    VARIANTS.append(("f32_w%d" % h, "f32", P, run_float(P, ZT)))
    P16 = [[W.astype(np.float16), b] for W, b in P]
    VARIANTS.append(("f16_w%d" % h, "f16", P16, run_float([[W.astype(f32), b] for W, b in P16], ZT)))
    Q = quant_i8(P)
    VARIANTS.append(("i8_w%d" % h, "i8", Q, run_i8(Q, ZT)))
VARIANTS.append(("f32_teacher", "f32", TEACH, run_float(TEACH, ZT)))
VARIANTS.append(("f32_struct8", "f32", STRUCT[8], run_float(STRUCT[8], ZT)))
VARIANTS.append(("f32_student", "f32", STUDENT, run_float(STUDENT, ZT)))
VARIANTS.append(("f32_prune90", "f32", PRUNED[0.9], run_float(PRUNED[0.9], ZT)))
for sp in (0.5, 0.75, 0.9):
    VARIANTS.append(("csr_p%d" % round(sp * 100), "csr", PRUNED[sp], run_float(PRUNED[sp], ZT)))
VARIANTS.append(("cl4_k16", "cl4", (CLUST[16][0], CLUST[16][1]), run_float(CLUST[16][0], ZT)))
VARIANTS.append(("cl4_deep", "cl4", (DEEP, DEEP_CL), run_float(DEEP, ZT)))
for name, kind, P, L in VARIANTS:
    unsure = int((softmax(L.astype(np.float64)).max(axis=1) < tau).sum())
    put("var_%s" % name, int((L.argmax(axis=1) == y[mte]).sum()), "%08x" % fnv(L), unsure)


# ══ 11. Заголовок для C ═══════════════════════════════════════════════════
def c_float(v):
    s_ = "%.9g" % v
    return (s_ if any(ch in s_ for ch in ".en") else s_ + ".0") + "f"


def c_arr(name, a, typ):
    flat = np.asarray(a).ravel()
    if typ in ("float", "half"):                # сначала во float32 (half — уже float16): тогда текст точен
        body = ", ".join(c_float(float(v)) for v in flat.astype(np.float32))
    else:
        body = ", ".join(str(int(v)) for v in flat)
    return "static const %s %s[%d] = { %s };\n" % (typ, name, len(flat), body)


def cl4_pack(a):
    v = a.ravel()
    return (v[0::2] | (v[1::2] << 4)).astype(np.uint8)


h_ = ["/* Раздел 119а: все варианты модели — сгенерировано train119b.py, не править */\n",
      "#define N_IN %d\n#define N_OUT 4\n#define N_TEST %d\n#define TAU %s\n" % (NB, int(mte.sum()), c_float(tau)),
      c_arr("MU", MU, "float"), c_arr("SD", SD, "float")]
descr, i8d = [], []
for name, kind, P, L in VARIANTS:
    u = name.upper()
    if kind == "i8":
        Q = P
        h_ += [c_arr(u + "_Q1", Q["q1"], "int8_t"), c_arr(u + "_BQ1", Q["bq1"], "int32_t"), c_arr(u + "_M1", Q["m1"], "float"),
               c_arr(u + "_Q2", Q["q2"], "int8_t"), c_arr(u + "_BQ2", Q["bq2"], "int32_t"), c_arr(u + "_M2", Q["m2"], "float")]
        i8d.append("    { \"%s\", %d, %s, %s_Q1, %s_BQ1, %s_M1, %s_Q2, %s_BQ2, %s_M2, 0x%08xu },\n"
                   % (name, Q["q1"].shape[0], c_float(float(Q["sx"])), u, u, u, u, u, u, fnv(L)))
        continue
    lay = []
    if kind == "cl4":
        P, cl = P
    for k, (W, b) in enumerate(P):
        n_out, n_in = W.shape
        relu = int(k < len(P) - 1)
        tag = "%s_L%d" % (u, k + 1)
        h_.append(c_arr(tag + "_B", b, "float"))
        if kind == "f32":
            h_.append(c_arr(tag + "_W", W, "float"))
            lay.append("{ L_F32, %d, %d, %d, %s_W, %s_B, 0, 0, 0 }" % (relu, n_in, n_out, tag, tag))
        elif kind == "f16":
            h_.append(c_arr(tag + "_W", W, "half"))
            lay.append("{ L_F16, %d, %d, %d, %s_W, %s_B, 0, 0, 0 }" % (relu, n_in, n_out, tag, tag))
        elif kind == "csr":
            nz = W != 0
            rowp = np.concatenate([[0], np.cumsum(nz.sum(axis=1))])
            h_ += [c_arr(tag + "_V", W[nz], "float"), c_arr(tag + "_COL", np.where(nz)[1], "uint8_t"),
                   c_arr(tag + "_ROWP", rowp, "uint16_t")]
            lay.append("{ L_CSR, %d, %d, %d, %s_V, %s_B, %s_ROWP, %s_COL, 0 }" % (relu, n_in, n_out, tag, tag, tag, tag))
        else:
            a, cen, _ = cl[k]
            cen = np.array([W[a == c][0] if (a == c).any() else cen[c] for c in range(len(cen))])  # центры после дообучения
            h_ += [c_arr(tag + "_IDX", cl4_pack(a), "uint8_t"), c_arr(tag + "_TAB", cen, "float")]
            lay.append("{ L_CL4, %d, %d, %d, %s_IDX, %s_B, 0, 0, %s_TAB }" % (relu, n_in, n_out, tag, tag, tag))
    h_.append("static const layer_t %s_LAYERS[%d] = { %s };\n" % (u, len(lay), ", ".join(lay)))
    descr.append("    { \"%s\", %d, %s_LAYERS, 0x%08xu },\n" % (name, len(lay), u, fnv(L)))
h_.append("static const model_t MODELS[%d] = {\n%s};\n" % (len(descr), "".join(descr)))
h_.append("static const i8model_t I8MODELS[%d] = {\n%s};\n" % (len(i8d), "".join(i8d)))
h_.append("#ifdef ML_DATA\n" + c_arr("TEST_X", X[mte], "float") + c_arr("TEST_Y", y[mte], "uint8_t") + "#endif\n")
open(OUT + "/model119b.h", "w", encoding="utf-8", newline="\n").write("".join(h_))
open(OUT + "/summary119b.txt", "w", encoding="utf-8", newline="\n").write("\n".join(lines) + "\n")
print("\n".join(lines))

infer119b.c — inference of all variants in C: float32, float16, CSR, clusters and int8.

/* Раздел 119а: вывод всех вариантов сети на C — плотный float32, веса float16, прореженный CSR,
 * кластеры (4-битные номера + таблица центров) и int8. Веса — из model119b.h (выгрузил train119b.py).
 * Вход — 64 бина спектра вибрации в дБ, ответ — 4 логита: норма, дисбаланс, перекос, подшипник.
 * На компьютере (-DML_HOST) main считает каждый вариант на проверочных окнах и сверяет логиты
 * с Python по контрольной сумме бит в бит. Собирать с -ffp-contract=off: без слитного
 * умножения-сложения порядок округлений тот же, что в numpy. Проверка — _verify/ml119b_check.py. */
#include <math.h>
#include <stdint.h>
#include <string.h>

#if defined(__arm__)
typedef __fp16 half;                    /* хранение: 2 байта, счёт — во float */
#else
typedef _Float16 half;
#endif

enum { L_F32, L_F16, L_CSR, L_CL4 };

typedef struct {
    uint8_t kind, relu;
    uint16_t n_in, n_out;
    const void *w;                      /* веса: float, half, значения CSR или упакованные номера */
    const float *b;                     /* смещения */
    const uint16_t *rowp;               /* CSR: начало строки каждого выхода */
    const uint8_t *col;                 /* CSR: номер входа каждого ненулевого веса */
    const float *tab;                   /* кластеры: 16 центров слоя */
} layer_t;

typedef struct {
    const char *name;
    uint8_t n_layers;
    const layer_t *layers;
    uint32_t hash;                      /* контрольная сумма логитов Python на проверке */
} model_t;

typedef struct {
    const char *name;
    uint16_t n_hid;
    float sx;                           /* масштаб входа int8 */
    const int8_t *q1;
    const int32_t *bq1;
    const float *m1;                    /* пересчёт суммы в масштаб скрытого слоя */
    const int8_t *q2;
    const int32_t *bq2;
    const float *m2;                    /* пересчёт суммы в логиты */
    uint32_t hash;
} i8model_t;

#if defined(ML_HOST) && !defined(ML_DATA)
#define ML_DATA
#endif
#include "model119b.h"

#define MAX_W 128

/* Плотный слой: каждый вес умножается, нули тоже */
static void dense_f32(const layer_t *L, const float *x, float *y)
{
    const float *w = (const float *)L->w;
    for (int j = 0; j < L->n_out; j++) {
        float s = L->b[j];
        for (int i = 0; i < L->n_in; i++)
            s += w[j * L->n_in + i] * x[i];
        y[j] = L->relu && !(s > 0.0f) ? 0.0f : s;
    }
}

/* Веса float16: вдвое меньше флеш-памяти, перед умножением — VCVTB в float */
static void dense_f16(const layer_t *L, const float *x, float *y)
{
    const half *w = (const half *)L->w;
    for (int j = 0; j < L->n_out; j++) {
        float s = L->b[j];
        for (int i = 0; i < L->n_in; i++)
            s += (float)w[j * L->n_in + i] * x[i];
        y[j] = L->relu && !(s > 0.0f) ? 0.0f : s;
    }
}

/* Разреженный слой CSR: только ненулевые веса, к каждому — номер входа */
static void dense_csr(const layer_t *L, const float *x, float *y)
{
    const float *v = (const float *)L->w;
    for (int j = 0; j < L->n_out; j++) {
        float s = L->b[j];
        for (int k = L->rowp[j]; k < L->rowp[j + 1]; k++)
            s += v[k] * x[L->col[k]];
        y[j] = L->relu && !(s > 0.0f) ? 0.0f : s;
    }
}

/* Кластеры: вес = таблица[номер], номер — 4 бита, два номера в байте */
static void dense_cl4(const layer_t *L, const float *x, float *y)
{
    const uint8_t *p = (const uint8_t *)L->w;
    for (int j = 0; j < L->n_out; j++) {
        float s = L->b[j];
        for (int i = 0; i < L->n_in; i++) {
            int k = j * L->n_in + i;
            s += L->tab[(p[k >> 1] >> ((k & 1) * 4)) & 15] * x[i];
        }
        y[j] = L->relu && !(s > 0.0f) ? 0.0f : s;
    }
}

/* Модель с весами float, float16, CSR или кластерами: нормировка и слои по очереди */
void model_run(const model_t *m, const float *x, float *out)
{
    float a[MAX_W], b[MAX_W], *in = a, *o = b, *t;
    for (int i = 0; i < N_IN; i++)
        a[i] = (x[i] - MU[i]) / SD[i];
    for (int l = 0; l < m->n_layers; l++) {
        const layer_t *L = &m->layers[l];
        float *dst = l == m->n_layers - 1 ? out : o;
        switch (L->kind) {
        case L_F32: dense_f32(L, in, dst); break;
        case L_F16: dense_f16(L, in, dst); break;
        case L_CSR: dense_csr(L, in, dst); break;
        default: dense_cl4(L, in, dst); break;
        }
        t = in;
        in = o;
        o = t;
    }
}

/* int8: вход и скрытый слой — байты, суммы — int32, пересчёт масштаба — одно умножение во float */
void i8_run(const i8model_t *m, const float *x, float *out)
{
    int8_t q[N_IN], h[64];
    for (int i = 0; i < N_IN; i++) {
        float z = (x[i] - MU[i]) / SD[i];
        int v = (int)floorf(z / m->sx + 0.5f);
        q[i] = (int8_t)(v < -128 ? -128 : v > 127 ? 127 : v);
    }
    for (int j = 0; j < m->n_hid; j++) {
        int32_t acc = m->bq1[j];
        for (int i = 0; i < N_IN; i++)
            acc += m->q1[j * N_IN + i] * q[i];
        int v = (int)floorf((float)acc * m->m1[j] + 0.5f);
        h[j] = (int8_t)(v < 0 ? 0 : v > 127 ? 127 : v);  /* ReLU — нижняя граница 0 */
    }
    for (int c = 0; c < N_OUT; c++) {
        int32_t acc = m->bq2[c];
        for (int j = 0; j < m->n_hid; j++)
            acc += m->q2[c * m->n_hid + j] * h[j];
        out[c] = (float)acc * m->m2[c];
    }
}

/* Уверенность: наибольшая вероятность softmax не меньше TAU, иначе ответ «не уверен» */
int confident(const float s[N_OUT])
{
    float mx = s[0], sum = 0.0f;
    for (int c = 1; c < N_OUT; c++)
        mx = s[c] > mx ? s[c] : mx;
    for (int c = 0; c < N_OUT; c++)
        sum += expf(s[c] - mx);
    return 1.0f / sum >= TAU;
}

int argmax4(const float s[N_OUT])
{
    int k = 0;
    for (int c = 1; c < N_OUT; c++)
        if (s[c] > s[k])
            k = c;
    return k;
}

/* Контрольная сумма FNV-1a по битам логитов: совпала — значит, совпало каждое число */
uint32_t fnv_add(uint32_t h, const float s[N_OUT])
{
    for (int c = 0; c < N_OUT; c++) {
        uint32_t u;
        memcpy(&u, &s[c], 4);
        h = (h ^ u) * 16777619u;
    }
    return h;
}

/* Все варианты под одним номером — чтобы ни одна таблица весов не выпала при сборке */
int n_models(void)
{
    return (int)(sizeof MODELS / sizeof MODELS[0] + sizeof I8MODELS / sizeof I8MODELS[0]);
}

void run_any(int k, const float *x, float *out)
{
    int nf = (int)(sizeof MODELS / sizeof MODELS[0]);
    if (k < nf)
        model_run(&MODELS[k], x, out);
    else
        i8_run(&I8MODELS[k - nf], x, out);
}

#ifdef ML_HOST
#include <stdio.h>

int main(void)
{
    int nf = (int)(sizeof MODELS / sizeof MODELS[0]), same = 0;
    for (int k = 0; k < n_models(); k++) {
        uint32_t h = 2166136261u, want = k < nf ? MODELS[k].hash : I8MODELS[k - nf].hash;
        int right = 0, unsure = 0;
        for (int n = 0; n < N_TEST; n++) {
            float s[N_OUT];
            run_any(k, &TEST_X[N_IN * n], s);
            h = fnv_add(h, s);
            right += argmax4(s) == TEST_Y[n];
            unsure += !confident(s);
        }
        same += h == want;
        printf("%s=%d,%08x,%d\n", k < nf ? MODELS[k].name : I8MODELS[k - nf].name, right, h, unsure);
    }
    printf("host_hash_ok=%d,%d\n", same, n_models());
    return 0;
}
#endif

Next: section 119б "The model on the board: TensorFlow Lite Micro from the inside — MicroInterpreter, op resolver, tensor arena, input buffer and the sample stream" — the model goes into the firmware: an array in flash, the kernel set, the arena, int8 input, the stream from DMA to the window, time per operator and building TFLM without make.