Embedded HandbookApps121в. STM32N6 and a battery-powered camera with a neural network: YOLO on the Neural-ART NPU, low power, an outdoor device
Section 121а covered OpenMV on the STM32H7: a frame, blobs, a small network through TensorFlow Lite Micro. YOLOv8n at 3.2 MB no longer fit there — the board table was left with the OPENMV_N6 row. This section is about that chip: the STM32N6 with the Neural-ART neural network accelerator, and the reason it goes into an outdoor device: the camera sleeps, wakes up on a motion sensor, takes a picture, finds people, animals or cars in the frame and sends a short message.
The section has three parts. The first is the STM32N6 itself and the path of a frame to the NPU, based on ST documentation and the OpenMV sources. The second is our own YOLO post-processing code for the Cortex-M55: the int8 network output is turned into boxes, and NMS removes the extra boxes. The code was run on a PC and on a Cortex-M55 in QEMU, with Helium and without, and compared against an independent numpy calculation — including the real output of the yolov8n_192 model from OpenMV on a photo of a person. The third is the energy budget on an 18650 and a requirements specification for the device.
What is not here. The section had no STM32N6 board at hand: the currents are taken from the datasheet; the phase times and the NPU, sensor and radio currents are estimates, and they are marked as such. The model was not run on the NPU — its output was obtained with the TensorFlow Lite interpreter on a PC.
References: the STM32N6x5/x7 datasheet —
DS14791 Rev 11,
the reference manual
RM0486 Rev 4,
the CMSIS header stm32n657xx.h and the SVD description from the
STM32N6 support pack. The OpenMV sources —
github.com/openmv/openmv, commit
62369938a94b: board files and examples are MIT; the ml module, the NPU wrapper
lib/stai/stai_backend.c, the ml.postprocessing post-processing and the camera
driver are under the OpenMV non-commercial license, so here they are only
paraphrased with a reference to the file and line; the ST library lib/stai/libstai is
SLA0044. int8 quantization — section 119, OpenMV and the memory pool — 121а,
DCMI and DMA — 22б, cache — 11б, sleep modes — 10 and 10а, Li-ion — 173б,
LoRa and time on air — 89, 93, 94, firmware update — 48, 48а–48г.
Terms used in the documentation
Term Meaning NPU (neural processing unit), Neural-ART accelerator a neural accelerator, ST's neural network accelerator CA (convolutional accelerator) the convolution unit in the NPU epoch a piece of the network that the NPU computes without the CPU, up to an interrupt stream engine, stream switch the accelerator's streaming DMA and the stream crossbar relocatable model a model image that can be placed at any address DCMIPP (pixel pipeline), ISP (image signal processor) the camera pixel pipeline, processing of the sensor signal CSI-2 (camera serial interface) the MIPI serial camera interface MVE (M-profile vector extension), Helium the Cortex-M55 vector instructions anchor a grid point (YOLOv8) or a box template (YOLOv2) objectness "there is an object here" NMS (non-maximum suppression) suppression of overlapping boxes IoU (intersection over union) the intersection divided by the union class-agnostic NMS NMS that ignores the class PIR (passive infrared) sensor a pyroelectric motion sensor duty cycle the share of time on air
121в.1 The STM32N6 in brief
The STM32N6 is a Cortex-M55 at 800 MHz with Helium vector instructions (MVE) and an
FPU for half, single and double precision. In the N6x7 line it is joined by
the Neural-ART NPU: up to 1 GHz, 600 Gops and 288 multiply-accumulate operations
per cycle according to the datasheet (DS14791, p. 1). We never measured the "Gops" ourselves —
this is ST's figure. Inside the NPU (RM0486, table 108): 4 convolution units, 2
pooling units, 2 activation units, 4 arithmetic units, 10 streaming DMAs with
built-in encryption, 2 AXI buses of 64 bits each. The same numbers appear in the
header of the ST library that OpenMV builds:
ATON_CONVACC_NUM 4, ATON_STRENG_NUM 10 (lib/stai/libstai/include/ATON.h).
There is no flash memory. In the part number decoding for the N6, the flash memory size is
"0 = 0–1 Kbytes" (DS14791, p. 262). After reset the ROM runs: depending on the
BOOT0 and BOOT1 pins and an OTP word, it loads the firmware from external memory —
XSPI NOR, HyperFlash, e.MMC or an SD card (RM0486, 9.1.1) — or over USB and
UART. Here is how it looks on the OpenMV N6 (boards/OPENMV_N6/board_config.mk,
lines 8–10): flash memory starts at 0x70000000, the firmware at
0x70080000, the OpenMV bootloader is in RAM at 0x34180400. The map of the external
32 MB flash memory on XSPI2 at 200 MHz is in boot_config.h
(lines 47–49 and 76–79): a 512 KB bootloader, the firmware, the file system,
ROMFS with models at 0x800000. In the CMSIS pack for the N6, "Flash" also
starts at 0x34180400 — this is the RAM where the ROM places the first-stage bootloader.
Memory. The AXI RAM consists of two 1 MB parts and four 448 KB parts
(SRAM1_AXI_SIZE … SRAM6_AXI_SIZE in stm32n657xx.h), plus
TCM 64 + 128 KB and 8 KB of backup RAM. The four 448 KB blocks starting at
0x34200000 are the RAM given to the NPU: in OpenMV this is a 1792 KB pool
(board_config.h, lines 172–174 and 187–188), in the ST compiler it is the pools
npuRAM3…npuRAM6 (lib/stai/scripts/stm32n6.mpool). The NPU cache for
external memory (CACHEAXI) is 256 KB (CACHEAXI_RAM_SIZE 0x40000UL).
The model weights live in external flash memory and are read in place; frames and
large buffers in OpenMV are in the external 64 MB PSRAM at 0x90000000
(lines 191–192, the 31 MB UMA pool — line 164).
Camera. The N6 has two paths for a frame (RM0486, 37.1): the older DCMI up to
24 Mpixel/s, and the DCMIPP — up to 5 MP at 30 frames/s, parallel up to 120 MHz
or over CSI-2 up to 2.5 Gbit/s per lane on two lanes, with an ISP (demosaicing,
exposure, white balance, contrast, bad pixels) and two application
pipelines with cropping and downscaling. The ISP output can go to the NPU over DMA
(DS14791, p. 14). The OpenMV N6 works through the DCMIPP: pipeline 1, a
24 MHz sensor clock from TIM1, and the CSI-2 path for MIPI sensors
(board_config.h, lines 134 and 307; ports/stm32/omv_csi.c).
Video and JPEG. A hardware JPEG codec and an H.264 encoder up to 1080p at 15
frames/s (DS14791, p. 1). In the SVD these are the JPEG block at 0x48023000 and
VENC ("Video encoder") at 0x48005000; in OpenMV the encoder is connected through
drivers/vc8000 (H264EncInit). A battery-powered device does not
need video — it needs a JPEG crop.
| STM32H743 (121а) | STM32U585 (198х) | STM32N657 | |
|---|---|---|---|
| core | Cortex-M7, 480 MHz | Cortex-M33, 160 MHz | Cortex-M55, 800 MHz, Helium |
| neural network | on the core | on the core | NPU, 600 Gops per ST |
| flash memory | 2 MB internal | 2 MB internal | none, external over XSPI |
| camera | DCMI | DCMI | DCMI and DCMIPP with ISP, CSI-2 |
| sleep with clock running | section 10 | Stop 2, Standby (10а) | Standby 90.4 µA at 1.8 V |
121в.2 Frame, NPU and YOLO output
How the model gets into the NPU. The .tflite model is run through the ST
Edge AI compiler: the OpenMV settings are in lib/stai/scripts/neuralart.json (a profile
with --enable-epoch-controller, pools from stm32n6.mpool, on chip —
up to 1024 KB). The result is a relocatable image: epoch code and weights. The network
is computed in epochs — pieces that the NPU goes through on its own, from being set up
by the CPU to an interrupt (RM0486, 20.1). What OpenMV does
(lib/stai/stai_backend.c, paraphrased):
- enables the clock and reset of the NPU and its cache (
ml_backend_npu_init); - moves the model image to external memory if it landed in internal memory:
the relocatable NPU pools must be in external memory — the boundary is
0x60000000(lines 53–58); - installs the model in XIP mode — the epoch code executes directly from the image (line 183) — and cleans the data cache: the NPU accesses memory bypassing the Cortex-M55 cache (section 11б);
- before a run it cleans the input buffers from the cache, spins
LL_ATON_RT_RunEpochBlock, and while the NPU is busy the core waits inWFE(lines 291–307); the output buffer is invalidated in the cache before reading (line 351).
YOLOv8 output. The yolov8n_192.tflite model from OpenMV lib/models is
3228400 bytes, input 192×192×3 uint8. Inside it is int8; at the output the
interpreter converts the last tensor to float. If you cut off this
conversion, the NPU returns an int8 tensor of shape [1, 5, 756] with scale
0.00507508637 and zero point −128. 756 is the anchors of three grids
24×24 + 12×12 + 6×6 (strides of 8, 16, 32 pixels). Rows 0–3 are the box center and size
as fractions of the input, row 4 is the score of the person class, already after the
sigmoid. The scale is shared by all five rows: a threshold of 0.4 is code −49,
a score of 0.9693 is code 63.
The second OpenMV model, yolo_lc_192.tflite (339448 bytes), is a
YOLOv2 variant: output [1, 12, 12, 30] int8 with scale 0.176762134 and zero
point 73: a 12×12 grid, 5 anchors of 6 numbers each (tx, ty, tw, th,
objectness, class). Here everything is logits: you need a sigmoid and an exponential, and
you need the anchor sizes the model was trained with.
Where to compute the post-processing. In OpenMV it is done in Python (ulab) in
ml.postprocessing.ultralytics.YoloV8: the threshold is converted to codes and
compared with the raw bytes (ultralytics.py, lines 108 and 116);
only the anchors that pass are converted to float. NMS in OpenMV is "soft":
the neighbors' scores are multiplied by exp(−IoU²/σ), classes are not distinguished
(ml-core/ml/utils.py, lines 91 and 119). In a device without MicroPython this
work is done by our own C function — it is below.
121в.3 Post-processing on the Cortex-M55
The task: get a few boxes out of 756 anchors (or 720 for YOLOv2). The rules of our code:
- the threshold is in int8 codes. The smallest code whose score is not below the threshold is found once by trying all 256 values; after that, bytes are compared;
- no more than 64 candidates. The list is kept in descending order of score; if it is full, a new one evicts the worst — memory stays constant with any amount of noise in the frame;
- NMS is greedy, per class (or class-agnostic — the
agnosticflag), IoU is in integers:inter · den > union · num, without division; - our own exponential, without libm: only float addition and multiplication, so the answer is the same on the PC and on the M55;
- Helium in the candidate search: 16 anchors per pass.
Threshold in codes:
/* Наименьший код q, у которого оценка уже не ниже порога. Сравнивать дальше можно
* прямо байты выхода: перевод в float нужен только прошедшим. Недостижимый порог
* даёт 127 — такой случай отсекает thr_reachable. */
int8_t yp_q_threshold(const yp_cfg_t *c, int logistic)
{
for (int q = -128; q <= 127; q++) {
float v = deq(c, q);
if (logistic) v = sigm(v);
if (v >= c->conf) return (int8_t)q;
}
return (int8_t)127;
}
Candidate list:
/* Кандидаты — по убыванию q; при равных раньше найденный выше. Полный список
* вытесняет худшего: в памяти не больше YP_MAX_CAND рамок при любом шуме. */
static void put(yp_box_t *cand, int *cnt, const yp_box_t *b)
{
int k = *cnt;
if (k == YP_MAX_CAND) {
if (b->q <= cand[k - 1].q) return;
k--;
}
while (k > 0 && cand[k - 1].q < b->q) {
cand[k] = cand[k - 1];
k--;
}
cand[k] = *b;
if (*cnt < YP_MAX_CAND) (*cnt)++;
}
The YOLOv8 decoder. On the M55 the loop goes 16 anchors at a time: vld1q_s8 loads 16
class scores, vmaxq_s8 takes the maximum over the classes, vcmpgeq_n_s8
compares with the threshold and gives a mask — one bit per anchor; only the set bits
are processed. The tail (756 = 47 · 16 + 4) and the build without MVE go through
an ordinary loop in the same order, so the result is identical:
static int8_t v8_best(const int8_t *out, int n, int nc, int i, int *best)
{
const int8_t *s = out + 4 * n + i;
int8_t q = s[0];
*best = 0;
for (int k = 1; k < nc; k++)
if (s[k * n] > q) {
q = s[k * n];
*best = k;
}
return q;
}
static void v8_take(const int8_t *out, int n, int nc, int i, const yp_cfg_t *c,
yp_box_t *cand, int *cnt)
{
int best;
yp_box_t b;
int8_t q = v8_best(out, n, nc, i, &best);
if (!make_box(deq(c, out[i]), deq(c, out[n + i]), deq(c, out[2 * n + i]),
deq(c, out[3 * n + i]), c, &b))
return;
b.q = q;
b.cls = (uint8_t)best;
put(cand, cnt, &b);
}
int yp_decode_v8(const int8_t *out, int n, int nc, const yp_cfg_t *c,
yp_box_t *cand, int *n_above)
{
int8_t t = yp_q_threshold(c, 0);
int cnt = 0, above = 0, i = 0;
if (!thr_reachable(c, 0, t)) {
*n_above = 0;
return 0;
}
#ifdef YP_USE_MVE
for (; i + 16 <= n; i += 16) { /* 16 якорей за проход */
int8x16_t m = vld1q_s8(out + 4 * n + i);
for (int k = 1; k < nc; k++)
m = vmaxq_s8(m, vld1q_s8(out + (4 + k) * n + i));
uint32_t p = vcmpgeq_n_s8(m, t); /* бит на каждый байт-якорь */
while (p) {
int lane = __builtin_ctz(p);
p &= p - 1;
above++;
v8_take(out, n, nc, i + lane, c, cand, &cnt);
}
}
#endif
for (; i < n; i++) {
int best;
if (v8_best(out, n, nc, i, &best) >= t) {
above++;
v8_take(out, n, nc, i, c, cand, &cnt);
}
}
*n_above = above;
return cnt;
}
The inner loop over classes after arm-none-eabi-gcc 12.2.1
(-mcpu=cortex-m55 -O2), disassembled with -m armv8.1-m.main:
4a6: ed93 7e00 vldrb.u8 q3, [r3, #0]
4aa: dd0c ble.n 4c6 <yp_decode_v8+0xc6>
4ac: f10b 3eff add.w lr, fp, #4294967295 @ 0xffffffff
4b0: f04e e001 dls lr, lr
4b4: 9b08 ldr r3, [sp, #32]
4b6: 4443 add r3, r8
4b8: ed93 5e00 vldrb.u8 q2, [r3, #0]
4bc: 4453 add r3, sl
4be: ef06 6644 vmax.s8 q3, q3, q2
4c2: f00f c807 le lr, 4b8 <yp_decode_v8+0xb8>
4c6: 9b05 ldr r3, [sp, #20]
4c8: fe07 1f43 vcmp.s8 ge, q3, r3
4cc: eefd 4a10 vmrs r4, p0
dls/le is the M55 hardware loop; vcmp.s8 writes the mask to p0, and
vmrs picks it up.
NMS and IoU in integers:
/* Пересечение больше доли num/den от объединения — в целых, без деления. */
static int iou_above(const yp_box_t *a, const yp_box_t *b, const yp_cfg_t *c)
{
int ax1 = a->x + a->w, bx1 = b->x + b->w, ay1 = a->y + a->h, by1 = b->y + b->h;
int ix = (ax1 < bx1 ? ax1 : bx1) - (a->x > b->x ? a->x : b->x);
int iy = (ay1 < by1 ? ay1 : by1) - (a->y > b->y ? a->y : b->y);
if (ix <= 0 || iy <= 0) return 0;
int64_t inter = (int64_t)ix * iy;
int64_t uni = (int64_t)a->w * a->h + (int64_t)b->w * b->h - inter;
return inter * c->iou_den > uni * c->iou_num;
}
int yp_nms(yp_box_t *b, int n, const yp_cfg_t *c)
{
int kept = 0;
for (int i = 0; i < n; i++) {
int drop = 0;
for (int j = 0; j < kept && !drop; j++)
if ((c->agnostic || b[j].cls == b[i].cls) && iou_above(&b[j], &b[i], c))
drop = 1;
if (!drop) b[kept++] = b[i];
}
return kept;
}
The exponential for YOLOv2 (the sigmoid is 1 / (1 + exp(−x))):
/* exp(x) = 2^m * exp(r), r = x - m*ln2, |r| <= 0,35; ряд до r^6. Только + - * и сдвиг
* порядка — результат одинаков на любом процессоре с IEEE-754 float. */
float yp_exp(float x)
{
if (x > 88.0f) x = 88.0f;
if (x < -87.0f) x = -87.0f;
float k = x * 1.44269504f;
int m = (int)(k + (k >= 0.0f ? 0.5f : -0.5f));
float r = x - (float)m * 0.693145752f; /* ln2, старшая часть */
r = r - (float)m * 1.42860677e-06f; /* ln2, младшая часть */
float p = 1.0f + r * (1.0f + r * (0.5f + r * (0.166666672f + r * (0.0416666679f
+ r * (0.00833333377f + r * 0.00138888892f)))));
union { float f; uint32_t u; } s;
s.u = (uint32_t)(m + 127) << 23; /* 2^m */
return p * s.f;
}
The YOLOv2 decoder (yp_decode_v2) works the same way: the objectness threshold is in
codes, the class is the argmax of the raw logits (softmax does not change it), the center is
(cell + sigmoid) / grid, the size is anchor · exp(t) / grid. It is
given in full at the end of the section.
121в.4 Testing: synthetic data, the real model, the M55 in QEMU
The test sets are in gen_tensors.py: synthetic outputs with our own generator and
the real yolov8n_192 output. The real one was obtained like this: the photo
scripts/unittest/data/person.bmp from OpenMV (400×400, RGB565) was scaled down
to 192×192 and run through the TensorFlow Lite 2.21.0 interpreter with the
reference kernels; the int8 tensor before the conversion to float was taken. The best
anchor has a score of 0.9693, a box covering the whole frame; 10 anchors are above the 0.4 threshold.
The same main_post.c was built with gcc 12.2.0 on the PC and
arm-none-eabi-gcc 12.2.1 for the Cortex-M55 — with Helium and with -DYP_NO_MVE.
Both M55 builds were run in QEMU 7.2 on the mps3-an547 machine (Cortex-M55),
with output over semihosting. Output on the PC:
CASE s1_three kind=8 conf=40 agn=0 thr=-49 above=17 cand=17 kept=3
BOX cls=0 q=45 x=44 y=58 w=29 h=79
BOX cls=1 q=30 x=100 y=117 w=69 h=39
BOX cls=2 q=10 x=93 y=48 w=22 h=19
CASE s2_overlap kind=8 conf=40 agn=0 thr=-49 above=11 cand=11 kept=2
BOX cls=0 q=49 x=62 y=60 w=59 h=103
BOX cls=1 q=20 x=67 y=55 w=57 h=90
CASE s2_overlap_agn kind=8 conf=40 agn=1 thr=-49 above=11 cand=11 kept=1
BOX cls=0 q=49 x=62 y=60 w=59 h=103
CASE s3_flood kind=8 conf=40 agn=0 thr=-49 above=240 cand=64 kept=58
CASE s4_v2 kind=2 conf=50 agn=0 thr=73 above=6 cand=6 kept=2
BOX cls=0 q=84 x=46 y=30 w=40 h=91
BOX cls=1 q=81 x=104 y=109 w=79 h=48
CASE r1_person_40 kind=8 conf=40 agn=0 thr=-49 above=10 cand=10 kept=1
BOX cls=0 q=63 x=0 y=0 w=190 h=190
CASE r1_person_25 kind=8 conf=25 agn=0 thr=-78 above=10 cand=10 kept=1
BOX cls=0 q=63 x=0 y=0 w=190 h=190
DONE
s1_three— three objects of three classes, each with 6 neighboring anchors above the threshold: 17 candidates, 3 boxes after NMS. Code 45 = score 0.88.s2_overlap— a person and a car in almost the same box: per-class NMS keeps both; class-agnostic NMS (agn=1, as in OpenMV) keeps only the person.s3_flood— 240 anchors above the threshold: 64 candidates, 58 after NMS.s4_v2— YOLOv2 on a 12×12 grid with three anchors of our own: 6 candidates, 2 boxes.r1_person_40andr1_person_25— the real model: 10 candidates, one 190×190 box with code 63 (0.9693) at thresholds 0.4 and 0.25. The OpenMV testscripts/unittest/tests/ml_yolov8.pyexpects the same on this photo: exactly one box with a score of at least 0.9 (lines 16–20).
Comparison:
| What | Against what | Result |
|---|---|---|
| C output on the PC | yolo_post.py — the same logic, every operation rounded to float32 |
matched line by line |
| C output on the PC | Cortex-M55 in QEMU, with Helium and without | matched line by line, both builds |
| C output on the PC | numpy 2.3.3: the whole tensor in float, np.exp, IoU as a matrix |
7 sets out of 7, boxes — a difference of 0 pixels |
yp_exp |
math.exp on −20…20 in steps of 0.01 |
relative error up to 1.15·10⁻⁶ |
Code size of yolo_post.c for the Cortex-M55 (-O2): 2488 bytes with
Helium and 2328 without; no data and no .bss. Of these,
yp_decode_v8 is 404 and 414 bytes, yp_decode_v2 is 684,
yp_nms is 374, yp_exp is 332. Instructions using the q vector
registers: 0 in the build without Helium, 4 with Helium: two
vldrb.u8, vmax.s8 and vcmp.s8 from the listing above. How many cycles this takes on a real M55 was not measured:
QEMU does not count cycles.
121в.5 Energy budget on an 18650
The cycle: Standby → wake-up by PIR or RTC alarm → boot from
external flash memory → sensor and exposure → NPU → post-processing →
LoRa → Standby. Plus an hourly timer "heartbeat" without the camera. The model is
energy121v.py; the current on the 1.8 V rail is converted to battery current through the
converter efficiency: I_bat = I · 1.8 / (3.6 · 0.85).
What comes from the documentation (DS14791, 25 °C, typical, internal SMPS):
| Mode | Current at 1.8 V | Where |
|---|---|---|
| Run at 800 MHz, all peripherals clocked | 99.93 mA (IDDSMPS) | table 40, p. 155 |
| Sleep, 800 MHz, everything clocked | 80.9 mA | table 42, p. 157 |
| Stop, SVOS low | 3.60 mA + 273.4 µA + 362.7 µA = 4.24 mA | table 44, p. 158 |
| Standby, backup RAM and RTC on LSE | 48.4 + 42 = 90.4 µA | table 45, p. 159 |
| the same at 85 °C | 69.0 + 80 = 149 µA | table 45, p. 159 |
| from Standby to the first instruction | 400 µs | table 47, p. 160 |
The core at 800 MHz alone draws 85 mA from VDDCORE with the peripherals off and 172 mA with them on (table 34, p. 149). The datasheet does not give the NPU current during computation.
What is estimated (measure it on your own board): NPU current 200 mA at 1.8 V,
boot 0.3 s, exposure 1.0 s (that is how long the OpenMV example
time_lapse_photos.py waits for the sensor, line 55: snapshot(time=1000)), NPU 0.1 s,
post-processing 0.05 s, image sensor 150 mW, external memory 20 mA,
LoRa — 45 mA when transmitting and 12 mA when receiving for 0.2 s, LTE-M — 120 mA for 6 s,
Wi-Fi — 180 mA for 2.5 s, PIR 15 µA, everything else in sleep 12 µA, 18650 —
3000 mAh, 90 % available at +20 °C and 60 % at −20 °C, self-discharge 2 % per
month. The script itself prints all the assumptions with their sources.
The LoRa message is 12 bytes of event and 13 bytes of LoRaWAN overhead: at SF9 and 125 kHz this is 205.8 ms on air. The formula is from section 89, and the script checks it: 23 bytes at SF9 — 205.8 ms, as in the table of section 89; 63 bytes at SF7 — 118.0 ms, as in 48в.
Run results:
| Value | |
|---|---|
| whole-board sleep in Standby | 80.2 µA from the battery, 1.92 mAh per day |
| the same when sleeping in Stop | 2518.8 µA, 60.5 mAh per day |
| one trigger with LoRa | 175.0 mA·s: sensor and exposure 96.6, radio 35.5, boot 21.2, NPU 18.8 |
| one trigger with LTE-M / Wi-Fi | 1145.1 / 708.5 mA·s |
| from PIR to the end of transmission | 1.66 s |
How many days the device will last:
| Link, triggers per day | mAh per day | 1×18650, +20 °C | 1×18650, −20 °C | 2×18650, −20 °C |
|---|---|---|---|---|
| LoRa, 10 | 2.8 | 567 | 378 | 533 |
| LoRa, 50 | 4.7 | 403 | 268 | 414 |
| LoRa, 200 | 12.0 | 193 | 129 | 225 |
| LTE-M, 10 | 5.5 | 362 | 241 | 381 |
| LTE-M, 50 | 18.2 | 134 | 89 | 162 |
| LTE-M, 200 | 65.9 | 40 | 27 | 52 |
| Wi-Fi, 10 | 4.2 | 433 | 288 | 437 |
| Wi-Fi, 50 | 12.1 | 191 | 128 | 223 |
| Wi-Fi, 200 | 41.6 | 62 | 41 | 79 |
Conclusions from the model. Sleep only in Standby: Stop on the N6 costs milliamps: with LoRa at 50 triggers, one 18650 at +20 °C lasts 41 days instead of 403. At 50 triggers, sleep and exposure take two thirds of the daily charge; the NPU is a small share if the network computes for 0.1 s. The radio decides the choice of link: one trigger over LTE-M is 6.5 times more expensive than over LoRa. At low load the lifetime is limited by self-discharge — 2.0 mAh per day per cell.
Sun. A 2 W panel, 0.5 h of full sun per day in December, 5 h in June, efficiency 0.7 (estimates): an intake of 194 and 1944 mAh per day. The December intake would be enough for 3911 LoRa triggers per day — if you can use it. In frost you cannot: Li-ion is charged only within the allowed temperature range (173б), and in winter the panel's energy during the day is lost while the cell is cold. So the winter reserve is calculated without sun — the "−20 °C" column of the table.
121в.6 Requirements specification for the device
Abridged, following the structure of GOST 15.016 (specification for a product). The numbers in the requirements are the author's proposals; the references to the 121в.5 model are there to show where the margin comes from.
1. Name and purpose. "Outdoor AI camera on the STM32N6 with battery power". Detects people, animals and cars in the field of view, transmits an event (class, number of objects, box, time, charge) and, on request, a JPEG crop. It does not record video and does not store frames.
2. Basis and stages. Under a contract; the stages are in item 11.
3. Composition.
| Unit | Requirement |
|---|---|
| processor | STM32N657 (N6x7 line with NPU); external XSPI NOR flash memory of 32 MB or more for the firmware, models and log |
| image sensor | CSI-2 or parallel, global or rolling shutter, IR illumination at the customer's request |
| motion sensor | PIR on the WKUP pin; the second wake-up source is the RTC alarm |
| communication | LoRaWAN (main) and LTE-M or Wi-Fi (depending on the version) |
| power | 1 or 2 protected 18650 cells; a solar charge controller with charging disabled by temperature (173б) |
| enclosure | outdoor, with a glass window in front of the lens and a PIR window |
4. Functional requirements.
| No. | Requirement | Verification |
|---|---|---|
| F1 | classes: person, animal, car; int8 model on the NPU | during testing |
| F2 | share of detected people in the zone up to 10 m in daytime — at least 0.9; false events — no more than 5 per day at the site | field tests |
| F3 | from PIR trigger to end of transmission — no more than 3 s (1.66 s per the model) | measurement |
| F4 | the score threshold and the NMS threshold are set remotely; per-class NMS | command test |
| F5 | hourly heartbeat: charge, temperature, counters | server log |
| F6 | the network output is checked at all thresholds: no more than 64 candidates, no memory allocation | unit tests (121в.4) |
5. Power and autonomy.
| No. | Requirement | Where the margin comes from |
|---|---|---|
| P1 | sleep only in Standby; board sleep current no more than 100 µA from the battery at +20 °C | 80.2 µA per the model |
| P2 | at least 365 days on 2×18650 at 50 LoRa triggers per day and −20 °C without sun | 414 days per the model |
| P3 | LTE-M version: at least 90 days on 2×18650 at 50 triggers and −20 °C | 162 days per the model |
| P4 | with a panel: operation without replacing the cells at 50 triggers per day all year round | December intake 194 mAh with consumption 4.7 |
| P5 | charge measurement, a low-battery message 14 days before shutdown | testing |
6. Operating conditions. Air temperature from −40 to +50 °C when running on cells that allow this range; the chip has the industrial range −40…125 °C (DS14791, p. 262). Enclosure protection — IP66 per GOST 14254. No condensation on the glass: a pressure equalization vent.
7. Communication. LoRaWAN EU868 class A, the event message is 12 bytes; duty cycle no more than 1 % (89, 93). LTE-M with PSM mode. Messages are signed with the LoRaWAN network keys (94) or TLS.
8. Firmware and model updates. Our own bootloader or MCUboot with two slots and rollback (48, 48б), image signing (48а), encryption (48г). Over LoRa — only delta firmware updates (48в). The model is not updated over LoRa: 3228400 bytes is 67259 frames of 48 bytes, at SF7 with 1 % on air no faster than 9.2 days, at SF12 — 217.5 days. Models are updated over LTE-M, Wi-Fi or on site.
9. Data storage and privacy. A frame lives only in RAM until the end of the cycle. Only metadata goes out; a crop only at the operator's request. Images of people can be personal data (152-FZ): the "metadata only" mode is the default. Keys are in OTP and a protected area, debug is closed (DS14791: life cycle with RDP and a debug password).
10. Tests. Unit: post-processing on the PC, in QEMU M55 and on the board — matching output (121в.4). Energy: sleep current, trigger charge by phase, comparison with the 121в.5 model. Climatic: −40, −20, +50 °C, cycles. Enclosure protection IP66. Field: a month at a site with event labeling, detection and false-trigger rates.
11. Stages.
| Stage | Result |
|---|---|
| 1. Mock-up | N6 evaluation board, sensor, PIR, LoRa; current measurements of all phases, a refined 121в.5 model |
| 2. Model | training and quantization of our own classes, compiling for the NPU, epoch timing |
| 3. Prototype | board, enclosure, firmware with update and logging |
| 4. Testing | items 10; test reports |
| 5. Documentation | manual, test procedure, product passport |
121в.7 Pitfalls
- Stop on the N6 is not a sleep mode for a battery. 4.24 mA at 1.8 V versus 90.4 µA in Standby (DS14791, tables 44 and 45). The habit from the U5 of sleeping in Stop 2 eats an 18650 here in 41 days.
- Leaving Standby is a reset. The ROM loads the firmware from
external flash memory again; after
machine.deepsleep()OpenMV restarts the whole script (onoff_wakeup.py, lines 11–13). It can be shortened: the TCM 64 + 128 KB is retained in Standby by thePWR_CR4_TCMRBSENbit, and on exit from Standby with retained RAM the ROM does not run — the core starts from the address in SYSCFG (RM0486, 9.2 and 13). The model weights are still read from flash memory. - A float threshold on every anchor. 756 float conversions and comparisons for the sake of a dozen that pass. Convert the threshold to a code once, then compare bytes; on the M55 — 16 per instruction.
- An unbounded candidate list. On a noisy frame there are 240 anchors
above the threshold (
s3_flood); without a limit the list grows with the noise. A limit of 64 and evicting the worst keep memory constant. - Class-agnostic NMS swallows neighbors. A person next to a car: class-agnostic NMS
kept one box out of two (
s2_overlap_agn). OpenMV's NMS is exactly the class-agnostic kind. - Cache and the NPU. The NPU reads memory bypassing the M55 cache: the input buffer is
cleaned before a run, and the output buffer is invalidated before reading —
as in
stai_backend.c(section 11б). - Models only in external memory. The ST library does not accept relocatable NPU pools below
0x60000000(stai_backend.c, lines 53–58). - YOLOv2 anchors are part of the model. Without the anchor sizes the network was trained with, the boxes come out the wrong size. Store them together with the model.
- Different results on the PC and on the board.
expffrom different libm builds and fusing a multiplication with an addition into an FMA change the last bits, and with them whole pixels of a box. Our own exponential and-ffp-contract=offgive a line-for-line match. - Charging in frost. The panel provides energy in the daytime, but Li-ion must not be charged in frost (173б): the winter reserve is calculated without sun.
- A model over LoRa. 67259 frames and 9.2 days at SF7 — that is not an update. Models are changed over LTE-M, Wi-Fi or on site.
121в.8 Summary
- STM32N6: Cortex-M55 at 800 MHz with Helium, Neural-ART NPU up to 1 GHz (600 Gops per ST; 4 convolution units, 10 streaming DMAs), no flash memory — firmware from XSPI; RAM 2 × 1 MB + 4 × 448 KB, DCMIPP with ISP, CSI-2 on 2 lanes at 2.5 Gbit/s, H.264 and JPEG.
- yolov8n_192: 3228400 bytes, int8 output [5 × 756], scale 0.00507508637, zero point −128; a threshold of 0.4 is code −49.
- Our own post-processing: threshold in codes, up to 64 candidates, NMS in integers, our own exponential; 2488 bytes of code for the M55 with Helium, 2328 without.
- Testing: 7 sets, PC, Python with float32 and M55 in QEMU with Helium and without — the output matched; numpy — 7 out of 7, boxes with no differences; on the real model output — one box with a score of 0.9693, as the OpenMV test expects.
- Energy: Standby 90.4 µA at 1.8 V, Stop 4.24 mA; board sleep 80.2 µA; a trigger with LoRa — 175.0 mA·s; 50 triggers per day — 4.7 mAh, on 2×18650 at −20 °C — 414 days (estimate).
- Specification: Standby, LoRa for events, models not over LoRa, do not store frames, calculate the autonomy margin without winter sun.
Full sources
Directory _verify/n6cam121v/. To build on a PC:
gcc -std=c11 -O2 -ffp-contract=off main_post.c yolo_post.c; for the M55 —
as in run.sh. tensors.h is generated by python gen_tensors.py tensors.h.
yolo_post.h — the post-processing interface.
/* Постобработка детектора YOLO на Cortex-M55: выход сети int8 -> рамки -> NMS.
* Свой код раздела 121в. Без malloc и без libm: одинаковый результат на ПК и на M55. */
#ifndef YOLO_POST_H
#define YOLO_POST_H
#include <stdint.h>
#define YP_MAX_CAND 64 /* сколько кандидатов держим до NMS: лучшие по оценке */
typedef struct {
int16_t x, y, w, h; /* рамка в точках входа сети */
int8_t q; /* оценка в кодах int8 выходного тензора */
uint8_t cls; /* номер класса */
} yp_box_t;
typedef struct {
float scale; /* квантование выхода: значение = (q - zp) * scale */
int32_t zp;
int in_w, in_h; /* размер входа сети, точек */
float conf; /* порог оценки, 0..1 */
int iou_num, iou_den; /* порог NMS дробью: 45/100 = 0,45 */
int agnostic; /* 1 — одно NMS на все классы, 0 — своё на каждый */
} yp_cfg_t;
/* YOLOv8 (выгрузка Ultralytics): out[(4 + nc) * n], строки cx, cy, w, h (доли входа),
* дальше nc строк оценок классов (уже после сигмоиды); n — число якорей. */
int yp_decode_v8(const int8_t *out, int n, int nc, const yp_cfg_t *c,
yp_box_t *cand, int *n_above);
/* YOLOv2 и YOLO-LC: out[gh][gw][na * (5 + nc)]: tx, ty, tw, th, объектность, nc логитов.
* anchors[2 * na] — ширина и высота якоря в клетках сетки. */
int yp_decode_v2(const int8_t *out, int gh, int gw, int na, int nc, const float *anchors,
const yp_cfg_t *c, yp_box_t *cand, int *n_above);
/* Жадное NMS на месте: рамки уже по убыванию q; возвращает, сколько осталось. */
int yp_nms(yp_box_t *b, int n, const yp_cfg_t *c);
int8_t yp_q_threshold(const yp_cfg_t *c, int logistic); /* порог в кодах int8 */
float yp_exp(float x); /* своя экспонента, без libm */
#endif
yolo_post.c — YOLOv8 and YOLOv2 decoders, threshold, candidates, NMS, exponential.
/* Постобработка YOLO: декодирование выхода int8 в рамки и NMS. Раздел 121в.
* Собирается gcc на ПК и arm-none-eabi-gcc -mcpu=cortex-m55; на M55 поиск
* кандидатов YOLOv8 идёт по 16 якорей за команду Helium (MVE). */
#include "yolo_post.h"
#if defined(__ARM_FEATURE_MVE) && !defined(YP_NO_MVE)
#include <arm_mve.h>
#define YP_USE_MVE 1
#endif
static float deq(const yp_cfg_t *c, int q) { return (float)(q - c->zp) * c->scale; }
/* exp(x) = 2^m * exp(r), r = x - m*ln2, |r| <= 0,35; ряд до r^6. Только + - * и сдвиг
* порядка — результат одинаков на любом процессоре с IEEE-754 float. */
float yp_exp(float x)
{
if (x > 88.0f) x = 88.0f;
if (x < -87.0f) x = -87.0f;
float k = x * 1.44269504f;
int m = (int)(k + (k >= 0.0f ? 0.5f : -0.5f));
float r = x - (float)m * 0.693145752f; /* ln2, старшая часть */
r = r - (float)m * 1.42860677e-06f; /* ln2, младшая часть */
float p = 1.0f + r * (1.0f + r * (0.5f + r * (0.166666672f + r * (0.0416666679f
+ r * (0.00833333377f + r * 0.00138888892f)))));
union { float f; uint32_t u; } s;
s.u = (uint32_t)(m + 127) << 23; /* 2^m */
return p * s.f;
}
static float sigm(float x) { return 1.0f / (1.0f + yp_exp(-x)); }
/* Наименьший код q, у которого оценка уже не ниже порога. Сравнивать дальше можно
* прямо байты выхода: перевод в float нужен только прошедшим. Недостижимый порог
* даёт 127 — такой случай отсекает thr_reachable. */
int8_t yp_q_threshold(const yp_cfg_t *c, int logistic)
{
for (int q = -128; q <= 127; q++) {
float v = deq(c, q);
if (logistic) v = sigm(v);
if (v >= c->conf) return (int8_t)q;
}
return (int8_t)127;
}
static int thr_reachable(const yp_cfg_t *c, int logistic, int8_t t)
{
float v = deq(c, t);
return (logistic ? sigm(v) : v) >= c->conf;
}
static float clampf(float v, float hi) { return v < 0.0f ? 0.0f : (v > hi ? hi : v); }
/* Центр и размер в долях входа -> рамка в точках; пустая рамка не нужна. */
static int make_box(float cx, float cy, float w, float h, const yp_cfg_t *c, yp_box_t *b)
{
float W = (float)c->in_w, H = (float)c->in_h;
float x0 = clampf((cx - w * 0.5f) * W, W), x1 = clampf((cx + w * 0.5f) * W, W);
float y0 = clampf((cy - h * 0.5f) * H, H), y1 = clampf((cy + h * 0.5f) * H, H);
int bw = (int)(x1 - x0), bh = (int)(y1 - y0);
if (bw <= 0 || bh <= 0) return 0;
b->x = (int16_t)(int)x0;
b->y = (int16_t)(int)y0;
b->w = (int16_t)bw;
b->h = (int16_t)bh;
return 1;
}
/* Кандидаты — по убыванию q; при равных раньше найденный выше. Полный список
* вытесняет худшего: в памяти не больше YP_MAX_CAND рамок при любом шуме. */
static void put(yp_box_t *cand, int *cnt, const yp_box_t *b)
{
int k = *cnt;
if (k == YP_MAX_CAND) {
if (b->q <= cand[k - 1].q) return;
k--;
}
while (k > 0 && cand[k - 1].q < b->q) {
cand[k] = cand[k - 1];
k--;
}
cand[k] = *b;
if (*cnt < YP_MAX_CAND) (*cnt)++;
}
static int8_t v8_best(const int8_t *out, int n, int nc, int i, int *best)
{
const int8_t *s = out + 4 * n + i;
int8_t q = s[0];
*best = 0;
for (int k = 1; k < nc; k++)
if (s[k * n] > q) {
q = s[k * n];
*best = k;
}
return q;
}
static void v8_take(const int8_t *out, int n, int nc, int i, const yp_cfg_t *c,
yp_box_t *cand, int *cnt)
{
int best;
yp_box_t b;
int8_t q = v8_best(out, n, nc, i, &best);
if (!make_box(deq(c, out[i]), deq(c, out[n + i]), deq(c, out[2 * n + i]),
deq(c, out[3 * n + i]), c, &b))
return;
b.q = q;
b.cls = (uint8_t)best;
put(cand, cnt, &b);
}
int yp_decode_v8(const int8_t *out, int n, int nc, const yp_cfg_t *c,
yp_box_t *cand, int *n_above)
{
int8_t t = yp_q_threshold(c, 0);
int cnt = 0, above = 0, i = 0;
if (!thr_reachable(c, 0, t)) {
*n_above = 0;
return 0;
}
#ifdef YP_USE_MVE
for (; i + 16 <= n; i += 16) { /* 16 якорей за проход */
int8x16_t m = vld1q_s8(out + 4 * n + i);
for (int k = 1; k < nc; k++)
m = vmaxq_s8(m, vld1q_s8(out + (4 + k) * n + i));
uint32_t p = vcmpgeq_n_s8(m, t); /* бит на каждый байт-якорь */
while (p) {
int lane = __builtin_ctz(p);
p &= p - 1;
above++;
v8_take(out, n, nc, i + lane, c, cand, &cnt);
}
}
#endif
for (; i < n; i++) {
int best;
if (v8_best(out, n, nc, i, &best) >= t) {
above++;
v8_take(out, n, nc, i, c, cand, &cnt);
}
}
*n_above = above;
return cnt;
}
int yp_decode_v2(const int8_t *out, int gh, int gw, int na, int nc, const float *anchors,
const yp_cfg_t *c, yp_box_t *cand, int *n_above)
{
int8_t t = yp_q_threshold(c, 1);
int cnt = 0, above = 0, st = 5 + nc;
if (!thr_reachable(c, 1, t)) {
*n_above = 0;
return 0;
}
for (int r = 0; r < gh; r++)
for (int col = 0; col < gw; col++)
for (int a = 0; a < na; a++) {
const int8_t *p = out + ((r * gw + col) * na + a) * st;
if (p[4] < t) continue; /* объектность ниже порога */
above++;
int best = 0;
for (int k = 1; k < nc; k++) /* softmax не нужен: argmax тот же */
if (p[5 + k] > p[5 + best]) best = k;
float cx = ((float)col + sigm(deq(c, p[0]))) / (float)gw;
float cy = ((float)r + sigm(deq(c, p[1]))) / (float)gh;
float w = anchors[2 * a] * yp_exp(deq(c, p[2])) / (float)gw;
float h = anchors[2 * a + 1] * yp_exp(deq(c, p[3])) / (float)gh;
yp_box_t b;
if (!make_box(cx, cy, w, h, c, &b)) continue;
b.q = p[4];
b.cls = (uint8_t)best;
put(cand, &cnt, &b);
}
*n_above = above;
return cnt;
}
/* Пересечение больше доли num/den от объединения — в целых, без деления. */
static int iou_above(const yp_box_t *a, const yp_box_t *b, const yp_cfg_t *c)
{
int ax1 = a->x + a->w, bx1 = b->x + b->w, ay1 = a->y + a->h, by1 = b->y + b->h;
int ix = (ax1 < bx1 ? ax1 : bx1) - (a->x > b->x ? a->x : b->x);
int iy = (ay1 < by1 ? ay1 : by1) - (a->y > b->y ? a->y : b->y);
if (ix <= 0 || iy <= 0) return 0;
int64_t inter = (int64_t)ix * iy;
int64_t uni = (int64_t)a->w * a->h + (int64_t)b->w * b->h - inter;
return inter * c->iou_den > uni * c->iou_num;
}
int yp_nms(yp_box_t *b, int n, const yp_cfg_t *c)
{
int kept = 0;
for (int i = 0; i < n; i++) {
int drop = 0;
for (int j = 0; j < kept && !drop; j++)
if ((c->agnostic || b[j].cls == b[i].cls) && iou_above(&b[j], &b[i], c))
drop = 1;
if (!drop) b[kept++] = b[i];
}
return kept;
}
main_post.c — runs the test sets on the PC and on the M55.
/* Прогон постобработки на наборах из tensors.h. Один и тот же файл собирается
* для ПК (вывод в stdout) и для Cortex-M55 в QEMU mps3-an547 (вывод полухостингом). */
#include <stdint.h>
#include "yolo_post.h"
typedef struct {
const char *name;
int kind; /* 8 — YOLOv8, 2 — YOLOv2/LC */
const int8_t *data;
int n, nc, gh, gw, na, conf_pct;
yp_cfg_t cfg;
} tcase_t;
#include "tensors.h"
#ifdef __arm__
static void out(const char *s)
{
register int r0 __asm__("r0") = 0x04; /* SYS_WRITE0 */
register const char *r1 __asm__("r1") = s;
__asm__ volatile("bkpt 0xAB" : "+r"(r0) : "r"(r1) : "memory");
}
#else
#include <stdio.h>
static void out(const char *s) { fputs(s, stdout); }
#endif
static char line[160];
static int pos;
static void put_s(const char *s) { while (*s) line[pos++] = *s++; }
static void put_i(int v)
{
char t[12];
int k = 0;
unsigned u = v < 0 ? (unsigned)(-v) : (unsigned)v;
if (v < 0) line[pos++] = '-';
do { t[k++] = (char)('0' + u % 10); u /= 10; } while (u);
while (k) line[pos++] = t[--k];
}
static void kv(const char *k, int v) { put_s(" "); put_s(k); put_s("="); put_i(v); }
static void flush(void) { line[pos++] = '\n'; line[pos] = 0; out(line); pos = 0; }
int main(void)
{
static yp_box_t cand[YP_MAX_CAND];
for (unsigned i = 0; i < sizeof cases / sizeof cases[0]; i++) {
const tcase_t *t = &cases[i];
int above, n;
if (t->kind == 8)
n = yp_decode_v8(t->data, t->n, t->nc, &t->cfg, cand, &above);
else
n = yp_decode_v2(t->data, t->gh, t->gw, t->na, t->nc, anchors_v2, &t->cfg, cand, &above);
int kept = yp_nms(cand, n, &t->cfg);
put_s("CASE "); put_s(t->name); kv("kind", t->kind); kv("conf", t->conf_pct);
kv("agn", t->cfg.agnostic); kv("thr", yp_q_threshold(&t->cfg, t->kind == 2));
kv("above", above); kv("cand", n); kv("kept", kept);
flush();
for (int k = 0; k < kept; k++) {
put_s(" BOX"); kv("cls", cand[k].cls); kv("q", cand[k].q); kv("x", cand[k].x);
kv("y", cand[k].y); kv("w", cand[k].w); kv("h", cand[k].h);
flush();
}
}
put_s("DONE");
flush();
return 0;
}
m55_boot.c — startup on the Cortex-M55 in QEMU: vectors, FPU and Helium, exit.
/* Запуск на Cortex-M55 в QEMU mps3-an547: таблица векторов, FPU и MVE, .data/.bss, main, выход. */
#include <stdint.h>
extern int main(void);
extern uint32_t _sidata, _sdata, _edata, _sbss, _ebss, _estack;
static void semi_exit(void)
{
register int r0 __asm__("r0") = 0x18; /* SYS_EXIT */
register int r1 __asm__("r1") = 0x20026; /* ADP_Stopped_ApplicationExit */
__asm__ volatile("bkpt 0xAB" : : "r"(r0), "r"(r1) : "memory");
for (;;) {}
}
void Reset_Handler(void)
{
*(volatile uint32_t *)0xE000ED88u |= 0xFu << 20; /* CPACR: CP10, CP11 — FPU и Helium */
__asm__ volatile("dsb\n isb");
for (uint32_t *s = &_sidata, *d = &_sdata; d < &_edata;) *d++ = *s++;
for (uint32_t *d = &_sbss; d < &_ebss;) *d++ = 0;
main();
semi_exit();
}
static void Fault_Handler(void) { semi_exit(); }
__attribute__((section(".vectors"), used))
static const void *vectors[16] = {
&_estack, Reset_Handler, Fault_Handler, Fault_Handler, Fault_Handler, Fault_Handler, Fault_Handler,
};
m55.ld — the mps3-an547 memory map.
/* QEMU mps3-an547: ITCM 512 КБ с 0x00000000 — код и константы, DTCM 512 КБ с 0x20000000 — данные и стек. */
MEMORY
{
ITCM (rx) : ORIGIN = 0x00000000, LENGTH = 512K
DTCM (rwx) : ORIGIN = 0x20000000, LENGTH = 512K
}
ENTRY(Reset_Handler)
SECTIONS
{
.text : { KEEP(*(.vectors)) *(.text*) *(.rodata*) . = ALIGN(4); } > ITCM
.data : { _sdata = .; *(.data*) . = ALIGN(4); _edata = .; } > DTCM AT > ITCM
_sidata = LOADADDR(.data);
.bss (NOLOAD) : { _sbss = .; *(.bss*) *(COMMON) . = ALIGN(4); _ebss = .; } > DTCM
_estack = ORIGIN(DTCM) + LENGTH(DTCM);
}
yolo_post.py — the same logic in Python with rounding to float32.
# -*- coding: utf-8 -*-
"""Та же постобработка YOLO, что в yolo_post.c, на Python: каждая операция с float
округляется до float32, поэтому вывод совпадает с C на ПК и на Cortex-M55 байт в байт.
python yolo_post.py — печатает то же, что main_post.c"""
import struct
from gen_tensors import cases, ANCH_V2
MAX_CAND = 64
def f32(x):
return struct.unpack("<f", struct.pack("<f", x))[0]
def s8(b):
return b - 256 if b > 127 else b
def yp_exp(x):
x = f32(x)
x = 88.0 if x > 88.0 else (-87.0 if x < -87.0 else x)
k = f32(x * f32(1.44269504))
m = int(f32(k + (0.5 if k >= 0.0 else -0.5)))
r = f32(x - f32(m * f32(0.693145752)))
r = f32(r - f32(m * f32(1.42860677e-06)))
p = f32(0.00138888892)
for c in (0.00833333377, 0.0416666679, 0.166666672, 0.5, 1.0, 1.0):
p = f32(f32(c) + f32(r * p))
return f32(p * 2.0 ** m)
def sigm(x):
return f32(1.0 / f32(1.0 + yp_exp(-x)))
def deq(c, q):
return f32(f32(q - c["zp"]) * c["scale"])
def q_threshold(c, logistic):
for q in range(-128, 128):
v = deq(c, q)
if logistic:
v = sigm(v)
if v >= c["conf"]:
return q
return 127
def reachable(c, logistic, t):
v = deq(c, t)
return (sigm(v) if logistic else v) >= c["conf"]
def make_box(cx, cy, w, h, c):
W, H = f32(c["in_w"]), f32(c["in_h"])
cl = lambda v, hi: 0.0 if v < 0.0 else (hi if v > hi else v)
x0 = cl(f32(f32(cx - f32(w * 0.5)) * W), W)
x1 = cl(f32(f32(cx + f32(w * 0.5)) * W), W)
y0 = cl(f32(f32(cy - f32(h * 0.5)) * H), H)
y1 = cl(f32(f32(cy + f32(h * 0.5)) * H), H)
bw, bh = int(f32(x1 - x0)), int(f32(y1 - y0))
if bw <= 0 or bh <= 0:
return None
return [int(x0), int(y0), bw, bh]
def put(cand, b):
if len(cand) == MAX_CAND:
if b["q"] <= cand[-1]["q"]:
return
cand.pop()
k = len(cand)
while k > 0 and cand[k - 1]["q"] < b["q"]:
k -= 1
cand.insert(k, b)
def decode_v8(d, n, nc, c):
t = q_threshold(c, False)
if not reachable(c, False, t):
return [], 0
cand, above = [], 0
for i in range(n):
q, best = s8(d[4 * n + i]), 0
for k in range(1, nc):
if s8(d[(4 + k) * n + i]) > q:
q, best = s8(d[(4 + k) * n + i]), k
if q < t:
continue
above += 1
box = make_box(*(deq(c, s8(d[r * n + i])) for r in range(4)), c)
if box:
put(cand, dict(box=box, q=q, cls=best))
return cand, above
def decode_v2(d, gh, gw, na, nc, anchors, c):
t = q_threshold(c, True)
if not reachable(c, True, t):
return [], 0
cand, above, st = [], 0, 5 + nc
for r in range(gh):
for col in range(gw):
for a in range(na):
p = [s8(v) for v in d[((r * gw + col) * na + a) * st:][:st]]
if p[4] < t:
continue
above += 1
best = 0
for k in range(1, nc):
if p[5 + k] > p[5 + best]:
best = k
cx = f32(f32(col + sigm(deq(c, p[0]))) / gw)
cy = f32(f32(r + sigm(deq(c, p[1]))) / gh)
w = f32(f32(f32(anchors[2 * a]) * yp_exp(deq(c, p[2]))) / gw)
h = f32(f32(f32(anchors[2 * a + 1]) * yp_exp(deq(c, p[3]))) / gh)
box = make_box(cx, cy, w, h, c)
if box:
put(cand, dict(box=box, q=p[4], cls=best))
return cand, above
def iou_above(a, b, c):
ix = min(a[0] + a[2], b[0] + b[2]) - max(a[0], b[0])
iy = min(a[1] + a[3], b[1] + b[3]) - max(a[1], b[1])
if ix <= 0 or iy <= 0:
return False
inter = ix * iy
return inter * c["iou_den"] > (a[2] * a[3] + b[2] * b[3] - inter) * c["iou_num"]
def nms(cand, c):
kept = []
for b in cand:
if not any((c["agnostic"] or k["cls"] == b["cls"]) and iou_above(k["box"], b["box"], c) for k in kept):
kept.append(b)
return kept
def run(cs):
lines = []
for t in cs:
c = dict(t, scale=f32(t["scale"]), conf=f32(t["conf"] / 100.0))
if t["kind"] == 8:
cand, above = decode_v8(t["data"], t["n"], t["nc"], c)
else:
cand, above = decode_v2(t["data"], t["gh"], t["gw"], t["na"], t["nc"], ANCH_V2, c)
kept = nms(cand, c)
lines.append("CASE %s kind=%d conf=%d agn=%d thr=%d above=%d cand=%d kept=%d" % (
t["name"], t["kind"], t["conf"], t["agnostic"], q_threshold(c, t["kind"] == 2), above, len(cand), len(kept)))
for b in kept:
lines.append(" BOX cls=%d q=%d x=%d y=%d w=%d h=%d" % ((b["cls"], b["q"]) + tuple(b["box"])))
lines.append("DONE")
return lines
if __name__ == "__main__":
print("\n".join(run(cases())))
energy121v.py — the energy budget model.
# -*- coding: utf-8 -*-
"""Энергобюджет уличной AI-камеры на STM32N6 (раздел 121в). Только стандартная библиотека.
python energy121v.py
Цикл: Standby -> пробуждение по PIR или таймеру -> загрузка -> кадр -> NPU -> отправка -> Standby.
У каждого числа — источник: «DS» — даташит DS14791 Rev 11 (таблица, страница, условия),
«оценка» — числа в документах нет, взято допущение; его и надо мерить на своей плате."""
# ── Параметры: имя -> (значение, единица, источник) ─────────────────────────────────
P = {
# STM32N6, токи при 1,8 В, 25 °C, типовые; SMPS внутренний
"n6_run_ma": (99.93, "мА", "DS табл. 40, с. 155: IDDSMPS, 800 МГц, все периферийные включены"),
"n6_sleep_ma": (80.9, "мА", "DS табл. 42, с. 157: IDDSMPS в Sleep, 800 МГц, все периферийные включены"),
"n6_stop_ma": (3.60 + 0.2734 + 0.3627, "мА", "DS табл. 44, с. 158: Stop SVOS low, IDDSMPS 3,60 мА + IDD 273,4 мкА + IDDA1V8 362,7 мкА"),
"n6_stby_ua": (48.4 + 42.0, "мкА", "DS табл. 45, с. 159: Standby, RAM резерва и RTC с LSE включены, IDD 48,4 + IDDA18ON 42"),
"n6_stby85_ua": (69.0 + 80.0, "мкА", "DS табл. 45, с. 159: то же при TJ = 85 °C: 69,0 + 80"),
"n6_npu_ma": (200.0, "мА", "оценка: тока NPU в вычислении в даташите нет"),
"v_rail": (1.8, "В", "DS табл. 25: VDD 1,8 В"),
# фазы одного срабатывания
"t_boot_s": (0.30, "с", "оценка: ПЗУ грузит прошивку с внешней флеш-памяти (DS: из Standby до первой команды 400 мкс)"),
"t_ae_s": (1.0, "с", "пример OpenMV: snapshot(time=1000) — дать датчику подстроиться"),
"t_npu_s": (0.10, "с", "оценка: YOLOv8n 192×192 на NPU"),
"t_post_s": (0.05, "с", "оценка: постобработка, JPEG вырезки"),
"sensor_mw": (150.0, "мВт", "оценка: датчик изображения во время съёмки"),
"flash_ma": (20.0, "мА", "оценка: внешняя флеш-память и PSRAM при работе, 1,8 В"),
# связь (токи от батареи)
"lora_tx_ma": (45.0, "мА", "оценка: модуль LoRa на передаче +14 дБм"),
"lora_rx_ma": (12.0, "мА", "оценка: модуль LoRa на приёме"),
"lora_rx_s": (0.2, "с", "оценка: окна приёма класса A"),
"lte_ma": (120.0, "мА", "оценка: модем LTE-M, средний ток сеанса"),
"lte_s": (6.0, "с", "оценка: выход из PSM, отправка вырезки 8 КБ"),
"wifi_ma": (180.0, "мА", "оценка: Wi-Fi, подключение и отправка"),
"wifi_s": (2.5, "с", "оценка: подключение к точке и отправка кадра"),
# сон всей платы помимо N6 (от батареи)
"pir_ua": (15.0, "мкА", "оценка: пироэлектрический датчик движения"),
"other_ua": (12.0, "мкА", "оценка: стабилизатор, флеш в глубоком сне, LoRa во сне, делитель батареи"),
# питание
"eta": (0.85, "", "оценка: КПД преобразователя батарея -> 1,8 В"),
"v_bat": (3.6, "В", "оценка: среднее напряжение Li-ion за разряд"),
"cell_mah": (3000.0, "мА·ч", "оценка: ёмкость одного 18650 по паспорту"),
"self_pct_mon": (2.0, "%/мес", "оценка: саморазряд элемента Li-ion"),
"use_warm": (0.90, "", "оценка: доступная доля ёмкости при +20 °C"),
"use_cold": (0.60, "", "оценка: доступная доля ёмкости при −20 °C"),
"pv_w": (2.0, "Вт", "оценка: солнечная панель"),
"sun_h_dec": (0.5, "ч/сут", "оценка: часов полного солнца в декабре"),
"sun_h_jun": (5.0, "ч/сут", "оценка: часов полного солнца в июне"),
"pv_eta": (0.70, "", "оценка: панель -> контроллер заряда -> элемент"),
"hb_per_day": (24, "раз/сут", "оценка: пульс по таймеру раз в час, без камеры"),
}
V = {k: v[0] for k, v in P.items()}
def lora_airtime_us(sf, bw, cr, payload):
"""Формула раздела 89: преамбула 8 символов, явный заголовок, CRC, DE при символе > 16 мс."""
tsym = (1000000 << sf) // bw
de = 1 if tsym > 16000 else 0
num = 8 * payload - 4 * sf + 28 + 16
den = 4 * (sf - 2 * de)
blocks = (num + den - 1) // den if num > 0 else 0
quarters = 4 * 8 + 17 + 4 * (8 + blocks * (cr + 4))
return quarters * tsym // 4
def bat_ma(ma_rail):
"""Ток шины 1,8 В -> ток батареи через преобразователь."""
return ma_rail * V["v_rail"] / (V["v_bat"] * V["eta"])
def event_mas(link):
"""Заряд одного срабатывания, мА·с от батареи, по фазам."""
ph = {
"загрузка": bat_ma(V["n6_run_ma"] + V["flash_ma"]) * V["t_boot_s"],
"датчик и экспозиция": (bat_ma(V["n6_sleep_ma"]) + V["sensor_mw"] / (V["v_bat"] * V["eta"])) * V["t_ae_s"],
"NPU": bat_ma(V["n6_run_ma"] + V["n6_npu_ma"] + V["flash_ma"]) * V["t_npu_s"],
"постобработка": bat_ma(V["n6_run_ma"]) * V["t_post_s"],
}
if link == "lora":
toa = lora_airtime_us(9, 125000, 1, 12 + 13) / 1e6
ph["радио"] = V["lora_tx_ma"] * toa + V["lora_rx_ma"] * V["lora_rx_s"] + bat_ma(V["n6_run_ma"]) * (toa + V["lora_rx_s"])
elif link == "lte":
ph["радио"] = V["lte_ma"] * V["lte_s"] + bat_ma(V["n6_sleep_ma"]) * V["lte_s"]
else:
ph["радио"] = V["wifi_ma"] * V["wifi_s"] + bat_ma(V["n6_sleep_ma"]) * V["wifi_s"]
return ph
def heartbeat_mas():
toa = lora_airtime_us(9, 125000, 1, 4 + 13) / 1e6
return bat_ma(V["n6_run_ma"] + V["flash_ma"]) * V["t_boot_s"] + V["lora_tx_ma"] * toa \
+ V["lora_rx_ma"] * V["lora_rx_s"] + bat_ma(V["n6_run_ma"]) * (toa + V["lora_rx_s"])
def sleep_ua(stop=False, hot=False):
n6 = V["n6_stop_ma"] * 1000 if stop else (V["n6_stby85_ua"] if hot else V["n6_stby_ua"])
return bat_ma(n6) + V["pir_ua"] + V["other_ua"]
def day_mah(events, link="lora", stop=False):
ev = sum(event_mas(link).values())
return (events * ev + V["hb_per_day"] * heartbeat_mas()) / 3600 + sleep_ua(stop) * 24 / 1000
def self_mah(n):
"""Саморазряд n элементов, мА·ч в сутки (месяц = 30 суток)."""
return n * V["cell_mah"] * V["self_pct_mon"] / 100 / 30
def days(n, use, d):
return n * V["cell_mah"] * V[use] / (d + self_mah(n))
def fmt(x, d=1):
return ("%.*f" % (d, x)).replace(".", ",")
def main():
print("== параметры")
for k, (v, u, src) in P.items():
print("PAR %s = %s %s | %s" % (k, fmt(v, 4).rstrip("0").rstrip(","), u, src))
print("== радио")
print("TOA sf9 23B = %s мс (раздел 89: 205,8)" % fmt(lora_airtime_us(9, 125000, 1, 23) / 1000))
print("TOA sf7 63B = %s мс (раздел 48в: 118,0)" % fmt(lora_airtime_us(7, 125000, 1, 63) / 1000))
print("TOA sf9 25B = %s мс — событие: 12 байт данных + 13 LoRaWAN" % fmt(lora_airtime_us(9, 125000, 1, 25) / 1000))
model = 3228400
frames = -(-model // 48)
for sf in (7, 12):
t = lora_airtime_us(sf, 125000, 1, 63) / 1e6 * frames
print("MODEL sf%d frames=%d air=%s ч, при 1 %% эфира не быстрее %s сут" % (sf, frames, fmt(t / 3600), fmt(t * 100 / 86400)))
react = V["t_boot_s"] + V["t_ae_s"] + V["t_npu_s"] + V["t_post_s"] + lora_airtime_us(9, 125000, 1, 25) / 1e6
print("REACT от PIR до конца передачи %s с (загрузка + экспозиция + NPU + постобработка + эфир)" % fmt(react, 2))
print("== сон")
print("SLEEP standby = %s мкА от батареи (N6 %s мкА при 1,8 В -> %s мкА + PIR %s + прочее %s)" % (
fmt(sleep_ua()), fmt(V["n6_stby_ua"]), fmt(bat_ma(V["n6_stby_ua"])), fmt(V["pir_ua"]), fmt(V["other_ua"])))
print("SLEEP standby85 = %s мкА от батареи" % fmt(sleep_ua(hot=True)))
print("SLEEP stop = %s мкА от батареи (N6 в Stop SVOS low: %s мА при 1,8 В)" % (fmt(sleep_ua(stop=True)), fmt(V["n6_stop_ma"], 3)))
print("SELF 1x18650 %s мА·ч/сут, 2x18650 %s мА·ч/сут" % (fmt(self_mah(1), 1), fmt(self_mah(2), 1)))
print("SLEEP day: standby %s мА·ч/сут, stop %s мА·ч/сут" % (fmt(sleep_ua() * 24 / 1000, 2), fmt(sleep_ua(stop=True) * 24 / 1000, 1)))
print("== срабатывание")
for link in ("lora", "lte", "wifi"):
ph = event_mas(link)
print("EVENT %s total=%s мА·с | %s" % (link, fmt(sum(ph.values())),
"; ".join("%s %s" % (k, fmt(v)) for k, v in ph.items())))
print("HEARTBEAT = %s мА·с" % fmt(heartbeat_mas()))
print("== сутки и автономность")
for link in ("lora", "lte", "wifi"):
for ev in (10, 50, 200):
d = day_mah(ev, link)
cells = []
for n in (1, 2):
for tag, use in (("+20", "use_warm"), ("-20", "use_cold")):
cells.append("%dx18650 %s: %s сут" % (n, tag, fmt(days(n, use, d), 0)))
print("DAY %s ev=%d %s мА·ч/сут | %s" % (link, ev, fmt(d, 1), "; ".join(cells)))
d = day_mah(50, "lora", stop=True)
print("DAY lora ev=50 STOP %s мА·ч/сут | 1x18650 +20: %s сут" % (fmt(d, 1), fmt(days(1, "use_warm", d), 0)))
print("== солнце")
for tag, h in (("dec", V["sun_h_dec"]), ("jun", V["sun_h_jun"])):
gain = V["pv_w"] * h * V["pv_eta"] / V["v_bat"] * 1000
print("PV %s приход %s мА·ч/сут; lora ev=50: баланс %s; lte ev=50: баланс %s" % (
tag, fmt(gain, 0), fmt(gain - day_mah(50, "lora"), 0), fmt(gain - day_mah(50, "lte"), 0)))
be = (V["pv_w"] * V["sun_h_dec"] * V["pv_eta"] / V["v_bat"] * 1000 - V["hb_per_day"] * heartbeat_mas() / 3600
- sleep_ua() * 24 / 1000 - self_mah(1)) / (sum(event_mas("lora").values()) / 3600)
print("PV dec breakeven lora 1x18650: %d срабатываний в сутки" % int(be))
print("== рисунок")
ph = event_mas("lora")
day = {"сон": sleep_ua() * 24 / 1000, "пульс": V["hb_per_day"] * heartbeat_mas() / 3600}
for k, v in ph.items():
day[k] = 50 * v / 3600
print("FIG " + "; ".join("%s=%s" % (k, fmt(v, 1)) for k, v in day.items()) + "; всего=%s" % fmt(sum(day.values()), 1))
if __name__ == "__main__":
main()
Next: section 122 "Integer types and conversions: promotion, sign, overflow".