Embedded HandbookApps
All sections › V. Debugging and quality
STM32F407, STM32F103 · level intermediate · 23 min read
Open in the interactive handbook

35в. A failure in the field: a state snapshot that survives a reset, and analysis after reboot

Section 35.2 taught you to read a HardFault while a debugger is connected: the handler puts PC, LR and CFSR into variables and spins in a loop until a person looks. A device in the field has no debugger. There a failure looks like this: the device rebooted, and that is all. The watchdog (section 12) brings it back to work, but takes the cause away with it — unless you save the cause somewhere it survives the reset.

Here is the complete path of such saving. The fault handler writes a 88-byte record into a RAM area that the startup code does not touch, and restarts the chip. The new firmware finds the record, checks it by CRC and outputs it over UART. On the PC a script turns PC and LR from the record into a function name and line number through arm-none-eabi-addr2line. Everything is shown by a run in QEMU: two deliberate faults in a row, two restarts, the analysis — and a check of the lines against the source.

Sources: the reference manuals RM0090 (F407) and RM0008 (F103) — reset types, the backup domain, flash memory; core_cm4.h from CMSIS — the addresses of the fault registers; the SVD files of the Keil packs — the RCC and USART1 addresses. The experiment is in the stm-arm:1 image: arm-none-eabi-gcc 12.2.1, QEMU 7.2.22 (the netduinoplus2 machine for F4 and stm32vldiscovery for F1), addr2line 2.40, gdb-multiarch. The same code builds with armclang for both chips. The stack frame and EXC_RETURN are covered in 7а, the CFSR and HFSR bits in 35.2, the linker script in 4в, the reset-cause flags in 12.4; here they are only used.

Terms used in the documentation

Term Meaning
crash record / crash dump a record of the failure / a snapshot of the state
.noinit, NOLOAD a section without initial values that the startup code does not zero
system reset a system reset: the NRST pin, the watchdogs, software
power reset (POR/PDR, BOR) a reset caused by power
SYSRESETREQ (AIRCR) the software reset bit in a core register
reset flags (RCC_CSR) the flags that tell the cause of a reset
GNU build ID a build fingerprint: a hash in an ELF note
backup SRAM / backup registers backup SRAM (F407) / backup registers (F103)
addr2line turns an address into a function and line using the debug information

35в.1 What to save: an 88-byte record

A failure happens once, and you often cannot repeat it on the desk. So the record must answer three questions at once: where it broke, why and with which firmware. Where is the exception frame: the eight words that the core pushed onto the stack (7а.1), among them the PC of the faulting instruction and LR — where it was called from. Why is the CFSR, HFSR, MMFAR, BFAR registers (35.2). Which firmware is the version number and the beginning of the build fingerprint: without it the addresses in the record have nothing to be checked against.

The path of a crash record: a fault, the HardFault handler writes the record into the .noinit area at the end of RAM, a SYSRESETREQ software reset, the startup code zeroes only .bss, main checks the magic and the CRC and outputs the record over UART, on the PC addr2line turns PC into a function and line; when power is removed RAM is lost and the record fails the check

The record layout is an ordinary structure, and its size is pinned down by a compiler check:

typedef struct {
    uint32_t magic;                      /* признак записи */
    uint16_t format;                     /* CRASH_FORMAT */
    uint16_t size;                       /* sizeof(crash_rec_t) */
    uint32_t count;                      /* сбоев подряд с последнего чистого старта */
    uint32_t fw_version;                 /* 0x00MMmmpp */
    uint8_t  build_id[8];                /* начало GNU build-id: какой ELF брать */
    uint32_t reset_csr;                  /* RCC->CSR при следующем старте */
    uint32_t exc_return;                 /* LR на входе в обработчик */
    uint32_t sp;                         /* где лежал кадр */
    uint32_t r0, r1, r2, r3, r12, lr, pc, xpsr;   /* кадр исключения */
    uint32_t cfsr, hfsr, mmfar, bfar;    /* почему */
    uint32_t crc;                        /* CRC-32 всех байтов до этого поля */
} crash_rec_t;

_Static_assert(sizeof(crash_rec_t) == 88, "record layout changed: bump CRASH_FORMAT");
_Static_assert(offsetof(crash_rec_t, crc) == sizeof(crash_rec_t) - 4, "crc must be last");

Three fields are for housekeeping. magic is the "a record is here" marker: 0x48535243, in memory these are the bytes C R S H. format and size are the layout number and its size: new firmware with a different structure will not take an old record for its own. crc is a CRC-32 (the same as zlib's, polynomial 0xEDB88320) over the first 84 bytes. The magic tells a record from emptiness, the CRC tells it from garbage and from a record cut off in the middle.

count counts failures in a row: if the old record is intact at a new failure, the counter grows. A device that crashes every five minutes and a device that crashed once a year give different records. reset_csr is filled not in the handler but after the restart: the reset-cause flags appear only then.


35в.2 The .noinit section: its own area at the end of RAM

The startup code from 4в does two things before main: it copies .data from flash and zeroes .bss. Everything that lies outside these two sections stays as it was after a reset. Section 7в showed this with a boot counter; section 48 with a reflash flag, and there the main condition was found: the address must not depend on how many variables the program has. So the record gets its own area at the end of RAM, and RAM is shortened by the same size:

MEMORY
{
  FLASH  (rx)  : ORIGIN = 0x08000000, LENGTH = 1024K
  RAM    (rwx) : ORIGIN = 0x20000000, LENGTH = 128K - 128
  NOINIT (rw)  : ORIGIN = 0x2001FF80, LENGTH = 128
}

Two rules are added to the linker script from 4в: the build fingerprint in flash with the label __build_id, and the section itself:

  .noinit (NOLOAD) :                     /* стартовый код её не обнуляет */
  {
    *(.noinit .noinit.*)
  } > NOINIT

The top of the stack is computed from the shortened RAM — _estack = ORIGIN(RAM) + LENGTH(RAM) — so the stack grows down from the start of the area and does not overwrite the record. The variable is declared with an attribute: crash_rec_t crash_rec __attribute__((section(".noinit")));.

The build confirms the layout. arm-none-eabi-nm for F4 shows crash_rec and _estack at the same address 2001ff80, and the size of .noinit is 88 bytes. The word NOLOAD is visible in the program headers: this segment has 0 bytes in the file and 0x00058 (88) in memory. So neither a loader nor a debugger writes anything into this area when flashing. For F1 in QEMU (8 KB of RAM) the area is 0x20001F80; on an STM32F103C8 with 20 KB of RAM it would be 0x20004F80.


35в.3 The handler: frame, fault registers, software reset

The first half of the handler is the same as in 35.2: bit 2 of EXC_RETURN selects the stack where the frame lies. One thing is added: LR on entry is passed as the second argument so that it gets into the record.

__attribute__((naked)) void HardFault_Handler(void)
{
    __asm volatile(
        "tst   lr, #4      \n"           /* бит 2 EXC_RETURN: какой стек */
        "ite   eq          \n"
        "mrseq r0, msp     \n"
        "mrsne r0, psp     \n"
        "mov   r1, lr      \n"           /* EXC_RETURN — вторым аргументом */
        "b     crash_save  \n");
}

The second half, instead of an endless loop, writes the record and restarts the chip:

__attribute__((used, noreturn)) void crash_save(const uint32_t *frame, uint32_t exc_return)
{
    uint32_t n = crash_valid() ? crash_rec.count + 1u : 1u;
    crash_rec.magic = CRASH_MAGIC;
    crash_rec.format = CRASH_FORMAT;
    crash_rec.size = sizeof(crash_rec_t);
    crash_rec.count = n;
    crash_rec.fw_version = FW_VERSION;
    for (int i = 0; i < 8; i++)
        crash_rec.build_id[i] = __build_id[16 + i];   /* 16 байт — заголовок заметки */
    crash_rec.reset_csr = CSR_NOT_READ;
    crash_rec.exc_return = exc_return;
    crash_rec.sp = (uint32_t)frame;
    crash_rec.r0 = frame[0];
    crash_rec.r1 = frame[1];
    crash_rec.r2 = frame[2];
    crash_rec.r3 = frame[3];
    crash_rec.r12 = frame[4];
    crash_rec.lr = frame[5];
    crash_rec.pc = frame[6];
    crash_rec.xpsr = frame[7];
    crash_rec.cfsr = SCB_CFSR;
    crash_rec.hfsr = SCB_HFSR;
    crash_rec.mmfar = SCB_MMFAR;
    crash_rec.bfar = SCB_BFAR;
    crash_reseal();                      /* CRC — последним: оборванная запись не пройдёт */
    __asm volatile("dsb" ::: "memory");
    SCB_AIRCR = AIRCR_RESET;             /* программный сброс: ОЗУ не трогается */
    __asm volatile("dsb" ::: "memory");
    for (;;) {
    }
}

The order matters. The CRC is computed last: if something interrupts the write in the middle, the check will reject it. The dsb before the reset waits until all writes to RAM have reached memory. AIRCR_RESET is the key 0x5FA in bits 31:16 and bit 2 SYSRESETREQ; the positions of both are from core_cm4.h, and NVIC_SystemReset() resets the same way.

The build fingerprint is an ELF note that the linker creates with the option -Wl,--build-id=sha1: 16 bytes of header (lengths, type 3, the string GNU) and 20 bytes of SHA-1 hash. In the run the note lies at 0x08000040, its size is 36 bytes; the record takes the first 8 bytes of the hash. The hash is computed over the contents of the build, so the F4 and F1 firmware from the same sources got different fingerprints: 4aba8274… and 6d8076eb….

The handler prints nothing and waits for no one: at the moment of the failure it is unknown whether the stack is intact, whether the UART is configured and whether it is the UART that broke. Output is the job of the next start-up.


35в.4 After the restart: check, reset cause, output

At the start of main the firmware checks the record — magic, format, size and CRC — and, if it is intact, adds the reset cause and outputs it:

    puts1("boot\n");
    if (crash_valid()) {
        if (crash_rec.reset_csr == CSR_NOT_READ) {
            crash_rec.reset_csr = RCC_CSR;           /* причина сброса после сбоя */
            crash_reseal();
        }
        crash_report();
    } else {
        puts1("no crash record\n");
    }
    RCC_CSR |= RCC_CSR_RMVF;

reset_csr is written exactly once — at the first start-up after the failure. If the device is later rebooted with the button, the record keeps the cause that belongs to the failure. The flags are cleared with the RMVF bit, otherwise they accumulate (12.4).

The experiment firmware is arranged so that a single run goes through everything. The first start-up finds no record and divides the sum of the samples by their count, which is still zero; the DIV_0_TRP bit in SCB->CCR is set, and the division becomes a fault. The second start-up finds record No. 1, outputs it and reads a "sensor" through the pointer 0x60000000, where there is nothing in QEMU. The third start-up outputs record No. 2 and waits for commands over UART: d — output the record again, c — clear it. The full F4 output:

boot
no crash record
boot
CRASH #1 pc=0x080004de lr=0x080004fb cfsr=0x02000000 hfsr=0x40000000 bfar=0x00000000 csr=0x00000000
REC 435253480100580001000000020401004aba8274789c0f3200000000f9ffffff40ff0120000000000800002008000020e803000000000000fb040008de0400080000004100000002000000400000000000000000d2e8e30b
boot
CRASH #2 pc=0x08000510 lr=0x08000525 cfsr=0x00008200 hfsr=0x40000000 bfar=0x60000000 csr=0x00000000
REC 435253480100580002000000020401004aba8274789c0f3200000000f9ffffff40ff012000000060000000000800002000000020d4ff012025050008100500080000006100820000000000400000000000000060dc0be734
ready: d - dump, c - clear
record cleared
no crash record

The CRASH line is for a person at the terminal, the REC line is all 88 bytes of the record in hex, for the script. The three boot words are three start-ups: the record survived two software resets in a row, and the counter reached 2. After the command c the magic is zeroed, and d honestly answers no crash record. F1 in QEMU gave the same picture with its own addresses: pc=0x0800053e and pc=0x08000570.

The csr field in the run is zeros: QEMU does not keep the reset-cause flags. On a board after SYSRESETREQ there will be SFTRSTF and PINRSTF here (table 12.4), after a watchdog — IWDGRSTF or WWDGRSTF.

In a product the record is output not only at start-up. A typical scheme is a d command over the same channel as the settings: the service program collects the record, makes sure the CRC matches, and only then sends c. Until the record has been collected, it waits in RAM.


35в.5 Analysis on the PC: addr2line and gdb using the ELF

decode35d.py takes the REC lines from the UART output and does four steps: it checks the CRC with the same algorithm (zlib.crc32), compares the record's fingerprint with the ELF fingerprint (readelf -n), decodes CFSR and HFSR bit by bit and turns the addresses into lines through arm-none-eabi-addr2line. Analysis of the F4 records:

record #1: 88 bytes, crc ok, fw 1.4.2, build-id 4aba8274789c0f32 = ELF
  CFSR 0x02000000: DIVBYZERO   HFSR 0x40000000: FORCED
  EXC_RETURN 0xfffffff9: frame on MSP, r0=0x00000000 r1=0x20000008
  PC 0x080004de -> average at app35d.c:58
  LR 0x080004fb -> call at 0x080004f8: scenario_div at app35d.c:70
record #2: 88 bytes, crc ok, fw 1.4.2, build-id 4aba8274789c0f32 = ELF
  CFSR 0x00008200: PRECISERR BFARVALID   HFSR 0x40000000: FORCED
  BFAR 0x60000000: address of the bad access
  EXC_RETURN 0xfffffff9: frame on MSP, r0=0x60000000 r1=0x00000000
  PC 0x08000510 -> read_sensor at app35d.c:65
  LR 0x08000525 -> call at 0x08000522: scenario_bus at app35d.c:76

The check compares the lines found with the source: line 58 of app35d.c is int32_t avg = sample_sum / count;, line 65 is return *reg + 1u;, lines 70 and 76 are the calls to average and read_sensor. The disassembler confirms: at 0x080004de there is sdiv r0, r3, r0, at 0x08000510 — ldr r0, [r0, #0]. The operands of the division are visible right in the frame: r0 is the divisor, zero; r3 is the dividend, 0x3e8, that is 1000. In the second record r0 is the bad pointer itself, and it is also in BFAR.

There is a subtlety with LR. The frame holds the return address — the next instruction after the call, with a one in the least significant bit (the Thumb marker). This instruction may already belong to the next line. So the script clears the Thumb bit and steps back another 2 bytes: 0x080004fb → 0x080004f8, the middle of the instruction bl 80004d8 <average>, which starts at 0x080004f6. This finds exactly the line of the call.

The same by hand — with one command, without the script:

average at /tmp/c35d/app35d.c:58
scenario_div at /tmp/c35d/app35d.c:70

This is the output of arm-none-eabi-addr2line -f -p -e fw35d_f4.elf 0x080004de 0x080004f8. The debugger answers the same from the same ELF — gdb-multiarch -batch -ex "info line *0x080004de":

Line 58 of "app35d.c" starts at address 0x80004de <average+6> and ends at 0x80004e2 <average+10>.

No board is needed for this: only the ELF of that very build and the 88 bytes of the record. That is why the ELF of every released version is kept next to the binary image — it is later found by the fingerprint.


35в.6 What the record survives and what it does not

System reset. RM0090 (7.1.1) and RM0008 divide resets into system and power resets. A system reset is a low level on NRST, either watchdog, a software reset through SYSRESETREQ, a reset on entering Stop or Standby set by the option bytes. It returns all registers to their reset values, except the flags in RCC_CSR and the backup domain. RAM is not on this list: its contents are spoiled only by the startup code, and that skips .noinit. The experiment showed this for a software reset; a watchdog leads to a reset of the same kind.

A watchdog without a fault. If the program hung rather than crashed, the fault handler is not called, and there is no one to write the record. The watchdog restarts the chip, and the new firmware sees only IWDGRSTF. To find out where it hung, the record is written in advance: from the WWDG early wakeup interrupt (12.3) — there is a frame with the PC of the hung code there — or "the last thing the program was doing" is periodically put into the same area.

Power removal. On a power reset RAM is not preserved: RM0090 says plainly for Standby that the contents of SRAM and registers are lost, except the backup domain. After power-on the area holds random bytes; the magic and the CRC-32 will filter them out — a CRC can match by chance with a probability of 1 in 2³².

How to survive power loss. Two ways, both with caveats.

The working scheme has two steps. The handler writes the record into RAM and resets the chip, as in the experiment. The already healthy firmware finds the record after start-up and copies it to flash with the journal from 11.8 — into an area erased in advance, in the normal way and with verification. Power removed after that will no longer erase the record.


35в.7 Pitfalls

After the restart the record is gone or has become garbage. The section got zeroed: the linker script has no rule for .noinit, it ended up inside .bss, or the startup code zeroes the whole RAM. The check: in the program headers the record's segment must have 0 bytes in the file and the address of the NOINIT area.

The record is corrupted even though everything is built correctly. The stack has grown into the area: the top of the stack is above the record. _estack is computed from the shortened RAM, as in 35в.2, not from the end of the actual RAM. A bootloader with its own script (section 48) must know about the same area.

Garbage at power-on taken for a failure. A check by the magic alone catches a random match of one word; for RAM after power-on that is not enough. The magic, format, size and CRC-32 together practically rule out such an error.

Analysis with the wrong ELF gives plausible nonsense. An F4 record analysed with the F1 ELF pointed to putc1 at app35d.c:10 and putc1 at app35d.c:8 — an existing function and existing lines, just not the right ones. The script caught this by the fingerprint: build-id 4aba8274789c0f32 != ELF 6d8076ebcf50f931. Without the fingerprint the error goes unnoticed. The same happens after reflashing with a debugger: it does not write into .noinit, and the old record survives the firmware change.

addr2line and gdb name different lines for one address. In the function return a / b; at -O2 the division instruction is at address 0, and the line table has three entries at that address: lines 6 ({) and 7 marked "start of statement", and line 8 (}) without the mark. addr2line takes the last one and answers quirk35d.c:8, gdb picks the start of the statement and answers Line 7. If the line points to }, look at the line above.

The call line from LR is wrong. LR holds the return address with the Thumb bit. Without the -1 correction and another -2 bytes the line may turn out to be the one after the call (35в.5).

The reset cause is always the same. The RCC_CSR flags accumulate until RMVF or a power reset (12.4); the record takes them once and clears them. In QEMU the field will be zero for any reset: the emulator does not keep them.

Writing the crash record to flash directly from the handler. Erasing a sector stops code execution from flash, and a reset in the middle of an erase leaves it in an unknown state. Write to flash after the restart, from healthy firmware.


35в.8 Summary

Full source code

crash35d.h — the crash record.

/* Раздел 35в: запись о сбое, которая переживает сброс. */
#ifndef CRASH35D_H
#define CRASH35D_H
#include <stddef.h>
#include <stdint.h>

#define CRASH_MAGIC   0x48535243u        /* байты C R S H в памяти */
#define CRASH_FORMAT  1u                 /* версия раскладки записи */
#define CSR_NOT_READ  0xFFFFFFFFu        /* причина сброса ещё не прочитана */

typedef struct {
    uint32_t magic;                      /* признак записи */
    uint16_t format;                     /* CRASH_FORMAT */
    uint16_t size;                       /* sizeof(crash_rec_t) */
    uint32_t count;                      /* сбоев подряд с последнего чистого старта */
    uint32_t fw_version;                 /* 0x00MMmmpp */
    uint8_t  build_id[8];                /* начало GNU build-id: какой ELF брать */
    uint32_t reset_csr;                  /* RCC->CSR при следующем старте */
    uint32_t exc_return;                 /* LR на входе в обработчик */
    uint32_t sp;                         /* где лежал кадр */
    uint32_t r0, r1, r2, r3, r12, lr, pc, xpsr;   /* кадр исключения */
    uint32_t cfsr, hfsr, mmfar, bfar;    /* почему */
    uint32_t crc;                        /* CRC-32 всех байтов до этого поля */
} crash_rec_t;

_Static_assert(sizeof(crash_rec_t) == 88, "record layout changed: bump CRASH_FORMAT");
_Static_assert(offsetof(crash_rec_t, crc) == sizeof(crash_rec_t) - 4, "crc must be last");

extern crash_rec_t crash_rec;
uint32_t crc32(const void *p, size_t n);
int crash_valid(void);
void crash_reseal(void);

#endif

crash35d.c — the fault handler: a record in .noinit and a restart.

/* Раздел 35в: обработчик отказа пишет запись в .noinit и перезапускает камень. */
#include "crash35d.h"
#include "regs35d.h"

#define FW_VERSION 0x00010402u           /* 1.4.2 */

crash_rec_t crash_rec __attribute__((section(".noinit")));
extern const uint8_t __build_id[];       /* из скрипта компоновщика */

uint32_t crc32(const void *p, size_t n)  /* CRC-32 как у zlib: полином 0xEDB88320 */
{
    const uint8_t *b = p;
    uint32_t c = 0xFFFFFFFFu;
    while (n--) {
        c ^= *b++;
        for (int k = 0; k < 8; k++)
            c = (c >> 1) ^ (0xEDB88320u & (0u - (c & 1u)));
    }
    return ~c;
}

int crash_valid(void)
{
    return crash_rec.magic == CRASH_MAGIC && crash_rec.format == CRASH_FORMAT
        && crash_rec.size == sizeof(crash_rec_t)
        && crash_rec.crc == crc32(&crash_rec, offsetof(crash_rec_t, crc));
}

void crash_reseal(void)                  /* после правки поля — пересчитать CRC */
{
    crash_rec.crc = crc32(&crash_rec, offsetof(crash_rec_t, crc));
}

__attribute__((used, noreturn)) void crash_save(const uint32_t *frame, uint32_t exc_return)
{
    uint32_t n = crash_valid() ? crash_rec.count + 1u : 1u;
    crash_rec.magic = CRASH_MAGIC;
    crash_rec.format = CRASH_FORMAT;
    crash_rec.size = sizeof(crash_rec_t);
    crash_rec.count = n;
    crash_rec.fw_version = FW_VERSION;
    for (int i = 0; i < 8; i++)
        crash_rec.build_id[i] = __build_id[16 + i];   /* 16 байт — заголовок заметки */
    crash_rec.reset_csr = CSR_NOT_READ;
    crash_rec.exc_return = exc_return;
    crash_rec.sp = (uint32_t)frame;
    crash_rec.r0 = frame[0];
    crash_rec.r1 = frame[1];
    crash_rec.r2 = frame[2];
    crash_rec.r3 = frame[3];
    crash_rec.r12 = frame[4];
    crash_rec.lr = frame[5];
    crash_rec.pc = frame[6];
    crash_rec.xpsr = frame[7];
    crash_rec.cfsr = SCB_CFSR;
    crash_rec.hfsr = SCB_HFSR;
    crash_rec.mmfar = SCB_MMFAR;
    crash_rec.bfar = SCB_BFAR;
    crash_reseal();                      /* CRC — последним: оборванная запись не пройдёт */
    __asm volatile("dsb" ::: "memory");
    SCB_AIRCR = AIRCR_RESET;             /* программный сброс: ОЗУ не трогается */
    __asm volatile("dsb" ::: "memory");
    for (;;) {
    }
}

__attribute__((naked)) void HardFault_Handler(void)
{
    __asm volatile(
        "tst   lr, #4      \n"           /* бит 2 EXC_RETURN: какой стек */
        "ite   eq          \n"
        "mrseq r0, msp     \n"
        "mrsne r0, psp     \n"
        "mov   r1, lr      \n"           /* EXC_RETURN — вторым аргументом */
        "b     crash_save  \n");
}

app35d.c — the experiment firmware: division by zero, a bad address, output of the record.

/* Раздел 35в: прошивка-опыт. Первый запуск — деление на ноль, второй — чтение по
 * неверному адресу, третий — ждёт команд: d — выдать запись, c — стереть. */
#include "crash35d.h"
#include "regs35d.h"

static void putc1(char c)
{
    while (!(USART1_SR & (1u << 7))) {
    }
    USART1_DR = (uint32_t)c;
}

static void puts1(const char *s)
{
    while (*s)
        putc1(*s++);
}

static void puthex(uint32_t v, int digits)
{
    static const char hex[] = "0123456789abcdef";
    for (int i = (digits - 1) * 4; i >= 0; i -= 4)
        putc1(hex[(v >> i) & 0xFu]);
}

static void put32(const char *name, uint32_t v)
{
    puts1(name);
    puts1("=0x");
    puthex(v, 8);
}

static void crash_report(void)
{
    const uint8_t *b = (const uint8_t *)&crash_rec;
    puts1("CRASH #");
    puthex(crash_rec.count, 1);
    put32(" pc", crash_rec.pc);
    put32(" lr", crash_rec.lr);
    put32(" cfsr", crash_rec.cfsr);
    put32(" hfsr", crash_rec.hfsr);
    put32(" bfar", crash_rec.bfar);
    put32(" csr", crash_rec.reset_csr);
    puts1("\nREC ");
    for (unsigned i = 0; i < sizeof(crash_rec_t); i++)
        puthex(b[i], 2);
    puts1("\n");
}

volatile int32_t sample_sum = 1000;                  /* сумма отсчётов */
volatile int32_t sample_count = 0;                   /* ошибка: ещё ни одного отсчёта */
volatile uint32_t *volatile sensor_reg = (volatile uint32_t *)0x60000000u;
volatile int32_t last_average;
volatile uint32_t sink;

__attribute__((noinline)) int32_t average(int32_t count)
{
    int32_t avg = sample_sum / count;                /* СБОЙ-1 */
    last_average = avg;
    return avg;
}

__attribute__((noinline)) uint32_t read_sensor(volatile uint32_t *reg)
{
    return *reg + 1u;                                /* СБОЙ-2 */
}

__attribute__((noinline)) static void scenario_div(void)
{
    sink = (uint32_t)average(sample_count);          /* ВЫЗОВ-1 */
    puts1("never\n");
}

__attribute__((noinline)) static void scenario_bus(void)
{
    sink = read_sensor(sensor_reg);                  /* ВЫЗОВ-2 */
    puts1("never\n");
}

int main(void)
{
    RCC_APB2ENR |= USART1_EN;
    USART1_BRR = BRR_115200;
    USART1_CR1 = (1u << 13) | (1u << 3) | (1u << 2); /* UE, TE, RE */
    SCB_CCR |= CCR_DIV_0_TRP;                        /* деление на ноль — отказ, а не тихий ноль */

    puts1("boot\n");
    if (crash_valid()) {
        if (crash_rec.reset_csr == CSR_NOT_READ) {
            crash_rec.reset_csr = RCC_CSR;           /* причина сброса после сбоя */
            crash_reseal();
        }
        crash_report();
    } else {
        puts1("no crash record\n");
    }
    RCC_CSR |= RCC_CSR_RMVF;

    uint32_t n = crash_valid() ? crash_rec.count : 0u;
    if (n == 0u)
        scenario_div();
    if (n == 1u)
        scenario_bus();

    puts1("ready: d - dump, c - clear\n");
    for (;;) {
        while (!(USART1_SR & (1u << 5))) {
        }
        char c = (char)USART1_DR;
        if (c == 'd') {
            if (crash_valid())
                crash_report();
            else
                puts1("no crash record\n");
        } else if (c == 'c') {
            crash_rec.magic = 0u;
            puts1("record cleared\n");
        }
    }
}

mem_f4.ld — F407 memory with an area for the crash record.

/* STM32F407: 1 МБ флеш-памяти, 128 КБ ОЗУ; последние 128 байт ОЗУ — под запись о сбое. */
MEMORY
{
  FLASH  (rx)  : ORIGIN = 0x08000000, LENGTH = 1024K
  RAM    (rwx) : ORIGIN = 0x20000000, LENGTH = 128K - 128
  NOINIT (rw)  : ORIGIN = 0x2001FF80, LENGTH = 128
}

Next: section 35г "A log on the device: levels, a ring buffer and binary records instead of printf" — a log you can write to from an interrupt: the device sends a message number, and the format string stays on the computer.

Open in the interactive handbook