All sections › XV. C in depth

STM32F103, STM32F407 · level advanced · 11 min read

123. Undefined behavior: where it hides and how to catch it

Undefined behavior is not a "random result" but a contract: the programmer promises that certain things will not happen in the program, and the compiler builds optimizations on that promise. Break the promise and the compiler is obliged neither to warn you nor to behave consistently. This is the source of the most mysterious firmware bugs: it works in the debug build and breaks in the release build, breaks because of an extra printf, behaves differently on two compilers.

This section shows it by experiment: one and the same text is built in five ways and gives different answers; then it looks at what the compiler's traps catch and what they miss, and what a trap costs on Cortex-M.

Terms used in the documentation

Term Meaning

undefined behavior (UB) the language requires nothing at all

unspecified behavior one of the allowed options, which one is not said

implementation-defined the implementation chooses and documents it

strict aliasing an object may be accessed only through a compatible type

type punning reading an object as a different type

undefined behavior sanitizer (UBSan) checking for UB at run time

trap stopping right at the place of the error

address sanitizer (ASan) checking memory accesses

static analyzer a tool that finds errors without running the code

sequence point / unsequenced a point where side effects are complete / no defined order

Term	Meaning
undefined behavior (UB)	the language requires nothing at all
unspecified behavior	one of the allowed options, which one is not said
implementation-defined	the implementation chooses and documents it
strict aliasing	an object may be accessed only through a compatible type
type punning	reading an object as a different type
undefined behavior sanitizer (UBSan)	checking for UB at run time
trap	stopping right at the place of the error
address sanitizer (ASan)	checking memory accesses
static analyzer	a tool that finds errors without running the code
sequence point / unsequenced	a point where side effects are complete / no defined order

123.1 Three kinds of "not defined"

The standard distinguishes three things, and confusing them is dangerous.

Implementation-defined: the compiler chooses and must document it: the sign of char, right shift of a negative number, the size of long. The result is constant for a given compiler (section 122).
Unspecified: one of the allowed options is chosen, but which one is not said: the order in which function arguments are evaluated.
Undefined: there are no requirements at all: signed overflow, going past the bounds of an array, dereferencing a null pointer, shifting by the width of the type, reading an object through a pointer of an incompatible type, modifying a variable twice without a sequence point.

It is the third kind that gives the optimizer its freedom. It does not "break" the program: it trusts the promise that the program has no such places.

123.2 Experiment: one text, different answers

Two probes from the experiment, built in five ways:

Build	`x + 1 > x` with `INT_MAX`	writing an `int` and a `float` at one address
`gcc -O0`	1	0
`clang -O0`	0	0
`gcc -O2`	1	1
`clang -O2`	1	1
ARM, `-O2`	1	1

None of the builds "made a mistake": with undefined behavior any answer is allowed. The defined version of the same task, the bits of a float copied with memcpy, gave the same result in all five builds: 0x3F800000.

123.3 Signed overflow

__attribute__((noinline)) int plus_one_bigger(int x)
{
    return x + 1 > x;
}

For INT_MAX the addition overflows. The compiler may assume this never happens and replace the expression with 1, and that is what gcc and clang did with optimization. clang without optimization honestly added, got a negative number and answered 0. Hence the classic bug: an overflow check after the operation (if (a + b < a)) disappears from the release build.

Check before the operation, within the range of the type:

int can_increment(int x)
{
    return x < INT_MAX;
}

or add with a built-in check, where no overflow happens at all and you get a flag instead:

return __builtin_add_overflow(a, b, out);

x + 1 > x with INT_MAX in five builds: answers 1 and 0, and how to check for overflow correctly

123.4 Strict aliasing

The standard allows an object to be accessed only through a compatible type (plus char). The compiler considers writing a float at the address of an int impossible:

__attribute__((noinline)) int alias(int *i, float *f)
{
    *i = 1;
    *f = 0.0f;
    return *i;             /* -O2 вправе вернуть 1: float «не может» менять int */
}

With -O0 the function returned 0: it read memory again. With -O2 all three compilers returned 1: a write through a float "could not" change an int. In firmware this happens when parsing packets, *(uint32_t *)&buf[4], and when reading the bits of a float. The correct way is memcpy; the compiler turns it into a single move:

uint32_t u;
memcpy(&u, &f, sizeof u);
return u;

The second honest way is a union (C allows it). The third is the option -fno-strict-aliasing: the Linux kernel and many firmware projects do this, paying with some of the optimizations.

123.5 Out-of-bounds access and shifts

Reading a[4] from an array of four elements may return a neighbouring variable, garbage, or cause a fault: nothing is promised. On a Cortex-M without memory protection this usually "works" until a sensitive variable happens to sit next to the array.

A shift by the width of the type or more is also UB: 1u << 32 on Cortex-M gives 0 in hardware, on x86 it gives 1, but the compiler is obliged to do neither. A safe shift checks the amount:

return n < 32u ? v << n : 0u;

And the index is checked before the access:

return i < n ? a[i] : fallback;

123.6 What the compiler's traps catch

What the undefined-behavior traps of gcc and clang see: the shift and the array overrun are caught, aliasing and pointers are not

The option -fsanitize=undefined inserts checks, and -fsanitize-trap turns whatever is found into an immediate stop. The experiment, with optimization:

Error	gcc	clang
shift by 32	yes	yes
overrun of a local array	yes	yes
overflow in `x + 1 > x`	no	yes
overrun through a pointer	no	no
strict aliasing violation	no	no

The main conclusion: "the trap is silent" does not mean "there is no error". gcc replaced x + 1 > x with 1 before it inserted the check, so there was nothing left to check. The size of the array behind a pointer is unknown, so there is nothing to put a check on. Such errors need other tools: ASan on the computer, static analyzers and coding rules (section 129).

123.7 Traps on the board itself

armclang can do the same for Cortex-M. The addition a + b with a trap becomes four instructions instead of two (ADDS, BX lr):

ADDS  r0, r0, r1
IT    VC
BXVC  lr        ; нет переполнения — возврат
UDF   #0xfe     ; есть — неопределённая команда

The function took 8 bytes instead of 4. On M3/M4 UDF raises a UsageFault with the UNDEFINSTR flag if it is enabled in SHCSR, otherwise a HardFault; on M0 it is a HardFault straight away. The fault happens at the place of the error: the address in the exception frame points to the line with the overflow (section 35 shows how to read it). No run-time library is needed. A sensible practice is to build the debug and test-bench firmware with traps, and the release firmware as you decide: the checks cost space and cycles, but a fault "on the spot" is better than silent data corruption.

123.8 Other places

An uninitialized variable: garbage is read, and the optimizer may assume any value. Caught by the warnings -Wall -Wuninitialized.
i = i++ + 1, f(i++, i++): modification without a sequence point; gcc and clang give different results. Warning -Wsequence-point.
Dereferencing a null pointer: on Cortex-M the vector table sits at address 0, so a read "works", and a write to flash is a fault. The compiler may remove an if (!p) check that comes after *p.
A function without return whose value is used is UB.
An empty loop with a non-constant condition: if it has no volatile, no atomic operations and no input/output, C11 allows it to be assumed to terminate: the compiler may throw away while (!flag) { } with an ordinary flag. An endless for (;;) { } with a constant condition is not covered by this rule and stays. Loops that wait for a flag are written with volatile (section 124).

123.9 How to live with it in firmware

Keep -Wall -Wextra on and read the warnings: half of UB is visible to the compiler in advance.
Build the debug build with UB traps; unit tests on the computer with ASan as well.
Types and bits go through memcpy, never through pointer casts.
Checks come before the operation: x < INT_MAX, i < n, n < 32.
Signed arithmetic that can go out of range goes through __builtin_add_overflow or is done in an unsigned type.
If the firmware is inherited and full of pointer casts, -fno-strict-aliasing is more honest than hope.

123.10 Pitfalls

Works with -O0, breaks with -O2. Almost always UB: overflow, aliasing, an uninitialized variable. Build with traps and -Wall.

The overflow check disappeared from the release build. The check comes after the operation: the compiler assumes overflow never happens. Check before.

Packet parsing gives garbage after the optimization level changes. *(uint32_t *)&buf[i] is an aliasing violation; on Cortex-M0 it is also an unaligned access, and on M3/M4 the compiler may merge two such reads into LDRD or LDM, which do not tolerate an unaligned address. Use memcpy.

The traps found nothing, but the error is there. The traps do not see everything: aliasing and overruns through a pointer are beyond them.

HardFault on UDF in the debug firmware. This is not a malfunction but a UB trap that fired: the address in the exception frame points to the guilty line.

123.11 Summary

Undefined behavior is a promise to the compiler; break it and any answer is possible: x + 1 > x gave 0 with clang -O0 and 1 with the others, the write through a float gave 0 without optimization and 1 with it.
The defined path is the same everywhere: the bits of a float via memcpy are 0x3F800000 in all five builds.
UB traps catch shifts and array overruns, clang also catches overflow; nobody sees aliasing and pointers.
On Cortex-M a trap is UDF and a HardFault at the place of the error: 8 bytes instead of 4 for an addition.
Check before the operation, bits through memcpy, warnings on.

Full source code

ub123.c contains the undefined-behavior probes for gcc and clang.

/* Раздел 123: неопределённое поведение — нарочно, чтобы увидеть, как оно выглядит.
 * Каждая проба выбирается аргументом командной строки; входные значения идут
 * через volatile, чтобы компилятор не посчитал всё заранее. Сборки: gcc и clang
 * с -O0 и -O2, ARM с -O2 и те же пробы с ловушками UB (_verify/c123_check.py).
 * Разные ответы одного и того же текста — и есть признак UB. */
#include <limits.h>
#include <stdio.h>
#include <string.h>

volatile int g_in = INT_MAX;

/* 1. Переполнение знакового: компилятор вправе считать, что x + 1 > x всегда. */
__attribute__((noinline)) int plus_one_bigger(int x)
{
    return x + 1 > x;
}

/* 2. Нарушение правила строгого псевдонима: int и float по одному адресу. */
__attribute__((noinline)) int alias(int *i, float *f)
{
    *i = 1;
    *f = 0.0f;
    return *i;             /* -O2 вправе вернуть 1: float «не может» менять int */
}

/* То же без UB: байты копируются, компилятор знает, что они общие. */
__attribute__((noinline)) unsigned float_bits(float f)
{
    unsigned u;
    memcpy(&u, &f, sizeof u);
    return u;
}

/* 3. Выход за границу массива — чтение соседней памяти или ловушка. */
__attribute__((noinline)) int read_at(const int *a, int i)
{
    return a[i];
}

/* 4. Сдвиг на ширину типа. */
__attribute__((noinline)) unsigned shift_by(unsigned v, int n)
{
    return v << n;
}

int main(int argc, char **argv)
{
    const char *probe = argc > 1 ? argv[1] : "all";
    if (!strcmp(probe, "overflow") || !strcmp(probe, "all"))
        printf("overflow=%d\n", plus_one_bigger(g_in));
    if (!strcmp(probe, "alias") || !strcmp(probe, "all")) {
        int storage = 0;
        printf("alias=%d\n", alias(&storage, (float *)&storage));
    }
    if (!strcmp(probe, "bits") || !strcmp(probe, "all"))
        printf("bits=0x%08X\n", float_bits(1.0f));
    if (!strcmp(probe, "bounds")) {
        int a[4] = { 1, 2, 3, 4 };
        volatile int i = 4;
        printf("bounds=%d\n", read_at(a, i));
    }
    if (!strcmp(probe, "bounds_direct")) {
        int a[4] = { 1, 2, 3, 4 };
        volatile int i = 4;
        printf("bounds_direct=%d\n", a[i]);       /* размер массива известен здесь же */
    }
    if (!strcmp(probe, "shift")) {
        volatile int n = 32;
        printf("shift=%u\n", shift_by(1u, n));
    }
    return 0;
}

ub123_arm.c is the same without undefined behavior, for the board, with and without UB traps.

/* Раздел 123: как писать без неопределённого поведения — то же на плате.
 * Собирается armclang для F103 и F407 дважды: обычно и с ловушками UB
 * (-fsanitize=undefined -fsanitize-trap=undefined): тогда нарушение превращается
 * в неопределённую команду, и ядро уходит в HardFault прямо на месте ошибки. */
#include <limits.h>
#include <stdint.h>
#include <string.h>

/* Биты float — через memcpy, а не через приведение указателя. */
uint32_t float_bits(float f)
{
    uint32_t u;
    memcpy(&u, &f, sizeof u);
    return u;
}

/* Сложение со знаком с проверкой: переполнения не случается, есть признак. */
int add_checked(int a, int b, int *out)
{
    return __builtin_add_overflow(a, b, out);
}

/* Проверка до операции, в пределах типа. */
int can_increment(int x)
{
    return x < INT_MAX;
}

/* Сдвиг с проверкой величины: сдвиг на 32 и больше — неопределённое поведение. */
uint32_t shl(uint32_t v, unsigned n)
{
    return n < 32u ? v << n : 0u;
}

/* Индекс проверяется до обращения. */
int read_checked(const int *a, unsigned n, unsigned i, int fallback)
{
    return i < n ? a[i] : fallback;
}

/* Эта функция с ловушками UB превращает переполнение в останов. */
int sum_unchecked(int a, int b)
{
    return a + b;
}

Next: section 124 "volatile, const and atomicity" — what volatile promises, what it does not, and how to share data with an interrupt.

Open in the interactive handbook