194. Link to the microcontroller: the port on the Linux side, COBS frames and CRC

In section 191.1 the device is split in two: the microcontroller handles real time, the Linux processor handles the network and storage. Most often they talk over UART. On the microcontroller side the port is simple (section 16), but on the Linux side a terminal driver sits between the wire and the program — and by default it corrupts binary data. This section is about how to turn that off and how to cut the byte stream into frames that survive noise.

The experiment uses a pair of pseudo-terminals (openpty) in the stm-linux:1 image: they have the same tty driver as /dev/ttyS0 and /dev/ttyUSB0, so the port behaves the same way. The program is _verify/linux194/link194.c, checked by _verify/linux194_check.py.

Terms used in the documentation

Term	Meaning
canonical mode	the driver hands over data line by line
raw mode	bytes as they are
termios	the structure of terminal settings
line discipline	stream processing by the terminal driver
software flow control (XON/XOFF)	flow control with the bytes 0x11 and 0x13
consistent overhead byte stuffing (COBS)	an encoding that removes zeros from the data
frame delimiter	the byte that separates frames
resynchronization	recovery after a fault in the stream

194.1 By default the port corrupts data

You opened /dev/ttyS0, the microcontroller wrote four bytes 11 0D 22 33 into it — and you read one byte 0A. That is exactly what happened in the experiment:

Three settings that are on by default for a port are to blame:

Plus ECHO: the driver sends what it received back down the line, and the microcontroller gets its own bytes. For a terminal with a human all this is convenient; for exchanging data with a device it is a disaster, and one that does not always show up, only on packets that happen to contain 0x0D, 0x11 or 0x13.

194.2 Raw mode

One function turns off almost everything unwanted:

#include <termios.h>

int fd = open("/dev/ttyS0", O_RDWR | O_NOCTTY);
struct termios t;
tcgetattr(fd, &t);
cfmakeraw(&t);                 /* ICANON, ECHO, ICRNL, IXON, OPOST — выключены */
cfsetispeed(&t, B115200);
cfsetospeed(&t, B115200);
t.c_cc[VMIN]  = 0;
t.c_cc[VTIME] = 5;             /* ждать до 0,5 с, в десятых долях секунды */
t.c_cflag |= CLOCAL | CREAD;   /* не ждать сигнала DCD, включить приём */
t.c_cflag &= ~(CSTOPB | CRTSCTS);   /* один стоп-бит, без RTS/CTS */
t.c_iflag &= ~(IXOFF | IXANY);
tcsetattr(fd, TCSANOW, &t);
tcflush(fd, TCIFLUSH);         /* выбросить то, что пришло до настройки */

After it, in the experiment ICANON, ECHO, ICRNL and OPOST are cleared, and the same four bytes arrive as they are: 11 0D 22 33.

cfmakeraw does not touch everything: in c_cflag it sets eight bits and clears parity, but leaves as they were two stop bits CSTOPB, hardware flow control CRTSCTS, CLOCAL and CREAD, and in c_iflag, IXOFF and IXANY. tcgetattr returns whatever the previous program left on the port, so these flags are set explicitly — the lines after VTIME. tcflush discards the bytes accumulated before the setup.

The baud rate is set not with a number but with a constant: B115200 is 0x1002, not 115200 (this is so in Linux with glibc; in BSD and macOS the constant equals the number). Passing the number 115200 to cfsetispeed is a common mistake: the compiler will not notice it, and the call returns −1. The constant works everywhere, so use it.

How long read waits is set by two fields:

For a device you usually take VMIN = 0, VTIME of a few tenths of a second, or poll with a timeout: the program does not hang forever if the microcontroller goes silent, and can notice that itself.

194.3 Frames: why a delimiter

UART carries a stream of bytes, not packets. If the receiver started in the middle of a transmission or noise corrupted a byte, you need to find the start of the next packet. Hence the rule: the stream must contain a byte that occurs only at a frame boundary. Zero is the most convenient.

But data almost always contains zeros: the packet number, a temperature, counters. The COBS encoding removes them. The data is cut into pieces up to the next zero, each piece is preceded by its length plus one, and the zero itself is dropped:

данные  11 00 22
COBS    02 11 02 22        02 — «один байт, потом был ноль», 02 — «один байт до конца»
кадр    02 11 02 22 00     00 — конец кадра, внутри нулей больше нет

The cost is predictable, hence the name (consistent overhead):

Some libraries do not write the last code 01 after a full block of 254 bytes and give 255 and 510 bytes; the decoder accepts both variants, but the frame lengths on the two sides may differ. No more than one byte for every 254 bytes of data plus one at the start — unlike "escaping" of special characters, where a packet made only of special characters grows to twice its size.

194.4 The whole frame: data, CRC, COBS, zero

The order on the sending side — on the microcontroller or on Linux, the code is one and the same:

1. append a CRC-16 to the data (section 43.1; here CRC-16/XMODEM: polynomial 0x1021, initial value 0 — for the string 123456789 it gives 0x31C3, as on the "CRC" tab of the calculators);
2. encode all of it in COBS;
3. append the 00 delimiter.

size_t frame(const uint8_t *data, size_t n, uint8_t *out)
{
    uint8_t buf[256];
    if (n > sizeof buf - 2) return 0;     /* данные + CRC должны поместиться */
    memcpy(buf, data, n);
    uint16_t c = crc16(data, n);
    buf[n] = c >> 8;
    buf[n + 1] = c & 0xFF;
    size_t m = cobs_encode(buf, n + 2, out);
    out[m++] = 0;                         /* конец кадра */
    return m;
}

The telemetry packet in the experiment is eight bytes with three zeros inside: 01 00 09 C4 0C E4 00 00 (number, temperature in hundredths of a degree, voltage in millivolts). The frame came out at 12 bytes:

02 01 05 09 C4 0C E4 01 03 86 4C 00

There are no zeros inside the frame; the last byte is the delimiter. At 115200 baud with 10 bits per byte this frame takes 1042 µs on the line.

Reception is the mirror image: collect bytes up to a zero, decode COBS, check the CRC. If it does not match, drop the frame, and from the next zero everything is fine again. In the experiment there were three frames in a row, with one byte corrupted in the second: the receiver accepted the first and the third and dropped the second on CRC — one loss, without breaking the whole stream.

for (;;) {
    int n = read(fd, rx, sizeof rx);      /* VMIN = 0, VTIME = 5 */
    for (int i = 0; i < n; i++) {
        if (rx[i] != 0) {
            if (al < sizeof acc) acc[al++] = rx[i];
            else overflow = 1;                /* длиннее допустимого — кадр испорчен */
            continue;
        }
        if (overflow) { overflow = 0; al = 0; errors++; continue; }
        long dl = cobs_decode(acc, al, dec);
        al = 0;
        if (dl >= 3 && crc16(dec, dl - 2) == (dec[dl - 2] << 8 | dec[dl - 1]))
            handle(dec, dl - 2);          /* целый кадр */
        else
            errors++;                     /* испорчен — ждём следующий ноль */
    }
}

The acc accumulator is sized for the longest frame: 254 bytes of data plus CRC after COBS is 258 bytes; anything longer is certainly garbage.

On the microcontroller it is convenient to receive the same thing through DMA with the "line went idle" flag (section 16.5): the 00 delimiter and the pause after a frame together give a reliable boundary.

194.5 Pitfalls

The program reads "the wrong thing" only sometimes. The port is in canonical mode: flow control swallows 0x11 and 0x13, 0x0D becomes 0x0A, data is handed over in lines. Packets without these bytes get through, and the error looks random. The cure is cfmakeraw once when opening the port.

The microcontroller gets its own bytes back. Echo is on (ECHO). The same cfmakeraw.

cfsetispeed(&t, 115200). The baud rate is the constant B115200, that is, 0x1002; the number 115200 gives a different rate or an error.

read hangs forever. VMIN > 0 with VTIME = 0: it waits until that many bytes have arrived. For a device use VMIN = 0 and a timeout, or poll.

The data contains kernel messages or a login prompt. The port is taken by the console: the kernel writes its messages to it, the login service responds to bytes. Remove the port from the console= parameter of the kernel command line and disable the login service on it: systemctl mask serial-getty@ttyS0.

open returns "Permission denied". The user is not in the dialout group that owns the serial ports: usermod -aG dialout name and log in again.

A frame got glued to its neighbor. There is no delimiter, and the receiver guesses boundaries from pauses — after the first delay in the transmission it guesses wrong. The 00 delimiter after COBS makes the boundary unambiguous.

You checked the CRC before decoding COBS. The checksum is computed over the original data, not over the encoded bytes.

The receive buffer overflowed on garbage. If zeros do not arrive for a long time (noise, wrong baud rate), the accumulator must be reset by length: a frame longer than the longest allowed one is certainly corrupted.

194.6 Summary

• By default a tty port works for a human: ICANON, ECHO, ICRNL and IXON are on. In the experiment, out of 11 0D 22 33 the program got 0A, the rest waited for a newline.
• cfmakeraw turns all of that off; the baud rate is the constant B115200 (0x1002).
• VMIN = 0, VTIME = 5 — wait no longer than half a second; VTIME = 0 — do not wait.
• The stream is cut into frames with the 00 delimiter, and COBS removes the zeros inside: no more than a byte per 254 bytes of data (254 → 256, 508 → 511).
• Frame = data + CRC-16 → COBS → 00. An 8-byte telemetry packet gave a frame of 12 bytes, 1042 µs at 115200 baud; a corrupted frame is dropped on CRC, the next one is received from the nearest zero.
• The same framing code works on both sides — on the STM32 and on Linux.

Full source code

link194.c — the section's experiment: raw port mode, VMIN/VTIME and COBS + CRC frames on a pair of pseudo-terminals.

/* Раздел 194: Linux-половина прибора говорит с микроконтроллером по UART.
 * Опыт на паре псевдотерминалов (openpty) — тот же драйвер tty, что и у /dev/ttyS0:
 *  1) что делает с байтами «обычный» режим порта и что меняет cfmakeraw;
 *  2) VMIN/VTIME: сколько ждёт read;
 *  3) кадры COBS + CRC-16/XMODEM: кодирование, накладные расходы, пересинхронизация
 *     после испорченного кадра.
 * Печатает строки ключ=значение; сверяет _verify/linux194_check.py. */
#define _DEFAULT_SOURCE
#include <errno.h>
#include <poll.h>
#include <pty.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <termios.h>
#include <time.h>
#include <unistd.h>

/* ── CRC-16/XMODEM: полином 0x1021, начальное 0 — как в калькуляторе и разделе 43 ── */
static uint16_t crc16(const uint8_t *p, size_t n)
{
    uint16_t c = 0;
    while (n--) {
        c ^= (uint16_t)(*p++) << 8;
        for (int i = 0; i < 8; i++)
            c = (c & 0x8000u) ? (uint16_t)((c << 1) ^ 0x1021u) : (uint16_t)(c << 1);
    }
    return c;
}

/* ── COBS: убирает нули из данных, чтобы ноль означал только «конец кадра» ── */
static size_t cobs_encode(const uint8_t *in, size_t n, uint8_t *out)
{
    size_t w = 1, code_at = 0;
    uint8_t code = 1;
    for (size_t i = 0; i < n; i++) {
        if (in[i] == 0) {
            out[code_at] = code;
            code_at = w++;
            code = 1;
        } else {
            out[w++] = in[i];
            if (++code == 0xFF) {
                out[code_at] = code;
                code_at = w++;
                code = 1;
            }
        }
    }
    out[code_at] = code;
    return w;
}

static long cobs_decode(const uint8_t *in, size_t n, uint8_t *out)
{
    size_t r = 0, w = 0;
    while (r < n) {
        uint8_t code = in[r++];
        if (code == 0)
            return -1;
        for (uint8_t i = 1; i < code; i++) {
            if (r >= n)
                return -1;
            out[w++] = in[r++];
        }
        if (code != 0xFF && r < n)
            out[w++] = 0;
    }
    return (long)w;
}

/* кадр = COBS(данные + CRC старшим байтом вперёд) + 0x00 */
static size_t frame(const uint8_t *data, size_t n, uint8_t *out)
{
    uint8_t buf[600];
    memcpy(buf, data, n);
    uint16_t c = crc16(data, n);
    buf[n] = (uint8_t)(c >> 8);
    buf[n + 1] = (uint8_t)c;
    size_t m = cobs_encode(buf, n + 2, out);
    out[m++] = 0;
    return m;
}

static double ms_since(const struct timespec *t0)
{
    struct timespec t;
    clock_gettime(CLOCK_MONOTONIC, &t);
    return (t.tv_sec - t0->tv_sec) * 1e3 + (t.tv_nsec - t0->tv_nsec) / 1e6;
}

int main(void)
{
    int m, s;
    if (openpty(&m, &s, NULL, NULL, NULL) != 0) {
        perror("openpty");
        return 1;
    }

    /* ── 1. порт «как есть»: канонический режим, эхо, перевод CR ─────────── */
    struct termios t;
    tcgetattr(s, &t);
    printf("default_icanon=%d\n", (t.c_lflag & ICANON) ? 1 : 0);
    printf("default_echo=%d\n", (t.c_lflag & ECHO) ? 1 : 0);
    printf("default_icrnl=%d\n", (t.c_iflag & ICRNL) ? 1 : 0);
    printf("default_ixon=%d\n", (t.c_iflag & IXON) ? 1 : 0);
    printf("B115200=0x%X\n", (unsigned)B115200);

    /* четыре двоичных байта. В каноническом режиме: 0x11 — это XON, его съедает
     * управление потоком; 0x0D превращается в 0x0A и заканчивает «строку»;
     * 0x22 0x33 ждут следующего перевода строки */
    const uint8_t bin[4] = {0x11, 0x0D, 0x22, 0x33};
    write(m, bin, sizeof bin);
    struct pollfd pf = {.fd = s, .events = POLLIN};
    uint8_t got[16];
    int k = poll(&pf, 1, 200) > 0 ? (int)read(s, got, sizeof got) : 0;
    printf("canon_first=");
    for (int i = 0; i < k; i++)
        printf("%s%02X", i ? " " : "", got[i]);
    printf("\n");
    printf("canon_waiting=%d\n", poll(&pf, 1, 200));   /* 0x22 0x33 без конца строки не отдаются */
    write(m, "\n", 1);
    k = poll(&pf, 1, 200) > 0 ? (int)read(s, got, sizeof got) : 0;
    printf("canon_second=");
    for (int i = 0; i < k; i++)
        printf("%s%02X", i ? " " : "", got[i]);
    printf("\n");
    tcflush(s, TCIOFLUSH);
    tcflush(m, TCIOFLUSH);

    /* ── 2. сырой режим: байты как есть ───────────────────────────────── */
    cfmakeraw(&t);
    cfsetispeed(&t, B115200);
    cfsetospeed(&t, B115200);
    t.c_cc[VMIN] = 0;
    t.c_cc[VTIME] = 5;      /* десятые доли секунды: 0,5 с */
    tcsetattr(s, TCSANOW, &t);
    struct termios r;
    tcgetattr(s, &r);
    printf("raw_icanon=%d\n", (r.c_lflag & ICANON) ? 1 : 0);
    printf("raw_echo=%d\n", (r.c_lflag & ECHO) ? 1 : 0);
    printf("raw_icrnl=%d\n", (r.c_iflag & ICRNL) ? 1 : 0);
    printf("raw_opost=%d\n", (r.c_oflag & OPOST) ? 1 : 0);
    printf("raw_speed_ok=%d\n", cfgetispeed(&r) == B115200);

    write(m, bin, sizeof bin);
    usleep(50000);
    k = (int)read(s, got, sizeof got);
    printf("raw_bytes=");
    for (int i = 0; i < k; i++)
        printf("%s%02X", i ? " " : "", got[i]);
    printf("\n");

    /* VMIN=0, VTIME=5: пустой порт — read вернёт 0 через полсекунды */
    struct timespec t0;
    clock_gettime(CLOCK_MONOTONIC, &t0);
    k = (int)read(s, got, sizeof got);
    double waited = ms_since(&t0);
    printf("vtime_read=%d\n", k);
    printf("vtime_ms_ok=%d\n", waited >= 450 && waited <= 1500);

    /* VMIN=0, VTIME=0: не ждёт вовсе */
    r.c_cc[VTIME] = 0;
    tcsetattr(s, TCSANOW, &r);
    clock_gettime(CLOCK_MONOTONIC, &t0);
    k = (int)read(s, got, sizeof got);
    waited = ms_since(&t0);
    printf("poll_read=%d\n", k);
    printf("poll_ms_ok=%d\n", waited < 50);

    /* ── 3. кадры COBS + CRC ──────────────────────────────────────────── */
    printf("crc_123456789=0x%04X\n", crc16((const uint8_t *)"123456789", 9));

    uint8_t enc[700];
    const uint8_t ex[3] = {0x11, 0x00, 0x22};
    size_t n = cobs_encode(ex, 3, enc);
    printf("cobs_example=");
    for (size_t i = 0; i < n; i++)
        printf("%s%02X", i ? " " : "", enc[i]);
    printf("\n");

    uint8_t big[600];
    for (int i = 0; i < 254; i++)
        big[i] = (uint8_t)(i + 1);            /* 254 байта без единого нуля */
    printf("cobs_254_nonzero=%zu\n", cobs_encode(big, 254, enc));
    memset(big, 0, 10);
    printf("cobs_10_zeros=%zu\n", cobs_encode(big, 10, enc));
    for (int i = 0; i < 508; i++)
        big[i] = (uint8_t)(i % 255 + 1);
    printf("cobs_508_nonzero=%zu\n", cobs_encode(big, 508, enc));

    /* пакет телеметрии: номер, температура ×100, напряжение в мВ — с нулями внутри */
    const uint8_t pkt[8] = {0x01, 0x00, 0x09, 0xC4, 0x0C, 0xE4, 0x00, 0x00};
    uint8_t f[40];
    size_t fl = frame(pkt, sizeof pkt, f);
    printf("frame_len=%zu\n", fl);
    printf("frame_zeros_inside=%d\n", (int)(memchr(f, 0, fl - 1) != NULL));
    printf("frame_bytes=");
    for (size_t i = 0; i < fl; i++)
        printf("%s%02X", i ? " " : "", f[i]);
    printf("\n");

    /* поток из трёх кадров, в среднем испорчен один байт — приёмник теряет только его */
    uint8_t stream[200];
    size_t sl = 0;
    for (int j = 0; j < 3; j++) {
        uint8_t p[8];
        memcpy(p, pkt, 8);
        p[0] = (uint8_t)(j + 1);
        sl += frame(p, 8, stream + sl);
    }
    stream[fl + 3] ^= 0x40;                  /* помеха в середине второго кадра */
    write(m, stream, sl);
    usleep(50000);
    uint8_t rx[200];
    int got_n = (int)read(s, rx, sizeof rx);
    printf("stream_len=%zu\n", sl);
    printf("stream_rx=%d\n", got_n);

    int good = 0, bad = 0;
    uint8_t acc[200], dec[200];
    size_t al = 0;
    printf("frames_ok=");
    for (int i = 0; i < got_n; i++) {
        if (rx[i] != 0) {
            acc[al++] = rx[i];
            continue;
        }
        long dl = cobs_decode(acc, al, dec);
        al = 0;
        if (dl < 3 || crc16(dec, (size_t)dl - 2) != (uint16_t)((dec[dl - 2] << 8) | dec[dl - 1])) {
            bad++;
            continue;
        }
        printf("%s%d", good ? " " : "", dec[0]);
        good++;
    }
    printf("\n");
    printf("frames_good=%d\n", good);
    printf("frames_bad=%d\n", bad);

    /* 115200 бод, 8N1 — 10 бит на байт: время кадра в линии */
    printf("frame_us_115200=%.0f\n", fl * 10.0 / 115200 * 1e6);
    printf("done=1\n");
    return 0;
}

Next: section 195 "A/B update: two slots, confirmation and rollback".