32. Stream and message buffers

A queue stores items of one fixed size — and that is its strength, as long as the data fits into such items. Bytes from a UART do not fit: they arrive as a stream without boundaries. Protocol frames do not fit either: their length varies. For both cases the kernel provides separate buffers — and they are simpler than a queue, because only half of its capabilities are left.

Terms used in the documentation

Term	Meaning
stream buffer	a ring buffer of bytes with waiting
message buffer	a buffer that keeps message boundaries
trigger level	the threshold at which the reader is woken
single writer / single reader	exactly one writer / exactly one reader
bytes available	how much is stored
spaces available	how much still fits
discrete message	a separate message with boundaries
completed callback	a callback on completion
inter-core	between processor cores

32.1 The stream buffer

A ring buffer of bytes with waiting. The writer puts in any number of bytes, the reader takes as many as it needs; record boundaries are not preserved.

static StreamBufferHandle_t sb_rx;

/* в main, до vTaskStartScheduler */
sb_rx = xStreamBufferCreate(128, 16);      /* размер, уровень триггера */
configASSERT(sb_rx != NULL);

The header is half the size of a queue's, and the bytes are packed densely — without rounding each item up to a word.

⚠️ One byte of storage is always empty — this is how a ring buffer with two indexes tells "full" from "empty". Dynamic creation adds this byte by itself: xStreamBufferCreate(128, …) holds exactly 128 bytes. Static creation takes the array as is: xStreamBufferCreateStatic(128, …) with a 128-byte array holds 127. To fit N bytes, declare the array as N + 1 and pass the size N + 1.

Writing and reading

/* прерывание: байт пришёл */
xStreamBufferSendFromISR(sb_rx, &byte, 1, &woken);
portYIELD_FROM_ISR(woken);

/* задача: забрать порцию */
n = xStreamBufferReceive(sb_rx, chunk, sizeof(chunk), pdMS_TO_TICKS(50));

Both functions return the number of bytes, not a success flag. A write returns less than requested if there was not enough space; a read returns as much as it could take, down to zero when the timeout expires.

⚠️ From an interrupt a byte can be lost silently. SendFromISR cannot wait; if it returned zero, the buffer is full and the byte is gone. The return value is usually not checked in a handler, and this is the only place where data loss leaves no trace. Count the failures.

32.2 The trigger level

The second creation argument is the reason to prefer a stream buffer over a queue for UART reception.

The reader wakes up not on the first byte, but when at least the trigger level has accumulated.

What this means in numbers. At 115200 baud, 11520 bytes arrive per second. A queue would wake the task on each one: 11520 wakeups and context switches per second. With a trigger of 16 — 720, sixteen times fewer; the processor is busy parsing, not waking up.

This is how it looks in time — the first three milliseconds of reception:

Two subtleties:

The trigger is a wakeup threshold, not a guaranteed chunk size. If the timeout expires, the reader gets everything that is stored — even a single byte. Data does not get stuck waiting for the threshold.

Zero turns into one silently. If you want "on the first byte", set 1 explicitly.

The trigger must not exceed the buffer size, otherwise the reader will never wake up — the kernel catches this with a check at creation.

32.3 The rule of one

⚠️ One writer and one reader. For the sake of speed the buffer is not protected against two tasks writing at the same time, or against two reading. Two tasks on one buffer corrupt it without any diagnostics.

This is not a flaw but a deliberate trade-off: a queue pays for many writers with critical sections on every operation, the buffer does not. Hence its purpose: a link between two sides — an interrupt and a task, a task and a task, a core and a core.

If you need many writers — either a mutex before each write, or a queue, which can do this by itself.

32.4 The message buffer

The same stream buffer, but with boundaries: each message is preceded by its length, and the reader gets exactly one whole message.

static MessageBufferHandle_t mb;

/* в main, до vTaskStartScheduler */
mb = xMessageBufferCreate(256);
configASSERT(mb != NULL);

/* отправитель: кадр любой длины */
xMessageBufferSend(mb, frame, len, pdMS_TO_TICKS(10));

/* получатель: len — длина именно этого кадра */
len = xMessageBufferReceive(mb, frame, sizeof(frame), portMAX_DELAY);

There is no trigger — a message is either stored whole or not at all.

The price of boundaries is four bytes per message: the length is stored as size_t. Hence the storage size: the sum of the lengths plus four for each one.

A comparison with a queue for variable-length frames shows why this is needed:

A queue reserves the maximum for each frame. A message buffer stores a frame at its actual length: short frames take less space, and seven times as many of them fit into the same 256 bytes.

All or nothing

if (xMessageBufferSend(mb, frame, len, 0) == 0) {
    /* места под ВСЁ сообщение нет — не положено ничего */
}

The buffer will not accept half a message: either all of it, or a refusal. This is more convenient than a queue, where an overflow cuts the stream at an item boundary rather than a frame boundary.

⚠️ The receive buffer must hold the longest message. If a message is longer than the receiver, the function returns zero, and the message stays in the buffer — and so on every call, forever. The device looks as if "data has stopped arriving". Size the receiver for the maximum frame, without economizing.

32.5 Monitoring and reset

size_t used = xStreamBufferBytesAvailable(sb);     /* сколько лежит  */
size_t free = xStreamBufferSpacesAvailable(sb);    /* сколько влезет */

A growing used with a constant free means the reader is not keeping up, and soon SendFromISR will start returning zeros. The same functions exist for the message buffer under the corresponding names.

xStreamBufferReset clears the buffer — but only if nobody is waiting on it; otherwise it returns a failure. Deletion is vStreamBufferDelete, and it is not allowed with waiting tasks either.

There are also callbacks: the kernel can call a function on every write and on every read. They are needed for one application — the next one.

32.6 Communication between cores

The buffers are designed so that no kernel functions are called from the wakeup handler, and this makes them suitable for communication between two processor cores with independent copies of FreeRTOS.

The scheme: the buffer lives in shared memory; core A writes, core B reads. The reader on core B is woken not by the scheduler of core A — it knows nothing about B's tasks — but by an inter-processor interrupt, which core A raises in the write callback, while the handler on core B calls xMessageBufferSendCompletedFromISR (for a stream buffer — xStreamBufferSendCompletedFromISR): "data written, wake the reader". The reverse path — …ReceiveCompletedFromISR — wakes the writer when the reader has freed space.

The F103 and F407 have a single core, and this subsection is here to explain why the buffer is built exactly this way. On dual-core families (H745/H747, WL5x) it is one of the standard ways of exchange; next to it are the hardware semaphores HSEM. On the M7 core with a data cache the shared region is made non-cacheable, otherwise the second core will see stale contents.

32.7 What to choose

A practical rule: UART reception — a stream buffer with a trigger, frame parsing from it — in a task; the parsing result, if it is needed further on, — into a message buffer or a queue.

32.8 Pitfalls

UART bytes are lost under load. The buffer is full, SendFromISR returns zero. Enlarge the buffer or raise the reader's priority.

The task wakes up on every byte. The trigger is 1 or 0. Set a reasonable threshold.

The task never wakes up. The trigger is larger than the buffer size — the kernel should have caught it at creation; or nobody is there to wait for — the writer is a different one and the data goes into a different buffer.

Data gets corrupted. Two writers or two readers on one buffer.

xMessageBufferReceive always returns zero. The receiver is smaller than the message; it is stuck forever. Enlarge the receiver.

xMessageBufferSend does not store, although there is space. There is no space for the whole message with its four length bytes.

Reset returns a failure. Someone is waiting on the buffer.

A static buffer holds one byte less than declared. One byte of storage is always empty, and …CreateStatic does not add it: the array and the size are N + 1.

32.9 Summary

• A stream buffer — bytes without boundaries, a 36-byte header, one writer and one reader.
• One byte of storage is always empty — it tells "full" from "empty"; Create adds it by itself, for CreateStatic the array and the size are N + 1.
• The trigger is the reader's wakeup threshold; at 115200 baud a trigger of 16 cuts wakeups from 11520 to 720 per second.
• The trigger does not delay data: when the timeout expires the reader gets everything there is; zero means one.
• The functions return the number of bytes, not a flag; SendFromISR loses bytes silently if the buffer is full.
• A message buffer adds boundaries at the price of four bytes per message; it stores the whole message or nothing.
• The receiver must hold the longest message, otherwise it gets stuck forever.
• The buffers do not call kernel functions from the wakeup — that is why they are suitable for communication between processor cores.

Next: section 32а "Active objects: an event-driven device architecture — a task, its own queue and a state machine" — how to assemble a whole device from tasks, queues and timers: each object waits only on its own event queue.