105. The network from the bottom up: Ethernet, MAC, IP, subnets, ARP

The sockets in section 78 used the network as a ready-made service: an address, a port, and the data goes. This section shows what lies beneath them. A device sends an Ethernet frame onto the wire, and to send it to a neighbour it needs the neighbour's MAC address, while all the device knows is its IP. Between the two sit the subnet mask, the gateway and ARP. Every frame in this section is built by portable C code, and on the PC the same code talks to a real Linux kernel in a neighbouring container: the ARP request and the ping go out through a raw socket, and the replies are parsed by the same functions. The kernel silently drops a frame with a wrong checksum, so if it replied, the checksums were computed correctly.

Sockets are in section 78, network types in section 86, containers in section 87.

Terms used in the documentation

Term	Meaning
frame	a unit of data on the Ethernet link
preamble, start frame delimiter (SFD)	the preamble, the start of the frame
MAC address	the physical address of a network interface
organizationally unique identifier (OUI)	the manufacturer code in a MAC address
locally administered address	an address assigned locally
EtherType	the type of the frame's payload
frame check sequence (FCS)	the frame checksum, CRC-32
interframe gap	the pause between frames
subnet mask, prefix length	the subnet mask, the prefix length
default gateway	the gateway used by default
broadcast address	the address of every node on the subnet
address resolution protocol (ARP)	finding a MAC address by an IP address
ones' complement checksum	a checksum in ones' complement arithmetic
time to live (TTL)	the packet's lifetime: how many more routers it may pass
raw socket	a socket for whole frames, bypassing the stack

105.1 The Ethernet frame

A frame is 64 to 1518 bytes long, and on the wire, with the preamble and the gap, it takes 84 to 1538 byte times. At 100 Mbit/s that is 148810 of the shortest frames per second or 8127 of the longest. Payload takes 97.53 % of line time in a long frame and 54.76 % in a short one. The shortest frame lasts 5.12 µs.

The F407 has a built-in MAC, while the transceiver (PHY) is a separate chip. Between them runs RMII — 50 MHz, 2 bits at a time — or MII — 25 MHz, 4 bits at a time: both give 100 Mbit/s. The F103 has no MAC — it is connected to a network through an external controller on SPI.

Building a frame means putting the fields together in order:

unsigned eth_frame(uint8_t *f, const uint8_t dst[6], const uint8_t src[6], uint32_t type,
                   const uint8_t *data, unsigned n)
{
    memcpy(f, dst, 6);
    memcpy(f + 6, src, 6);
    put16(f + 12, type);
    memcpy(f + ETH_HDR, data, n);
    unsigned len = ETH_HDR + n;
    if (len < ETH_MIN) {
        memset(f + len, 0, ETH_MIN - len);
        len = ETH_MIN;
    }
    uint32_t fcs = eth_crc32(f, len);
    for (unsigned i = 0; i < ETH_FCS; i++)
        f[len++] = (uint8_t)(fcs >> (8 * i));     /* FCS — младшим байтом вперёд */
    return len;
}

Numbers in the fields go most significant byte first, while a Cortex-M stores them least significant byte first — put16 splits them into bytes explicitly. On the board the F407 MAC adds the FCS and the padding to 60 bytes itself, and the PHY adds the preamble.

105.2 FCS: CRC-32

uint32_t eth_crc32(const uint8_t *p, unsigned n)
{
    uint32_t c = 0xFFFFFFFFu;
    while (n--) {
        c ^= *p++;
        for (unsigned k = 0; k < 8; k++)
            c = (c >> 1) ^ (0xEDB88320u & (0u - (c & 1u)));
    }
    return ~c;
}

The check value for the string "123456789" is 0xCBF43926, as in every implementation of this CRC-32. The receiver does not strip the FCS from the frame: the CRC over the frame together with it always equals 0x2144DF1C. Corrupt one bit and the sum is already different (for this frame — 0x0B6CE77E): anything other than 0x2144DF1C means an error. The frame is dropped silently, and retransmission is the job of the protocols above. The CRC unit in the F1, F2, F4 and L1 uses a different bit order and has no final inversion (section 12), so it is no good for the FCS without reversing the bits. On the F0, F3, F7, L4, G0, G4 and H7 the order is set by the REV_IN and REV_OUT bits: with input reversal by byte, output reversal and an inverted result, the unit computes the FCS itself.

105.3 The MAC address

A MAC is 6 bytes. In a purchased address the first three are the manufacturer code. The two low bits of the first byte are special: bit 0 marks a group address, and it is zero in a device address; bit 1 means the address was assigned locally, not by the manufacturer. An STM32 has no MAC of its own: addresses are bought as a block, taken from a chip with a programmed address, or assigned locally. For a device on your own network a local address is built from the chip's unique ID:

#define MAC_LOCAL  0x02u                /* бит 1 первого байта: адрес назначен локально */
#define MAC_GROUP  0x01u                /* бит 0: групповой; у адреса устройства — ноль */

void mac_from_uid(const uint8_t uid[12], uint8_t mac[6])
{
    uint64_t h = 0xCBF29CE484222325u;   /* FNV-1a, 64 бита: 96 бит номера → 46 бит адреса */
    for (unsigned i = 0; i < 12; i++)
        h = (h ^ uid[i]) * 0x100000001B3u;
    for (unsigned i = 0; i < 6; i++)
        mac[i] = (uint8_t)(h >> (8 * i));
    mac[0] = (uint8_t)((mac[0] & ~(MAC_LOCAL | MAC_GROUP)) | MAC_LOCAL);
}

void net_mac(uint8_t mac[6])
{
    uint8_t uid[12];
    memcpy(uid, (const void *)NET_UID_ADDR, sizeof uid);
    mac_from_uid(uid, mac);
}

The ID is at address 0x1FFFF7E8 on the F103 and 0x1FFF7A10 on the F407. The example ID 30 00 3C 00 0F 51 33 30 35 39 32 37 gives the address 06:42:8F:70:AA:C7: the low bits of the first byte are 10, a local individual address. A collision is not ruled out but unlikely: 46 bits give 70368744177664 addresses, and the probability of a collision in a network of 1000 devices is about 7.1·10⁻⁹.

105.4 IPv4: header and checksum

The checksum is the 16-bit words added with end-around carry and inverted at the end:

uint32_t ip_checksum(const uint8_t *p, unsigned n)
{
    uint32_t s = 0;
    for (unsigned i = 0; i + 1 < n; i += 2)
        s += get16(p + i);
    if (n & 1u)
        s += (uint32_t)p[n - 1] << 8;
    while (s >> 16)
        s = (s & 0xFFFFu) + (s >> 16);
    return ~s & 0xFFFFu;
}

The header 45 00 00 73 00 00 40 00 40 11 00 00 C0 A8 00 01 C0 A8 00 C7 with zero in the checksum field gives 0xB861. In it "40 00" is "don't fragment" with offset 0, "40" in byte 8 is TTL 64, "11" is UDP. The receiver computes the sum over the header including this field and gets zero. The same function computes the ICMP, UDP and TCP checksums — the last two also with a pseudo-header made of the addresses.

105.5 Subnets and the gateway

An address is written with its prefix length: 192.168.1.130/26 means 26 ones in the mask from the left. The network is the address ANDed with the mask bit by bit; the broadcast address is the network with ones in all host bits:

A /31 has no network or broadcast address — it is two hosts on a point-to-point link.

uint32_t ip_mask(unsigned prefix)
{
    return prefix ? 0xFFFFFFFFu << (32u - prefix) : 0u;
}

uint32_t next_hop(uint32_t me, unsigned prefix, uint32_t gw, uint32_t dst)
{
    return ((me ^ dst) & ip_mask(prefix)) ? gw : dst;
}

The decision "directly or through the gateway" is a single operation: if the addresses match under the mask, the destination is nearby and the frame goes to it; if not, the frame goes to the gateway's MAC address, while the IP header keeps the destination's address. Device 192.168.1.130/26 with gateway .129 reaches .150 directly and .20 through the gateway.

105.6 ARP: MAC by IP

Before sending a frame you need a MAC. ARP asks everyone: "who has 192.168.1.129? Tell 192.168.1.130".

unsigned arp_build(uint8_t *a, const arp_t *r)
{
    put16(a, 1);                        /* оборудование: Ethernet */
    put16(a + 2, ETH_IPV4);             /* протокол: IPv4 */
    a[4] = 6;                           /* длина MAC */
    a[5] = 4;                           /* длина адреса IPv4 */
    put16(a + 6, r->op);
    memcpy(a + 8, r->sha, 6);
    put32(a + 14, r->spa);
    memcpy(a + 18, r->tha, 6);
    put32(a + 24, r->tpa);
    return ARP_LEN;
}

ARP is 28 bytes; with the frame header 42, with padding to 60 and the FCS 64. The whole request:

ff ff ff ff ff ff  06 42 8f 70 aa c7  08 06     кому — всем, от кого, тип ARP
00 01  08 00  06  04  00 01                     Ethernet, IPv4, длины 6 и 4, запрос
06 42 8f 70 aa c7  c0 a8 01 82                  MAC и IP отправителя
00 00 00 00 00 00  c0 a8 01 81                  искомый MAC — нули, искомый IP
00 × 18                                         дополнение до 60 байт
3b 6c 8b c2                                     FCS, младшим байтом вперёд

The reply goes not to everyone but to the one who asked: operation 2, with the owner's MAC in the sender field. The reply is kept in the ARP table for several minutes so as not to ask before every packet.

105.7 Experiment: ARP and ping to a real kernel

Two containers on their own network 10.105.105.0/24: .10 runs the section's code, .20 is ordinary Linux. The code opens a raw AF_PACKET socket on eth0 and sends frames built by the section's functions — without the FCS: the network interface adds it. First an ARP request for .20, then a ping to the MAC address from the reply. The replies are parsed by the same functions:

• ARP reply from 10.105.105.20: operation 2, 42 bytes, addressed to our MAC. The frame is shorter than 60 bytes: on a virtual interface nobody pads it; the network card does that before the wire.
• Echo reply: type 0, TTL 64, a 60-byte packet — 20 of header, 8 of ICMP and 32 of ours. Both checksums of the reply matched our ip_checksum, and the same data came back.

The kernel drops a packet with a wrong checksum without replying, so the header and checksum building were verified by the recipient itself. TTL 64 is Linux's initial value; had the reply passed through a router, it would have arrived as 63.

105.8 Pitfalls

Frames go out but nobody replies. The type was written least significant byte first: 0x0608 instead of 0x0806. Numbers in the fields go most significant byte first.

Two devices fight over an address. Identical MACs: both use the address from the example. Build the address from the chip ID with the "local" bit set.

The device sees its neighbours but not the internet. No gateway, or a mask wider than the real one: the device looks up a foreign address with ARP itself, and nobody replies.

ARP works but ping is silent. A wrong IP or ICMP checksum — the kernel drops the packet silently. Compute the checksum with zero in its field.

The FCS computed by the STM32 CRC unit does not match. On the F1 and F4 the unit has a different bit order and no final inversion; on families with REV_IN/REV_OUT these were not enabled.

A short frame is not accepted. Under 60 bytes without the FCS: pad with zeros. The F407 MAC does this itself, software frame building does not.

105.9 Summary

• A frame is a 14-byte header, 46–1500 bytes of data and a 4-byte FCS, 64–1518 bytes in all; on the wire another 20: 148810 short frames per second at 100 Mbit/s.
• The FCS is a CRC-32; over the frame together with it the result is 0x2144DF1C.
• MAC: bit 0 means group, bit 1 means local; on an STM32 the address is built from the chip ID.
• IPv4: a 20-byte header and a ones' complement checksum; over the header together with the checksum the result is zero.
• The mask decides whether to send to the destination or to the gateway, and ARP finds the MAC by the IP.
• A real Linux kernel answered our ARP and ping: 42 bytes, TTL 64, checksums correct.

Next: section 106 "TCP and UDP from the inside: handshake, windows, retransmissions, NAT" — what the transport adds on top of IP and why a connection survives packet loss.