107. What happens when you type an address: DNS, DHCP, HTTP, TLS

Between the address in the browser bar and the page there are six steps, and a device that sends data to a server by name goes through all of them. This section builds and parses the messages of each protocol with its own C code. In the experiment the code asks Docker's built-in DNS for the name of a neighbouring container and fetches a page from its server, and the TLS greeting is built by real OpenSSL, with the section's function reading the server name out of it, in plain text.

Frames and ARP are in section 105, TCP in section 106, the device clock in section 12.

Terms used in the documentation

Term	Meaning
dynamic host configuration protocol (DHCP)	handing out addresses on a network
lease	an address lease
renewal (T1), rebinding (T2)	extending the lease, looking for any server
resolver	the name resolver
resource record	a DNS record
name compression	compressing names with pointers
time to live (TTL)	how long a reply is kept in the cache
NXDOMAIN	"no such name"
status line	the HTTP status line
header field	an HTTP header
chunked transfer encoding	sending the body in chunks
ClientHello	the TLS client greeting
server name indication (SNI)	the server name in the greeting
application-layer protocol negotiation (ALPN)	choosing the protocol on top of TLS
certificate	a certificate

107.1 Six steps

1. DHCP: at power-up the device gets an address, a mask, a gateway and a DNS address.
2. ARP: the gateway's MAC address.
3. DNS: the server name turns into an IP address.
4. TCP: the handshake with the server.
5. TLS: key exchange and certificate verification.
6. HTTP: the request and the response.

The device does the first three steps rarely: the address lives for the lease time, a DNS reply for its own time in the cache. The last three happen on every connection.

107.2 DHCP: an address for a while

There are four messages: DISCOVER, "is there a server here?", OFFER, "take this address", REQUEST, "I'll take it", ACK, "it's yours". The client has no address yet, so it sends from 0.0.0.0 to 255.255.255.255 and asks for a broadcast reply:

unsigned dhcp_discover(uint8_t *d, uint32_t xid, const uint8_t mac[6])
{
    memset(d, 0, 240);
    d[0] = 1;                           /* сообщение клиента */
    d[1] = 1;                           /* Ethernet */
    d[2] = 6;                           /* длина MAC */
    pu32(d + 4, xid);                   /* номер обмена: ответ несёт тот же */
    pu16(d + 10, 0x8000);               /* адреса ещё нет — отвечать всем */
    memcpy(d + 28, mac, 6);
    pu32(d + 236, DHCP_COOKIE);
    static const uint8_t opt[] = { 53, 1, DHCP_DISCOVER,   /* тип сообщения */
                                   55, 4, 1, 3, 6, 51,    /* просим маску, шлюз, DNS и срок */
                                   255 };                 /* конец */
    memcpy(d + 240, opt, sizeof opt);
    return 240 + sizeof opt;
}

DHCPDISCOVER is 250 bytes: a 236-byte header, the "magic cookie" 0x63825363 and 10 bytes of options. In practice the message is padded with zeros to at least 300 bytes, as was customary for DHCP's predecessor, BOOTP: some servers and relay agents drop short messages. lwIP, for example, pads the options to 68 bytes, and the message comes out at 308. The server's offer in the model is parsed as follows: address 10.107.107.50, mask 255.255.255.0, gateway and DNS 10.107.107.1, lease 86400 s, one day. Renewing the lease with the same server starts after 43200 s, half the lease, and looking for any server after 75600 s, 7/8 of the lease. A reply with someone else's transaction number is rejected: the number ties the question to the answer.

107.3 DNS: name → address

The query is a 12-byte header, the name as labels, the type and the class:

unsigned dns_query(uint8_t *q, uint32_t id, const char *name, uint32_t type)
{
    pu16(q, id);
    pu16(q + 2, 0x0100);                /* обычный запрос, «нужна рекурсия» */
    pu16(q + 4, 1);                     /* один вопрос */
    pu16(q + 6, 0);
    pu16(q + 8, 0);
    pu16(q + 10, 0);
    unsigned n = 12;
    while (*name) {                     /* «stm.example» → 3 stm 7 example 0 */
        const char *dot = strchr(name, '.');
        unsigned len = dot ? (unsigned)(dot - name) : (unsigned)strlen(name);
        q[n++] = (uint8_t)len;
        memcpy(q + n, name, len);
        n += len;
        name += len + (dot != 0);
    }
    q[n++] = 0;
    pu16(q + n, type);
    pu16(q + n + 2, DNS_IN);
    return n + 4;
}

The name stm107-peer in the query is 0b 73 74 6d 31 30 37 2d 70 65 65 72 00: the label length, its letters and a zero at the end; the whole query is 29 bytes. In the reply the server repeats the question, and does not write the name in the record a second time: instead there is the pointer c0 0c, "the name from byte 12". The whole reply is 45 bytes. The parser skips names of both kinds:

static int dns_skip(const uint8_t *m, unsigned n, unsigned *p)
{
    while (*p < n) {
        unsigned l = m[*p];
        if (l == 0) {
            *p += 1;
            return 1;
        }
        if ((l & 0xC0u) == 0xC0u) {
            *p += 2;
            return *p <= n;
        }
        *p += 1u + l;
    }
    return 0;
}

A record carries a lifetime: how many seconds the reply may be kept in the cache. Code 3 in the header means "no such name", and no addresses are taken from such a reply.

107.4 HTTP/1.1: text, an empty line, the body

The request is lines of text, each ending with \r\n, with an empty line at the end:

GET / HTTP/1.1
Host: stm107-peer
Connection: close

The Host header is mandatory in HTTP/1.1: many sites live at one address. The whole request is 56 bytes of text. The response is the status line, the headers, an empty line and the body; header names are compared case-insensitively, the body length is in Content-Length:

unsigned http_response(const char *r, unsigned n, unsigned *status, unsigned *clen)
{
    const char *end = 0;
    for (unsigned i = 0; i + 3 < n && !end; i++)
        if (memcmp(r + i, "\r\n\r\n", 4) == 0)
            end = r + i + 4;
    if (!end || memcmp(r, "HTTP/1.", 7) != 0)
        return 0;
    *status = (unsigned)(r[9] - '0') * 100u + (unsigned)(r[10] - '0') * 10u + (unsigned)(r[11] - '0');
    const char *cl = http_header(r, end, "content-length");
    *clen = 0;
    while (cl && *cl >= '0' && *cl <= '9')
        *clen = *clen * 10u + (unsigned)(*cl++ - '0');
    return (unsigned)(end - r);
}

The body is read by length, not "until close": the connection may stay open for the next request. If there is no length but there is Transfer-Encoding: chunked, the body comes in chunks, each with its own length.

107.5 TLS: what is visible before encryption

TLS starts with the client greeting, and it travels in plain text: versions, cipher suites, the key for the exchange and extensions. One of them is the server name, so that a server with many sites knows which certificate to present:

        if (type == 0 && len >= 5 && v[2] == 0) {           /* server_name */
            unsigned k = be16(v + 3);
            if (k < sizeof h->sni && 5 + k <= len) {
                memcpy(h->sni, v + 5, k);
                h->sni[k] = 0;
            }
        } else if (type == 43 && len >= 1) {                /* supported_versions */
            for (unsigned i = 1; i + 1 < 1u + v[0] && i + 1 < len; i += 2) {
                uint32_t ver = be16(v + i);
                if ((ver & 0x0F0Fu) != 0x0A0Au && ver > h->max_version)
                    h->max_version = ver;                   /* 0x?A?A — пустышки GREASE */
            }

The greeting was built by OpenSSL, the same one used in browsers and on servers: the greeting is 517 bytes, OpenSSL padded the record body to 512. The section's function found in it the name "stm32.example", the highest version 0x0304, TLS 1.3, and the protocols h2 and http/1.1 for ALPN. Everyone along the path sees the site name, even though everything else is encrypted, unless encrypted client hello (ECH) is enabled. And the DNS query usually goes out in plain text anyway.

The certificate is also checked for its validity period, so the device needs a correct clock: without a real-time clock and synchronization the certificate will turn out "not yet valid".

107.6 Experiment: Docker DNS and a page

Two containers on the network 10.107.107.0/24: the neighbouring one runs a Python server (python -m http.server), ours runs the section's code on ordinary sockets.

• Docker's built-in DNS at 127.0.0.11 knows the names of containers on its network. The reply was parsed by the section's function: stm107-peer is 10.107.107.20, lifetime 600 s.
• The GET request went over TCP to port 8080. The response is code 200 and Content-Length: 111, exactly the file size, and the body matched it byte for byte.

107.7 On the device

DHCP, DNS and HTTP are within reach of small stacks too: it is byte parsing, as in this section. TLS is the most expensive part: memory for keys and certificates and time for the handshake. Experiment: a TLS 1.2 client on mbedTLS 3.6.4, cut down to one cipher suite (ECDHE-ECDSA with AES-128-GCM) and one curve, P-256, built for Cortex-M4 with -Os and run in QEMU against a server in the same program:

On an F407 at 168 MHz the handshake takes at least half a second, but there is one per connection. 4 KB buffers are fine only if the server agrees to short records (the max_fragment_length extension), otherwise a 16 KB record will not be accepted. The biggest flash consumers are table-based AES, the TLS core, curve arithmetic and certificate parsing. The F103C8 with its 64 KB of flash and 20 KB of RAM cannot fit TLS: the library alone almost fills the flash. There TLS is handed off to a communication module.

107.8 Pitfalls

The device does not get an address via DHCP. The "reply to everyone" flag is not set: the reply went to an address the device does not have yet.

The address "disappeared" after a day. The lease was not renewed: renew from half of the lease time.

DNS answers, but there is no address. The first record in the reply is a CNAME, an alias; the address is in the next record.

The server changed its address, but the device goes to the old one. Keep a DNS reply in the cache no longer than the lifetime from the record.

The server answers 400. The Host header is missing.

Receiving the response hangs. The body is read until close, but the server keeps the connection open: read by Content-Length or ask for Connection: close.

TLS fails certificate verification. The device clock is set to the year 2000: the certificate is "not yet valid".

107.9 Summary

• Six steps: DHCP, ARP, DNS, TCP, TLS, HTTP; the first three rarely, the last three on every connection.
• DHCP gives an address for a limited time: with a one-day lease, renew after 43200 s, 12 hours.
• DNS: the name as labels, the reply compressed with pointers, the cache lifetime in the record.
• HTTP/1.1: text, Host is mandatory, the body by Content-Length.
• TLS: the greeting is open, the server name is visible; the certificate needs a correct clock.
• A TLS client on mbedTLS: 57 KB of flash, 12–36 KB of heap, the handshake is 85 million instructions; the F103C8 cannot fit it.

Next: section 108 "Routing, VLAN, Wi-Fi and VPN": how a packet finds its way between networks.