109. Git for the engineer: branches, history, firmware versions

Firmware lives for years: it gets edited, released, rolled back, and half a year later a complaint arrives about a board with a version nobody remembers any more. Git answers three questions at once: what is in the device now, what has changed since the previous version and exactly when things broke. This section does not go through commands for their own sake but through how it works: why an object's name is its content, how a version string inside the firmware comes out of the history and how to find the breaking commit in five checks instead of twenty.

The experiments use a real git 2.43 in a container, and the object IDs are computed alongside by our own C code, which builds for both F103 and F407.

Terms used in the documentation

Term	Meaning
repository	the whole history of the project
working tree / index (staging area)	the files on disk / what goes into the next commit
commit	a snapshot of the tree with an author and a parent
blob / tree	file content / a directory
branch	a movable label on a commit
tag	a fixed label, usually a version
merge / merge conflict	joining two lines of history / both sides changed the same place
fast-forward	a merge without a new commit
rebase	moving commits onto a different base
git describe	a version string from a tag and a commit count
dirty tree	unsaved edits in the working tree
bisect	binary search for the breaking commit
detached HEAD	HEAD that is not on a branch

109.1 Three places, not one

Git has three stores, and half of all misunderstandings come from mixing them up: the working tree (files on disk), the index (what has already been picked for the next commit) and the history itself. git add moves things from the first to the second, git commit from the second to the third. This also explains the answers: git status shows what lies where, git diff shows the difference between the working tree and the index, git diff --staged between the index and the last commit.

For firmware this matters also because nothing extra should get into the repository: build directories, .o, .axf, IDE files. The rule is simple: the history holds only sources and what they are built from; everything generated goes into .gitignore.

109.2 An object's name is its content

Git stores not "files with versions" but objects, and an object's name is the SHA-1 of its type, length and content:

SHA-1("blob" + " " + длина + "\0" + содержимое)

Everything else follows from this. Two identical files in different commits are one object. An empty file has the same name in every repository in the world: e69de29bb2d1d6434b8b29ae775ad8c2e48c5391. Change a single byte and the name is different, so the content cannot be swapped unnoticed.

Our code computes it itself:

void git_object_id(const char *type, const void *data, unsigned n, uint8_t id[20])
{
    char head[32];
    unsigned k = (unsigned)strlen(type);
    memcpy(head, type, k);
    head[k++] = ' ';
    k += put_dec(head + k, n);
    head[k++] = 0;                      /* нулевой байт — часть заголовка */
    sha1_t s;
    sha1_init(&s);
    sha1_update(&s, head, k);
    sha1_update(&s, data, n);
    sha1_final(&s, id);
}

Experiment: the real git and this code computed the same thing for an empty file, for README and for main.c — 8252973…. A tree of one file is the string "100644 main.c", a zero byte and the 20-byte ID of the content, 34 bytes in all; its name is 958ed88…, and again it matched.

A commit on top of this is an ordinary text object: tree, parent, author, date, message. That is why a merge commit has two parents and the first commit has none.

109.3 Branches and merging

A branch is not a copy but a label: a name pointing to a commit, and it moves with every new commit. That is why branches in git are cheap: git switch -c fix is one line in a file.

A merge finds the common ancestor and applies both sides. If the edits are in different places, git copes on its own. If both branches edited the same place, that is a conflict, and git leaves the choice to a human:

<<<<<<< HEAD
void fix(void) { }
=======
void feature(void) { }
>>>>>>> feature

That is exactly what happened in the experiment: both branches appended to the end of main.c, git reported a conflict in one file and inserted three marker lines. After resolution both lines are in place, and the merge commit has two parents.

The important thing is that a conflict is not a breakage but a question. The danger is elsewhere: resolving it "quickly" by throwing away someone else's edit. So after resolving a conflict you are expected to build the firmware and run the tests — this is exactly where lost lines get caught.

109.4 The firmware version from the history

A release gets a tag: git tag -a v1.2.0 -m "version 1.2.0". After that git describe --tags --dirty builds the version string by itself:

This string is passed to the compiler as a macro and printed when the firmware starts. Then the question "what is in the device" is answered by the device itself, and -dirty honestly says: built from who knows what, such firmware has no counterpart in the history.

109.5 Parsing the version on the board

The string is handy not only to print but also to parse: compare your own version with one received over the radio, refuse to install older firmware. Parsing takes a few lines and works without libraries:

if (s[0] == '-' && s[1] >= '0' && s[1] <= '9') {    /* «-3-g5b1f0c2»: после тега были коммиты */
    if (!(s = get_num(s + 1, &v->ahead)) || s[0] != '-' || s[1] != 'g')
        return 0;

You must not compare versions as text: "1.10.0" is less than "1.9.3" alphabetically, although 1.10.0 is newer than 1.9.3. The comparison goes by numbers:

const unsigned x[4] = { a->major, a->minor, a->patch, a->ahead };
const unsigned y[4] = { b->major, b->minor, b->patch, b->ahead };
for (unsigned i = 0; i < 4; i++)
    if (x[i] != y[i])
        return x[i] < y[i] ? -1 : 1;

The fourth number is the commits after the tag: build v1.2.0-3 is newer than v1.2.0 itself. But this number only makes sense on a single branch: v1.2.0-3 from two different branches are "equal" by it. So the rollback ban on the device compares release versions (tags), and builds with -N-g… are installed only on the test bench. Strings that should not exist ("v1.2", "v1.2.0-x") are rejected by the parser rather than interpreted at random.

109.6 What to store and what not

• Sources, build scripts, project files — yes.
• Packages and libraries — as a reference to a version (submodule, a manager from section 115), not as a copy.
• Build outputs — no: .o, .axf, .hex, build/ directories. The exception is released images, but they belong in an artifact store (section 110), not in the history.
• Secrets — never. Removing them later is almost impossible: the commit stays in the history and with everyone who has fetched it. A leaked key is considered compromised and is replaced.
• Large binary files (oscilloscope captures, display images) — with care: already compressed files (images, archives, firmware images) barely compress as deltas, every version is stored in full and bloats the repository forever. For them there is Git LFS: the history keeps a reference, and the file itself lives in a separate store.

109.7 Finding the breaking commit in five checks

The most useful command that people most often do not know about. You know that a month ago it built, and now it does not. Twenty commits in between — going through them by hand takes long. git bisect does a binary search, and git bisect run also runs the check by itself:

git bisect start HEAD "$FIRST" > /dev/null 2>&1
git bisect run sh -c 'gcc -fsyntax-only app.c' > /tmp/bisect.log 2>&1 || true

Experiment: 21 commits, the thirteenth lost a semicolon. git bisect run with a real compiler found exactly that commit 13 in 5 checks instead of twenty — that is ⌈log₂ 21⌉. The check can be anything as long as it returns zero on a good commit: a build, a test on the computer, a run on the board.

The order of checks in the experiment: commit 10 builds, 15 does not, 12 does, 14 does not, 13 does not. Each answer discards half of the remaining interval.

The rule is the same as for tests: the smaller the commits, the more precise the answer. A commit "rewrote the driver" will be found just as quickly, but it narrows the search to a thousand lines rather than five.

109.8 When history is rewritten

git rebase moves commits onto a new base so that the history reads as a straight line. This is convenient as long as the commits have not been handed to anyone: the move creates new objects with new names, and a colleague is left with the old branch — up to diverging histories.

The rule is simple: your own, not yet pushed — edit as much as you like; pushed — only with new commits. Fixing a mistake with a separate commit is more honest: the history shows what was wrong, and that is exactly what people look for half a year later.

109.9 How it looks when working on firmware

• A branch per task, a short life, a merge into the main branch after the build and tests (section 110).
• A tag on every release, the version in the firmware comes from git describe.
• The commit message says what and why; "fixed" is useless half a year later. The first line is short, details follow.
• A release is marked with a tag, not a branch: a branch will move, a tag will stay.
• The firmware prints its version at start-up, and a complaint starts not with "probably an old one" but with an exact string.

109.10 Pitfalls

-dirty in the version of released firmware. Built from unsaved edits: such firmware does not exist in the history and cannot be reproduced. Build a release only from a clean tree (section 110 — in a container).

Versions are compared as text. 1.10.0 turns out "older" than 1.9.3. Compare by numbers.

A secret in the history. Deleting the file with a new commit does not help: the old commit is still there. Replace the key.

The repository has bloated. The history holds binary build files. They belong in an artifact store.

A conflict resolved "quickly". Someone else's edit was thrown away. After resolution — build and tests.

Pushed history was rewritten. Colleagues have a different history of the same branch. What has been pushed is fixed with new commits.

Detached HEAD. Commits were made not on a branch and "disappeared": they exist, but nothing points to them. git switch -c name right there, or git reflog to find them.

109.11 Summary

• An object's name is the SHA-1 of its type, length and content; our C code and the real git compute it the same way, an empty file is always e69de29bb2d1d6434b8b29ae775ad8c2e48c5391.
• A tree of one file is 34 bytes: mode, name, zero and the 20-byte ID.
• A branch is a movable label, a tag is a fixed one; a merge commit has two parents.
• The firmware version comes from git describe: v1.2.0-3-g<hash>, and -dirty means the build cannot be reproduced.
• Versions are compared by numbers: 1.10.0 is newer than 1.9.3.
• git bisect run finds the breaking commit 13 out of 21 in 5 checks instead of trying them all.

Full sources

git109.c — SHA-1 and Git object names, parsing git describe and comparing versions.

/* Раздел 109: Git для инженера — объекты и их имена, версия прошивки.
 * SHA-1 по стандарту, ID объектов Git — blob, tree, commit — как их считает
 * сам Git: SHA-1 от заголовка «тип длина\0» и содержимого. Разбор строки
 * git describe в поля версии и сравнение версий по числам, а не по тексту.
 * Переносимый C: собирается и для платы — прошивка может сама проверить
 * хэш образа и сравнить свою версию с присланной (_verify/git109_host.c
 * считает ID, которые проверка сверяет с настоящим git).
 * Проверка — _verify/git109_check.py. */
#include <stdint.h>
#include <string.h>

/* ── SHA-1 ───────────────────────────────────────────────────────────── */
typedef struct {
    uint32_t h[5];
    uint64_t len;                       /* байт принято */
    uint8_t buf[64];
    unsigned used;
} sha1_t;

static uint32_t rol(uint32_t x, unsigned n)
{
    return x << n | x >> (32u - n);
}

static void sha1_block(sha1_t *s, const uint8_t *p)
{
    uint32_t w[80];
    for (unsigned i = 0; i < 16; i++)
        w[i] = (uint32_t)p[4 * i] << 24 | (uint32_t)p[4 * i + 1] << 16 | (uint32_t)p[4 * i + 2] << 8 | p[4 * i + 3];
    for (unsigned i = 16; i < 80; i++)
        w[i] = rol(w[i - 3] ^ w[i - 8] ^ w[i - 14] ^ w[i - 16], 1);
    uint32_t a = s->h[0], b = s->h[1], c = s->h[2], d = s->h[3], e = s->h[4];
    for (unsigned i = 0; i < 80; i++) {
        uint32_t f, k;
        if (i < 20) {
            f = (b & c) | (~b & d);
            k = 0x5A827999u;
        } else if (i < 40) {
            f = b ^ c ^ d;
            k = 0x6ED9EBA1u;
        } else if (i < 60) {
            f = (b & c) | (b & d) | (c & d);
            k = 0x8F1BBCDCu;
        } else {
            f = b ^ c ^ d;
            k = 0xCA62C1D6u;
        }
        uint32_t t = rol(a, 5) + f + e + k + w[i];
        e = d;
        d = c;
        c = rol(b, 30);
        b = a;
        a = t;
    }
    s->h[0] += a;
    s->h[1] += b;
    s->h[2] += c;
    s->h[3] += d;
    s->h[4] += e;
}

void sha1_init(sha1_t *s)
{
    static const uint32_t h0[5] = { 0x67452301u, 0xEFCDAB89u, 0x98BADCFEu, 0x10325476u, 0xC3D2E1F0u };
    memcpy(s->h, h0, sizeof h0);
    s->len = 0;
    s->used = 0;
}

void sha1_update(sha1_t *s, const void *data, unsigned n)
{
    const uint8_t *p = data;
    s->len += n;
    while (n--) {
        s->buf[s->used++] = *p++;
        if (s->used == 64) {
            sha1_block(s, s->buf);
            s->used = 0;
        }
    }
}

void sha1_final(sha1_t *s, uint8_t out[20])
{
    uint64_t bits = s->len * 8u;
    static const uint8_t one = 0x80, zero = 0;
    sha1_update(s, &one, 1);            /* бит «1», затем нули до 56 байт в блоке */
    while (s->used != 56)
        sha1_update(s, &zero, 1);
    uint8_t len[8];
    for (unsigned i = 0; i < 8; i++)
        len[i] = (uint8_t)(bits >> (56 - 8 * i));      /* длина в битах, старшим байтом вперёд */
    sha1_update(s, len, 8);
    for (unsigned i = 0; i < 20; i++)
        out[i] = (uint8_t)(s->h[i / 4] >> (24 - 8 * (i % 4)));
}

/* ── ID объекта Git: SHA-1 от «тип длина\0» и содержимого ────────────── */
static unsigned put_dec(char *p, unsigned v)
{
    char t[10];
    unsigned n = 0, k = 0;
    do
        t[n++] = (char)('0' + v % 10u);
    while (v /= 10u);
    while (n)
        p[k++] = t[--n];
    return k;
}

void git_object_id(const char *type, const void *data, unsigned n, uint8_t id[20])
{
    char head[32];
    unsigned k = (unsigned)strlen(type);
    memcpy(head, type, k);
    head[k++] = ' ';
    k += put_dec(head + k, n);
    head[k++] = 0;                      /* нулевой байт — часть заголовка */
    sha1_t s;
    sha1_init(&s);
    sha1_update(&s, head, k);
    sha1_update(&s, data, n);
    sha1_final(&s, id);
}

/* дерево из одного файла: «режим имя\0» и 20 байт ID его содержимого */
unsigned git_tree_one(uint8_t *t, const char *mode, const char *name, const uint8_t blob[20])
{
    unsigned k = 0;
    size_t m = strlen(mode), f = strlen(name);
    memcpy(t, mode, m);
    k += (unsigned)m;
    t[k++] = ' ';
    memcpy(t + k, name, f);
    k += (unsigned)f;
    t[k++] = 0;
    memcpy(t + k, blob, 20);
    return k + 20;
}

/* ── версия прошивки из git describe ─────────────────────────────────── */
typedef struct {
    unsigned major, minor, patch;       /* из тега vX.Y.Z */
    unsigned ahead;                     /* коммитов после тега */
    char hash[16];                      /* короткий ID коммита */
    int dirty;                          /* собрано из изменённого дерева */
} fw_version_t;

static const char *get_num(const char *s, unsigned *v)
{
    if (*s < '0' || *s > '9')
        return 0;
    *v = 0;
    while (*s >= '0' && *s <= '9')
        *v = *v * 10u + (unsigned)(*s++ - '0');
    return s;
}

/* «v1.2.0-3-g5b1f0c2-dirty» → поля; 1 — разобрано */
int fw_describe(const char *s, fw_version_t *v)
{
    memset(v, 0, sizeof *v);
    if (*s == 'v')
        s++;
    if (!(s = get_num(s, &v->major)) || *s++ != '.' || !(s = get_num(s, &v->minor)) || *s++ != '.'
        || !(s = get_num(s, &v->patch)))
        return 0;
    if (s[0] == '-' && s[1] >= '0' && s[1] <= '9') {    /* «-3-g5b1f0c2»: после тега были коммиты */
        if (!(s = get_num(s + 1, &v->ahead)) || s[0] != '-' || s[1] != 'g')
            return 0;
        s += 2;
        unsigned k = 0;
        while (((*s >= '0' && *s <= '9') || (*s >= 'a' && *s <= 'f')) && k < sizeof v->hash - 1)
            v->hash[k++] = *s++;
        v->hash[k] = 0;
    }
    if (strcmp(s, "-dirty") == 0)
        v->dirty = 1;
    else if (*s)
        return 0;
    return 1;
}

/* сравнение по числам: 1.10.0 новее 1.9.3, хотя как текст «1.10» меньше «1.9» */
int fw_cmp(const fw_version_t *a, const fw_version_t *b)
{
    const unsigned x[4] = { a->major, a->minor, a->patch, a->ahead };
    const unsigned y[4] = { b->major, b->minor, b->patch, b->ahead };
    for (unsigned i = 0; i < 4; i++)
        if (x[i] != y[i])
            return x[i] < y[i] ? -1 : 1;
    return 0;
}

Next: section 110 "CI/CD for firmware" — the same history, but the build and tests run by themselves, and the output is an image with the version from the tag.