143. Static and dynamic libraries: .a, .so, soname, rpath, LD_PRELOAD, dlopen

Section 74 covered the standard library — the one you are given. Here is how to make your own: build code into a library, hand it to other programs and update it without rebuilding them. In firmware a library is always static: its code is pasted into the image. Linux has a second way — a dynamic library, one file on disk for all programs at once — and around it a whole household: soname, ldd, rpath, LD_PRELOAD, dlopen. The library libgeom of four small functions (two public, one hidden and one static) goes the whole way in _verify/sys143/ (stm-sys:1, gcc 12, glibc 2.36).

Terms used in the documentation

Term	Meaning
static library, archive (.a)	a static library, an archive of object files
shared library, shared object (.so)	a dynamic (shared) library
position-independent code (PIC)	code that does not depend on the load address
dynamic linker / loader (ld.so)	the program that loads libraries at startup
soname	the library name recorded in the program
NEEDED, RUNPATH	what the program needs and where to look for it
symbol visibility	whether a symbol is seen outside the library
interposition	replacing a symbol by a library loaded earlier
dlopen, dlsym	open a library and find a name in it at run time
plugin	a library loaded by name
ABI	the binary interface: what you can change without breaking old programs

143.1 A library of four functions

static double square(double x) { return x * x; }                 /* static: не видна даже компоновщику */

double GEOM_HIDDEN geom_internal(double x) { return square(x) + 1; }   /* видна в .a, скрыта в .so */

GEOM_API double geom_dist(double x1, double y1, double x2, double y2)
{
    return sqrt(square(x2 - x1) + square(y2 - y1));
}

Three levels of visibility: static — the name does not leave the file; an ordinary external one — visible to everyone who links against it; GEOM_HIDDEN — the visibility("hidden") attribute: the name exists in the object file but is not exported from the dynamic library. The program app.c calls geom_dist and geom_version (the fourth function, it returns the version string) and does not know which library it will get.

143.2 Static: an archive of object files

gcc -O2 -c geom.c -o geom.o
ar rcs libgeom.a geom.o
gcc -O2 -o app_static app.c -L. -lgeom -lm

libgeom.a is just an archive: ar t lists geom.o. The linker takes from the archive only the object files that contain the needed symbols and pastes them into the program: app_static is 16152 bytes and does not depend on libgeom.a after the build. nm shows what is in the archive:

T geom_dist
T geom_internal
T geom_version
U sqrt

T — defined in the code section, U — undefined, needed from outside (here sqrt from libm). square is not in the list: at -O2 the compiler inlined it into its callers; at -O0 it is there, marked t — local.

The order on the command line matters: the linker goes from left to right and takes from an archive what has already been needed. gcc -lgeom app.c gives the error undefined reference to geom_dist: at the moment libgeom.a is read, nobody needs the symbol yet. gcc app.c -lgeom builds. Libraries go after the objects, and the ones that depend on others come before the ones they depend on.

143.3 Dynamic: PIC, soname, three names

gcc -fPIC -fvisibility=hidden -c geom.c -o geom_pic.o
gcc -shared -Wl,-soname,libgeom.so.1 -o libgeom.so.1.0 geom_pic.o -lm
ln -s libgeom.so.1.0 libgeom.so.1
ln -s libgeom.so.1 libgeom.so

-fPIC — code without absolute addresses: the library is loaded into each process at its own address. -fvisibility=hidden hides everything that is not marked GEOM_API (visibility("default")): exactly two functions are exported —

geom_dist
geom_version

and geom_internal does not exist from the outside. This is the library's ABI: the less is exported, the fewer the obligations.

Three names of one file:

-Wl,-soname records the name libgeom.so.1 in the library, and that is exactly what the linker writes into the program:

(NEEDED) [libgeom.so.1]
(NEEDED) [libc.so.6]

The number in the soname is the ABI version: 1.0 → 1.1 are compatible, and the program gets the new library without a rebuild; 1 → 2 are incompatible, and old programs keep looking for libgeom.so.1. The experiment: build libgeom.so.1.1 with GEOM_VERSION "1.1", switch the libgeom.so.1 link — and the old program prints:

geom 1.1: расстояние (0,0)-(3,4) = 5.0

The program with the dynamic library is 16048 bytes, almost the same as the static one: the geom code is not in it, only the name and a table of the symbols it needs.

143.4 Where to look: ld.so, LD_LIBRARY_PATH, rpath

Starting a program means starting /lib64/ld-linux-x86-64.so.2: it reads NEEDED, finds the libraries, maps them and binds the symbols. It searches in this order: RPATH in the program itself (the old entry; used only if there is no RUNPATH), the LD_LIBRARY_PATH variable, RUNPATH in the program, the cache /etc/ld.so.cache (built from the directories in /etc/ld.so.conf.d/), then /lib and /usr/lib. A modern linker writes RUNPATH, so LD_LIBRARY_PATH overrides it. The current directory is not in the list:

./app_shared: error while loading shared libraries: libgeom.so.1: cannot open shared object file: No such file or directory

LD_LIBRARY_PATH=. ./app_shared works, and ldd shows where each library was taken from:

libgeom.so.1 ./libgeom.so.1
libc.so.6 /lib/x86_64-linux-gnu/libc.so.6
libm.so.6 /lib/x86_64-linux-gnu/libm.so.6

The environment variable is for development; for an installed program, either put the library in a system directory and run ldconfig, or record the path in the program: -Wl,-rpath,'$ORIGIN' writes RUNPATH [$ORIGIN] — "next to the executable", and the program can be moved as a whole directory.

143.5 LD_PRELOAD: replacing a function without a rebuild

A library from LD_PRELOAD is loaded before all others, and its symbols override the same-named ones in libc. This is how you count malloc calls in any program:

void *malloc(size_t n)
{
    if (!real_malloc) real_malloc = dlsym(RTLD_NEXT, "malloc");   /* RTLD_NEXT: настоящий malloc из libc */
    calls++;
    return real_malloc(n);
}

[preload] malloc вызван 994 раз      ← python3 -c pass
[preload] malloc вызван 1 раз        ← app_rpath: буфер printf
[preload] malloc вызван 0 раз        ← /bin/true

Inside the replacement you must not call printf — it calls malloc itself, and you get recursion; the report is written with snprintf into a static buffer and write. The same way people replace malloc entirely (jemalloc, section 135), catch memory errors (libasan), and fake time and the network in tests. For setuid programs LD_PRELOAD has no effect: the kernel marks such a launch as secure, and the dynamic loader ignores the variable (except for libraries from system directories).

143.6 dlopen: a library by name at run time

A plugin is a library whose name is known only at startup:

    void *h = dlopen(path, RTLD_NOW);                          /* RTLD_NOW: все символы сразу, ошибка — здесь */
    if (!h) { printf("dlopen: %s\n", dlerror()); return 1; }
    double (*dist)(double, double, double, double) = dlsym(h, "geom_dist");

плагин ./libgeom.so.1.1: версия 1.1, dist = 10.0, geom_internal скрыт
плагин ./libgeom.so.1.0: версия 1.0, dist = 10.0, geom_internal скрыт
dlopen: ./nope.so: cannot open shared object file: No such file or directory

One program opened both versions in turn, and dlsym did not find geom_internal — hidden visibility applies here too. RTLD_NOW binds everything when the library is opened, RTLD_LAZY on the first call; dlerror gives the text of the last error. This is how editors, servers and format handlers load plugins; a library meant for dlopen usually exports one "entry" function that returns a table of pointers to the rest.

143.7 Windows: DLL

In Windows a dynamic library is a .dll, and next to it the linker creates an import library .lib (.dll.a with MinGW): the program is built against it, and at startup it looks for the .dll in its own directory, then along PATH. Exporting is done with __declspec(dllexport) on a function or with a .def file; by default nothing is exported, as with -fvisibility=hidden. The counterpart of dlopen/dlsym is LoadLibrary/GetProcAddress. The file format is PE instead of ELF, and there is no soname: versions are handled by the file name and by a directory next to the program.

143.8 Pitfalls

143.9 Summary

• .a is an archive: the code is pasted in, -l goes after the objects.
• .so: -fPIC, -shared, -Wl,-soname; three names; only GEOM_API is exported with -fvisibility=hidden.
• Version 1.0 → 1.1 under the same soname — no rebuild of the program.
• Search: LD_LIBRARY_PATH → RUNPATH ($ORIGIN) → cache → /lib, /usr/lib; the old RPATH comes before the variable.
• LD_PRELOAD replaces symbols (994 malloc calls for python3 -c pass); no printf inside.
• dlopen/dlsym — plugins; hidden hides symbols from them too.

Full sources

geom.c — the "geometry" library: two public functions and one hidden one.

/* Раздел 143: библиотека «геометрия» — две открытые функции и одна скрытая. */
#include <math.h>
#include "geom.h"

static double square(double x) { return x * x; }                 /* static: не видна даже компоновщику */

double GEOM_HIDDEN geom_internal(double x) { return square(x) + 1; }   /* видна в .a, скрыта в .so */

GEOM_API double geom_dist(double x1, double y1, double x2, double y2)
{
    return sqrt(square(x2 - x1) + square(y2 - y1));
}

GEOM_API const char *geom_version(void) { return GEOM_VERSION; }

preload_malloc.c — replacing malloc through LD_PRELOAD.

/* Раздел 143: подмена malloc через LD_PRELOAD — считаем вызовы в любой программе без пересборки.
   Внутри — только write и dlsym, без printf: printf сам зовёт malloc. */
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <dlfcn.h>

static void *(*real_malloc)(size_t);
static long calls;
static char buf[64];

void *malloc(size_t n)
{
    if (!real_malloc) real_malloc = dlsym(RTLD_NEXT, "malloc");   /* RTLD_NEXT: настоящий malloc из libc */
    calls++;
    return real_malloc(n);
}

__attribute__((destructor)) static void report(void)
{
    int len = snprintf(buf, sizeof buf, "[preload] malloc вызван %ld раз\n", calls);
    write(2, buf, len);
}

plugin_host.c — loading a library at run time through dlopen and dlsym.

/* Раздел 143: загрузка библиотеки во время работы — dlopen/dlsym; так устроены плагины. */
#include <stdio.h>
#include <dlfcn.h>

int main(int argc, char **argv)
{
    const char *path = argc > 1 ? argv[1] : "./libgeom.so";
    void *h = dlopen(path, RTLD_NOW);                          /* RTLD_NOW: все символы сразу, ошибка — здесь */
    if (!h) { printf("dlopen: %s\n", dlerror()); return 1; }
    double (*dist)(double, double, double, double) = dlsym(h, "geom_dist");
    const char *(*ver)(void) = dlsym(h, "geom_version");
    void *hidden = dlsym(h, "geom_internal");
    printf("плагин %s: версия %s, dist = %.1f, geom_internal %s\n", path, ver(), dist(0, 0, 6, 8),
           hidden ? "найден" : "скрыт");
    dlclose(h);
    return 0;
}

Next: section 144 — x86-64 assembly through the eyes of C.