diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 033a9d71..94075d42 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -330,11 +330,25 @@ jobs: # Every bundled object must resolve once AppRun's LD_LIBRARY_PATH is # applied; an unresolved soname here is a user-visible launch crash. + # + # This runs on a HEADLESS runner on purpose, and that is the point. + # The first version of this build bundled Debian's SDL2, which + # hard-links libpulse/libasound/libX11/libwayland, so it only ever + # started on a full desktop -- a bare runner is what exposed it. missing="$(LD_LIBRARY_PATH="$PWD/squashfs-root/lib" \ ldd squashfs-root/bin/love squashfs-root/lib/*.so* 2>/dev/null \ | grep 'not found' || true)" [ -z "$missing" ] || { echo "::error::unresolved deps:"; echo "$missing"; exit 1; } + # Nothing may hard-link a driver, session or audio-stack library: + # those must be reached through dlopen so the AppImage runs on a box + # with only ALSA, only Wayland, or only KMSDRM. + linked="$(for f in squashfs-root/bin/love squashfs-root/lib/*.so*; do + objdump -p "$f" 2>/dev/null | awk '/NEEDED/{print $2}' + done | sort -u | grep -E '^lib(pulse|asound|X11|wayland|GL|EGL|drm|gbm|xcb|cairo|sndio|dbus)' || true)" + [ -z "$linked" ] \ + || { echo "::error::these must be dlopened, not linked:"; echo "$linked"; exit 1; } + # The whole point of compiling on bullseye. If a future change moves # the builder to a newer base, the glibc floor silently rises and # every user on an older distro gets "GLIBC_2.xx not found" -- catch diff --git a/README.md b/README.md index 45afadb6..ce866511 100644 --- a/README.md +++ b/README.md @@ -217,11 +217,12 @@ chmod +x gen1recomp-*-linux-arm64.AppImage ``` LÖVE publishes no aarch64 binary of any kind, so this artifact compiles the -engine from source inside a Debian bullseye arm64 container; the result needs -only glibc 2.29+, which covers Raspberry Pi OS bullseye through trixie and -Ubuntu 20.04 onward. Build instructions, the host requirements, and why the -font stack is deliberately left unbundled are in -[docs/linux-arm64-build.md](docs/linux-arm64-build.md). +engine — and SDL2, OpenAL and the codecs — from source inside a Debian +bullseye arm64 container. It needs only glibc 2.29+, libstdc++, freetype and +zlib on the host; OpenGL, X11, Wayland, KMSDRM, ALSA and PulseAudio are all +dlopened, so the same image runs on a full desktop, a Wayland-only session or +a KMSDRM handheld with no X server. Build instructions and the reasoning are +in [docs/linux-arm64-build.md](docs/linux-arm64-build.md). ## iOS diff --git a/docs/linux-arm64-build.md b/docs/linux-arm64-build.md index c40a8bef..b02e0ced 100644 --- a/docs/linux-arm64-build.md +++ b/docs/linux-arm64-build.md @@ -31,17 +31,22 @@ install it (`sudo apt install libfuse2`) or run without it: ### What the host has to provide -The AppImage bundles LÖVE, SDL2, OpenAL and the audio/video decoders. It -deliberately does **not** bundle the graphics drivers, the audio server -client libraries, or the font stack — those have to come from your system, -because bundled copies would either bypass your GPU driver or disagree with -libraries your desktop already has loaded (see -[Why the font stack is not bundled](#why-the-font-stack-is-not-bundled)). +Very little, and this is enforced by an assertion in the build rather than by +good intentions. The only libraries the AppImage requires at startup are: -In practice any arm64 system with a working desktop already satisfies this. -The requirements are glibc 2.29 or newer, plus Mesa/GL, X11 or Wayland, -ALSA or PulseAudio, and freetype/fontconfig — i.e. `libgl1`, `libfreetype6`, -`libfontconfig1`, `libpng16-16`, `libx11-6`. +``` +glibc 2.29+ libstdc++ libfreetype6 zlib +``` + +Everything else — OpenGL/Mesa, X11, Wayland, KMSDRM, ALSA, PulseAudio — is +**dlopened**, so it is used when present and skipped when absent. That means +one image runs on a full desktop, on a Wayland-only session, on a +KMSDRM-only handheld with no X server, and on a box with ALSA but no +PulseAudio, without a different build for each. + +That property does not come for free from Debian's packages, and getting it +is most of what the build below is doing; see +[Why five libraries are built from source](#why-five-libraries-are-built-from-source). ## For builders @@ -64,8 +69,8 @@ downloads and the compiled LÖVE prefix. ### Requirements An **aarch64 host** with **docker or podman**. A Raspberry Pi 5 is the -reference machine (a full build takes about 3.5 minutes on one; rebuilds -reuse the cached LÖVE prefix and take seconds). Apple Silicon with Docker +reference machine (a cold build takes about 10 minutes on one — six libraries +plus the engine; rebuilds reuse the cached prefix and take seconds). Apple Silicon with Docker Desktop and GitHub's `ubuntu-24.04-arm` runner both work too. The script refuses to run on x86_64 rather than falling back to qemu-user @@ -80,10 +85,10 @@ trick is not available here — **LÖVE publishes no aarch64 binary at all.** Th and that is the entire list. So this build compiles LÖVE 11.5 from the official `linux-src` tarball and -assembles the AppImage from scratch. Both pinned inputs (the LÖVE source -tarball and the AppImage type-2 runtime) are SHA-256 verified on the host -before the container ever sees them, and the container itself runs with no -network access. +assembles the AppImage from scratch. Every pinned input — the LÖVE source, the +five libraries built alongside it, and the AppImage type-2 runtime — is +SHA-256 verified on the host before the container ever sees it, and the +container itself runs with no network access. ### Why the build happens in a Debian bullseye container @@ -103,42 +108,59 @@ floor and strand every user on an older one, with no symptom until they download it. CI enforces the floor: `linux-arm64-build` fails if the highest required glibc symbol version climbs above 2.31. -### Why the font stack is not bundled +### Why five libraries are built from source -The dependency walker copies in what LÖVE needs and leaves everything else to -the host. Three categories are excluded, and the third one is subtle enough -to be worth writing down, because it is a real crash that shipped in an early -version of this build: +SDL2, OpenAL, libtheora, libogg/libvorbis and libmpg123 are compiled rather +than installed from bullseye. In every case the reason is *correctness*, not +a newer version number — Debian builds these for a system where every +dependency is installed and co-versioned, which is the opposite of an +AppImage's situation. Each one broke the build in a different way, and all +three failure modes are now assertions that fail the build instead of +shipping. -1. **Driver and session coupled** — GL/EGL/gbm/drm, X11/xcb/Wayland, D-Bus, - PulseAudio, ALSA, systemd/udev. A bundled `libGL` would bypass Mesa's V3D - driver on the Pi; a bundled `libpulse` would fight the running sound server. -2. **Loader coupled** — glibc's own pieces cannot be mixed with the host's - `ld.so`, and `libstdc++`/`libgcc_s` must be at least as new as the compiler - that built us (bullseye's gcc 10 is older than any supported host's, so the - host copy always satisfies us). -3. **Shared with the host font stack** — freetype, fontconfig, libpng, brotli, - zlib. +**1. Hard-linked backends (SDL2, OpenAL).** Debian's `libSDL2` lists +`libpulse`, `libasound`, `libX11` and `libwayland-client` as `DT_NEEDED` — +resolved by the loader at startup, not dlopened. An AppImage bundling it +refuses to start unless the host has *all four*. It appeared to work in +testing only because a desktop Pi has all four; a headless CI runner is what +exposed it. Debian's OpenAL does the same via `libsndio`, which itself +hard-links `libasound`. Built from source with `--enable-*-shared` and +`ALSOFT_DLOPEN`, both dlopen their backends instead. -That third one exists because Debian's `libtheoradec.so.1` is, oddly, linked -against `libcairo.so.2`. LÖVE needs theora for `love.video`, so the host's -cairo gets pulled into our process. The dynamic loader resolves one SONAME -exactly once per process, so a host cairo then binds to whatever -`libfreetype.so.6` *we* bundled: +**2. A stray link (libtheora).** Debian's `libtheoradec.so.1` is linked +against `libcairo.so.2` — a packaging artifact, since a video decoder has no +business drawing vector graphics — and cairo drags in X11, xcb, fontconfig +and freetype. `--disable-examples` produces a `libtheoradec` needing only +`libogg`. + +**3. SONAME collision with the host (ogg, vorbis, mpg123).** The subtle one. +OpenAL dlopens ALSA, ALSA's config loads its PulseAudio hook plugin, and that +plugin pulls the *host's* `libsndfile` into our process. `libsndfile` links +`libogg`, `libvorbis` and `libmpg123` — the same three we bundle. The loader +resolves a SONAME exactly once per process, so the host's `libsndfile` binds +to *our* copies: ``` -love -> liblove -> libtheoradec -> libcairo (host, new) - `-> FT_Get_Transform -> libfreetype (ours, bullseye 2.10.4) +openal -> libasound -> libasound_module_conf_pulse -> libsndfile (host, new) + `-> mpg123_info2 -> libmpg123 (ours, bullseye 1.26) ``` -`FT_Get_Transform` arrived in FreeType 2.11, so cairo 1.18 on a trixie host -fails to relocate and the game dies at startup with a symbol lookup error. -Bundling a *newer* freetype only moves the arms race one release along. -Excluding the whole font/compression stack instead makes the process -self-consistent: cairo, fontconfig and freetype all come from one host and -agree with each other, while `liblove` — compiled against 2.10.4 — only ever +`mpg123_info2` arrived in mpg123 1.32, so the plugin failed to relocate, ALSA +config collapsed, and the game ran with **no audio device at all**. Not +bundling these instead would make `libogg`/`libvorbis`/`libmpg123` mandatory +host packages; building them current means our copies *satisfy* the host's +`libsndfile` rather than starving it. + +The same collision is why the font stack — freetype, fontconfig, libpng, +brotli, zlib — is left to the host entirely. Bundling a bullseye freetype +2.10.4 meant a host `libcairo` could not find `FT_Get_Transform` (added in +2.11) and the game died at startup. Leaving the whole stack to the host keeps +it self-consistent, while `liblove` — compiled against 2.10.4 — only ever asks for symbols every supported host already has. +The general rule this all reduces to: **never bundle a library the host's own +stack may also load, unless yours is at least as new as theirs.** + ### CI Three jobs, path-gated on `scripts/build_linux_arm64.sh`, @@ -165,11 +187,16 @@ it runs on fork PRs too. Both pins live in `scripts/linux-arm64/common.sh`: -- `LOVE_VERSION` / `LOVE_SRC_SHA256` — bumping the LÖVE version invalidates - the cached prefix automatically (it is keyed by version). Check that - bullseye still has `-dev` packages new enough for the new release; - `build_appimage.sh` asserts every optional module actually linked, because - LÖVE's `configure` exits 0 and silently drops a module when one is missing. +- `LOVE_VERSION` / `LOVE_SRC_SHA256` — bumping any version invalidates the + cached prefix automatically (its name is keyed by every source version at + once, so a partial rebuild cannot mix vintages). Check that bullseye still + has `-dev` packages new enough for the new release; `build_appimage.sh` + asserts every optional module actually linked, because LÖVE's `configure` + exits 0 and silently drops a module when one is missing. +- `SDL2_*`, `OPENAL_*`, `THEORA_*`, `OGG_*`, `VORBIS_*`, `MPG123_*` — the + source-built libraries. Bumping these is usually safe and occasionally + necessary: `libmpg123` in particular must stay at least as new as what a + target host's `libsndfile` expects, which is asserted for `mpg123_info2`. - `APPIMAGE_RUNTIME_TAG` / `APPIMAGE_RUNTIME_SHA256` — always a dated tag from [AppImage/type2-runtime](https://github.com/AppImage/type2-runtime/releases). The selftest fails the build if this ever points at `continuous`. diff --git a/scripts/build_linux_arm64.sh b/scripts/build_linux_arm64.sh index e8e94a83..4df2ff00 100755 --- a/scripts/build_linux_arm64.sh +++ b/scripts/build_linux_arm64.sh @@ -104,6 +104,12 @@ cp "$GAME_LOVE" "$IN_DIR/game.love" # Fetched on the host and checksum-pinned here so the container never needs # network access and every input is verified in exactly one place. download_pinned "$LOVE_SRC_URL" "$CACHE/$LOVE_SRC_TARBALL" "$LOVE_SRC_SHA256" +download_pinned "$SDL2_URL" "$CACHE/$SDL2_TARBALL" "$SDL2_SHA256" +download_pinned "$OPENAL_URL" "$CACHE/$OPENAL_TARBALL" "$OPENAL_SHA256" +download_pinned "$THEORA_URL" "$CACHE/$THEORA_TARBALL" "$THEORA_SHA256" +download_pinned "$OGG_URL" "$CACHE/$OGG_TARBALL" "$OGG_SHA256" +download_pinned "$VORBIS_URL" "$CACHE/$VORBIS_TARBALL" "$VORBIS_SHA256" +download_pinned "$MPG123_URL" "$CACHE/$MPG123_TARBALL" "$MPG123_SHA256" download_pinned "$APPIMAGE_RUNTIME_URL" "$CACHE/$APPIMAGE_RUNTIME_NAME" \ "$APPIMAGE_RUNTIME_SHA256" @@ -129,6 +135,18 @@ fi say "compiling and packaging inside $BUILDER_BASE_IMAGE" "$RUNTIME" run --rm ${user_args[@]+"${user_args[@]}"} \ -e LOVE_VERSION="$LOVE_VERSION" \ + -e SDL2_VERSION="$SDL2_VERSION" \ + -e SDL2_TARBALL="$SDL2_TARBALL" \ + -e OPENAL_VERSION="$OPENAL_VERSION" \ + -e OPENAL_TARBALL="$OPENAL_TARBALL" \ + -e THEORA_VERSION="$THEORA_VERSION" \ + -e THEORA_TARBALL="$THEORA_TARBALL" \ + -e OGG_VERSION="$OGG_VERSION" \ + -e OGG_TARBALL="$OGG_TARBALL" \ + -e VORBIS_VERSION="$VORBIS_VERSION" \ + -e VORBIS_TARBALL="$VORBIS_TARBALL" \ + -e MPG123_VERSION="$MPG123_VERSION" \ + -e MPG123_TARBALL="$MPG123_TARBALL" \ -e APP_NAME="$APP_NAME" \ -e VERSION="$VERSION" \ -v "$CACHE:/cache" \ diff --git a/scripts/linux-arm64/Dockerfile b/scripts/linux-arm64/Dockerfile index 65990884..ab12a3b7 100644 --- a/scripts/linux-arm64/Dockerfile +++ b/scripts/linux-arm64/Dockerfile @@ -14,16 +14,32 @@ ENV DEBIAN_FRONTEND=noninteractive # build-essential/autoconf: LÖVE 11.5's linux-src tarball is autotools. # squashfs-tools: packs the AppDir into the AppImage payload. -# The lib*-dev set is LÖVE's full optional-module surface — a missing one +# +# Note what is deliberately ABSENT: libsdl2-dev, libtheora-dev and +# libopenal-dev. All three are built from source instead (see common.sh for +# why), and having Debian's copies installed would let pkg-config hand LÖVE's +# configure the system ones and silently undo it. +# +# The remaining lib*-dev set is LÖVE's optional-module surface. A missing one # does not fail configure, it silently drops a module (love.sound decoders, -# love.font, love.video), so they are pinned here deliberately. +# love.font, love.video), so they are pinned here deliberately and asserted +# after the build. +# +# The X11/Wayland/audio -dev packages are here for SDL2's *build*, not for +# runtime linkage: SDL detects each backend at compile time and then dlopens +# it, so these headers decide which backends exist at all while adding no +# DT_NEEDED entry to the shipped library. RUN apt-get update -qq \ && apt-get install -y --no-install-recommends \ - build-essential pkg-config autoconf automake libtool \ - ca-certificates curl file xz-utils zip unzip squashfs-tools \ - libsdl2-dev libopenal-dev libogg-dev libvorbis-dev libtheora-dev \ + build-essential pkg-config autoconf automake libtool cmake \ + ca-certificates curl file xz-utils bzip2 zip unzip squashfs-tools \ + libogg-dev libvorbis-dev \ libmodplug-dev libmpg123-dev libfreetype6-dev libluajit-5.1-dev \ - zlib1g-dev libgl1-mesa-dev libgles2-mesa-dev \ + zlib1g-dev libgl1-mesa-dev libgles2-mesa-dev libegl1-mesa-dev \ + libasound2-dev libpulse-dev libudev-dev libdbus-1-dev \ + libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev \ + libxinerama-dev libxss-dev libxkbcommon-dev \ + libwayland-dev wayland-protocols libdrm-dev libgbm-dev \ && rm -rf /var/lib/apt/lists/* WORKDIR /work diff --git a/scripts/linux-arm64/build_appimage.sh b/scripts/linux-arm64/build_appimage.sh index 420d586c..53fc8fdc 100755 --- a/scripts/linux-arm64/build_appimage.sh +++ b/scripts/linux-arm64/build_appimage.sh @@ -15,6 +15,18 @@ set -euo pipefail LOVE_VERSION="${LOVE_VERSION:?}" +SDL2_VERSION="${SDL2_VERSION:?}" +SDL2_TARBALL="${SDL2_TARBALL:?}" +OPENAL_VERSION="${OPENAL_VERSION:?}" +OPENAL_TARBALL="${OPENAL_TARBALL:?}" +THEORA_VERSION="${THEORA_VERSION:?}" +THEORA_TARBALL="${THEORA_TARBALL:?}" +OGG_VERSION="${OGG_VERSION:?}" +OGG_TARBALL="${OGG_TARBALL:?}" +VORBIS_VERSION="${VORBIS_VERSION:?}" +VORBIS_TARBALL="${VORBIS_TARBALL:?}" +MPG123_VERSION="${MPG123_VERSION:?}" +MPG123_TARBALL="${MPG123_TARBALL:?}" APP_NAME="${APP_NAME:?}" VERSION="${VERSION:?}" JOBS="${JOBS:-$(nproc)}" @@ -29,16 +41,171 @@ fail() { printf '\033[1;31merror:\033[0m %s\n' "$*" >&2; exit 1; } mkdir -p "$WORK" +# --------------------------------------------------------------- prefix +# Everything we compile lands in one prefix, cached because the compiling is +# the only slow part (~5 min cold on a Pi 5) and is identical for every game +# version. The key includes every source version, so bumping any of them +# invalidates the cache instead of silently reusing a stale mix. +PREFIX="$CACHE/prefix-love$LOVE_VERSION-sdl$SDL2_VERSION-al$OPENAL_VERSION-theora$THEORA_VERSION-ogg$OGG_VERSION-vorbis$VORBIS_VERSION-mpg$MPG123_VERSION" +export PKG_CONFIG_PATH="$PREFIX/lib/pkgconfig" +# Our own libraries must win over the system ones during LÖVE's configure and +# link, or the whole point of building them is lost. +export LD_LIBRARY_PATH="$PREFIX/lib" + +# ------------------------------------------------------ compile audio codecs +# Ordered by dependency: vorbis needs ogg, and theora needs ogg too. All three +# are small, plain autotools builds -- well under a minute each. +build_autotools() { # $1 = label $2 = version $3 = tarball $4 = probe lib $5.. = configure args + local label="$1" version="$2" tarball="$3" probe="$4"; shift 4 + if [ -f "$PREFIX/lib/$probe" ]; then + say "reusing cached $label $version" + return 0 + fi + say "compiling $label $version" + local src="$WORK/$label-src" + rm -rf "$src"; mkdir -p "$src" + case "$tarball" in + *.tar.bz2) tar -xjf "$CACHE/$tarball" -C "$src" --strip-components=1 ;; + *) tar -xzf "$CACHE/$tarball" -C "$src" --strip-components=1 ;; + esac + ( + cd "$src" + # Several of these tarballs predate aarch64's entry in config.guess; the + # distro's copies know about it, so refresh them or configure bails out + # with "cannot guess build type". + for helper in config.guess config.sub; do + [ -f "$helper" ] && cp "/usr/share/misc/$helper" . 2>/dev/null + done + ./configure --prefix="$PREFIX" --disable-static "$@" >/dev/null + make -j"$JOBS" >/dev/null + make install >/dev/null + ) +} + +build_autotools ogg "$OGG_VERSION" "$OGG_TARBALL" libogg.so.0 +build_autotools vorbis "$VORBIS_VERSION" "$VORBIS_TARBALL" libvorbis.so.0 +# mpg123's ports/ tree and the command-line player are irrelevant here; only +# libmpg123 gets linked, and --disable-modules keeps the output-backend +# plugins (and their dlopen of ALSA/pulse) out of the shipped library. +build_autotools mpg123 "$MPG123_VERSION" "$MPG123_TARBALL" libmpg123.so.0 \ + --disable-modules --with-audio=dummy --disable-lfs-alias + +# The symbol that was missing when this was bullseye's copy. Assert it, so a +# version bump that quietly regresses below the host's expectations fails the +# build instead of silently killing audio again. +objdump -T "$PREFIX/lib/libmpg123.so.0" | grep -q 'mpg123_info2' \ + || fail "bundled libmpg123 lacks mpg123_info2; the host's libsndfile will fail to relocate" + +# ------------------------------------------------------------ compile SDL2 +# --enable-*-shared (the defaults, made explicit so a future SDL release +# cannot flip them under us) is the entire reason this is built from source: +# each backend is dlopened at runtime rather than becoming a DT_NEEDED entry, +# so the AppImage starts on a host with only ALSA, or only Wayland, or only +# KMSDRM, instead of demanding all of them at once the way Debian's build does. +if [ -f "$PREFIX/lib/libSDL2-2.0.so.0" ]; then + say "reusing cached SDL2 $SDL2_VERSION" +else + say "compiling SDL2 $SDL2_VERSION (jobs: $JOBS)" + rm -rf "$WORK/sdl-src"; mkdir -p "$WORK/sdl-src" + tar -xzf "$CACHE/$SDL2_TARBALL" -C "$WORK/sdl-src" --strip-components=1 + ( + cd "$WORK/sdl-src" + ./configure --prefix="$PREFIX" --disable-static \ + --enable-alsa --enable-alsa-shared \ + --enable-pulseaudio --enable-pulseaudio-shared \ + --enable-video-x11 --enable-x11-shared \ + --enable-video-wayland --enable-wayland-shared \ + --enable-video-kmsdrm --enable-kmsdrm-shared \ + --enable-libudev --disable-sndio --disable-jack --disable-esd \ + --disable-arts --disable-nas --disable-oss >/dev/null + make -j"$JOBS" >/dev/null + make install >/dev/null + ) +fi + +# Prove the dlopen intent actually took. If SDL ever hard-links an audio or +# video backend again, the AppImage silently regains a startup dependency on +# the host having that exact stack -- which is the bug this replaced. +sdl_lib="$PREFIX/lib/libSDL2-2.0.so.0" +[ -f "$sdl_lib" ] || fail "SDL2 build produced no libSDL2-2.0.so.0" +for forbidden in libpulse libasound libX11 libwayland libdrm libgbm libsndio; do + if objdump -p "$sdl_lib" | grep -q "NEEDED.*$forbidden"; then + fail "SDL2 hard-links $forbidden; it must dlopen its backends (--enable-*-shared)" + fi +done + +# ---------------------------------------------------- compile openal-soft +# ALSOFT_DLOPEN keeps the ALSA and PulseAudio backends behind dlopen, and +# sndio is switched off outright -- Debian enables it, which is what chained +# libopenal -> libsndio -> libasound into a mandatory startup dependency. +if [ -f "$PREFIX/lib/libopenal.so.1" ]; then + say "reusing cached openal-soft $OPENAL_VERSION" +else + say "compiling openal-soft $OPENAL_VERSION (jobs: $JOBS)" + rm -rf "$WORK/openal-src"; mkdir -p "$WORK/openal-src" + tar -xzf "$CACHE/$OPENAL_TARBALL" -C "$WORK/openal-src" --strip-components=1 + ( + cd "$WORK/openal-src" + cmake -S . -B build \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="$PREFIX" \ + -DALSOFT_DLOPEN=ON \ + -DALSOFT_BACKEND_SNDIO=OFF \ + -DALSOFT_BACKEND_OSS=OFF \ + -DALSOFT_BACKEND_JACK=OFF \ + -DALSOFT_EXAMPLES=OFF \ + -DALSOFT_UTILS=OFF \ + -DALSOFT_TESTS=OFF \ + -DLIBTYPE=SHARED >/dev/null + cmake --build build -j"$JOBS" >/dev/null + cmake --install build >/dev/null + ) +fi + +openal_lib="$PREFIX/lib/libopenal.so.1" +[ -f "$openal_lib" ] || fail "openal-soft build produced no libopenal.so.1" +for forbidden in libsndio libasound libpulse libjack; do + if objdump -p "$openal_lib" | grep -q "NEEDED.*$forbidden"; then + fail "openal hard-links $forbidden; backends must stay behind dlopen" + fi +done + +# --------------------------------------------------- compile libtheora +# --disable-examples is what drops Debian's libcairo link (and with it libX11, +# libxcb, libfontconfig and libfreetype as startup dependencies). The encoder +# is dead weight for a player, but libtheoradec is what LÖVE actually links. +if [ -f "$PREFIX/lib/libtheoradec.so.1" ]; then + say "reusing cached libtheora $THEORA_VERSION" +else + say "compiling libtheora $THEORA_VERSION" + rm -rf "$WORK/theora-src"; mkdir -p "$WORK/theora-src" + tar -xjf "$CACHE/$THEORA_TARBALL" -C "$WORK/theora-src" --strip-components=1 + ( + cd "$WORK/theora-src" + # theora 1.1.1 predates the aarch64 config.guess, so refresh the autotools + # helper scripts or configure rejects the host outright. + for helper in config.guess config.sub; do + cp "/usr/share/misc/$helper" . 2>/dev/null || true + done + ./configure --prefix="$PREFIX" --disable-static \ + --disable-examples --disable-spec --disable-doc >/dev/null + make -j"$JOBS" >/dev/null + make install >/dev/null + ) +fi + +theora_lib="$PREFIX/lib/libtheoradec.so.1" +[ -f "$theora_lib" ] || fail "libtheora build produced no libtheoradec.so.1" +if objdump -p "$theora_lib" | grep -q "NEEDED.*libcairo"; then + fail "libtheoradec still links libcairo (--disable-examples stopped working)" +fi + # ------------------------------------------------------------ compile LÖVE -# The prefix is cached because this is the only slow step (~3 min on a Pi 5, -# and it is identical for every game version). Keyed by LÖVE version so a -# LOVE_VERSION bump cannot silently reuse the old build. -PREFIX="$CACHE/love-$LOVE_VERSION-prefix" if [ -x "$PREFIX/bin/love" ] && [ -f "$PREFIX/lib/liblove-$LOVE_VERSION.so" ]; then say "reusing cached LÖVE $LOVE_VERSION aarch64 build" else say "compiling LÖVE $LOVE_VERSION for aarch64 (jobs: $JOBS)" - rm -rf "$PREFIX" "$WORK/love-src" + rm -rf "$WORK/love-src" mkdir -p "$WORK/love-src" tar -xzf "$CACHE/love-$LOVE_VERSION-linux-src.tar.gz" \ -C "$WORK/love-src" --strip-components=1 @@ -46,8 +213,11 @@ else cd "$WORK/love-src" # No --disable-* flags on purpose: configure silently drops a love module # when its -dev package is absent, so the Dockerfile pins the full set and - # the assertions below prove each one actually linked. - ./configure --prefix="$PREFIX" --disable-static >/dev/null + # the assertions below prove each one actually linked. CPPFLAGS/LDFLAGS + # point at our prefix so the SDL2 and theora just built above win over + # anything the base image might still provide. + ./configure --prefix="$PREFIX" --disable-static \ + CPPFLAGS="-I$PREFIX/include" LDFLAGS="-L$PREFIX/lib" >/dev/null make -j"$JOBS" >/dev/null make install >/dev/null # Keep LÖVE's license inside the cached prefix: the unpacked source tree @@ -97,29 +267,23 @@ chmod +x "$APPDIR/bin/love" # 1. Driver/session coupled. A bundled libGL would bypass Mesa's V3D driver # on the Pi; a bundled libpulse/libdbus would fight the user's running # session. GL/EGL/gbm/drm, X11/xcb/wayland/xkbcommon, dbus, pulse, alsa, -# systemd/udev. +# systemd/udev. Note that after the source builds above, none of these are +# DT_NEEDED of anything we ship -- SDL2 and OpenAL dlopen them, so they are +# used when present and skipped when absent. # # 2. Loader coupled. glibc's pieces cannot be mixed with the host's ld.so at # all, and libstdc++/libgcc_s must be at least as new as the compiler -- # bullseye's gcc 10 is older than any supported host's, so the host copy # always satisfies us. # -# 3. Shared with the host's font stack -- the subtle one, and the reason -# this list is longer than LÖVE's own AppImage manifest. Bullseye's -# libtheoradec is (bizarrely, a Debian packaging artifact) linked against -# libcairo, so the HOST's cairo gets loaded into our process. Because the -# dynamic loader resolves one SONAME once per process, that host cairo -# then binds to whatever libfreetype.so.6 we bundled -- and a bullseye -# freetype 2.10.4 has no FT_Get_Transform, which cairo 1.18 needs: -# -# love -> liblove -> libtheoradec -> libcairo (host, new) -# `-> FT_Get_Transform -> libfreetype (ours, old) BOOM -# -# Bundling a newer freetype only moves the arms race. Excluding the whole -# font/compression stack instead makes it self-consistent: cairo, -# fontconfig and freetype all come from one host and agree with each -# other, while liblove -- compiled against 2.10.4 -- only ever asks for -# symbols every supported host already has. +# 3. The font/compression stack: freetype, fontconfig, libpng, brotli, zlib. +# These are shared with whatever the host's own graphics libraries have +# already loaded, and mixing vintages inside one process breaks the older +# copy. Bundling a bullseye freetype 2.10.4 is what made a host cairo fail +# to find FT_Get_Transform (added in 2.11) and killed the game at startup. +# Leaving the whole stack to the host keeps it self-consistent, and +# liblove -- compiled against 2.10.4 -- only ever asks for symbols every +# supported host already has. EXCLUDE_RE='^(ld-linux-aarch64\.so\.1|libc\.so\.6|libm\.so\.6|libdl\.so\.2|libpthread\.so\.0|librt\.so\.1|libresolv\.so\.2|libutil\.so\.1|libanl\.so\.1|libnsl\.so\.[0-9]+|libstdc\+\+\.so\.6|libgcc_s\.so\.1|lib(GL|GLX|GLdispatch|OpenGL|EGL|GLESv[12]|glapi|gbm|drm)\..*|libX[a-z0-9]*\..*|libxcb.*|libwayland-.*|libxkbcommon.*|libdbus-1\..*|libpulse.*|libasound\..*|libsndfile\..*|libFLAC\..*|libopus\..*|libsystemd\..*|libudev\..*|libselinux\..*|libcap\..*|libgcrypt\..*|libgpg-error\..*|liblzma\..*|libzstd\..*|liblz4\..*|libffi\..*|libexpat\..*|libbsd\..*|libmd\..*|libuuid\..*|libg(lib|object|module|thread)-2\..*|libfontconfig\..*|libfreetype\..*|libpng[0-9]*\..*|libbrotli.*|libz\.so\..*|libwrap\..*|libasyncns\..*|libtirpc\..*|lib(gssapi_krb5|krb5|k5crypto|com_err|krb5support|keyutils)\..*|libpcre.*)$' # soname -> absolute path, harvested from the full ldd closure of both roots. @@ -156,6 +320,33 @@ bundle_needed "$APPDIR/bin/love" bundle_needed "$APPDIR/lib/liblove-$LOVE_VERSION.so" say "bundled $(ls "$APPDIR/lib" | wc -l) libraries: $(ls "$APPDIR/lib" | tr '\n' ' ')" +# ------------------------------------------------- host dependency contract +# The portability promise, stated as an assertion instead of a paragraph in a +# README: these are the ONLY sonames the shipped objects may require from the +# host. Everything driver-, session- or audio-related has to be reached +# through dlopen, so the AppImage starts on a box with no PulseAudio, no X11 +# or no ALSA and simply uses whatever it does find. +# +# The original build failed exactly here and nobody noticed until CI ran on a +# headless runner: Debian's SDL2 hard-links libpulse/libasound/libX11/ +# libwayland, so the image only ever started on a full desktop. +HOST_ALLOWED_RE='^(ld-linux-aarch64\.so\.1|libc\.so\.6|libm\.so\.6|libdl\.so\.2|libpthread\.so\.0|librt\.so\.1|libstdc\+\+\.so\.6|libgcc_s\.so\.1|libatomic\.so\.1|libfreetype\.so\.6|libpng[0-9]*\.so\.[0-9]+|libz\.so\.1|libbrotli(dec|common)\.so\.1)$' + +unexpected="" +for object in "$APPDIR/bin/love" "$APPDIR"/lib/*.so*; do + while read -r soname; do + [ -n "$soname" ] || continue + # Satisfied from inside the AppDir, so not a host requirement at all. + if [ -n "${BUNDLED[$soname]:-}" ]; then continue; fi + if [[ "$soname" =~ $HOST_ALLOWED_RE ]]; then continue; fi + unexpected="$unexpected $(basename "$object") -> $soname"$'\n' + done < <(objdump -p "$object" | awk '/NEEDED/ {print $2}') +done +[ -z "$unexpected" ] || fail "$(printf '%s\n%s' \ + "these objects hard-require host libraries outside the allowed set (they must be dlopened, not linked):" \ + "$unexpected")" +say "host dependency contract holds (glibc, libstdc++ and the font stack only)" + # LÖVE loads jit.* (jit.status, the profiler) through LUA_PATH; without these # the modules are simply absent, so ship them the way upstream's image does. jit_share="$(ls -d /usr/share/luajit-* 2>/dev/null | head -1)" diff --git a/scripts/linux-arm64/common.sh b/scripts/linux-arm64/common.sh index 210c67c2..39e196b4 100755 --- a/scripts/linux-arm64/common.sh +++ b/scripts/linux-arm64/common.sh @@ -18,6 +18,78 @@ LOVE_SRC_TARBALL="love-$LOVE_VERSION-linux-src.tar.gz" LOVE_SRC_URL="https://github.com/love2d/love/releases/download/$LOVE_VERSION/$LOVE_SRC_TARBALL" LOVE_SRC_SHA256="066e0843f71aa9fd28b8eaf27d41abb74bfaef7556153ac2e3cf08eafc874c39" +# SDL2 is built from source rather than taken from bullseye, and this is a +# correctness requirement, not a version preference. Debian's libSDL2 lists +# libpulse, libasound, libX11 and libwayland-client as DT_NEEDED -- hard links +# resolved by the loader at startup -- so an AppImage bundling it refuses to +# launch unless the host has ALL FOUR installed. That is wrong for an artifact +# whose whole job is to run on arbitrary arm64 systems: an ALSA-only handheld +# or a minimal Wayland box would die before main(). Built from source, SDL +# defaults to dlopening every audio and video backend (--enable-*-shared), so +# it loads whichever the host actually has and degrades gracefully. +# The newer version is a bonus: 2.30 has a far better controller database and +# real KMSDRM support, both of which matter on Pi-class and handheld hardware. +SDL2_VERSION="2.30.12" +SDL2_TARBALL="SDL2-$SDL2_VERSION.tar.gz" +SDL2_URL="https://github.com/libsdl-org/SDL/releases/download/release-$SDL2_VERSION/$SDL2_TARBALL" +SDL2_SHA256="ac356ea55e8b9dd0b2d1fa27da40ef7e238267ccf9324704850d5d47375b48ea" + +# libtheora likewise. Debian's libtheoradec.so.1 is linked against libcairo -- +# a packaging artifact, since a video decoder has no business drawing vector +# graphics -- and cairo drags in libX11, libxcb, libfontconfig and libfreetype +# as hard dependencies. LOVE needs theora for love.video, so that link would +# put the entire X11 and font stack on the critical path at startup, and it is +# what caused the FT_Get_Transform crash this build hit on a trixie host. +# Upstream's tarball with --disable-examples produces a libtheoradec that +# needs only libogg. +THEORA_VERSION="1.1.1" +THEORA_TARBALL="libtheora-$THEORA_VERSION.tar.bz2" +THEORA_URL="https://downloads.xiph.org/releases/theora/$THEORA_TARBALL" +THEORA_SHA256="b6ae1ee2fa3d42ac489287d3ec34c5885730b1296f0801ae577a35193d3affbc" + +# OpenAL for the same reason as SDL2, one level down. Debian's libopenal is +# openal-soft built with the sndio backend enabled, so it hard-links +# libsndio, which itself hard-links libasound -- reintroducing exactly the +# mandatory-ALSA dependency the SDL2 source build exists to remove. Upstream +# openal-soft dlopens its backends, so building it here leaves the shipped +# library with no audio-stack dependency at all. +OPENAL_VERSION="1.23.1" +OPENAL_TARBALL="openal-soft-$OPENAL_VERSION.tar.gz" +OPENAL_URL="https://github.com/kcat/openal-soft/archive/refs/tags/$OPENAL_VERSION.tar.gz" +OPENAL_SHA256="dfddf3a1f61059853c625b7bb03de8433b455f2f79f89548cbcbd5edca3d4a4a" + +# The audio codecs are built from source for a third, different reason: SONAME +# collision with the host's audio stack. +# +# OpenAL dlopens ALSA, ALSA's config loads its PulseAudio hook plugin, and that +# plugin pulls the HOST's libsndfile into our process. libsndfile links +# libogg, libvorbis and libmpg123 -- the same three we bundle. The loader +# resolves a SONAME once per process, so the host's libsndfile binds to OUR +# copies, and a bullseye libmpg123 has no mpg123_info2 (added in 1.32): +# +# openal -> libasound -> libasound_module_conf_pulse -> libsndfile (host) +# `-> mpg123_info2 -> libmpg123 (ours, bullseye) +# +# which failed to relocate and left the game with no audio device at all. +# Not bundling them instead would make libogg/libvorbis/libmpg123 mandatory +# host packages; building them current means our copies satisfy the host's +# libsndfile rather than starving it. libvorbisfile ships in the vorbis +# tarball. +OGG_VERSION="1.3.5" +OGG_TARBALL="libogg-$OGG_VERSION.tar.gz" +OGG_URL="https://downloads.xiph.org/releases/ogg/$OGG_TARBALL" +OGG_SHA256="0eb4b4b9420a0f51db142ba3f9c64b333f826532dc0f48c6410ae51f4799b664" + +VORBIS_VERSION="1.3.7" +VORBIS_TARBALL="libvorbis-$VORBIS_VERSION.tar.gz" +VORBIS_URL="https://downloads.xiph.org/releases/vorbis/$VORBIS_TARBALL" +VORBIS_SHA256="0e982409a9c3fc82ee06e08205b1355e5c6aa4c36bca58146ef399621b0ce5ab" + +MPG123_VERSION="1.32.10" +MPG123_TARBALL="mpg123-$MPG123_VERSION.tar.bz2" +MPG123_URL="https://www.mpg123.de/download/$MPG123_TARBALL" +MPG123_SHA256="87b2c17fe0c979d3ef38eeceff6362b35b28ac8589fbf1854b5be75c9ab6557c" + # AppImage type-2 runtime: the ~900 KB static-pie ELF that gets prepended to # the squashfs payload. Pinned to a dated tag, never "continuous", so a # rebuild months from now produces the same bytes. diff --git a/scripts/linux-arm64/selftest_build_linux_arm64.sh b/scripts/linux-arm64/selftest_build_linux_arm64.sh index 1a25b177..81ac3008 100755 --- a/scripts/linux-arm64/selftest_build_linux_arm64.sh +++ b/scripts/linux-arm64/selftest_build_linux_arm64.sh @@ -43,7 +43,8 @@ printf '%s' "$guard_out" | grep -q 'aarch64 host' \ say "checking pinned inputs" # Pins must be real digests, and the AppImage runtime must come from a dated # tag: "continuous" is a moving target and would make rebuilds unreproducible. -for pin_name in LOVE_SRC_SHA256 APPIMAGE_RUNTIME_SHA256; do +for pin_name in LOVE_SRC_SHA256 SDL2_SHA256 OPENAL_SHA256 THEORA_SHA256 \ + OGG_SHA256 VORBIS_SHA256 MPG123_SHA256 APPIMAGE_RUNTIME_SHA256; do pin_value="${!pin_name}" printf '%s' "$pin_value" | grep -Eq '^[0-9a-f]{64}$' \ || fail "$pin_name is not a sha256 digest: $pin_value" @@ -113,6 +114,48 @@ for soname in libSDL2-2.0.so.0 libopenal.so.1 libfreetype.so.6 libmodplug.so.1 \ || fail "build_appimage.sh no longer asserts liblove links $soname" done +say "checking the dlopen guarantees" +# SDL2, OpenAL and libtheora are compiled from source for correctness, not for +# a newer version number: Debian's builds hard-link libpulse/libasound/libX11/ +# libwayland (SDL2), libsndio (OpenAL) and libcairo (libtheora), each of which +# turns an optional runtime capability into a mandatory startup dependency. +# If a future edit drops the source build and reaches for the -dev package +# again, the AppImage silently stops starting on lean systems. +for forbidden_pkg in libsdl2-dev libtheora-dev libopenal-dev; do + if grep -qE "^ +.*\b$forbidden_pkg\b" "$SCRIPT_DIR/Dockerfile"; then + fail "Dockerfile installs $forbidden_pkg; that library is built from source on purpose" + fi +done +grep -qF -- '--enable-alsa-shared' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "SDL2 is no longer configured to dlopen its audio backends" +grep -qF -- '--enable-x11-shared' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "SDL2 is no longer configured to dlopen its video backends" +grep -qF 'ALSOFT_DLOPEN=ON' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "openal-soft is no longer configured to dlopen its backends" +grep -qF -- '--disable-examples' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "libtheora is no longer built with --disable-examples (it regains the libcairo link)" + +say "checking the host dependency contract" +# The shipped objects may require nothing from the host beyond glibc, +# libstdc++ and the font stack. Everything driver-, session- or audio-related +# has to be dlopened. This is the invariant a headless CI runner proved was +# broken the first time round. +HOST_ALLOWED_RE="$( + grep -m1 "^HOST_ALLOWED_RE=" "$SCRIPT_DIR/build_appimage.sh" \ + | sed "s/^HOST_ALLOWED_RE='//; s/'\$//" +)" +[ -n "$HOST_ALLOWED_RE" ] || fail "could not read HOST_ALLOWED_RE out of build_appimage.sh" +for soname in libpulse.so.0 libasound.so.2 libX11.so.6 libwayland-client.so.0 \ + libGL.so.1 libcairo.so.2 libsndio.so.7.0 libdbus-1.so.3; do + if [[ "$soname" =~ $HOST_ALLOWED_RE ]]; then + fail "$soname is allowed as a hard host dependency; it must be dlopened" + fi +done +for soname in libc.so.6 libstdc++.so.6 libfreetype.so.6 libz.so.1; do + [[ "$soname" =~ $HOST_ALLOWED_RE ]] \ + || fail "$soname must be allowed as a host dependency but the contract rejects it" +done + say "checking the shared game.love payload" temp_dir="$(mktemp -d "${TMPDIR:-/tmp}/gen1recomp-linux-arm64-selftest.XXXXXX")" trap 'rm -rf "$temp_dir"' EXIT