From f92364a0027bbf14c8a9a54bbb187a8ad48bc33f Mon Sep 17 00:00:00 2001 From: ratherDashing <1879051+ratherDashing@users.noreply.github.com> Date: Wed, 5 Aug 2026 15:24:53 -0400 Subject: [PATCH] Build SDL2, OpenAL and the codecs from source for the arm64 AppImage CI on a headless ubuntu-24.04-arm runner caught what a desktop Pi could not: the AppImage only started on a machine that already had a full desktop stack installed. Three distinct causes, all from bundling Debian's builds of libraries that Debian builds for a co-versioned system, which is the opposite of an AppImage's situation. 1. Hard-linked backends. Debian's libSDL2 lists libpulse, libasound, libX11 and libwayland-client as DT_NEEDED rather than dlopening them, so the loader demanded all four at startup; the CI job failed with "libpulse.so.0 => not found". Debian's OpenAL does the same through libsndio, which itself hard-links libasound. Built from source with --enable-*-shared and ALSOFT_DLOPEN, both dlopen their backends, so the image now runs on a Wayland-only session, a KMSDRM handheld with no X server, or a box with ALSA and no PulseAudio. 2. A stray link. Debian's libtheoradec is linked against libcairo, which drags in X11, xcb, fontconfig and freetype for a video decoder. --disable-examples leaves it needing only libogg. 3. SONAME collision with the host. OpenAL dlopens ALSA, ALSA's config loads its PulseAudio hook plugin, and that plugin pulls the host's libsndfile into the process. libsndfile links libogg, libvorbis and libmpg123 -- the same three we bundle -- and since the loader resolves a SONAME once per process it bound to our bullseye copies. A bullseye libmpg123 has no mpg123_info2 (added in 1.32), so the plugin failed to relocate, ALSA config collapsed, and the game ran with no audio device at all. Building them current means our copies satisfy the host's libsndfile instead of starving it. The general rule, now stated as an assertion instead of a comment: never bundle a library the host's own stack may also load unless ours is at least as new as theirs. build_appimage.sh fails if any shipped object hard-requires anything beyond glibc, libstdc++ and the font stack, and CI re-checks it on the extracted artifact. Host requirements drop from "a working desktop" to glibc 2.29+, libstdc++, libfreetype6 and zlib. Bundled libraries drop from 13 to 10: libcairo, libpixman and libsndio are gone entirely. Verified on a Raspberry Pi 5 (trixie, Wayland): boots, imports, plays, and audio works -- SDL 2.30 now picks the native Wayland backend rather than falling back to XWayland as bullseye's 2.0.14 did. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 14 + README.md | 11 +- docs/linux-arm64-build.md | 123 +++++---- scripts/build_linux_arm64.sh | 18 ++ scripts/linux-arm64/Dockerfile | 28 +- scripts/linux-arm64/build_appimage.sh | 239 ++++++++++++++++-- scripts/linux-arm64/common.sh | 72 ++++++ .../linux-arm64/selftest_build_linux_arm64.sh | 45 +++- 8 files changed, 466 insertions(+), 84 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 033a9d71..94075d42 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -330,11 +330,25 @@ jobs: # Every bundled object must resolve once AppRun's LD_LIBRARY_PATH is # applied; an unresolved soname here is a user-visible launch crash. + # + # This runs on a HEADLESS runner on purpose, and that is the point. + # The first version of this build bundled Debian's SDL2, which + # hard-links libpulse/libasound/libX11/libwayland, so it only ever + # started on a full desktop -- a bare runner is what exposed it. missing="$(LD_LIBRARY_PATH="$PWD/squashfs-root/lib" \ ldd squashfs-root/bin/love squashfs-root/lib/*.so* 2>/dev/null \ | grep 'not found' || true)" [ -z "$missing" ] || { echo "::error::unresolved deps:"; echo "$missing"; exit 1; } + # Nothing may hard-link a driver, session or audio-stack library: + # those must be reached through dlopen so the AppImage runs on a box + # with only ALSA, only Wayland, or only KMSDRM. + linked="$(for f in squashfs-root/bin/love squashfs-root/lib/*.so*; do + objdump -p "$f" 2>/dev/null | awk '/NEEDED/{print $2}' + done | sort -u | grep -E '^lib(pulse|asound|X11|wayland|GL|EGL|drm|gbm|xcb|cairo|sndio|dbus)' || true)" + [ -z "$linked" ] \ + || { echo "::error::these must be dlopened, not linked:"; echo "$linked"; exit 1; } + # The whole point of compiling on bullseye. If a future change moves # the builder to a newer base, the glibc floor silently rises and # every user on an older distro gets "GLIBC_2.xx not found" -- catch diff --git a/README.md b/README.md index 45afadb6..ce866511 100644 --- a/README.md +++ b/README.md @@ -217,11 +217,12 @@ chmod +x gen1recomp-*-linux-arm64.AppImage ``` LÖVE publishes no aarch64 binary of any kind, so this artifact compiles the -engine from source inside a Debian bullseye arm64 container; the result needs -only glibc 2.29+, which covers Raspberry Pi OS bullseye through trixie and -Ubuntu 20.04 onward. Build instructions, the host requirements, and why the -font stack is deliberately left unbundled are in -[docs/linux-arm64-build.md](docs/linux-arm64-build.md). +engine — and SDL2, OpenAL and the codecs — from source inside a Debian +bullseye arm64 container. It needs only glibc 2.29+, libstdc++, freetype and +zlib on the host; OpenGL, X11, Wayland, KMSDRM, ALSA and PulseAudio are all +dlopened, so the same image runs on a full desktop, a Wayland-only session or +a KMSDRM handheld with no X server. Build instructions and the reasoning are +in [docs/linux-arm64-build.md](docs/linux-arm64-build.md). ## iOS diff --git a/docs/linux-arm64-build.md b/docs/linux-arm64-build.md index c40a8bef..b02e0ced 100644 --- a/docs/linux-arm64-build.md +++ b/docs/linux-arm64-build.md @@ -31,17 +31,22 @@ install it (`sudo apt install libfuse2`) or run without it: ### What the host has to provide -The AppImage bundles LÖVE, SDL2, OpenAL and the audio/video decoders. It -deliberately does **not** bundle the graphics drivers, the audio server -client libraries, or the font stack — those have to come from your system, -because bundled copies would either bypass your GPU driver or disagree with -libraries your desktop already has loaded (see -[Why the font stack is not bundled](#why-the-font-stack-is-not-bundled)). +Very little, and this is enforced by an assertion in the build rather than by +good intentions. The only libraries the AppImage requires at startup are: -In practice any arm64 system with a working desktop already satisfies this. -The requirements are glibc 2.29 or newer, plus Mesa/GL, X11 or Wayland, -ALSA or PulseAudio, and freetype/fontconfig — i.e. `libgl1`, `libfreetype6`, -`libfontconfig1`, `libpng16-16`, `libx11-6`. +``` +glibc 2.29+ libstdc++ libfreetype6 zlib +``` + +Everything else — OpenGL/Mesa, X11, Wayland, KMSDRM, ALSA, PulseAudio — is +**dlopened**, so it is used when present and skipped when absent. That means +one image runs on a full desktop, on a Wayland-only session, on a +KMSDRM-only handheld with no X server, and on a box with ALSA but no +PulseAudio, without a different build for each. + +That property does not come for free from Debian's packages, and getting it +is most of what the build below is doing; see +[Why five libraries are built from source](#why-five-libraries-are-built-from-source). ## For builders @@ -64,8 +69,8 @@ downloads and the compiled LÖVE prefix. ### Requirements An **aarch64 host** with **docker or podman**. A Raspberry Pi 5 is the -reference machine (a full build takes about 3.5 minutes on one; rebuilds -reuse the cached LÖVE prefix and take seconds). Apple Silicon with Docker +reference machine (a cold build takes about 10 minutes on one — six libraries +plus the engine; rebuilds reuse the cached prefix and take seconds). Apple Silicon with Docker Desktop and GitHub's `ubuntu-24.04-arm` runner both work too. The script refuses to run on x86_64 rather than falling back to qemu-user @@ -80,10 +85,10 @@ trick is not available here — **LÖVE publishes no aarch64 binary at all.** Th and that is the entire list. So this build compiles LÖVE 11.5 from the official `linux-src` tarball and -assembles the AppImage from scratch. Both pinned inputs (the LÖVE source -tarball and the AppImage type-2 runtime) are SHA-256 verified on the host -before the container ever sees them, and the container itself runs with no -network access. +assembles the AppImage from scratch. Every pinned input — the LÖVE source, the +five libraries built alongside it, and the AppImage type-2 runtime — is +SHA-256 verified on the host before the container ever sees it, and the +container itself runs with no network access. ### Why the build happens in a Debian bullseye container @@ -103,42 +108,59 @@ floor and strand every user on an older one, with no symptom until they download it. CI enforces the floor: `linux-arm64-build` fails if the highest required glibc symbol version climbs above 2.31. -### Why the font stack is not bundled +### Why five libraries are built from source -The dependency walker copies in what LÖVE needs and leaves everything else to -the host. Three categories are excluded, and the third one is subtle enough -to be worth writing down, because it is a real crash that shipped in an early -version of this build: +SDL2, OpenAL, libtheora, libogg/libvorbis and libmpg123 are compiled rather +than installed from bullseye. In every case the reason is *correctness*, not +a newer version number — Debian builds these for a system where every +dependency is installed and co-versioned, which is the opposite of an +AppImage's situation. Each one broke the build in a different way, and all +three failure modes are now assertions that fail the build instead of +shipping. -1. **Driver and session coupled** — GL/EGL/gbm/drm, X11/xcb/Wayland, D-Bus, - PulseAudio, ALSA, systemd/udev. A bundled `libGL` would bypass Mesa's V3D - driver on the Pi; a bundled `libpulse` would fight the running sound server. -2. **Loader coupled** — glibc's own pieces cannot be mixed with the host's - `ld.so`, and `libstdc++`/`libgcc_s` must be at least as new as the compiler - that built us (bullseye's gcc 10 is older than any supported host's, so the - host copy always satisfies us). -3. **Shared with the host font stack** — freetype, fontconfig, libpng, brotli, - zlib. +**1. Hard-linked backends (SDL2, OpenAL).** Debian's `libSDL2` lists +`libpulse`, `libasound`, `libX11` and `libwayland-client` as `DT_NEEDED` — +resolved by the loader at startup, not dlopened. An AppImage bundling it +refuses to start unless the host has *all four*. It appeared to work in +testing only because a desktop Pi has all four; a headless CI runner is what +exposed it. Debian's OpenAL does the same via `libsndio`, which itself +hard-links `libasound`. Built from source with `--enable-*-shared` and +`ALSOFT_DLOPEN`, both dlopen their backends instead. -That third one exists because Debian's `libtheoradec.so.1` is, oddly, linked -against `libcairo.so.2`. LÖVE needs theora for `love.video`, so the host's -cairo gets pulled into our process. The dynamic loader resolves one SONAME -exactly once per process, so a host cairo then binds to whatever -`libfreetype.so.6` *we* bundled: +**2. A stray link (libtheora).** Debian's `libtheoradec.so.1` is linked +against `libcairo.so.2` — a packaging artifact, since a video decoder has no +business drawing vector graphics — and cairo drags in X11, xcb, fontconfig +and freetype. `--disable-examples` produces a `libtheoradec` needing only +`libogg`. + +**3. SONAME collision with the host (ogg, vorbis, mpg123).** The subtle one. +OpenAL dlopens ALSA, ALSA's config loads its PulseAudio hook plugin, and that +plugin pulls the *host's* `libsndfile` into our process. `libsndfile` links +`libogg`, `libvorbis` and `libmpg123` — the same three we bundle. The loader +resolves a SONAME exactly once per process, so the host's `libsndfile` binds +to *our* copies: ``` -love -> liblove -> libtheoradec -> libcairo (host, new) - `-> FT_Get_Transform -> libfreetype (ours, bullseye 2.10.4) +openal -> libasound -> libasound_module_conf_pulse -> libsndfile (host, new) + `-> mpg123_info2 -> libmpg123 (ours, bullseye 1.26) ``` -`FT_Get_Transform` arrived in FreeType 2.11, so cairo 1.18 on a trixie host -fails to relocate and the game dies at startup with a symbol lookup error. -Bundling a *newer* freetype only moves the arms race one release along. -Excluding the whole font/compression stack instead makes the process -self-consistent: cairo, fontconfig and freetype all come from one host and -agree with each other, while `liblove` — compiled against 2.10.4 — only ever +`mpg123_info2` arrived in mpg123 1.32, so the plugin failed to relocate, ALSA +config collapsed, and the game ran with **no audio device at all**. Not +bundling these instead would make `libogg`/`libvorbis`/`libmpg123` mandatory +host packages; building them current means our copies *satisfy* the host's +`libsndfile` rather than starving it. + +The same collision is why the font stack — freetype, fontconfig, libpng, +brotli, zlib — is left to the host entirely. Bundling a bullseye freetype +2.10.4 meant a host `libcairo` could not find `FT_Get_Transform` (added in +2.11) and the game died at startup. Leaving the whole stack to the host keeps +it self-consistent, while `liblove` — compiled against 2.10.4 — only ever asks for symbols every supported host already has. +The general rule this all reduces to: **never bundle a library the host's own +stack may also load, unless yours is at least as new as theirs.** + ### CI Three jobs, path-gated on `scripts/build_linux_arm64.sh`, @@ -165,11 +187,16 @@ it runs on fork PRs too. Both pins live in `scripts/linux-arm64/common.sh`: -- `LOVE_VERSION` / `LOVE_SRC_SHA256` — bumping the LÖVE version invalidates - the cached prefix automatically (it is keyed by version). Check that - bullseye still has `-dev` packages new enough for the new release; - `build_appimage.sh` asserts every optional module actually linked, because - LÖVE's `configure` exits 0 and silently drops a module when one is missing. +- `LOVE_VERSION` / `LOVE_SRC_SHA256` — bumping any version invalidates the + cached prefix automatically (its name is keyed by every source version at + once, so a partial rebuild cannot mix vintages). Check that bullseye still + has `-dev` packages new enough for the new release; `build_appimage.sh` + asserts every optional module actually linked, because LÖVE's `configure` + exits 0 and silently drops a module when one is missing. +- `SDL2_*`, `OPENAL_*`, `THEORA_*`, `OGG_*`, `VORBIS_*`, `MPG123_*` — the + source-built libraries. Bumping these is usually safe and occasionally + necessary: `libmpg123` in particular must stay at least as new as what a + target host's `libsndfile` expects, which is asserted for `mpg123_info2`. - `APPIMAGE_RUNTIME_TAG` / `APPIMAGE_RUNTIME_SHA256` — always a dated tag from [AppImage/type2-runtime](https://github.com/AppImage/type2-runtime/releases). The selftest fails the build if this ever points at `continuous`. diff --git a/scripts/build_linux_arm64.sh b/scripts/build_linux_arm64.sh index e8e94a83..4df2ff00 100755 --- a/scripts/build_linux_arm64.sh +++ b/scripts/build_linux_arm64.sh @@ -104,6 +104,12 @@ cp "$GAME_LOVE" "$IN_DIR/game.love" # Fetched on the host and checksum-pinned here so the container never needs # network access and every input is verified in exactly one place. download_pinned "$LOVE_SRC_URL" "$CACHE/$LOVE_SRC_TARBALL" "$LOVE_SRC_SHA256" +download_pinned "$SDL2_URL" "$CACHE/$SDL2_TARBALL" "$SDL2_SHA256" +download_pinned "$OPENAL_URL" "$CACHE/$OPENAL_TARBALL" "$OPENAL_SHA256" +download_pinned "$THEORA_URL" "$CACHE/$THEORA_TARBALL" "$THEORA_SHA256" +download_pinned "$OGG_URL" "$CACHE/$OGG_TARBALL" "$OGG_SHA256" +download_pinned "$VORBIS_URL" "$CACHE/$VORBIS_TARBALL" "$VORBIS_SHA256" +download_pinned "$MPG123_URL" "$CACHE/$MPG123_TARBALL" "$MPG123_SHA256" download_pinned "$APPIMAGE_RUNTIME_URL" "$CACHE/$APPIMAGE_RUNTIME_NAME" \ "$APPIMAGE_RUNTIME_SHA256" @@ -129,6 +135,18 @@ fi say "compiling and packaging inside $BUILDER_BASE_IMAGE" "$RUNTIME" run --rm ${user_args[@]+"${user_args[@]}"} \ -e LOVE_VERSION="$LOVE_VERSION" \ + -e SDL2_VERSION="$SDL2_VERSION" \ + -e SDL2_TARBALL="$SDL2_TARBALL" \ + -e OPENAL_VERSION="$OPENAL_VERSION" \ + -e OPENAL_TARBALL="$OPENAL_TARBALL" \ + -e THEORA_VERSION="$THEORA_VERSION" \ + -e THEORA_TARBALL="$THEORA_TARBALL" \ + -e OGG_VERSION="$OGG_VERSION" \ + -e OGG_TARBALL="$OGG_TARBALL" \ + -e VORBIS_VERSION="$VORBIS_VERSION" \ + -e VORBIS_TARBALL="$VORBIS_TARBALL" \ + -e MPG123_VERSION="$MPG123_VERSION" \ + -e MPG123_TARBALL="$MPG123_TARBALL" \ -e APP_NAME="$APP_NAME" \ -e VERSION="$VERSION" \ -v "$CACHE:/cache" \ diff --git a/scripts/linux-arm64/Dockerfile b/scripts/linux-arm64/Dockerfile index 65990884..ab12a3b7 100644 --- a/scripts/linux-arm64/Dockerfile +++ b/scripts/linux-arm64/Dockerfile @@ -14,16 +14,32 @@ ENV DEBIAN_FRONTEND=noninteractive # build-essential/autoconf: LÖVE 11.5's linux-src tarball is autotools. # squashfs-tools: packs the AppDir into the AppImage payload. -# The lib*-dev set is LÖVE's full optional-module surface — a missing one +# +# Note what is deliberately ABSENT: libsdl2-dev, libtheora-dev and +# libopenal-dev. All three are built from source instead (see common.sh for +# why), and having Debian's copies installed would let pkg-config hand LÖVE's +# configure the system ones and silently undo it. +# +# The remaining lib*-dev set is LÖVE's optional-module surface. A missing one # does not fail configure, it silently drops a module (love.sound decoders, -# love.font, love.video), so they are pinned here deliberately. +# love.font, love.video), so they are pinned here deliberately and asserted +# after the build. +# +# The X11/Wayland/audio -dev packages are here for SDL2's *build*, not for +# runtime linkage: SDL detects each backend at compile time and then dlopens +# it, so these headers decide which backends exist at all while adding no +# DT_NEEDED entry to the shipped library. RUN apt-get update -qq \ && apt-get install -y --no-install-recommends \ - build-essential pkg-config autoconf automake libtool \ - ca-certificates curl file xz-utils zip unzip squashfs-tools \ - libsdl2-dev libopenal-dev libogg-dev libvorbis-dev libtheora-dev \ + build-essential pkg-config autoconf automake libtool cmake \ + ca-certificates curl file xz-utils bzip2 zip unzip squashfs-tools \ + libogg-dev libvorbis-dev \ libmodplug-dev libmpg123-dev libfreetype6-dev libluajit-5.1-dev \ - zlib1g-dev libgl1-mesa-dev libgles2-mesa-dev \ + zlib1g-dev libgl1-mesa-dev libgles2-mesa-dev libegl1-mesa-dev \ + libasound2-dev libpulse-dev libudev-dev libdbus-1-dev \ + libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev \ + libxinerama-dev libxss-dev libxkbcommon-dev \ + libwayland-dev wayland-protocols libdrm-dev libgbm-dev \ && rm -rf /var/lib/apt/lists/* WORKDIR /work diff --git a/scripts/linux-arm64/build_appimage.sh b/scripts/linux-arm64/build_appimage.sh index 420d586c..53fc8fdc 100755 --- a/scripts/linux-arm64/build_appimage.sh +++ b/scripts/linux-arm64/build_appimage.sh @@ -15,6 +15,18 @@ set -euo pipefail LOVE_VERSION="${LOVE_VERSION:?}" +SDL2_VERSION="${SDL2_VERSION:?}" +SDL2_TARBALL="${SDL2_TARBALL:?}" +OPENAL_VERSION="${OPENAL_VERSION:?}" +OPENAL_TARBALL="${OPENAL_TARBALL:?}" +THEORA_VERSION="${THEORA_VERSION:?}" +THEORA_TARBALL="${THEORA_TARBALL:?}" +OGG_VERSION="${OGG_VERSION:?}" +OGG_TARBALL="${OGG_TARBALL:?}" +VORBIS_VERSION="${VORBIS_VERSION:?}" +VORBIS_TARBALL="${VORBIS_TARBALL:?}" +MPG123_VERSION="${MPG123_VERSION:?}" +MPG123_TARBALL="${MPG123_TARBALL:?}" APP_NAME="${APP_NAME:?}" VERSION="${VERSION:?}" JOBS="${JOBS:-$(nproc)}" @@ -29,16 +41,171 @@ fail() { printf '\033[1;31merror:\033[0m %s\n' "$*" >&2; exit 1; } mkdir -p "$WORK" +# --------------------------------------------------------------- prefix +# Everything we compile lands in one prefix, cached because the compiling is +# the only slow part (~5 min cold on a Pi 5) and is identical for every game +# version. The key includes every source version, so bumping any of them +# invalidates the cache instead of silently reusing a stale mix. +PREFIX="$CACHE/prefix-love$LOVE_VERSION-sdl$SDL2_VERSION-al$OPENAL_VERSION-theora$THEORA_VERSION-ogg$OGG_VERSION-vorbis$VORBIS_VERSION-mpg$MPG123_VERSION" +export PKG_CONFIG_PATH="$PREFIX/lib/pkgconfig" +# Our own libraries must win over the system ones during LÖVE's configure and +# link, or the whole point of building them is lost. +export LD_LIBRARY_PATH="$PREFIX/lib" + +# ------------------------------------------------------ compile audio codecs +# Ordered by dependency: vorbis needs ogg, and theora needs ogg too. All three +# are small, plain autotools builds -- well under a minute each. +build_autotools() { # $1 = label $2 = version $3 = tarball $4 = probe lib $5.. = configure args + local label="$1" version="$2" tarball="$3" probe="$4"; shift 4 + if [ -f "$PREFIX/lib/$probe" ]; then + say "reusing cached $label $version" + return 0 + fi + say "compiling $label $version" + local src="$WORK/$label-src" + rm -rf "$src"; mkdir -p "$src" + case "$tarball" in + *.tar.bz2) tar -xjf "$CACHE/$tarball" -C "$src" --strip-components=1 ;; + *) tar -xzf "$CACHE/$tarball" -C "$src" --strip-components=1 ;; + esac + ( + cd "$src" + # Several of these tarballs predate aarch64's entry in config.guess; the + # distro's copies know about it, so refresh them or configure bails out + # with "cannot guess build type". + for helper in config.guess config.sub; do + [ -f "$helper" ] && cp "/usr/share/misc/$helper" . 2>/dev/null + done + ./configure --prefix="$PREFIX" --disable-static "$@" >/dev/null + make -j"$JOBS" >/dev/null + make install >/dev/null + ) +} + +build_autotools ogg "$OGG_VERSION" "$OGG_TARBALL" libogg.so.0 +build_autotools vorbis "$VORBIS_VERSION" "$VORBIS_TARBALL" libvorbis.so.0 +# mpg123's ports/ tree and the command-line player are irrelevant here; only +# libmpg123 gets linked, and --disable-modules keeps the output-backend +# plugins (and their dlopen of ALSA/pulse) out of the shipped library. +build_autotools mpg123 "$MPG123_VERSION" "$MPG123_TARBALL" libmpg123.so.0 \ + --disable-modules --with-audio=dummy --disable-lfs-alias + +# The symbol that was missing when this was bullseye's copy. Assert it, so a +# version bump that quietly regresses below the host's expectations fails the +# build instead of silently killing audio again. +objdump -T "$PREFIX/lib/libmpg123.so.0" | grep -q 'mpg123_info2' \ + || fail "bundled libmpg123 lacks mpg123_info2; the host's libsndfile will fail to relocate" + +# ------------------------------------------------------------ compile SDL2 +# --enable-*-shared (the defaults, made explicit so a future SDL release +# cannot flip them under us) is the entire reason this is built from source: +# each backend is dlopened at runtime rather than becoming a DT_NEEDED entry, +# so the AppImage starts on a host with only ALSA, or only Wayland, or only +# KMSDRM, instead of demanding all of them at once the way Debian's build does. +if [ -f "$PREFIX/lib/libSDL2-2.0.so.0" ]; then + say "reusing cached SDL2 $SDL2_VERSION" +else + say "compiling SDL2 $SDL2_VERSION (jobs: $JOBS)" + rm -rf "$WORK/sdl-src"; mkdir -p "$WORK/sdl-src" + tar -xzf "$CACHE/$SDL2_TARBALL" -C "$WORK/sdl-src" --strip-components=1 + ( + cd "$WORK/sdl-src" + ./configure --prefix="$PREFIX" --disable-static \ + --enable-alsa --enable-alsa-shared \ + --enable-pulseaudio --enable-pulseaudio-shared \ + --enable-video-x11 --enable-x11-shared \ + --enable-video-wayland --enable-wayland-shared \ + --enable-video-kmsdrm --enable-kmsdrm-shared \ + --enable-libudev --disable-sndio --disable-jack --disable-esd \ + --disable-arts --disable-nas --disable-oss >/dev/null + make -j"$JOBS" >/dev/null + make install >/dev/null + ) +fi + +# Prove the dlopen intent actually took. If SDL ever hard-links an audio or +# video backend again, the AppImage silently regains a startup dependency on +# the host having that exact stack -- which is the bug this replaced. +sdl_lib="$PREFIX/lib/libSDL2-2.0.so.0" +[ -f "$sdl_lib" ] || fail "SDL2 build produced no libSDL2-2.0.so.0" +for forbidden in libpulse libasound libX11 libwayland libdrm libgbm libsndio; do + if objdump -p "$sdl_lib" | grep -q "NEEDED.*$forbidden"; then + fail "SDL2 hard-links $forbidden; it must dlopen its backends (--enable-*-shared)" + fi +done + +# ---------------------------------------------------- compile openal-soft +# ALSOFT_DLOPEN keeps the ALSA and PulseAudio backends behind dlopen, and +# sndio is switched off outright -- Debian enables it, which is what chained +# libopenal -> libsndio -> libasound into a mandatory startup dependency. +if [ -f "$PREFIX/lib/libopenal.so.1" ]; then + say "reusing cached openal-soft $OPENAL_VERSION" +else + say "compiling openal-soft $OPENAL_VERSION (jobs: $JOBS)" + rm -rf "$WORK/openal-src"; mkdir -p "$WORK/openal-src" + tar -xzf "$CACHE/$OPENAL_TARBALL" -C "$WORK/openal-src" --strip-components=1 + ( + cd "$WORK/openal-src" + cmake -S . -B build \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="$PREFIX" \ + -DALSOFT_DLOPEN=ON \ + -DALSOFT_BACKEND_SNDIO=OFF \ + -DALSOFT_BACKEND_OSS=OFF \ + -DALSOFT_BACKEND_JACK=OFF \ + -DALSOFT_EXAMPLES=OFF \ + -DALSOFT_UTILS=OFF \ + -DALSOFT_TESTS=OFF \ + -DLIBTYPE=SHARED >/dev/null + cmake --build build -j"$JOBS" >/dev/null + cmake --install build >/dev/null + ) +fi + +openal_lib="$PREFIX/lib/libopenal.so.1" +[ -f "$openal_lib" ] || fail "openal-soft build produced no libopenal.so.1" +for forbidden in libsndio libasound libpulse libjack; do + if objdump -p "$openal_lib" | grep -q "NEEDED.*$forbidden"; then + fail "openal hard-links $forbidden; backends must stay behind dlopen" + fi +done + +# --------------------------------------------------- compile libtheora +# --disable-examples is what drops Debian's libcairo link (and with it libX11, +# libxcb, libfontconfig and libfreetype as startup dependencies). The encoder +# is dead weight for a player, but libtheoradec is what LÖVE actually links. +if [ -f "$PREFIX/lib/libtheoradec.so.1" ]; then + say "reusing cached libtheora $THEORA_VERSION" +else + say "compiling libtheora $THEORA_VERSION" + rm -rf "$WORK/theora-src"; mkdir -p "$WORK/theora-src" + tar -xjf "$CACHE/$THEORA_TARBALL" -C "$WORK/theora-src" --strip-components=1 + ( + cd "$WORK/theora-src" + # theora 1.1.1 predates the aarch64 config.guess, so refresh the autotools + # helper scripts or configure rejects the host outright. + for helper in config.guess config.sub; do + cp "/usr/share/misc/$helper" . 2>/dev/null || true + done + ./configure --prefix="$PREFIX" --disable-static \ + --disable-examples --disable-spec --disable-doc >/dev/null + make -j"$JOBS" >/dev/null + make install >/dev/null + ) +fi + +theora_lib="$PREFIX/lib/libtheoradec.so.1" +[ -f "$theora_lib" ] || fail "libtheora build produced no libtheoradec.so.1" +if objdump -p "$theora_lib" | grep -q "NEEDED.*libcairo"; then + fail "libtheoradec still links libcairo (--disable-examples stopped working)" +fi + # ------------------------------------------------------------ compile LÖVE -# The prefix is cached because this is the only slow step (~3 min on a Pi 5, -# and it is identical for every game version). Keyed by LÖVE version so a -# LOVE_VERSION bump cannot silently reuse the old build. -PREFIX="$CACHE/love-$LOVE_VERSION-prefix" if [ -x "$PREFIX/bin/love" ] && [ -f "$PREFIX/lib/liblove-$LOVE_VERSION.so" ]; then say "reusing cached LÖVE $LOVE_VERSION aarch64 build" else say "compiling LÖVE $LOVE_VERSION for aarch64 (jobs: $JOBS)" - rm -rf "$PREFIX" "$WORK/love-src" + rm -rf "$WORK/love-src" mkdir -p "$WORK/love-src" tar -xzf "$CACHE/love-$LOVE_VERSION-linux-src.tar.gz" \ -C "$WORK/love-src" --strip-components=1 @@ -46,8 +213,11 @@ else cd "$WORK/love-src" # No --disable-* flags on purpose: configure silently drops a love module # when its -dev package is absent, so the Dockerfile pins the full set and - # the assertions below prove each one actually linked. - ./configure --prefix="$PREFIX" --disable-static >/dev/null + # the assertions below prove each one actually linked. CPPFLAGS/LDFLAGS + # point at our prefix so the SDL2 and theora just built above win over + # anything the base image might still provide. + ./configure --prefix="$PREFIX" --disable-static \ + CPPFLAGS="-I$PREFIX/include" LDFLAGS="-L$PREFIX/lib" >/dev/null make -j"$JOBS" >/dev/null make install >/dev/null # Keep LÖVE's license inside the cached prefix: the unpacked source tree @@ -97,29 +267,23 @@ chmod +x "$APPDIR/bin/love" # 1. Driver/session coupled. A bundled libGL would bypass Mesa's V3D driver # on the Pi; a bundled libpulse/libdbus would fight the user's running # session. GL/EGL/gbm/drm, X11/xcb/wayland/xkbcommon, dbus, pulse, alsa, -# systemd/udev. +# systemd/udev. Note that after the source builds above, none of these are +# DT_NEEDED of anything we ship -- SDL2 and OpenAL dlopen them, so they are +# used when present and skipped when absent. # # 2. Loader coupled. glibc's pieces cannot be mixed with the host's ld.so at # all, and libstdc++/libgcc_s must be at least as new as the compiler -- # bullseye's gcc 10 is older than any supported host's, so the host copy # always satisfies us. # -# 3. Shared with the host's font stack -- the subtle one, and the reason -# this list is longer than LÖVE's own AppImage manifest. Bullseye's -# libtheoradec is (bizarrely, a Debian packaging artifact) linked against -# libcairo, so the HOST's cairo gets loaded into our process. Because the -# dynamic loader resolves one SONAME once per process, that host cairo -# then binds to whatever libfreetype.so.6 we bundled -- and a bullseye -# freetype 2.10.4 has no FT_Get_Transform, which cairo 1.18 needs: -# -# love -> liblove -> libtheoradec -> libcairo (host, new) -# `-> FT_Get_Transform -> libfreetype (ours, old) BOOM -# -# Bundling a newer freetype only moves the arms race. Excluding the whole -# font/compression stack instead makes it self-consistent: cairo, -# fontconfig and freetype all come from one host and agree with each -# other, while liblove -- compiled against 2.10.4 -- only ever asks for -# symbols every supported host already has. +# 3. The font/compression stack: freetype, fontconfig, libpng, brotli, zlib. +# These are shared with whatever the host's own graphics libraries have +# already loaded, and mixing vintages inside one process breaks the older +# copy. Bundling a bullseye freetype 2.10.4 is what made a host cairo fail +# to find FT_Get_Transform (added in 2.11) and killed the game at startup. +# Leaving the whole stack to the host keeps it self-consistent, and +# liblove -- compiled against 2.10.4 -- only ever asks for symbols every +# supported host already has. EXCLUDE_RE='^(ld-linux-aarch64\.so\.1|libc\.so\.6|libm\.so\.6|libdl\.so\.2|libpthread\.so\.0|librt\.so\.1|libresolv\.so\.2|libutil\.so\.1|libanl\.so\.1|libnsl\.so\.[0-9]+|libstdc\+\+\.so\.6|libgcc_s\.so\.1|lib(GL|GLX|GLdispatch|OpenGL|EGL|GLESv[12]|glapi|gbm|drm)\..*|libX[a-z0-9]*\..*|libxcb.*|libwayland-.*|libxkbcommon.*|libdbus-1\..*|libpulse.*|libasound\..*|libsndfile\..*|libFLAC\..*|libopus\..*|libsystemd\..*|libudev\..*|libselinux\..*|libcap\..*|libgcrypt\..*|libgpg-error\..*|liblzma\..*|libzstd\..*|liblz4\..*|libffi\..*|libexpat\..*|libbsd\..*|libmd\..*|libuuid\..*|libg(lib|object|module|thread)-2\..*|libfontconfig\..*|libfreetype\..*|libpng[0-9]*\..*|libbrotli.*|libz\.so\..*|libwrap\..*|libasyncns\..*|libtirpc\..*|lib(gssapi_krb5|krb5|k5crypto|com_err|krb5support|keyutils)\..*|libpcre.*)$' # soname -> absolute path, harvested from the full ldd closure of both roots. @@ -156,6 +320,33 @@ bundle_needed "$APPDIR/bin/love" bundle_needed "$APPDIR/lib/liblove-$LOVE_VERSION.so" say "bundled $(ls "$APPDIR/lib" | wc -l) libraries: $(ls "$APPDIR/lib" | tr '\n' ' ')" +# ------------------------------------------------- host dependency contract +# The portability promise, stated as an assertion instead of a paragraph in a +# README: these are the ONLY sonames the shipped objects may require from the +# host. Everything driver-, session- or audio-related has to be reached +# through dlopen, so the AppImage starts on a box with no PulseAudio, no X11 +# or no ALSA and simply uses whatever it does find. +# +# The original build failed exactly here and nobody noticed until CI ran on a +# headless runner: Debian's SDL2 hard-links libpulse/libasound/libX11/ +# libwayland, so the image only ever started on a full desktop. +HOST_ALLOWED_RE='^(ld-linux-aarch64\.so\.1|libc\.so\.6|libm\.so\.6|libdl\.so\.2|libpthread\.so\.0|librt\.so\.1|libstdc\+\+\.so\.6|libgcc_s\.so\.1|libatomic\.so\.1|libfreetype\.so\.6|libpng[0-9]*\.so\.[0-9]+|libz\.so\.1|libbrotli(dec|common)\.so\.1)$' + +unexpected="" +for object in "$APPDIR/bin/love" "$APPDIR"/lib/*.so*; do + while read -r soname; do + [ -n "$soname" ] || continue + # Satisfied from inside the AppDir, so not a host requirement at all. + if [ -n "${BUNDLED[$soname]:-}" ]; then continue; fi + if [[ "$soname" =~ $HOST_ALLOWED_RE ]]; then continue; fi + unexpected="$unexpected $(basename "$object") -> $soname"$'\n' + done < <(objdump -p "$object" | awk '/NEEDED/ {print $2}') +done +[ -z "$unexpected" ] || fail "$(printf '%s\n%s' \ + "these objects hard-require host libraries outside the allowed set (they must be dlopened, not linked):" \ + "$unexpected")" +say "host dependency contract holds (glibc, libstdc++ and the font stack only)" + # LÖVE loads jit.* (jit.status, the profiler) through LUA_PATH; without these # the modules are simply absent, so ship them the way upstream's image does. jit_share="$(ls -d /usr/share/luajit-* 2>/dev/null | head -1)" diff --git a/scripts/linux-arm64/common.sh b/scripts/linux-arm64/common.sh index 210c67c2..39e196b4 100755 --- a/scripts/linux-arm64/common.sh +++ b/scripts/linux-arm64/common.sh @@ -18,6 +18,78 @@ LOVE_SRC_TARBALL="love-$LOVE_VERSION-linux-src.tar.gz" LOVE_SRC_URL="https://github.com/love2d/love/releases/download/$LOVE_VERSION/$LOVE_SRC_TARBALL" LOVE_SRC_SHA256="066e0843f71aa9fd28b8eaf27d41abb74bfaef7556153ac2e3cf08eafc874c39" +# SDL2 is built from source rather than taken from bullseye, and this is a +# correctness requirement, not a version preference. Debian's libSDL2 lists +# libpulse, libasound, libX11 and libwayland-client as DT_NEEDED -- hard links +# resolved by the loader at startup -- so an AppImage bundling it refuses to +# launch unless the host has ALL FOUR installed. That is wrong for an artifact +# whose whole job is to run on arbitrary arm64 systems: an ALSA-only handheld +# or a minimal Wayland box would die before main(). Built from source, SDL +# defaults to dlopening every audio and video backend (--enable-*-shared), so +# it loads whichever the host actually has and degrades gracefully. +# The newer version is a bonus: 2.30 has a far better controller database and +# real KMSDRM support, both of which matter on Pi-class and handheld hardware. +SDL2_VERSION="2.30.12" +SDL2_TARBALL="SDL2-$SDL2_VERSION.tar.gz" +SDL2_URL="https://github.com/libsdl-org/SDL/releases/download/release-$SDL2_VERSION/$SDL2_TARBALL" +SDL2_SHA256="ac356ea55e8b9dd0b2d1fa27da40ef7e238267ccf9324704850d5d47375b48ea" + +# libtheora likewise. Debian's libtheoradec.so.1 is linked against libcairo -- +# a packaging artifact, since a video decoder has no business drawing vector +# graphics -- and cairo drags in libX11, libxcb, libfontconfig and libfreetype +# as hard dependencies. LOVE needs theora for love.video, so that link would +# put the entire X11 and font stack on the critical path at startup, and it is +# what caused the FT_Get_Transform crash this build hit on a trixie host. +# Upstream's tarball with --disable-examples produces a libtheoradec that +# needs only libogg. +THEORA_VERSION="1.1.1" +THEORA_TARBALL="libtheora-$THEORA_VERSION.tar.bz2" +THEORA_URL="https://downloads.xiph.org/releases/theora/$THEORA_TARBALL" +THEORA_SHA256="b6ae1ee2fa3d42ac489287d3ec34c5885730b1296f0801ae577a35193d3affbc" + +# OpenAL for the same reason as SDL2, one level down. Debian's libopenal is +# openal-soft built with the sndio backend enabled, so it hard-links +# libsndio, which itself hard-links libasound -- reintroducing exactly the +# mandatory-ALSA dependency the SDL2 source build exists to remove. Upstream +# openal-soft dlopens its backends, so building it here leaves the shipped +# library with no audio-stack dependency at all. +OPENAL_VERSION="1.23.1" +OPENAL_TARBALL="openal-soft-$OPENAL_VERSION.tar.gz" +OPENAL_URL="https://github.com/kcat/openal-soft/archive/refs/tags/$OPENAL_VERSION.tar.gz" +OPENAL_SHA256="dfddf3a1f61059853c625b7bb03de8433b455f2f79f89548cbcbd5edca3d4a4a" + +# The audio codecs are built from source for a third, different reason: SONAME +# collision with the host's audio stack. +# +# OpenAL dlopens ALSA, ALSA's config loads its PulseAudio hook plugin, and that +# plugin pulls the HOST's libsndfile into our process. libsndfile links +# libogg, libvorbis and libmpg123 -- the same three we bundle. The loader +# resolves a SONAME once per process, so the host's libsndfile binds to OUR +# copies, and a bullseye libmpg123 has no mpg123_info2 (added in 1.32): +# +# openal -> libasound -> libasound_module_conf_pulse -> libsndfile (host) +# `-> mpg123_info2 -> libmpg123 (ours, bullseye) +# +# which failed to relocate and left the game with no audio device at all. +# Not bundling them instead would make libogg/libvorbis/libmpg123 mandatory +# host packages; building them current means our copies satisfy the host's +# libsndfile rather than starving it. libvorbisfile ships in the vorbis +# tarball. +OGG_VERSION="1.3.5" +OGG_TARBALL="libogg-$OGG_VERSION.tar.gz" +OGG_URL="https://downloads.xiph.org/releases/ogg/$OGG_TARBALL" +OGG_SHA256="0eb4b4b9420a0f51db142ba3f9c64b333f826532dc0f48c6410ae51f4799b664" + +VORBIS_VERSION="1.3.7" +VORBIS_TARBALL="libvorbis-$VORBIS_VERSION.tar.gz" +VORBIS_URL="https://downloads.xiph.org/releases/vorbis/$VORBIS_TARBALL" +VORBIS_SHA256="0e982409a9c3fc82ee06e08205b1355e5c6aa4c36bca58146ef399621b0ce5ab" + +MPG123_VERSION="1.32.10" +MPG123_TARBALL="mpg123-$MPG123_VERSION.tar.bz2" +MPG123_URL="https://www.mpg123.de/download/$MPG123_TARBALL" +MPG123_SHA256="87b2c17fe0c979d3ef38eeceff6362b35b28ac8589fbf1854b5be75c9ab6557c" + # AppImage type-2 runtime: the ~900 KB static-pie ELF that gets prepended to # the squashfs payload. Pinned to a dated tag, never "continuous", so a # rebuild months from now produces the same bytes. diff --git a/scripts/linux-arm64/selftest_build_linux_arm64.sh b/scripts/linux-arm64/selftest_build_linux_arm64.sh index 1a25b177..81ac3008 100755 --- a/scripts/linux-arm64/selftest_build_linux_arm64.sh +++ b/scripts/linux-arm64/selftest_build_linux_arm64.sh @@ -43,7 +43,8 @@ printf '%s' "$guard_out" | grep -q 'aarch64 host' \ say "checking pinned inputs" # Pins must be real digests, and the AppImage runtime must come from a dated # tag: "continuous" is a moving target and would make rebuilds unreproducible. -for pin_name in LOVE_SRC_SHA256 APPIMAGE_RUNTIME_SHA256; do +for pin_name in LOVE_SRC_SHA256 SDL2_SHA256 OPENAL_SHA256 THEORA_SHA256 \ + OGG_SHA256 VORBIS_SHA256 MPG123_SHA256 APPIMAGE_RUNTIME_SHA256; do pin_value="${!pin_name}" printf '%s' "$pin_value" | grep -Eq '^[0-9a-f]{64}$' \ || fail "$pin_name is not a sha256 digest: $pin_value" @@ -113,6 +114,48 @@ for soname in libSDL2-2.0.so.0 libopenal.so.1 libfreetype.so.6 libmodplug.so.1 \ || fail "build_appimage.sh no longer asserts liblove links $soname" done +say "checking the dlopen guarantees" +# SDL2, OpenAL and libtheora are compiled from source for correctness, not for +# a newer version number: Debian's builds hard-link libpulse/libasound/libX11/ +# libwayland (SDL2), libsndio (OpenAL) and libcairo (libtheora), each of which +# turns an optional runtime capability into a mandatory startup dependency. +# If a future edit drops the source build and reaches for the -dev package +# again, the AppImage silently stops starting on lean systems. +for forbidden_pkg in libsdl2-dev libtheora-dev libopenal-dev; do + if grep -qE "^ +.*\b$forbidden_pkg\b" "$SCRIPT_DIR/Dockerfile"; then + fail "Dockerfile installs $forbidden_pkg; that library is built from source on purpose" + fi +done +grep -qF -- '--enable-alsa-shared' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "SDL2 is no longer configured to dlopen its audio backends" +grep -qF -- '--enable-x11-shared' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "SDL2 is no longer configured to dlopen its video backends" +grep -qF 'ALSOFT_DLOPEN=ON' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "openal-soft is no longer configured to dlopen its backends" +grep -qF -- '--disable-examples' "$SCRIPT_DIR/build_appimage.sh" \ + || fail "libtheora is no longer built with --disable-examples (it regains the libcairo link)" + +say "checking the host dependency contract" +# The shipped objects may require nothing from the host beyond glibc, +# libstdc++ and the font stack. Everything driver-, session- or audio-related +# has to be dlopened. This is the invariant a headless CI runner proved was +# broken the first time round. +HOST_ALLOWED_RE="$( + grep -m1 "^HOST_ALLOWED_RE=" "$SCRIPT_DIR/build_appimage.sh" \ + | sed "s/^HOST_ALLOWED_RE='//; s/'\$//" +)" +[ -n "$HOST_ALLOWED_RE" ] || fail "could not read HOST_ALLOWED_RE out of build_appimage.sh" +for soname in libpulse.so.0 libasound.so.2 libX11.so.6 libwayland-client.so.0 \ + libGL.so.1 libcairo.so.2 libsndio.so.7.0 libdbus-1.so.3; do + if [[ "$soname" =~ $HOST_ALLOWED_RE ]]; then + fail "$soname is allowed as a hard host dependency; it must be dlopened" + fi +done +for soname in libc.so.6 libstdc++.so.6 libfreetype.so.6 libz.so.1; do + [[ "$soname" =~ $HOST_ALLOWED_RE ]] \ + || fail "$soname must be allowed as a host dependency but the contract rejects it" +done + say "checking the shared game.love payload" temp_dir="$(mktemp -d "${TMPDIR:-/tmp}/gen1recomp-linux-arm64-selftest.XXXXXX")" trap 'rm -rf "$temp_dir"' EXIT