Add opaque byte storage to mod API

This commit is contained in:
Shane McGovern
2026-08-14 21:42:45 +01:00
parent 797a6bebfe
commit 9fab992d42
6 changed files with 321 additions and 31 deletions
+33 -11
View File
@@ -308,6 +308,25 @@ local keys, code, message = mod.storage:list(game, "history/quick")
local deleted, code, message = mod.storage:delete(game, "history/quick/q0001")
```
For independently generated binary data, use the opaque byte methods. They
accept and return the exact Lua string of bytes, including NUL bytes and bytes
that are not valid text:
```lua
local ok, code, message = mod.storage:writeBytes(
game, "cache/maps/pallet/terrain", encodedMesh)
local encodedMesh, code, message = mod.storage:readBytes(
game, "cache/maps/pallet/terrain")
```
Opaque values are limited to 512 MiB per key. The engine stores them without
decoding, compression, or an engine-defined file format, and never executes
them. A consuming mod owns validation of its format, fingerprint, checksum,
and compression metadata. Byte writes are staged and compared byte-for-byte
before replacement, and reads can recover a valid backup after an interrupted
write. Existing table values and opaque byte values use one shared logical key
space; delete a key before changing its value from one type to the other.
`context` returns `{ engineVersion, gameVersion, playthroughId }`. The engine
version is compatibility metadata; physical launcher-slot and path identity stays
private. A title-selected context may additionally contain `normalSavedAt`, the
@@ -316,19 +335,22 @@ progress or a slot/path handle.
At the title screen only, `mod.storage:selected(game)` returns a bound storage
facade for the launcher-selected existing playthrough, or `nil, code, message`.
Resolving this facade is read-only: it never allocates an identity, adopts a
Resolving this facade is non-allocating: it never allocates an identity, adopts a
fresh New Game, or exposes a slot id/path. Its `context()`, `read(key)`,
`write(key, value)`, `list(prefix)`, and `delete(key)` methods have the same
data-only and transaction contract as `mod.storage`, but remain restricted to
the calling mod's selected existing namespace. It is intended for title tools
that need to browse or manage durable history before the first normal SAVE.
`write(key, value)`, `readBytes(key)`, `writeBytes(key, bytes)`,
`list(prefix)`, and `delete(key)` methods have the same scoped and
transactional contract as `mod.storage`, but remain restricted to the calling
mod's selected existing namespace. It is intended for title tools that need to
browse or manage durable history before the first normal SAVE.
Values must be tables containing serializable data only. Keys are conservative
slash-separated segments (letters, digits, `_`, `-`); paths and filesystem
handles are never exposed. Writes are staged and decode-verified, reads recover
from a valid staged/backup generation, and methods return structured errors for
normal data or I/O failures. The playthrough identity is allocated lazily on the
first storage/checkpoint call, so an unused API changes no save bytes.
Table values must contain serializable data only. Opaque values must be Lua
strings. Keys are conservative slash-separated segments (letters, digits, `_`,
`-`); paths and filesystem handles are never exposed. Table writes are staged
and decode-verified; opaque writes are staged and byte-verified; reads recover
from a valid staged/backup generation. Methods return structured errors for
normal data, byte validation, and I/O failures. The playthrough identity is
allocated lazily on the first storage/checkpoint call, so an unused API changes
no save bytes.
`mod.checkpoints` captures and reconstructs engine-owned semantic runtime state:
+29 -6
View File
@@ -26,8 +26,8 @@ wiki's Save Model. `mod.save` and `mod.options` keep their existing behavior.
## The exact API delta
Backward-compatible, additive-only. `Loader:_api` binds a new `mod.storage`
facade to the calling mod id. Mods receive logical keys and decoded values, never
filesystem handles or physical paths.
facade to the calling mod id. Mods receive logical keys and either decoded table
values or exact opaque byte strings, never filesystem handles or physical paths.
### Lazy opaque playthrough identity
@@ -75,6 +75,25 @@ Returns a freshly decoded table, or `nil, code, message`. It tries main, staged,
then backup data. A valid staged/backup value is returned and promoted
best-effort; corrupt bytes are never executed.
### `mod.storage:writeBytes(game, key, bytes)`
Accepts a Lua string containing opaque bytes and returns `true`, or
`false, code, message`. Empty strings are valid. Payloads are limited to 512
MiB per key. The engine writes the supplied bytes exactly as received, without
decoding, compression, checksums, or an engine-defined envelope. The consuming
mod owns semantic validation of its format.
Byte records use private `.bin`, `.bin.tmp`, and `.bin.bak` witnesses. A staged
and replacement write is read back and compared byte-for-byte before it is
committed. A failed write leaves the previous verified generation readable.
Byte storage never passes its payload to the Lua serializer, loader, or module
resolver.
Table and byte records share one logical key namespace and a key has one type.
Writing one type over the other returns `type_conflict`; callers must delete the
key before changing its type. `mod.storage:selected(game)` exposes the same
`readBytes` and `writeBytes` operations for the selected playthrough facade.
### `mod.storage:list(game[, prefix])`
Returns sorted logical keys beneath a valid prefix, an exact key when the prefix
@@ -92,9 +111,9 @@ Physical records are scoped as:
`persistence root / mod_storage / game version / playthrough id / mod id`
Stable error codes are `not_in_playthrough`, `storage_unavailable`,
`invalid_key`, `encode_failed`, `write_failed`, `verify_failed`, and
`not_found`. Ordinary data and I/O failures are return values, not callback-
terminating errors.
`invalid_key`, `encode_failed`, `invalid_bytes`, `size_limit`, `type_conflict`,
`type_mismatch`, `write_failed`, `verify_failed`, and `not_found`. Ordinary
data and I/O failures are return values, not callback-terminating errors.
The restricted serializer's recursive writer runs outside LuaJIT traces. A
1,000-process GC stress regression found compiled recursion could intermittently
@@ -107,6 +126,8 @@ boundary.
**Nothing.** No API is removed, no manifest field changes, and no storage path or
playthrough id is created unless a mod invokes `mod.storage` or
`mod.checkpoints`. Existing save bytes remain unchanged on the no-caller path.
Existing files outside the scoped storage contract are not imported; a caller
must rebuild them through `writeBytes`.
## Parity tests
@@ -115,8 +136,10 @@ playthrough id is created unless a mod invokes `mod.storage` or
- **Engine identity:** lazy allocation, save/load preservation, stable legacy
mapping, fresh-playthrough replacement, and version/slot isolation.
- **Public Mod API:** two real API-2 entry chunks prove data-only roundtrip,
opaque byte roundtrip including NUL bytes, no execution, size/type rejection,
deterministic listing, key rejection, mod/game/playthrough isolation,
corrupt-main recovery, failure retention, exact delete, and no-mod no-write.
corrupt-main recovery, failure retention, selected-playthrough access, exact
delete, and no-mod no-write.
## Deprecation etiquette