Merge pull request #1304 from ShaneMcGovernIE/agent/mod-storage-opaque-bytes

Add opaque byte storage to mod API

docs/modding.md

@@ -322,6 +322,25 @@ local keys, code, message = mod.storage:list(game, "history/quick")
 local deleted, code, message = mod.storage:delete(game, "history/quick/q0001")
 ```
 
+For independently generated binary data, use the opaque byte methods. They
+accept and return the exact Lua string of bytes, including NUL bytes and bytes
+that are not valid text:
+
+```lua
+local ok, code, message = mod.storage:writeBytes(
+  game, "cache/maps/pallet/terrain", encodedMesh)
+local encodedMesh, code, message = mod.storage:readBytes(
+  game, "cache/maps/pallet/terrain")
+```
+
+Opaque values are limited to 512 MiB per key. The engine stores them without
+decoding, compression, or an engine-defined file format, and never executes
+them. A consuming mod owns validation of its format, fingerprint, checksum,
+and compression metadata. Byte writes are staged and compared byte-for-byte
+before replacement, and reads can recover a valid backup after an interrupted
+write. Existing table values and opaque byte values use one shared logical key
+space; delete a key before changing its value from one type to the other.
+
 `context` returns `{ engineVersion, gameVersion, playthroughId }`. The engine
 version is compatibility metadata; physical launcher-slot and path identity stays
 private. A title-selected context may additionally contain `normalSavedAt`, the
@@ -330,19 +349,22 @@ progress or a slot/path handle.
 
 At the title screen only, `mod.storage:selected(game)` returns a bound storage
 facade for the launcher-selected existing playthrough, or `nil, code, message`.
-Resolving this facade is read-only: it never allocates an identity, adopts a
+Resolving this facade is non-allocating: it never allocates an identity, adopts a
 fresh New Game, or exposes a slot id/path. Its `context()`, `read(key)`,
-`write(key, value)`, `list(prefix)`, and `delete(key)` methods have the same
-data-only and transaction contract as `mod.storage`, but remain restricted to
-the calling mod's selected existing namespace. It is intended for title tools
-that need to browse or manage durable history before the first normal SAVE.
+`write(key, value)`, `readBytes(key)`, `writeBytes(key, bytes)`,
+`list(prefix)`, and `delete(key)` methods have the same scoped and
+transactional contract as `mod.storage`, but remain restricted to the calling
+mod's selected existing namespace. It is intended for title tools that need to
+browse or manage durable history before the first normal SAVE.
 
-Values must be tables containing serializable data only. Keys are conservative
-slash-separated segments (letters, digits, `_`, `-`); paths and filesystem
-handles are never exposed. Writes are staged and decode-verified, reads recover
-from a valid staged/backup generation, and methods return structured errors for
-normal data or I/O failures. The playthrough identity is allocated lazily on the
-first storage/checkpoint call, so an unused API changes no save bytes.
+Table values must contain serializable data only. Opaque values must be Lua
+strings. Keys are conservative slash-separated segments (letters, digits, `_`,
+`-`); paths and filesystem handles are never exposed. Table writes are staged
+and decode-verified; opaque writes are staged and byte-verified; reads recover
+from a valid staged/backup generation. Methods return structured errors for
+normal data, byte validation, and I/O failures. The playthrough identity is
+allocated lazily on the first storage/checkpoint call, so an unused API changes
+no save bytes.
 
 `mod.checkpoints` captures and reconstructs engine-owned semantic runtime state:
 

docs/rfcs/0003-playthrough-storage.md

@@ -26,8 +26,8 @@ wiki's Save Model. `mod.save` and `mod.options` keep their existing behavior.
 ## The exact API delta
 
 Backward-compatible, additive-only. `Loader:_api` binds a new `mod.storage`
-facade to the calling mod id. Mods receive logical keys and decoded values, never
-filesystem handles or physical paths.
+facade to the calling mod id. Mods receive logical keys and either decoded table
+values or exact opaque byte strings, never filesystem handles or physical paths.
 
 ### Lazy opaque playthrough identity
 
@@ -75,6 +75,25 @@ Returns a freshly decoded table, or `nil, code, message`. It tries main, staged,
 then backup data. A valid staged/backup value is returned and promoted
 best-effort; corrupt bytes are never executed.
 
+### `mod.storage:writeBytes(game, key, bytes)`
+
+Accepts a Lua string containing opaque bytes and returns `true`, or
+`false, code, message`. Empty strings are valid. Payloads are limited to 512
+MiB per key. The engine writes the supplied bytes exactly as received, without
+decoding, compression, checksums, or an engine-defined envelope. The consuming
+mod owns semantic validation of its format.
+
+Byte records use private `.bin`, `.bin.tmp`, and `.bin.bak` witnesses. A staged
+and replacement write is read back and compared byte-for-byte before it is
+committed. A failed write leaves the previous verified generation readable.
+Byte storage never passes its payload to the Lua serializer, loader, or module
+resolver.
+
+Table and byte records share one logical key namespace and a key has one type.
+Writing one type over the other returns `type_conflict`; callers must delete the
+key before changing its type. `mod.storage:selected(game)` exposes the same
+`readBytes` and `writeBytes` operations for the selected playthrough facade.
+
 ### `mod.storage:list(game[, prefix])`
 
 Returns sorted logical keys beneath a valid prefix, an exact key when the prefix
@@ -92,9 +111,9 @@ Physical records are scoped as:
 `persistence root / mod_storage / game version / playthrough id / mod id`
 
 Stable error codes are `not_in_playthrough`, `storage_unavailable`,
-`invalid_key`, `encode_failed`, `write_failed`, `verify_failed`, and
-`not_found`. Ordinary data and I/O failures are return values, not callback-
-terminating errors.
+`invalid_key`, `encode_failed`, `invalid_bytes`, `size_limit`, `type_conflict`,
+`type_mismatch`, `write_failed`, `verify_failed`, and `not_found`. Ordinary
+data and I/O failures are return values, not callback-terminating errors.
 
 The restricted serializer's recursive writer runs outside LuaJIT traces. A
 1,000-process GC stress regression found compiled recursion could intermittently
@@ -107,6 +126,8 @@ boundary.
 **Nothing.** No API is removed, no manifest field changes, and no storage path or
 playthrough id is created unless a mod invokes `mod.storage` or
 `mod.checkpoints`. Existing save bytes remain unchanged on the no-caller path.
+Existing files outside the scoped storage contract are not imported; a caller
+must rebuild them through `writeBytes`.
 
 ## Parity tests
 
@@ -115,8 +136,10 @@ playthrough id is created unless a mod invokes `mod.storage` or
 - **Engine identity:** lazy allocation, save/load preservation, stable legacy
   mapping, fresh-playthrough replacement, and version/slot isolation.
 - **Public Mod API:** two real API-2 entry chunks prove data-only roundtrip,
+  opaque byte roundtrip including NUL bytes, no execution, size/type rejection,
   deterministic listing, key rejection, mod/game/playthrough isolation,
-  corrupt-main recovery, failure retention, exact delete, and no-mod no-write.
+  corrupt-main recovery, failure retention, selected-playthrough access, exact
+  delete, and no-mod no-write.
 
 ## Deprecation etiquette