fix(mod-api): validate dataset roots lazily

This commit is contained in:
MaxTomahawk
2026-08-24 09:57:57 +02:00
parent ca700c44d7
commit 7e069df740
9 changed files with 442 additions and 80 deletions
+16 -6
View File
@@ -226,18 +226,28 @@ end
`view.content` exposes the same registry names, aliases, generation routing,
and data-only record shapes as `mod.content`, but only `get`, `has`, and
`each`. Returned records are detached copies and cannot mutate either dataset.
Every generated base record passes the selected generation's existing public
schema before it is returned; extractor metadata beside record maps stays out
of the registry id space. A malformed record makes `get` return nil, `has`
return false, and `each` return no rows, and invalidates that dataset view.
Records containing functions, userdata, threads, metatables, or cycles are not
exposed. Each open call receives an independent facade, so one mod cannot
replace another mod view method. Canonical boot shaping is included, such as
Gen 1 defaults and Yellow corrections, and Gold's Foresight matchup rows and
derived `held_items`.
The marker and every required generated module for the selected version are
rechecked on each open. Missing, partial, and stale imports return
`nil, "not_imported"`; malformed or resource-limit-breaking generated data
returns `nil, "invalid_cache"`. Generated modules are decoded with a bounded
literal-only grammar and are never executed. No raw ROM bytes or generated
source are exposed. `view.assets:path(relative)` and
`open` checks the completion marker, exact version-specific file inventory,
source/cache boundary, and file-size bounds without reading or decoding the
semantic modules. A root is read, bounded-decoded, normalized, and cached only
when a content operation first needs it. Each later view operation rechecks
readiness and the source bytes behind already cached roots; unchanged roots are
not decoded again. Missing, partial, and stale imports return
`nil, "not_imported"`. Malformed syntax, a resource-limit violation, or a
record that fails the public schema is discovered on first root access and
fails that operation closed; a later `open` against the same source returns
`nil, "invalid_cache"`. Generated modules use a bounded literal-only grammar
and are never executed. No raw ROM bytes or generated source are exposed.
`view.assets:path(relative)` and
`view.assets:info(relative)` accept only `assets/generated/...` paths and
keep them under the selected version cache prefix. The API never changes
`mod.game`, the active `Data` table, `GameVersion`, or cache mount state.
+48
View File
@@ -510,6 +510,26 @@ mod.content.item_effects:register("MOON_FLUTE", { use = fn, field = true })
mod.content.items:patch("POTION", { price = 100 })
```
### On Gold (Gen 2)
- semantics: `record`
- target: `Data.items`
The record differs; the registry name, the verbs and the id space
do not.
| field | type | required |
|---|---|---|
| `ball` | balls id | no |
| `effect` | item_effects id | no |
| `id` | string | yes |
| `index` | integer 0..255 | no |
| `machine` | {kind, move, number} | no |
| `name` | string | yes |
| `needsTarget` | boolean | no |
| `price` | integer >= 0 | yes |
| `tossable` | boolean | no |
## landmarks
- semantics: `record`
@@ -674,6 +694,34 @@ mod.content.move_effects:register("DRAIN_PP_EFFECT", { kind = "primary", run = f
mod.content.moves:patch("BLIZZARD", { accuracy = 70 })
```
### On Gold (Gen 2)
- semantics: `record`
- target: `Data.moves`
The record differs; the registry name, the verbs and the id space
do not.
| field | type | required |
|---|---|---|
| `accuracy` | integer 0..100 | yes |
| `anim` | any value | no |
| `category` | one of "physical" | "special" | "status" | no |
| `chargeText` | string | no |
| `counterable` | boolean | no |
| `effect` | move_effects id | yes |
| `fixedDamage` | integer >= 1 | function | no |
| `highCrit` | boolean | no |
| `id` | string | yes |
| `index` | integer 0..255 | no |
| `multiHit` | integer >= 1 | list of integer >= 1 | no |
| `name` | string | yes |
| `power` | integer 0..255 | yes |
| `pp` | integer 0..64 | yes |
| `priority` | integer -7..7 | no |
| `semiInvulnerable` | boolean | no |
| `type` | type_chart id | yes |
## music
- semantics: `record`
+39 -16
View File
@@ -70,9 +70,13 @@ semantic presence.
the selected version's generation routing and the engine's existing
`Schemas`, `Registry`, and `Builtins` normalization, so structured sources
such as type matchups retain the same public ids used by the active
`mod.content` facade. No register, patch, override, or remove verb is exposed. Each call returns an
independent facade over the cached internal dataset, so facade mutation cannot cross
mod boundaries.
`mod.content` facade and extractor metadata beside record maps is not exposed
as a record id. Every generated base record is checked with that selected
generation's existing public schema before it can cross `get`, `has`, or
`each`; there is no dataset-specific duplicate schema. No register, patch,
override, or remove verb is exposed. Each call returns an independent facade
over the cached internal dataset, so facade mutation cannot cross mod
boundaries.
`assets:path` returns the selected cache-prefixed virtual path.
`assets:info` returns sanitized `type` and optional `size` metadata. Both
@@ -80,16 +84,29 @@ accept only relative paths below `assets/generated/`, reject control
characters, absolute paths, backslashes, and traversal, and expose no byte
reader.
An unknown version returns `nil, "unknown_version"`. A missing, partial, or
stale cache returns `nil, "not_imported"`. A required module that is malformed
or exceeds the limits (8 MiB per module, 48 MiB aggregate, depth 64, 500,000
values, 2 MiB per string, or 250,000 entries per table) returns
`nil, "invalid_cache"`; actionable detail is engine-logged but not exposed to
the mod. Generated Lua is decoded with the existing restricted
literal grammar and never executed. Functions, userdata, threads, metatables,
cycles, non-table roots, binary chunks, and trailing syntax cannot cross the
facade. Successful roots are decoded lazily and cached per selected version.
The API never exposes raw ROM bytes, generated source, host paths, or a mount.
An unknown version returns `nil, "unknown_version"`. `open` checks the current
completion marker, exact version-specific file inventory, source/cache
boundary, and available file sizes; it does not read or decode semantic
modules. A missing, partial, or stale cache returns `nil, "not_imported"`.
The first content operation that needs a root reads it once, applies the
limits (8 MiB per module, 48 MiB aggregate, depth 64, 500,000 values, 2 MiB
per string, and 250,000 entries per table), decodes the restricted literal
grammar, applies canonical selected-version normalization, and caches the
detached root. Each later content or asset operation rechecks marker/file and
source-bound readiness. It also rereads the source bytes for already cached
roots; unchanged roots are not decoded again, while a changed root clears the
derived registry cache and is decoded on its next use.
Malformed syntax, non-table roots, binary/trailing content, resource-limit
violations, and records that fail the public schema are therefore discovered
on first access rather than during `open`. The triggering `get` returns nil,
`has` returns false, or `each` returns no rows; the whole internal view is
invalidated, and a subsequent `open` against the unchanged source returns
`nil, "invalid_cache"`. Actionable detail is engine-logged but not exposed to
the mod. Generated Lua is never executed. Functions, userdata, threads,
metatables, and cycles cannot cross the facade. The API never exposes raw ROM
bytes, generated source, host paths, or a mount.
## Migration and compatibility
@@ -106,7 +123,9 @@ The completion marker and per-version required-file rules live in the pure,
injected `CacheContract` shared by the importer and dataset service. It also
defines source-tree behavior. Neither consumer mutates `CacheFs.prefix` while
checking readiness. Every `open` revalidates the contract and required module
shapes; a stale/remove/reimport transition evicts the previous semantic view.
inventory without semantic decoding. Every view operation rechecks that
readiness before serving cached state; a stale/remove/reimport transition
evicts the previous semantic view.
## Verification
@@ -117,8 +136,12 @@ shapes; a stale/remove/reimport transition evicts the previous semantic view.
version-prefixed generated assets,
traversal rejection, stable failure reasons, and stale-marker rejection.
- It also proves missing/empty/partial/stale/remove/reimport behavior, hostile
generated-source rejection, canonical Gen 1/Yellow/Gold hydration, and the
approved Kanto+ Gold records and assets.
generated-source and malformed-record rejection, canonical Gen
1/Yellow/Gold hydration, active Red/Blue/Yellow isolation while reading Gold,
and the approved Kanto+ Gold records and assets.
- `tests/engine/dataset_views_lazy_validation.lua` counts semantic reads and
decoder calls to prove `open` decodes nothing, unused roots stay unread, and
repeated access does not re-decode an unchanged cached root.
- `tests/modkit/cases/dataset_views_nontermination.lua` proves generated code
is rejected rather than executed; `tests/engine/generated_data_decoder_test.lua`
proves every decoder resource bound.