Merge pull request #1767 from MaxTomahawk/adaptive-trainers/dataset-view-api

feat(mod-api): expose imported dataset views

docs/modding.md

@@ -200,6 +200,60 @@ the adapter's own coverage table:
 python3 tools/modkit.py gen2check mods/my_mod
 ```
 
+## Imported version datasets
+
+A mod can inspect semantic content from another game the player has already
+imported without switching the active game or reaching into engine cache
+internals:
+
+```lua
+local gold, reason = mod.datasets:open("gold")
+if not gold then
+  -- reason is "unknown_version", "not_imported", or "invalid_cache"
+  return
+end
+
+local chikorita = gold.content.pokemon:get("CHIKORITA")
+local normalVsGhost = gold.content.type_chart:get("NORMAL>GHOST")
+local spritePath = gold.assets:path(chikorita.spriteFront)
+
+for id, record in gold.content.pokemon:each() do
+  -- ids are returned in deterministic lexical order
+end
+```
+
+`view.version` and `view.generation` identify the selected dataset.
+`view.content` exposes the same registry names, aliases, generation routing,
+and data-only record shapes as `mod.content`, but only `get`, `has`, and
+`each`. Returned records are detached copies and cannot mutate either dataset.
+Every generated base record passes the selected generation's existing public
+schema before it is returned; extractor metadata beside record maps stays out
+of the registry id space and is reserved against `register`, `override`,
+`patch`, and `remove` writes through the active registry. A malformed record
+makes `get` return nil, `has`
+return false, and `each` return no rows, and invalidates that dataset view.
+Records containing functions, userdata, threads, metatables, or cycles are not
+exposed. Each open call receives an independent facade, so one mod cannot
+replace another mod view method. Canonical boot shaping is included, such as
+Gen 1 defaults and Yellow corrections, and Gold's Foresight matchup rows and
+derived `held_items`.
+
+`open` checks the completion marker, exact version-specific file inventory,
+source/cache boundary, and file-size bounds without reading or decoding the
+semantic modules. A root is read, bounded-decoded, normalized, and cached only
+when a content operation first needs it. Each later view operation rechecks
+readiness and the source bytes behind already cached roots; unchanged roots are
+not decoded again. Missing, partial, and stale imports return
+`nil, "not_imported"`. Malformed syntax, a resource-limit violation, or a
+record that fails the public schema is discovered on first root access and
+fails that operation closed; a later `open` against the same source returns
+`nil, "invalid_cache"`. Generated modules use a bounded literal-only grammar
+and are never executed. No raw ROM bytes or generated source are exposed.
+`view.assets:path(relative)` and
+`view.assets:info(relative)` accept only `assets/generated/...` paths and
+keep them under the selected version cache prefix. The API never changes
+`mod.game`, the active `Data` table, `GameVersion`, or cache mount state.
+
 ## Editing maps in Tiled
 
 Maps are data, not assets, so they can be authored in a real map editor and

docs/modding/reference/registries.md

@@ -342,6 +342,7 @@ accepted and merged as-is.
 | `source` | string |
 | `swarmGrass` | map of string -> {map?, rates, slots} |
 | `swarmWater` | map of string -> {map?, rate, slots} |
+| `timeFishGroups` | map of string | integer 0..255 -> {day, nite} |
 | `treeSets` | map of string -> {common, rare} |
 | `trees` | map of string -> string |
 | `water` | map of string -> {map?, rate, slots} |
@@ -509,6 +510,26 @@ mod.content.item_effects:register("MOON_FLUTE", { use = fn, field = true })
 mod.content.items:patch("POTION", { price = 100 })
 ```
 
+### On Gold (Gen 2)
+
+- semantics: `record`
+- target: `Data.items`
+
+The record differs; the registry name, the verbs and the id space
+do not.
+
+| field | type | required |
+|---|---|---|
+| `ball` | balls id | no |
+| `effect` | item_effects id | no |
+| `id` | string | yes |
+| `index` | integer 0..255 | no |
+| `machine` | {kind, move, number} | no |
+| `name` | string | yes |
+| `needsTarget` | boolean | no |
+| `price` | integer >= 0 | yes |
+| `tossable` | boolean | no |
+
 ## landmarks
 
 - semantics: `record`
@@ -598,7 +619,7 @@ mod.content.map_songs:override("PALLET_TOWN", "Music_Routes1")
 | `id` | string | yes |
 | `index` | integer >= 0 | no |
 | `label` | string | no |
-| `objects` | list of any value | no |
+| `objects` | list of {pokemon?, ...} | no |
 | `palette` | string | no |
 | `signs` | list of any value | no |
 | `tileset` | tilesets id | yes |
@@ -673,6 +694,34 @@ mod.content.move_effects:register("DRAIN_PP_EFFECT", { kind = "primary", run = f
 mod.content.moves:patch("BLIZZARD", { accuracy = 70 })
 ```
 
+### On Gold (Gen 2)
+
+- semantics: `record`
+- target: `Data.moves`
+
+The record differs; the registry name, the verbs and the id space
+do not.
+
+| field | type | required |
+|---|---|---|
+| `accuracy` | integer 0..100 | yes |
+| `anim` | any value | no |
+| `category` | one of "physical" | "special" | "status" | no |
+| `chargeText` | string | no |
+| `counterable` | boolean | no |
+| `effect` | move_effects id | yes |
+| `fixedDamage` | integer >= 1 | function | no |
+| `highCrit` | boolean | no |
+| `id` | string | yes |
+| `index` | integer 0..255 | no |
+| `multiHit` | integer >= 1 | list of integer >= 1 | no |
+| `name` | string | yes |
+| `power` | integer 0..255 | yes |
+| `pp` | integer 0..64 | yes |
+| `priority` | integer -7..7 | no |
+| `semiInvulnerable` | boolean | no |
+| `type` | type_chart id | yes |
+
 ## music
 
 - semantics: `record`

docs/rfcs/0015-imported-dataset-views.md

@@ -0,0 +1,154 @@
+# RFC 0015: Read-only imported dataset views
+
+## Status
+
+Proposed.
+
+## Motivation
+
+A cross-version content mod can read only the active merged dataset through
+`mod.content`. Even when the player has already imported another supported
+game, a mod cannot inspect that version's semantic species, moves, items, type
+chart, or generated asset namespace. The available alternatives are private:
+mutating `CacheFs.prefix`, mounting another cache over the active one, loading
+generated Lua directly, or asking for a second raw-ROM import. They compose
+poorly with the active game, expose unstable import layout, and make safe
+read-only use impossible from the sandbox.
+
+The concrete consumer is **Adaptive Trainers**, whose approved Phase G Kanto+
+sidecar must derive nine Kanto-line continuations, Steel/type and move
+definitions, and generated sprites from the player's existing verified Gold
+cache while Red, Blue, or Yellow remains active. `mod.content` exposes only the
+active R/B/Y dataset; `mod.imports` can read only separately declared raw mod
+imports; and `mod.cache` is mod-private generated output. None can inspect the
+launcher-owned Gold semantic dataset without a second ROM import and mod-side
+ROM interpretation. The engine API remains generic and contains no Adaptive
+Trainers policy.
+
+## Decision and plan extended
+
+This implements **D-AT-004: optional Kanto+ content consumes an
+active-independent semantic dataset view**. The consuming design is tracked in
+the Adaptive Trainers implementation plan,
+[`docs/superpowers/plans/2026-08-14-adaptive-trainers.md`](https://github.com/MaxTomahawk/gen1recomp-adaptive-trainers/blob/main/docs/superpowers/plans/2026-08-14-adaptive-trainers.md),
+Task 8. The delta is a generic, additive public API and contains no trainer
+pools, scaling, boss identities, version-mixing rules, or Adaptive Trainers
+policy.
+
+## Exact API delta
+
+Every sandboxed mod receives:
+
+```lua
+local view, reason = mod.datasets:open("gold")
+```
+
+A known version with the current completed import marker returns a read-only view:
+
+```lua
+view = {
+  version = "gold",
+  generation = 2,
+  content = {
+    pokemon = {
+      get = function(self, id) end,
+      has = function(self, id) end,
+      each = function(self) end,
+    },
+    -- every public registry name and alias
+  },
+  assets = {
+    path = function(self, generatedPath) end,
+    info = function(self, generatedPath) end,
+  },
+}
+```
+
+`get` returns a bounded detached data-only copy or nil. `has` reports data-only
+semantic presence.
+`each` returns ids in lexical order and detached values. The registries use
+the selected version's generation routing and the engine's existing
+`Schemas`, `Registry`, and `Builtins` normalization, so structured sources
+such as type matchups retain the same public ids used by the active
+`mod.content` facade and extractor metadata beside record maps is not exposed
+as a record id or writable through the active registry's mutation verbs. Every
+generated base record is checked with that selected generation's existing
+public schema before it can cross `get`, `has`, or
+`each`; there is no dataset-specific duplicate schema. No register, patch,
+override, or remove verb is exposed. Each call returns an independent facade
+over the cached internal dataset, so facade mutation cannot cross mod
+boundaries.
+
+`assets:path` returns the selected cache-prefixed virtual path.
+`assets:info` returns sanitized `type` and optional `size` metadata. Both
+accept only relative paths below `assets/generated/`, reject control
+characters, absolute paths, backslashes, and traversal, and expose no byte
+reader.
+
+An unknown version returns `nil, "unknown_version"`. `open` checks the current
+completion marker, exact version-specific file inventory, source/cache
+boundary, and available file sizes; it does not read or decode semantic
+modules. A missing, partial, or stale cache returns `nil, "not_imported"`.
+
+The first content operation that needs a root reads it once, applies the
+limits (8 MiB per module, 48 MiB aggregate, depth 64, 500,000 values, 2 MiB
+per string, and 250,000 entries per table), decodes the restricted literal
+grammar, applies canonical selected-version normalization, and caches the
+detached root. Each later content or asset operation rechecks marker/file and
+source-bound readiness. It also rereads the source bytes for already cached
+roots; unchanged roots are not decoded again, while a changed root clears the
+derived registry cache and is decoded on its next use.
+
+Malformed syntax, non-table roots, binary/trailing content, resource-limit
+violations, and records that fail the public schema are therefore discovered
+on first access rather than during `open`. The triggering `get` returns nil,
+`has` returns false, or `each` returns no rows; the whole internal view is
+invalidated, and a subsequent `open` against the unchanged source returns
+`nil, "invalid_cache"`. Actionable detail is engine-logged but not exposed to
+the mod. Generated Lua is never executed. Functions, userdata, threads,
+metatables, and cycles cannot cross the facade. The API never exposes raw ROM
+bytes, generated source, host paths, or a mount.
+
+## Migration and compatibility
+
+Existing mods change nothing. `mod.datasets` is additive and requires no
+permission. The service is allocated lazily on the first explicit
+`mod.datasets:open` call. A boot with no mods, or with mods that do not call
+it, performs no cross-version cache reads.
+
+Opening a view does not change `GameVersion`, `CacheFs.prefix`, the active
+`Data` table, PhysFS mounts, save state, or the selected game's behavior.
+Red, Blue, Yellow, Gold, Silver, and Crystal keep their existing active data paths.
+
+The completion marker and per-version required-file rules live in the pure,
+injected `CacheContract` shared by the importer and dataset service. It also
+defines source-tree behavior. Neither consumer mutates `CacheFs.prefix` while
+checking readiness. Every `open` revalidates the contract and required module
+inventory without semantic decoding. Every view operation rechecks that
+readiness before serving cached state; a stale/remove/reimport transition
+evicts the previous semantic view.
+
+## Verification
+
+- `tests/modkit/cases/dataset_views.lua` loads sandboxed fixture mods through
+  the public API and covers Red, Blue, Yellow, Gold, Silver, and Crystal independently.
+- The test proves semantic registry normalization, deterministic iteration,
+  detached records, read-only facades, cross-mod facade isolation,
+  version-prefixed generated assets,
+  traversal rejection, stable failure reasons, and stale-marker rejection.
+- It also proves missing/empty/partial/stale/remove/reimport behavior, hostile
+  generated-source and malformed-record rejection, canonical Gen
+  1/Yellow/Gold hydration, active Red/Blue/Yellow isolation while reading Gold,
+  and the approved Kanto+ Gold records and assets.
+- `tests/engine/dataset_views_lazy_validation.lua` counts semantic reads and
+  decoder calls to prove `open` decodes nothing, unused roots stay unread, and
+  repeated access does not re-decode an unchanged cached root.
+- `tests/modkit/cases/dataset_views_nontermination.lua` proves generated code
+  is rejected rather than executed; `tests/engine/generated_data_decoder_test.lua`
+  proves every decoder resource bound.
+- `tests/engine/dataset_views_no_mod_parity.lua` is the separate guarded no-mod
+  parity suite and proves the service stays unallocated with zero cache reads.
+
+## Deprecation etiquette
+
+Nothing is removed, renamed, superseded, or deprecated.