Merge upstream dev into dataset view API

docs/rfcs/0013-map-occupancy-and-active-block.md

@@ -0,0 +1,142 @@
+# RFC 0013: Conditional map occupancy and active-block reads
+
+## Status
+
+Proposed.
+
+## Motivation
+
+The Gen 1 Vermilion Dock script correctly ejects a player who enters after the
+S.S. Anne has departed. A content mod can add a city-side route back to that
+empty harbor, but it cannot preserve the dock visit: replacing the complete
+dock handler would discard vanilla behavior and peer handlers, while an added
+handler cannot cancel the base handler's ejection.
+
+A mod that changes one active map block also needs to prove that it is looking
+at the expected Red, Blue, or Yellow layout before it acts. `mapOverview()` is
+intentionally presentation-oriented and does not expose block identity.
+Requiring internal `Map` state or generated ROM data would cross the public mod
+boundary and make a wrong-version edit difficult to fail closed.
+
+## Decision and plan extended
+
+This extends Route B in `CONTRIBUTING-mods.md`: new behavior is additive,
+ordinary hook composition remains the authority, an empty hook chain is a
+provable no-op, and mods receive copied or scalar data rather than mutable
+engine state. It supports the approved Mew-under-the-truck implementation plan
+without adding any Mew-specific rule, asset, flag, or content to the engine.
+
+## Exact API delta
+
+### `map.occupancy_allowed`
+
+The post-departure `VERMILION_DOCK` script calls this hook after it replaces the
+ship blocks with water and immediately before it would display the departure
+message and warp the player to Vermilion City.
+
+A wrapper has this shape:
+
+```lua
+function(next, game, context) -> boolean
+```
+
+The context is a new table with these fields:
+
+| Field | Meaning |
+|---|---|
+| `mapId` | `"VERMILION_DOCK"` at this call site |
+| `reason` | Stable reason key `"ss_anne_departed"` |
+| `gameVersion` | Active save version (`red`, `blue`, or `yellow`) when present |
+| `x`, `y` | Current player cell coordinates when present |
+
+Vanilla returns `false`. The player remains only when the final chain result is
+exactly `true`. Absent, throwing, or malformed wrappers therefore preserve
+ejection. A wrapper composes by calling `next(game, context)` and returning
+true when either downstream or its own narrow rule permits occupancy. The hook
+does not replace `MapScripts` registration, merging, or dispatch, and does not
+change the departure flag or reconstruct the ship.
+
+As with every wrapper hook, a callback that does not call `next` intentionally
+owns the final answer and does not run lower-priority callbacks. Permission
+wrappers must call downstream to compose. A false, non-forwarding wrapper
+safely denies occupancy and can suppress downstream permission by this normal
+rule. A malformed final result also fails closed and cannot permit occupancy.
+
+The call is guarded by `Runtime.wantsHook`, so an empty chain allocates no
+context and follows the prior branch exactly.
+
+### `WorldAPI:activeBlockAt`
+
+Gen 1's public `mod.world` facade adds:
+
+```lua
+activeBlockAt(mapId, blockX, blockY) -> blockId
+                                      | nil, reason
+```
+
+`mapId` must equal the active map ID. Coordinates are finite, integral,
+zero-based block coordinates. A successful result is a numeric scalar copied
+from the active runtime map. The method never returns the map's mutable block
+array and never writes game or save state.
+
+Failure reasons are stable:
+
+| Condition | Reason |
+|---|---|
+| No active overworld map | `no overworld` |
+| `mapId` differs from the active map | `map is not active` |
+| Coordinate has the wrong type, is non-finite, or is fractional | `invalid block coordinates` |
+| Coordinate is negative or outside the active map | `block coordinates out of bounds` |
+| Active block data is absent or malformed | `block unavailable` |
+
+The block slot and the active map accessor must both contain the same valid
+nonnegative integer. Missing or sparse storage and inconsistent accessor data
+return `block unavailable`.
+
+Requiring the expected map ID and rejecting all ambiguous input lets a mod
+compare every cell in its version-specific signature before it calls an
+existing mutation API. Red, Blue, and Yellow each use their own loaded map
+data. Gold does not gain this method in this RFC.
+
+## Migration and compatibility
+
+Existing mods change nothing. No hook, event, registry, manifest field, save
+field, map handler, or WorldAPI method is removed or renamed. With no hook
+subscriber the departed-dock behavior is unchanged. Existing callers cannot
+invoke the new WorldAPI method accidentally.
+
+The occupancy answer is not persisted by the engine. Disabling or uninstalling
+a mod removes its wrapper through normal owner cleanup, so a later dock entry
+uses vanilla ejection. The engine writes no new save state and neither API
+returns or serializes ROM or save data.
+
+## Verification requirements
+
+The parity gate must prove:
+
+- vanilla departed-dock message and warp remain with no subscriber;
+- one permission wrapper can allow occupancy without removing the ship-erasure
+  work or any map handler;
+- multiple cooperative wrappers preserve downstream permission;
+- absent, throwing, nil, false, malformed, and non-forwarding hook answers fail
+  closed according to the normal wrapper-chain rule;
+- Red, Blue, and Yellow contexts keep their version identity separate;
+- disabling or removing the owner restores vanilla behavior without a save
+  migration;
+- `activeBlockAt` accepts only the active map and valid in-range integral
+  coordinates, returns a scalar, and never mutates map or save state; and
+- the test fixtures and resulting changes contain no ROM or save payload.
+
+The hook must be driven through a real public `hooks:wrap` chain. The block API
+must be exercised through a public `WorldAPI` instance. Tests must not replace
+the production map handler with a test-only implementation.
+
+## Docs with the change
+
+`docs/modding.md` documents both contracts, their failure behavior, the
+cooperative wrapper pattern, and the Gen 1-only block method. No registry or
+schema changes occur, so generated registry documentation is unchanged.
+
+## Deprecation etiquette
+
+Nothing is removed, superseded, or deprecated.

docs/rfcs/0014-mod-driven-actors-and-adopted-link-sessions.md

@@ -0,0 +1,158 @@
+# RFC 0014: Mod-driven world actors and adopted link sessions
+
+## Status
+
+Proposed.
+
+## Motivation
+
+A mod can already spawn a runtime object with `mod.world:spawnNpc` and can
+already start a link battle. It cannot make either of them behave like
+something the mod itself owns.
+
+Three walls, each of which stops a mode rather than inconveniencing it:
+
+**An actor cannot move on its own schedule.** The only public way to animate a
+spawned object is `Handle:scriptMove`, which queues onto
+`OverworldState.scriptMoves`. A non-empty `scriptMoves` is how the overworld
+knows a cutscene is running -- `handleInput` gates on it -- so anything
+animated that way freezes the player's controls for as long as it walks. That
+is correct for Oak marching into his lab and wrong for an ambient walker or a
+networked player's ghost, which move continuously and must not lock anyone out.
+
+**A spawned object cannot answer the A press.** Talking is resolved from the
+map's text tables by `TEXT_*` id. A runtime object has no id, so the vanilla
+path has nothing to say for it, and the mod that created it has no way to say
+anything either.
+
+**A link battle cannot ride a connection the mod already has.** `LinkState`
+owns pairing, so a mode that already has a socket to its peer must either open
+a second connection for the battle or reimplement lockstep. And when the
+battle ends, cable rules leave the real party untouched, so the damage exists
+only in the battle's own copies -- which are gone by the time the state
+unwinds. A mode built on link battles cannot learn what the fight cost.
+
+The immediate consumer is an overworld multiplayer mode, but nothing here is
+specific to it: the first two are wanted by any mod with an actor that moves
+itself, and the third by any mode that runs battles over its own transport --
+a tournament ladder, a draft, a gauntlet.
+
+## The decision it extends
+
+This adds nothing to the compatibility surface's shape; it extends the
+**additive, guarded seam convention** that Route B in `CONTRIBUTING-mods.md`
+documents, and is gated by the parity guarantee `tests/engine/gate_meta_coverage.lua`
+enforces ("21-testing-and-ci: a parity gate for every extension point"; M14).
+
+There is no in-repo D-number registry to amend; the consuming design lives
+outside this repository, as it did for RFC 0011.
+
+## Exact API delta
+
+### New hook: `world.talk`
+
+```lua
+mod.hooks:wrap("world.talk", function(next, ow, target)
+  -- ow     = the OverworldState raising it
+  -- target = the object on the faced cell
+  if mine(target) then
+    say(target)
+    return                    -- the mod answered; the text path is skipped
+  end
+  return next(ow, target)     -- anything else falls through unchanged
+end)
+```
+
+Call site: `OverworldState:interact`, on the branch that has already resolved
+an object on the faced cell (including across a counter) and confirmed it is
+not mid-step and not the Pikachu follower. It runs before the map's text
+tables are consulted. With no subscriber, `Runtime.call` invokes the vanilla
+fallthrough, which is a file-local function rather than a per-press closure,
+so an unhooked A press allocates nothing it did not allocate before.
+
+### New event: `link.battle_ended`
+
+```lua
+mod.events:on("link.battle_ended", function(ev)
+  -- ev = {
+  --   result     = "win" | "lose" | "draw" | "ended",
+  --   myParty    = lockstep copy of our party,
+  --   theirParty = lockstep copy of theirs,
+  --   peerName   = string,
+  --   role       = "host" | "guest",
+  -- }
+end)
+```
+
+Call site: `LinkState:update`, in the `battleRunning` stage, once the battle
+state has been popped and before `exitWith` unwinds the link stack. Guarded by
+`Runtime.wants("link.battle_ended")`, so with no subscriber no payload table
+is allocated and the branch runs exactly as before.
+
+The party copies are the point of the event. They are the lockstep records the
+battle actually fought with, which is where the damage lives under cable rules.
+
+### New `WorldAPI` handle methods
+
+```lua
+handle:stepNow(dir)        --> true when the step started
+handle:canStep(dir)        --> would stepNow land somewhere legal?
+handle:placeAt(x, y, dir)  --> snap, no animation, clears a step in flight
+handle:isMoving()          --> true while a step is animating
+handle:setPassable(flag)   --> may the player walk through this object?
+```
+
+`stepNow` sets the same per-tile state `scriptMove` does, without the queue and
+therefore without the input lockout. It does **not** consult collision: the
+intended caller is replaying a move already decided elsewhere (validated on a
+peer, or authored), and re-judging it locally would let two copies of the same
+actor disagree about where it is. `canStep` is the separate opinion for callers
+that want it. `setPassable` sets the flag `Collision.occupied` already honours
+for Yellow's companion Pikachu; a passable object still draws and still talks.
+
+### Two supporting changes
+
+`Game:startNewGame(opts)` -- the title screen's NEW GAME closure, made callable,
+with `opts.intro = false` to land directly in the world. A mode that issues its
+own starting state has no use for the intro cutscene.
+
+`CodeEntry.new(shape)` -- accepts an optional `{ length =, charset = }`, so the
+slot-scrub widget that enters a link code can also carry a room code or a
+host:port address. Called with no argument it is byte-for-byte the previous
+widget.
+
+## Migration and compatibility
+
+**Existing mods change nothing.** Every item above is a new name or a new
+optional argument; no existing name, payload, signature, or default changes.
+
+The v1 surface is unaffected: `content.X:register/override/get`, `events:on`,
+`hooks:wrap`, `mod.log`, `mod:read`, the manifest v1 fields and
+`pokemon.before_give` all behave as before. `mods/example_mew_starter` -- api 1,
+`category = "GAMEPLAY"`, whole-species copy -- loads unchanged, which
+`tests/run_modkit.lua` proves on every run.
+
+With no subscriber to either seam, Red, Blue, Yellow, Gold and Silver behave
+exactly as they did: the A press reaches `talkTo`, the link battle unwinds
+without building an event payload, and no handle method is reachable unless a
+mod calls it.
+
+## Verification
+
+- `tests/modkit/cases/world_talk.lua` drives the real `OverworldState:interact`
+  path. It asserts the unhooked build first (the A press reaches `talkTo`),
+  then that a mod owning the object suppresses the text path by not calling
+  `next`, that an object the mod ignores still falls through, and that an
+  object mid-step raises no hook at all.
+- `tests/modkit/cases/link_battle_ended.lua` drives the real `LinkState:update`
+  path. It asserts that with nothing subscribed the event is not wanted (so no
+  payload is built) and the battle still unwinds, then that a subscriber
+  receives the result, the role, the peer name and both party copies including
+  the damage the real party never took, from both sides of the cable.
+- `tests/engine/gate_hooks.lua` and `tests/engine/gate_events.lua` walk the live
+  catalog, so both seams are parity-gated structurally; `gate_meta_coverage`
+  passes 208/208 with both names covered and no DEBT entry added.
+
+## Deprecation etiquette
+
+Nothing is removed, renamed, superseded or deprecated.

docs/rfcs/0015-imported-dataset-views.md

@@ -118,7 +118,7 @@ it, performs no cross-version cache reads.
 
 Opening a view does not change `GameVersion`, `CacheFs.prefix`, the active
 `Data` table, PhysFS mounts, save state, or the selected game's behavior.
-Red, Blue, Yellow, Gold, and Silver keep their existing active data paths.
+Red, Blue, Yellow, Gold, Silver, and Crystal keep their existing active data paths.
 
 The completion marker and per-version required-file rules live in the pure,
 injected `CacheContract` shared by the importer and dataset service. It also
@@ -131,7 +131,7 @@ evicts the previous semantic view.
 ## Verification
 
 - `tests/modkit/cases/dataset_views.lua` loads sandboxed fixture mods through
-  the public API and covers Red, Blue, Yellow, Gold, and Silver independently.
+  the public API and covers Red, Blue, Yellow, Gold, Silver, and Crystal independently.
 - The test proves semantic registry normalization, deterministic iteration,
   detached records, read-only facades, cross-mod facade isolation,
   version-prefixed generated assets,

docs/rfcs/0016-battle-field-residual.md

@@ -0,0 +1,123 @@
+# RFC 0016: Engine-owned field residual descriptors
+
+## Status
+
+Proposed.
+
+## Motivation
+
+Battle-rule mods can keep deterministic data-only state in the public battle
+field and observe `battle.turn_ended`, but that event fires after the engine's
+residual and faint pipeline. A listener cannot safely deal end-of-round field
+damage: directly changing live HP bypasses bar drains, faint messages,
+experience, replacements, double-faint resolution, and checkpoint continuation.
+Putting callbacks into `battle.field` is also rejected by the checkpoint
+serializer, correctly, because executable state is not save-safe.
+
+## Decision and plan extended
+
+This implements **D-AT-004: public engine-owned field residual execution**, the
+consuming decision required by Adaptive Trainers capability `ENGINE-FIELD-
+RESIDUALS`. The plan is
+[`docs/superpowers/plans/2026-08-14-adaptive-trainers.md`](https://github.com/MaxTomahawk/gen1recomp-adaptive-trainers/blob/main/docs/superpowers/plans/2026-08-14-adaptive-trainers.md),
+Task 8. The engine delta defines only a generic end-of-round extension point;
+it contains no weather names, immunities, damage formula, trainer identity, or
+Adaptive Trainers policy.
+
+## Exact API delta
+
+Add the guarded Gen 1 hook:
+
+```lua
+mod.hooks:wrap("battle.field_residual", function(next, context)
+  local rows = next(context)
+  rows[#rows + 1] = {
+    side = "enemy",
+    amount = 7,
+    message = context.battlers.enemy.name .. " is buffeted!",
+  }
+  return rows
+end)
+```
+
+The hook runs once during an undecided battle's end-of-round processing, after
+vanilla status residuals and before field/side token expiry and
+`battle.turn_ended`. With no subscriber, the guarded site builds no context and
+changes nothing. Vanilla contributes an empty list.
+
+`context` is `{ field, battlers, turn }`. `field` is a strictly data-only view
+with the same `{ weather, tokens }` shape captured by battle checkpoints. Its
+recursive projection retains raw tables and finite numbers, strings, and
+booleans under scalar keys. It strips metatables and omits functions, userdata,
+threads, unsupported keys, and cyclic edges. Thus it exposes neither
+`field.sides` nor a graph or executable callback back into live engine state.
+`battlers.player` and `battlers.enemy` are detached snapshots with `{ side,
+name, hp, maxHp, types, vanished }`. Changing either detached view cannot
+change the live battle. `turn` is the current Gen 1 turn counter.
+
+A result row is `{ side = "player"|"enemy", amount =
+positive_finite_integer_number, message = optional_string }`. Numeric strings,
+zero, negatives, fractions, NaN, infinities, malformed sides, and non-string
+messages fail closed. Damage is clamped to current HP. The engine owns
+mutation, HP-bar drain rows, and its existing faint pipeline. It applies every
+accepted row before scheduling newly fainted battlers. If this hook batch
+terminally faints the player with no healthy reserve, it queues only the player
+faint authority, so descriptor order cannot race a blackout against an enemy
+EXP/replacement path. Otherwise it schedules newly fainted battlers in fixed
+player/enemy order. This precedence is scoped to this hook response; native
+faint paths, including `enemyMonFainted`, keep their existing no-hook behavior.
+Wrappers compose by calling `next(context)` and appending their own rows.
+Callbacks are not part of the descriptor contract and hook functions are never
+stored in the battle.
+
+Gold already owns native weather and a generation-specific between-turn order;
+this first additive call site is Gen 1-only. A future Gold site must keep the
+same context and descriptor contract and choose its native ordering explicitly.
+
+## Migration and compatibility
+
+Existing mods change nothing. No hook name or payload changes. With no wrapper,
+Gen 1 performs the same residual, token, event, and faint work as before,
+including native simultaneous-faint resolution, and allocates no context. Gen
+2 is unchanged. Existing and new checkpoints keep
+serializing only the data stored in `battle.field`; hook callbacks remain
+process-local loader state and are never serialized.
+
+The v1 surface remains unchanged: `content.X:register/override/get`,
+`events:on`, `hooks:wrap`, `mod.log`, `mod:read`, manifest v1 fields, and
+`pokemon.before_give` keep their existing behavior.
+
+## Verification
+
+- The catalog hook parity gate proves null and live-empty buses return the
+  vanilla list unchanged.
+- A sandboxed fixture mod exercises the seam through `mod.hooks`, verifies the
+  detached checkpoint-shaped context, applies damage, and reaches the engine
+  faint pipeline.
+- Engine validation tests cover strict number validation (including numeric
+  strings, NaN, infinities, zero, and negatives), nested mutation isolation,
+  omission of functions/userdata/threads/cycles and metatables from the public
+  field projection, optional messages, non-table results, clamping, and
+  settled-battle suppression.
+- Both descriptor orders are driven through queue completion for a simultaneous
+  terminal residual. Each proves player blackout loss with no EXP event,
+  enemy replacement, or replacement UI.
+- A disabled-bus sentinel proves the guard performs no `Runtime.call` or field
+  context construction. An ordering probe proves the enabled hook runs after
+  vanilla status residuals and before token expiry and `battle.turn_ended`.
+- A no-hook native regression proves a simultaneous zero-HP state still enters
+  the pre-existing enemy-faint EXP and win authority outside this hook batch.
+- Capture/restore/capture evidence proves checkpointed field state round-trips
+  while the enabled process-local hook remains installed and callable.
+
+## Docs with the change
+
+`docs/modding.md` documents the Gen 1 timing, detached payload, descriptor
+validation, simultaneous-terminal result, and checkpoint boundary.
+`docs/mod-api-gen2-compat.md` records that Gold does not yet expose the hook.
+No registry or schema changes are involved, so generated registry docs do not
+change.
+
+## Deprecation etiquette
+
+Nothing is deprecated. The hook is additive.

docs/rfcs/0017-mod-developer-mode.md

@@ -0,0 +1,102 @@
+# RFC 0017: Public mod developer-mode signal
+
+## Status
+
+Proposed.
+
+## Motivation
+
+The loader already derives a boot-time developer-mode flag for its permission
+diagnostics and headless test seam. Gen 1's `Game` independently derives a
+similarly sourced flag for its console and hot reload. A sandboxed mod cannot
+read either one. `mod.commands` can register a diagnostic command but cannot
+say whether the current boot is a developer boot. `mod.exports` only publishes
+values to other mods. `game.ready` fires after entry registration and carries
+only the game. `mod.options` is player configuration, not engine mode, and
+`mod.log` logs unconditionally. The pre-sandbox compatibility
+`os.getenv("POKEPORT_DEV")` deliberately returns `nil`, because the process
+environment is hidden from mods.
+
+The concrete consumer is **Adaptive Trainers**. Its approved Chapter 30 and
+Phase H require trainer, boss, Rival, and League diagnostic views plus
+seed-label tracing to exist only when `POKEPORT_DEV` is active. Without a
+public signal, the mod must either ship those registrations in production,
+misuse a player option, or import loader/Logger internals. All three violate
+the approved observability boundary or the sandbox/public-API policy.
+
+## Decision and plan extended
+
+This implements **D-AT-005: diagnostics and seed tracing are admitted only by
+the engine's developer-mode decision**. The consuming design is tracked in the
+Adaptive Trainers implementation plan,
+[`docs/superpowers/plans/2026-08-14-adaptive-trainers.md`](https://github.com/MaxTomahawk/gen1recomp-adaptive-trainers/blob/main/docs/superpowers/plans/2026-08-14-adaptive-trainers.md),
+Task 9. The engine delta is generic and contains no trainer, balancing,
+diagnostic-layout, seed-label, or Adaptive Trainers policy.
+
+## Exact API delta
+
+Every sandboxed mod object adds one field:
+
+```lua
+mod.developer -- boolean
+```
+
+The loader copies its existing `dev` decision into this field before invoking
+the mod's entry chunk. It is therefore available for load-time registration:
+
+```lua
+if mod.developer then
+  mod.commands:register("my_mod:diagnostics", diagnostics_command)
+end
+```
+
+The value is a plain boolean snapshot, not a loader reference or environment
+facade. `false` is the normal player-build answer. `POKEPORT_DEV=1` and the
+`--developer` command-line path make the loader flag true; on Gen 1 those inputs
+separately make `Game`'s own developer flag true for its console and hot
+reload. The loader's existing injected `opts.dev` test seam changes only the
+loader flag and diagnostics, not `Game` or its hotkeys. It grants no permission
+and does not expose environment variables. The answer is fixed for the life of
+that loader; changing a field on a mod's own table cannot change engine mode.
+
+The field is generation-independent and has identical semantics on Red, Blue,
+Yellow, Gold, and Silver. Gold and Silver do not gain Gen 1's developer console
+or hot-reload hotkeys from this field.
+
+## Migration and compatibility
+
+Existing mods change nothing. `mod.developer` is additive, requires no
+permission, and does not bump the integer mod API. Existing API-v1 and API-v2
+entry chunks receive one extra scalar field and retain all prior fields and
+methods unchanged. No name is removed or shadowed.
+
+With no mods installed, `Loader:_api` is never called, so the delta allocates no
+mod object and changes no data, save, options, event, hook, command, or file.
+With mods installed in a normal boot, the new field is `false` unless an author
+explicitly reads it. Existing registration and logging behavior is unchanged.
+
+An adopting mod should gate developer-only registrations and verbose logging
+directly on `mod.developer`. Player-facing behavior belongs behind
+`mod.options`, not this signal.
+
+## Verification
+
+- `tests/engine/mod_developer_mode_test.lua` loads a real sandboxed mod through
+  the public SDK with developer mode both on and off. It proves the boolean is
+  available during entry execution and that the same source registers its
+  diagnostic command only for the developer load. It also covers the
+  command-line global path and a Gen 2 load.
+- `tests/engine/mod_developer_mode_parity_test.lua` is the separate no-mod
+  parity suite. It proves both developer answers discover no mods, create no
+  files, and leave injected vanilla data unchanged. It also loads an unchanged
+  API-v1 probe and verifies identity, `mod:read`, exports, and options behavior.
+- The full ROM-free engine and modkit tiers remain the compatibility proof for
+  `content.X:register/override/get`, `events:on`, `hooks:wrap`, `mod.log`,
+  `mod:read`, manifest v1 fields, and `pokemon.before_give`.
+
+No registry or schema changes are involved, so generated registry documentation
+is unaffected.
+
+## Deprecation etiquette
+
+Nothing is removed, renamed, superseded, or deprecated.