DramaticShapeVoxelMod/assets/docs/buidling_to_voxel/sprite_to_voxel_methodology.md

Sprite → Voxel Methodology

Translating Game Boy ¾-view overworld sprites into 3D voxel models

Worked example: Red's house exterior — PALLET_TOWN blocks 56/57/60/61, 64×48 px (reds_house_voxel.html). The same pipeline applies to any GB building sprite.

---

Core principle

A Game Boy overworld sprite is a fake-3D projection: it packs several different 3D facings into one flat image. Roof tops are drawn as if seen from above, walls as if seen from the front, and sloped surfaces as diagonal silhouettes. Voxelization is therefore not one operation — it is (1) classifying each region of the sprite by which 3D surface it depicts, then (2) applying the matching geometric operation per region.

Two invariants govern everything:

1. Every visible voxel color comes from a sprite pixel wherever the lookup lands inside the silhouette. Synthesis is allowed only for geometry the sprite implies but never paints (undersides, depth extensions, interiors), and synthesized cells must continue the sprite's own periodic patterns and 4-shade palette.
2. The sprite is ground truth, not the tile documentation. The tile-grid doc described 1px ground flanks; the actual sprite has 5px insets on the ground floor and a 3-level wedding-cake silhouette. Always extract from the real pixels and verify any doc claims against the extracted mask.

---

Stage 1 — Extraction

Sample the sprite down to its native resolution (uploads are usually integer upscales; sample the center of each scale×scale block, never bilinear). Quantize to the palette — for GB art expect exactly #ffffff / #aaaaaa / #555555 / #000000. Anything else in the histogram means the sampling grid is misaligned.

Compute the silhouette with a flood fill from the image border that spreads only through light pixels (luminance > 130). This threshold is load-bearing: the black outline (#000) and the dark shading (#555) together form the boundary. A naive "not black" threshold lets the fill eat the #555 shaded flanks and corrupts the silhouette — that bug produced a 398-vs-148 pixel discrepancy on the first pass here.

Output: a JSON of the palette plus one string per row (. = outside the outline, digit = palette index). Dump the mask as ASCII and read it — this is where the building's real structure becomes visible, and it is the input to Stage 2. Tooling: sprite_extract.py.

Stage 2 — Band classification

Segment the sprite rows into horizontal bands and assign each a facing. The cues generalize:

Cue in the mask / pixels	Reads as	3D treatment
Top band, plain fill with full-width course lines	Roof top seen from above	Lay horizontal (flat)
Silhouette tapering at a constant rate (here 2 px per row) with slat/stripe texture	Sloped roof surface; the taper rate is the slope	Stepped diagonal surface, 1 down per (taper rate) out
Band containing window/door frames	Vertical facade	Straight extrusion
Full-width band with a black underline sitting above an inset band	Ledge / awning overhang	Extrusion + protrusion
Dark #555 runs beside a facade under a taper	Shadow on the wall beneath an eave	Leave as wall — the geometry above produces the shadow's meaning

The band table for Red's house, which Blue's house shares verbatim:

Sprite rows	Content	Treatment
0–15, x16–47	Coursed grey panel, black rims, highlight course	Flat horizontal top
0–15, flanks	White/dark slats, 2 px-per-row taper	Sloped ends, 2:1
16–23	Gable wall, 3 windows, slat flank pixels	Extrude; roof solid overwrites the corners
24–31	Awning slab, double black underline	Extrude + 2-voxel ledge front/back
32–47	Ground floor siding, door (own inner pane), 2 windows	Extrude; recess panes

Interpretation rules settled during this build — do not relearn them: a top-facing band must end up level everywhere (no synthesized skirts rising through it); a tapering silhouette on a top band means sloped surfaces in elevation, not chamfered corners in plan; and the slope's eave tips should land where the sprite's taper pixels stop (here: just above the awning band).

Stage 3 — Geometry construction

Vertical bands extrude straight back to depth D (26 here). Front and back layers take the sprite pixel; interior layers take a de-outlined color — if the pixel is black, walk inward up to 3 px for the first non-black color. Without this, the side faces of the model are solid outline-black slabs. With it, sides read as material with black front/back corner edges, which is the correct GB-cartoon look. The back face becomes a mirror of the front for free; that is sprite-pure and acceptable.

Ledges replicate the band's front pixels ±2 voxels in z past the walls. The sprite's own black-underline rows become the visible dark underside.

Recesses delete the front voxel of every non-black pixel inside a window/door rect. Frames stay proud; the identically-colored voxel behind becomes the pane, one voxel deep. Rects with nested frames (the door's inner window) produce layered relief automatically.

The flat top lays the top-facing rows horizontal. The band is shallower than the house (16 rows vs 34+ of depth), so extend it by cycling a mid-row band whose period matches the course rhythm (rows 5–8 here, period 4) — this continues both course lines and slat columns seamlessly. Map the outermost sprite rows to the front/back rims so the black-line/fascia trim survives.

Sloped ends are driven by an elevation profile T(x): flat at YTOP over the plateau, dropping 1 voxel per (taper rate = 2) columns outward to the eave tips. Build a solid of constant vertical thickness (4) following T(x), spanning the full roof depth including overhangs. Build order matters: walls first, then the roof solid overwrites wall voxels it intersects, then trim any wall voxel above T(x) so nothing pokes through the surface. The wall strip left exposed beneath the slope shows the sprite's own #555 shadow pixels — the sprite encodes this geometry.

Outline pass: cells on the roof's outer boundary get black (top layer), grey (second layer), black (below) — reproducing the sprite's black-grey-black fascia — and all interior undersides are dark.

Parameters used here, to tune per building: depth D=26; roof overhang 4 in x and z beyond the walls; slab thickness 4; YTOP=35, derived so the eave tips (after the 10-step drop) land one voxel above the awning band, matching where the sprite's taper ends.

Stage 4 — Color sourcing off the sprite

For any roof cell, map z to a sprite row (roofSy), then look up (x, row). If that lands inside the silhouette, use the pixel. If it lands outside (overhang extensions), continue the sprite's periodic texture: the slat rhythm is period 3 with a per-side phase (left flank white at x % 3 == 1, right at x % 3 == 2 — derive the phases from the actual pixels, and mind negative-modulo semantics in JS). Plain top areas fall back to the mid grey. Ground plane and path are presentation-only and the single place non-palette colors are permitted.

Stage 5 — Verification (non-negotiable)

Every bug in this build was caught by one of these, none by eyeballing alone:

1. Dual implementation parity. Build the identical algorithm in a reference implementation (Python) and in the shipping runtime (JS in the HTML). Diff total voxel count and post-cull shell count — they must match exactly (final build: 58,356 / 14,169).
2. Numeric asserts on intent. Flatness: the set of y-layers above the walls must be exactly the slab layers. Slope: the top-surface profile at mid-depth must read tips → 1-per-2 steps → flat plateau, mirrored. Zero wall voxels with y > T(x). Full wall coverage by the roof footprint.
3. Isometric preview. A ~60-line painter's-algorithm render (sort by (x+z, y), draw top/left/right faces of shell voxels) catches texture and layering mistakes cheaply before touching the runtime.
4. Hidden-face culling. Drop voxels whose 6 neighbors all exist before instancing; render the shell as one InstancedMesh.

Tooling: voxel_build_verify.py (builds, asserts, renders previews).

Repeat checklist

1. Obtain the sprite; sample to native resolution via block centers.
2. Extract palette + silhouette (light-only flood fill, threshold 130); review the ASCII mask.
3. Segment rows into bands using the Stage-2 cues; write the band table before writing any geometry code.
4. Measure taper rates from the mask; derive T(x), YTOP, overhangs, D.
5. Build: extrude verticals (de-outlined interiors) → ledges → recesses → flat top (mid-row cycling) → sloped solids (overwrite, then trim) → outline pass → ground presentation.
6. Verify: parity counts, profile/flatness/poke asserts, iso preview.
7. Ship: embed palette + row strings in the HTML; the builder runs client-side and doubles as the reference implementation of the algorithm.

Applying this in the mod

Because tilesets are shared, the band table can be keyed by tile id rather than by sprite: each id gets a treatment record (extrude / lay-flat / slope, plus ledge, recess, warp flags) in the spirit of mods/DRAMATIC_SHAPE/data/voxel_heights.lua. Pinning blocks 56/57/60/61 with the profile above voxelizes Blue's house identically for free and propagates to the Fuchsia City, Route 25, and Saffron City instances of the same art. The door's lower-left tile (27) is the warp/walkable tile — keep its recessed front face aligned with the collision cell so the 3D doorway matches REDS_HOUSE_1F's warp.

What shipping it settled

The mod implements this as mods/DRAMATIC_SHAPE/lib/Buildings.lua, driven by a buildings list in the profile. Three things changed from the sketch above, each for a reason worth keeping:

Key the band table by the building's tile GRID, not by tile id. A tile id is not a band: tile 23 is the house's awning course and the top of its roof and the eave course that ends Oak's lab's much taller roof. Matching the exact grid (../buildings/ catalogues one per building, with every map that places it) is unambiguous, still shares one entry across all seven placements of Red's house, and cost nothing to verify — a scan of all 222 maps returns exactly the catalogued placements.

Measure everything measurable. Only the band table needs a human to read the drawing. The silhouette, the taper rate, the eave height and every window and doorway come off the pixels: a pane is a non-black region the drawing seals behind its own black frame, and T(x) = YTOP - topRow(x) falls straight out of the mask — which is also what makes YTOP stop being hand-tuned. The eave tips landing one voxel above the awning, tuned by hand here, then happens by itself.

Depth is the plot, and the drawn row → depth mapping has a direction. In a diorama D is free; on a map it is the building's footprint (48px for Red's house, 64 for the lab), which is why the roof band has to be cycled so far. And the drawing looks at the roof from the north: its top rows are the FAR edge and its bottom rows the eave over the facade. Getting that backwards is invisible in the counts and in a symmetric preview — it shows up in game as a fascia along the wrong rim.

One colour note: the outline pass's grey fascia band (GREY at t-1) is right for the raw GB palette but comes out white once the atlas is recoloured, turning every sloped end into a black-and-white zip. The drawing's own eave is black / #555 / black, and using that reads correctly under every palette.