mirror of
https://github.com/bryanthaboi/gen1recomp.git
synced 2026-08-12 08:21:02 +02:00
Fix Windows crash: decode LuaJIT dump output as UTF-8, not the locale codepage
subprocess.run(..., capture_output=True, text=True) with no explicit encoding falls back to locale.getpreferredencoding(False) -- the OS default codepage. On Windows that's a legacy single-byte codepage (e.g. cp1252), never UTF-8. When the LuaJIT dump contains a byte with no mapping in that codepage, subprocess's internal _readerthread crashes with an uncaught UnicodeDecodeError in a background thread; the thread dies silently and the caller gets back stdout=None instead of a string, crashing one line later with AttributeError: 'NoneType' object has no attribute 'splitlines'. Concretely, the Yellow-side imported dataset contains: "_ColosseumHeightText" -> "...6’8” tall!" The right double quotation mark (U+201D) encodes in UTF-8 as E2 80 9D; 0x9D has no defined character in cp1252, so decoding as cp1252 fails outright. Verified against the real imported dataset: the Red/Blue-only dump has zero bytes outside cp1252's defined range; the Yellow dump has exactly one, at this row. UTF-8 is the actual encoding these dumps are produced in -- the driver Lua sources are read/written as UTF-8 throughout this file, and LuaJIT writes those source strings' bytes back out verbatim -- so passing encoding="utf-8" explicitly at the three affected call sites (run_loader, check_data_dump, dump_dataset) is a no-op on platforms whose default codepage is already UTF-8 (Linux/macOS) and a correctness fix on Windows. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
+3
-3
@@ -716,7 +716,7 @@ def run_loader(repo, mod_dir, findings, base="fixture", notes=None):
|
||||
driver_path = handle.name
|
||||
try:
|
||||
proc = subprocess.run([LUAJIT, driver_path], cwd=repo,
|
||||
capture_output=True, text=True, timeout=120)
|
||||
capture_output=True, text=True, encoding="utf-8", timeout=120)
|
||||
except FileNotFoundError:
|
||||
findings.append(Finding("MK100", "error",
|
||||
f"cannot run {LUAJIT} (install luajit or "
|
||||
@@ -1056,7 +1056,7 @@ def check_data_dump(repo, path, base, rel):
|
||||
driver = DUMP_DRIVER % (lua_quote(path), lua_quote(vanilla))
|
||||
try:
|
||||
proc = subprocess.run([LUAJIT, "-e", driver], cwd=repo,
|
||||
capture_output=True, text=True, timeout=60)
|
||||
capture_output=True, text=True, encoding="utf-8", timeout=60)
|
||||
except FileNotFoundError:
|
||||
# the gate must fail closed: a missing interpreter is a broken
|
||||
# environment, not a clean mod
|
||||
@@ -1416,7 +1416,7 @@ def dump_dataset(repo, base):
|
||||
handle.close()
|
||||
try:
|
||||
proc = subprocess.run([os.environ.get("LUA", "luajit"), handle.name],
|
||||
cwd=repo, capture_output=True, text=True)
|
||||
cwd=repo, capture_output=True, text=True, encoding="utf-8")
|
||||
finally:
|
||||
os.unlink(handle.name)
|
||||
if proc.returncode != 0:
|
||||
|
||||
Reference in New Issue
Block a user