Path 2 of the pattern-fragment dispatch architecture: each variant of
a [[input.decompressed_section_pattern]] now gets a unique link-time
ram_addr from a synthetic vram pool (0xC0000000+, KSEG2/KSEG3 — unused
by N64 software so it can't collide with engine-resident sections like
RSP at 0xA4000000+).
Why: when multiple variants share a single canonical link bucket
(e.g. all stadium_models pattern variants at 0x8FF00000), runtime
fragment-vaddr resolution via gFragments[id] is single-pointer and
ambiguous when more than one variant is host-resident at the same
time. Per-variant synthetic ram_addrs make each variant's RELOC_HI16
/ RELOC_LO16 emit produce a unique 0xCXXXXXXX literal at runtime,
giving variant-internal references unambiguous identity without
depending on caller PC, host stack walks, or data-context tracking.
Implementation:
- add_decompressed_section accepts an override_link_ram_addr param.
The bytes-encoded `vram` (= canonical link bucket) is passed to
parse_fragment_relocs and discover_function_bounds (so jump tables
resolve correctly against the body's encoded references), while
section.ram_addr is set to the override. The two roles of vram are
cleanly separated.
- New original_pattern_id field on Section. Populated for synthetic-
link variants with the original game-side fragment id derived from
the pattern's canonical bucket (e.g. 0xEF for stadium_models).
Lets the runtime candidate filter know which game id should
include this synthetic section as a candidate, eliminating cross-
pattern hash-collision misregistration.
- main.cpp emit: section_load_table now writes original_pattern_id
into the SectionTableEntry initializer.
- decompressed.cpp pattern loop: every unique variant now gets
synthetic ram_addr = 0xC0000000 + variant_idx * 0x100000 (1 MB
stride, ~286 KB largest observed variant). For Stadium's 279
unique variants the pool occupies 0xC0000000..0xCDB00000, well
within the runtime-side 512-bucket capacity.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Section::content_hash, populates it on pattern-synthesized
sections with FNV-1a-64 of the first 0x100 bytes of the decompressed
body, and emits it into recomp_overlays.inl's SectionTableEntry. The
runtime side hashes the same window over the bytes Stadium loads at
fragment_ptr and looks up the matching section by hash.
Build-time and runtime use:
- SAME hash algorithm: FNV-1a-64
- SAME window: 0x100 bytes (95% uniqueness across Stadium's 282
distinct fragment bodies; falls back to first-candidate on the
residual ~5%)
- SAME byte source: pre-relocation decompressed bytes (link-time
form, before Stadium's R_MIPS_32 patches run)
Section table emit gains the .content_hash field; non-pattern sections
get hash=0, runtime-side condition `sec.content_hash != 0` filters
them out of the candidate set.
Pairs with the runtime-side change in
lib/N64ModernRuntime/librecomp/src/overlays.cpp.
Activation in PokemonStadiumRecomp's game.toml is gated on a
follow-up: pattern-synthesized impl bodies currently get a basic
forward-CFG-walked size which produces invalid C for fragments with
internal jump tables (data interpreted as code). Future fix: emit
pattern-section impl bodies as runtime-dispatched stubs instead of
trying to statically recompile each body. Until then, fragment78
stays declared as a single static [[input.decompressed_section]];
the engine's pattern infrastructure is in place, ready to be flipped
on once the impl-body emit is reshaped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add function hooks to mod symbol format
* Add function sizes to section function tables
* Add support for function hooks in live generator
* Add an option to the context to force function lookup for all non-relocated function calls
* Include relocs in overlay data
* Include R_MIPS_26 relocs in symbol file dumping/parsing
* Add manual patch symbols (syms.ld) to the output overlay file and relocs
* Fix which relocs were being emitted for patch sections
* Fix sign extension issue with mfc1, add TODO for banker's rounding
This commit implements the "live recompiler", which is another backend for the recompiler that generates platform-specific assembly at runtime. This is still static recompilation as opposed to dynamic recompilation, as it still requires information about the binary to recompile and leverages the same static analysis that the C recompiler uses. However, similarly to dynamic recompilation it's aimed at recompiling binaries at runtime, mainly for modding purposes.
The live recompiler leverages a library called sljit to generate platform-specific code. This library provides an API that's implemented on several platforms, including the main targets of this component: x86_64 and ARM64.
Performance is expected to be slower than the C recompiler, but should still be plenty fast enough for running large amounts of recompiled code without an issue. Considering these ROMs can often be run through an interpreter and still hit their full speed, performance should not be a concern for running native code even if it's less optimal than the C recompiler's codegen.
As mentioned earlier, the main use of the live recompiler will be for loading mods in the N64Recomp runtime. This makes it so that modders don't need to ship platform-specific binaries for their mods, and allows fixing bugs with recompilation down the line without requiring modders to update their binaries.
This PR also includes a utility for testing the live recompiler. It accepts binaries in a custom format which contain the instructions, input data, and target data. Documentation for the test format as well as most of the tests that were used to validate the live recompiler can be found here. The few remaining tests were hacked together binaries that I put together very hastily, so they need to be cleaned up and will probably be uploaded at a later date. The only test in that suite that doesn't currently succeed is the div test, due to unknown behavior when the two operands aren't properly sign extended to 64 bits. This has no bearing on practical usage, since the inputs will always be sign extended as expected.
