This eliminates duplicate code. - Only maintain the code in one place!
To make sure the submodule is getting cloned too after cloning Poke_Transporter_GB, execute:
git submodule update --init --recursive
To update the submodule to a newer commit/different branch:
cd PCCS
git pull
git checkout <commit_or_branchname>
cd ..
git add PCCS
git commit
git push
The way it works is that a specific commit is tied to your Poke_Transporter_GB repository's PCCS folder.
LZ10 decompression is builtin to the GBA's bios, so we don't need ZX0. It's also significantly faster
(618 usec instead of 2311 usec in my personal benchmark code for decompression of the same data)
And it seems like by doing so, we saved 1 KB as well!
So, seems like replacing ZX0 is the right move.
The reason I didn't initially is because I misunderstood the documentation. I assumed LZ77UnCompWram could only uncompress into EWRAM, not IWRAM.
But it turns out it can do both.
And using standardized tools is usually better than using a custom implementation.
The only downside of this right now, is that we can no longer stream text tables through a smaller buffer than the entire decompressed size.
Anyway, things seem to work fine, so bye bye ZX0. It's been fun.
This commit moves payload_builder and the z80_asm code to the data-generator subproject in order to generate the gameboy payloads
at compile time instead of at runtime.
In addition, we select a couple of base payloads (more than 1 for compressibility's sake) and generate binary patches to transform them into
other payloads. We then generate a binary file with both the base payload and binary patches and compress these files with zx0.
This reduces the rom size by about 8 KB.
To protect the world from the soft float library...
To unite all arithmetic within our binary...
To denounce the evils of floating point precision...
To save more kilobytes - that's our vision....
(god this is cringe)
All floating point math has been eliminated, and replaced with
equivalent or near-equivalent fixed-point math.
sprite_data.cpp uses Q16, and get_rand_range uses a full Q32 to
ensure that the exact same results are generated as before, at
the cost of some inline assembly to do a umull (__aeabi_lmul is a
little excessive when the lower 32 bits are discarded)
This eliminates all of the expensive double precision float library,
saving a few kilobytes.
Additionally, the unneccessary parts of nanoprintf have been
disabled. There is no need for precision specifiers, long longs, or
floats.
I eliminated these fonts by redefining code from tte_init_chr4c.c and tte_main.c files in libtonc.
Both the original tte_init_chr4c and tte_init_base functions had some code to use either verdana9 or sys8Font as default if the user
did not specify a font instance.
But we don't need it.
To eliminate this code, we not only had to define a custom version that omits this code, but also had to redefine other functions from those .c files
in order to make the linker not try to pull in those .o files from libtonc and use our versions instead.
I found another way to optimize the rom space by implementing a custom malloc, free, realloc and calloc
function.
This reduces rom size by 3 KB and IWRAM usage by 1 KB. (elimination of __malloc_av). The original
implementation is much more complex and larger than it needs to be.
The custom malloc is implemented as a bitmap allocator. It keeps a bitmap to track which pages of the
heap are allocated. Like the original allocator, it uses the free space in EWRAM after the multiboot gba
rom. But unlike the original allocator, we control the size with CUSTOM_MALLOC_POOL_SIZE.
The custom malloc can be disabled with USE_CUSTOM_MALLOC.
There was a crash happening with ptgb::vector when you'd press A on the CONFIRM button of the box screen. It only occurred on actual gba hardware and
was a real heisenbug: as soon as you'd add code to display logs on screen, the problem would disappear. So it was very difficult to figure this one
out. We're not even entirely sure why, but it looks like the malloc/realloc/free use in ptgb::vector would cause issues.
Maybe it was alignment, but after messing with the code we also saw a warning appear in the terminal telling us that realloc wouldn't properly
deal with non-POD types. It complained about this very thing while referring to the add_track() function, which stores ptgb::vectors inside another
ptgb::vector. We also didn't have a custom copy constructor yet to actually copy the buffer instead of its pointer.
All of these could potentially have led to the crash. But debugging during the link cable flow was difficult, so we were never able to confirm it in
a debugger, log or dump.
Because I suspected the high IWRAM consumption (especially now with ZX0 decompression) for a while, I also did an optimization in mystery_gift_builder
to pass global_memory_buffer as its section_30_data buffer instead. This reduces IWRAM consumption by 4 KB.
There was another problem I discovered during my crash hunt: the out_array (now payload_buffer) was allocated as a 672 byte array, but the payloads
were actually 707 bytes. Therefore writing this to the buffer caused a buffer overflow, thereby corrupting the global variables appearing after it in
IWRAM. It turned out eventually that none of these variables were really critical, but it could explain some minor bugs GearsProgress has seen.
I also did a few performance optimizations:
- At various stages in the code, for loops were used to copy data from one buffer into another byte-by-byte. This was far from optimal because the gba
cpu can load/copy 4 bytes at a time if you ask it to. So I replaced those with memcpy(), which is a hand-optimized assembly function to copy data
using this principle.
- generate_payload was being called twice: once at start_link and once at continue_link, giving the exact same result, even though it was already
being stored in a global buffer allocated in IWRAM. This was also a fairly heavy function. So I optimized the code to only initialize it once in
the script chain and then just retrieve the buffer.
- generate_payload was constructing the eventual payload twice even within the same call. That's because it first merged z80_rng_seed, z80_payload
and z80_patchlist into a full_data ptgb::vector, after which it then copied the data again to out_array (now called payload_buffer). I eliminated the
full_data vector now.
Add a binary table format and convert the text entries into this format in text_helper/main.py. It then gets compressed with zx0.
The new text_data_table and streamed_data_table classes exist to read the various entries from this binary table. streamed_data_table specifically
exists to use a decompression buffer that is smaller than the actual binary table. But it requires a decompression buffer that is
still larger than ZX0_DEFAULT_WINDOW_SIZE (default: 2048 bytes) and will only be able to decompress in
chunks of (<decompression_buffer_size> - <ZX0_DEFAULT_WINDOW_SIZE>) bytes
Try to keep the binary text tables sufficiently small though, because since zx0 doesn't actually support random access,
getting to the last entry is significantly more expensive than reading the first one. And unless you use streamed_data_table,
it also requires <uncompressed_size> bytes of stack space, therefore IWRAM to decompress them.
I also had to rework script_array because it can no longer reference the strings directly. Instead we now reference the DIA_* "enum" values.
We also no longer store an array of script_obj instances, because these were getting stored in IWRAM since they're non-const global variables
originally. Instead we now have const arrays of script_obj_params structs, which should end up in .rodata -> therefore EWRAM.
Right now, script_obj only supports the PTGB text table (originally the dialogue array). But if the need arises to support other tables as well,
I'd consider adding a separate enum to script_obj_params to indicate the specific table.
The compilation process will also output .su files in the build folder from now on. These files indicate the stack frame size for every function in
every compilation unit, so be sure to check them from time to time. Note that they will only show the stack consumption for that specific function.
So to get the worst case stack consumption, you need to manually add all the functions in a certain stack flow.
libsysbase_libsysbase_a-handle_manager.o
So, I optimized the MOVESETS table to only store the "overriding" bits in the movesets of the evolutions
in relation to their base forms. That only improved compression slightly (about 300 bytes)
I also eliminated 4 KB of IWRAM usage by libsysbase_libsysbase_a-handle_manager.o because of the "handles"
buffer. We're not using it and we REALLY need our IWRAM. (and it also reduces the rom size with 4KB too!)
