Due to recent development of tools and discussions, this
is something that came up and has probably been overdue
for ages.
Document key concepts of how the iidx engine works regaring
rendering, game loop and (song) sync.
Add a “syncbook” that provides context and guidelines how
to configure any (modern) system to work properly with iidx.
Due to recent development of tools and discussions, this
is something that came up and has probably been overdue
for ages.
Document key concepts of how the iidx engine works regaring
rendering, game loop and (song) sync.
Add a “syncbook” that provides context and guidelines how
to configure any (modern) system to work properly with iidx.
Due to recent development of tools and discussions, this
is something that came up and has probably been overdue
for ages.
Document key concepts of how the iidx engine works regaring
rendering, game loop and (song) sync.
Add a “syncbook” that provides context and guidelines how
to configure any (modern) system to work properly with iidx.
When running this in a script on startup to ensure
whatever configuration is required to be created is
created, this avoids several commands to first
check if the profile exists, delete it, then re-create it.
When running this in a script on startup to ensure
whatever configuration is required to be created is
created, this avoids several commands to first
check if the profile exists, delete it, then re-create it.
When running this in a script on startup to ensure
whatever configuration is required to be created is
created, this avoids several commands to first
check if the profile exists, delete it, then re-create it.
When running this in a script on startup to ensure
whatever configuration is required to be created is
created, this avoids several commands to first
check if the profile exists, delete it, then re-create it.
When running this in a script on startup to ensure
whatever configuration is required to be created is
created, this avoids several commands to first
check if the profile exists, delete it, then re-create it.
Adding two more tests that have been proven very useful
for debugging monitor issues that caused problems with IIDX.
The vsync test flickers a text snippet between red and cyan on
every even and uneven frame. If vsync works correctly, the text
appears grey to the human eye. If there are issues like re-displaying
a previous frame or a dropped frame, this appears as the text
rendering red or cyan.
The response time test scrolls two simple blocks/lines vertically
to check if they creating ghosting/smearing on the display.
The tool now supports an interactive and command line mode.
Interactive mode is useful for running different tests on the setup
while the command line mode can be used to ran these tests
with fixed parameters and for a fixed amount of time, e.g. in a shell
script, for automation purpose.
The tool still has a few obvious imperfections like scaling issues on
different resolutions. Considering the basic functionality is given,
further improvements can follow in future iterations.
Have a checkbox to switch to a frame graph focused view
which saves a bunch of math when it’s more useful to
focus on the frame/refresh rate than the frame time values.
GPU scaling allows high quality upscaling of older games
to the native resolution of the screen. The output frame
quality is significantly sharper than the fairly simple
frame buffer upscaling of bemanitools.
10 seconds is what the nvidia control panel also
has for the test timeout. Can still be changed if
shorter or longer timeouts are desired, but 10
seconds should be a general fine timeout value
to have this optional.
Use more user-friendly seconds for total and warm-up time
instead of frames. Use the refresh rate to calculcate the
total frame count internally for the test.
Use more user-friendly seconds for total and warm-up time
instead of frames. Use the refresh rate to calculcate the
total frame count internally for the test.
A separate application to run the infamous IIDX “monitor check”.
The on-screen output has been enhanced to provide more
detailed information about the on-going process. The tool
furthermore allows basic querying of adapter and adapter
modes information from the command line. These additional
options can be useful to gather more debug information
or select correct parameters for the monitor test from actually
supported mode parameters by the adapter.
The tool has been tested on an actual cabinet with nvgpu
setting different custom timings. The accuracy seems to be
even higher than what IIDX’s monitor check is actually showing.
For example, with a custom timing of 59.900, the tool yields 59.902.
Meanwhile, the monitor check of iidx 29 59.8981 hz to 59.8997 hz
on screen.
This doesn’t mean the game’s getting inaccurate values. The game
actually syncs and plays just fine. However, the game’s monitor check
cannot be used as a reference for “getting the avg. refresh rate”
value as an input parameter for other tools, e.g. chart patching with
btools.
Co-authored-by: icex2 <djh.icex2@gmail.com>
Use more user-friendly seconds for total and warm-up time
instead of frames. Use the refresh rate to calculcate the
total frame count internally for the test.
A separate application to run the infamous IIDX “monitor check”.
The on-screen output has been enhanced to provide more
detailed information about the on-going process. The tool
furthermore allows basic querying of adapter and adapter
modes information from the command line. These additional
options can be useful to gather more debug information
or select correct parameters for the monitor test from actually
supported mode parameters by the adapter.
The tool has been tested on an actual cabinet with nvgpu
setting different custom timings. The accuracy seems to be
even higher than what IIDX’s monitor check is actually showing.
For example, with a custom timing of 59.900, the tool yields 59.902.
Meanwhile, the monitor check of iidx 29 59.8981 hz to 59.8997 hz
on screen.
This doesn’t mean the game’s getting inaccurate values. The game
actually syncs and plays just fine. However, the game’s monitor check
cannot be used as a reference for “getting the avg. refresh rate”
value as an input parameter for other tools, e.g. chart patching with
btools.
A overlay window showing the a frame time graph with
the current frame time, ~10 seconds of history as well
as reference lines for the current avg. and a configurable
target frame time.
This is considered a debug tool to support in any efforts
that are related to understanding current frame times
of the games. More specifically this can be used to asses
impact of any bemanitools hooking to the game’s main
(render) loop.
For now, we focus on internal overlays for bemanitools
that hook into an existing d3d9 context. The current
abstraction is fairly thin and should be fine as a start.
Implementations are called “components” and hook
up with a single frame_update function to execute
and logic and drawing updates with imgui
Version 1.91.7 of the imgui library with the cimgui wrapper.
imgui allows us to easily create minimal and powerful UI for
use-cases like in-game overlays or separate tooling.
The additional wrappers cimgui_impl_dx9 and cimgui_impl_win32
were added to provide a full C linkage integration for the rest
of the code base.
Tweaks to the makefile were kept to a minimum but enable
compilation of C++ since imgui is C++ based.
Remark: At this point bemanitools itself is still to be kept a pure
C codebase. Due to the lack of proper module/library management
with the current build system in bemanitools 5, proper separation
of concerns and clean integration with external libraries isn’t
possible with reasonable effort.
An open source re-implementation of the “NvDisplayConfigLDJ"
tool with additional enhancements.
This can be used to tweak your nvidia GPU driver settings to
create custom display timings to address IIDX’s requirement
if expecting proper display timings. This can also be used for
any legacy IIDX versions that even expect very specific display
timings, e.g. 59.95 or 60.05 hz.
Furthermore, creating application profiles allows further tweaks
to important GPU settings such as the current performance mode
setting. This is crucial to ensure the GPU is not going into any
kind of power saving states which results in non-smooth
scrolling during gameplay and micro stuttering that cannot
be measured on application level.
An open source re-implementation of the “NvDisplayConfigLDJ"
tool with additional enhancements.
This can be used to tweak your nvidia GPU driver settings to
create custom display timings to address IIDX’s requirement
if expecting proper display timings. This can also be used for
any legacy IIDX versions that even expect very specific display
timings, e.g. 59.95 or 60.05 hz.
Furthermore, creating application profiles allows further tweaks
to important GPU settings such as the current performance mode
setting. This is crucial to ensure the GPU is not going into any
kind of power saving states which results in non-smooth
scrolling during gameplay and micro stuttering that cannot
be measured on application level.
Summary:
Test Plan:
A shim library implementing the same concept as the already
existing ddrio-async library. The iidxio implementation takes
another iidxio library and runs it asynchronously which may
improve performance for certain iidxio implementations,
e.g. if the send and receive functions are driving actual IO
calls synchrously and are expensive.
This is not a replacement for a well engineered and proper
implementation of a iidxio library for any specific use-case.
It does not fix bad performance of existing implementations,
i.e. if the poll rate is too low because actual IO is too slow.
Use with caution and know why and when you need to use
it.
A separate application to run the infamous IIDX “monitor check”.
The on-screen output has been enhanced to provide more
detailed information about the on-going process. The tool
furthermore allows basic querying of adapter and adapter
modes information from the command line. These additional
options can be useful to gather more debug information
or select correct parameters for the monitor test from actually
supported mode parameters by the adapter.
The tool has been tested on an actual cabinet with nvgpu
setting different custom timings. The accuracy seems to be
even higher than what IIDX’s monitor check is actually showing.
For example, with a custom timing of 59.900, the tool yields 59.902.
Meanwhile, the monitor check of iidx 29 59.8981 hz to 59.8997 hz
on screen.
This doesn’t mean the game’s getting inaccurate values. The game
actually syncs and plays just fine. However, the game’s monitor check
cannot be used as a reference for “getting the avg. refresh rate”
value as an input parameter for other tools, e.g. chart patching with
btools.
A separate application to run the infamous IIDX “monitor check”.
The on-screen output has been enhanced to provide more
detailed information about the on-going process. The tool
furthermore allows basic querying of adapter and adapter
modes information from the command line. These additional
options can be useful to gather more debug information
or select correct parameters for the monitor test from actually
supported mode parameters by the adapter.
The tool has been tested on an actual cabinet with nvgpu
setting different custom timings. The accuracy seems to be
even higher than what IIDX’s monitor check is actually showing.
For example, with a custom timing of 59.900, the tool yields 59.902.
Meanwhile, the monitor check of iidx 30 yields ~XXX TODO.
Summary:
Test Plan:
A separate application to run the infamous IIDX “monitor check”.
The on-screen output has been enhanced to provide more
detailed information about the on-going process. The tool
furthermore allows basic querying of adapter and adapter
modes information from the command line. These additional
options can be useful to gather more debug information
or select correct parameters for the monitor test from actually
supported mode parameters by the adapter.
The tool has been tested on an actual cabinet with nvgpu
setting different custom timings. The accuracy seems to be
even higher than what IIDX’s monitor check is actually showing.
For example, with a custom timing of 59.900, the tool yields 59.902.
Meanwhile, the monitor check of iidx 30 yields ~XXX TODO.
An open source re-implementation of the “NvDisplayConfigLDJ"
tool with additional enhancements.
This can be used to tweak your nvidia GPU driver settings to
create custom display timings to address IIDX’s requirement
if expecting proper display timings. This can also be used for
any legacy IIDX versions that even expect very specific display
timings, e.g. 59.95 or 60.05 hz.
Furthermore, creating application profiles allows further tweaks
to important GPU settings such as the current performance mode
setting. This is crucial to ensure the GPU is not going into any
kind of power saving states which results in non-smooth
scrolling during gameplay and micro stuttering that cannot
be measured on application level.
Summary:
Test Plan:
A overlay window showing the a frame time graph with
the current frame time, ~10 seconds of history as well
as reference lines for the current avg. and a configurable
target frame time.
This is considered a debug tool to support in any efforts
that are related to understanding current frame times
of the games. More specifically this can be used to asses
impact of any bemanitools hooking to the game’s main
(render) loop.
For now, we focus on internal overlays for bemanitools
that hook into an existing d3d9 context. The current
abstraction is fairly thin and should be fine as a start.
Implementations are called “components” and hook
up with a single frame_update function to execute
and logic and drawing updates with imgui
Version 1.91.7 of the imgui library with the cimgui wrapper.
imgui allows us to easily create minimal and powerful UI for
use-cases like in-game overlays or separate tooling.
The additional wrappers cimgui_impl_dx9 and cimgui_impl_win32
were added to provide a full C linkage integration for the rest
of the code base.
Tweaks to the makefile were kept to a minimum but enable
compilation of C++ since imgui is C++ based.
Remark: At this point bemanitools itself is still to be kept a pure
C codebase. Due to the lack of proper module/library management
with the current build system in bemanitools 5, proper separation
of concerns and clean integration with external libraries isn’t
possible with reasonable effort.
Summary:
Test Plan:
Integrate the previously created frame time graph component
into iidxhook 1 to 8. iidxhook9 uses d3d9ex which still requires
a separate hook implementation for imgui.
Guard the feature with a switch as it might have some performance
impact and it’s something that is only required to be turned on
when needed.
A overlay window showing the a frame time graph with
the current frame time, ~10 seconds of history as well
as reference lines for the current avg. and a configurable
target frame time.
This is considered a debug tool to support in any efforts
that are related to understanding current frame times
of the games. More specifically this can be used to asses
impact of any bemanitools hooking to the game’s main
(render) loop.
For now, we focus on internal overlays for bemanitools
that hook into an existing d3d9 context. The current
abstraction is fairly thin and should be fine as a start.
Implementations are called “components” and hook
up with a single frame_update function to execute
and logic and drawing updates with imgui
Version 1.91.7 of the imgui library with the cimgui wrapper.
imgui allows us to easily create minimal and powerful UI for
use-cases like in-game overlays or separate tooling.
The additional wrappers cimgui_impl_dx9 and cimgui_impl_win32
were added to provide a full C linkage integration for the rest
of the code base.
Tweaks to the makefile were kept to a minimum but enable
compilation of C++ since imgui is C++ based.
Remark: At this point bemanitools itself is still to be kept a pure
C codebase. Due to the lack of proper module/library management
with the current build system in bemanitools 5, proper separation
of concerns and clean integration with external libraries isn’t
possible with reasonable effort.
Summary:
Test Plan:
Version 1.91.7 of the imgui library with the cimgui wrapper.
imgui allows us to easily create minimal and powerful UI for
use-cases like in-game overlays or separate tooling.
The additional wrappers cimgui_impl_dx9 and cimgui_impl_win32
were added to provide a full C linkage integration for the rest
of the code base.
Tweaks to the makefile were kept to a minimum but enable
compilation of C++ since imgui is C++ based.
Remark: At this point bemanitools itself is still to be kept a pure
C codebase. Due to the lack of proper module/library management
with the current build system in bemanitools 5, proper separation
of concerns and clean integration with external libraries isn’t
possible with reasonable effort.
Version 1.91.7 of the imgui library with the cimgui wrapper.
imgui allows us to easily create minimal and powerful UI for
use-cases like in-game overlays or separate tooling.
The additional wrappers cimgui_impl_dx9 and cimgui_impl_win32
were added to provide a full C linkage integration for the rest
of the code base.
Tweaks to the makefile were kept to a minimum but enable
compilation of C++ since imgui is C++ based.
Remark: At this point bemanitools itself is still to be kept a pure
C codebase. Due to the lack of proper module/library management
with the current build system in bemanitools 5, proper separation
of concerns and clean integration with external libraries isn’t
possible with reasonable effort.
An open source re-implementation of the “NvDisplayConfigLDJ"
tool with additional enhancements.
This can be used to tweak your nvidia GPU driver settings to
create custom display timings to address IIDX’s requirement
if expecting proper display timings. This can also be used for
any legacy IIDX versions that even expect very specific display
timings, e.g. 59.95 or 60.05 hz.
Furthermore, creating application profiles allows further tweaks
to important GPU settings such as the current performance mode
setting. This is crucial to ensure the GPU is not going into any
kind of power saving states which results in non-smooth
scrolling during gameplay and micro stuttering that cannot
be measured on application level.
An open source re-implementation of the “NvDisplayConfigLDJ"
tool with additional enhancements.
This can be used to tweak your nvidia GPU driver settings to
create custom display timings to address IIDX’s requirement
if expecting proper display timings. This can also be used for
any legacy IIDX versions that even expect very specific display
timings, e.g. 59.95 or 60.05 hz.
Furthermore, creating application profiles allows further tweaks
to important GPU settings such as the current performance mode
setting. This is crucial to ensure the GPU is not going into any
kind of power saving states which results in non-smooth
scrolling during gameplay and micro stuttering that cannot
be measured on application level.
Summary:
Test Plan:
An open source re-implementation of the “NvDisplayConfigLDJ"
tool with additional enhancements.
This can be used to tweak your nvidia GPU driver settings to
create custom display timings to address IIDX’s requirement
if expecting proper display timings. This can also be used for
any legacy IIDX versions that even expect very specific display
timings, e.g. 59.95 or 60.05 hz.
Furthermore, creating application profiles allows further tweaks
to important GPU settings such as the current performance mode
setting. This is crucial to ensure the GPU is not going into any
kind of power saving states which results in non-smooth
scrolling during gameplay and micro stuttering that cannot
be measured on application level.
A shim library implementing the same concept as the already
existing ddrio-async library. The iidxio implementation takes
another iidxio library and runs it asynchronously which may
improve performance for certain iidxio implementations,
e.g. if the send and receive functions are driving actual IO
calls synchrously and are expensive.
This is not a replacement for a well engineered and proper
implementation of a iidxio library for any specific use-case.
It does not fix bad performance of existing implementations,
i.e. if the poll rate is too low because actual IO is too slow.
Use with caution and know why and when you need to use
it.
Move everything to new launcher.xml configuration
files. Adjust the bootstrapping of launcher in the
.bat files. Features such as copying the default
props/ files to nvram are now handled by launcher.
Using the PATH variable, bemanitools binaries can
live in their own dedicated bemanitools/ subfolder
next to props/ and modules/ now. All original
binaries are expected to be kept in a modules/
folder like on stock data.
Kudos to Shiz for providing the groundwork for this.
Fundamentally re-think how launcher operates and
bootstrapping the games is managed and configured.
This brings it significantly closer to how the original
bootstrap is doing the job: launcher now utilizes the
data (structures) provided by the bootstrap.xml configuration
file. This creates compatibility with vanilla data dumps
and original stock images. Note that bemanitools does not
include any code or means to run DRM'd data, only decrypted.
But, this allows users to keep decrypted dumps as stock as
possible which means:
* No copying around of property files anymore
* Keep the modules/ folder with the binaries
* Have bemanitools binaries separate in the data
* No need to edit/customize the original configuration files
A list of key features of the "new" launcher:
* Boostrap games by following the configuration provided by
stock game's bootstrap.xml files
* Custom launcher.xml configuration file that adds further
launcher configurable features, composability of
bootstrap.xml configuration(s) as well as configuration
overriding/stacking of selected types of configurations,
e.g. eamuse config, avs-config. The latter eliminates
the need for modifying stock config files in the prop/
folder
* Unified logging system: launcher and AVS logging uses
the same logger, all output can now be in a single file
* Original features such as various hook types still
available
Due to the significant architectural changes, this also
breaks with any backwards compatibility to existing
launcher setups. Thus, users need to migrate by re-applying
the new configuration format and migrating their config
parameters accordingly.
Further migration instructions and updated documentation
will be provided upon release.
Co-authored-by: Shiz <hi@shiz.me>
Keep this a separate commit because this also removes
inject's own logging engine and replaces it with the
streamlined core API. The core API provides all the
features of inject's own logging engine which also
performed horribly. The entire logging operation
was locked which included expensive operations
that formatted the log messages and required
memory allocations and copying around data.
The core API's implementation at least only
synchronizes the actual IO operations
(though this can be improved further with an
actual async logging sink, TBD)
The log API stopped scaling already a while ago and needs
considerable refactoring to consider the various use-cases
that emerged since it was first created on alpha versions
of bemanitools.
Boils down to:
- Include headers
- Reduce boiler plate with helpers
- Swap out explicit usages with core API layer
and ensure the right API is configured beforehand
This module contains the "core" (API) of
bemanitools which includes an abstraction
layer for threads and logging at this time.
The threads API is very close to what
util/thread already was with some structural
enhancements which make it easier to understand
and work with the API, I hope. Some additional
helpers (*-ext module) support in doing common
tasks, e.g. setting up the thread API with other
modules.
The log(ging) part receives a major overhaul to
address known limitations and issues with the
util/log module:
- Cleaner API layer
- Separate sinks from actual logging engine
- Sinks are composable
- Improved and cleaner compatibility layer
with AVS logging API
Additional "extensions" (*-ext modules) add
various helper functions for common tasks like
setting up the logging engine with a file and stdout
sink.
The sinks also improved significantly with the file
sink now supporting proper appending and log rotation.
Logging to stdout/stderr supports coloring of log
messages which works across logging engines.
Overall, this refactored foundation is expected to
support future developments and removes known
limitations at the current scale of bemanitools such as:
- Reducing boiler plate code across hooks
- Interop of bemanitools and AVS (and setting the foundation
for addressing currently missing interop, e.g. for
dealing with property structures without AVS)
- Addressing performance issues in the logging engine
due to incorrect interop with AVS
Improve the development experience by providing
an additional docker container that can be started
and used as an interactive development environment.
It provides all the tools and a stable environment
for building (identical to the build container).
Move everything to new launcher.xml configuration
files. Adjust the bootstrapping of launcher in the
.bat files. Features such as copying the default
props/ files to nvram are now handled by launcher.
Using the PATH variable, bemanitools binaries can
live in their own dedicated bemanitools/ subfolder
next to props/ and modules/ now. All original
binaries are expected to be kept in a modules/
folder like on stock data.
Kudos to Shiz for providing the groundwork for this.
Fundamentally re-think how launcher operates and
bootstrapping the games is managed and configured.
This brings it significantly closer to how the original
bootstrap is doing the job: launcher now utilizes the
data (structures) provided by the bootstrap.xml configuration
file. This creates compatibility with vanilla data dumps
and original stock images. Note that bemanitools does not
include any code or means to run DRM'd data, only decrypted.
But, this allows users to keep decrypted dumps as stock as
possible which means:
* No copying around of property files anymore
* Keep the modules/ folder with the binaries
* Have bemanitools binaries separate in the data
* No need to edit/customize the original configuration files
A list of key features of the "new" launcher:
* Boostrap games by following the configuration provided by
stock game's bootstrap.xml files
* Custom launcher.xml configuration file that adds further
launcher configurable features, composability of
bootstrap.xml configuration(s) as well as configuration
overriding/stacking of selected types of configurations,
e.g. eamuse config, avs-config. The latter eliminates
the need for modifying stock config files in the prop/
folder
* Unified logging system: launcher and AVS logging uses
the same logger, all output can now be in a single file
* Original features such as various hook types still
available
Due to the significant architectural changes, this also
breaks with any backwards compatibility to existing
launcher setups. Thus, users need to migrate by re-applying
the new configuration format and migrating their config
parameters accordingly.
Further migration instructions and updated documentation
will be provided upon release.
Co-authored-by: Shiz <hi@shiz.me>
Keep this a separate commit because this also removes
inject's own logging engine and replaces it with the
streamlined core API. The core API provides all the
features of inject's own logging engine which also
performed horribly. The entire logging operation
was locked which included expensive operations
that formatted the log messages and required
memory allocations and copying around data.
The core API's implementation at least only
synchronizes the actual IO operations
(though this can be improved further with an
actual async logging sink, TBD)
The log API stopped scaling already a while ago and needs
considerable refactoring to consider the various use-cases
that emerged since it was first created on alpha versions
of bemanitools.
Boils down to:
- Include headers
- Reduce boiler plate with helpers
- Swap out explicit usages with core API layer
and ensure the right API is configured beforehand
This module contains the "core" (API) of
bemanitools which includes an abstraction
layer for threads and logging at this time.
The threads API is very close to what
util/thread already was with some structural
enhancements which make it easier to understand
and work with the API, I hope. Some additional
helpers (*-ext module) support in doing common
tasks, e.g. setting up the thread API with other
modules.
The log(ging) part receives a major overhaul to
address known limitations and issues with the
util/log module:
- Cleaner API layer
- Separate sinks from actual logging engine
- Sinks are composable
- Improved and cleaner compatibility layer
with AVS logging API
Additional "extensions" (*-ext modules) add
various helper functions for common tasks like
setting up the logging engine with a file and stdout
sink.
The sinks also improved significantly with the file
sink now supporting proper appending and log rotation.
Logging to stdout/stderr supports coloring of log
messages which works across logging engines.
Overall, this refactored foundation is expected to
support future developments and removes known
limitations at the current scale of bemanitools such as:
- Reducing boiler plate code across hooks
- Interop of bemanitools and AVS (and setting the foundation
for addressing currently missing interop, e.g. for
dealing with property structures without AVS)
- Addressing performance issues in the logging engine
due to incorrect interop with AVS
Improve the development experience by providing
an additional docker container that can be started
and used as an interactive development environment.
It provides all the tools and a stable environment
for building (identical to the build container).
Move everything to new launcher.xml configuration
files. Adjust the bootstrapping of launcher in the
.bat files. Features such as copying the default
props/ files to nvram are now handled by launcher.
Using the PATH variable, bemanitools binaries can
live in their own dedicated bemanitools/ subfolder
next to props/ and modules/ now. All original
binaries are expected to be kept in a modules/
folder like on stock data.
Kudos to Shiz for providing the groundwork for this.
Fundamentally re-think how launcher operates and
bootstrapping the games is managed and configured.
This brings it significantly closer to how the original
bootstrap is doing the job: launcher now utilizes the
data (structures) provided by the bootstrap.xml configuration
file. This creates compatibility with vanilla data dumps
and original stock images. Note that bemanitools does not
include any code or means to run DRM'd data, only decrypted.
But, this allows users to keep decrypted dumps as stock as
possible which means:
* No copying around of property files anymore
* Keep the modules/ folder with the binaries
* Have bemanitools binaries separate in the data
* No need to edit/customize the original configuration files
A list of key features of the "new" launcher:
* Boostrap games by following the configuration provided by
stock game's bootstrap.xml files
* Custom launcher.xml configuration file that adds further
launcher configurable features, composability of
bootstrap.xml configuration(s) as well as configuration
overriding/stacking of selected types of configurations,
e.g. eamuse config, avs-config. The latter eliminates
the need for modifying stock config files in the prop/
folder
* Unified logging system: launcher and AVS logging uses
the same logger, all output can now be in a single file
* Original features such as various hook types still
available
Due to the significant architectural changes, this also
breaks with any backwards compatibility to existing
launcher setups. Thus, users need to migrate by re-applying
the new configuration format and migrating their config
parameters accordingly.
Further migration instructions and updated documentation
will be provided upon release.
Co-authored-by: Shiz <hi@shiz.me>
Keep this a separate commit because this also removes
inject's own logging engine and replaces it with the
streamlined core API. The core API provides all the
features of inject's own logging engine which also
performed horribly. The entire logging operation
was locked which included expensive operations
that formatted the log messages and required
memory allocations and copying around data.
The core API's implementation at least only
synchronizes the actual IO operations
(though this can be improved further with an
actual async logging sink, TBD)
Move everything to new launcher.xml configuration
files. Adjust the bootstrapping of launcher in the
.bat files. Features such as copying the default
props/ files to nvram are now handled by launcher.
Using the PATH variable, bemanitools binaries can
live in their own dedicated bemanitools/ subfolder
next to props/ and modules/ now. All original
binaries are expected to be kept in a modules/
folder like on stock data.
Kudos to Shiz for providing the groundwork for this.
Fundamentally re-think how launcher operates and
bootstrapping the games is managed and configured.
This brings it significantly closer to how the original
bootstrap is doing the job: launcher now utilizes the
data (structures) provided by the bootstrap.xml configuration
file. This creates compatibility with vanilla data dumps
and original stock images. Note that bemanitools does not
include any code or means to run DRM'd data, only decrypted.
But, this allows users to keep decrypted dumps as stock as
possible which means:
* No copying around of property files anymore
* Keep the modules/ folder with the binaries
* Have bemanitools binaries separate in the data
* No need to edit/customize the original configuration files
A list of key features of the "new" launcher:
* Boostrap games by following the configuration provided by
stock game's bootstrap.xml files
* Custom launcher.xml configuration file that adds further
launcher configurable features, composability of
bootstrap.xml configuration(s) as well as configuration
overriding/stacking of selected types of configurations,
e.g. eamuse config, avs-config. The latter eliminates
the need for modifying stock config files in the prop/
folder
* Unified logging system: launcher and AVS logging uses
the same logger, all output can now be in a single file
* Original features such as various hook types still
available
Due to the significant architectural changes, this also
breaks with any backwards compatibility to existing
launcher setups. Thus, users need to migrate by re-applying
the new configuration format and migrating their config
parameters accordingly.
Further migration instructions and updated documentation
will be provided upon release.
Co-authored-by: Shiz <hi@shiz.me>
Keep this a separate commit because this also removes
inject's own logging engine and replaces it with the
streamlined core API. The core API provides all the
features of inject's own logging engine which also
performed horribly. The entire logging operation
was locked which included expensive operations
that formatted the log messages and required
memory allocations and copying around data.
The core API's implementation at least only
synchronizes the actual IO operations
(though this can be improved further with an
actual async logging sink, TBD)
The log API stopped scaling already a while ago and needs
considerable refactoring to consider the various use-cases
that emerged since it was first created on alpha versions
of bemanitools.
Boils down to:
- Include headers
- Reduce boiler plate with helpers
- Swap out explicit usages with core API layer
and ensure the right API is configured beforehand
This module contains the "core" (API) of
bemanitools which includes an abstraction
layer for threads and logging at this time.
The threads API is very close to what
util/thread already was with some structural
enhancements which make it easier to understand
and work with the API, I hope. Some additional
helpers (*-ext module) support in doing common
tasks, e.g. setting up the thread API with other
modules.
The log(ging) part receives a major overhaul to
address known limitations and issues with the
util/log module:
- Cleaner API layer
- Separate sinks from actual logging engine
- Sinks are composable
- Improved and cleaner compatibility layer
with AVS logging API
Additional "extensions" (*-ext modules) add
various helper functions for common tasks like
setting up the logging engine with a file and stdout
sink.
The sinks also improved significantly with the file
sink now supporting proper appending and log rotation.
Logging to stdout/stderr supports coloring of log
messages which works across logging engines.
Overall, this refactored foundation is expected to
support future developments and removes known
limitations at the current scale of bemanitools such as:
- Reducing boiler plate code across hooks
- Interop of bemanitools and AVS (and setting the foundation
for addressing currently missing interop, e.g. for
dealing with property structures without AVS)
- Addressing performance issues in the logging engine
due to incorrect interop with AVS
Improve the development experience by providing
an additional docker container that can be started
and used as an interactive development environment.
It provides all the tools and a stable environment
for building (identical to the build container).
# IIDX "Syncbook": A guide to ensuring sync and performant gameplay
This is a guide applicable to all IIDX versions that are native Windows PC-based. The goal is to
enable proper configuration of those games to ensure the game engine "runs in-sync" with any of its
IO (visual GPU/display output, audio output, user input) resulting in high and stable performance
for smooth game-play.
This document won't go into the implementation details of the game engine. Please refer to the
[IIDX engine details journal](../dev/journal/2025-02-09-iidx-engine.md) for a more in-depth
information.
## Understanding the problems and options to address them
First, let's understand what the key problems are and how they contribute to different issues
that impact game-play.
### Song de-syncing
Symptom: Throughout playing a song, the audio is de-syncing even it was fine when the song started.
Cause: The game engine is fundamentally designed to run on a fixed and consistent frame
rate/time per engine step. A mismatch of expected frame time by the game engine and actually delivered
frame time by the GPU driver. This can come either as a stable frame time that just mismatches in
general or (highly) fluctuating frame times.
* G-sync on modern displays causes known fluctuations because it adjusts dynamically the monitor
refresh rate
* V-sync not correctly configured/enforced or even vsync-off override in driver is causing
inconsistent frame times
* Incorrect display timings causing refresh rate fluctuations resulting in inconsistent frame times
Stable frame times but not having the correct ones that the game engine expects is a different
problem that can be addressed by:
* Knowing the refresh rates expected by the game engine of the different game versions
* Configuring your GPU driver's display timings to match the expected frame times
* Or, patch the game's chart data with your custom display timings so the game engine gets uses
*your* "correct" frame times
* Note: Offline patching the chart data doesn't work for several game versions as the monitor
check does override this data, but not as flexible as required
* Use the chart patch feature of bemanitools instead
### Micro-stuttering and non-smooth note scrolling
This is a trickier problem as it can be caused by a few different things.
* GPU related
* Fluctuating refresh rates result in fluctuating frame times in the main game loop
* GPU power configuration in non-performance mode (e.g. "power saving") causes processing latency
due to sleep states
* Display related
* Display firmware buggy and/or internal scalers that expect to run at very specific refresh rates
and cannot adapt to any custom display timings enforced by the GPU
* A known example is the official
[PENDUAL LCD monitor](https://github.com/shizmob/arcade-docs/blob/6ac99975cdb2bf668362f65fa9fa3ffb2127308b/konami/product/GULDJ-JI.md#notes-and-known-issues) that does not v-sync correctly to any other refresh-rate than 60.000 hz
* CPU related
* CPU power configuration on Windows in non-performance mode (e.g. "power saving") causes
processing latency due to sleep states
* Correct CPU C-states configuration in BIOS/UEFI
* These cannot be configured in Windows and are separate from Windows's power configuration
* Other stuff running in the background of the system causing fluctuations
* (Synchronous) Disk I/O heavy applications stealing CPU time
* Or any other process creating a considerable amount of CPU or GPU load
## Configuration guide for different game versions
### Measuring monitor refresh rate
Let's start with configuring the system to meet the game engine's expectation regarding the
target monitor refresh rate and requiring vsync.
First, use the [d3d9-monitor-check tool](../tools/d3d9-monitor-check.md) to measure the currently
configured refresh rate of your monitor. Note that your monitor might yield different timings on
different resolutions and different game versions run on
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
