Leahnaya/bemanitools
1
0
Fork 0
mirror of https://github.com/djhackersdev/bemanitools.git synced 2026-09-30 18:46:34 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity

chore: Apply formatting to all markdown docs

Browse Source
This commit is contained in:
icex2
2024-01-29 23:18:21 +01:00
committed by icex2
parent 0897a25174
commit 0a29e031ff
75 changed files with 3200 additions and 2786 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
doc/dev/README.md
View File

@@ -1,3 +1,4 @@
# Development documentation and notes
This folder contains various lose documentation snippets created by the developers. These can be
helpful as future reference.
helpful as future reference.

17
doc/dev/device-list.md
View File

@@ -6,21 +6,20 @@
Separate boxed unit containing with one card reader slot and pin key pad
- BeatmaniaIIDX DistorteD to Lincle: grey slotted readers hanging below the
side speakers next to the monitor
- DDR SN 1/2: slotted readers red/black supernova cover mounted to the side of
the cabinet next to the monitor (left and right)
- BeatmaniaIIDX DistorteD to Lincle: grey slotted readers hanging below the side speakers next to
the monitor
- DDR SN 1/2: slotted readers red/black supernova cover mounted to the side of the cabinet next to
the monitor (left and right)
# ICCB
Single card reader slot (no separate pin pad) built into the cabinet.
Pin entry using game controls.
Single card reader slot (no separate pin pad) built into the cabinet. Pin entry using game controls.
- (First gen) Jubeat cabinets before replaced with wave pass readers
# ICCC
Single card reader wave pass unit without separate pin pad. Pin entry using
game controls. Still supported by newer versions.
Single card reader wave pass unit without separate pin pad. Pin entry using game controls. Still
supported by newer versions.
- (Second gen) Jubeat cabinets with wave pass readers
- (Second gen) Jubeat cabinets with wave pass readers

101
doc/dev/journal/2018-02-10-logging-breakdown-avs.md
View File

@@ -2,75 +2,116 @@ Copy/pasted from chat with tau (2018/02/10):
alright then. so for modern iidx.
we want to do logging inside iidxhook and we also want to pass AVS-style log functions to iidxio which in turn passes them on to geninput in order for those to do logging too
we want to do logging inside iidxhook and we also want to pass AVS-style log functions to iidxio
which in turn passes them on to geninput in order for those to do logging too
so iidxhook connects to the AVS log API: log_body_misc and friends, which I assume are invoked using a log_misc() macro in Konami's source code that adds some sort of module tag.
so iidxhook connects to the AVS log API: log_body_misc and friends, which I assume are invoked using
a log_misc() macro in Konami's source code that adds some sort of module tag.
anyway yeah this we already know.
libutil has four function ptrs: log_impl_misc and co. These are static variables which are statically initialized to some no-op functions. Except log_impl_fatal, whose implementation just calls libc abort()
libutil has four function ptrs: log_impl_misc and co. These are static variables which are
statically initialized to some no-op functions. Except log_impl_fatal, whose implementation just
calls libc abort()
at startup you call log_to_external(), supplying four function ptrs to wire these up to. As the name suggests, this causes Bemanitools libutil to talk to something that is compatible with the AVS log sink API.
at startup you call log_to_external(), supplying four function ptrs to wire these up to. As the name
suggests, this causes Bemanitools libutil to talk to something that is compatible with the AVS log
sink API.
alternatively you can log_to_writer(), which initializes Bemanitools to use its own, internal logging system, and you give it a log writer function that takes strings and writes them somewhere.
alternatively you can log_to_writer(), which initializes Bemanitools to use its own, internal
logging system, and you give it a log writer function that takes strings and writes them somewhere.
So you have log sinks and you have log writers. The path is [application code] -> [log sink] -> [logging engine] -> [log writer]
19:29
So you have log sinks and you have log writers. The path is \[application code\] -> \[log sink\] ->
\[logging engine\] -> \[log writer\] 19:29
inside config.exe (or generally outside of modern AVS games) this path looks like [bemanitools application code] -> [log sinks passed across dlls] -> [bemanitools logging engine] -> [bemanitools log writer]
inside config.exe (or generally outside of modern AVS games) this path looks like \[bemanitools
application code\] -> \[log sinks passed across dlls\] -> \[bemanitools logging engine\] ->
\[bemanitools log writer\]
inside modern AVS game the path looks like [bt hook dll / bt iodev dll] -> [avs log_body_whatever log sinks] -> [avs logging engine] -> [launcher.exe log writer]
inside modern AVS game the path looks like \[bt hook dll / bt iodev dll\] -> \[avs log_body_whatever
log sinks\] -> \[avs logging engine\] -> \[launcher.exe log writer\]
note that I tried to keep the log writer API consistent with the AVS log writer API but then Konami went and broke it repeatedly so now Bemanitools has its own stable log writer API. Launcher tracks the AVS log writer API, which breaks constantly, so that's not the same thing.
note that I tried to keep the log writer API consistent with the AVS log writer API but then Konami
went and broke it repeatedly so now Bemanitools has its own stable log writer API. Launcher tracks
the AVS log writer API, which breaks constantly, so that's not the same thing.
anyway that's the background story. Now for the details about IIDX in particular.
up until about iidx19 we did things the obvious way: iidxhook would log_to_external() to hook into the AVS log sinks and then call those directly and all was well. Then one fine day I was given a IIDX19 data dump and tried running iidxhook and it crashed with a stack overflow. hmm.
up until about iidx19 we did things the obvious way: iidxhook would log_to_external() to hook into
the AVS log sinks and then call those directly and all was well. Then one fine day I was given a
IIDX19 data dump and tried running iidxhook and it crashed with a stack overflow. hmm.
the problem boils down to this: iidx19 AVS added those log timestamps. And for for whatever reason the AVS logging engine needs to access some mutexes and condition variables to make this work properly
the problem boils down to this: iidx19 AVS added those log timestamps. And for for whatever reason
the AVS logging engine needs to access some mutexes and condition variables to make this work
properly
but AVS of course in grand Konami tradition has its own threading and concurrency primitive API which wraps the Win32 API. tbf this is kind of understandable in some sense, because win32 actually did not have condition variables until Windows Vista! in 2006! seriously, I'm not kidding.
but AVS of course in grand Konami tradition has its own threading and concurrency primitive API
which wraps the Win32 API. tbf this is kind of understandable in some sense, because win32 actually
did not have condition variables until Windows Vista! in 2006! seriously, I'm not kidding.
there's all sorts of articles out there describing in fine detail how to use Win32's event objects to implement your own condition variables and the multitudinous pitfalls that this entails
there's all sorts of articles out there describing in fine detail how to use Win32's event objects
to implement your own condition variables and the multitudinous pitfalls that this entails
but anyway one fun thing about the AVS concurrency API is that you can't actually use concurrency primitives unless you're calling that API from a thread launched using the AVS threading API
but anyway one fun thing about the AVS concurrency API is that you can't actually use concurrency
primitives unless you're calling that API from a thread launched using the AVS threading API
and that's a problem in the case of iidxhook, because iidx is old as balls relatively speaking and its EZUSB driver code has I think two worker threads, which it launches using the MS libc's _beginthreadex() function
and that's a problem in the case of iidxhook, because iidx is old as balls relatively speaking and
its EZUSB driver code has I think two worker threads, which it launches using the MS libc's
\_beginthreadex() function
this in turn is a wrapper around the win32 CreateThread function, but it also boots up stdio on whatever new thread gets launched and basically is responsible for guaranteeing that the libc will operate correctly on the newly launched thread
this in turn is a wrapper around the win32 CreateThread function, but it also boots up stdio on
whatever new thread gets launched and basically is responsible for guaranteeing that the libc will
operate correctly on the newly launched thread
so yeah when you do windows programming, never call CreateThread, always call _beginthreadex. otherwise stuff will break. maybe.
so yeah when you do windows programming, never call CreateThread, always call \_beginthreadex.
otherwise stuff will break. maybe.
point is, IIDX predates modern AVS so it just uses Windows threading directly. So, IIDX worker thread starts up, iidxhook does its thing, writes a log message, calls into AVS logging, which in turn grabs an AVS mutex, the implementation of which says "omg this isn't an AVS thread aaaaaa" and ... attempts to call back into the logging system to log this fact. whereupon a stack overflow condition proceeds in a predictable manner.
point is, IIDX predates modern AVS so it just uses Windows threading directly. So, IIDX worker
thread starts up, iidxhook does its thing, writes a log message, calls into AVS logging, which in
turn grabs an AVS mutex, the implementation of which says "omg this isn't an AVS thread aaaaaa" and
... attempts to call back into the logging system to log this fact. whereupon a stack overflow
condition proceeds in a predictable manner.
so, there are a few ways to deal with this problem. bemanitools 4 dealt with it in a fairly stupid way.
so, there are a few ways to deal with this problem. bemanitools 4 dealt with it in a fairly stupid
way.
and very elaborate way too
bt4 intercepted IIDX's calls to create Windows threads and then redirected those calls to go via the AVS threading API
bt4 intercepted IIDX's calls to create Windows threads and then redirected those calls to go via the
AVS threading API
so now the worker threads are AVS threads and logging works as expected
which is all well and good but the problem is that these threads are quite timing critical. it probably worked fine, but i didn't want to risk affecting those threads in a weird way and introducing latency and jitter. i wanted to keep the threading pristine and not mess with it just for the sake of diagnostic messages
which is all well and good but the problem is that these threads are quite timing critical. it
probably worked fine, but i didn't want to risk affecting those threads in a weird way and
introducing latency and jitter. i wanted to keep the threading pristine and not mess with it just
for the sake of diagnostic messages
so bemanitools 5 uses the log server approach
at an appropriate time, it creates its own AVS thread, the logging server. Since it is an AVS thread, it can call the AVS logging API
at an appropriate time, it creates its own AVS thread, the logging server. Since it is an AVS
thread, it can call the AVS logging API
and then we have log_post_misc() and friends, implemented in log-server.c
we initialize the Bemanitools logging system to log "externally" to those funcs and we also propagate those to iidxio.dll and eamio.dll which in turn pass them to geninput.dll
we initialize the Bemanitools logging system to log "externally" to those funcs and we also
propagate those to iidxio.dll and eamio.dll which in turn pass them to geninput.dll
so what do log_post_misc and friends do
they lock a "mailbox" using the win32 concurrency primitives and write the log severity and a pointer to the string to be logged into the mailbox, then signal the log server thread, again using win32 concurrency primitives
they lock a "mailbox" using the win32 concurrency primitives and write the log severity and a
pointer to the string to be logged into the mailbox, then signal the log server thread, again using
win32 concurrency primitives
then they do a synchronous wait for an acknowledgement from the logging server: since we're holding a string pointer, we cannot return until that string pointer has been consumed or it may be concurrently invalidated
then they do a synchronous wait for an acknowledgement from the logging server: since we're holding
a string pointer, we cannot return until that string pointer has been consumed or it may be
concurrently invalidated
so the log server wakes up, locks the mailbox, calls AVS log_body_misc() to write the log message, then once that returns it asserts a signal in the mailbox (again, win32 event object because lol what are condition variables) and releases the lock.
so the log server wakes up, locks the mailbox, calls AVS log_body_misc() to write the log message,
then once that returns it asserts a signal in the mailbox (again, win32 event object because lol
what are condition variables) and releases the lock.
the caller gets the signal, wakes up, and returns to whatever Bemanitools code is running on the IIDX IO worker thread that wanted to write a log message.
the caller gets the signal, wakes up, and returns to whatever Bemanitools code is running on the
IIDX IO worker thread that wanted to write a log message.
end of essay.

126
doc/dev/journal/2019-07-29-c02-driver-on-w7-crash.md
View File

@@ -1,68 +1,86 @@
# Follow-up (14th August 2019)
After publishing this post-mortem, I got messaged by a user on sows who was able to shed some more light on this issue.
The user was experiencing the same symptoms on a Win7 setup: blue screen once the firmware was flashed to the C02 IO.
This user's solutions was to use the USB2 ports on the PC instead of the USB3 ones. This is good to know and kinda
aligns with the weird things happening in the driver (see below).
After publishing this post-mortem, I got messaged by a user on sows who was able to shed some more
light on this issue. The user was experiencing the same symptoms on a Win7 setup: blue screen once
the firmware was flashed to the C02 IO. This user's solutions was to use the USB2 ports on the PC
instead of the USB3 ones. This is good to know and kinda aligns with the weird things happening in
the driver (see below).
# Post-mortem: C02 IO kernel module crash on Windows 7 (29th July 2019) by icex2
## Background
The original ezusbsys.sys kernel module, which is required to run the C02 IO, was compiled for Windows XP 32-bit, only.
There is a newer driver by Cypress, cyusb3.sys, which could be used to IO2 boards on newer Windows platforms, but does
not work with the C02 IO in combination with Konami's propriatery firmware. Thus, it was not possible to run the C02 IO
on anything than Windows XP 32-bit. But, with newer IIDX games running on Windows 7 64-bit, the C02 IO wasn't usable
anymore. Leaving aside, that the newer games actually require a BIO2 board and do not support C02 nor IO2 boards
anymore.
The original ezusbsys.sys kernel module, which is required to run the C02 IO, was compiled for
Windows XP 32-bit, only. There is a newer driver by Cypress, cyusb3.sys, which could be used to IO2
boards on newer Windows platforms, but does not work with the C02 IO in combination with Konami's
propriatery firmware. Thus, it was not possible to run the C02 IO on anything than Windows XP
32-bit. But, with newer IIDX games running on Windows 7 64-bit, the C02 IO wasn't usable anymore.
Leaving aside, that the newer games actually require a BIO2 board and do not support C02 nor IO2
boards anymore.
## The goal
I still wanted to use my cabinet with a C02 board on newer games which is possible with BT5 adding an emulation layer
and an interface (iidxio). This IO interface can be used to implement a driver that talks to a real IO again. Thus,
implementing a ezusb iidxio driver library, we can run newer games with an C02 IO as well.
However, there was no ezusbsys.sys driver that works on newer platforms required to run the newer games. But, Cypress
was nice and included the source code of the ezusbsys kernel module. With a few tweaks and a very recent version of
visual studio, it was quite easy to build this driver for newer platforms, including Windows 7, 8 and 10 in both
32-bit and 64-bit variants.
I still wanted to use my cabinet with a C02 board on newer games which is possible with BT5 adding
an emulation layer and an interface (iidxio). This IO interface can be used to implement a driver
that talks to a real IO again. Thus, implementing a ezusb iidxio driver library, we can run newer
games with an C02 IO as well.
However, there was no ezusbsys.sys driver that works on newer platforms required to run the newer
games. But, Cypress was nice and included the source code of the ezusbsys kernel module. With a few
tweaks and a very recent version of visual studio, it was quite easy to build this driver for newer
platforms, including Windows 7, 8 and 10 in both 32-bit and 64-bit variants.
## The problem
But, when using this driver on certain combinations of newer hardware (max. 1-2 years old) and Windows 7, the kernel
module might crash after the Konami C02 firmware got flashed to the ezusb board. The result was a bluescreen and reboot.
However, the hardware was fine and the kernel module worked fine on another piece of hardware, the stock PC that was
used with iidx 20 to 24. However, this hardware is not powerful enough to run iidx 25 and newer without stuttering
issues.
But, when using this driver on certain combinations of newer hardware (max. 1-2 years old) and
Windows 7, the kernel module might crash after the Konami C02 firmware got flashed to the ezusb
board. The result was a bluescreen and reboot.
However, the hardware was fine and the kernel module worked fine on another piece of hardware, the
stock PC that was used with iidx 20 to 24. However, this hardware is not powerful enough to run iidx
25 and newer without stuttering issues.
## The analysis/debugging
Note: The full source code can be found in the bemanitools-supplement package.
Setup:
* Native hardware with Windows 7 that was crashing
* Vmware with Windows 10 and Visual Studio 2019 to compile the kernel module. Target platform Windows 7 64-bit
* Booting Windows 7 in test mode to allow unsigned kernel modules to run and with debug output turned on
* dbgview on Windows 7 machine to get local kernel dbg output
Because I wanted to stick to Windows 7 in the beginning (refer to the solution section), I started debugging the kernel
module by enabling the debug message output that was already available in the code. However, since kernel debug message
printing can be very delayed, the kernel could not print various messages before the kernel crashed.
- Native hardware with Windows 7 that was crashing
- Vmware with Windows 10 and Visual Studio 2019 to compile the kernel module. Target platform
Windows 7 64-bit
- Booting Windows 7 in test mode to allow unsigned kernel modules to run and with debug output
turned on
- dbgview on Windows 7 machine to get local kernel dbg output
Thus, I started stripping the kernel module step by step to narrow down the possible spots causing the crash. After a
few hours, I got the (first) issue tracked down:
Because I wanted to stick to Windows 7 in the beginning (refer to the solution section), I started
debugging the kernel module by enabling the debug message output that was already available in the
code. However, since kernel debug message printing can be very delayed, the kernel could not print
various messages before the kernel crashed.
After the firmware was flashed, the device had to re-enumerate. When this happens, the function *Ezusb_PnPAddDevice*
is called to create a new instance of the device. Since this kernel module is acting as a filter driver, it has to
trap this call, and add a filter device before the real device in the device stack. Thus, each call to the ezusb device
hits the filter device first and the filter device calls the real device after doing some magic.
Thus, I started stripping the kernel module step by step to narrow down the possible spots causing
the crash. After a few hours, I got the (first) issue tracked down:
*Ezusb_PnPAddDevice* calls *Ezusb_CreateDeviceObject*. Afterwards, it checks the status of the call to
*Ezusb_CreateDeviceObject* and if successful, it tries attaching the device to the device stack. However, instead of
using *IoAttachDeviceToDeviceStackSafe* it uses the unsafe variant *IoAttachDeviceToDeviceStack* which can lead to a
race condition on newer Windows Systems. Furthermore, all initialization of further variables of the *deviceObject*
needs to happen BEFORE doing that. Again, this is a race condition.
After the firmware was flashed, the device had to re-enumerate. When this happens, the function
*Ezusb_PnPAddDevice* is called to create a new instance of the device. Since this kernel module is
acting as a filter driver, it has to trap this call, and add a filter device before the real device
in the device stack. Thus, each call to the ezusb device hits the filter device first and the filter
device calls the real device after doing some magic.
*Ezusb_PnPAddDevice* calls *Ezusb_CreateDeviceObject*. Afterwards, it checks the status of the call
to *Ezusb_CreateDeviceObject* and if successful, it tries attaching the device to the device stack.
However, instead of using *IoAttachDeviceToDeviceStackSafe* it uses the unsafe variant
*IoAttachDeviceToDeviceStack* which can lead to a race condition on newer Windows Systems.
Furthermore, all initialization of further variables of the *deviceObject* needs to happen BEFORE
doing that. Again, this is a race condition.
Next issue: Once the kernel calls *Ezusb_StartDevice* -> *Ezusb_ConfigureDevice* ->
*Ezusb_SelectInterfaces*, it tries to use *USBD_ParseConfigurationDescriptorEx* to get the interface
from the configuration descriptor. However, that fails for some unknown reason. I checked the data
structure and it is perfectly fine and everything is there. Thus, I wrote my own version
*Ezusb_GetInterfaceFromConfigurationDescriptor* which does all the magic required to get this part
fixed:
Next issue: Once the kernel calls *Ezusb_StartDevice* -> *Ezusb_ConfigureDevice* -> *Ezusb_SelectInterfaces*, it tries
to use *USBD_ParseConfigurationDescriptorEx* to get the interface from the configuration descriptor. However, that
fails for some unknown reason. I checked the data structure and it is perfectly fine and everything is there. Thus,
I wrote my own version *Ezusb_GetInterfaceFromConfigurationDescriptor* which does all the magic required to get this
part fixed:
```
PUSB_INTERFACE_DESCRIPTOR Ezusb_GetInterfaceFromConfigurationDescriptor(
IN PUSB_CONFIGURATION_DESCRIPTOR ConfigurationDescriptor
@@ -89,15 +107,17 @@ PUSB_INTERFACE_DESCRIPTOR Ezusb_GetInterfaceFromConfigurationDescriptor(
}
```
And next issue is just up ahead: Following the above, we have to call *Ezusb_USBD_CreateConfigurationRequestEx* to
create a USB configuration request to set the interface we want to use. This is executed with a *Ezusb_CallUSBD* call
which sends request to the real hardware. However, this request always fails. The call *IoCallDriver* inside
*Ezusb_CallUSBD* always returns an NTSTATUS code that is not documented anywhere (can't find the exact status code
anymore, but once you get it, try to find it in the header file).
And next issue is just up ahead: Following the above, we have to call
*Ezusb_USBD_CreateConfigurationRequestEx* to create a USB configuration request to set the interface
we want to use. This is executed with a *Ezusb_CallUSBD* call which sends request to the real
hardware. However, this request always fails. The call *IoCallDriver* inside *Ezusb_CallUSBD* always
returns an NTSTATUS code that is not documented anywhere (can't find the exact status code anymore,
but once you get it, try to find it in the header file).
At this point, I had to give up. I already wasted too many hours and this is clearly a dead end.
## The solution
Once I realized that I got stuck with Windows 7 and I didn't want to buy (more) new hardware, I gave Windows 10 a try.
Surprisingly, this solved all the issues and the kernel module runs fine. The C02 board is flashable without crashing
and works with newer IIDX games.
Once I realized that I got stuck with Windows 7 and I didn't want to buy (more) new hardware, I gave
Windows 10 a try. Surprisingly, this solved all the issues and the kernel module runs fine. The C02
board is flashable without crashing and works with newer IIDX games.

82
doc/dev/journal/2019-10-07-iidx-gfx-rendering-loops.md
View File

@@ -1,25 +1,31 @@
# Notes outlining some aspects of the DirectX calls which were required to know to implement a up-/downscaling feature
As the title says, this outlines some aspects I needed to figure out in order to implement a up-/downscaling feature
within the d3d9 hook module that works on all currently available IIDX versions (9 to 26).
To analyze the rendering loops, I have used a tool called apitrace which traces the calls of many graphic APIs:
https://github.com/apitrace/apitrace
As the title says, this outlines some aspects I needed to figure out in order to implement a
up-/downscaling feature within the d3d9 hook module that works on all currently available IIDX
versions (9 to 26).
Defintely recommended to quickly figure out what is going on regarding rendering. It also allows you to let you render
parts of a scene after the application exited because it records all API calls and data passed to them.
To analyze the rendering loops, I have used a tool called apitrace which traces the calls of many
graphic APIs: https://github.com/apitrace/apitrace
Anyway, considering the various iterations in (GPU) hardware the game had to undergo combined with weird quirks and
"fixes" Bemanitools is undoing, I wouldn't have guessed that their rendering engine was nearly the same until IIDX 20.
That's when they introduced SD/HD mode.
Defintely recommended to quickly figure out what is going on regarding rendering. It also allows you
to let you render parts of a scene after the application exited because it records all API calls and
data passed to them.
In this case, that's great news because I had to craft a solution that allows up-/downscaling the final frame to
different resolutions (see the iidxhook-util/d3d9 module for more details about the feature).
Anyway, considering the various iterations in (GPU) hardware the game had to undergo combined with
weird quirks and "fixes" Bemanitools is undoing, I wouldn't have guessed that their rendering engine
was nearly the same until IIDX 20. That's when they introduced SD/HD mode.
In this case, that's great news because I had to craft a solution that allows up-/downscaling the
final frame to different resolutions (see the iidxhook-util/d3d9 module for more details about the
feature).
But first, we need a breakdown of the render loop's most relevant parts for this:
## IIDX pre 20
Using apitrace, we can see the following outline of a frame (not counting the first one that does a lot of setup in
the beginning):
Using apitrace, we can see the following outline of a frame (not counting the first one that does a
lot of setup in the beginning):
```
BeginScene
Clear
@@ -31,12 +37,14 @@ Present
No render target switching, simply render everything to the back buffer...plain and simple.
Note: The viewport size is determined by the size returned by GetClientRect, wtf.
Welp, no official Konmai seal of approval without that. ¯\_(ツ)_/¯
Note: The viewport size is determined by the size returned by GetClientRect, wtf. Welp, no official
Konmai seal of approval without that. ¯\_(ツ)\_/¯
## IIDX 20+
Using apitrace, we can see the following outline of a frame (not counting the first one that does a lot of setup in
the beginning):
Using apitrace, we can see the following outline of a frame (not counting the first one that does a
lot of setup in the beginning):
```
BeginScene
// tex1 is a render target texture with size 1280x720 (also in SD mode)
@@ -63,26 +71,30 @@ EndScene -> ErrInvalidCall return code
Present
```
This is quite a different flow to implement HD and SD mode but the solution to solve that particular problem is straight
forward and easy to understand. This means that the game will always render in HD mode and only downscale the final
frame to SD resolution for 640x480 output.
This is quite a different flow to implement HD and SD mode but the solution to solve that particular
problem is straight forward and easy to understand. This means that the game will always render in
HD mode and only downscale the final frame to SD resolution for 640x480 output.
Also, why the fuck do they call BeginScene and EndScene twice? Looks like they wanted to do this in two separate scenes
for some reason. Checking the return values would have revealed to them that something's not right...lucky them that
this code works nevertheless.
Another Konmai seal of approval, a job well done. ¯\_(ツ)_/¯
Also, why the fuck do they call BeginScene and EndScene twice? Looks like they wanted to do this in
two separate scenes for some reason. Checking the return values would have revealed to them that
something's not right...lucky them that this code works nevertheless. Another Konmai seal of
approval, a job well done. ¯\_(ツ)\_/¯
## iidxhook's up-/downscaling solution
The initial solution simply hooked into BeginScene and EndScene and let the game render to an intermediate render
target texture. The texture was scaled according to the actual target frame buffer size before getting presented. This
solution worked fine for pre IIDX 20 games but created a black screen on IIDX 20+.
In order to avoid two different scaling flows, the final solution that works for both does the following:
* Create a render target texture with native resolution and let the game render to it
* Set the render target to that intermediate render target texture on BeginScene
* Before Present
* Scale the intermediate render target texture to the back buffer
* Set the back buffer as the render target
* Present frame
The initial solution simply hooked into BeginScene and EndScene and let the game render to an
intermediate render target texture. The texture was scaled according to the actual target frame
buffer size before getting presented. This solution worked fine for pre IIDX 20 games but created a
black screen on IIDX 20+.
Just an outline which follows the actual implementation that you can find in the iidxhook-util/d3d9.
In order to avoid two different scaling flows, the final solution that works for both does the
following:
- Create a render target texture with native resolution and let the game render to it
- Set the render target to that intermediate render target texture on BeginScene
- Before Present
- Scale the intermediate render target texture to the back buffer
- Set the back buffer as the render target
- Present frame
Just an outline which follows the actual implementation that you can find in the iidxhook-util/d3d9.

37
doc/dev/journal/2020-12-16-acio-bio2-iidx-package-dump.md
View File

@@ -1,21 +1,25 @@
# ACIO BIO2 IIDX package dump
Package dump excerpt of the init sequence of a original Konami IIDX BIO2 with sub IO connected.
This was used to identify a missing piece of information that needs to be communicated to the BIO2
for IIDX to initialize the sub IO correctly.
Package dump excerpt of the init sequence of a original Konami IIDX BIO2 with sub IO connected. This
was used to identify a missing piece of information that needs to be communicated to the BIO2 for
IIDX to initialize the sub IO correctly.
The dump was cut off after two polls as the sequence just keeps on repeating from that point on.
## Findings for problem to solve
With the previous implemention of the BIO2 driver, which was created off references of SDVX KFCA,
a BIO2 used with IIDX and the sub IO (to upgrade older C02, IO2 cabinets) connected didn't
initialize properly. This resulted in no inputs/outputs other than 14 keys working.
With the previous implemention of the BIO2 driver, which was created off references of SDVX KFCA, a
BIO2 used with IIDX and the sub IO (to upgrade older C02, IO2 cabinets) connected didn't initialize
properly. This resulted in no inputs/outputs other than 14 keys working.
The problem identified was a different byte, exact meaning not known, that is sent in
[exchange 4](#exchange-4-ac-io-cmd-clear). Instead of `0x3B` from the SDVX KFCA based
implementation, it needs to be set to `0x2D`.
## Exchange 1: AC_IO_CMD_ASSIGN_ADDRS
### Write
```text
AA 00 00 01 00 01 00 02
@@ -29,6 +33,7 @@ data: 00
```
### Read
```
AA AA 00 00 01 00 01 01 03
@@ -43,7 +48,9 @@ AA: SOF
```
## Exchange 2: AC_IO_CMD_GET_VERSION
### Write
```
AA 01 00 02 00 00 03
@@ -56,6 +63,7 @@ AA: SOF
```
### Read
```
AA AA 81 00 02 00 2C 0D 06 00 00 ...
@@ -70,7 +78,9 @@ XX: checksum
```
## Exchange 3: AC_IO_CMD_START_UP
### Write
```
AA 01 00 03 00 00 04
@@ -83,6 +93,7 @@ AA: SOF
```
### Read
```
AA AA 81 00 03 00 01 00 85
@@ -97,7 +108,9 @@ AA: SOF
```
## Exchange 4: AC_IO_CMD_CLEAR
### Write
```
AA 01 01 00 00 01 2D 30
@@ -111,6 +124,7 @@ AA: SOF
```
### Read
```
AA AA 81 01 00 00 01 00 83
@@ -125,7 +139,9 @@ AA: SOF
```
## Exchange 5: BIO2_BI2A_CMD_WATCHDOG
### Write
```
AA 01 01 20 00 02 00 00 24
@@ -139,6 +155,7 @@ AA: SOF
```
### Read
```
AA AA 81 01 20 00 01 00 A3
@@ -153,7 +170,9 @@ A3: checksum
```
## Exchange 6: BIO2_BI2A_CMD_POLL
### Write
```
AA 01 01 52 00 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 84
@@ -167,6 +186,7 @@ AA: SOF
```
### Read
```
AA AA 81 01 52 00 2E 00 00 B0 00 F0 00 F0 F0 00 00 00 00 00 02 00 5F 11 FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 F3
@@ -181,12 +201,15 @@ F3: checksum
```
## Exchange 7: BIO2_BI2A_CMD_POLL
### Write
```
AA 01 01 52 00 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 84
```
### Read
```
AA AA 81 01 52 00 2E 00 00 B0 00 F0 00 F0 F0 00 00 00 00 00 02 00 64 11 FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 F8
```
```

37
doc/dev/journal/2023-04-03-d01-ezusb-io-boot-security.md
View File

@@ -1,30 +1,29 @@
# Beatmania IIDX 10th Style - D01 IO boot code and security init
Date: 2023-04-03
Author: icex2
Date: 2023-04-03 Author: icex2
Documenting decompiled and reverse engineered code snippets from the D01 JAE `bm2dx.exe`. These
helped me figuring out the two different security boot modes the game supports.
In summary, it supports booting with the C02 IO with a C02 black dongle and a D01 IO board with
a D01 dongle. No other combination is valid because they didn't make sense back then. You either
had an upgraded old style/twinkle cabinet to C02 (GEC02) or you bought a new dedicated cabinet
(GQD01). With 10th style supporting the old C02 dongle, it appears that all owners of a C02 cabinet
with IO board and C02 dongle received a free software update/HDD. This might make sense considering
the short life span of C02 and the game being super buggy, especially in earlier/initial revisions.
In summary, it supports booting with the C02 IO with a C02 black dongle and a D01 IO board with a
D01 dongle. No other combination is valid because they didn't make sense back then. You either had
an upgraded old style/twinkle cabinet to C02 (GEC02) or you bought a new dedicated cabinet (GQD01).
With 10th style supporting the old C02 dongle, it appears that all owners of a C02 cabinet with IO
board and C02 dongle received a free software update/HDD. This might make sense considering the
short life span of C02 and the game being super buggy, especially in earlier/initial revisions.
The game expects the following "configurations" from bemanitools:
* Booting as upgraded C02 with free D01 upgrade
* `sec.boot_version=GEC02 `
* `sec.boot_seeds=0:0:1`
* `sec.black_plug_mcode=GEC02JAA`
* "D01 IO pin" on IO board not active
* Booting as dedicated
* `sec.boot_version=GEC02 `
* `sec.boot_seeds=0:1:1`
* `sec.black_plug_mcode=GQD01JAA`
* "D01 IO pin" on IO board ACTIVE
- Booting as upgraded C02 with free D01 upgrade
- `sec.boot_version=GEC02 `
- `sec.boot_seeds=0:0:1`
- `sec.black_plug_mcode=GEC02JAA`
- "D01 IO pin" on IO board not active
- Booting as dedicated
- `sec.boot_version=GEC02 `
- `sec.boot_seeds=0:1:1`
- `sec.black_plug_mcode=GQD01JAA`
- "D01 IO pin" on IO board ACTIVE
All the above was derived from reading and understanding the documented code excerpts below.
@@ -286,4 +285,4 @@ int io_init_security()
set_io_error_type(-1, aNgSecurity);
return -1;
}
```
```

25
doc/dev/journal/2023-04-13-iidx-24-gfx-upscaling.md
View File

@@ -1,18 +1,17 @@
# IIDX 24 GFX upscaling notes
Date: 2023-04-15
Author: icex2
Date: 2023-04-15 Author: icex2
Notes about my work on fixing the upscaling/downscaling feature of bemanitools for IIDX 20 to 26.
I realized that the render backend changed significantly that the old method that worked fine
doesn't work anymore.
Notes about my work on fixing the upscaling/downscaling feature of bemanitools for IIDX 20 to 26. I
realized that the render backend changed significantly that the old method that worked fine doesn't
work anymore.
The tool used in the screenshots is [apitrace](https://github.com/apitrace/apitrace).
## IIDX 24
The GFX engine in IIDX from 20 to 26 has a changed render loop that includes built-in scaling to
implement the SD and HD/HD* screen settings that are selectable in the operator menu
implement the SD and HD/HD\* screen settings that are selectable in the operator menu
### Frame 0 - GFX init part
@@ -28,15 +27,15 @@ Start the scene and set the render target to the intermediate texture.
![](2023-04-13-iidx-24-gfx-upscaling/beginscene.png)
After done drawing the scene, the intermediate texture is blended to the framebuffer. With a target
2D plane having the size of the target resolution, the blending applies linear scaling to either
up- or downscale the final image.
2D plane having the size of the target resolution, the blending applies linear scaling to either up-
or downscale the final image.
![](2023-04-13-iidx-24-gfx-upscaling/scaling.png)
## IIDX 10
A recap of the old stuff, see also [my previous notes](2019-10-07-iidx-gfx-rendering-loops.md),
as I had to look at everything again to properly understand the differences.
A recap of the old stuff, see also [my previous notes](2019-10-07-iidx-gfx-rendering-loops.md), as I
had to look at everything again to properly understand the differences.
### Frame 0 - GFX init part
@@ -47,11 +46,11 @@ buffer.
### Frame 1 - A clean main render path
Beginning the scene excerpt. The viewport needs to match the target resolution to display the
final image correctly.
Beginning the scene excerpt. The viewport needs to match the target resolution to display the final
image correctly.
![](2023-04-13-iidx-24-gfx-upscaling/beginscene10.png)
Ending the scene excerpt, nothing fancy here, just swapping the back buffer.
![](2023-04-13-iidx-24-gfx-upscaling/endscene10.png)
![](2023-04-13-iidx-24-gfx-upscaling/endscene10.png)

183
doc/dev/journal/2023-05-28-ddr-p3io-driver-work.md
View File

@@ -1,7 +1,6 @@
# DDR p3io driver work, various notes
Date: 2023-05-28
Author: icex2
Date: 2023-05-28 Author: icex2
Notes about my work on writing a DDR P3io driver.
@@ -13,114 +12,116 @@ PCB breakout interfaces/connectors on the side of the boards.
Descriptions assume cabinet hardware of an upgraded DDR "black SD cabinet"
* `LINE OUT 1`: Primary audio out to amplifier
* `LINE OUT 2`: N.C.
* `RGB`: Video out to monitor, supports 15khz/31khz based on hardware switch for video freq/mode selection
* `COM1`: Virtual com port that connects to the EXTIO
* `COM2`: Virtual com port that connects to a pair of ICCA card readers
* `LAN`: Network connection
* `PWR`: 100V power in
* `ANALOG`: N.C.
* `PORT 1`: Lights output for cabinet lights, e.g. menu button lights, top header lights
* `PORT 2`: N.C.
* `DIPSW`
* `1`: ???
* `2`: On = force all sensor polling mode and allow running the game without an EXTIO
* `3`: ???
* `4`: On = force 15 khz monitor output
* `PLUG 1`: Black dongle
* `PLUG 2`: White dongle
- `LINE OUT 1`: Primary audio out to amplifier
- `LINE OUT 2`: N.C.
- `RGB`: Video out to monitor, supports 15khz/31khz based on hardware switch for video freq/mode
selection
- `COM1`: Virtual com port that connects to the EXTIO
- `COM2`: Virtual com port that connects to a pair of ICCA card readers
- `LAN`: Network connection
- `PWR`: 100V power in
- `ANALOG`: N.C.
- `PORT 1`: Lights output for cabinet lights, e.g. menu button lights, top header lights
- `PORT 2`: N.C.
- `DIPSW`
- `1`: ???
- `2`: On = force all sensor polling mode and allow running the game without an EXTIO
- `3`: ???
- `4`: On = force 15 khz monitor output
- `PLUG 1`: Black dongle
- `PLUG 2`: White dongle
### P3IO GF/DM
Descriptions assume usage of a P3IO from a GF/DM in a "chimera style PCB build" on an upgraded
DDR "black SD cabinet"
Descriptions assume usage of a P3IO from a GF/DM in a "chimera style PCB build" on an upgraded DDR
"black SD cabinet"
* `PWR`: 100V power in
* `12V-OUT`: +12V out for external devices
* `PORT 1`: Lights output for cabinet lights, e.g. menu button lights, top header lights
* `PORT 2`: N.C.
* `COM1`: To EXTIO
* `COM2`: To card readers
* `RGB`: Video out to monitor, supports 15khz/31khz based on hardware switch for video freq/mode selection
* `LINE OUT 1`: Primary audio out to amplifier
* `LINE OUT 2`: N.C.
* `DIPSW`
* `1`:
* `2`:
* `3`:
* `4`: On = force 15 khz monitor output
* `PLUG 1`: Roundplug black dongle
* `PLUG 2`: Roundplug white dongle
- `PWR`: 100V power in
- `12V-OUT`: +12V out for external devices
- `PORT 1`: Lights output for cabinet lights, e.g. menu button lights, top header lights
- `PORT 2`: N.C.
- `COM1`: To EXTIO
- `COM2`: To card readers
- `RGB`: Video out to monitor, supports 15khz/31khz based on hardware switch for video freq/mode
selection
- `LINE OUT 1`: Primary audio out to amplifier
- `LINE OUT 2`: N.C.
- `DIPSW`
- `1`:
- `2`:
- `3`:
- `4`: On = force 15 khz monitor output
- `PLUG 1`: Roundplug black dongle
- `PLUG 2`: Roundplug white dongle
### P3IO DDR(X)
Descriptions assume cabinet hardware of an upgraded DDR "black SD cabinet"
* `PWR`: 100V power in
* `USB`: USB memory card readers on cabinet
* `PORT 1`: Lights output for cabinet lights, e.g. menu button lights, top header lights
* `COM3-4`: Virtual COM ports
* COM3 (VCOM1): Pins 1,3,5 on connector -> To card readers
* COM4 (VCOM0): Pins 2,4,6 on connector -> N.C.
* `PLUG`: Breakout to security round plugs (black and white dongles)
* `COM1`: To EXTIO
* Pinout (pins left to right)
* 1: TXD1
* 2: RXD1
* 3: N.C.
* 4: N.C.
* 5: GND
* `COM2`: N/A (light spires on black HD cabinet)
* Pinout (pins left to right)
* 1: TXD2
* 2: RXD2
* 3: GND
* `RGB`: Video out to monitor, supports 15khz/31khz based on hardware switch for video freq/mode selection
* `LINE OUT1`: Primary audio out to amplifier
* `LINE OUT2`: N.C.
* `LAN`: Network
* `DIP SW`
* `1`:
* `2`:
* `3`:
* `4`: On = force 15 khz monitor output
- `PWR`: 100V power in
- `USB`: USB memory card readers on cabinet
- `PORT 1`: Lights output for cabinet lights, e.g. menu button lights, top header lights
- `COM3-4`: Virtual COM ports
- COM3 (VCOM1): Pins 1,3,5 on connector -> To card readers
- COM4 (VCOM0): Pins 2,4,6 on connector -> N.C.
- `PLUG`: Breakout to security round plugs (black and white dongles)
- `COM1`: To EXTIO
- Pinout (pins left to right)
- 1: TXD1
- 2: RXD1
- 3: N.C.
- 4: N.C.
- 5: GND
- `COM2`: N/A (light spires on black HD cabinet)
- Pinout (pins left to right)
- 1: TXD2
- 2: RXD2
- 3: GND
- `RGB`: Video out to monitor, supports 15khz/31khz based on hardware switch for video freq/mode
selection
- `LINE OUT1`: Primary audio out to amplifier
- `LINE OUT2`: N.C.
- `LAN`: Network
- `DIP SW`
- `1`:
- `2`:
- `3`:
- `4`: On = force 15 khz monitor output
## Python IO boards and differences
### P3IO DDR(X)
* Connects to USB and actually enumerates as a USB device and not a virtual COM port
* COM ports 1-4 on breakout of the PCB are being passed through as actual COM ports to the operating system
- Connects to USB and actually enumerates as a USB device and not a virtual COM port
- COM ports 1-4 on breakout of the PCB are being passed through as actual COM ports to the operating
system
### P3IO GF/DM
* Connects to USB and actually enumerates as a USB device and not a virtual COM port
* COM ports 1-2 on breakout of the PCB are just virtual COM ports
* These do not show up as COM ports on the operating system
* The game drives the COM ports through the main P3IO protocol with additional P3IO commands
to open, read, write and close these virtual COM ports
- Connects to USB and actually enumerates as a USB device and not a virtual COM port
- COM ports 1-2 on breakout of the PCB are just virtual COM ports
- These do not show up as COM ports on the operating system
- The game drives the COM ports through the main P3IO protocol with additional P3IO commands to
open, read, write and close these virtual COM ports
### P2IO DDR
* Connects to USB and actually enumerates as a USB device and not a virtual COM port
* COM ports 1-2 on breakout of the PCB are just virtual COM ports
* These do not show up as COM ports on the operating system
* The game drives the COM ports through the main P3IO protocol with additional P3IO commands
to open, read, write and close these virtual COM ports
- Connects to USB and actually enumerates as a USB device and not a virtual COM port
- COM ports 1-2 on breakout of the PCB are just virtual COM ports
- These do not show up as COM ports on the operating system
- The game drives the COM ports through the main P3IO protocol with additional P3IO commands to
open, read, write and close these virtual COM ports
## Pinout card reader P1 -> P2 mini din8 male to mini din8 male
Port 1 2 and 3:
Port 1 2 and 3:
* Pin 3: TX
* Pin 4: GND
* Pin 5 : RX
- Pin 3: TX
- Pin 4: GND
- Pin 5 : RX
Pin 3 and 5 need to be reversed inbetween readers
They are all the same, so your cable needs to bridge them over.
A standard male to male mini din 8 cable does not do that.
Pin 3 and 5 need to be reversed inbetween readers They are all the same, so your cable needs to
bridge them over. A standard male to male mini din 8 cable does not do that.
### Pinout card reader stock cable s-sub9 to round pin9
@@ -135,16 +136,16 @@ dsub-9 female -> mini-din8 male
Default or incorrectly configured?
* `COM1` -> on mainboard
* `COM2` -> COM2 on P3IO breakout
* `COM3` -> ???
* `COM4` -> COM1 on P3IO breakout
- `COM1` -> on mainboard
- `COM2` -> COM2 on P3IO breakout
- `COM3` -> ???
- `COM4` -> COM1 on P3IO breakout
## P3IO command init sequence on DDR 18
From the ddrio-python23 library
```text
````text
.data:1002D5D0 g_init_pakets db 0AAh, 2, 0, 1, 29h dup(0); field_0.field_0
.data:1002D5D0 ; DATA XREF: initialize_and_send_pakets+5↑o
.data:1002D5D0 db 0AAh, 2, 0, 2Fh, 29h dup(0); field_0.field_0
@@ -191,4 +192,4 @@ From the ddrio-python23 library
// 29: get video freq
// 5: set watchdog
// 27: get cab type or dipsw
```
````