So, I've seen information here and there about 21FX, though it's been heavily focused on its use for ROM dumping. What I'm more interested in is its use for development; for instance, a level editor that can edit the current level (loaded into RAM) during gameplay, allowing for rapid testing. But this leaves some questions I'm not really sure the answer to.
From what I understand, the 21FX overrides both reset and NMI vectors unconditionally. That is, every frame (once the game turns on NMIs), the SNES will jump to the IPLROM at $2184.
So, my questions are these:
1. When the USB cable is unplugged, the IPLROM jumps to the original reset vector at $fffc. This is meant to allow the game to run normally without having to remove the 21FX from the SNES, but since as far as I know $2184 is run every NMI as well, wouldn't this just cause the SNES to continually reset as soon as it hits the first NMI? Hopefully I'm just missing something here; have you tested this yet?
2. For my use case, I need the game to run normally except when I have data to send. Having the USB cable unplugged and only plugging it in when I need to send data would be very inconvenient (and unsuitable if unplugging between sets of data resets the console), so I need to send a jump command every frame I don't have data to get out of the IPLROM and into the normal NMI code. That said, how do I know when an NMI has happened to send the command? Would the read/write flags help with that?
3. It'd be helpful to be able to set which address it forces a jump to on NMI, for e.g. homebrew games running out of RAM, or else the 21FX would again have to receive a jump command every frame to jump to the game's NMI routine, or indeed just to keep the game running without getting stuck in the IPLROM. Is that at all feasible?
EDIT: Well, come to think of it, the third question's problem could be sidestepped if the homebrew game didn't actually activate NMIs, but instead just continuously checked the vblank flag in $4210. Though, that's not exactly ideal...
This (attn: byuu) thing is a fun new trend :D
> From what I understand, the 21FX overrides both reset and NMI vectors unconditionally.
Currently, it only overrides the reset vector. However, defparam has NMI/IRQ hooking working. The thought is to possibly offer it via $21fe writes to enable/disable them (they will break certain games, guaranteed.) We'll also want separate data loaded into $2184-21fd for each. The reset vector version alone requires all 122 bytes available, and would take too much time to do stuff with.
We'd probably want to discuss the design of NMI/IRQ hooking for a while before actually deploying it in a real device.
> When the USB cable is unplugged, the IPLROM jumps to the original reset vector at $fffc.
The IPLROM also tests the cable connection, so you can disconnect the USB cable post-poweron to start the game if you like.
> since as far as I know $2184 is run every NMI as well, wouldn't this just cause the SNES to continually reset as soon as it hits the first NMI?
It most certainly would, if we indiscriminately hooked NMI as well.
> Would the read/write flags help with that?
In a hypothetical reset+NMI+IRQ hooker, we could make $21fe reads have bits for "IRQ/NMI" acknowledge (one bit for each.)
> It'd be helpful to be able to set which address it forces a jump to on NMI
That's actually a really neat way of handling things. Given writes to $2184-21fd are ignored currently, we could easily use $21fa,b as NMI hijack address and $21fc,d as IRQ hijack address. I'll run that idea by defparam.
> EDIT: Well, come to think of it, the third question's problem could be sidestepped if the homebrew game didn't actually activate NMIs, but instead just continuously checked the vblank flag in $4210. Though, that's not exactly ideal...
The point of NMI/IRQ hijacking was for TASers and speedrunners to have ways to looking into the state of games they are actually playing (read: unmodified, retail SNES cartridges.) I don't think this is a smart way to handle things for homebrew development at all. Nor is it a personal goal of the 21fx idea. This is more in line with defparam's idea for the SNES-Tap. He was gracious enough to help make a sort of "limited SNES-Tap" (the 21fx) for my sake.
Instead, I would suggest your NMI routine poll $21fe to see if there's data available to be read. When there is, then you can execute your own custom routine to pull down data and do whatever.
Or alternatively, have the NMI routine write a "check status" command, then read back from the PC to see if it has some events pending, and act on them when you want to.
Really ... the sky is the limit with the 21fx. You can load level data dynamically (infinitely sized ROMs), pull data from and send data to the internet (netplay, online high scores, etc), link multiple SNES consoles together (as many as you want ... with a host computer), connect the audio in/out cables for audio mixing fun (infinite channel MSU1 audio), use your keyboard as a controller, load and store save game data, send debug messages, hook it up to an RPi with an LCD screen and have a "dual screen" game, use the PC as a glorified "3GHz coprocessor" for math computations, etc etc. Anything you can think of.
Even though I do emulate the device in higan, I doubt we'll see a lot of use out of it, though. The SNES expansion port connectors are too hard to source and require manual dremel work, so we're never going to see mass production be an option for these things =(
Gotcha, thanks for the response! A lot of my concerns came from the idea that I'd have to handle the IPLROM every frame, but if I can just deal with reads/writes myself in my NMI handler, that's no problem.
All of my concerns are pretty much taken care of then; setting the NMI handler override address was more of a side-concern as I don't actually plan to execute entirely out of RAM (in fact, I don't need to intercept NMIs/IRQs at all), but if that does get included, that's great too.
Unless you are keen on doing this through real hardware, you could just modify an open source emulator to do this kind of thing. I like hacking mednafen and bsnes/higan.
For instance you could write a new "hardware" port. For example, I did this to run the testing framework for my PCE engine. I created a new port that interpreted the first null-terminated string written to it as a filepath, and all following bytes are written directly to that file.
It's really fun hacking features into an emulator.
bazz wrote:
Unless you are keen on doing this through real hardware
Nintendo has been cracking down on YouTube videos of TASes of its games that aren't recorded from an authentic copy of the game on an authentic console.
bazz wrote:
Unless you are keen on doing this through real hardware, you could just modify an open source emulator to do this kind of thing.
Well, for me, programming for the SNES at all is entirely motivated by the cool factor
(It's certainly not practicality!)
For me, it's just not the same if I can't do it "for real", so to speak. Not to mention the 21fx is very cool itself.
tepples wrote:
Nintendo has been cracking down on YouTube videos of TASes of its games that aren't recorded from an authentic copy of the game on an authentic console.
TASes would be another good application given Nintendo being... well, Nintendo, though it's not something I'm personally interested in.
So I haven't spoken too much about NMI/IRQ/*-vector hooking because 1) The primary intention of 21FX is for cart dumping and cart research and 2) hooking non-reset vectors is a very experimental feature. As byuu put it, there is no 1-size fits all solution to hooking non-reset vectors for all carts, attempts to redirect program flow away and back to the cart will definitely cause lock up with some carts (especially with carts that have heavy DMA use, tight NMI routine timing and cart co-processor synchronization). Now that being said I still like the idea of hooking non-reset vectors for a couple of reasons (which may or may not interest some). Firstly, load and save state on console which would be a useful feature for speedrun practice. There are devices that can perform load/save state (i.e. GameSaver, SD2SNES rom hacks,etc...) but games with co-processors become a trickier problem (i.e SMRPG, Yoshi's Island) which may be possible on 21FX. Secondly, being able to analyze RAM values on console would be another speedrun practice feature. Emulator can do it all the same but input lag becomes a problem.
So my unreleased IPLROM does allows you to READ/WRITE/EXEC/GOTONMI upon hooking NMI, however it doesn't hook all NMI indiscriminately. The CPLD will search for NMI but will only override the NMI address if it sees there are commands waiting in buffer from the USB interface. This allows the user only to hook the vector if it is requesting access and otherwise leave the console execution completely alone. With this functionality I was able to perform RAM writes on SMW to affect gameplay with no softlocks. I was also able to save and load state for SMW in similar fashion to SNES-Tap. At the moment i'm trying to get the same functionality in SMRPG/Yoshi's Island. Dealing with SA-1/SuperFX at the moment to serialize its state.
Lastly, if the intention is to use the 21FX as a debug device or extension to a homebrew ROM then you absolutely don't need non-reset hooking to accomplish that. 21FX's bootloader IPLROM (byuu's rom) can work as an extension to your homebrew code to provide the 21FX as a USB interface for your ROM at runtime. Essentially in your ROM you can write directly to USB via $21FF in your NMI routine to wake up some host code and jump to $2184 to provide some host side functionality.
> tight NMI routine timing
The worst of it is IRQ timing. Some games use them as a poor man's HDMA for some reason, and invoke them every single scanline. I believe Battle Blaze is a good example of this. If you add even one extra opcode of overhead to these, you will completely break games that do this.
NMI alone, given it's once per frame, you'll probably be okay in 98% of games, so long as you don't use more than ~30-60 opcodes. After that, the breakage rate will start soaring upward. Especially on NTSC.
> Firstly, load and save state on console which would be a useful feature for speedrun practice.
I still don't really get how this can work. The SMP is pretty much a black box. There's no way to reset it at will, short of the horrendous /RESET line assertion, which would cause lots of other issues if you were trying to deserialize to a clean state. And there's definitely no way to read out its current execution state.
But... clearly, people have gotten it somewhat working, so it must be possible to some extent.
> The CPLD will search for NMI but will only override the NMI address if it sees there are commands waiting in buffer from the USB interface.
Hah, that's clever :D
Another idea is if you hook NMI for a really long time, you can make your hook wait until right before the next NMI to return. It may cause some horrible graphics glitches for that one frame, but theoretically the game should recover most of the time. Worst effect would probably be timed audio events desyncing slightly.
> Essentially in your ROM you can write directly to USB via $21FF in your NMI routine to wake up some host code and jump to $2184 to provide some host side functionality.
Another option would be to use the reset vector to shunt a special NMI routine somewhere else, like in some spare SRAM or WRAM space. That way you cut down on most of the overhead. The IPLROM's meant to be paranoid with a signature check and lots of register initialization stuff that's usually not needed.
> I still like the idea of hooking non-reset vectors for a couple of reasons
Yeah, it's definitely a neat idea with lots of potential :D
Just different usage categories.
But if you implement it into a rev E board of 21fx, then I'll support emulating the device in higan. Even if the goal is to use it on real hardware, it's immensely valuable to debug this sort of code under an emulator first :D
>I still don't really get how this can work. The SMP is pretty much a black box. There's no way to reset it at will, short of the horrendous /RESET line assertion, which would cause lots of other issues if you were trying to deserialize to a clean state. And there's definitely no way to read out its current execution state.
Yes, it is a black box
... There a couple of ideas I've had to perform some attempt of loading the SMP (like creating a shadow SMP to keep track) but honestly the effort isn't worth the function. Besides people who practice that use GameSaver seem to Tolerate the audio de-sync because in practice it doesn't matter too much if your runs aren't heavy in audio cues. GameSaver makes no attempt to save and load SMP state but it does have a system of "patch codes" when you use it on specific carts. This is because since it doesn't touch the SMP on save/load state it needs a way to preserve the area in RAM dedicated to CPU/SMP synchronization. These handshakes between processors use sequence numbers that cannot be overridden during a load state otherwise your audio control loop routines will go into infinite loop. The patch codes tell the device which parts of ram to not overwrite.
> Another idea is if you hook NMI for a really long time, you can make your hook wait until right before the next NMI to return. It may cause some horrible graphics glitches for that one frame, but theoretically the game should recover most of the time. Worst effect would probably be timed audio events desyncing slightly.
I agree, I believe one can leverage a lot of these tricks to regain stability in the cart execution, however I do believe hook implementations will be cart specific. Save and load state would have to be tweaked ever so slightly based on the cart name. If not for the timing differences, certainly for the support of serializing carts with co-processor state and critical SMP sync state.
> But if you implement it into a rev E board of 21fx, then I'll support emulating the device in higan. Even if the goal is to use it on real hardware, it's immensely valuable to debug this sort of code under an emulator first
Well I have been re-writing the schematic in Altium =P almost layout time
By the way, where does higan look for 21fx.rom and 21fx.so by default on Windows? Trying to figure out what interface->path(ID::System) and interface->path(ID::SuperFamicom) resolve to took me on a journey through higan's code that didn't really clear things up.
FYI, since I mentioned it above. Here is some results of NMI hooking for load/save state of RAM and SuperFX.
https://t.co/wDfukT6LbRhttps://t.co/VhwuNqdQpj
> By the way, where does higan look for 21fx.rom and 21fx.so by default on Windows?
The path system is a nightmare, yeah.
path(ID::System) refers to the Super Famicom.sys folder.
path(ID::SuperFamicom) refers to the Game.sfc folder.
Although in the future, I want to extend controllers to have their own folders, so I can support things like the Super TurboFile (has persistent storage.) Once that goes in, the 21fx.rom file will move there.
The idea was that the IPLROM was a fixed property of the device, and the program you run on it would change per game.
> FYI, since I mentioned it above. Here is some results of NMI hooking for load/save state of RAM and SuperFX.
That is super impressive! Amazing to see the SuperFX serialized on real hardware! :D
It wouldn't be 21FX without Super FX.