Using VS Code as an IDE for debugging assembly with source

[note that this topic is a bit of a fork from some previous discussion in the "bsnes-plus" thread here]

One thing I'm working on - in fits and starts - is trying to improve the debugging experience of SNES homebrew and patch development, especially by leveraging Visual Studio Code as an IDE of sorts, to provide an experience similar to debugging Python script or compiled C code. For example, being able to drop breakpoints in source ASM files, and step through the program counter and see the PC reflected in the source ASM, as demonstrated here:

https://www.youtube.com/watch?v=hkuMV-1LLmI

Status of this project as of Dec 16:

• v9.8 of wla-dx can now output the address-to-line data, and other source file information, required for mapping ROM addresses to file and line numbers. Both the spec for WLA's symbols, as well as information on how to generate the data in question, is described here.
  
  
• Head of Asar (point release TBD) can also output the same data.
  
  
• I'm working on local modifications to bsnes-plus to load in and process the symbols and source files, and accept many interactions over VS Code's Debug Adapter Protocol. WIP material is available here. This still has a lot of little things I need to address and fix. e.g. performance issues, code quality, and some occasional issues when using the emulator as a debugger (esp. with breakpoints). This is still a ways away from being prepped as a PR to Revenant/devinacker's bsnes-plus but I'm not looking to, like, fully implement the DAP before that point. I just want to get to a point where it's possible to comfortably debug 65816 ROM data.
  
  
• I'm working on an extension for VS Code to serve as a general tool for debugging external applications in VS Code. I have a repo for it, and the only thing it does right now is just kick an executable of your choice and establish a debug adapter connection with that exe. It is currently available on the VS Code marketplace, here: https://marketplace.visualstudio.com/it ... -app-debug. There are a lot of wishlist items I have on this extension that I'd like beyond just establishing that connection, though - for example, I think it should be possible to even have the extension create multiple views in VS Code for things memory watches from the external application.

One particular thing I wanted to create this topic for - besides promotion of this project - was also to ask a longer-form question from, perhaps, other folks here that are more experienced in patch development.

This is an issue I ran into while testing the Asar implementation of the addr-to-line data against a patch, specifically, which you can see demonstrated here (and I'll be referring back to for context). The assembly listing here is from dotsarecool's SMWPracticeCart.

The subroutine every_frame, is explicitly specified in the ASM to be located at PC 0x978000 (!_F is defined in patch.asm as 0x800000). In the symbol file, this lines up nicely, without a problem. The following is from the output symbols:



So, no problem. 0x978000 maps to every_frame.asm with instructions at line 5, 6, 7, and so on.

However, there's another case where the SMWPracticeCart patches existing, non-zero, code: hijacks.asm. In this case, it writes out new subroutine jumps starting at PC 0x809510, and this is reflected in the symbols file:



But, the original SMW ROM actually has the PC go not to address 0x809510, but rather 0x009510, due to fastrom/slowrom memory mapping and shadowing. The addr-to-line mapping doesn't have information to map 0x009510 to any source line, so the address doesn't resolve, even though that area in memory does have source line data to map to. Of course, setting !_F from 0x800000 to 0x000000 fixes the problem here: hijack's addr-to-line and label data points to 00:9510, and the mapping matches how the program is executing.

Theoretically, it should be possible to modify how the PC or other addresses are interpreted to try and guess at intended mappings (either when trying to get source line locations from addresses, or address from source line locations) but I can't help but imagine that that will come out the other end looking gnarlier than I'd like - it doesn't seem like it'd be possible to just mask all addresses by 0x7FFFFF and call it a day.

So, I want to try and get a read as to whether this is a "legitimate" issue to be concerned about: If a developer writes some code that is mapped to one location of ROM but executed at a different address in reality, due to shadowed memory banks, should they understand that they're doing something non-standard, and expect the symbols to not line up? Should the emulator attempt to bridge that gap automatically? Should it be recommended to just not have the fastrom offset be applied in a case like this? Is this even a common enough thing that it's worth being concerned about, i.e. is this only an issue when setting up entry points on ROM patches?

Would it be possible to use some sort of manifest to map each shadowed address to its canonical address? I know Mega Man X, for example, relies on shadowing.

Every game that executes out of banks $80-ff will run into this, because reset/NMI/IRQ vectors always jump to bank $00.

For most cartridges you could probably do that. Just give it a try.
And best keep some option in your source code (or an option in user interface) for disabling the feature if it doesn't work.
There are some special carts (with coprocessors etc) where it won't work, though you could detect such carts, and auto-disable the mirroring feature for them.

The main reason for using bank 00h instead of 80h is probably that exception vectors (reset/irq/nmi/etc) always point to bank 00h.
Small irq/nmi handlers might execute right in bank 00h, and larger handlers should manually far-jump to fast rom in bank 80h.
And don't forget hirom's also having exception vectors in bank 00h mirrored to bank 40h and C0h.

For fast rom source code, it might be best/easiest be to use bank 80h and up for the whole program (when doing it that way, exception vectors would be defined as being in bank 80h rather than bank 00h).
But people might do it this way, or another way. And, yeah, maybe some homebrew patches won't care about anything at all : )

I imagine that treating $000000-$7DFFFF and $800000-$FDFFFF the same should work for the vast majority of LoROM (mode $20/$30) and HiROM (mode $21/$31) carts. It won't work for the uncommon memory maps of ExHiROM (mode $25/$35, Tales of Phantasia) and coprocessor carts (as nocash mentioned), and there will be issues for battery RAM and $7E0000-$7FFFFF.

HiROM introduces another wrinkle that reset, NMI, and IRQ handlers will end up executed from $008000-$00FFFF as a mirror of $C08000-$C0FFFF.

Oh. Huh. Okay, this is definitely going on the feature list, then. Thanks for the info!


Well, digging a little bit more, it looks like (of course) bsnes has a bunch of internal support for memory mapping already set up, including definitions across a slew of different (all of the officially released?) chips: https://github.com/devinacker/bsnes-plu ... tridge.hpp

It'll take some investigation to see how/if I can piggyback off of that information, but that'll be my first check.

Edit: Maybe the existing code for breakpoint tests across mirrored memory would be an even better starting point

I'm not working on any snes project yet but any tools that will help remote debug from vs code will be greatly appreciated. Once it is becoming more concrete, I will be more than happy to try it.

Would love to have the same thing for ca65/cc65 too sometime ^^;;

Can work on assemblies? I use the WDC tool chain for the c compiler.

Having asar and wla-dx export the required data are the only things on my todo-list right now, just because those seemed to be the two most popular and actively-maintained assemblers out there.

As well, it'd be great to have this for a C-centric toolchain, but every time I look at the state of C toolchains for SNES I instantly find myself morose and miserable

At the risk of being that guy saying "Hey if you have a problem with this project, submit a PR!", I certainly won't have to be a bottleneck in getting that stuff up and running, in the case of someone wanting to stand up the support required to export WLA-style symbols in ca65 (or cc65). The only things that should be necessary will be having a tool that exports a .sym file with the appropriate file spec, which would be the to-do on whatever compiler/assembler toolchain you've got; and an emulator that reads in that .sym file (with support for the Debug Adapter Protocol) which I'm working on.

Well, it's pretty brute-force (e.g. address resolutions to labels and source lines all, each and every time, generate a list of all possible mirror addresses across the CPU bus) but this seems to work pretty well: https://i.imgur.com/brunZOZ.png.

Before, the callstack could not resolve the appropriate label for address 0x00951b because every_frame_hijack was supposed to be at 0x80951b, and, similarly, the source file/line number for 0x00951b could not have been determined, but now both are intact.

Even compiling the emulator in debug - though admittedly on a decent desktop PC - I can't glean an apparent performance loss when stepping through code due to this change given the afore-mentioned brute-force nature of it. In any case, there's obvious low-hanging fruit like caching potential mirrors, and that sort of thing, which could be done that would collapse any noted performance concerns.

In any case, I think cracking this nut may have been my only concern with the WIP stuff I had for Asar (even though technically that was present for the wla impl, too ) so next task will be polishing up the PR for that.

Just a small update (OP has been updated with current events). Prepped the PR for Asar, and decided to scrub the few nicks off of the VS Code extension so that others can use it.

So, if you're feeling intrepid and you do want to try this all out against whatever stuff you may already have:

• Fetch the respective Asar or wla-dx repo above.
• Build the assembler, point your build scripts to it, and rebuild your ROM to generate a companion .sym file.
• (if you use wla-dx, note the extra instructions you need to add to your ROM build script to generate symbols)
• Fetch my vscode-stackframes branch of bsnes-plus and build it.
• Fetch the .vsix file from the vscode-retrorom-debug repo and install that into VS Code.
• Set the emulator path in VS Code's settings to the output of bsnes above, and create a standard "RetroROM Debug" debug configuration.

...and I think that's all you'd need to head off to the races.

Man, if we could get this and the disassembler enhancements here merged upstream, we'd have a seriously powerful debug environment.

Oh, awesome! I'll definitely be keeping an eye on when you submit that into your trunk, because that'll be an interesting merge on my part

Ah, I just noticed that the 'newdebugger' branch is based off of the qt5 branch. Think that'll all be done in one swoop?

Most likely. The qt5 branch is basically finished (pending some tiny makefile tweaks or something, possibly).

Just to give a small update:

I've been porting my work on bsnes-plus to a branch off of the Qt5+newdebugger branch and everything seems to be working nicely, now. The only huge deviation compared to what was there before was my own rewrite to the symbol map - on top of the re-write already done by Revenant Most notably, instead of the symbol map storing a list of Symbols that could operate in a vaguely polymorphic way each 'symbol' is now just an address with some text associated with it, and Labels, Comments, Source lines, all have their own lists. Similarly, access of symbol data is just done by asking for, specifically, a Label at some addr or a Comment at some addr, as opposed to fetching a Symbol and trying to get a comment or line from it manually. Of course, this had a small effect on the other systems that read and write to the symbol map, but it's all fine - and I would argue the code is a bit simpler to read and compute, anyway.

The only functional difference is that the Symbols list (accessed from the Debugger window) doesn't display Comments - it's only Labels - and I wonder if it's worth trying to keep Comments in there? Would that still be valuable for developers (I guess esp. people trying to disassemble/understand existing ROMs?) to search specifically for Comments when appending existing ROM code, or would Labels be sufficient?

Anyway, that aside, I've got just a couple i's to dot and t's to cross for an initial release/PR to the bsnes-plus branch. I've got two big things left I want to rework:

1) breakpoint handling in the emulator, to better distinguish between "user defined breakpoints" and "debug-adapter specific breakpoints", and allow for a more flexible number of breakpoints in general, instead of a flat eight.

2) rework some of the IPC to not simply rely on stdin and stdout, or at least not require the emulator executable to have to be launched from the VS Code extension (i.e. allow for attach to a running process). If nothing else, it would be to try and ease the debugging of the debug-adapter-protocol itself. Right now, the only way I have to debug bsnes-plus itself is to launch it from gdb, so when I launch it from the VS Code extension, there's no facility to attach a debugger to a running instance of the bsnes-plus process, which has made moments of debugging issues a bit bothersome, to say the least. Even to help my own sanity for future development of the debug adapter protocol, it would be good to help make sure other people don't end up a creek without a paddle.

Work on the vscode-newdbg branch is continuing, checking off a few more things here and there.

One major thing recently, though, was the IPC rewrite I wanted to do: Communication now happens over Tcp Sockets so it's possible to, e.g., attach VS Code to a running emulator instead of launching it directly.

Related to this, I also had to expand how my VS Code debug adapter works so that a connection is established over tcp, of course, but as a part of that, I figured it's just non-trivial enough now that it might as well exist on the VS Code Marketplace: https://marketplace.visualstudio.com/it ... orom-debug (or just search for "retrorom" in VS Code's Extensions viewliet ) Note that, unless you're really up on your updates, you'll likely have to update VS Code itself: the extension relies on some newly-finalized debugging functionality.

So, trying this whole thing out for yourself has one less major step in it!

Written a little bit of snes asm recently... won't be needing something as elaborate as this, but I just have to say I find this project really amazing!
So cool to be able to breakpoint snes asm and all that jazz!

A small update before the holidays. Nothing user-facing or substantial from the last post, but wanted to give this because it'll be at least an extra week and a half before anything else.

For one, I do want to note one thing I failed to previously: for bsnes-plus itself on Windows if you want to fetch the vscode-newdbg branch linked in the OP, things are set up there right now so that you only need to have mingw64 installed, just fetch the repo, and run the 'cc' script to build everything: The necessary Qt5 dependencies are contained in a submodule in the project right now, so you do not need to do a separate install of Qt5 and re-set your PATH envvar or whatever to get things to work.

Two, I've been doing a lot of work refactoring a lot (all?) of the stuff related to breakpoints, which is going well. The main intent for the work is:

-Allow for an unlimited number of breakpoints, not just 8. This will involve a rework of the Breakpoint Editor window, of course, to show a list of breakpoints as opposed to a grid of widgets. Still hemming and hawwing over how to go about this, but I think some regression of usability (at least, measured in terms of # of clicks to add/modify a breakpoint) might be inevitable. Trying to find ways to minimize that, though.

-Fix a couple big issues related to how the external debug adapter system was using the breakpoints: breakpoints are set during Launch events from Code properly, and the list of breakpoints are not unintentionally cleared out when you enable or disable them in Code (including breakpoints from files not related to the one you just touched).

-Fix some other design issues behind-the-scenes, where systems tended to directly query the BreakpointEditor UI for information about breakpoints and call it to modify breakpoints (effectively treating the window and its widgets as a data model...tut tut).

I've got most of it done, now: I need to do some more testing to identify and address some bugs around it, do a final performance pass on it especially for the breakpoint_test function (need to get down to ~O(1) complexity from O(n)), and whipping up the new UI for the Breakpoint Editor. I might whiddle away at this over the break, but there are some other things I want to distract myself with in the meantime.

Apologies for taking so long to get back to this, but I'm definitely interested in helping it move forward now that I've been able to have the time for serious work on the emulator again.

Incidentally, I did end up reworking the breakpoint editor in an unrelated effort just because I wasn't particularly satisfied with how it used to be either. There's still some stuff that the new debugger had introduced to it which I kind of want to rework a little, though, and I assume it'll probably need to change a bit more anyway pending all of this stuff.

Anyway, consider this stuff officially back on my radar now.



Did you have any thoughts about how to potentially go about this? Supporting ranged breakpoints, address mirroring, etc. kind of eliminates the possibility of just using a (address -> breakpoint) hash map...

Yeah, ditto.



So, practically speaking, the address space of the 65816 is small enough that I think you could just have a 2MB buffer for each of R/W/X where every byte of address space is marked with a bit indicating if it should be broken on or not. A range breakpoint could "just" set multiple bits, and I think bank shadowing could be accomplished by replicating bits across banks as needed. It'd be a fair bit more complicated than the simpler setup already in place but that'd be my take on it.