How to debug a nes crash?

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
How to debug a nes crash?
by on (#163820)
I have a rom that reliably crashes. It's all written in C, so I can probably blame cc65, but I need to know why it crashes first.

The thing is, I'm on Linux. I have several NES emulators, but none of them offer debugging. And even if they did, my 6502 assembly skills are on the level "I can read it, if I google every single instruction and before-unseen syntax" :P

Sound stops playing. It looks like this (sometimes different colors, sometimes black screen), but keeps scrolling:
Image

I can post the code and the ROM, but I'd also like to know how you'd approach this.
Re: How to debug a nes crash?
by on (#163822)
This is the function that crashes, but only in special circumstances.

When run the first time, everything is perfect, and I can run the function 10 times in a loop without a crash. I can tell it to load any level, and it successfully does so.

However, when I load level 0 and "win the level", moving to level 1, it reliably crashes in this function, in the final ppu_on_all call (tracked using sound effects). It still crashes if I tell it to always load level 0. Very strange.

Code:
void loadlevel(const u8 num) {
   static u8 x, y, q, i, scrollx, scrolly, t, tx, ty, qx, qy;
   static s16 winx, winy, winxmax, winymax;
   static u16 bigx, bigy, maxx, maxy;

   ppu_off();

   LZ4_decompress_fast(complevels[num], (char *) levelbuf, 684);

   memfill(attr, 0, 64);

   lapsleft = laps[num];
   timeleft = times[num];
   leveldir = directions[num];

   bankswitch(levelbank[num]);

   vram_adr(NAMETABLE_A);
   vram_fill(0, 1024);

   // Where do we start?
   goalx = checkfinish[num * 4 + 0];
   goaly = checkfinish[num * 4 + 1];
   checkpointx = checkfinish[num * 4 + 2];
   checkpointy = checkfinish[num * 4 + 3];

   // Aim the camera so it centers on the finish line.
   camx = goalx * 64 + (256 - 128 + 32);
   camy = goaly * 64 + (256 - 120 + 32);

   scrollx = camx % 256;
   scrolly = camy % 240;
   scroll(scrollx, scrolly);

   scrollxoff = scrollx / 8;
   scrollyoff = scrolly / 8;

   // Init area. Which tiles are visible, and at which positions?
   maxy = camy + 256;
   maxx = camx + 272;

   winx = (camx - 256) / 8;
   winy = (camy - 256) / 8;
   winxmax = winx + 32;
   winymax = winy + 30;

   if (maxy > 2304)
      maxy = 2304;
   if (maxx > 2304)
      maxx = 2304;

   bigy = camy;
   if (bigy < 256)
      bigy = 256;

   for (; bigy <= maxy; bigy += 64) {
      y = (bigy - 256) / 64;

      bigx = camx;
      if (bigx < 256)
         bigx = 256;

      for (; bigx <= maxx; bigx += 64) {
         x = (bigx - 256) / 64;

         gqx = x;
         gqy = y;
         q = getquad();

         x *= 4;

         for (i = 0; i < 16; ++i) {
            const u8 tile = quads[q][i];

            qx = x + i % 4;
            qy = y * 4 + i / 4;

            qx *= 2;
            qy *= 2;

            for (t = 0; t < 4; ++t) {
               const u8 val = tiles[tile][t];

               tx = qx + t % 2;
               ty = qy + t / 2;

               if (tx < winx || tx >= winxmax)
                  continue;
               if (ty < winy || ty >= winymax)
                  continue;

               // tx and ty are hw-tile world coordinates,
               // 0-255. Where is our window?

               tx -= winx;
               ty -= winy;

               // Now in window space. Where's the hw window?

               tx = (tx + scrollxoff) % 32;
               ty = ty + scrollyoff;
               while (ty >= 30) ty -= 30;

               // Place the tile
               vram_adr(NTADR_A(tx, ty));
               vram_put(val);
            }
         }
      }
   }

   // Load palette for level
   pal_bg(levelpal);
   pal_spr(sprpal);

   pal_bright(4);
   ppu_on_all();
}
Re: How to debug a nes crash?
by on (#163823)
In my experience, most of the time when I have a crash or glitchy screens, it's one of a few things:

-An address of something is wrong when loading chr or nametable data
-My bankswitching code has a bug or I just forgot to update a bank number in a table
-My vblank code is running too long and causing glitches by writing to the ppu outside of vblank
-Maybe I have a stack error (forgot a pla where I had a matching pha)
-Maybe I have a buffer overrun error and I'm trampling on variables outside of an array (this could easily happen in C, too, no safety net there). With local stack space in C, that could even mess up return addresses...eek! *edit* I see you are not using local stack space in your function, here, so this likely does not apply---could still apply with global arrays though. *edit* To debug this last type of bug, you can experiment with moving arrays around in RAM or changing the size of buffers to see if it affects the behavior of the bug.

*edit* I'm unfamiliar with neslib (assuming that is what you are using). Maybe you need to set a global vram address so the nmi routine resets the vram address and scroll correctly when the ppu is turned back on?

To debug, I can often reason about my code if it is organized enough, but when it is enough of a mystery I can often help narrow down the problem with git bisect (assuming I've been tracking many small changes since the beginning of the project) *edit* note, this only helps if it was previously working and has mysteriously broken. If it's your first revision, all you can do is reason about the code you've written and try to narrow down the problem some other way, possibly by trying simpler cases first. Maybe just load the palette and nothing else and verify it works, then bankswitch and see if it works, (use FCEUX ppu and memory viewer) keep introducing small bits of code until you observe the crash---to help find which code is a suspect for the source of your bug.

I'd be reluctant to blame cc65 itself...I'll let C users here comment further.
Re: How to debug a nes crash?
by on (#163827)
From just the picture alone, it looks like it is repeatedly running the code to update graphics that would usually be in NMI.
Re: How to debug a nes crash?
by on (#163828)
Thanks for the suggestions. I don't think some of them apply though, they're asm-specific, which I'm not using.

The bankswitch is not responsible. When I move it to the start of main, things crash here the same. The PPU is off in this function, and every cpu-heavy function has been tested under valgrind on the host pc.

Quote:
*edit* I'm unfamiliar with neslib (assuming that is what you are using). Maybe you need to set a global vram address so the nmi routine resets the vram address and scroll correctly when the ppu is turned back on?


No need for that, I believe. Here's the part of its NMI routine dealing with scrolling:
Code:
        lda #0
        sta PPU_ADDR
        sta PPU_ADDR

        lda <SCROLL_X
        sta PPU_SCROLL
        lda <SCROLL_Y
        sta PPU_SCROLL

        lda <PPU_CTRL_VAR
        sta PPU_CTRL


Bisect is not useful, since this is working code, I just now added a second level. Also the Linux version of fceux does not have a memory or ppu viewer.

I've hit several valid cc65 compiler bugs, all of which I've reported and some are already fixed. It's the quality of this compiler that always leads me to suspect it first ;)
Re: How to debug a nes crash?
by on (#163829)
Dwedit wrote:
From just the picture alone, it looks like it is repeatedly running the code to update graphics that would usually be in NMI.


That's not possible. Well unless something corrupted the state it keeps for that in the ZP. Which I have no way of testing :(
Re: How to debug a nes crash?
by on (#163830)
calima wrote:
I've hit several valid cc65 compiler bugs, all of which I've reported and some are already fixed. It's the quality of this compiler that always leads me to suspect it first ;)


Fascinating...further reinforces my recent decision to abandon C permanently and simply continue to build idioms (and make good non-overengineered use of macros) in 6502. The only suggestion I made that was asm specific was a stack error (specifically one involving a manual pha .... pla)---all those other things could easily happen in C code, just in different ways.
Re: How to debug a nes crash?
by on (#163831)
Well, I don't have any nmi/vblank code. It's all Shiru's, which is well tested, and I've used it myself in several games successfully.
Re: How to debug a nes crash?
by on (#163832)
There are many ways a game can crash...
-infinite loop
-NMI within an NMI
-stack overflow
-bank switch and program calling data from the wrong bank
-corrupted C library variables
-corrupted C stack

Are you able to run an emulator with debugging features in Linux?

My first step would be to look at the RAM, see if the stack (100-1ff) is going crazy. Then I would export labels from the linker, and use that to locate the probable error function, set a breakpoint in an emulator, and go through it step by step (would help if you knew 6502 ASM).
Re: How to debug a nes crash?
by on (#163833)
AFAIK no emulator for Linux supports any debugging, though I've only tried the most popular ones. That's making this quite hard.
Re: How to debug a nes crash?
by on (#163834)
Quote:
Well, I don't have any nmi/vblank code. It's all Shiru's, which is well tested, and I've used it myself in several games successfully.


(it looks to me like...)
Shiru's code uses NMI code to time VRAM/Palette changes (and music) while rendering is on. With rendering off 'ppu_off()' it skips doing that in the NMI code, and lets you make VRAM changes with the main code. (but it will still go to NMI code to do music updates).
Re: How to debug a nes crash?
by on (#163835)
calima wrote:
Also the Linux version of fceux does not have a memory or ppu viewer.

Unless you're using Linux on a non-x86 platform, you can do what I did: sudo apt-get install wine and then use the Windows version.
Re: How to debug a nes crash?
by on (#163836)
Well, it definitely looks to me like your game is repeatedly writing the same chunk of data to the PPU over and over while rendering is on. This would also give a 'scrolling' effect, as PPU changes also effect scroll position.

There's nothing in the code you posted that would indicate why this is so, because you have ppu_off(); near the top of the function.

Is there any other code that would also happen at level change, that writes to the PPU? Is it perhaps doing it while rendering is on?
Re: How to debug a nes crash?
by on (#163838)
Here's another guess---are you decompressing a level/screen etc. into RAM and then decoding that into vram writes? If so---perhaps your nes.cfg allocates too much ram space to CC65 and this decompression is actually overwriting local/cc65 stack. I recall the default nes.cfg uses 3 pages, which is kinda huge. I asked about this in Efficiency of development process using C versus 6502 and the general response (rainwarrior's response actually) was it can be much smaller, perhaps as small as 64 or even 32 if you're careful. I imagine you already checked into this, just thought I'd throw that out there.
Re: How to debug a nes crash?
by on (#163843)
tepples wrote:
Unless you're using Linux on a non-x86 platform, you can do what I did: sudo apt-get install wine and then use the Windows version.

Pure 64-bit, so Wine is not a possibility.

dougeff wrote:
Is there any other code that would also happen at level change, that writes to the PPU? Is it perhaps doing it while rendering is on?


No, that is all. And it baffles me how I can run it several times over with no ill effects, but run after a second of gameplay, that function crashes.

It's very curious how the full chain works the first time. The outer loop literally goes

Code:
while (1) {
    loadlevel(foo);
    gameplay();
}


The same path works the first time, but fails the second.

Quote:
Well, it definitely looks to me like your game is repeatedly writing the same chunk of data to the PPU over and over while rendering is on. This would also give a 'scrolling' effect, as PPU changes also effect scroll position.


Why would the music stop in that case? It would only be possible if there was an eternal loop in the NMI itself, so it never reached the music code.

GradualGames wrote:
Here's another guess---are you decompressing a level/screen etc. into RAM and then decoding that into vram writes? If so---perhaps your nes.cfg allocates too much ram space to CC65 and this decompression is actually overwriting local/cc65 stack. I recall the default nes.cfg uses 3 pages, which is kinda huge. I asked about this in Efficiency of development process using C versus 6502 and the general response (rainwarrior's response actually) was it can be much smaller, perhaps as small as 64 or even 32 if you're careful. I imagine you already checked into this, just thought I'd throw that out there.


The decompression goes into a statically allocated BSS buffer. There's almost 200 bytes free for cc65's stack, and no function uses more than a couple variables local stack. No deep call chains either. If it's stack corruption, it's not because of my code.
Re: How to debug a nes crash?
by on (#163845)
I would rather use Windows than try to write an NES game without a debugger :|

If it works repeatedly at the beginning of the game

and crashes every time at the end of the level

then something happens during gameplay that causes the crash

(in particular, some global state (either in memory or in the hardware) is changed)

If you know it happens during ppu_on_all(), then you know that the
correct behaviour of ppu_on_all() depends on the the global state that is
changed during gameplay

What does ppu_on_all() look like?
Re: How to debug a nes crash?
by on (#163849)
calima wrote:
tepples wrote:
Unless you're using Linux on a non-x86 platform, you can do what I did: sudo apt-get install wine and then use the Windows version.

Pure 64-bit, so Wine is not a possibility.

Then go impure (that is, multi-arch) with i386 and amd64. Or install a VM and run 32-bit Linux and Wine in the VM. Or buy a copy of Windows to use on another computer, compile FCEUX for Windows in 64-bit mode, and run that in 64-bit Wine.
Re: How to debug a nes crash?
by on (#163850)
Something's gotta be stepping on something. While I was experimenting with C, I could swear I got into a situation once where something was mysteriously stepping on a zp variable of mine. It made me wonder at one point if the linker configuration was set up so that my zp variables actually shared space with cc65's zp space. I never found out what that particular problem was as that bug just hid after a while. With 6502, I've been able to consistently find the root cause of bugs throughout the deveopment of each game (and know for sure it was my own code...the assembler itself only caused trouble in 3 instances I can recall and were minor problems)---something I can't say for any other language I've worked with (sometimes a workaround is the only thing you can find in the face of immense complexity).
Re: How to debug a nes crash?
by on (#163852)
tepples wrote:
Then go impure (that is, multi-arch) with i386 and amd64. Or install a VM and run 32-bit Linux and Wine in the VM.
Or debootstrap yourself a 32-bit chroot and use schroot.
Re: How to debug a nes crash?
by on (#163855)
Maybe you could just post the Source Code and/or the ROM, and one of the people here can look at it with a debugger.

(And if cc65 source code, please indicate what version of cc65 you use).

Or perhaps, buy a $20 used windows laptop from ebay.
Re: How to debug a nes crash?
by on (#163858)
The Debian version of mednafen has a debugger. It has a really obtuse interface, but it's something.
Re: How to debug a nes crash?
by on (#163859)
Not having a debugger at your disposal is crazy, solving bugs will take you 10 times longer if you have to keep guessing where the problem is... Come on, just install VirtualBox and a 200MB stripped down version of XP so you can do things properly.
Re: How to debug a nes crash?
by on (#163860)
I know I'm being completely unhelpful here, but I find it depressing that the answer to "how to debug a NES game on Linux" is "use Windows".

Seriously though, why is the Linux port of FCEUX so pathetic?
Re: How to debug a nes crash?
by on (#163861)
Rahsennor wrote:
Seriously though, why is the Linux port of FCEUX so pathetic?

You act like this is some sort of professional project. The reason is obvious: nobody is sufficiently interested to do it for free.

If it's something you care about, why not contribute to the project yourself?
Re: How to debug a nes crash?
by on (#163865)
I was not intending to act like it was a professional project, and you're right that the answer is obvious. I should have kept my mouth shut.
Re: How to debug a nes crash?
by on (#163867)
Sorry, I didn't want to admonish you for speaking about it; what I was trying to do (perhaps with too bitter a tone) was encourage you to join the FCEUX project and help bring it up to snuff.
Re: How to debug a nes crash?
by on (#163875)
Rahsennor wrote:
I find it depressing that the answer to "how to debug a NES game on Linux" is "use Windows".

No, it's "use Wine". That works for NES (FCEUX), and it works for Super NES (NO$SNS).

Quote:
Seriously though, why is the Linux port of FCEUX so pathetic?

Is it necessarily more pathetic than wanting to adhere to some standard of 64-bit purity?
Re: How to debug a nes crash?
by on (#163894)
tepples wrote:
Is it necessarily more pathetic than wanting to adhere to some standard of 64-bit purity?


And waste several gigs? I have better use for that space.

To the people telling me to pirate or pay for Windows, screw you. That is never the solution, it's a problem.

Quote:
(in particular, some global state (either in memory or in the hardware) is changed)


I know, but there is no such state change in my code. So it must be a compiler bug, which is rather hard to track. If the cc65 stack underflows, it scribbles on the 0x2000 registers, if the hw stack overflows, it scribbles on ZP vars.

Quote:
What does ppu_on_all() look like?


Code:
;void __fastcall__ ppu_on_all(void);

_ppu_on_all:

        lda <PPU_MASK_VAR
        ora #%00011000

ppu_onoff:

        sta <PPU_MASK_VAR
        jmp _ppu_wait_nmi


Quote:
While I was experimenting with C, I could swear I got into a situation once where something was mysteriously stepping on a zp variable of mine


They are allocated the same, so that was a bug in the compiler (or the standard library, if you used it).

dougeff wrote:
Maybe you could just post the Source Code and/or the ROM, and one of the people here can look at it with a debugger.

(And if cc65 source code, please indicate what version of cc65 you use).


I use cc65 from git master. Will post the ROM in a bit.

Quote:
No, it's "use Wine". That works for NES (FCEUX), and it works for Super NES (NO$SNS).


I won't use closed-source software on principle. Yes, fceux Windows would be open source, but that's a huge waste of both time and disk space.

I'm thinking about building fceux with debug symbols, finding where it keeps the RAM, and setting watches by debugging the emulator on the host. Crazy, but far less crazy than installing Windows.
Re: How to debug a nes crash?
by on (#163897)
ROM link:
https://u.pomf.is/vstins.gz

Wait a second or two and press select.
Re: How to debug a nes crash?
by on (#163902)
calima wrote:
AFAIK no emulator for Linux supports any debugging, though I've only tried the most popular ones. That's making this quite hard.

NESICIDE does. But it's been *forever* since I built and provided a Linux binary. Not that it won't work. Just will take me a little bit to get my Linux VM back.
Re: How to debug a nes crash?
by on (#163903)
It is not a hw stack overflow. 0x100 is never accessed.
Re: How to debug a nes crash?
by on (#163904)
Quote:
I know, but there is no such state change in my code. So it must be a compiler bug, which is rather hard to track. If the cc65 stack underflows, it scribbles on the 0x2000 registers, if the hw stack overflows, it scribbles on ZP vars


Not exactly what happens...

The hardware stack will wrap back on itself...102, 101, 100, 1ff. Overwriting itself. But, noone ever uses that much HW stack, so this generally means an infinite loop has occurred, and the program is crashing.

Some people put variables in the stack. I've seen Shiru's music engine put variables in the HW stack (at the low end where it's very unlikely to be overwritten). It's possible that these could be overwritten.

The C stack...if STACK_SIZE is correct, should also wrap back onto itself...thus crashing the game. I've never seen this happen, btw.
I suppose it could also start overwriting BSS section of RAM, if the STACK_SIZE or SP (C stack pointer) got corrupted.
Re: How to debug a nes crash?
by on (#163906)
cpow wrote:
calima wrote:
AFAIK no emulator for Linux supports any debugging, though I've only tried the most popular ones. That's making this quite hard.

NESICIDE does [EDIT: Specifically supports C/assembly source stepping, etc.]. But it's been *forever* since I built and provided a Linux binary. Not that it won't work. Just will take me a little bit to get my Linux VM back.

Sorry...I hit reply instead of edit. Oops.
Re: How to debug a nes crash?
by on (#163907)
dougeff wrote:
The hardware stack will wrap back on itself...102, 101, 100, 1ff. Overwriting itself. But, noone ever uses that much HW stack, so this generally means an infinite loop has occurred, and the program is crashing.


Thanks for the correction.
Re: How to debug a nes crash?
by on (#163909)
Doesn't look like the C stack is bonkers either. SP does not become bad at any point.
Re: How to debug a nes crash?
by on (#163910)
calima wrote:
To the people telling me to pirate or pay for Windows, screw you.

No, screw you. Have fun posting a ROM every time you hit a bug so others can find the cause for you. That sure sounds like a productive workflow.
Re: How to debug a nes crash?
by on (#163911)
calima wrote:
tepples wrote:
Is it necessarily more pathetic than wanting to adhere to some standard of 64-bit purity?

And waste several gigs?

It's not necessarily several GB. I haven't tried it myself, but you can probably install a 32-bit userland smaller than 1 GB in a chroot and then install Wine and FCEUX into that. The existence of Puppy Linux makes me think this is possible.

Quote:
I have better use for that space.

It's not a "waste" if you are getting something useful in exchange for said space, such as the ability to debug NES games. How big is the HDD in your existing machine? Which distro? I need this detail so that I can eventually post a question to Unix & Linux Stack Exchange on your behalf.

(And if the answer to that is "it doesn't matter how big it is; it's the principle of the thing", then my reply is as follows: If the effort of creating your own debugger for the SDL version of FCEUX is worth "several gigs of disk space" to you, go ahead.)

Quote:
To the people telling me to pirate or pay for Windows, screw you. That is never the solution, it's a problem.

Then download MinGW-w64, cross-compile FCEUX for 64-bit Windows, and run that in 64-bit Wine.

Quote:
I won't use closed-source software on principle.

Then how do you connect to the Internet? The firmware of your machine's Wi-Fi radio is probably not free software. And how did you get into the NES in the first place? NES games published prior to 1997 were not free software.

Quote:
Yes, fceux Windows would be open source, but that's a huge waste of both time and disk space.

Is using FCEUX in Wine necessarily more of a waste of time than writing your own debugger?
Re: How to debug a nes crash?
by on (#163912)
You have an infinite loop. I don't have the labels, or the asm with inlined C in comments, so this will look like crap...

Code:
            sta $6a            ; $a542: 85 6a     
            stx $6b            ; $a544: 86 6b     
__a546:     ldy #$00           ; $a546: a0 00     
__a548:     lda ($6a),y        ; $a548: b1 6a     
            iny                ; $a54a: c8       
            cmp #$40           ; $a54b: c9 40     
            bcs __a561         ; $a54d: b0 12     
            sta $2006          ; $a54f: 8d 06 20 
            lda ($6a),y        ; $a552: b1 6a     
            iny                ; $a554: c8       
            sta $2006          ; $a555: 8d 06 20 
            lda ($6a),y        ; $a558: b1 6a     
            iny                ; $a55a: c8       
            sta $2007          ; $a55b: 8d 07 20 
            jmp __a548         ; $a55e: 4c 48 a5 

;-------------------------------------------------------------------------------
__a561:     tax                ; $a561: aa       
            lda $7c            ; $a562: a5 7c     
            cpx #$80           ; $a564: e0 80     
            bcc __a570         ; $a566: 90 08     
            cpx #$ff           ; $a568: e0 ff     
            beq __a596         ; $a56a: f0 2a     
            ora #$04           ; $a56c: 09 04     
            bne __a572         ; $a56e: d0 02     
__a570:     and #$fb           ; $a570: 29 fb     
__a572:     sta $2000          ; $a572: 8d 00 20 
            txa                ; $a575: 8a       
            and #$3f           ; $a576: 29 3f     
            sta $2006          ; $a578: 8d 06 20 
            lda ($6a),y        ; $a57b: b1 6a     
            iny                ; $a57d: c8       
            sta $2006          ; $a57e: 8d 06 20 
            lda ($6a),y        ; $a581: b1 6a     
            iny                ; $a583: c8       
            tax                ; $a584: aa       
__a585:     lda ($6a),y        ; $a585: b1 6a     
            iny                ; $a587: c8       
            sta $2007          ; $a588: 8d 07 20 
            dex                ; $a58b: ca       
            bne __a585         ; $a58c: d0 f7     
            lda $7c            ; $a58e: a5 7c     
            sta $2000          ; $a590: 8d 00 20 
            jmp __a548         ; $a593: 4c 48 a5 

;-------------------------------------------------------------------------------
__a596:     rts                ; $a596: 60   


The error is this...

cpx #$ff
beq __a596

it seems to be looking for a bit of data, equal to $ff (perhaps an END OF DATA marker), but it's not finding one.

The data set it's looking at goes like this...

00000000000000000000000000000000000000000000E8E8E8E8E8E8E8E8E8E8E8ABABABABABABABABABABAB00000000

You can see it has no $ff in it.


And, as I suspected, it's writing that data to the PPU, over and over.
Re: How to debug a nes crash?
by on (#163913)
Not on topic at all, but...
dougeff wrote:
I've seen Shiru's music engine put variables in the HW stack (at the low end where it's very unlikely to be overwritten).

Is this really true? Because I use that space for a current project that I plan to use with Famitone2... hmmm... well worst case, I can modify Famitone2, I guess.
Re: How to debug a nes crash?
by on (#163914)
Re:Shiru...FT...variables in HW stack...

Yes, the Chase game example code...

Code:
FT_BASE_ADR      =$0100   ;page in RAM, should be $xx00


but, you can change the base address to whatever page you want.
Re: How to debug a nes crash?
by on (#163915)
tokumaru wrote:
No, screw you. Have fun posting a ROM every time you hit a bug so others can find the cause for you. That sure sounds like a productive workflow.


Yes, that is exactly what I have done, and intend to do. /s

Telling a Linux user to install Windows is awfully unhelpful. "just install VirtualBox and a 200MB stripped down version of XP so you can do things properly."

Thanks to this thread, I learned about mednafen, and my current strategy of gdb-on-the-host is working great so far.

Quote:
Then how do you connect to the Internet? The firmware of your machine's Wi-Fi radio is probably not free software. And how did you get into the NES in the first place? NES games published prior to 1997 were not free software.


This is quite offtopic. I'll talk about this in a separate thread, if you like?

Quote:
If the effort of creating your own debugger for the SDL version of FCEUX is worth "several gigs of disk space" to you, go ahead.

Quote:
Is using FCEUX in Wine necessarily more of a waste of time than writing your own debugger?


Yes :)

Using gdb on fceux, I have proper access to RAM watchpoints, and I've found the issue. It's indeed a compiler bug. cc65 is not setting the A register properly when calling a function that takes a pointer.

This took me much less time than setting up a 32-bit chroot, and wasted no space.

@dougeff

Thanks! We got it the same time, but I appreciate your help.

@Kasumi

It's true.
Re: How to debug a nes crash?
by on (#163916)
Oh. Well, that's more that that specific game does it, rather than the engine. It's not even the default.
Re: How to debug a nes crash?
by on (#163917)
Funny, it actually loads that data set into the HW stack at the start of the game, but never uses it. Then, when you press 'SELECT'...

when it crashes, it IS accessing it, to load to the PPU.
(sometimes it doesn't crash)
when it doesn't crash, it ISN'T using it at all.
Re: How to debug a nes crash?
by on (#163918)
dougeff wrote:
Funny, it actually loads that data set into the HW stack at the start of the game, but never uses it. Then, when you press 'SELECT'...

when it crashes, it IS accessing it, to load to the PPU.
(sometimes it doesn't crash)
when it doesn't crash, it ISN'T using it at all.


The cc65 bug causes garbage to be passed, when the C code passed NULL. Since Shiru's update loop uses an u8 loop index, it will go over the same 256 bytes forever if there's no EOF marker there.
Re: How to debug a nes crash?
by on (#163920)
I have created a new topic for the Wine installation issue:
Getting into 8-bit NESdev with only 64-bit free software
Re: How to debug a nes crash?
by on (#163921)
I'm curious to see the simplest possible example of a C routine that cc65 will generate incorrect code for in the case you are examining. I'm not using C anymore but maybe it'd help others avoid the same issue until someone fixes the bug.
Re: How to debug a nes crash?
by on (#163924)
https://github.com/cc65/cc65/issues/263

It happens to both pointers and 16-bit variables, but nobody's made a minimal example yet. A workaround is to disable optimizations.
Re: How to debug a nes crash?
by on (#164200)
Quote:
Thanks to this thread, I learned about mednafen, and my current strategy of gdb-on-the-host is working great so far.

Same here. I was using Nestopia to visually "debug" and tried fceux in wine but was annoyed by the multitude of windows and input focus issues. Since I prefer using my Mac for development, mednafen seems to be the best solution.

Version control has also saved me a lot of headaches. I make frequent commits, using git, so that if I break something I can roll back to a previous version and test it. That usually means I only have to look at a small portion of code instead of making guesses.
Re: How to debug a nes crash?
by on (#164206)
heardtheword wrote:
Since I prefer using my Mac for development

Yay, someone else! We're few and far between. :)
Re: How to debug a nes crash?
by on (#164227)
As mentioned. Mednafen works, though it's a bit pre-GUI in how it works. (Also nota bene that you need about x3 minimum zoom for there to be sufficient resolution to display the entire CPU debug screen!)
Rahsennor wrote:
I know I'm being completely unhelpful here, but I find it depressing that the answer to "how to debug a NES game on Linux" is "use Windows".

Seriously though, why is the Linux port of FCEUX so pathetic?

To my knowledge (from being on TASvideos forums for a long time), the Windows version's memory-watch dialogue (etc.) was ported from Gens.

Gens's dialogue is Windows-only.
Re: How to debug a nes crash?
by on (#165244)
After further research, the 16-bit pointer bug (that caused my crash) is a bug in neslib, not cc65. Further, half of neslib is buggy. I will make a new topic about it.
Re: How to debug a nes crash?
by on (#165249)
It's *both* a neslib and a cc65 bug. Whee this is fun.