Hblank on line 153?

So I recently worked on emulating the GBVideoPlayer.

Source: https://github.com/LIJI32/GBVideoPlayer
Result: https://twitter.com/byuu_san/status/686436982407315456

Aside from obviously needing a pixel-based renderer (already had one), the trick seemed to be that it was expecting 145 Hblank IRQs to fire. And if you were one short, it'd fetch bad control bytes and go completely haywire, displaying nothing but gibberish.

Obviously, lines 0-143 trigger Hblank. But upon adding another to line 153, on the theory of it being a way to set up special effects for the first scanline, the video began to work correctly.

But it kind of bothers me that gambatte doesn't emulate this ROM. The gibberish in it is identical to what I get without the extra Hblank at line 153. However, the video ROM has been shown running on a real GBC, so obviously it's an emulation issue.

So, does anyone know anything about this? Can anyone confirm the existence of an extra Hblank IRQ?

Perhaps this is related to some undocumented (or rather, super poorly documented) behavior I stumbled upon while debugging Line 0 LYC-STAT IRQs. Apparently, Line 153 can be treated as Line 0 (I assume at this time, the LCD has actually returned to the top of the screen), and this is the appropriate time to fire LYC = 0 STAT IRQs. Without it, my IRQs always fired late, causing issues like these for me and other devs.

So, essentially, what we call Line 153 is Line 0, but technically still in VBlank state, I think. After Line 153/Line 0 runs for 456 cycles, it exits VBlank, leaves LY at 0, then returns to Mode 2, so it "appears" as if Line 0 is run twice. This is the only thing I can come up with that makes sense from a hardware point of view, and it works in emulation. It's easy to verify on HW, just haven't gotten around to it lately.

Anyway, here's my theory. What if we suppose that the LCD controller just does something else weird on Line 153/Line 0? What if it, even though it's in VBlank, it treats Line 153 as if it were some sort of dummy Line 0, e.g. triggering STAT and HBlank IRQs? From what I can see, we can already assume that such a dummy Line 0 triggers STAT IRQs in commercial games, so what's to stop it from doing HBlank IRQs as well? byuu, if you want, I can make a quick test ROM and verify it on real HW. Actually, I'll just do it now since I'm off today

Having this question in the back of your mind might help you come up with hypotheses: When does it fetch the pattern data for a sprite on the top scanline of the screen?

> Perhaps this is related to some undocumented (or rather, super poorly documented) behavior I stumbled upon while debugging Line 0 LYC-STAT IRQs. Apparently, Line 153 can be treated as Line 0 (I assume at this time, the LCD has actually returned to the top of the screen), and this is the appropriate time to fire LYC = 0 STAT IRQs. Without it, my IRQs always fired late, causing issues like these for me and other devs.

Well, I certainly agree that you want to be able to do effects on line 0. The SNES has an entire "dummy" scanline at line 0 for that exact purpose, so the behavior makes sense. I can see the Game Boy kind of messing up this clarity by calling SNES 0 == GB 153, SNES 1 == GB 0, etc.

What I'm curious about here is whether or not the real LY=0 line triggers these STAT (Hblank, etc) IRQs then. My guess is it'd have to.

Thus, you'd have them firing on LY={153,0,1,2,3,...,143}; or 145 times per frame. The final 143 one is rather useless. However, without firing exactly 145 Hblank IRQs, this GBVideoPlayer doesn't work.

> After Line 153/Line 0 runs for 456 cycles, it exits VBlank, leaves LY at 0, then returns to Mode 2, so it "appears" as if Line 0 is run twice.

Are you sure it's not meant to allow start/end effects on each line of the display, rather than "running twice"?

It feels more likely to me that it's intentional and not a duplication. And line 153 is simply just not rendering at all.

> What if it, even though it's in VBlank, it treats Line 153 as if it were some sort of dummy Line 0, e.g. triggering STAT and HBlank IRQs?

Exactly what I'm thinking.

> byuu, if you want, I can make a quick test ROM and verify it on real HW. Actually, I'll just do it now since I'm off today

Awesome, thank you very much :D

I don't really know LR35902 assembler syntax, don't have a cross assembler table for the syntax, etc. So I'm pretty useless for test ROMs right now, unfortunately. I get the feeling that'll probably change in the future. Like you said in your Github commit comment ... DMG documentation is just pitiful. There's really not much choice but to break out the hardware if you want to write a good emulator.

I'm borrowing your style of quoting (it's just so much quicker!)

>Are you sure it's not meant to allow start/end effects on each line of the display, rather than "running twice"?
>
>It feels more likely to me that it's intentional and not a duplication. And line 153 is simply just not rendering at all.

The thing is, I believe the LY counter simply reads as 0 on Line 153 and Line 0, that's why I say Line 0 "appears" to run twice. It'd technically be Line 153 as far as the LCD itself is concerned. But during those 912 cycles (from Line 153 start to Line 0 end), LY would be 0, so to GB software it might look like Line 0 takes twice as long to finish. Like I said, I haven't had time to test it in my GBs. I debugged this behavior by observing what BGB did. In software at least, that's how BGB interprets hardware behavior, and it several commercial games appear to rely on this trick.

> I don't really know LR35902 assembler syntax, don't have a cross assembler table for the syntax, etc. So I'm pretty useless for test ROMs right now, unfortunately. I get the feeling that'll probably change in the future. Like you said in your Github commit comment ... DMG documentation is just pitiful. There's really not much choice but to break out the hardware if you want to write a good emulator.

Well, up until recently, I had been doing nearly all of my test ROMs in a hex editor + a text file to keep track of stuff (not kidding). Recently made my own minimal assembler that's only barely a step up from that. I call it LAZ80. It's a pun on my laziness to properly parse strings. The only thing it actually reads are hex values prefaced by "!" or "$", then builds the ROMs bit by bit. Basically the whole GB binary with comments, all in a text file. I have separate graphics tools for pushing data to ROMs though.

Anyway, I made this quickly. Source is attached as well. I tried to count how many HBlank interrupts occurred in a single frame. Code should start counting just as it enters VBlank, so if an HBlank interrupt is fired on Line 153, it should technically catch it. Unfortunately, ran it on my GBC, and it gave me a negative result (the ROM prints a giant "N" for only 144 HBlanks, and a giant "Y" for 145). So, as long as the code does what we think it does, it says only 144 HBlanks are fired. If you run it higan, you should see a Y, but apparently that's not supposed to happen

So where to go from here now?

EDIT - Perhaps there is a pending HBlank IRQ somewhere in VBlank for GBVideoPlayer? If the VBlank IRQ is serviced first and finishes, the pending STAT interrupt could still be triggered towards the end of VBlank. It would still have total access to VRAM at that point too.

> I'm borrowing your style of quoting (it's just so much quicker!)

Isn't it, though? :D

Worked for years with e-mail. But phpBB made everything a pain to do.

Whenever I get around to finishing that custom forum of mine, I'm definitely going to parse them into block quotes. And "<name> foo" for named quotes.

> The thing is, I believe the LY counter simply reads as 0 on Line 153 and Line 0

... that would be crazy. I'd very much like to see a test around this.

> So where to go from here now?

You may be right. I tried firing the Hblank on line 153 and NOT on line 0, and the GBVideoPlayer works. Of course, it could also be that line 143 -doesn't- fire an Hblank IRQ (why would it? You aren't going to be doing any scanline-level effects when line 144 doesn't render anything.) I'm inclined to believe the latter is correct, because it's a lot more sensible.

Doesn't matter whether I return 0 from $ff44 (LY) for line 153 or not, it still works with just the first change.

I would say next up is we'd want to just have a dumb loop that reads LY and writes it to save RAM. Fill the entire save RAM with a dumb loop. Then we can see if 0 appears twice as much as other values, and if 153 is indeed missing.

It'll be trickier to time the Hblank IRQs, but I think it's doable: have a main loop that just keeps incrementing a 16-bit number. Every time Hblank fires, write the value to RAM. As long as we have two frames worth, we should be able to determine where the Hblank events are firing. If we see a gap in one value that's twice as wide, we confirm the theory for LY=0 skips Hblank. If not, I believe that'll confirm it's LY=143 that's missing. And our emulators should get relatively close to the same numeric values as the save RAM files from real hardware, too.

Use CommonMark instead of BBCode, and you get that for free. Reddit, Discourse, and Stack Exchange do this, for instance.

> Use CommonMark instead of BBCode, and you get that for free. Reddit, Discourse, and Stack Exchange do this, for instance.

That's most likely what I'm going to do. My website runs on DML, which is basically simplified CommonMark. But I'll need to add some specialization for eg "<name> quote" style.

I'll have to make it an option, though. Some people don't like CommonMark.

Well, I must say byuu, you certainly know how to entertain people with mysteries

I think I've cracked this one though. It's not HBlanks you want to go after, but you're close. You still want to target STAT IRQs, but you want the OAM ones (with Bit 5 set in the STAT register). If you run the ROM of the Pokemon anime intro, you'll see it constantly uses this interrupt. Here's what I observed through BGB (it's one of the few emulators that "works" with it) and I noticed that OAM-STAT IRQs were being fired on Lines 0 through 144. That's up and including Line 144. That means the demo in question runs and services that interrupt 145 times.

Curious, I tried to make a test ROM that replicates and demonstrates that behavior. Like before, it counts every instance of a certain interrupt (in our case OAM-STAT ones). It runs one frame (until VBlank) then draws a "Y" for 145 successful instances, or "N" for only 144 instances (or a blank screen if something entirely different happens). Initially, I used VBlank IRQs to stop the test from counting. Strangely, I kept getting the same results as before. Real hardware showed only 144 instances of the OAM-STAT interrupt firing.

However, I saw that the GB Video demo does not use VBlank interrupts, so I wondered if that was somehow blocking the 145th OAM-STAT interrupt in my test from firing. Rewriting the test to read LY manually, suddenly the 145th OAM-STAT triggered! I verified this in BGB (displays the "Y"), VBAM (displays the "N" and runs like garbage on the GB Video demo), my own GBE+ (displays a blank screen, oops...), and finally my GBC (displays the "Y").

So here's what it looks like. OAM-STATs can fire 145 times a frame, from Lines 0 to 145. Even though Line 145 is technically in VBlank, perhaps the LCD hardware hasn't had enough time to tell itself to stop searching OAM and generating interrupts? I can't say, but I don't believe Lines 146 - 153 trigger STAT interrupts of any sort. What's more, this behavior is only observable when a VBlank interrupt does not occur. As soon as a VBlank IRQ is fired, the LCD fails to fire an OAM-STAT interrupt on Line 145; you don't even see the interrupt as pending in the IF register. I'm really interested in what logic the LCD controller is following internally that makes it go this route.

Here is the new source code and the new test ROM. Like I said, I already verified the result on real hardware, but please double check the code to make sure it does what we think it does. At any rate, I'm pretty sure we've pegged another undocumented quirk about the GB. Again, apparently some emu authors (*cough*BGB*cough*) knew about this stuff ages ago, but failed to enlighten anyone else...

Thanks, this seems like we're definitely on the right track here! Still lots of questions remaining, though.

What about LYC interrupts? Do they affect this behavior at all if they do/don't trigger?

Does this behavior happen only for the STAT-VBLANK interrupt, or the regular VBLANK one?

Does the display enable flag matter at all for this? (Do all STAT interrupts require the display to be enabled?)

What about the "read LY as 0 on LY=153" behavior?

Does a LYC of 0 trigger on LY 153 only, or on both 153 and 0? Testing Kirby's Dreamland 2, both work.

> Again, apparently some emu authors (*cough*BGB*cough*) knew about this stuff ages ago, but failed to enlighten anyone else...

No surprise there. What else would you expect from someone that hides their source?

> What about LYC interrupts? Do they affect this behavior at all if they do/don't trigger?
>
> Does this behavior happen only for the STAT-VBLANK interrupt, or the regular VBLANK one?

If you had an LYC interrupt on line 144, you would have to read the STAT register to tell if an LYC interrupt would technically block the OAM interrupt like regular VBlank interrupts do. OAM and LYC interrupts trigger at the same time from what I understand, so you couldn't use a simple counter like this test does. You'd have to see what bits get flipped in STAT itself. It's something I'll keep in mind when I work on the next versions.

Now about other STAT interrupts (VBLANK especially), I don't know. I only tested OAM interrupts. I would like to know if I enabled STAT-OAM and STAT-VBLANK, would it still hit 145 instances for the STAT interrupt? That is to say, is the STAT-VBLANK just like the regular VBlank interrupt, blocking the 145th instance? If so, we can say there's something weird going on with the GB's LCD and how it checks VBlank interrupts of any kind. Ultimately, what I'd like to know for sure is, what kinds of STAT interrupts can occur on Line 144. I'll leave that for tomorrow though. Today's been pretty exciting already

> Does the display enable flag matter at all for this?

Pretty sure the LCD doesn't clock any cycles when it's turned off. Maybe I misremembered something, but that's what I recall. In that case no LCD interrupts can be triggered, so I don't imagine it would affect us here. Just have to be careful when counting lines because LY is reset to 0 again when switching the LCD off then on.

> What about the "read LY as 0 on LY=153" behavior?

I'll still have to check that out. I suspect that "dummy" Line 0 is what happens on a real GB. Tomorrow I'll make a new test ROM specifically for that stuff.

> Pretty sure the LCD doesn't clock any cycles when it's turned off.

... holy shit. That would be absolutely chaotic if true.

You could effectively control the refresh rate completely with that.

I know Nintendo disallows turning it off while the screen is rendering. But if you turn it off in Vblank, and then wait for one full frame, you've turned your Game Boy into a 30fps display.

It would absolutely require an adaptive sync monitor to emulate. And even then, you could easily push it below the minimum value they support (not that any game would.)

I can easily emulate it under the caveat that you'll never get smooth scrolling, but ... yuck.

Would love a test ROM to confirm this, if you don't mind.

Nintendo DS can also have its scanline reset during vblank time.
And some GBC games did turn off the screen during render time, like Conker's Pocket Tales.

As I understand it, this is intended for synchronizing multiple systems in WLAN multiplayer where one system's crystal may be slightly faster or slower than another's.


GBC also had a radically different screen technology from the monochrome Game Boy. I wonder if ability to safely turn off rendering was one of the changes between the Game Boy's screen and the GBC's.

@tepples - I think you're correct on both accounts. Echoes what I've looked into.

@byuu - Not enough time to make the rest of the Line 144 STAT IRQ tests, but I managed to replace OAM-STAT interrupts with HBLANK-STAT interrupts. Even with VBlank interrupts (regular and STAT based ones) disabled, HBlank never fires on Line 144. It seems OAM-STAT is a bit of an oddball. I still have to test it to see if VBLANK-STAT interrupts mess it up like regular VBlank interrupts. I think the 145th OAM-STAT interrupt probably triggers (when in theory it shouldn't) because of how early in a scanline it's supposed to fire. Like I said, I think the LCD controller doesn't properly recognize it's in VBlank as soon as it enters Line 144, so it gives OAM-STAT interrupts the OK. HBLANK-STAT interrupts happen later in the scanline, so maybe by that time the LCD controller correctly figures out VBlank is taking place. Just a theory, since I can't see what's going on with the actual circuits and whatnot.

Anyway, I did have time to get the LCD clocking demo done. Here's how it works. It creates 2 16-bit counters (C1 and C2), then increments them in two loops. The test waits for Line 152 (tail end of VBlank), then it shuts off the LCD (which resets LY to 0). The first loop updates C1 for 0x390 times (taking 32 cycles each time, so approximately 72 scanlines worth of looping). The values are written to cart RAM, then LCD is turned on. Now the next counter, C2, increments in a loop (also 32 cycles long) all the way until it reaches VBlank. On VBlank, it writes the counter values to RAM, then loops endlessly.

The idea is, if the LCD isn't clocking at all, LY isn't incremented at all during this period. If the loop for C1 finishes and the LCD doesn't clock anything, when it turns the LCD back on, LY is still 0 and the LCD starts clocking with a fresh start. In that case, when C2 counts up, it will be noticeably bigger than C1 because it has to run for all 144 scanlines.

If the LCD is clocking when turned off, however, it should still update LY. When C1 finishes its loop, LY should be 72 when the LCD is turned back on. Since C2 counts until the next VBlank, its value should be nearly equivalent to C1 (since both are supposed to run for 72 scanlines).

I tested it on my GBC, but the results indicate the former rather than the latter. The LCD doesn't clock cycles when its turned off. At least, it doesn't do anything with them. I'll leave the source code along with the ROM + save file. C1 is 0x1 - 0x2 of the save (MSB first), and C2 is 0x3 - 0x4 (again MSB first). C1 will always be 0x390, but real hardware shows C2 being 0x720 (so exactly 2x C1, meaning it ran for all 144 scanlines). So... what this means is you can delay how long you leave the LCD off for an arbitrary amount of time. When you turn it on, it always starts fresh from zero. Chaos as you said

Hey!

First of all I must say I'm flattered to see people actually invested time in properly emulating my GBVideoPlayer. (You should have contacted me though! I could have helped!)

Second, the source code of the OAM interrupt in the player (It's incorrectly labeled as HBlank, sorry!) states the problem:


When I released the player, I haven't investigated this issue that much, after I start writing my own Gameboy Color emulator (Will release it sooner or later ) I decided to investigate further. And yes, my own player gave me a pretty much hard time when writing the emulator.

I discovered that the extra interrupt only occurs for OAM interrupts, but that extra "line" is neither 153, nor is it 144 (as stated in the comment). BGB emulates this hardware bug incorrectly, an interrupt being fired for LY=144. This was EXTREMELY confusing to figure that out, since I used BGB to debug my player when I was making it.

I wrote a simple test ROM that had an OAM interrupt that simply added 1 to an array cell, with LY as an index (i.e. array[rLY]++). I ran the test on my GBC and found out that the extra interrupt occurs for LY=151.

This can also be verified by modifying GBVideo Player by changing $8F in the above code snippet. Setting anything higher that 143, except 151, will break it on a real Gameboy. 144 will work on BGB, but not on a real Gameboy, and 153 will work on higan, but not on a real Gameboy. Also, any value greater than 143 will work on an emulator that does not emulate this bug in the first place.

And while we're at it, my local "build" has a double resolution, which will later get another 50% increase. I attached a comparison as a teaser, I hope the coming soon updates won't break any of your emulators!

Hello LIJI, great to have you join us!

Could you double check your results on whether the 145th OAM-STAT interrupt happens on Line 151? I only ask because I modified my tests to try to replicate it. The source code and the binary are attached for review. What I did was use Register B as a counter for OAM interrupts, and HL pointing to Cart RAM to write the results of the current LY value. The program waits until the start of a new VBlank (manually testing if LY == 144), then it spins, waiting for OAM-STAT interrupts. Every OAM-STAT interrupt, B is incremented, and the current value of LY is stored in Cart RAM. It technically starts waiting for OAM-STAT interrupts somewhere on line 144, so it should catch any OAM-STAT interrupts that occur from 145-153, then from 0-144. Essentially, I believe I'm testing for OAM-STATS on every line. The results of the Cart RAM show 0x0 through 0x90 (Lines 0 through 144).

I tested it on my GBC, and the save attached (just rename it from .txt to .sav or something) is the dump from my test. Could you share your test here as well, and perhaps take a look at my code? I'm pretty sure I'm doing this correctly, but I want to reconcile getting different results than you.

> Second, the source code of the OAM interrupt in the player (It's incorrectly labeled as HBlank, sorry!) states the problem:

The comment definitely helped a whole lot. But it sounds like we still don't have the exact behavior down 100% just yet, but Shonumi seems to be closing in on it :D

Right now, it looks like my interrupt system is shot. Altered Space is doing something really weird with IRQs that I don't yet understand. And the exact way they trigger is really poorly explained (like everything else on the DMG ...) So my emulator's probably not the best place to test things, but ... oh well.

> the extra interrupt occurs for LY=151

Given this contradicts Shonumi's research ... I wonder if it varies per Game Boy, or if perhaps yours is an odd exception that's acting slightly erratic?

I've also heard another person state they couldn't run your GBVideoPlayer on their real hardware with a flashcart.

Maybe Game Boys are all full of micro-variations like this. Wouldn't that be a treat? :P

> And while we're at it, my local "build" has a double resolution, which will later get another 50% increase. I attached a comparison as a teaser, I hope the coming soon updates won't break any of your emulators! :P

Looking forward to seeing how you do it! :D

And by the way, I'm considering extending my MBC3 support to allow full 16-bit bank selection, which would give you 1GiB of storage. It's a silly, stupid thing, but maybe that could be useful for you in some way? Obviously won't work on real hardware, of course.

Weird, Shonumi's test ROM passes on my Gameboy, yet fails on emulator, meaning I haven't got it either.

I uploaded the test ROM I used to get that 151 timing. It's different than what I remembered but it's still pretty much the same: It logs every OAM interrupt by it's LY value into an array, "restarting" every HBlank. On my Gameboy Color and on my own emulator, the output starts with 97 (151) and continues with "00, 01, 02, 03...". It does not display the entire array, because that would take too much CPU and break the timing.

BGB's handling of this is quite weird: When in CGB mode, the output does not start with 97; it starts with 00 and continues. Debugging easily shows the extra interrupt happens on LY=144 - incorrect. In DMG mode however, it's the same as the correct CGB output. However, I don't have a DMG to verify if this behavior is correct.

As for higan, I can't get the ROM to even run on it — for some reason it makes icarus crash both on OS X and my Windows VM.

Edit: This is really weird, I managed to get it to run on higan, and it actually displays the correct result, even though it should have been 99 in higan. Got to do more tests!

> Maybe Game Boys are all full of micro-variations like this. Wouldn't that be a treat?

Like some of the audio quirks? That would be wonderful! not really

Anyway, I thought about this, so I added a new check to my test. It now verifies each LY value at the start of every OAM-STAT interrupt before writing it to Cart RAM. It expects it to be 0 through 0x90 incrementing by one. If it encounters any other value, it now displays an "NN" screen. So this gives us the results on-screen in addition to the saved RAM. If any Game Boy models deviate from the GBC, we should be able to tell with this new "NN" screen, since it will mess up the expected pattern of LYs.

I was a spoiled kid, so I have quite a few GB models to work with: MGB-001, CGB-001, AGB-001, AGS-001, and a Game Boy Player for the Gamecube. Tested on everything but the GBP, but the results were the same. They passed this new test, so as long as this test is written correctly, it means this behavior should be consistent, OAM-STATs on Lines 0 - 144.

@LIJI - I'll take a look at your test more in-depth in a bit. Tuesdays + Wednesdays are the busiest days of my week, however, so I might not finish up till later. Thanks for sharing though, and keep up the awesome work on the GB Video Player!

EDIT - FWIW, I can replicate your results with your test ROM on my GBC, first value is 0x97. I'll dig into the assembly tonight if I get time.

I tested oam_count_v5.gb with all handheld devices I own (listed here here) and the test passed on all of them. Can't test SGB/SGB2 at the moment, but I'm 99.99% sure the test will pass on them as well.

Just curious... are you the same LIJI from the Clickteam / TDC forums?

Yes.