blargg's SPC test ROMs

Apparently byuu managed to find blargg's long-lost SPC test ROMs on an old thumb drive. They were posted to his board, but I figured somebody around here might be interested in them.

- spc_dsp6.sfc
- spc_mem_access_times.sfc
- spc_spc.sfc
- spc_timer.sfc

Download mirror 1
Download mirror 2

I may play around with disassembling them if I find the time.

Thanks for posting those.

Now I have what I believe is a somewhat late PAL SNES that seems to hate a lot of blargg's tests.

The DSP test seems to deviate during the "KON then KOFF" test. KON should immediately silence the volume envelope, set the state to attack and start increasing the volume envelope and playing the sample after the initial KON delay. Setting KOFF should set the state to release and decrease the envelope by eight every sample.



I would imagine this test examines the envelope value, so it wouldn't need a sample. It looks as though 00-07 are the channels with the next ten values being the envelope output. According to anomie_dsp there the high bit should always be zero, so that column of 80 seems rather mysterious. It might be reading the value as it is getting set and picking up noise, but perhaps there is something else going on.


Now the memory access test, this one has both a standalone version as well as being included in the SMP test. My SNES fails both. The CRC is different from one run to the next.

This is the output from one particular run of the standalone version:



The output is a breakdown of opcodes that contain cycles that involve memory accesses, but the output here is notably wrong in a lot of places. "R" is "read", "W" is "write". "D" is data which can be either read or write depending on mode. The numbers are first read, second read, and so on.

I wouldn't expect for there to be actual differences in the cycles themselves. It's far more likely there is some inconsistency with DSP writes on my system that the test doesn't account for. The technique should be to have the echo buffer write at or around the point where the SMP would perform a memory access.

Timing is absolutely critical here, so if the method used to synchronise the SMP and DSP is off by one on some systems then it's going to cause problems. There might be a few different ways, but I would suspect watching for timer updates may have been appealing.

I did stitch that image together, but the intermittent black lines indicate the screen is updated using forced blanking as opposed to vblank, so either test failing does not seem at all likely to be caused by anything timing related on the CPU/PPU side.

I haven't looked at what either test is actually doing, so my interpretations could be very wrong. They might even be doing something even more exotic.


blargg's mul/div tests never behaved on my system either. The multiplication one at least produced a consistent CRC, but the division one was very erratic producing one of thirty or so different hashes at random. Neither test ever passed.

Perhaps a few people with a few different consoles could test all these and note whether they pass all tests or fail any of them? I'd at the very least like some reassurance that these anomalies aren't completely unique to my own SNES.

If anyone has better/different explanations for any of these, that would be nice too. ^_^

I'm pretty sure the tests were written for NTSC consoles, so it doesn't surprise me that they fail on PAL consoles.

I do not believe there is any relevant difference between NTSC and PAL modes that could explain any of this.

There is different frame timing, but that only really affects interrupts and when one can write to PPU registers. As mentioned, these tests bypass that entirely by enabling forced blanking whenever they want to write to the PPU. To do that repeatedly on demand is not a technique I've ever seen but when I saw the black lines I knew what it was doing right away. In this context it makes sense.

The components of the APU are of course completely oblivious to such things. The SMP and DSP are clocked together independently of the master clock, so region means nothing to them.

The CPU clock relative to the SMP is the only potential issue, but to run both sides independently like that would be catastrophic on any system. If the CPU side was timed to run on its own, it wouldn't output an error, it would probably just hang indefinitely. Normal protocol would be to communicate back and forth and to wait for responses as appropriate.

So yes, I'm of the opinion that unless my own SNES is particularly strange, it might stand to reason that some consoles would exhibit the same behaviour regardless of region.

Only the spc_dsp6 test fails for me:

Thanks for testing. My own system is dated 1995 with the 2-in-1 APU. In that post you mention having two consoles so I might assume you tested the 1992 console? At any rate I would expect either one to have separate SMP and DSP. It's nice to know there is some variance on other systems too.

Now "brr addr wrap-around". BRR is decoded in blocks of nine bytes and that output contains addresses ending in 8, so it might start each test from xFFF with the end flag in the subsequent block, at either x008 or x009.

There doesn't seem to be any way to directly know where the BRR is being fetched from at a given time, but if you know the pitch settings and SMP/DSP alignment, you can count cycles, keep a separate counter and wait for EndX to get set.

In this scenario every entry should end in 008, so the ones that don't... I'm not really sure. Maybe EndX is getting set late or something. Although 514E and BE44 are all kinds of crazy. Perhaps if the decoder missed the end flag for whatever reason and just kept going until it hit one somewhere else.

Although who knows. Perhaps the test does something else entirely.

blargg is still around and contactable via Email. Asking certainly wouldn't hurt! For all we know maybe the test ROMs were WIP and had bugs.

No, the 1993 one (Super Famicom, switch set to "NTSC"). The PAL console is not connected often.

I'll do some more tests later.

Oh, well that's even better.

One thing that was bothering me is the purpose of a "BRR wrapping" test in the first place when it seems like such a basic thing like memory layout couldn't fluctuate at all. The explanation that makes sense to me is that it's not in any way intended as a system diagnostic, but as an emulator test intended to aid development.

One of the discoveries in the recent era is that the DSP ticks at three times the rate of the SMP. So instead of the 32-step sequence outlined in anomie_dsp to generate a stereo sample, it's a finer 32*3=96-step sequence. It's wishful thinking, but I wonder if all this could be the result of a finer alignment issue?

That theory would give us three fixed outcomes. So far we've observed two different "fails" and hopefully the test actually passed to completion on someone's console at some point. That gives us three states. If other consoles can reproduce something close to either observed fail without giving us a completely different one then we might be on to something.

That still wouldn't necessarily explain the particular output of the fails we've observed though. The different hashes I observed in the memory timing test are particularly disturbing. I wouldn't have expected alignment to be persistent either, I would have instead expected it to fluctuate rather like what the NES CPU and PPU do.

And of course if a different fail turns up the alignment theory goes out the window and we're back to the "some consoles are just weird" theory unless someone can come up with something better.


I'm under the impression that it was somewhat a work in progress and this is one reason it was only distributed privately.

Then again, this area of the system is largely perceived to be deterministic so the same test producing different outcomes consistently is...notable.

Suddenly quizzing him about the incredibly fine details of something he worked on almost a decade ago would be uh, rude. With regards to the role of the test in this puzzle, I'm feeling fairly content, For, Now. Mistakes are possible though and I'm open to ideas.

"Seems like", at least until you read "0-days hitting Fedora and Ubuntu open desktops to a world of hurt" by Dan Goodin.

Super Famicom (switched to PAL):
spc_dsp6 either fails with the same error as above, or it hangs at (after?) the "Misc/counter rate synchronizations" step; the other tests pass.

Super Nintendo (PAL):
Same as above, except that the error is "Failed 02".

I didn't expect the 96:32 theory to fit, but the fact you have two separate consoles that produce similar errors is very encouraging.

If a completely different error had cropped up, it still would have been possible for separate SMP+DSP to produce three different states than the 2-in-1 package, but that theory would certainly become more uncomfortable.

The small variations are still unaccounted for though. The only other thing I can think of that might be relevant is loosely related to the Soul Blader problem. That one doesn't initialise RAM properly so it has a track (or tracks?) that sound different depending the contents of RAM at boot. It turned out there were a handful of different manufacturers of said RAM with different boot characteristics. It's almost certainly not uninitialised RAM causing problems here, but perhaps there is another factor at play? Maybe something very subtle like speed, or perhaps some of them just don't like writing then reading from the same address on adjacent cycles?

These are all just ideas though, and without more samples we can't really draw a definitive conclusion.


Right, these are things an emulator might fail but it was expected that a real console should not. There is at least one thread discussing that particular bug and if I recall, the conclusion was that it's caused by indexed instructions being able to index out of bounds. If the effective address is limited to 16-bits like a real console presumably does, that particular bug would never occur.

The test says "this is how (my) console behaves, so it's how emulators should behave too", but now we're facing the problem where we have some consoles all with linear RAM that behave differently. How emulators might approach these things are outside of our scope for the moment. One would have to have a firm idea of what's going on before it could be emulated effectively and there are too many guesses just yet.

Oh no, I really hope that I don't cause blargg any more trouble with these ...

It's just, when I initially asked if anyone had them, it ended up all over social media, and people continually asked me about them.

So when I found them, I thought I'd let everyone know. And then multiple people asked me for the files. All of blargg's other WIP tests are online, so I felt it would be okay give them out. I really hope that was okay now ...

...

Yes, they were work-in-progress test ROMs. The 6 in the filename is because blargg would send me a new version periodically, and that was the final version I received back at the time.

There are a huge amount of variations in real-world SNES consoles. Indeed, passing these tests are not a proof of correctness as such.

What they are, to me, is a proof of correctness of blargg's DSP core. Which is invaluable. I need to make an extremely drastic DSP core change due to recently discovered peculiarities in Magical Drop: it seems the initial register values are non-deterministic and yet reading from the register ports are. But blargg shared the RAM and registers as a 128-byte array. I've been very afraid to attempt rewriting the core to split the two, because without these invaluable test ROMs, I could not be certain I hadn't introduced a painful regression.

As to the hardware variances, I have been emulating them as I find them (HDMA init timing, DRAM refresh position timing, etc), but I have not exposed them to the GUI just yet. We should do the same for variations we discover in the DSP as well.

Because Nintendo stopped incrementing the CPU/PPU chip version numbers (and the SMP/DSP never provided one), I think the best bet will be to use the board PCB serials instead (eg the RGB, GPM, 1CHIP, etc designations and their revisions.)

...



Strongly agreed. If you need to, please contact me instead and I'll do my best. I will need a few more weeks to get set up here first.

Hi byuu,

Is there any form of documentation retarding these roms? I am writing a SNES in VHDL to run on an FPGA and these files are ideal to test the APU )CPU opcodes, DSP, etc.). However I get the following messages when running spc_smp.sfc and spc_dsp6.sfc.

I can see I have issues with my spc700 opcodes but I can't work out where they go wrong and the 8 digits to the right of each opcode don't make sense to me.

It keeps a running hash for all the opcodes it tests. If it detects an irregularity, it outputs the hash it has between each opcode loop.

In your case there is divergence at opcode BE; DAS.

Great, thank you. So what you sent me is what I should be outputting. Let's have a look at DAS.

Hi Kingizor,

How did you get this output. Is it via an emulator?

I can now get a pass but then it then freezes. Is it possible to get hold of the assembler code to see what the rom does.

It seems to run one pass with the running hash, then checks the final result against what it expects. If the two hashes don't match it runs a second pass but logs each interim result along the way. Therefore, one can make it produce that list by damaging the running hash as the first pass is being run. By doing that in an emulator that would otherwise pass, it willl produce the list of "correct" hashes for each step.

Your output (in particular the "Passed 01" message at the bottom), indicates all the listed opcodes are fine so the issue seems to lie somewhere else. This is where things get tricky because there are a lot of reasons things might go wrong. It looks as though it completes the first pass successfully but doesn't manage to convey that back to the CPU, so it ends up running the second pass anyway and uploads the results.

I'd check that writes to $F1 are only clearing the ports on the SMP side, and that the 4-in, 4-out behaviour is correct on both sides.

If it's neither of those, then I'm not really sure. Whatever it is, it's probably not something that would affect most games, so I'd be happy to ignore it for the time being.

Thank you again Kingizor. I have checked the APU IOs and they work fine. There must be something else wrong and I suspect it comes from my CPU code. I need to to test/simulate each opcode with a fine tooth comb.

Really aggressive CPU instruction validation is a large missing component in the entire emulation scene.

The test suites all rely on the CPU core to be mostly functional, and some of the video circuitry implemented, in order to even run them.

But if you have a bug in a critical CPU instruction, you're in for a world of pain. The best I've found is to find another emulator and generate an instruction trace log from it, and then go line by line looking for bad flag / accumulator calculations.

A testing framework needs to be able to evaluate each instruction, without any other part of the CPU being present. That's not an easy problem, and one I'm not up for trying to solve right now, but at least you can be ~95% sure that someone evaluating a CPU interpreter is going to be using C or C++, so C would be the language of choice for the evaluation framework.

Then perhaps the answer is to start with a direct port of "nestest" (a basic 6502 test suite by kevtris) to SPC700, so that an emulator developer can detect where PC/AXY diverge from a known good trace. This should help shake out the most obvious problems with "critical" instructions.