Sound synthesizer using $4011, 2a03 only

I just got an idea that would be awesome for NES development. Basically it would allow an undefined number of extra sine audio channels, probably 1, 2 or 4 channels, with volume control, and that using $4011. I probably won't ever be able to put this idea to good use (as to, make a decent game using music that uses this), so I prefer to share the idea and hope someone else use it. If they give me credit, even better

The main idea is to use a mapper that supports timed IRQs that are fired at a programmable arbitrary rate. I think VRC mappers, and the FME-7 supports this, and the Famicom Disk System. Perhaps other more obscure mappers as well, I don't know. The rate can be anything really.

Now, if we can generate IRQs periodically, it's possible to write to $4011 periodically without monopolizing the CPU for this task alone, and thus get an extra sound channel, that doesn't sound terrible like DMC does. The problem is that doing it the "trivial" way is just copying data from ROM to $4011, but it wastes tremendous amount of ROM to replay samples, so this is not practical for a NES game - only for tech demoes.

Another idea would be to synthesize sound, but this requires CPU power, and the NES bascially doesn't have that. But I think I found a way to synthesize sound using very low CPU. The idea is to generate sine waves, and take values from a sine table which is stored in 256 bytes of ROM. Sine waves have no harmonics, and no harmonics aliasing, so it's very easy to resample them to any rate with a fixed sample rate, and it will continue to sound good, unlike any other waveform such as square or saw waves.

So the IRQ mixing code looks like something like that:



The mixing code is thus relatively simple and fax, and executes in constant time. But that's not all, get a bunch of sine waves is not very exciting in itself. What is exciting is that you can control their volume, and that without touching the code above, by pairing two dephased sines of the same frequency. In the sound code, something like that:


Will combine Chan1 and Chan2 sines to produce a sine wave of controllable volume, and that without using any multiplication anywhere. I just tought this idea was very cool, so I wished to share it. I have no idea how much "channels" would be feasible in practice, it's a tradeoff between amount of channels, CPU usage and sampling frequency (and thus, maximum sine pitch). I think expecting 4 sine channels without volume control, or 2 sine channels with volume control, could already greatly expand what is possible to do for NES music !

I didn't mention it, but of course the sine table is pre-shifted so that it doesn't overflow, so maybe it's range will be -64..64 or something like that instead of -256..256.

Aside from every other issue with using $4011, sprite DMA is a big latency hog. It takes 4.5 scanlines of uninterrupted time to perform sprite DMA.

Back to random ideas about the code...
If you updated at ~5000Hz, that would be running the sound code once every ~3 scanlines. If the update code ate up 113 cycles, that would be a loss of about 33% speed overall.
There would be aliasing, since it changes at discrete levels.

This is a serious problem. The only sane solution I can think of is doing nothing and let the IRQ handler run late, which will create a hearable distortion, but hopefully still tolerable...

My count is 49 cycles for the constant part and 39 cycles for the looped part. With 4 channels w/o volume or 2 channels with volume, that makes a total of 204 cycles. This is a bit optimizable by unrolling the mixing loop and storing registers in ZP instead of the stack.

Mockup of what a song at 5512Hz (aliased) might sound like.
Just a mockup, some simple song made with a sine wave and downsampled to 5512Hz, and resampled to 44100 with aliasing.

So, with this code, we get 2 sine waves of controllable volume in a 110 cycle IRQ routine. It could be possible to save a few cycles by placing it in RAM and use selfmod-constants for variables used only once, that is the Freq and Volume variables. It is not possible anymore to get 4 sines without volume control, but I think the mega speedup makes it worth the deal. I ignore how many cycles it takes for the 6502 to enter IRQ, does anyone have any idea?

This creates a situation close to the one described by Dwedit, where code could be run every 3 lines with 1/3 CPU usage.

I don't think the demo sounds very good, but it doesn't sound horrible either, and remember, it's NES, and 2 extra sound channels in addition to the normal 2a03 stuff. We could use only 1 channel and update faster, too, it could sound better.

You'd get less distortion if you could skip the update and catch up instead. Shifting the whole timeline back causes worse problems than a sample-and-hold over a missed sample while retaining consistent timing.

IRQ is 7 cycles in (same timing as BRK) and 6 out (RTI).

Wow, yeah that's harsh. I think the NES' lowpass filter would help slightly, but the SNR here seems way too high for it to help enough. I think partly the problem is just that the sine wave is the one signal that will be most noticably affected by aliasing, since it has no other component frequencies to cover up the aliasing. Maybe a brighter waveform (e.g. saw) would make it slightly less noticeable, but 5kHz is really prominent to most human ears. This samplerate might just be too low to be practical.

8kHz was a common samplerate for a while, but even that was usually used in conjunction with a suitable lowpass filter.

Anything else than a sinewave rules out the oportunity of smooth volume control while using only 256 bytes of ROM, and make playback at any given frequency very difficult without major aliasing problems.

Yes, I think by mathematical definition no other waveform could use that phase trick to control volume.

There are probably other waveforms that have a smooth volume control at low cost, though, by other means. A square wave, for example, could trivially do it, and wouldn't need a lookup table either.

The mockup doesn't sound too bad to me. Call me crazy but I might even prefer a slightly harsher sound to a proper (overly clean) sine wave. However, the OAM DMA will distort it further. In specific cases (demos or simple games) one might be able to get by without OAM DMA (at 4+4 cycles for LDA/STA per byte, it'd take exactly one vblank to upload all sprites manually).

The trick to control the volume is pretty cool.

Actually, I'm curious about Dwedit's example now; was going by how it sounded, but looking at the OGG in an editor the resampling seems to have emphasized the aliasing?

When you use a harsh brick-wall low-pass filter for resampling, the Gibbs phenomenon causes overshoot by 10 percent, the height of the first sidelobe on each side of sinc. The visual effect of this ringing is to emphasize the aliasing. To minimize ringing, use a less sharp low-pass filter, such as a filter with a sinusoidal rolloff between 3f/8 and f/2.

Yes, I know it's gibbs ringing, but I was not expecting this much ringing from the kind of simple conditions I presumed Dwedit was simulating. It might have been introduced by the OGG compression method too, unsure. The thing is, you can't build a suitable lowpass filter on the NES, the only possible resampling is sample-and-hold. There is a filtering from the "ideal" sample-and-hold to the 44kHz signal, but the effects of that are so small here as to be irrelevant; the primary problem of 5kHz aliasing is already dominating.

I wrote a quick python program to generate an example of my own, easy to modify to add other conditions. Naive resampling, sine table quantized to 8 bits, frequency sweep, you can modify it in other ways if you like (e.g. 60hz interruptions of various kind), but you can hear from the example already that the aliasing is quite strong.

So... Dwedit's example is not far off from mine, now that I've tested it; I just wasn't sure what was going on because of that ringing, which was unexpected. The aliasing is pretty strong.

Can an all-up or all-down DMC sample reduce sample and hold artifacts?

Interesting idea. I guess at best it would be linear resampling, and at worst it would be no better than sample-and-hold. I think between the +/- 2 increment granularity of DPCM, having only a few useful frequencies to choose from, and latency of the DMC channel, the practicality seems dubious to me.

You could modify my example for a linear resampling just to inspect the upper bound of quality improvement, if you like.

Next question...
Let's say you synthesize an entire frame's worth sound data once per frame. Then your IRQ handler just becomes something that spits out one byte then increments the counter. Sine Wave synthesis would still be fast, only it could be done ahead of time rather than in time with the IRQ routine. You could also run the output handler every scanline or every other scanline.

Once per scanline (15720Hz) is 262 bytes per frame, once every other scanline (7680Hz) is 131 bytes per frame. (yeah, I'm multiplying by 60 here...)

Yeah, possibly a nice memory for speed tradeoff.

I was thinking that this might be a reasonable use case for $2003/$2004 OAM updates. You could use it to update a small amount of sprites each frame, allowing animation. Obviously wouldn't be great for gameplay, but you could still do some animated sequences with limited sprite update bandwidth.

I ended up getting really interested in this idea, so I decided to make a weekend project of it.

I wrote a player that software mixes 2 square channels, and outputs their result every 102 cycles (~17547 Hz), and interleaves it with updates of the other channels. No IRQs or anything, just cycle counted code making full use of the CPU (very stable timing this way). Still working on a way to make the data stream a bit smaller, but the player itself is about 750 bytes.

The resulting aliasing at this frequency is not too bad, and it almost adds a little interesting slightly squelchy texture. I made it to take input as a VRC6. It ignores the saw channel, the DPCM channel (obviously), sweep, and the duty on the two extra channels is always square. I could probably add a saw channel and duty control for a full VRC6 recreation, but the samplerate would be reduced (and aliasing increased); not sure if I really care to do that at this point; I really wanted something to make new music with, not duplicate VRC6 specifically. I used VRC6 just cause it's easy to make data for in Famitracker (and since the squares are 16-bit phase oscillators, they have extended frequency range, so the VRC6's extra bit helps).


Here is a demo playing a modified version of a tune from Akumajou Densetsu (famicom Castlevania III) but with my player substituting for the VRC6. (I also added some extra percussion to the noise channel to make up for the DPCM being lost, and moved the saw channel to the triangle which was unused.)

http://rainwarrior.ca/projects/nes/ad_sound_demo.nsf

I was thinking of hacking something up based on this thread too. But that NSF sounds brilliant, rainwarrior.

Bregalad's volume control trick is pretty cool, as a composer I've never been a big fan of the sine wave sound though, it's kinda.. naked. I can remember doing some tracks where every channel used sine waves and I liked that well enough, but I've never felt the urge to mix a sine wave with other wave channels. Though I suppose in practice it doesn't sound all that different from a triangle wave (but I've never tried multi-channel triangle waves either). I'm sure others could do something cool with it though.

Anyways the idea I had was to try 4-bit artibrary waveform with 4-bit volume control. That way you could just OR them together and use a 256-byte LUT.



Yeah, definitely. I remember expecting my original Squeedo synth to sound like crap with sprite DMA, but it was hardly noticeable for just that reason. The synth itself kept running, the NES just missed the samples and held the last one during DMA. With just the 2A03 I suppose we'd need the update rate to be divisible into 512 (or 513) to keep things kinda sane. I think Dwedit's pre-calculating suggestion is probably easier to deal with.

Actually one of my WIP mapper designs (another el-cheapo one) could have an 8-bit CPU cycle counter at no extra cost really, I figured it'd be perfect for something like this if someone was crazy enough to use it in something.

For those without a 2ear03, was there any sweep, and were the extra channels nonsquare in the original piece?

I made a ROM version in case that's more fun:
http://rainwarrior.ca/projects/nes/ad_sound_demo.nes


The original piece didn't use sweep, but it definitely used various non-square duties, and it had a saw channel too (moved in my arrangement to the triangle, which it didn't use for anything). It was also using DPCM for drums, which I added stuff to the noise channel to replace. It's not supposed to be an exact replica, or anything, just a proof of concept.

Years ago, I did composition with multi-channel triangle waves, albeit with others for percussion (ogg | tracker source). I eventually remade it on the NES as a stress test for Pently's channel interruption features (ogg | nsf | Pently source).

This demo indeed sounds very good ! I am surprised that you were able to get square waves sound that good, my personal experience was that square waves sound awful when at low frequency and are neither filtered nor a frequency which evenly divide the sampling rate.

However, correct me if I'm wrong but you used a sample rate much, much higher than what Dwedit suggessted.

I am not against the idea of buffering samples and replaying them back during the frame, but it creates some other issues, such as the sampling rate needs to evenly divide the VBlank rate. It's not an absolute need, but is very helpful (hem hem somehow this REALLY remind me the GBA).

My goal was to make the samplerate as high as I could make practical. I figured 2 squares is enough (sort of like a compromise between MMC5 and 5B expansions), and started with something like this:

So I started with these 78 cycles, and tried to think what the minimum I needed between them was. At first I figured I should update registers in groups of 3, to get a whole channel at once, so I set a goal of 27 cycles (LDA zp + STA (zp), Y x 3) in between each sample. That was sort of the initial basis for the choice of samplerate. (Actually, I also forgot that STA (zp), Y was 1 cycle longer than LDA (zp), Y, so I mistakenly believed it was going to be 24 cycles while working on it.)

After I wrote this version, though, I realized that it didn't really matter if I updated all 3 values at once, and I could make the data a lot smaller by treating each register as its own RLE'd data stream. At that point I rewrote everything from scratch, and arbitrarily stuck with 24 cycles between sample updates. That's what determined the samplerate in the end.

I could have picked a number larger or smaller than 24, but 24 was enough. It's hard to change that now without a complete rewrite. The sample loop is easy to change, though, if that number 78 changes, the cycle count of the loop is just a constant I change in the exporter and then I just add or subtract a few "killing time" samples from the main loop to hit a target close to 60 Hz.

The main loop is pretty simple:

At this point, though, it'd be hard to add new features without sacrificing something else, probably samplerate would have to drop significantly. With any more data streams (e.g. 3 more streams for a saw channel) it's easy to just add a couple of new ones to the main loop, but what's not easy at this point is fitting anything else within 60 Hz. There's only 28 "spare" samples right now and each stream takes 14 samples to update. If I use a longer loop length, I get less samples, though... so basically I'd have to increase the in-between (24 cycle) code until I can do more per sample too.

Eventually I could find a balance that fits a given set of features, but I figure I'm already getting a lot of musical capability out of just two squares, and rewriting the in-betweens is tedious, so I don't really want to add more features at this point. (I'll release source eventually, in case others want to play with it.) Rewriting the in-betweens isn't too bad, actually, maybe 150 lines of code, but I'm not much interested in doing that over and over, maybe I'd do it if I had a cool idea.

The one last thing I want to do is add some internal repeat detection to the exporter, which should help reduce the data stream sizes a lot more. Updating a channel really only takes about 7 samples ideally, but I basically made it twice as long to accomodate this repeat feature. Currently it's just looping at the end of the song, but it can actually be used for arbitrary repeats.

Oh, also I should also mention that I borrowed shiru's lightweight 2A03 emulator from famitone2/nsf2data to make my exporter. It came in very handy.

Also, since it's entirely CPU driven, if you run it on PAL everything adjusts to the PAL CPU rate, so it stays in tune but slows down by ~7% (56 Hz). Much less of a change than the usual 20% 60/50 Hz difference, so for cross-region code you might even just leave it as-is, or you could just adjust the number of empty sample loops per frame to bring it back up to 60 Hz.

If you're using almost 100% CPU usage, it gets a lot more competitive. SuperNSF does GBA-like mod playback on a NES.

Yes, I've used SuperNSF before. It is quite good for NSF demos.

I'm not "competing", just having fun making my own alternative. I wanted something that I could use with Famitracker, doesn't require 4k NSF bankswitching, and could fit in a small enough space to maybe use as a title screen or something. Probably I could modify SuperNSF to meet these goals, if I wanted to, but why would I want to?

Here's a beta test and full source of my "two squares" engine, if anybody wants to play around with it.

Edit: I ended up adding duty control, so it's basically VRC6 without saw now, with some aliasing.

http://rainwarrior.ca/projects/nes/dmmc_beta.zip

Was thinking about the idea of subtracting two waves to control volume. We mentioned before that this only works for sine, but I got to thinking about a similar approach with saw waves, or whether there are other practical saw waves?


If you could reset the phase of a lower pitched saw wave every time a higher pitched one rolls over, you could subtract one from the other to control volume; in this case the difference in pitch would control the difference in volume.

There's a jitter problem, though, since if your saw wave is just the high bits of a phase accumulator, it will normally roll over "in between" samples, so there's a difficulty in propagating the phase reset to other saw with correct phase. I think it would slip by a fixed amount each time, causing some sort of cross-modulation. The slip amount could be pre-calculated, though, and added during the phase reset to compensate, but it seems a bit of a complicated solution.


As an alternative, you could just do a single volume-controlled saw as a phase accumulator with a threshold; i.e. every time the threshold is exceeded, subtract the threshold from the phase. This would combine pitch and volume into the same accumulator increment, so the value you added per sample depends on both at once. Probably this would be a bit more practical than the "two saws" idea above.


The VRC6 itself has a very nice way of doing it, i.e. volume/output is an accumulator, but phase is just 7 fixed steps locked to a division of the samplerate (not really a "true" saw, but close enough?). It operates at a much higher frequency than you could do in software, of course, so tuning might be an issue if you did something similar in software (i.e. ~150x lower samplerate).

I haven't got around to testing it myself, but I'm pretty sure you can implement a divisor of the full CPU clock rate in software:


It only gives you one channel, and you'll need the Deflemask trick for timing since the speed of execution changes with pitch. The minimum period would also be quite high, depending on how often the routine above gets called (the exact interval doesn't matter as long as it's always the same), but anything below ~172 cycles starts to sound out of tune anyway, and you can exchange table length for pitch range at runtime.

Yeah, but that's just a phase accumulator again, just you've replaced the saw wave with a wavetable lookup, so it could be any wave. (With the saw, the "wavetable index" is your wavetable already.)

Though, if you did 4-bit waves, you'd only need a 16 x 16 table to do sample * volume * output level in a single lookup. That's probably a better way to do volume.

I was just coming at it trying to think if the "subtract 2 waves" idea could apply to anything else. Maybe not.

Subtracting two saws gives you a variable duty cycle pulse wave, for some SID or VRC6 style sound.

I have the same communication difficulties as tepples, so let me clarify. I was replying to this:

I believe this to be incorrect, and that the code I posted implements the described behaviour: a software implementation of a clock divisor running at the same samplerate as the VRC6, with the caveat that it is limited to periods greater than approximately 172 cycles. The benefit of such an algorithm being that it eliminates audible aliasing.

If any of that is not the case, please point out the error in my logic.

Oh, sorry, I misunderstood what you were implying with the sample. So the "clockslide" is really a variable entry point that determines the length of the sample loop?

So... that lets you adjust the samplerate directly to control the pitch of your generator, instead of having a fixed samplerate. Makes sense. I guess the pitch resolution would then be half the CPU rate. The high end of the pitch range might be a bit limited, though probably fine.

Limited to one channel, though, as you said. Hmm. Interesting approach though.

A clockslide produces a single cycle of delay per byte, down to a minimum of two cycles, so the pitch resolution is the full CPU rate.

EDIT: that site seems to be unstable, and I can't find another good explanation, so here's the general idea:


A jmp (indirect) or self-modifying jmp absolute into any point in a sequence of {C9} C5 EA will delay one cycle for every byte of clockslide executed, plus a constant (the cost of the jump plus the final nop, 5 or 7 cycles).

Ah, I understand. I was just thinking of a line of NOPs.

Hmm, noticed this technique is actually described on the Wiki, but in an inappropriate article for it. Ha ha. Weird.

Blargg got working saw and square waves with DMC channel long ago, combiniing $4011 writes and DPCM it doesn't even require a mapper with a CPU-cycle timer IRQ, and can be done on NROM, and uses much less CPU.

The whole point I had making it on a fixed sample rate was to mix 2 or more channels.

I do not see how substracting saw waves get a saw wave with controllable volume. Anyhow, I fear aliasing might be awful with saw waves, and with general-purpose-4-bit waves as well.

What, polling $4015 for frame counter with interrupts masked?

Yes, that. The code I gave uses 100% CPU but runs at a variable rate, so it can't time notes and envelopes by itself. Interrupts would cause audible distortions, but polling can be synchronized with the PCM.

(Move to "NES Music" please. )

Why? This thread is about programming, not music. (More thorough reponse here)

The only piece of music in this thread is the CV3 test I posted a ways in.

"Discuss NSF files, FamiTracker, MML tools, or anything else related to NES music."

"Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems."

This is very much a thread about a technichal issue related to programming the NES imho.

Much more so than related to NES music.

If sampling rate is an issue, you could try the method I used to get double the normally achievable rate with DMC interrupts (8 272Hz) by sitting in the IRQ handler enough to pump out another sample right in-between two consecutive IRQ-s. It takes more CPU, but you can also spend the time after the first sample to do your calculations and it requires no IRQ source to be in the cartridge.

Interesting idea, though the DPCM sample would be interfering with the samples we're trying to play (unless you're playing PCM on another channel, somehow).

My solution to that problem is to use a byte of $AA or $55 for the timing, so the underlying square wave should be inaudibly high at rate $F. The 1-bit fluctuations caused by this sample shouldn't be too bad.

It's not inaudibly high for me, but depends on your hearing and also your video setup. (Maybe not very noticeable with RF output's filtering.)

Might this be the chance for my old PCM on square demo to shine?