Converting Simple PCM Samples to Pulse / Triangle / Noise?

Well, I thought it would be fun to look at using FamiTracker, not because I want to work on the NES, but because I just think it would be interesting, and I saw that the options that you have when creating an instrument are Volume, (except Triangle, which apparently doesn't have volume control) Arpeggio, (Not even sure what that does or means...) Pitch, (same thing as the keyboard?) High Pitch (Just a more extreme version of pitch?) and Duty / Noise (I'm not sure what this does, other than make the sound different somehow). Now, I'm no sound expert, (I'm not even sure as to what PCM, Pulse, Triangle, Noise, Square, Sawtooth, or FM even mean or work) but since you can make pretty much any sound into PCM, can you somehow reverse this process, even if it will obviously be "lossy"? Or will this just sound completely terrible and unreasonable to do for a machine, like trying to convert assembly code to C? I'm just trying to make an electric guitar type noise, if that's possible.

It's lossy. Very lossy. A few years ago, lidnariq experimented with converting drums to noise by estimating the frequency and volume of each 1/60 second segment.

For electric guitar, you'll probably want to create a duty envelope that starts on 2, goes to 0 for one or two frames, stays on 1 for a few more frames, and ends on 2.

The Arpeggio envelope changes the pitch of a pulse or triangle channel down or up each frame by a number of notes. I use it to create more interesting drums (arpeggio + note), tom-toms, and "plinky" attacks. Others might use the Pitch envelope for things like that, but I prefer the Arpeggio envelope because it more closely matches the data model of my own music engine.

Oh yeah, I hadn't even thought of the fact that you could put multiple channels together to make a sound effect. That adds a whole other layer of complexity...

Anyway though, is there a way to make it to where the sound will repeat for the frame instead of just ending on the last note, or would you just have to keep copying it over and over again?

Oh yeah though, one thing I'm curious about though is is this actually exactly how the NES sound hardware works, or is it "simplified" (like Mario Paint or something)?

Its the shape of the sound wave...

Square _|``|_|``|_|``|_
Triangle /\/\/\/\
Sawtooth |\|\|\|\|\
Noise - pseudorandom wave

FM is short for frequency modulation, and is a style of music synthesis circuit. (not used by the NES, I'm not sure how this came up)



Each note using this instrument will pitch shift (relative to the base note). High pitch is faster. I think for nearly everything you will NOT want to use these... but we had a conversation recently about drums that used slight pitch down, could be set here.

Arpeggio - rapidly alternates notes (to simulate chords, I suppose).



Not sure what you mean. The NES is a simple waveform generator. Not at all like any more modern system. The DPCM channel is closer to what newer systems do (play samples)

The way the audio hardware actually works is described on the page "APU" on NESdev Wiki.

As for sound effects using multiple channels, that's common. In fact, drums in a lot of games use both the triangle channel and the noise channel. You can experiment with multichannel sound effects using my sound effects editor. It's actually an NES program, which you can run in an emulator or even on your NES if you have a PowerPak, EverDrive N8, or the like.


To get started with making drums, here's the drum kit I usually use in FamiTracker:

Kick/Snare
Play at 0-# for kick or A-# for snare
Volume: 12 10 9 8 7 6 5 4 3 2 1 1 0
Arpeggio: 10 0

Closed hat
Play at C-#
Volume: 5 4 3 2 1 1 0
Duty: | 0 1

Open hat
Play at C-#
Volume: 5 5 4 4 3 3 3 2 2 2 2 1
Duty: Same as closed hat

Oh, screw it. I can't use this tool if my life depended on it, I've gotten frustrated trying to replicate sounds that probably just can't be replicated, (I simply can't stay in track of volume and archipelago and pitch and whatever duty/noise is at the same time) and running out of sound channels in like 5 seconds. It's no wonder VRC6 songs seem to be more common than stand alone 2A03, just like how Mario Composer songs seem to be more common than Mario Paint songs, although I think when you stack a bunch of crap onto it, the point is kind of lost. (Although it is harder to make, (Just like 2b) so I'm not trying to take credit away from the people who have made awesome expansion chip songs.) It's like songs from Genesis games using the VRC7. Isn't that in conjunction with the 2A03 practically the Genesis's sound hardware? I'd be like if someone made the "VRC8" that had 8 PCM channels and you recreated the entire DKC soundtrack. You still have to admit that this is awesome though: https://www.youtube.com/watch?v=WlaDLLdvGhc

and speaking of the Kirby Super Star and the SA-1, I found this by the same guy: https://www.youtube.com/watch?v=2Alq81a_wV0 Wow...

You know, isn't there supposed to be a "SuperFamiTracker" for the SNES being made? That way, you could have any instrument sound exactly like you want it to, as long as it fits in audio ram. I really hope there is an option to swap out data during the song, as I plan to do. The biggest problem with the SNES's audio is that because of the small space for audio, it's often fairly low quality and muffled sounding as a result, which I plan to try and avoid. Some audio samples also seem to hold up better than others, or maybe it's just that they have more space dedicated toward them. I mean, hear this: The beginning sounds like it came right of a CD, while the symbol/drum sounds things in the background latter in the song sound muffled in comparison, and considering the instrument at the beginning isn't even found in the rest of the song, they could have just swapped it out.

Anyway, one last thing about the NES's audio, I thought of a potentially stupid but potentially good idea: Instead of just using one audio channel to try to replicate one instrument, why not just try using all the audio channels in conjunction to try to replicate the whole song? Easily the largest problem with this would be that it would take up a ton of memory considering just about every second of the song would have to be a unique instrument, and each channel would have to be a unique instrument. The other problem would be that some "instruments" in the song could make other "instruments" sound different, kind of like when I suggested creating the best 256 color palette out of what would be a 16bpp imagine in an 3D SNES game vs. having the same 256 color palette like Doom. At this point, it seems like it would just be best to use the DPCM channel, and speaking of which, I forgot, what besides memory is restricting you from just recording a whole song and playing it in DPCM like an MP3 file or something vs. a MIDI? I just remember hearing that there's almost no CPU time for anything if you do this approach for some reason, and I don't think it had to do with compression.

Sometimes there are hard decisions to make when arranging music for 2A03. If you can prototype your music in ModPlug Tracker, perhaps I can try arranging it.


One approach is to convert XM music composed in FastTracker or ModPlug Tracker.


Very few Super NES music engines supported that, as far as I can tell. Streaming in samples needs attention from the CPU, which causes slowdown if not done very carefully. Tales of Phantasia does, I think The Lord of the Rings (which uses ARDI) does, and I think some fighting game engines support streaming in vocal sound effects. In any case, games that dynamically load samples would be listed as "undumpable" on snesmusic.org, as an SPC700 save state is insufficient.


That's possible if you are willing to take memory use to the max (300 that is). But the (NES) PowerPak supports only up to 512K of PRG ROM, and only 248K in NSF mode.

DPCM isn't good for complex sounds because at the rate of 1 bit per sample the quality is really bad.


It's PCM that consumes all of the CPU time, not DPCM. The CPU can fetch 1-bit DPCM samples from ROM automatically, but you can achieve much better quality if you manually feed 7-bit samples to the APU. Doing that at a steady rate will consume most of your CPU time.

Don't you need to stream some information to the SPC700 anyway in order to play sound effects at certain times? Wait, it's like transferring one byte vs. a thousand...


What's that?


Doesn't the NES not support PCM samples though? And I'm assuming the reason PCM samples are processor intensive is the same reason as swapping PCM samples on the SNES?

Yes you can do PCM, but like tokumaru said...



And, DPCM consumes almost none, and at the highest sample rate, is not so bad. You can have understandable speech ("fight") and decent drum samples.

The VRC7 is a cut-down version of the FM chip used in the Japanese Master System. Compare JP SMS FM music (on YouTube, 'natch) to the usual great Genesis stuff, and you should be able to hear the relative timbral simplicity of the SMS' FM.

For the technical reason as to why they're different, look up OPLL versus OPN2 FM.

A MIDI-based Super NES sound engine developed by Interplay.


It does, just not in the background. Big Bird's Hide and Speak uses it, as does the Rickroll demo.

I said earlier that NES doesn't use FM. But, now that I looked up VRC7 i see that exactly 1 Japanese game used an FM chip...Lagrange Point.

https://youtu.be/drwX7MbB_IE

Weird, it has such an un-NES quality. Especially those bass lines.

Yes, but like I said, there's no automatic mechanism to stream the data from memory, like there is for DPCM. You must make a program that will read the samples from ROM/RAM and write them to the appropriate APU register in a timely fashion, and while the program is doing that it can't possibly run a game engine at the same time.


Not really... If my understanding of how SNES audio works is correct, sending the samples to the audio RAM is the slow part, but once that's done, the samples can be played back at no cost. And even if you need to change the data over time, you at least have a buffer to make it easier. On the NES, you can never stop sending bytes to the APU, otherwise the gaps would cause audible glitches. Even the sprite DMA, which consumes around 514 consecutive CPU cycles, is troublesome.

So really though, switching instruments mid song on the SNES shouldn't be that hard, that is, if these instruments aren't played right after each other. There are 60 frames in a second, so if you where to about 32 bytes over in a frame, you'd send over 1920 bytes a second, so about 2KB, which should be about perfect. (When changing an instrument, it's always all right to start the instrument before actually finishing writing it to audio ram, just as long as you finish writing it in time.) I don't think sending 32 bytes over a frame should be difficult. You know, at 32000hz, how many KB of instrument data would you go through in one second?

The SNES APU's BRR gets sixteen samples stored into nine bytes. So

32000 samples/second × 9 bytes/16samples = 18000 bytes/second.

Well, even if it does take up that much memory, I imagine you could actually be able to get away with many instruments being half a second long (9000 bytes) and just keep looping it and changing the pitch or something. (Of course, many samples will sound almost exactly the same at a lower sample rate too.) I looked at the Gunforce 2 instruments again (I was trying to look at waveforms and seeing if there was any hope of copying it on the NES) and although the guitar sound kind of loops for whatever reason, (and the pitch is a little higher and it's louder at the beginning), I saw that it fits in a little less than half a second at 20000hz. (Which sounds like the correct speed.) Considering that that's about 2/3 of the highest sample rate on the SNES, if you multiply 9000 by 3/2, you get 13500 bytes, which is a little more than a 5th of audio ram. Although that isn't to astronomically terrible, I'm sure you could also still make the sound effect shorter by shortening the loop or just decreasing the quality by a little. What would have been a great idea for Nintendo would have been to taken 32KB of main ram and put it toward audio ram, and put the other 32KB to vram.

You know though, what's actually so CPU intensive about transferring data to vram? I heard it's that the SPC700 runs at a slower clock speed than the 65816. Because of this, you have to somehow have both chips be in sync with one another, but how do you do this? Anyway, if you try to do this, wouldn't just one write cause you to have to wait for both chips to be in sync again? Does the 65816 not have the option to run at the same clock speed as the SPC700? If so, you'd just turn it on, transfer the data, and then set the speed back to normal again.

The thing that is so CPU intensive is that neither the SNES CPU nor the APU CPU have the ability to fire an interrupt in the other.

As a result, there's only a few ways to manage transfers:
* S-CPU writes one to three bytes and a handshake byte, and waits for the A-CPU to write an acknowledgement. The A-CPU waits for the handshake, transfer the data accordingly, and writes an acknowledgement.
* S-CPU writes one to three bytes and a clock byte, assume the other end will keep up. The A-CPU waits for the clock byte to change, then transfers the data accordingly.
* S-CPU/HDMA writes data at a fixed rate and the A-CPU maybe drops it on the floor. (The two CPUs run off entirely disjoint clocks, one of which can and will drift relative to the other, so fixed timing loops become trickier)

Unfortunately, using a timed IRQ to transmit data adds so much overhead to the S-CPU that it's not clear it's worth it.

Perhaps the best hope for a rapid transfer is to use a 3+clock loop with an HDMA channel. That doesn't use quite as much S-CPU time as a fully synchronized PIO loop, but the S-CPU has to copy the data from ROM to RAM in order to insert the clock bytes, and the S-SMP has about 64 cycles to stash 3 bytes and wait for next clock.