PCM sound requirements?

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
PCM sound requirements?
by on (#163990)
I was thinking about the capabilities of the PCM channel in the NES, and I started to wonder if I could create some samples that meet the requirements/restrictions. I don't think I will truly understand what kind of sounds the PCM channel can handle until I am condensing the samples myself.

So, what kind of specs do I need to reduce a sample to where it could be reasonably played on the NES?
And second to that, just how many samples (or I guess, what's the total length of samples) a game could reasonably hold in its data?
By reasonable, I mean assuming that an actual game is being played, and the system isn't dedicating resources exclusively to playing a sound or some special event. I know some games clearly use more sound data than others, but they tend to come with some sort of cost. Gauntlet 2 has outstanding voices for the NES, but not much else.
What kind of methods are commonly employed to get around these restrictions? Like can a game quickly unload sounds and load in new sounds?

My questions are in regard to both sounds used for music and for sound effects. I would assume that the system doesn't really differentiate between the two, so the only consideration is "how much space do I need for sounds" vs "how much space do I need for instruments." But if I am incorrect in that assumption, please elucidate me.

Also, would I be correct in assuming that the NES is incapable of manipulating the PCM data on the fly? I was playing around with an .nsf player and listened to some songs with only the PCM channel playing, and I noticed none of the instruments ever changed pitch. If so, this would explain why the PCM channel only really used percussion instruments. Even if the chip itself cannot change the pitch of a PCM sample, is it too much for the system to try to manipulate the data directly while a game is running? I would guess so since I can't see any examples of it, but it doesn't hurt to ask.
Re: PCM sound requirements?
by on (#163991)
Marscaleb wrote:
So, what kind of specs do I need to reduce a sample to where it could be reasonably played on the NES?
As a very simple approximation, reduce it to 6 bits per sample at 33144 Hz. Note that volume 15 on the pusle channels is approximately 1/4 as loud (-12dB) as full scale on the DPCM channel.

Quote:
And second to that, just how many samples (or I guess, what's the total length of samples) a game could reasonably hold in its data?
Each sample takes a bit, but the amount of ROM you want to dedicate to DPCM is up to you.

Quote:
What kind of methods are commonly employed to get around these restrictions? Like can a game quickly unload sounds and load in new sounds?
Fine-grained bank switching is more-or-less required for any significant amount of sound. Only MMC3-class and better mappers usefully support the ability to bankswitch DPCM, so it's not that common. The rest used at most a KiB or two of fixed DPCM samples.

Quote:
I was playing around with an .nsf player and listened to some songs with only the PCM channel playing, and I noticed none of the instruments ever changed pitch. If so, this would explain why the PCM channel only really used percussion instruments.
It can, but they're not all that convenient. Only 16 different sample rates were provided, and they're not quite right.
See: nesdevwiki:User:Lidnariq/DPCM_mistuning
and: viewtopic.php?p=92494#p92494

Quote:
is it too much for the system to try to manipulate the data directly while a game is running?
The region of memory that DPCM streams from is almost always ROM (the only exceptions are FDS and MMC5), so, no, the CPU can't really modify the data. (Also, it's compressed, which makes it harder to make modifications)
Re: PCM sound requirements?
by on (#163994)
If you have enough memory available, you can make music with lyrics. And Memblers is about to introduce a board that makes enough memory available.

Breaking the Law
Re: PCM sound requirements?
by on (#164000)
lidnariq wrote:
Each sample takes a bit, but the amount of ROM you want to dedicate to DPCM is up to you.

Huh? Just one bit?
Oh jeez, I totally forgot what "sample" refers to with PCM data. That's just any one of the dots in my wave, when I zoom in real close. I was thinking of sample as in a clip of sound.
But this information is even more useful.

lidnariq wrote:
Fine-grained bank switching is more-or-less required for any significant amount of sound. Only MMC3-class and better mappers usefully support the ability to bankswitch DPCM, so it's not that common. The rest used at most a KiB or two of fixed DPCM samples.

Wasn't the MMC3 the most commonly-used mapper though?

tepples wrote:
If you have enough memory available, you can make music with lyrics. And Memblers is about to introduce a board that makes enough memory available.

Breaking the Law

OMG I love this version better than the original!
Do some Metallica!
Re: PCM sound requirements?
by on (#164001)
Marscaleb wrote:
Wasn't the MMC3 the most commonly-used mapper though?

It's one of the most common mappers, yes, but having one entire PRG slot permanently dedicated to audio means you lose 8KB of your addressing space, and there's only 1 switchable slot left for the game engine to use. This makes the engine harder to design, and it will mostly likely perform worse than it would if it could map game code and data more freely. Most programmers aren't willing to make that sacrifice.
Re: PCM sound requirements?
by on (#164003)
Marscaleb wrote:
My questions are in regard to both sounds used for music and for sound effects.

PCM requires full use of the CPU while samples are playing. In general, games that use PCM sound effects have to pause gameplay briefly while the sound effect plays. Music is more difficult.

You can simply run samples whenever the CPU is otherwise idle during the frame, but if a game is going to take, say, 50% of every frame to do updates, your sample will be interrupted during that time, and you'll get at strong 60 Hz buzz cutting into the sound. This is why the PCM drums in Ultimate Stuntman have such a cruddy sound.

Games that mix PCM and animation successfully do it very carefully. Battletoads uses PCM drums during its title screen and intro, but mutes them whenever the animation gets busy. It also uses PCM sound effects sparingly (they pause gameplay and break raster effects when they occur). Big Bird's Hide and Speak carefully cuts samples into small pieces and uses the moment of transition to make small animation updates.

NSF examples are not representative of games, since they can dedicate the full CPU to just music.

Basically what I'm saying is that PCM samples aren't usually very practical. Space for the data isn't really the biggest consideration.


Wait, are we talking about PCM or DPCM right now? I'm kinda confused by the other answers. DPCM is practical, but it has only one format, and it's 1 bit per sample as stated above, and only available at 16 specific samplerates. PCM you can store any way you think is worthwhile, compressed or otherwise.
Re: PCM sound requirements?
by on (#164004)
rainwarrior wrote:
Wait, are we talking about PCM or DPCM right now? I'm kinda confused by the other answers. DPCM is practical, but it has only one format, and it's 1 bit per sample as stated above, and only available at 16 specific samplerates.
I answered the question I think he intended (i.e. about DPCM), rather than the one he said, because of
Marscaleb wrote:
samples [...] a game could reasonably hold in its data?
Marscaleb wrote:
assuming that an actual game is being played, and the system isn't dedicating resources exclusively to playing a sound or some special event
Marscaleb wrote:
the PCM channel only really used percussion instruments

I should have explicitly stated that, but sometimes I rush to answer to avoid getting ninja'd.
Re: PCM sound requirements?
by on (#164005)
With a mapper IRQ you could get a sampling rate of ~11kHZ for about 50% CPU time, though the biggest concern is that even if your NMI handler is not very busy and you allow IRQs to interrupt it, you still have to deal with OAM DMA every frame, during which a few IRQs should happen. Maybe it could work in a text adventure game or something like that where you can even get away with not using any sprites at all.
Re: PCM sound requirements?
by on (#164011)
rainwarrior wrote:
NSF examples are not representative of games, since they can dedicate the full CPU to just music.

"Breaking the Law" happens to be though. The tone generators are driven by Pently, the same music engine I used for RHDE, and the DPCM driver isn't very big. The least representative thing about it technically is the cost of memory before 1994-ish.
Re: PCM sound requirements?
by on (#164015)
Yes, I was referring to PCM examples, and the one you posted was DPCM. (Didn't mean to imply I was talking about yours; was confused about whether the subject was PCM or DPCM.)

Example PCM NSFs: viewtopic.php?t=7309
Re: PCM sound requirements?
by on (#164016)
Marscaleb, can you clarify which you mean...PCM samples or DPCM?

PCM requires constant input from the main program, but gives the highest quality sound. And, generally not used during regular gameplay.

DPCM is an automatic process that allows the main program freedom to do whatever it wants. And is frequently used for background music, sound fx, etc.

Because, most of the answers here seem to be specific to PCM use, and not DPCM samples.
Re: PCM sound requirements?
by on (#164018)
Guys, you can't expect someone who's not familiar with the inner workings of the system to know the difference between PCM and DPCM. It's our job to explain the difference.

The NES has a PCM channel, which plays 7-bit samples. You can manually write raw 7-bit values for the APU to play at any rate you want (the amount of time between each write determines the rate), but this takes nearly all of the CPU time because you're the one responsible for timing the writes. The APU also allows DPCM samples to be played automatically, without intervention from the program after the sample is started. DPCM samples have much lower quality though, because they're only 1-bit (each bit makes the wave go up or down) and can be played at a few pre-determined rates.
Re: PCM sound requirements?
by on (#164019)
Marscaleb wrote:
Also, would I be correct in assuming that the NES is incapable of manipulating the PCM data on the fly?

7-bit PCM playback requires the full attention of the CPU in most cases. (Gauntlet II is an exception, using an IRQ trick to push samples at the cost of CPU time available to the game.) But as long as the CPU pushes out samples to the digital to analog converter (DAC) at audio frequency (usually at least 6000 samples per second), the CPU can do whatever it needs.

But with this cost in CPU time comes a benefit: PCM can be higher quality than DPCM. Run Hello and listen to how "Raw" sounds less muffled than "DMC".

This "manipulating the PCM data" usually takes one of two forms. One is varying the rate at which samples are sent out to the DAC. This produces the guitar part in the theme from EA's Skate or Die 2: The Search for Double Trouble. The other is using more sophisticated audio codecs than your typical 4-bit LPCM. I made a custom audio codec called Quadratic Delta Pulse Code Modulation (QuaDPCM) in the Action 53 intro. It uses a nonlinear step size as well as a frequency inversion trick to make the letter S sound better. And there's a Famicom game by Sunsoft called Tenka no Goikenban: Mito Koumon, whose intro uses the ESS MX speech codec for fairly long speech. But don't expect the CPU to decode MP3, as the 1.8 MHz 6502 CPU limits the complexity of a format that can be decoded in software.

Quote:
I was playing around with an .nsf player and listened to some songs with only the PCM channel playing, and I noticed none of the instruments ever changed pitch.

These instruments might be using DPCM, the 1-bit format that can be played in the background at less than 1 percent CPU use. Because the same DAC is shared by both PCM and DPCM, soloing the channel in an NSF player will allow both to play. Due to cost cutting, the APU can play DPCM only at a small set of sample rates. This is why Sunsoft games such as Batman: Return of the Joker and Gimmick store multiple copies of the bass wave at different pitches, to allow playing notes that lie between the defined rates.

Is Bee 52 ripped?

Quote:
Do some Metallica!

Come up with some good parody lyrics about how MP3 pirates are taking food out of band members' kids' mouths, and I might.
Re: PCM sound requirements?
by on (#164020)
I did some Metallica, but with VRC6 expansion for the vocals instead of DPCM.

Chibi-Tech did some really good DPCM guitars in Blood-stained Gothic Shota Doll.
Re: PCM sound requirements?
by on (#164050)
dougeff wrote:
Marscaleb, can you clarify which you mean...PCM samples or DPCM?


Okay, I was not really aware of these differences. I was reading on some other thread and the gyst I got was that "DPCM" was just being more specific. I was under the impression that the NES could not do formal PCM (as compared to DPCM) so the two were effectively identical in this context.
I would not have heard of DPCM at all were it not that I noticed my nsf player referred to that channel as "DMC" and I *thought* it was supposed to be PCM. When I successfully got a google search that pointed me toward the sound and not DeLoreans I was given the impression that DMC = DPCM and the NES technically does DPCM but nobody really gets picky about the terms.
I was wrong.

So yeah, when I say "PCM" what I am referring to is the sound that NES games play that is neither a square wave, sawtooth, nor noise. The sounds where you can effectively use a real sound effect but at horribly reduced quality. That played in a normal game, during gameplay. That I could use in my normal game, during gameplay. Like the drum sounds in the music for Mario 3 or Contra; like the glass crashing sound in Castlevania II.

So with THAT in mind, to what sort of quality would I need to convert a wave file to in order for it to be played during normal gameplay? And about how much room would a normal game from 1989-1991 (with probably an MMC3) be able to reasonably devote to storing this PCM-like audio? I'm not talking about a game that makes a special effort because it needs special sound, but just a game that wants a few sound effects that sound better.
Re: PCM sound requirements?
by on (#164057)
Still not 100% clear, but the drums in SMB3 are DMC samples.

I was able to fit 32 DMC samples of OK quality into an NROM game (NEStalk), each about .25 seconds, that's about 8 seconds. A better mapping could fit 8x as much, for a total of 64 seconds at an OK sample rate. At the best sample rate, half that... So, about 30 seconds of high quality sound could fit in a normal game.

(Correct me if I'm wrong, guys).

My current method for DMC sound samples, is to get a MONO 44100 rate audio sample, cut it down to abou 0.2- 0.5 seconds, import it into Famitracker, and save it as a DMC file (which can be included into a the game code).

I find that DMC is much better at low to mid range frequencies, as the upper range is lost in DMC hum noise (artifacts of the DMC sample rate being in the audible spectrum). The highest sample rate has the least noticeable hum/noise.

Also, DMC sounds tend to be much quieter than other channels, which will have to be volumed down to balance this issue.

DMC is also the only way to get really low Bass notes.
Re: PCM sound requirements?
by on (#164060)
DPCM = delta pulse code modulation
DMC = delta modulation channel

The DMC has a 7-bit "current value" register. Raw PCM playback feeds individual samples to this register. A program may also write once to this register before starting background playback of a DPCM sample, producing the pops common in Konami NSFs.

I've written tools to encode and decode DPCM in Python. This should give you an idea of exactly how it would sound. Are you capable of running Python programs on your computer? What operating system (Windows, Linux, or OS X) does your computer run?
Re: PCM sound requirements?
by on (#164069)
dougeff wrote:
I find that DMC is much better at low to mid range frequencies, as the upper range is lost in DMC hum noise (artifacts of the DMC sample rate being in the audible spectrum). The highest sample rate has the least noticeable hum/noise.
The DMC playback is equivalent to a first-order lowpass at frequency 0 (i.e. it's an integrator). Frequency X can be represented as twice as loud as frequency 2X.

So as you found, it's good for bass guitars, bass drums, toms, and deep rumbly synth samples, and it's rather suboptimal at cymbals, high hats, and other broadband noise sources. (Fortunately, there's the noise channel)

Quote:
Also, DMC sounds tend to be much quieter than other channels, which will have to be volumed down to balance this issue.
Each individual step in DPCM is pretty close to the quietest pulse channel volume; 15 steps are close to pulse channel maximum volume. The triangle channel is equivalent in volume to a DPCM stream of 20 ↑ followed by 20 ↓ (and repeat).

IME, I found that companding the audio first is vital towards getting good volume (and lack of hiss) out of DPCM-encoded recordings.
Re: PCM sound requirements?
by on (#164087)
Marscaleb wrote:
So with THAT in mind, to what sort of quality would I need to convert a wave file to in order for it to be played during normal gameplay? And about how much room would a normal game from 1989-1991 (with probably an MMC3) be able to reasonably devote to storing this PCM-like audio? I'm not talking about a game that makes a special effort because it needs special sound, but just a game that wants a few sound effects that sound better.

The most DPCM I've seen in an NES game was Gimmick! which has less than 32k (4 x 8k banks) dedicated to samples. It had a mapper that could swap out the C000-DFFF region which helps a lot there. Most of these samples are used for a DPCM bass instrument that plays constantly. Sunsoft put DPCM bass in several games (look up "sunsoft bass"); Journey to Silius makes do with a much smaller collection of DPCM samples in a single bank (MMC1), but its bassline is less varied and a bit out of tune compared to their later games which afforded more data to it.

The majority of DPCM using games just use them for a couple of percussion sounds (e.g. bass drum, snare drum), and often they are sharing precious space in a fixed bank with important code. The amount of data devoted too samples tends to be rather small. A couple of kilobytes, maybe ~2k or so? 8k would probably be excessive if you can't bankswitch $C000-DFFF.

Really there is a wide variety of options available, what games did was about how they wanted to manage their resources. Sunsoft wanted their soundtracks to stand out, and they certainly paid for it in their game's data budget.
Re: PCM sound requirements?
by on (#164122)
tepples wrote:
DMC = delta modulation channel

Ooooohhhhhhh!! That makes sense.

tepples wrote:
The DMC has a 7-bit "current value" register. Raw PCM playback feeds individual samples to this register.


When I read that i thought... Could you feed the DMC the result of an algorithm to produce another simple wave, like another square or another sawtooth, to get an extra channel but without the expense that an actual DPCM file would carry?

tepples wrote:
Are you capable of running Python programs on your computer? What operating system (Windows, Linux, or OS X) does your computer run?

Probably. I run Windows 7 64-bit.

(Redacted)
Re: PCM sound requirements?
by on (#164124)
Marscaleb wrote:
tepples wrote:
The DMC has a 7-bit "current value" register. Raw PCM playback feeds individual samples to this register.

When I read that i thought... Could you feed the DMC the result of an algorithm to produce another simple wave, like another square or another sawtooth, to get an extra channel but without the expense that an actual DPCM file would carry?

PCM is the expensive one, not DPCM. So, no. This is backwards.

Marscaleb wrote:
rainwarrior wrote:
The most DPCM I've seen in an NES game was Gimmick! which has less than 32k (4 x 8k banks) dedicated to samples.

Gimmick, Gauntlet II, Sesame Speak and Spell...
Did any game that DIDN'T suck do anything impressive with PCM?

Gimmick! was an example of DPCM, not PCM, and also if you think it sucks I don't know what game you would think is impressive.
Re: PCM sound requirements?
by on (#164125)
Marscaleb wrote:
Could you feed the DMC the result of an algorithm to produce another simple wave, like another square or another sawtooth, to get an extra channel but without the expense that an actual DPCM file would carry?

That would need one of two things: a mapper that maps RAM to $C000-$DFFF (which means Famicom Disk System or MMC5), or feeding one byte at a time through (ab)use of the completion IRQ as in blargg's saw demo.

Marscaleb wrote:
tepples wrote:
Are you capable of running Python programs on your computer? What operating system (Windows, Linux, or OS X) does your computer run?

Probably. I run Windows 7 64-bit.

Are you also familiar with the Windows command prompt? If so, try the Python installation instructions and let me know what messes up.

Marscaleb wrote:
rainwarrior wrote:
The most DPCM I've seen in an NES game was Gimmick! which has less than 32k (4 x 8k banks) dedicated to samples.

Gimmick, Gauntlet II, Sesame Speak and Spell...
Did any game that DIDN'T suck do anything impressive with PCM?

Does Wheel of Fortune also suck? Does SCAT suck? And do the two community multicarts known as Action 53 (volume 1 and volume 2) also suck? All three use PCM. Usually if a game could afford to stop the action for speech, it would use PCM for higher quality.
Re: PCM sound requirements?
by on (#164128)
I've made a square wave generator using the DMC IRQ. I play a 1-byte sample, which is the value $AA, so an alternating bit pattern of 1s and 0s. I play the sample at the fastest possible DMC sampling rate. This is so that the DPCM sample doesn't interfere with the rest of the sound more than it has to. You get an inaudibly high, very quiet square wave as an artifact, which is no big deal, and if I remember correctly it takes about 3500 cycles per frame to handle this, with the cost getting lower, the lower you go with the frequency. The issue is that in a game this would be interrupted by OAM DMA, and you'd get a 60Hz buzz in the sound. So the only practical use of this would be an extra channel for sound effects, or tone drums where the interference doesn't matter that much.
I screwed up something with the controller routine so sometimes you get dropped input, and sometimes you get double reads. Couldn't be bothered to fix it because this is just a proof of concept-type thing.

Every x IRQs the state bit of the "sequencer" is flipped, and I write a zero to the "current value" or the current volume based on this bit, which makes a square wave.

Up-Down changes the divisor value, Left-Right changes the volume.
It might work incorrectly if your emulator doesn't have accurate DMC IRQ emulation.
Edit: Got the code here that makes it happen, not very complex...