Cracking Sound for APU

I've mostly implemented the APU, but I'm unsure of how to implement the timing of actually outputting crisp sound.

First, I tried to calculate the number of APU cycles per sample (NTSC_FREQ / sampleRate), and that worked pretty well, but there was minor crackling when VSync was on. When there was no frame limiting, it sounded perfect, but it was much slower than the video. I've attached the audio in cycle.mp3


Then, I tried to calculate the time between each cycle (1 / sampleRate), but in that case, there was significantly more crackling, to the point where it was practically unrecognizable. I've attached the audio in time.mp3


By looking at Audacity, I've realized that the crackling is caused by the period of silence after one segment of audio has finished playing but before the next one is queued. How would I get crisp sound without any crackling while still having the audio speeding up if the framerate increases?

Sadly, you've discovered that you're torn between two masters. (Different clock domains). You will have to pick one that will lose.

One recent emulator - I think nintaco? nemulator - dynamically resamples audio by a couple permille to make the audio and video rates match.

Other emulators just accept tearing, stuttering, or dropped frames in video.

An emulator that is specifically targeting a device that emits both audio and video via HDMI has both video and audio in the same clock domain, so in this narrow case, you can carefully adjust the emulated clock rates to keep the two in sync.

If I'm forced to choose one, I'd rather choose the one timed with a timer instead of counting the clock cycles. However, the audio is practically unrecognizable if I do it that way because there is almost as much crackling as there is useful sound. Am I using the wrong function or algorithm to time it or is it just unfeasible to do it that way?

You seemed to imply that it can be done even with a timer with only slight mistakes, so I'm assuming that something in my code is causing the extremely excessive crackling.

So, the problem with "time.mp3" isn't the dropouts. (You're dropping waaaay too many samples for that to be what's wrong)... you've got to be doing the math wrong instead.

How do you determine cycleCount? I'll point out that 1789773Hz ÷ 48000Hz isn't particularly close to any integer.

(Nor, for that matter, is "slowed down NES CPU clock for exact 60Hz video" meaningfully closer - 60 × 262 × 341 × 4 ÷ 12 = 1786840 Hz)

I can't tell what the heck is going on with time.mp3. With cycle.mp3, this honestly sounds like an audio playback buffer overrun or underrun, i.e. you're either being too aggressive with your latency (I suspect it's this) or not aggressive enough; I can reproduce this when using something like FCEUX and setting the latency slider to something extremely low, then letting the emulator lose/regain focus. Other emulators for all sorts of consoles (NES, SNES, everything) have this exact problem too, while others do not. It's all about implementation, all the way down to what OS-level timer functionality you're using.

You probably want to read this thread and the references linked within it; James (author of Nemulator) goes over his model, which I believe is what lidnariq is describing: viewtopic.php?f=3&t=15405

From what I understand, the "most stable" way to keep A/V in sync is to actually "sync off the audio", not off the video. Video refresh rate is too susceptible to variance (see: monitor refresh rate, Vsync on/off, full-screen vs. windowed, GSYNC/FreeSync, etc.).

I also hope that Sour provides some details about how he did his A/V synchronisation as well as "overall timing sync" in Mesen, since quite honestly it's the one emulator I use that is perfect in this regard -- no tearing, stable framerate, no audio anomalies. Nestopia comes in at a close second (it worked better for me on XP, honestly), followed by FCEUX, followed by anything else I've tried (most of the latter have some degree of audio crackle/oddities).

One technique I found helpful is outputting the sound to a WAV file. But not just the sound. With a WAV file you can create multiple channels of...anything. In one channel, for example, I output a ramp. Effectively a sawtooth that is just an integer I increment every time I create an audio sample. I also output the produce/consume pointers of my audio buffer so I can see how they move. Then you can use Audacity or just write yourself a quick script to parse out that channel data into some analysis tool. I used Excel to graph long times and look for anomalies.

Also, to take care of the "fractional" nonsense, I use a float to represent how many samples I need to generate at 1.789whateverMHz before I need to *consume* one for 44.1KHz. This works out to something like 40.58godawful. But then every time I produce a sample I add 1.0 to a float counter. Then every time that float goes above 40.58godawful, I *subtract* 40.58godawful from it. I see you are resetting your counters to 0. That might lead to misses in the fractional world. Because things don't line up perfectly, when I subtract from the counter my counter will go back to "nearly" zero. But not (usually ever) *exactly* zero. So that means that it'll take a varying amount of time to get back to the point where I need to generate another 44.1KHz audio sample.

My approach isn't perfect by any means...but I think it sounds pretty nice.

It's fairly simple, all things considered. The FPS is regulated by a high precision timer, once a frame is done rendering and sent to the video card, the emulation thread sleeps and waits ~16.66ms minus the time it took to render the frame. If there's a slowdown for some reason, the core will compensate and run some frames faster than 60fps to catch up. The entire core runs in a separate thread from the UI, so UI actions have no impact on it (e.g moving the window, using menus, etc.), which makes this somewhat simpler to manage, too.

For audio, it monitors how far ahead of the playback the sound card is (by checking the current play position vs the last write we did) to calculate latency. If average over the last second deviates too much from the target latency (e.g by more than 3ms), it will dynamically adjust the sample rate by a very small amount (something like 0.025% for each millisecond of gap above 3ms, up to a max of 0.2% or so). It also adjusts the resampling rate a bit more aggressively the longer the average latency is outside the +/- 3ms target range).

That being said, the end result feels relatively simple, but I rewrote both the video and audio timing code something like 3 times each over the past 3 years, so...

The clock ratio can be expressed without "godawful" floats if you think rationally.

NES CPU rate: 39375000/22 Hz
audio rate 44k: 44100 Hz
audio rate 48k: 48000 Hz

Factor each:
NES CPU: 2^2 * 3^2 * 5^7 * 7 / 11 Hz
audio rate 44k: 2^2 * 3^2 * 5^2 * 7^2 Hz
audio rate 48k: 2^7 * 3 * 5^3

NES CPU to audio rate 44k ratio: 5^5 / (7 * 11) = 3125/77
NES CPU to audio rate 48k ratio: 3 * 5^4 * 7 / (2^5 * 11) = 13125/352

So at 44k, you can add 77 to a counter every CPU cycle, wrap it at 3125, and leave the floating for Balloon Fight and Kirby's Adventure.

Yeah that's my problem. I tend to float through life.
Thanks for Doing The Math for me.

There's even one more option: Mess with your monitor refresh rate to get a true 60.098Hz refresh rate. There is a tool called "Custom Resolution Utility" which will let you tweak your monitor timings.

I assume you meant "Nemulator" there...

Yup, sorry! I'll edit it. (Quietust here is the author of Nintendulator :-) )

I decided to just output an audio sample every number of cycles and increase the sample rate to 48000 from 44100, and it seemed to get rid of most of the crackling. To get rid of the last bit, I decided to add CPU timing to limit the framerate to 60.09, and that seems to get rid of the other crackling.

But the way I implemented the frame limiting isn't ideal. I tried using std::this_thread::sleep_until or sleep_for but it always overslept and resulted in around 55 fps. So I used a while loop to just check if enough time has passed, and it was extremely accurate, but I've read that it's cpu intensive and not recommended. Is a while loop the only way to do it since the time interval is so small or are there better ways to do it?