Regarding DSP, when attenuating audio or mixing audio (which involves attenuation and addition), should audio dithering be applied? Is seems like a similar process as bit depth reduction?
Audio dithering is only really appropriate as part of bit depth reduction. It's approximately delta-sigma modulation on the LSbits to increase the effective bit depth at lower frequency in exchange for more noise at higher frequency.
Dithering definitely makes a slight improvement for 8-bit output, but it's probably not worth doing for 16-bit or higher, except for very high fidelity applications.
Like it's a reasonable operation for Audacity to do when you're saving a file to disk (the application is entirely about audio, and overhead is drowned by disk access anyway), but for an emulator I wouldn't even bother thinking about it.
In general high-end music production, you only dither when you need to convert your project from 32-bit float to 16 or 24 bit media targets, as a final step once all the other mastering tweaks are done. A cargo cult idea is that it makes the mix sound more "analog", which is in a sense is true since you mitigate the rounding errors which are very digital domain, but the amount of noise added (compared to tape hiss) shouldn't be noticeable.
Elsewhere... i guess the following could be a test. Are you bit depth reducing? can you hear the rounding error affect the sound as a result (in the case of audio - elsewhere, does it provide significant inaccuracies)? Is it unacceptable? If yes to all three, dithering may be a worthwhile feature.
Some excellent explanations of dither.My current emulator has 64-bit float DSP and 8/16-bit integer output, so I add simple TPDF dither at the final downsample. Scale and clamp rail-to-rail signal range to to +/- 3276
6 (or +/- 12
6), add two random numbers in the range +/- 0.5 and cast to int. Practical performance overhead is nil compared with resampling, and it makes an audible difference because the signal is often very quiet (headroom needed to prevent clipping and/or tracks intentionally not using full range).