This fixes an issue with famitone

I believe we discussed this before some time... but the code was never posted.

The issue = if sound effects are quiter than the music, they won't play...and if they have a volume envelope that fades out, they will cut off as soon their envelope is quieter than the music.

Here's the fix...



Although you will get the opposite problem... the music might cut out abruptly to play a sound effect.

Although, on second glance, I'm not sure what "FT_TEMP_VAR1" is supposed to be doing. That might be removable.

So I guess not everyone appreciates the "louder of the two" behavior that both FamiTone and Pently have.

My own preference is for sound effects to cancel the currently playing note and override the channel for its duration. The channel resumes to silence after the sound effect until another note occurs.

Cancelling the current note prevents notes from resuming in the middle, which I find disrupts the rhythm of the music unpleasantly.

I don't think volume is a sufficient "priority"; do you really want to hear less of the sound effects if the music is louder? Is there no important information in the quiet parts of sounds? I think there's very good reason for quiet parts of SFX to be heard instead of music.

Sound effects have an important gameplay function, and I don't want this to ever be masked by music, really. The note-cancel also helps with aesthetic interruption of music, letting it gracefully resume with a new note, rather than hearing only part of a note cut back in with a non-musical timing.

(This also means I don't compose long held notes or the most important musical lines on the channels to be interrupted. Notes on the shared channel tend to be frequent and quieter to fit the interruptable role.)

I'm ok with music notes resuming from the middle after a sound effect ends, but only if the envelope is able to resume from a middle point, rather than a brand new note triggering from the start, off cue. If the engine doesn't support that, then it's better to just drop the note and leave silence until the next note starts, but you'll need to pay attention to your music and what channel you interrupt since you'll be introducing gaps that way.

There's a similar line of reasoning behind which sound channel to interrupt. If the sound effect is for some minor action, it should interrupt the lesser important music track. That would be Mario getting a coin, and having the coin interrupt the harmony track. There are some cases where it's ok to interrupt the main melody with a sound effect though, that's when something really important happens. That would be Mario getting a mushroom or a 1up. Those things don't occur nearly as often, plus (and more importantly) they're pretty noteworthy when they do occur, so that's what sets those things apart from other non-frequent things, like Mario going down a pipe, which shouldn't interrupt the main melody.

Another interesting case occurs in Kid Icarus and Metroid 2. In both games, the characters actually have footstep noises, but not while noteworthy music is playing. A similar thing in Gradius 1, the iconic "Challenger 1985" song of the first stage plays on all sound channels, with sound effects completely muted until after the first phrase of the song has completed. That could also be due to a limited music engine that cannot handle multiplexed sound channels (as is the case of Spelunker, any sound effect will permanently mute its respective channel of music until the music loops), but you could also counter that with the idea that a game like Gradius has so much action and sound effects going on that it makes sense to reserve one sound channel for just that (like racing games where one sound channel plays the engine noise only).

With all this in mind, there isn't one simple mechanical way to solve music/sfx multiplexing, it all comes down to careful planning, and although the volume-based priority method can work for some, it's not always going to be the right solution for everyone.

What?? Why would anyone start a new note at the end of the sound effect? What engine does that?

These are the real methods I've seen/tried:

• Silence until next note.
• Resume from middle of note (i.e. it kept playing but has been masked by SFX).
• Fade into middle of note (more involved variant of resume).
• Volume priority.

I've already stated my personal preferences, but my objection to resume isn't about starting a new note envelope where there is none; it's simply that the sound of resuming is a sudden contrast, so it makes a rhythmic mark in the music (it doesn't matter whether it's the start of the envelope or not, it still stands out). Having musical sounds always in the same rhythm keeps them coherent and they hang together to feel like they're separate from sound effects. I only want the sound effects interrupt things and happen at non-musical rhythms.

I also tried fading in after a sound effect instead (for square or noise), but I found this had the same problem. The fade actually draws attention to the resuming sound, and it's still out of rhythm with the music. (I remember hearing fade resume in someone else's engine, but I forget whose.)


Those are interesting examples, though I think this falls in a different category. Footsteps are being suppressed at a higher level, a special flag on the music track or something. This is not any kind of generic priority or resume system, there's no comparison against the actual sounds being played, it's just an on/off switch for the code that triggers the footstep sound. (i.e. any game could implement this in tandem with any of the SFX priority methods we're discussing)

Like I guess the generic extension of this would be that every music track has a boolean list of which sound effects in your game it's allowed to disable? Probably in most games it's easier to make a special case of it for the any sounds that want to do this. (You could similarly flag some sound effects as having volume priority or not, and implement both systems, perhaps? Getting a little complicated, heh...)


I'm interested in the opinions of people who favour the volume priority. I asked some questions about this, maybe they sounded like rhetorical argument but I actually want to hear the rationale and why they like it better. (It surprises me that it seems to be the majority here, but maybe that's sort of a consequence of most people using famitone.)

Vice: Project Doom restarts an interrupted pitch slide after a sound effect ends. Around 0:12 and 2:45


One argument is somewhat specific to the architecture of Pently: it lets the player repurpose the sound effect code for drums that override a pitched channel. You may, for example, want to have a triangle kick end earlier when there's a note on the same channel than when it's alone. I added volume priority in Pently 3, used for Thwaite, because the "box breaking into 3 missiles" effect was so echoey that its long tail could be dispensed with if a note is playing at the same time. The attack track added in Pently 4, on the other hand, uses "resume from middle of note" because the envelope system is separate from the sound effect and drum system.

But now I'm aware that some prefer strict "resume from middle of note" for everything, and I've added a PENTLY_USE flag for it to the Git repository that will become Pently 5.

So in some sense you wanted sounds that have an optional ending that can be discarded or terminated early, and you feel that volume is a good enough way to express which part is optional?

Yes. Volume priority is an approximation of illusory continuity and other forms of auditory masking. If you calculate the overall mix as if all sounds were on separate channels, and then search for what tone generator parameters produce the closest sound, the parameters will probably resemble those of volume priority.

I think the "closest" sound to a mix would be to double both volumes and alternate between the two sounds on successive frames (i.e. arps, or sorta low frequency PWM), but but that's probably not a desirable sound texture for most people.

Maybe this approach lends insight into what your goals are with a sound effect system, though. It seems that your goal is primarily to mix another sound on top of the playing music.

For me, I don't want sound effects to mix with the music. I consider them a functional part of user feedback, and generally I want them to be heard clearly and distinguished from the music, which is entirely subordinate to the sound effect. My goal is to make sure sound effects are not mistaken for music, and vice versa. I really don't like the idea of compromising both sounds by approximating a mixed version of them.

Within the music itself, I use lots of techniques like the ones you're talking about (e.g. interrupting a channel for a short time period to create another musical line, sticking echo tails in between notes, etc.) but I only do it within the music which is pre-planned and deterministic. I don't think I'd normally want to do this kind of sleight-of-hand for sound effects, which are usually for the purpose of communicating the state of the game to the player. I'd generally want to ensure that a sound effect always behaves the same so the player can get used to is as a cue.

I also think there's useful sound expressions that may be a lower volume than the prevailing music. "Louder" is far from the only way to make something stand out, but also often complex sounds have both quiet and loud frames mingling together, if your system overrides the quiet frames you're losing part of that sound. For example, a slow attack on a dog barking sound effect is important to make it sound like a dog and not, say, a bell, which would have a very fast attack. However, if there were music playing a tone at the same time, under a volume priority system suddenly the bark cuts in just at the middle and the bark becomes more of a stifled beep as you hear only the part that peaks above the tone. You really lose the character of some kinds of sounds if you start masking them in this way.

In songs with a note with an especially long hold time, it's kind of weird-sounding if a sound effect makes it so that the next several seconds lack this otherwise-constant note.

I can't think of an example song at the moment, though.

Yeah, but the question is why you'd implement it to restart the note instead of just masking/resuming. I can't think of any situation where you would want to restart the note instead.

Oh.

No, I think we're all in agreement: drag's saying that, when you DO resume a note, you should make sure that the engine goes to the appropriate middle portion of the envelope/patch rather than starting anew. Naïve implementations of "resume note" might restart it entirely, but it's a nota bene/precaution, not a "people do this", I think…

If you need a test case for holding a long note, try the first minute of "Vogue" by Madonna.

I guess many "classic" engines couldn't handle resuming an envelope from the middle because developers were trying to conserve RAM by reusing the instrument envelope position bytes for the sound effect envelope position. So they'd either leave the channel silent until the next note (older games) or restart the envelope (Vice: Project Doom pitch slides). But in an engine with enough RAM, we agree that picking up at the appropriate point in the envelope is more desirable.

That leaves priority. Both Pently and FamiTone default to the volume comparison paradigm, but a configuration option in Pently and this patch by dougeff to FamiTone allow the "sound effect trumps instrument" behavior that some sound designers may prefer.

I said it before, but it's really simple to solve this by just putting the sound effeects on the other square channel only, and let there be one dedicated music channel. You can also play notes on every quarter note that "resume" the sound (seamlessly at the same level it was prior) in time with the music, or a number of other techniques to keep it going without envelope resume.


I don't agree with this at all. Mine doesn't support envelope resume intentionally, because I prefer note cancellation. I also implemented several things (including envelope resume), and there wasn't any difference in RAM usage between those two cases for me. (Even if there was, I would probably gladly trade a little RAM for something that sounds better to me.)

Do you really think the creators of SMB3 used note cancelling to save RAM, or because they preferred the way it sounded? The former seems like an absurd suggestion when you've got 8k WRAM. (I also don't think it actually saves RAM, but I don't want to have a pointless argument about hypothetical implementations.)

Vice: Project Doom's slides seem like a bug that was just not important enough to fix before shipping. Caused by a poor decision to use hardware slides + resume behaviour. I don't think it's even "note restart", I think it's just regular envelope resume, but it unfortunately uses hardware slides that destroy the information they need for resuming properly.

What classic games (besides V:PD) do you know of that resume envelopes? What classic games do you know of that implement volume priority? Which ones implement note cancelling?

I thought note cancelling was the most common behaviour, but I certainly haven't taken a good survey (and maybe I'm biased toward it because it's the method I strongly prefer). This is part of why I may seem so surprised that it's not the most common behaviour for NESDev homebrew.


That leaves out the third option of note cancelling, which I think is a really good option.

OK, I had to share this. I've been working on rewriting the source code for famitone2's text2data.

Today I've been working on adding the 'volume column' into the data. And, big success!! I still need to rewrite famitone.s to accommodate, but the data is looking right. I'm so happy!!

Also managed to add full note range today.

I had to sacrifice PAL mode and efficiency.

There is room for more effect support, but I likely won't be doing that.

Famitone 3 on its way? :3

Would love to see Famitone supporting more effect columns from Famitracker, since it would make it much easier to work with musicians who aren't happy about limitations. Especially pitch modulation would be a huge improvement, but I'm not sure it can be done without sacrificing a bunch CPU cycles?

As for PAL mode, we don't need that

I'm basically done with my first edition. Need to make a NESASM and ASM6 version.

Size differences... (data in a test song)
Original text2data - 505 bytes
My version with volume column - 615 bytes (22% increase)
My version minus all volume changes - 565 bytes (12% increase)

Need 6 more RAM variables.

Runs about [estimated in my head] 150-200 more lines of code per frame, that'd be about 500 cycles more per frame. [again, in my head estimate]. That's because I have to run a multiplication routine 3 times [Sq1, Sq2, Nz].

You could implement the volume column without slowing anything down, at the expense of data space - have 15 variants created for every volume macro, pre-scaled, and variant instruments to point to them. Maybe that can work?