Opinon needed: Data structure for my sound engine

I did most of the Nerdy Nights sound tutorials. I'm at chapter 7 now. But now the tutorial includes all the really advanced features, like volume envelopes, opcodes etc.
But my music engine is supposed to be simple since it is intended for a simple game.

So, I created my own sound data structure that I want to base my code on. And I'd like you to judge it. Is this a reasonable attempt?


O.k., the first data structure is a tune.
A tune consists of the following values:

1. Volume and duty cycle (1 byte)
Just 1:1 the value that you would write into the corresponding APU port.

2. Note or length (1 byte, can occur multiple times)
If bit 7 of this value is 0, then it is a note. Or rather: It is the index to a lookup table that includes the 96 frequency values from this list, each of them consisting of two bytes: www.freewebs.com/the_bott/NotesTableNTSC.txt
If bit 7 of the value is a 1, then this value (or rather: value AND %01111111) is the length (in frames) of all the following notes until the next length value is read.

3. $FF as an indicator for the end of the tune.

That was the "inner" data structure.


The second data structure, the "outer" data structure, is a song.
A song consists of the following values:

1. Address to a tune on square wave channel 1 (2 bytes, can occur multiple times)
2. Value $0000 (2 bytes) as an indicator that the song needs to be looped now, i.e. it starts at the first tune again.
3-8: Like 1 and 2, only for square wavel channel 2, triangle channel and noise channel.
If any of the tune address values is 0, this channel is deactivated for the song.

So, if we have the song "For he's a jolly good fellow", we would have the following tunes:

Tune 1: "For he's a jolly good"
Tune 2: "fellow."
Tune 3: "fehellooow."
Tune 4: "Which nobody can deny."

And the song array (for one channel) would look like this:
Tune 1
Tune 2
Tune 1
Tune 2
Tune 1
Tune 3
Tune 4
0


Sound effects are not included yet. But they're basically the same, only that they don't loop and will probably just consist of one tune anyway. When they happen, they "steal" one of the sound channels from the corresponding music stream.


What do you say?

You could increase the range of the length (from 128 to 256 frames) if only the note byte used bit 7 as a flag: if the bit is 0, play the note using the previous length, if the bit is 1, play the note using the length specified in the next byte (all 8 bits of it), which becomes the new default length.

You can represent the same information in 36 bytes. To do that, you'd use a table of 12 frequency values and 12 "rounding" values. The frequency value is right-shifted to the correct octave, and the "rounding" either keeps the result or subtracts 1 to arrive at the correct value.

I also worked out tables that round to the nearest pitch* instead of the nearest frequency. I can give you a copy if you're interested. (Really, it probably belongs on the wiki.)

*If you have an offset the same number of Hz above and below the target frequency, below will sound further away because pitch does not map linearly to frequency.

I dont understand this. Bit 7 is already there to check whether it's a length at all or whether it's a note.
Can you write down what you mean in pseudo code with a few ifs and elses please?


I don't think I will do this. The lookup table is merely 96 bytes for the whole program. So, with your method I would save 30 bytes. But it would add logic to the function. And time per frame is a much more serious issue than a few constants in the ROM.


Sure, post it. I'd need some of the music experts on this forum to compare which one is better, though. Since I have no idea.

Which rounding method does the Famitracker use? (Since my composer works with that program, so if he enters a certain note and I paste it into my game, then it should of course still sound the same in my own engine.)


Another question: How important is it to be able to change the volume and duty cycle of the square waves in the middle of the song? I'm thinking about putting these values at the beginning of the whole song instead of the start of every tune. Saves me a few bytes, but most important: It saves me time since those values only need to be set once in the beginning instead writing (and even checking for them) every tune.
So, do songs with a constant duty cycle per square wave stream still sound good?

96 - 36 = 60


Fair enough. It's an option you can consider if ROM space becomes more important than CPU time.


Here's the NTSC table. It looks like I stopped when I started getting duplicate values for adjacent notes, so the (basically useless) highest values are missing. There might be some typos since I did all of the math with a graphing calculator.



That's probably up to your composer. For something like this, it's very important.

Yeah, that's all a matter of opinion. That's what I like so much about mathematics: There isn't the one correct answer.

So, what do the others say? Which list is better? Joe's or this one?

Also, which list does the Famitracker use? How do they convert the notes into 11 bit hex values?

That table is wonky. Are you using A440? Or are you doing something clever with tuning harmonics?
All five periods here are within two cents of 12-TET A55, but 7F0 is not what I'd've picked if I was trying to be as close as possible.

On the NES, with its NTSC-based timing, there may be something to be said for detuning to 13 or 15 cents flat; you get more in-tune notes in the highest octaves that way. There may also be something to be said for using D-6=1202.8Hz (40 cents sharp, period of exactly 93): then the noise channel's tonal noise mode is in tune.


Anyway, here's some simple math to convert from a divisor (as used by the NES) to a note name and a detuning in cents.

My point is you stop using it to tell notes and lengths apart, and assume all bytes are notes, but when the bit is set, you load a new length, which will be specified in the next byte.

I find it redundant to have a bit identifying notes and lengths, because there's no need (as far as I can tell) to have consecutive length bytes, since you're using the concept of persistent lengths that are changed sporadically. Notes on the other hand are absolutely necessary all the time, and since you have the free bit, you can simply have a note "request" that a new length be loaded when the current one is not what it needs.

I wasn't trying to do anything clever beyond rounding to the nearest pitch, so you probably found a mistake.


Isn't that off by one? The wiki says this:

No, the length won't be specified in the next byte, but in the current byte:



Is that alright?

I know how it is, I just think that using a bit to identify the length byte is a waste of a perfectly good bit. Here's what I mean:


There's no need to identify the byte as a length byte if it implicitly comes after the note when needed. The meaning of the high bit of the note is:

0: play this note using the previous length;
1: play this note use a new length, provided in the next byte;

And then you don't need to "sign" each byte as being either a note or a length... They're all notes, except for the bytes that follow notes that request new lengths.

I don't know why this is better than the other method. You still have a bit to indicate that a length is coming and you still need an additional byte for the length. So, why does this make a difference?

The only thing that I noticed here is that you have a strange order in your song data:

My data: LengthA, NoteA1, NoteA2, NoteA3, LengthB, NoteB1, NoteB2, LengthC, NoteC1, NoteC2 ...
Your data: NoteA1, LengthA, NoteA2, NoteA3, NoteB1, LengthB, NoteB2, NoteC1, LengthC, NoteC2 ...

I guess I'm missing something here, but I just don't know what exactly. What do I gain from switching the two values around?

The length can be 1..256 instead of 1..128, that's all. You shouldn't need such long lengths very often, but if you do...

Oh, right. Now I understand it. Yeah, that's better. Thanks.

I think this whole conversation points out that Famitracker could use a better tool (like Famitone) to specifically use Famitracker songs in a game. The easiest approach (IMHO) would be to modify Famitone to accept more/all Famitracker features.

If I were smarter, I would write a tool to convert MIDI files directly to usable game song data. (And write an engine designed to play it). But that may be above my skill level.

As someone who recently did that, I'd say it is really not the easiest approach. The "easiest" approach would be to just use Famitracker's existing driver, and add a sound effect override on top of it. If you don't need sound effects, it's actually very easy to use Famitracker's driver to play Famitracker tunes (why wouldn't it be?).

In a game, though, there are lots of reasons why supporting every Famitracker feature might not be the best idea:

1. The Famitracker driver is about 5k of code. A more restrained engine like Famitone is more like 1k. This can be huge if space is a problem.
2. A lot of the extra features require extra RAM usage to track their state.
3. Some of the extra features increase CPU usage.

Those are actually very simple and are not advanced features at all.

The only remotely efficient way to store music is using some sort of byte code. I did not look in the details you proposed, but it looks like at least two bytes per note is required. With a properly deisgned byte code, you have only one byte per note, and this is capital if you're going to have many music in the game.

Advanced or not, but what I wanted to say:
You have 11 bits of frequency values + volume + duty cycle in the APU.
And then you have 26 Famitracker effects that do different stuff to these bits.
So, transforming a Famitracker song into a compressed version of the APU values, you're really better off using the debug log file instead of the text file with the Famitracker syntax.


Storing the music is not the problem for me.
And yes, one byte is enough as long as you restrict yourself to just using the frequencies that correspond to actual keys on a keyboard. (Which is what Famitracker does.) Because then you have 96 values.
As shown above, I designed my own little data format, so that's not the issue. The issue is: We have a composer who knows almost nothing about 6502 or the NES APU. And we have me, the programmer, who knows almost nothing about Famitracker.
And I want to convert his songs into my format.

That's why I was hoping that there's a tool that converts Famitracker data into APU data. But not just all APU data per frame. But instead:
1. All five channels treated separately.
2. Only the values that have changed since the last frame.

So, the file would tell you:
Square wave
Frame 1: Volume 5, Duty cycle 1, Frequency 385
Number of frames for these values: 12
Frame 13: Volume 5, Duty cycle 1, Frequency 209
Number of frames for these values: 20

It's still not my own format, of course. But it would help. I could do some search and replace and then use copy and paste.

But it looks like I have to write my own program that checks the log file and then transforms the whole song directly into my data format.

What I also don't understand: You have to fiddle with Famitracker's source code and have to uncomment a #define to enable the logging. Why didn't they include that feature a least as a command line option if they didn't want to include it into the GUI?

If you're going to just be playing back an APU write stream, I don't see why you'd bother arguing about whether a note takes 1 or 2 bytes. Logged formats are huuuuuuge by comparison.

The idea of an "instrument" is actually a compression method. For example, having a volume macro says "you're going to see this pattern of volume writes '10 6 4 3 2 2 1 1 1 0' many times in this song, so let's just store it once". This is the problem instruments are intended to solve. They make the data smaller.

If you refuse to get to know the tool your composer is using, you're probably going to find it a huge source of frustration trying to implement their songs in your game.

My own personal limited subset of Famitracker is:
No effects except Dxx/Bxx/Fxx (skip/loop/speed control). Volume column allowed. Volume, pitch, arpeggio, and duty macros supported (relative only).

Most other things that effects do can already be duplicated somehow with instrument macros (e.g. vibrato 4xx can be replaced by a pitch macro).

If you can tell your composer not to use any effects, you can cut out a large amount of implementation, and it's easy for the exporter to check if any forbidden effects are used and warn the composer. Give them an export tool that will tell them if they're doing something bad, and will make them an NSF/NES they can test their music in, and they'll be able to validate their music as they go along.

Well, you don't need EVERY feature. Simply adding volume column and tempo shifts and maybe a few more things (to Famitone2) should suffice for DRW's needs. And just ban the use of the other effects. If his composer is using Famitracker, than it might be the best direction for him.

I don't play back the stream 1:1. I have a minimalistic format:

I store each note as a one byte value which serves as the index for a lookup array that contains all the frequency values for the 96 possible notes.
Furthermore, I store the length of the following notes whenever it changes.
This is what I call a tune. One tune is basically an array of notes, lengths and an end character ($FF).

And then I have an array of addresses to a bunch of tunes. This array is the song.

So, in each frame, the inner loop iterates through the tune and plays the current note and counts down the length counter. When the length counter is 0, the next note (or note and length) is played.
And when the tune has ended, then the outer loop increments the index in the song array and the inner loop can start anew with the new tune. If the outer array ends, the music either ends or it jumps back to the beginning, depending on the end character.

So, every repeating part of the song only needs to be saved once.

I asked my composer not to change the volume and the duty cycle in the middle of the song. I could implement this as well, but I want the music function to be quick and small, so in my game, the volume and duty cycle for each square wave is only set once in the beginning of the song. Saves me a few ifs.


Well, what does refusing mean? It's simply that I don't have the time to get to know a completely new tool (Famitracker) for a task that I don't know very much about anyway (making music).
I want to create a little game, but I don't want to become an expert in music because of that.


I can understand that many effects might make the songs bigger. But why should the implementation be a source of frustration?
Every effect that you can use in the Famitracker has a direct equivalent in the NES APU. So, what kind of frustration could arise?

From my point of view, I simply take the APU writes of every frame.
Then I calculate for how many frames a note was played and store this as a length/note combination.
Then I see how many consecutive notes have the same length to further compress it into a length/notes combination.
Then maybe checking for repititions, so that I can put them into separate tune arrays.
And that's it.

So, yeah, the size issue is something that I definitely have to communicate to my composer. But why should anything be difficult when it comes to the actual implementation of my engine?


Not really possible in my case. As I said, my implementation is nothing but a collection of notes (or rather, note indices for a lookup table), lengths per frame and an outer array that stores these collections to avoid redundancies.

I know that you can implement a lot of muscial stuff. The Nerdy Nights tutorials had volume envelopes and opcodes that trigger certain specific functions. But I consciously decided against that. My function shall be quick and simple.

The format you're describing is not even complex enough to play the music of Donkey Kong. A talented composer might be able to make interesting music nonetheless, but it's going to sound strangely minimal compared to most NES music.


I literally meant refuse: v. indicate or show that one is not willing to do something.

Anyhow, good luck with however you want to try to go about this. I highly suspect you won't be happy with what you've proposed, but you can decide that for yourself once you have it working. Maybe I'm wrong.



Because it's a feature jsr added entirely for debugging. He didn't think it would be useful to anybody. You are probably the first person besides him to ever use it.

Alternatively you could export to NSF first, and then use something else. NSFPlay has an NSF logging option you can turn on in the .ini file, and shiru's famitone SFX exporter has code that emulates and logs an NSF to do something similar to what you're describing.

Why not? It should be complex enough to play any music because it saves any note.
Or do you mean it takes too much space for each song?

What is my intended engine missing?

And what kind of music engine do the more simple commercial games use?
Does "Super Mario Bros." have 2000 lines of code just for the music functions? Don't they just put the notes from ROM to the APU, maybe having some functions to jump between tunes for repetition and maybe something to change the volume?

I really want to know this. Because I want to create a working sound engine. So, if my idea is crap, I'd like people to tell me what to do better.


Alright, so the ability to transform Famitracker stuff into the actual NES APU values is something that nobody would ever need? Sounds a bit strange to me. Isn't this the first thing that should be featured? The ability to see what the program actually does to create NES sounds?

The most important thing you are missing is volume envelopes. Without these, the only sound you can produce is a solid beep with a sharp start and stop. Just about every NES game implements volume envelopes so that a note can have a shape to it.

In decreasing importance you have things like vibrato or pitch slides. Not every engine has these, but they transform a flat sound into one with some pitch shape. You need to be able to represent pitches between the notes of your scale to do this; a pitch envelope is effective enough.

Being able to change duty in the middle of the song, or middle of a note is also pretty useful. Then there's the idea of arpeggios, where you can change the note on every frame; this lets you make sounds of large chords despite having only a few channels to work with.

These things are often implemented as "macro" envelopes, which are just a series of values that get ticked through each frame. When a note starts, all of its macro envelopes reset to their starting position, and each frame it just reads the next value from the envelope for each of these parameters. Conveniently, you can reuse the same code for updating all of the envelopes.

In general, the idea of an "instrument" is a set of envelopes to be applied. When you change instruments, any notes played after the instrument change will use one set of envelopes until the instrument is changed. Being able to change instrument mid-song is very helpful for composing.

Anyhow, as I said, the biggest omission here is volume envelopes. This is the one thing from that list that will have the most impact. Which gets to...


APU logs take a lot of space because of their inability to repeat patterns effectively (see what I said about instruments above). If your playback is truly limited to just turning the channels on and off at a regimented set of pitches, maybe that's not much overall.

However, as soon as you track volume changes, or any kind of envelope, your data will get huge.

We haven't even touched effects, though as I said above, once you have envelopes, you can do pretty much anything famitracker effects do already with just envelope macros, so they're not really important, IMO.



Famitracker is a program for composing. That's its first thing, to be able to make and hear music that sounds like the NES. APU writes are a detail that are entirely hidden from the user throughout the program. They're important to a programmer maybe, but not the composer, usually. The majority of people using Famitracker just want to be able to trust that the program is accurately producing the NES sound; knowing what values are written to what register is precisely the kind of thing they are using Famitracker to be sheltered from.

Is a volume envelope supposed to have the same sequence of volume values for each note throughout the whole song? So, when I declare the volume values
1, 2, 4, 5, 6, 7, 6, 5, 4, 2, 1
does every note use them and do these values never change throughout the song?
What happens if a note is shorter than this sequence? Does the next note's volume start from the beginning or does it take over where the previous note left off?
And if a note is too long? Does it simply set the first volume after reaching the last?


Why isn't this possible with my attempt? Simply set the length to 1, then store the notes.


Isn't it a huge performance hit when your music engine has to go through 20 envelopes and adjust each note according to these values?
Also, what happens when two values clash?
Say you have a volume envelope, but you can also change the volume mid-song. What happens then? Which value does the program take?
Or take an envelope that might change actual notes instead of just the volume or the duty cycle.


But this would also require to go through the whole song manually and painstakingly check what could be created with a macro, right? Or intensively studying the Famitracker to get to know each and every feature.


So, how about just using the Famitracker library in my program and letting the composer do whatever he wants? Would this be an option?
Especially: Is that code quick enough? I don't need a sound library that takes one third of my frame update time to do the background music.
And how much space would this plus the soundtrack occupy? (My ROM is supposed to be 32 + 8 KB.)

In Famitracker you can change instruments as often as you like. You can tell your composer they can only use one instrument, if you want, but that's pretty restrictive.


Yes.


Macros envelopes usually have a loop point. If no loop is specified, you can just loop on the end of it.


The data for doing this becomes very large very quickly. (This principle applies to all use of envelopes.) An instrument would store the arpeggio in a couple of bytes, and it be applied to a thousand notes at no extra cost.


It takes CPU time, but it doesn't necessarily take a lot of it.


You'd have to be specific, as I think each type of envelope is applied differently.


Specifically Famitracker multiplies the volume column value with the volume envelope value (rounding up to 1 if nonzero). (Table lookup implementation.)

You're asking a lot of questions about how Famitracker works. This dialogue is really not a very good way for you to get this information, though. A lot of this would probably very quickly become clear if you read Famitracker's documentation or followed a tutorial, or read its wiki. Actually using the program will explain this better than asking me sparse questions about its implementation.


No, if you want to use Famitracker's features, you should not try to reverse detect them from a register write log. You should just take the exported data as Famitracker produces it. You can check right there if any effects are used, and warn the composer if anything you don't support it used. You don't have to try and detect envelope macros from register logs, instrument definitions are right there in the exported data.


Its base size is about 5k. If you went through it carefully you could probably trim a lot of stuff you don't need. I can't possibly tell you how much space your soundtrack would occupy. Depends on how much music you make for it.

I don't know what your performance requirements are, here's what it says in its readme.txt:


The main thing that Famitracker is lacking for use in games is sound effect overrides. You'll need to handle that yourself.

Alternatively, you can just use Famitone2, which takes Famitracker music files and turns them into its own format. Famitone2 only implements a subset of Famitracker features and your composer would need to know exactly what's missing. Its biggest missing feature is the volume column; it only has volume envelopes. (It's missing a bunch of other stuff, but that's the one that stands out for most people.)

There's also Muse Tracker. I don't know much about it, but I believe it is well featured and does have a sound effects feature. Famitone2 seems to be more popular, but that might have more to do with all the tutorial stuff Shiru has written than its feature set.


Stepping back a bit, there's other ways to implement some of these concepts; you're asking specifically about Famitracker, and if your composer is using Famitracker you might as well just implement things that work the same way. That's one of Famitone2's problems; it's not really based on a subset of Famitracker, it's just that a lot of it was similar enough to what Famitracker does that Shiru could translate Famitracker tunes into Famitone2 data (as long as the composer doesn't try stuff that would expose the differences); many things about it are fundamentally different than Famitracker's implementation though (e.g. how it handles pitch).

If you look at other NES music engines, you'll find all sorts of ways of doing things. As I mentioned, almost all of them have something that works like a volume envelope macro, but other than that, all bets are off, pretty much. Famitracker has a very versatile set of features, and just the 4 envelope types I mentioned are extremely powerful together, but lots of engines don't have pitch/arpeggio/duty envelopes.

My own approach was to write my own engine that worked fundamentally the same as Famitracker, just as a strict subset. That way there aren't surprises when I am exporting, everything that can be exported sounds and works the same, and everything that can't gives me a warning/error on export.

One of the things that you can't do well with envelopes is sliding pitches between notes; I can't think of an commercial NES games that do this, but Famitracker's pitch slide effects (e.g. Qxx/Rxx/3xx) are very expressive, and well liked by Famitracker composers.

O.k., I can watch that tutorial.
Reading the wiki would be a bit too much, though. As I said, I didn't intend to become a music expert.
And it wouldn't even stop with the Famitracker. Not only do I not know the Famitracker effects. I don't even know what the regular musical terms mean. What is a pitch, what is a vibrato? I would have to study music 101 just to understand the words.

After having a look at it, I'm playing with the thought of simply using Famitone.
I actually wanted to do everything myself, but if I really need to implement the equivalent of a music player instead of just a function for simple data copying, I guess using an established third party library doesn't go against my principles.

Pitch refers to the frequency of the sound wave. Higher pitches have more rapid frequency. We call certain frequencies by name (middle C, A1, etc.)

Vibrato - imagine a violinist. He has to press his finger on the fretboard to shorten the length of the string (and increase its frequency). Now, imagine he's shaking his finger rapidly... which effectively lengthens and shortens the string rapidly. The frequency shifts up and down rapidly.

Tremolo - the volume is now rapidly shifting up and down.

Portamento - similar to a pitch slide, but with a specific ending pitch.

Arpeggio - quickly alternating frequencies in a pattern, usually low, middle, high, low, middle, high, etc.

Thanks, but the question wasn't really intended to be answered. I just asked it to demonstrate that learning about the Famitracker would require learning about another general topic as well.

By the way, I will definitely use Famitone now.

"Sappy" from Tetrisphere is the first one that comes to mind immediately. The second is "Radio Star" from Super Mario Bros. 3. There are note slides in the first 10 seconds of each, though those of SMB3 might be enveloped because there's only one distance. But that's without getting into the style that can only be described as "Follin".

I was specifically talking about targeted pitch slides, i.e. starting at one note, and then sliding toward another and stopping on it. That's difficult to pull off with just a pitch envelope, but very easy to do with Famtracker's Qxx/Rxx/3xx. Pitch slides using envelopes generally either start or end at an indefinite place, i.e. easy to bend-into a note or fall off, but not slide from note to note.

Silver Surfer does sound like it has this capability. Good example. (I think Pictionary and Wolverine have the same capability, though maybe not Solstice.)

I don't think Mario 3's engine does. That could easily just be an envelope.

My own soundtrack has plenty of pitch slides but they're all just envelope slides.