NSF --> MIDI

Hey experts, can I get your take on this idea?

I have used nsf2midi and its English translation before, and would like to replicate the behavior in javascript for a project I am working on. If my amateur understanding is correct, this will mean emulating the NES processor, rendering the nsf file through that emulator, and capturing the resultant channel data as appropriate midi messages. My guess is that this means each NES channel would map to a MIDI channel, so 5 channels in total: SQ1, SQ2, SAW, WN, DPCM (is my understanding correct here? should there instead be channels for each voicing of the NES channels? Are the NTSC/PAL/FDS variations a consideration? What about the audio expansion chips, like VRC VI, N163, and MMC5?). I recognize that this process will not produce high quality MIDIs, and would be ecstatic with performance similar to nsf2midi.

Because writing an entire emulator is beyond me (for now! if need be, I'm willing to learn), it seems that the best approach to this problem is to hack an existing emulator to suit my purposes, then cross-compile to javascript using Emscripten, and pray that everything works out ok. This is the same approach used by 'pure javascript-nsf player', which cross-compiles gme and writes a simple html5 player to render the resultant waveform. Do you think this is the right way to approach the problem? Should I instead have a go at hacking a native javascript NES emulator, like Ben Firsh's JSNES?

Any insight/explanations as to why I am stupid and this will never work would also be appreciated, because you guys know much much more about this stuff than I do. Thanks!

tl;dr: I am trying to convert NSF files to MIDI using client-side javascript, have a few (bad?) ideas about how to do it and want YOUR advice.

Doing this the way NSF2MIDI does is stupid and completely useless in my opinion.

The "proper" way to do that is to reverse engineer a game's sound engine, and understand it's format how the music is stored in binary. Then you write a program that convert that into MIDI.
The only problem is that every company has different sound engine, and usually a company upgrades it's sound engine quickly enough to make it incompatible with previous version, so you can do a couple of games at best with such a converter. BUT it will convert music data properly, and be usable.

Alternatively, you can use a tool like the Famitracker NSF Importer to make a track easy to see/read, and then just transcribe it by hand from there. A human reading it can see very easily what is a note and where they begin, in a way that it would be very hard to write a computer program to do reliably.

My opinion mirrors that of Bregalad.

Don't bother turning NSFs into MIDIs -- it's ridiculous. iNES (the old/original emulator) used to have an option to try and do this (specifically output the game's audio ""as MIDI"") and it was horrible. I assume anyone doing anything in JavaScript probably wasn't around during that era... :)

The only "useful" piece of information that relates to MIDI and NSFs, IMO, is the sequencing data (i.e. the notes and when they're played). But as Bregalad said, this varies per every single game and per company (there is no universal sound engine on the NES, sorry), so it requires a tremendous amount of reverse-engineering per-game.

The only way to really "transpose" it all would be to use an actual NSF player like Famitracker's NSF Importer that rainwarrior mentioned, or nsfplay/the Winamp plugin -- both which contain 6502 emulators and necessary audio emulation bits -- and then use that to determine actual notes (e.g. C-sharp in the 3rd octave, etc.). Famitracker would make for easier understanding, I think. nsfplay will actually show you notes and even a little keyboard/piano indicating what's what. It's more for amusement/cute-factor than it is for what you're wanting. But the code is open-source, so you could go look at it and go from there...

So if you're looking to basically "MIDI-ise" a particular game's music, e.g. shove it into Cakewalk or something and then clean it up + do your own instruments, I'd suggest maybe looking at that. But it sounds like what you're writing is some kind of "en-masse" conversion thing, which really isn't going to go very well, IMO. I mean give it a shot if you want, but really to do it correctly you're going to have to implement a 6502 emulator in JavaScript in combination with proper emulation of the audio bits of the NES (not for audio playback, but for calculation of notes and so on).

And don't expect to get any "effects" -- while MIDI (as a format) has all of that, I really don't know how you'd convert some of what the games do (to get the sounds they do) into MIDI accurately, or even remotely accurate for that matter.

And if you do attempt this, take my advice up front: focus on just the main NES audio. Stop thinking about expansion chips right now. That's just going to hurt your head given that (from what I can tell?) you don't even have a grasp of the console's native audio.

Man all this just makes me think of old iNES. God that MIDI option was such garbage... Sorry, I just can't get over that. It's remarkable how history tends to repeat itself.

I remember wanting to get that emulator just for that feature alone, before most NSFs were ripped and channel-splitting either wasn't a thing or I didn't know about it. Good to hear it sucked(s).

By the way, why does iNES cost money? Is there any redeeming features it has that other NES emulators don't?

That's more of a question for the author. I'm sure he'd be delighted to answer that if you mailed him. (I'm being sarcastic BTW -- your question implies you already know the answer :-) )

The primary problem for translating an NSF playback stream into MIDI is things like:

- how to detect the start of a note (sudden increase in volume? sudden change in pitch?)
- how to detect the end of a note
- how to deal with changing pitch (MIDI pitch bend is limited)
- how to deal with changing volume (MIDI volume adjustment? instrument choice? well-placed note-release?)
- how to clean up situations where the detector is tricked into producing too many notes, etc.

There are approaches to all of these problems, but no approach will work universally. To do it well, it needs to be interactive and adjustable for every tune, more or less, and even then it may still be hard to get it working cleanly across the whole tune (different settings for different sections?).

I believe G-NSF has a MIDI output option, if you want to try playing with that.

If I was going to make the most basic support, I would try to treat it like an NES channel. I'd set the velocity to max, note-on only at the beginning (except when needed, like for duty cycle changes to select instrument), and try do handle all the changes afterwards with channel volume and pitch bend (and maybe pitch bend sensitivity, but I've never seen a MIDI change that after initialization). I'd bet that it's usually OK to note-on with writes to $4003/$4007, but if envelopes/timers are disabled, it's certainly possible for a new note to not write to it. If the envelopes and timers are enabled (most stuff doesn't use it), then writes to $400B/$400F would be note-ons as well.

I've had the idea of adding NSF support into my synth as sort of a bonus feature, but I haven't worked on that part of it. What I had in mind was to dig into the NSF code instead of having the player try to guess. I would look for things like, the code's equivalent to a note-on command, and the actual instrument selection that could be mapped to MIDI instruments. That would take care of the biggest NSF to MIDI problems.

Sorry for late reply, IRL has been keeping me too busy to work on this project but I've been ghosting and reading responses and really appreciate your input; there's some very good advice here, and it seems like I've got my work cut out for me. I've found a few 6502 emulators implemented in javascript online, and they may prove invaluable if I figure out how to work with them. Investigation pending.

For now, I'm going to start by reading through some tutorials on using sound in custom roms, which should hopefully give me a more complete idea of what I am working with, and help me to start reversing actual sound engines (though I am still inclined to implement a 'nsf2midi'-style automatic converter).

I will update here when I have progressed more.