Project: make a software synth MIDI driver from SPC player

Project: make a software synth MIDI driver from SPC player
by tcaudilllg on 2010-10-22 (#68983)

Imagine playing a FFIV midi and it sounding just like FFIV. Or Beethoven with FFVI instruments. Yes, I know this is somewhat possible with soundfonts... but the results are not particularly striking.

How much work would it be to code an SPC core as a Windows MIDI interface driver? I imagine it would involve intercepting MIDI OUT messages; processing/mixing them with the DSP simulation code; and sending the mix to the WAVE OUT driver. The actual SPC700 implementation would be unneeded; only the samples and especially, the DSP-based modulation and filtering would have to be figured in.

by Bregalad on 2010-10-22 (#68988)

Quote:
and especially, the DSP-based modulation and filtering would have to be figured in.

I'm afraid that you can't (easily) record exact information about the modulation in MIDI files. The best you can do as far I know is use NRPNs to specify vibrato rate, speed and dept and try to get something as close as the original. This will vary from synth to synth unfortunately. Most synth use sine-wave vibratoes, while I think FF4 uses square-wave vibratoes. You could try to simulate it with pitch wheel events but it would be a bit tedious to do.

Also the SPC only support filtering on reverb, which can be tricked to be only a 1-sample delay and act as a IIR filter, but no SNES game I know ever did that.

It would probably make more sense to listen to FF4 SPCs (if you want it to sound like the original), or hack FF4 to insert your Bethoven music. I know a little about the music format FF4 uses, and I was able to hack SPCs to make my own music with them, so you could just ask about it if you want.

EDIT : Hopefully it wasn't a spammer - this is his first post and it not about nesdev...

by tcaudilllg on 2010-10-22 (#69010)

No, I want to replace my Microsoft GS software synth with an emulated SPC700 wavetable engine.

Indeed, this isn't about SNES dev directly, but it does apply SNES-specific knowledge. Maybe there is a better board for discussions of this type (which has a strong SNES developer presence) and if so, maybe you can direct me to it. Else, this is probably the best place to discuss the project.

I'm thinking I can rip the guts out of Timidity++, and replace them with the SPC playback engine. The question is what details would be involved. It might be better to try the NES or Genesis first, I'm not sure. But I do know that I eventually want to be able to change my instrument presets by loading in SPC files and beyond that I suppose the sky is the limit if a Timidity++ fork could be transformed into a base for chiptune synthesis. It would be a new way to appreciate MIDI synth.

I would not be necessary to convert any elements of the MIDI to SPC700 instructions. All that would be necessary would be to construct the instrument and play it back at the right pitch. By filtering I meant Gaussian filtering, Cubic-spline, etc... the methods used to add smoothness and gloss to the output waveform.

by Bregalad on 2010-10-22 (#69012)

If you have a card that supports soundfonts, then its VERY likely it uses some kind of filtering to the samples. I don't know whenever they use Gaussian, Cubic or whatever.

So in other words you'd like to develop a VST synth that would use something like an emulated DSP, but without the SPC part.
I have no idea how to develop VST synths personally, but I can already see that you'd need a way to tell the synth which patches correspond to which BRR samples...

This sounds like a cool idea I have to say, but you'd want to know exactly what to do.

I have already a VST plugin supposedly for chiptunes, but it's not freeware, so you can't save your settings without paying and this basically sucks.

by tcaudilllg on 2010-10-22 (#69014)

Yeah we could put it under a non-commercial license so it wouldn't compete with the pay synths. No need to destroy the entire market.

I'll post an update when I know more.

by tepples on 2010-10-22 (#69015)

tcaudilllg wrote:
Yeah we could put it under a non-commercial license so it wouldn't compete with the pay synths. No need to destroy the entire market.

Linux and GCC wouldn't have taken off if they were under a non-commercial license. Instead, they were under a copyleft license, which allows commercial or non-commercial distribution so long as source code is available for all derivatives.

by Memblers on 2010-10-22 (#69024)

The main problem I see is the instrument mapping. General MIDI of course has the standard 128 instruments, but the SPC instrument map is 100% up to the programmer. So it seems like it would be a decent amount of work creating config files to get things sounding the way one might want (especially if people want to download it and use it with any random SPC file).

Also there's the issue that General MIDI calls for 24 audio channels, compared to the SNES's 8. More channels than that, and it just won't sound like an SNES anymore (same thing with playing MIDI files with NES sound). Though it would sound cool, no doubt.

Anyone ever use GSXSCC by Gashisoft?
http://www.geocities.co.jp/SiliconValley-SanJose/8700/P/GsorigE.htm
Kind of reminds me of this thread (but for the Konami SCC), it's an excellent player though and I must admit kind of an inspiration for my own synth's MIDI support.

by tcaudilllg on 2010-10-22 (#69027)

Memblers wrote:
The main problem I see is the instrument mapping. General MIDI of course has the standard 128 instruments, but the SPC instrument map is 100% up to the programmer. So it seems like it would be a decent amount of work creating config files to get things sounding the way one might want (especially if people want to download it and use it with any random SPC file).

This is an important point. The soundfont attempts tend to sound bad for this very reason.

Quote:
Also there's the issue that General MIDI calls for 24 audio channels, compared to the SNES's 8. More channels than that, and it just won't sound like an SNES anymore (same thing with playing MIDI files with NES sound). Though it would sound cool, no doubt.

Main idea is to play songs with SNES sound samples.

It seems we would all stand to benefit from this, so I think this would be a sustainable community enterprise. First you'd have to make the program, and then volunteers would create the instrument config files.

Quote:
Anyone ever use GSXSCC by Gashisoft?
http://www.geocities.co.jp/SiliconValley-SanJose/8700/P/GsorigE.htm
Kind of reminds me of this thread (but for the Konami SCC), it's an excellent player though and I must admit kind of an inspiration for my own synth's MIDI support.

Checking it out.

I checked out Timidity's source. Most of the comments for the synth are in Japanese... I don't think it would be worth the effort to modify. I am however investigating MIDI driver samples from the old MS DDKs. They may not be too difficult to work with.

The first operation would be to create an application that played MIDIs with SPC instruments. I think there is such an application on Zophar, if I'm not mistaken.

EDIT: found something: mid2agb. I'll look at how they did that, and then compare with the SNES.

Update: mid2agb = closed source. GXSCC also, appears to be closed source. But I do like it. I like it A LOT.

As I understand it, each game has a sound playback engine. The SPC700 just plays back samples at frequencies, which are programmed in the sound DSP's language. For the first version, we want to get MIDI notes; read their pitches; and playback samples at the rates associated with those pitches. (if I'm not mistaken) That would take out a lot of the CPU processing and improve performance, compared to the sound engine on the actual SNES. As for the peculiars of the sounds, that will only come from SPC hacking... can use a BASIC variant for that, I guess. Make it easy. Or even a plug-in system, eventually... but for now, just BASIC because I've got a decent BASIC interpreter core already.

by Jedi QuestMaster on 2010-10-23 (#69050)

Memblers wrote:
The main problem I see is the instrument mapping.
Anyone ever use GSXSCC by Gashisoft?
http://www.geocities.co.jp/SiliconValley-SanJose/8700/P/GsorigE.htm
Kind of reminds me of this thread (but for the Konami SCC), it's an excellent player though and I must admit kind of an inspiration for my own synth's MIDI support.

I love GXSCC!

I dislike, however, how some people record a MIDI off of it, upload it to youtube, & say "Here's my 8-bit rendition of..."

http://www.youtube.com/watch?v=689BST7t ... re=related

by Bregalad on 2010-10-24 (#69052)

Jedi QuestMaster wrote:
it kind of an inspiration for my own synth's MIDI support.

I love GXSCC!

I dislike, however, how some people record a MIDI off of it, upload it to youtube, & say "Here's my 8-bit rendition of..."

http://www.youtube.com/watch?v=689BST7t ... re=related[/quote]
I agree 100%
I woulnd't have any problem if they just said the truth and said "played using GXSCC" rather than wanting to show off and make people belive they did a remix (that doesn't sound like a NES at all btw, too much precision on note frequencies, too much waveforms available, too much channels).

EDIT : Also chromatic percussions instruments are emulated very badly. Any MIDIs using timanis woodblocks or taikos is guaranteed to sound terrible.
I like how the kick and snare sounds however (done with noise + square + pitch bend instead of DPCM).

by Memblers on 2010-10-24 (#69054)

If you haven't seen the thread about my synth, it's here:
http://nesdev.com/bbs/viewtopic.php?t=6586

Using MIDI-Yoke, I can go into Windows configuration and select my Squeedo synth as the main MIDI device so it works with lots of stuff. If GXSCC can like live MIDI data, that could be used like that also.

I sort of have been toying with the idea of making my synth code open-source.. not sure if it's any use, but I haven't wanted to release anything until I finish making the hardware version (it targets a 32-bit MCU, however I'm also using the SDL library so it can work on PCs).

When I first implemented samples into it, first thing I did was rip some out of SPC files and play the corresponding MIDI files. Castlevania 4 MIDIs sound pretty cool with it's original organ sound in it, however most samples are really grainy and un-SNES-like without the Gaussian interpolation. With the main target platform CPU speed I don't think I'll be able to do interpolation for all the audio channels needed (without taking shortcuts maybe), but I guess it would be no problem on a PC. So far I haven't wanted to make it sound identical on all platforms, but maybe an exception would be useful?

The way I'm loading samples into it now is pretty absurd. For SPC ones I had converted them to .IT files, saved the instruments as XI files and I import them from that format (to preserve looping and tuning info).

by tepples on 2010-10-24 (#69067)

Memblers wrote:
With the main target platform CPU speed I don't think I'll be able to do interpolation for all the audio channels needed (without taking shortcuts maybe)

I guess linear interpolation is an acceptable shortcut on a microcontroller, if even that's doable.

by tcaudilllg on 2010-10-26 (#69124)

So Memblers you say you have a software synthesizer that can play back NES samples? How does it work, and will it be capable of working with only NES functions or also with SNES DSP functions?

by Memblers on 2010-10-26 (#69152)

tepples wrote:
Memblers wrote:
With the main target platform CPU speed I don't think I'll be able to do interpolation for all the audio channels needed (without taking shortcuts maybe)

I guess linear interpolation is an acceptable shortcut on a microcontroller, if even that's doable.

You can here the linear interpolation (lerp) here, compared to just straight output, with the Super Mario World song. That's not the sample from the game though, just a random piano sample I had sitting around. http://membler-industries.com/squeedo/. It improves things somewhat I think, not a huge difference, but a comparison to SPC would sound like night and day.

tcaudilllg:

Basically the synth is designed around the code being fast (hopefully for more polyphony, and as little latency as possible). It's designed to have multiple sound modes.

1 - wavetable synth (like GXSCC) but with 256 samples of 8-bits. So you can put whatever shape waveform in there.

2 - samples, with loop points.

3 - Yamaha sound (FM)

4 - Casio sound (Phase distortion)

Some other fun stuff, would be neat to emulate the Atari POKEY sound, or Konami's VRC6.

An instrument will be able to use any of those modes, there is also an envelope system that can be used to control any parameters, which includes triggering other instruments/channels.

More DSP functions could be added in, I thought about the SNES-style echo-buffer, but if it was done optimally it would have to be really short because RAM is limited (128kB in total, lots of it used for stuff). Or at least limited to a certain number of channels.

Right now it uses PortMIDI - portmidi.dll, for MIDI input. Supposedly that library will work in win/linux/mac. I'll have to come up with my own for the MCU I'm using.

It works the same as MIDI right now, 128 instruments, you just give it note-on and note-off and it assigns the sound channels to your notes dynamically. It also uses MIDI NRPNs to edit the instrument parameters (not all of it yet, I'm still sorta doing that in a hex editor, heheh).

by kode54 on 2010-10-26 (#69157)

For something similar to GSXSCC, you can look at "Emu de MIDI."

Also, FluidSynth is probably a more complete SoundFont synthesizer than Timidity.

by tcaudilllg on 2010-10-31 (#69314)

kode54 wrote:
For something similar to GSXSCC, you can look at "Emu de MIDI."

Also, FluidSynth is probably a more complete SoundFont synthesizer than Timidity.

True, but FluidSynth is also only for Linux and is kind of difficult to use besides. Timidity is way easier.

Memblers I listened to your tunes. They are pretty good but the eight-bit sound is rough on the ears. (brings back memories of THOSE days, when to hear decent game music you had to listen to MODs!)

We'd want to make as much RAM available as the user cared to use. Even more than eight instruments, possibly. After all, we want to use this to play back Beethoven Uematsu style.

The idea is not to create an SNES DSP clone, but a DSP that uses the SNES sound architecture.

But I'll look at your release when it's ready, and see what can be done with that.

by tomaitheous on 2010-10-31 (#69342)

Quote:
They are pretty good but the eight-bit sound is rough on the ears. (brings back memories of THOSE days, when to hear decent game music you had to listen to MODs!)

Wait.. what?