Family BASIC Data Recorder emulation

When the SAVE command is issued in Family BASIC, $4016.0 is written to as described by the wiki. If you record the $4016.0 writes to a file, the result is just a stream of alternating zero's and one's: 0, 1, 0, 1, 0, 1, etc. However, if you record timestamps along with each write and plot the data against time, you'll find that you are looking at square waves of 2 different frequencies (2100 Hz and 4200 Hz) encoding the binary data.

Since the square waves are generated by running software, it seems necessary for an emulator to record $4016.0 values along with the timestamps, as opposed to simply converting the aforementioned frequencies to a binary stream of data. Such conversion would be sufficient for Family BASIC, but it will not necessarily work for all programs that use the Data Recorder.

That said, I'm not sure what is required to read back the written data. The wiki includes a circuit diagram, but I don't fully understand it. Does the circuit mean that the software must always use those 2 specific frequencies mentioned above? Does that op-amp convert the frequencies back into zero's and one's automatically? And, if so, does that mean that the emulator would not have to store the timestamps as long as it played back the data stream at the expected read rate?

You should be looking at Enri's reverse-engineered schematic instead.

The answer seems to be: for well-behaved programs, the bits they get back in via the keyboard's "ADC" is the same as the bits they sent out via the keyboard's "DAC". Analog constraints mean that there's a minimum and maximum spacing between field reversals (i.e. bit changes); maximum is very approximately 1.6ms between reversals; minimum is probably around 70us?

There are emulators that already implement the data recorder interface: VirtuaNES at least.

Do you mean if I just record the raw values and play it back at the same rate, it will work? There is no need to detect which frequency is being used?

Correct.

There's lots more little random details here, but just playing the bitstream back should work.

(Details like "converting the bitstream back into the transmitted bytes" or "the bandwidth of the tape might mean you want a deglitching filter")

I finally got around to testing this and sure enough it works! The minimal sampling rate that reliably enables the tape to function appears to be 97 CPU cycles per bit sample (NTSC). I'll experiment further to see if that can be pushed up. At that sample rate, BASIC programs will take up 25x the space on disk when compared to a raw text file.

At 97cy/bit, you're recording a bit stream at 18.5kHz. That's not too dissimilar from the bandwidth supported by analog tape (albeit tremendously less bitdepth), but the bit quantization means there's probably an uncomfortable amount of jitter that could interfere if any decoders use frequencies above 1/4 that sample rate

Is this more space efficient than storing time deltas? Do generic compressors do a decent job with either the uncompressed or a time-delta format?

Have you looked into what VirtuaNES is doing? Compatibility isn't important, but it might be nice.

Out of curiosity, do you have a random small sample of the Family BASIC encoding?

As for disk space, I'm sure if the files are zipped, they'll shrink considerably. There is probably tons of redundancy in the stream.

I'll investigate VirtuaNES to see if I can make a compatible format.

I'll try to provide some sample data in the near future. At the moment, my virtual audio cassette table only exists in memory. I have yet to add an import/export option. I wonder if it can be saved to .wav file to see if it sounds like a modem.

Also, I tested out Excitebike. Amazingly, save and load work there too! However, I had to increase the sample rate. I'll experiment further to find the minimal rate that works with that game.

Of course that would work. I don't know of any encodings that you could use that are denser (and error-free) than 1 byte/sample, though.

I was naïvely wondering if minimodem would just randomly happen to be able to decode some of these.

For Excitebike, the minimal number of CPU cycles per bit sample is 88 (NTSC). This going to end up producing a lot of data. I was hoping to capture the audio cassette data into the ROM save state. But, now I'm reconsidering this to avoid needlessly bloated saves.

What's so bloated if you pack the bits? At 88cy, that's only 2542B/sec ...

Then again, given that with the actual physical data recorder the user could swap between different saved things by picking different locations on the tape, or different physical tapes, I'm not convinced that treating this memory as equivalent to a SRAM snapshot is actually the right UI.

2.5K/s is not that much data. But, compared to the save state file size, it's bloated. I'm also considering auto-saves. So, the total memory is multiplied across multiple saves and the time required to generate each save increases.

As you suggest, I'll probably ended up treating the tape as separate data outside the save state. It'll keep the menu as simple as possible: Record, Play, Stop, Export, Import and Erase. There is no need for Forward or Rewind since the user can just save each virtual tape segment to a separate file. Play and Record will both effectively do a full Rewind before their standard operation. Record has the consequence of erasing and overwriting the tape. But, a separate Erase might be a useful option too to free up the memory.

Also, 88 cycles/sample works for Wreaking Crew. This is the first time I'm messing around with these level editors. It's really unfortunate that Nintendo abandoned the idea until recently with things like Mario Maker. Level editors are really fun to play with.

Edit: 88 cycles/sample also works for Mach Rider.

I wonder if you can automatically detect when a game is writing a save?

Normally OUT0 should only be being toggled a few times with rather close spacing for the keyboard and/or controller every vblank; audio encoding implies instead toggling it not more often than every ≈100 and not less often than every ≈3000 cycles.

Detecting when the game is looking for audio data in seems both more straightforward (when is it paying attention to the 2s bit of $4016?) and more obnoxious ( how do you know when it's paying attention to just that one bit short of doing weird things to your 6502 emulation?).