More NSF Requests

Well, here's a explenation taken from the readme for SID detection tool by Michael Schwendt. It's obviously more SID related, but should give you an idea about what's necessary. Not sure if it's even feasible to do with NSF, someone with a better NES knowledge would have to look into that.

A friend of mine whipped a time detector for his Amiga music emulator using similar principles (he added sound register state logging I think), which worked pretty good.

Also, the sc68 Atari ST sound emulator has a time detector, which I think uses an similar algorithm.

Article:

I've promised I would either document my current song-length
detection or release it as a module or library, respectively. After
separating some parts of siddiag to get a better overview, I've
decided I would rather document my method in order to suggest a
concept instead of an clean implementation. Here's a small draft.
See what you can get out of it.

[As a side note: if anyone has further ideas on finding a song-end
or loop position by analyzing sequences of normalized frequency
values, this might still be an option, too.]

DETECTING SILENCE?

On the SID emu side, I won't discuss the obvious things here. One
can easily determine when a sidtune turns silent. For instance, when
Master Volume turns zero, chosen waveforms turn inaudible (TEST=1 |
WAVE=0 | FREQ=0), ADSR envelopes reach a near-zero level and digis
are not active. One might even take into account invalid filter
settings which turn off voices. But I doubt the latter is relevant
for song-end detection.

The only issue here is that this kind of "silent song-end detection"
wouldn't help with sidtunes that restart playback after some silent
period of time. Raises the question how many seconds of silence a
song-end detector should allow for? 5 seconds? 10 s? 20 s? I would
look for a period of 10-20 s of silence, so detected get only tunes
that "seem" to have ended for some time at least, leaving undetected
only those which restart or play a next part after a short pause.

However, all this would still not be enough with sids that loop
forever to some arbitrary song position. These never become silent
and hence require a more powerful song-end detection.

MEMORY USAGE PROFILING

The "memory usage dump" feature of sid2crc has lead to the idea of
observing a music player's activity in memory. My first assumption
is, that a sidtune -- which consists of e.g. 2.5 KiB of machine code
and 1.8 KiB of data -- towards its song-end would have accessed all
or most of its data bytes at least once and would begin recycling
previously accessed memory locations when it loops. So, why not
modify the code emulator to observe how often memory locations are
accessed?

Of course, this makes some fundamental assumptions about how the
sidtune is stored in memory. For instance, that a part of a song yet
to be played requires the music player to read new memory locations
that have not been read before. Fortunately, music players are built
like this usually. Imagine a track/step table which lists the
patterns to be played. Entries in that table and the corresponding
patterns of music data have not been played until they have been
accessed. And when they are accessed it means the music player is
still playing something new and has not reached end of music.

THE EXPERIMENTAL IMPLEMENTATION

What I have currently implemented can be described as follows:

* A function that tells the current playtime (e.g. in seconds), so
when there is some kind of event (e.g. the music turning silent),
we know exactly at which place in time we are.

* A modified SID emulator which signals the different "silence"
events outlined above. A front-end measures when these events
start and how long they last. The song-end detector can stop
right away when silence has been detected for 10-20 seconds.

* A slightly modified interpreter in libsidplay tells whether each
accessed memory location is READ, WRITTEN, or EXECUTED,
respectively. While it executes the sidtune, it transfers this
information to the back-end for each address that it accesses.

[That back-end is part of siddiag/sid2crc and maintains a log
with additional information for each C64 memory address, whether
it is READ_FIRST, WRITTEN_FIRST, etc. This allows more detailed
analysis of whether a sidtune utilizes uninitialized memory, uses
self-modifying code, or relocates memory areas, and things like
that.]

Distinguishing between READ/WRITE/EXE memory access allows for
sids that relocate memory areas before they use them
(WRITE_FIRST, then READ and/or EXE), and future song-end
detection extensions regarding self-modifying code and random
values.

* A "memory activity analyzer" (class ActivityWatch) counts how
often each address has been READ/EXE. Outside the initial sidtune
loading range, it makes sure a byte has been WRITTEN FIRST before
it is counted as being read or executed. Additionally, it
maintains five separate counters on how many addresses have been
read so far the FIRST TIME, the SECOND TIME, the THIRD TIME, etc.
up to the FIFTH TIME. When these five counters stop increasing
for some period of time, it means all recently accessed addresses
have been accessed before at least five times, which is likely an
indication that the music player is looping. Sidtune emulation
can stop, and further code be run to determine the song-length.
One could use less than five such counters. But it might be that
a sidtune reads new addresses for the first time after it has
read other addresses many times, i.e. it plays a new part after
it has looped on previous parts for some time. Processing a
sidtune long enough helps here -- "long" being 15 minutes or so
unless the detector kicks out sooner. If none of the five
counters has changed for some long period, it seems safe to
assume that all recently accessed addresses have been read more
than five times, i.e. the machine code has been recycling old
data.

I assume the song-length in seconds to be the last second in which
an address was counted as being accessed for the FIRST TIME, keeping
any eye on above silence events, of course (any objections,
anyone?).

BAD RIPS AND BUGS IN MUSIC PLAYERS

What is left is the case of bad rips (which read unused new bytes
sporadically) or sids that loop X times, reading a few new bytes
each time, before they restart for good (e.g. /BOGG in HVSC).

This is where additional code kicks in. It divides time into small
UNITS -- e.g. 10 seconds -- and counts how many new bytes per unit
are accessed the FIRST TIME. The threshold of how many new bytes per
unit mark a sidtune as still playing new parts needs to be
determined empirically. I'm using 8 bytes per 10 seconds, but have
not examined this deeply (only with sids with suspiciously long
playtimes). When such a low activity in memory has lasted for
several minutes, I assume the sidtune does bad things in memory (the
"B" attribute in the song-lengths database). In such as case, as
song-end is taken the first second when such low activity started.


When detection can fail:

- the sidtune loops, but player keeps reading many new bytes from
somewhere (bad rip or unused machine code)

- machine code stays in a long loop, while modifying memory
addresses and reading them to feed the SID, it neither reads nor
executes any _new_ addresses, but creates new data