Good news! I got bored so I whipped up a Python program to compress FCEUX FM2 input movies into the format I documented above. I tested this with a ~18 minute run of SMB1 and a ~6 minute run of SMB1. Both came out to about 6% of the uncompressed (one byte per frame) size.
Download Link (GPL V3, Requres Python 2.5.4 or Higher)
Edit: Oh yea, I forgot to mention. This program only supports first player, only with a Game Pad, and there are *no* sanity checks. I may add that stuff later, or anyone else is free to.
I am very excited to work on my AVR-based playback thing-a-majig now. Only problem is I only have *very* small AVRs with 2K Flash space. I'll have to order bigger ones to see full speed runs
Here's the Stats! Yay!
Code:
Title: NES Super Mario Bros (JPN/USA PRG0) "warpless" in 18:41.7 by Lee (HappyLee)
Download URL: http://tasvideos.org/movies/fm2/happylee3-smbwarpless.zip
Total Frames: 67413
Output Length: 4391 Bytes
Compressed %: 6.513580%
Title: NES Super Mario Bros (JPN/USA) "walkathon" in 06:47.2 by Lee (HappyLee).
Download URL: http://tasvideos.org/movies/fm2/happylee4-smb-sidestroller.zip
Total Frames: 24472
Output Length: 1498 Bytes
Compressed %: 6.121281%
Python Program (WARNING: Ugly Code
)
Code:
'''fm2bin.py
Convert an FCEUX .FM2 Emulator Movie File into a binary form for playback on
NES hardware.
Copyright (C) 2011 Norman B. Lancaster
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
'''
import os
import sys
import re
import struct
input_bytes = []
rle_words = []
word_count = dict()
dict_words = dict()
output_bytes = []
def main():
if len(sys.argv) != 3:
print '''Usage: fm2bin.py input output
input *MUST* be a .fm2 version 3 file. This is created by FCEUX. Only port 0
input will be processed. Port 0 must have been a game pad. Save states
are not supported either.
output The output binary file. See "fileformat.txt" for details.
'''
sys.exit(1)
# Load the input file
input = open(sys.argv[1], "r")
while True:
line = input.readline()
if line == "":
break
line = line.strip()
# Input record line
if line.startswith('|'):
byte = 0
match = re.search('\|([^\|]*)\|([^\|]*)\|', line)
token = match.group(2)
for char in token:
byte = byte << 1
if char != " " and \
char != ".":
byte += 1
input_bytes.append(byte)
input.close()
# REL-compress the data
last_byte = -1
count = 0
for byte in input_bytes:
if ( byte != last_byte and \
count > 0 ) or \
count >= 128:
rle_words.append(((count - 1) << 8) | (last_byte & 0xff))
count = 0
last_byte = byte
count += 1
rle_words.append(((count - 1) << 8) | (last_byte & 0xff))
# Build the word dictionary
for word in rle_words:
if not word in word_count:
word_count[word] = 0
word_count[word] += 1
# Sort the word dictionary
tupples = []
for key in word_count:
tupples.append((word_count[key], int(key)))
tupples.sort()
tupples.reverse()
# Take the top 128 elements as the compression dictionary
tupples = tupples[0:128]
for i in range(0, len(tupples)):
dict_words[tupples[i][1]] = i
tupples[i] = tupples[i][1]
# Output the dictionary
# NOTE: tupples is now an array of INTs, should have thought that one through :D
output_bytes.append(len(tupples))
for word in tupples:
# Just like in the record stream, count first, then byte
output_bytes.append(word >> 8)
output_bytes.append(word & 0xff)
# Output the input record stream
for word in rle_words:
# Compress-able record
if word in dict_words:
# Bit 7 is the "this is a dictionary word" flag
output_bytes.append(dict_words[word] | 0x80)
# Non-compress-able record
else:
# Count first, then byte
output_bytes.append(word >> 8)
output_bytes.append(word & 0xff)
# Output the file
# Note: there may be a much faster way to do this.
output = open(sys.argv[2], "wb")
for i in range (0, len(output_bytes)):
output.write(struct.pack("B", output_bytes[i]))
output.close()
print "Total Frames:\t%d" % len(input_bytes)
print "Output Length:\t%d Bytes" % len(output_bytes)
print "Compressed %%:\t%f%%" % float((float(len(output_bytes)) / float(len(input_bytes))) * float(100))
if __name__ == "__main__":
main()
File Format Documentation:
Code:
File format description for the binary output file created by fm2bin.py
by Norman B. Lancaster
Offset Length Desscription
0 1 (N) Length of the compression dictionary in entries
1 2*N Compression entries.
Byte Meaning
0 The number of frames to hold this data in the output latch for controller 1
1 The data that should be loaded into the output latch for controller 1
2*N+1 XXX Input record entries.
Byte Meaning
0 The number of frames to hold this data in the output latch for controller 1
If bit 7 is set this is an index into the compression entries table (expressed
as an offset in compression entires, NOT bytes) to load.
1 The data that should be loaded into the output latch for controller 1
If bit 7 of byte one is set, skip this byte.
Results for Reference FM2 Files:
Title: NES Super Mario Bros (JPN/USA PRG0) "warpless" in 18:41.7 by Lee (HappyLee)
Download URL: http://tasvideos.org/movies/fm2/happylee3-smbwarpless.zip
Total Frames: 67413
Output Length: 4391 Bytes
Compressed %: 6.513580%
Title: NES Super Mario Bros (JPN/USA) "walkathon" in 06:47.2 by Lee (HappyLee).
Download URL: http://tasvideos.org/movies/fm2/happylee4-smb-sidestroller.zip
Total Frames: 24472
Output Length: 1498 Bytes
Compressed %: 6.121281%