bsnes and headers

I would, except there was this emulator that demanded I remove the SMC header from all of my images, so they no longer work on my SWC DX/32.

Oh no, how stupid! I hope that you and everyone else will loudly protest that change everywhere. Maybe the developer will see the logic in supporting the dozens of incompatible floppy disk copiers from the early '90s at the expense of file size/checksum and patching verifications.

If only it were possible to write a two-line batch script to copy to disk and then add a header with uCON/inSNESt.

Although it's not as complicated as programming SNES homebrew and setting up/managing online networks, it's probably above our abilities, sadly.

Can anyone here help out me and koitsu with something like that? I can start a Kickstarter project for it if that helps.

I'm agreeing with byuu here. If you want to add a header that a particular copier needs, add it just before you put the game onto the copier. For example, the ROM builder for the PocketNES emulator expands each 16-byte iNES header into a 64-byte PocketNES header.

Losing the sarcasm ...

I get that many SNES coders are still rocking their copiers. I'm not saying you have to get rid of them. But you have to realize you're a minority. I have an old Win9x box that I use for older Falcom games. When the RAM died, I had to go out of my way to find SDRAM for it. My GDSF7 has a parallel port, my comm cable uses serial, and yet my modern computers don't have serial/parallel ports, or even floppy connectors. If you want to fight change for 20+ years, expect to be inconvenienced. Don't expect the right to inconvenience everyone else for your benefit.

Most people now have flash carts for the SNES, where you can put 100% of your games on a single SD stick and load them in two seconds. And they're mostly cheaper than copiers were, and far more readily available and still made. And smaller, and don't need another power brick, and they don't need copier headers because the designers weren't lazy, and some can even run DSP-{1-4}, ST010, Cx4, etc.

Copier headers are hurting -everyone-. Many ROM hacking tools only work with headers (Lunar Magic), many tools work only without headers (mosaic), some emulators auto-remove copier headers (Geiger's Snes9X debugger) or don't support loading them at all (bsnes) or try and parse them and will fail to play the game if they are malformed (Super Sleuth), some tools auto-add copier headers (Lunar Magic), many ROM hackers make IPS patches against headered ROMs, and many do not. Every patch release on RHDN deals with patching errors due to this. End users don't know how to add and remove these. Sets like GoodSNES are inconsistent where half the ROMs have headers, and half do not. Groups like No-Intro and Zapatabase remove headers entirey. Groups like NSRT subvert the copier header by creating ZSNES-specific emulator headers. The images distributed have copier headers in 20+ differing and incompatible formats. Later SNES copiers support 5+ formats, but never all of them. Many, many, many copiers won't work with just a copier header, anyway: they expect interleaved images. No emulator supports widescale interleaving of image data. Floppy disks are limited to 1.44MB, so you can't play 2MB games without doing work on your end anyway. You can't rely on the CRC32 of a file inside a ZIP to identify a game, file scanning tools can't detect duplicates because you might have the same game in two copier formats, sha256sum/md5sum/etc tools are useless. Everything has to be aware of copier headers. Emulators have to do odd math to try and detect internal headers now not just at $7fc0, $ffc0, $40ffc0; but also at $81c0, $101c0, $4101c0. You can't even do filesize&0x7fff==512 due to older homebrew that doesn't pad the ROMs out. ROM hackers that use hex editing tools have to manually add and subtract 512 in their heads or with a calculator constantly. And on and on ... I have had to deal with ALL of this. Literally.

I've even made this easy. A simple tool will purge all headers in two clicks. My cross assembler easily supports an ifdef statement to include a copier header. That's how I used to test my own stuff while still running it in my emulator. bass -DHEADER -o a:/test.smc test.asm && bass -o test.sfc test.asm. Now all you need is a batch file to drag your games onto to play them. The same work you'd have to do -anyway- for the 80% of your games that are >=12mbit and/or HiROM.

I don't know why so many highly intelligent individuals are so hellbent against any change at all. I'm not out to be the Grinch who Stole Copier Headers. I'm really trying to help make it easier for everyone, but I have to constantly be fought tooth and nail. I wish it were just the usual nobodies with no experience or knowledge complaining. That wouldn't bother me nearly as much. But you should know all of the above very well. Having such highly regarded individuals as yourself oppose this is exactly why we can't get the traction to make real progress.

I took a jab at you 70% serious / 30% in jest because (IMO) you deserved it. You're too busy to test something on real hardware, but you're not too busy to kill two birds with one stone.

I'm trying to keep this as terse as possible, but the more I write more I realise it has to be verbose (I can't cover all the angles in a single paragraph) and I don't want to get verbose in this thread because it's unrelated to the DMA register topic.

We can talk about it in another thread if one is started. But if you start that thread, I strongly suggest you go back and read what you wrote me in PM circa February 2011 then compare it to your words + actions of today. TL;DR -- it's very hard for me to believe someone who talks one way but walks another, and that's coming from a guy (meaning me) who has a well-established history of being an absolute jerk.

I could care less about BSNES not loading SMC, SWC, FIG, etc files with copier headers or interleaved ROMs. But it is annoying that atleast last I heard it had dropped support for loading Zipped ROM files. I forget whatever reason there was for dropping support but it's annoying to waste space like that.

Dealing with Copier formats and headers couldn't be easier with UCON64 around.

If you or tepples want to split this thread, that's fine.

I am genuinely interested in hearing your arguments about why copier headers should be supported, without bringing in "because that's the way things used to be done."



The IRQ/IO->read cycle finding took me two weeks of full-time work to fully understand. The HDMA->MDR update issue took me another two weeks. The DMA/HDMA<>CPU bus sync took three or four weeks. Figuring out this DMA crash is easily a 2+ week job. It's easy when you know the answer, but to find the answer you have to test -every possibility ever-. Writing the iSNS post took me ten minutes. Which is probably the same amount of time Marat put into iNES.



I really dislike the insinuation that people can't change their positions over time without it being a problem. Reflection and admitting when one is wrong is a sign of character development. But that said, let's take a look ...

02-2011:
"I have basically begged people for several years to take over the bsnes GUI. I definitely want someone to make a bsnes GUI that supports all the copier extensions and interleave formats and nonsense in the world. Hell, I tried to do it myself, bsnes v073 is the ONLY SNES emulator to load multi-archive 7-zip ROMs."
"As I said, it is a philosophical contradiction to me to say "I hate SMC and copier headers" while implicitly endorsing them by supporting it. I don't want that shit in my codebase."
"The only proper way to do it is to score ${ 7fc0, 81c0, ffc0, 101c0, 40ffc0, 4101c0 } and pick the most likely candidate."
"Your code would not boot your own iNFINITY demo ROM, nor a bunch of older PD ROMs. I believe Cowering manually padded many of them for compatibility with ZSNES."

10-2012:
http://board.byuu.org/viewtopic.php?p=76050#p76050
"The thing is, I don't care if it's modified to load ZIP files, remove copier headers, deal with SMC/SWC/FIG/BIN, etc. I just don't want to write and maintain the code in my codebase."
"Emulators have to do odd math to try and detect internal headers now not just at $7fc0, $ffc0, $40ffc0; but also at $81c0, $101c0, $4101c0"
"You can't even do filesize&0x7fff==512 due to older homebrew that doesn't pad the ROMs out."



To reiterate, my issues were that it required a 300+KB dependency on zlib, and infighting with Debian upstream over using system zlib on Linux despite Windows not having a system zlib, having to accept zlib's license, as forgiving as it is, and with losing the association between extension=system, meaning you'd get false-positives so that every .zip shows up as a possible game to load, even though few ZIPs are actually SNES games.

You may be in luck there. I wrote a ZIP implementation based on miniZIP that's <10KB in size. It's just a matter of whether or not I can parse the ZIP headers, find the SFC images inside, and show them in a file browser quickly enough without lagging the emulator out.

The harder part will be finding ZIP files with .sfc extensions inside and no copier headers. It's a done-deal for No-Intro and Zapatabase sets, at least. NSRT pushes JMA, and GoodSNES pushes SMC+cointoss copier headers.

I do want to point out that there are, at best, 4000 SNES games, including all revisions and all countries. That means you are talking about a space difference of ~1GB, which costs approximately 5 cents/mechanical, and 50 cents/SSD. Modern file systems like NTFS can natively compress folders too, in a way that's transparent to the application.

The only advantage I see to supporting ZIP is so that people don't have to extract their illegally downloaded ROMs (and noting that some browsers force decompress your archives, like Safari.)

I remember some good arguments against supporting .zip archives:

• HDD space difference is small compared to sizes of current drives and media people store.
• It's a drain for a developer who doesn't care about archives.
• The OS should do this, not every program.

My argument for supporting (skipping) ROM headers is that it makes the program handle a very common situation that many people have (having lots of headered ROMs) with zero effort, and the implementation is only a few lines of code (I'm only referring to merely skipping them, not things like interleaving etc.). This is the bread-and-butter of what programs do: take on slight costs that benefit the user many times over and eliminate wasting time on routine tasks.

There's a larger picture as well, that of each user's machine being a unique environment within a unique process they use for all the activities they do. With ROMs, they might use lots of tools that need headers, and prefer to use headered ROMs for this purpose. Supporting the user's choice makes sense to me in this case.

As for encouraging adoption of headerless ROMs, I'd prefer approaches that give users new options that are preferable to headered ROMs, rather than leaving support limited where it could be added trivially.

I guess you tried the approach of having someone else manage the GUI, and add all these things that people want but that don't interest you in the least. I don't know the specifics and the reasons for the way things are done. I take it nobody took you up on making a front-end that satisfies more people. I find this the most compelling argument that there's not much to come back with: nobody wanted to help with the GUI and I personally don't care for headered ROMs, so it doesn't support it.

True. My arguments against it were above, but most poignantly:
* the entire ROM hacking scene won't move on from IPS. 50% of patching fails and requires users to add or remove copier headers and try again.
* some tools force you to have headers, some tools force you to not have headers, some forcefully add them, some forcefully remove them.
* it's not just two lines of code if you want to support unpadded homebrew ROMs, soft-patching, and file hash verifications.

Yes, killing off copier headers is painful. But we -can- do it, and everyone will be better off after it's done. We killed IE6, people said that couldn't be done too.
We just need pessimistic leaders in the community to stop acting like it's their Second Amendment right to bear copier headers.
Honestly, I know this sounds arrogant, but it really feels like there are many people who are against things solely because I'm involved.

I refuse to believe we can't make progress. No-Intro dropped SMC and copier headers, and they're the predominant set. All we need is Snes9X to seal the deal.

I've gone out of my way to help. I wrote a launcher/DLL thread injector that forcefully modifies the closed-source Lunar Magic to support headerless ROMs, since its author refuses to. And then jumped through hoops to get around his inane license rules. I even wrote it generically enough that it'll work on any Windows application.



That's true, and look at Microsoft. They can't abandon Win32 and its 50,000 undocumented "bugs" because old programs rely on it. Wine is a hopeless project that has been trying to replicate those quirks for decades. Do we want our emulation scenes to be supporting copiers 30 years after the last physical copier has died?



Correct. SSNES/RetroArch was almost there, but then went on to support as many emulators as possible, puts more focus on Snes9X these days, and won't support the formats I am proposing at all. In fact, they've even extended my shader format so that half of them no longer with with my emulator. The features are nice, but extending a spec that isn't yours is not.



Pretty much. I've said time and time again, it's a personal project. The only reason I keep it going is because I can maintain interest. The only way I can maintain interest is to be true to myself and what I want to see. I'm not looking for popularity (although it's of course nice for furthering goals), integrity matters more. Yet at the same time, I recognize that I can't force everyone to do what I'd like them to. So I'm just going to keep espousing my views when provoked, keep doing what I want with my codebase, and let people do what they want with my open source code.

One use I can see for supporting ZIP files would be what I did with Project MD, which was distribute both the binary and the source code in the same ZIP. These are the side effects of such a move:

• If you just want to play the game in an emulator, make the emulator open the ZIP.
• If you want to mess with the source code, you can extract the ZIP file.
• If you distribute the ZIP, you're automatically complying with its license, without even needing to know about it (which can matter if somebody decides to upload it somewhere without caring about it, then at least downloaders won't have to worry about it).

Of course, given most homebrewers only want to release binaries, maybe it isn't worth it... And it'd be nice if operating systems allowed access to archive files as if they were directories - and by this I don't mean in the shell, but rather the file I/O functions themselves (including the C standard library), then this wouldn't be an issue at all.

From the romhacker point of view, I really prefer the header not to be supported and I really don't care about ZIP support. Why?

1) When reverse-engineering the code in a ROM, I prefer the ASM code to be referred to address $0 in the file, not $200 or whatever: easy to make calculations without calculators (even for LoROM, I did all calculation in my head).

2) Headers are useless when running on debuggers or emulators so, why preserving them till the end of time? Evolving is inherent to mankind, and when we talk about emulation, it makes sense too.

3) ZIP files are hard to handle when it comes to romhacking: if the ZIP contains several versions of the same code, you may choose the wrong ROM inside it when you open it from time to time. It's pretty more secure to unzip the required ROM and keep it as the "master source code".

4) I don't use bsnes for gaming, I use it for accuracy issues. My laptop even can't run bsnes faster than 35fps, so it's impossible to me playing games. But when it comes to test hombrew code, it is the best emulator, just like running the code on the SNES. From my point of view, bsnes is aimed to developers like me, and I don't need ZIP files or headers or anything else but 100% accuracy.

I think byuu's work deserve our admiration since it will be a reference in the future: whenever somebody needs to know how the SNES worked internally, he will check bsnes source code to get the answer.

The below is my viewpoint, and it is shared by multiple peers of mine who have equal "stance" or "seniority" (as if that somehow means something -- it damn well shouldn't!) in the community.

The situation is this:

The author of a SNES emulator doesn't like headers which are intended for copiers (ex. SMC, FIG, split-SMC files, etc.). I'm going to reiterate this for folks with a slight rephrase: one man who happens to be an emulator author doesn't like them. This one individual has some kind of personal vendetta against them, even taking jabs at them, as well as other emulator authors in the process (hence my two-birds-one-stone comment).

The technical reasons are nil -- there is absolutely no technical reason that has been presented so far that justifies this stance. The headers can technically (with software) can be ignored. The files do not need to be changed for the sake of one man's crusade. This therefore implies that a person who has a copier (SWC DX, etc.) would need to keep two copies of their ROM set around -- one with headers, and one without -- all because of one person.

Since there is no technical justification or merit to this, we can only conclude that the reasons are personal. This conclusion is also supported by facts (keep reading).

The author himself has stated repeatedly that he finds the headers to be difficult to deal with from a software perspective. Again: this is not a technical limitation, but rather a personal one -- the author is annoyed by them, is annoyed by having to deal with them, doesn't like writing the code to deal with them (keep reading, these are not assumptions, they are accurate statements). The reasons behind why he feels that way don't matter either -- it's his opinion, and he has a right to it (and I fully support his right to have an opinion).

These reasons are backed by his own words, in which every single statement made to justify the removal of these headers involves the use of the word I. As in "me", as in "it affects me", as in "it upsets me". Him. Just him. So we have further proof that this is purely just one person's opinion -- and again, having that opinion is just fine.

It's not the opinion that is offensive -- it is the fact that the opinion turns into a crusade. A mission. A witch-hunt of sorts. The opinion becomes something forced upon other people, almost in a political or lobbyist fashion.

Phrased less eloquently: "To use my emulator you must modify all of the existing files you have. I do not care if they are 30 years old and hold nostalgia value. I do not care if you own hardware copiers that rely on these headers. I do not care if from a technical standpoint these headers can be ignored, because writing the code that deals with that bothers me. You will adhere to what my requirements are, even if those requirements (changing all your existing files) will impact other programs (emulators, etc.) or impact your ability to enjoy these on a real console (sending SMC files to a SWC/DX, for example)."

Historically the standpoint has been "it's my emulator and I will do what I want" -- which is fine. The problem becomes when what one person wants mandates everyone else comply with what that person wants as well, especially when there is absolutely no technical justification for it (if you've forgotten this fact, re-read my post from the beginning).

So from this we can conclude that the author does not like something, therefore he feels everyone else must comply with his lifestyle/choices/issues/demands. Rather than focus on the negative, let's ask the question: what does the individual like?

The author appears to enjoy one thing: writing a decent SNES/SFC emulator. He likes working on the emulation part of the console, he likes getting his hands dirty with regards to that aspect. He does not enjoy dealing with file formats. He enjoys emulation. And that is something he does amazingly well (better than most others out there, hands down, no argument).

Thus we can conclude that the author should be doing that: focusing on the emulation part. Not file formats. Not going on religious anti-header tirades. Not arguing or insulting other emulator authors. Not forcing people to adhere to his standards / his opinions / his likes and dislikes, thus modifying their entire ROM sets to comply with his views.

The author has even admitted privately to me (it's excluded from the quote set above) that his libsnes library doesn't even require a file handle nor does it interface with the filesystem at all -- you just hand it pointers to linear pieces of memory and it takes care of the rest. So on a technical level he has already begun (and continues to work on) the part that he enjoys and does not frustrate him. This also does not require the rest of the world adhere to his beliefs (i.e. changing all of their ROMs to remove headers).

The author has also been adamant in admitting that the "front-end" part (GUI, file format handling, etc.) he would much rather leave in the hands of someone else so he can focus just on what makes him happy. This statement in itself should make any intelligent reader wonder why the author is on a crusade at all -- there's nothing forcing him to work on the "front-end" part especially when it makes him so miserable. (This is also justified by the removal of many GUI and visualisation options in bsnes, which is something that peers of mine have complained about for months (years?) -- specifically when those features were removed "just because they were annoying to deal with").

The answer and solution then becomes quite obvious and simple:

1. The author should stop the tirade/crusade and stop forcing people to adhere to his demands/standards/opinions while making two-faced comments (quote: "I'm really trying to help make it easier for everyone"),
2. The author should focus on his emulation library and emulation bits, as that makes him happy, and it's something he does amazingly well,
3. The author should leave the "file parsing" bits (for lack of better term) and "front-end" (GUI, etc.) to someone who isn't bothered so much by the complexities of dealing with them, supporting multiple file formats, etc..

Leave the front-end to someone else. Can't find someone else? Doesn't matter -- not your problem (honest!). Focus on what makes you happy and stop forcing everyone (through your emulator) to comply with your beliefs.

I won't be responding to this thread past this point. I've covered every single base and have stated the facts as well as interwoven my own opinions. But just in case it's not clear: there's no technical merit behind the removal of the headers (requiring everyone to modify their files). It is purely personal. And because it's personal, byuu should focus on his library and leave the "messy stuff" that he doesn't like (which is his right/opinion and that's fine!) to someone else.

Why are you acting like this is only byuu's personal stance? byuu has little to do with the fact that headers have fallen out of favor (Hardly anybody listens to him anyway ). I hated headers back in 1996, before byuu was on anybodies radar. I didn't use a copier then and don't use one now. Back then headers were an annoyance for us to work around and only served a purpose for the elite that could afford a copier (Being a poor teenager back then prevented me from getting one). Let me use your own words against you... There is NO technical reason that currently justifies the use of headers from an emulation, rom hacking/translation or preservationists stand point. byuu challenged you to state one and you failed to do so and proceeded to make a long drawn out argument about how it is byuu's personal crusade against headers.

Most of the current ROM preservation community doesn't like headers. People want purity in their ROMS and this has existed BB (Before byuu), evident by cowerings creation of the (!) years ago to signify a "pure" ROM. I obviously appreciate the work of the people back then that used copiers, but I see no reason to continue passing around ROM images with headers on them. I doubt the old timers with copiers are still regularly using them anyways. Everyone that I know with a copier tells me that it is in a closet somewhere collecting dust.

It sounds like you are taking this personally... The community has evolved and you are being left behind. Are you basically against evolution/change and want things to remain as they were back in the 90s for no logical reason (other than you might use your copier sometime in the future if you have time to dig it out of the closet )?

P.S. Nobody is forcing you to update your emulator, just continue to use an older emulator with your aging ROM set .

As a developer of both homebrew software and ROM-handling tools (the Forbidden Four and Action 53 builders) for a closely related console, let me add my two bells:


Another thing I would be in favor of requiring is that homebrew be padded to the next 32 KiB (for LoROM) or the next 64 KiB (for HiROM), just as the iNES format requires it to be padded to the next 16 KiB.


Microsoft has used the introduction of the Surface tablet to deprecate Win32 in favor of WinRT and at the same time grab its $99/yr + 30% pound of flesh from anybody who distributes an application compatible with Surface.


Netscape extended a spec that wasn't its (tables, forms, and JavaScript added to HTML), and web pages became interactive. Microsoft extended it further (XMLHttpRequest), and web pages became applications.


You make a good point. What's a better way to encourage end users to comply with a copyleft license?


At a fairly substantial cost in time in the case of running on a modern CF- or SD-based copier that understands the internal header, such as the SNES PowerPak: the copier has to load the first 65536+512 bytes and check for the internal header in four places as opposed to two.

So perhaps the best compromise, given that byuu doesn't want to deal with headers or front-ends, is to give the front-end the job of formatting ROMs correctly for bsnes: copy a ROM with a header to the temporary folder without a header before running it.

I want to be clear. I am not 100% opposed to copier headers. I get the pragmatic approach of not breaking things that already work.

But I am weighing their good versus their harm, and believe it's in our best interest to remove them from the general circulation.



Why do you assume it's only me? Just in this very thread: tepples, magno, Mottzilla, and HJRodrigo don't seem to like or care about them.



I wasn't aware Marat Fayzullin invented SNES copier headers.



I listed over a dozen reasons. Many of which were technical.

"Copier headers are hurting -everyone-. Many ROM hacking tools only work with headers (Lunar Magic), many tools work only without headers (mosaic), some emulators auto-remove copier headers (Geiger's Snes9X debugger) or don't support loading them at all (bsnes) or try and parse them and will fail to play the game if they are malformed (Super Sleuth), some tools auto-add copier headers (Lunar Magic), many ROM hackers make IPS patches against headered ROMs, and many do not. Every patch release on RHDN deals with patching errors due to this. End users don't know how to add and remove these. Sets like GoodSNES are inconsistent where half the ROMs have headers, and half do not. Groups like No-Intro and Zapatabase remove headers entirey. Groups like NSRT subvert the copier header by creating ZSNES-specific emulator headers. The images distributed have copier headers in 20+ differing and incompatible formats. Later SNES copiers support 5+ formats, but never all of them. Many, many, many copiers won't work with just a copier header, anyway: they expect interleaved images. No emulator supports widescale interleaving of image data. Floppy disks are limited to 1.44MB, so you can't play 2MB games without doing work on your end anyway. You can't rely on the CRC32 of a file inside a ZIP to identify a game, file scanning tools can't detect duplicates because you might have the same game in two copier formats, sha256sum/md5sum/etc tools are useless. Everything has to be aware of copier headers. Emulators have to do odd math to try and detect internal headers now not just at $7fc0, $ffc0, $40ffc0; but also at $81c0, $101c0, $4101c0. You can't even do filesize&0x7fff==512 due to older homebrew that doesn't pad the ROMs out. ROM hackers that use hex editing tools have to manually add and subtract 512 in their heads or with a calculator constantly. And on and on ... I have had to deal with ALL of this. Literally."

You have not refuted a single statement. This is not how you debate someone.



So if I am applying an IPS patch to a ROM: do I remove the header and then patch; or do I patch and then remove the header?

If the ROM is not padded to a multiple of 32KB, how do I know if there is a header or not?



As you already have to do for games >=12mbit that need to be split onto several floppies (unless you have a working SPP parallel port), and for games in HiROM format that need to be interleaved (which no emulator supports.)

Of course, you don't have to do this for any games. You just have to make a simple batch file to transfer games to your copier with the same amount of work it'd take to copy an unmodified file.



What gun am I holding to peoples heads, demanding they use my emulator?



Is that a serious statement? Are you nostalgic for 512 bytes of garbage at the top of your files?

Should I emulate the layering issues that caused trees to poke through text boxes and sprites to disappear underwater? Should I emulate the wind sounding like a seal being clubbed to death? You know, for nostalgia to the good old days of SNES emulation?



Cost/benefit analysis requires one to take in account how many people use SNES images (100%) versus how many use oldschool copiers (LESS THAN 1%.) Seriously, in all my years, I've met about 30 people with SNES copiers. My last software release has had over 40,000 downloads. And I am the underdog.

Should they all deal with patching issues to appease copier users?

Even the few I've known with copier headers are moving on to flash carts. They are infinitely superior. They can still be purchased new, they are usually cheaper, they can hold the entire game library, they don't require copier headers nor interleaving, and they can load games almost instantly.



This is untrue. Patching and unpadded ROMs.



Every public emulator supports headerless ROMs. Geiger's Snes9X Debugger requires it.

I wrote a Win32 API hooking library to allow any tools that require headers to work without them. Without having to make hidden copies of files. It hooks right into the file read/size commands to offset them by 512. I even made a bundle just for the most popular tool, Lunar Magic.



The 02-2011 and 10-2012 quotes show this is untrue. I am fine with people adding copier header support to my emulator if they want to.

But I'm going to share my opinion on why they shouldn't, and hope I can convince more people. That is what human beings do: they advocate their beliefs.



Now who's acting like the politician? Do we need a counter for how many times you've said the same thing? Saying it again and again doesn't make it true.

Try re-reading my posts, and addressing my technical justifications instead of pretending they don't exist.



Another repeated statement. I hope that everyone will follow suit and drop copier headers, but I'm not forcing anyone to.



I haven't insulted anyone over copier headers.

I haven't even insulted Marat, as you seem to think I have. It's called a sense of humor.



Another repeated statement.



I see you're not above quoting things written to you in private.

Yet this is no secret. I did this to allow Ryphecha to implement bsnes into Mednafen. I've had months of discussion on my forum about this API.



So because I don't like GUI programming, I shouldn't be allowed to discuss why I dislike copier headers?



If I don't have a front-end, nobody can use my work. There is no front-end that puts bsnes front and center. They're all multi-emulator paradigms, mostly defaulting to the faster, less accurate cores. Make a really fantastic emulator that can be called "bsnes", and I will happily stop releasing binaries.



Please tell me what it is I owe you and your friends. Why I am required to implement and maintain features you want.

Did you all pay me? No. Did any of you donate money to me in return for a favor? No.

Was helping improve the accuracy of ZSNES, Snes9X, SNESGT, SNEeSe, Super Sleuth and no$sns not enough for you?

Was spending $15,000 of my own money to buy and verify the dumps of all US games (I've already found a few hacked images) not enough for you?

Was raising $2,500 and burning all the goodwill donations I've raised over the last eight years to get all of the SNES coprocessors decapped and dumped, allowing SNES emulation to finally reach 100% compatibility, not enough for you?

What will be enough? When did I become your indentured servant?



Two-faced? What does that even mean in this context?

Ask any ROM hacker how sick they are of dealing with patching problems.

It really will make things easier. Less complexity (one format instead of two) = easier. It's a basic axiom.



"The author should shut up and do what I like."



Another repeated statement.



"I'm getting the last word in here."



"by ignoring every single argument byuu presented against copier headers completely."



"I'll repeat another statement."



What really surprises me is just how amazingly centered it is on the SNES devs (homebrewers, ROM hackers, translators.) Casual gamers don't even care. They run purify, and three seconds later they are gaming. My download count hasn't taken much of a hit at all.

But the ROM hackers, holy shit do they ever hate change. Most are still using ZSNES, IPS, xkas v06, copier headers ...

In koitsu's case, it's basically the backfire effect:

http://youarenotsosmart.com/2011/06/10/ ... re-effect/



(And you know, maybe it's true for me too here. I like to think I am sympathetic to all logical arguments. And I admit there is a pragmatic benefit to just removing the headers where you are able to. I am trying my best to weight it all logically. To make a big column of pros versus cons. So far, I have 2 pros to copier headers (established practice/convenience and in some cases copier users), and 19 cons.)

It's a very well understood phenomena. When peoples' beliefs are challenged, they double down. It actually becomes even harder to convince them that they are wrong.

koitsu doesn't have a single poster in this thread agreeing with him, and yet he thinks it's just big-bad-byuu out to take away his rights unilaterally.

He's a smart guy, though. I just need him to seriously consider my arguments without emotion being brought into this.

I'll give you a side story.

Long ago, I was really huge on hating Cowering because he was including fan translations that got propagated to ROM sites. I joined the hate bandwagon with every other ROM hacker.

But I was wrong. Translating a game without the author's permission is a violation of the Berne Convention (of which the US and many other nations are signatories.) Distributing the original games pre-translation is a violation of copyright *no different* than distributing the fan translation is also a violation of copyright. There is no honor among thieves. To wantonly disregard copyright when it suits you, and then demand it in turn, is the height of hypocrisy.

You couldn't convince me I was wrong back then. Eventually though, all the people pointing this out to me stuck in my head. Eventually, I reflected on things, and realized how arrogant I had been. And I reversed my position. Earning me the scorn of the ROM hacking community. IST went apeshit on me over it just recently even.

But if I could re-evaluate my deeply held beliefs, I'm confident koitsu and others can do the same.

Interestingly enough this has a direct relationship to the issue of headers. If it were not for Cowering's work there would be many translations/hacks that I would have not have even tried, due to poor documentation. Authors of these patches often failed to specify whether the ROM images required headers or interleaving. I would just say screw it to these works, as I did not want to try multiple IPS/ROM combinations. Then I discovered that Cowering did the leg work for me, so I would just visit one of those sites that propagated these authors works and try out the games. This is of course a problem that need not occur this day an age with people adopting no intros ROM standards.

P.S. I belonged to a different type of hacker, one that viewed ROM sites and acknowledgement in goodtools as a good way of getting our crappy work out there . Only the big shots seemed bothered by having their work cataloged by goodtools. Many of us small time guys would actually submit our work for inclusion into goodtools .

My beef with Cowering was mostly when he misspelled my name in some GoodNES entries.

My name is Damian YERRICK. It's listed as such at the bottom of every page on my web site and on my contributor page on English Wikipedia. Yet I've seen "Yeppick" and "Terrick" on some of my works in GoodNES, possibly due to my attempts to stuff an half-uncial or insular font into 8x8 pixels. If you've wondered why I've used the same plain-jane Chicago font (see "Who's Cuter" in this sampler) for just about everything since then, now you know.

Another point ... let me show how easy it is to have a complete collection of headerless ROMs, and still use them with your copier.

First, download and install kaijuu (http://byuu.org/programming/kaijuu)
Add a new rule: for files of type "*.sfc" execute: "ucon64 -swc {file}"; optionally make it the default.

If you set it to the default, you can double-click your files and play them on your SWC.
If you didn't, all you have to do is right-click the file, choose "Transfer to Copier" and enjoy.
No need to ever have two copies of each game.

kaijuu will also let you associate all of your SNES emulators, so you can easily play them.
But if you won't use it because I made it, you can add the action to HKEY_CLASSES_ROOT\*.sfc manually.

Linux file managers have kaijuu's functionality built-in, too. Thunar calls them custom actions, Dolphin and Nautilus have them as well.

...

So I create a two-click tool to remove all headers, a two-click tool to play games on copiers, a drop-in replacement for tools that only work with copier headers, give out my full emulator source code under an FSF/OSS license, and I'm the bad guy.

FuSoYa makes a closed source tool that only works with copier headers, because that's what he likes, and nobody has any problem with that, and he doesn't take any grief for it, and he's the good guy. Go figure.



It's another point of stubbornness against change.

BPS is so vastly superior to IPS, and is in Snes9X, but nobody will use it for their works.

BPS verifies the checksum of the game before and after patching. It even verifies the patch itself is undamaged.
BPS offers metadata authorship. So you can embed your contact info, website, and even the readme itself, into the patch.
BPS is a delta format, so you can insert, move and delete data. Substantially smaller patches on non-ROM based systems.
BPS' delta encoding can also result in significantly smaller patches on certain ROM types (eg Mystic Ark is half the size this way.)
BPS has no limit on file size: not 16MB, not 4GB, not 100 Googol Exabytes. No limit, period.
Writing a BPS patch applier or linear patch creator is doable in ~20-30 lines of code; only ~30% more complex than IPS.
Writing a BPS delta patch creator is of course tough, but only one person needs to make that.
It also supports BPM, which allows you to patch entire folders. Allowing the same format to handle ROMs, CDs, PC games, etc.
The spec itself is many, many times smaller than Xdelta and bsdiff. It's possible for people to embed soft-patchers with this format.

All that's needed here is an incremental approach.
Start by allowing IPS and BPS in emulators, and offering all patches in both IPS and BPS on RHDN.
Continue by releasing new patches only in BPS format.
Eventually, start making soft-patchers offer to convert IPS to BPS.
Create compelling advantages for BPS: like emulators that can check a database when you load a game and find translations for you to download.
Next, stop offering IPS patches for download at all.
Finally, remove IPS support for emulator when its usage drops below 1%. Be willing to deal with being called a "religious crusader" for it.
Done =)



Whoever indexed Der Langrisser attributed it to [D+byuu], which is unfortunate for the 20 other people who also helped out.

BPS seems great, but I am under the impression that the ppl at RHDN think you have abandoned it like was done with UPS. Release a minor update to beat and I will add a news update on RHDN, so everyone will know you are still backing this format. If you plan to release your newest hack with it (which I thought you mentioned would be a necessity for some reason) then this could also be mentioned in the news update as a selling point of how good the format is. Technically you could do the news update on RHDN yourself, but I believe you left the site and a log in is required to post news updates.

We screwed up with UPS, yeah.
Focused too much on the niche feature of patch reversibility and didn't factor in its costs: substantially larger patches post-compression.
Wasn't going to stick with a bad format just to bolster public confidence.

But yes, BPS is a done deal. Extremely well vetted, tested against thousands of files. It's not going anywhere.

Newest version was posted in August, there's little to be done at the moment: http://byuu.org/programming/beat/

We just need someone who -really- understands delta encoding to help make a more effecient delta algorithm for files > 100MB.

I suppose if you guys want to push it there, great, I'm not going to. That would be like a hen in a foxhouse. May want to wait for the next Snes9X release so that a mainstream release supports it.

Most emulators spazz out when there are multiple ROMs in a single ZIP so those ZIPs already would be unusable for starters.


If you want 100% accuracy then ditch the emulator and use real hardware. Sorry, but I have learned that emulators shouldn't be trusted at all - even when they emulate things like cycle-level operations and such, they still won't emulate issues like the hardware crashing if you try to access it too quickly (or if they do, they won't emulate in what way this happens) and such. On systems with caches they also won't emulate the caches properly, which can cause issues depending on whether caches are flushed or not (or which parts of the cache). Also different revisions of the hardware have different issues and emulators rarely will emulate them.

Emulators are good to debug game logic - and to be fair, this is what you're going to do most of the time. They are also good to see if you're uploading the right data to memory before you can see anything. But if you need to see if some new code dealing with hardware is working, you have no option but to use real hardware to be sure (and even then you have to be careful - if you're using a flashcart that has a firmware instead of booting directly chances are the hardware initialization done by the firmware will later mess up debugging, I already know of one flashcart firmware that likes to clear RAM upon boot).


Reminds me of how many Sonic hacks were still being made for Gens no matter what (not to mention how almost every single hack seems to have an address error somewhere). These days that kind of dropped out thanks to the EverDrive making it easy to use flashcarts with modern systems (so people want to play hacks on real hardware), but yeah.

(says the guy who still sticks to asm68k, but honestly this has more to do with about every single assembler having its own variant of the syntax making things completely incompatible with each other unless you want to drop the most useful features)


LOL my biggest issue is how about every homebrew game seems to get labeled public domain -_-'

EDIT: also why is it that on the Mega Drive end headerless ROMs were the standard (with the only copier ever supported being Super Magic Drive, and only back when it was still somewhat popular) but this isn't the case with the SNES? Heck, Mega Drive ROMs even have .bin as their standard extension rather than a console-specific one (there's .gen, but nobody uses it).

I will agree that hardware is always preferred where possible. But bsnes is good enough for prototyping that you don't have to run every small change on hardware.

For cache, take the SuperFX. I emulate the 256-byte instruction cache, and the way it's setup as 16x16-byte stripes that fill any time a byte in that block is fetched. I emulate its ROM buffer cache, its RAM buffer cache, and its primary -and- secondary pixel buffer caches. And all that for a chip used by ten games.

No, it's not perfect. But it's an order of magnitude more precise than any other emulator. I'm really serious about getting things as close to perfect as possible.



I do emulate the differences found between the CPU revisions 1 and 2, with exception of the DMA crash. I handle the different pattern to DRAM refresh, to HDMA init timing, etc. These things are -really- hard to find, it's needle in a haystack stuff, so certainly I've probably missed some things.



A nice part of mine is a table-based syntax. So you can use any opcode syntax you want, you just have to write the table.
Nobody's done a 680x0 table yet, though.



Well, at least interleaving mostly didn't take off because only ZSNES tried to support it initially. Thankfully Snes9X was popular enough that their not doing so lead to ROMs being deinterleaved widescale.

The SMD file format is pretty annoying, but at least it's easily detectable thanks to that forced "SEGA" text.

I really admire the Genesis memory map, though. ROM starts from 0 and goes upward.

Here is SNES mapping, by comparison: http://byuu.org/snes/database/

There's about 50 boards that all do things slightly differently. The ROM header does not give you enough info to tell which board it uses.

I considered waiting, but then I realized Snes9x may never get a new release (like ZSNES). My fiancée has an account in her nickname over there, so I just went ahead and got it submitted (It is currently pending acceptance). One day, I will make an account for myself .

I had to paraphrase/shorten some of the information you had written on your site and I combined it with some of what you wrote here for the sake of easy legibility and since I am no editor (or a native speaker) you should forgive any errors I may have committed in the process... You know what they say, "If you want something done right do it yourself" .

When I wrote my interpretation of the GoodCodes for Pocket Heaven, I didn't mention public domain at all in my explanation of "(PD)". Instead, I wrote "(PD): Internet release authorized for free redistribution".


Early on, GBA also used .bin, but it fairly quickly switched to .gba for programs designed to run in ROM or .mb for programs designed to run in RAM. I prefer .gen for MD because it's distinct from any Atari product. At one time, I planned to make a launcher for .bin that would peek into the file, detect whether it's Atari 2600, Genesis, or GBA, and launch the appropriate emulator, like the .py launcher for Windows introduced in Python 3.3.


My sick vision of a 68000 table includes letter names for registers (A-H for D0-D7 and Z-S for A0-A7) and 6502-style load/store instructions as aliases for MOVEs in and out of registers. Imagine LDA.B, LDX.W, and STY.L. Then imagine 65816-style REP and SEP, but instead of emitting an instruction, they switch the default size between .B and .W for subsequent MOVEs without a dotted size in the same scope. Either that or ape the syntax of Thumb.


Then perhaps it's time to design our own copier header: jSNS. It'd be like iSNS but one better, as it would encode the missing info as a JSON object stored in a separate file: Super Mario World.sfc and Super Mario World.jsns.

Heh. I created a 6502 mnemonic set for the SPC700. You could tell 90% of the opcodes were "inspired" by 6502 anyway. They probably just redid it all last minute for legal reasons. mov x,#$00? Come on. ldx #$00.

nocash remakes every instruction set into x86 for some reason.

So there's definitely precedent if you want to make your own. Unlikely to catch on, but we usually do these things for our own benefit.



That was part of the joke, yeah. For all the good iNES does, you could just make a 2-byte mapper ID and uniquely specify every tiny variant of every board out there. The same is true for the SNES. Drop the maker ID (SHVC-, MJSC-, MAXI-, EA-) as irrelevant, and then store the rest as the header. "1A3B-20" ... and an internal DB of the 50 boards. That's all you need for 100% perfect mapping.

My vision though is to leave the binary files as pure ROM contents, no user-supplied stuff that is due to be argued about for generations like iNES 2.0 and UNIF are, no compression archives, no existing containers like ZIP/TAR with all the stuff that doesn't matter to emulators. That gives emulator authors two choices.

Option 1: keep a database of SHA256={mapping layout}.
I'm making that database now at byuu.org/snes/database

Option 2: provide a manifest file that describes the PCB.
Just like your .jsns, I support this as well. Only instead of JSON, I went with something far simpler to parse. I call it BML.

Basically it's indent-level sensitive ala Python. You can do attributes like XML, but internally they're child nodes just like anything else.
You can assign with name=data, or name="data with spaces", or name:data with spaces and "quotes"
= runs to the next space
="..." runs to the second "
: runs to the next line feed (if it's a child node, it appends multiple lines)
And that's it. No comments for the same reason as JSON. No escape character/entities to decode. No Billion Laughs by entity expansions (XML) or linked nodes (YAML). Can be decoded with no memory allocation other than the child nodes (inserting zeroes into the loaded document, nothing ever expands.)

Here's an example:



Only cartridge/board is needed. cartridge/information is optional, and cartridge/information/configuration tells you how to split a ROM file into individual chunks. For the NES, it'd help you split PRG and CHR. For the SNES, it helps you split program from data ROMs, and coprocessor firmware ROMs.

Emulators would be free to support whatever they wanted in cartridge/information, since it's extensible.

Aside: title is for the native language, so it'd be Japanese letters if from Japan. name is always using Roman letters (A-Za-z), and avoids NTFS restricted characters like : and \, and is sort friendly ("Brainies, The" and not "The Brainies" like the title.)

So my long-term vision ...

Most people will just have their folder full of .sfc files. They will pick games and play them. I find them in my database of verified software, and load the mapping from there.

ROM hacks will be distributed as BPS patches, that have the manifest.bml embedded in the patch. The emulator will apply the patch, pull the manifest out, and use that mapping (some games change mapping rules.)

Users can also distribute ZIP containers that have SFC+BML files, and it'll use that mapping.

Developers can have folders with game.sfc+game.bml files, and it'll use that mapping.

Behind the scenes, I make game folders. It's a folder named game.sfc/, and inside you have all the files related to it. program.rom, save.ram, bsnes/state-1.bst, cheats.bml, box/front.png, etc.

The emulator cores only load game folders. The frontend makes them before sending them to the core, completely transparently to the user.

For people who -really- agree with my vision, they can use game folders directly as well. They're my favorite approach. No compression, no merged files at all. Anything emulator-specific in its own subfolder. A folder is equivalent to a real game. When you move it, you move everything related to that game with you. There is no need for the emulator to have separate path configurations for every file type.

Well, regarding iNES, I think the problem is that with NES games you can't do away with headers because there are multiple ROMs in the cartridge (two, more specifically) and then you have a myriad of mappers and no header in the ROM at all to retrieve them. A header providing all this information is pretty much a must (or you could take the MAME or ZOMG approach and make a ZIP file with multiple files inside, with the filename telling what each file is meant to contain).

Also this discussion reminds me about when once on IRC we were discussing about Chinese bootleg Mega Drive games and how every one of them seem to include copy protection. Somebody joked about that at this rate we'd need to include headers in the ROMs (since sooner or later it'll become impossible to distinguish between two schemes just going by how the program behaves). The problem is that of course this information is never present in the ROMs (in fact most of the time they don't even have valid data, period). An even worse problem is that the schemes seem to be unique to each game most of the time, so we wouldn't be talking about specifying which mapper to use in the header, but outright storing how to emulate them if we ever pretend to keep sanity around.


But how far can it reach? Because some things get really tricky. For example, each assembler has its own way to do local variables, and it isn't just different symbols but outright different semantics (e.g. just a local named label, a fancy named label that is actually part of the global scope, forward/backward nameless labels, etc.). Also there's how strings are parsed (e.g. using " and ' for any string vs. using " for integers and ' for dc.b text). Then there are assembler-specific features (e.g. asm68k has the filesize function, as well as the concept of sections though I never figured out how they actually work...). Also operator precedence (e.g. asm68k for some stupid reason will put binary operations above arithmetic operations - I just use redundant parenthesis to keep myself sane >_>). Etc.

Though even then it may be still useful, where is this tool? Maybe I could just go make a table and use that.


Yeah, one thing about Sega's hardware is that it's a lot saner to program for in comparison to Nintendo stuff >_> That makes hardware abuse on the former much easier than on the latter.

Why wouldn't the same checksum approach work? Checksum the PRG+CHR file and use your database to get the mapper, CHR and PRG sizes, etc.

In fact, this concept of an application bundle is so good that Apple uses it for applications on Mac OS X. The problem then becomes one of how to change the default intent when such a folder is double-clicked to "Open this game in an emulator" instead of "Open this folder in the file manager".


WSZ (Winamp skins), SMZIP (StepMania songs), JAR (Java applications), and APK (Android applications) also use the pattern of a bunch of files in a Zip archive. And when unzipped into a new folder, this becomes byuu's application bundle concept.

Think of it this way: Ultimately, iNES is an archive format analogous to tar.


I read that the N64 was a pain to program too. But after that, Nintendo started designing things more sanely. The Game Boy Advance, for example, is wonderfully more orthogonal than the Super NES, even though it is missing priority per tile, subtractive blending, and hardware audio resampling and mixing.


Emulator authors would need to be notified of the checksum for each of the NES test ROMs that you have written, for one thing. A database of checksums is fine for the ROM images of the cartridges released during the Super NES's original commercial era, but not new productions in the homebrew era. Homebrew is why hacks might provide a new .bml, or why I eventually convinced the ZapFC guy to add an optional board descriptor file alongside the PRG and CHR files in fc.zip archives.

These issues apply to the SNES version of this too. Sik raised the objection that this wouldn't work on the NES because of multiple ROMs, and my point was that multiple merged ROMs in the file don't make it any less feasible for the NES than for the SNES. The same problems occur as on the SNES, with cartridges outside the commercially-released set from the console's life needing additional files with mapper information.

His point stands though, you're hardcoding the data into the emulators and whenever a new ROM comes out (could be homebrew or just a new dump, for all we know - there are still rare revisions of games out there that didn't get dumped) you need to update the emulator. Besides we're talking about keeping track of a database the emulator has no reason to know about - it should be part of the dump itself, not the emulator.

As for NES vs. SNES, with the latter you can still make a game that's just the ROM and that's it, with the former you have no option even for the simplest games, so the concept of raw binaries can't be used at all in that case regardless of situation.

EDIT: which reminds me, NES dumps not only are two ROMs, but also the direction in which the tilemap is mirrored. Fun stuff.

Splitting the files isn't nearly enough. It's the mapping info you need.

What kind of mapping chips are in there? (74LS chips? MMC-1?) How are the pins wired up? (mirroring? VRC-4 pinout?)

With iNES, the mapper #s are arbitrary enough (sometimes one mapper describes 2+ board types; sometimes the same board type has 3+ mapper IDs) that you basically have a database of boards already. May as well drop the rest, go for a 16-bit ID, and describe every variation at that point.



Basically yeah. A ZIP file to me is just a compressed folder.

The problem with the MAME approach is that the filenames don't describe intent, they describe what was written on the IC. Useful for the three people on the planet replacing bad ROM chips on NES/SNES carts, not useful for anyone else.

Here is Contra 3 for the NSS:

contra3.ic2
contra3.ic3
contra3.ic8
dsp1data.bin
nss-ic14.02
nss-c.dat
spc700.rom

How do you make a standard SFC ROM that plays on most emulators? Why you combine contra3.ic3 + contra3.ic2 in that order, of course.
Why is the DSP-1 data ROM and SPC700 IPLROM included in the ZIP, despite neither being present on this board? Good question!

The idea of a ZIP file representing a game folder is a good one. Then the emulator can extract it and use it. Once extracted to a folder, you can easily store save RAM files without rebuilding the ZIP file all the time, and without damaging the archival, pristine ZIP archive.

I just don't like MAME's handling. The ZIP includes irrelevant files, awful file names, and no mapping information. The mapping is embedded in the emulator in their own proprietary format under a non-GPL compatible license.



Yeah, I was following Haze's progress on that. SFC has a few too, but thankfully not as many. Genesis was easier to program for.

Personally, the way I'd handle that would be to include patch files that hack out the protections. But that's just me



Yeah, opcode mnemonics only. And even then, you'd have a really hard time doing something like the full amd64 instruction set.
The cool thing is you can use the assembler's macro system to inject new opcodes into the table, so you can build your table using recursive functional programming.



http://byuu.org/files/bass_v08r01.tar.xz



Exactly the comparison I use =)



That's why I made kaijuu =)
[with huge help from OV2, he wrote the initial app, then extended it greatly with icon support]

http://byuu.org/programming/kaijuu/

Works on Windows XP - Windows 8, 32-bit and 64-bit.

Once installed, you can double-click to play a folder that matches your filter (eg *.sfc) in bsnes/higan.
You can still right-click and explore the folder contents. Or you can reverse that, double-click to browse, right-click to play.

It also does many other cool things. You can set a rule to open Makefile in a text editor (Windows can't assign it because it has no extension), and right-click to run make all, make clean, etc. You can associate .htaccess files as well.

The argument passing is really advanced: you can pass in a filename, a folder name, the extension, multiple files and/or folders, etc.



Exactly! Once extracted, the core no longer has to deal with recompressing the save files, save states, etc.

ZIP is the conduit to get the folder out there, since web browsers lack the ability to download folders. And it's also good for archival.



Yes, but it gains a substantially more powerful processor that can be targeted with C compilers easily, and it has hardware sprite rotation.

It's really hard to draw a clear winner, but I still like SNES more. It felt like more love went into the games. GBA games always felt like half-assed ports and quick money grabs to me. Only a few real gems there. It had a much better homebrew scene though.



Yeah. The idea is we try and drive all commercial software with a database, so that they're immune to future changes in the markup format, and so that existing games can work without people having to redownload them.

Homebrew/ROM hacks/translations will continue to be produced, and we can't grow our database forever. Plus I really don't want to add games like SM Choukyoushi (lolicon rape dungeon "game" ... I wish I was making that up) to my database ...

Of course, what we really need is a finalized board format with 100% of commercial games working on it.
I think I have it now, finally, after half a decade of trying. But yeah, the last thing I want to do is say it's all done and finalized, then end up unhappy with a missing feature or a flaw in the markup syntax :/

But we can't strongly push ZIP { game file + board file } without the board file being finalized.



I agree in principle. If we could perfect a board format, and get it to be universal across emulators, that would be fantastic.
I just don't know how we can do that. Nobody ever agrees on markup board formats. I have lots of reservations over Marty's markup in Nestopia, for instance. Many people probably dislike mine.

Nobody's interested in board formats, and anytime I do things entirely myself, people seem especially opposed to it.
But yeah, I do have five years of experience working with SNES board layouts. So I am kind of the foremost expert there



Yes, mostly. You do end up with a half-dozen really nasty hacks, but it's possible.
See: http://gitorious.org/bsnes/bsnes/blobs/ ... amicom.hpp



Agreed.



Tilemap mirroring is part of the board wiring. Belongs in a board layout description, along with ROM/RAM size, mapper chips present, mapper chip wiring, etc.

I forgot to mention that I am against headers in ROMs even though I DO use copiers (GameDoctor 7) to test my code. But chopping the ROM image in 4 disks, copy them with my USB floppy and then go to the sitting room to load each floppy in the GD7 unit is a pain in the ass to be done each change I made. So I develop all code debugging in Geiger's SNES9x, then I test it all in bSNES and finally, I play the game on the real hardware using GD7.
When release candidate is ready, I put it on my homemade cartridge and play it again on the real SNES.

Wasn't there a new format for NES ROMs that instead of assigning IDs to each mapper it contained information about every single part of the wiring? Including what chips were included, how the ROM gets mapped and such.


You forgot that they have a tendency to make dumps incompatible about every version with the naming scheme =P

But you get the idea, ZOMG would be a better comparison. It's a new savestate format for Mega Drive emulators, where each piece of information (e.g. RAM, VRAM, CRAM, CPU statuses, etc.) is in its own file with a specific name, and the emulator then just figures out what each thing does based on the filename. It also makes expansion extremely easy as you can just add a new file if needed.

In our case the ROMs would be their own files, and the mapping information would just be another file (possibly a text file?).


And then you don't have pure dumps anymore. Luckily in just about every one of those games it's just a mapper, but what about coprocessors? (remember for how long the SVP remained unemulated)


Yeah, then it isn't very helpful in my case, the opcodes seems to be the only thing every 68000 assembler agrees on =/


Thanks :v


Even then the processors are the relatively easy part... The video and sound hardware are usually where Nintendo liked to screw up ^.^; (like how in the original Game Boy DMA won't stop the CPU yet it still wants the bus so the CPU has to run off a dedicated chunk of 128 bytes of RAM just to let the DMA run - oh, and DMA can only overwrite all of video memory, it can't just load small chunks of data)

With the Game Boy Advance it seemed like they finally got back on their senses, it's even easier to program than the Mega Drive and that's assuming you aren't using any help from devkits at all other than the hardware documentation.


Yes I know, WTF was Nintendo thinking on? They literally made the Famicom like an arcade machine, using CHR-ROM and hardwiring lots of stuff instead of providing some leeway to accomodate multiple games.

EDIT: forgot

I think the whole point of newer flashcarts is that you don't have to go through all that to test on real hardware =P

One mapper number describing multiple board types that are not functionally equivalent is exactly the problem that NES 2.0 was designed to fix. For example, among MMC1 boards, SOROM and SXROM have more RAM on the cart than the "normal" versions of those boards (SNROM and SUROM), so NES 2.0 explicitly specifies the amount of RAM on the PRG and CHR buses. And among VRC2/VRC4 variants, there's a "submapper" field that describes how the address lines are wired up.


And that's what NES 2.0 does: expand to a 16-bit ID. Four more bits are added to the mapper number for a total of 12, and four bits are dedicated to disambiguating incompatible variants of existing mappers.


Which is why the board file would describe intent, using "href" attributes or the like to link to the filenames.


Seeing as Nintendo was fresh off success with Donkey Kong, Donkey Kong Jr., and Mario Bros., I guess it sort of figures. Nintendo was looking for arcade quality graphics, and it saw that the TMS9918 in the ColecoVision was almost there, with nearly pixel-perfect Donkey Kong. So it set out to make something like the the TMS9918, except with scrolling (a feature useful for things that were becoming common in arcade games at the time) and various ways to keep down the cost of memory (CHR ROM and the attribute table). This is what led to the NES PPU. And yes, it was arcade-like, with a view toward Vs. System releases.

I suppose you're correct for a <=32K SNES ROM, where low versus high ROM doesn't matter. Larger, and the SNES emulator needs the mapper. Perhaps possible on the NES, if you had say a 24K file, which would be an 8K CHR and 16K PRG.


Yes, and I wasn't saying otherwise; I'd make the same point. I just didn't see it useful to inject general criticisms randomly when the focus was on a particular point. To me it reframes the discussion into a battle between things, scoring points for one or the other, instead of exploring something. When you're doing the latter, you're often focusing, not summing.

Since we're fully distracted from the focus, my big objection to a database is that it comes from a place of being all-knowing, and is very centralized. It seems the opposite of the emulation/preservation community, where work is decentralized, and new things are being discovered constantly. This to me calls for self-contained things that ask the cartridge what its features are, and don't presume to know of every one in existence. And yes, I know that the database approach generally has a fallback, but at that point you might as well just use the fallback for everything if it works. If the fallback is separate files, then it's cumbersome and you've made it a second-class approach, when it could be first-class (self-contained).

I guess one of the big draws of a database of commercial ROMs is legality. If you update the database, the updates are lawful to distribute, but if you update board definitions that are included with commercial ROMs, that's not.

I've been asked by 4 of my peers to continue responding in this thread, focusing specifically on the technical items listed here as justification for modifying all existing SNES ROM images to have their copier headers removed (or for split-images, be joined and have their headers removed). Three of my peers, for what it's worth, are against modification of the existing files; 1 is on the fence. I've also received a PM from a forum member (will remain anonymous) who agrees with me that this is purely a "personal crusade" of some sort, simultaneously agreeing with me that the "issues" can absolutely be addressed without modification of existing ROM files.

IPS patches
- Issue: emulator or front-end is unsure whether to apply patch to the entire image (including header) or just the non-header portion
- Removing the header from the file does not solve the problem -- IPS patch is still likely to fail to "do the right (desired) thing" 50% of the time
- IPS file format is at fault for not providing (bare minimum) MD5/SHA1 of the source file the IPS was meant to be applied to
- People using IPS patches need to provide the filename of the ROM and a MD5/SHA1 of the ROM the patch is intended to be used against
- Issue is not the responsibility of the emulator/library or front-end author -- cannot reliably (100% of the time) be solved with software
- Conclusion: not valid justification for modifying ROM / removing header from file
- Alternate conclusion: emulator or front-end can ignore/skip the header and apply the IPS patch to the rest of the ROM -- effectively same thing as modifying ROM except no modification needs to be done

Third-party tool header support inconsistencies
- Issue: some tools found online are inconsistent with their header support. Some work only with headers, others work only without headers
- Removal of header from file would cause some existing tools not to work. Likewise, for some other tools, keeping the header would cause some existing tools not to work
- Cannot reliably be solved (i.e. some tools are not going to work no matter if header removal is done or not)
- Issue is not the responsibility of the emulator/library or front-end author
- Conclusion: not valid justification for modifying ROM / removing header from file -- issue is irrelevant to topic

Third-party tool behavioural inconsistencies
- Issue: some tools found online attempt to parse headers and reject playback of the ROM if header found to be invalid (criteria unknown/unspecified)
- Issue: some tools found online attempt to add headers themselves
- Irrelevant to issue -- has nothing to do with bsnes or any other emulator or library, nor front-ends to those emulators/libraries
- Issue is not the responsibility of the emulator/library or front-end author
- Conclusion: not valid justification for modifying ROM / removing header from file -- issue is irrelevant to topic

Third-party ROM verifiers (ex. GoodSNES) and complexities
- Issue: some tools used to verify authenticity of a ROM are inconsistent
- Not enough precise detail to reach conclusion -- quote seems to imply that if there was only one "type" of ROM (i.e. headerless), this would somehow make ROM verifiers better or decent or good in some way
- Issue is not the responsibility of the emulator/library or front-end author
- Conclusion: not valid justification for modifying ROM / removing header from file

Online groups creating their own emulator-specific headers
- Issue: some online cliques or groups, quote, "subvert the copier header by creating ZSNES-specific emulator headers"
- Not enough precise detail to reach conclusion -- have not seen this myself -- need more technical specifics please
- Conclusion at this time (may change): not valid justification for modifying ROM / removing header from file

Online ROM distribution groups being inconsistent with which header format they use
- Issue: some online cliques or groups distribute ROM images that, quote, "have copier headers in 20+ differing and incompatible formats"
- If the ROM distribution group didn't exist, the same problem would already be happening, re: FIG vs. SMC vs. SWC vs. GD3 vs. GD7 vs. UFO vs. [0-9][0-9][0-9] (split-image SMC) vs. lots of other formats
- ROM distribution groups, much like romhackers, operate independently of emulators/libraries and front-ends
- Conclusion at this time: not valid justification for modifying ROM / removing header from file -- issue is mostly irrelevant to topic

End-users with limited skill sets
- Issue: many end-users / game players do not know how to properly remove a header (lack of technical knowledge/familiarity with formats)
- Applies only if the individual is attempting to use a tool or an emulator/library which mandates "no headers"
- Conclusion: not valid justification for modifying ROM / removing header from file -- if anything this argument supports not modifying the ROMs (i.e. end-users will mess this process up and make things worse)

SNES copier and file format support
- Issue: some newer SNES copiers support a limited number of file formats, or alternately, too many formats
- Issue: same copiers expect, quote, "interleaved images" -- need more technical specifics on this please; as such I am assuming this means "split image SMC", as in split across multiple 1MByte files
- Conclusion: none at this time; need more specifics as stated

Emulator complexities; ROM/game identification nuances
- Issue: very difficult for an emulator to determine the type of game or features it has
- Issue: checksums (CRC32, SHA256SUM, MD5SUM, etc.) not reliable way to distinguish
- This issue would exist whether copier headers existed in the ROM or not
- Conclusion: not valid justification for modifying ROM / removing header from file

Emulator complexities; ROM (not copier!) header identification nuances
- Issue: emulators/libraries/etc. have to, quote, "do odd math to try and detect internal headers ... not just at $7fc0, $ffc0, $40ffc0; but also at $81c0, $101c0, $4101c0"
- Issue: quote: "you can't even do filesize&0x7fff==512 due to older homebrew that doesn't pad the ROMs out"
- 1st problem described is personal; coder/programmer "does not like doing odd math" effectively. I touched base on this already in my previous posts. Rephrased: the "odd math" annoys the programmers, "makes his code messy". I also offered proper workarounds for this (focus on emulation and stop focusing on the front-end / file parsing bits)
- 2nd problem described is not the fault of the emulator author/etc. -- there are ways to solve this problem other than using said formula, but they may result in the reason for the 1st problem ("makes code messy")
- Recommendation for 2nd problem: don't bother supporting this junk -- yes, that includes my own demo with my old SNES docs! You can absolutely 100% blame me for not padding the ROM and I'll take full responsibility for that. However, give us some credit -- those of us who wrote demos on the console did so back in 1991-1996, where copiers were the only thing we had and disk space mattered more. I ask kindly that folks retain context (time period, etc.) when bitching!
- Conclusion: not valid justification for modifying ROM / removing header from file -- issues are either personal to the programmer/coder (i.e. this is just how it goes, welcome to programming), or are related to "non-compliant" software (demos not padded, etc.) which is not the fault of the emulator author

Copier headers cause annoyances for romhackers
- Issue: romhackers must subtract 0x200 (512 bytes) from their file offsets when working with headered ROM images
- Social problem; i.e. "this hurts me to have to do"
- Workaround: romhackers can remove the header from their images before working on them, to ease their pain, then re-apply header before creating + releasing IPS patch. This is what I do with my own romhacks (i.e. FF2j/e for Neo Demiforce (NES/Famicom however), and private work I've done on Otogirisou (SFC), etc.), so if you're expecting sympathy.... :-)
- Conclusion: partial validation for modifying ROM / removing header -- I will accept this as a valid point but it doesn't hold much ground (with me anyway -- YMMV)

If anyone has other reasons/justifications for removal of copier headers, please provide them, as I will be happy to analyse each one individually. But as you can see, some of the points above are refuted, and in others I see some legitimacy.

Now for the slightly more personal part...

As such, I still have yet to see justification for requiring everyone to modify their ROMs (copier header removal) rather than just ignoring them in software (emulator/library or front-end). As of this writing, I still see this as byuu's personal crusade, and that's justified by his own words. Quoting:



So it's not about the technical benefits at all (despite saying it's about "technical merits" and "logical options" -- there's nothing logical, even remotely, about asking every person with a SNES ROM image to modify it!). It's about philosophy, it's about opinion -- fuelled by one man. It doesn't matter if it's shared by 5 or even 500 people -- quantity does not justify demanding everyone change something that can be dealt with in software. How can a person say it's not a crusade given the above?!

I'm perfectly fine with change -- I advocate changing things all the time (go look at the FreeBSD mailing lists sometime). So don't try and troll me with a bunch of rhetoric and demand I "refute the arguments" when I'm not the one proposing everyone who has a SNES ROM modify it for philosophical reasons. And for what it's worth, I was a poor teenager back when I bought a copier as well (this is back when they were going for US$500-600); so what?! I'm not "clinging to an old format" because I like it, I'm stating there's no reason to demand the world modify a billion files (filecount*person) when it doesn't have to be that way.

This is what I think about headers and this argument that makes no sense, I don't care about what anyone thinks:

They are useless, and more, they aren't part of the ROM in the cartridge. Why should it be there?
And better yet: Why so many formats? Emulators aren't emulating copiers as far as I know.
There should only be a header if the emulator finds a good use for it. iNES is an exemple. Even iSNS seems more useful than a copier header in an emulator.


I will now quote in no particular order and just say what's wrong to me.


I don't care if I'm bothering someone by stating a fact:
We are not in the 90s anymore.


Just this means to me one thing: Why no one thinks about a standard ROM format that everyone should use? It CAN be a justification to change everyone's ROMs.


Sure, he doesn't feel like it, so why arguing? Especially when YOU make it personal to yourself, as seen here, in this FIRST POST:

Oh, you don't feel like it? That means personal to me. In that case, deal with it. Add/Remove the header yourself. Or just don't use bsnes/higan. And more: byuu still made purify that can run ROMs, headered or not.

Just from this, I'm just saying this argument has no end because of personal stuff. And lazy people.
Or this is a pretty bad joke. In that case, continue. I laughed many times.

Headers are vestiges of the past. And they aren't even accurate. Plus there isn't even *one* standard header. Every copier had his own. So, which one would be the "real" header? SWC ? Pro Fighter's ? Doctor's ?
ROM files sent to Nintendo for approval don't have headers. ROMs read using an eprom reader don't have headers (and I don't see any more accurate way to extract a game). ROMs extracted using a Retrode don't have a header. sd2snes doesn't use the header. etc...
So why keep using them ? If byuu doesn't want his emu to support header roms, that's his choice. Geiger's snes9x did the same thing, as Byuu mentionned. NoIntro team releases headerless roms. Most roms sites no longer uses header roms.

They are really useless, and I don't see any good reason to support them. If older softwares don't support them, there is always a workaround.

(I'm glad the 640k limit for PC has been removed )

But they did come with a page describing the board the exact same way NES headers are made to do. I hate to say it, but XML is not the future, headered ROMS are here to stay. I won't use an emulator that doesn't support them, simple. What good is an emulator if it can't play everything you throw at it? Audacious plays NSF, SPC, Midi, MOD, etc files. They don't complain it's not a .flac file, they just support it. Not saying most formats don't suck (PasoFAMI, whatever else) but the community has to come down and say THIS is what we're supporting as the main format. Period.

@LuigiBlood:

The copier headers are useless to someone who doesn't use a copier, but that's not the point. The point is these headers can be safely ignored by software (emulators, etc.). This thread is not discussing whether or not the copier headers are viable -- it's discussing whether or not there is technical justification behind removing them (read: every person with a ROM image having to modify it). Why the need to modify something if the underlying software ignores what it doesn't care about? Not rocket science.

You're right -- we're not in the 90s. I'm sorry that I can't go back in time and change all the demos that were done by all the demo groups, properly padding their size. I'm not asking people to do that either (you intentionally removed the most relevant part of my quote) -- I asked that people at least try to remember the era when these were made / what we had to work with then. It doesn't mean our choices were right (far from it!), but having some historical context + education at least allows a person to say "ah I see, yeah, I understand now".

Regarding "a standard ROM format" -- xkcd covers this quite well.

Regarding "why I'm arguing" -- because the point I was given was that there are technical reasons that justify everyone modifying their ROM images to remove copier headers. However so far that doesn't appear to be the case. All that's been shown far is what I already said: it's an opinion. A "philosophy". I'm all for someone having an opinion/philosophy, but demanding that I comply with that opinion/rule/whatever is unreasonable, especially when the justification for it is presented as "technical" (again, nothing presented so far indicates it's technical). So you're bitching at the wrong guy. :P

Also, you quoting me ("I would except there was this emulator that demanded...") is silly -- that was partially serious, partially a joke. I said this for two reasons, and I already explained what those were. Please go back and read my explanation.

I fully agree with you on this point:


...because it's exactly what I said from the beginning. It's purely an opinion, a personal choice. Yet I see crap like this (quote: "There is NO technical reason that currently justifies the use of headers from an emulation") and it's utter nonsense. If people would just say "you're right, it's totally about opinion and it's personal", and stop trying to claim there's technical justification behind their removal, I'd be quite content.

You're right they don't -- and do you know how that information is provided to Nintendo? I do: on paper. Do you want proof of my statement? I'm happy to show you the pages of the official SNES developers manual ("SNES Software Submission Requirements") which contain 2 pages of fields/data one has to fill out when submitting a ROM to Nintendo for review. Would you like to see it or will you just trust me?


Good fucking lord people! The topic at hand is whether or not ***REMOVING THE HEADERS FROM THE ROM ITSELF (MODIFYING THE FILE)*** is justified, or if they can just be ignored by software. This isn't about "why keep using them?" or other philosophical/opinion-based shit. I was told there are technical reasons why they should be removed from the file (vs. just ignored) and I have yet to see those reasons. This is not difficult to understand -- I'm sure all of us here are technical and intelligent and can understand this.

Programs can ignore those headers -- that's completely 100% legitimate and I couldn't care less if a program does or doesn't do that. What I'm against is requiring people to mass-modify all their existing ROM images to comply with an opinion, specifically when there's no technical reason the headers can't just be ignored/skipped over.

Capicé?

Let me try to provide a few "more technical specifics" per koitsu's request:


It's not only emulators but also solid-state copiers (e.g. SNES PowerPak, Super EverDrive) that have to seek into files to detect internal headers. You can see this in the current PowerPak firmware: if you run a game that has a copier header, it has to load the game to determine whether or not a header is present, and then it has to load the game again. Because these newer copiers rely on the Super NES's 2.7-3.6 MHz CPU, which is a thousand times slower than a PC CPU, this increase in loading time becomes substantial. Do I need to provide a video of loading a large HiROM with no header, a large HiROM with header, a large LoROM with no header, and a large LoROM with header?

To anyone arguing that an emulator shouldn't support any representation variations, should only support the latest and best, I simply point you to your web browser or image viewer.

I am glad you are willing to change, but the issue is why is byuu NOT allowed to make HIS emulator do whatever the hell he wants it to? Nobody is forcing you to use his emulator. If people don't want their ROMs modified, then here is a clue... DON'T use a program that modifies your ROMs.



It is called courtesy.
byuu said, "I am genuinely interested in hearing your arguments about why copier headers should be supported". You responded by ignoring him and attacking his personal philosophical stance and implying it is wrong for him to implement what he wants in his emulator. Nobody is forced to use it and if they don't like something about it they are free to change it. If you didn't notice his emulator is OPEN source.



Poor, yet you brought a copier when is was going for $500-600?! Only a US American can say such a foolish comment and actually mean it. For regular poor people, spending that kind of money means no food AND place to live for at least two months. As Inigo Montoya would say, "I do not think [poor] means what you think it means."



I am glad you did reply and clarify that it is the modification of ROMs, that the user wishes to keep headers on, that bothers you and not whether ROMs should have headers or not. Kudos on responding and helping us understand what your issue was. Unfortunately, I think it means we can never come to an agreement. I think byuu is entitled to make his emulator run however he wants as its default. If somebody wants to change something then they have to get off their derriere and actually write the code themselves.



Web browsers are the perfect example of what a cluster f**k a programs code can become when trying to support all the possible web pages out there. I like the new trend towards following standards that has begun to emerge after all these years of web browsers doing whatever they wanted.

Wouldn't the thing to do be to have people that distribute ROMs update what they are distributing so that eventually most people have ROMs without various useless headers? Most ROMs have a SMC or SWC header which contains no useful information really. Except for the copier. And if you are using a copier (I have several) you can't trust the file extension anyway and you need to have a program like UCON64 convert the ROM for your copier anyway.

Maybe someone does need to go and take every homebrew/demo rom that has a header or odd ROM size and cut the header and pad the ROM. You are right that it's crazy to ask *everyone* to do that for every ROM but if just someone does it and passes it along, eventually the problem is gone.

I think it's a reasonable suggestion to try to get rid of the copier headers on images just because they are useless to emulators and flash carts and copiers as well because you can't trust a file actually has a *valid* header for your particular copier either.

That's why Byuu's approach would be the most accurate one (xml file for mapping information).
The analogy with web browser is accurate : why keeping an old, inaccurate and deprecated thing, useful only to 0.0001 % of rom users (and no, I'm not exaggerating).



Only one word : accuracy.
How do you know if the header is valid or not ? And what about games that can't be dumped using a copier ? Do you add header ?
Byuu's work has always be for more accuracy, to be closer to the "real hardware". He doesn't force people to use his tools. But sometimes, to make people forget their old habits, you have to force them. And he even offers utilities to keep headers...

Headers are useless. Can they be ignored by software ? Sure, they can. So can be an overdumped ROM. Or any unuseful data at the end of the rom.
Majority of people doesn't need them. So why keep them ?

Anyway, all this is taking us nowhere. NoIntro people are redumping roms, sometimes without even using copiers. So, in a few years, most of, if not all, the roms will be dumped by anything but copier.

I wrote up a very lengthy reply to each point here: http://pastebin.com/y9RcR0J7

But really, there is no point in having a discussion when you have clearly made up your mind. You reject every problem I point out as having a workaround. That does not dismiss that it's still a problem. We can live in a world where we do not need workarounds. You claim that because I'm an emulator author I have no say in the file format my emulator uses. That's complete nonsense.

I'll summarize things, since we're at a standstill. Here is the bottom line, for the fence sitters:

Less than 1% of people own copiers that require headers to work. Of those people, distributed ROMs (which make up 99.9% of how ROMs are obtained) are already going to have to be modified to run on their specific copiers. And of those modified ROMs, at least 30% will not work on both their emulator and copier with the same image. And if they use a floppy-based copier, it's closer to 70%. And even if they copied every SNES game in every region for every revision, they'd only eat up five extra cents worth of disk storage space. Although this duplication is trivial to avoid with a simple batch file. But the truth is, copiers are obsolete. Computers don't even have floppy drives or parallel ports anymore. Most stores don't even sell floppy disks anymore. In ten years, almost nobody will be using them at all. $69 or less will get you a flash cart that is vastly superior in every possible way.

There is the pragmatic approach of supporting copier headers because they are out there. This approach means we will have to support them forever. Even though they have not been for sale in decades, and will eventually be impossible to obtain. This approach continues the problems with patching, with hashing, with verification, with conflicting tools, and everything else I've said. Yes, there are workarounds. And they are annoying, and they are unnecessary. This approach has, does, and will continue to cause small amounts of pain points for everyone. Whether you acknowledge or trivialize it or not. Patching is still a problem in 2012.

blargg's point on browsers was an interesting one. Sure, they support JPEG, GIF and PNG. But do they still support PCX, TIFF and BMP? Should they support PCX until the heat death of the universe? What about all those people with Allegro graphics files?

There is the idealist approach of standardizing on exactly one format. This approach will cause a lot of pain for everyone, yes, this is true. But it's a one-time cost. We do it like taking off a band-aid, and it's done forever.

I truly do understand your pragmatic appeal, and I've done everything possible to ease the transition. I wrote a two-click, ten second batch convertor for all your ROMs. I wrote a launcher that forces header-only tools to also work with headerless ROMs. I've made a superior patching format to fix IPS' problems. I've spent dozens of hours writing posts like this to educate people. And so on.

And there is the compromose approach of slowly pulling off the band-aid. Allow emulators to load headered ROMs, but offer to remove them, then require they be removed, convert IPS patches, try and get RHDN to update their archives to all be based on headerless images, and so forth. This makes it a little less painful, but it'll take many times as long to finish, and we'll have to deal with threads like this a lot more. This is likely how things are going to end up, because we can't get everyone to agree.

Which is exactly the point of all this. Nobody's ever going to agree on whether or not to use or require headers or no headers. The pain points will continue forever until people finally say enough is enough and put an end to it.

The NES scene may be hopelessly stuck with iNES 1.0 forever, but I will spend the rest of my life trying to improve the SNES scene. I am against headers, and not only do I advocate that, I write the software and tools to make it happen. Perhaps you should do more than post tirades on a BBS if you want to make your vision a reality.

...

So, here's the bottom line. Support is growing every day. I have 40,000+ users who have all removed their copier headers already. No-Intro and Zapatabase have abandoned them completely. 80% of the ROMs distributed on the internet today have no headers at all. Every emulator and every known tool (thanks to my launcher) can currently work without headers.

Whether ROM hackers, patch makers and tool authors like it or not, SNES games are more likely to not have copier headers today. And that trend is only accelerating. If they choose to enforce them, they will be the ones causing the most inconvenience to their userbase.

There will always be those who will fight tooth and nail to the very end. But rest assured, unless you start taking these arguments seriously, you're going to lose this battle. In many ways, you already have.

...

And you know what? Even if I fail, so what? At least I tried, and advocated for something I believed in.

The people who give a shit about the issues I've brought up agreed and moved over. If people really want to cling to headers for their dear lives despite how illogical it is? So fucking what. Let them, I don't really care. They get what they deserve with all the problems they bring.

I am amazed you took the time to do that, but you need to get back to coding. I need my Far East of Eden Zero fix and you are the only dealer in town -j/k

I appreciate the lengthy response and the time you spent writing it, byuu. I especially appreciated the part about the UFO format for mode21/hirom images -- I had absolutely no idea they were interleaved to that degree. The UFO copier was one of the harder-to-find units, by which time most of us back in the 90s had already saved up or gotten deals on SWC units (I had to do both -- I got mine from one of the guys at Anthrax in exchange for some code, but I also had to haul wood and hay for a summer to afford it even after the discount). I did have the "pleasure" of using a MGH (Multi Game Hunter) unit a friend of mine had -- you know, the dual Genesis/SNES copier -- which had its own file formats too (can't remember which ones it used). That thing actually fried (as in burnt up) after about a year.

I am in full agreement that you and I will probably never agree on this topic. I'm fine with that, as I'm sure you are.

You're absolutely correct that it's your emulator, your code, you can do what you want, etc. -- my point this entire time has been that there is nothing concrete that from a pure technical standpoint removing the copier headers solves a problem (sans what tepples just mentioned -- yup, in full admission that one is 100% justified!). It may make things a lot easier (keep reading please), but what I was told was that there were technical facts that justified modifying the ROM images vs. skipping/ignoring the headers via software (front-end, emulator, etc).

What you've listed off are lots of excellent points justifying your opinion/philosophy on the subject, based on experience as an emulator author, as to why you feel they should be removed -- in fact, it might surprise you to know I sympathise with many of them and can understand how they cause complications for you, or annoy you. I get it, I really do! But those are opinions/philosophies/personal reasons, not something that says to me as an engineer "there's no other way to deal with this, we have to remove the headers from all the ROMs. It's the only choice for everyone". There's a huge difference between that and "well, there might be some annoyance, but supporting all, or at least some, of the existing formats means we don't have to ask everyone on the planet to change something that's been out there for 20 years".

I hope we can at least come to terms in the sense that we see the standpoint one another takes, even if we don't agree.

One final point (or answer to your question I guess) -- you asked who the peers were whom share my opinion that the ROMs should just be left alone (with copier headers) as you'd like to talk to them. I reached out to all of them this afternoon. Of the off-forum four (4), two (2) of those stated very rudely that they don't want to talk to you as they have tried communicating with you in the past (about other things emulation-oriented) -- and as one put it "you shit down his neck". One of the 2 presented a single-paragraph viewpoint of you which one of the others said represented his opinion as well, and the former was quite happy to have me post his opinion on his behalf. So here it is:

Byuu is just a self-important spergelord. He's a talented coder, he wrote C++ code that didn't make me want to commit self-harm when I read it. But I wouldn't concern myself with his fits of whimsy. I imagine this is him holding breath and whining over header files in ROMs and him threatening to go home unless people undertake effort for his sake.

All I got from the 3rd: "Haha. No." What's interesting is that this individual actually uses one of your older emulation libraries in a private project of his own, so I'm not sure what the history is there, nor am I going to ask.

I can't imagine you would want to sit down and waste your time talking to people who feel this strongly about you. I know I wouldn't, at least not when the conversation would absolutely be destructive vs. constructive, you know?

The one who's on the fence does not care to involve himself in this matter at all. Quoting him: "arent you and him [byuu] done beating a dead horse already? it doesnt matter how i see headers if i did id post on that bbs").

As for the anonymous forum PM person, I'm sorry but that's going to remain that way -- not only because I'm a site admin, but also because that person has already been through enough drama in their life and I'm doing it out of respect. I hope you can understand that.

You don't need a say. It's the front-end's job to make sense of headers, which may involve stripping them to a temporary folder before launching the emulator. If a PC based front-end wants to support the bsnes engine and ROMs with copier headers, it's just four reads for the common cases.


PCX I'm not sure about, but some browsers support a subset of TIFF, perhaps through a plug-in, because USPTO.gov demands it. Others support BMP because IE supports BMP, and BMP isn't really that much more code, especially compared to libpng. In fact, indexed BMP might even be as efficient over the wire as GIF due to HTTP Content-encoding: gzip. This analogy might work if you're familiar with the old compress tool that was popular on UNIX before Unisys started flexing its patents: .bmp.gz is to .gif as .gz is to .Z. As far as the web is concerned, the big gain of PNG over BMP is support for scanline differencing and support for alpha from day one (apart from old IE).


The Allegro library can save_bitmap() in the BMP format, and browsers can read BMP.


Or at least until someone figures out how to push a firmware update for NES 2.0 support on the PowerPak. There's 8 KiB of repeated CHR data in the PowerPak firmware.


By what factor does this exceed the number of people still using their floppy or parallel copiers?

Not at all what I expected to hear in response to that, but I'll take it! :D



For me it was the only one. If you'll believe it, I have never been able to get an SWC DX2. Even with all my connections, nobody would ever sell me one. eBay of course deletes copier auctions. Mind, I don't want one anymore, of course :P
The sd2snes is the new Cadillac of the SNES scene. And I have one made by ikari himself.

So my only choice was to pay $250 total to get one imported from Japan through a deputy. A long time later, I also bought a GDSF7 from Tomy, but it broke right away (not his fault, it worked when I received it, they're just shitty products.)



We disagree on what's a problem.

I personally find it to be a huge problem that people can't apply IPS patches directly. They have to know whether the patch was made with a copier header, and whether their game has a copier header, and they have to be able to add and remove them.

That, to me, is a technical problem. It's a problem we can solve. If we disallow headers, people will make patches on ROMs without headers going forward. Problem solved.

I've released multiple game translations. I have dealt with this issue personally. I have been in the translation scene for a long time, and seen many people deal with this. And it's all so silly. Nobody uses copiers anymore. These headers serve no purpose.

What I will agree with you on is that it is going to cause headaches for a bit. But only for a bit. Everyone who has used bsnes since 02-2011 has had a bit of trouble, gotten over it, and all is good now. Some guys on my forum even converted every game translation patch to BPS and posted a ZIP of them all.

You say it's fine for ROM hackers to keep adding and removing headers to work around flawed tools like Lunar Magic. Why is it then not okay for you to add and remove headers for your copier usage? Do you at least acknowledge you are in the extreme minority (copier users) here?



I have admitted many times that you are right, small amounts of code can get copier headers to work. And I've also said many times that others are free to add this support to my emulator if they want it. One guy on aep-emu.de posts a build with ZIP support each time. I've only ever thanked him for his trouble.

My point has always been: I dislike headers, I think they are absurd in 2012. Clearly, as evidenced by this thread, many people agree with me. I have a right to speak my mind, and I do. I control my software, and it's a reflection of my values.

If I were trying to force you to remove headers, then I would have a closed source emulator, and I would have coerced the Snes9X devs to lose their copier support, eg "remove them or you can't use the APU I wrote for you." That's disgusting and I would never, ever do that.

But if you want to use -my- software, remove the headers or add support for them yourself. You're even free to share your modified build with everyone. That nobody has done this tends to show you how little most people value headers.



As a fellow technical person, I don't see how you can feel this way. What does age have to do with it? The whole point of computing is advancement. Getting better and better. We used to use BASIC 20 years ago, and now we have better languages. Things evolve, people change. That's how technology works.

An argument for keeping copier headers is an argument for keeping IE6 support around forever.

This is not change for the sake of change. I showed you 19 different things that would be improved if they were gone. As minor or trivial as you take them to be, they're big deals for myself and many others.



I do understand your side. You are pragmatic. You think that if software can work around an issue, it's better than inconveniencing the user.

I am idealistic. I think that if something can be done better, we should do it the better way.

There's nothing inherently evil or wrong about either approach. Just as there's nothing wrong with my focus on accuracy versus ZSNES' focus on speed.



That sounds anatomically impossible, but impressive nonetheless.



That's exactly my point.

I hope you realize how childish it is to bring up anonymous comments in a discussion. There's no proof that what you are saying is true. What if I told you that someone PM'ed me and told me you "shot (his) grandmother and raped (his) dog?" What if I told you that seventeen people I spoke with on my IRC channel told me you were a horrible monster who eats babies? Even if it were true, what does it have to do with our conversation other than trying (poorly) to piss you off?

I don't doubt your honesty, everyone I've banned from my forum has formed a personal vendetta against me. I can name a dozen people who hate my guts. And I couldn't care less. If they are too afraid to state their names, then don't bring them up.

Either they can speak for themselves, or they can stay cowards who insult people in private. In which case, it's no wonder we don't get along. I despise cowards.



Worst case, 100:1. Best case, 1000:1.

The solution is for the IPS to have the byte offset of the ROM and not just the file with whatever else is in it. Then you can have the tool decide where to patch it and it would do it right. The problem is in the IPS system, not the file format. The format shouldn't matter and is easily manipulated if the IPS had better information.

Some IPS patches actually try modifying the copier header to work on the author's copier type. A good example of this was Final Fantasy V.

But yes, if emulators made IPS treat offsets as only applying to the ROM content, that would solve the problem with IPS patches. After a bit of pain. Everyone would have to recode all the old IPS patches that relied on headers, and everyone who downloaded them would have to update their patch collections. But it would be a one-time expense to solve the problem. We could make tools to recode existing patches in two clicks, we could have emulators use heuristics to try and minimize disruptions. We could update RHDN's entire repository at once. If people were willing to attack the problem head-on.

BPS makes it even easier. You could apply the patch assuming no header. If the checksum fails, try again with a header. If it fails again, you say they have the wrong ROM. And even better, both bsnes and Snes9X enforce BPS application with no header, so no patches will be made with headers if people want them to be soft-patchable. BPS patches are guaranteed to either work, or tell you that you are using the wrong ROM.

If you took this one-time pain point, you would eliminate IPS patches from my list of objections to copier headers. You'd still have 18 more arguments to go, but yes. The most important point would be eliminated.

I guarantee you that if people are fighting me over copier headers, they will also fight me over a rule saying that IPS should only assume ROM offsets. Why? Because it breaks 20 years of history involving IPS patches errantly made against headered ROMs. People will have to change their existing files.

Apparently, change is a really big problem when it's not absolutely necessary.

BTW, beat has been added to RHDN. If you want anything corrected, just let me know.

It'd be easier to fix the old IPS patches. Keep them the same just with a new format. Label them old. Then put a flag that says it just works on the raw file with no ROM separation. Then run them through a single converter. As long as you have every file in the world anyone would need, conversion is cake.

I'm on your side here, let's fix the old IPS patches. Not as nice as using a format with a built-in checksum, but I'll take it as a positive step.

But I'm telling you, there will be just as many people upset about removing copier offsets from IPS patches as there are from people upset about removing copier headers from ROMs.

Both require modifications to existing files. That's the whole crux of koitsu's arguments, "why change files that can work now and have been around for a while? People already have headered ROM+headered IPS and unheadered ROM+unheadered IPS pairs for all their files and they currently work with existing emulators." In fact I'm curious: koitsu, what's your take on recoding IPS patches?

So by all means, please go forth and convince people to recode IPS patches based on headered ROMs. You have my full support and gratitude if you do.

I mean I'm not saying I should do anything. Like I said, there may not be one stanard. But we, as a community who are dedicated to building these ROMS and maintaining their formats and such, we just need to get together and say this is how it will be done. Then I'd say that's when we need to go on the final crusade and piss everyone off, hehe. I myself believe that we need headered ROMS. Even if people don't like them, as long as it DOES work with everything and has a good quality format, you can always tell them just to split the ROM and CRC/MD5/whatever they desire from there. Just my $0.02 on this subject.

Just to be clear, you're aware of the difference between iNES and SNES copier headers, right?

iNES tells emulators how to play NES games. Emulators can't play the games without them (or a database.)
SNES copier headers do not. Emulators completely ignore them.

The reason SNES copier headers exist is so that copiers can read the first sector of the file and determine where to place all subsequent sectors. This shaved off less than one second of seek/read time to read the internal SNES header at $7fc0/ffc0 first. These headers are nothing more than poor duplications of the internal header. So it'd be like if NES ROMs already had iNES footers, and we decided to add another one so that it was at the top.

I wasn't 100% sure, but I understand that. I didn't know there was a huge difference but eh.

The iSNS thread was a joke about rushing through a universal standard without fully understanding the problem domain first.
(UPS was a personal example of my own similar failure, in fact.)

When I listed the ROM and RAM multipliers as increments larger than the smallest examples, it was a jab at the issue with Galaxian.
Separating the RAM size into two fields mirrored the issue with "16 mappers should be enough for everybody."
Leaving dead space was just asking for "DiskDude!" and other people to continue extending the spec.

But to be fair, although I am against including mapping information in with ROM data directly (SNES mapping is -really- nuanced, it's unlikely anyone would agree on a format to use); I would prefer it over having no precise mapping information at all. That also won't happen because it requires changing existing ROMs.

So NES is stuck with (inadequate) headers because we can't change files, and SNES is stuck with (useless) headers because we can't change files, and the GBA is stuck with no headers because we can't change files.

Doesn't it get tiresome that we can never make things better for fear of change?

Ironically, we might be able to subvert copier headers to add mapping info. But that too will probably get people complaining.

I learned a long time ago that nothing you do will go without criticism. Sometimes you just have to ignore people.

I'm surprised nobody has mentioned yet that IPS has several variants that are incompatible with each other, and not always possible to distinguish other than for producing erroneous output. This just makes things even worse and even more of a reason to drop that format =P


Actually, 4-bit and 8-bit BMPs are compressed. I'm not making this up, BMP does indeed support compression (some form of RLE).

I'm just offering my bigger take on things because it's an interesting exercise and I think it's the only way to shed light on the topic. I encourage anyone not interested to just skip the rest.

I'm also highly, HIGHLY amused by how such an insignificant-seeming thing, 512 essentially padding bytes (no useful information for the main case we're talking about, just a nuisance to skip) at the beginning of the file, are able to weave a web between so many things at so many levels. I love it.


The issue isn't whether we can snap our fingers and suddenly have no headers on any SNES ROMs on everyone's machines, all IPS patches everywhere made for headerless ROMs, all emulators and tools supporting headerless ROMs, and everyone's play-on-copier flow made to add headers appropriately. If we could, I think most people would agree that this power would allow us to fix countless problems with file formats, bad dumps, etc.

The argument here is about what we can do, and which of those things are respectful of users' unique setups and the larger ecosystem. It goes beyond technicalities to the intent of people choosing particular ways of handling things. If the intent is to get people to do something that one considers superior, end of discussion, then it won't be received well. This always shows up in the product; it has to, as the technical approach is chosen to implement this intent. This kind of thinking lacks humility and necessarily disregards that reality might differ or others might have different ideas and like to explore them. Yes, one isn't forced to use a piece of software, but there is still an element of control in being given the choice to be constrained or use something else.

Unless I'm misunderstanding, the code to handle skipping an optional 512-byte header from a padded ROM is trivial. Maintaining complex code one feels is unnecessary is clearly a burden, but if it's just a few lines and doesn't require revisiting, it's not a technical burden. If you supported properly-padded ROMs with a 512-byte header (that is, acted as if the file on disk had that removed), would that handle most of the headered-ROMs out there? I'm not asking about IPS patches, mind you, just the ROMs. If supporting this knocks out the need to remove headers on most ROMs, then you've eliminated a lot of unnecessary modifications of files all over the planet. Basically, you get the same effect as if you had removed them with regards to them working, acting the same as the unheadered ones (in particular, requiring IPS patches that assume no header), completely transparently and without a technical burden. All that's left are the unpadded, headered ROMs, and if they're relatively uncommon, they can be rejected without causing much inconvenience.


No disagreement here. I think software exists as a tool to submit to the will of the user, to extend what the user can do. It should never be used as a form of control of the user.

My experience with idealism is that it never works in reality like it works in my model of reality. This is an opportunity to learn more about reality and perhaps actually make it work there too.

And what's considered a "better" way is by a certain set of what's considered valuable, and what's considered costly. There are costs to changing things and some have greater ones, perhaps much greater than the benefit it gives. I can't help but connect this to free markets and centrally-controled ones, where the former is decentralized and has no person on top of everything, who understands everything, since it's the entire entity that's constantly making local decisions to maximize value, and able to take into account every individual's unique situation in maximizing value. Central control has to constantly disregard everything outside its model, to throw away the useful information encoded into the problems when trying to implement plans. It's outside the paradigm of coming up with a master plan that everyone will follow.

I don't see how there is any benefit to having copier headers, except to not break things (in a way that is trivial to fix) that have worked in the past.

I explained how the floppy disk size, interleaving, and multiple header formats makes the headers unreliable anyway; so it's hard to imagine a legitimate scenario where one is seriously keeping one ROM set that can both be used in an emulator and on hardware. I will acknowledge that it's possible to make such an emulator, but it doesn't currently exist.

However, I would like to entertain hearing more about why people want the headers. If I have missed something, please let me know.

From my perspective, it's about making executive decisions. Life is such that people often want conflicting things, and you can't please everyone. For everyone who really wants copier headers, there's another two who are tired of the problems they bring. The needs of the many outweigh the needs of the few.

I am tasked with weighing which position holds more merit. Supporting both with and without is my implicit endorsement of copier headers' continued existence. Those who want to abolish copier headers will never be able to do so while I stand in the way. Example: FuSoYa has stated he won't support headerless ROMs with his tool unless Snes9X stops supporting them as well.



True, and I have gone pretty far to give people that choice. Yet I seem to be required to provide compiled binaries with header support here for some reason.



Yes, it is. The unpadded ROM count has dropped substantially from where it used to be back in the '90s.



Yes, it would.



But you are skipping the IPS patching issue. The usage of command-line tools like sha256sum to check the integrity of or compare a file. The ability to rely on a ZIP archive's CRC32 to identify a file by hash without decompressing it first. ROM hackers manually adding and subtracting 512 to their file offsets. These are all intertwined.

If I remove header support, I encourage tool authors and patch creators to agree on a consistent choice. No emulator enforces copier headers, so headerless is the default choice to work everywhere. If I leave them in anyway, the situation will remain chaotic forevermore.

I'm not holding a gun to peoples heads forcing them to remove the headers, but I am doing my part to advocate my position. Others are free to create their own software, debate people as I have in favor of the headers, etc. We make our case and let the chips fall where they do, keeping our own autonomy and choices our own.



The problem is that this would cause issues as well. There are hundreds of patches out there that will only work when applied to a ROM with a header. If I were to support IPS patches but only after the header were removed, I would have people stating that I am forcing people to change their already-working IPS patches to suit my arbitrary decisions. What gives me the right to dictate/change how IPS patches are created?

And if I only support a format like BPS that stores the checksum, I would be told I'm trying to impose a new standard and take away everyone's IPS patches entirely.



The divide is that I honestly do not believe I am trying to hurt the user. I am trying to make things easier for everyone.

I recognize that trying to state how things should be done comes off as arrogant. But I have a decade of experience working with this. I understand the issues very well. Uninformed people do have a tendency to work against their own interests, eg "A little knowledge is a dangerous thing", and "What's the Matter With Kansas?"

Should we have kept PCX and IE6 bugs around forever, because a very vocal minority didn't want to change? Was DEP a mistake because it required a few software programs to be redesigned? Growing pains are a natural part of evolution.

I feel that as software authors, we have a certain responsibility to usher things in the right direction. I will admit that I do so in a very heavy-handed manner. Much nicer would be to allow headered ROMs and offer to convert them. Thus allowing choice but increasing the ratio of unheadered to headered ROMs. But like I said, I am not very pragmatic.



I don't believe that is the case with removing copier headers, but if one were to convince me, I would reverse course and offer copier header support again.

I just want to say something...
If you're trying to rid the world of copier headers, that's fine, copier headers are stupid. Just don't try to make people switch to "Rom Folders" at the same time. Windows just isn't the same as Mac OS. People don't interact with a singular "Icon" that contains a hidden internal directory structure.
If you really want to pursue Rom Folders, at least throw a shiny "Desktop.ini" file in there, give the folder a cool icon, and provide something to double click on to launch the emulator. And provide a good readme file with the emulator explaining that things are different in this town.

I just downloaded Bsnes recently, read the homepage, and saw that it emulates the GBA. So I was thinking "Oh cool, it now emulates the GBA! Let's try PocketNES". Then I tried to open a file, and wondered why the file open screen wasn't showing any GBA files. I couldn't drag-drop GBA files onto the main window either.
Then I looked for a readme file, but there was no readme file bundled with the emulator explaining Rom Folders or how to open files.
I couldn't even drag-drop a ROM file onto the EXE.
I thought that the program was extremely buggy, and the code for the custom-style rom menu must be broken or something. Which is really strange for an emulator with such a good reputation for accuracy.
Finally, I used the "purifier" tool to load a game, and that worked.
I was wondering "What the hell, GBA files have no copier headers on them. Why do they need to be purified?"

Later I read the documentation on the main website that explains what Rom Folders are, and realized why I couldn't open anything. Before reading this (and after testing out the emulator), I had no clue that the emulator worked with Rom Folders.

Needing to use a separate program just to try out an emulator is not user-friendly. It reminds me of the MAME for MS-DOS days, where you needed to run a Mame Frontend just to use Mame.

Putting the save file inside a Rom Folder is also a bad idea, because then it doesn't work on read-only media, network shares, or folders with restricted write permissions. If I was trying to use these, I'd be forced to make a ton of symbolic links, and that would be a pain.

-----

On a completely different note, PocketNES doesn't work very well in the emulator. The graphics blending is incorrect, and the Green channel is completely missing in the backgrounds. And the sound is missing too. But the GBA emulation looks like it's still in the early stages, so bugs are expected.

NES emulation looks really good now, for the mappers that are supported. Smooth and glitch-free. But I noticed that the purifier named any unsupported mapper as "NROM" in the manifest file.

Thanks for checking it out and leaving feedback.

> Just don't try to make people switch to "Rom Folders" at the same time.

Don't worry, bsnes v091 was a bit heavy-handed because it was the first release of a new UI.

higan v092 will let you load individual SFC images.

> Windows just isn't the same as Mac OS. People don't interact with a singular "Icon" that contains a hidden internal directory structure.

That is true, but I do want to stress how seamlessly I've made that to indeed be the case with kaijuu. And Windows users are used to the paradigm as well, double-clicking the CD-drive executes autorun.inf, and you have to right-click to browse a CD's contents.

> If you really want to pursue Rom Folders, at least throw a shiny "Desktop.ini" file in there, give the folder a cool icon, and provide something to double click on to launch the emulator

That's exactly what kaijuu does. And it also works on XP, where desktop.ini wasn't very powerful yet.

> I was wondering "What the hell, GBA files have no copier headers on them. Why do they need to be purified?"

The main issue with GBA is not being able to detect the save RAM type at launch. You can use heuristics of running code to guess, most of the time, but then you can't know things like save state format and size until after the code has run for a while.

> Needing to use a separate program just to try out an emulator is not user-friendly.

Integration is poor, certainly. If you drop a .gba file onto bsnes.exe, it will shell purify and play it. But I haven't yet been able to make a DLL that hooks into the GUI to add a standard file dialog interface to purify.

It's still planned, though. Long ago with the Qt GUI this was really easy, and I had a separate DLL that did stuff like SMC, ZIP, 7z, copier header, etc loading. I liked that approach a lot: it kept the code I hated out of my core. People could just delete the DLL if they agreed with me and wanted to do things my way. And I didn't have to maintain code I didn't like.

> Putting the save file inside a Rom Folder is also a bad idea, because then it doesn't work on read-only media, network shares, or folders with restricted write permissions.

The way the system is intended to work ... you have your ROM file that is your archival image. Put it in a read-only place, whatever.

To play in higan/bsnes, somehow that gets copied into somewhere (defaults to your home folder, can be pointed anywhere) and that is where all files related to that game are now stored. You can keep opening the game with the archival image if you want.

This is going to be important for some things. FlashROM and EEPROM in some instances are rewritable. So the idea is that internally, I'm going to use game folders, but the user won't have to care about it at all. It'll be behind the scenes.

If you're upset about the duplication of the image, I'm sorry, but I'm just not going to worry about it with today's multi-TB hard drives.

> On a completely different note, PocketNES doesn't work very well in the emulator.

It's not a very great GBA emulator yet.

> NES emulation looks really good now, for the mappers that are supported.

Only because NES is so easy, and I had help from Ryphecha. But thanks =)

The last system I want to support is FDS. Then it'll just be about refining what's there. Anything else (like NDS) will have to be other people writing the cores, or they won't happen.

Ultrabooks have 128-256 GB SSDs, not multi-TB hard drives. When higan starts supporting disk based consoles or someone else's DS core, the limitation of one emulated memory card per multi-hundred-MB application bundle will start to hurt.

I doubt that'd be reliable at all, though. There are games that will fail to run if you give them a larger amount of RAM (most likely to deter flashcarts), and they will make the emulator think there should be more RAM than there really is. Heuristics will fail miserably with those.

Even 128GB is enough to store 32 complete SNES ROM sets. Or buy a $6 SD card / USB drive.

And the thing is, it only eats the space on games you actually play. I'll bet you a million dollars that 99% of people do not actually play even half of complete SNES game sets. Let's face it, 90% of SNES games suck royally. For every Zelda 3 there are twenty <Sports Genre> <Year> and Bass Fishin' games.

NDS could become an issue, sure. Those games are huge.

> Heuristics will fail miserably with those.

Yeah, I hate heuristics. I want an internal database for every commercial GBA game, and a manifest board for everything else. I am amazed there's not a simple one pre-made, but I'll make one. Eventually. SNES database comes first.

A news update about beat has been posted, so the last step of pushing it is complete . There have already been ten downloads, so hopefully we will begin to see people leaving feedback about their experience with beat/BPS.

I like the post, but it doesn't mention one of the most important (IMO) features, and the one most relevant to this discussion, file checksums

EDIT: I see the actual project page has it, that works

BPS is just so jam packed with positive features that it is a shame that all of them can't be mentioned in the news update

The response in the forums has been fairly positive and the download count has gone up to 33, so maybe down the road we will actually see some hacks getting released as BPS files.

The strongest point will also be the hardest. This is a solid format that's finished.

The whole thing is ISC licensed (with the caveat that extending the format is strictly forbidden unless you change the patch signature/extension), and aside from the optional metadata, is absolutely set in stone. There won't be a BPSv2, or a new ?PS format in a few months.

At some point I'll probably make a general advisory format for metadata. Like a "translation scene / ROM hack scene info template" or something.

People are naturally going to be weary because UPS failed. I screwed up there, but I learned from the mistakes and avoided them with BPS. The BPS spec has been around for over a year now and much more extensively tested than UPS. Now, BPM is a bit new, but I've done a few tests there and everything seemed okay ...

Also probably worth noting that currently the delta encoder sucks. It's O(n^2) time. So you can't do files >100MB or so in a reasonable amount of time (if ever, people have left 700MB files running for days without completing.) This is temporary, although it'll probably be a long while before we can solve it. Very, very tough problem to compare all possible differences in two files without it being quadratic. But all the older ROM-based systems work just fine with the linear coder.

What's important is that the format is just fine. Patch appliers will just work and remain fast. We just need to make a faster patch creator.

Also be sure to mention that Snes9X v1.54 will support it for soft-patching if you haven't.

I hear "BPS" and think of the intro to Yoshi's Cookie for Super NES. It used to mean "Bullet-Proof Software" but apparently changed its name to "Blue Planet Software" in 1996 when Henk Rogers moved it to Hawaii in the process of establishing The Tetris Company. That and Pipe Dream, inspiration for half the hacking minigames out there.

Someone needs to make a Yoshi's Cookie hack that just changes the B on the logo screen to an I, then distribute it exclusively as a .bps patch.

It has been mentioned in the forum. beat has sparked the interest of quite a few people in the forums. Download count has gone up to 59, so the odds of seeing BPS patches has gone up slightly .

Of note, somebody even offered to do a Nimrod version.

You'll definitely want to tack down a basic metadata spec before people start using it, so patch applicators can agree on field names to parse. Obviously, being xml, it's meant to be extensible, but nail down the basics as soon as possible so it can be standardized.

Seems byuu's "friends" at RHDN, decided to step in an educate people that IPS is good enough and for the times it is not we should be using xdelta. But at least one ROM hacker stated he would use BPS for his N64 projects, so hopefully he will share his thoughts on the format.

I have only tried BPS on a small scale, I need to find a project that will really let me get to use it .

There have been 88 downloads, so maybe one of those people will share their thoughts on the format. byuu's "friends" have shared their thoughts on the format, but it somehow ends up being about byuu . byuu was not being sarcastic when he says he is a polarizing figure.

That's because you're pushing religion. Your own post focuses on people as well, with several scare quotes.

Religon? I don't follow your logic, but yes my post is mentioning people because at first it was a discussion of BPS, then people brought their feelings about byuu into the discussion. It is like comparing operating systems and having people bringing in feelings of Gates or Jobs into the discussion. It is not scientific or based on facts. If you feel, me wanting a discussion to be on the merits of a format without having its creator thrown in is religion, then you are confused. I need facts not feelings, if you prefer feelings then so be it. This has nothing to do with headers, if that is your motive for throwing in that jab of pushing religion.

What you call scare quotes, I call trying to be civil. I would rather refer to those people in a hash manner.

I legitimately find BPS interesting and think it could be a good thing in the long run. I don't care how many past failures byuu has had or if he has OCD, I want to hear technical flaws with the format. I heard three and I acknowledged them.

Flaw number 1
Patch stacking is not possible

Flaw number 2
Delta patching is slow on large files

Flaw number 3
No other implementations

I think judging a format based on your feelings for a man, sounds like the same logic used by religious fanatics. It is that same fanaticism that delayed man's technological progress, in many fields of science. In fact, explain to me how the following posts are technical assessments of the format instead emotionally driven comments and I will admit that I was in the wrong.



Because he doesn't see a point in a revision, that makes it a valid technical reason. Let us not forget the jab at the end towards byuu.



So he discredits delta patching when it is the very advantage that BPS has over UPS and states that delta patching is only needed for big games. It had already been mentioned that hacks with insertions or deletions benefit greatly from delta patching. He doesn't care because he wants to prove byuu's format is a failure, regardless of the facts.



He is stating here that there is no reason that anyone should implement their own delta patcher (except for soft-patching). Very good technical reason, no wait that is called an opinion with no scientific basis.



Very technical analysis of the format. BPS is bad because nobody cares about those feature. Very scientific approach.



I am at a loss for words. There are absolutely no other programmers in the planet that care about a programs complexity or license. Of course absolutes are part of a scientific mind.



Wow, the ONLY thing a new format can bring to the table is that. How very open minded, this reminds of the same open mindedness seen in religious fanatics.

I told you there was a reason I stopped wasting my time with them entirely.

You can't make people care about their own self-interests. All you can do is ignore them, and help out the people who do care. If we have to form a giant divide between { game folders, BPS, headerless images } vs { SMC, IPS, copiers }, then so be it. I'm not going to live in the past forever.

> but it somehow ends up being about byuu

Exactly. Whenever I'm involved, it becomes nothing but an Ad Hominem fest with no technical merit in the discussion.

Intelligent people will realize that, and stupid people I could care less about.

It doesn't matter who I am, or what I did or said in the past. That has no bearing on the technical description of the file format involved.

> Patch stacking is not possible

This is the same thing I've explained countless times. I am so god damn tired of repeating myself.

Patch stacking with IPS and BPS are equally possible. BPS has a checksum to verify you applied the patch correctly. It's impossible to support an arbitrary list of patching and have checksums for them all. So if you want to stack patches, ignore the checksum. Problem solved.

Right now, right this instant, 98% of patches on RHDN are meant to turn file A into file B, and not to be stacked. That's not hyperbole, that's not an exaggeration, that's fact. So the smart thing to do is default to validation.



Looky there, a check box to bypass validation, too.

But this is like long-form birth certificates, the truth doesn't matter, it's just a talking point.

> Delta patching is slow on large files

Yes, it is. That's a valid criticism, and one I'm working on.
It's the hardest technical challenge I've faced in my 15 years of programming, in fact.

I currently use a list of 64K nodes that keep 16-bit IDs of data at each address, and each node is a linked list of addresses that match, cutting the overhead to O((n/65536)^m), which is still quadratic. I don't know how Xdelta or bsdiff does it. Their technical descriptions use the entire Greek alphabet. bsdiff's author used it as his doctoral thesis: http://www.daemonology.net/papers/thesis.pdf

When you understand this:


You'll understand just how hard a problem this is.
Yes, it's my problem, and I have to solve it. But it's going to take some time.

Now to be fair, it only affects delta patches, which aren't made for ROM-based systems (NES, SNES, GBA, etc.)
And it's only an issue when the file size is >16MB.
And it's only patch creation, which happens one time.
And we're only talking ~15 minutes for a 16MB file.
Patch application is lightning fast.

> No other implementations

So far python-bps and Snes9X. Chicken and egg problem. How many other implementations are there of Xdelta? (hint: zero.)

beat isn't Xdelta nor bsdiff. It's goal is to be as simple as possible to implement a patcher for, at the cost of being slightly less efficient (typically ~1% difference in size)

Here is Xdelta3, over 500KB of math.

beat is 20KB of code to both compress and decompress. It's like comparing ZIP against PAQ/KGB.
And amusingly enough, in many instances, it beats Xdelta.

> explain to me how the following posts

In the future, I'd appreciate if you didn't quote that forum. It undermines my efforts to avoid them.

> That's because you're pushing religion. Your own post focuses on people as well, with several scare quotes.

I don't know what he said and I don't want to know. But that's unfortunate if it is true. Still, two wrongs don't make a right.

...

This is why I didn't bother posting there. I'm happy doing my own thing.

That is good to know. That version you are using, I notice it is v03, is it available somewhere?



Valid point, but I suppose the fear was the perceived loss of an ability already possible with IPS



I wish you much success in that venture, especially considering all the other aspects of you hobby that you have on your plate.



Understood. Alot of your comments make sense now, so I apologize for putting any salt in any old wounds.



I have no clue why blargg used that terminology. Nobody mentioned any religions, churches, ceremonies, gods or goddesses. I think it was a reference to the discussions here that referred to your philosophies as a "crusade", which is a religious term. I think it was a jab at me for backing two of your philosophies here in this forum. He was basically using "pushing religion" as a euphemism for "crusade". Headers and the like was not even the issue of discussion over there. BPS, beat, byuu, xdelta, UPS, and soft patching were the items of discussion.

It is sad to hear that your views are now considered a religion. I am not even following your beliefs I am just agreeing with them, as I had those feeling toward headers before I ever even knew of you. I like BPS patches not because you made them, but because I was legitimately impressed with its features. There were others in that forum that were also impressed, so I don't see what is so strange with liking BPS as a format.

> That is good to know. That version you are using, I notice it is v03, is it available somewhere?

It's the standalone, non-BPM patcher, designed more for when I was catering ROM hacking with direct-file patching.
It's available here: http://byuu.org/programming/bps

> I suppose the fear was the perceived loss an ability already possible with IPS

The problem is that I already addressed this when UPS first came out. About six times. To these same exact people.
And I've also addressed it more than once with BPS.

The person saying this knows for a fact it's not true. They are spreading FUD.

> I apologize for putting any salt in any old wounds.

It doesn't bother me. I've dealt with these people since 1997, so I have a rather thick skin.
They haven't discouraged me to stop working on things yet, and they never will.
In fact, for better or worse, I've come to use their negativity as motivation.
What I don't like is this forum having to deal with it too.

> Headers and the like was not even the issue of discussion over there.

Speaking of that, it's worth noting that beat itself doesn't push that issue.
With the checksum known, you can apply a beat patch to a headered ROM.
Now with the soft-patching in bsnes and Snes9X, we made it require headerless for simplicity and consistency. May as well, since there are no legacy patches to break, and emulators do not use copier headers at all.

HDRodrigo, I'd like to apologize on my religion comment. I reviewed your comments in this thread and elsewhere and saw that it was just that one message that set me off. There were two things going on:

You were referring to people as byuu's "friends", which I interpreted as an underhanded put-down of them (the meaning of the scare quotes). I tend to respond poorly to attacks on people.

I take it you just meant that you posted an announcement of BPS, and some other members of the forum replied with their thoughts that IPS was good enough and that BPS' advantages for their purposes didn't warrant the drawbacks of a format change.

My religion comment was how the focus is on how many people you get to use this format, as opposed to what problems it's solving for people (and what new ones it's creating), which is a focus not on what format(s) they are using, but on their entire workflow. It's also how an issue of what file format people uses becomes something almost moral, like you're purifying things. It rises above it simply being something that solves some technical issues, to something that must happen.

I would love it if technical discussions focused on technical issues and never brought people into it. I'm guessing that they brought byuu into the discussion because people arguing for new formats have unnecessarily brought their own character into the discussion in the past, or have pushed something technical that they had given signals that they weren't interested in.

I've emailed byuu about a possible speed improvement in the delta patcher, and am eager to hear whether it helped any.

> It rises above it simply being something that solves some technical issues, to something that must happen.

Yeah, I never intend to force the entire world to do things my way. I only ask that they give me discretion when designing my own software to work how I want it to.

In this case, it's really not a big deal. If authors want to keep getting complaints about header vs no header patching problems, they're more than welcome.

But the general reasoning is that less tools are easier, so if people were to unify around one format that solves all problems, it would make things easier. Again, not really an issue in this case. Easy to make a tool that can apply all patch format types (even if you need a separate binary for Xdelta.)

> I'm guessing that they brought byuu into the discussion because people arguing for new formats have unnecessarily brought their own character into the discussion in the past

Basically there's probably around 20 people who have a grudge with me, at least half for more than 10+ years now. So whenever my name is brought up they go into instant Ad Hominem mode.

I'm not going to say I did nothing wrong. I can be extremely crass on occasion, and I'm all to happy to argue back with asshats (rather than ignore them as more refined individuals do.) But it is amazing how long people can hold a grudge for.

But yeah, it doesn't matter how much you like or hate someone. The only thing that matters is the technical merits. It's sad to see otherwise bright individuals reduced to howling obscenities.

> I've emailed byuu about a possible speed improvement in the delta patcher, and am eager to hear whether it helped any.

I'm having a bit of trouble with the implementation, but I didn't want to keep bothering you with yet another one of my problems.

But yeah, I can keep a hash of eg "offset A-{A+N}" where A={source 0-file size}, and N={arbitrary length to match} values. Now if I have a 700MB file, and a 32-bit hash, that's a tree of {32-bit addr, 32-bit hash} * 700MB = 2.8GB tree. I would have to keep it sorted to do logarithmic searches, so I'd have to use a fancy self-balancing binary tree. I could reduce the memory size by incrementing A by N each time, rather than by 1. So for 16-bytes, we'd be down to 175MB of RAM, which isn't horrendously bad at least.

But we'd miss some matches entirely now, like say if we wanted to match 8+ bytes, then a pattern from 9-22 wouldn't register. Although my goal isn't to beat Xdelta's efficiency (it has advanced compression techniques built-in; and my variable-length integers aren't ideal for post-compression efficiency); I'd still like to be somewhat competitive. I'm already somewhere between Xdelta3 -9 and Xdelta3 -9.7z, meaning I can't beat the best-case Xdelta already.

Video game storage media larger than 64 MB, the size of the biggest N64 Game Pak, tend to be block devices such as optical discs or DS cards. So perhaps for matches farther than 64 MB, you can optimize for 512-byte offsets, as that's the size of a sector on most block devices.

No problem, but thanks for the apology. It is fully accepted.



It was the only civil way I could think of referring to them (without actually using their forum names). I guess I could have used acquaintances, but that seemed even more odd. I wanted to avoid saying enemies (or any similar term) or any expression that would connote my frustration with them.



I am fine with people stating that point. IPS does have its perks. It is widespread and is relatively simple to work with. I was disappointed with the dismissing of all of BPS' merits, based on feelings and emotions. It was pretty obvious that several individuals had not even looked at the specs of the format and were declaring that the format had no use.



My focus was to get as many people to try it out, not switch to it, in the hopes that people would begin discussing what problems it's solved or created for them and much more. It seemed to me that certain people, that had no plans to ever try out BPS, were only willing to give opinions instead of trying it out and give an actual thoughtful analysis of BPS' pros and cons. Basically they only demonstrated an interest to tear it down and not offer any constructive criticism.

I will give my philosophical view on the matter. One reason, I feel supporting or evaluating new open source solutions is important is because technology is in a constant state of evolution/progress and it would be nice if for certain things, we controlled the direction it was headed, as opposed to businesses with commercial/financial interests. A random example of my view in practice is my backing of the Ouya project. I don't believe people should switch to that "open" platform, but I feel if more people try it out it may bolster more open source projects into the public's eyes. Theoretically, the more people trying out open source projects, the faster they can evolve to be competitive with closed projects. Closed projects will in turn try to surpass them as to not to fall behind, thus the open source projects can direct the growth. Here is a proverb I was told that illustrates how I feel competition can improve technology, "As iron sharpens iron, so one person sharpens another". There, that is the most religion, you will get out of me on this thread



That is good to know. Fresh ideas and points of views are good for progress. Maybe the speed barrier can get cracked sooner rather than later, if more individuals would just come together and help support the project (regardless of whether they will use it or not) rather than tear it down. Reading the discussions between authors of video encoders has always impressed me. They each have their own format that they are backing/specialize in, yet they still share ideas and discuss the different ways something could be implemented (This is based on the discussions I have seen take place on Doom9's forum).

Thanks for sharing more about your philosophy. I really like it and the benefits it has for open development. I take you as basically asking people to step out of their comfort zones a bit and give some things a try and really scrutinize them, so that the developers can fix problems and then put them out there again for further evaluation. This is the only way, as you say, that they can reasonably match the proprietary things out there, which have had a lot of this kind of evaluation behind the scenes. It's also made me think of my own approach, which is sadly more the "what, that's crap, nevermind I'll go back to what I was using before" kind. This makes me want to at the very least give a semi-detailed report of why I switched back so the developers can fix that for the next person.

This is all the more reason to seriously avoid any judgments of people who don't want to try a new format, to be as absolutely technical as possible when presenting it, and always allowing for it not being the end-all format, that current ones might really work better for some people. I personally have low patience for people who even have hint that they think their solutions are superior and could have no objections or shortcomings for particular uses. Humility and openness to hearing about problems win big points with me. I'd like to be less short with the former people, but eh this is how I am currently.

> I personally have low patience for people who even have hint that they think their solutions are superior and could have no objections or shortcomings for particular uses.

I always try to list pros and cons, the pros are usually much longer, but the one recurring con that always seems to dominate is legacy support.

But for fun ... correct me if I've missed anything.

Against IPS:

BPS pros:
* validates patch, input and output files are correct (a huge deal thanks to copier headers)
* embeds authorship metadata if desired
* supports >16MB, and >4GB files
* supports data insertion, deletion, and move deltas
* smaller patch sizes
* no extensions exist, no extensions are possible that will break old patchers
* no ambiguities (EOF as an address versus an end file marker, etc)

IPS pros:
* can be implemented with half the code (both are under 4KB)
* ZSNES and older Snes9X releases can soft-patch these files
* some people already have IPS patches or tools

Against Xdelta3:

BPS pros:
* can be implemented in one twentieth (or more) of the code; allows embedded soft-patching support in emulators
* embeds authorship metadata

Xdelta pros:
* lightning fast delta patch creation (temporary advantage, but a significant one all the same)
* an Xdelta3 -9 (max compression) inside a 7z -mx=9 (max compression) archive is typically 5-10% smaller

I guess an IPS to BPS converter would help to gain some popularity to the format, as it would allow to get back catalog available in the new format and allow to make comprasions between file sizes in different formats in an easy way.

If creation of Xdelta3 patches is lightning-fast, then couldn't one create an Xdelta3 patch, parse it, and represent the same data in BPS format? Or are the data models too different, like GIF and JPEG?

Shiru, an IPS-to-BPS converter would be like converting a JPEG to PNG; you'd have to fill in blanks and provide a false sense of security. It'd have to be done semi-manually, with the user verifying that the IPS-patched file worked properly before making a BPS patch for it.

tepples, I took the comparison to be about the formats themselves, rather than implementations. So like you, I see Xdelta3's speed as mainly that it has a mature implementation, rather than a benefit of the format itself.

So a BPS patch has three checksums: that of input file, patched output file, and patch itself? If so, wonderful, as that gives a high degree of confidence, and if there's an error, points to the cause.

Ambiguities are a nasty problem with IPS. I remember writing an IPS decoder and having to use heuristics in decoding some. It reminds me of the issue with HTML parsing, where the maxim of being lenient on what you accept was taken the wrong way and taken to mean to not even complain if something was out of spec, leading to people never even knowing that their pages were out of spec since most things didn't make a peep.

Yes, it does. And yes, it is.

Yes, but if they don't wish to step out of their comfort zones a bit, I would rather they not say things that would needlessly discourage any further innovation. They should just ignore the subject, and let those that are interested discuss and criticize (constructively) the subject matter. Then again I am for free speech, so I guess I am not one to talk .


I agree. I worked for a time as a researcher (In genetics and microbiology, nothing of use here) and became an idealist of sorts, when it comes to the sharing and expanding of knowledge/information. I refuse to believe that anything is an end-all solution, when it comes to technology. The moment we think like this, then progress stops. I believe that in any scientific field (including computer science) we should always attempt to test and/or build upon the work of others. I think BPS is an interesting approach because, in a sense, it builds upon the prior work of two patching formats. It is not a revolutionary idea, but it is an "evolutionary" one that can have a practical use for many.

Sometimes researchers do need to "pimp" their ideas/discoveries or else all the work they have done may not receive any scrutiny. Finding the balance to getting interest in ones work with out being a little bit controversial or grandiose is a tough one.

> If creation of Xdelta3 patches is lightning-fast, then couldn't one create an Xdelta3 patch, parse it, and represent the same data in BPS format?

That may or may not be possible, the problem is that Xdelta3 is a monstrously Byzantine format. It would probably be harder to learn how to parse it than how to do the delta comparison >_<

> Shiru, an IPS-to-BPS converter would be like converting a JPEG to PNG; you'd have to fill in blanks and provide a false sense of security. It'd have to be done semi-manually, with the user verifying that the IPS-patched file worked properly before making a BPS patch for it.

Correct. Also on my forum, someone already converted every completed SNES translation from RHDN into a big BPS patch archive.

> So a BPS patch has three checksums: that of input file, patched output file, and patch itself?

Yes. But one choice I made was to use CRC32 instead of SHA256. The idea was that CRC32 can be implemented in a one-line function, whereas SHA256 is not so simple. But aside from malicious intent to deceive, that gives you only a one in four billion chance of a false pass in the worst case, and a 1:2^64'ish chance in the best case. I favored simplicity of the spec over cryptographic-level security. And besides, SHA256 will likely be broken eventually, anyway.

Another point to show how much thought I put into the patching format: the hashes always come after the data, that way you can calculate the hashes at the same time as you read/write the files, if you so choose. For creating patches, it means not having to seek back and rewrite data, and not having to buffer the whole patch in memory.

> It reminds me of the issue with HTML parsing, where the maxim of being lenient on what you accept was taken the wrong way and taken to mean to not even complain if something was out of spec, leading to people never even knowing that their pages were out of spec since most things didn't make a peep.

Yep. Another fun one we've found was that some patches would seek backward in the file (use a lower offset after a higher previous offset), and at least one patcher couldn't handle that when it was streaming the output.

> I refuse to believe that anything is an end-all solution, when it comes to technology.

Definitely. If a more suitable format than mine emerges, I'm all for it.

Really, the primary driving force for BPS over Xdelta was that I wanted to see soft-patching support in emulators. And we have that, Snes9X had no trouble supporting it. Whereas I know of no third-party implementation of Xdelta.

When fighting about sizes, it becomes like compression. Do you use ZIP (ubiquity and simplicity), 7zip (great compression, somewhat popular), or KGB Archiver (ten hours to make an archive, but they are certainly the smallest)? bsdiff -destroys- Xdelta (by 3:1 or better) on x86 program diffs. Google's Chrome diff destroys bsdiff even. But those are highly specialized.

So honestly, it depends on what you're doing. I think Google's Chrome diff is wonderful for how it is used and what it does. But I'd never use it for an SNES fan translation.

I tried finding it and failed , how big is the download or do you have a link address? I would love to compare the file sizes between the IPS files and the BPS files.