Checksum Craziness

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Checksum Craziness
by on (#161411)
If there's one thing that's been bothering me about SNES development, it's that all checksum fixing programs are closed-source, except for ucon64, which is bloated and crazy. (The help menu can't even fit in the cmd).

This has been bothering me, so I've tried writing a program to fix the checksum, and have had some luck, (dead link) but the problem comes around when applying the checksum (note that the code is only meant for 2mbit lorom games for now).

When using this rom, at first the program gets the checksum just right. Then it puts in the checksum, and snes9x suddenly says that it's actually supposed to be a little higher. I noticed that it was actually adding in the checksum bytes, and had my program do so. On a fresh build of the rom with the fix, suddenly the checksum needs to be a bit higher.

Basically, I'm asking that someone explain to me in the most remedial way possible how the checksum is calculated. Just for 2mbit roms, I don't care about the rest for now. I just don't get it.
Re: Checksum Craziness
by on (#161413)
The checksum is calculated by adding every byte in the ROM together and keeping the lowest 16-bits of the total.

There is both a checksum and an inverse checksum in the header that complement each other. The difference is just a XOR, so together they should always add up to 0xFFFF.

The inverse checksum is there so that changing the checksum in the header won't change the calculated checksum of the ROM itself.
Re: Checksum Craziness
by on (#161420)
Okay, so currently my program has these steps. (Note, it correctly implements the checksum and complement now).

1. Add up all the bytes.
2. Get the lower 16.
3. Add in the checksum bytes.
4. Add in the complement bytes.
5. Put the stuff into the sfc.

This worked once. That's it. Then it started screwing up. Is this even right?
Re: Checksum Craziness
by on (#161422)
Let's try a test-driven approach. Have you written a program to verify checksums and run it on good dumps of the same size? If you have, try running it on the output of your program.
Re: Checksum Craziness
by on (#161424)
I'm not really sure what you're asking for here. My progress is up, if anyone can follow along. I'm so confused.
Re: Checksum Craziness
by on (#161425)
UPDATE! It works now. I read two things:
1. A post by byuu on the zsnes forum saying not to add the checksum and inverse bytes to the checksum.
2. Something in the ucon64 source about running the program twice for correct checksum.

I took these things in mind, ran the program twice and it works perfectly. Edited the source so it does so on its own from now on.
Re: Checksum Craziness
by on (#161426)
The hardcoded 0x7FDC and 0x7FDE offsets are correct for LoROM (mode $20/$30) but incorrect for the HiROM (mode $21/$31) and ExHiROM (mode $25/$35) mappers.
Re: Checksum Craziness
by on (#161427)
Yeah, I'm aware but don't plan on modifying the program to be more versatile just yet. For now imma just be happy that ucon64 isn't a dependency for me anymore.
Re: Checksum Craziness
by on (#161428)
nicklausw wrote:
I took these things in mind, ran the program twice and it works perfectly. Edited the source so it does so on its own from now on.

Why run it twice when you could be running it once?

The four bytes of the checksum and inverse checksum will always add up to 0x1FE once they've been correctly set, so you can skip them and add 0x1FE to the final value to get the correct checksum in one pass.

Skipping them might be more work than pre-setting them to some fixed value (for example, 0x0000 and 0xFFFF) before calculating the checksum.
Re: Checksum Craziness
by on (#161429)
Joe wrote:
The four bytes of the checksum and inverse checksum will always add up to 0x1FE once they've been correctly set, so you can skip them and add 0x1FE to the final value to get the correct checksum in one pass.

Neat! :)
Re: Checksum Craziness
by on (#161435)
Joe wrote:
nicklausw wrote:
I took these things in mind, ran the program twice and it works perfectly. Edited the source so it does so on its own from now on.

Why run it twice when you could be running it once?

The four bytes of the checksum and inverse checksum will always add up to 0x1FE once they've been correctly set, so you can skip them and add 0x1FE to the final value to get the correct checksum in one pass.

Skipping them might be more work than pre-setting them to some fixed value (for example, 0x0000 and 0xFFFF) before calculating the checksum.

Thanks, fixed.

While I said earlier that I have no plans on implementing further support (lorom, hirom, roms > 2 mbit,) I figure I might as well at least try now.

Update: nah.
Re: Checksum Craziness
by on (#161488)
Okay, (yes I know I can't decide whether I want to add full support or not) I read somewhere that to do games larger than 4mbits, you add the bytes up in 4mbit chunks, add these all together and get the lower 16 bytes of that. Or do you get the lower 16 bits of the chunks, add them up and do it again? Can anyone confirm this?

EDIT: I'm pleasantly surprised to see that the method I was already using worked perfectly with Super Mario RPG. The problem I'm facing now is roms such as Final Fantasy 3 and Super Metroid which claim to have 32 mbits, but really only have 24. No idea what to do about those.
Re: Checksum Craziness
by on (#161498)
nicklausw wrote:
Okay, (yes I know I can't decide whether I want to add full support or not) I read somewhere that to do games larger than 4mbits, you add the bytes up in 4mbit chunks, add these all together and get the lower 16 bytes of that. Or do you get the lower 16 bits of the chunks, add them up and do it again? Can anyone confirm this?

They're equivalent. All additions for checksum are modulo 65536, and applying this modulus before or after combining partial sums matters not.

Quote:
EDIT: I'm pleasantly surprised to see that the method I was already using worked perfectly with Super Mario RPG. The problem I'm facing now is roms such as Final Fantasy 3 and Super Metroid which claim to have 32 mbits, but really only have 24. No idea what to do about those.

For 6, 12, 24, and 48 Mbit ROMs, count the last third twice. For 10, 20, and 40 Mbit ROMs, count the last fifth a total of four times. This corresponds to doubling up the last part of the ROM until it adds up to a power of two.
Re: Checksum Craziness
by on (#161680)
Is it possible to distribute this as a standalone executable? I tried making a SNES checksum fixer in C++ a while ago that I could use as part of the tool chain when assembling (ucon64 is a bit much for this indeed), but I ran into some weird problems I couldn't figure out and put it on hold.
Re: Checksum Craziness
by on (#161682)
Pokun wrote:
Is it possible to distribute this as a standalone executable? I tried making a SNES checksum fixer in C++ a while ago that I could use as part of the tool chain when assembling (ucon64 is a bit much for this indeed), but I ran into some weird problems I couldn't figure out and put it on hold.

Here you go. I included the source, maybe its useful. I actually used ucon64 all the time before reading this post to fix the checksum of my programs and had no problems, but i guess it doesnt hurt to have my own program do it :).
I tested it with 4mbit, 32mbit and 48mbit ROMs, and had positive results. If there is a bug let me know.

Edit: Now it's licensed; recompiled the executeable; now using static libs in order to run on without any dependencies on windows
Re: Checksum Craziness
by on (#161692)
Thanks. Do you plan on putting the source code under a free software or open source license? If you need help choosing one, the GNU project has an article about Apache, LGPL, and GPL.
Re: Checksum Craziness
by on (#161693)
elseyf wrote:
Pokun wrote:
Is it possible to distribute this as a standalone executable? I tried making a SNES checksum fixer in C++ a while ago that I could use as part of the tool chain when assembling (ucon64 is a bit much for this indeed), but I ran into some weird problems I couldn't figure out and put it on hold.

Here you go. I included the source, maybe its useful. I actually used ucon64 all the time before reading this post to fix the checksum of my programs and had no problems, but i guess it doesnt hurt to have my own program do it :).
I tested it with 4mbit, 32mbit and 48mbit ROMs, and had positive results. If there is a bug let me know.

Oh thanks a lot!
Did you link the library? It says I'm missing dlls when trying to start the included exe. I tried to compile it myself with static linking but Codeblocks just decied to refuse doing anything I tell it. :(
Re: Checksum Craziness
by on (#161704)
tepples wrote:
Thanks. Do you plan on putting the source code under a free software or open source license? If you need help choosing one, the GNU project has an article about Apache, LGPL, and GPL.


Thanks for the suggestion. After reading the article a bit, the APACHE License seemed appropriate.
I actually never used a license before to publish some code.
Pokun wrote:
Oh thanks a lot!
Did you link the library? It says I'm missing dlls when trying to start the included exe. I tried to compile it myself with static linking but Codeblocks just decied to refuse doing anything I tell it. :(

I used Code Lite to compile the code. I assumed it statically link the libraries so only the executeable would be required to work. I recompiled sns_chk, this time i explicitly stated the -static flag, so it would include the libraries statically.
Here are the switches used by Code lite to run g++ (c++ compiler):
Code:
-O2 -Wall -static -static-libgcc -DNDEBUGĀ  -o ./Release/sns_chk.cpp.o -I. -I.

Please try the updated zip and report.
Re: Checksum Craziness
by on (#161710)
Still doesn't work. "The program can't start because libgcc_s_dw2-1.dll is missing..."

Maybe try add "-static-libstdc++" in addition to the other flags.


Edit: Now it works perfectly! Thanks again!
Re: Checksum Craziness
by on (#162292)
This thread made me put some polish to the small tool I've used for setting checksums on my own builds... I just uploaded it to GitHub: SuperFamicheck

It was a pretty interesting, however useless, detour! It should display info and optionally fix most ROMs without too many false positives (at least on even the most exotic stuff I've tested).
Re: Checksum Craziness
by on (#162295)
Jeez, there's so many of these programs around apparently yet I wasn't able to find any without asking...about a year ago (I've been a member for a year now!), tepples mentioned that we need to gather and organize all the SNES development tools out there because of how scattered they are, and this is making me want to attempt that.