Tilemap/colormap from Super Mario World looks like garbage?

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Tilemap/colormap from Super Mario World looks like garbage?
by on (#237452)
So I downloaded YY-CHR (.NET version) to see what the tilemap and colormap look like from an actual SNES game. I downloaded Super Mario World.smc from the internet (and it works perfectly on the BSNES emulator). I opened it up in YY-CHR and got this weird thing that looks almost like random noise...

When I did the same thing for Super Mario Bros.nes I got a much nicer result that matches my expectations.

Why does the SNES game tilemap and colormap look like that? Did they just optimize the graphics so much that it ends up becoming non-human-readable? Or am I just using YY-CHR incorrectly...
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237453)
Most NES games put their graphics data in a separate physical IC, and the NES's Picture Processing Unit can only work with uncompressed data. However, almost every other console ever instead provides RAM for the PPU or its equivalent, and to save space the game will compress that graphics data.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237455)
Try using vSNES to look at ZSNES / SNES9x savestates. You can then also load the ROM and use the savestate's color palette to search for graphics in the ROM. There are games that don't compress them because they still fit into the designated ROM chips, or because they need to be loaded very quickly.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237456)
lidnariq wrote:
to save space the game will compress that graphics data.


How do they compress it, and how does the SNES uncompress it?

How can I see/manipulate the tilemap? Is YY-CHR capable of it? Not sure why it has SNES mode if it's not gonna uncompress the data... Is there another program I can use?
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237457)
Any game can compress/decompress graphics however it wants, using its own decompression code. The format of the graphics in one game may be completely different from another game.

Because of that, it is more-or-less impossible to write a tool that can handle graphics from all SNES games unless it's programmed to recognize every individual game and decompress that game's format.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237458)
For Super Mario World specifically, try Lunar Magic. It can decompress and export the graphics.

Also, the word "tilemap" doesn't refer to a bunch of tiles. It specifically refers to the data structure that tells the PPU where to put background tiles when rendering a layer. SMW (like every Mario game) builds its tilemaps from compressed level data, not that that matters in this case because even an uncompressed tilemap in YY-CHR would look like garbage and would not be distinguishable from program code or any other kind of data.

I'm not sure what you mean by "colormap", but you can't see the palette in a tile editor. The matchup between indexed graphics data and actual colours is done at runtime, which is why so many games have different-coloured characters/enemies that are otherwise identical (it's cheap). And there's no easy way to figure out what data in an arbitrary game corresponds to palette data, so tile editor programs don't bother.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237459)
rchoudhary wrote:
Not sure why it has SNES mode if it's not gonna uncompress the data...
The primary purpose of YY-CHR is for writing games, not reverse-engineering them. It's entirely coincidence that the NES requires uncompressed data, which makes it possible to use this general purpose tool to view its picture data.

During the development of many games, the assets will often remain uncompressed for ease of editing; compression is often only used when building the ROM itself.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237462)
rchoudhary wrote:
How do they compress it, and how does the SNES uncompress it?

Custom uncompression software runs on the SNES CPU.

Compression would have been done by their custom development tools.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237464)
I think this is the coding system.

http://old.smwiki.net/wiki/LC_LZ2
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237465)
This compression format is not tied to the SNES right? Like the code to decompress the graphics is in the Mario game itself?

Are there any SNES games with uncompressed graphics?
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237479)
rchoudhary wrote:
This compression format is not tied to the SNES right? Like the code to decompress the graphics is in the Mario game itself?

Correct. Every game can do it however they wish. There is no "standard". There may be algorithms used that are commonplace, but there's zero guarantee of it; companies certainly used their own compression algorithms based on whatever there needs were.

As for the decompression: it's done on-the-fly (into RAM) natively in 65816 assembly code. You get to reverse-engineer the game and its code to figure out how its done, then apply that knowledge to write a tool/program to decompress it (from the ROM directly, or say, if you were to extract the compressed data using a hex editor into a file, then decompress that file).

Likewise, if you planned on re-inserting it, you'd need to write a compressor tool. On top of that, you'd need to know "how" to reinsert it; if it's larger than the previous data, this is going to cause major problems for the game because you can't just "extend the size" of something and have it work -- if you shift everything up/over/whatever (in the ROM) by even 1 byte, everything breaks. And depending on how the game code references the data, it might even crash/break if you inserted recompressed data that was smaller (in length) than the original -- it all depends on all sorts of variables/nuances that are defined by the code itsef. Thus, you have to reverse-engineer the game in pieces (eventually as a whole) to get the full picture. That leads me to this statement:

Welcome to romhacking. There is a site dedicated to exactly that: http://www.romhacking.net/ . The forums here are more "development-focused", but asking technical questions is always encouraged too. If you're new to the whole romhacking thing, asking there might be a better start, as there may be tutorials people can refer you to that can help with the general understanding.

rchoudhary wrote:
Are there any SNES games with uncompressed graphics?

Plenty. You should spend some time going through all sorts of games for this purpose, for your own education.

Also remember: not every game has to "use compression everywhere" (i.e. there are many games that use a combination of both compressed and uncompressed data). Again: every game is different.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237481)
BTW, I've seen games use unexpected bit depth.

SNES can use 2bpp, 4bpp, and 8bpp, and mix them. But, I've even seen 1bpp. That's right. 1 bit. The game doubled each byte to produce a 2bpp tileset that was white text on black background.

And some games even did a weird 3bpp, which isn't even a supported format... and did some decompression algorithm to turn it into 4bpp.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237482)
Graphics in Super Mario All-Stars were uncompressed last I checked.
Re: Tilemap/colormap from Super Mario World looks like garba
by on (#237483)
dougeff wrote:
... I've even seen 1bpp. That's right. 1 bit. The game doubled each byte to produce a 2bpp tileset that was white text on black background.

In case an example of this is needed: Otogirisou does this... plus more.

It's Japanese text font is a 1bpp VWF in kind of a weird RLE-like format. The font is loaded from ROM and decompressed into RAM, while simultaneously doing some shifting + math + masking work to make the 2nd bitplane (thus 2pp) which is used for a 1 pixel-wide shadow effect on the right and bottom parts of the actual font, giving it depth + standing out on brighter screens (though the game is quite dark, figuratively and literally). Said graphics are then DMA'd to PPU RAM. In other words: the shadow effect is done purely in software, in real-time, rather than stored as part of the font/graphics.

Took me quite some time to figure out what was going on (the decompression took me a while to understand, once I got the routine figured out, I was like "wait, what is all of this other nonsense? Why are they doing this? For what pur...... oh! OH. Ha."

Few pictures of that work on grid/graph paper, as well as some results once I figured out just enough of how its game text bits worked to replace text (extremely complicated). The last picture/text means nothing to anyone really, kind of an inside joke at the place I was working at. Most of my notes are hand-written on paper, including working out some of the routines in 65816.

I really don't like RE'ing Chunsoft games. Their code is really, really dense and good (read: hard).