S-DD1 graphics compression

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
S-DD1
by on (#28703)
In this post, tokumaru wrote:
I'm still thinking of a way to make better use of the 2D redundancy present in the tiles... generic compression algorithms are 1D, and will ignore that.

Star Ocean for Super Famicom uses "S-DD1" compression. This XORs each row of pixels with the one above it, XORs each pixel of the residue with the pixel to the left of it, and does arithmetic coding on the rest.

by on (#28705)
tepples wrote:
This XORs each row of pixels with the one above it, XORs each pixel of the residue with the pixel to the left of it

I have thought about this before, but never implemented... I didn't even know this was already used and named! :shock:

Quote:
and does arithmetic coding on the rest.

That would be a bit hard to implement on the NES, right? But I don't know what other kind of compression could be used instead. I guess this was why I never used the idea, because I couldn't figure out how to properly compress the transformed data.

by on (#28707)
tokumaru wrote:
tepples wrote:
This XORs each row of pixels with the one above it, XORs each pixel of the residue with the pixel to the left of it

I have thought about this before, but never implemented... I didn't even know this was already used and named! :shock:

And fully documented.

Quote:
Quote:
and does arithmetic coding on the rest.

That would be a bit hard to implement on the NES, right? But I don't know what other kind of compression could be used instead.

Huffman coding of 8-bit units would probably stand in for arithmetic coding.

by on (#29368)
Quote:
This XORs each row of pixels with the one above it, XORs each pixel of the residue with the pixel to the left of it


No. You have missed the point there. What the algo is doing is to separate a stream of bits corresponding to a bi-dimensional image into several substreams on a per-pixel base according to the "surrounding" pixels that we have previously encountered in the stream. Each of those substream are then independently compressed.

That is the idea of the "contexts" in the S-DD1 algorithm, and you can decouple it from the arithmetic encoding so as to use it with whatever other compression of your choice.

EDITED: After re-reading it, i thought this comment was somewhat obscure even for me, especially this:

Quote:
Each of those substream are then independently compressed.


What i meant is that the idea of the "contexts" is just what allow you to take into account the bi-dimensional info on the stream, allowing you to cut the stream into substreams so as to take adventage of that fact. One you have done so, you can apply the compression. What i wanted to stress (and i think it wasn't clear at all) is that, while the S-DD1 can mix the compression of the substreams in one single "output" stream due to the nature of the compression it uses, you can too choice to compress them independently by whatever compression you choose. Of course, if you do so, you have the problem of determining where are the limits of the output substreams. To mix them, the S-DD1 use the technique of storing a MPS and a probability estimation per context. The MPS allow it to XOR the substream so as to have as much zeros in the sequence as possible; the probability estimation allow it to interleave the compressed substreams in a clever way so as to group parts of the substreams that have similar entropy. Look the documentation for details.