Defining a cross-emulator save state format

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Defining a cross-emulator save state format
by on (#238959)
In this post, mkwong proposed designing a save state format that is portable across different NES emulators. Let me describe some snags that an effort to design such a format spec would encounter:

How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.

As we come to understand the NES better, we may discover more state that we need to store. Either it'll have to be a name-value pair format (except for bulk RAM contents) or there'll need to be some sort of version number for the Control Deck's state and the mapper's state.
Re: Defining a cross-emulator save state format
by on (#238962)
Cross-emulator save state formats have been tried in the past. They don't really work.
Re: Defining a cross-emulator save state format
by on (#238968)
A Q post from 2005 which I wholeheartedly agree with.
Re: Defining a cross-emulator save state format
by on (#238970)
tepples wrote:
How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.

Yes, that is definitely a lot of work. But as all authors of emulators with save state functionality have done it already, why not spare newcomers from reinventing the wheel?
Re: Defining a cross-emulator save state format
by on (#238986)
Well, if nothing else, attempt to export and import savestates from the "popular" open-source emulators. Maybe FCEUX, Nestopia, Nintendulator, etc...

You might not have complete information needed for another emulator, but some things can be extrapolated.
For example, if you have the current CPU or PPU memory map, you can fill in what the values for mapper latches should be.
If you know the PPU raster position that the save state would happen at, you can ensure that the save state is made at that time.

Even if you can't get a 100% accurate state conversion that would survive TAS-like validation, you might still get something that can let you proceed to play the game without a crash.
Re: Defining a cross-emulator save state format
by on (#238989)
mkwong98 wrote:
tepples wrote:
How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.

Yes, that is definitely a lot of work. But as all authors of emulators with save state functionality have done it already, why not spare newcomers from reinventing the wheel?

Sparing them from reinventing the wheel? Why exactly are they making a new emulator for a system that has hundreds? :P

Dwedit wrote:
Well, if nothing else, attempt to export and import savestates from the "popular" open-source emulators. Maybe FCEUX, Nestopia, Nintendulator, etc...

I think the idea of a savestate converter tool might be fun.

You could do this as its own separate utility without trying to burden any emulator authors with it. It might also become a nice point of reference for the actual savestate binary formats used by various emulators, since that stuff tends not to be well documented at all.

(On the other hand, the benefits of having such a thing are probably so small that nobody would ever want to undertake a project like this.)
Re: Defining a cross-emulator save state format
by on (#239025)
Dwedit wrote:
You might not have complete information needed for another emulator, but some things can be extrapolated.
For example, if you have the current CPU or PPU memory map, you can fill in what the values for mapper latches should be.
If you know the PPU raster position that the save state would happen at, you can ensure that the save state is made at that time.

Even if you can't get a 100% accurate state conversion that would survive TAS-like validation, you might still get something that can let you proceed to play the game without a crash.

My idea is similar to yours. A minimum amount of info required for a save state is defined. Beyond that, values are defined with a default value.

rainwarrior wrote:
Sparing them from reinventing the wheel? Why exactly are they making a new emulator for a system that has hundreds? :P

Because they want to. :wink:
Anyway we can't ignore the possibility that there are people out there with new ideas and ability to create a good emulator. Mesen is a very good example of a relatively new emulator when there are already hundreds.

rainwarrior wrote:
I think the idea of a savestate converter tool might be fun.

You could do this as its own separate utility without trying to burden any emulator authors with it. It might also become a nice point of reference for the actual savestate binary formats used by various emulators, since that stuff tends not to be well documented at all.

(On the other hand, the benefits of having such a thing are probably so small that nobody would ever want to undertake a project like this.)

If we have a save state converter tool, the internal state of the tool itself can be used as the base of a cross-emulator save state format. Then we can change this tool into a lib for emulators to use and we are done! :beer:
Re: Defining a cross-emulator save state format
by on (#239392)
The basic NES memory can't be different, along the CPU state. The problem is all about mappers, I suppose.
Re: Defining a cross-emulator save state format
by on (#239395)
Different emulators use different models of PPU state, which may cause part of the problem even apart from mappers. That's why I recommended requiring portable save states to be taken in the post-render line, as it's when the PPU state is least in flux.
Re: Defining a cross-emulator save state format
by on (#239400)
tepples wrote:
Different emulators use different models of PPU state, which may cause part of the problem even apart from mappers. That's why I recommended requiring portable save states to be taken in the post-render line, as it's when the PPU state is least in flux.


Emulators can be used for at least four different purposes:

  • Running as efficiently as possible a game that's known to use only a certain set of hardware features and programming techniques.
  • Validating that a game only relies upon a particular set of hardware features and programming techniques.
  • Running a game that uses an unknown set of hardware features and programming techniques.
  • Experimenting with new programming techniques.

Given that some behaviors can vary among different real systems, no emulator can be ideal for all purposes. An emulator that is focused on the first two purposes may be much faster than one which needs to be suitable for the fourth, or needs to accomplish the third without trial and error (typically, one could accomplish the third by making a guess about features and then testing to see if one's guess was correct).

When running programs which don't exploit too many corner cases, an APU/PPU emulator which is written in an interpreted language but skips time-consuming corner cases could probably be faster than a more "accurate" emulator written in native code. Among other things, if the interpreted language included "sync" instructions to indicate when an action by the main CPU might have side effects that are observable on the PPU or vice versa, and if such side effects are rare, the emulator could reduce the rate at which it has to switch between processing CPU-side logic and PPU-side logic, thus improving cache performance.

If save-game files could include emulator *logic* in a portable format, then they could be processed by any emulator that can interpret the bytecode language.
Re: Defining a cross-emulator save state format
by on (#239483)
About the savestate, is there a problem of setting APU (and possibly) PPU timings to zero instead of saving cycle counters? Good about saving on scanline -1. I believe it's no so hard to create a standard savestate format with tagged blocks.
Re: Defining a cross-emulator save state format
by on (#239522)
Zepper wrote:
About the savestate, is there a problem of setting APU (and possibly) PPU timings to zero instead of saving cycle counters? Good about saving on scanline -1. I believe it's no so hard to create a standard savestate format with tagged blocks.


The DMC timer can be used to generate raster-synchronized interrupts without requiring a special mapper, but losing the APU state would totally break things.
Re: Defining a cross-emulator save state format
by on (#239524)
In addition to the DPCM IRQ...

The APU frame counter can also be polled or used as an interrupt.

The APU status bits will change e.g. when a DPCM sample reaches its end.

So there's a few APU things that a game might rely on. Probably a lot of games wouldn't have any discernable effect of losing APU state at a save, but I'm sure there are notable ones that would be affected.


There are some expansion audio devices that are effectively "write only" though. Those at least might potentially have no non-audio consequences. ...and of course if you care about audio consequences, this whole idea isn't even on the table anyway.