Overlap World Map image with Video of gameplay

There's some "ATLAS" videos where someone took a speedrun and moved the gameplay video on top of an image of the map so you could see the character move throughout the entire map. It is awesome and there are a few like this one: https://www.youtube.com/watch?v=9CoQ-wXR_iw

But that was done manually frame by frame.

What I'd like to do is some digital processing on the video to figure out where the player is. So every frame(ish), you compare the video image with all possible places on the map and get an exact location (and obviously could do smart things like only search nearby the last position and spawn points, etc). Once you have a location per frame, it should be relatively easy to generate a video like the one above. BUT WHYYY???

In speedrunning, one of the more fascinating/fun things to do is compare runs. Like the video seen here: https://www.youtube.com/watch?v=b2I0j1_biZU

So I guess a run comparison tool would also be cool, but I think having the full map, with more than one character moving on the same map (think when you finish a Super Meat Boy Level or when you race a ghost in mario kart). I wanted to post here to see if anyone knew of any similar projects or knew of any existing tools that would make it easier (or would be interested in helping). My initial goal would be to do this for Metroid (bc I actually run the game), but could be extrapolated for other games. Thoughts?

The closest thing I can think of is Bisqwit's "animation merger" tool:
https://bisqwit.iki.fi/source/animmerger.html

I don't think that would quite do the job though. I think if I wanted to make "NES Atlast" style stuff I'd write my own tools for it.

As a start I'd record the footage as an emulator input movie so I can play it back in different ways, like in FCEUX you could turn off the sprites and just output the background layer, which might be better for tracking the current map position of each frame.

There's this: https://prilik.com/ANESE/wideNES
Which is an emulator with a similar goal. I confess I haven't had a chance to try it.

Thanks for the links! The animation merger info could DEFINITELY prove helpful. Basically I'd be starting like half way through (already have the static image), but doing very close to the same thing and can certainly learn from some things like the HUD, etc.

And thanks for the WideNES page. That was what actually made me switch from its an idea I'll prolly never do to a.. this might be fun to do...

The one caveat is that I don't want this to be emulator specific. I want to be able to analyze video feeds (ie via capture card) because most top level speedruns tend to be recorded on console. That obviously means some of the matching is a lot harder because the screen size and colors can be quite different, but agree that starting with emulator video (especially if I tuned my emulator to have the same colors as the map) would be a great way to minimize the amount of work to prove the concept.

I'm not sure how much time/motivation I can devote to this idea any time soon, but figured I'd collect any data folks had and see if there was an interest. Thanks!

Haha a friend *just* linked me wideNES completely unrelated to this topic! Was just about to post it. Pretty cool idea!