Netplay strategies

Any suggestions for implementing Netplay?

First you have to traverse whatever network address translation (NAT) layers lie between the players. Common methods for this include ICE and IGD.

Once the machines are connected, they send input data back and forth. Over this connection, the machines exchange input data, each with a timestamp saying on which frame it was valid. Because these packets represent real-time data, the emulator disables output buffering on the TCP socket with TCP_NODELAY.

When a button is pressed locally, the emulator does not act on it instantly. Instead, it uses input data from about four frames back, giving time for the other player's input data to arrive. (This delay should be a bit over half the ping.) If input data is late in arriving, the emulator predicts the other player's input (usually by assuming that the input has not changed) and emulates that, but it saves the state beforehand so that it can re-emulate missed frames should the other player's input data differ from the prediction.

Typically you want to use UDP as your method for sending packets for these reasons...


http://gafferongames.com/networking-for ... dp-vs-tcp/


Also most NES games aren't set up to take advantage of prediction since re-emulating the frame really isn't an option in a some games. Especially if more than a few frames have gone by.


The basic structure is

1. Determine number of frames to delay input (typically half the ping rounded up to the nearest integer or +1 frame for margin) on local machine by measuring average ping time over a time interval.
2. Synchronize emulator startup/loading of rom
3. Ignore first X frames where X = frame delay since you can't possibly have any input from the other player.
4. Send input every frame with the frame it is to execute on. Frame to execute = current frame + delay
5. Save own input to buffer for local machine
6. On current frame check get own input from buffer and check to make sure input from other player has arrived. If it hasn't arrived emulation freezes until it does

By the time you've implemented reliable delivery of input packets over UDP, with error detection and retransmission and the like, you'll end up having reimplemented most of TCP. Or what am I missing?


How not? It's not like the NES has some huge @$$ amount of RAM like the quarter megabyte of a Super NES save state. There's 20K at the most because games using SOROM, SXROM, ETROM, and EWROM boards tend to be single-player and turn-based. And with the Wii Menu-inspired nemulator emulating over a dozen NES instances at once on modern PC hardware, speed shouldn't be too much of a problem.


I was describing the technique used by GGPO. In twitch games, I imagine that prediction and reemulation disrupts the experience less than a momentary freeze.

Nothing. You're absolutely 100% correct. And for sake of example, WoW and many other modern games use TCP. The "use UDP because it's fast" mentality is only applicable in one scenario, and that's the handshaking nature of the protocol -- which for large payloads isn't even used any more given how the TCP sliding window algorithm works (RFC 1323): you no longer need to send an ACK for every single packet). TCP selective acknowledgement (SACK) per RFC 2018 further extends this in the case of packet loss.

Use UDP if you want, but you're going to end up reimplementing pieces/parts of TCP like Tepples said. Those reimplementations may be better for your application -- it's up to you to decide and do the analysis -- but it's time not well-spent, IMO.

The biggest problem I find is that the programmers using the underlying syscalls/code for sockets do not actually understand 1) how individual descriptor flags affect what is going across the wire, and 2) have no real familiarity with packet analysis to determine what is going on and correlating that behaviour with actual code in their application. (The latter is somewhat difficult, which is why using a language like C makes this easier -- anything that is abstracted is going to make that task even harder). A common example are packets that arrive out-of-order due to intermediary routers and load balancing on the Internet (and to some degree even NAT routers); there's a common misconception that if machine A sends packets to machine B in the order of 1,2,3 that machine B will receive them in the order of 1,2,3. I will not pontificate on all this this past this point; my statements (especially about the code aspect) sound anecdotal but honestly I'm really preaching fact.

All that said: if you find something online somewhere that actually documents the design and architecture of a realtime gameplay protocol (TCP or UDP, I don't care which), I'd love to read it. I have yet to see anyone publish anything like this. All I've seen are commercial companies writign their own proprietary methodologies (which makes sense to some degree), and open-source nutbags saying "look at some crap I put on github; the code is the documentation" (wrong). Possibly this type of thing is discussed in an actual published game development book. I have a few, but they're all dedicated to graphics, level design, algorithms, and things of this nature.

The buffering methodology Tepples describes (re: the emulator working on input 4 frames behind what's active) is commonly used, but the problem is that 4 frames is sometimes too little, or in other cases too much. It's going to vary based on several variables, absolutely none of which your emulator/game/whatever has control over.

My general suggestion to people is: don't bother implementing netplay in emulators. You're entering a completely different world of completely different pain. And I have seen people implement things (including in commercial games) so utterly wrong that it's baffling.

With those network buffering strategies, there are at least 2 different architectures that could be implemented:

1) Master-slave: The master runs at realtime and the slave has a delayed view of the master. The slave transmits button statuses to the master and they take effect there first. Network lag would only be perceptible on the slave.

2) Symmetric: Both emulators transmit button statuses to each other and each can be potentially delayed waiting for the other. Its a configuration where they are both equally kept in sync by each other. Network lag could manifest itself on either.

Any thoughts on which is better or if there are other architectures? Thanks.

There's also history-changing, where you retroactively apply lagged joystick input. Made possible by logging input, saving state every frame, and running faster than full speed.

1 is also known as server/client.

Remember that you can have more than two players. In that case, server-client has the server take the brunt of the communications load compared to symmetric.

3) Dedicated server
Like 1) except all players are slaves and the server is a computer that isn't playing...probably not that useful for NES-emulators. Has advantage of being harder to hoodwink/cheat; doesn't have to (but can) render nor process input, so had more processing power available...in the days when 3d graphics cards were not the norm.
4) Terminal-mainframe
Like 3, except the only thing clients send/receive is button presses/graphics information. (Or like 1.)

Again, probably not terribly useful in this age nor for NES emulators...though the idea of a way to encode NESvideos particularly would be nice for other purposes...hmm.

Surely there's a library of some sort to deal with the low-level stuff, right?

The design of GGPO is buffering plus history-changing.


I chose four as a starting point because it's an approximation of the ideal buffer depth, which is a little over half a ping time.


If emulators don't include netplay, then how[Digression detected; taken to another topic]


This has at least two advantages. First, NAT traversal can be simpler because the server has a public IP address. Second, the copy of the emulator running on the server can encode the game's output as a live video stream, so long as games' publishers don't use copyright to shut down the streams.


GGPO used to be available as a library by Tony Cannon, but I don't know how it's licensed. And I'm not quite sure how to find out, seeing as GGPO's web site has become a soft redirect to FightCade, a version of the FBA emulator.

Is there an easy way to stream NES video output? If the clients are just remote views into a server, that would make life easier. E.g. only transmit tile, nametable, palette data and PPU register changes per frame with their associated timing.

I've enjoyed emulators with netplay occasionally, especially over LAN, so I think it's not a bad feature to consider for one.


You only need a symmetrical experience if the game is competitive in a way that short scale timing makes a difference. Letting the "server/master" run normally with all the lag/unreliability pushed onto the other players can sometimes be much more straightforward to implement than other techniques. The "slave" has their input acknowledged only when the "master" later tells them it happened.

UDP is often very suitable for networked games, but generally for this case the game engine is built in such a way that data loss is acceptable. If you miss a few seconds of input, the data coming in is enough to place the player where they should be without the game having to go back in time and re-simulate, etc. In the general purpose emulation idea, the game state diverges irrecoverably if you don't have identical inputs, so loss of anything is forbidden. If your game is designed for networked play, though, you have a lot of different ways to deal with lag/unreliability/divergence. If you can recover well from short-term divergence, dealing with lost UDP data is often better than dealing with the extra latency of TCP (all latency causes divergence, unless you halt the game for everybody). It's a question of which method will cause a greater disparity for players, and on what time scale.

(Strategies for networked game design is a whole topic of its own I won't get into here, since we're just talking about emulating, right now.)

In the asymmetrical case, you can at least send save-states from "master" to "slave" as a recovery method if data is lost. (Obviously you can't "merge" savestates, so this isn't an option for fair/symmetrical play. You have to go back and re-emulate.) If you've already ruled out competitive play, you probably might as well go with TCP to make your implementation simpler.

^^^I do not believe there is a NES video format/protocol; that was what this bit addressed:
I think that deserves its own topic, though.

That could work in theory. But in practice, you'd have to develop an app for Windows desktop that receives and interprets this data, develop an app for OS X that receives and presents this data, develop an app for X11/Linux that receives and presents this data, develop an app for iOS that receives and presents this data, develop an app for Android that receives and presents this data, develop an app for Windows Runtime (8.1/10, 8.1 RT, and Phone 8.1/10 Mobile) that receives and interprets this data, and develop an app for each major set-top box platform that receives and presents this data. This is why most people just encode to video and stream it through a service like Twitch: the bandwidth wasted by video's theoretical inefficiency costs less than developing and deploying the native viewers.

I guess I haven't seen how prediction would look on sports game on the nes where things happen every frame. If too many frames go by and the actual input is different isn't it going to look like the game got "rewound"

If the emulator predicts that I didn't press a button to say pass the ball but I actually did press a button to pass the ball and it has to go back and re-do that if too many frames have gone by. Correct? That seems like a worse experience


From that same article on why you shouldn't use TCP

http://gafferongames.com/networking-for ... dp-vs-tcp/







Kailerra which Nestopia and other emulators used to help implement netplay has worked well. It uses UDP to provide a less laggy experience. We've used it for our online NES leagues to great success. It figures out the needed frame delay by taking a ping average or a certain period of time.

It has a server based client as well as direct peer to peer based client (the peer to peer) is the one we use. It does require people in the league to port forward in order to be able to host games. The kaillera C++ source is out there. You can see their reasoning for using UDP in the faq. The typical frame delay is 1-5 frames depending on where the two players are located.

In the dial-up days the frame delay was 11 frames which made for a very different gaming experience as you can imagine.

http://www.kaillera.com/faq.php

Now obviously kaillera was developed YEARS ago so maybe the same logic doesn't apply. The code for kailerra is open source however the comments are VERY sparse in the code and there is no high level document so you are left to mostly reverse engineer it.




I also helped work on getting the netplay working for an updated implementation of NES game on a different platform and most of their netplay options end up sending the information via UDP either in unreliable or reliable mode. I'm currently not at liberty to post the code for proprietary reasons but I don't think it would be much help for the emulator. It used the same basic principles I outlined in my first post.

Usually taking a measurement over X seconds before you load the game provided the connection you are on isn't experiencing heavy packet loss will give you an accurate measurement of the typical delay.

In peer to peer, its typically ping/2 rounded up to the nearest frame integer. Given the small amount of data that needs to be sent this is usually good enough 99% of the time and not a very difficult calculation to do.

Sorry last post...

For game that require near frame level responsiveness you don't want the extra delay that TCP by its very nature demands unless I'm understanding things wrong... you ALWAYS have to wait for the acknowledge. Versus using UDP you just have ask for a re-transmit on the rare times the packet does not arrive. On good connection packet loss should be very low. So yes you need to implement a good part of what TCP does but its worth it for twitch type games.

USing UDP with some minimal checking for re-transmit input delay is:

ping/2 +1

With TCP always requiring and acknowledge its

ping +1

Say I'm playing someone on the east coast from the west coast and I have an effective ping of 130

At 60 FPS With TCP I'm looking at 130ms/ (16.6ms per frame) +1 = 9 frames

At 60 FPS With TCP I'm looking at (130ms/2) / (16.6ms per frame) +1 = 5 frames

4 frames may not seem like much but in a sports, FPS, twitch type game its very noticeable.


If you just expect netplay to be used for LAN then TCP is fine as it would be a 1 frame difference.

In practice, changing the past feels closer to lagless than an occasional freeze.


If your opponent launched a rocket at you a second ago, then you do care about what happened a second ago. But if you're not seeing the other player's keypresses for a whole second, then your pings are in the satellite range, and you'd have problems with SSH as well.


So what happens if the player presses a button during a momentary freeze?


TCP ultimately requires an acknowledge, but several packets can be in flight at once so long as they fit in a receive window. As I understand it, each packet is added to the receiver's stream once it arrives at the receiver, which is ideally half a ping after it is sent.

Another good article on the subject of TCP vs UDP


https://1024monkeys.wordpress.com/2014/04/01/game-servers-udp-vs-tcp/