Microsoft aims to make streaming gaming better with predictive modeling

Microsoft researchers are developing a predictive model that could make the game streaming services more tolerable. One of the big issues of these services is the latency caused by the total round-trip time (RTT) it takes to send user input to a remote server and receive a frame of game data from that server. The software giant hopes to make this experience more tolerable via a predictive modeling system that tries to predict what you're about to do in-game and generate those speculative frames.

The system promises an experience much closer to that of a game running on a local machine but the downside is that the bandwidth overhead increases by anywhere from 1.5 to four times that of a normal streaming game client. Additionally, it also requires coding changes to games.

By the time those predicted frames get back to the user, the system can see which input was actually entered, then immediately show the appropriate predicted frame for that situation rather than waiting for another round-trip to the server. The DeLorean system also improves performance by "supersampling" inputs at a faster rate than the game normally does, and it applies a Kalman filter to reduce the shakiness of the predicted frames.

Even if the input doesn't match the prediction precisely, the DeLorean system uses a misprediction compensation technique to alter a predicted frame so that it matches what should actually be shown. By sending extra depth and rotational environment information with each frame, the local machine can immediately tweak the predicted image to match the actual inputs; showing a scene a bit to the left if the user turned farther left than expected, for instance.

More details at ARS Technica.