Google IO 2019: Stadia team talks about latency in cloud gaming

Google is the newest player in the world of video games and the company has taken a non-traditional approach for the same. The company has entered the gaming industry with a focus on cloud gaming. So, be it a Chromebook, a Chromecast, an entry-level Windows PC or even a smartphone or a tablet, players will be able to play the latest and greatest games over the internet where the processing workload is taken care of by a computer sitting in a room halfway across the globe. At the Stadia launch event, Google also announced a Stadia controller. You can read more about the game streaming service and the controller in our deep dive here. 

At Google, I/O 2019, the team behind stadia shed some light on how it is tackling latency in cloud gaming. The thing is, it is easy to stream content to a user over the internet as the only interaction a user has is with the play, pause, rewind, forward, etc. gaming isn’t as linear. It is a dynamic environment in which the player’s actions, though in a controlled environment, are unpredictable. The biggest problem lies with the solution of cloud gaming – remote processing. Because the heavy lifting is being done elsewhere, there is the problem of latency. For those that aren’t aware, latency is the time delay between a player pressing a button on the controller and the action appearing on the screen. Before you go all pro gamer about latency, please note that there is a latency between a person seeing an image on the screen and the brain telling your fingers to perform a certain action on the screen. According to the Google I/O talk, a casual gamer can be okay with a slightly higher amount of latency when compared to a pro player. A casual gamer may not need the precision of a pro player to enjoy a game. However, there is a threshold beyond which even a casual gamer will be affected by latency. 

Google Stadia Product Manager Khaled Abdel Rahman said, “Latency perception is closely tied to the player’s level of experience with a particular title. A casual player can fully enjoy a sports game without significant latency perception even when we dialled those numbers up to absurd levels. On the other hand, a more experienced player may be looking to do some skill shots, special passes, and things that are not feasible at high or inconsistent latency levels.”

The Google I/O talk about Stadia was given by Guru Somadder, Rob McCool and Khaled Abdel Rahman. During the talk, they highlighted the difference between cloud gaming and traditional gaming. They said, “First of all, it’s interactive, which means that latency requirements are simply too stringent to allow buffering of any kind. We also cannot afford any computationally intensive based stream compression to accomplish quality. The open world nature of games also makes caching of content a priori simply not possible or very difficult. And network impairments can be highly visible and must be avoided at all costs. A cloud gaming platform must maintain an optimal experience under all possible conditions.”

Talking about latency, they spent a lot of time doing extensive research on the effects of latency. Here is a look at some real-world examples of latency. Typical HDMI transfers to your TV take anywhere between 16 and 33 milliseconds. Typical monitor refresh rates are on the order of 4 to 16 milliseconds. USB ports for peripherals are about eight milliseconds. Even in the USB polling case, gaming peripherals provide specialized hardware and software that try to mitigate this latency but is not zero.

Cloud game streaming is a complex system. During the talk the speakers said cloud gaming “has many actors and many variables that are not directly observable”. Talking about RTS games like Warcraft, Starcraft they said, “there’s a mechanic called the fog of war where you can’t see something until you send a unit to go observe it. Cloud gaming is actually a lot like that. There’s a bunch of information, but you haven’t seen it until you go observe it and you don’t know if it’s changed since the last time you looked at it. And just like a real-time strategy game, you have to make decisions all the time. The key is, what are you going to focus on?”

Moving to building games, currently, developers build for the platforms and to overcome the limitations of the platforms that are available to users. With Stadia, since the computing power can be increased to suit the requirement of the developer, such limitations can be overcome.

Google isn’t the only one getting into the streaming game. Microsoft has also announced that it would begin trials of its cloud streaming service, Project XCCloud in 2019. Sony has had its PlayStation Now game streaming service live for some time now but it is available in limited regions only.