Companies like Google and Microsoft have been investing heavily in the concept of cloud gaming, where a player uses their computer or a mobile device to stream the video feed of a game that’s running on powerful machine tucked away in a data center somewhere. With this technology you can play the latest and greatest titles, even if the device you’re using doesn’t have the processing power to run it locally.
Considering the Switch is already a portable system, it’s not too surprising Nintendo doesn’t seem interested in the technology. But that didn’t stop [Stan Dmitriev] from doing a bit of experimentation on his own. With little more than a Raspberry Pi 4 and Trinket M0, he’s demonstrated that users can remotely interact with the Switch well enough to play games in real time.
The setup is fairly straightforward. A cheap HDMI capture device is used to grab the video from the Nintendo Switch dock, which is then streamed out to web with the help of the Pi’s hardware video encoder. Input from the user is sent over the Pi’s UART to the Trinket, which itself is running a firmware specifically developed for mimicking Nintendo Switch controllers. With so many elements involved, naturally some latency comes into play. The roughly 100 millisecond delay [Stan] is reporting isn’t exactly ideal for fast-paced gaming, but is certainly adequate for more relaxed titles.
On the software side of things, the project is using a SDK developed by [Stan]’s employer SurrogateTV. Right now you need to apply if you want to get your game or other interactive gadget up on the service, though he says it will be opened up to the public next year. But even without all the details, we’ve got a clear idea of how both the video capture and user input sides of the equation are being handled. For personal use, all you’d really need to do is put together a simple web interface to tie it all together.
This isn’t the first time we’ve seen a microcontroller used to interface with the Switch. Other consoles are a bit more selective about what kind of hardware they will talk to, but the Microsoft Adaptive Controller could potentially allow you to do something similar on the Xbox.
” play games in real time.” … ” roughly 100 millisecond delay” …..
You keep using that word. I do not think it means what you think it means.
I believe the real time is a response can interact with the problem with efficacy, but 100ms is very long time.
And you as well :)
https://en.wikipedia.org/wiki/Real-time
You didn’t have to “interrupt” anon with all that “context switching” ;-)
If I tried to sell a wireless keyboard that had “real time” response of only 100ms I would have a lot of pissed off customers. So yes the word is malliable to the problem. In this context, the word clearly does not fit unless you are playing a game that doesn’t need real time responses.
Have you ever played Pokemon? Nobody is worried about 100 ms delay in a turn-based fighting game.
I like Nintendo consoles, especially Switch although there was a time they cared nothing for casual gamers… When their only experience of humanity was a toilet seat coming at them down a steel corridor.
I’ve just bought a Steam Link controller, it’s “just” a remote desktop to play Steam games. Wish it was compatible with the other gaming platforms from the desktop. Or a Rpi4 to! :-)