The Raspberry Pi line of single-board computers are great little devices, and they can do great things with cameras and video. However, there can be a fair bit of latency involved in these tasks depending on the board you’re using and just what you’re doing. You need to have things set up just so to get peak performance. [MattKC] has tangled with this issue in a personal project, and recently had somewhat of a breakthrough.
The issue came up during [MattKC’s] work on cloning the WiiU gamepad. The idea was to receive the video stream from the WiiU console and display it using a Raspberry Pi Zero 2W. He had some problems with latency, wherein there was a strange 3 frames of latency in the video pipeline that just wouldn’t go away. Even substituting in some dummy frames into the pipeline and ignoring them at output time didn’t work—the latency stuck around. Eventually, [MattKC] realized the delay wasn’t about a certain number of frames—it was about time. About 50 milliseconds, in fact.
Drilling down further revealed that there was a problem in the way frames were being fed into and received from the decoder. The first frame would take about 50 ms to decode, while later frames would take far less—as little as 5ms. However, [MattKC’s] code wasn’t set up to grab frames as soon as they were done, so the lag carried forward. The video explains it in greater detail, and how polling the decoder regularly helped solve the issue. The final result was a Raspberry Pi Zero 2W that could process and display the WiiU video feed as quickly as the original Nintendo WiiU gamepad.
If you dug this, it’s worth going back and checking out where the WiiU gamepad project started, too. Video after the break.
Continue reading “Troubleshooting Video Delay On The Raspberry Pi”








