TinyPilot Provides KVM-over-IP, With Low Cost And Even Lower Latency

Remote access is great, but if the machine stops booting, ceases to connect to the network, or needs low-level interaction like BIOS settings or boot management, remote access is worthless because it’s only available once the host computer is up and running. The usual solution is to drag a keyboard and monitor to the machine in question for physical access.

Ubuntu laptop (right) being accessed over IP, via web browser on the left.

For most people, swapping cables in this way is an infrequent task at best. But for those who work more closely with managing hardware or developing software, the need to plug and unplug a keyboard and monitor into machines that otherwise run headless can get tiresome. The modern solution is KVM (keyboard, video, mouse) over IP, but commercial options are expensive. [Michael Lynch]’s TinyPilot on the other hand clocks in at roughly $100 of parts, including a Raspberry Pi and USB HDMI capture device. It does have to drop the ‘M’ from KVM (meaning it does not support a mouse yet) but the rest of it hits all the bases, and does it all from a web browser.

What exactly does TinyPilot do? It provides remote access via web browser, but the device is an independent piece of hardware that — from the host computer’s point of view — is no different from a physical keyboard and monitor. That means keyboard and video access works before the host machine even boots, so even changing something like BIOS settings is no problem.

[Michael] demonstrates his design in the video embedded below, but we encourage you to check out the project page for a fascinating exploration of all the challenges that were part of TinyPilot’s development.

Interested? Make one yourself, or as an alternative [Michael] has made a parts kit available. TinyPilot doesn’t provide an interface to the host machine’s power switch, but if you need to add that you can use this other KVM project’s method of integrating a relay module with some DIY of your own.

43 thoughts on “TinyPilot Provides KVM-over-IP, With Low Cost And Even Lower Latency

      1. They did, it is called Intel vPro and people hate it and the rest of the Intel ME (management engine) because it is a HUGE backdoor if not properly configured, and in 95% of the places I’ve worked it is rarely configured at all which makes you vulnerable to somebody plugging in a USB and configuring it FOR you to give them persistent system access.

    1. They present as a USB Video Class device, and therefore use the standard UVC driver. Look for the Twitter thread referenced in the linked page (Arsenio Dev) for a teardown. I believe the chip was unmarked, however.

    1. It is using the Pi4’s USB Device Controller to appear as an arbitrary USB device, in this case a keyboard. I think he just hasn’t figured out how to capture mouse movements in the browser, and translate those to USB reports. The second part is simple, I’ve got code linked above that does it. The first part is why I chose to use VNC rather than making up my own protocol: availability of MANY client implementations!

        1. MaMe82’s P4wnP1 has implemented a USB HID Absolute Positioning Device (not sure of the exact term, but close enough) that would be a precise match for VNC. My code subtracts the previous position from the current one to obtain a relative movement to emulate a more conventional mouse. Obviously there is a fairly big jump if the mouse exits the VNC window, and re-enters from a different direction.

      1. The USB Device Controller is already built-in to the Pi4, and several other inexpensive SBCs. You’re going to need something fairy capable to deal with the HDMI stream in the first place, of course! Might as well be a Pi 4, or a BeagleBoneBlack, or an Orange/Nano Pi

      1. I did a quick newegg check and saw at least one $11
        “VGA to HDMI Adapter/Converter with Audio (Old PC to New TV/Monitor with HDMI),”

        So that might also work to turn VGA output into HDMI output, which the HDMI dongle could then capture.

  1. Is this fast enough catching up when changing display modes?

    When I need a KVM adapter, it is usually to enter the BIOS, select another boot device and similar stuff. When rebooting, most systems tend to change graphics modes in quick succession, also you just have short timespans in which you need to hit the correct key sequence to abort the regular boot and go into the bios menu or boot menu.

    I have tried several other homebrew solutions and hdmi capture modules. Unfortunately they either take 3-5 seconds to change the display mode (in which time you don’t see anything) or even completely disconnect on mode change (Elgato Camlink 4k).

    1. If it don’t work how you like, hack it…. have you tried sending screenlock/pause every couple of seconds or something? To slow it down until the mode change takes.

      One monitor I have is a bit like that, it’s goes “Oh, you’ve changed the screen mode, let me just show you a scrambled corner of it a second while I realise that, now I’m going to blank the screen and pop up a ‘screen mode change’ box from my internal UI. Now I’m going to display a couple of scrambled screens and make some weird clicks while I hunt through screen modes, then I’m going to go TOTALLY black for a count of 2 making you think the weird clicks blew me up, TADA!!! there’s the screen mode you wanted, oh wait, shuffle it left… TA…ooops, shuffles it down and up… whew, finally, there you go.” ….. and it happens at about the speed that takes to read.

  2. I had the same idea to use this HDMI->USB dongle for KVM-type applications.

    But when I got the HDMI->USB dongle, I had an interesting first experience with it.

    I plugged it into a Windows 10 PC and tried to use VLC to ‘play’ the capture device.

    It displayed the boot screen output, but it was unreadably blocky and low-res. I figured the device was just unsuitable for lower-than-1080p captures.

    However, after seeing this example of it actually working properly, I tried the Windows 10 built-in ‘Camera’ app, and the output looked *great*.

    It was a night-and-day difference.

    I’m guessing it must just be an issue and/or setting on the VLC app, but I’m not sure if anyone knows how to get VLC to display the dongle output at a readable resolution.

    1. First thought is a codec issue, I’ve got the impression that VLC doesn’t by default use proprietary codecs, just free implementations maybe… possibly an updated one or a plugin or something might fix things.

  3. It’s a nice intellectual exercise, but it doesn’t scale. You can buy a one user 16-port IP KVM (where the mouse works) for $1,000 or $63/port. A BOM of $100 for a DIY solution without the mouse isn’t reasoable.

    1. Use a cheaper SBC, and the price per port comes right down! e.g. any of the OrangePi/NanoPi boards with a USB Device Controller should do the job just fine, if all they are doing is relaying MJPG to a browser, and those SBC’s are available from $20 or so.

      Or even, get a slightly more capable SBC, and more HDMI dongles per SBC. I’m pretty sure the Pi 4 could support 2-4 active HDMI dongles at once. Then you are talking about $35 for the cheapest Pi 4, plus $12 per port for the HDMI dongle, plus another small amount for e.g. an atmega32u4 or CH552, etc to implement the USB Device Controller per port, connected via I2C or SPI to the host SBC.

      If you want to be fancy, the MAX3420E even has a proper Linux kernel driver, so it could be a fully supported UDC. Not sure if the driver (or Linux) supports having multiple concurrent UDC’s, though. And the MAX3420E is pretty expensive, unfortunately, around $10 per chip!

      And mind you, for each HDMI dongle and UDC you have, those are essentially “seats” that you are adding, or “users” in IP KVM terminology. i.e. these can all be active at the same time, so you could have 2 users each accessing a different system without stomping on each other. A lot of the commercial IP KVM implement this sort of thing with a single capture device, and single UDC, and then a big HDMI switch to choose which system it is connected to. i.e. a “single user” device.

      All of which tells me that a lot of the commercial IP KVM players have been taking their customers for a ride for far too long!

    1. Yup, been using this one for a while…seems fairly mature, rock solid, and has a fairly polished web interface. Only required hardware is the rpi4 and the HDMI csi-2 board and you can be up and running with keyboard and mouse support within an hour or two. There’s also support for wiring gpio pins to remotely hit the physical power switches on the machine, or read power/HDD light status, and the emulated CD for transferring files is a nice touch.

      As an aside, for those that say it doesn’t scale, supporting additional machines would be a simple matter of hooking an HDMI and USB switch up to the gpio pins, then just setting which is active to transfer control between them.

    2. Yup, been using this one for a while…seems fairly mature, rock solid, and has a fairly polished web interface. Only required hardware is the rpi4 and the HDMI csi-2 board and you can be up and running with keyboard and mouse support within an hour or two. There’s also support for wiring gpio pins to remotely hit the physical power switches on the machine, or read power/HDD light status, and the emulated CD for transferring files is a nice touch.

      As an aside, to those who say it doesn’t scale, supporting additional machines would be a simple matter of hooking up an HDMI and USB switch to the gpio pins and selecting corresponding inputs on each to transfer control between machines.

      1. I’ve found Belkin 4-port HDMI switches that have a little dopey remote control, so you could probably put an IR LED on the GPIO pins and control it that way.

        -or-

        The Belkin HDMI switches I have come with what looks like a fairly common “LED-looking sensor in a cage” IR receiver on-a-cable that plug into them via a headphone-jack.

        So if you know what voltages those IR sensor modules usually take, you could probably do:

        RPi->GPIO->(some diodes/resistors)->Headphone plug->HDMI-Switch

        I want to do that, but I’m not quite versed enough in the voltages involved to make it sound easy enough for me to do in a single attention-span session.

        I also kinda wonder if you could pass USB KB/Mouse signal from/to each PC across extra HDMI-port pins in the HDMI switch, but I don’t know what beyond “minimal # of switched pins to even be called an HDMI switch” those switches actually pass.

  4. maybe this will be helpful. Because I am remote working and have a 34 inch curved 4k screen(emphasis onlots of screen real estate), i wanted my works IT to install Microsoft mouse without borders which would allow me to use both my personal pc and work with same mouse and keyboard. they rejected the request due to the nature of server/client relationship. Mouse without borders maybe a solution for you. my work around is using chrome remote desktop on my work laptop, connected to my work laptop in dual monitor mode. That way i can switch to my personal pc also connected to the monitor. mouse and keyboard are still connected to work pc.

  5. That dingle is $8.07 on eBay, but for $5 more you can get the USB3 version with 60fps instead of 30fps.

    And how about wiring some GPIO pins into the hardware reset switch of the PC for forcing reboot on hung systems?

    1. those are SCAM

      There are real 60fps USB 3.0 grabbers on ebay, $65 Wiistar USB 3.0. older bigger multi chip design. You can find them on aliexpress at $25 so might still be worth it.

  6. while writing this I remember that I had the dongle and an orange pizer0, 2hrs to download the image and
    update os, plus the instructiosn from https://github.com/pikvm/ustreamer for debian and it works (with some problems).

    I conencted the USB and it gets recognized, but it does not work, since I also had the “Video capture with loop”
    (and yes is the same chip insede) , because it requieres more power it has a secondary usb for power and it has
    a nice tv color palet screen when it does not recive any imput source…
    that was just a simple test but if you can manage to power sbc from GPIO you coud use the ultra cheap
    NanoPi NEO-LTS that starts 10USD and if those guys develop a hdmi capture hat with external powersource
    they would have a nice product.

    else you would need to have external power supply, and for that you can go with:

    aliexpress.com/item/32640565321.html (25.14)
    aliexpress.com/item/4001115514333.html (8.49)
    aliexpress.com/item/33038355026.html (3.90)
    aliexpress.com/item/4001058817302.html (0.98)

    the rest is software configuratio, and since that was all ready done, it’s just time

    One last note, the CPU got really hot, so heatsink is crucial for this… that make the friendlyarm a better
    choice in my mind (not that I have tried their products)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.