Sometimes you have a whole bunch of computers that you need to work with, and having a keyboard, monitor, and mouse for each one becomes too much to deal with. There are a multitude of solutions to this problem, but [Fmstrat] went the hacker route, and built their own.
The build is a rather unique way of controlling PCs remotely, but it does the job. A Raspberry Pi 3 is pressed into service as the core of the operation. It’s accessible over IP for remote control. Video is captured from the controlled machines through the combination of an HDMI-to-S-Video adapter and an analog video capture card plugged into the Pi. Keystrokes are sent in a roundabout way, first sent to a Pi Zero over a USB-to-Serial adapter. From there, the Pi acts as an emulated mouse and keyboard to the PC under control.
One caveat of remotely controlling computers over a network is that if things go pearshaped, it can become necessary to power cycle the machine. [Fmstrat] deals with this by fitting a relay board to the Pi 3, which is connected to the reset buttons of the machines under control.
It may not be the quickest, easiest, or industry standard way of controlling remote computers, but it works. [Fmstrat] tells us this build was primarily designed to get around the fact that there aren’t any decent cheap IP-KVM systems, and consumer motherboards don’t support the IPMI standard that would otherwise be useful here.
We particularly like the hard-wired relays for rebooting a machine – great for when a network dropout is stopping Wake-on-LAN packets from achieving their goal. While the conversion of HDMI outputs into analog video for capture is unusual and somewhat costly on a per-machine basis, it’s functional and gives the system the ability to work with any machine capable of outputting a basic analog video signal. With the Pi Zero keyboard emulation and analog video capture, we could see this being used with everything from modern computers to vintage 80s hardware. If you’ve ever needed to control an Amiga 2000 remotely for whatever reason, this could be the way to do it.
We’ve seen plenty of other KVM builds over the years, too – like this low-cost HDMI switcher.
Neat – I’m in the middle of a similar thing using a teensy for keyboard side and Elgato cam-link for the video, which is a bit pricier. After digging a little, though – why does this writeup repeatedly mention *mouse* support? Fmstrat’s page doesn’t mention it, or as far as I can tell, implement it – as far as I know, linux “gadget” support can do one device, not a hub, so figuring out how to do both mouse and keyboard with a single pi-zero would be *really* interesting. (Not faulting Fmstrat’s project, still very useful, but I *am* faulting your misleading writeup. Please try and be a little more careful about not inventing features…)
I’se seen usb mice-leopard hybrids before. It’d be trivial to do so if the origonal code dosn’t.
A mouse-leopard hybrid would be quite a sight indeed! :-)
But in this case, you can do a keyboard and mouse in a single USB endpoint. You just have two different reports, one with keystrokes, and another with mouse movements. Think Logitech unifying receiver, for an example.
I was of course dubious of any “trivial” case that doesn’t actually include examples… but it turns out CircuitPython does exactly this – comes up as both `cdc_acm` and `usb-storage`, so you can write python code to the storage while talking to the Python REPL (or your own code) over the “modem”. That also opens up possibilities for including install isos directly on the same usb device…
Why the USB-to-Serial adapter and not Pi3 UART (TX+RX) wired to Pi0 UART (RX+TX).
Seems to be so it can be easily expanded with a USB hub and additional Pi Zeros.
It seems the idea is he can easily expand to multiple machines with a USB Hub and additional Pi Zeros.
While I do think that a homebuilt KVM over IP solution is a nice and useful project, and this apparently works well for the creators, it really doesn’t feel like a very elegant solution to me (yet). Converting the video signal to analog and then back to digital just seems rather convoluted. As does the involvement of a “secondary” raspberry pi solely for mouse and keyboard emulation… it’d be really neat to have a bit more of a compact solution.
Still, nice hack :-)
I don’t think the firmware (or is it the USB chipset?) in the pi3 supports gadget mode, so you need a pi0 (or to be fair, a teensy or some other tiny hid-with-second port embedded board) to emulate the keyboard at all. And you can get a less convoluted video side – HDMI digitizers for gaming are great, and support higher resolutions – and cost $100-$250 instead of the $40 or so for this set of parts (much credit to Fmstrat for including a *shopping list* in the README :-)
https://www.youtube.com/watch?v=SAvGylthpkU
$40 roundabout
ooh, that’s apparently a straight up HDMI->RTP box with a web interface on it, nice.
You could talk PS2 mouse and keyboard out of the Pi and get a USB ps dongle quite cheaply.
Bluetooth/ radio based keyboard/mouse hardware (I have an infra red unit too) is also a posible option – tho I like wires.
Yup it would be much nicer to use an fpga hat and take the hdmi.
What if the pi acted as a serial console (using usb and voltage shifted uart) and piped the resulting code online, through ssh, on a webserver, etc . . .
Now that I think of it, I saw someone doing it with an esp.
HDMI input solutions for the pi are rare – the only one I’m aware of is the Auvidea B101 (which seems to be not-too-well-maintained given that their pages are still unclear about pricing – they appear to still be advertising a sale that they claim ended in 2015???) and appears to have significant software limitations.
I’d have gone with a teensy for the HID…
I *did* go with a teensy for the HID part of a similar project (couldn’t find *any* chipset that could be two USB *clients* at once, the “use the USB port as HID-client then use the console serial port to pass in the commands” trick is about the only option I could find using off-the-shelf boards – though it helped that I actually had some teensys on-hand from the original kickstarter.) Using a pi-zero for that bit does give you more options for “scripting” the typing side of things, for example you could probably do some clever password-vault tricks on the pi-zero, using standard keychain-tools…
Use a Beaglebone Black instead of 2 Pis. The BBB supports USB gadget mode very well and is directly accessible via ethernet.
Do any of the current-gen HDMI capture widgets have decent Linux support? It has been a little while since I checked, this was for an older project; but at the time the MythTV types had some older capture cards working; some new(though mostly awful and analog only) showed up as mostly normal UVC cameras with a couple of extra knobs to twiddle to do input select; but HDMI capture was mostly unsupported; either “we would but HDCP licensing something something” or just “we don’t care, and it hasn’t been re-implemented by a third party” depending on the vendor.
If that situation has improved, definitely a consideration, using a console at composite resolutions is nasty; but a lot of the HDMI grabbers(when last I checked) both didn’t have any Linux support and required a moderately punchy system if you wanted them to work well.
I’m building a similar system to emulate an expensive IP KVM system. It uses HDMI over ethernet extenders as the HDMI capture and will probably use a microcontroller for the input emulation. You can buy the transmitter side seperate for the ethernet extenders and they produce a nice h264 RTP stream I can redirect to the controlling machine.
Interesting – should be possible to DIY with an FPGA. Another project to add to the list.
Last I saw – not directly, no. All of the USB solutions on the market are either severely outdated/discontinued or not supported in Linux. However a year or two ago HaD covered a guy who reverse engineered some cheap Chinese HDMI extenders and found out:
1) They output H.264 over RTP or raw UDP (configurable depending on exact firmware version)
2) They could be configured to output unicast
I used such a device to feed H.264 to a Pi3, which then scaled and recompressed the stream (the extender itself behaved badly if you set the bitrate low enough to be sent directly to any sort of livestreaming solution, so decompression followed by recompression with the Pi’s encoder was necessary)
https://blog.danman.eu/new-version-of-lenkeng-hdmi-over-ip-extender-lkv373a/
Audio or just video ?
I’ve recently used both the “elgato cam link” and the “avermedia live gamer portable 2”, and they were “plug HDMI cable into target system, plug usb cable into my laptop, fire up guvcviewer, see the target system’s output”. Under ubuntu 16.04, there wasn’t even any configuration needed, they just showed up as /dev/video0.
Part of this is probably that you’re not going to have a PC generating HDCP, as far as I know (certainly not the kind of machines I want a “modern crash-cart” for) so that’s not an issue…
I’ve done HDMI to VGA and then run the video threw a networking cable for the video. Worked great for me.
And for the USB all I’ve done is add a small USB hub in line 2 or 3 times to get the length that I needed. It works great for the TV connected to my Computers about 40′ away. and then the arduino to capture the ALT,CTRL, and what ever key you want to turn the servo on a old VGA switcher that I had hanging around.
I have not tried it yet but I think a ESP8266 would work for the switcher as well to control the servo. And I guess it could do the keyboard and mouse as well. When my setup brakes down or I need more computers to control I’ll look at adding the ESP8266 or now the ESP32.
Not to be too far off-topic, but I use “pies” as the plural of “pi” after a lot of thought and discussion. Pis just looks like a dropped character from something you do in the washroom.
I have a lot of machines, pies and others, here on my homestead that require remote control. I use TightVNC, which is no biggie to install (and nearly obsolete…there are others newer) for that. If I had windows, I could also use RDP and friends. Remmina in linux can talk to any VNC or RDP server, for example. All free and no extra hardware needed. Did I miss something in my quick skim of the article?
I call them Pi’ for plural.
Or Poxen.
So instead of saying to Romeo, “A pox on both your families!”, his dying friend could have just said, “Family poxen!”
How do you change the BIOS / EFI settings through VNC/RDP? Or power-cycle a device that won’t communicate over the network?
Or, for that matter, install the OS from a PXE server?
I would have used this: https://hackaday.com/2016/06/09/hdmi-extender-reverse-engineered/ and some sort of Teensy thing for the usb keyboard/mouse. Can the firmware on those do both at once? Of course there’s VNC and RDP and such, but that stuff doesn’t work until the computer is booted up and running an OS.