Downloading Data Through The Display

HIPAA – the US standard for electronic health care documentation – spends a lot of verbiage and bureaucratese on the security of electronic records, making a clear distinction between the use of records by health care worker and the disclosure of records by health care workers. Likewise, the Federal Information Security Management Act of 2002 makes the same distinction; records that should never be disclosed or transmitted should be used on systems that are disconnected from networks.

This distinction between use and disclosure or transmission is of course a farce; if you can display something on a screen, it can be transmitted. [Ian Latter] just gave a talk at Kiwicon that provides the tools to do just that. He calls it ThruGlassXfer (TGXf), and it does exactly what it says on the tin: anything that can be displayed on a screen can be transmitted. All you need are the right tools.

How is [Ian] doing this? With QR codes, strangely enough. [Ian] has designed a protocol and application that allows people to download files through a screen. By using TGXf, anyone can load a file stored locally on a computer, have the binary data displayed through QR codes, and record that data with a smartphone or tiny video camera. This video is then analyzed, the data is recovered, and the file is transmitted, defeating all security measures a sysadmin has in mind.

ThruKeyboardXfer (TKXf) keyboard stuffer
ThruKeyboardXfer (TKXf) keyboard stuffer

Displaying binary data as a QR code presents another problem. How do you put an application that will convert raw data to QR codes on a locked-down system? That’s another trick up [Ian]’s sleeve called ThruKeyboardXfer (TKXf). This requires a hardware device to emulate a USB HID keyboard, pushing data up to a computer simply by emulating a keyboard.

TKXf encodes binary data that are sent out the serial port of one computer (or smartphone) and enters them via the keyboard of another. Either a single file (i.e. an app that encodes data as a QR code) or a continuous stream of data can be sent into a computer through the a USB HID keyboard interface.

For a demonstration of his system, [Ian] put up a video of a smartphone downloading a PDF from YouTube through a laptop screen. The only requirement for this file transfer are pointing the phone directly at the screen; no WiFi or cellular network is necessary to send data from a computer to a smartphone.

If this sounds like something torn from the pages of a yet-to-be-written [Cory Doctorow] YA novel, you’re probably not far off: nearly all official recommendations for security and privacy controls, including publications published by NIST, place a distinction between use of a file, and distribution or disclosure of a file. There is a marked difference between displaying information on a screen and sending it over a network. By transmitting binary data through a display, [Ian] has kicked that door down, turning every monitor and every employee into a security risk.


Thanks [Roman] for sending this in.

75 thoughts on “Downloading Data Through The Display

  1. A little sensationalized, no? If an employee wants to break the law, and reveal confidential information, they hardly need to resort to such a suspicious-looking method that would immediately raise questions with his or her supervisor.

    1. Yeah. The short version is “If somebody malicious can view information on a computer then they can possibly steal it!”.

      The USB HID dongle for the QR app is variation on it but again, you’re using a computer that allows you to plug in random devices and run arbitrary software.

      I’ve seen people taking photos of a screen to grab the information or the basic “look at it and remember it” method.

      “every employee into a security risk” <– employees already are security risks.

      1. “you’re using a computer that allows you to plug in random devices ”

        Every computer allows HID devices by default, ive never come across one that doesnt, without it, USB keyboards and mice wouldnt work.

        As a method of getting stuff onto the computer its pretty good.

        Open a text document, plug in the HID device, and it will transmit an file into the “locked down” machine….
        Its clever…

        1. Some places install software that explicitly blocks drivers being installed for any new device inserted. Other places will just physically restrict access to the USB ports by either locking it up or gluing them.

          1. But those would hardly block a regular USB keyboard, would they. A regular keyboard in the regular keyboard port are all those would see.

            Blocking software from running outside certain (more or less) locked down directories is another thing though. ;-)

        2. Hurting computers with a malicious HID devices is definitely a hit this season. It appears USB protocol lacks (it always has, but it just didn’t matter before) cryptographically secured pairing procedure which allows both parties (a host and a device) to establish a trust relationship before accepting any data/control from one another.

          Of course attaching the first keyboard and/or mouse to a system poses some chicken and egg problem.

          1. Cue the DRM’d Keyboards folks… I can see it now, ‘Your keyboard has been used the maximum number of hours per your purchased allotment. To continue using your keyboard, please go to our website to purchase more hours…’ Anybody remember the DRM chair? Yes it was an art piece, but it was making a point. Lock down keyboards with crypto keys, and its only a matter of time before this happens…

        1. And then plug in a USB barcode scanner (the old CueCats were great for this) and use blink patterns that emulate the bright-dark-bright sequence of scanning a barcode. In and out, through the keyboard port!

          Bonus points: It’s optoisolated. :P

    2. actually it is a little more insidious. Our agency allows limited public access of systems that can access data that is proprietary or confidential. Sure the only people supposed to access it are the individual whose data it is, but some people have nothing better to do, (or some reason) to get roomates/ex’s/whatever’s access credentials and copy the rest for whatever nefarious nogoodnik plot they may hatch. Custody issues, revenge, mental illness (which may be redundant) whatever-we’ve seen it all. The more data -both personal AND case connected (other individuals not actually involved in a given case but recorded for federal/state requirements) we are required to put into databases, along with the increased cry of “make it convenient!” from all levels of bossdom, the more stuff there is that can be abused.

      Problem here is, like most security issues, is physical access. When you’re required to make data publicly accessable, and management does not see the need to physically enclose the PC, insist on disconnection of external USB ports, or have enough personnel with the time and attitude to keep a constant eye on potential sneaky clients (kids crawling under the desk, just being uncontrollable, or a diversion?) is a recipe for disaster.

      One that, as far as we’re aware of in the rank ‘n file, has fortunately not occurred yet.

      Or DID it? Doom doom DOOOOOOOM! :)

  2. Accuracy and speed is probably the key factor here, since that same smartphone could simply photograph the plain-text content of the screen, assuming the data being stolen were patient records.

    True, but I’m a lot more afraid that federalizing medical care means those records are open to nefarious government activities much like the IRS has been exposed as using its records under the Obama administration. Never forget that the violation of your privacy by a powerful government is likely to be far more dangerous than any private abuse.

    This demonstration suggests another scary possibility. It’s already been demonstrated that ultrasound from a speaker on one computer to a microphone on another can be used to transfer data over an otherwise air-gapped computer. Modulation of screen brightness at a rate faster than human eyes can follow might accomplish much the same thing.

    1. I like the way you think, those are cool ideas. And I agree, people seems to forget its governments that caused atrocities like the holocaust, the slaughters of Soviet Russia, and many many other massive killings. People seems to forget the largest corporation in any country is always the government, usually on the order of 1000x of times larger and throwing around that much more money. The US government alone spends 4 Trillion a year, much much much larger than any other corporation that I know of, and people are people, if corporations are super greedy, super corrupt, nepotism, plutocracy, cronyism, etc, etc, why would government be any less.

    1. Depending on the camera system you can build a HD Parallel Bus (is it a good solution; certainly not).

      Nice thing would be filsharing a binary video file through youtube with this method :-D

  3. The real security breach is intercepting data intended for a display screen which could easily be reassembled into human-readable documents/images. If memory serves – this goes way back – this sort of thing was demonstrated on very early PCs (which admittedly dealt with much less complex data sets going to the screen).

    The rest (though it looks very cool and would be fodder for the next James Bond flick) seems only intended to obscure the data a bit so that most people wouldn’t recognize the type of data as it’s being transmitted. Anybody remember the Cauzin Softstrip in Byte magazine? Anybody? Anybody….?

    http://i.imgur.com/nIDv0ME.png

    http://en.wikipedia.org/wiki/Cauzin_Softstrip

    1. It cost as much back in the day, for the strip barcode reader as it did for a floppy drive, so I spent my hard earned pennies on the latter (and made a case for it out of folded aluminium). The bit density of Cauzin Strip is not mich higher than punched paper tape, perhaps a couple of orders of magnitude… to encode a 1.44Mb floppy you would need about (rapid back of envelope calculation coming up… so probably wrong) 3.6 square meters of paper.

      1. How plausible is it to have something like a Cauzin Softstrip, but higher density to be read off by a smartphone perhaps? Or is it still better to just have a sequence of QR codes?

  4. It is a neat hack, but transferring data through the monitor isn’t exactly new. There are various one-time-pad generators commonly used for securing things like internet banking that use this exact method to transmit the challenge code to the security token and to maintain it in sync with the application. This has been around since at least 1999 or so when I have seen it in the bank I was working for back then. Of course, those were transmitting just a short string of numbers and didn’t use a camera but a phototransistor that you held against a flashing box on the screen. I believe those were the RSA SecurID tokens, but I could be wrong there.

    It went out of use with the advent of LCD screens, those were way too slow to flip the pixels compared to the CRTs and it was making the transfers unreliable. I guess that is why he is using the QR codes – better robustness.

  5. Users are by definition the biggest security risk. Furthermore if the user is in a position of trust, then they are far more of a security risk. Someone on a “secure” PC with limited access to a system is unlikely to be able to substitute a data tape, and steal the entire enterprise, however a disgruntled sysadmin certainly can. Its all a question of scale.

    This kind of hack might give the user on the “secure” PC more access than they would otherwise be allowed, but the big hacks usually involve more subtle tricks, often with more social engineering than software engineering. This looks like an interesting variation on a theme, but bear in mind that data has been leaking from your screen since the days of CTRs and VSU terminals.

  6. If the computer has an accessible USB port, you could just push back the data through the same USB HID interface. Even if it is very tightly locked down to keyboard-only, you can get several kbps through the capslock/scrollock/numlock led bits.

    At that point, why not build it into a keyboard, which will type up the transmission program and run it on the background. If anyone sees, say that the keyboard was “behaving strange” and you replaced it with one you found in the closet. Then a few hours later replace it back.

    1. The smart hacker would have identified the kind of keyboard needed in advance, procured an identical one and done the substitution when everyones back was turned…. furthermore the attack keyboard would be doing its magic on someone else’s PC, so they get the blame when security turns up…. errr… allegedly…. assuming you were to try this sort of thing… which naturally you wouldn’t….

      1. … besides… as anybody who has secure access to anything will tell you secure means you dont get to carry a big mess of wires or a hacked keyboard with you to work.. in fact most secure computer resources dont let you take your phone, keys, anything not on the list of banned items, and subject you to metal detector scan before you get past the first electonic lock…

        1. Secure access IME also means personal phones to be deposited in a locker by the door and discovery of unsanctioned NV storage of any sort means you’re fired (or worse).

          Impressive hack though. Kinda reminds me of that hack for getting a surreptitious two way data link through a submarine hull (by ultrasonics most likely).

          1. @nes – Yes a NLJD detector at the door will stop any electronic gadgets too. But they are expensive unless you know how to home-brew one (servv89pn0aj(dot)sn(dot)sourcedns(dot)com/~gbpprorg/mil/non) [replace all (dot) with periods first.]

            A device to communicate from a modern nuclear sub to the outside world is a bit problematic. Not only do you need to get by the security officer with his NLJD device, you have to get by the Comm Officer who would detect it as soon as you fired it up. That’s not to mention that all nuclear subs have a special outer coating which your signal could not penetrate unless you found the special exterior screen door to go outside while underway to send your signal (Screen door is an old Jeff Foxworthy joke! LOL).

            One little nefarious device is “the buzzer”. It is a ultrasonic device that makes a ultrasonic signal while water passes through it’s tiny propellers. It has to be place on the exterior hull somewhere and it makes the boat noisy as hell to passive sonar detectors. It will drive a sonar mate insane until he finds it. And he will if he’s any good.

        2. Little, if any of that security actually exists in hospital settings, where doctors, nurses, nurse aids, interns, patients, guests, security officers, janitors, IT staff, clergy, etc etc etc… all have access to patient rooms, and can easily swap out the “official” keyboard, with their hacked keyboard. If the target was specifically HIPAA data, there’s your weak point, the “convenient” terminal in each patient’s room.

          1. My local hospitals all use laptops on personal rolling carts. The carts are like tall desks. They never leave the side of the user. They wheel them everywhere. When they do leave then unattended it’s usually near the nurse’s station where security, nurse’s, other personnel, and CCTV can clearly see what’s going on. I recommend removable plastic or cloth privacy shrouds when left unattended. You’d better have a darn good reason to be looking under the shroud if you’re not the authorized user.

            It would be easier to just wear a button camera or camera glasses and just video the laptop transactions while standing right next to the screen. A good analyst can figure out what’s on the screen taking screen shots and stuff. Believe it or not Walmart sells stealth camera sunglasses in the toy section that will fit an adult head for only $40 USD. They look really cool too. However, wearing sunglasses inside seems a little suspicious. They use a micro sdcard or usb cable to download images or video. You could dangle them from behind your head like some hipsters do, or prop them on your forehead. You’d have to do some pretty weird head movements to capture anything though.

            I like the car key-fob camera myself. You can make gestures like your pointing at something and video just about anything.Or just have it in your crossed arms in between your fingers sticking out. Who ever suspects a person carrying his car keys openly? Brookstone has the stealth camera writing pen but I know someone that got busted by a woman with that once, because she recognized it from the store. Who has a fat writing pen in their shirt pocket anymore?

      2. or dropped on a USB wireless dongle for their keyboard, and hoped the system installs it without any screen messages. :) then type away, make the next client think “there’s something wrong with the computer” and while they walk off to try to find someone to help them and leave the system open, hackhackhack

      1. @F – That’s called “Eidetic memory”. It is very rare but quite usable in espionage scenarios. However, it is only good for a few minutes of stuff and usually only found in children from 6 to 12. Only adults like Sheldon Cooper of the TV show The Big Bang Theory claims to have it but comically prove false in the show. Johnny Mnemonic is a sci-fi myth.Elizabeth Stromeyer was the only one to show promise. But even she doesn’t seem to really have it after experiments where done at Harvard by her husband in 1970’s.

    1. If you’ve already built custom hardware and connected it to the computer in question, then you might as well use said hardware to copy data in both directions. Of course, there are less nefarious uses for QR-code data download; for instance, anything where you want to easily configure a phone app without involving a central server.

    1. Sadly that relied on CRT signal timing. I recall they made an adapter for laptops, which was sort of an LED cradle that was driven by bitbanging a serial port flow control line, probably DTR. Anyone still got one of those to tear apart? It occurs to me with a GPS-locked clock now, I could make a Datalink sync cradle…

  7. This type of exploit is easily mitigated by restricting what executables the user can run. Even if the user is able to save the transmit program as approved-program.exe, the checksum won’t match and any decent security management system will reject it. As for the TKXf, depending on the policies in place, it may have to match the USB vendor and product identifiers exactly for it to be accepted or risk alerting infosec of an unauthorized device. However, that is all dependent on getting your device(s) by security at the entrance to start with.

    I will give Ian credit for thinking outside the box and coming up with unusual methods of getting data on and off of a system.

    1. But can you run a macro-enabled Office Document?

      I have generated layeres of QR codes snippets in Excel before using manually enterred data and that even included a layer of simple encryption (used to randomise start condition) without any Visual Basic.

      Similar to Random name’s suggetion below.

    2. I’ve always circumvented restricted PC’s like that by finding a way to run DOS executables or like a batch file. You could type the batch file in NOTEPAD and then save it as a .bat file somewhere where bat files are normally launched by the system like the old autoexec,bat on older systems. I think it is still used however in newer systems. If you can get the RUN command in Windows to come up then you can simply type CMD and [Enter] to get a DOS screen up. You could also double-click CMD.EXE in the system folder too – or your saved batch file too. Then you can run almost any executable you want. I don’t think security management systems pay much attention to antiquated DOS any more – it’s like paying attention to Morse Code on a secure radio system.

  8. There are ways to lock down Windows to only allow specific programs to run and to hide or disable various functions and features.

    For 32bit XP and Vista, Microsoft had Windows Steady State. Set it up and lock it. Then upon rebooting all differences from the locked state get wiped out. Microsoft didn’t make a 64 bit version and the 32bit version doesn’t work with win 7 or 8. It’s been discontinued.

    Back in the 3.1x years there was a kiosk mode setup to lock Windows to launching directly into a single program, and exiting that program would just relaunch Windows right back into it. The basics of it were replacing program manager as the shell program with a different one. It also blocked quitting to DOS. Using a keyboard minus Ctrl and/or Alt blocked using Ctrl Alt Del to reboot and stop the DOS bootup while processing config.sys or autoexec.bat. (Note that most PC based kiosks still use missing keys keyboards to block this reboot/logoff attack.)

    Many PCs designed for business use allow disabling individual or groups of USB and other ports. I have some Dell Optiplex boxes that can disable the front pair of ports, the rear two or the rear four ports, plus the LPT and COM ports. Every built in port can be disabled, but then it would need some kind of expansion card to communicate with the outside world.

    1. One thing to note with the steady state approach (my employer uses Deep Freeze, which pretty much is a variant of this) is it doesn’t inherently block you installing anything, it just “saves” your OS image from being destroyed over time by users. It doesn’t do anything for your network security – in fact it makes it more awesome for hackers in that when you’re down pwning the kiosk, just reboot and the evidence is gone! =D

    2. Yes I think the reboot/logoff attack with autoexec.bat/config.sys is still pretty effective. However, if you take away my CTRL ALT DEL what’s to stop me from just pulling the plug and letting it cold reboot? A locked cabinet won’t stop a paperclip in the nearby outlet to trip the breaker. When the maintenance guy resets breaker the newly keyboard modified autoexec.bat or config.sys is waiting to be auto executed.

  9. Someone remembers ZX Spectrum game binaries were broadcasted over FM public radio?
    So this a very old technology, just in modern form.
    We were copied ZX Spectrum games over the phone with “no technology”. Just a cassette recorder and a phone.

  10. Years ago I had a tiny little “PDA”, mostly it was an address and phone number storage plus a calendar or somesuch. It had a photosensor and an LED on the back.

    The Windows 9x software would put a black square on the screen to align the device, then it would blink a spot under the sensor to upload data, which was easier than using the tiny buttons on the gizmo, which IIRC was not much more than 1″ square. For some reason the uploading software either wouldn’t work at all with LCDs or would only work with a few.

    The extra nifty feature was how the sensor and LED had opposite angled bezels so two of the things could be held together to exchange data.

    1. See my comment above – the LCDs of the era were way too slow to flip the pixels and that was ruining the data transfer. That basically killed this method for consumer level devices.

      Today’s LCD could probably handle it, but there are easier and faster ways to transfer data now.

      1. I always thought that it was a matter of light wave-length transmission, not pixel flip-speed.
        a CRT has a significantly higher light output with a much wider bandwidth then the standard LCD… so the photosensor needed to recognize the difference between “off” and “on” states from a CRT would not need to be as sensitive and therefore likely be cheaper (at the time) than one that could reliably detect the LCD “on/off” states.

        I had a timex that read data from a crt and if I remember correctly it was triggering off of IR, which the LCDs that I had access to did not transmit at high enough levels to trigger recognition. iirc, there was an optional plug-in dongle with an IR transmitter if your the watch couldn’t detect your display.

  11. Over the last year and a bit, you’ve managed to make hackaday more and more like a low-quality tabloid newspaper every day. This was honestly one of my favourite sites and inspired me as a teenager with the no-fluff approach to tech. I now read less and less often, and honestly sometimes find things on here a bit distasteful. You generate faaaar more content, but it is of much lower quality. It now seems like you’re crafting posts to fit facebook and twitter more than anything else.
    The line between “hacking” to improve gadgets and gizmos we own and inventing things with parts that people had lying around, – and – l337 hax0r culture has been totally blurred on here, and your site is making it worse.

  12. Actually if you had access to the TEMPEST system (like the NSA and et al) have, which exploits the Wm Van Eck Radiation published in 1985, you would not need such a system to gather intelligence from a PC or laptop. Of course the USG already knew of this in WWII with teleprinters.

    Also the old dial up modem can be exploited with a telescope and a photo-diode by simply aiming at the activity LED on the front panel from across the street. But this is nothing compared to the newest system developed this year called the AirHopper (of course devised by the Israelis ever vigilant at spying on their “friends”). It has a range of 1-7 meters and bandwidth of up to 60 bps. It somehow uses the PC or laptop’s monitor via Trojan Horse (e.g. Stuxnet) to create RF signal in the FM broadcast band to send data to a remote cell phone with FM broadcast receiver installed. Some cell phones do have that built in. Or you could just use a standard FM receiver or SDR dongle on your netbook.

    http://www.slideshare.net/mordechaiguri/air-hoppermalwarefinale?redirected_from=save_on_embed

    I’ve used a much simpler method once at an old Fortune 50 company I used to work for in IT department. We just put a DOS batch file in the global login scripts on the corporate file server (never touched the targeted computer). When the targeted computer logged in in the AM, while he was drinking his morning Joe, it also loaded our spy script. It pulled up all of his Internet cookie and history files and automatically FTP’d the text file to our secret FTP server on same network. We found that he was visiting porn sites and we presented them to him as a practical joke – which he didn’t think was very funny. We didn’t fire him we just wanted him to appreciate BIG BROTHER is always watching. Later a real server app was installed to curtail this type of improper use of company assets and promptly caught our director of security and he was fired! :-)

  13. Your last paragraph, and the point you try to make is … kinda lame. The reason for laws like this is that you can’t stop people from stealing info that they need access to do their job. Thats why they have to make it illegal. Oh and if you do this your very likely to lose your job, no matter how much you try to argue that you only displayed the info on the screen. Unless you hire Jonny Cochran to defend yourself, you clearly are transmitting data to another device (networking) for unauthorized use.

  14. @OtherPeople – I don’t think Brian’s point was lame at all. I do not think he was saying it was “legal” to exploit this method. I think he is only saying that someone is thinking outside the box with this exploit and the laws need to play catch up now by specifying more details as to what is an actual violation.

    Any way you look at it someone needs to put something on the computer to make this exploit work either by Trojan Horse virus or using the sneaky keyboard entry method he mentioned. That in and of itself violates laws and company policies already. I don’t think Mr. Cochran can help from beyond the grave. His extant law firm is no more crafty than any other law firm either. If you violate the law and break company policy (in where you signed a NDA and a employment-at-will) you would have no legal leg to stand on. You could be terminated and you could not file wrongful termination litigation successfully.

    I think this exploit only makes industrial and military espionage easier but not more legal. There are much easier ways to get medical records that don’t even involve a computer. But all still very illegal. That’s a risk any operative takes when trying to collect such protected information. A TEMPEST exploit seems very legal to me as it only involves collecting free-air-space emanations or radiations you should have protected yourself against with Faraday Shielding like how the US President does when he is out and abroad.

  15. Just the other day a pharmacy here send an excel sheet with client information through a mass email … making it impossible by design to disclose the records without intentionally subverting the system is not without merit.

  16. Lets say you can write a payload to execute on the target machine that can convert files to QR…..there is a less conspicuous method. You can just write a payload use a num lock/capslock call back to a modified keyboard and encode the data that way and you could probably encode it fast enough that the light just looks like its on and you wouldn’t even have to stand there while it was transmitting

  17. Seriously although the use of a screen is funny the whole ” blabla” around security is completely out of context.

    quote/”distinction between use of a file, and distribution or disclosure of a file. There is a marked difference between displaying information on a screen and sending it over a network. By transmitting binary data through a display, [Ian] has kicked that door down, turning every monitor and every employee into a security risk”/quote

    you just misunderstood the law or you play the idiot. File doesnt mean necessarily computer/electronic file. it refers to a bunch of documents which might be written on paper, and/or on computer.
    USE of information means a doctor or pharmacist can SEE on a display, or print that info but for its sole own use!
    “Distribution/disclosure” means you cant talk to a third party about what you saw on the file.Therefore you cant make a copy and give it to a 3rd party. Besides laws always use terms such as ‘ by any electronic or non electronic means’ which includes any new medium or technology invented or to be invented.
    If you display a file and decide to transmit it to a third party thanks to this thing you are clearly distributing it and disclosing its content to an unauthorized party.
    So please dont act silly to ” sell ” the technology – it doesnt need it.

  18. This is exactly what I am looking for!

    Is there by chance a video tutorial of how to create this feature in Unity? I am attempting to make a unity project and need to use such a method to share large amounts of string data across users.

Leave a Reply to Jonathan WagnerCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.