A Lightgun For LCDs – Thanks To Maths!

Light guns were a fun way to learn to shoot things on consoles, enjoying their heyday in the 80s and 90s. The original designs largely relied on the unique characteristics of CRT televisions and the timing involved in the drawing of their frames. Unfortunately, due to a variety of reasons (dependent on the exact techniques used), they typically do not work at all with modern LCD & plasma screens.

The light gun contains a camera, and reportedly works by using the distortion of the rectangular image of the screen to calculate the position of the light gun itself.

Recently, there has emerged a new project called the Sinden Lightgun. In the How It Works video, it seems to use a fairly standard 30fps camera inside the gun to image the television screen being used by the game. The display is then rendered in 4:3, letterboxed on a 16:9 aspect ratio display, within a rectangular bezel. The image from the camera is then processed, and the distortion of the game image is used to calculate the position of the gun and the direction of its aim. Processing is handled by the host computer running MAME and the requisite coordinates are fed back in to the game code.

The basic concept seems sound, though as always, there’s a healthy amount of skepticism around the project. We’d love to hear your take, on whether the concept is plausible, and whether the lag figures stated are cromulent. We’re always excited to see new developments in the lightgun space! Video after the break.

Our own [Will Sweatman] penned an excellent piece on a variety of ways one could resurrect the venerable game of Duck Hunt, too.

47 thoughts on “A Lightgun For LCDs – Thanks To Maths!

        1. These are the people who–rightly–are adherents of the Adams postulate, namely–

          “There is a theory which states that if ever anyone discovers exactly what the Universe is for and why it is here,‭ ‬it will instantly disappear and be replaced by something even more bizarre and inexplicable.‭ ‬There is another theory which states that this has already happened.”–Douglas Adams

    1. From the west side of the pond–I can think of only one problem with the word “maths”:
      it’s one more word to which the mouth-breathing air-heads will think an apostrophe should be added before the final “s”. Simply because it ends with an “s”.

      1. He he +100 about the apostrophe. MP3’s and DVD’s are the worst.

        There is no problem with “maths”. Presumably youngsters in North America learn mathematic, or maybe mathematick, rather than mathematics?

        1. Do you abbreviate economics as “econ” or “econs”?

          In the end it doesn’t matter which species of English you speak, any effort to make this hodge-podge of a language self-consistent and logical is utterly vain. It still works fine, though.

  1. “The center of the camera image is exactly where the camera is pointing.”

    Sure, but that may not be exactly where the gun is pointing. Mounting tolerances in the gun, camera tolerances on its own board, and plain ol’ getting thrown around may affect this. Hopefully there’s a calibration function to correct this.

    Also, I know the final project may have very different innards, but it would be nice to have *any* idea about what’s in that gun. Is it just a USB webcam and everything else handled by software on the PC? Is any processing handled in the gun itself?

    1. That is my objection to cheap laser levels. Sure, a laser projects a straight beam, and a spirit (bubble) level is an accurate way to see if the surface is level, but how does a customer know if the laser beam is parallel with the surface?

  2. It’s when you build a box that takes a video signal and converts it to be drawn line by line really fast so a standard light gun still works.

    I guess it’s a lot easier to read the angles of a large rectangle than it is to keep track of the cursor position with a camera in the gun.

    Just wait not light guns have cameras the government will start using them to spy on you… (;

    1. NES light gun games don’t track the timing of the beam actually. When you pull the trigger it turns the entire screen black with a white rectangle at the target, during one frame. During this frame the light gun measures the amount of light entering it (darkness = pointing at the black area = no hit, and light = pointing at the white area = hit). This doesn’t work with LCDs because of the latency (by the time the LCD switches to the black and white frame the light gun has already unsuccessfully made its light measurement).

  3. There is a physical wire leaving the bottom of the gun going into at a guess a USB 2.0 port, if a cheap $3 640x480x30fps web camera, is being used in the Proof Of Concept (no fire buttons were used during the video). And some web cameras do have an single button (I/O pin) that are typically used for taking 2MP still image.
    I suspect that for the POC that they are using the OpenCV library to calculate where the web camera is pointing using the corners of the image on the screen as reference points.

    It may not be a scam, but … a €283,407 goal with guns selling for $100 each, that are probably going to be cheap web cameras put inside in a plastic gun shaped case. And all the heavy duty image processing will be done on the PC.

    1. That seems a little cynical. even old off-the-shelf camera modules like the ov7670 already have multiple framerates and resolutions, and basic image processing built in. I see no reason why you couldn’t combine a more modern sensor with an fpga or arm microcontroller and get very good accuracy and responsiveness, with minimal lag, over USB HID.

      The proof of concept does what it says – proves that you can do lightgun-style aiming with a webcam. Kickstarter money will go to engineers and programmers to get proper prototypes up and running, and hopefully a first small run. Tooling for injection moulding is expensive though, and demand is likely to be fairly niche. Ultimate best case scenario I guess is that they deliver on the prototypes, then get bought by SEGA who go on to package a pair in with a new House Of The Dead remastered collection *drools*…

      1. thats how trackir and wiimotes work. the trackir is just a camera and an fpga slabed together with a usb interface. idk the internals of the wiimote but they both have internal spot detection and only is concerned about sending the coordinates of the spots not the entire image. both those devices are also dependant on ir pass filters and well defined targets.

        using the screen as a target is a rather novel approach and you only need to send the coordinates of the 4 corners to the pc. i think the problem here is going to be isolating the screen from the rest of the environment. with ir markers and a filter thats pretty easy, but if you can do that with software it makes the whole thing simpler, and any off the shelf camera can do.

        some of these arm microcontrollers out in the wild now might be fast enough to do the screen detection algorithm, abstract it down to a few coordinates, then stick a few buttons on the gpio and a bluetooth interface and you got a light gun. possibly at the same price point that the trackir is at.

        1. TrackIR doesn’t have an FPGA, AFAIK. The ones I have seen were nothing else but a webcam with an IR filter and a wide-angle lens. The processing is all done by the PC.

          Wiimote has a special camera that can track up to 4 targets by itself (in hardware), so that any video doesn’t need to be sent over the low bandwidth Bluetooth connection nor any video processing done by the host (console in this case).

    2. That is an overly complicated and unreliable nonsense, IMO. It is doable but the vision approach used is going to be super fragile, dependent on lighting, requiring calibration and what not.

      You can achieve the same thing with a $20 Wiimote and two IR LEDs (or the original “Sensor Bar”) and a bluetooth dongle. It will be a lot more accurate and also faster because most of the image processing is done by the Wiimote camera already. Oh and it could track the gun even when not pointing at the screen or not seeing the LEDs to a certain degree using the built-in accelerometer/gyro combo.

      1. The Wiimote requires calibration, it’s accuracy isn’t great, and has troubles when the player moves outside of the calibrated area. There are also some FoV issues. And picking up other IR spots, like light bulbs and the reflection of the sun…

        I like this idea. Take the PiZero wireless, take the Pi Camera. And you have a wireless, zero calibration, zero setup pointer device. Yes, it has cases where it won’t work correctly. But so has WiiMote option. Accuracy can be improved with a better camera, unlike the WiiMote. There is just lots of more room to play with here.

  4. i thought about just using an ir camera and sticking ir markers in the corners of the display. its the same kind of math that makes trackir possible, or wiimotes. its pretty much rendering in reverse, start with a polygon of known dimentions and try to figure out where the camera is.

  5. Why so skeptical. I do not see any complicated calculations to be done here to determine the cursor position. The only thing that has to work is the detection of the screen edge, which in most cases should be relatively easy since there is an high gradient in brightens from the screen to the monitor-frame. Further you can assume that in the center of the image you are inside the monitor since it does not make a lot of sense to point somewhere else in the room.
    (you might “hack” it by pointing on a white paper on a dark background but so what)
    And i guess the distortion just can be ignored since its not a preciosion-pointing-device.

    1. To be fair, it may not be this project, it may be my scam detector running on overdrive, Kickstarter have allowed the recently cancelled “BattBump – a mobile app to share and receive battery”. The people running that scam then within an hour of cancelling went on to expunge any evidence of their campaign happening through privacy complaints.

      Usually when I do not see the name of an actual individual person mentioned anywhere in a campaign my scam sense starts to tingle. But now with scammers purging their online history …

      1. Ok, I have not looked at the Kickstarter campaign , nor do I have a lot of experience on detecting scam campaigns therefore I don’t want to do any scoring on the scam probability of this campaign.
        It might be a scam, but I think the technology itself (recording the screen by a camera, throw some openCV on it to detect the edge, applying some coordinate transforms to calculate the center of the pointer as well as the direction relatively to the screen and outputting it over usb does not seem like a hard problem to me)

        1. I’ve thrown in a pledge. Looking at YouTube clips, this is a genuine project from an enthuastist. Fantastic and simple solution. Want to see this take off. Positive thinking ????

  6. I’ve thrown in a pledge. Looking at YouTube clips, this is a genuine project from an enthuastist. Fantastic and simple solution. Want to see this take off. Positive thinking ????

  7. I’ve got the Aimtraks as well and they are probably the best option out there at the moment. The problem with them is that if you move position at all they of out of alignment and passing the gun to someone else is out of the question. These look like they work as the creator says. There is another project out at them minute by PDP (mars light gun) that seems to be similar (and different) in that the light guns fire a laser at the screen which is picked up be a base unit with camera. The idea looks pretty similar byt the module removes the need for extra software to create the frame.
    I’ve done a write up of the 2 projects:

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.