It sure sounds like “laser speckle imaging” is the sort of thing you’d need grant money to experiment with, but as [anfractuosity] recently demonstrated, you can get some very impressive results with a relatively simple hardware setup and some common open source software packages. In fact, you might already have all the components required to pull this off in your own workshop right now and just not know it.
Anyone who’s ever played with a laser pointer is familiar with the sparkle effect observed when the beam shines on certain objects. That’s laser speckle, and it’s created by the beam reflecting off of microscopic variations in the surface texture and producing optical interference. While this phenomenon largely prevents laser beams from being effective direct lighting sources, it can be used as a way to measure extremely minute perturbations in what would appear to be an otherwise flat surface.
In this demonstration, [anfractuosity] has combined a simple red laser pointer with a microscope’s 25X objective lens to produce a wider and less intense beam. When this diffused beam is cast onto a wall, the speckle pattern generated by the surface texture can plainly be seen. What’s not obvious to the naked eye is that touching the wall with your hand actually produces a change in the speckle pattern. But if you take high-resolution before and after shots, the images can be run through OpenCV to highlight the differences and reveal a ghostly hand-print.
[anfractuosity] then uses the same technique on a calculator before and after some buttons have been pressed on it. Not only does the final cleaned up image clearly show the numbers on the display, but it highlights the individual buttons which were touched. Seeing this example, it’s not much of a stretch to think there could be some nefarious application for this technique. Could an attacker use laser speckle imaging to determine which buttons have been pressed on a lock keypad or alarm panel?
Luckily, it sounds like putting an attack like that into practice would be quite difficult. For one thing, the camera and laser need to be in exactly the same position when the before and after shots are taken, which would be all but impossible for a clandestine operation. Secondly, as evidenced in the video below, the imprints tend to decay fairly rapidly. The after shot has to be taken within a few minutes of the keypad being touched, making it even more difficult to pull off in the wild.
It’s not immediately obvious what practical applications this technique may have among the hacker and maker crowd, so we’d love to hear about any you might come up with. In any event, it’s an impressive accomplishment and an excellent example of what’s possible for the modern hobbyist.
This makes me wonder how sensitive this is to temperatures and similar, since thermal expansion and other effects might also create changes in the specles.
So… What about using this to detect temperature changes on a PCB with high spatial resolution, compared to a IR thermal camera that is really broad.
Putting the PCB down and leaving it in the exact same place relative to laser and camera is pretty easy~
It’s nearly unbelievably sensitive. As mentioned in the very first sentence, a research colleague of mine worked along these huge grants. She used it to find difference in single cell activities in cell cultures for quantification and it was impressively consistent. I’ve had to provide some cv-code here and there and always wondered if the really expensive hardware needs to be really that precise since most of the work comes from the software… Interestingly on one occasion the vendor of the hardware destroyed their laser-diode. I’ve had laying around a cheap chinese part, we changed it… no difference at all but the price divided by some 1000 factor.
If the marks fade after a few minutes, you wouldn’t need a shot before and after, just one soon after and another a few minutes later once it’s faded, then load them in reverse order? On that basis, it seems obtaining the keys pressed on a code lock or access panel of some sort would be fairly easy as long as you’ve got a few minutes to hang around after someone has used it!
Good thinking- it may not even be necessary to wait, a quick wipe with alcohol might clear the pattern enough to show a difference to to the freshly touched panel.
In the image above, I think the buttons were showing a microscopic difference in height, so I’m not sure that would work.
But, I think you could probably also image fingerprints using the original approach, and perhaps see the thermal print using the later image as you suggested.
I can think of a simple setup where an arduino has control of a camera, IR laser diode (e.g. GDLD980100H4T) (why make it obvious?), and either a PIR sensor or microphone to trigger (much like a camera trap actually). It could all be housed in a small box opposite the keypad.
So we had better be looking out for these things, because if _I_ can think of it, so can n.e.one else (spooks or crooks and everyone in between). Could be applied to ATMs, access code locks etc etc.
Would the IR laser (with its longer wavelength) be able to show the minute differences picked up by the visible light laser?
(just thinking out loud).
Well if it can’t then you could probably use a UV laser instead.
Good point, but I don’t see why it would need an IR laser to begin with. If you are pointing at a keypad, those are usually outside of doors. Once the numbers have been pressed, people tend to go through those doors and out of sight. You can easily get the shot you need with no one around. This become problematic if there are lots of people around constantly. Keypads on the outsides of buildings are probably easier.
An IR laser is still a coherent in-phase light source, and would have the same kind of speckle pattern. The problem is the camera might not be sensitive at that wavelength range, so the speckle might not be observable.
That’s a good point, assuming you were there to witness when the buttons were pressed and could immediately set up the camera/laser.
But that’s still a pretty tall order in the real-world. Something like that could maybe be pulled off if the keypad is on an external door, and there’s nobody patrolling the area or watching with security cameras. But good luck doing it inside an office building in the middle of the work day.
That said, if it turns out this technique can be pulled off at a distance with telephoto optics and a more targeted beam…
The distance isn’t the issue, but thermal fluctuations in air currents would mean the pattern fluctuates naturally much more quickly.
Possibly a poor man’s lidar…? Could this technique be used in conjunction with larger “samples” [images, etc] to extract details from various perspectives? Again, like processing the results of a lidar scan?
I wonder if the order in which one touches the buttons affects the fade-time. I’d almost expect that the first button wipes more oil off your fingers than the last one. So it might even be possible to get the order of numbers with this technique by taking photos 30 seconds apart after someone has touched it.
This sounds logical to me, but the images of the calculator don’t seem to support the theory (at least not to my eyes and brain). The 7 key would have to have been pressed before the 8, but the 8 key seems to look more perturbed.
Of course, I’m sure that in some cases this theory could work. Definitely worth looking into for anyone interested in putting this method to work, IMO.
I’m pretty sure the reason the buttons appear so well-defined is that after they were released and their springs returned them to their normal positions, they were not returned to the exact same point they started from. A push button would have to be an ultra-precision machine in order to return to the exact same point (within 20nm!) and they simply aren’t made to do that – they’re made to cost as little as possible.
After they spring back to a slightly different position, they then slowly drift back to their original position? Only that would explain the fading. That is not very likely, is it?
I put my dollar on either oily substance from the finger evaporating (not likely) or the heating up and subsequent cooling of the keys, with accompanied expansion and shrinking of the material as reason for the fading. Thermoreflectance (@macona) sounds interesting, too. Makes one wonder what the 3D face sensors in some smartphones are capable of.
After the article about taking a thermal image of a credit card machine with a FLIR add-on published here on HAD, I always lay my hand over the whole keyboard on whenever machine I have to type a password
This is probably the result of thermoreflectance. Materials change their reflectivity depending on the temperature and you can use this to measure the temperature of stuff with visible light. There is one company that make a microscope system that will show temperatures over the area of an IC similar to what you would see with a thermal camera but much higher resolution. I am actually a co-inventor on a patent relating to this.
Please share pat.number or at least keywords, thx!
In the general case, I’d be far less worried about this showing the code for an alarm keypad than the clean spots on the keys.
Or the over-worn keys.
Make a small part that needs to carry loads.
Take one shot with no loading, then a series of shots with increasing loads.
Compare the images, look for deflection.
I love this idea. Very useful and inexpensive mechanical testing.
You could use this to detect gestures on a surface. Ir laser with an ir camera could do it with a relatively low frame rate. You could make anything into a multitouch sensor with some delay.
Or maybe detect footprints for presence sensing. Maybe this could make a good depth camera?
I remember the Xbox camera would get better results if you attached a vibration motor to it. Maybe something like a piezo element could vibrate the laser or detector to get minute variations.
Xbox cameras aren’t using speckle, so destroying speckle pattern with vibrations helps. In this case, and motion of the optics will destroy the speckle pattern and you won’t see the difference pattern.
Things this would work in. Ai motion tracking system for iot. Glass panel hologram integrated touch sensor. Ant trail detector. Clean room detector. Theft prevention systems.
The reporter on “Max Headroom” used a similar technique to read keypad numbers and sequence on a lock. I’ve worked with laser interferometers. They are very sensitive to heat. If I touched one of the mirrors, I could see interference fringes changing for five minutes until temperature equilibrium was reached. You would have a brief time window to perform such detection but it pays to shield keypads from external view.
A smart phone would have the camera and processing power (I assume) to take these pictures, build a small laser into a phone holder, connect it to the phone and write a program to flash the laser twice while taking two photos in rapid succession, then just dump to photos to processing software on your phone and Bob’s your Uncle!