Typically, when we want to take images, we use an image sensor paired with some sort of lens assembly to make a picture that’s sharply in focus. However, [okooptics] is here to show us there’s another way—using Scotch tape in place of a typical lens element.
If you just put Scotch tape over an image sensor without a lens, you’ll just get a blurry image, whatever you point it at. With the right algorithms, though, it’s possible to recover an image from that mess, using special “lensless imaging” techniques. In particular, [okooptics] shows how to recreate the so-called coded aperture techniques which were previously demonstrated in [Laura Waller]’s DiffuserCam paper.
It’s complicated stuff, but the video does a great job of breaking down the optics into understandable chunks. Armed with a Raspberry Pi HQ camera covered in a small amount of Scotch and electrical tape, [okooptics] is able to reconstruct a viable image from what initially looks like a blurry mess of nothingness, with the aid of the right deconvolution maths. It’s all about understanding the point spread function of the tape versus a regular lens, and figuring out how to fight off noise when reconstructing the image.
We’ve featured previous work from [okooptics] before, too, like this impressive demonstration of light transport and reconstruction. Video after the break.
There is another very interesting paper on Fourier ptychography from Laura Waller’s group using pi camera and a LED array HAT. https://doi.org/10.1038/s41598-019-43845-9
The collected raw data images are similar-looking blobs of noise, and both techniques use a lot of compute horsepower to produce the output image, but they are quite different processes. Both interesting, but not a lot of similarity between them.
Does this effectively have infinite depth-of-focus, or is that limited by some factor?
Because distance-independent imaging could be very useful even if the result is not as sharp as a lens would be at correct focus.
But maybe the PSF actually depends on the distance of the object too, meaning that you have to pick a focus distance at which you do the reconstruction. It would still allow after-the-fact focusing, like light-field cameras.
Yeah this does look a bit like a variation of lightfield photography with a super super super low budget lens array.
Light field photography has some different principals going on. Although the tape likely has some scattering information from the image, I don’t see how much of it could meaningfully be extracted. Lightfield cameras usually have tons of lenses, tape at best gives us a diffuser with some fraction of a wave polarizer
I don’t think so. I think of the tape as an aperture or a pinhole. A very dirty large one. The trick isn’t that a rough aperture on its own allows us to never need a lens again and also allows for a replacement of any lens. It’s moreso that “wow we can actually still see an image”. The image resolved from doing this depends pretty heavily on the aperture size as well as the size of the ccd. We could very quickly run out of pixels or the opposite need a large ccd/cmos array to start changing the “focal length”.
It’s still really cool and it has use cases. The obvious trade off here is the quality of the image.
Did you watch the video at all?
Yes. I saw them create an electrical tape aperture, and use a single light source against a dark background. I’m not saying it isn’t cool, it is. But I do believe my understanding of the physics here is correct.
But they mention that it’s based on the variations in the tape that enables you to recreate the image, and to my understanding it works in a sense like a collection of small deformed poor quality lenses. Hence the comparison to lightfield stuff.
I can see how you are thinking about it and it is based on the variations in the tape. But it’s important to consider the optical properties of many lenses with different focal lengths vs a uniform thickness “retarder” or a diffuser.
In light field photography, at least in a lens array variant, each lens has a different polarization and focal length. This turns each photo capture into many sub images. The trick is calibrating each sub image to correspond to different focuses, angles, etc.
With the tape we don’t have this property, especially with the experiment they set up. We have 1 psf, 1 image plane, and 1 broad band light source. Scatterer/diffusers themselves have interesting properties but the way we are doing things here it’s best to just think about this as a Fourier optics problem in my opinion.
Well obviously it’s not a lightfield camera per se, but it’s related in concept.
Somehow this reminds me of folks who have demonstrated “seeing around corners” by reconstructing light rays scattered from a painted wall.
Why does it look like there’s a person standing on the left side of the pic. With a stripped sweater on. Kind of like Freddy kruger. In all seriousness.?
Good imagination. It’s a can of air duster.
Seriously. Good imagination. Maybe go see a therapist?
And nobody mentions the silhouette of Sherlock Holmes on the right?
Right there, next to Darth Vader’s Mask?? 😮
Everyone remain calm. I am a doctor from a prominent psyche ward. Brandon may or may not have escaped. Everything is okay. Brandon come here buddy we have your favorite sock puppet in the car and we will not hurt you I promise.
Well, this won’t get you shot like having a Minox on you overseas
Next up, bug-detector proof recording device on vinyl?
Does this defeat the old tape over the webcam trick then?
If you remove the webcam’s lens and replace it with a particular type of diffusing tape a specific distance from the sensor, and go through the painstaking calibration process then, yes, it would defeat that particular security measure. Provided you have a single well-lit target in the middle of a dark field of view, and no other stray light falling on the tape.
It doesn’t. The trick with the tape is to use electrical tape, duct tape, or for the ultra paranoid people who don’t mind cutting their fingers furnace tape. The idea is if no light comes through to the detector, there is no recovery. Can’t recover photons that never made electrical signals.
If people are using cellotape then they have been doing it wrong.
No, this doesn’t impact using (ideally black or at least opaque) tape to block the lens. This technique only works with tape right up against the bare image sensor.
Actually it requires the tape to be offset from the sensor a certain critical distance to maximize the contrast of the caustics produced. IIRC it’s 3 mm in this case with his particular choice of tape.
Can this be used to see behind fog or smoke screen in automotive or military applications?
Yes, for sufficiently thin layers of fog, and if the fog droplets remain stationary between the time you calibrate the scattering layer and the time you take the data to produce the image.
Civilian fog won’t work, but I’m certain military versions of fog exist enough to support a grant application.
That was amusing snark :)
I love the way you are thinking. Unfortunately, no this wouldn’t work for that. One of several issues but the most important would be, what is the point source of a dynamic system like fog? It isn’t characterizable. Though fog works somewhat like tape, it scatters light, it has heterogenous densities at even the square inch scale. If the tape in this device was subjected to temperature variations or stress/strain or even dust over time it would require new calibrations.
The other issue, though related to the above is distinctly different. That is, complete loss of signal. Once the light is gone via absorption, or redirected randomly enough like in mie scattering for long enough, any hope at signal recovery is lost. The computation to recover an image from such a system would be incredible, you would have to know so much about the water in front of the imaging system a super computer would look like a toy. That said… If you could pull it off you could possibly image things all around your vehicle, even 90 degrees from where your camera was facing. Cool thought.
I should say. There’s little to no hope at recovery but what could be recovered would require immense compute and knowledge about the state it would be impractical
Would something like this work to clean up really degraded VHS tapes?