Hackaday Prize 2022: Multispectral Smartphone Camera Reveals Paintings’ Inner Secrets

A Google Pixel 3a with a filter wheel attached to its camera

Multispectral imaging, or photography using wavelengths other than those in ordinary visible light, has various applications ranging from earth observation to forgery detection in art. For example, titanium white and lead white, two pigments used in different historical eras, look identical in visible light but have distinct signatures in the UV range. Similarly, IR imaging can reveal a painting’s inner layers if the pigments used are transparent to IR.

Equipment for such a niche use is naturally quite pricey, so [Sean Billups] decided to transform an older model smartphone into a handheld multispectral camera, which can help him analyze works of art without breaking the bank. It uses the smartphone’s camera together with a filter wheel attachment that enables it to capture different spectral ranges. [Sean] chose to use a Google Pixel 3a, mainly because it’s cheaply available, but also because it has a good image sensor and camera software. Modifying the camera to enable IR and UV imaging turned out to be a bit of a challenge, however.

Image sensors are naturally sensitive to IR and UV, so cameras typically include a filter to block anything but visible light. To remove this filter from the Pixel’s camera [Sean] had to heat the camera module to soften the adhesive, carefully remove the lens, then glue a piece of plastic to the filter and pull it out once the glue had set. Perfecting this process took a bit of trial and error, but once he managed to effect a clear separation between camera and filter it was simply a matter of reattaching the lens, assembling the phone and mounting the filter wheel on its back.

The 3D-printed filter wheel has slots for four different filters, which can enable a variety of IR, UV and polarized-light imaging modes. In the video embedded below [Sean] shows how the IR reflectography mode can help to reveal the underdrawing in an oil painting. The system is designed to be extendable, and [Sean] has already been looking at adding features like IR and UV LEDs, magnifying lenses and even additional sensors like spectrometers.

We’ve seen a handful of multispectral imaging projects before; this drone-mounted system was a contestant for the 2015 Hackaday Prize, while this project contains an excellent primer on UV imaging.

18 thoughts on “Hackaday Prize 2022: Multispectral Smartphone Camera Reveals Paintings’ Inner Secrets

  1. Smartphone cameras do have a lot of features, but that was a lot of work to customize. Wouldn’t it be easier to start with the Pi camera module (IR version), which is already a bare sensor? (Assuming you already own a Pi.)

    1. customizing a smartphone camera might take a lot of work but you have the bonus of having a working and compact mobile platform to start from. basing the project on a Pi camera would require a lot of work to make the Pi portable and give it a working UI.

  2. This works fairly well for USB webcams too. You’ll see hot things glow, such as soldering iron tips etc. But don’t expect a lot of sensitivity beyond that. Yu may even see hidden markings on flowers.
    Fun fact, coca-cola is transparent when viewed using IR. Old floppy disks make fairly good visible light filters in a pinch and 3D cinema glasses are a cheap way to get polarizing filters.

      1. You both have ‘see through clothes’ hacked up cameras, admit it (‘supposed’, LOL)…Only reason anybody would know these tricks.

        Except me of course. I did it for the science. Pure science.

      2. Long time ago, I mod’d old B&W webcam with CCD also used in astronomy. I used overdeveloped slide film for the IR filter – blocked visible light. Worked like a charm, top notch stuff. Hey, do you sell these kits? You SHOULD.

    1. You’ll see near infrared from a lot of cheap cameras, like IR LEDS or TV remotes, but you won’t see the far infrared of hot glowing objects unless the objects are extremely hot indeed. A 400 degree soldering iron is not going to give of the kind of near IR a cheap camera without a filter can see.

  3. Ok, what am I missing? The sensor is still a RGB device fitted with a set of Bayer filters, There will be a bit of leaked UV and some common IR but the RGB filters will still dominate.

    1. the response curves of Bayer filters are basically identical in the near infrared. Or put it the other way around: IR-light is basically recorded with very similar amplitudes. No need to rip the Bayer filters off when using a normal image sensor in the near IR. It’s enough to get rid of the IR-blocking filter (usually a blueish piece of polished optical glas in front of the sensor). In the UV, it’s different. People working in the UV routinely rip off the Bayer filter layer in front of the sensor.

    2. I did something similar with a compact camera, removing the IR filter and replacing it with the ends of a developed exposed negative film as an visible filter. If the white-balance is left as normal (not manually set), the IR light comes out as pink. Without the visible light filter, greens look washed-out, and the image looks like a frame from a 70s super-8 movie. This is because most green stuff is plants and they reflect a lot of near-IR this goes straight through all the colours in the Bayer filters.

      As RW points out the Bayer filter is not that sharp and will pass a lot of light outside the colour it’s filtering for. Cameras fix this up digitally. If Bayer filters effectively filtered IR light, there would be no need for the IR filter.

  4. Also relevant: I “invented” a modified slide and film viewer.
    I found by accident that even a deteriorated slide looks useful again if you use different wavelength LEDs rather than a simple white light source as supplied.
    The diodes you want are easy enough to get, even pulling them from broken Bluetooth headsets and suchlike.
    They can be purchased online, and the method I used to build my setup is take the old white strip or PCB out (its rarely if ever glued in) and replace it with your ROYGBV strip or other illumination method like programmable LEDs, glued into the plastic block with Bondic or clear craft glue.
    By adjusting the relative brightness of each LED strip, you can then record each wavelength separately with a B/W camera thus avoiding horrible distortion common with phone cameras.
    Now working on a larger version for viewing of old negatives and projector film.

  5. >Multispectral imaging, or photography using wavelengths other than those in ordinary visible light

    No…multispectral can include invisible wavelengths, but it more properly refers to breaking the spectrum, visible or otherwise, into narrow bands (e.g., 20 nm). There’s much more information in 20 bands of colour than in three…

    1. THIS! while it its a neat project calling it multispectral imaging is just wrong. also polarisation is not a part of the spectrum it a property of light.
      if a perfect multispectral camera would exist it would create a curve from ir to uv for every pixel in the camera. like having a few million spectometers in a box, thats what multispectral imaging is.

Leave a Reply to limrohCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.