Neural Nets And Game Boy Cameras

Released in 1998, the Game Boy camera was perhaps the first digital camera many young hackers got their hands on. Around the time Sony Mavica cameras were shoving VGA resolution pictures onto floppy drives, the Game Boy camera was snapping 256×224 resolution pictures and displaying them on a 190×144 resolution display. The picture quality was terrible, but [Roland Meertens] recently had an idea. Why not use neural networks to turn these Game Boy Camera pictures into photorealistic images?

Neural networks, deep learning, machine learning, or whatever other buzzwords we’re using require training data. In this case, the training data would be a picture from a Game Boy Camera and a full-color, high-resolution image of the same scene. This dataset obviously does not exist so [Roland] took a few close up head shots of celebrities and reduced the color to four shades of gray.

[Roland]'s face captured with the Game Boy Camera (left), and turned into a photorealistic image (right)
[Roland]’s face captured with the Game Boy Camera (left), and turned into a photorealistic image (right)
For the deep machine artificial neural learning part of this experiment, [Roland] turned to a few papers on converting photographs to sketches and back again, real-time style transfer. After some work, this neural network turned the test data back into images reasonably similar to the original images. This is what you would expect from a trained neural network, but [Roland] also sent a few pics from the Game Boy Camera through this deep machine artificial learning minsky. These images turned out surprisingly well – a bit washed out, but nearly lomographic in character.

We’ve seen a lot of hacks with the Game Boy Camera over the years. Everything from dumping the raw images with a microcontroller to turning the sensor into a camcorder has been done. Although [Roland]’s technique will only work on faces, it is an excellent example of what neural networks can do.

11 thoughts on “Neural Nets And Game Boy Cameras

    1. This is exactly what I was thinking too, however I think it’s still miles away surely this would have to be calibrated for any brand of camera out there as although a lot of shop CCTV footage does look like someone glued a gameboy camera to the corner of the ceiling most camera’s will have slight and or major differences.

      1. Not useful for security cameras, unfortunately.
        RNNs introduce a creative element, and these enhancement algorithms, while totally awesome, are an application of style transfer, i.e. reconstructing the image based on what it’s learned from other images.

        It’s about as useful as a police sketch.

    1. That is not what photo realism means though, if you’re going to split hairs. Photorealism was originally a movement wiithin painting, to paint scenes has realistically as photography could render them.

      This is attempting to do the same with the low res output of the gameboy, so the term is applicable. I’m pretty sure you knew this and was just trying to be cheeky.

      1. Photorealism was not just to paint realistically, but to also paint from a specific photograph. In other words, the photograph that inspires the realistic painting is part of the artwork. It would be inane to say that a photo is photorealistic, because a photo is nothing more than photo.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s