Hallucinating Machines Generate Tiny Video Clips

Hallucination is the erroneous perception of something that’s actually absent – or in other words: A possible interpretation of training data. Researchers from the MIT and the UMBC have developed and trained a generative-machine learning model that learns to generate tiny videos at random. The hallucination-like, 64×64 pixels small clips are somewhat plausible, but also a bit spooky.

The machine-learning model behind these artificial clips is capable of learning from unlabeled “in-the-wild” training videos and relies mostly on the temporal coherence of subsequent frames as well as the presence of a static background. It learns to disentangle foreground objects from the background and extracts the overall dynamics from the scenes. The trained model can then be used to generate new clips at random (as shown above), or from a static input image (as shown in pairs below).

Currently, the team limits the clips to a resolution of 64×64 pixels and 32 frames in duration in order to decrease the amount of required training data, which is still at 7 TB. Despite obvious deficiencies in terms of photorealism, the little clips have been judged “more realistic” than real clips by about 20 percent of the participants in a psychophysical study the team conducted. The code for the project (Torch7/LuaJIT) can already be found on GitHub, together with a pre-trained model. The project will also be shown in December at the 2016 NIPS conference.

LED goggles make you trip out?

Who knows if this works and should you really want to try to induce hallucinations by flashing colors in front of your eyes? But we do love the zaniness of the project. [Everett’s] homemade hallucination goggles come in two flavors, the small swimming-goggle-type model and the heavy-duty trip visor made from welder’s goggles. Each brings together the same components; a half ping-pong ball for each eye to diffuse the light from an RGB LED. The system is controlled by an Arduino with some buttons and 7-segment displays for a user interface. Put this together with some homemade EL wire and you’re ready for Burning Man.
[Thanks Evan]