Neural Networks Walk Better Than Humans For Game Animation

May 5, 2017 by Inderpreet Singh 40 Comments

Modern day video games have come a long way from Mario the plumber hopping across the screen. Incredibly intricate environments of games today are part of the lure for new gamers and this experience is brought to life by the characters interacting with the scene. However the illusion of the virtual world is disrupted by unnatural movements of the figures in performing actions such as turning around suddenly or climbing a hill.

To remedy the abrupt movements, [Daniel Holden et. al] recently published a paper (PDF) and a video showing a method to greatly improve the real-time character control mechanism. The proposed system uses a neural network that has been trained using a large data set of walking, jumping and other sequences on various terrains. The key is breaking down the process of bipedal movement and its cyclic behaviour into a series of sub-steps or phases. Each phase translates to a natural posture for the character while moving. The system precomputes the next-phases offline to conserve computational resources at runtime. Then considering user control, previous pose of the character(including joint positions) and terrain geometry, the consequent frame of the animation is computed. The computation is done by a regression network that calculates future position of the joints and a blending function is used for Motion Matching as described in a presentation (PDF) and video by [Simon Clavet]. Continue reading “Neural Networks Walk Better Than Humans For Game Animation” →

Learn Neural Network And Evolution Theory Fast

April 5, 2017 by Jack Laidlaw 32 Comments

[carykh] has a really interesting video series which can give a beginner or a pro a great insight into how neural networks operate and at the same time how evolution works. You may remember his work creating a Bach audio producing neural network, and this series again shows his talent at explaining the complex topic so anyone may understand.

He starts with 1000 “creatures”. Each has an internal clock which acts a bit like a heart beat however does not change speed throughout the creature’s life. Creatures also have nodes which cause friction with the ground but don’t collide with each other. Connecting the nodes are muscles which can stretch or contract and have different strengths.

At the beginning of the simulation the creatures are randomly generated along with their random traits. Some have longer/shorter muscles, while node and muscle positions are also randomly selected. Once this is set up they have one job: move from left to right as far as possible in 15 seconds.

Each creature has a chance to perform and 500 are then selected to evolve based on how far they managed to travel to the right of the starting position. The better the creature performs the higher the probability it will survive, although some of the high performing creatures randomly die and some lower performers randomly survive. The 500 surviving creatures reproduce asexually creating another 500 to replace the population that were killed off.

The simulation is run again and again until one or two types of species start to dominate. When this happens evolution slows down as the gene pool begins to get very similar. Occasionally a breakthrough will occur either creating a new species or improving the current best species leading to a bit of a competition for the top spot.

We think the series of four short YouTube videos (all around 5 mins each) that kick off the series demonstrate neural networks in a very visual way and make it really easy to understand. Whether you don’t know much about neural networks or you do and want to see something really cool, these are worthy of your time.

Continue reading “Learn Neural Network And Evolution Theory Fast” →

Neural Nets And Game Boy Cameras

February 20, 2017 by Brian Benchoff 11 Comments

Released in 1998, the Game Boy camera was perhaps the first digital camera many young hackers got their hands on. Around the time Sony Mavica cameras were shoving VGA resolution pictures onto floppy drives, the Game Boy camera was snapping 256×224 resolution pictures and displaying them on a 190×144 resolution display. The picture quality was terrible, but [Roland Meertens] recently had an idea. Why not use neural networks to turn these Game Boy Camera pictures into photorealistic images?

Neural networks, deep learning, machine learning, or whatever other buzzwords we’re using require training data. In this case, the training data would be a picture from a Game Boy Camera and a full-color, high-resolution image of the same scene. This dataset obviously does not exist so [Roland] took a few close up head shots of celebrities and reduced the color to four shades of gray.

[Roland]'s face captured with the Game Boy Camera (left), and turned into a photorealistic image (right) — [Roland]’s face captured with the Game Boy Camera (left), and turned into a photorealistic image (right)

For the deep machine artificial neural learning part of this experiment, [Roland] turned to a few papers on converting photographs to sketches and back again, real-time style transfer. After some work, this neural network turned the test data back into images reasonably similar to the original images. This is what you would expect from a trained neural network, but [Roland] also sent a few pics from the Game Boy Camera through this deep machine artificial learning minsky. These images turned out surprisingly well – a bit washed out, but nearly lomographic in character.

We’ve seen a lot of hacks with the Game Boy Camera over the years. Everything from dumping the raw images with a microcontroller to turning the sensor into a camcorder has been done. Although [Roland]’s technique will only work on faces, it is an excellent example of what neural networks can do.