Arduino, Accelerometer, And TensorFlow Make You A Real-World Street Fighter

A question: if you’re controlling the classic video game Street Fighter with gestures, aren’t you just, you know, street fighting?

That’s a question [Charlie Gerard] is going to have to tackle should her AI gesture-recognition controller experiments take off. [Charlie] put together the game controller to learn more about the dark arts of machine learning in a fun and engaging way.

The controller consists of a battery-powered Arduino MKR1000 with WiFi and an MPU6050 accelerometer. Held in the hand, the controller streams accelerometer data to an external PC, capturing the characteristics of the motion. [Charlie] trained three different moves – a punch, an uppercut, and the dreaded Hadouken – and captured hundreds of examples of each. The raw data was massaged, converted to Tensors, and used to train a model for the three moves. Initial tests seem to work well. [Charlie] also made an online version that captures motion from your smartphone. The demo is explained in the video below; sadly, we couldn’t get more than three Hadoukens in before crashing it.

With most machine learning project seeming to concentrate on telling cats from dogs, this is a refreshing change. We’re seeing lots of offbeat machine learning projects these days, from cryptocurrency wallet attacks to a semi-creepy workout-monitoring gym camera.

Continue reading “Arduino, Accelerometer, And TensorFlow Make You A Real-World Street Fighter”

Turning A Cadenza Into A Finishing Move

What do you get when you mix dueling pianos with a 2D fighting game? Undoubtably some complex controls, but also an awesome platform for showmanship! The “Sound Fighter” installation by artists [Cyril] and [Eric] was built with the exact intention that two opposing parties could duke it out in a Street Fighter match with their piano playing abilities mapping into attack combos and dragon-punches.

In order to turn a piano into a glorified arcade stick, [Cyril] and [Eric] would need a way to register when and what notes were being played and then translate that data into commands for the fighting game itself. To start, they did their homework on the inner workings of different piano types. Whatever digital augmentation they were to design would have to work without inhibiting the piano’s function.

There were many possible methods of registering when the piano was being used and though several would have worked for their intended purpose, it took writing down and discussing the pros and cons of each sensor before they made a decision. Some of the options they considered included pressure sensors for the keys themselves, accelerometers to detect the movement of the individual hammers within the piano, and even a microphone to computationally analyze the sound heard from either instrument. In the end they chose to implement small and accurate piezo knock sensors tethered to the internal mechanism of each key. These could register both faint and strong notes when played without altering the natural sound of the instrument.

fighterPiano_2

After deciding on a Street Fighter iteration for the PS2 to develop the rest of the project around, they had to play the actual game a bit to get a feel for the command list of moves. They wanted to conceive of a way to map the notes played to the controller, but not in the direct “key to button” sort of way. The idea was that if someone was good at playing piano, they would also be good at executing moves in game. So they had to sort out how groups of notes and chords would translate into moving the character or attacking.

I highly suggest checking out their in depth play-by-play as they built the installation from the ground up. In addition to being fascinating (they prepared this project in a fight against time for the reopening of a historical site in Paris), you’ll find that everything they developed is opened source. The completed installation is as awesome as it sounds. You can see it in action in an actual duel below:

Continue reading “Turning A Cadenza Into A Finishing Move”

Shredding To Street Fighter

Strumming to punch and changing frets to move, [Alan Chatham] plays Street Fighter using his guitar. It’s been modified to use OpenChord, an open source guitar controller package he developed. This was originally meant to be used with Guitar Hero and the like but as he mentions in the video after the break, it is open source so you can do whatever you want with it. In this case, he’s patched into a PS3, showing yet another way to use your own hardware on that console. Unlike alternative guitar-like interfaces you won’t have to relearn how to play. You just need to adapt your favorite songs to fall in line with butt-kicking controller combos. For the adventurous you can build your own but [Alan’s] got kits available too. Continue reading “Shredding To Street Fighter”