Using Kinect To Play Super Mario Bros 3 On NES Ensures Quick Death

Kinect4NES

Why do only the new game consoles get all the cool peripherals? Being a man of action, [Paul] set out to change that. He had a Kinect V2 and an original Nintendo and thought it would be fun to get the two to work together.

Thinking it would be easiest to emulate a standard controller, [Paul] surfed the ‘net a bit until he found an excellent article that explained how the NES controller works. It turns out that besides the buttons, there’s only one shift register chip and some pull up resistors in the controller. Instead of soldering leads to a cannibalized NES controller, he decided to stick another shift register and some resistors down on a breadboard with a controller cable connected directly to the chip.

Kinect4NES wiring

An Arduino is used to emulate the buttons presses. The Arduino is running the Firmata sketch that allows toggling of the Arduino pins from a host computer. That host computer runs an application that [Paul] wrote himself using the Kinect V2 SDK that converts the gestures of the player into controller commands which then tells the Arduino which buttons to ‘push’. This is definitely a pretty interesting and involved project, even if the video does make it look very challenging to rescue Princess Toadstool from Bowser and the Koopalings!

If you’d like to help the project or just build one for yourself, check out the source files on the Kinect4NES GitHub page.

5 thoughts on “Using Kinect To Play Super Mario Bros 3 On NES Ensures Quick Death

    1. Agreed! And the fact that the port was always on the motherboard itself always kept you on your toes a bit when doing homebrew projects!

      Of course you could always build one of those USB -> LPT projects implemented with….you guessed it…an Arduino! :D

  1. The 4021 shift register is a parallel to serial IC. The AVR has a shift register internally.

    If I recall correctly, the NES isn’t going to poll the 4021 any faster than every 60Hz or so….

    But the Arduino hides so much it never occurs to him to just skip the the 4021 entirely and go directly to the NES using the AVR SPI. Probably can optimize a bunch of other stuff away but I quit reading the article after I realized he was still using a 4021 in the m8x.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.