First the robots took our jobs, then they came for our video games. This dystopian future is brought to you by [Little French Kev] who designed this adorable 3D-printed robot arm to interface with an Xbox One controller joystick. He shows it off in the video after the break, controlling a ball-balancing physics demonstration written in Unity.
Hats off to him on the quality of the design. There are two parts that nestle the knob of the thumbstick from either side. He mates those pieces with each other using screws, firmly hugging the stick. Bearings are used at the joints for smooth action of the two servo motors that control the arm. The base of the robotic appendage is zip-tied to the controller itself.
The build targets experimentation with machine learning. Since the computer can control the arm via an Arduino, and the computer has access to metrics of what’s happening in the virtual environment, it’s a perfect for training a neural network. Are you thinking what we’re thinking? This is the beginning of hardware speed-running your favorite video games like [SethBling] did for Super Mario World half a decade ago. It will be more impressive since this would be done by automating the mechanical bit of the controller rather than operating purely in the software realm. You’ll just need to do your own hack to implement button control.
Personally I do not have a nNintendo account on file!
Nice, between this type of output design, and the camera based input featured the other day, these will go far in ML training and human-vs-computer play.
There is always that loud group of people that to this day still think computers can’t play any games because they aren’t limited to eyes and slow fingers like people are.
On the other practical side of automation however, I’ve wondered why analog was excluded from the xbox adaptive controller. Instead of direct connections like all the buttons have, the only provided way to get analog stick input is via generic USB HID, and two separate ones at that (left vs right on separate USB jacks, it isn’t able to use a single 2-stick gamepad)
Especially strange since the two analog triggers do in fact have breakout connectors on it…
Working from camera input is my next challenge .
I am not sure I completely understand your question in the second part but I didn’t tape directly into the breakout connector as I wanted to see how the network would handle the imperfection of the physical mechanism . Usually it is a lot more precise and efficient to directly control the game through software without using any peripheral at all .
My apologies, it wasn’t so much a question as an “out loud bafflement”
I see advantages of both methods of input control, but of course for different purposes.
If you search for “xbox adaptive controller”, a device I’ve used for the direct control method, this device mostly excludes the very analog sticks you automated mechanically. It’s confusing to me why Microsoft chose to do that is all.
I have given this some idle thought before, playing Gran Turismo and doing the endurance races over and oever for that 50/50 shot of getting the car you want to add to your garage. One of the tracks in the game was just a big wide loop. I’d rubber band and tape the controls so that it would bump off the walls and eventually you’d win as you’d never stop to pit. Letting it run for 24 hours or whatever it was (long time ago).
Scrolled into the comments to consider posting exactly this. Did exactly the same thing as you. Had to check up on it every half hour or so to make sure the car was still driving the right way around the track. Game balancing was pretty much broken after you won that open-wheel race car though…
I wonder if the twitchyness and the ball loss on digital training might both be related to the dead band setting on the analog stick.
I reduced the dead band to nearly 0 but I think it tries to compensate for the play in the mechanism (backlash in the servos gearing, twisting of plastic, play between parts…)
i want to implement this into an adaptive flight controller yoke. where it can click onto a controller whether its a playstation, statdia, or xbox, and control the sticks