The first person shooter genre found its feet in the PC world, relying on the holy combination of the keyboard and mouse for input. Over time, consoles have refined their own version of the experience, and the gamepad has become familiar territory for many FPS fans. [Tech Yesterday] was a die hard controller player, but after trying out a mouse, didn’t want to go back. Instead, he built a truly impressive hybrid device.
The build begins with a standard Xbox 360 wired controller, somewhat of a defacto standard for PC gamepads. The left analog stick and triggers remain untouched, however the face buttons are all relocated using mechanical keyboard switches. The D-pad has been relocated to the left hand side with tactile switches, and the right analog stick removed entirely. In its place, a cut-down optical mouse is used on a flat 4″x4″ mousepad attached to the controller, strapped to the player’s thumb.
The resulting controller combines the benefit of analog stick movement and the precision aiming of a mouse. We’re amazed at how comfortable the controller looks to use, particularly in the improved second revision. While currently only used on PC, we can imagine such controllers shaking up the console FPS scene in a serious way.
We see some great controller hacks around these parts; the force-feedback mouse is a particularly amusing example. Video after the break.
[Thanks to Baldpower for the tip!]
Clever, similar to how the controls worked for Metroid Prime: Hunters on the Nintendo DS, but with an analog stick instead of a regular ol’ D-pad. He could actually make even more improvements by getting closer to the MP:H control scheme by swapping the tiny mouse for a touchscreen and thumb stylus, this would allow him to place programmable buttons for control macros on the edges of the touchscreen.
Sadly no thats the issue with things like the wii-u and ds you are required to lift hand from your controller surface and grab the stylus to function and with no static mounting place the left hand is the only thing providing stability its very hard to get very precise touches.the steam controller may it rest in peace as a true innovation it was the refinement of this concept.
The DS had a “thumb stylus” which was a strap with a plastic nub that wrapped around your thumb so you’d also be holding the controller (which was also the body of the system) with both hands, very similar to this strapped-on thumb mouse but more compact.
This is freaking awesome – using a gamepad is the #1 reason I don’t game on consoles. I hope he has patented it, but I also hope someone like Logitech or Razer picks this up.
Search for Xim Apex.
Been using Xim products for years and those are godly.
I had a Xim before switching to PC, and I now realize it’s straight up cheating. Having aim assist with a mouse is like an aimbot-lite.
Yes, using a mouse and keyboard against players with controllers gives you a hugely unfair advantage. You should be very proud of yourself.
Wouldn’t work in fortnite I’ve already built one.
That looks fantastic, but I’m wondering if a touchpad surface isn’t more practical?
i hate touch pads, give me a damn trackball.
ah yes the trackball, prime pick of all FPS champions since Thresh!
For this use I think the best options is a trackball, virtually implemented on the touchpad – in the same way steam controllers do. Personally I love it – all the fine precision of a mouse (better even), and flick for rapid movements (which is why it can be better fine precision similar to the ‘aim’ button some mice have – setup so the controlled movement is very, very small – go as far as you like on movement to input distance, and then use the flick and stop for bigger moves).
I like trackballs anyway, but in a game controller a real physical trackball I don’t think is ideal. There is keeping the ball properly seated as the body tilts, cleaning out the gunk that will build up, and the fact the ball has real mass. Which leads to a trade off between a bigger a ball for a larger range in precision movements, and the weight of the ball requiring more force to accelerate and stop. Where a trackpad playing trackball (or even a curved touch surface if you really want it to feel like a trackball) has no real weight, so you can define how easily it flicks and decelerates in software, being a touch surface only needs that surface cleaning, and the ball can’t fall out…
Watch the whole video
Kind of like a pad with a gyro, like the defunct Steam controller.
The steam controller was a good idea, but I could easily see something about the tactile feedback with a hard object registering better with our hands for tiny movements that would be lost in the massive pickup of a touch sensor. To counter the large surface area of the thumb I think if you had something like a rubber thimble / band with a metal dot for that the sensor picked up would bring these closer together in responsiveness.
The Steam Controller trackpad is supremely configurable, you can make it quite accurate: overall sensitivity plus acceleration, along with haptic feedback, and active edges to continue motion, and you can set it up to make very fine moves without sacrificing large movement ability. Combine that with the gyro for super-fine movements: set the gyro to activate upon partial trigger pull and/or when not touching the pad: so awesome.
Agreed.
The only thing it doesn’t do well, which I’d have liked is the touchpads don’t always register well on things like blisters, scabs etc or while wearing gloves – but that’s not a failing of the controller at all really, almost all touch powered tech have the same issues.
I think the only reason it didn’t really catch on hugely is that you can easily spend hours tinkering the controller to give the input you really want in each game. Because its so possible to make it better, but all that work is on you. Where a keyboard and mouse or game controller has a small amount of configuration and then you just have to deal with it as it is.
I made a config for deus ex:md and just imported that for each new game. Some broken games like fallout4, alien isolation, cyberpunk cant do mouse and keyboard at the same time, so there’s a 2nd config for those.
The customization is one of the big reasons I picked it up, and it really countered my carpal.
… I am one to spend hours tinkering on things though (ie getting Fallout 4 playable on an intel atom handheld pc)
Defunct? I use a Steam controller for all FPS. It’s easy enough to adjust the haptics and sensitivities. I’ve used a mouse and keyboard but I found that I was more focused on keys then the game play
i got this media center keyboard with a great thumb trackball. i wish someone made a game pad with one of those. i did try one with a blackberry trackball but those are horrible for high res screens. maybe i can dig through my dead mice for an optical sensor where they didn’t scrub the chip, so i can find a proper datasheet for it.
Why not Zoidberg? umm a trackball?
Come to think of it, you could rig that with barely any hacking, using one of those presentation type hand grip trackballs. Just splice or tape stuff together.
The trackball doesn’t move with your thumb, so it’s more like a different sort of joystick.
You gotta try getting out of that box once in a while both thumb and fingertip trackballs exist.
It’s still a stationary ball that you cannot move around, and making larger motions means you have to flick it repeatedly.
Can’t really make super large movements with the thumb mouse either. You can even see them picking up and resetting the thumb mouse in the vids. A trackball can actually make much bigger movements since it has momentum and can be flicked and continue spinning.
The way I see trackballs are as a reverse-mice of some sort. Instead of moving the sensor over the surface, you’re moving the surface (ball’s surface) over a stationary sensor. I don’t see how reversing the two would be different.
With that being said, there are commercially available optical trackballs that exist, or you can make your own. It’s as easy as turning an optical mouse upside down and suspending a smooth ball on the optical sensor (albeit you need to reverse the axis).
> and making larger motions means you have to flick it repeatedly
Also not true. Compared to conventional mice, you can actually perform a better/smoother SINGLE flick with more travel due to the ball’s moment of inertia.
Another trick… I found myself needing to use a mouse in mid air temporarily, and a playing card sized piece of card, moved with my thumb on the inverted mouse seemed to work pretty well. So I am considering that scheme for an attached pointing device in future, inverted mouse sensor with movable surface retained by a bezel. It’s easier when you can flip left/right or up/down according to which way round you’re holding it. If you get a superbly trackable surface, and turn sensitivity way up, while ensuring motion is friction free, and keep dust out, you could have very fine control over the range of movement of your thumb. You know like touchpads are meant to work but don’t. (Hey, I’ve detected that the tip of your thumb is exerting slightly less pressure now than the bottom of the pad, let me leap halfway across the screen, oh, now you’ve twitched and being a multitouch, now I perceive your thumb to be two points of contact so I’m gonna randomly zoom in or out.)
Some of the Steam Controller prototypes had trackballs. Worked pretty well from what I heard.
https://www.wired.com/2013/11/valve-steam-controller/
Interesting, but the accuracy of a mouse comes fro the amount of space you have to move it. I have nearly two feet of space, this has 4’. I doubt it’s really more accurate than a good joystick player.
With my middling 800 DPI mouse, I can move the cursor across a standard HD monitor within 2.4 inches. There’s plenty or resolution in a mouse, since you cannot orient your view in the game in less than one pixel increments. It’s your ability to physically move your finger that accurately that’s the limit, and for the thumb joystick user the range of motion is even less.
The joystick player also suffers from the fact that they have to move the stick to the middle to stop the motion and then turn the other way. It’s like pulling things along on a string, where you first have to pull the string tight before the object follows along.
Yeah, good point. Also after watching the video, I take it back. That looks a lot better than a joystick
I think it’s important to note that this isn’t going to work on a console unless mouse input is enabled from the get go. Claiming this controller is “OP” or gives any sort of advantage over other PC gamers who are most definitely using a Keyboard+Mouse is sensationalism and completely misleading.
Full HID emulation could be used with an esp32 (for example). I don’t know if it’s possible to emulate a Xbox or PS pad anyway.
That’s not where the accuracy of a mouse comes from…So does that mean I’ll be more accurate if I use the floor as a mousepad?
Dance Dance Mousepad™
The accuracy of the human, not the input device. Consider the number of neurons involved in using just the muscles to move your thumb vs fingers, wrist, and arm. There’s a sweet spot where you have a largest bandwidth in the human-machine interaction with a small enough reaction time. Using the floor wouldn’t change your biodynamics, but having a large enough mousepad can remove impediments like falling off the edge. There are practical limits on how fast you can move different parts of your body, and how precisely. Using a mouse is easier to get high resolution from the brain than using an analog stick. Imagine having to write with a pen tip stuck to your thumb vs holding it with thumb and fingers.
The hack factor is awesome. Great job combining all the bits into a coherent whole.
But the innovation factor is diminished a bit by the fact that the Steam Controller basically already did this. Sure the trackpad is smaller, but it was extremely configurable with tracking speed & acceleration, trackball momentum simulation, continued movement at the edges, haptic feedback, remapping (act like a mouse, trackpad, trackball, joystick, joymouse), and more. All that configuration plus extra buttons on the back and the fact that there is _zero_ extra weight strapped to your thumb… and the Steam Controller still didn’t catch on (though I still use mine!)
The accuracy seems to be pretty good but I never saw him spin around using this once. All of his demo footage has him pointing mostly in the same direction and never turning all that much. Neat idea though. I wonder why it’s so much better than a trackpad? Maybe the optical sensor can have a much higher dpi?
I don’t play games though I did in the standup coin days. I need a joystick I can grasp and has firing buttons and a hat switch as well. Being right handed like most I use that stick in the right hand. Like a piano player, the lead is in the right hand. I might play a game that just used a mouse.
We are all different, so adopting only one way of interface is poor design in the long term. Qwerty. Take the case of inverted vs. normal sticks, adaptability is needed. My thumbs like most are not all that agile compared to the finesse of fingers. The thumb’s best act is to follow the opposing finger. When it leads “he’s all thumbs” comes to mind. Grip not control.
Why? Half of the hand is being used to hold the “console” piece instead of being part of playing with the console. The thumb is enabled as a last ditch effort to make a piece of plastic usable for cheap. I took apart a wireless “console” thinking I could hack the pots in the “stick” to my Arp, only to find that they were tiny trim-pots meant for an occasional screwdriver with a piece of plastic coupling to the “stick”.
The kind of control needed in music on a variable control is the envy of any gamer if they could try it. My pitch bender on a Casio is really a bender as in a leaf spring that protrudes thru the top of the keyboard. No dead-band in the center position and more and more tension as pressed either way. Vibrato is made with the bender like in real life not a separate control to an LFO.
Nice to realize that simulating a brake on a driving sim requires a force sense and a position pot reading to get to the real feel. A piece of stiff piano wire as a airplane joystick, attach magnets and hall sensors. Hmm… No mechanical dead-band response!
This is definitely a good hack, despite there being a separate wire to plug in for the mouse. I would have probably tried to fit a small hub in there so there’s only one wire to plug in. Alternatively you could rewire all the buttons and the mouse to an arduino pro mini or a teensy and avoid all that and just assign the buttons as you wish.
It is probably also better than a regular game pad due to the increased accuracy, but I doubt it is better than a keyboard and mouse. It’s in the middle somewhere. The thumb is limited in some directions of movement compared to your entire arm. Your palm gets in the way, your joints don’t move in certain directions as well, etc.
Nice job though. He keeps saying “We” built it. I wonder who else worked on it.
This is actually really stupid sorry to say. The entire and only reason controllers are used is because they are supplemented with auto aim to make up for their bad design. So the entire point of using a controller in any modern fps is to get access to the games built in auto aim. This device takes loses that and also makes for a low quality mouse. And loses access to the keyboard. So it literally destroys every good in favor of making every part less effective
The loosing the keyboard part is for me a big win to a device like this – letting you have many more axis with variable control so you can strafe for instance a little fast/slower perhaps to help match where you are aiming or stay obscured behind an object.
Most games have very few buttons that do anything else, most of the keys you press are just really low resolution human PWM signals to move in a certain direction. Probably a few other buttons to control tool selections, interact with the world, but only a few so they can be on the controller too.
Don’t personally think this implementation makes any sense – the steam controller does the same idea better IMO. And if you want to aim with a mouse but gain the other advantages of the controller on PC you can almost certainly just do that. Though I will say some games I’ve played are annoying in that if they get given a ‘controller’ movement input at any point the auto-aim type crap turns on, but might also just turn itself off – which doesn’t sound so bad as you don’t want it on anyway, you have a lovely precise input, but the inconsistency really ruins the playing experience (not to mention most auto-aims are rubbish if you are not expecting and thus playing to use them, locking on to the wrong critter that just happened to walk almost in front of the one you were targeting etc).
This isn’t a new concept. Its basically the same thing as the Assassin 3D joystick created in the mid 90s.
heuuu… it was using a trackball….
Sadly, motion controls still work better for aiming but people still think its okay to say mouse is better and not count motion controls
The reality is, mouse is damn close to motion controls, to the point where I bet if you could use both at the same time there’d be nothing standing between you and your aim. Even better, this hypothesis doesn’t require anything but off the shelf parts to try- a mouse and left joycon should be sufficient.
How about if you make the optical sensor stationary and you move a peice of the pad around attached to your thumb? I think that is probably an idea yall already had. Maybe not…lol i hope i win the prize
Hey its looking good. It sounds like you have a few problems from the reply’s. If you want something similar back in 07 I think the Frag FX was released if you can get your hands on one I can tell you that i loved it for the old PS3. Looks good so far. Here check it out on IGN
https://www.ign.com/articles/2007/06/25/splitfish-fragfx-ps3-review
I used to do something like this in PC games. Mouse for right hand and a playstation Move Navigation controller in the left (like a wiimote with a joystick). For games that has controls configurable enough to use both it was a dream.