Anyone who has played an online shooter game in the past two or three decades has almost certainly come across a person or machine that cheats at the game by auto-aiming. For newer games with anti-cheat, this is less of a problem, but older games like Team Fortress have been effectively ruined by these aimbots. These types of cheats are usually done in software, though, and [Kamal] wondered if he would be able to build an aim bot that works directly on the hardware instead.
First, we’ll remind everyone frustrated with the state of games like TF2 that this is a proof-of-concept robot that is unlikely to make any aimbots worse or more common in any games. This is mostly because [Kamal] is training his machine to work in Aim Lab, a first-person shooter training simulation, and not in a real multiplayer videogame. The robot works by taking a screenshot of his computer in Python and passing the information through a computer vision algorithm which recognizes high-contrast targets. From there a PID controller is used to tell a series of omniwheels attached to the mouse where to point, and when the cursor is in the hitbox a mouse click is triggered.
While it might seem straightforward, building the robot and then, more importantly, tuning the PID controller took [Kamal] over two months before he was able to rival pro-FPS shooters at the aim trainer. It’s an impressive build though, and if one of his omniwheel motors hadn’t burned out it may have exceeded the top human scores on the platform. If you would like a bot that makes you worse at a game instead of better, though, head over to this build which plays Valorant by using two computers to pass game information between.
This is very impressive, and something I couldn’t do. I do have a question, kind of along the lines of, “You could have done that with a 555.” Was there a need for the physical mouse and all the motors? Could this have been done with a 34U4? It seems like it would have been easier to do. But, then again, on some of my own projects, I do them, “not because they are easy, but because they are hard.”
games could always report your vendor/device ids to the moderators if they suspect you of cheating. you could at least detect known dev boards, virtual mouse drivers, and both hardware and software cheat tools.
i think id have done this by using the pcb from an old ball mouse. simply remove the encoders and buttons and wire them directly to the arduino. then just stick it in a project box and plug it into the computer. or cut the mouse entirely and wire directly to the ps2 port. i dont think the ps2 protocol has any provisions for firmware device ids.
no method would hold up to behavioral analysis. you simply play differently when you have to operate the cheat tools. and you play way differently if you let the computer do it for you.
You can set the VID/PID on most micro controllers with USB. Faking the whole discriptor to be exactly like a mouse is easy.
But your movement would still be very exact and not human like. By going the full physical route, it might be very hard to distinct from a top human player.
You can introduce a bit of jitter and randomness in software, no problem.
/u/jfedor has made a passthrough remapper/ dpi scaler/ overclocker device for usb mice and trackballs from a raspberry pico. it’s the perfect candidate for this.
Software random is never as good as real world random though – bound to be a detectable pattern if you look for it.
As this is interfacing with a physical mouse, working on an imperfect surface the random baked into its function aught to make it much harder to prove its not human.
Software randomness can be statistically processed to be indistinguishable from true randomness, especially when there is so little data for the adversary to observe (and any reasonable amount of game playing is only going to be a “small” amount of data.)
Wrt /u/Bryce+Schroeder, wouldn’t approaching true randomness be a fairly detectable signal in and of itself? I know from the heatmap vizualization from a touchpad driver (Synaptics, maybe?) that my inputs are nowhere near random. I think the lack of randomness comes from ergonomic considerations (some areas of a touchpad or mousepad are just less comfortable).
I remember seeing something about a website that uses physical dice being “rolled” by a machine that brings the dice past a camera on a vertical conveyor. The system uses AI of some sort to identify the dice and get the numbers on the dice to seed a random number generation system. I also remember a system where a wall of lava lamps were “watched” by an AI that turned the blobs of the lava lamps into random numbers.
I don’t think either process would be detectable.
I’m wondering if attaching the sensor to a x axis on top of a conveyor belt would do anything? Or perching it on a ball.
Or dare I say putting LCD displays in the optic sensors of a ball mouse?
Hmm, interesting, interesting.
In the past I’ve seen a “reverse mouse ball” in some projects.
Instead of the encoders, there were small electro motors to rotate the (also reversed) mouse ball, and it could rotate the ball in both X and Y axis for long distances.
It was a quite professionally made product and it was used in research on insects and how they find their direction to get attracted to crop fields, by logging the direction in which they try to move after receiving some stimuli.
I would have done it with a stripped down old mechanical ball mouse and then all you would have to do is drive the 2 slotted wheels for the 2 optical sensors. It would cut the hardware down by half and it would make the PID easier to control
Actually the sample rate of a mouse with rotary encoders is too slow and moving too fast will lead to incorrect speed and direction. The cursor will just bounce around or will even go in the wrong direction. This reminds me on my first gaming experience back in the days.
I’m thinking of the Android/Ipad game, Golf Clash. You press down on the screen to activate a swinging needle, and release when the needle in the EXACT centre. Variations to left or right give you hook or slice.
You’d need an XY plotter gantry to do this on a phone or tablet. Or, you could run it in Windows using an emulator.
Either way, you’d need an option to deliberately *NOT* hit exact centre, both for deliberate hooks / slices and to help evade detection.
Android supports USB mice.
very cool ! Curious – instead of moving the mouse, what about moving the pad below it ? A pair of linear actuators might be easier ? ( steal it from a printer ! )
Beat me to it :)
Me too. Great minds.
Fun demonstration, cheaters would go way out of their lengths to get their “fix”.
Like the most professional cheats for OW/Valorant/Apex can go as high as thousands of dollars subscription per month and you will get banned at the end guaranteed.
Cheating is like a drug once you got the hang of it even in a single player game you will go in all the way you wont turn it off. However in these modern team shooters even cheating won’t guarantee for you to win, far from it. We regularly humiliated with aimbotters in both ow and valo and in apex I actually played some games together with an aim/esper regardless that he got lucky in one we lost 5 in a row after that because the odds go against you.
The time and energy you put into cheating could be well spent on trying to actually get “better” instead of being worried all the time when to turn on and off your hack and when to fake your movement etc.
Interesting!
But why not just move the mouse pad instead of the mouse?
Maybe put the mouse in a gantry over the print bed of a 3d printer and control it this way…
or lay a dual monitor down flat and display a moving picture of a mouse pad to trick the mouse into moving.
Yesd….
Why move the whole mouse around instead of just, for instance, mounting a ball beneath the sensor with two motors to drive the ball to make the sensor think it’s moving around?
lol. i just made this same comment before i read yours. oops. oh well.
would be faster and easier to control if you just put a ball underneath the sensor. moving the whole mouse is commendable. however my best and fastest aim was always with a ball mouse where my thumb controlled the ball. too bad they dont make em with high quality sensors. if i had time to tinker id rip a sensor out of a modern mouse and stuff it in the case for the old logitech ergo mouse.
less mas, faster response, tighter controls, less surface area to navigate.
how about jsut using one of the erg ball mice. .( ball on top ) and make a bot that sits on that.. this way it doesnt need to move and you can just use rubber wheels instead of omniwheels. 2 would be enough . and its fully phsical .. if the control wheels get a bit dirty that would add the jitter and randomness. and be non moving so easy enough to have a physical clicker to press teh buttons.
a camera or a capture card to translate teh video ..
i can see this being a goo dmanual miner on dual universe for surface mining. as its tedious time consuming work
This is an awesome project! i was able to do this with an arduino leonardo but without the physical hardware. this would be so fun to replicate! just need to add those wheels or maybe like a mini 3d printer with the mouse sensor: https://www.slyautomation.com/blog/arduino-ide-aimbot-in-aimlab-using-arduino-leonardo/