It might not seem too impressive these days, but when microcontrollers with hardware USB support were more expensive and rare, the VUSB library was often used to create USB devices with an ATtiny85. It became so popular that the ATtiny85 even got packaged into USB dongle formfactors, like the DigiSpark boards. Well, you might not know this, but your Android smartphones can also work with USB mice and touchscreens in lieu of the built-in touchscreen display. [ErfanSn] combined these two ideas, creating a library to automate smartphone touchscreen events and keyboard input with an ATtiny85 — open for all of us to use, and with examples to spare.
The library is called DigiCombo, and it comes with plenty of examples for any screen touch event emulation that you might want. For instance, check out the README — it has video examples for Instagram page scrolling, unlock screen brute-forcing with random coordinates, playing the Stack rhythm game, and pinch zoom — all the building blocks for your smartphone touch emulation needs are covered pretty well! Of course, all of these have example code corresponding to them, that you can download and base your own ideas on. What’s more, the library is available in current Arduino IDE under the DigiCombo name. So if you need to, say, make a quick autoclicker for your phone, the library is a few steps away!
If your smartphone project was stalled because you needed to emulate touchscreen input, this library is your chance to get it done! We appreciate projects that let us get more from smartphones — there’s a lot of those laying around, they’re pretty functional and self-sufficient devices, so it makes sense that some projects of ours could do with a phone instead of a Raspberry Pi. Some manufacturers let us get a bit more of our phones, but this hasn’t really caught on, which means we have to make do with help of libraries like these. Or, perhaps, you rely on your phone day-to-day, and you’d like to add a touchpad to its back?
this is so amazing, it even works on ios without a hitch, beware where you plug your phone into ;)
very cool!
You have been able to plug devices into Android phones for a while. Assuming the device is supported natively or you have an app that works with it then you can plug in all sorts of devices. Mice, keyboards, flash drives or hard drives, even webcams and drone video receivers. Or an unusual one, my USB oscilloscope.
Apple devices used to only work with certain USB devices but that has changed now, even on an older iPad (air 2) I can now use a mouse, keyboard, USB hub and storage devices. The iPad has a power limit though so only very low power devices work unless you use a powered USB hub.
Links for example code and video are reversed.
Not clear to me as to how useful it is to have a tiny Arduino controlling the touchscreen of an old phone. Unless one created a program for the old phone that you wanted to control from an Arduino. If we are writing a program for the old phone wouldn’t we control it from the phone? Or couldn’t the Arduino run the program instead of running it on the old phone.
Now interfacing an ESP32 with WiFi and Bluetooth to your old phone might be interesting. But using the ESP32 to run existing apps on your old phone doesn’t make a lot of sense. Until we create a full phone touchscreen interface accessible from the web using an ESP32. Assuming you want to access a phone using a web interface into it. Android emulation already runs on Linux and Windows so you don’t need an old phone to run Android applications on a laptop or pc. Though you might not have the phones GPS, gyro, and Compass and a touchscreen on your laptop or pc.
You could program the Tiny to make a call or send a text message using the phone’s cellular connection. That might be cool if you have a spare active cellphone. Or you could periodically send the phones GPS location. Though one could write Android apps that run on the phone to do both of these.
If you didn’t have fingers to use the phone’s touchscreen this might be useful.
Thank you, fixed the links! and yeah, I’m pretty sure it’s for the cases when the program is not under your control. Also, if anyone’s interested in more control than this solution gives, they should check out scrcpy!
I have a perfect use case for this, many thanks
Really cool stuff, great job
Seems like a useful potential exploit if a USB device can drive the touchscreen interface to make unlock attempts or other such things.
One idea comes to mind.i dropped my phone once (some years ago) and it seemed to be fine at first, but then the digitizer was broken, so I could no longer unlock it. That phone also did not support OTG back then, so also no external mouse. With this I might have used it to unlock the phone and from there see what I could have done.
Sadly, this does rely on OTG. However, I do wonder if scrcpy would’ve help you – assuming existing adb trust being set up or whatever it is that’s needed
Doesn’t OTG need to be enabled from the keyboard? So this wouldn’t be possible if the touch screen was broken.
Will this let me cheat at Golf Clash, or other similar games?
Pressing a button on screen can started manually, but RELEASE must coincide with a swinging needle on screen being in the correct position. An external camera sensor needs to be added for that.