The first optical mice had to be used on a specially printed mousepad with a printed grid that the four-quadrant infrared sensor could detect. Later, mice swapped the infrared sensor for an optoelectric module (essentially a tiny, very low-resolution camera) and a powerful image processing. [8051enthusiast] was lying in bed one day when they decided to crack the firmware in their gaming mouse and eventually start streaming frames from the camera inside.
Step one was to analyze the protocol between the mouse and the host machine. Booting up a Windows VM and Wireshark allowed him to capture all the control transfers to the USB controller. Since it was a “programmable” gaming mouse that allowed a user to set macros, [8051enthusiast] could use the control transfers that would normally query that macro that had been set to return the memory at an arbitrary location. A little bit of tinkering later, and he now had a dump of the firmware. Looking at the most abundant bytes, it seems to match a profile similar to the Intel 8051. In a fascinating blur of reverse engineering, he traced the main structure of the program back from the function that sets the LED colors for the scroll wheel (which is dependent on the current DPI setting). Unfortunately, the firmware prevented the same macro mechanism from writing to arbitrary locations.
Looking through the code, a good old buffer overflow exploit seemed possible, but it caused the system to reset via watchdog. So he took another approach, invoking recovery mode and loading an entirely new firmware on the device, which a set_report control transfer can invoke.
Next, he moved onto the ADNS-9800 optical sensor (pictured in the top image provided by JACK Enterprises), which had a large encrypted blob in the firmware. Some poking around and deduction lead to a guess that the optical sensor was another 8051 system. With some clever reasoning and sheer determination, [8051enthusiast] was able to crack the XOR stream cipher encryption with a program that showed him versions of the disassembled assembly and allowed him to pick the one that was the most likely. With the firmware decrypted, he was able to see the encryption code and confirm his deducted algorithm.
With the sensor now cracked open, it was onto the 30 x 30 240 fps video stream. The sensor communicates over SPI, and the USB controller has to bit-bang the connection as it doesn’t have the hardware. Putting two custom firmware images on with a few extra functions was easy enough, but the 7 fps was somewhat lacking. The first optimization was loop unrolling and removing some sleeps in the firmware, which bought it up to 34 fps. By measuring the cycle counts of individual instructions, he was able to find some alternatives such as a mov instead of a setb that took one less cycle. Going from a 17 cycle loop to an 11 cycle loop and some other optimizations gave him 54 fps. Not content to stop there, he modified the ADNS-9800 firmware to continuously sample rather than waiting for the USB controller to finish processing. While this yielded 100 fps, there was still more to do: image compression. At a whopping 230 fps, [8051enthusiast] decided to call it done.
However, there was one last thing he wanted to do: control the mouse with the video stream. Writing some image processing into his Python-based program that received the image files allowed him to use the mouse, however impractically.
All in all, it’s an incredible journey by [8051enthusiast], and we would highly recommend reading the whole journey yourself. This isn’t the first time he’s modified the firmware of 8051-based devices, such as modifying the firmware of the WiFi chipset in his laptop.
[Thanks to JACK Enterprises over at Tindie for the use of the image of an ADNS9000].
Ever since sprite_tm’s optical mouse hack, I thought this might be possible. Sounds like an awesome project, will watch.
never mind that, what’s the point of the embedded video?
https://8051enthusiast.github.io/2020/04/14/002-Sensor_Firmware.html explains it
+1
Interesting hack though.
The 8051 is the ISA that will not die. It’s now over forty years old, and even though Intel have stopped making real ones (in 2007!) there are like a bajillion derivatives and clones available everywhere, and some of them are terrifyingly fast (450MHz, one instruction per clock cycle). The memory architecture is weird, with main memory only accessible indirectly via I/O instructions. Sadly they’re strictly Harvard architecture making them tricky to run real operating systems on, or they’d make an interesting homebrew 8-bit computer.
I am amazed that it still lives, given that there are better options for everything it can do, capability wise and price wise, but when I first used one, I was amazed at what it could do. I still have a few 8031 and 8751 around, but have not touched one in maybe 30 years.
I really don’t miss dealing with this architecture, but am deeply impressed with how far this project went.
Actually, I’m an idiot — it’d be totally possible to wire the external data and program buses together and connect them to a single SRAM chip; then a CP/M-like operating system would be completely viable; or, with memory banking, Fuzix might work. It looks like Maxim still make romless 8051 clones in 40-pin DIP for about $10 a piece. It’d be the most pointless retro computer ever!
Pointless and now, sir, mandatory.
You wizards that have learned how to reverse engineer systems like this simply amaze me. Most of us “aren’t worthy”! :)
The 8051 has bit addressable memory space. Using that one could unroll a SPI loop to read/write each bit to/from GPIO directly without needing the bit shift instruction.
There are some clever instructions in the 8051, but the ISA suffers from only having 1 index pointer. Most of the clones have added additional ones and use pipelining to increase the IPC.
Brings me back to the great old 1990s: In that tiny slice of history between when optical imaging mice became available and DOS became extinct, INT 0x33 is (was) the mouse interface, and one helpful function dumped the whole 32×32 image array. You could repeat that as fast as your serial port could inhale it (several frames/sec).
The field of view on the mouse pad (or piece of paper) was quite small (a couple of millimeters), but it was pretty easy to cross-correlate successive frames and build up a large image of whatever the mouse was over: I made it into kind of a poor-man’s page scanner. Fun times. Yeah, I had no life.
(Wow)^n
(Wow)^n+1
wow↑↑↑↑↑wow
I thought the image format was well documented…? I have code that displays the ADNS-3080/9500/9800 video in a browser to debug an optical flow project as well as to focus the lens.
I liked those mousepads. They looked like the crystalline scifi version of carbon fiber.
And fun to play with: rotate all the pads 90 degrees first thing on a Monday morning, and watch the confusion when people come in and all the mice don’t work. That only worked a few time though, and stopped entirely when someone clued in and taped all the pads down :-)
Oh dear, my comment was deleted. Useful to know that that actually happens. Still, could someone fix “lead” -> “led”. That would be great. Thanks.
first_time?.jpg
Now it just needs a 2 axis CNC control to become an image scanner. Image sensor is 30 pixels wide, 240 frames per second. If the positioning is precise that could be a 7200 x 30 pixel stripe in 1 second. Use bi-directional scanning and it could do a 720×720 image in around 120 seconds.
sure, but it would actually be slower as there is no mov bit, bit instruction and while the rlc a instruction takes just one cycle on the controller, the mov bit, c instruction takes three cycles and the mov direct, a instruction takes 2 cycles, so it’s faster to keep it in the A register and not directly write it to memory