Flute Now Included On List Of Human Interface Devices

April 24, 2024 by Bryan Cockfield 13 Comments

For decades now, we’ve been able to quickly and reliably interface musical instruments to computers. These tools have generally made making and recording music much easier, but they’ve also opened up a number of other out-of-the-box ideas we might not otherwise see or even think about. For example, [Joren] recently built a human interface device that lets him control a computer’s cursor using a flute instead of the traditional mouse.

Rather than using a MIDI interface, [Joren] is using an RP2040 chip to listen to the flute, process the audio, and interpret that audio before finally sending relevant commands to control the computer’s mouse pointer. The chip is capable of acting as a mouse on its own, but it did have a problem performing floating point calculations to the audio. This was solved by converting these calculations into much faster fixed point calculations instead. With a processing improvement of around five orders of magnitude, this change allows the small microcontroller to perform all of the audio processing.

[Joren] also built a Chrome browser extension that lets a flute player move a virtual cursor of sorts (not the computer’s actual cursor) from within the browser, allowing those without physical hardware to try out their flute-to-mouse skills. If you prefer your human interface device to be larger, louder, and more trombone-shaped we also have a trombone-based HID for those who play the game Trombone Champ.

Latency Meter For Accurate Gaming

September 19, 2023 by Bryan Cockfield 15 Comments

The gaming world experienced a bit of a resurgence in 2020 that is still seen in the present day. Even putting aside the effects from the pandemic, the affordability and accessibility has arguably never been better. Building a gaming PC can have its downsides, though, and a challenging issue to troubleshoot is input lag or input latency. This is something that’s best measured with standalone hardware, and if this is an issue on your setup you may want to take a look at this latency meter.

Unlike other measurement devices that use the time between a mouse button input and the monitor’s display of a bullet or shooting event, this one looks at mouse movement and the change in the scene instead. This makes it much more versatile than other methods since it’s independent of specific actions, and can be used in any game without any specific events needed to perform the measurement. A ~~camera~~ phototransistor is placed on the monitor’s top edge and the Arduino-based device sends mouse commands to the computer while measuring the time between those commands and the shift in the image on the monitor.

The project is open source, so with the right hardware it’s possible to build one to troubleshoot latency issues or just to learn more about a particular hardware configuration’s behavior. Arduinos and other microcontrollers have been doing all kinds of things by pretending to be human interface devices like this for a while now. One of our favorites of late was this effects pedal that replicates musical effects on mice and keyboards.

Vintage Remote Control Gets Bluetooth Upgrade

July 4, 2021 by Tom Nardi 20 Comments

This swanky Magnavox remote is old enough to predate the use of infrared, and actually relies on ultrasound to communicate with the television. It’s a neat conversation starter, but not terribly useful today. Which is why [Chad Lawson] decided to gut the original electronics and replace it with a Adafruit Feather 32u4 Bluefruit LE that can actually talk to modern devices.

We know, we know. Some in the audience will probably take offense to such a cool gadget being unceremoniously torn apart, but to be fair, [Chad] does say he has a second one that will remain in its original state. Plus a quick check on eBay shows these old remotes don’t seem to be particularly rare or valuable. In fact, after some browsing through the recently concluded auctions, we’re fairly sure he paid $27 USD for both of these remotes.

Anyway, [Chad] found that a piece of perfboard in his collection just happened to be nearly the same size as the PCB from the remote, which made the rest of the conversion pretty straightforward. He simply had to mount tactile switches on one side of the perfboard so the remote’s original buttons would hit them when pressed, and then wire those to the Adafruit on the other side. We know there’s a 3.7 V 500 mAh pouch battery in there someplace as well, though it’s not immediately clear where he hid it in the images.

The code [Chad] came up with tells the Adafruit to mimic a Bluetooth Human Interface Device (HID) and send standard key codes to whatever device pairs with it. That makes it easy to use as a media remote on the computer, for example. We’ve seen something similar done with the ESP32, if you’ve already got one in the parts bin and are looking to revamp a remote control of your own.

At the end of the write-up, [Chad] mentions he may try developing an ultrasonic receiver that can pick up the signals from the unmodified remote control. That would be a nice way to bring this whole thing full circle, and should appease even the most hardcore vintage remote control aficionados.

This ESP32 Bluetooth Page Turner Can’t Get Any Easier

May 27, 2021 by Tom Nardi 8 Comments

Commercial Bluetooth pedals, designed to allow musicians to flip pages of sheet music on a tablet, have the sort of inflated price tag you’d expect for a niche electronic device. Rather than forking as much as $100 USD over for the privilege of hands-free page flipping, [Joonas Pihlajamaa] decided to build his own extremely low cost version using an ESP32 and a cheap foot pedal switch.

In terms of hardware, it does’t get much easier than this. All [Joonas] had to do was hook the pedal up to one of the ESP32’s digital pins, and plug the microcontroller into a USB power bank. From there, it became a software project. With the ESP32-BLE-Keyboard library, it only took a few lines of code to send RIGHT_ARROW or LEFT_ARROW depending on whether the pedal was quickly tapped or held down for a bit; allowing him to navigate back and forth through the pages with just one button.

[Joonas] mentions that the ESP32 development board he’s using is too large to fit inside the pedal itself, though we wonder if the bare module could get slipped in there someplace. Of course you could always build your own pedal with a bit of extra room to fit the electronics, but for less than $2 USD on AliExpress, it’s hard to go wrong with this turn-key unit.

Looking for an alternate approach? We covered a Bluetooth page turner last month that doubled the inputs and packed it all into a handsome wooden enclosure.

Continue reading “This ESP32 Bluetooth Page Turner Can’t Get Any Easier” →

Flashpen Is A High Fidelity Pen Input Device

April 1, 2021 by Lewin Day 10 Comments

Pen input has never really taken off in the computing mainstream, though it’s had somewhat of a renaissance in the last decade or so. Various smartphones and tablets are shipping with the technology, and some diehard users swear by it as the best way to take notes on the go. Recently, researchers at the Sensing, Interaction and Perception Lab at ETH Zurich have been working on Flashpen, a high-fidelity pen interface for a wide range of applications.

The fundamental technology behind the pen is simple, with the device using an optical flow sensor harvested from a high-end gaming mouse. This is a device that uses an image sensor to detect the motion of the sensor itself across a surface. Working at an update rate of 8 KHz, it eclipses other devices in the market from manufacturers such as Wacom that typically operate at rates closer to 200Hz. The optical sensor is mounted to a plastic joint that allows the user to hold the pen at a natural angle while keeping the sensor parallel to the writing surface. There’s also a reflective sensor on the pen tip which allows cameras to track its position in space, for use in combination with VR technology.

The team show off the device being used in several ways, primarily in VR tasks, but also in simple handwriting and coloring work. It’s a project that could readily be replicated by any eager experimenter by gutting a gaming mouse and getting down to work; our writers will expect six of your submissions by June 1st to the tipsline. Those eager to learn more can check out the project paper, and may also find the team’s TapID technology interesting. Video after the break. Continue reading “Flashpen Is A High Fidelity Pen Input Device” →

Bone Vibration Brings Typing Into VR

March 16, 2021 by Lewin Day 17 Comments

Virtual reality is becoming more of a thing, now that we have high quality headsets and the computing power to generate attractive environments. Many VR systems use controllers held in mid air, or camera-based systems that track limbs and hands for interaction. However, productivity scenarios often require prolonged interactions over a long period of time, which typically necessitates working at surfaces that allow the body to rest intermittently. To help facilitate this, a group of researchers at ETH Zurich developed TapID, including a preprint paper (PDF) that will be presented at IEEE VR 2021 later this month.

TapID consists of a wristband that carries two motion sensors, with one worn on each wrist. This allows TapID to detect taps from each of the user’s fingers individually, thanks to a machine learning algorithm that analyses the unique vibrations through your skeletal system. This is demonstrated as being useful for VR environments, where the user can type into a virtual keyboard, or interact with virtual objects on a surface, using their fingers as they would in the real world. This is a sensor fusion with the features of modern VR headsets that include hand tracking. The TapID wristbands deliver granularity and detection of small motions that is not nearly as accurate through headset-mounted senors and camera-based detection.

Test hardware includes 4 accelerometers. Two on flexible PCBs are the sensing hardware used by the system, the other two on the rigid PCB are used as a baseline during testing but do not contribute to the tap detection.

We’re not entirely convinced of the utility of sitting down in a virtual environment to type at a fake keyboard when monitors and real keyboards are more tactile and cheaper. However, having a device that can accurately determine individual finger interactions is sure to have applications in VR. And whether or not the demonstrated use cases are viable, the technology does indeed work.

It’s exciting to see the wrist-band form factor. It brings to mind the possibility of improving tap interactions in smart watches for non-VR uses. We envision chorded keyboard type gestures that detect which fingers are tapping but don’t need positional accuracy.

Those experimenting in VR interfaces may find it useful to reverse engineer what’s already out on the market, as we’ve featured before. Or, you can simply build your own! Video after the break.

Continue reading “Bone Vibration Brings Typing Into VR” →

A Gesture Recognizing Armband

January 2, 2021 by Danie Conradie 4 Comments

Gesture recognition usually involves some sort of optical system watching your hands, but researchers at UC Berkeley took a different approach. Instead they are monitoring the electrical signals in the forearm that control the muscles, and creating a machine learning model to recognize hand gestures.

The sensor system is a flexible PET armband with 64 electrodes screen printed onto it in silver conductive ink, attached to a standalone AI processing module. Since everyone’s arm is slightly different, the system needs to be trained for a specific user, but that also means that the specific electrical signals don’t have to be isolated as it learns to recognize patterns.

The challenging part of this is that the patterns don’t remain constant over time, and will change depending on factors such as sweat, arm position, and even just biological changes. To deal with this the model can update itself on the device over time as the signal changes. Another part of this research that we appreciate is that all the inferencing, training, and updating happens locally on the AI chip in the armband. There is no need to send data to an external device or the “cloud” for processing, updating, or third-party data mining. Unfortunately the research paper with all the details is behind a paywall.

Continue reading “A Gesture Recognizing Armband” →