Controlling A Mouse With Your Voice

It’s entirely possible to use a computer without the aid of a mouse or trackpad. Shift and arrow keys will get you very far, but that is entirely too taxing. [Stephen] came up with a really neat way to control a mouse with your voice, a project that is sure to find its way onto the desktops of those with mobility issues very quickly

The voice controlled mouse works in conjunction with the voice recognition built into OS X, a little AppleScript, and a touch of Python. When the user says, ‘show grid’ a 10 by 10 grid numbered 1 to 100 is displayed on the screen. By saying ‘thirty five,’ the cursor moves to the 35th cell in the grid. From there, the mouse can be controlled by speaking cardinal directions such as South and Northwest.

[Stephen] put up a very clever demo of his Voice Mouse project available after the break. Even though he did have a little difficulty with his mac recognizing a few of his spoken commands its light years ahead of trying to navigate the web with just shift and arrow keys.

[youtube=http://www.youtube.com/watch?v=NvX57ubQHiE&w=470]

25 thoughts on “Controlling A Mouse With Your Voice

  1. I was just thinking about that idea a couple of days ago. What a boon for quadriplegics or other (temporarily) incapacitated/bedridden people by allowing them to communicate via computer with people or robotic assistants.

  2. Not to disparage [Stephen], but this is exactly how the built-in voice recognition in Windows 7 works. One minor difference in that the Win7 voice control uses subdivided grids, not cardinal directions after the rough target is selected.

      1. just goto wase of access in control panel. i used it for the better part of a year after my hp mini drank a pepsi then a day later drank some coffee and refused to let me type on it anymore. works great. nice to see a homebrewed variety.

    1. Yeah I was going to say, I remember this exact process being called up through the “mouse grid” voice command several years ago with Dragon.

      I understand the value in doing something yourself, but this seems mostly inferior.

  3. Anyone else here ever read Snowcrash by Neal Stephenson? There is a character that is severely disabled and drives his vehicle using his voice. Not actual spoken words but various sounds. This seems like it would be much better for moving a mouse around. It could be customized for each person so as not to conflict with other people working together in an office. i.e “veet”/”voot” for verticle movement and “sheet”/”shoot” for horizontal. The grid concept is interesting for large grained moves but I’ll bet the user would’t even need to see the grid after a while, he would just know the relative locations so something like “gee 35” would jump there. Shortcuts like this could make it very usable.

Leave a Reply to VtCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.