It’s entirely possible to use a computer without the aid of a mouse or trackpad. Shift and arrow keys will get you very far, but that is entirely too taxing. [Stephen] came up with a really neat way to control a mouse with your voice, a project that is sure to find its way onto the desktops of those with mobility issues very quickly
The voice controlled mouse works in conjunction with the voice recognition built into OS X, a little AppleScript, and a touch of Python. When the user says, ‘show grid’ a 10 by 10 grid numbered 1 to 100 is displayed on the screen. By saying ‘thirty five,’ the cursor moves to the 35th cell in the grid. From there, the mouse can be controlled by speaking cardinal directions such as South and Northwest.
[Stephen] put up a very clever demo of his Voice Mouse project available after the break. Even though he did have a little difficulty with his mac recognizing a few of his spoken commands its light years ahead of trying to navigate the web with just shift and arrow keys.
25 thoughts on “Controlling A Mouse With Your Voice”
I was just thinking about that idea a couple of days ago. What a boon for quadriplegics or other (temporarily) incapacitated/bedridden people by allowing them to communicate via computer with people or robotic assistants.
Useful for interacting with elements that don’t accept tabs and such like. E.g flash!
I love the grid within a grid. A great idea.
I can’t help but respond with “You Sunk My Battle Ship”
I tried, I really tried.
Not to disparage [Stephen], but this is exactly how the built-in voice recognition in Windows 7 works. One minor difference in that the Win7 voice control uses subdivided grids, not cardinal directions after the rough target is selected.
This is built-in win7? how do I access it?!
It actually works really well, I used it a while back to interface with the spaceship sim X3 because I thought yelling at digital spaceships would be awesome.
Fact: yelling at digital spaceships is awesome (but that’s a tangent).
Press “Start” and type “Windows Speech Recognition”
“Speech” works just fine as well.
I remember playing about with this in the Vista Beta, it was quite interesting! More info on the mouse grid can be found here:
Control Panel\Ease of Access\Speech Recognition
just goto wase of access in control panel. i used it for the better part of a year after my hp mini drank a pepsi then a day later drank some coffee and refused to let me type on it anymore. works great. nice to see a homebrewed variety.
Dragon dictation software has had this feature for some years. Nevertheless this is some good coding.
Shame it is still quicker to use a mouse.
Yeah I was going to say, I remember this exact process being called up through the “mouse grid” voice command several years ago with Dragon.
I understand the value in doing something yourself, but this seems mostly inferior.
It seems like with a slightly more accurate speech recognition module (like Dragon), you could get used to it after awhile.
make it accessable via keyboard and this is a must-have for even potent people.
And nobody has yet referred to this – http://www.youtube.com/watch?feature=player_detailpage&v=QkcU0gwZUdg#t=39s
And you call yourself geeks? Shame on you!
Few days ago we have had Voight-Kampff test and now this.
Geek-fu is week in summer, I would say.
something windoze 7 had for quite long. I liked it there but at least with my mics the recognition quality sucked a bit.. but usable enough.
“Show grid. 47. 3. South. Click left”
Just leaving it here. Blade runner as f**k
This has already been done as part of the open source project ‘simon’
Watch from 3:18
Dragon: “Naturally Speaking” has been using that method for over 15 years
nancy: i want this project as its my final yr project topic.. can you plz provide me the source code and SRS of this project??
Is there a way to voice control just the mouse clicks? Not the movement.
Anyone else here ever read Snowcrash by Neal Stephenson? There is a character that is severely disabled and drives his vehicle using his voice. Not actual spoken words but various sounds. This seems like it would be much better for moving a mouse around. It could be customized for each person so as not to conflict with other people working together in an office. i.e “veet”/”voot” for verticle movement and “sheet”/”shoot” for horizontal. The grid concept is interesting for large grained moves but I’ll bet the user would’t even need to see the grid after a while, he would just know the relative locations so something like “gee 35” would jump there. Shortcuts like this could make it very usable.
Please be kind and respectful to help make the comments section excellent. (Comment Policy)