Controlling a mouse with your voice

It’s entirely possible to use a computer without the aid of a mouse or trackpad. Shift and arrow keys will get you very far, but that is entirely too taxing. [Stephen] came up with a really neat way to control a mouse with your voice, a project that is sure to find its way onto the desktops of those with mobility issues very quickly

The voice controlled mouse works in conjunction with the voice recognition built into OS X, a little AppleScript, and a touch of Python. When the user says, ‘show grid’ a 10 by 10 grid numbered 1 to 100 is displayed on the screen. By saying ‘thirty five,’ the cursor moves to the 35th cell in the grid. From there, the mouse can be controlled by speaking cardinal directions such as South and Northwest.

[Stephen] put up a very clever demo of his Voice Mouse project available after the break. Even though he did have a little difficulty with his mac recognizing a few of his spoken commands its light years ahead of trying to navigate the web with just shift and arrow keys.


  1. Ren says:

    I was just thinking about that idea a couple of days ago. What a boon for quadriplegics or other (temporarily) incapacitated/bedridden people by allowing them to communicate via computer with people or robotic assistants.

  2. RobinJood says:

    Useful for interacting with elements that don’t accept tabs and such like. E.g flash!

  3. Zee says:

    I love the grid within a grid. A great idea.

  4. Cabe says:

    I can’t help but respond with “You Sunk My Battle Ship”

    I tried, I really tried.

  5. Not to disparage [Stephen], but this is exactly how the built-in voice recognition in Windows 7 works. One minor difference in that the Win7 voice control uses subdivided grids, not cardinal directions after the rough target is selected.

  6. Vt says:

    Dragon dictation software has had this feature for some years. Nevertheless this is some good coding.
    Shame it is still quicker to use a mouse.

    • Xyroze says:

      Yeah I was going to say, I remember this exact process being called up through the “mouse grid” voice command several years ago with Dragon.

      I understand the value in doing something yourself, but this seems mostly inferior.

    • thatcherc says:

      It seems like with a slightly more accurate speech recognition module (like Dragon), you could get used to it after awhile.

  7. kidna says:

    make it accessable via keyboard and this is a must-have for even potent people.

  8. wholostwhat says:

    And nobody has yet referred to this –

    And you call yourself geeks? Shame on you!

  9. jc says:

    something windoze 7 had for quite long. I liked it there but at least with my mics the recognition quality sucked a bit.. but usable enough.

  10. brad says:

    “Show grid. 47. 3. South. Click left”

    Boom headshot!

  11. Wizzard says:

    Just leaving it here. Blade runner as f**k

  12. spacevs says:

    This has already been done as part of the open source project ‘simon’

    Watch from 3:18

  13. Rick Keller says:

    Already done.

    Dragon: “Naturally Speaking” has been using that method for over 15 years

  14. Kevin B says:

    norf east

  15. debjani tripura says:

    nancy: i want this project as its my final yr project topic.. can you plz provide me the source code and SRS of this project??

  16. Don Leever says:

    Is there a way to voice control just the mouse clicks? Not the movement.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 96,687 other followers