Ha, good timing. Yesterday and today I’ve been knocking together a similar idea, but using face tracking and a webcam to control where the player is looking in an FPS. Move your head up and left to look up and left, for example. It works pretty well, although it definitely needs a bit more smoothing in order to be usable. I’m gonna post it somewhere once it is “finished” and hopefully people can make some improvements.

I like the idea of using face tracking simply because nobody likes wearing stuff on their head so that they can play, although I reckon that’ll change as 3D TV becomes more mainstream.

Posted at 12:04 pm on Feb 6th, 2009 by dan

@dan

How do you find the speed/response of the face tracking? Are you using openCV?

Posted at 12:17 pm on Feb 6th, 2009 by max

the common problem with these setups is that, while you are in fact moving your head to change your view, your “window” into the world stays stationary. You end up looking sideways at your screen most of the time.

Wearing an HMD, or doing dome projection are the only ways I can think of to overcome this.

Posted at 12:29 pm on Feb 6th, 2009 by Caleb Kraft

i’m actually using a commercial SDK called VeriLook, made by Neurotechnology:

http://www.neurotechnology.com/pc-based-face-recognition.html

I’m mainly using this because I use it at work every day and am very familiar with it. It provides detection and recognition, but I’m only using the detection classes. It also handles webcams rather nicely, which is good. I’m writing the code in C#.

Anyway, face detection works fantastically; on my Core2Duo system at work the limiting factor is the webcam’s framerate. It isn’t so great on my 1.66Ghz Atom netbook when running side by side with Quake 3, but without Quake running it is fine.

I’m basically just using user32.dll to send mouse movement events when the face moves, so you can use it to mouse around the windows GUI. This actually works better than within a game right now; when the mouse cursor is moving you don’t notice the jerkiness too much, whereas in Quake you notice it a lot – jumping from view to view sort of disrupts the optical illusion of motion and makes it look like the game is running at a low framerate.

The problem with jerkiness isn’t inherent to the concept, it is just resulting from me recycling old classes that i was originally using to control a couple of servos rather than a mouse. once i get around to sorting it out it should be great.

Posted at 12:37 pm on Feb 6th, 2009 by dan

“gaming visuals and interactivity have come leaps and bounds while the controllers themselves have basically just added a few buttons, he explains his goals. ”

thats retarded. controllers havnt changed much because they just plain work. food has changed so much in the last 100 years, maybe we should reinvent the fork too!

Posted at 1:01 pm on Feb 6th, 2009 by hamburglar

“thats retarded. controllers havnt changed much because they just plain work. food has changed so much in the last 100 years, maybe we should reinvent the fork too!”

Your completely right about controllers working, but what I mean is that your still just sitting on your couch twiddling a couple fingers. Sure, there are incredible advancements in graphics, but your still interacting the same as you did 20 years ago, albeit with a couple more controls.

You can choose to keep on going using a controller, or play them in a new interactive and immersive way. But its true, controllers will most likely always be the standardized way of playing, and as you can see in the original xbox design, the controller is still the form of interaction, the choice to use a DDR pad is as simple as hooking it up when you want and disconnecting it at the more difficult boss. Its completely up to the player.

Posted at 6:52 pm on Feb 6th, 2009 by MikeFez

I have been thinking of a similar project as face tracking but would not eye tracking be better? At least as far as aiming is concerned in an FPS.

Posted at 8:09 pm on Feb 7th, 2009 by Sethgb

I think a sort of Wrap around or multiple tv array would be better for Aiming and tracking instead of the head motions sensor. Perhaps some kind of a giant mouse pad for the movement. Suspend someone by the waist and place a kind of slippery surface under them. Perhaps a velvet on velvet or an inflated ball on ball bearing, so you could walk and still be kind of flat footed… Maybe a kind of air-track. Like a fabric pad over an air-hockey table would work.

Posted at 1:56 am on Feb 8th, 2009 by GuyPaterson

With the slippery surface thinger, you could just place a Mouse right on it for the movement. I don’t know, Brainstorming. Seems like a good project!

Posted at 2:00 am on Feb 8th, 2009 by GuyPaterson

But yeah, we do need a better controller for consuls… Going from an FPS on a computer and to a consul, Murder.. A good 3Hour learning curve before you get your “groove” back.

Posted at 2:02 am on Feb 8th, 2009 by GuyPaterson

This looks similar to my project that hack a day covered last year:
http://blog.insightvr.com/?p=13

I don’t have a ddr pad, but I do have a laser gun for shooting and wiiMote headtracking. Oh, and I wrote my own 3d engine.

Posted at 8:34 pm on Feb 8th, 2009 by arandomjohn