Facial recognition software can tell when you’re frustrated with Xbox Live

Most of us have been faced with the anguish of being shot in the head repeatedly by 12-year-olds. There are also the times when we’re overjoyed by defeating the Mother Brain and making it out of the caverns of┬áZebes. If we wanted to scientifically quantify how happy, sad, or angry we are while playing video games, we wouldn’t know what to do. [Dale] came up with a very interesting way to gauge someone’s state of mind while either playing Xbox, or watching TV.

To get a measure of how happy or sad he is, [Dale] put a webcam underneath his TV and pointed it towards his couch. Every 15 seconds or so, the webcam snaps a picture and sends if off to the face.com API. After face.com sends a blob of JSON containing information about all the faces detected in the photo, a short Python script plots it on a graph.

[Dale] admits he’s not entirely scientific with this project; the low resolution of the webcam, coupled with images being captured every 15 seconds means he runs into the limitations of his hardware very quickly. Also, there’s the confound of [Dale] paying attention to something else in the room – like his kids – rather than the TV. Still, it’s an interesting use of hardware and software that would be loved by a market researcher or QA designer.

20 thoughts on “Facial recognition software can tell when you’re frustrated with Xbox Live

    1. I’d say it’s an accurate graph of my gaming experience, most of the time.

      One day I’ll wake up and realize my gaming time is more stressful than my average work day. ;)

  1. So game makers should look at that graph with horror. Basically games make you angry 94% of the time and make you happy 4% of the time. I dont think surprise is really worth while to have on there since surprise is an instantaneous reaction that quickly fades more then likely before the 15 second interval between pics.

    Great idea.

    1. I am reminded of a time I was walking down the Vegas strip, and somebody stopped and offered me dinner and free show tickets to participate in a “survey.” The “survey” was actually the review of the pilot of a proposed television sitcom.

      I was led into a cubicle farm, each cube equipped with a computer, a set of headphones, and a hand-held switch. My instructions were to watch the show. When I felt happy, amused, or entertained, I was to push the green button. If I was offended, bored, or otherwise disgruntled, I was to push the red button.

      At the end of the exercise, I they tallied the button presses of many hundreds of people, and created a moving strip chart graph that was superimposed on the program. This would allow developers to review the pilot and simultaneously see what “humor” worked with the audience and what did not.

      It seems that the gizmo in this HAD post could be used to automate or auto-collect the same kind of information.

      The sitcom, by the way, was unimaginative and unfunny crap. That is to say, you should expect to see it on network TV any day now ;)

  2. I could see this technology being used in a Game like deadspace. If the game thought you were not scared SH**less from scary monsters it could throw more your way.

  3. This would be useful for a lot of games. If a particular point in a level has gotten too difficult and you just can’t get past it, it would be nice if the games were smart enough to know to make it just a little easier. Of, course then go back to normal.

    Then again, the game could simply track that by the number of repeated tries.

    Nothing is worse than being stuck at a point in a game and just finally giving up and not playing that game anymore.

    1. This is what happened to me with the original Alice :( There was probably tons of beautiful game I never got to play cause I didn’t have the skills.

  4. Yeah, now ms will be tracking for market research, and all we’ll ever get are happy Smiley game 7… not realizing that the journey , story and drama are more to a good game than happy

  5. Would be a fantastic use of the kinect! I always assumed games were supposed to frustrate a certain amount, otherwise there’s no satisfaction from actually completing it. Look at vvvvvv!

  6. Very good thing, if the player gets VERY frustrated AND a vibrational sensor inside the console (The player beats it), some automatic difficulty level decrease could be triggered, or an extra life (action) or deal (card) could be given. Better spoiling a cheap game than the expensive hardware, (or even life as the console contains 120 Volts!) But how to know if the player is truely frustrated and wants to break the console?

  7. I bet it depends on what game you are playing. I react different to a first person shooter than to a driving game.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s