Facial Recognition Software Can Tell When You’re Frustrated With Xbox Live

Most of us have been faced with the anguish of being shot in the head repeatedly by 12-year-olds. There are also the times when we’re overjoyed by defeating the Mother Brain and making it out of the caverns of Zebes. If we wanted to scientifically quantify how happy, sad, or angry we are while playing video games, we wouldn’t know what to do. [Dale] came up with a very interesting way to gauge someone’s state of mind while either playing Xbox, or watching TV.

To get a measure of how happy or sad he is, [Dale] put a webcam underneath his TV and pointed it towards his couch. Every 15 seconds or so, the webcam snaps a picture and sends if off to the face.com API. After face.com sends a blob of JSON containing information about all the faces detected in the photo, a short Python script plots it on a graph.

[Dale] admits he’s not entirely scientific with this project; the low resolution of the webcam, coupled with images being captured every 15 seconds means he runs into the limitations of his hardware very quickly. Also, there’s the confound of [Dale] paying attention to something else in the room – like his kids – rather than the TV. Still, it’s an interesting use of hardware and software that would be loved by a market researcher or QA designer.

20 thoughts on “Facial Recognition Software Can Tell When You’re Frustrated With Xbox Live

    1. I’d say it’s an accurate graph of my gaming experience, most of the time.

      One day I’ll wake up and realize my gaming time is more stressful than my average work day. ;)

  1. So game makers should look at that graph with horror. Basically games make you angry 94% of the time and make you happy 4% of the time. I dont think surprise is really worth while to have on there since surprise is an instantaneous reaction that quickly fades more then likely before the 15 second interval between pics.

    Great idea.

    1. I am reminded of a time I was walking down the Vegas strip, and somebody stopped and offered me dinner and free show tickets to participate in a “survey.” The “survey” was actually the review of the pilot of a proposed television sitcom.

      I was led into a cubicle farm, each cube equipped with a computer, a set of headphones, and a hand-held switch. My instructions were to watch the show. When I felt happy, amused, or entertained, I was to push the green button. If I was offended, bored, or otherwise disgruntled, I was to push the red button.

      At the end of the exercise, I they tallied the button presses of many hundreds of people, and created a moving strip chart graph that was superimposed on the program. This would allow developers to review the pilot and simultaneously see what “humor” worked with the audience and what did not.

      It seems that the gizmo in this HAD post could be used to automate or auto-collect the same kind of information.

      The sitcom, by the way, was unimaginative and unfunny crap. That is to say, you should expect to see it on network TV any day now ;)

  2. This would be useful for a lot of games. If a particular point in a level has gotten too difficult and you just can’t get past it, it would be nice if the games were smart enough to know to make it just a little easier. Of, course then go back to normal.

    Then again, the game could simply track that by the number of repeated tries.

    Nothing is worse than being stuck at a point in a game and just finally giving up and not playing that game anymore.

  3. Yeah, now ms will be tracking for market research, and all we’ll ever get are happy Smiley game 7… not realizing that the journey , story and drama are more to a good game than happy

  4. Would be a fantastic use of the kinect! I always assumed games were supposed to frustrate a certain amount, otherwise there’s no satisfaction from actually completing it. Look at vvvvvv!

  5. Very good thing, if the player gets VERY frustrated AND a vibrational sensor inside the console (The player beats it), some automatic difficulty level decrease could be triggered, or an extra life (action) or deal (card) could be given. Better spoiling a cheap game than the expensive hardware, (or even life as the console contains 120 Volts!) But how to know if the player is truely frustrated and wants to break the console?

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.