The Tens Of Millions Of Faces Training Facial Recognition; You’ll Soon Be Able To Search For Yourself

In a stiflingly hot lecture tent at CCCamp on Friday, Adam Harvey took to the stage to discuss the huge data sets being used by groups around the world to train facial recognition software. These faces come from a variety of sources and soon Adam and his research collaborator Jules LaPlace will release a tool that makes these dataset searchable allowing you to figure out if your face is among the horde.

Facial recognition is the new hotness, recently bubbling up to the consciousness of the general public. In fact, when boarding a flight from Detroit to Amsterdam earlier this week I was required to board the plane not by showing a passport or boarding pass, but by pausing in front of a facial recognition camera which subsequently printed out a piece of paper with my name and seat number on it (although it appears I could have opted out, that was not disclosed by Delta Airlines staff the time). Anecdotally this gives passengers the feeling that facial recognition is robust and mature, but Adam mentions that this not the case and that removed from highly controlled environments the accuracy of recognition is closer to an abysmal 2%.

Images are only effective in these datasets when the interocular distance (the distance between the pupils of your eyes) is a minimum of 40 pixels. But over the years this minimum resolution has been moving higher and higher, with the current standard trending toward 300 pixels. The increase is not surprising as it follows a similar curve to the resolution available from digital cameras. The number of faces available in data sets has also increased along a similar curve over the years.

Adam’s talk recounted the availability of face and person recognition datasets and it was a wild ride. Of note are data sets by the names of Brainwash Cafe, Duke MTMC (multi-tracking-multi-camera),  Microsoft Celeb, Oxford Town Centre, and the Unconstrained College Students data set. Faces in these databases were harvested without consent and that has led to four of them being removed, but of course, they’re still available as what is once on the Internet may never die.

The Microsoft Celeb set is particularly egregious as it used the Bing search engine to harvest faces (oh my!) and has associated names with them. Lest you think you’re not a celeb and therefore safe, in this case celeb means anyone who has an internet presence. That’s about 10 million faces. Adam used two examples of past CCCamp talk videos that were used as a source for adding the speakers’ faces to the dataset. It’s possible that this is in violation of GDPR so we can expect to see legal action in the not too distant future.

Your face might be in a dataset, so what? In their research, Adam and Jules tracked geographic locations and other data to establish who has downloaded and is likely using these sets to train facial recognition AI. It’s no surprise that the National University of Defense Technology in China is among the downloaders. In the case of US intelligence organizations, it’s easier much easier to know they’re using some of the sets because they funded some of the research through organizations like the IARPA. These sets are being used to train up military-grade face recognition.

What are we to do about this? Unfortunately what’s done is done, but we do have options moving forward. Be careful of how you license images you upload — substantial data was harvested through loopholes in licenses on platforms like Flickr, or by agreeing to use through EULAs on platforms like Facebook. Adam’s advice is to stop populating the internet with faces, which is why I’ve covered his with the Jolly Wrencher above. Alternatively, you can limit image resolution so interocular distance is below the forty-pixel threshold. He also advocates for changes to Creative Commons that let you choose to grant or withhold use of your images in train sets like these.

Adam’s talk, MegaPixels: Face Recognition Training Datasets, will be available to view online by the time this article is published.

Your Face Is Going Places You May Not Like

Many Chinese cities, among them Ningbo, are investing heavily in AI and facial recognition technology. Uses range from border control — at Shanghai’s international airport and the border crossing with Macau — to the trivial: shaming jaywalkers.

In Ningbo, cameras oversee the intersections, and use facial-recognition to shame offenders by putting their faces up on large displays for all to see, and presumably mutter “tsk-tsk”. So it shocked Dong Mingzhu, the chairwoman of China’s largest air conditioner firm, to see her own face on the wall of shame when she’d done nothing wrong. The AIs had picked up her face off of an ad on a passing bus.

False positives in detecting jaywalkers are mostly harmless and maybe even amusing, for now. But the city of Shenzhen has a deal in the works with cellphone service providers to identify the offenders personally and send them a text message, and eventually a fine, directly to their cell phone. One can imagine this getting Orwellian pretty fast.

Facial recognition has been explored for decades, and it is now reaching a tipping point where the impacts of the technology are starting to have real consequences for people, and not just in the ways dystopian sci-fi has portrayed. Whether it’s racist, inaccurate, or easily spoofed, getting computers to pick out faces correctly has been fraught with problems from the beginning. With more and more companies and governments using it, and having increasing impact on the public, the stakes are getting higher.

Continue reading “Your Face Is Going Places You May Not Like”

Simon Says Smile, Human!

The bad news is that when our robot overlords come to oppress us, they’ll be able to tell how well they’re doing just by reading our facial expressions. The good news? Silly computer-vision-enhanced party games!

[Ricardo] wrote up a quickie demonstration, mostly powered by OpenCV and Microsoft’s Emotion API, that scores your ability to mimic emoticon faces. So when you get shown a devil-with-devilish-grin image, you’re supposed to make the same face convincingly enough to fool a neural network classifier. And hilarity ensues!

Continue reading “Simon Says Smile, Human!”

Face Recognition For Your Next Con

[jwcrawley] is busy planning for the Makevention coming up in Bloomington, Indiana in late August. One problem when working any con is manning the door; it’s a good idea to know how many people are there, and you can’t double count people. Previously, the volunteers used dead trees to estimate how many people have turned up. This year they might go with a more technological solution: face recognition and tracking.

The project is called uWho, and it uses the faceRecognizer class in OpenCV. The purpose of the entire project is to identify who someone is from previous frames. If your face is unknown to the program, your likeness – rather, a few points of data – are added to the database of faces. It’s simple, and according to [jwcrawley], it works.

While this is technically the best way to count how many unique people show up to Makevention, there will be some discussions to see if this solution is appropriate. The program only saves unique data from a face locally, and does nothing online. It’s less evil than whatever Facebook does, but there are obvious privacy implications here.

Link to the Makevention.

Face Tracking With An Android Device

This Android device can recognize faces and move to keep them in frame. It’s a proof of concept that uses commonly available parts and software packages.

The original motivation for the project was [Dan O’s] inclination to give the OpenCV software a try. OpenCV is an Open Source Computer Vision package that takes on the brunt of the job when it comes to discerning meaning from images. To give the phone the power to move he designed and printed his own mounting brackets for the phone and a couple of hobby servos. An IOIO board connects to the Android device in order to control the motors. On the software side all [Dan] needed to do was write some code to interface the output of the OpenCV face tracking modules with the input of the IOIO. See the finished project demonstration after the jump.

This system can easily be implemented with other hardware, like this Arduino-based version we looked at earlier in the year.

Continue reading “Face Tracking With An Android Device”

Smile, Your Face Is On The Internet

[Kyle McDonald] is up to a bit of no-good with a little piece of software he wrote. He’s been installing it on public computers all over New York City. It uses the webcam found in pretty much every new computer out there to detect when a face is in frame, then takes a picture and uploads it to the Internet.

We’ve embedded a video after the break that describes the process. From [Kyle’s] comments about the video it seems that he asked a security guard at the Apple store if it was okay to take pictures and he encouraged it. We guess it could be worse, if this were a key logger you’d be sorry for checking your email (or, god forbid, banking) on a public machine. Instead of being malicious, [Kyle] took a string of the images, adjusted them so that the faces were all aligned and the same size, and then rolled them into the latter half of his video.

Continue reading “Smile, Your Face Is On The Internet”

Cheap And Reliable Portable Face Recognition System

faceaccess_portable_facial_recognition

For their senior ECE 4760 project, engineering students [Brian Harding and Cat Jubinski] put together a pretty impressive portable face recognition system called FaceAccess. The system relies on the eigenface method to help distinguish one user from another, a process that the pair carried out using MatLab.

They say that the system only needs to be hooked up to a computer once, during the training period. It is during this period that faces are scanned and processed in MatLab to create the eigenface set, which is then uploaded to the scanner.

Once programmed, the scanner operates independently of the computer, powered by its own ATmega644 micro controller. Users enroll their face by pressing one button on the system, storing their identity as a combination of eigenfaces in the onboard flash chip. Once an individual has been enrolled, a second button can be pressed to gain access to whatever resources the face recognition system is protecting.

The students say that their system is accurate 88% of the time, with zero false positives – that’s pretty impressive considering the system’s portability and cost.

Stick around to see a quick demo video of their FaceAccess system in action.

Continue reading “Cheap And Reliable Portable Face Recognition System”