Adversarial Makeup: Your Contouring Skills Could Defeat Facial Recognition

Facial recognition is everywhere these days. Cloud servers churn through every picture uploaded to social media, phone cameras help put faces to names, and CCTV systems are being used to trace citizens in their day-to-day lives. You might want to dodge this without arousing suspicion, just for a little privacy now and then. As it turns out, common makeup techniques can help you do just that.

In research from a group at the Ben-Gurion University of the Negev, the team trialled whether careful makeup contouring techniques could fool a facial recognition system. There are no wild stripes or dazzle patterns here; these techniques are about natural looks and are used by makeup artists every day.

The trick is to use a surrogate facial recognition system and a picture of the person who intends to evade. Digital techniques are used to alter the person’s appearance until it fools the facial recognition system. This is then used as a guide for a makeup artist to recreate using typical contouring techniques.

The theory was tested with a two-camera system in a corridor. The individual was identified correctly in 47.57% of frames in which a face was detected when wearing no makeup. With random makeup, this dropped to 33.73%, however with the team’s intentionally-designed makeup scheme applied, the attacker was identified in just 1.22% of frames. (PDF)

The attack relies on having a good surrogate of the facial recognition system one wishes to fool. Else, it’s difficult to properly design appropriate natural-look makeup to fool the system. However, it goes to show the power of contouring to completely change one’s look, both in front of humans and the machines!

Facial recognition remains a controversial issue, but nothing is stopping its rollout across the world. Indeed, your facial profile may already be out there.