Changed Your Hair Style? Grown a Beard? Your Phone Might Not Unlock

Who are you? Your phone now has the power to decide. JL

Tripp Mickle reports in the Wall Street Journal:

The rise of facial-recognition technology such as Microsoft Corp.’s Windows Hello and Apple Inc.’s Face ID means computers now seem to be passing judgment on users’ appearances. When a face doesn’t measure up, people are left to ponder whether they look their best, whether they use too much makeup, why they changed their hairstyle—and perhaps whether they even look like themselves. Users report their devices won’t unlock unless they wear the same makeup as when they set it up. Others complain it can’t identify them in the morning when they first tilt phone to face on the pillow.
When Karen Cummings’s new Microsoft Surface computer captured her image for its facial-recognition system, her blond hair was curled and her eyes were accentuated by mascara and green eyeliner.
Early the next morning—hair scraggly, makeup off, eyeglasses on—the computer looked again. It refused to unlock.
“The computer was like: ‘Oh my God, girl. You got to go back and fix yourself up. You’re looking bad,’ ” recalls Ms. Cummings, 72 years old, of Fryeburg, Maine. “When I get up in the morning, I know I look bad, but I don’t want the computer telling me that.”
The rise of facial-recognition technology such as Microsoft Corp.’s Windows Hello and Apple Inc.’s Face ID means computers now seem to be passing judgment on users’ appearances. When a face doesn’t measure up, people are left to ponder whether they look their best, whether they use too much makeup, why they changed their hairstyle—and perhaps whether they even look like themselves.
Users report their devices won’t unlock unless they wear the same makeup as when they set it up. Others complain it can’t identify them in the morning when they first tilt phone to face on the pillow. Men who shave their beards say their phones suddenly treat them like a passing stranger.
“It’s a big decision to shave your beard. You’re emotionally attached to it, and next thing you know, your phone doesn’t recognize you,” said Mason Estep, a 25-year-old entrepreneur whose iPhone refused to unlock when he looked at it after shaving his thick, dark beard in July. “It was an existential crisis.”
Josh Engroff, 49, discovered a hairy issue when his barber shook up his hairstyle last summer, shortening the sides and reversing his part to sweep from right to the left. Friends and family complimented his new look, but his Face ID refused to unlock his iPhone for three straight days, forcing him to resort to using his passcode.
“So maybe my haircut wasn’t so good after all,” said Mr. Engroff, who finally reset the system to account for his new look.
Kelly Coon, 40, a young-adult novelist outside Tampa, Fla., began to reassess how much makeup she uses after her Microsoft Surface didn’t recognize her without mascara, eyeliner and lipstick. “I definitely slap some paint on the old barn door, but I wouldn’t say it’s a ridiculous amount. Or at least, I didn’t think it was,” she said.
Facial-recognition technology, which generally takes just minutes to set up, is now available on more than 20 smartphone models world-wide, mostly introduced in the past year, according to research firm IHS Markit . After introducing Face ID last year on the iPhone X, Apple released two new iPhones with the system last week; another is due out next month.
Engineers have been trying to train computers to size up people’s appearance since the 1960s, writing software to measure nose position, eye shape and other features. The early algorithms’ identification processes were easily tripped up by the angle of a face, its expression or the amount of light, said Kevin Bowyer, a computer science professor at the University of Notre Dame.
Recent developments in artificial-intelligence techniques and three-dimensional sensors have given computers better depth perception and a refined ability to analyze images.
The advances led Toronto-area plastic surgeon Philip Solomon to devote a page on his website to the crucial question: “Will getting facial plastic surgery alter the facial recognition of the iPhone X?” Those who get Botox, neck lifts or chin implants likely needn’t worry, the page says. Those contemplating eyelid surgery, cheek implants or fat transfers to the face may need to take more drastic measures.
“The people we’ve done nose jobs for have gone back and done a reset,” said Dr. Solomon.
Windows Hello aims to balance security with usability, said Dave Bossio, a Microsoft program manager. An algorithm uses the infrared camera on laptops and other devices to create a mathematical model based on facial “landmarks” like the eyes, nose and mouth. Makeup, glasses, beards, lighting and other factors can affect the system, and widening the range of acceptability too much creates a security risk, he said.
To avoid the makeup problem, Apple’s engineers designed a camera system that projects 30,000 infrared dots across a user’s face to create a 3-D model stored on the phone, according to people familiar with the project. Apple said the chances the iPhone X could be unlocked by a random person’s face are one in a million. (The probabilities change for family members such as twins or other siblings.) That compares with one in 50,000 for Touch ID, the fingerprint sensor used by previous iPhone models.
Shortly after the iPhone X was released last November, makeup artist James Charles took to his YouTube channel to test how much makeup he needed to apply before his phone no longer recognized him. He “contoured the crap” out of his cheeks, glued on huge “drag lashes,” applied blue lipstick and donned a Marilyn-Monroe-style wig. His iPhone X continued to recognize him throughout the process.
“I really thought [I] was going to have Apple quaking as soon as I got to the nose contour,” he said.
Torri Lovett of Bushnell, Fla., successfully unlocked her mother’s new iPhone X with her own face late last year—upending the way she saw herself.
Most of her life, friends and family told the 20-year-old photographer that she looked like her father. “We always made the joke that if we put a goatee on me that I would look just like my dad,” she said. The iPhone revelation validated what people, including her mother, had been saying more recently: You’re starting to look like your mom.
Her mother found it funny, said Ms. Lovett. “She had gotten annoyed that all her kids looked like my dad. She was like, ‘I told you that you look like me,’ ” Ms. Lovett said. “It was so cool. My mom’s beautiful and hardly ages.”