'Deep Nostalgia' AI Makes Ancestors' Photos Wink, Nod, Dance

Harmless fun or potentially disruptive deepfake, messing with old photos and videos appears to be gaining in popularity. JL 

Vanessa Ramirez reports in Singularity Hub:

A service called MyHeritage has added a unique feature: bringing photos of your deceased relatives “alive” by adding animations like blinking, nodding, head-bopping, and other gestures. Far from being weirded out, MyHeritage users seemed to embrace the feature; in five weeks, people created 72 million photo animations. A deep learning algorithm analyzes a photo to determine which way the subject is looking and which direction their head is facing. Then it finds a compatible “driver”—a video of a face doing whatever the person in the photo will end up doing—to guide how the photo is animated.

Genealogy services have grown in popularity in recent years. Ancestry.com and 23andMe let users analyze their DNA to show their ethnic makeup, connect with relatives they didn’t know they had, create online family trees, and search databases for information about ancestors. Now a third service, called MyHeritage, has added a particularly unique feature to this list: bringing photos of your deceased relatives “alive” by adding animations like blinking, nodding, head-bopping, and a number of other gestures.

The feature is called Deep Nostalgia, and it was launched in late February of this year. It initially let users animate one person in a photo for a few seconds, in a way that made it seem like you could see their face right before or after the photo was taken—sort of like the iPhone’s live photos feature. Photo subjects would move their heads, blink, or smile in a way that looked pretty darn realistic. Far from being weirded out, MyHeritage users seemed to embrace the feature wholeheartedly; the site claims that in just five weeks, people created over 72 million photo animations.

This week MyHeritage added 10 new “drivers,” or videos of actions you can make the pictures do, to the Deep Nostalgia service; people in photos can now be made to blow a kiss, nod approval, and bop their heads in a dance-y way, to name just a few.

Deep Nostalgia uses a deep learning algorithm created by a company called D-ID, short for de-identification. The software analyzes a photo to determine which way the subject is looking and which direction their head is facing. Then it finds a compatible “driver”—a video of a face doing whatever the person in the photo will end up doing—to guide how the photo is animated.

You can use Deep Nostalgia for free to animate up to five photos if you have a MyHeritage account. The site’s blog says its users “were moved to tears to see their ancestors look around and smile at them.” Though it’s on a whole different level, this brings to mind the chatbots or digital avatars of deceased people that can be created using their old texts, emails, and recorded conversations.

We can call these things bizarre, creepy, or ethically questionable, as they’re using technology to play to peoples’ emotions with products that, at the end of the day, aren’t real. But if they make people feel good or help ease the loss of a loved one, perhaps they’re not all bad (the chatbot is up for debate, IMO). Many MyHeritage users tweeted their photo animations and expressed how moved they were by them.

The animated photos don’t quite qualify as “deepfakes.” Deepfakes typically map someone’s facial expressions (e.g., an actor) onto someone else to make it seem like the person said or did something they did not in fact say or do. In this case, there’s no actor and no intention to deceive. But Deep Nostalgia shows how this sort of technology is slowly creeping into unexpected places; who’d have thought you’d ever be able to make a photo of your great-grandmother wink at you?

Some uses of the technology, like this one, will be fun and harmless. While we enjoy these, we must also be cautious of applications that are meant to deceive, and keep a close eye on whether even the harmless uses are emotionally healthy for us.

In the meantime, why limit the photo animations to deceased relatives? They seem like the perfect way to spice up a school picture, add some flare to a birthday or anniversary card, or just make someone laugh.