The Billboard That Recognizes You

The marriage of technology and advertising appears to know no boundaries. Image recognition is an extension of voice recognition and is being used experimentally, to test consumers' reactions to various inducements. As usual in these cases, there is a debate about efficiency/productivity versus the potential invasion of privacy. I am inclined to argue that more data are needed before that determination can be made. Are people being sold shoody merchandise? Inappropriate services (think adjustable rate mortgages with balloon payments)? Inadequately or misleadingly labelled products? The opportunities for abuse are manifold and the ability of the market - as mediated by a vigilant media - remain uncertain.

Emily Steel has the story in the Wall Street Journal:


"In the labs of some tech-savvy advertising agencies, engineers are testing new ways to use advanced technologies to make ads that can recognize human gestures and facial expressions.

Marketers envision billboards that could tell if a passerby is paying attention, and whether that person is male or female, then alter its images and message accordingly. Other goals include interactive television commercials that viewers could control with a wave of the hand via a sensor connected to their TV sets, or targeted TV spots geared to the age and gender of the viewer.

The technology "actually recognizes faces. If you raise your eyebrow, it can track that," says Jonathan Nelson, chief executive at Omnicom Digital, a unit of Omnicom Group Inc., New York. "We're exploring the applications, and they are endless."

Gesture- and facial-recognition technologies have been in development for years, especially for security purposes, and more recently for video gaming, but prices have dropped sharply in the past few years, encouraging the ad industry to experiment. The new systems can detect and interpret motions as subtle as nodding or frowning. Some facial-recognition technologies can even identify individuals, one of the reasons the industry's progress in the field is likely to raise privacy concerns.

"What we are trying to do is figure out what your brain is doing," says Benjamin Palmer, CEO of Barbarian Group, an interactive-ad agency owned by Cheil Worldwide Inc. "If your eyes are the window into the soul, we're paying attention to what you are paying attention to."

Though most of the activity remains in the lab, some of the fruits of the research are close to reality or have already made their debut. This summer Barbarian plans to introduce a device designed to allow digital billboards to spot the consumers who are paying attention to them. It hopes the device will be a step toward more sophisticated billboards that can show a consumer more of the particular images or text that have drawn his interest.

Basic applications of the technology already have been tested in stores, such as mirrors that scan an outfit and suggest accessories.

The technologies, however, come with major privacy implications that companies, and possibly policy makers, will need to sort out. The devices could potentially read a person's face the moment he walks into a store, then link that information to a profile of that individual. The information could include the type of home that person lives in, the car he drives, social-networking profiles and other data.

"If you want to do something evil with it, I'm sure you could. It is the same thing with anything else technology-wise," says Barbarian's Mr. Palmer.

"There's a thin line here around privacy," says Inon Beracha, chief executive of Prime Sense Ltd., which makes gesture-recognition technologies.

Many consumers may get their first taste of the technology from Microsoft Corp.'s motion-sensor gaming device, called Kinect. Released in November, the device lets people play videogames without holding a traditional controller. The Kinect, which has sold more than 8 million units, plugs into Microsoft's Xbox 360 consuls. It includes a camera, a microphone and an infrared depth sensor that analyzes a player's body movements. The technology makes possible games that score players on such skills as dance moves.

The gaming device is changing consumers' ideas about how they should interact with televisions, computers and other digital screens, says John Mayo-Smith, chief technology officer at R/GA, an interactive-ad agency owned by Interpublic Group of Cos. "It will be less of a novelty and more of an expectation."

Some ad companies have tapped the hardware behind Kinect to explore other potential uses. Publicis Groupe SA's digital-advertising agency Razorfish used it to build the "DaVinci Experience," which lets people virtually grab and move objects on a screen. New York-based digital-ad agency Big Spaceship is using Kinect as a three-dimensional scanner and connecting it to a 3-D printer. The tool could allow consumers to walk into a store and create, for example, a custom piece of jewelry, says Michael Lebowitz, the agency's CEO.

Recognition technology was featured in a project R/GA worked on last year for 77kids, the children's brand of American Eagle Outfitters Inc. The technology allowed children to stand in front of a digital screen and try on a virtual outfit.

The ad industry is creating other applications that would automatically analyze individual stores, similar to the way companies track where people click on websites, to determine which marketing messages worked best and which products drew the most interest, says Mr. Mayo-Smith.

The ultimate application from the industry's point of view: mirrors in a department store or a device connected to a TV in a consumer's home that could calculate a woman's dress measurements or a man's trouser size and instantly place an order.