The experience to date reflects concerns familiar to users of AI in broader realms of experience. The first is that the AI has no sense of context so has to be curated by humans, which means human biases regarding culture - what's humorous, what's offensive - is dependent on whichever human is programming or editing it.
The second is that it is also dependent on whatever previous data have been inputted, in this case, including previous Super Bowl ads, reflecting the historical record more than current or future possibilities. So more ads today will use AI as a punchline than actually embody it. JL
David Griner interviews Patrick Kulp in Ad Week:
This bot is unpredictable. It takes quite a bit of curating to sift out something presentable. Often its thoughts are unpublishable because they’re gibberish (or) disturbing. The topics it touches on often are off-limits: domestic violence and racism, lewdly sexual. The bot takes topics from ads and expands them out of context. It will take an NFL PSA on domestic violence and apply it to a Doritos commercial. (And) it picks up on cliched, outdated, ad tropes. "It feels like someone threw pop culture in a blender and hit pulse five times."
Key insights:
- The bot's output definitely needed to be curated.
- It's oddly obsessed with war, dystopia and awkward situations.
- Human copywriters aren't in trouble, but AI could still offer value.
When we launched Adweek’s AI-powered Super Bowl Bot, designed to create ad pitches for the Big Game, we truly didn’t know what to expect. Would it be funny? Insightful? Disturbing? Offensive?The answer to all of the above definitely proved to be “yes,” but the process of training an AI on such a specific task was also fascinating in ways we couldn’t have expected. The bot—which you can find at @SuperBowlBot on Twitter or @adw.ai on Instagram—has generated more than 200 ad ideas so far and continues to evolve as we expand its data set.During the brief calm before the storm of (actual) Super Bowl ads, the bot’s creators decided to touch base on how it’s going, what we’ve all learned and where this project seems to hint that AI is headed, at least in the realm of creativity.Here’s the conversation between Adweek emerging tech reporter Patrick Kulp and creative and innovation editor David Griner:David Griner: Patrick, thanks again for co-parenting Adweek’s Super Bowl Bot with me. Much like with my real children, I like to take 50% of the credit despite my partner doing 95% of the effort in actually creating them.Patrick Kulp: I don’t think you’re giving yourself enough credit. You really sell the bot, for one thing.Griner: I love this bot. It’s my Baby Yoda.Kulp: Baby Yoda would actually be an excellent concept to feed the bot. [Editor’s note: Patrick followed through and actually generated the Baby Yoda pitch.]Griner: So we’ve written quite a bit about how the bot generally works, but I’m curious about a few things. Namely, how much work was it to build and get up and running? You handled all that on the back end after we came up with the concept.Kulp: It was a gradual process, building up the dataset. I used web scrapers to download a ton of descriptions of ads in bulk from various sources and just kept adding to it over time.As far as the training of the bot goes, I found it surprisingly easy. I used a guide from a BuzzFeed data scientist named Max Woolf and ran it in a free cloud program called Google Colab, in part because I don’t have the hardware needed to handle the processing demands of AI—typically graphics processing units (GPUs). That backend guide is written in Python code, but you don’t have to understand much Python at all (I don’t) to make it work. The training process itself takes about an hour or two depending on how extensively you do it.Griner: We chose not to automate this bot, meaning you hand-feed it prompts, or it just spits out a random Super Bowl ad idea based on the data you’ve fed into it. But I assume we could have made this an automated bot that shoots things directly onto Twitter without us moderating or curating it?"This bot is wildly unpredictable. It takes quite a bit of curating to sift out something presentable."—Patrick Kulp, Adweek emerging tech reporterKulp: Yeah, there is also an API section in the guide that explains how to build a web app interface for the bot. And I’m sure the Twitter automation would have been doable from there.That would’ve taken a bit more figuring out, but the main reason we chose not to do it was that this bot is wildly unpredictable. It takes quite a bit of curating to sift out something presentable. To be clear, that’s selecting which generated output to post—I never actually adjust any of its text.Griner: You get to see all the raw content it creates from a prompt. You and I originally worried that, without a moral compass, the bot might create some truly unfortunate or problematic ad concepts that we wouldn’t feel good about putting out in the world. I’m sure the word “Tay” still haunts AI folks from Microsoft’s failed 2016 experiment with a Twitter AI bot. But now that you’ve seen what it actually produces, were we right to worry? How often does it get legitimately … unpublishable?Kulp: We were absolutely right to worry. I don’t have concrete stats on how often its thoughts are unpublishable because they’re gibberish versus downright disturbing, but the latter definitely tends to stand out.You and I have had conversations about the topics it touches on surprisingly often that are absolutely off-limits: domestic violence and racism, mostly. At other times, it gets over-the-top lewdly sexual. The problem is that the bot takes these topics from ads and expands on them out of context. So for example, it will take the theme from an NFL PSA on domestic violence and apply it to, say, a Doritos commercial in some horrible way.Aside from those line-crossing ones, I do enjoy the bot’s dark sense of humor, though.Griner: And to be clear, the AI already started with a knowledge base that included some, what, 8 million web pages? Lots of weird shit on the internet. I never know how much to blame Super Bowl ads or just, you know, humanity, when it cranks out weird connections.Kulp: Yeah, that also contributes to it for sure. The underlying knowledge base, GPT-2, teaches the bot the mechanics of language and how context works and allows it to riff on subjects when there isn’t anything available in our limited training data. So who knows from which dark corners of the internet it gets some of these ideas.Griner: Let’s talk about trends we’ve noticed in the ad concepts it creates. If I were to list the Top 3 that come to mind, it’d be dystopia, military and… I guess “people being forced into awkward confrontations”?Kulp: I would say that’s a good summary. It definitely makes you appreciate how much supporting veterans or active-duty troops and patriotism have been huge themes throughout Super Bowl ad breaks.Another thing I’ve been surprised that it picks up on is cliche, sometimes outdated, ad tropes—a bumbling guy and his annoyed wife, for instance. I feel like you don’t tend to see those quite as much in Super Bowl commercials these days, but then I remember that its training data encompasses Super Bowl commercials from all time, though it’s weighted much more heavily toward more recent ones.As for the dystopia, there have been a few real commercials—Apple’s “1984” of course, then Avocados From Mexico’s ad about a secret society—that it likes to draw on.Griner: Do you have a favorite of all the ads it’s pitched so far?Kulp: Hmm, that’s hard. I do like some of the simpler, succinct ones. Maybe: “Jeff Bridges plays a musical number titled ‘Monsters Unforgiven’ to celebrate deceased loved ones.” I like it because it suggests a lot more going on. Who are these monsters who have presumably killed Jeff Bridges’ loved ones?Griner: And I love that you actually then had the bot write the lyrics to the song. How did you do that?Kulp: For that, I actually used a version of GPT-2 that wasn’t fine-tuned, so it could draw on anything in its 8-million-website training data. I prompted it with the ad concept, then something like “Here are the full lyrics:”Griner: You know my favorite.Kulp: Is that still the incel killer whale?Griner: Well, first of all, how dare you. But yes, I don’t think the bot has written a line that cracked me up more than, “Oh no, the killer whale doesn’t have a girlfriend.” It’s the period at the end that really nails that this is a bot. Any human would have put an exclamation mark. It’s also just such a classically absurdist, impossible-to-film premise.Kulp: Yes, I love when you can read some kind of tone into its actual pitching of the ad, like how it sounds so breathless in the “Lethal Weapon video game” concept.But I think you actually called me after the killer whale one went up—because we’re based in different cities—and we had a whole conversation wondering about the details of this killer whale’s situation.Griner: Well yeah, I mean, there’s a lot going on here. The bot really turns 180 after the first line. It sets up this great narrative about a single killer whale, but then promptly tells us it’s been in a long-term relationship with a circus horse and also the killer whale is now a dog? But I’d argue it’s still the best Pringles ad in years.Kulp: I would have to agree. I think they should really be jumping on this one.Griner: Take that, Bill Hader. OK so that begs the real question here, kind of the existential question behind this whole bot. Can an AI write an ad? Or at least pitch one that could be viable for a brand?"I could see it being useful as a way to prompt creativity."—Patrick KulpKulp: I think the bot’s pitches are a lot of fun, but they are really fun because we know they come from an AI. If a human were to pitch all the ideas the bot does, I think I could see maybe 1% being even a seed of an actual campaign. They would probably be fired.On the other hand, because this bot is supposed to be fun, I’ve sometimes actually passed over its more boring, straightforward ideas in favor of its more entertaining, ridiculous ones. Sometimes I’ve been impressed when it pitches a straight ad.Griner: There’s some irony here in the fact we made a bot to pitch ads and then refuse to listen when it pitches a believable ad. We’re awful parents.Kulp: In any case, I could see it being useful as a way to prompt creativity. It definitely spits out a lot of things I never would have thought of in a million years.Griner: I think you can see the potential value in having something that digests a bunch of information relevant to your audience—celebs, themes, fears, relatable scenarios—and then shows how they might be combined. But it feels like we’re probably a ways away from that not looking like someone threw pop culture in a blender and hit pulse five times.Kulp: Yeah, absolutely. I don’t think it’ll be taking anyone’s job anytime soon. I do almost worry, though, what might happen if they find a way to hone some of its rough edges and constrain it to something more useful. This tech is still in its very earliest stages. GPT-2 was only released a little less than a year ago.Griner: Well, I’m sure this isn’t the last time we’ll be playing with the potential for AI. We’ve gotten quite a bit of interest and inquiries about taking it in a few other interesting directions. Stay tuned! But for now, thanks again for building my favorite bot of all time and for explaining what we’ve learned along the way, Patrick.Kulp: Thank you! It’s been a blast!
0 comments:
Post a Comment