culture

Late last week, The Hollywood Reporter ran a story about an “AI Stan Lee hologram” that would be appearing at the LA Comic Con this weekend. Nearly seven years after the famous Marvel Comics creator’s death at the age of 95, fans will be able to pay $15 to $20 this weekend to chat with a life-sized, AI-powered avatar of Lee in an enclosed booth at the show.

The instant response from many fans and media outlets to the idea was not kind, to say the least. A writer for TheGamer called the very idea “demonic” and said we need to “kill it with fire before it’s too late.” The AV Club urged its readers not to pay to see “the anguished digital ghost of a beloved comic book creator, repurposed as a trap for chumps!” Reactions on a popular Reddit thread ranged from calling it “incredibly disrespectful” and “in bad taste” to “ghoulish” and “so fucked up,” with very little that was more receptive to the concept.

But Chris DeMoulin, the CEO of the parent company behind LA Comic Con, urged critics to come see the AI-powered hologram for themselves before rushing to judgment. “We’re not afraid of people seeing it and we’re not afraid of criticism,” he told Ars. “I’m just a fan of informed criticism, and I think most of what’s been out there so far has not really been informed.”

“It’s unfortunate that a few people have really negative things to say about it, sight unseen, just the level of it being a concept,” DeMoulin continued. “It’s not perfect. I’m not sure something like this can ever be perfect. But I think what you strive to do is feed enough information into it and test it enough so that the experience it creates for the fans is one that feels genuine.”

“It’s going to have to be really good or we’re all going to say no”

This isn’t the first time LA Comic Con has featured an interactive hologram (which for the Stan Lee experience means a life-sized volumetric screen-in-a-box that can show different views from different angles). Starting in 2019, the convention used similar technology to feature Boffo the Bear, a 7-foot-tall animated blue ursid who served as the MC for a live talent show featuring famous voice acting talent. But Boffo was powered by a real-time motion-captured improv performance from actor Mark DeCarlo rather than automated artificial intelligence.

In the years since Boffo’s introduction at the con, DeMoulin said he’s kept up with the team behind that hologram and “saw the leaps and bounds that they were making in improving the technology, improving the interactivity.” Now, he said, it’s possible to create an AI-powered version that ingests “all of the actual comments that people made during their life” to craft an interactive hologram that “is not literally quoting the person, but everything it was saying was based on things that person actually said.”

DeMoulin said he called Bob Sabouni, who manages the Stan Lee Legacy brand, to pitch the AI Stan Lee avatar as “kind of an entry point into people asking questions about the Marvel universe, the stories, the characters he created.” Sabouni agreed to the idea, DeMoulin said, but added that “it’s gonna have to be really good or we’re all going to say no.”

With that somewhat conditional approval, DeMoulin reached out to Proto Hologram, the company that had developed the Boffo the Bear experience years earlier. Proto, in turn, reached out to Hyperreal, a company that describes itself as “powering ownership, control, performance, and monetization of identity across digital ecosystems” to help develop the AI model that would power the Lee avatar.

Hyperreal CEO and Chief Architect Remington Scott tells Ars that the company “leverages a customized ecosystem of cutting-edge AI technologies” to create “bespoke” and “custom-crafted” AI versions of celebrities. To do that for Stan Lee, DeMoulin said they trained a model on decades of content he had left behind, from tapes of dozens of convention panels he had appeared on to written and spoken content gathered by the managers of the Stan Lee Universe brand.

Scott said Hyperreal “can’t share specific technical details” of the models or training techniques they use to power these recreations. But Scott added that this training project is “particularly meaningful, [because] Stan Lee had actually begun digitizing himself while he was alive, with the vision of creating a digital double so his fans could interact with him on a larger scale.”

After incurring costs of “tens of thousands into six figures” of dollars, DeMoulin said he was finally able to test the Lee hologram about a month ago. That first version still needed some tweaks to get the look and feel of Lee’s delivery just right, though.

“Stan had a considered way of speaking… he would pause, he had certain catch phrases that when he used them he would say them in a certain way,” DeMoulin said. “So it took a while to get to the hologram to be able to say all that in a way that [Sabouni] and I and others that work with Stan felt like, ‘Yeah, that’s actually starting to sound more like him.’”

“The only words that are gonna be in Stan’s mouth are Stan’s words”

Anyone who is familiar with LLMs and their tendency to confabulate might be worried about the potential for an AI Lee avatar to go off-script or make things up in front of a live audience. And while DeMoulin said he was concerned about that going in, those concerns have faded as he and others who worked with Lee in his lifetime have spent hours throwing “hundreds and hundreds and hundreds” of questions at the hologram “to sort of see where the sensitivities on it are.”

“The only words that are gonna be in Stan’s mouth are Stan’s words,” DeMoulin said. “Just because I haven’t personally seen [the model hallucinate] doesn’t mean that it’s impossible, but that hasn’t been my experience.”

While a moderator at the convention will be on hand to repeat fan questions into a microphone (to avoid ambient crowd noise from the showfloor), DeMoulin said there won’t be any human filtering on what fans are allowed to ask the Lee avatar in the 15- to 20-minute group Q&A sessions. Instead, DeMoulin said the team has set up a system of “content governors” so that, for instance, “if you ask Stan what he thought of the last presidential election he’s gonna say ‘That’s not what we’re here to talk about. We’re here to talk about the Marvel universe.'”

For topics that are Marvel-related, though, the AI avatar won’t shy away from controversy, DeMoulin said. If you ask the avatar about Jack Kirby, for instance, DeMoulin said it will address the “honest disagreements about characters or storylines, which are gonna happen in any creative enterprise,” while also saying that “‘I have nothing but respect for him,’ which is I think largely what Stan would have said if he was asked that question.”

Hyperreal’s Scott said the company’s approach to training digital avatars on verified content “ensures responses stay true to Stan’s documented perspectives and values.” And DeMoulin said the model is perfectly willing to say when it doesn’t know the answer to an appropriate question. In early testing, for instance, the avatar couldn’t answer a question about the Merry Marvel Marching Society, DeMoulin said, because that wasn’t part of its training data. After a subsequent update, the new model provided a relevant answer to the same question, he said.

“We are not trying to bring Stan back from the dead”

Throughout our talk, DeMoulin repeatedly stressed that their AI hologram wasn’t intended to serve as a replacement for the living version of Lee. “We want to make sure that people understand that we are not trying to bring Stan back from the dead,” he said. “We’re not trying to say that this is Stan, and we’re not trying to put words in his mouth, and this avatar is not gonna start doing commercials to advertise other people’s products.”

DeMoulin said he sees the Lee avatar as a kind of futuristic guide to a library of Marvel information and trivia, presented with a fun and familiar face. “In the introduction, the avatar will say, ‘I’m here as a result of the latest developments in technology, which allow me to be a holographic representation of Stan to answer your questions about Marvel and trivia’ and this, that, and the other thing,” DeMoulin said

Still, DeMoulin said he understands why the idea of using even a stylized version of Lee’s likeness in this manner could rub some fans the wrong way. “When a new technology comes out, it just feels wrong to them, and I respect the fact that this feels wrong to people,” he said. “I totally agree that something like this–not just for Stan but for anyone, any celebrity alive or dead–could be put into this technology and used in a way that would be exploitative and unfortunate.”

That’s why DeMoulin said he and the others behind the AI-powered Lee feel a responsibility “to make sure that if we were going to do this, we never got anywhere close to that.” Moreover, he said he’s “disappointed that people would be so negative about something they’ve not seen. … It’s not that I think that their point of view is invalid. What I think is invalid is having a wildly negative point of view about something that you haven’t actually seen.”

Scott said concerns about respect for the actual human celebrity are why they “partner exclusively with authorized estates and rights holders like Stan Lee Universe.” The “premium, authenticated digital identities” created by Hyperreal’s system are “not replacing artists” but “creating respectful digital extensions that honor their legacy,” Scott said.

Once fans actually see the AI-powered Lee avatar in person, DeMoulin said he’s confident they’ll see the team behind the convention is “trying to do it in a way that will actually be delightful and very much be consistent with Stan’s legacy… We clearly have to set our sights on doing this right, and doing it right means getting people that knew and loved the guy and worked with him during his career to give us input, and then putting it in front of enough fans to know if we’re doing it in a way that lives up to his standards.”

And if he’s wrong about the expected reception? “I suppose if we do it and thousands of fans interact with [it] and they don’t like it, we’ll stop doing it,” he said. “I saw firsthand the impact that Stan had in that [convention] environment, so I think we have a team of people together that love and respect that and are trying to do something which will continue that. And if it turns out, for some reason, this isn’t that, we won’t do it.”