Hollywood AI-listers: is the technology ready to rival flesh-and-blood film stars?

‘Really, really scary’ … AI actor Tilly Norwood. Computer-generated: Particle6 View image in fullscreen ‘Really, really scary’ … AI actor Tilly Norwood. Computer-generated: Particle6 Guardian journalism is independent. We earn a commission if you buy something through an affiliate link. Learn more.

As a freaky Val Kilmer doppelganger stars in a new western and faux starlet Tilly Norwood is set for her feature debut, experts are split on whether mainstream adoption is a long way down the line – or just around the corner

Prefer the Guardian on Google ‘D on’t fear the dead – and don’t fear me.” An exact facsimile of a young Val Kilmer, not quite dead and not quite alive, utters that baiting line in the trailer for the new western As Deep as the Grave. Director Coerte Voorhees knows exactly what he is doing – this is an AI-created version of the late actor, approved by his family but bringing with it a heap of controversy about whether such technology could make flesh-and-blood thespians redundant.

Not just yet, judging by the extended glimpse of the film released earlier this year: it looks like a thirtysomething Kilmer, if a robot version had been sent back to the 1920s American wilderness by Skynet and, as well as being strangely rigid, had forgotten to download his voice, let alone his signature air of spacey shrewdness. Kilmer’s estate signed off on it, citing his enthusiasm for new technology that expands storytelling, but surely he would have been dismayed by this freaky doppelganger.

Many reacted with revulsion to last year’s unveiling of AI “starlet” Tilly Norwood. After her appearance in an introductory “comedy” sketch, Emily Blunt described her as “really, really scary” – underscoring the depth of feeling around the idea of synthetic performers supplanting real ones. What you might call the Norwood injunction – the feeling of a tech-driven agenda imposing something no one asked for – deepened with the announcement the AI-lister will be soon appear in feature-length film Misaligned.

Tim Webber, the chief creative officer at British VFX powerhouse Framestore, is sceptical about whether fully AI-generated actors can ever capture the complexity of our inner lives. “AI models are trained on the surface of things – the pixels of internet images, if they’re image models. To get to the level where it’s a human performance, where there’s genuine emotion involved, is a long way further down,” he says. So AI acting, scraping the external texture of past performances, would seem to be the inverse of something like method acting – whose acolytes excavate their internal psyche and emotions in the interests of authenticity.

Read more But AI acting is undeniably becoming more realistic. Higgsfield AI – one of many companies currently pushing the possibilities of AI video – recently claimed to have “eliminated” the uncanny valley: that nagging phoneysimilitude effect that has plagued even sophisticated recent CGI reincarnations, such as the de-aged Indiana Jones in The Dial of Destiny.

Having released two full-length, entirely AI-generated features this year, Higgsfield’s content lead Adil Alimzhanov is bullish. This new frontier of completely AI-generated films is liberating for artists, he says, and shortcuts laborious Hollywood production structures. The two online-only releases – Hell Grind, a 95-minute action fantasy, and The Cully Hill Boys, a 115-minute Guy Ritchie-esque caper comedy – completed the equivalent of shooting and post-production in a fortnight and a month respectively (not counting time spent on the scripts).

Despite actors’ qualms, the future can’t be resisted, says Alimzhanov, 23, who used his own face for Hell Grind’s protagonist. “We can’t fight new tools simply because they transform industries. If you draw a parallel with when cars first came out, if we’d fought that, everybody would still be riding horses today.”

Hell Grind and The Cully Hill Boys are essentially extended ads for Higgsfield’s suite of AI content-creation tools. But they show swift progression on the AI acting front. In Hell Grind, released in May, the skate-rat characters only seem to be able to crank out a single emotion per 15-second shot (the limit of the Chinese video-AI model used, Seedance 2.0).

The Cully Hill Boys, which came out three months later, is far more engaging, with enough convincing repartee and comic timing to qualify it as quite funny. Its lead trio of grime rappers on the lam are AI-generated likenesses of real-life streetball player Matt Kiatipis, MMA fighter Israel Adesanya and livestreamer N3on – testament to the gen Z, AI-native short-form creators Higgsfield aims to attract. Though there is something quaint about the idea of an “actor” in the age of AI.

If bespoke characters can be generated for every film, why have an intermediary play a “role”?

Written by a human being, Tim Planagan, The Cully Hill Boys was assembled in 30-second chunks, made possible by this summer’s release of Seedance 2.5. But prompting your way towards something dramatically convincing is labour-intensive. First, visual “assets” are created of the characters – detailed body-length shots and facial closeups that are used to establish continuity when fed into video-generation prompts. Then, using the assets as an anchor point, each 30-second segment is defined in a 4,000-word prompt (itself partly generated by an LLM) defining the key parameters: action, camera movement, lighting and, of course, the emotions on display. The prompts are revised and the scene regenerated until the uncanny valley is hopefully bridged.

It becomes a case of treating the model like a marionette doll and tweaking individual strings Where, say, Klaus Kinski could summon torrents of rage by tapping into his personal demons, an AI programmer has to achieve the same effect in much more approximate ways. “An AI is not an actor, so how do you communicate the internal state of a character?” says Zack London, who as Gossip Goblin makes highly baroque AI-generated sci-fi shorts. “You can say: they look confused, or they say this in jest, or they say this sternly. But it’s such a broad stroke. The model will probably come out with a very over-the-top telenovela-type performance, or will just not understand.” At that point, it becomes a case of treating the model like a “marionette doll” and tweaking individual strings. “You could say: there’s a slight quiver on the left side of their lip, and then their gaze diverts slightly downwards, and they look up again and blink.”

London chooses to keep a human ghost in the machine by first recording dialogue with real actors, around which he retro-fits the animation. He is candid about the shortcomings of the current prompting process, saying the results suit a certain type of film: “The acting is not going to be particularly subtle, like some grand Anthony Hopkins or Olivia Colman performance. It’s more like an old-school Terry Gilliam or Guy Ritchie performance, where the characters are larger than life.” But maybe that’s where the danger lies for actors: many blockbuster performances, meting out simple beats of easily comprehensible emotion, have a cut-and-paste quality – and are therefore susceptible to AI.

Then again, a mundane magic still happens even on blockbuster sets. “Sometimes the most brilliant moment happens in the fraction of a second. And you can’t put your finger on why an actor gives a certain look,” says Tye Sheridan, the co-founder (in 2016) of Wonder Dynamics, who developed the AI production tool Flow Studio. He should know: he was the star of Steven Spielberg’s 2018 film Ready Player One, which itself patrolled the frontier between real and digital.

View image in fullscreen ‘You can’t put your finger on it’ … Tye Sheridan in Ready Player One. Photograph: courtesy of Warner Bros Pictures/AP Sheridan shares Higgsfield’s belief that AI-driven methods can democratise film-making. But he is adamant that performances should retain a human core. Flow Studio uses AI for an augmented and far cheaper version of motion capture, layering CG characters and environments on top of filmed footage. At Framestore, Webber is exploring similar AI tools that allow him to apply what he calls “digital makeup” to traditional performances, without the guesswork involved in generative prompts.

He, London and Sheridan agree that this hybrid form of film-making still allows actors to make specific choices essential to authentic creation. “You have to ask, with these generative models: are they really designed to deliver legendary performances, or are they designed to cut to an average?” asks Sheridan. Alimzhanov, though, insists something akin to an actor’s internal process is achievable via AI. The prompting used on The Cully Hill Boys was different in one key regard: instead of replicating emotions by specifying the physical mechanics, the AI model generated sequences based on contextual input about what the characters were experiencing.

These two camps of AI acting – technological and human-centric – are fighting the latest bout in the struggle between the forces that have shaped the craft: external v internal; reproducing v experiencing; Laurence Olivier telling Dustin Hoffman that, instead of staying awake for 72 hours, he should “try acting, dear boy”. But where Stanislavsky promoted the internal quest to revitalise acting, this could be the first time in the modern era when the formalist approach – the mannered reproduction of emotions – looks like the future.

Opening the digital floodgates could lead to William Gibson territory, where deathless virtual stars seduce the masses, as in his 1996 novel Idoru. But it’s still not clear whether the box office demand, and thus a true economic case, for AI cinema exists. And we’re yet to see a compelling script that crystallises the possibilities of the tech in story form, as The Matrix did for CGI. If we’re inundated with unloved AI abominations, the words of Val Kilmer in Tombstone, playing that icon of human mortality, Doc Holliday, will ring anew: “There’s just something about him. Something around the eyes reminds me of … me. I’m sure of it – I hate him.”