Panel Explores AI-Assisted Video Workflows

It’s still early days for artificial intelligence (AI) and machine learning in media, but already major players like HBO and Deluxe are exploring how the technology can improve video production.

Weyron Henriques, vice president of product development at Deluxe Technology, and Barbara Ford Grant, senior vice president, Digital Production Services at HBO, were joined on stage Tuesday by Google Cloud Solutions Architect Adrian Graham and moderator Kylee Peña from Netflix to tackle AI and production workflow during the session “Machine Intelligence: From Dailies to Master — Machine Intelligence Comes to Video Workflow.”

“AI by itself is statistically impressive but individually unreliable,” Henriques told an overflow crowd attending the session. He noted that additional image processing methods are needed to tag objects and actions.

Henriques recounted the example of a green-screen shot of an actor who was supposed to be falling. Air blowing at high speed from the bottom of the shot made the actor’s hair stand on end. However, without the intervention of a human to supplement the data available, AI would deduce that the actor is being electrocuted. Such an error could throw off an AI-assisted content search for a specific shot, he said.

However, this challenge doesn’t diminish the fact that AI can assist mightily in content searches, said Henriques. Without AI, a visual effects supervisor might spend days with a visual effects editor looking for background shots for a particular show; however, that task can be completed in seconds with the help of artificial intelligence, he added.

At HBO, there is a lot of enthusiasm around AI, said HBO’s Ford Grant. For the time being, however, HBO is “just dipping our toes and exploring some of the ways that we can use it across the pipeline,” she said. That pipeline, which extends from dailies through to masters, includes structured data as well as an enormous amount of data associated with visual effects, scripts and call sheets.

“One of the things we are looking at is building up taxonomies and dynamic ontologies around this other material and how that might get utilized,” said Ford Grant.

HBO is looking at AI not only to boost efficiencies, but as a means to delight audiences with new experiences, such as immersive and interactive, as well as to extend its streaming services, she added.

Google’s Graham identified AI as a means to understand patterns that exist throughout the production pipeline and to fill in missing data. Recounting an earlier time in his career as a visual effects artist, Graham said a personal pet peeve was trying to put effects on a shot when camera data was sketchy.

“Can you train a [machine learning] model to understand the length of a lens … where a camera is, its elevation and tilt … a wide shot versus a zoom shot, or a dolly versus a track?” he asked rhetorically.

Once that data is available, the next task is to embed it in footage so it can be carried through the production pipeline, understood and grabbed at any point, he added.

AI also has an important role to play in the localization of content, said Deluxe’s Henriques. It’s not uncommon for Deluxe to send content to dubbing facilities in 30 different countries to be localized.