In the Picture

Books, movies and games that completely absorb one’s attention have long been described as ‘immersive’. But advances in VR production and headset technology are taking absorption to a whole other level.

In Encounter Dinosaurs, developed by Hollywood director Jon Favreau, viewers find themselves in a prehistoric landscape where they come eye to eye with dinosaurs and react to each other.

In The Book of Distance, a VR documentary film about artist Randall Okita’s grandfather’s experiences in a Japanese internment camp in Canada in World War II, the viewer is dropped into family scenes, interacts with objects such as a camera, and is asked to take photos at different moments in the family’s history to help ‘create’ a visual record.

For film specialist Professor Tim Gruenewald, who has been following the key events in cinematic virtual reality, this represents a turning point for storytelling.

“Every new medium draws heavily from the previous media and adds something else – print offered mass distribution, film brought moving images. With VR media, for the first time, you can have an embodied, immersive and interactive experience with the fictional world,” he said.

“Conventional Hollywood would never acknowledge your presence because that breaks the suspension of disbelief. But in VR, the characters can look at you and talk to you, and you have the impression that they see you, that you are in their space. This can strengthen the emotional impact of the story.”

Holding viewers’ attention

AI-enhanced VR is making such engagement easier to achieve. Until recently, huge effort and expense were required to create virtual worlds and embodied experiences. High-end productions, such as the acclaimed game Half-Life: Alyx, cost tens of millions of US dollars each. This limited the growth of the VR market, which nonetheless encompasses about 200 million users, generated about US$20 billion in revenue in 2024, and is growing at 10–20 per cent annually.

Much faster adoption is likely to occur in the future as VR headsets become lighter and easier to use and, importantly, virtual world-building becomes much cheaper. All it takes is a few strokes on a keyboard, using AI such as Google DeepMind’s Genie 3, which was launched as a limited release in summer 2025. While the technology can only produce brief VR experiences at this stage, it is possible to imagine that almost anyone could one day produce a film-like VR story with a fixed timeline, or an open-world sandbox where the user takes more control of the story or events.

Yet these developments also raise new challenges for storytelling. When the user can walk around the virtual world and gaze anywhere, instead of at the characters or the action, extra effort will need to be made to keep their attention, for example, through lighting, sound or staging. The social enjoyment of watching a movie and sharing opinions might be undermined when everyone is shaping their own virtual experiences. “Challenges are what usually demand creativity,” he said.

Imitating styles

AI’s involvement also gives rise to the question of just who the creator is – the human or the AI. Professor Gruenewald takes a nuanced view. There will always be co-creations by humans and AI, but AI is also not simply a tool. For one thing, it is able to conjure a dense amount of visual information from just a few words in a prompt, using processes that are not so different from humans.

“When AI creates worlds, it takes input from all the images it is trained on. So do we. If I make a film with a 2D camera, nobody will deny that that’s an act of creativity. The way I put it together is informed by my conscious and subconscious viewing of numerous archives of photography and film in the preceding years, but I create something that you cannot find in the archive.

“AI can do the same. It may create something new that you won’t find in the archive. Yes, it imitates styles, but so do humans. The AI models are moving very fast and – for now – they are waiting for our prompts. We can push them to be more creative and more inventive.”

The downside to this is that realistic VR and AI images are already used to create false images, accelerating current social divisions and undermining our trust in what we see and hear, putting entire political systems at risk, he said. “It’s depressing. It’s also exciting. I can’t believe what this technology can already create, how it can also make things better. I find it’s an exciting time to work with AI right now, with so many challenges.”