M&E Journal: Virtual Production: Cloud to Set and Back
We are in the middle of a virtual production phenomenon that accelerates the creation of digital content for theatrical and streaming productions.
In a way, this is not all new. Creators have employed clever techniques such as front and rear projections to manipulate what’s being seen by the camera for the better part of the last century. One thing that is common to these tricks is that the principal photography from set resembles what’s going to be seen by the audience.
However, in the last couple of decades, visual effects have moved the creative center of gravity away from the set in the post-production process where what’s going to be on the screen isn’t available until final compositing. It works great, but creative spontaneity suffers. Virtual production moves the creative energy back to the set! What the directors and photographers see through the monitors on-set is close to what’s going to be on an audience screen.
The flexibility of “fixing it in the post” remains available as a toolkit if need be.
Cloud-based production techniques are liberating creatives in a virtual production revolution to produce content where they want and when they want by leveraging a global talent pool of artists.
DESIGNED TO BE SECURE
Virtual production puts a new point of view for the creatives on set and remote streaming. A more immersive environment is brought to your stage, with the ability to create instance changes with the visualizations.
It also leading to a pattern where medium to large productions use several different independent entities in the workflow during pre-production, principal photography, and post-production.
In a globally connected production, independent companies find their creative talent all over the world. These truly global productions create a truly global attack surface for production assets, original camera files (OCFs), dailies, and other inflight materials for production.
Creative members of a virtual art department located in Manila need to be confident in their ability to collaborate securely with their counterparts in Toronto while building the 3D scene for a production in L.A. Regardless of their location, they are participating in a workflow controlled by a pipeline that spans continents. Securing the pipeline from the start ensures a studio’s ability to have full control and visibility of where assets are, who has permissions to them, and how those specific assets are used in the production.
The same security constructs are applicable for post-production, media supply chain and distribution workflows.
Using Amazon Web Services (AWS), you gain control and confidence to securely run your production pipeline with the most flexible and secure cloud computing environment available today. You can prevent unauthorized access to production assets by assigning fine-grained permissions to roles and dynamically assign users to roles depending on their participation in the production process at any given point in time.
You can gain detailed visibility into users and system actions by using logging and monitoring services including Amazon CloudWatch and incorporating other industry partner integrations.
You can automate your incident response in case of a potential breach that preemptively isolates a rogue actor in the system to prevent further damage. AWS resources for cloud-native patterns and best practices for securing a virtual production pipeline is available on the AWS website.
AWS professional services or partners with specialized competencies can help you implement these patterns for your studio.
EFFICIENT DATA FLOW TO AND FROM THE SET
A production opts to use virtual production for greater flexibility and creative control over what’s on the LED volume, VP stage, and other facilities. If you were to choose one metric that determines a truly positive experience for the director and producer, it would be the ability to move assets and content effortlessly and securely between the set and the cloud. In an ideal world, a virtual art department comes ready to the set with a perfect blueprint and 3D assets for the scene(s) on the day of the shoot.
All they have to do is to sync the local perforce edge server with the latest from the cloud, load that into Unreal Engine (UE), if that is their choice of game engine, light up the LED walls, and go on with their day.
The reality is multiple last-minute edits to the scene by the director of photography and/or the creative director. Therefore, a creative professional who is likely thousands of miles away will make changes in real time by working with a supervisor on set.
The success of this process depends on hyper-fast remote syncing of production assets. Perforce-enhanced studio pack, when deployed with best practice guidance on AWS, offers advanced collaboration to enable this workflow.
The moment the OCFs are made available to editors, creative iterations can start. A shortened dailies cycle could shave hours from a day’s schedule.
This, when combined with editors working in another time zone, could make production a 24/7 operation.
This ultimately puts invaluable time back into the schedule and saves production budget from overages.
A direct and timely ingestion of camera originals to a cloud storage endpoint creates the possibility that the dailies can pick up the take soon after “print” is called on set.
AWS DataSync is a secure, online service that accelerates moving OCNs between set and AWS storage services.
It can copy OCFs from a digital imaging technician’s (DIT) machine to Amazon Simple Storage Service (Amazon S3) buckets, Amazon FSx for Windows File Server file systems, and Amazon FSx for Lustre file systems.
AWS Direct Connect partners can help ensure that cloud-bound traffic remains efficient, highly avail- able, and resilient.
A virtual production requires multiple collaborators to work closely in real time during pre-production, production, and in post-production. AWS Partners including Epic Games offer native capabilities through collaboration plugins in the case of Unreal Engine.
AWS offers the ability to make transport secure, resilient, and reliable with a cloud-native VPN.
More importantly, the creative work, when supported by a virtual workstation in the cloud, enables the ability to remotely share pixel streaming session by more than one individual thousands of miles apart.
AWS offers virtual workstation solutions like Amazon Nimble Studio, which is a purpose-built service for content production workloads.
If you want to build your own, AWS offers a broad selection of graphics-capable virtual machines in the cloud including recent generation NIVDIA GPUs.
Moreover, several AWS partners offer purpose-built solutions for cloud-native virtual workstations on Amazon Elastic Compute Cloud (Amazon EC2) for Digital Content Creation (DCC) applications.
FUTURE OF VIRTUAL PRODUCTION WITH THE CLOUD
Tomorrow’s landscape is uncharted, but there are hints of technology that will help push the boundaries of what is seen on the silver screen.
LED panels will become denser and offer more lifelike depth.
The supporting technology to display pixels will become more performant and bridge the gap between new digital techniques.
Many of these pieces are already in place, including having a Wacom Tablet at your desk connected to an Amazon EC2 GPU-enhanced instance running Autodesk Maya.
AWS Local Zones create a low-latency connection between the artist and the machine, building towards a seamless experience.
The possibility of having an internet connection and working anywhere is a reality. Now, adding in remote panels with onboard caches, you can move a virtual art department into your living room.
One thing is certain, tomorrow will bring new and exciting technology to build visual delight.
* By Provanshu Dey, Solutions Architect Leader, Virtual Production, and Matt Herson, Solution Architect Leader, Content Production, Amazon Web Services