Virtual production is gaining more traction in the motion picture business. One example? The recent collaboration of several companies came together for both practical and creative reasons to bring the Disney+ feature the Muppets Haunted Mansion to life. “The Muppets and ARwall are a perfect fit,” says Rene Amador, co-founder and CEO of ARwall. His company creates virtual production technology for its own use and in turn markets it to other filmmakers. Its first product is ARFX, a suite of real-time tools developed natively for using real-time in-camera LED effects in scripted filmmaking.
“We [were] very lucky the studio and filmmakers believed in us, involved us early in development as the script was being written, and gave us free reign to educate the other department heads on how moving to in-camera LED effects was going to change their workflow,” Amador continues. “As filmmakers ourselves, we worked with Kirk Thatcher the director to make sure all deptartments were on board, and I think it shows in the frame.”
ARwall calls the resulting virtual stage at Soapbox Films that was built for the show The Cuboid, because of its rough resemblance to a cube. The equipment included a primary 20.5-foot by 11.5-foot LED main wall using Ledman COB M series 1.5mm pixel-pitch LED panels; two 15-foot by 10-foot sidewalls; a 2-foot by 15-foot ceiling; and two 20-inch by 40-inch mobile walls with Absen VN 8mm panels for additional lighting.
The sets themselves were a collaboration between four teams: ARwall, Dastoli Digital, Allucinari, and the production designer Darcy Prevost’s team of seasoned set designers. Special attention was given to weave in a spate of easter eggs to satisfy the most die-hard fans of both The Muppets and Haunted Mansion, some of whom happened to be on the actual team designing the environments in blissful glee.The main wall appeared on camera compositing the virtual sets with the actors and set pieces, while the five other screens provided immersive lighting and reflections that shift according to the needs of the scene. Need a quick dash of lighting for a spooky moment? It’s just one keystroke away. Delivering the real-time visuals for all the six screens was a single powerful ARFX Pro server running Unreal Engine; controlled by a keyboard, gamepad, and Midi board.
The collaboration didn’t end there, as the unique needs of The Muppets performers, the pandemic’s new concerns, and the speed of film production required ARwall to build entirely new toolsets specifically for the production. The result allowed virtual production to be used in nearly every scene, filling in the halls, rooms, and mysterious grounds of the familiar Haunted Mansion.
“We found that the usual method for changing and saving a scene while playing weren’t a fit for the speed of production, so we coded a workaround that allowed us to make changes to a scene while it was still active,” says Jocelyn Hsu, co-founder and chief product officer at ARwall, “The big revelation on this show was moving the tools into the real-time engine itself, and out of Unreal Editor, to get the best performance on set.”
Together, Amador and Hsu served as virtual production supervisors on the production, orchestrating the flurry of technicians, artists, and equipment required to achieve real-time effects for a studio with expectations as high as Disney’s. ARwall, and the technology behind it, has been the brainchild of its founders for nearly five years, beginning as an experiment in a kitchen that eventually became ARFX.
“An emerging tech like this needs a team that is crazy enough to be working on both the technical and creative sides of it,” says Amador. “That’s ARwall. Our model is to create jaw-dropping results, then release the tools we built for others to use on their own productions.”
Muppets Haunted Mansion is available to stream now only on Disney+ and currently has a 90 percent score on Rotten Tomatoes.