Performance capture -- the method of digitally recording an actor's facial expressions and body movements -- is evolving to become a bona fide filmmaking technique for creating CG characters in both animated and live-action productions. A preview of Paramount Pictures' "Beowulf" should underscore that message and make the technique a hot topic at Siggraph, the annual computer graphics confab that opens this weekend in San Diego.
A clip from "Beowulf" will be screened during Siggraph's Electronic Theater computer animation festival; it includes performance-captured Angelina Jolie, Anthony Hopkins and Ray Winstone. Said festival chair Paul Debevec: "To capture a human performance and bring it into the digital world in a way that represents the acting -- we are just on the threshold of showing that it is possible."
Numerous technology manufacturers plan to demonstrate performance capture advancements for face and/or body during the confab, but Hollywood already is demonstrating a keen level of interest.
"When one or more characters has to be synthetic by design, there are many discussions about using performance capture for live action as well as animation," said Debbie Denise, executive VP production infrastructure and executive producer at Sony Pictures Imageworks. "It frees up live-action directors to cast whomever they want in whatever role they want. I think it will become popular as long as the character design supports the rationale. We are talking to three or four companies about significant characters for upcoming films that need to be synthetic by design."
Imageworks -- the company behind performance capture-based "Beowulf," "The Polar Express" and "Monster House" -- already is developing three new projects to use its proprietary Imagemotion performance-capture technologies. They include two projects from Sony Pictures Animation, "Neanderthals" and an untitled feature from VFX supervisor Jerome Chen; and a movie based on James Patterson's "Maximum Ride."
Meanwhile, "Beowulf" director Robert Zemeckis' performance-capture studio ImageMovers Digital is using the technique to make an adaptation of "A Christmas Carol" for Disney that stars Jim Carrey as Scrooge and the three ghosts.
Industrial Light + Magic, creators of a performance capture-based Davy Jones and crew on the second and third films in the "Pirates of the Caribbean" franchise, plans to use its iMoCap performance-capture system on several upcoming productions. In addition, director David Fincher and Digital Domain are known to have been examining capture methods for production of "The Curious Case of Benjamin Button," which would show Brad Pitt aging in reverse.
Performance capture, combined with highly skillful computer animation, can be used to help breathe life into a CG human or character. Said ImageMovers executive VP Doug Chiang: "This medium can liberate the director."
In addition, performance capture has found other applications, like previsualization.
An unusual use was identified recently at Mova, maker of the Contour reality capture system. Company president Steve Perlman said that an "A-list actor" that he declined to identify recently asked the company to capture a collection of the actor's performance at present age. Perlman said Mova is essentially creating a data library of the actor, which will be stored on a hard drive.
This offers stunning possibilities. Perlman suggested, for instance, that if the actor is to appear in a flashback in a future project, the actor at a younger age is stored and available for use.
In the case of CG humans, there already have been successes, largely for stunt doubles or performances where the use of a live actor would not be practical or possible. But for actual acting -- like sitting at a table having a conversation -- most agree that the industry is not there yet.
In some cases, these efforts have resulted in an area known as the "uncanny valley," the point were the CG human is realistic but not quite right, introducing a perceptual zone known to cause a dip in empathy from an audience. VFX pros agree that the industry is getting closer to overcoming this hurdle, but Digital Domain VP advanced strategy Kim Libreri believes it remains a year or two away.
From a technical standpoint, the removal of markers from the motion-capture process is one direction innovation is heading. Instead of markers, Mova's Contour uses an FDA-approved phosphorescent makeup mixed with a base that is sponged onto the actor and used to capture motion data. Also, Organic Motion is readying a markerless body motion capture system that uses a mocap stage. "We use new types of computer vision to track and digitize the human shape without the need to attach markers or any types of devices," Organic Motion founder and CEO Andrew Tschesnok said. Meanwhile, Image Metrics is developing technology to enable markerless facial animation by capturing data directly from video. Mova will demonstrate techniques at Siggraph with motion capture system developer Vicon.
Efforts also are in place to eliminate the mocap stage. ILM's iMoCap, for instance, allows the actors to perform on set or on location while a director is shooting the scene. "It's not about technology; it's about the creative process," ILM's Michael Sanders said. "It's about not interrupting the filmmaking process."
Education initiatives also are on the rise. Earlier this year, USC launched a performance-capture class taught by Zemeckis and USC digital system specialist Ed Furie. Furie said that the school already is expanding the curriculum, and three performance capture-based student films are in production.