Invisible face replacement of „The social network“

posted in: essay | 0

 

In the film, Marc Zuckerberg is asked to work on the idea of Havard Connection, a social network featuring the exclusive nature of Harvard students, by Tyler and Cameron Winklevoss. Later, Zuckerberg is sued by his fellow students, the Winklevoss twins‘, for stealing their concept and developing it into Facebook.

The interesting part is that director David Fincher didn’t cast twins for the roles of Tyler and Cameron and neither solely relied on the „classical“ techniques to create the illusion of twins.

Seeing twins on screen is nothing new, in fact Hollywood didn’t shy away from it as early as the musical Here Come The Waves from 1944. The classical way the „twinning effect“ is done, is a mix of an intelligent denouement of the scene (for example using over the shoulder to fake both characters being in one shot) and the use of static or motion controlled split screen where one actor plays the scene as both characters. Later, the multiple passes which are added together with masks.

 

A good example for this techniques can be seen in showreel of Orphan Black, a series about clones, by Intelligent Creatures https://vimeo.com/79259597

Orphan Black Season One – VFX Reel from Intelligent Creatures on Vimeo

However, in The Social Network, David Fincher and Lola VFX managed to raise the bar higher than ever with the creation of the Winklevoss twins through face mapping. What qualifies this effect for a presentation about invisible effects is the perfect execution. You simply cannot identify whether the shot was done in the traditional way with actor Armie Hammer acting out both parts on location or via a face mapping.

 

So, how was it done?
The illusion already began in pre-production as actors Pence and Hammer underwent physical training to replicate the similar body shapes of identical twins. This was so important because the aim was to only replace the face and keeping the body.
On set, Hammer and Pence (with tracking markers on his face) then acted out the scene.

One challenge they had to overcome was, that if Pence moved through the scene the continuous lighting changes on his face would need to match those of the future face replacement map. Lola VFX only had the actual footage and extensive coverage in means of production stills to recreate the lighting.

To capture the necessary lighting data and check it’s accuracy, Lola VFX scanned Pence’s head for a rough digital model, then object tracked his performance (when he wore tracking markers) with pftrack and then applied the tracking data onto the Pence’s digital scan. Just to again use this for analyzing and obtaining lighting information they would need, when they capture Hammer’s face map performance in a light stage (http://gl.ict.usc.edu/lightstages/) .

After the principal photography, they recorded Hammer inside bespoke light stage for his face texture map performance with an array of 4 RED cameras, where the surrounding lights would reemit the set lighting conditions on his face through a DMX style controlled lighting board (https://www.youtube.com/watch?v=tmWXD2MpuGU and https://www.fxguide.com/featured/the-art-of-digital-faces-at-ict-from-digital-emily-to-digital-ira/).


Following this, a scanned version of Hammer’s head is now carefully hand-animated to match the performance filmed by the camera array inside the light stage. Once the animation was finished, they projected the face texture map onto the animated CG head.

Finally, the Pence head tracking data is now applied to this structure (cg head + animation + face texture map). Further manipulations were needed such as correcting facial differences between Pence and Hammer, which falls right into Lola’s USP (known for beauty, digital makeup etc.). Moreover, they made lighting adjustments and extensive color matching. What David Fincher called the hockey mask technique got them most of the way, only the last 15% were done in compositing.

 

So, why did Fincher choose this route?

Complex HDR sampling on set takes a lot of time. This can break the mood and pace for actors as the production workflow is interrupted which could pose a problem in a very strong performance driven piece. Fincher is a very technical and precise director who wants everything to be perfect, which puts a lot of pressure on the crew and the actors. This method allowed the actors to only focus on their performance and ignore technical factors such as memorizing walking paths and landing on his marks. It even allows for natural human interaction that otherwise can only be achieved through trickery.

Moreover the film had 1/3 of the budget of Fincher’s earlier film “The curious case of Benjamin Button”, so intelligent and cost-benefit minded VFX might have also played a role in the invention of this method. Because of the realistic nature of this film (in contrast to the reverse aging of Benjamin Button) this method allowed for an even more seamless movie experience than the groundbreaking but still not perfect CGI head from Benjamin Button. Using face map projections, they could bypass all the problems cutting edge CGI still has, such as photoreal skin textures and muscle movement, sub-surface scattering and lighting. The fact that people believed that Armie Hammer had a twin brother is proof for the effectiveness and invisibility of this visual effect.

 

 

Sources:

http://cinefex.com/blog/twinning/

https://www.fxguide.com/featured/twice_the_social_network/

https://www.fxguide.com/featured/the-art-of-digital-faces-at-ict-from-digital-emily-to-digital-ira/

 

Leave a Reply