- Matchmoving a Stereo Project using 3DEqualizer and Maya
Matchmoving is one of the invisible effects in this industry. It is vital to combine CG and live-action. Weta Digital's camera departmenthad a challenging role to deliver over 300 stereoscopic matchmoves forAvatar.
"I have always been fascinated by integrating CG into real life footage. Robots or any type of creature interacting with real actors. Extending film sets with CG environments to make them look the size of a city or bigger. For Avatar it was mostly the other way round and we had to integrate real actors into a CG environment. And that in stereo which means instead of one plate we had to deal with two plates mimicking the human perception. We hadn't worked on a stereo project before and the fact that the cameras' interocular, convergence and focal length could change at the same time was a very new concept.
We had to sensibly recreate a stereo-rig that reflected the real cameras'motion, interocular-shift, change in convergence and zooming which took a good deal longer than matchmoving plates for a non-stereo show.
We are very lucky to have survey for most of our sets. Occasionally we have shots with only a limited amount of survey. For such situations extensive photo-lineups within 3DE helped a lot: additional reference pictures taken by the onset team 'fill-up' our survey. To be sure we always use the correct lens settings for this photogrammetry within 3DE we read out the reference pictures'metadata and compare them with previously taken grid shots that we useto measure the lens used by the onset team.
For one sequence the survey and additional camera information had become unreadable. Since we needed a CG camera for set extensions and critical contact on moving objects within the scene, an iterative approach was needed to establish the right relation focal length, interocular and recalculated points. Comparing the plates with shots of other sequences we could already guess where we would have to start. We used these 'guestimates' as a starting point in 3DE then checked if the recalculated result still made sense.
One sequence was particularly exciting to work on:
We got a filmed set element and separately an actor filmed in front of a green screen. Both had to work perfectly together since we had to implement Neytiri with critical contact in both plates. Additionally the glass on a flexible rebreather that is first in Neytiri's hand then put onto Jake's face had to be added in CG and needed accurate matchmoving. The CG cameras for each plate were almost identical. However, since every tiny error in stereo shows up so much more than in anon-stereo film we had to correct for the slight difference in motion and convergence.
We decided to use one plate as the main plate and adjust the second plate by pre-stabilizing using an elaborate 3D/2D setup. That locked the plates together, corrected all differences in convergence and allowed for set extensions and critical contact of our CG character. Both plates could be combined now and a new CG stereo camera that worked for Neytiri with critical contact in both plates was created.
We work with a variety of tools to be able to deliver 3D setup scenes with CG cameras, dewarped plates, set geometry, rigid object matchmoves like guns or helicopters or matchmoves of deforming objects like skin or even water. But we also deliver 2d setups to combine various 2D elements (different miniature passes, etc.) into one shot. Cardrigs are something in between. With those we can place live action elements onto CG creatures or in CG environments anywhere we want to. The resulting changes in scale and perspective due to different distance sand angles to the CG camera are piped back to our compositing software.
In some instances we tried to pre-vertically align the plate in comp. This proved to be useful before the release of 3DEqualizer4 since we couldn't properly calculate vertical offset or depth shift between the cameras then. Since 3DEqualizer4 we can calculate all these values in relation to each other at once and deliver a CG camera setup matching the original plate. This is a huge leap forward in solving cameras in stereo for critical contact. The vertical alignment happens as a last step in comp now.
In the old days we struggled with long shots in full resolution. Today we can easily load in huge stereo plates in HD or 2k without loosing any performance. That way we work with the most detail available and have a greater freedom to extract tracking markers off a blue/green screen using either the image controls within 3DE or alternatively Shake or Nuke if we believe the original scans have more image detail than those we use in 3DE. For animation we hand over CG cameras and dewarped plates with smaller resolutions to keep their workflow fast and easy.
We had quite a steep learning curve for this project and were stoked to see Avatar on the big screen in stereo."
Weta Digital Ltd