Friday
Sep022011

Portal: No Escape - Hallway sequence

Earlier this year I worked on a dozen shots for the short film "Portal: No Escape", directed by Dan Trachtenberg, together with a remote team of vfx artists supervised by Jon Chesson. The film, with the talented actress Danielle Rayne in the leading role, premiered at this year's Comic Con in San Diego and was released online last week, reaching viral proportions of over 5 million views in less than 6 days, and marked as the 'most liked video' in August (recommended viewing in 1080p HD in FULLSCREEN with sound UP):

 

After the initial contact with Dan was made via Twitter, I was asked together with my good friend Marijn Eken to do the matchmoving for this project, where I would be responsible for the Hallway-sequence and most of the Roof-shots. In this blog post I'm hoping to give a little insight in some of the matchmove work that was done on this amazing short film.

The film was shot using a Red One camera, using a set of Lomo Anamorphic lenses of 35, 50 and 75mm, where most shots were done with a 50mm. The tracking plates were handed to us in the form of a (quite dark) 1080p unsqueezed and letterboxed Targa sequence, with no additional info on the original Red and anamorphic modes that were used during the shoot, nor how the footage was cropped to HD.

Hallway sequence thumbnails and shot numbers

Image-distortion

One of the first things that stood out when I studied the tracking plates, was the very prominent high frequency rolling shutter, especially at the end of the sequence (see: image wobbles in shot 17 and 18).

A logical tool for tackling this image-distortion problem, that is often experienced in CMOS cameras, is the Rolling Shutter plug-in by The Foundry. However, using their suggested settings (a correction value of 0.32 for the Red One camera) introduced serious artefacts in the corrected footage, even after experimenting with the other input parameters of the plug-in:

Example of rolling shutter correction artefacts in shot 18

These artefacts could be 'fixed' by a paint artist later in the vfx pipeline, but they can also mess up a 3D track, because tracking points might be 'jumping' on bad frames and therefore not deliver accurate results. This led us to decide to work with the original tracking plates and see how far these would get us...

Next step in the 3D tracking workflow is removing a possible lens distortion from the tracking plates. Because no lens distortion grids were available here, I used 'checklines' to unbend lines in the plate that were assumed to be straight in reality. Unexpectedly this led me to conclude that this unsqueezed anamorphic footage had no significant lens distortion.

Camera tracking

To get a good first solve for the camera motion, I decided to ignore the whole rolling shutter problem for a moment, and just tried to track the original tracking plates...

Using SynthEyes this turned out to be pretty straightforward. For most shots it was nothing more than increasing the gamma to make possible features in dark areas more aparent, use the Auto-tracker to track the shot, manually delete the bad trackers, solve the shot with a known 50mm lens on a 24.4 x 13.725 mm sensor and finally orient the ground plane in a similar way for each shot to simplify Hallway reconstruction later on. The only shot that needed a different approach was shot 13a, which was almost completely out of focus. By carefully placing 11 manual trackers (using SynthEyes' very efficient workflow for supervised tracking), I was able to solve the shot with the same lens and sensor.

However, even though the camera solve wasn't corrupted by the rolling shutter, the wobble in the plates of course remained... and thus some of the solved 3D positions for the 2D trackers were still sliding in areas were the high frequency rolling shutter appeared. Realizing that I would probably have to face this problem at some point, I decided to build a proxy model for the Hallway first...

Hallway reconstruction

My initial idea was to build one Hallway model that matched the plates in all the shots, so the scene scale and a position within the Hallway would be consistent from shot to shot, which would be a great benefit for the particle simulations that needed to be done later on. This proved to be quite difficult, partially due to the green screen Portal stand-in on the wall changing from shot to shot, and because the walls seemed to be slanting more in one shot than the other.

I started out building a proxy model for shot 14, which had the best view through the Hallway towards the start. Then I copied this model to shots 13 and 16 which have the same viewing direction, changed the scale, position and orientation of the new scene (where needed) to match the model from shot 14 as close as possible, and finally fine-tuned the geometry in areas where differences between shots would appear... In this way the proxy model could vary a little from shot to shot, but consistency in scene scale, position and orientation would be maintained.

Proxy Hallway model for shot 14 (with scene origin)

Continuing with the shots viewing at the other end of the Hallway, again I used the proxy model from shot 14 to build forward on, and used the same principles to reconstruct a matching geometry for the remaining shots.

3D rolling shutter simulation

Once I had a proxy model for the entire Hallway in place, the idea was born to simulate the effect of the high frequency rolling shutter in 3D to get a better match, by deforming the model slightly from frame to frame. Obviously this was a cheat that might help with compositing, but one that should never be used as input for any particle simulations, because they can be influenced by the vibrating walls and ceiling.

An important thing to realize here is that high frequency rolling shutter can be a very local problem. The ceiling might be vibrating up and down while the floor is not, the left wall might be vibrating left and right while the right wall is not, or the right wall might be skewing close to the camera while in the distance it is not. But because in this sequence the camera was always looking along the length of the Hallway, I suspected this image distortion could be simulated by using a lattice deformer.

After some experiments showed that a 3x3x3 lattice around the complete Hallway model was the ideal setup, I was able to find a perfect match for every frame of the sequence by animating individual lattice points in areas were the high frequency rolling shutter appeared:

3D rolling shutter simulation (using a 3x3x3 lattice) in shot 17 played at half speed

Segment of the Hallway matchmove in shot 17 before and after 3D rolling shutter simulation

Portal gun reconstruction

For making believable gun blasts, particle work and contact lighting on the Portal gun, a seamless object track and an accurate reconstruction of the Portal gun were essential.

Before reconstruction I would normally place trackers on the Portal gun and calculate a motion path and point cloud first, but because of the lack of trackable features on the gun and the fact that in a lot of frames the black parts of the gun almost disappeared in the background, this workflow was bound to fail.

So I started with building a proxy model for the Portal gun based on available reference photos:

First rough version of Portal gun model based on available reference photos of a damaged gun

Reference didn't get me all the way there though, because the gun was slightly different from the one in the tracking plates... the barrel was damaged (probably during the shoot) and I found out it was able to slide back and forth, also causing the cables to bend.

So to be 100% sure my proxy model for the Portal gun would match, I needed to revert to the tracking plates and fine tune my first rough version:

Fine tuned version of Portal gun model based on 3 viewing angles in shot 14

Basically this meant that I needed to setup the Portal gun model for rotomation (more on this later), and position and orient it on a number of 'key' frames in a shot with clearly different viewing angles on the gun, giving me enough information to fine tune the proxy model.

Unfortunately though this model couldn't be used for all shots, because it appeared that they used a second 'stunt' version of the gun in shots 17 and 18 where she's fighting off the guard. The shape of this 'stunt' version was so different from the 'original' Portal gun, that I had to build a separate model for it:

Adapted 'stunt' version of Portal gun model based on 5 viewing angles in shot 18

Portal gun rotomation

While normally, having an accurate proxy model for the Portal gun would be an ideal starting point for a geometry track, just like the regular object track for this sequence, it was bound to fail... In some shots the black parts of the gun were almost indistinguishable from the background, and in others the gun was moving in and out of sight entirely. Alltogether this made rotomation the best way to go.

First thing to do before starting any rotomation, is making sure the pivot point and rotation order are chosen wisely, and the scaling is worked out properly. Luckily I got some measurements from set which gave me good starting point to set this up:

  • width of hallway = 5' = 1.524 m
  • length of actress = 5' 3½'' = 1.613 m
  • length of gun = 27" = 0.686 m

Using this information, I scaled, positioned and oriented the model of the Portal gun inside the reconstructed Hallway model on a number of 'key' frames in shot 14. Then, while everything seemed to look right in frame, I did the same thing in a couple of other shots, to make sure the scaling I found in shot 14 would place the gun in the Hallway correctly in all the other shots of the sequence as well. To illustrate the importance of this step, check out how much influence a proper scaling has on the positioning of the gun, noting that the rotomation of the gun matches perfectly in frame for both scaling values:

TIP: Motion path shift occurs when the wrong Portal gun scale is chosen before rotomation

Once the scaling for the Portal gun was locked down, the process of rotomation was basically just animating the gun by hand to make it match on every frame, while making sure that the motion path of the gun was looking realistic in 3D. For every shot I would start with a correctly positioned and oriented gun on a number of 'key' frames in the shot, and fill in the areas in between from there...

While doing this, it proved to be very important to keep an eye on the animation curves... Working from 'key' frames can sometimes result in 'unnatural' jumps in the animation curves, which might look fine in both 2D and 3D, but they can cause strange problems when particle simulations are attached to the gun. Reason for this is that particle simulations not only look on-the-frame only, but also calculate substeps between frames, which look a lot different in both versions of the curves:

TIP: Rotation curves with 'unnatural' jumps seem correct but can have a strange effect on particle sims attached to the Portal gun

Scene delivery

One more thing to mention before showing the final result is that, while FBX worked fine for getting all this tracking data from Maya to Softimage, Nuke was having a bit of trouble importing all animation curves correctly. Luckily I ran into this very handy FromMaya2Nuke Python script, that automatically prepares a Nuke scene including animated camera and obj-sequences for all moving geometry, making both my life and the compositors' lives a lot easier...

Final result

Of course I can't finish this post without showing the final result... so here it goes, with compliments to Dan Trachtenberg, Danielle Rayne, Jon Chesson, Paul Griswold and others who all did a great job on this sequence (recommended viewing in 1080p HD in FULLSCREEN):

 

Also check out what other team members say about working on this project:

 

References (52)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    Response: Sims 3
    Marco de Goeij - 3D character setup
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    NFL is truly one of the biggest sports in America. It has a significant following.
  • Response
    Response: useful reference
    Fantastic Webpage, Stick to the wonderful job. Many thanks.
  • Response
    Response: シューズ通販
    Whenever you think of buying a new pair of s oakleyサングラス hoes, you just go to market and buy the shoes that look very stunning and often come at reduced prices. Actually selecting right kind of girls dress up shoes is an art which may require enough experience an
  • Response
    Response: louis vuitton uk
    The requirements for immigration to UK for the married couples are always been louis vuitton uk taken into consideration following by the published UK government rules for the given policy. A person dreaming to marry in the UK has to prove their nationality which is
  • Response
    Athletic shoes are designed a ショートエンジニアブーツ nd manufactured based on features of people participating in sports or traveling. The soles of athletic shoes are different from those of ordinary leather shoes, casual shoes, the soles of athletic shoes are generally sof
  • Response
    Response: サングラス
    Men's shoes are highly important for any man in his daily life. Footwear is ショートエンジニアブーツメンズ important for various purposes. Apart from their functionality and the protection factor, shoes are now the most important fashion accessory for men. Shoes make the whole att
  • Response
    Shoes are an indu アディダススニーカー 6 stry with a glorious future. Everyone, everyday wears shoes. There are shoes for 偏光サングラス 12 all occasions and can be found in a varied price range. The shoes maybe the hardy sort or could be highly fashionable and trendy. Then there are the st ジバッグ ...
  • Response
    The shoe designers and manufacturers throughout the スニーカーレディース 16 world have ブーツレディース通販 realized the influence of Internet as an ultimate selling tool. The online shoe shops have thus become a very common place to shop shoes. A shoe レザースニーカー shop is not bound by location, time or any
  • Response
    Response: ジバッグ
    In the popular footwear, a kind of such arguments by 2010 ジバッグ is China shoe "2012 レザースニーカー ". This is a delibrately sensationalizing, or are displaying the terrible reality? "Power" crisis As the labor-intensive industries, shoemaking lik adidasスニーカー e "migratory birds" same,
  • Response
    The word durability goes wit シュプリームtシャツ 8 h some classic shoe brands today. CAT shoes or caterpillar shoes brand briefingバッグ 17 is one of the most applauded shoe brands for its amazing durability. The people who believe that shoes are for life should buy caterpillar shoe ニューバランス576 s! Well,
  • Response
    Shoe Goo is an indus シュプリームtシャツ trial strength rubber cement that may be exactly what you are lo gravisスニーカー通販 18 oking for when it comes to repairing your old shoes. Invented in 1972,シュプリームtシャツ,gravisスニーカー通販 18, S dr.martensブーツ hoe Goo was originally designed to combat a problem known as “tenn
  • Response
    The word durability goes with some classic shoe brands today. CAT shoes or chippewaブーツ通販 c アディダススニーカー aterpillar shoes brand is one of the most applauded shoe brands for its amazing durability. The people who believe that shoes are for life should buy caterpillar shoes! Well メンズ靴 11 ,
  • Response
    With platform shoes out of fashion it is unli briefingバッグ kely that you will find them in the men's section in any shoe store. Platforms, however, are still available for women and continue to be in fashion. For men who are looking to buy these shoes to gain heig
  • Response
    As we all kn oakleyサングラス ow by now skateboarding was not a recognized sort when it first began, which is when the metal wheels were made. This was understandable at first since all one could do was roll forward on a skateboard and try not to fall down. For that s ...
  • Response
    What you wear on your feet says an a ニューバランススニーカー wful lot about you – and when it comes right down to it, your shoes can make or break your day as well. Nothing is as nice as having a comfortable, easy to wear, and great looking pair of shoes. No ...
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Winter comes and leather boots become 靴 popular gradually. As the new fashion this year, soft leather boots get more and more attention of people. What are the advantages of soft leather boots? What makes them differ from other boots? In order to make
  • Response
    Response: program make
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: ラコステ靴
    Oprah loves her ugg boots so a good deal that she featured the アディダススニーカー 6 m on an ep アディダスランニングシューズ isode of her well-liked display entitled Oprah's Favourite items 2005. Kate Hudson wore a pair of trendy ugg boots for the poster to the motion picture Raising Helen. Pamela And メンズブーツブランド ...
  • Response
    Response: adidas通販
    Are you in need of such the dog shoes boots that can give comfo オフィシャル 13 rt and protection to your pet? If yes then call us or visit to our web ナイキ通販 site for the Waterproof Anti-Slip Sports Pet Dog Boots Shoes. These are such the dog shoes boots that ...
  • Response
    You can find b ファッション通販 4 ecause スニーカー several styles of climbing boot styles because you can find walkers. The particular specific things in order to think about if deciding upon ones mountain c ブーツメンズ 14 limbing boots will depend on the sorts of climbing you might be planning
  • Response
    Response: web page
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Clark Philippines operates under a slightly ブーツメンズブランド set of rules from the rest of the country. There are many advantag ブーツメンズブランド es but as usual, there are also areas that investors need to be well apprised to avoid unnecessary problems.Clark Freeport Zone offers man ニューバランススニーカー 6 y
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Wholesale LV handbags are one in the most in demand and also t Lacoste アウトレット he most preferred manufacturer bag across the globe these days. Its looks that most of the women who personal an LV bag just loves to take them around with them regardless of time and loca
  • Response
    Comfort is the most important factor that you should be looking for when spxスニーカー通販 you are purchasing a pair of shoes. ニューバランス After all, this is something that will support you whenever you are walking, running or even standing. So, you must be comfortable with your GRAVIS アウトレット
  • Response
    If you are searching for a perfect brand of shoe that ca Dr.Martens アウトレット n cater to your requirement of both com ニューバランス576 fort and style, you can go for the Rieker shoes. If you have not yet purchased a pair of these shoes, it is high time that you purchase ...
  • Response
    Networking is known to occur through the com スポーツサングラス munication between new individuals. Some underestimate the power of networking between companies and individuals. Innovation Process Management helps support networking and collaboration within a business w
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: Supreme Garcinia
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: ResVibrant Reviews
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: ResVibrant Reviews
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: ripped rx Reviews
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: jeux steam gratuit
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: My Score Reviews
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: Digest it
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: Digest it Reviews
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: outdoor dog bed
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: outdoor dog bed
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: Digest it
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Response: Digest it
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence
  • Response
    Marco de Goeij  —  Matchmoving, Rotomation & On-set mocap - Mad-id weblog - Portal: No Escape - Hallway sequence

Reader Comments (2)

Wow! No Escape is a great film. Thanks for sharing the secrets behind this film. Good Job!

September 23, 2011 | Unregistered CommenterMarc Azada

I watched it over and over again. I hope this will be in a full movie. I will surely see it in cinema. Amazing!

April 17, 2012 | Unregistered Commentersustainable architects

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>