Skip to main content

How to remove a coffee cup from a video — for instance, Game of Thrones

How to remove a coffee cup from a video — for instance, Game of Thrones


The quick way to track and replace content using After Effects

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

In the latest release of Adobe’s After Effects, the company added a tool long desired by users: the Content-Aware Fill, which provides a similar function to the Content-Aware Fill tool in Photoshop. Now you can easily remove objects from moving video, just like you’ve been able to do in still images for years. It works on all kinds of objects, from still to moving, and even coffee cups. I wonder where that might come in handy?

Using the Content-Aware Fill in After Effects is very similar to how it works in Photoshop: You simply select an area you want to remove and have replaced with magically generated imagery that looks like it was meant to be there all along. But that isn’t to say it will always produce the best results: I used the tool to remove the coffee cup from this scene but was not completely satisfied with the results. I’m sure that with some more tinkering I could achieve something I’d like, but instead I defaulted back to a process I’ve used for years by simply tracking motion and using the original clips footage to mask over the unwanted object.

Original footage
Original footage
Clip: HBO
Footage using Content-Aware Fill
Footage using Content-Aware Fill

That process is a little more complicated than the nearly one-click process that Content-Aware Fill provides, but it’s not hugely difficult to do once you know the steps.

  • Open After Effects
  • Import your footage
  • Create a new composition with your footage
  • Trim your work space to the area you need to correct
  • Right click on your layer and select Track & Stabilize > Track Motion
  • In the Track Motion panel along with Position select Rotation and Scale
  • Use the two Trackers to select pixels near the object you want to remove. Edges and corners or areas of high contrast are great to track
  • In the Track Motion panel select Analyze Forward and let it track the motion and create keyframes.
  • If the track seems off you can try tracking different areas until you get a track that matches the cameras movements
  • Create a new Null Layer in the composition
  • In the Track Motion panel go to Edit Target… and select the Null, then apply post the X and Y positions
  • In this clip there is a section of table that looks like we can use it to cover the object so let’s find a frame that works the best and then duplicate the footage layer
  • On that frame, right click and freeze the frame by selecting Time > Freeze Frame
  • Using the Pen Tool, make a mask on the Freeze Frame layer, then move the entire layer to cover the object and align as best as possible. You might need to refine the mask after placing; I feathered the edges of the mask to blend in better.
  • On the Mask layer, use the Pick Whip to select the Null, this will parent the Mask layer to the Nulls attributes, including the keyframes for position, rotation and scale.
  • Previewing the work area you can see the Mask layer move along with the Null, match the position of the table.
  • Next form the effects & Presets panel find the effects Exposure and Add Grain and add to the Mask layer
  • Adjust the exposure to better match the position of the table
  • Adjust the grain to match the grain from the camera (since we made a freeze frame there is none so it would stand out)

And there you go! Now you just need to upload your coffee cup-free footage to your massive content company’s distribution servers.

No more cup!
No more cup!

You can also use the same Null to parent other layers, too. Say if you wanted to add something back in…

A little addition...
A little addition...

Vox Media has affiliate partnerships. These do not influence editorial content, though Vox Media may earn commissions for products purchased via affiliate links. For more information, see our ethics policy.