Video Interpolation

📘

We recommend reviewing Video Sequences before Interpolation

Video Sequences

👍

Run Interpolation Often

After every sequence creation.
Interpolation detects sequences with changes and it's a lot faster to run after each sequence creation then later.

Interpolate uses keyframes you created to fill in the gaps in other frames.

Interpolate allows you to save time annotating. Used well you may only need to annotate every 10th image.

In this example, we have key frames on the left and right with interpolated frames in the middle. In order to interpolate a video that has multiple objects we use a sequence for each object.

The keyframes are the same keyframes created during normal Sequences.

Getting started

1. Annotate a sequence
2. Press Interpolate button
3. Review results and edit as desired

Be sure to select the same sequence for the same object.
For example, the car in sequence #1 at frame 0 should be the same car in sequence #1 at frame 5.
Please see the introduction for a more in depth look at this.

FAQ

📘

Run time

The first run on new frames is expected to be slower.
After that, a single new sequence of up to 50 frames will usually run in a few seconds.

How many key frames do I need

Usually every 3-7 frames.
If the object is stationary for a duration: Set a keyframe when it becomes stationary and a second keyframe the frame prior to it resuming motion.

When should I run interpolation

After annotating at 2 least key frames in at least 1 sequence. In general a sequence usually has at least 3-7 keyframes, so normally it's better to run it after you are done all the expected keyframes in a sequence.

How often should I run interpolation per sequence

Often 1 or 2 runs per sequence is enough.
For faster motion the iterative approach of running it then correcting it may require 2-3+ runs (with edits in between).
You can run it as many times as you like, but it does take some time, and time to go back and review.

Interpolated Display

Interpolated Icons are displayed in the instance detail with the extra icon:

Human Edited Frames are Promoted

These frames will likely show in keyframe list once saved.

📘

Human edited interpolated frames are promoted to keyframes when edited.

Edit instances like normal (including after interpolation).

Every time interpolation is run it refreshes the interpolations based on all new data. So changing a single keyframe at the worst point, will improve both frames before and after it.

You can also add more instances to the end of start of a sequence at any time.

Best practices

Motion

• There should be a keyframe as soon as the Time Series concept starts. In some cases this may be when the object enters the video and in the last frame that it leaves
• More rapidly changing motion should have more keyframes. If there is a lot of motion between frames you may need a keyframe every few frames.
• If the object is not moving, you can create a keyframe at the start of the video and at the end.

Controls

• Use a slower playback speed to better see interpolated annotations
• Only human annotated frames show up under the keyframe list. Interpolated frames will render as part of the visual, and show up and the normal instance list.
• Interpolation runs on the entire video (all sequences), only sequences with changes are effected.

Concepts

• Don't expect interpolated annotations to be perfect - if your goal is higher quality you will need more keyframes.
• A great way to work with interpolation is with multiple passes. For example, you could do your best effort, run interpolation, then review the results and correct the largest errors. Each correction is typically faster than the previous, and you can usually get good results this way within a few passes.
• If interpolation is not working well try using pre-labeling to help too, pre-labeling works even when the object is displaying non-linear motion

Success Case

No changes.

Error Handling

A sequence must have at least 2 keyframes. A keyframe in this context is any human created instance.

Specs

• The system interpolates key-frames in a linear fashion if there are only 2. If there are at least 3 keyframes within the sliding window of the keyframe it will using a Zeroth degree non-linear spline.

Interpolate works with video - so your file must be a video to see the interpolate option appear.