Gemini Omni video extension and frame interpolation: API workflow and continuity checks

You may already have a short clip that ends before the action is complete, or only a first and last frame with no motion between them. With gemini-omni-1.1-flash, you can append a 3–10 second continuation or generate a transition between two boundary images; by the end, you will know which mode to use, how to call the API, and how to decide whether the result is deliverable.

Use tail extension when you want to preserve the motion, camera, and sound of an existing clip; use first/last-frame interpolation when the start and finish are fixed and only the middle is missing; an MP4 response only completes generation, so you still need to inspect the subject, motion, seam, and audio.

Should you extend the clip or interpolate two frames?

If you have source video, start with tail extension; if a specific final image must be reached, start with interpolation. Extension is not bounded by a target final frame, so using it for that job can add another generation and review cycle.

Job	Input	Best use	Continuity to inspect most closely
Tail extension	One video, optionally with reference images	Continue an action, camera move, or scene for a few seconds	Pose, speed, camera direction, lighting, and audio around the seam
First/last-frame interpolation	A first frame, a last frame, and a transition prompt	Fill in motion between two defined states	Identity, spatial relationships, motion path, and arrival at the target frame

Uploaded-video extension is designed for short increments: the official guide describes a 3–10 second continuation, and the prompt guide also documents 10-second steps up to 40 seconds total. Your uploaded clip must be 10 seconds or shorter, and footage can only be appended at the tail; use a conventional editor if you need to insert a shot at the beginning or in the middle.

Decide these four things before you call the API

Fix the model, output specification, cut behavior, and source material you will use for comparison before generating. Missing one of these decisions often means rerunning the job or being unable to tell whether a seam change came from the model or the source.

Use the GA model ID

Use the GA model ID gemini-omni-1.1-flash. It became generally available on August 27, 2026; the older gemini-omni-flash-preview endpoint is scheduled for deprecation on September 30, 2026.

Validate continuity at 720p first

For the first pass, use the default 720p so continuity debugging stays separate from high-resolution delivery. The resolution field in response_format accepts 360p, 720p, 1080p, and 4k; the documentation identifies 1080p and 4k as upscaled outputs, with 16:9 as the default aspect ratio and 9:16 available on request. For results larger than 4 MB, especially above 720p, use delivery="uri", wait until the file becomes active, and then download it.

Choose between one continuous shot and a deliberate cut

Decide whether the extension is one continuous shot or an intentional cut. In an extension prompt, 0s means the start of the newly generated portion, not the start of the source clip. Ask for a “single continuous shot” and “no scene cuts” when the action must remain unbroken. When a cut is intentional, state when it should happen within the extension.

Keep the source clip and boundary images

Retain the source clip and both boundary images. The official prompt guide says Omni uses the last 10 seconds of the original video as context and may edit some final input frames to make the transition. That makes a source-versus-output comparison part of the continuity review, not an optional backup step.

Have a source clip? Extend it from the tail

Use tail extension when you already have video and want the action, camera move, or scene to continue for a few seconds. Start with a minimal 720p request, then tighten the subject, motion, camera, and audio constraints.

Run a minimal 720p extension request

The Python example below uploads the clip through the Files API, waits for processing, and passes the video URI and continuation instructions to interactions.create. It stays at 720p so you can solve continuity before dealing with high-resolution file delivery.

import base64
import time
from google import genai

client = genai.Client()
video_file = client.files.upload(file="my_video.mp4")

while video_file.state == "PROCESSING":
    time.sleep(10)
    video_file = client.files.get(name=video_file.name)

if video_file.state == "FAILED":
    raise ValueError(video_file.state)

interaction = client.interactions.create(
    model="gemini-omni-1.1-flash",
    input=[
        {"type": "video", "uri": video_file.uri},
        {
            "type": "text",
            "text": (
                "Continue the scene as one unbroken shot. "
                "Keep the same subject, wardrobe, camera direction, lighting, and motion speed. No scene cuts and no new dialogue."
            ),
        },
    ],
    response_format={"type": "video", "resolution": "720p"},
)

with open("extended.mp4", "wb") as f:
    f.write(base64.b64decode(interaction.output_video.data))

Write prompts as observable conditions

Do not stop at “continue the scene.” State at least five observable dimensions: what must remain unchanged about the subject, how the action should develop, how the camera should move, whether light or environment may change, and whether cuts or new sound are allowed. “The right hand keeps rising at the same speed” is testable; “make it natural” is not.

Set task only when prompting selects the wrong mode

If prompting does not select extension mode, add {"video_config": {"task": "extend"}} under generation_config. The guide recommends prompting first because setting task adds stricter constraints. For a video generated in the preceding turn, previous_interaction_id can continue the sequence without another upload. That path requires stored interaction state; a result created with store=false cannot be edited later through that ID.

Change the plan when dialogue or region limits apply

There are important dialogue and region boundaries. An uploaded video in which someone is already speaking cannot currently be extended with additional dialogue; a silent continuation or a prompt that adds no dialogue is allowed. Spoken continuation is supported for multi-turn extension of model-generated video through previous_interaction_id. Editing and extending uploaded video are currently unavailable to users in the EEA, Switzerland, and the United Kingdom, while extension of model-generated video remains supported in available regions.

Only have boundary frames? Generate the motion between them

Use first/last-frame interpolation when the start and finish are clear but the motion between them is missing. Submit the two images in order and describe how the subject, camera, and environment should move from the first state to the last.

Submit the two boundary images first

Interpolation fits a job with clear boundary states but no middle motion. Put the two images in order in input, then describe how the subject, camera, and environment should travel from the first state to the last. This complete example reads local JPEGs, encodes them, and requests a 720p result.

import base64
from pathlib import Path
from google import genai

client = genai.Client()

first_frame_b64 = base64.b64encode(Path("first.jpg").read_bytes()).decode()
last_frame_b64 = base64.b64encode(Path("last.jpg").read_bytes()).decode()

interaction = client.interactions.create(
    model="gemini-omni-1.1-flash",
    input=[
        {"type": "image", "data": first_frame_b64, "mime_type": "image/jpeg"},
        {"type": "image", "data": last_frame_b64, "mime_type": "image/jpeg"},
        {
            "type": "text",
            "text": (
                "Transition naturally from the first frame to the last frame. "
                "Preserve subject identity and camera direction, keep the motion path continuous, and avoid abrupt cuts."
            ),
        },
    ],
    response_format={"type": "video", "resolution": "720p"},
)

with open("interpolation.mp4", "wb") as f:
    f.write(base64.b64decode(interaction.output_video.data))

Use image_to_video only if the mode is still unclear

After submitting the images in order, state that the first is the start and the second is the finish. If the mode is still wrong, use image_to_video as a task value; it supports a precise description of subject, camera, and motion rather than replacing one.

The larger the difference, the more you should sequence the changes

Compatible boundary frames make the problem easier to specify. Keep subject identity, aspect ratio, camera axis, and spatial layout reasonably consistent, or explain the order of change: turn the subject first, move the camera second, and change the light last. That is production guidance, not a guarantee of model behavior; the returned video still has to be inspected.

Is the result deliverable? Inspect the seam, subject, motion, and audio

The standard is not whether the result “looks close enough,” but whether defects can be pointed out and converted into actionable revision instructions. Watch once at normal speed, inspect the seam slowly or frame by frame, and listen to the audio in a separate pass.

Dimension	What to inspect	Actionable revision wording
Subject	Face, hair, wardrobe, accessories, body proportions, hands, and held objects	“Keep the coat color and the object in the right hand unchanged.”
Motion	Direction, speed, balance, contact points, gaze, and camera trajectory	“Continue the current stride to the right without accelerating or reversing.”
Edit and image	Jump cuts, exposure, color temperature, depth of field, perspective, and background geometry at the seam	“Keep the same camera axis and focal length; do not cut.”
Audio	Ambience, music beat, dialogue, reverb, volume, clicks, and new noise	“Continue the ambience, add no dialogue, and keep the music beat unchanged.”

Check the seam for extension and both endpoints for interpolation

For extension, concentrate on roughly the last second of the source and the first second of the generated continuation, and verify whether the output altered the original tail frames. For interpolation, inspect the middle path but also confirm that the start really corresponds to the first image and the finish reaches the second image, rather than merely producing something stylistically similar.

Tighten one variable per revision

On revision, tighten one major variable at a time. Fix identity first, then motion, then camera and audio. This makes each prompt change easier to evaluate. Describe the location and behavior of the defect—“the left sleeve changes color at the seam”—instead of asking for something vague such as “more cinematic” or “smoother.”

Do not force this endpoint into these jobs

This endpoint is not a one-call editor when you need to insert a shot into the middle of a long program, continue existing dialogue, or reason across several source clips. If any limit below affects the job, split the work into shots or switch to a conventional editing workflow first.

• Uploaded clips can only be extended at the tail and must be 10 seconds or shorter.
• Additional dialogue cannot be generated from an uploaded clip in which someone is speaking; voice editing itself is unsupported.
• Audio references cannot be uploaded. Audio in a video reference is ignored.
• Video references support at most three clips of up to three seconds each. Reasoning across multiple videos is unsupported.
• System instructions, temperature, top_p, stop sequences, and a separate negative-prompt field are unsupported. Put negatives in the ordinary prompt, such as “Do not cut.”
• Every generated video includes invisible SynthID, and both inputs and outputs are subject to content-safety filtering.
• The documentation marks English prompts as fully supported; other languages have not been evaluated and may vary.

A practical fallback is to generate short shots separately, then assemble and mix them in a conventional editor. That is usually faster than repeatedly asking a tail-extension endpoint to perform mid-program insertion, dialogue continuation, or multi-clip reasoning.

Run this checklist before delivery

A successful generation call is not a quality approval. Before delivery, confirm that:

• you retained the input video, output video, and both boundary images;
• you recorded the model ID, resolution, and complete prompt;
• you inspected the extension seam frame by frame and checked whether source tail frames changed;
• you confirmed that interpolation starts at the first image and actually reaches the second;
• you listened to the audio separately for gaps, clicks, or unexpected new noise;
• you checked regional, input-duration, dialogue, and reference-media limits.

If any item is unexplained, revise the result instead of approving it from the API status alone.

Official references

• Generate and edit videos with Gemini Omni Flash
• Gemini API release notes