← Back to pipelines

Video Reframe

Auto-crop horizontal video to vertical with AI active-speaker framing. TikTok-ready in one call, frames the speaker in every shot and cuts cleanly at every boundary.

2

Still needed: Video, Target Aspect Ratio

Best for

  • •Reframing 16:9 podcast / interview / documentary footage to 9:16 TikTok/Reels/Shorts
  • •Tracking a specific subject (host, product, screen) when content has multiple focal points
  • •Producing 1:1 Instagram squares from horizontal masters with active-speaker tracking
  • •Closing the long-form-to-shorts loop when chained with video-trim and captions

When to use

  • •After video-trim, trim picks the moment, reframe picks the framing
  • •Before captions, reframe first so caption text fits the 9:16 canvas
  • •Standalone for users with already-edited horizontal clips that need vertical versions

Tips

  • ✓Supply a transcript with speaker labels for interviews and panels, it frames the active speaker far more reliably than the visible faces alone
  • ✓Shots with no clear single subject (screen-shares, wide establishing shots) letterbox by design, that is the safe choice, not a miss
  • ✓There are no pan / zoom / smoothing knobs to tune, the camera locks to one crop per shot and cuts cleanly at each boundary

Recipes using this pipeline

Articles about this pipeline

Video Reframe API

Call this pipeline from your own code. One request dispatches a run; the model is an input field, not a separate endpoint.

3 inputs
2 required
2 to 5 credits
bun add @pipe2-ai/sdk 
import { createClient } from "@pipe2-ai/sdk";

const client = createClient(process.env.PIPE2_TOKEN!);

const { run_pipeline } = await client.RunPipeline({
  pipeline_slug: "video-reframe",
  input: {
    video_url: "https://example.com/interview.mp4",
    target_aspect_ratio: "9:16",
  },
});
pip install pipe2 
import asyncio
import os

from pipe2 import Pipe2Client

client = Pipe2Client(token=os.environ["PIPE2_TOKEN"])

run = asyncio.run(client.run_pipeline(
    pipeline_slug="video-reframe",
    input={
        "video_url": "https://example.com/interview.mp4",
        "target_aspect_ratio": "9:16",
    },
))
go get github.com/pipe2-ai/sdk-go 
client := pipe2.NewClient(os.Getenv("PIPE2_TOKEN"))

input := json.RawMessage(`{
  "video_url": "https://example.com/interview.mp4",
  "target_aspect_ratio": "9:16"
}`)

run, err := pipe2.RunPipeline(context.Background(), client, "video-reframe", input)
if err != nil {
    log.Fatal(err)
}
brew install pipe2-ai/tap/pipe2 
pipe2 pipelines run \
  --pipeline video-reframe \
  --input-json '{"video_url": "https://example.com/interview.mp4", "target_aspect_ratio": "9:16"}' \
  --wait

Get an API token →

Parameters

target_aspect_ratio required

Aspect ratio to reframe to — vertical for shorts, square for feeds.

9:161:14:516:9 default 9:16
video_url required

Public URL of the video to reframe.

string
transcript_url

Public URL of a transcript. Used to keep the speaker in frame.

string

Use it from an AI agent

Point any MCP client at pipe2 and your agent gets these tools. It finds this pipeline, reads the same schema above, then runs it.

  • list_pipelines
  • get_pipeline_schema
  • run_pipeline
  • get_pipeline_run_status
  • request_upload
https://mcp.pipe2.ai/mcp 
{
  "mcpServers": {
    "pipe2ai": {
      "url": "https://mcp.pipe2.ai/mcp",
      "headers": { "Authorization": "Bearer YOUR_TOKEN" }
    }
  }
}

Full API reference MCP setup CLI reference

Video Reframe: Horizontal to Vertical with Speaker Tracking

Most viral video lives at 9:16 (TikTok, Reels, Shorts) but most footage is shot 16:9. Video Reframe closes the gap: it finds the active speaker in every shot and frames them in a clean vertical crop, held perfectly still through the shot, cutting only where the source cuts. No manual editing, no camera knobs to tune.

How It Works

  1. Detect shot boundaries: a scene-cut detector splits the video into separate shots, so every framing decision stays within one continuous shot.
  2. Find and identify the cast: the pipeline locates and groups the faces in each shot, giving every person a stable identity.
  3. Resolve who to follow per shot: audio-first: if you supply a transcript with speaker labels, it names who is talking; otherwise the visible faces decide. The speaker gets the frame; an ambiguous shot letterboxes (shows everyone) rather than guessing.
  4. Plan the camera: each shot gets one fixed crop with hard cuts at the boundaries, and very short shots are merged into their neighbours. No pans, no drift, no motion artifacts, and the same video always reframes the same way.
  5. Render: the video is cut to the chosen crop per shot (or a centered letterbox for whole-frame shots), keeping the audio intact.

Supply a transcript for multi-speaker tracking

For interviews and panels, pass a transcript (SRT) with speaker labels covering the same time range as the video. Its speaker turns drive who gets framed, so a two-shot follows whoever is actually talking, not just the largest face on screen. Without a transcript it still works, falling back to the faces it can see in each shot.

Frequently Asked Questions

What aspect ratios are supported?
9:16 (TikTok/Reels/Shorts), 1:1 (Instagram square), 4:5 (Instagram portrait), and 16:9 (passthrough, no reframing applied).
What if my source is already vertical?
The pipeline detects this and returns the source unchanged, no reframing applied.
How does it pick who to follow?
Audio-first. If you supply a transcript with speaker labels it frames the active speaker in each shot; otherwise the visible faces decide. Shots with no clear single subject, screen-shares, wide establishing shots, letterbox to show the whole frame rather than guessing.
Does it ever pan or zoom around?
No. Each shot gets one static crop held perfectly still; the camera only ever cuts at a shot boundary. This makes the jittery-pan / hunting-camera artifact class structurally impossible.
What does it cost?
2 to 5 credits, scaling with clip length: short clips sit at the low end and longer clips at the top.
Can I chain it with video-trim?
Yes, typical chain is video-trim (length) → video-reframe (aspect) → captions (burn-in). Most users will want all three for a finished short.

Explore more pipelines

See all →
Video Generator
from 9.5
Video Generator →
Image Generator
0.2-6.4
Image Generator →
Audio Generator
0.7-1.3
Audio Generator →
Music Generator
2.5-45.1
Music Generator →