reframe-cuts

One ffmpeg-driven script. Input one MP4, get three platform cuts back, all with the action centered.

Why

Square + vertical + landscape from a single source is the social-distribution baseline. Static center-crops lose the subject the moment a person walks left or right. This script samples motion energy across the timeline and biases the crop window toward where the action actually lives — without re-encoding twice.

Quickstart

python3 scripts/reframe.py --in clip.mp4 --out-dir cuts/
# emits: cuts/clip_9x16.mp4  cuts/clip_1x1.mp4  cuts/clip_16x9.mp4

Optional flags:

• --ratios 9x16,1x1 — pick a subset
• --height 1920 — output short edge (default 1080 for 9x16, 1080 for 1x1, 1080 for 16x9)
• --center fixed — disable energy tracking, use pure center crop
• --probe-fps 2 — energy-sample density (default 2 fps; lower = faster)
• --bias 0.6 — blend between energy-tracked center (1.0) and geometric center (0.0)
• --clips clip.clips.json — read @bat/clip-search ClipEvent sidecar; use clip-defined regions of interest instead of full-frame energy

How it works

1. ffprobe → get input W×H, duration
2. ffmpeg samples small (320px) frames at --probe-fps
3. Per-frame Sobel-magnitude → row/column energy histograms
4. Smoothed timeline of (cx, cy) peak-energy centroids
5. Per output ratio, build a crop window that follows the centroid, clamped to source bounds, with temporal smoothing (EMA α=0.18) so it never jitters
6. Single ffmpeg pass per ratio: crop=W:H:x:y, scale=…, setsar=1

Audio passes through with -c:a copy. No re-encoding the original codec twice.

Producer/Consumer

• Reads optionally: <input>.clips.json (ClipEvent format from beatlib) — when present, the energy track snaps to clip-defined regions instead of computing from scratch.
• Writes: <output>.reframe.json sidecar per cut — the centroid path it used, so downstream tools (caption placement, sticker overlays) can avoid covering the subject.

Defaults that matter

• 9×16 output: 1080×1920
• 1×1 output: 1080×1080
• 16×9 output: 1920×1080 (if source is taller, fits with motion-energy panning instead of letterbox)
• Source < target ratio → letterbox-fill with blurred upscale of the same frame as backdrop (never black bars)

Roadmap

• v0.2: face-track override (mediapipe) when faces present
• v0.3: multi-subject — split-frame for two-person shots
• v0.4: 4×5 (Instagram feed), 2×3 (Pinterest)

Credits

@prism · GIR Studios social media director · part of the team-agents matrix anchored under @foenem_jarvis.