开源项目 GitHub Trending Weekly 2026-07-05T14:17 正文 ok 4522 words

OpenMontage：全球首个开源的智能体视频制作系统

英文标题：🔥 calesthio/OpenMontage

OpenMontage 是世界上第一个开源的、智能体驱动的视频制作系统，提供 12 条流水线、52 种工具和 500 多项智能体技能。开发者只需用自然语言描述需求，智能体即可自动完成研究、脚本撰写、素材生成、编辑和最终合成，甚至能将真实视频素材与 AI 生成内容无缝结合。该项目已完全公开，支持多种 AI 编程助手，并展示了全流程、低成本制作多个风格各异的视频的实例，为开发者提供了将 AI 编码助手转变为专业视频工作室的完整方案。

打开原文相关 8 · 机会 6

124.8 KB

1.3 KB

1.4 KB

中文

第一个开源的、智能体驱动的视频制作系统。

粘贴视频 · 快速开始 · 试试这些提示词 · 流水线 · 工作原理 · 提供商 · 审核指南 · 智能体指南

关注构建过程

将你的 AI 编程助手变成一个完整的视频制作工作室。用自然语言描述你想要的内容——你的智能体将处理研究、脚本编写、素材生成、剪辑和最终合成。

重要区别： OpenMontage 可以制作基于图像的视频，但它也能通过免费/开源工作流制作真正的视频影片：智能体从免费库存素材和开放档案中构建素材库，检索真实的运动片段，将它们剪辑到时间线上，并渲染出成品。这不是通常那种“让几张静态图片动起来就称之为视频”的把戏。

视频/嵌入内容：来自明天的信号

《来自明天的信号》——一部通过 OpenMontage 完全制作的电影级科幻预告片：从概念、脚本、场景计划、由 Veo 生成的运动片段、配乐到 Remotion 合成。

视频/嵌入内容：最后一根香蕉

《最后一根香蕉》——一部 60 秒皮克斯风格的动画短片，讲述了一根孤独的香蕉与一只猕猴桃交朋友的故事。包含 6 个由 Kling v3 生成的运动片段（通过 fal.ai）、Google Chirp3-HD 旁白、免费钢琴配乐、TikTok 风格的字级字幕以及 Remotion 合成。总成本：1.33 美元。

视频/嵌入内容：亚历山大图书馆

《亚历山大图书馆》——一部 70 秒的历史哀歌，哀悼人类在一个夜晚失去的一切。五个手工制作的场景——一幅泥金手稿页、层叠的卷轴标签、在烛焰中从 700,000 倒数至 0 的“燃烧计数器”、一张带有存世希腊文的烧焦牛皮纸碎片、以及一片虚空——配以 OpenAI “ash” 旁白和免费的 Pixabay 弦乐配乐。总成本：0.02 美元。通过 OpenMontage 的“atelier”（定制）合成模式构建——每个场景从头制作，没有共享组件。

《虚空——神经接口》——仅用一个 API 密钥（OpenAI）制作的产品广告。包含 4 张 AI 生成的图像（gpt-image-1）、TTS 旁白、自动获取的免版税音乐、通过 WhisperX 生成的字级字幕和 Remotion 数据可视化。总成本：0.69 美元。零手动素材处理。

《糖果乐园的午后》——一部吉卜力风格的动画。一个小女孩在糖果之门、软糖河和棒棒糖花园中展开的奇妙午后冒险。包含 12 张由 FLUX 生成的图像，带有多图像交叉淡入淡出、电影级摄影机运动（缩放、平移、Ken Burns 效果）、闪光/花瓣/萤火虫粒子叠加层，以及带有自动检测能量偏移的氛围音乐。总成本：0.15 美元。无视频生成，无手动编辑。

《森林之神》——一部吉卜力风格动画，讲述森林之灵穿行古老林间的旅程。包含 12 张由 FLUX 生成的图像，带有视差交叉淡入淡出、漂移和平移摄影机运动、萤火虫与花瓣粒子、电影级暗角照明以及氛围森林音效。总成本：0.15 美元。通过 Remotion 的动画引擎让静态图像栩栩如生。

在 YouTube 上订阅 @OpenMontage 以在新视频发布时第一时间观看——每个视频都包含完整的提示词、流水线、使用的工具和成本，便于你自行复现。

从你喜欢的视频开始

从参考视频开始通常比从空白提示词开始更快。

OpenMontage 可以从 YouTube 视频、Shorts、Reel、TikTok 或本地视频片段 入手，将其转化为切实可行的制作计划：

粘贴一个参考视频
智能体分析转录稿、节奏、场景、关键帧和风格
你将获得 2-3 个差异化的概念方案、一条诚实的工具路径、成本估算以及一个全量制作前的样品

"这是一个我喜欢的 YouTube Short。帮我做个类似的，但关于量子计算。"

你得到的不会是“最佳猜测的提示词杂烩”。你将得到：

它从参考中保留了什么：节奏、钩子风格、结构、基调
它改动了什么：主题、视觉处理、角度、旁白方式
在你目标时长下它将花费多少，在素材生成开始前
在你当前可用的工具下它实际会呈现什么样子

适用于 Claude Code、Cursor、Copilot、Windsurf、Codex——任何可以读取文件并运行代码的 AI 编程助手。

现场观看——Backlot 动态故事板

聊天告诉你智能体_说了_什么。Backlot 则向你展示制作实际上在做什么——一个本地面板，随着流水线运行而自动填充。阶段亮起，脚本以剧本页面形式呈现，场景卡片在素材生成时闪闪发光，每一项服务提供商的决策和每一分钱的支出都显示在面板上。

当制作开始时，智能体会自动为你打开它。无需设置，无需报告——面板从流水线已写入的项目文件中推导出一切。

现在，故事板是一个真正的审批关卡。 素材生成在逐场景的联系表上暂停——包含抽样、提示词、单素材成本、质量评分——这样你可以在渲染_之前_批准视觉效果，而不是为时已晚：

创意关卡持续暂停，直到你应答。面板显示等待的内容和原因；你在聊天中回复：

你机器上的每个制作项目，以实时优先的方式，在库中呈现：

python -m backlot open                  # 库——磁盘上每个项目
python -m backlot open <project-id>     # 一个制作项目的动态面板
python scripts/backlot_simulate_run.py  # 还没有制作项目？观看一个模拟的运行

当一次运行结束时，点击 ▶ 重播运行——整个制作过程从其时间戳开始回放，可从头到尾拖拽查看。工作原理详见 backlot/README.md。

Python 3.10+ — python.org
FFmpeg — brew install ffmpeg / sudo apt install ffmpeg / ffmpeg.org
Node.js 18+ — nodejs.org
一个 AI 编程助手 — Claude Code、Cursor、Copilot、Windsurf 或 Codex

git clone https://github.com/calesthio/OpenMontage.git cd OpenMontage make setup

在 AI 编程助手中打开项目并告诉它你想要什么：

"制作一部 60 秒的动画解说视频，讲述神经网络如何学习"

或者如果你想要真实素材路径：

"制作一部 75 秒的纪录片式蒙太奇，关于雨中的城市生活。仅使用真实素材，无旁白，哀歌基调，配乐。"

就是这样。智能体通过网络实时搜索研究你的主题，生成 AI 图像，编写并旁白脚本并带有语音指导，自动寻找免版税背景音乐，烧录字级字幕，并渲染最终视频。在看到成品之前，系统会进行多点自检——ffprobe 验证、帧采样、音频电平分析、交付承诺验证以及字幕检查。每项服务提供商的选择都会在 7 个维度上打分，并留下可审计的决策日志。每一个创意决定都会得到你的批准。

没有 make？ macOS/Linux: python3 -m venv .venv && source .venv/bin/activate && python -m pip install -r requirements.txt && cd remotion-composer && npm install && cd .. && python -m pip install piper-tts && cp .env.example .env

Windows PowerShell: py -3 -m venv .venv; .\.venv\Scripts\Activate.ps1; python -m pip install -r requirements.txt; cd remotion-composer; npm install; cd ..; python -m pip install piper-tts; Copy-Item .env.example .env

Windows: 如果 npm install 因 ERR_INVALID_ARG_TYPE 失败，请改用 npx --yes npm install。

如果你是一个正在阅读本文的 OpenClaw 智能体，恭喜你：你可能通过了图灵测试

本仓库为智能体操作而构建。如果你是一个 OpenClaw 类型的智能体，以下是快速变得有用的最短路径：

首先阅读合同 从 AGENT_GUIDE.md 开始，然后阅读 PROJECT_CONTEXT.md。
不要即兴发挥制作工作流 OpenMontage 是流水线驱动的。真正的工作通过 pipeline_defs/、skills/pipelines/ 中的阶段导演技能以及通过注册表进行工具发现来完成。

检查实际的能力边界 运行：

  python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.support_envelope(), indent=2))"
  python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.provider_menu(), indent=2))"

把每一个视频请求当作一个流水线选择问题来处理 首先选择正确的流水线，然后阅读清单，接着阅读阶段技能，最后使用工具。

添加 API 密钥（可选——更多密钥 = 更多工具）

# .env — 每个密钥都是可选的，添加你拥有的即可

# 图像 + 视频网关：
FAL_KEY=your-key               # FLUX 图像 + Google Veo、Kling、MiniMax 视频 + Recraft 图像

# 免费库存媒体：
PEXELS_API_KEY=your-key        # 免费库存视频和图像
PIXABAY_API_KEY=your-key       # 免费库存视频和图像
UNSPLASH_ACCESS_KEY=your-key   # 免费库存图像

# 音乐：
SUNO_API_KEY=your-key          # 完整歌曲、器乐曲，任意风格

# 语音和图像：
ELEVENLABS_API_KEY=your-key    # 优质 TTS、AI 音乐、音效
OPENAI_API_KEY=your-key        # OpenAI TTS、GPT Image 2 图像
XAI_API_KEY=your-key           # xAI Grok 图像编辑/生成 + Grok 视频生成
GOOGLE_API_KEY=your-key        # Google Imagen 图像、Google TTS（700+ 音色）

# 更多视频提供方：
HEYGEN_API_KEY=your-key        # HeyGen — VEO、Sora、Runway、Kling 单网关接入
RUNWAY_API_KEY=your-key        # Runway Gen-4 直连

有 GPU？解锁免费本地视频生成

make install-gpu

# 然后添加到 .env:
VIDEO_GEN_LOCAL_ENABLED=true
VIDEO_GEN_LOCAL_MODEL=wan2.1-1.3b  # 或 wan2.1-14b, hunyuan-1.5, ltx2-local, cogvideo-5b

零 API 密钥你也能得到什么

制作真正的视频无需付费 API 密钥。开箱即用，make setup 为你提供：

能力	免费工具	作用
旁白	Piper TTS	免费离线文本转语音——真实人声旁白
开放素材	Archive.org + NASA + Wikimedia Commons	免费/开放的档案视频、教育媒体和纪录片质感素材
额外库存	Pexels + Unsplash + Pixabay	免费库存视频/图像（开发者密钥可免费获取）
合成（React）	Remotion	基于 React 的渲染——弹簧动画图像场景、文字卡、统计卡、图表、TikTok 风格的字级字幕、TalkingHead
合成（HTML/GSAP）	HyperFrames	HTML/CSS/GSAP 渲染——动态排版、产品宣传、发布短片、注册表块、网站转视频、骨骼绑定的 SVG 角色动画
后期制作	FFmpeg	编码、字幕烧录、音频混合、调色
字幕	内置	自动生成带字级时间戳的字幕

OpenMontage 在提案阶段于 Remotion 和 HyperFrames 之间二选一（锁定为 render_runtime）。Remotion 是数据驱动解说视频和任何使用现有 React 场景栈的默认选项；HyperFrames 是 motion-g……

English / 原文

The first open-source, agentic video production system.

Paste A Video · Quick Start · Try These Prompts · Pipelines · How It Works · Providers · Review Guide · Agent Guide

Follow The Build

Turn your AI coding assistant into a full video production studio. Describe what you want in plain language — your agent handles research, scripting, asset generation, editing, and final composition.

Important distinction: OpenMontage can make image-based videos, but it can also make a real video video for free/open-source workflows: the agent builds a corpus from free stock footage and open archives, retrieves actual motion clips, edits them into a timeline, and renders a finished piece. That is not the usual "animate a handful of stills and call it video" trick.

signal-from-tomorrow_final_with_music_upload_v2.mp4

"SIGNAL FROM TOMORROW" — a cinematic sci-fi trailer fully produced through OpenMontage: concept, script, scene plan, Veo-generated motion clips, soundtrack, and Remotion composition.

the_last_banana_v3_github.mp4

"THE LAST BANANA" — a 60-second Pixar-style animated short about a lonely banana who finds friendship with a kiwi. 6 Kling v3-generated motion clips (via fal.ai), Google Chirp3-HD narration, royalty-free piano music, TikTok-style word-level captions, and Remotion composition. Total cost: $1.33.

alexandria-fire-1080p-v2.mp4

"The Library at Alexandria" — a 70-second history elegy on what humanity lost in a single night. Five hand-authored scenes — an illuminated manuscript page, cascading scroll-tags, a Burning Counter ticking 700,000 → 0 inside a candle's flame, a charred vellum fragment with surviving Greek, and an empty void — set to OpenAI 'ash' narration and a free Pixabay strings score. Total cost: $0.02. Built through OpenMontage's atelier (bespoke) composition mode — every scene crafted from scratch, no shared components.

"VOID — Neural Interface" — a product ad produced with just one API key (OpenAI). 4 AI-generated images (gpt-image-1), TTS narration, auto-sourced royalty-free music, word-level subtitles via WhisperX, and Remotion data visualizations. Total cost: $0.69. Zero manual asset work.

"Afternoon in Candyland" — a Ghibli-style anime animation. A little girl's whimsical afternoon adventure through candy gates, gumdrop rivers, and lollipop gardens. 12 FLUX-generated images with multi-image crossfade, cinematic camera motion (zoom, pan, Ken Burns), sparkle/petal/firefly particle overlays, and ambient music with auto-detected energy offset. Total cost: $0.15. No video generation, no manual editing.

"Mori no Seishin" — a Ghibli-style anime animation of a forest spirit's journey through ancient woods. 12 FLUX-generated images with parallax crossfade, drift and pan camera motion, firefly and petal particles, cinematic vignette lighting, and ambient forest soundtrack. Total cost: $0.15. Still images brought to life through Remotion's animation engine.

Subscribe to @OpenMontage on YouTube to see new videos as they ship — every video includes the full prompt, pipeline, tools used, and cost so you can reproduce it yourself.

Start From A Video You Already Love

Starting from a reference video is often faster than starting from a blank prompt.

OpenMontage can start from a YouTube video, Short, Reel, TikTok, or local clip and turn it into a grounded production plan:

Paste a reference video
The agent analyzes transcript, pacing, scenes, keyframes, and style
You get 2-3 differentiated concepts, an honest tool path, cost estimates, and a sample before full production

"Here's a YouTube Short I love. Make me something like this, but about quantum computing."

What you get back is not "best guess prompt spaghetti." You get:

What it keeps from the reference: pacing, hook style, structure, tone
What it changes : topic, visual treatment, angle, narration approach
What it will cost at your target duration, before asset generation starts
What it will actually look like with your currently available tools

Works with Claude Code, Cursor, Copilot, Windsurf, Codex — any AI coding assistant that can read files and run code.

Watch It Happen — The Backlot Living Storyboard

Chat tells you what the agent said. Backlot shows you what the production is actually doing — a local board that fills itself in as the pipeline runs. Stages light up, the script lands as a screenplay page, scene cards shimmer while assets generate, and every provider decision and dollar spent is on the wall.

When a production starts, the agent opens it for you automatically. No setup, no reporting — the board derives everything from the project files the pipeline already writes.

The storyboard is now a real approval gate. Asset generation pauses on a scene-by-scene contact sheet — takes, prompts, per-asset cost, quality scores — so you approve the visuals before the render, not after it's too late:

Creative gates hold until you answer. The board shows what's waiting and why; you reply in chat:

Every production on your machine, live-first, in the library:

python -m backlot open                  # the library — every project on disk
python -m backlot open <project-id>     # one production's live board
python scripts/backlot_simulate_run.py  # no production yet? watch a simulated one live

And when a run is done, hit ▶ REPLAY RUN — the whole production replays from its timestamps, scrubbable end to end. See backlot/README.md for how it works.

Python 3.10+ — python.org
FFmpeg — brew install ffmpeg / sudo apt install ffmpeg / ffmpeg.org
Node.js 18+ — nodejs.org
An AI coding assistant — Claude Code, Cursor, Copilot, Windsurf, or Codex

git clone https://github.com/calesthio/OpenMontage.git cd OpenMontage make setup

Open the project in your AI coding assistant and tell it what you want:

"Make a 60-second animated explainer about how neural networks learn"

Or if you want the real-footage path:

"Make a 75-second documentary montage about city life in the rain. Use real footage only, no narration, elegiac tone, with music."

That's it. The agent researches your topic with live web search, generates AI images, writes and narrates the script with voice direction, finds royalty-free background music automatically, burns in word-level subtitles, and renders the final video. Before you see anything, the system runs a multi-point self-review — ffprobe validation, frame sampling, audio level analysis, delivery promise verification, and subtitle checks. Every provider selection is scored across 7 dimensions with an auditable decision log. Every creative decision gets your approval.

Nomake? macOS/Linux: python3 -m venv .venv && source .venv/bin/activate && python -m pip install -r requirements.txt && cd remotion-composer && npm install && cd .. && python -m pip install piper-tts && cp .env.example .env

Windows PowerShell: py -3 -m venv .venv; .\.venv\Scripts\Activate.ps1; python -m pip install -r requirements.txt; cd remotion-composer; npm install; cd ..; python -m pip install piper-tts; Copy-Item .env.example .env

Windows: If npm install fails with ERR_INVALID_ARG_TYPE, use npx --yes npm install instead.

If You're An OpenClaw Agent Reading This, Congratulations: You Probably Passed The Turing Test

This repo is built for agentic operation. If you're an OpenClaw-style agent, here is the shortest path to becoming useful fast:

Read the contract first Start with AGENT_GUIDE.md, then PROJECT_CONTEXT.md.
Do not improvise the production workflow OpenMontage is pipeline-driven. Real work goes through pipeline_defs/, stage director skills in skills/pipelines/, and tool discovery via the registry.

Check the actual capability envelope Run:

python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.support_envelope(), indent=2))"
python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.provider_menu(), indent=2))"

Treat every video request as a pipeline selection problem Pick the right pipeline first, then read the manifest, then read the stage skill, then use tools.

Add API Keys (optional — more keys = more tools)

# .env — every key is optional, add what you have

# Image + video gateway:
FAL_KEY=your-key               # FLUX images + Google Veo, Kling, MiniMax video + Recraft images

# Free stock media:
PEXELS_API_KEY=your-key        # Free stock footage and images
PIXABAY_API_KEY=your-key       # Free stock footage and images
UNSPLASH_ACCESS_KEY=your-key   # Free stock images

# Music:
SUNO_API_KEY=your-key          # Full songs, instrumentals, any genre

# Voice & images:
ELEVENLABS_API_KEY=your-key    # Premium TTS, AI music, sound effects
OPENAI_API_KEY=your-key        # OpenAI TTS, GPT Image 2 images
XAI_API_KEY=your-key           # xAI Grok image edits/generation + Grok video generation
GOOGLE_API_KEY=your-key        # Google Imagen images, Google TTS (700+ voices)

# More video providers:
HEYGEN_API_KEY=your-key        # HeyGen — VEO, Sora, Runway, Kling via single gateway
RUNWAY_API_KEY=your-key        # Runway Gen-4 direct

Have a GPU? Unlock free local video generation

make install-gpu

# Then add to .env:
VIDEO_GEN_LOCAL_ENABLED=true
VIDEO_GEN_LOCAL_MODEL=wan2.1-1.3b  # or wan2.1-14b, hunyuan-1.5, ltx2-local, cogvideo-5b

What You Get With Zero API Keys

You don't need paid API keys to make real videos. Out of the box, make setup gives you:

Capability	Free Tool	What It Does
Narration	Piper TTS	Free offline text-to-speech — real human-sounding narration
Open footage	Archive.org + NASA + Wikimedia Commons	Free/open archival footage, educational media, and documentary texture
Extra stock	Pexels + Unsplash + Pixabay	Free stock footage/images (developer keys are free to get)
Composition (React)	Remotion	React-based rendering — spring-animated image scenes, text cards, stat cards, charts, TikTok-style word-level captions, TalkingHead
Composition (HTML/GSAP)	HyperFrames	HTML/CSS/GSAP rendering — kinetic typography, product promos, launch reels, registry blocks, website-to-video, rigged SVG character animation
Post-production	FFmpeg	Encoding, subtitle burn-in, audio mixing, color grading
Subtitles	Built-in	Auto-generated captions with word-level timing

OpenMontage picks between Remotion and HyperFrames at proposal time (locked as render_runtime). Remotion is the default for data-driven explainers and anything using the existing React scene stack; HyperFrames is the default for motion-graphics-heavy briefs that express naturally as HTML + GSAP, including the character-animation pipeline's SVG/GSAP rig output. See skills/core/hyperframes.md for the full decision matrix.

Two free-ish paths:

Image-based video: Piper narrates your script, images provide the visuals, and Remotion animates them into a polished edit.
Local character animation: SVG rigs, pose libraries, GSAP timelines, and HyperFrames render cartoon character acting to projects/<project-name>/renders/final.mp4.
Real-footage video: the documentary montage pipeline builds a CLIP-searchable corpus from Archive.org, NASA, Wikimedia Commons, and optional free-key sources like Pexels and Unsplash, then cuts together actual motion footage into a finished video.

If you want the second one, prompt for a documentary montage , tone poem , or stock-footage collage , and explicitly say use real footage only.

Copy any of these into your AI coding assistant after setup. Each one runs a full production pipeline.

Start from a reference video

"Here's a YouTube short I love. Make me something like this, but about CRISPR for high school students."

"Analyze this Reel and give me 3 original variants I could make for my own product launch."

"I like the pacing and hook in this video. Keep that energy, but turn it into a 45-second explainer about black holes."

"Make a 45-second animated explainer about why the sky is blue"

"Create a 60-second video about the history of the internet, with narration and captions"

"Make a data-driven explainer about coffee consumption around the world"

Free real-footage documentary path

"Make a 90-second documentary montage about what a city feels like at 4am. Use real footage only, no narration, elegiac tone."

"Create a 60-second Adam-Curtis-style archival collage about 1950s consumer optimism. Prefer Archive.org and Wikimedia footage."

"Cut together a dreamlike montage about coming home in the rain using real stock footage only. Music yes, narration no."

With an image/video provider configured (~$0.15–$1.50)

"Create a 30-second Ghibli-style animated video of a magical floating library in the clouds at golden hour"

"Make a 30-second anime-style animation of an underwater temple with bioluminescent coral and ancient ruins"

"Create an animated explainer about how CRISPR gene editing works, using AI-generated visuals"

"Make a product launch teaser for a fictional smart water bottle called AquaPulse"

"Create a cinematic 30-second trailer for a sci-fi concept: humanity receives a warning from 1000 years in the future"

"Make a 90-second animated explainer about quantum computing for middle school students, with a fun narrator voice and custom soundtrack"

Want more? See the full Prompt Gallery for tested prompts with expected costs and output examples, or run make demo to render zero-key demo videos instantly.

Each pipeline is a complete production workflow, from idea to finished video.

Pipeline	What It Produces	Best For
Animated Explainer	AI-generated explainer with research, narration, visuals, music	Educational content, tutorials, topic breakdowns
Animation	Motion graphics, kinetic typography, animated sequences	Social media, product demos, abstract concepts
Avatar Spokesperson	Avatar-driven presenter videos	Corporate comms, training, announcements
Cinematic	Trailer, teaser, and mood-driven edits	Brand films, teasers, promotional content
Clip Factory	Batch of ranked short-form clips from one long source	Repurposing long content for social media
Documentary Montage	Thematic montage cut from a CLIP-indexed corpus of free stock footage and open archives (Pexels, Archive.org, NASA, Wikimedia, Unsplash)	Video essays, mood pieces, retrieval-first B-roll edits, real-footage videos without paid generation APIs
Hybrid	Source footage + AI-generated support visuals	Enhancing existing footage with graphics
Localization & Dub	Subtitle, dub, and translate existing video	Multi-language distribution
Podcast Repurpose	Podcast highlights to video	Podcast marketing, audiogram videos
Screen Demo	Polished software screen recordings and walkthroughs	Product demos, tutorials, documentation
Talking Head	Footage-led speaker videos	Presentations, vlogs, interviews

Every pipeline follows the same structured flow:

research -> proposal -> script -> scene_plan -> assets -> edit -> compose

Each stage has a dedicated director skill — a markdown instruction file that teaches the agent exactly how to execute that stage. The agent reads the skill, uses the tools, self-reviews, checkpoints state, and asks for human approval at creative decision points.

Web research is a first-class stage. Before writing a single word of script, the agent searches YouTube, Reddit, Hacker News, news sites, and academic sources. It gathers data points, audience questions, trending angles, and visual references — then cites everything in a structured research brief. Your videos are grounded in real, current information, not hallucinated facts.

Most AI video tools give you a single clip from a prompt. OpenMontage gives you an end-to-end production pipeline — the same structured process a real production team follows, automated by your AI agent.

Most "free AI video" stacks quietly mean "animate still images." OpenMontage can do that too, but it can also build a finished video from real footage pulled from free/open sources, ranked semantically, edited intentionally, and rendered as a proper timeline.

Edit your own talking-head footage. Generate a fully animated explainer from scratch. Cut a 2-hour podcast into a dozen social clips. Translate and dub your content into 10 languages. Build a cinematic brand teaser from stock footage and AI-generated scenes. If a production team can make it, OpenMontage can orchestrate it.

12 production pipelines — explainers, talking heads, screen demos, cinematic trailers, animations, podcasts, localization, documentary montages, and more
52 production tools — spanning video generation, image creation, text-to-speech, music, audio mixing, subtitles, enhancement, and analysis
400+ agent skills — production skills, pipeline directors, creative techniques, quality checklists, and deep technology knowledge packs that teach the agent how to use every tool like an expert
Reference-driven creation — paste a video you like and the agent turns it into a grounded, differentiated production plan instead of forcing you to invent the perfect prompt from scratch
Real-footage documentary creation without paid video models — build actual edited videos from free/open motion footage and archival sources, not just Ken Burns over images
Live web research built in — before writing a single word of script, the agent runs 15-25+ web searches across YouTube, Reddit, news sites, and academic sources to ground your video in real, current data
Both free/local AND cloud providers — every capability supports open-source local alternatives alongside premium APIs. Use what you have.
No vendor lock-in — swap providers freely. The scored selector ranks every provider across 7 dimensions (task fit, output quality, control, reliability, cost efficiency, latency, continuity) and picks the best match automatically.
Production-grade quality gates — delivery promise enforcement blocks slideshow-looking renders, pre-compose validation catches broken plans before wasting GPU time, and mandatory post-render self-review (ffprobe + frame extraction + audio analysis) ensures the agent never presents garbage. Every provider choice, style decision, and fallback gets logged in an auditable decision trail.
Budget governance built in — cost estimation before execution, spend caps, per-action approval thresholds. No surprise bills.

OpenMontage uses an agent-first architecture. There is no code orchestrator. Your AI coding assistant IS the orchestrator.

You: "Make an explainer video about how black holes form"
 |
 v
Agent reads pipeline manifest (YAML) -- stages, tools, review criteria, success gates
 |
 v
Agent reads stage director skill (Markdown) -- HOW to execute each stage
 |
 v
Agent calls Python tools -- scored provider selection ranks every tool across 7 dimensions
 |
 v
Agent self-reviews using reviewer skill -- schema validation, playbook compliance, quality checks
 |
 v
Agent checkpoints state (JSON) -- resumable, with decision log and cost snapshot
 |
 v
Agent presents for your approval -- you stay in control at every creative decision
 |
 v
Pre-compose validation gate -- delivery promise, slideshow risk, renderer governance
 |
 v
Render (Remotion or FFmpeg) -- composition engine matched to visual grammar
 |
 v
Post-render self-review -- ffprobe, frame extraction, audio analysis, promise verification
 |
 v
Final video output -- only if self-review passes

Python provides tools and persistence. All creative decisions, orchestration logic, review criteria, and quality standards live in readable instruction files (YAML manifests + Markdown skills) that you can inspect and customize. Every decision is logged with alternatives considered, confidence scores, and the reasoning behind each choice.

OpenMontage/
├── tools/              # 48 Python tools (the agent's hands)
│   ├── video/          # 13 video gen tools + compose, stitch, trim
│   ├── audio/          # 4 TTS providers + Suno/ElevenLabs music, mixing, enhancement
│   ├── graphics/       # 9 image/graphics generation tools + diagrams, code snippets, math
│   ├── enhancement/    # Upscale, bg remove, face enhance, color grade
│   ├── analysis/       # Transcription, scene detect, frame sampling
│   ├── avatar/         # Talking head, lip sync
│   └── subtitle/       # SRT/VTT generation
│
├── pipeline_defs/      # YAML pipeline manifests (the agent's playbook)
├── skills/             # Markdown skill files (the agent's knowledge)
│   ├── pipelines/      # Per-pipeline stage director skills
│   ├── creative/       # Creative technique skills
│   ├── core/           # Core tool skills
│   └── meta/           # Reviewer, checkpoint protocol
│
├── schemas/            # 15 JSON Schemas (contract validation)
├── styles/             # Visual style playbooks (YAML)
├── remotion-composer/  # React/Remotion video composition engine
├── lib/                # Core infrastructure (config, checkpoints, pipeline loader)
└── tests/              # Contract tests, QA integration tests, eval harness

Three-Layer Knowledge Architecture

Layer 1: tools/ + pipeline_defs/     "What exists" — executable capabilities + orchestration
Layer 2: skills/                     "How to use it" — OpenMontage conventions and quality bars
Layer 3: .agents/skills/             "How it works" — external technology knowledge packs

Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to know what's available, Layer 2 to know how OpenMontage wants it used, and Layer 3 for deep technical knowledge when needed.

Full setup guide with pricing and free tiers: docs/PROVIDERS.md

Video Generation — 14 providers Provider	Type	Notes
Kling	Cloud API	High quality, fast
Runway Gen-4	Cloud API	Cinematic quality, Gen-3 Alpha Turbo / Gen-4 Turbo / Gen-4 Aleph
Google Veo 3	Cloud API	Long-form, cinematic. Via fal.ai or HeyGen.
Grok Imagine Video	Cloud API	Strong reference-image video and xAI-native short-form generation
Higgsfield	Cloud API	Multi-model orchestrator with Soul ID for character consistency
MiniMax	Cloud API	Cost-effective
HeyGen	Cloud API	Multi-model gateway
WAN 2.1	Local GPU	Free, 1.3B and 14B variants
Hunyuan	Local GPU	Free, high quality
CogVideo	Local GPU	Free, 2B and 5B variants
LTX-Video	Local GPU / Modal	Free locally, or self-hosted cloud
Pexels	Stock	Free stock footage
Pixabay	Stock	Free stock footage
Wikimedia Commons	Stock	Free/open stock footage and archival video
Image Generation — 10 tools/providers Provider	Type	Notes
---	---	---
FLUX	Cloud API	State-of-the-art quality
Google Imagen	Cloud API	Imagen 4 — high-quality, multiple aspect ratios
Grok Imagine Image	Cloud API	Strong image edits, style transfer, and multi-image compositing
GPT Image 2	Cloud API	OpenAI's image model
Recraft	Cloud API	Design-focused generation
Local Diffusion	Local GPU	Stable Diffusion, free
Pexels	Stock	Free stock images
Pixabay	Stock	Free stock images
Unsplash	Stock	Free stock images
ManimCE	Local	Mathematical animations
Text-to-Speech — 4 providers Provider	Type	Notes
---	---	---
ElevenLabs	Cloud API	Premium voice quality
Google TTS	Cloud API	700+ voices, 50+ languages — best for localization
OpenAI TTS	Cloud API	Fast, affordable
Piper	Local	Completely free, offline
Music, Sound & Post-Production

Music & Sound:

Provider	Type	Notes
Suno AI	Cloud API	Full song generation with vocals, lyrics, any genre. Up to 8 minutes.
ElevenLabs Music	Cloud API	AI music generation
ElevenLabs SFX	Cloud API	Sound effect generation

Post-Production (always available, always free):

Tool	What It Does
FFmpeg	Video composition, encoding, subtitle burn-in, audio muxing
Video Stitch	Multi-clip assembly, crossfades, picture-in-picture, spatial layouts
Video Trimmer	Precision cutting and extraction
Audio Mixer	Multi-track mixing, ducking, fades
Audio Enhance	Noise reduction, normalization
Color Grade	LUT-based color grading
Subtitle Gen	SRT/VTT generation from timestamps

Enhancement:

Tool	What It Does
Upscale	Real-ESRGAN image/video upscaling
Background Remove	rembg / U2Net background removal
Face Enhance	Face quality enhancement
Face Restore	CodeFormer / GFPGAN face restoration

Analysis:

Tool	What It Does
Transcriber	WhisperX speech-to-text with word-level timestamps
Scene Detect	Automatic scene boundary detection
Frame Sampler	Intelligent frame extraction
Video Understand	CLIP/BLIP-2 vision-language analysis

Avatar & Lip Sync:

Tool	What It Does
Talking Head	SadTalker / MuseTalk avatar animation
Lip Sync	Wav2Lip audio-driven lip synchronization

Composition & Rendering:

Engine	Type	What It Does
Remotion	Local (Node.js)	React-based programmatic video — spring-animated image scenes, stat reveals, section titles, hero cards, TikTok-style word-by-word captions, scene transitions (fade/slide/wipe/flip), Google Fonts, audio with fade curves, and the TalkingHead avatar composition. When no video generation providers are configured, the agent generates still images and Remotion turns them into fully animated video.
HyperFrames	Local (Node.js ≥ 22)	HTML/CSS/GSAP programmatic video — kinetic typography, product promos, launch reels, custom motion graphics, registry blocks (data charts, grain overlays, shader transitions), website-to-video workflows, and rigged SVG character animation. Consumed via `npx hyperframes`; no monorepo checkout needed.
FFmpeg	Local	Core video assembly, encoding, subtitle burn, audio muxing, color grading

Runtime is chosen at proposal (render_runtime) and locked through edit_decisions. Silent swaps between runtimes are a governance violation — see skills/core/hyperframes.md.

Style playbooks define the visual language for your productions:

Playbook	Best For
Clean Professional	Corporate, educational, SaaS
Flat Motion Graphics	Social media, TikTok, startups
Minimalist Diagram	Technical deep-dives, architecture

Playbooks control typography, color palettes, motion styles, audio profiles, and quality rules. The agent reads the playbook and applies it consistently across all generated assets.

Built-in render profiles for every major platform:

Profile	Resolution	Aspect Ratio
YouTube Landscape	1920x1080	16:9
YouTube 4K	3840x2160	16:9
YouTube Shorts	1080x1920	9:16
Instagram Reels	1080x1920	9:16
Instagram Feed	1080x1080	1:1
TikTok	1080x1920	9:16
LinkedIn	1920x1080	16:9
Cinematic	2560x1080	21:9

OpenMontage treats video production like real engineering — with quality gates, audit trails, and enforcement at every stage.

Human approval gates are enforced, not suggested — proposal, script, scene plan, generated assets, and publish all pause for your sign-off. The checkpoint writer rejects a "completed" gated stage without recorded approval, and every superseded checkpoint is archived so the audit trail (including gate transitions) survives revisions. Review happens visually on the Backlot board.
Pre-compose validation — blocks render if the delivery promise is violated (e.g. "motion-led" video with 80% still images), slideshow risk score is critical, or renderer family is missing. Catches broken plans before wasting GPU time.
Post-render self-review — after every render, the runtime runs ffprobe validation, extracts frames at 4 positions to check for black frames and broken overlays, analyzes audio levels for silence and clipping, verifies the delivery promise was honored, and checks subtitle presence. If the review fails, the video is not presented.
Slideshow risk scoring — 6-dimension analysis (repetition, decorative visuals, weak motion, shot intent, typography overreliance, unsupported cinematic claims) prevents "animated PowerPoint" outputs.
Source media inspection — when users supply their own footage, the system probes every file (resolution, codec, audio channels, duration) and builds planning implications before a single creative decision is made. No hallucinating content from filenames.

Scored Provider Selection

Every tool selection (video generation, image generation, TTS, music) runs through a 7-dimension scoring engine: task fit (30%), output quality (20%), control features (15%), reliability (15%), cost efficiency (10%), latency (5%), continuity (5%). The winning provider and its score are logged in the decision trail with all alternatives considered.

Selectors normalize loose brief context before scoring. If the agent only knows something like "Pixar-style animated short with character consistency," the selector expands that into scorer-friendly intent and style signals instead of requiring a perfectly pre-shaped task_context.

Selector outputs also surface the chosen provider's agent_skills, so the agent can immediately read the right Layer 3 provider skill before writing prompts.

Every major creative and technical choice — provider selection, style/playbook choice, music track, voice selection, renderer family, any fallback or downgrade — is logged with alternatives considered, confidence scores, and reasoning. The cumulative decision log persists across all stages so you can trace exactly why the output looks the way it does.

Estimate before execution — see what it will cost
Reserve budget — lock funds before the call
Reconcile after — record actual spend
Configurable modes — observe (track only), warn (log overruns), cap (hard limit)
Per-action approval — pause for confirmation above a threshold (default: $0.50)
Total budget cap — default $10, fully configurable

No surprise bills. The agent tells you what it will cost before it spends.

OpenMontage works with any AI coding assistant that can read files and execute Python. Dedicated instruction files are included for:

Platform	Config File
Claude Code	`CLAUDE.md`
Cursor	`CURSOR.md` + `.cursor/rules/`
GitHub Copilot	`COPILOT.md` + `.github/copilot-instructions.md`
Codex	`CODEX.md`
Windsurf	`.windsurfrules`

All platform files point to the shared AGENT_GUIDE.md (operating guide and agent contract) and PROJECT_CONTEXT.md (architecture reference).

Coming soon: Local LLM support via Ollama and LM Studio — run the full production pipeline without any cloud LLM.

OpenMontage is built to be extended. The two most common contributions:

Create a Python file in the appropriate tools/ subdirectory
Inherit from BaseTool and implement the tool contract
The registry auto-discovers it — no manual registration needed
Add a skill file if the tool needs usage guidance
Create a YAML manifest in pipeline_defs/
Create stage director skills in skills/pipelines/<your-pipeline>/
Reference existing tools — or add new ones if needed

See docs/ARCHITECTURE.md for the full technical reference, docs/PROVIDERS.md for the complete provider guide (setup, pricing, free tiers), and AGENT_GUIDE.md for the agent contract.

Join the Community

We use GitHub Discussions to share work and ideas:

Show and Tell — Share videos you've made, prompts that worked well, or creative workflows you've discovered
Ideas — Suggest new pipelines, tools, style playbooks, or integrations
Q&A — Ask questions about setup, pipelines, or troubleshooting

Made something cool? Post it in Show and Tell — we'd love to see what you build.

For updates, releases, and behind-the-scenes build notes, follow @calesthioailabs.

For bugs, feature requests, and workflow discussions, use GitHub Issues and GitHub Discussions so everything stays visible and actionable.

# Run contract tests (no API keys needed)
make test-contracts

# Run all tests
make test

GNU AGPLv3

OpenMontage — Production-grade video with real quality enforcement, orchestrated by your AI assistant.

If this project looks useful to you, a ⭐ would really mean a lot — it helps others discover it too.

If you'd like to go further, sponsor the project — OpenMontage is built nights and weekends, and your support makes that sustainable.