An open platform for agentic video intelligence applications.
VideoIndex indexes hours of video once, into speech, on-screen text, frames and scenes at a shared timebase, and
gives your agents tools to search it, look at it and answer with the exact moment cited. A Rust core,
agnostic to the vision and language models you run it with.
22×real time indexing on one A100, speech, on-screen text and frame embeddings included
93%answer quality on library-wide questions, level with Gemini agentic video at 2.6× lower cost
84%on LVBench, ahead of Gemini agentic video (80.8%); 78.1% against 76.5% on MINERVA
[demo]
The whole idea
Watch the demo 1:34A narrated walkthrough: the SDK, one library-wide question over 36 hours of talks, and how VideoIndex compares with Gemini agentic video on LVBench, MINERVA, cost and latency.
The demo is one application. The platform underneath is what you build on: a persistent index, agent tools
over it, your choice of models, and every integration surface an agent framework expects.
Index once
A Rust pipeline decodes video, runs speech recognition, OCR, shot detection and frame embeddings, and stores everything with timestamps at one timebase. An optional fine pass adds scene descriptions and entity and event extraction. The index is a directory you keep.
Agents that cite
Search, view and describe tools let an agent read transcripts and on-screen text, look at frames when text does not settle it, and answer with hh:mm:ss citations. Retrieval-only and custom policies when you need cheaper or deterministic behaviour.
Any VLM, any LLM
Model roles, not model names: speech, OCR, embeddings, description, extraction and the agent are each bound to a provider you choose. OpenAI-compatible servers, Anthropic, Gemini or local ONNX models, swapped in one config file, priced and rate-limited per role.
Integrate anywhere
A library from Python or Node, an HTTP API with streaming answers, or an MCP server that Claude Code, LangGraph and other agent frameworks call as a tool. One binary, one index directory, no services to run for the local case.
[results]
Where it stands
VideoIndex against Gemini 3.8 Flash agentic video, with the same questions, the same videos and the same scoring
for both: two public long-video benchmarks, then questions that span a whole library.
Public long-video benchmarks, runs of 2026-10-04: VideoIndex answers more questions correctly than Gemini's
agentic video mode on both, 84.1% against 80.8% on LVBench and 78.1% against 76.5% on MINERVA.
Questions that need the whole library, 2026-09-17: 28 questions over the 30-talk demo library. VideoIndex matches
Gemini's agentic mode on answer quality (93% each) and finds every expected video (recall 100% against 97%), at 2.6×
lower cost and 1.6× lower median latency (under a third at the 95th percentile), because the index answers a
library-wide scan with one query where Gemini reads most of the library on every question. The
results page has the per-question table, the judge protocol and the run files.
The docs carry the results pages: intervals, per-type tables and
the reading of each run.
[roadmap]
Where it is going
Real-time understanding of live video streams
The same index, built continuously from a live stream, so interactive agentic AI applications can ask about what is happening now and what happened a minute ago, with the same citations. Follow progress on our Discord.
Demo
The reference app built on the platform: a chat over 36 hours of AI Engineer talks and Berkeley lectures. Sign in and click any timestamp.
Watch the demo 1:34
A narrated walkthrough: the SDK, one library-wide question over 36 hours of talks, and how VideoIndex compares with Gemini agentic video on LVBench, MINERVA, cost and latency.