Extracting Structured Data
Turn any online video or uploaded file into clean, validated JSON using a JSON Schema you define, or from a plain-words description of what you need.
If transcripts are for humans, extraction is for software. The Extract API lets you define the exact fields you want as a standard JSON Schema (or describe them in plain words), then returns structured JSON that matches your schema so you can send the result straight into a database, CRM, workflow, or AI pipeline.
What This Guide Helps You Do
By the end of this guide, you will know how to:
- choose the right extraction endpoint
- design a schema the model can follow reliably, or reuse a Pydantic model or Zod schema you already have
- use
what_to_extractto improve focus, or on its own when you have no schema yet - understand auto-transcription, caching, and billing
- move from one test video to a repeatable production workflow
Why This Is More Than Just an LLM Prompt
The real challenge is not only extracting structured data. It is getting the video content into a usable text layer in the first place.
With VidNavigator, the workflow is end to end:
- fetch the transcript when the platform already exposes one
- auto-transcribe videos with speech-to-text when no transcript exists, YouTube included
- run AI extraction on that text using your schema
- return validated JSON you can use immediately in software
That is the main difference. You are not expected to manually retrieve captions, run a separate speech-to-text step, clean the text, and then prompt a model yourself.
Compared with building that flow manually, VidNavigator gives you:
- transcript retrieval and speech-to-text built into the same extraction workflow
- one API call instead of stitching together multiple tools
- a fixed response shape defined by your schema
- validated output instead of free-form text
- prompt compilation that is cached for 2 hours for repeated schemas
This is what makes video data extraction practical at scale, especially for platforms where transcripts are inconsistent, missing, or not easily accessible.
Choose the Right Endpoint
Online Videos
Use /v1/extract/video when you have a public video URL from YouTube, Instagram, TikTok, Facebook, X, Vimeo, Dailymotion, Loom, Snapchat, and similar platforms.
Uploaded Files
Use /v1/extract/file when the content is already in your VidNavigator library and you want to extract from a file_id.
extract/video can auto-transcribe videos that have no transcript, YouTube included, when transcribe=true (the default). extract/file does not transcribe for you, so the file must already have a transcript.
Long videos: when speech-to-text is needed, the synchronous /v1/extract/video accepts media up to 10 minutes and rejects longer media with 400 video_too_long. Use /v1/extract/video/async — same body, no duration cap, result by polling or webhook. See Async Jobs.
How Extraction Works
The Extract API runs a simple 2-step pipeline:
- You send a schema, a
what_to_extractinstruction, or both. Withwhat_to_extractalone, the schema is written from it first. - VidNavigator compiles an optimized extraction plan and caches it for 2 hours.
- The plan is applied to the transcript for the current video or file.
- The response is validated so the output matches your schema.
That cache matters. The first call for a new schema usually has a small compilation overhead. Reusing the same schema on later calls is faster because the compiled prompt is reused automatically.
Start with a Small Schema
The fastest way to succeed is to start with 2-4 fields, validate the output, then expand.
A schema is a standard JSON Schema whose root is an object: the fields go under properties. Here is a good starter schema for product review videos:
{
"type": "object",
"properties": {
"products_mentioned": {
"type": "array",
"description": "Products mentioned in the video",
"items": {
"type": "string",
"description": "Product name"
}
},
"main_claim": {
"type": "string",
"description": "The single most important claim or conclusion made in the video"
},
"sentiment": {
"type": "string",
"description": "Overall sentiment toward the main product discussed",
"enum": ["positive", "neutral", "negative"]
}
}
}Quickstart
Extract from an Online Video
Use this when you have a public video URL:
from vidnavigator import VidNavigatorClient
client = VidNavigatorClient() # reads VIDNAVIGATOR_API_KEY
resp = client.extract_video_data(
video_url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
what_to_extract="Focus on product names, pricing, and the speaker's overall verdict",
schema={
"type": "object",
"properties": {
"products_mentioned": {
"type": "array",
"description": "Products mentioned in the video",
"items": {"type": "string", "description": "Product name"},
},
"pricing_signals": {
"type": "array",
"description": "Any price points, pricing plans, or cost references",
"items": {"type": "string", "description": "Pricing detail"},
},
"verdict": {
"type": "string",
"description": "Final recommendation or overall verdict from the speaker",
},
},
},
)
print(resp.data) # a dict shaped like your schema
print(resp.video_info.title) # the video's metadataThe 2.1.0 SDKs run video extraction through /extract/video/async, so videos of any length work with the same call. To run many extractions in parallel, use client.submit_extract_video_data(...) (Python) or vn.extractVideo.submit(...) (JavaScript). The SDK result contains the extracted data and the video's metadata in video_info (resp.video_info in Python, video_info in JavaScript).
Extract from an Uploaded File
Use this when the transcript already lives in your VidNavigator workspace:
resp = client.extract_file_data(
file_id="file_abc123",
what_to_extract="Extract action items and owners from this meeting recording",
schema={
"type": "object",
"properties": {
"action_items": {
"type": "array",
"description": "Tasks agreed on during the meeting",
"items": {"type": "string", "description": "Action item"},
},
"owners": {
"type": "array",
"description": "People assigned to specific tasks",
"items": {"type": "string", "description": "Person name and task if available"},
},
"deadline_mentions": {
"type": "array",
"description": "Any due dates or timing commitments mentioned",
"items": {"type": "string", "description": "Deadline detail"},
},
},
},
)
print(resp.data)Reuse a Pydantic Model or a Zod Schema
Because schema is standard JSON Schema, the models you already have can be passed as they are: a Pydantic model's model_json_schema() in Python, a Zod schema converted to JSON Schema in JavaScript. Field descriptions become the instructions for each field, Literal and z.enum become enums, and nested models (sent as $ref / $defs) are followed.
from typing import Literal, Optional
from pydantic import BaseModel, Field
class Product(BaseModel):
name: str = Field(description="Product name")
price: Optional[float] = Field(None, description="Price in the currency shown")
class Review(BaseModel):
products: list[Product] = Field(description="Products shown or named in the video")
sentiment: Literal["positive", "negative", "neutral"] = Field(
description="Overall sentiment toward the main product"
)
resp = client.extract_video_data(
video_url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
schema=Review.model_json_schema(),
)
print(resp.data) # {"products": [...], "sentiment": "positive"}A field the video does not give comes back as null, even when the schema lists it as required. Validate data with your model only if its fields accept null.
Without a Schema
When you do not have a schema yet, send what_to_extract alone, in plain words. The schema is written from it and returned beside the data, as a JSON Schema: schema in the API answer, resp.extraction_schema in Python, schema in JavaScript. It needs version 2.6.0 of the SDKs.
resp = client.extract_video_data(
video_url="https://www.tiktok.com/@user/video/1234567890",
what_to_extract="Every product shown or mentioned, with its price and whether it is sponsored",
)
print(resp.data) # {"products": [{"name": "Wireless earbuds", "price": 49.99, "sponsored": True}]}
print(resp.extraction_schema) # {"type": "object", "properties": {"products": {...}}, ...}
# Later requests: pass the schema back to get exactly the same fields
resp = client.extract_video_data(video_url=other_url, schema=resp.extraction_schema)The answer carries the data and the schema it follows:
{
"status": "success",
"data": {
"products": [
{ "name": "Wireless earbuds", "price": 49.99, "sponsored": true }
]
},
"schema": {
"type": "object",
"properties": {
"products": {
"type": "array",
"description": "Every product shown or mentioned in the video",
"items": {
"type": "object",
"properties": {
"name": { "type": "string", "description": "Name of the product" },
"price": { "type": "number", "description": "Price stated for the product" },
"sponsored": { "type": "boolean", "description": "Whether the product is presented as sponsored" }
},
"required": ["name", "price", "sponsored"]
}
}
},
"required": ["products"]
},
"video_info": { "title": "My favorite tech this month" }
}- The same text gives the same fields, so repeated calls with the same
what_to_extractstay consistent. - To edit the fields (rename one, add an enum, tighten a description), start from the returned
schemaand send it asschema. - With both a
schemaandwhat_to_extract, the schema sets the fields and the text guides the extraction. With neither, the request is rejected with400 missing_parameter. - The async endpoint returns it as
schemabesideresultonGET /extract/video/{task_id}.
Fields That Are on Screen, Not Said
By default the fields are filled from what is said. With include_scenes the extraction also reads what the video shows — its scenes, with the text readable on screen — so a field can come from a price tag, a product on a shelf, a caption or a slide. It needs version 2.5.0 of the SDKs.
resp = client.extract_video_data(
video_url="https://www.dailymotion.com/video/x84243e",
schema={
"type": "object",
"properties": {
"dish": {"type": "string", "description": "Name of the dish"},
"ingredients": {
"type": "array",
"description": "Every ingredient, with its quantity",
"items": {"type": "string", "description": "One ingredient and its quantity"},
},
},
},
include_scenes=True,
)
print(resp.data["ingredients"]) # seven ingredients, read from the text on screen
print(resp.frames_analyzed) # 30This video only has music: without include_scenes, the same schema comes back empty.
With include_scenes, the picture is also looked at a second time with your fields in mind, so that a name on a lower third, a price on a label or a logo in a corner is read even when the scene descriptions leave it out. This second look is part of the extraction and costs nothing more.
The frames are charged on top of the extraction (1 credit per 2,000 frames analyzed, with at least 667 frames per hour of video) and given back if the extraction fails. On an uploaded file, include_scenes uses the scenes kept with the file: see Scenes of a file.
Where Each Value Comes From
Every extraction says where its values come from. Beside data, evidence gives, for each field that has a value, the moments of the video it was taken from: start and end in seconds, and in, which says how it was found:
speech: said in the video;picture: shown or written on screen (withinclude_scenes);post: read in the post's own text, its title, channel or description. These have no times.
{
"data": {"people": ["Reid Wiseman", "Christina H. Koch"], "summary": "Two astronauts talk about the mission."},
"evidence": {
"people": [{"start": 0.0, "end": 37.0, "in": "picture"}, {"in": "post"}],
"summary": [{"start": 0.0, "end": 24.5, "in": "speech"}]
}
}Use it to check a value, or to send someone to the exact moment in a player. A field left empty has no entry. The SDKs read it from version 2.7.0 (resp.evidence in Python, evidence on the JavaScript result).
When to Use what_to_extract
Your schema defines the output shape. what_to_extract tells the model where to focus. On its own, without a schema, it also decides the fields: see Without a Schema.
Use it next to a schema when:
- the transcript covers multiple topics and you care about one of them
- you want the model to prioritize a specific lens such as pricing, claims, objections, or action items
- the field descriptions are correct but still too broad
Good examples:
Focus on feature claims, competitor mentions, and pricing strategy.Extract only factual claims and cited statistics. Ignore jokes and off-topic banter.Prioritize buyer intent, objections, and next steps from the sales call.
Schema Format
schema is a standard JSON Schema whose root is an object, {"type": "object", "properties": {...}}, as Pydantic, Zod or a structured-output definition produce it. It is the only format accepted.
type | Extra keys | Example value |
|---|---|---|
string, number, integer, boolean | — | "upbeat", 4.5, 3, true |
array | items: the schema of one element | ["love", "commitment"] |
object | properties: the sub-fields | {"name": "Rick", "role": "singer"} |
- Enums:
enumrestricts a field to a list of values:{"type": "string", "enum": ["positive", "negative", "neutral"]}. Aconstis an enum of one value. - Descriptions: give each property a
descriptionsaying what to put in it. Without one, the field'stitleor its name is used. - References: local
$refs (#/$defs/...or#/definitions/..., whatmodel_json_schema()emits for nested models) are followed. - Nullable types such as
["number", "null"]oranyOf: [{"type": "number"}, {"type": "null"}]count as their type. - Hints:
format,pattern,minimum,maximumand similar keywords are passed on to guide the extraction. requiredandadditionalPropertieschange nothing: every field is returned, and a field the video does not give isnull.
Limits:
- maximum
10fields per level (root fields, and sub-fields of eachobject) - maximum
3nesting levels - not supported:
anyOf/oneOfbetween several types, recursive schemas
A schema that breaks these rules, or is not a JSON Schema with an object at its root, is rejected with 400 invalid_schema before anything is transcribed or billed. The error message says what to fix.
Write descriptions like instructions, not labels. Primary pricing strategy discussed, in one sentence is much better than pricing.
Best Practices That Improve Results
1. Be specific in field descriptions
Weak:
{
"type": "object",
"properties": {
"topic": {
"type": "string",
"description": "Topic"
}
}
}Better:
{
"type": "object",
"properties": {
"primary_topic": {
"type": "string",
"description": "Primary topic discussed in the video, in 5 to 10 words"
}
}
}2. Use enums when the answer should come from a fixed list
If a field is really a classification, add an enum with the allowed values instead of leaving a free string. That makes downstream automation much easier.
3. Start simple, then expand
Do not begin with a 10-field nested schema unless you already know it works. Start with the 2-3 fields that matter most, test them on a few videos, then add more.
4. Write descriptions in the output language you want
The response is returned in the same language as your schema descriptions. If you write field descriptions in French, the output will also come back in French.
Ready-to-Use Ideas
Common extraction patterns include:
- lead generation: companies, buyer intent, pain points, next steps
- market research: competitor mentions, feature claims, pricing strategy, objections
- creator analysis: hooks, quotes, sponsored mentions, content format
- RAG ingestion: dense summary, entities, claims, topics, language code
- fact-checking: claims, statistics, cited sources, controversy markers
Online Videos vs. Uploaded Files
POST /extract/video
Choose this endpoint when:
- you already have a public video URL
- you want VidNavigator to fetch the transcript for you
- you want automatic speech-to-text fallback when a video has no transcript
Important behavior:
transcribe=trueby default- automatic transcription applies to every platform, YouTube included
- when speech-to-text runs, media longer than 10 minutes must go through
POST /extract/video/async
POST /extract/file
Choose this endpoint when:
- the media is already uploaded to your VidNavigator account
- you want to extract from private or internal content
- you already completed transcription earlier in your workflow
Typical file workflow:
- Upload the file.
- Generate a transcript if needed.
- Call
/v1/extract/filewith the resultingfile_id.
Billing and Performance
Each extraction consumes analysis_request units. One unit covers up to 15,000 total tokens; longer transcripts scale with:
ceil(total_tokens / 15000)
A few practical notes:
- the first call for a new schema includes prompt compilation overhead
- repeated calls with the same schema reuse the cached plan for 2 hours
extract/videomay also triggertranscription_hourbilling when auto-transcription is used- failed requests are reverted and are not billed
Troubleshooting
transcript_not_available
This usually means the source does not have a usable transcript and auto-transcription is either disabled or unavailable.
- Keep
transcribe=trueon/extract/video: a video without a transcript is then transcribed from its audio. - For uploaded files, transcribe the file before calling
/extract/file.
video_too_long
The video needs speech-to-text and is longer than 10 minutes. Send the same request to /v1/extract/video/async and collect the result by polling or webhook.
invalid_schema
This usually means the schema is not a JSON Schema with "type": "object" and properties at its root, a field has no type or an unsupported one (types are the lowercase JSON Schema types listed above), nesting is too deep, or the schema exceeds the limits above. The error message says which field to fix.
missing_parameter
The request has neither a schema nor a what_to_extract. Send at least one of them.
schema_generation_failed
A request with what_to_extract alone could not get its schema written. This is temporary: retry, or pass a schema.
Results are too broad
Tighten the schema descriptions and add a more targeted what_to_extract instruction.
First request is slower than later requests
That is expected when a schema is new. The compiled extraction plan is cached for 2 hours, so repeated schemas are faster.
Next Steps
- Extract from an online video
- Extract from an uploaded file
- Upload a file first
- Schema files, token usage and batch extraction with the Python and JavaScript SDKs