Analyzing Videos
Learn how to analyze videos to extract summaries, topics, and other insights.
Overview
VidNavigator's analysis capabilities allow you to go beyond simple transcription and understand the content of a video at a deeper level. You can get an AI-generated summary, identify key topics and speakers, and even ask specific questions about the video's content.
Every analysis returns a transcript_analysis object with:
summary: an overview of the contentpeople: people mentioned, with the context they appear inplaces: locations referencedkey_subjects: important topics, with a description and importancequery_answer: when you pass aquery, the directanswerplus therelevant_segmentsof the transcript that support it
Prerequisites
- A valid VidNavigator API key.
- The Python or JavaScript SDK, version 2.1.0 (
pip install vidnavigator==2.1.0ornpm install vidnavigator@2.1.0).
Billing
AI analysis consumes analysis_request units based on context size. One unit covers up to 15,000 total tokens, and longer contexts are billed with ceil(total_tokens / 15000). For example, a video that takes 17,000 total tokens costs 2 analysis_request units.
Pass include_usage=True (Python) or include_usage: true (JavaScript) to receive a usage block with the exact charges and LLM token counts.
Analyzing a Video
from vidnavigator import VidNavigatorClient, VidNavigatorError
client = VidNavigatorClient() # reads VIDNAVIGATOR_API_KEY
try:
resp = client.analyze_video(
video_url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
query="What is the main theme of this song?",
)
analysis = resp.data.transcript_analysis
print("Summary:", analysis.summary)
print("Answer to your query:", analysis.query_answer.answer)
# people, places and key_subjects are lists of plain dicts
for person in analysis.people or []:
print("Person:", person) # {"name": ..., "context": ...}
for subject in analysis.key_subjects or []:
print("Subject:", subject) # {"name": ..., "description": ..., "importance": ...}
except VidNavigatorError as e:
print(f"An error occurred: {e}")query is optional. Without it you still get the summary, people, places and key subjects.
Using What the Video Shows
By default the analysis reads what is said. With include_scenes it also reads what the video shows: its scenes, with the text readable on screen. Ask for it when the answer may be on screen rather than spoken (the quantities of a recipe, a price, a slide, a caption), or when the video has little or no speech. It needs version 2.5.0 of the SDKs.
resp = client.analyze_video(
video_url="https://www.dailymotion.com/video/x84243e",
query="What are the ingredients and their quantities?",
include_scenes=True,
)
print(resp.data.transcript_analysis.query_answer.answer)
# "Shredded cooked chicken 500 g; mayonnaise 300 g; requeijão 250 g; ..."
print(resp.data.frames_analyzed) # 30
for scene in resp.data.scenes or []:
print(f"[{scene.start:.0f}s] {scene.description} | on screen: {scene.on_screen_text}")This video only has music: without include_scenes, the same question gets no answer.
- The scenes are the same as those of watching a video, and are returned with the analysis.
- A video without a transcript is analyzed from its scenes, and
transcript_errorsays why there is no transcript. A video whose picture cannot be read is analyzed from its transcript, andscenes_errorsays why. - The frames are charged on top of the analysis: 1 credit per 2,000 frames analyzed, with at least 667 frames per hour of video.
- For a video longer than 10 minutes, run the analysis as a background job:
client.submit_analyze_video(...)in Python,vn.analyze(...)in JavaScript, orPOST /analyze/video/async.
Analyzing an Uploaded File
Files you have uploaded are analyzed the same way, by file_id:
resp = client.analyze_file(
file_id="your_file_id",
query="What are the key action items from this meeting?",
)
print(resp.data.transcript_analysis.summary)
print(resp.data.transcript_analysis.query_answer.answer)include_scenes works on a file too, once the scenes of the file have been analyzed: see Scenes of a file.
Next Steps
- Need specific fields rather than a free-form summary? Use structured data extraction.
- See the Analyze Video and Analyze File API references.