Skip to Content

Analyzing Videos

Learn how to analyze videos to extract summaries, topics, and other insights.

Overview

VidNavigator's analysis capabilities allow you to go beyond simple transcription and understand the content of a video at a deeper level. You can get an AI-generated summary, identify key topics and speakers, and even ask specific questions about the video's content.

Every analysis returns a transcript_analysis object with:

  • summary: an overview of the content
  • people: people mentioned, with the context they appear in
  • places: locations referenced
  • key_subjects: important topics, with a description and importance
  • query_answer: when you pass a query, the direct answer plus the relevant_segments of the transcript that support it

Prerequisites

  • A valid VidNavigator API key.
  • The Python or JavaScript SDK, version 2.1.0 (pip install vidnavigator==2.1.0 or npm install vidnavigator@2.1.0).

Billing

AI analysis consumes analysis_request units based on context size. One unit covers up to 15,000 total tokens, and longer contexts are billed with ceil(total_tokens / 15000). For example, a video that takes 17,000 total tokens costs 2 analysis_request units.

Pass include_usage=True (Python) or include_usage: true (JavaScript) to receive a usage block with the exact charges and LLM token counts.

Analyzing a Video

from vidnavigator import VidNavigatorClient, VidNavigatorError

client = VidNavigatorClient()  # reads VIDNAVIGATOR_API_KEY

try:
    resp = client.analyze_video(
        video_url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
        query="What is the main theme of this song?",
    )
    analysis = resp.data.transcript_analysis

    print("Summary:", analysis.summary)
    print("Answer to your query:", analysis.query_answer.answer)

    # people, places and key_subjects are lists of plain dicts
    for person in analysis.people or []:
        print("Person:", person)    # {"name": ..., "context": ...}
    for subject in analysis.key_subjects or []:
        print("Subject:", subject)  # {"name": ..., "description": ..., "importance": ...}
except VidNavigatorError as e:
    print(f"An error occurred: {e}")

query is optional. Without it you still get the summary, people, places and key subjects.

Using What the Video Shows

By default the analysis reads what is said. With include_scenes it also reads what the video shows: its scenes, with the text readable on screen. Ask for it when the answer may be on screen rather than spoken (the quantities of a recipe, a price, a slide, a caption), or when the video has little or no speech. It needs version 2.5.0 of the SDKs.

resp = client.analyze_video(
    video_url="https://www.dailymotion.com/video/x84243e",
    query="What are the ingredients and their quantities?",
    include_scenes=True,
)
print(resp.data.transcript_analysis.query_answer.answer)
# "Shredded cooked chicken 500 g; mayonnaise 300 g; requeijão 250 g; ..."

print(resp.data.frames_analyzed)  # 30
for scene in resp.data.scenes or []:
    print(f"[{scene.start:.0f}s] {scene.description} | on screen: {scene.on_screen_text}")

This video only has music: without include_scenes, the same question gets no answer.

  • The scenes are the same as those of watching a video, and are returned with the analysis.
  • A video without a transcript is analyzed from its scenes, and transcript_error says why there is no transcript. A video whose picture cannot be read is analyzed from its transcript, and scenes_error says why.
  • The frames are charged on top of the analysis: 1 credit per 2,000 frames analyzed, with at least 667 frames per hour of video.
  • For a video longer than 10 minutes, run the analysis as a background job: client.submit_analyze_video(...) in Python, vn.analyze(...) in JavaScript, or POST /analyze/video/async.

Analyzing an Uploaded File

Files you have uploaded are analyzed the same way, by file_id:

resp = client.analyze_file(
    file_id="your_file_id",
    query="What are the key action items from this meeting?",
)
print(resp.data.transcript_analysis.summary)
print(resp.data.transcript_analysis.query_answer.answer)

include_scenes works on a file too, once the scenes of the file have been analyzed: see Scenes of a file.

Next Steps

On this page