"Summarize Video with Gemini API in Python: Full Guide"
"Summarize video with the Gemini API in Python: upload user clips via the Files API, wait for ACTIVE state, send structured summary prompts, clip segments with start/end offsets, and control costs with count_tokens. One complete runnable script, plus the limits and mistakes that matter."
To summarize video with Gemini API Python, forget the pipeline you were about to build. No ffmpeg frame extraction, no thumbnail grids stitched into a collage, no separate transcription service whose output you then summarize. Gemini watches video natively — about 1 frame per second, audio included — straight from an uploaded file or even a YouTube URL. The DIY frame pipeline isn't just extra work; it produces worse results than handing the model the actual video. This guide gives you the complete production pattern: upload, wait for processing, summarize, clip segments, control costs, and reuse the file. The complete pipeline Upload the user's clip with the Files API (mandatory above 20 MB) Poll until the file state is ACTIVE — and handle FAILED properly Send the video part plus a structured summary prompt ( prompt goes after the video ) Summarize a single segment with start_offset / end_offset instead of re-uploading Pre-check costs with count_tokens (~258–300 tokens per second of video at 1 fps) Reuse the same file_uri for follow-up questions within the 48-hour window Shortcut: pass a YouTube URL directly for public videos Step 1 — Upload the clip with the Files API Inline video bytes work only for small files (under ~20 MB). For user uploads, the Files API is the correct path — it also gives you a reusable file_uri and a 48-hour retention window: import os from google import genai client = genai.Client(api_key=os.environ["GEMINI_API_KEY"]) MODEL = "gemini-2.5-flash" # fast and cheap; use gemini-2.5-pro for deep analysis video_file = client.files.upload(file="user_clip.mp4") print(video_file.name, video_file.state.name) # files/abc123xyz PROCESSING The upload returns immediately while Google transcodes in the background. Never call generate_content with…