Pegasus 1.5 from TwelveLabs is now available in Amazon Bedrock

On Oct 9, 2026 we announced the availability of TwelveLabs’ Pegasus 1.5 on Amazon Bedrock. Pegasus 1.5 is a video language model that turns long-form video into structured, timestamped metadata. You describe the moments that matter to your workflow, and the model returns every matching segment as JSON.

Video represents a significant and complex category of unstructured data, yet most of it remains unsearchable and underutilized. Extracting structured, actionable insights from that content has historically required labor-intensive manual review or fragmented toolchains.

With earlier versions of Pegasus, you can point the model at a clip, ask a question, and get an answer. Now, Pegasus 1.5 analyzes the entire video, finds every segment that matches user-defined criteria, and returns structured, timestamped metadata. It also introduces video segmentation, multimodal prompting with reference images, and on-the-fly processing.

With Pegasus 1.5 on Amazon Bedrock, you can analyze videos directly from an Amazon Simple Storage Service (Amazon S3) URI or a base64 string with no pre-indexing required, and your data remains within the AWS data boundary.

In this post, we cover what’s new in Pegasus 1.5, how the model is packaged on Amazon Bedrock, and how to send your first request. We then walk through video segmentation, structured JSON output, and multimodal prompting.

Pegasus 1.5 has the following changes compared to previous versions:

Pegasus 1.5 retains and enhances the core capabilities that made Pegasus 1.2 a powerful tool for video understanding, including:

Pegasus 1.5 takes a video and either a text prompt or a set of segment definitions, and returns text or JSON. You call it on the bedrock-runtime endpoint with the InvokeModel API, or with InvokeModelWithResponseStream when you want the response streamed as it’s generated.

Each request runs in one of two analysis modes, chosen with the analysisMode field:

You can point the model at video in two ways. An S3 location ( s3Location ) takes the object’s uri and the bucketOwner account ID, and suits long-form video. A base64 string ( base64String ) embeds the video in the request body, up to 25 MB, and suits short clips.

To track cost by application, create an application inference profile from the Pegasus 1.5 model, tag it, and call it by its Amazon Resource Name (ARN) in place of the model ID.

The following section outlines how to get started with Pegasus 1.5.

The following policy grants both invoke actions on Pegasus 1.5.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream" ], "Resource": [ "arn:aws:bedrock:us-west-2::foundation-model/twelvelabs.pegasus-1-5-v1:0" ] } ] } Boto3 signs requests with your usual AWS credentials. You can also authenticate with an Amazon Bedrock API key by setting it in the AWS_BEARER_TOKEN_BEDROCK environment variable. For production, prefer a short-term key generated from your IAM credentials with the aws-bedrock-token-generator package, because it expires automatically. Bearer-token authentication also requires the bedrock:CallWithBearerToken permission.

pip install boto3 The following example analyzes a video stored in Amazon S3 in general mode. This example uses the global Cross Region Inference (CRIS) endpoint. To target the US geographic endpoint instead, replace “global.” with “us.” in the endpoint URL.

import boto3 import json from botocore.config import Config AWS_REGION = "us-west-2" MODEL_ID = "global.twelvelabs.pegasus-1-5-v1:0" bedrock_runtime = boto3.client( service_name="bedrock-runtime", region_name=AWS_REGION, config=Config( # keep a longer read_timeout for lengthier invocations such as video segmentation read_timeout=900, retries={"max_attempts": 0}, connect_timeout=10, ), ) # Analyze a video stored in Amazon S3 payload = { "inputPrompt": "Summarize this video and identify the key topics discussed.", "mediaSource": { "s3Location": { "uri": "s3://amzn-s3-demo-pegasus-15/your-video.mp4", "bucketOwner": "your-account-id" } }, "temperature": 0.2, "maxOutputTokens": 4096, "includeMetering": True } response = bedrock_runtime.invoke_model( modelId=MODEL_ID, body=json.dumps(payload) ) result = json.loads(response["body"].read()) print(result["message"]) The response carries the answer in message, the token counts in input_tokens and output_tokens, and the reason generation ended in stopReason. A stopReason of length means the answer reached maxOutputTokens before it finished, so check it before you rely on the output:

if result["stopReason"] == "length": print("The answer was cut off. Raise maxOutputTokens and try again.") print("Tokens in:", result["input_tokens"], "out:", result["output_tokens"]) Request fields for general mode:

For a short clip, you can send the video inline instead of from S3:

import base64 with open("your-video.mp4", "rb") as f: video_b64 = base64.b64encode(f.read()).decode("utf-8") payload["mediaSource"] = {"base64String": video_b64} response = bedrock_runtime.invoke_model(modelId=MODEL_ID, body=json.dumps(payload)) print(json.loads(response["body"].read())["message"]) Note that the preceding example uses the optional includeMetering attribute. includeMetering: True makes Pegasus 1.5 add two billing-usage fields to the non-streaming response: outputTokenCount and inputVideoSecond (the analyzed video’s duration in seconds).

For interactive applications, InvokeModelWithResponseStream returns the answer in chunks as it’s generated. Each chunk carries the next piece of text in message:

response = bedrock_runtime.invoke_model_with_response_stream( modelId=MODEL_ID, body=json.dumps(payload) ) for event in response["body"]: chunk = json.loads(event["chunk"]["bytes"]) print(chunk.get("message", ""), end="", flush=True) print() Return structured JSON In general mode, you can constrain the answer to a JSON Schema so downstream code can parse it directly. Set responseFormat.type to json_schema and put the schema under jsonSchema. The answer arrives in message as a JSON string:

payload = { "inputPrompt": "Analyze this video and return a structured summary.", "mediaSource": { "s3Location": { "uri": "s3://amzn-s3-demo-pegasus-15/your-video.mp4", "bucketOwner": "your-account-id" } }, "temperature": 0, "maxOutputTokens": 2048, "includeMetering": True, "responseFormat": { "type": "json_schema", "jsonSchema": { "type": "object", "properties": { "summary": {"type": "string"}, "key_scenes": {"type": "array", "items": {"type": "string"}}, "setting": {"type": "string"} }, "required": ["summary", "key_scenes", "setting"] } } } response = bedrock_runtime.invoke_model(modelId=MODEL_ID, body=json.dumps(payload)) summary = json.loads(json.loads(response["body"].read())["message"]) print(summary["summary"]) for scene in summary["key_scenes"]: print(" -", scene) Video segmentation: Turning video into structured data Time-based metadata generation is where Pegasus 1.5 differs most from earlier versions. Instead of a prompt, you send one or more segment definitions, and the model returns every segment of the video that matches each one.

Set analysisMode to time_based_metadata and responseFormat.type to segment_definitions. The definitions take the place of inputPrompt, so leave that field out. Each definition has:

The following example identifies each editorial section in a news broadcast and returns a title, summary, and key topics for each one:

payload = { "analysisMode": "time_based_metadata", "mediaSource": { "s3Location": { "uri": "s3://amzn-s3-demo-pegasus-15/news-broadcast.mp4", "bucketOwner": "your-account-id" } }, "responseFormat": { "type": "segment_definitions", "segmentTimeFormat": "hh:mm:ss", "segmentDefinitions": [ { "id": "editorial_section", "description": "A distinct editorial section of the broadcast, " "such as a new story, topic, or segment of the program.", "fields": [ {"name": "segment_title", "type": "string", "description": "A descriptive title"}, {"name": "summary", "type": "string", "description": "A brief summary of the segment content"}, {"name": "key_topics", "type": "array", "items": {"type": "string"}, "description": "Main topics discussed"} ] } ] }, "temperature": 0, "maxOutputTokens": 32768, "includeMetering": True } response = bedrock_runtime.invoke_model( modelId=MODEL_ID, contentType="application/json", accept="application/json", body=json.dumps(payload) ) result = json.loads(response["body"].read()) for segment in result["segments"]: print(segment["start_sec"], "-", segment["end_sec"], segment["segment_title"]) print(" ", segment["summary"]) print(" topics:", ", ".join(segment["key_topics"])) The previous sample code when tested against a sports clip may produce an output like the following sample.

{ "segments": [ { "start_sec": "00:04:24", "end_sec": "00:05:22", "segment_title": "Substitutions and Added Time", "summary": "The referee announces three minutes of added time, and both teams make substitutions in preparation for a potential penalty shootout.", "key_topics": ["Substitutions", "Added time", "Penalty shootout preparation", "Tactical changes"] }, { "start_sec": "00:05:22", "end_sec": "00:08:57", "segment_title": "Last Gasp Attempts and Final Whistle", "summary": "The final minutes of the match see intense action from both sides, with several close calls before the referee blows the final whistle, ending the game in a 3-3 draw.", "key_topics": ["Final whistle", "Draw"] }, { "start_sec": "00:08:57", "end_sec": "00:09:29", "segment_title": "Match Conclusion and Penalty Shootout Setup", "summary": "The match concludes with a 3-3 draw, leading to a penalty shootout. Players from both teams shake hands and prepare for the decisive moments.", "key_topics": ["Match conclusion", "Penalty shootout", "Player handshakes", "Final"] } ], "definition_errors": [], "input_tokens": 249408, "output_tokens": 802, "stopReason": "stop", "finish_reason": "stop" } Each segment carries start_sec, end_sec, and the fields you defined. segmentTimeFormat controls how the times are written: seconds returns numbers you can do arithmetic on, while hh:mm:ss and hh:mm:ss.fff return readable timestamps as strings. The response also includes a definition_errors list, which is empty when every definition was processed.

A single request can include multiple segment definitions, for example one for editorial sections and another for on-screen brand appearances. Pegasus 1.5 returns the matches for all of them in one segments list, and tags each segment with _definition_id so you can tell them apart. The following example adds a second definition to the previous request and groups the results:

from collections import defaultdict payload["responseFormat"]["segmentDefinitions"].append({ "id": "brand_appearance", "description": "A shot where a company logo or branded product " "is clearly visible on screen.", "fields": [ {"name": "brand", "type": "string", "description": "The brand shown"}, {"name": "is_prominent", "type": "boolean", "description": "Whether the brand is the main focus of the shot"} ] }) response = bedrock_runtime.invoke_model(modelId=MODEL_ID, body=json.dumps(payload)) result = json.loads(response["body"].read()) by_definition = defaultdict(list) for segment in result["segments"]: by_definition[segment["_definition_id"]].append(segment) for definition_id, segments in by_definition.items(): print(definition_id, len(segments), "segments") Because the schema is defined once, the same definitions can be applied across an entire video library, producing consistent metadata for every asset.

Pegasus 1.5 introduces multimodal prompting, so you can include reference images alongside your text prompts. This is useful when you want to ask the model about specific objects, people, or scenes that are easier to show than describe.

For example, you can provide a product image and ask the model to identify every moment where that product appears in a video. Or you can provide a reference image of a specific logo and ask the model to detect brand appearances and their timestamps. This capability enhances the precision of video analysis, and opens up use cases such as visual brand monitoring, product placement analysis, and image-guided content search.

Reference images attach differently depending on the mode. In general mode, put them in promptV2.mediaSources, alongside the question in promptV2.inputText. In segmentation mode ( analysisMode: time_based_metadata ), attach them to the mediaSources of each segment definition inside responseFormat.segmentDefinitions. A definition can then match a visual reference as well as a text description. Each image needs a name, a media_type of image, and exactly one source: either an inline base64String or an s3Location with uri and bucketOwner. A single request can carry up to 30 reference images.

A general-mode request asks the model about an object shown in a reference image. The following sample payload and response provide an example in general mode.

payload = { "analysisMode": "general", "mediaSource": {"s3Location": {"uri": "s3://...", "bucketOwner": "..."}}, "promptV2": { "inputText": "Does the object in the reference image appear in this video?", "mediaSources": [ {"name": "ref1", "media_type": "image", "base64String": "..."} ] } … } { "input_tokens": 144, "output_tokens": 11, "message": "Yes, the green canoe appears in the video.", "stopReason": "stop" } In segmentation mode, the reference image lives inside the definition, so Pegasus returns only the segments where that product appears. The following sample payload and response provide an example in segmentation mode.

payload = { "analysisMode": "time_based_metadata", "mediaSource": {"s3Location": {"uri": "s3://...", "bucketOwner": "..."}}, "responseFormat": { "type": "segment_definitions", "segmentTimeFormat": "hh:mm:ss", "segmentDefinitions": [ { "id": "product_appearance", "description": "A shot showing the product in the reference image", "fields": [ {"name": "what", "type": "string", "description": "what is on screen"} ], "mediaSources": [ {"name": "ref1", "media_type": "image", "s3Location": {"uri": "s3://...", "bucketOwner": "..."}} ] } ] } } { "segments": [ {"start_sec": "00:00:00", "end_sec": "00:00:02", "what": "A family of four in a green canoe, paddling on a calm river surrounded by towering red rock formations."}, {"start_sec": "00:00:09", "end_sec": "00:00:10", "what": "The family back in the canoe, paddling away from the shore, with the children smiling and enjoying the serene surroundings."} ], "definition_errors": [], "input_tokens": 4498, "output_tokens": 292, "stopReason": "stop", "finish_reason": "stop" } Monitoring usage Pegasus 1.5 publishes Amazon CloudWatch metrics in the AWS/Bedrock namespace under its model ID, including Invocations, OutputTokenCount, InvocationLatency, and InvocationClientErrors. Use them to track volume and latency. Pair them with application inference profiles when you need usage broken down by application.

Pegasus 1.5 is designed for organizations that manage large volumes of long-form video:

In each case, people remain part of the workflow. Their work shifts from finding and tagging every moment by hand to reviewing and refining the metadata generated by the model.

Pegasus 1.5 supports the following video specifications:

TwelveLabs Pegasus 1.5 is available today on Amazon Bedrock in the US West (Oregon) AWS Region. Check the full Region list for future updates, and visit the Amazon Bedrock pricing page for pricing information. Pegasus 1.5 is billed through AWS Marketplace, so its charges appear on your AWS bill and in AWS Cost Explorer under TwelveLabs rather than Amazon Bedrock.

We’re excited to see what you build with TwelveLabs Pegasus 1.5 in Amazon Bedrock.

Ron is a Machine Learning Engineer at TwelveLabs, where he leads inference deployment of TwelveLabs models through third-party platforms, including Amazon Bedrock. He has spent more than a decade building machine learning platforms and production inference infrastructure, from founding ML platform teams to serving foundation models at scale.

Shannon is a Product Manager at TwelveLabs, where she leads the development of Pegasus, a video language model. She is Research Affiliate of the Oxford Martin School AI Governance Initiative and has conducted research on democratic governance of AI with Open Future Foundation and Creative Commons. She started her career as a Product Manager at Scale AI, where she led technical teams to build data labeling products for autonomous vehicles. Shannon holds an M.B.A from Oxford University, and a B.A. in Development Studies and Data Science at the University of California, Berkeley.

Nate is a Senior Partner Development Manager at AWS, where he helps AI ISVs build and execute go-to-market strategies. With 10 years at AWS, Nate focuses on helping AI model providers build, market, and sell their solutions – spanning Amazon Bedrock integrations, AWS Marketplace, and field enablement. In his spare time, Nate enjoys being outdoors. You can connect with or follow Nate on LinkedIn.

Narcisse is a Principal Solutions Architect based in Boston. He helps customers in the Northeast U.S. accelerate their business transformation through innovative, and scalable solutions, on the AWS Cloud. He is passionate about enabling organizations to transform their business, using advanced analytics and AI. When Narcisse is not building, he enjoys spending time with his family, traveling, and road racing.

Suheel is a Principal Solutions Architect at AWS, specializing in artificial intelligence, machine learning, and generative AI. He helps Foundation Model Provider customers design, build, modernize, and scale their AI/ML and generative AI workloads on AWS — from bringing foundation models to Amazon Bedrock to optimizing training and inference and building Agentic AI solutions. His experience spans the AWS AI/ML and generative AI portfolio, particularly Amazon Bedrock, Amazon Bedrock AgentCore, and Amazon SageMaker AI. In his free time, Suheel enjoys working out and hiking.

Badri is a Software Development Engineer at Amazon SageMaker AI. He helps bring foundation models from leading model providers to customers by building scalable, secure and observable hosting infrastructure behind them on Amazon SageMaker. Beyond AWS, he is deeply passionate about robotics, especially autonomous vehicles. When not working, Badri likes to go on hikes, swimming and cafe-hopping.