AI Agent Tool Pagination: Stop One Response From Swallowing the Context Window

An agent asks a search tool for invoices, tickets, or documents. The tool returns 20 tidy JSON objects. The model writes a confident answer.

Those 20 objects may be page one of 2,000. Or they may be the only records the model can afford to read before its context becomes mostly old tool output. In both cases, a perfectly valid API response creates a bad decision.

The fix is not telling the model to “remember pagination.” Give it a result contract that makes incompleteness visible, bounds how much data enters context, and lets the agent choose the next useful retrieval step. This guide shows a practical pattern for paginated tool results that works with ordinary HTTP APIs, internal tools, and MCP-backed workflows.

Conventional APIs can assume a program will loop until nextCursor disappears. An agent cannot safely make that assumption. It has a task, a limited context window, and a tool response whose meaning is partly described in natural language.

Three failure modes repeat in production:

Pagination is therefore a reasoning and interface-design problem, not just an API detail. The right question is not “How can the agent load everything?” It is “What is the smallest next slice of evidence that can change the answer?”

That change in framing also improves user experience. A chat interface can show progress such as “I found 1,840 matches; I’m comparing the top accounts by open-ticket count” rather than silently issuing ten calls. The user can then refine the task before the system spends time and tokens browsing records that will not affect the conclusion.

Keep the complete response in a durable, access-controlled result store. Put only a compact page summary in the model-visible response. Give the model a reference it can use to fetch a narrow follow-up slice.

The result store is not long-term agent memory. Treat it as short-lived evidence for one task: tenant-scoped, expiry-bound, auditable, and deleted or redacted according to your data policy.

This separation also makes retries safer. A transport retry can retrieve the same page by reference without asking the model to reconstruct a query or blindly repeat a broad search. Keep retrieval state deterministic; let model reasoning operate on a compact, explicitly scoped view of that state.

Offset pagination is easy for a UI but fragile for changing data. Cursor pagination is usually safer because the server can preserve ordering and make the cursor opaque. The agent should never manufacture or inspect a cursor.

More important than the cursor is the surrounding contract. A useful response says what the page proves and what it does not prove:

type ToolPage < T > = { resultRef: string; // opaque, tenant-scoped, short lived items: T []; // hard-capped, e.g. 10–25 records returnedCount: number; knownTotal?: number; // omit if expensive or misleading hasMore: boolean; nextCursor?: string; // opaque; valid only with resultRef sort: { field: string; direction: " asc " | " desc " }; appliedFilters: Record < string, string | number | boolean >; coverage: { complete: boolean; reason?: " more_results " | " time_budget " | " permission_limited "; }; summary: string; // factual, generated by the server suggestedNext: Array < | " get_next_page " | " search_within_results " | " get_aggregate " | " get_record " >; }; Enter fullscreen mode Exit fullscreen mode coverage.complete matters. items.length < limit is not sufficient evidence that the task has complete data: permissions, timeouts, server caps, or a moving query can also shorten a page.

The summary should be factual rather than interpretive. Good: “Showing 20 newest open tickets, sorted by updated time; more matching tickets exist.” Bad: “These are the most important tickets.” Let the model decide importance, but make the available evidence explicit.

Paging every result is rarely the best plan. A search tool should offer alternatives that map to common user questions.

For example, a request such as “Which customers had the most unresolved billing issues?” can usually be answered by an aggregate, then a narrow drill-down:

// First: a cheap grouped view, not 500 ticket rows. const groups = await tickets. aggregate ({ resultRef, groupBy: " customerId ", metrics: [ " count " ], filters: { status: " open ", category: " billing " }, limit: 10, }); // Then: inspect only the evidence behind the leading group. const details = await tickets. searchWithin ({ resultRef, customerId: groups. items [ 0 ]. customerId, fields: [ " id ", " subject ", " updatedAt ", " priority " ], limit: 10, }); Enter fullscreen mode Exit fullscreen mode Design these moves deliberately:

This is also a useful dividing line for MCP. Protocol-level pagination can enumerate tools, prompts, and resources. Your business tool still needs its own domain-level result contract for searches, CRM records, logs, or catalog items.

Do not rely on a prompt rule such as “continue until complete.” Give the runtime a small, deterministic continuation policy. It should consider the task, evidence quality, page count, and resource budget.

function chooseNextStep ( input: { taskNeedsExhaustiveList: boolean; page: ToolPage < unknown >; pagesRead: number; maxPages: number; canAnswerFromCurrentEvidence: boolean; }) { if ( input. canAnswerFromCurrentEvidence ) return " answer_with_scope "; if (! input. page. hasMore ) return " answer_with_scope "; if ( input. pagesRead >= input. maxPages ) return " ask_or_narrow "; if (! input. taskNeedsExhaustiveList ) return " aggregate_or_search "; return " get_next_page "; } Enter fullscreen mode Exit fullscreen mode The final answer should carry the same scope honestly. If the tool only inspected the 50 newest records, say so. If it used an aggregate across every matching record and sampled five examples, say that too.

Scope language is not a disclaimer added at the end; it is part of correct reasoning.

Here is a simplified server implementation. Notice that the server owns the cursor and the result reference. It also limits fields before any data reaches the agent.

const PAGE_SIZE = 20; export async function searchTickets ( input: { tenantId: string; query: string; cursor?: string; resultRef?: string; }) { const state = input. resultRef? await resultStore. load ( input. resultRef, input. tenantId ): await resultStore. create ({ tenantId: input. tenantId, query: input. query }); const page = await ticketRepo. search ({ tenantId: input. tenantId, query: state. query, after: input. cursor, limit: PAGE_SIZE + 1, select: [ " id ", " subject ", " status ", " priority ", " updatedAt " ], }); const hasMore = page. rows. length > PAGE_SIZE; const items = page. rows. slice ( 0, PAGE_SIZE ); const nextCursor = hasMore? encodeCursor ( items. at ( - 1 )!. updatedAt, items. at ( - 1 )!. id ): undefined; return { resultRef: state. ref, items, returnedCount: items. length, hasMore, nextCursor, sort: { field: " updatedAt ", direction: " desc " as const }, appliedFilters: { query: state. query }, coverage: { complete:! hasMore, reason: hasMore? " more_results ": undefined }, summary: `Showing ${ items. length } newest matches ${ hasMore? "; more matches exist ": "; complete result set " }.`, suggestedNext: hasMore? [ " get_next_page ", " search_within_results ", " get_aggregate " ]: [ " get_record " ], }; } Enter fullscreen mode Exit fullscreen mode In a real system, sign or encrypt cursors, bind resultRef to the actor and tenant, and expire both quickly.

Never let a cursor from one query, user, or tenant open another result set.

Every tool that can return a collection needs explicit limits. Pick them before you tune prompts:

These limits make cost and correctness testable. They also prevent a user request such as “show me all activity” from turning into an unbounded retrieval loop.

Unit tests for nextCursor are necessary but incomplete. Add workflow tests where the agent has to choose an information-gathering action.

Log page count, result bytes, continuation reason, aggregate use, and final-answer scope. Those signals tell you whether pagination is reducing tool cost or merely hiding a retrieval problem.

“Return everything and let the model summarize it.” This is expensive and makes the model less reliable as the output grows. Put raw records in the result store instead.

“Always follow nextCursor.” A cursor is an option, not an instruction. An aggregate or a narrower filter is often the next best action.

“Put pagination instructions in the tool description.” Keep the description, but enforce caps and state binding in code. A model can miss a sentence; the runtime should not.

“Use page length to infer completeness.” A partial page can be caused by permissions or server limits. Return explicit coverage metadata.

Only when the task needs an exhaustive, ordered list and the budget allows it. For rankings, counts, trends, or a single record, an aggregate or targeted search is normally better.

Choose a small, measured cap—often 10 to 25 compact records—then test with your real schemas. The byte size and field selection matter more than the row count.

Usually. Opaque cursors better support stable ordering and discourage the agent from inventing page numbers. They still need tenant binding, expiry, and explicit completeness metadata.

MCP has pagination conventions for protocol list operations. A custom tool that searches business data should define its own response schema with a cursor, bounded items, and clear continuation semantics.

Store the full result server-side, return a short factual summary plus a scoped result reference, and expose follow-up actions such as search, aggregate, and record lookup.

State the evidence scope plainly: for example, “I reviewed the 50 most recently updated matching tickets” or “I used an aggregate over all matches and inspected five examples.”

Pagination is successful when the agent can explain both what it found and how complete that evidence is. Build that truth into the tool response, and the context window stops being a dumping ground for records it never needed.