Your AI agent doesn't need a frontier model. It needs a smaller one.

A position paper making the rounds this year comes from an unusual messenger. NVIDIA Research, the research arm of the company selling the GPUs that frontier models run on, argues in Small Language Models are the Future of Agentic AI that the agentic era belongs to small models. Their claim: for most of the calls an AI agent makes, a model under 10 billion parameters is powerful enough, better suited, and dramatically cheaper.

The headline number: serving a 7-billion-parameter model is 10 to 30 times cheaper than a 70-to-175-billion-parameter model, measured in latency, energy and compute. And specializing a small model takes a few GPU-hours of fine-tuning, not weeks.

This matters because agents change what a model is for. A chatbot holds open-ended conversation, and generality earns its keep. An agent, in the paper's framing, performs a small number of specialized tasks repetitively: plan, call a tool, check the result, repeat. Paying frontier-model prices for encyclopedic generality on every one of those calls is like hiring a professor to stamp forms. Surveys cited in the paper say more than half of large IT enterprises already run AI agents, with 21% having adopted just within the last year. The bill for all those repetitive calls is becoming a board-level topic.

NVIDIA's paper is an argument, so I went looking for independent measurements. The strongest I found is a 2026 study in the IJCAI proceedings from DFKI, Germany's AI research center, which benchmarked small models on accuracy and energy use across five tasks.

On GSM8K, a math reasoning benchmark, the best large model they tested, MistralLarge-3 at 675 billion parameters, consumed about 32 times the energy of the best small model, Qwen-3-4B, for roughly a 2-point absolute accuracy gain. The diminishing returns show up inside the small range too: on another benchmark, moving from Qwen-3-8B to Qwen-3-14B for a single percentage point of accuracy nearly doubled the inference energy.

The same study delivers the honest counterpoint, and it is the part most "small is better" takes skip. Smaller is not automatically cheaper. Comparing each small model against every larger one, the smaller model burned more energy in 14 to 43 percent of pairwise comparisons, depending on the benchmark. The reason is output length: a small model that rambles spends its savings on extra tokens. Size and cost-per-token alone do not determine efficiency. Behavior does.

Meanwhile, the enterprise constraint is real. Analyst firm Ecosystm found that 54% of organizations in Asia Pacific identify infrastructure limitations as the biggest barrier to AI deployment, and only 21% say they have the compute and infrastructure capabilities to support AI growth. Small models attack exactly that constraint. They run on consumer-grade hardware, at the edge, even offline, which also keeps data in-house.

In NVIDIA's tests on three open-source agent frameworks, specialized small models could potentially replace 40 to 70 percent of the LLM calls. The pattern is routing: a small model handles the routine steps, and the frontier model sees only the hard residue. Add fine-tuning that costs GPU-hours and finishes overnight, and each step of an agent can have its own specialist instead of renting one generalist.

This section is my opinion, built from the sources above.

NVIDIA's paper is a position paper, meaning advocacy, from a company whose hardware powers both ends of this market. The IJCAI numbers come from specific benchmarks, models and one hardware setup, and the authors note results can vary with deployment. Ecosystm's figures cover Asia Pacific only. What I trust is the direction across all three: for repetitive agent work, model size is a cost decision more than a capability one. Exact savings will vary by workload.