Modal says Nvidia’s Vera CPU doubles throughput for its AI sandboxes

Modal plans to run its Sandboxes product on Nvidia’s Arm-based Vera CPU, its first step away from x86 chips. In Modal’s own tests, Vera handled up to twice as many tasks as an Intel server when packed with containers. The results have not been independently checked.

Modal plans to run its AI agent sandboxes on Nvidia’s new Vera CPU. In its own tests, the chip handled up to twice as many tasks as an Intel server. The AI cloud company announced the plan in a blog post on Monday.

It will be Modal’s first move away from x86 chips, the kind made by Intel and AMD. Vera uses Arm’s design instead, so Modal will need to change its infrastructure and the system it uses to build software images. Nvidia hand-delivered the company’s first Vera server to its San Francisco office last week, Modal said.

Sandboxes are isolated computers where AI agents run code they write. The product makes $300M a year, Modal said last week when it raised $750M at a $15.75B valuation.

Sandbox workloads are hard to plan for, Modal said. An agent may sit idle while it waits for an AI model, then use ten times more computing power during a software install. Hundreds of these run side by side on each server, so one customer’s spike can slow down another’s work.

To test the chip, Modal built a benchmark of 50 coding tasks, such as compiling software and searching files. Run one at a time, Vera was often faster than chips from Intel, AMD and Amazon, but not always, and not by much.

The gap grew when Modal packed the server with containers. Vera kept finishing tasks at the same speed, while rival chips hit a ceiling, Modal said. The company credits Vera’s high memory bandwidth.

That matters for two uses, Modal said: background coding agents and reinforcement learning. In reinforcement learning, expensive GPUs often sit idle until the CPU work finishes.

Nvidia unveiled Vera in March as a CPU built for AI agents, with 88 cores, and has named Anthropic and OpenAI among its first users. Perplexity has said it will run agent work on the chip, after reporting 1.5 times faster coding tasks. Rivals are building up their own sandbox offerings too, with Baseten buying Blaxel in September.

Modal did not say when the Vera-powered sandboxes would reach customers.

With expertise in digital marketing, product management, and branding & identity, Ana Maria Constantin develops strategies that resonate (show all) With expertise in digital marketing, product management, and branding & identity, Ana Maria Constantin develops strategies that resonate with our target audience in the software/SaaS industry. Collaboration and teamwork are paramount to her, as she loves empowering her colleagues to achieve outstanding results and unlock their full potential.

Get the most important tech news in your inbox each week.

Thanks! Check your inbox to confirm your subscription.

By subscribing you agree to our Privacy Statement.