Should your company buy an AI server? Hardware tiers, prices and payback vs the cloud

Posted on Oct 27 • Originally published at klukyanov.ru

Companies keep asking me for an AI model on their own hardware. Usually the honest answer is "not yet": for short questions, a cloud API stays cheaper than a GPU workstation for about a year and a half. The picture changes when the model has to read a lot of internal documents on every request, or when the data simply can't leave the building. Then a single-GPU machine can pay for itself in roughly four months.

Below is how I size the hardware, what it costs (I'm based in Russia, so prices are from the Russian market in October 2026), and how to do the payback math for your own workload.

If none of these apply, start with a cloud API. It's faster and cheaper, and you can move to your own server once you know your real request volume.

The key spec is GPU memory, not CPU power. The model has to fit entirely in VRAM, otherwise it runs many times slower or doesn't start.

Local models are almost always quantized to 4 bits per parameter, with barely noticeable quality loss. That gives a simple rule of thumb: about 0.5 GB per billion parameters, plus headroom for context.

The headroom matters. Long documents and several concurrent users add gigabytes on top of the weights.

I ran the same task on my own laptop, a MacBook with Apple M5 and 16 GB of memory, using Ollama: write a 150-word business email about a three-day delivery delay.

The small model writes faster than a person reads. The model three times larger generates almost half as fast, and prompt processing drops twentyfold: on 16 GB it's competing with the OS and open apps. For a short email you barely notice. Hand it a twenty-page contract and an employee waits for minutes.

The lesson for a server: buy 1.5–2x more memory than the model weighs. A 20 GB model on a 24 GB card is a squeeze; on 32 GB it's comfortable.

Sources: Digital Razor for RTX 5090 and PCs, the RTX PRO 6000 launch at DNS at 1,079,999 RUB in February 2026, Megabait for Mac Studio, zeroquant.ru for GPU rental, Yandex AI Studio pricing.

Two notes. NVIDIA cards have faster memory, so they answer quicker and handle several users at once better. A Mac gives you more memory per ruble and is nearly silent, but serves fewer people concurrently. For a team of 10–30 I'd pick a GPU; for one analyst or lawyer with huge documents, a Mac. And the RTX 5090 draws up to 575 W, the RTX PRO 6000 up to 600 W: this is a hot, loud machine that belongs in a server room, not under a meeting-room desk.

Example: a team of 20, each making 30 requests a day, 22 working days a month.

YandexGPT Pro 5.1 costs 0.8 RUB per 1,000 tokens including VAT. Open models in the same cloud are cheaper (gpt-oss-120b at 0.3 RUB, gpt-oss-20b at 0.1 RUB), which pushes payback out another 2.5–8x.

Renting is the middle ground. An RTX 5090 at 66 RUB an hour, kept only during working hours (about 200 hours a month), costs around 13k RUB a month; around the clock, about 48k. An H100 around the clock is about 255k RUB a month, which only makes sense under heavy load.

Not included: electricity, setup and maintenance, and model quality. A 32B model that fits on an RTX 5090 trails YandexGPT Pro on some tasks and matches it on others. Test that on your own data before buying anything.

My default for a company starting out: don't buy a server right away. A month on a cloud API shows the real volume. If data can't leave the office or the model works over a document base, start with one RTX 5090 machine: it handles models up to 32B for a team of a few dozen people. The RTX PRO 6000 and a 512 GB Mac Studio are for tasks where a smaller model has proven not to be enough.

Is your team running models on its own hardware yet? What tipped the decision, cost or security?

Shorter weekly write-ups (in Russian) — on Telegram.