Introducing Superfluid: the local-AI native LLM server for multi-agent inference

The future of local AI is agentic and requires multiple agents to share one machine: one agent summarising your emails overnight, one watching your health data and one doing the shopping, while you are still able to chat and ask questions in between. Today's inference servers were built for throughput on data centre racks, not for one person sharing a laptop with eight agents. Without prioritisation, your question waits behind whatever is already running and without logging, a restart takes every session with it.

Superfluid is an open-source (Apache-2.0) server for running LLMs on your own machine. Our server supports the OpenAI, Anthropic and Ollama APIs and various runtimes: llama.cpp for GGUF models, MLX for MLX directories, the BaseRT engine for.base bundles.

Superfluid was built for multi-agent workloads on one machine:

curl -fsSL https://superfluid.sh/install.sh | sh superfluid serve unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M Enter fullscreen mode Exit fullscreen mode

Eight agents working on 7.5k-token documents, then a chat question arrives after two seconds. Superfluid produces an answer in 0.36s, llama-server takes over 10 seconds. That's a 30x faster response for the same model on the same Mac.

Install for macOS (Apple silicon) or Linux (x86-64, arm64). On Apple silicon and Linux, the BaseRT engine is installed per default, while llama.cpp and MLX are installed automatically when needed.

curl -fsSL https://superfluid.sh/install.sh | sh Enter fullscreen mode Exit fullscreen mode Or cargo install superfluid.

Serve a GGUF from Hugging Face. The tag picks the quantization:

superfluid serve unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M Enter fullscreen mode Exit fullscreen mode Call it with anything that speaks the OpenAI API. Anthropic's SDK and Ollama clients pointed at the server address equally work:

from openai import OpenAI client = OpenAI ( base_url = " http://127.0.0.1:8453/v1 ", api_key = " not-needed " ) stream = client. chat. completions. create ( model = " unsloth/Qwen3.8-27B-GGUF ", messages = [{ " role ": " user ", " content ": " Explain KV caching in two sentences. " }], stream = True, ) for chunk in stream: print ( chunk. choices [ 0 ]. delta. content or "", end = "", flush = True ) Enter fullscreen mode Exit fullscreen mode An MLX model works the same way: superfluid serve mlx-community/Qwen3-4B-4bit.

superfluid launch claude --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M Enter fullscreen mode Exit fullscreen mode launch starts the server if none is running and runs the agent pointed at it. It works for Claude Code, Codex, pi, OpenCode, Hermes Agent, Cline, OpenClaw and the Ollama CLI.

One machine is the head server. Create a token there and serve with a listener for nodes:

superfluid fleet token superfluid serve unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M --http 0.0.0.0:8453 --fleet-listen 0.0.0.0:8454 Enter fullscreen mode Exit fullscreen mode On each other machine:

superfluid node join --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M --token <the token> Enter fullscreen mode Exit fullscreen mode The node finds the head on the local network, pulls the model, and serves it over a link encrypted with the token. The head places whole sessions on nodes and only sampled tokens come back. If a node dies mid-stream, its session is replaced and no token is delivered twice. Nodes can run different runtimes for the same model: BaseRT on a Mac next to llama.cpp on a Linux box.

We measured performance with the native superfluid/tools/suite on an Apple M5 Pro with Qwen3.8-27B at 4 bits, where each server deploys the same runtime.

The serving layer adds no overhead, but increases performance in several scenarios: at one and eight concurrent requests, llama.cpp served through Superfluid matches llama-server on tokens per second. Superfluid batches MLX requests that mlx_lm.server serves less efficiently one at a time. Find every method and evaluation on the website: superfluid.sh/latest/performance/benchmarks.

Try it, break it, and tell us what you find.