A tool that tells you which local LLM your PC can actually run
For my final degree project I wanted to know what hardware a small company would need to run language models locally. My method was to download a model, watch it crash or crawl, and try another quantization. So I built Jaull to do that work properly. Jaull is an open-source Python tool with a termi
For my final degree project I wanted to know what hardware a small company would need to run language models locally. My method was to download a model, watch it crash or crawl, and try another quantization. So I built Jaull to do that work properly.
Jaull is an open-source Python tool with a terminal UI. It detects your CPU, RAM, GPU and backends, asks a few plain questions (what you'll use it for, what matters most, languages, commercial use) and searches Hugging Face using metadata only. You get up to five concrete plans, each with an exact file, runtime, backend and GPU offload, and an explanation of its position.
Jaull estimates memory by component (weights, KV cache, overhead, margins) and treats RAM and VRAM as two pools, so it can tell you which one you'll run out of. If you want to check, you can run, benchmark or score any plan on your machine. Each result is saved with the file's SHA-256, the machine and the protocol, and only results that match a plan can reorder recommendations. A benchmark of a Q5_K_M file says nothing about the Q6_K next to it.
The 1.2B model beat the 1.5B one by a wide margin on instruction following, which is a good reason not to rank by parameter count.
My first offload policy left most of the GPU unused. On Qwen2.5-7B it gave 21.5 tok/s, and putting every layer on the GPU gave 59.4 for 133 MB more VRAM. After fixing the formula, the same run reached 44.9.
I also built a tokens/second predictor. A simple version estimated 170 tok/s for a model that measured 18. I removed predictions from the UI, and Jaull now shows a speed only after measuring it.
Some numbers can't be measured at all. On a GeForce that drives your display, Windows won't report VRAM per process, so Jaull says "not comparable" there.
git clone https://github.com/Ton-Llop/Jaull.git cd Jaull uv sync --locked uv run jaull Enter fullscreen mode Exit fullscreen mode You don't need llama.cpp to get recommendations, only to run plans. I haven't tested Intel (Vulkan) or AMD support on real hardware yet. If you try it, I'd like to hear what it recommends for your machine.