How to Run a Free Local AI Model with LM Studio and Use It from Python

Posted on Nov 5 • Originally published at garystow.co.uk

Most of the AI work my home automation teams do runs on a model on my own PC, not in the cloud. My website team's weekly review, my news digest and my assistant, which I named Wade, all talk to a local model through LM Studio, so none of that work costs me a penny in tokens. This guide shows you how to set up the same thing from scratch and get your first answer back in Python. The screenshots are from my own copy of LM Studio, and the red boxes show you exactly where to click.

This is for marketers and other non-developers who want to try AI on their own computer, keep their data at home, or automate small jobs without paying per request. You don't need to know how to code, because you will copy one short script and run it. By the end you will have a model running on your PC, a local server that other programs can talk to, and a Python script that sends it a question and prints the answer.

Cost: nothing, because LM Studio and Python are both free.

You will see model names like GPT-OSS 20B or Qwen2.5 Coder 7B. The number with a B is the model's size in billions of parameters. Parameters are the numbers a model learned during training, and they are what it uses to work out its answers. As a rough rule, more parameters means a more capable model, but also a bigger download and more memory needed to run it. A 20B model has 20 billion parameters.

Models are usually shrunk for home use with a process called quantization. You will see labels like Q4_K_M next to a model, which means each parameter has been squeezed into about four bits. In practice, the download size LM Studio shows you is the best guide to how much memory the model needs.

The model has to fit into memory to run, and it runs far faster when it fits into your graphics card's own memory, called VRAM, rather than your PC's normal memory.

What you should see: a figure such as 8GB, 12GB or 16GB. Mine is an AMD Radeon RX 7800 XT with 16GB, alongside 32GB of normal system memory. Write your number down, because you will use it in Step 4.

Python is the programming language your script will use.

What you should see: a line starting with the word Python followed by a version number. If you see that, Python is ready.

On Windows I always type py rather than python to run Python, because the Microsoft Store can grab the word python and open the Store instead of running your script. Every command in this guide uses py.

LM Studio now makes two different apps, and the download page offers both. LM Studio is the classic app this guide uses, with the model library and the local server. Bionic is a newer, separate app from the same team, built as an AI agent. LM Studio's own announcement describes Bionic as a separate app and says you can keep using LM Studio alongside it for advanced configuration, so download the classic app for this guide.

What you should see: the LM Studio window, with a column of icons down the left-hand side. Here is what each one does, because you will use them in the next steps.

In my screenshot the model is 17.74GB and the badge says Partial GPU Offload Possible. That means it is too big for my 16GB graphics card on its own, so LM Studio would run part of it on the card and the rest in normal memory. It works, but it runs noticeably slower.

My simple rule is to pick a model whose download size is a few gigabytes smaller than your VRAM from Step 1, because the model needs some room to work as well. That is why my main model is openai/gpt-oss-20b, an 11.28GB download that fits comfortably on my 16GB card. For help with code I use qwen2.5-coder-7b-instruct, which is only 4.36GB. If you have 8GB of VRAM, start with a 7B or 8B model. Look for the word instruct in the name, because those models are built to follow instructions.

What you should see: a reply appears in the chat. The picker also shows Memory Consumption, marked 2, so you can see how much memory the loaded model is using, and box 3 lists the size of every model you have downloaded. If you get a reply, the model works on your machine, and everything from here is about letting other programs use it.

The local server lives in LM Studio's developer tools, which are hidden until you switch them on.

Box 5 shows the types of connection LM Studio supports: its own LM Studio API, OpenAI-compatible and Anthropic-compatible. The OpenAI-compatible option is the one this guide uses. It means LM Studio understands the same requests as OpenAI's service, which is why so many tools can use it.

Before writing any code, prove the server answers.

(Invoke-RestMethod -Uri "http://localhost:1234/v1/models").data.id Enter fullscreen mode Exit fullscreen mode What you should see: a list of your models' identifiers, one per line, such as openai/gpt-oss-20b. Write down the one you want to use exactly as shown, because your script must match it letter for letter.

py -m pip install requests Enter fullscreen mode Exit fullscreen mode requests is a small, popular library for talking to web addresses, and it is all you need here. When it finishes, you will see a line starting with Successfully installed, or a message saying it is already installed.

import sys import requests sys.stdout.reconfigure(encoding="utf-8", errors="replace") URL = "http://localhost:1234/v1/chat/completions" MODEL = "openai/gpt-oss-20b" question = "Give me three ideas for a blog post about PPC reporting." response = requests.post(URL, json={ "model": MODEL, "messages": [ {"role": "system", "content": "You are a helpful marketing assistant. Answer in British English."}, {"role": "user", "content": question}, ], "temperature": 0.3, }, timeout=400) if response.status_code >= 400: print("LM Studio said:", response.text[:1000]) response.raise_for_status() print(response.json()["choices"]["message"]["content"]) Enter fullscreen mode Exit fullscreen mode In PowerShell, move into that folder by typing cd C:\Projects\local-ai and pressing Enter. Run the script by typing py ask_local_ai.py and pressing Enter.

What you should see: after a short wait, three blog post ideas print in the window. That is your PC answering, with nothing sent to the cloud.

A few lines in that script are there because of problems I hit, and they are worth keeping. The sys.stdout.reconfigure line stops a crash on Windows when the answer contains characters such as curly quotes or emoji. The timeout=400 setting gives the model time to finish, because local models are slower than cloud ones. Printing response.text when something fails shows you LM Studio's own explanation, which is far more useful than a bare error code.

Once the basic script works, change the question line to something you do every week, such as summarising a pasted report or tidying up a list of keywords. My website team reads its own data files, puts them into the question, and asks the local model to spot what changed, and that whole review runs every Friday evening without costing anything.

My biggest disappointment with local AI is that the really strong models need much better hardware than most of us have at home. The models that do fit are still more than good enough for everyday jobs like these.

If something isn't working and the list above doesn't fix it, paste this prompt into Claude, Copilot or ChatGPT. Fill in the square brackets, and attach a screenshot of LM Studio or the PowerShell window.

I am following a beginner's guide to run a local AI model in LM Studio on Windows and call it from a Python script. I am not a developer, so please explain each fix in simple numbered steps and tell me exactly where to click or what to type. Where I am stuck: [which step, for example "Step 10, running py ask_local_ai.py"].

The exact error or what I see: [paste the full error message from PowerShell]. My setup: Windows [10 or 11], graphics card [name] with [number]GB of VRAM, LM Studio version [shown at the top of the window], model [identifier from Step 8]. The Local Server page says: [Status: Running or not, and the Reachable at address].

What I have already tried: [anything you have changed]. Ask me questions if you need more information before suggesting a fix. Enter fullscreen mode Exit fullscreen mode Where to go next If you want the bigger picture of why I run AI at home, read why I run AI at home, and LM Studio tips for beginners.

For the cost side, local AI without the token bill explains what running locally saves, and build your own local AI assistant shows how I turned this into Wade.

I write about practical AI for marketers and building with free tools at garystow.co.uk. This piece first appeared there.