Understanding AI Beyond the Headlines (1): AI Explained

Posted on Nov 3 • Originally published at Medium

Models, Training, Inference — and Where It All Runs*Part 1 of 4 — Understanding the technology behind the headlines.*

AI ExplainedYou ask an AI assistant a question. A few seconds later, you have an answer.

Simple enough. But what happened in between? Was it searching the internet? Did it learn something from your question? And was the work happening on your computer or somewhere else?

We often call all of this “AI,” even though several different things are involved. Understanding the basics makes it easier to see what these tools can do — and what we are actually trusting them with.

The app you open and the model answering you are different things.

The app handles the conversation, lets you attach documents, and displays the reply. The model is the part that generates the answer.

Think of a model as something that has already gone through an enormous amount of practice. During that process, it developed patterns that help it work with language: explaining a topic, translating a sentence, or suggesting how to finish a piece of code.

It doesn’t need to search the internet for every answer. An app may offer web search, but that is an additional tool. A convincing reply on its own is no guarantee that anything was checked.

Before a model can be useful, it has to be trained.

During training, the system repeatedly processes examples and adjusts its internal settings. Over many repetitions, those adjustments shape what it can do.

Building a large model this way can require substantial computing power. An existing model can also receive additional training to improve particular capabilities.

But training is not what normally happens every time you type into a chatbot.

Once a trained model receives your question and generates an answer, you are using it. The technical name for that is inference.

Ask it to translate an email: inference.

Ask it to explain a confusing paragraph: inference.

Tell it that the explanation was too long and ask for a shorter one: still inference.

That last example can look like learning. The model responds to your correction, so something clearly changed. Usually, what changed was the information available in the conversation — not the model itself.

An app might also remember your preference for short answers and include it in future conversations. Keeping that preference does not require retraining the model.

Whether a provider uses conversations for later training is a separate matter.

A chat window doesn’t tell you where the answer is being generated.

With cloud AI, the request goes to remote servers. They run the model and return the result.

With local AI, the model runs on your own device.

The same basic arrangement applies to both: software gives the model something to work with, and the model produces an output. The difference is where the computing happens.

A model can be trained in a data center and later used on a laptop. It does not have to stay where it was trained. And data centers also run models for everyday requests — they do far more than training.

If the necessary model and software are on your computer, some tasks can work without an internet connection. Summarising a document stored on that computer, for example.

But a local application might still use online services. Asking it for today’s news requires access to fresh information, even if the model itself runs locally.

So “local AI” is useful information, but it does not answer every privacy question. You still need to know whether the application sends your files or requests elsewhere.

So far, we have talked about producing answers.

Now imagine asking an assistant to read an email, check your calendar, and arrange a meeting. It needs more than the ability to write a good reply. It needs software connected to those services, and permission to use them.

The model may help decide what to do next. The surrounding software carries out the actions through the tools available to it.

And agents have a history that starts long before today’s AI assistants.

Next: Agents existed before AI. What changes when they start using it?