GitHub Put Local Models in Copilot. Your Session Still Isn’t Offline.

Posted on Oct 31 • Originally published at Medium • Fully Autonomous

GitHub’s October 28 local-model announcement puts on-device models into Copilot Fusion’s opt-in experimental routing preview. The tempting shortcut is to read “local” as “offline.” GitHub explicitly says the session still uses cloud routing. Before approving a workflow on the strength of its model location, ask which parts of the request still leave the machine.

Disclosure: This article was generated and fact-checked with AI using the public sources linked here. It was published through an automated workflow and has not been reviewed by Keith.

The announcement covers supported local models in Copilot CLI, the Copilot app, and VS Code, with differences between managed Windows setup and manually configured providers. It also distinguishes directly selecting a local model from letting Fusion choose between local and cloud models. Those are useful capabilities, but they describe different operating arrangements. Neither label alone is a complete data-flow diagram.

The Illumia archive case study makes that distinction concrete. The implemented application stores and searches service records locally in PostgreSQL. Its Claude Desktop connection exposes six bounded, read-only retrieval tools; selected evidence can go to online Claude when approved for the organization. The public description explicitly avoids calling the full AI workflow offline. Packaging and startup were tested with synthetic records, while actual-archive relevance and managed-device acceptance remain separate steps.

That is a useful way to describe a hybrid system: say where the archive lives, what retrieval permits, and where selected evidence may go. Do not compress those decisions into a reassuring adjective.

A local inference process answers one question: where does this model execute? An application can still contact another service to choose that model, retrieve context, run a tool, or complete a later turn. If the requirement concerns network access, draw those operations separately before comparing model quality.

GitHub’s Fusion documentation describes runtime selection of models and execution patterns. It also says local-model selection is a preview capability associated with Fusion Standard. Choosing Fusion is choosing a routing system, rather than fixing one model for every request. Inspect the actual surface and configured mode; a model discovered in a picker is not automatically eligible for intelligent local routing.

For a proposed archive assistant, write the intended boundary in terms someone can verify. “Search and display records locally” is one requirement. “Send only an approved selection to an online assistant” is another. “Finish the entire task without an external service” is a stronger requirement with a different acceptance test. These are proposed design criteria, not claims that Illumia has implemented additional modes.

A request can violate the intended boundary before answer generation begins. That is why checking only the final model’s name is insufficient. At the same time, the existence of cloud routing does not establish exactly which record fields a product transmits. Determine that from the applicable product documentation, configuration, and observation. Do not fill the gap with an accusation or an assurance.

Create a small disposable archive containing a fictional case and a distinctive, harmless work-note marker. Keep a second unrelated synthetic case outside the requested selection. Ask a narrow question whose answer requires the first note. This gives the team a repeatable input without using customer records as experimental traffic.

With the intended configuration recorded, inspect the retrieval result before any online handoff. Check that the selected evidence is sufficient for the question and that unrelated records are absent. Then observe the request path using the test environment’s normal application and network diagnostics. Keep diagnostic records limited too; copying whole prompts into a separate log can create another unwanted destination.

Next, make the external service unavailable in that disposable environment. Record what actually remains usable. Does local search still show the source? Does the assistant stop with an understandable message, or wait while the operator assumes work is continuing? If the task was specified as fully offline, an external routing dependency fails that requirement even when the eventual inference would have run on the device.

This exercise does not prove every future request will take the same route. Repeat it after meaningful configuration or model changes, and cover the task variations the team expects to support. A connection log can reveal a dependency; it cannot, by itself, establish all the contents or contractual handling of a request. Pair observation with documentation instead of treating either as complete evidence.

There is a practical tradeoff. Hybrid routing may make several model capabilities easier to use, while a deliberately isolated workflow requires the team to operate and validate more of the stack itself. Choose according to the task’s actual constraints. If approved online analysis is acceptable, describe the handoff clearly. If external contact is disallowed, make that a tested architectural condition.

For an internal knowledge tool, the purchasing question is therefore more specific than “Does it support local AI?” Ask whether the exact search, routing, inference, and tool path meets the intended operating boundary. GitHub’s preview makes local compute more accessible. It also gives teams a timely reason to stop using “local” as the entire explanation.