LanesLanes
Docs/Desktop/Agentic Coding/Local LLMs

Local LLMs

Run coding agents against a model on your own machine. Lanes manages Ollama end to end, sizes models to your hardware, and sets a context window that survives a real agent prompt.

Lanes can run a coding session against a model on this machine. Nothing leaves it: no tokens bought, no prompt sent anywhere. Ollama is the first provider Lanes manages directly, so installing it, starting it, pulling models, and wiring a session to it all happen from one page.

Open Settings, then Agentic Coding, then Local LLMs. It is marked Research Preview: it works, and the rough edges are the ones we most want to hear about.

Managing the Ollama Server

The status card tells you what is true right now: Serving on localhost:11434, Installed but not running, or Not installed, with the version when there is one. An emerald Sessions use this chip appears when Ollama is the provider your sessions currently launch against.

  • Install installs Ollama with Homebrew.
  • Start server and Stop server use the right lever for how Ollama got here. A Homebrew formula is driven through brew services; the desktop app is opened and quit as an app. You do not have to remember which one you have.
  • Recheck re-runs the detection.
  • Uninstall is a two-step button that stops the service and removes Ollama. Models you pulled stay in ~/.ollama, so reinstalling does not re-download them.

Every one of these runs in a real terminal, so you see exactly what happened.

Your Installed Models

The Installed list shows what is on disk: size, parameter count, quantization, capabilities, and the context length the model was trained with.

It works with the daemon stopped. When Ollama is not serving, Lanes reads the model manifests in ~/.ollama and sums the layer sizes rather than showing you an empty page. Manifests carry no parameter count or quantization, so those columns read -- until the daemon is up again.

Finding a Model

Browse the library holds two things.

Recommended is a short curated list, with a checked download size and a minimum memory figure per model, compared against the RAM this machine actually has. It is the only list carrying a hardware claim, because ollama.com does not publish download sizes.

Below it is the full ollama.com library, searchable, with capability and parameter chips but no sizing advice. If you already know what you want, Pull any model by name takes any reference, including hf.co/user/repo:tag.

Both lists share one table: Model, Details, Size, and Fit.

The Fit Column

Fit is the answer to "will this work on this machine", before you spend the download:

  • A context figure, for example 128k context, means it fits with room.
  • Fits at 64k, little headroom means it works, but you are close to the edge.
  • Needs ~38k, this machine fits ~24k means the model cannot hold a real session prompt here. See below for why that matters.
  • No tool support means the model cannot run an agent session at any context size. An agent that cannot call tools cannot edit files.

Fit warns, it does not block. Pull it anyway if you know what you are doing.

Why Context Is the Thing That Decides

A real session sends roughly 38,000 tokens before you type anything: the harness's own instructions, its built-in tool definitions, and the tool definitions of every MCP server you have connected.

Ollama picks a context size from available memory, commonly far below that, and when a prompt overflows it truncates instead of failing. It keeps the head and the tail and drops the middle. What reaches the model is tool-definition JSON with the instructions cut out, so the model answers with something that looks like nonsense and nothing anywhere reports the loss.

So Lanes sizes the context itself and bakes it into the model.

  • On pull, Lanes creates a variant tagged -lanes with num_ctx set, using your total memory and that model's own key/value geometry to pick the number. It reads the running daemon's cache setting too, since that halves or doubles the per-token cost.
  • The variant shares its data with the base model, so the Size column reads shared and it costs nothing extra on disk. It survives reboots and service restarts, and ollama rm undoes it.
  • The re-bake action on a row rebuilds the variant when the estimate changes, for instance after you close something large or change the daemon's cache setting.
  • The Context window field overrides the estimate. Leave it on Auto unless you have a reason. The subtext tells you what the loaded model is serving with right now.

If a model does not fit, the lever that helps most is connecting fewer MCP servers. Their tool definitions are a large share of that 38,000.

Using a Local Model in a Session

Once Ollama is serving and has at least one model, Lanes registers it as a Gateway provider automatically, and sessions start using it. Three details worth knowing:

  • Registration waits for the first model. Activating an empty daemon would break every new session, so Lanes does not.
  • It never overrides a provider you picked yourself, and it matches on the base URL, so a profile you renamed or hand-made for localhost:11434 is adopted rather than duplicated.
  • If you delete the profile, it stays deleted. Nothing re-adds it behind your back. The page shows a Not in Gateway row with an Add to Gateway button if you want it back.

Ollama is the one provider Lanes can interrogate, so the session picker's Model row lists the tags your daemon really has. Switching to Ollama rewrites a leftover first-party model id to a real tag, and switching away clears the Ollama tag. Default is hidden while a local model is active, because it is the one guaranteed-wrong choice: it would send a model id your daemon does not serve.

There is also a shortcut in the app header, next to the keep-awake button, that shows live daemon state and can start or stop the server without opening Settings.

Why LM Studio Is Not Managed

LM Studio serves the same wire format and works fine as a Gateway provider, as does a vLLM box on your network. What Lanes cannot do for them is the lifecycle: install, start, stop, and pull a model with a checked size. Ollama exposes all of that to a command line and a local HTTP API, which is why it is the one that gets a page.

Point a profile at anything serving the Anthropic Messages format and it will work. You just do the driving.