Local AI models through Ollama #20

What

Run small language models on the Agent's GPU, and use them from the Client as if Ollama were local.

Plan, In Order

1. Automatic port forwarding #9, automatic port forwarding. Built: run and attach forward Ally's ports, and ally forward 11434 opens Ollama's port on request.
2. Local models: offer Ollama in the tools step #38, offer Ollama in the tools step, based on the Agent's GPU. Done in feat: offer Ollama and OpenCode in the tools step #131.
3. Local models: ally models #39, ally models: list models, show what fits in free VRAM, and pull with a warning.
4. Local models: show the loaded model in the menu bar #40, show the loaded model and its VRAM use in the menu bar.
5. Local models: offer OpenCode in the tools step #60, OpenCode in the tools step. The code landed in feat: offer Ollama and OpenCode in the tools step #131; only its real test is left.
6. Local models: a chat window for the Agent's models #57, a chat window for the Agent's models.

Found On October 27, 2026

• archbox runs ollama-cuda with ollama.service enabled, and Ollama finds the GPU. Models live in /var/lib/ollama.
• OpenCode 2.0.12 is installed from pacman and opens inside ally attach.
• No model is pulled yet, so Local models: ally models #39 and Local models: offer OpenCode in the tools step #60 wait for one.
• On October 18 the service was stopped, so ally models must still notice a stopped Ollama, print the fix, and ask before starting anything. Since feat: warn when the Agent's Ollama is stopped or has no GPU support #152 the tools step warns about this too.

Rules

• Ollama runs only on the Agent, and listens only on the Agent's 127.0.0.1. The Client reaches it through the ssh forward from Automatic port forwarding #9.
• Ally wraps the ollama program. It never runs models itself.
• Nothing is installed without showing the command and asking once.

Done When

• On a Client with no GPU, a tool pointed at localhost:11434 uses a model running on the Agent's GPU, and ally models shows what is loaded.