Adam Blackwell

Local inference · reused hardware · explicit trust boundaries

Two Macs, one local model pool

NVIDIA PAIR can let an older Mac hold the model while a newer one keeps the editor, browser, and build tools. The useful part is not raw speed. It is separating two workloads that fight over the same memory.

ClientApps talk only to localhost
HostEngine and model weights
StatusExperimental, plain completions only

An old laptop can be more useful as a quiet model host than as a second complete development machine.

The client sends an Ollama-compatible or OpenAI-compatible request to its own loopback interface. PAIR can forward that request over the LAN to a paired host, where an inference engine owns the model weights. The application still sees one local endpoint; the expensive memory footprint lives elsewhere.

The point is resource separation: keep the model on the machine that has spare memory, and keep interactive work on the machine in front of you.

The request path

Client app OpenAI request to 127.0.0.1:11434 PAIR proxy + mTLS over the LAN Host proxy
Host proxy Loopback request to the engine Ollama loads the selected model Response

Discovery uses mDNS, so both machines need to be on the same trusted network without guest-network or access-point isolation. The client should not run its own engine unless local serving is intentional; otherwise a request that looks healthy may never leave that machine.

Three jobs, three skills

1 · Cluster lifecycle

/local-ai:pair-setup

Preflight, pinned installation, pairing guidance, read-only diagnosis, routing verification, and complete removal.

2 · Model fit

/local-ai:local-model-pick

Choose a conservative Ollama model from total and currently available memory before downloading weights.

3 · Client routing

/local-ai:litellm-setup

Create an authenticated loopback gateway that names the local route and can translate client protocols.

The limits matter more than the happy path

Structured tool calls are currently broken through PAIR. The upstream proxy can return an intended tool invocation as ordinary text instead of a structured tool_calls object. That makes plain completions useful, but blocks the agent workflow most people are tempted to try first. Check issue #94 before installing for an agent use case.

A healthy endpoint is not routing proof. Bare Ollama can answer /v1/models on the same port. The authoritative evidence is a real request followed by a PAIR Jobs card that says Ran on and names the other machine.

PAIR changes the machine. The desktop application installs a persistent privileged helper for firewall management. Pairing begins with a six-digit PIN. PAIR's own security documentation treats the LAN as a meaningful trust boundary and warns against exposing local inference endpoints to untrusted interfaces.

Safe pilot shape: use two machines you own or are authorized to administer, a trusted private LAN, public or synthetic prompts, and an explicit uninstall test. Do not forward the ports or put a public reverse proxy in front of them.

Why put LiteLLM in front?

PAIR chooses which machine serves a request. It does not choose between models or translate every client protocol. LiteLLM is a separate layer: it exposes a named route such as local-pair, keeps credentials in environment variables, and can present OpenAI-compatible or Anthropic-shaped endpoints to local clients.

The plugin writes a secret-free starter configuration and can install a pinned LiteLLM version inside a dedicated virtual environment. It binds to loopback and requires a master key. That translation layer is useful, but it cannot repair PAIR's structured-tool-call defect.

Install the plugin

Add the marketplace, install local-ai, then run the PAIR setup on the host Mac first.

Claude Code

1. In your terminal

claude plugin marketplace add adzuci/claude-plugins
claude plugin install local-ai@adzuci-plugins

2. In a session, on the host Mac

/local-ai:pair-setup host verify
/local-ai:pair-setup host install

Codex

1. In your terminal

codex plugin marketplace add adzuci/claude-plugins
codex plugin add local-ai@adzuci-plugins

2. In a session, on the host Mac

/local-ai:pair-setup host verify
/local-ai:pair-setup host install

host verify is read-only; on a fresh machine it should report a FAIL because PAIR is not installed yet. host install runs a read-only --check first, lists every planned write, and asks for your confirmation before applying anything. Then repeat on the client Mac with client install and client verify. Do not run host install for agent or tool-call workloads while issue #94 is open. The PAIR installer currently targets Apple Silicon macOS. Model names, releases, and upstream limitations are checked again before mutation.