JDS5 No-BS AI

NVIDIA's AI router says it supports two engines. It routes to a third if you lie to it in one file.

By Daniel S. · September 5, 2026

NVIDIA Personal AI Router (PAIR) is a beta that turns a house full of GPUs into one local OpenAI-style endpoint. Officially it speaks to Ollama and LM Studio. We put llama.cpp behind it on a Windows gaming PC in an afternoon, and the interesting part is not that it worked. It is where it pushed back.

What it actually is

PAIR is a small Electron app plus thirteen Go services. Every machine runs the same software; there is no server. Two reverse proxies on each box claim the ports your apps already know, :11434 for the Ollama API and :1234 for the OpenAI-shaped one, and the real engines get moved behind them. That holds only when the port is free: on a box already running Ollama, Ollama keeps :11434 and PAIR settles for :11435, so a client left pointing at the old port bypasses the router entirely. Nodes find each other over mDNS, pair with a six-digit PIN, and talk over mutual TLS pinned per peer. When an app asks the local :1234 for a model, the proxy looks at which nodes currently advertise that exact model id and forwards the request there.

Two things it does not do, and the docs say so: it does not pool GPUs or split a model across machines, and it does not serve models itself. It adds concurrent capacity, not per-request speed.

The engine list is the catch. Ollama and LM Studio. That is the whole roster, and if you already run llama.cpp's llama-server because you like owning your quantisations, you are not on it.

Why "just add llama.cpp" is not one setting

We read the source before touching the box, and the answer is split in two.

The routing layer is closed. The desktop app's engine type is a literal union of two strings. The OpenAI proxy subscribes to discovery for nodes tagged lm, the Ollama proxy for nodes tagged ol, and the broker registers exactly those two tags off a five-second poll of the two engines it knows. A note in the constants file says llama.cpp was once carried as a placeholder and was removed. The registry will accept a manifest for an engine it has never heard of. It will not matter: both proxies subscribe only to nodes tagged lm or ol, so nothing would ever route to it. (Whether an unknown engine is ever even probed is a separate gate, and it is roadblock one of part two of this series.) XDA's reviewer hit precisely this wall with vLLM.

The engine layer is open. The engine manager is manifest-driven: each engine is a JSON file that says how to detect, start, stop, probe, and query it. Better, it reads a per-user engines/ directory and deep-merges any file there onto the bundled manifest with the same engine name. Objects merge, arrays and scalars replace, and a JSON null clears a key.

Put those together and the move is obvious and slightly dishonest: keep the name lmstudio, replace the body. PAIR thinks it is running LM Studio. What it is actually running is llama-server.

What the file says

The override is about sixty lines. The parts that matter:

The four roadblocks

1. A listener that is not the engine blocks the engine. LM Studio's own server was already on the backend port. PAIR probed it, did not get the answer it expected, and correctly refused to spawn over it. It also refuses to kill a process it did not start. You stop LM Studio yourself first. This is the right behaviour and it cost fifteen minutes.

2. Manifests are read once, at start, and the app does not quit when you ask it to. Every edit means a PAIR restart. A polite close hides the window to the tray. A tree kill of the main process took down the broker and all thirteen services cleanly, and the relaunch rebound every port and rejoined the ten-node cluster without a second PIN. Fine, once you know.

3. The port control deletes your file. In the source, setting an engine's port back to the bundled default removes the override file entirely, on the theory that an override with nothing in it is clutter. Ours has everything in it. XDA's reviewer made the file immutable at the filesystem level to survive this; we simply never touch that engine's port from the UI and wrote the rule down.

4. The tooling ate the backslashes. Windows paths in JSON need doubled backslashes. The shell heredoc we generated the file with collapsed them, producing a file the registry would have skipped with a one-line warning and no visible symptom except "the engine never appears". We caught it by validating the JSON before copying it into place. Generate config from a real language, not a shell.

What it does now

A chat request to the local :1234 naming one of the GGUFs comes back in about four seconds cold, under a second warm, and PAIR's workload ledger records it as scheduled on this node. Video memory goes from 0.8 GB idle to 5.6 GB with the 4B model loaded and back to 0.7 GB after an unload. The merged model list on :1234 shows the four local GGUFs next to an embedding model a peer is serving from real LM Studio. Two different engines, one slot name, one endpoint, and the router does not care.

One pleasant surprise: the llama.cpp router forwards the parent's --ctx-size and GPU-layer flags to each child instance. Read off the child's command line, not assumed.

What it does not do

How we actually run it now

  1. The override file lives in PAIR's per-user engines/ directory, with a reference copy and a README beside the binaries.
  2. Models go in one directory; the file name is the model id; add one, bounce the engine.
  3. One resident model at a time. The engine switch in PAIR is the "give me my VRAM back before the game starts" button.
  4. Never change that engine's port in the UI.
  5. Every manifest edit is a restart. Validate the JSON first.

The verdict

PAIR's public surface is narrow on purpose; a beta that only promises two engines can make both of them work. Underneath, NVIDIA built an engine manager general enough that the third engine is one file away, and the routing layer is the only thing standing on the name. That is a good sign for where this is going and a fair warning about where it is: an unsupported edge that works today because the product was built cleanly, not because anyone said it would.

If you already run llama.cpp and want one endpoint across several machines, this is worth an afternoon. Keep the file in version control and expect to revisit it after every update.

Sources