JDS5 No-BS AI

The router deleted my config file twice. Both times it was working as designed.

By Daniel S. · September 5, 2026

NVIDIA's Personal AI Router turns several machines into one local OpenAI-style endpoint, but it only knows two engines and it insists on starting them itself. Our biggest model runs under vLLM, split across two DGX Sparks, and nothing is going to restart that on a whim. Getting the router to adopt an engine it cannot start took four attempts. Every failure was one line of the router's own source, and none of the lines were bugs.

The problem, concretely

Our two Sparks serve one model between them, tensor-parallel over the interlink (that build is its own post). The head node answers on port 8888. The worker answers on nothing; it exists to hold half the weights. When we put NVIDIA's router (PAIR) on the fleet, every llama.cpp box joined in an afternoon by letting PAIR spawn llama-server under an overridden manifest (part one of this series). The Sparks could not do that. No llama-server, no GGUFs, and about 8 GB of 121 free under the live pair. There was nothing to spawn. The only shape that could work was the one XDA's reviewer sketched for vLLM: tell PAIR the engine is LM Studio, make its start and stop commands do nothing, point its health probe at the endpoint that already exists, and let it route.

I did exactly that at one in the morning and PAIR showed nothing. Two restarts later, still nothing, and no log line anywhere, because the headless unit throws the TUI's stderr away. I reverted and wrote "cause unknown", which is the correct thing to write and the wrong place to stop.

Roadblock one: it will not look unless a file exists

The next morning I read the engine manager instead of restarting it. The presence check opens with this:

// A command-mode engine needs its control CLI. A compatible HTTP endpoint
// alone (for example another OpenAI server on LM Studio's port) is not an
// installation and must not suppress the installer.
if !pathInstalled && st.plat.Runtime.modeOrDefault() != "process" {
    return presenceResult{}
}

In command mode PAIR never probes the port unless one of the manifest's detect paths exists on disk. The bundled LM Studio manifest detects the lms binary, which a Spark does not have, and my override had replaced everything except detect. So PAIR accepted the file, decided LM Studio was not installed, and never sent a single probe. The empty list was correct.

The part that stung: the same override had worked on three other machines that morning. Those manifests pointed detect at the llama-server binary, chosen because it looked tidy, not because anyone knew it was load-bearing. Three green nodes were hiding an undocumented precondition. A green you did not understand is a precondition you have not found yet.

Fix: detect names a path that exists. I used the docker binary, since docker is what actually holds the engine.

Roadblock two: the port you choose is a port it will erase

Attempt two lasted five seconds. PAIR started, and five seconds later the override file was gone. Nobody ran the undo. The engines directory's timestamp said PAIR did it.

setport.go:

// Bundled engine: back to default ⇒ drop the override; else persist
// just the delta so bundled upgrades to everything else still apply.
if def == port {
    if err := os.Remove(path); ...

At startup PAIR pushes its two engines onto its own reserved backend ports; for LM Studio that is 1235. My manifest said 8888, vLLM's real port. PAIR called set-port back to 1235, and a port equal to the bundled default means "no override needed", so it deleted the file. The three working nodes survive this for the same accidental reason as before: their overrides already said 1235.

So the manifest must say 1235, and something on 1235 must answer as the engine. systemd ships a socket proxy for exactly this. A socket unit listens on loopback 1235 and hands connections to systemd-socket-proxyd, which forwards to 8888. PAIR probes 1235, sees vLLM's /v1/models, and adopts. Loopback only, idle-exits, nothing new exposed.

Roadblock three: it cannot see your forwarder, so it moves the engine away from it

Attempt three also lasted five seconds. This time the file was not deleted. It was replaced by three lines:

{ "engine": "lmstudio", "runtime": { "port": 1236 } }

Same code path, other branch: a non-default port is persisted as a delta, and for a bundled engine the delta overwrites the whole file. Every custom key was gone from disk. The question was why PAIR wanted 1236 at all, when 1235 was answering perfectly.

The answer is in the UI broker, which runs a port-ownership step once at startup, before it spawns the LM Studio proxy. It asks the engine manager for status while probing port 1234 only, the compatibility port where a stock LM Studio would sit. Nothing is on 1234 at that moment. So the engine reads "not running", and the plan is one line:

if !st.Running && !available(st.Port) {
    backend := nextAvailablePort(managedLMStudioBackendStart, available)

Not running, and its port is occupied? Move it to the next free port. My forwarder was invisible to the check by construction and guilty of squatting by the same construction. The router was defending its own port against an unknown process, which is what you want it to do.

Two shortcuts fail on the same code. XDA made the file immutable; here that makes set-port fail, and the broker's response to a failed move is to give up the compatibility port and put the proxy itself somewhere else. That breaks the 1234 endpoint, not just the engine. A forwarder on 1234 gets adopted by the probe and then fails the move in the same way.

The real rule is narrower than "use 1235": the configured port must be free at the instant the broker looks, and set-port must never be invoked on this engine. The three working nodes pass because PAIR spawns their engine after the broker looks.

Fix: the forwarder binds 1235 only after the broker is done. The precise signal is the LM Studio proxy listening on 1234, which the broker spawns only after the ownership step. The socket unit gets After= and PartOf= the router's unit, and an ExecStartPre that waits for 1234, then five more seconds. A drop-in on the router's unit pulls the socket back in after every restart. If the gate never fires the socket never binds, so the failure direction is "nothing happens", not "config clobbered".

Attempt four

File intact at 2846 bytes a minute after start. Model listed. The engine on the dashboard. A chat request to the head node's own 1234 naming the model:

content: 'routed'   finish_reason: stop   11 -> 63 tokens   1.2 s

Then the one that matters. A workstation across the room, which holds no copy of the model, posted to its own 1234 naming the same id:

content: 'remote'   finish_reason: stop   21 -> 242 tokens   6.2 s

PAIR looked up which node advertises that model, sent the request over its pinned mTLS ingress to the head node, the forwarder handed it to vLLM, both Sparks computed it, and the answer came back as if it were local. The 6.2 seconds are not routing overhead; the model is a reasoning model and spent 240 tokens thinking before it said one word. Quote a latency without its token count and you have quoted nothing.

The recipe, in order

The files below, with an install | verify | uninstall script, are on GitHub as pair-adopt (Apache-2.0). The steps are what the script does.

  1. Confirm the engine already answers the router's two probes on loopback. GET /health and GET /v1/models returning 200 on 127.0.0.1. If it only binds a LAN address you are done before you start; PAIR probes loopback and nothing else.
  2. Write the override under the engine name PAIR already routes to. lmstudio, command mode, start and stop both /bin/true, bin unset so PAIR owns no process, detect a path that exists, port equal to the bundled default (1235), ready and health probes on /v1/models, model listing re-pointed to data[].id, pull and delete disabled. Validate the JSON with a real parser.
  3. Forward loopback 1235 to the engine's real port with a systemd socket plus systemd-socket-proxyd. Make the socket After= and PartOf= the router's unit, gate its start on the router's proxy being up on 1234, and add a Wants= drop-in so it follows every restart.
  4. Make sure 1235 is free, then restart the router. Any forwarder already running must stop first or the broker will move the engine and rewrite your file.
  5. Verify in this order: the override is still your file (check the size), the socket is active and 1235 answers as the engine, the router lists the model (an empty list in the first minute is the poll interval, not a failure), a completion through 1234 locally, a completion through 1234 from a node that does not have the model. Reasoning models need a few hundred max_tokens or the reply is empty and looks like a routing fault.
  6. Undo is delete one file, remove two units and one drop-in, restart the router.

What it does not do

The verdict

Four attempts for one file and two units sounds like a fight with a bad product. It was the opposite. Every one of the three roadblocks was PAIR refusing to do something dangerous: probe an engine that was not installed, keep a port override it had not been asked to keep, leave an unknown process on a port it was told to own. The router was right each time and told me nothing each time, and the two facts are connected: the headless unit discards the logs where each refusal is written. The turn came when I stopped restarting and started reading. Two blind restarts taught nothing; one read of status.go explained every empty list that night.

If you have an engine the router cannot start and you want it on the same endpoint as everything else, it is one file and two units, in that order, with the port free when the broker looks. Expect the router to delete your config at least once while you learn where its edges are. That is not it misbehaving. That is it showing you the rule.

Sources