dsh-llm-manage

Configuration

Every value lives in the plugin row's config, so your own patch layer keeps overrides across upgrades. Defaults:

- id: llm-manage
  name: 'dsh-llm-manage'
  config:
    selector:
      mode: managed     # 'native' withdraws the shadowed picker
    probe:
      intervalMs: 1200  # delay between deep probes
      timeoutMs: 30000
      batch: 40         # probes accepted per request
    directory:
      ttlMs: 1800000    # how long a gateway directory read is trusted
    fallback:
      enabled: true
      chain: []         # 'route/model' entries, tried first
    usage:
      enabled: true     # local proficiency/heat counters
      seed: true        # one pass over existing session logs at startup
      flushMs: 5000     # counters are batched to disk on this interval

The fallback chain order is: fallback.chain, then pinned models, then favorites, then the remaining models on the same route, then other routes. Models already known to be retired, protocol-incompatible, hidden, or inside their quota cooldown are skipped. At most three automatic switches happen per turn.

Which failures switch, and which never do

The rule is scope, not severity:

Failure	Scope	Switches?	Recorded against the model?
model not found / retired	one model	yes	yes
wrong protocol for this route	one model	yes	yes
per-model quota exhausted	one model	yes	yes
other per-model failures	one model	yes	yes
no adapter registered for this route	the whole route	yes	no
account-wide rate limit	the whole API key	never	no
account-wide quota	the whole API key	never	no

A model-scoped failure moves to another model, because a different model can serve the turn. An account-scoped failure never moves: every model shares that key, so switching would only hide the real problem and burn another request.

A route with no adapter is its own scope. Switching models on that route cannot help — every model there fails the same way — but switching to another provider can, so it still switches. What it must not do is land in the model's health record: error is in the set of reasons that make the fallback chain skip a model, and unlike quota that verdict has no staleness escape, so a single route misconfiguration would have removed an innocent model from every future chain permanently.

A probe is not a routing event

"Unusable" and "move the conversation" are two different conclusions, and only a failed request reaches the second one:

• A failed request (agent/request-error) is the live conversation hitting a wall. If the failure is model-scoped, the turn is retried on the next usable model and a banner names the model it moved for.
• A failed probe is an answer to a question you asked about a model. It records that model's health — so the next real switch skips it — but it does not touch whichever session happens to be using it. Moving a live conversation because of a diagnostic you ran would be switching models behind your back, which is the thing this design forbids.

Concretely, with model-gamma selected and its quota exhausted:

Event	Selected model afterwards
you deep-probe model-gamma, and the probe fails	still model-gamma
a real request on model-gamma fails with per-model quota	model-beta (next usable), banner shown
a real request fails with an account-wide key limit (2008)	still model-gamma

At most three automatic switches happen per turn, and if the chain finds nothing usable the failure is passed through untouched rather than retried in a way that cannot succeed.

The switch happens, and the banner proves it

Two mechanisms move a model, and the difference matters:

• The session's own selection (sessionController.selectModel) is what makes the change real. It validates the target against a registered adapter and appends a durable model/selection event, so the model the conversation uses actually changes and every surface agrees. The saved default model is put back afterwards: an automatic recovery has no business permanently changing which model new sessions start on.
• A per-request override, bound to the exact turn and step it was decided for, is what recovers the turn already in flight. It is needed because the harness applies the session selection during prompt assembly, which happens once per step — before the retry loop — so the session change alone would only take effect on the next turn.

The banner then reports what was proven, not what was decided. A switch is recorded as pending, and only promoted to something you are shown once a committed assistant message names the model that actually produced the response:

What the answering model was	What the banner says
the model we switched to	Model switched, with Switch back
a different model	Switch did not take effect, naming the model that actually ran
no message — the turn ended	Switch attempted, turn not recovered

The earlier version wrote the banner when the switch was decided, so a retry that failed, was aborted, or ran on a third route produced the same "switched automatically" text. It also hardcoded "exhausted quota" as the cause for every failure, including failures that were not about quota at all. Each reason now has its own wording, and the provider's raw message is one clamped line with the full text on hover — inlined in full it was up to 400 characters, which made the banner taller than the conversation around it.

Two controls sit on the banner: Switch back (offered only once a switch was confirmed, since it moves the session's model back) and Stop switching, which suppresses automatic switching for that session until the harness restarts.

Some gateways report a model's quota exhaustion as HTTP 429, which the harness's own classifier maps to a retryable RATE_LIMIT — indistinguishable from an account-wide rate limit. The two need opposite handling, so this plugin reads the gateway's numeric business code out of the error body instead of trusting the mapped code. The code table is at the top of index.js; adjust it there if your gateway numbers things differently.