DSH Model Probe

Persistent model-health monitoring for the DeepSeek Harness (DSH) Web UI.

DSH Model Probe discovers the models advertised by active DSH providers, probes them through DSH's own LLM runtime, keeps a persistent Alive / Dead health ledger, observes failures from normal DSH model usage, and automatically rechecks unhealthy models.

This is a third-party DSH plugin maintained by AGSQ11. It is not an official DeepSeek project.

Screenshots

Alive models

Dead models and retry diagnostics

Features

• Native DSH Web UI integration under Settings → Model Probe.
• Uses DSH semantic --dsw-* theme tokens and follows Light, Dark, and System themes.
• Higher-contrast diagnostic/secondary text for dense health data in both light and dark palettes.
• Enumerates every model advertised by every currently registered DSH provider adapter.
• Alive / Dead tabs with provider, model, latency, TTFT, result/error, attempt count, and next retry time.
• Manual Probe all with configurable timeout and concurrency.
• Persistent host-side health state that survives browser and DSH restarts.
• Runtime monitoring of normal DSH llm/stream calls.
• Two independent health retries after a real model error before declaring the model Dead.
• Dead models are rechecked automatically every 30 minutes.
• Successful normal DSH traffic immediately revives a previously Dead model.
• Explicit user cancellation/abort is not treated as model death.
• Bounded health history (latest 500 outcomes).

Health policy

Manual probe

A manual probe has up to three attempts:

1. Initial minimal probe.
2. Retry #1 if the initial probe fails.
3. Retry #2 if retry #1 also fails.
4. The model is marked Dead only if all three attempts fail.

Failure during normal DSH use

If a normal DSH request ends in a terminal provider/model error:

1. That failed user request counts as the initial failure.
2. Model Probe sends health retry #1.
3. If necessary, it sends health retry #2.
4. The model becomes Dead only if both health retries fail and no successful normal request has revived it in the meantime.

The plugin intentionally does not replay the user's original request. Replaying a partially streamed response could duplicate assistant output or tool calls. Health verification uses a separate, minimal request:

Reply with exactly OK.

with maxTokens=8.

Dead-model recovery

A Dead model receives one background health probe every 30 minutes:

• success → Alive immediately;
• failure → remains Dead and gets another check 30 minutes later.

The scheduler runs in the DSH host process, so the Model Probe page does not need to remain open. If DSH is stopped, overdue checks resume after DSH starts again.

Persistence

The host stores current health state and a bounded result history in state.json.

Default paths:

Platform	Path
Windows	%LOCALAPPDATA%\\dsh-model-probe\\state.json
macOS	~/Library/Application Support/dsh-model-probe/state.json
Linux/Unix	$XDG_STATE_HOME/dsh-model-probe/state.json or ~/.local/state/dsh-model-probe/state.json

Override the location with:

DSH_MODEL_PROBE_STATE_FILE=/custom/path/state.json

No provider credentials are stored by Model Probe. Probe requests use the provider adapters and credentials already configured in DSH.

Install

Clone or download this repository, then add the local plugin directory to the DSH profile:

dsh plugin --profile default add ./dsh-model-probe

Replace default with the profile actually in use, restart DSH, and open:

Settings → Model Probe

Update

Replace the plugin directory with the newer release/source, keep the same DSH profile entry, and restart DSH. Persistent health state is kept separately from the plugin directory.

Model discovery limitation

DSH provider catalogs are advisory: an adapter may accept model IDs that it does not advertise. Model Probe therefore checks all advertised models, but it cannot infer arbitrary hidden model IDs that no provider exposes through its catalog.

Health is keyed by provider + model, not model name alone. For example, the same model ID can be Alive on one provider and Dead on another.

Development

The distributed package is intentionally small and prebuilt. The browser and host entry points live in both src/ and lib/ so DSH can load the package directly.

Run the test suite with:

npm test

The tests cover failure classification, persistence, manual retry policy, runtime-error verification, dead-model scheduling/recovery behavior, API routes, and DSH theme integration.

Repository topic

The canonical GitHub repository should carry the topic:

dsh-plugin

so it is discoverable at GitHub's dsh-plugin topic page.

License

MIT © 2026 AGSQ11. See LICENSE.