dsh-tool-local-model

dsh-local-model

A DeepSeek Harness plugin that lets the model start, stop, and inspect local inference servers — Strata, Ollama, llama.cpp, or anything else that answers an OpenAI-compatible model listing.

The agent asks for a server by name and gets back provider/model references it can select on the model route that fronts that server. No shell script in the middle, no per-machine hardcoding in the plugin: the servers are deployment configuration.

> 帮我把本地的 ollama 拉起来，用 qwen2.5:7b

local_model start ollama
→ ollama: answering on port 11434 - ollama-local/qwen2.5:7b

Install

Not on npm yet. Vendor it into a harness checkout or link it from a profile:

git clone https://github.com/miloye7711-dotcom/dsh-local-model.git
cd dsh-local-model
pnpm install
pnpm run build

Then mount the built package by absolute path, or pnpm link it into the harness that will load it.

Configure

Declare the servers this composition may start. command, args, and cwd go to the subprocess provider verbatim — no shell, so anything shell-shaped has to be spelled out.

- id: local-model
  name: 'dsh-tool-local-model'
  config:
    servers:
      ollama:
        command: 'ollama'
        args: ['serve']
        cwd: '%USERPROFILE%'
        port: 11434
        provider: 'ollama-local'

      strata:
        command: 'D:\Strata\.venv\Scripts\python.exe'
        args:
          - 'D:\Strata\serve\server.py'
          - '--engine'
          - 'strata'
          - '--config'
          - 'D:\Strata\strata-iq2_xs.json'
          - '--port'
          - '8080'
        cwd: 'D:\Strata'
        port: 8080
        provider: 'strata-local'

    startTimeoutMs: 900000   # cold start budget (default 15 min)
    probeTimeoutMs: 5000     # one readiness probe (default 5 s)

Field	Meaning
command	executable name or absolute path
args	arguments, passed verbatim
cwd	working directory for the child process
env	extra environment entries, merged onto the provider's environment
port	the port the server listens on
provider	route name in provider/model refs; defaults to the config key
modelsPath	HTTP path answering the model list; defaults to /v1/models

An empty servers table is valid: the tool is registered with nothing to start.

Actions

action	Behaviour
list	only servers that are answering
status	every configured server, answering or not
start	spawn if down, then wait until the model listing answers; idempotent when already up; terminates the child if the budget runs out
stop	terminate, wait for exit — only for servers this plugin started

Readiness is one HTTP GET : a server counts as up when it lists at least one model. That rule works for OpenAI-compatible servers and for Ollama-shaped listings (data[].id or models[].name).

What this plugin does not do

It owns process lifecycle, not routing. To actually send a model request to a server it started, add an LLM adapter route whose baseURL is that server's port — for example an @deepseek-ai/dsh-llm-pi-ai provider:

- id: llm-pi-ai
  name: '@deepseek-ai/dsh-llm-pi-ai'
  config:
    providers:
      ollama-local:
        api: openai-completions
        baseURL: http://127.0.0.1:11434/v1
        headers: { Authorization: 'Bearer ollama' }   # placeholder: keyless server
        models:
          - id: qwen2.5:7b
            name: 'Qwen2.5 7B (Ollama)'
            contextWindow: 32768
            maxTokens: 8192

refs in the result gives exactly the provider/model spelling to select there.

Design notes

• No new capability seam. This is a Consumer over the existing subprocess capability; a second service definition would have no second consumer.
• Termination is process-tree-wide, because it goes through the same subprocess provider the rest of the harness uses — a killed server leaves no orphan worker processes.
• No KV-cache churn. The tool description and parameter schema are fixed text; nothing here changes between turns.
• Ownership is enforced, not assumed. stop refuses a server that answers on the port but was started elsewhere, rather than killing a process the plugin did not create.

Development

pnpm install
pnpm run typecheck   # strict, no emit
pnpm test            # 7 tests over a real ToolRuntime
pnpm run build       # lib/ + declarations

The subprocess provider is faked in tests at the seam's shape, so the shipping plugin body runs unmodified against a real ToolRuntime and a real HTTP listener.

License

MIT