dsh-plugin-warroom-garak

A DeepSeek Harness plugin that adds one model-facing tool, garak_scan — an authorized garak baseline red-team sweep against a target LLM endpoint, with a built-in authorization gate, a budget cap, and an auto-written evidence report.

Part of the war-room approach: garak owns the probes (the "weapons"); this plugin is the orchestrator that runs them against an authorized target, keeps the run in the dsh session log (auditable/replayable), and produces a compliance-ready evidence artifact.

v0 / developer preview. Built against @deepseek-ai/dsh-* 0.1.0-rc.x and modeled on the official dsh-tool-web plugin. Two seams are version-sensitive and may need a small fix on first build in your environment — both are commented in the source:

1. defineTool parameter/output schema DSL (src/index.ts)
2. garak report JSONL field names / hit detection (src/garak.ts)

What it does

1. Refuses to run unless authorization.authorized === true with a non-empty scope.
2. Caps attempts per probe at maxGenerations (budget / circuit-breaker).
3. Spawns garak against your REST target, parses the JSONL report into per-probe hit rates.
4. Writes a one-page Markdown evidence report (target, scope, tool+time, per-probe table, overall hit rate, pointer to the raw garak report for reproduction).
5. Returns a summary + report path + metrics as the tool result (a durable session event).

Prerequisites

• A working DeepSeek Harness install.
• garak on PATH (pipx install garak or pip install garak; verify garak --version).

Build

# pin dsh-* deps to your installed dsh version first:  npm ls @deepseek-ai/dsh-tools
npm install
npm run build      # tsdown -> lib/

Install into dsh

Add the plugin as an entry in your dsh profile / cordis config (the composition file that lists plugins), then restart dsh:

# in your dsh profile's cordis config (e.g. cordis.patch.yml / cordis.yml)
- name: 'dsh-plugin-warroom-garak'
  config:
    garakBin: 'garak'        # or 'python' to use `python -m garak`
    maxGenerations: 5
    reportDir: '.warroom/reports'