yan77-h/dsh-agent-evaluator ↗★ 0
dsh-agent-evaluator
Agent benchmark evaluation plugin for DeepSeek Harness: web panel, slash commands, headless CLI, test-set import, model/judge switching, orchestration, scoring, and reports.
安装
$
npx -p @deepseek-ai/dsh dsh plugin --profile web add github:yan77-h/dsh-agent-evaluator说明文档
阅读完整 README ↗Usage
Web UI: Settings → Agent Evaluation, or run /eval commands:
/eval import /path/to/suite.jsonl
/eval model provider/model
/eval judge provider/model
/eval run suite --parallel 4 --retries 1
/eval progress
/eval report
Subcommands: import, list, drop, model, models, judge, run, resume, kill, runs, progress, report.
Headless / CI:
cd /path/to/deepseek-harness
pnpm dsh --profile eval eval import suite.jsonl
pnpm dsh --profile eval eval model provider/model
pnpm dsh --profile eval eval run suite --parallel 4