DeepSeek HarnessHub
플러그인 제출

yan77-h/dsh-agent-evaluator ↗★ 0

dsh-agent-evaluator

Agent benchmark evaluation plugin for DeepSeek Harness: web panel, slash commands, headless CLI, test-set import, model/judge switching, orchestration, scoring, and reports.

AI 분석

核心用途是对 DSH 中的 Agent 进行自动化基准测试与性能评估。适合需要导入测试集、切换不同模型或裁判进行批量评测与生成报告的开发者。

에이전트
패키지
dsh-agent-evaluator
버전
0.10.0
라이선스
MIT
최근 업데이트
2026. 8. 16.

설치

$npx -p @deepseek-ai/dsh dsh plugin --profile web add github:yan77-h/dsh-agent-evaluator

문서

전체 README 읽기 ↗

Usage

Web UI: Settings → Agent Evaluation, or run /eval commands:

/eval import /path/to/suite.jsonl
/eval model provider/model
/eval judge provider/model
/eval run suite --parallel 4 --retries 1
/eval progress 
/eval report

Subcommands: import, list, drop, model, models, judge, run, resume, kill, runs, progress, report.

Headless / CI:

cd /path/to/deepseek-harness
pnpm dsh --profile eval eval import suite.jsonl
pnpm dsh --profile eval eval model provider/model
pnpm dsh --profile eval eval run suite --parallel 4