uson1x/dsh-plugin-llm-verifier ↗★ 0
dsh-plugin-llm-verifier
LLM-as-a-Verifier for DeepSeek Harness (dsh): continuous reward signals via select / compare / track, using Monte Carlo repeated grading over decomposed criteria
安装
此插件尚未提供可验证的 bundle，或兼容性检查未通过。请先阅读仓库说明。 阅读完整 README ↗
说明文档
阅读完整 README ↗Configuration
| Key | Default | Meaning |
|---|---|---|
provider | — (required) | Registered ctx.llm provider route to grade with |
model | — (required) | Model id on that route |
granularity | 20 | Integer score scale 1..G |
repetitions | 3 | Grading passes per criterion (K) |
temperature | 1 | Sampling temperature for the Monte Carlo estimate |
criteria | specification / output / errors | Array of { name, description } sub-criteria |
maxOutputTokens | 2048 | Output cap per grading call |
timeoutMs | 120000 | Deadline per grading call |
concurrency | 4 | Parallel grading calls |
tieMargin | 0.02 | compare margin below which the verdict is tie |
Cost note: verify_select makes N × C × K model calls (default 9 per candidate); verify_track makes steps × K.