uson1x/dsh-plugin-llm-verifier ↗★ 0
dsh-plugin-llm-verifier
基于 LLM 评审器的 DSH 插件：通过多维度蒙特卡洛重复评分提供连续奖励信号。
AI 分析
核心用途是利用大模型作为评审器，对 Agent 的输出进行多维度、高精度的自动打分与对比。适合需要对模型生成质量进行自动化评估和对齐训练的开发者。
安装
此插件尚未提供可验证的 bundle，或兼容性检查未通过。请先阅读仓库说明。 阅读完整 README ↗
说明文档
阅读完整 README ↗Configuration
| Key | Default | Meaning |
|---|---|---|
provider | — (required) | Registered ctx.llm provider route to grade with |
model | — (required) | Model id on that route |
granularity | 20 | Integer score scale 1..G |
repetitions | 3 | Grading passes per criterion (K) |
temperature | 1 | Sampling temperature for the Monte Carlo estimate |
criteria | specification / output / errors | Array of { name, description } sub-criteria |
maxOutputTokens | 2048 | Output cap per grading call |
timeoutMs | 120000 | Deadline per grading call |
concurrency | 4 | Parallel grading calls |
tieMargin | 0.02 | compare margin below which the verdict is tie |
Cost note: verify_select makes N × C × K model calls (default 9 per candidate); verify_track makes steps × K.