shengbinxu/dsh-llm-latency ↗★ 0
dsh-llm-latency
Per-vendor/per-model LLM latency telemetry and cross-vendor benchmark plugin for DeepSeek Harness
安装
$
npx -p @deepseek-ai/dsh dsh plugin --profile web add github:shengbinxu/dsh-llm-latency说明文档
阅读完整 README ↗Usage
- Dashboard — switch between 总览 / 时段对比 / 会话对比 / 请求日志:
- 时段对比: pick a model, pick a window, compare vendors side by side.
- 会话对比: pick two sessions that each used a single model, compare them.
- 请求日志: search and filter every model call by request id, vendor, model, session, credential ref, or status.
- Model tool — ask the agent "帮我看看各厂商延迟对比" (
latency_report); it acceptsmodel,vendors,from/to, andsessionIds.
Configuration
Set in cordis.patch.yml (or override the row):
| Key | Default | Meaning |
|---|---|---|
retentionDays | 30 | Data retention window in days |
recentLimit | 2000 | Per-key exact-sample ring cap |
sessionLimit | 500 | Sessions retained (most recent first) |
spikeFloorMs | 10000 | TTFT above this counts as a spike |
modelAliases | {} | Canonical model → provider model ids |
minSamplesForComparison | 20 | Minimum ok samples before a median CI is reported |
logLimit | 5000 | Request-log mirror cap (recent records kept) |
logRetentionDays | 7 | Request-log retention window in days |