shengbinxu/dsh-llm-latency0

dsh-llm-latency

Per-vendor/per-model LLM latency telemetry and cross-vendor benchmark plugin for DeepSeek Harness

包名
dsh-llm-latency
版本
0.1.0
最近更新
2026年8月29日

安装

$npx -p @deepseek-ai/dsh dsh plugin --profile web add github:shengbinxu/dsh-llm-latency

Usage

  • Dashboard — switch between 总览 / 时段对比 / 会话对比 / 请求日志:
    • 时段对比: pick a model, pick a window, compare vendors side by side.
    • 会话对比: pick two sessions that each used a single model, compare them.
    • 请求日志: search and filter every model call by request id, vendor, model, session, credential ref, or status.
  • Model tool — ask the agent "帮我看看各厂商延迟对比" (latency_report); it accepts model, vendors, from/to, and sessionIds.

Configuration

Set in cordis.patch.yml (or override the row):

KeyDefaultMeaning
retentionDays30Data retention window in days
recentLimit2000Per-key exact-sample ring cap
sessionLimit500Sessions retained (most recent first)
spikeFloorMs10000TTFT above this counts as a spike
modelAliases{}Canonical model → provider model ids
minSamplesForComparison20Minimum ok samples before a median CI is reported
logLimit5000Request-log mirror cap (recent records kept)
logRetentionDays7Request-log retention window in days