shengbinxu/dsh-llm-latency0

dsh-llm-latency

Per-vendor/per-model LLM latency telemetry and cross-vendor benchmark plugin for DeepSeek Harness

AI 分析

核心用途是统计和对比不同模型及厂商的响应延迟。适合需要评估模型性能、进行时段或会话延迟对比,并希望通过智能体直接生成延迟报告的用户。

パッケージ
dsh-llm-latency
バージョン
0.1.0
最終更新
2026/08/29

インストール

$npx -p @deepseek-ai/dsh dsh plugin --profile web add github:shengbinxu/dsh-llm-latency

ドキュメント

README 全文を読む ↗

Usage

  • Dashboard — switch between 总览 / 时段对比 / 会话对比 / 请求日志:
    • 时段对比: pick a model, pick a window, compare vendors side by side.
    • 会话对比: pick two sessions that each used a single model, compare them.
    • 请求日志: search and filter every model call by request id, vendor, model, session, credential ref, or status.
  • Model tool — ask the agent "帮我看看各厂商延迟对比" (latency_report); it accepts model, vendors, from/to, and sessionIds.

Configuration

Set in cordis.patch.yml (or override the row):

KeyDefaultMeaning
retentionDays30Data retention window in days
recentLimit2000Per-key exact-sample ring cap
sessionLimit500Sessions retained (most recent first)
spikeFloorMs10000TTFT above this counts as a spike
modelAliases{}Canonical model → provider model ids
minSamplesForComparison20Minimum ok samples before a median CI is reported
logLimit5000Request-log mirror cap (recent records kept)
logRetentionDays7Request-log retention window in days