@xidong-ai/dsh-rate-limiter

Configuration

Configure the token bucket per provider in the profile's cordis.patch.yml (or this plugin's cordis.patch.yml):

- id: rate-limiter
  config:
    enabled: true
    providers:
      nvidia:
        rate: 2          # tokens/second (long-term average QPS)
        burst: 5         # bucket capacity (allowed burst requests)
      oc-zen:
        rate: 1
        burst: 3

• rate: refill rate (tokens/second), i.e. the long-term average request rate.
• burst: bucket capacity, the number of burst requests allowed.
• Providers not listed are not rate-limited; requests pass through untouched (zero intrusion).
• enabled: false disables the plugin entirely.