dsh-bilibili

🔧 Configuration

Defaults live in cordis.patch.yml; override any field in $DSH_HOME/profiles/web/cordis.patch.yml (later layers win per row):

- override:
    - id: bilibili
      config:
        sessdata: ''                 # optional Bilibili SESSDATA (logged-in subtitles / more comments)
        commentLimit: 20             # max comments to fetch
        maxFrames: 6                 # max keyframes
        extractFrames: true          # false = text-only mode
        downloadVideo: true          # download video locally before capture (recommended)
        keepVideo: false             # true = keep downloaded video files
        maxVideoMinutes: 30          # videos longer than this are captured remotely per frame
        maxDownloadMb: 800           # download size cap (MB)
        quality: 32                  # 16=360p 32=480p 64=720p 80=1080p
        detectScenes: true           # scene-change detection (sampled pass beyond 20 min)
        sceneThreshold: 0.4          # scene threshold 0-1, higher = stricter
        sharpFrames: true            # sharp-frame preference: blurdetect picks the sharpest frame within ±1.5s
        asrProvider: 'bcut'          # ASR engine: bcut (default) | sherpa-onnx (Chinese) | whisper-local | auto | none
        sherpaBin: ''                # sherpa-onnx-offline binary path
        sherpaModel: ''              # sherpa model onnx path (SenseVoice/Paraformer)
        sherpaModelType: 'sense-voice'  # sense-voice | paraformer | zipformer2-ctc
        sherpaTokens: ''             # sherpa tokens.txt path
        sherpaThreads: 0             # sherpa CPU threads (0 = auto)
        whisperBin: 'whisper-cli'    # whisper.cpp binary (PATH or absolute path)
        whisperModel: 'medium'       # small / medium / large-v3, or a ggml-*.bin path
        whisperModelDir: ''          # model dir; empty = /models
        whisperLanguage: 'zh'        # transcription language
        whisperThreads: 0            # whisper CPU threads (0 = auto)
        visionProvider: 'none'       # frame vision: none (default) | ollama | llama-cpp | openai-compatible
        visionBaseUrl: ''            # vision endpoint; empty + ollama = http://localhost:11434/v1
        visionModel: 'medium'        # low(2B) / medium(4B) / high(8B), or an explicit model name
        visionApiKey: ''             # cloud vision API key (empty for local)
        visionPrompt: ''             # vision prompt (empty = built-in per-model default)
        visionPromptByModel: {}      # per-model prompt overrides (explicit model name / low / medium)
        visionMaxFrames: 6           # max frames to describe (aligned with maxFrames)
        framesDir: ''                # frame output dir; empty = system temp/dsh-bilibili/
        summaryTemplate: ''          # template path; empty = bundled templates/summary.md
        timeoutMs: 300000            # overall tool timeout (ms)

Local ASR transcription (optional; sherpa-onnx recommended for Chinese)

No-subtitle videos default to Bijian (zero config, anonymous, China-friendly). To go fully offline or when Bijian fails, switch to a local engine. The plugin ships no models — only the interface; models and binaries are downloaded by the user (no API keys, quotas, or fees involved).

Recommended: sherpa-onnx (Chinese, SenseVoice)

Bilibili is mostly Chinese content, and SenseVoice beats Whisper on Chinese accuracy while being faster and smaller; official models are hosted on ModelScope (fast in China).

Tier	Recommended model	Size (approx)	For
Low	SenseVoiceSmall (int8)	~230 MB	low-end machines
Mid	SenseVoiceSmall (fp32)	~900 MB	mainstream (recommended)
High	Paraformer-large	~2.5 GB	high-end / maximum accuracy

Steps:

1. Download the sherpa-onnx-offline binary for your OS from sherpa-onnx;
2. Download a model (model.onnx + tokens.txt) — SenseVoice models are on ModelScope or the sherpa-onnx model list;
3. Set asrProvider: 'sherpa-onnx' and fill sherpaBin / sherpaModel / sherpaTokens (sherpaModelType defaults to sense-voice).

Alternative: whisper.cpp (general / English)

Tier	whisperModel	Model file	Size	For
Low	small	ggml-small.bin	~466 MB	low-end / fast drafts
Mid	medium	ggml-medium.bin	~1.5 GB	mainstream
High	large-v3	ggml-large-v3.bin	~3 GB	high-end

1. Download whisper-cli from whisper.cpp;
2. Download the matching ggml-*.bin model into a models/ directory;
3. Set asrProvider: 'whisper-local' and fill whisperBin / whisperModel.

Note: asrProvider: 'auto' falls back in order Bijian → sherpa-onnx → whisper-local. For Chinese, use at least medium (whisper) or pick SenseVoice (sherpa) directly. sherpa-onnx CLI flags vary slightly across versions — check --help of your build if something errors.

🔍 Frame vision descriptions (optional)

When the main model has no vision, enable this feature: each captured frame is sent to a vision model and returned with a description field, so the main model can decide which images to cite in the report — cite only when visual confirmation matters (charts / UIs / demo details); pure talking-head frames are not cited. Off by default; a vision failure never breaks the main flow (frame paths are still returned).

Local (recommended): install Ollama and pull a model — no keys, offline, free:

Tier	visionModel	Ollama model	RAM (approx)	For
Low	low	qwen3-vl:2b	~2 GB	ultra low-end
Mid	medium (default)	qwen3-vl:4b	~4 GB	low-end to mainstream (recommended)
High	high	qwen3-vl:8b	~6-8 GB	mainstream, best quality

Larger models can be passed as explicit names (e.g. qwen3-vl:32b) — they are just no longer a default tier.

Mid-tier alternative MiniCPM-V 4.0 (OpenBMB, 2026; officially claims to surpass GPT-4.1-mini, runs on phones; official GGUF/int4 releases — check the official library for its Ollama tag). visionModel also accepts explicit model names (Ollama tags or cloud model ids).

Other non-Qwen models (verified on the Ollama library, 2026-08): minicpm-v:8b (OpenBMB MiniCPM-V 2.6, strong Chinese OCR), moondream (1.9B, English-first), gemma3n (Google, English-first). Kimi-VL / InternVL / GLM-4V are not in the official Ollama library — use community GGUFs via llama.cpp or cloud OpenAI-compatible APIs (e.g. Moonshot / Zhipu). MiniCPM-V 4.0's official GGUF works on the llama-cpp route.

Selection rationale: this task is understanding frame content + emitting a citation hint, not OCR transcription — the weights are on Chinese scene understanding and instruction-following, so the default tiers use the Qwen3-VL family (consistent behavior, shared prompts); MiniCPM-V 4.0 for maximum edge efficiency.

llama.cpp (local alternative): run llama-server with a vision GGUF (model + mmproj); it exposes an OpenAI-compatible API, and visionModel is simply the --alias you set at launch — matching the alias to a tier keyword reuses the tier config directly:

llama-server -m qwen3-vl-8b-q4_k_m.gguf --mmproj mmproj-qwen3-vl-8b.gguf --port 8080 --alias qwen3-vl:8b