dsh-video-lens

Configuration

All options are DSH config values:

Key	Default	Meaning
visionBaseUrl	https://api.siliconflow.cn/v1	Vision endpoint (OpenAI-compatible)
visionModel	Qwen/Qwen3-VL-8B-Instruct	Vision model name
visionApiKeyEnv	VIDEO_LENS_API_KEY	Env var holding the vision key
asrBaseUrl	https://api.siliconflow.cn/v1	ASR endpoint (OpenAI-compatible /audio/transcriptions)
asrModel	FunAudioLLM/SenseVoiceSmall	ASR model name
asrApiKeyEnv	VIDEO_LENS_ASR_KEY	Env var holding the ASR key
maxFrames	12	Frame budget cap (1–max); actual count is duration-adaptive (~1 frame per 30s, denser for short videos)
frameMaxWidth	768	Max frame width; keeps payloads small
frameQuality	4	JPEG quality (ffmpeg -q:v)
sceneThreshold	10	scdet threshold (0–100); higher = fewer cuts
askPaddingSec	2	video_ask window padding around matched transcript segments
vlmMaxTokens	1500	Vision model max output tokens
vlmTimeoutMs	90000	Vision call timeout
asrTimeoutMs	120000	ASR call timeout

Usage

Ask the agent:

"What's in /tmp/demo.mp4?"

The agent calls video_probe first, then video_analyze. Evidence includes:

{
  "metadata": { "container": "mov,mp4,m4a,3gp,3g2,mj2", "durationSec": 268.4, "...": "..." },
  "shots": [{ "timeSec": 12.3, "score": 45.2 }],
  "framesSampled": [{ "timestampSec": 5.5, "jpegBytes": 12345 }],
  "transcript": {
    "text": "…",
    "segments": [{ "start": 0.0, "end": 2.4, "text": "…" }],
    "language": "zh"
  },
  "visionModel": "Qwen/Qwen3-VL-8B-Instruct",
  "analysis": { "overall_summary": "…", "timeline": [{"timestamp_sec": 5.5, "description": "…"}], "on_screen_text": "…", "visual_style": "…", "notable_moments": "…" }
}