xiaozhengdeng/dsh_omnivision ↗★ 1
dsh_omnivision
OmniVision for DeepSeek Harness: an OmniParser-powered GUI agent plugin — screen capture, element recognition, click/type automation and a browser vision dock with recognition history, diffing and summary
安装
$
npx -p @deepseek-ai/dsh dsh plugin --profile web add github:xiaozhengdeng/dsh_omnivision说明文档
阅读完整 README ↗Usage
Call the gui_* tools from the model side; the OmniVision dock in the browser provides the visual surface:
| Control | Description |
|---|---|
| 智能识别 | Live element list + SOM-annotated image (hover to highlight, click to zoom) |
| 🖼 解析图片 | Pick a local image file to parse (bypasses model multimodal limits) |
| 📋 总结 | Send the current recognition info to the session for the model to summarize |
| 识别记录 | History with thumbnails + diff vs latest (added / removed / moved) |
| 调用记录 | Recent gui_* tool calls and action log |
| 探测 | Check OmniParser server connectivity |
使用
模型侧直接调用 gui_* 工具；浏览器里的 OmniVision Dock 提供可视化操作：
| 控件 | 说明 |
|---|---|
| 智能识别 | 实时元素列表 + SOM 标注图（悬停高亮、点击放大） |
| 🖼 解析图片 | 选择本地图片文件解析（绕开模型多模态限制） |
| 📋 总结 | 把当前识别信息发到会话，由模型按提示词总结 |
| 识别记录 | 历史缩略图 + 与最新对比（新增 / 移除 / 位移） |
| 调用记录 | 最近 gui_* 工具调用与动作日志 |
| 探测 | 检查 OmniParser 服务连通性 |