dsh-computer-use

dsh-computer-use is a plugin for DSH (DeepSeek Harness) that gives your agents local macOS computer use capabilities — see the screen (screenshot), drive mouse/keyboard/scroll, and control the browser at DOM level, with an OCR fallback for models without vision.

• Tools: screenshot / screen_ocr (macOS Vision) / desktop_mouse_move|click|drag|scroll (cliclick) / keyboard_type|key / browser_open|click|type|scroll|extract|close (Playwright-core + system Chrome)
• Enforces an act → verify loop: screenshot before and after every operation; text-only models read the screen via screen_ocr / browser_extract
• Safety by design: on-screen coordinate guards, keyboard shortcut whitelist, TCC permission boundaries (never bypassed)
• Mounts via profile cordis.patch.yml; does not modify the official dsh repository

---

以下为中文文档 / Chinese documentation below:

为 dsh（DeepSeek Harness） 提供本地 macOS computer use 能力：模型可以看屏幕（截图）、操作屏幕（鼠标/键盘/滚动）、控制浏览器（DOM 级），并在模型无视觉能力时用 OCR 文本兜底。

独立插件包，通过 profile cordis.patch.yml 挂载，不改动 dsh 官方仓库。

能力总览

场景	工具	实现
看屏幕	screenshot（全屏/区域/窗口 → image block）	/usr/sbin/screencapture + attachment 服务
看屏幕（无视觉模型）	screen_ocr（文本 + 像素坐标）	macOS Vision framework（swift）
桌面键鼠	desktop_mouse_move/click/drag/scroll	cliclick
键盘	keyboard_type（含中文）/ keyboard_key（白名单快捷键）	cliclick + osascript
浏览器（主）	browser_open/click/type/scroll/extract/close	playwright-core + 系统 Chrome

操作纪律（模型侧）：每次操作前 screenshot 确认状态，操作后 screenshot 验证（act → verify 闭环）。无视觉模型用 screen_ocr / browser_extract 读屏。

前置条件

依赖	状态	安装/授权