GreyRaphael/dsh-web-fetch-moli ↗★ 2

dsh-web-fetch-moli

利用轻量级Moli浏览器渲染并爬取网页 适合需要在低内存占用下抓取复杂单页应用和动态网页的用户。

包名
dsh-web-fetch-moli
兼容性
待验证
Harness 依赖范围
>=0.1.2-alpha.0 <0.3.0
Cordis 依赖范围
>=4.0.1-rc.1 <5
版本
0.5.1
许可证
MIT
最近更新
2026年9月23日

安装

$npx -p @deepseek-ai/dsh dsh plugin --profile web add github:GreyRaphael/dsh-web-fetch-moli

Configuration

The settings card (设置 → 插件 → 插件配置 → Moli 网页爬取) configures web-fetch-moli in real time:

FieldDefaultDescription
backendlocallocal (managed Moli CDP daemon) or cdp (remote CDP endpoint).
moliPath(auto)Path to moli binary. Blank = auto-discover on $PATH and standard locations.
cdpEndpoint127.0.0.1:9222Remote CDP endpoint (host:port or URL, used in cdp mode).
bypassCsptrueBypasses Content Security Policy via CDP Page.setBypassCSP, unlocking micro-frontend dynamic script loading.
autoScrollSentineltrueProgrammatically triggers IntersectionObserver load-more sentinels for structure-first geometry.
shareBrowserContexttruePreserves cookies and localStorage across fetches in cdp mode. Unchecked (and always in local mode): fresh isolated context per fetch.
denoisetrueRuns LinkeDOM + Readability + mdream to strip ads, navbars, and footers before LLM-optimized Markdown conversion.
timeoutMs90000Total deadline budget (ms) for one fetch; heavy micro-frontend pages may need several layout + scroll rounds. The bundled layer ships 60000, the schema default is 90000.
maxConcurrency(auto)Max concurrent rendering slots (auto: local 20, remote CDP 50).
challengeWaitMs5000Bounded wait (ms) for Cloudflare verification to clear naturally (0 disables).
challengeRetries0Same-page retry attempts after challenge wait.