Nature biochemistry/bioinformatics daily paper picker: scrapes nature.com, ranks by journal impact factor, shows 3 papers/day in a bottom-right panel with dated history.
A DSH Web plugin that scrapes nature.com in real time for the latest papers on biochemistry / bioinformatics, ranks them with journal impact factor (IF) as the primary quality criterion, and shows 3 papers per day in a floating panel at the bottom-right corner of the page — with direct links to the originals, journal IF, and content summaries. Every daily recommendation is archived by date (year-month-day) and can be browsed later. The panel footer also offers a one-click Exit Harness action.
This document is the full technical implementation guide (open-source documentation): architecture, per-feature implementation details, the Nature anti-bot automatic fallback mechanism, data formats, and API contracts.
🧬 3 papers per day: updates automatically each day (re-scrapes across day boundaries); cached within a day so it never flickers.
🏆 Impact-factor first: ships a reference table of approximate JCR impact factors for ~90 journals; candidates are sorted by "IF descending → publication date descending", and each card shows .
dsh-nature-papers · DSH Hub
IF x.x
🔗 Direct links: the title opens the nature.com original in a new tab, plus a DOI link.
📄 Content summaries: prefers the article-page Abstract, falls back to the search-page excerpt; expandable/collapsible.
📜 History: archived per YYYY-MM-DD, expandable to review any past day.
🔄 Real-time refresh: the button re-scrapes immediately and re-picks (preferring papers never recommended before).
⏻ Exit Harness: gracefully shuts down the DSH server process after a two-step confirmation.
🛡 Anti-bot automatic fallback: when nature.com serves a Client Challenge, the plugin automatically switches to the PubMed E-utilities mirror (still scoped to Nature Portfolio journals; links still point to nature.com).
Companion scripts live in the repository root (run the commands below from there):
Script
Purpose
install.ps1
Syncs code into the install dir + writes the cordis.patch.yml entry (supports -Uninstall)
start-dsh.ps1
Starts dsh web in the background (hidden window, logs to disk)
stop-dsh.ps1
Stops the dsh web process precisely by the listening port
restart-dsh.ps1
Kill old process → confirm port free → start new process → self-check the plugin
test-scraper.mjs
Standalone scraper-core test (no Cordis dependency)
Architecture: The DSH Plugin Model
DSH Web is a Cordis plugin tree: the web profile's empty root config cordis.yml is composed from several patch layers (@deepseek-ai/dsh-base, @deepseek-ai/dsh-web-app bundle layers + the user layer cordis.patch.yml). Mounting a plugin = inserting a loader entry into the composed tree and having it imported and activated.
Mounting chain (host side)
Place the package: copy this directory to $DSH_HOME\profiles\node_modules\dsh-nature-papers (a real directory). The DSH server resolves bare Node module specifiers by walking up from the profile directory, so both import('dsh-nature-papers') and its dependency @deepseek-ai/schemastery resolve.
Register the entry: write to cordis.patch.yml (maintained automatically by install.ps1):
The insert patch appends the entry to the composed tree; the !!js expression is evaluated at entry activation (the eval scope is injected with dshHomePath by boot()).
Activation: the loader import()s the module by name → unwrapExports takes the default export { name, inject, Config, apply } → Cordis resolves inject: ['webServer'] (waits for the service) → validates config via the schemastery Config → runs apply(ctx, config): registers 5 HTTP routes, loads/persists the history store, and starts the daily-rollover timer. When the entry is stopped or updated, the disposers inside ctx.effect unregister the routes and clean up timers.
Mounting chain (client side)
lib/client.js is a browser bundle, recognized through the package's dsh.client declaration (platform: web) and exports["./client"]:
dsh-client-modules (host side) scans loader entries for packages declaring dsh.client → hashes the bundle content into the window.__DSH_BOOT__ manifest → serves it at /plugins/dsh-nature-papers/client.js.
On page load the shell reads __DSH_BOOT__ → loads the bundle script → the bundle calls window.__ModuleLoader__.load({ id, factory }) to register the factory (lazy CJS: registering ≠ executing).
The Cordis client loader materializes the factory (factory(require), where require('react') is available) → plugin object { name, inject: ['slots'], apply }.
apply() registers the panel component into the shell.overlay slot via ctx.slots.inject('shell.overlay', ...) — that slot is rendered by ui-layout's AppFrame overlay layer (position:absolute; inset:0), and the panel is positioned with position:absolute; right:16px; bottom:16px — i.e. the bottom-right corner of the page.
Background: for scriptless crawlers nature.com serves a JavaScript challenge page ("Client Challenge", ~3 KB of HTML containing a loadScript routine); a plain HTTP client cannot pass it (a real browser must execute JS to obtain a cookie).
Detection (two places):
Search page: html.includes('Client Challenge') || !html.includes('app-article-list-row') → throws nature.com 触发了反爬校验（Client Challenge），已切换备用源 (nature.com served a Client Challenge; switched to the fallback source).
Article page: when Client Challenge is hit, that paper's abstract is treated as unavailable, returns null, and the excerpt fallback is used.
Fallback flow (generateEntry):
Try source A first (live nature.com scraping);
On error, automatically switch to source B (PubMed E-utilities) without interrupting the user's request;
Source B query construction:
esearch: term = ("Nature"[ta] OR "Nature Communications"[ta] OR …) AND (bioinformatics[tiab] OR biochemistry[tiab] OR "computational biology"[tiab] OR …), with sort=date&retmax=90 — the [ta] journal field pins the search to ~55 Nature Portfolio journals, and the [tiab] topic terms keep topical relevance;
esummary: fetches titles, full journal names, publication dates and DOIs for all hits (up to 90) in one call;
News filtering: Nature news items have DOIs like 10.1038/d41586-…; rows where doi.startsWith('10.1038/d4') are dropped — only research-type papers remain;
Link restoration: DOIs with the 10.1038/ prefix map back to https://www.nature.com/articles/ (links still point to nature.com); other prefixes go to https://doi.org/;
Abstracts: one batched efetch (retmode=xml&rettype=abstract) for the top count*2 PMIDs; the XML is split on blocks to extract / (multiple sections joined) / / ` ` /.
The panel labels the source of the batch ("来源：Nature 实时" / "来源：PubMed 镜像" — Source: Nature live / Source: PubMed mirror) and surfaces the switch reason in the response (sourceError), displayed in a notice bar at the top of the panel.
Proactive anti-bot measures: browser UA, request delay (default 1.5 s), per-request timeout, retry with backoff, page cap, fetching article pages only when the excerpt is insufficient (fewer requests), and same-day caching (no repeated scraping within a day).
4.3 Impact-factor-first ranking
Data (if-data.js): approximate JCR impact factors (mostly the 2023 release) for ~90 journals — covering Nature (flagship), Nature research journals, Nature Reviews journals, the npj series, and high-IF Springer Nature/BMC journals hosted on nature.com (Signal Transduction and Targeted Therapy 40.8, Cell Research 28.1, Molecular Cancer 27.7, Genome Biology 10, etc.).
Name normalization: lowercase → strip non-alphanumerics (including "&" and spaces) → strip a leading "the". This way nature.com's Communications Biology and PubMed's Nature reviews. Molecular cell biology both hit the same table entry.
Sort rule (rankRows):
sort((a, b) =>
(b.journalIf - a.journalIf) || // ① IF descending ("impact factor first")
b.pubDate.localeCompare(a.pubDate) || // ② same IF: newest publication first
a.title.localeCompare(b.title)) // ③ stable tiebreak
Journals not in the table get IF 0 (below every known journal); cards show IF — when unknown. The numbers are for ranking and display only, not licensed data.
4.4 Content summaries (abstract fetch chain)
Source A: use the search-page excerpt first; only when the excerpt is missing or shorter than 60 characters, fetch the article-page Abstract — locate the content after the id="Abs1-content" opening tag, cut before ``, then stripTags and collapse whitespace; a result under 40 characters is considered invalid. Fetches are serial with requestDelayMs spacing (polite rate limiting).
Source B: one efetch returns full abstracts for the top candidates (PubMed abstract coverage is ~100%).
Final fallback: abstract || snippet || '（暂无简介，请点击标题查看原文）' (no summary available; click the title to read the original).
The client truncates summaries longer than 140 characters to three lines with an "expand/collapse" toggle.
4.5 Three papers per day and caching
Storage: a single JSON file (config.storageFile, default $DSH_HOME/storages/nature-papers.json); see Storage Format. Writes are atomic (write .tmp then rename); a corrupt file is renamed to .bak and rebuilt.
Generation triggers:
GET /today on the first request of the day (lazy generation, includes the live scrape);
the daily-rollover timer (checks every 30 minutes whether the local date changed; pre-generates in the background when a new day has no data yet);
the user pressing "refresh" (forced generation).
Idempotence: within a day, /today always returns the same day's entry (cache first) — no flicker on page reload; the set rotates automatically on the next day.
Concurrency coalescing: state.generating holds the in-flight generation promise, so concurrent requests share one generation instead of scraping repeatedly.
4.6 History and de-duplication
After each successful generation the entry is inserted at the head of the history (dates descending); same-day entries are replaced; capped at historyCap (400 days).
De-dup (never recommend the same paper twice): the candidate pool filters out every URL already recommended in the history; if fewer than count papers remain, relax to excluding only the last 30 days; if still insufficient, allow repeats (take the current best).
The client "History" view groups entries by date; clicking a date expands that day's cards (TOP number, journal, IF, title link, summary).
4.7 Real-time refresh
POST /refresh → generate(force=true): re-scrapes immediately and counts today's existing entry as already-seen too (rotates in a fresh set).
On failure returns 502 { ok:false, error }; the client shows the error with a retry button.
4.8 The bottom-right panel (client side)
Mounting: the shell.overlay slot (list kind, root scope), registration id nature-papers, order: 100.
State machine: collapsed (collapses to a pill, remembered in localStorage) / view (today ↔ history) / loading / error / entry / history / expanded (summary expansion) / exitState (exit confirmation).
Daily self-healing: the window focus event plus a 10-minute interval check whether "local date ≠ panel date" → automatically re-fetch /today.
Styling: uses the app theme CSS variables (--dsw-alias-*) with fallbacks, adapting to light/dark mode; styles are injected via `` and cleaned up by the HMR machinery when the plugin is removed.
4.9 Exit Harness
Host side: POST /shutdown → respond 200 {ok:true} first (so the panel can show its final state), then after 200 ms call the launcher-injected appExit service (ctx.get('appExit')), which runs the graceful shutdown sequence: dispose the plugin tree, close the HTTP server, exit the process; falls back to process.exit(0) when appExit is unavailable.
Client side: the footer button "⏻ 退出 Harness" → first click enters "⚠ 确认退出？" (auto-resets after 4 seconds if not confirmed) → second click POST /shutdown and shows "正在退出…".
HTTP API Contract
Common prefix /plugins/dsh-nature-papers; responses are { ok: boolean, data?: any, error?: string }.
Method
Path
Description
Error codes
GET
/today
Today's picks (first request of the day triggers a live scrape)
502 upstream failure; 405 wrong method
GET
/history
Full history (dates descending)
405
POST
/refresh
Force re-scrape and re-pick today's picks
502; 405
GET
/info
Runtime status: date, history days, source, last generated at, last error
requestDelayMs between pagination and article-page fetches
Timeout
AbortSignal.timeout(requestTimeoutMs) per request
Retry
one retry on network failure with 1.2 s backoff
Anti-bot detection
Client Challenge marker string + missing result rows (double check)
Dual-source failover
nature.com failure → PubMed; both fail → 502 with a clear error message
Request-volume control
same-day caching; article pages only when the excerpt is insufficient; PubMed abstracts fetched in one batched efetch
Storage safety
atomic writes (tmp + rename); corrupt files auto-rebuilt via .bak
Concurrency
generation promises coalesced (shared in-flight)
Lifecycle
all routes/timers registered inside ctx.effect; cleaned up automatically on unload
Known Limitations
Impact factors are approximate JCR reference values (mostly the 2023 release), used only for ranking and display, not licensed data; the numbers go stale over time.
nature.com may serve a Client Challenge to high-frequency access; the plugin then uses the PubMed mirror automatically (see 4.2) and labels the source on the panel.
The endpoints are exposed on loopback only, with no authentication (consistent with DSH's other /plugins routes).
Hot reload of code is limited: cordis.patch.yml is watched by watchUserPatches, but that watcher proved unreliable in practice (BOM encoding, file deletion, and chokidar exact-file watch breakage all made it stop responding), so plugin code changes (lib/*.js) require restarting dsh web (see Operations below); entry-level config changes should also be followed by a restart to be safe.
The panel is a fixed-position overlay (not draggable); it stacks with other shell.overlay registrations in registration order.
Run the commands below from the repository root (powershell -ExecutionPolicy Bypass -File .\xxx.ps1).
The scripts resolve $DSH_HOME automatically (env var, falling back to ~/.dsh); the dsh launcher is auto-detected (PATH or npx caches, pinnable via -DshBin); the port defaults to 3080 and is configurable via -Port. No machine-specific paths are hardcoded.
# Install / update (syncs code + writes the patch entry; falls back to in-place
# overwrite when the server is running and the directory is locked)
powershell -ExecutionPolicy Bypass -File .\install.ps1
# Uninstall
powershell -ExecutionPolicy Bypass -File .\install.ps1 -Uninstall
# Start / stop (a restart is required for code changes to take effect)
powershell -ExecutionPolicy Bypass -File .\stop-dsh.ps1
powershell -ExecutionPolicy Bypass -File .\start-dsh.ps1
powershell -ExecutionPolicy Bypass -File .\restart-dsh.ps1 # kill old → confirm port → start new → self-check
# The panel's "⏻ 退出 Harness" is equivalent to stop-dsh.ps1 (graceful exit)
Development tips: the scraping core scraper.js has no Cordis dependency, so you can validate it standalone with node test-scraper.mjs; after changing lib/*, run install.ps1 and restart.