dsh-agnes-image

dsh-agnes-image

English · 中文

Agnes image and video generation for DeepSeek Harness. It adds two agent tools (agnes_generate_image, agnes_generate_video), an AI prompt optimizer, and a sidebar panel with four tabs: Images, Videos, Library, Settings. Progress reporting is honest, cards are poster-first, and files are handed off to the desktop.

The plugin ships without an API key. Install it, open the panel's Settings tab, paste your Agnes key once and save. Only then do the generation buttons light up; until then they are disabled and explain why. Credentials live in your own %DSH_HOME% data directory, never in this repository.

Both halves are hand-written against the official DSH bundle format. There is no build step: index.js is the host half, and client.js is the browser bundle.

Agnes APIs used:

Purpose	Model	Endpoint
Image generation	agnes-image-2.5-flash	POST https://api.agnes-ai.cn/v1/images/generations
Video generation	agnes-video-2.5-flash (default; 720P only)	POST https://api.agnes-ai.cn/v1/videos, poll GET https://api.agnes-ai.cn/agnesapi?video_id=&model_name=
Prompt optimization	agnes-3.0-flash	POST https://api.agnes-ai.cn/v1/chat/completions

Docs: image · video · text

Screenshots

Real panel in a desktop profile, captured from the running harness.

Images tab	Videos tab

Image library	Video library, posters

Files

File	Role
package.json	bundle manifest: dsh.bundle.patch + dsh.client (platform web) + exports["./client"]
cordis.patch.yml	bundle patch layer. One insert row with id: agnes-image, which is also the settings namespace. No credential is written here.
index.js	host half: Config, key resolution (panel only), Agnes calls, disk output, tool registration, /api/agnes-image/* routes (video, video covers, prompt optimize, local file open)
client.js	browser half: sidebar.panellist icon plus the whole main page (Images / Videos / Library / Settings)
icon.svg	sidebar icon

Documentation

Longer notes live in docs/:

• Index
• Install · Usage · Configuration · Architecture · Contributing
• 中文版：docs/index.md · README_ZH.md

Capabilities

1. Agent tool agnes_generate_image

Call it straight from a conversation.

Parameters: prompt (required), size (1K/2K/3K/4K), ratio (1:1, 3:4, 4:3, 16:9, 9:16, 2:3, 3:2, 21:9), count (1–4), reference_images (image-to-image or composition: a public HTTPS URL or a data:image/...;base64 string).

It returns absolute paths on disk, for example:

Agnes generated 1 image (agnes-image-2.5-flash, 2K 1:1).
Saved to disk — absolute paths:
  C:\...\default-workspace\agnes-images\2026-10-16\190212-测试：一只戴宇航头盔的柴犬.png  (836.9 KB)

2. Agent tool agnes_generate_video

This one blocks: create task, poll, download. The default ceiling is 15 minutes.

Parameters: prompt (required), mode (text / keyframe / reference), seconds ("4"–"12", default "5"), size (720P; with agnes-video-2.5 also 1080P/1K/2K), aspect_ratio (16:9/9:16/1:1/4:3/3:4/21:9), first_frame / last_frame (keyframe mode), images[] / audios[] (reference mode), seed.

Agnes enforces the per-mode media rules, and this plugin validates them locally before the request, then re-throws the upstream error verbatim. text carries no media. keyframe needs at least one frame and no images or audios. reference needs at least one image or audio clip and no first or last frame. In reference mode local file paths are inlined into data: URLs automatically: single file under 15 MB, edge length 256–5760 px, and anything outside that errors. Public HTTPS URLs are accepted too.

3. AI prompt optimizer

Both prompt boxes, image and video, have an AI Optimize button. It hands the current prompt to a language model and shows a rewrite card first. The card holds only the rewritten prompt, expanded in place right under the prompt box and above the button. The original stays in the input above, the card does not repeat it, and it has no card title. Adopt writes it back, Discard throws it away, Retry asks again. The rewrite keeps subject and intent and only adds visual detail: composition, light, palette, material, mood, style. It follows the input language. The video side additionally asks for motion, camera movement and pacing.

Two backends, picked from a dropdown and persisted to settings:

Backend	Behavior
agnes (default)	agnes-3.0-flash via Agnes Chat Completions, same key as generation
dsh	Reuses the harness' current conversation model (agentDefaultModel.currentSelection()). If the host has no LLM service or the profile has no model configured, the option shows as unavailable and nothing else is affected

4. Sidebar panel

Four tabs, left to right: Images, Videos, Library, Settings. The panel shares the same host code path as the tools, so behavior is identical.

• Images: prompt, size, ratio, count, optional reference images. Thumbnails appear immediately.
• Videos: prompt, mode, seconds, aspect ratio, resolution. Keyframe mode reveals first/last frame inputs, reference mode reveals image and audio rows. After submit the button area shows queue and render progress and polls automatically. On completion it never plays in the panel; it shows a poster plus "Open with default app". History is not on this tab. Everything is collected in the Library.
• Progress bar: a running image job shows an indeterminate striped bar plus elapsed seconds. Image generation is a single call with no percentage, so no number is invented. A running video job shows the real upstream percentage (0/10/70/100), plus status and elapsed seconds, for example Rendering: in_progress · 10% · 84s elapsed. When the upstream percentage has not moved for more than 12 seconds, which is very common because the 10% step alone can sit for two minutes, the bar switches from a fill to flowing stripes and the caption becomes Upstream still reports 10% — rendering · 132s elapsed. It says where the number comes from instead of faking motion or inventing digits, and snaps back to a real fill the moment upstream jumps to 70 or 100. The bar occupies the slot where the old status card was.
• Library: a segmented switch at the top toggles Image library and Video library. Each has its own count and refresh button. Both render as small card grids, about 116 px per image column and 182 px per video column. Videos show a poster only and never play here.
• Cards carry a thumbnail by default. Collapsed, a card shows a thumbnail on top and the file name below, 96×96 for images and 160×90 for video posters. Clicking the file name or the chevron expands the card to reveal the prompt, size, ratio, duration, bytes and the action buttons. Hovering the thumbnail scales it slightly, 1.03×, and draws an orange ring as a hint that it can be double-clicked.
• Opening local files: double-click the thumbnail or poster, or use "Open with default app" in the expanded area. The file goes to the desktop's default association, images to the image viewer and videos to the default player, which is semantically the same as double-clicking in Explorer. "Reveal in folder" selects the file instead. The expand button and the file name only toggle expansion; they never open files.
• Settings: API key (saved into the plugin data directory), model, default size and ratio, video and optimizer defaults, output directory. No key ships with the plugin. Without a key the image, video and optimize (agnes backend) buttons are disabled and a hint appears at the top of Settings.
• Action buttons. Per image: Insert into composer, copy path, open with default app, reveal in folder, delete. Per video: Insert link, copy path, open with default app, reveal in folder, delete.

Output layout:

• Images /agnes-images//- .
• Videos /agnes-videos//- .mp4
• Video poster /-cover.png, produced by the panel's frame grab

Repeated names within the same second get -2, -3. CJK prompts are preserved as-is.

Video posters: browser frame grab, no ffmpeg

Video cards must show something even when collapsed, but the plugin does not decode video on the host. The author's machine has no ffmpeg, and the core packages expose no host-side thumbnail service, so the frame grab happens in the browser:

1. When a video task completes, on the poll that first yields file, the panel builds a hidden `` in grabFrame(), waits for loadeddata, seeks to min(0.2s, duration/2), and on seeked runs canvas.drawImage() plus toDataURL('image/png'). Max width 640 px, 8-second timeout, and any failure resolves to an empty string.
2. The frame is posted to POST /api/agnes-image/video-cover as { file, dataUrl }. The host writes -cover.png next to the video and records cover back into video-history.json and, for a matching file, into video-tasks.json.
3. Cards prefer record.cover as an . Without a poster they fall back to the first frame of a , with no controls and never playing. If even that fails they show "No cover yet (double-click to open with the default player)".

A failed grab only logs console.warn('[agnes-image] cover capture failed:'). A missing poster never blocks anything. Refreshing the Library also back-fills at most 3 old records without a poster, each path attempted once and deduplicated with useRef(new Set()).

video-cover reuses the same safety boundary as /open: only a file recorded in video-history.json is accepted, otherwise 404 no history record for that file. The dataUrl must start with data:image/png;base64,, be non-empty, and be at most 6 MB.

Interaction & motion

All of it is CSS, and everything degrades to no animation when the OS asks for reduced motion (@media (prefers-reduced-motion:reduce)):

Where	Effect	How
Tab switch (Images / Videos / Library / Settings)	450 ms horizontal slide, 40 px, decelerating cubic-bezier(.16,1,.3,1)	The tab body remounts on a keyed wrapper and replays .dsh-agnes-slide / @keyframes dsh-agnes-slide-in. Direction follows tab order: moving forward (Library → Videos) enters from the right, moving back from the left. The left variant only changes animation-name via .is-left; duration and curve are shared.
Active-tab "top" ring	An orange inset-ring pill slides to the active tab in step with the body	.dsh-agnes-pill with transition: transform 450ms cubic-bezier(.16,1,.3,1). Width and offset are computed with calc() from the equal-width tab columns (translateX(calc(*100% + *gap))). No DOM measurement and no ResizeObserver, so it stays aligned when the sidebar is resized.
Library segmented switch	Same feel: body slides, pill slides	The key includes libraryKind, so direction follows the switch. It reuses the same pillStyle().
Card expand / collapse	200 ms true-height transition	The expanded area stays in the DOM, so its height is stable. The wrapper toggles .dsh-agnes-expand{grid-template-rows:0fr} against .is-open{grid-template-rows:1fr}, with overflow:hidden;min-height:0 on the child. No height measurement and no extra layout pass.
Thumbnail / poster hover	1.03× scale and a 2 px orange ring	.dsh-agnes-media:hover. Under reduced motion only the transition is disabled; the affordance stays.
Indeterminate progress	Flowing stripes	.dsh-agnes-bar.is-indeterminate>i plus @keyframes dsh-agnes-bar-slide. Under reduced motion it is a static translucent bar.

Horizontal overflow from the slide is clipped at the panel root with overflow-x:hidden, otherwise a horizontal scrollbar appears mid-animation. The pill is already in place on first render, so opening the panel never shows it sliding in from the first tab.

Tab switching only animates transform, never opacity. The tab body remounts, and starting from opacity:0 made tall form tabs flash the bare dark panel background for about 100 ms before growing in, which was far more jarring than the animation itself. It is fully opaque from frame one, direction comes from the 40 px offset alone, and there is no gap.

The collapsed expand area stays in the DOM but is neither focusable nor clickable. CardShell uses a ref + useEffect to set inert on the container while collapsed and remove it when expanded. That is the DOM API rather than a JSX prop, so React 18 and 19 behave the same.

Key policy: panel only, no shipped credential

The only source is the key saved in the panel's Settings tab, written to %DSH_HOME%\agnes-image\settings.json.

Config has no apiKey field and there is no environment-variable fallback. The plaintext key row and apiKeyEnv that early versions had are gone. resolveKey() reads the settings panel only. Without a key, status.hasKey === false and keySource === 'none', and every generation or optimize call on the agnes backend fails with:

No Agnes API key configured: open the panel Settings tab and save one first (the plugin ships without a key).

The reasoning: plugins get distributed and copied around, so a credential may only ever live in the user's own data directory, never inside plugin source. status reports just keySource (panel / none) and a masked keyHint holding the first six and last four characters. It never reports the full key.

Data files

Path	Contents
%DSH_HOME%\agnes-image\settings.json	configuration saved by the panel
%DSH_HOME%\agnes-image\history.json	image history (newest first, capped by historyLimit)
%DSH_HOME%\agnes-image\video-history.json	completed video history (newest first)
%DSH_HOME%\agnes-image\video-tasks.json	video task table (deduplicated by videoId, last 100 kept). Polling resumes after a page reload.

The task table is what makes "close the page and come back" work. The host remembers baseDir per videoId and downloads the mp4 into the original workspace when it finishes. The download happens exactly once.

HTTP routes

The panel uses these. Trust fence and cookie auth are automatic.

Method	Path	Purpose
GET	/api/agnes-image/status	key source (panel/none), model, defaults, available options, optimizer and LLM availability
GET	/api/agnes-image/history?limit=	image history
POST	/api/agnes-image/generate	{ prompt, size, ratio, count, referenceImages, sessionId, cwd }
POST	/api/agnes-image/settings	save configuration (clearApiKey: true clears the panel key)
POST	/api/agnes-image/delete	{ file }, only a file recorded in history
POST	/api/agnes-image/open	{ file, action: 'default'|'reveal' } → open with the OS default association / reveal in folder
POST	/api/agnes-image/video-create	{ prompt, mode, seconds, size, aspectRatio, firstFrame, lastFrame, images, audios, seed, sessionId, cwd } → { ok, task }
GET	`/api/agnes-image/video-