Vmarcelo49/dsh-local-models ↗★ 0
dsh-local-models
dsh addon: a Local Models settings tab that starts and stops a llama-server child process living with the dsh host process.
AI 分析
核心用途是便捷地在本地加载和运行 GGUF 格式的开源模型。适合需要在本地离线运行大模型、微调上下文及草稿头参数的开发者。
インストール
$
npx -p @deepseek-ai/dsh dsh plugin --profile web add github:Vmarcelo49/dsh-local-modelsドキュメント
README 全文を読む ↗Usage
- Choose GGUF… — pick a model file (Home / Models shortcuts, Up navigation).
- Tune context, Max MTP head (fixed draft; capped at 3 — deeper collapses at large ctx), thinking level, optional mmproj and MoE settings.
- Load model, watch the status card, inspect output via Open terminal.
- Register in dsh — the route (default
local-) appears in the Models picker. - Alternatively, save profiles and Start router (from profiles) for a multi-model endpoint.
Configuration
| Variable | Default | Meaning |
|---|---|---|
LOCAL_MODELS_PORT | 8080 | llama-server port |
LOCAL_MODELS_BIN | ~/Projetos/llama.cpp/build/bin/llama-server | server binary |
LOCAL_MODELS_SHORTCUTS | /mnt/raid0/GGUF | colon-separated file-browser shortcut dirs (name=path for custom labels) |
LOCAL_MODELS_MMPROJ_CPU | 1 | vision projector weights in RAM (0 = offload to GPU) |
LOCAL_MODELS_ROUTER_MAX | 1 | max simultaneously resident router models |
LOCAL_MODELS_MAX_IMAGE_BYTES | 10485760 | vision image guard |
LOCAL_MODELS_IMAGE_PIXEL_BUDGET | 4194304 | vision pixel budget |
DSH_HOME | ~/.dsh | data dir (local-models/profiles.json, llama-server.log) |
Launch flags are fixed to the validated daily config: full offload, -b 2048 -ub 512 -t 4 -np 1, --flash-attn on --kv-unified, --cache-type-k q8_0 --cache-type-v q4_0, MTP --spec-type draft-mtp --spec-draft-n-max N --spec-draft-p-min 0.75 (dropped above 131072 ctx).