dsh-llm-verifier-pro

Configuration

Zero-config default: with no explicit baseUrl / apiKey / model (and the panel's Verifier fields empty), the verifier follows the session — same provider route, endpoint and model as the conversation. Turning on Best-of-N alone gives the paper's self-verification experience: candidates are sampled as variants of the conversation's own model, and that same model grades them. The endpoint for the session provider is read from its settings namespace (llm-pi-ai.providers. style). Only when the session provider is unknown does the resolution fall back to:

plugin config → the verifier settings section → session provider endpoint → OPENAI_BASE_URL / OPENAI_API_KEY / DEEPSEEK_API_KEY → api.deepseek.com.

The verifier must sit on an endpoint that returns token-level logprobs (vLLM, SGLang, OpenAI, DeepSeek, and modern Ollama all do; a plain gateway that strips logprobs will not). Non-DeepSeek servers get the optional vLLM/SGLang prefill pass so score tags land exactly at the label position.