mirror of
https://github.com/NCBM/plyngent.git
synced 2026-11-07 07:17:54 +08:00
docs: describe the thinking-strength settings
This commit is contained in:
@@ -52,15 +52,17 @@ lmproto/deepseek/anthropic/ ← DeepSeek Anthropic client (extends Anthropic
|
||||
- **`openai/client.py`** — platform `OpenAIClient` (`kind="responses"`): chat completions + `responses` / `get_response` / `delete_response`.
|
||||
- **`anthropic/model.py`** — Anthropic Messages request/response + SSE event models.
|
||||
- **`anthropic/client.py`** — `AnthropicClient` (`kind="messages"`): `POST /messages` + `GET /models`.
|
||||
- **DeepSeek** — `DeepseekOpenAIClient` (`kind="chat_completions"`); models add `reasoning_content`, `prefix`, `ThinkingOptions`. Config default model ids: `deepseek-v4-flash`, `deepseek-v4-pro`. `DeepseekResponsesClient` (`kind="responses"`) extends the OpenAI Responses client for `convention = "responses"` (SSE stops at terminal events — DeepSeek sends no `[DONE]`). `DeepseekAnthropicClient` (`kind="messages"`) extends `AnthropicClient` for `convention = "anthropic"` (base `https://api.deepseek.com/anthropic`; `models()` returns `[]`).
|
||||
- **DeepSeek** — `DeepseekOpenAIClient` (`kind="chat_completions"`); models add `reasoning_content`, `prefix`, `ThinkingOptions`, and it injects the `thinking` flag (`enabled`, or `disabled` for `reasoning_effort = "none"`) after encoding. Config default model ids: `deepseek-v4-flash`, `deepseek-v4-pro`. `DeepseekResponsesClient` (`kind="responses"`) extends the OpenAI Responses client for `convention = "responses"` (SSE stops at terminal events — DeepSeek sends no `[DONE]`). `DeepseekAnthropicClient` (`kind="messages"`) extends `AnthropicClient` for `convention = "anthropic"` (base `https://api.deepseek.com/anthropic`; `models()` returns `[]`).
|
||||
|
||||
### Config (`config/`)
|
||||
|
||||
TOML load/store (`ConfigStore`): `[providers]` tagged union presets, `[database]` section. Default path via platformdirs. Providers may set `timeout` as a float or `{ connect, read }` (`HttpTimeoutConfig`); omitted → product defaults (10s connect / 600s read). `DeepseekProvider.convention` (Literal: `""`/`"openai"`/`"anthropic"`/`"responses"`) selects the DeepSeek API surface — `"responses"` → OpenAI Responses, `"anthropic"` → Anthropic Messages on `https://api.deepseek.com/anthropic` (default url follows the convention); a per-model `ModelConfig.convention` overrides it (empty = inherit). `[mcp]` (`McpConfig`): `servers.<name>` stdio server definitions (`McpServerConfig`: `command`/`args`/`env`/`cwd`/`timeout`/`read_only`, `url` reserved for streamable HTTP) + `disable` gate; defining a server opts it in. `[skills]` (`SkillsConfig`): `paths` (explicit roots first), `discover` (`plyngent` user dir + `claude` dirs) with `include_project_roots`, `inject_catalog` / `max_catalog_skills`, `enabled`, and `allow_write` (standing write grant); a malformed section degrades to defaults like `[agent]` / `[mcp]`.
|
||||
|
||||
`reasoning.py`: thinking strength as `reasoning_effort` (`ReasoningEffortConfig`: `""` = send nothing | `none` … `high` … `max`) and `thinking_budget_tokens`, settable on `[agent]`, a provider, or a model entry — `resolve_reasoning(agent, provider, model)` merges the layers per field (model > provider > agent) and clamps `max` (DeepSeek chat only) to `xhigh` for every other surface, returning a `ReasoningConfig` the agent loop and `create_client` consume.
|
||||
|
||||
### Runtime (`runtime/`)
|
||||
|
||||
`create_client(provider)` maps config `Provider` → protocol client by effective preset (model-level `preset`/`url` overrides apply). `normalize_http_timeout` feeds OpenAI-compatible configs and Anthropic. Mapping: `openai` → `OpenAIClient` (agent uses Responses), `openai-compatible` / `deepseek` → chat-completions clients (deepseek with effective `convention = "responses"` → `DeepseekResponsesClient`, `"anthropic"` → `DeepseekAnthropicClient`), `anthropic` → `AnthropicClient`.
|
||||
`create_client(provider, *, model=None, reasoning=None)` maps config `Provider` → protocol client by effective preset (model-level `preset`/`url` overrides apply); `reasoning` (resolved `ReasoningConfig`) carries the Anthropic `thinking_budget_tokens` onto `AnthropicConfig` — the other surfaces read the effort off the request param — and is derived from the provider/model layers alone when omitted. `normalize_http_timeout` feeds OpenAI-compatible configs and Anthropic. Mapping: `openai` → `OpenAIClient` (agent uses Responses), `openai-compatible` / `deepseek` → chat-completions clients (deepseek with effective `convention = "responses"` → `DeepseekResponsesClient`, `"anthropic"` → `DeepseekAnthropicClient`), `anthropic` → `AnthropicClient`.
|
||||
|
||||
MCP (`runtime/mcp_client.py`): `McpManager` owns one `McpServerConnection` per `[mcp]` server — a spawned stdio subprocess speaking newline-delimited JSON-RPC 2.0: initialize handshake, `tools/list` (cursor pagination), `tools/call`, `notifications/tools/list_changed` → `tools_stale`. Spawn/handshake failures record `connection.error` (status string), never raise; `ensure_started` / `restart(config)` / `aclose` / `refresh_tools`; `statuses()` → `(server, status, tool_count)` rows; `instructions()` → `(server, text)` for every connected server that returned optional `InitializeResult.instructions` (usage guidance captured per connection); per-request timeout from config (default 30s).
|
||||
|
||||
@@ -76,15 +78,15 @@ Async SQLAlchemy + aiosqlite. `MemoryStore`: schema init (+ lightweight SQLite `
|
||||
|
||||
- **Kind-based dispatch**: clients expose `kind` (`chat_completions` | `responses` | `messages`). Agent history stays chat-completions-shaped; transport adapters return synthetic `ChatCompletionResponse` / stream chunks.
|
||||
- **`responses_bridge` / `responses_dispatch`**: chat ↔ OpenAI Responses; stream text/reasoning deltas; tool calls + usage on `response.completed`.
|
||||
- **`messages_bridge` / `messages_dispatch`**: chat ↔ Anthropic Messages; system fold; tool_use/tool_result; stream text + tool argument fragments.
|
||||
- **`messages_bridge` / `messages_dispatch`**: chat ↔ Anthropic Messages; system fold; tool_use/tool_result; stream text + tool argument fragments; `thinking` from the request's `reasoning_effort` (level → token budget, floor 1024, `max_tokens` grown when the budget would leave no room) plus the client's configured `thinking_budget_tokens`.
|
||||
- **Provider-side tools**: `OpenAIProvider.provider_tools` (list of dicts; default `[{type="web_search"}]` when omitted; `[]` disables) threaded via `ChatAgent`/`run_chat_loop` into Responses only; never executed by `ToolRegistry`.
|
||||
- **`@tool` / `ToolRegistry`**: decorator infers JSON Schema from type hints; execute tools by name. Before running, `execute` answers a call that omits a required argument with the accepted names (`_missing_required_error`) — the handler's bare `TypeError` never reaches the model, and no confirm prompt fires for a call that cannot run. Shape mistakes inside a *present* argument are the tool's own job (see the `chat` bullet).
|
||||
- **`run_chat_loop`**: multi-round tool loop; default **streaming** text deltas + stream tool-call merge; parallel tools; tool-result char budget; soft context compact on request (**API-calibrated** after first usage when available); cooperative cancel points; optional `on_limit`.
|
||||
- **`run_chat_loop`**: multi-round tool loop; default **streaming** text deltas + stream tool-call merge; parallel tools; tool-result char budget; soft context compact on request (**API-calibrated** after first usage when available); cooperative cancel points; optional `on_limit`; resolved `reasoning_effort` on every request (see Config).
|
||||
- **`ChatAgent`**: optional `MemoryStore` (user message persisted immediately; **completed tool batches checkpointed** mid-turn; unfinished assistant suffix rolled back on failure); `stream`; system prompt; `provider_tools`; `retry()` continues incomplete turns (user-only **or** after committed tools — does not re-run those tools).
|
||||
- **`/compact`**: soft-compact tool dumps → model summary (no tools) via the same kind dispatch → **new** session seeded with summary message. Soft-compact keeps the last **12** tool results full-size by default (`DEFAULT_RECENT_TOOL_RESULTS`); older tool payloads shrink first.
|
||||
- Events: text_delta, **reasoning_delta**, assistant_message, tool_call/result, max_rounds, **error** (`retryable`/`source`), **cancelled** (`reason`), **usage** (`TokenUsage`).
|
||||
- Usage: API `usage` from completions (stream with `include_usage`); **char≈token fallback** (~4 chars/token) when omitted; **context size** = last request ``prompt_tokens`` (API preferred); `last_turn_usage` / `session_usage` are **billed sums** (tool rounds re-send history); CLI end-of-turn + `/status`.
|
||||
- Config ``[agent]``: `system_prompt` (persona; default `DEFAULT_SYSTEM_PROMPT`; `""` omits persona), `tool_directives` (tool playbook; default `DEFAULT_TOOL_DIRECTIVES`; `""` omits playbook; both empty → no system), `mcp_instructions` (fold connected MCP servers' initialize ``instructions`` into the system prompt when tools on; default true), `[skills] inject_catalog` (fold the skill catalog — name/source/description only — the same way), `allow_paths` (static out-of-workspace pre-allow `path → read|write|exec`; never prompts), `auto_continue_limits` (auto-raise tool/PTY limits without prompting; default false), `max_tool_result_chars`, `parallel_tools`, `confirm_destructive`, `path_denylist`, `max_context_tokens` (default 200k est. tokens). Truncated tool results carry a model hint (`[Truncated (N chars max; M omitted). You may ask for user to increase the limit via configuration file.]`) so the model can ask the user to raise `max_tool_result_chars`; request-time compact shrinks (`hint=False`) keep the terse `...[truncated N characters]` marker.
|
||||
- Config ``[agent]``: `system_prompt` (persona; default `DEFAULT_SYSTEM_PROMPT`; `""` omits persona), `tool_directives` (tool playbook; default `DEFAULT_TOOL_DIRECTIVES`; `""` omits playbook; both empty → no system), `mcp_instructions` (fold connected MCP servers' initialize ``instructions`` into the system prompt when tools on; default true), `[skills] inject_catalog` (fold the skill catalog — name/source/description only — the same way), `allow_paths` (static out-of-workspace pre-allow `path → read|write|exec`; never prompts), `auto_continue_limits` (auto-raise tool/PTY limits without prompting; default false), `reasoning_effort` / `thinking_budget_tokens` (thinking strength; see Config), `max_tool_result_chars`, `parallel_tools`, `confirm_destructive`, `path_denylist`, `max_context_tokens` (default 200k est. tokens). Truncated tool results carry a model hint (`[Truncated (N chars max; M omitted). You may ask for user to increase the limit via configuration file.]`) so the model can ask the user to raise `max_tool_result_chars`; request-time compact shrinks (`hint=False`) keep the terse `...[truncated N characters]` marker.
|
||||
|
||||
### Tools (`tools/`)
|
||||
|
||||
|
||||
@@ -154,6 +154,11 @@ max_context_tokens = 200000
|
||||
# allow_paths = { "/data/datasets" = "read", "/tmp/build" = "exec" }
|
||||
# Auto-raise tool/PTY limits without prompting (same as answering yyy).
|
||||
# auto_continue_limits = false
|
||||
# Thinking strength: "" (provider default) | none | minimal | low | medium |
|
||||
# high | xhigh | max. See "Thinking strength" below; providers and single
|
||||
# models may override either value.
|
||||
# reasoning_effort = "medium"
|
||||
# thinking_budget_tokens = 8000 # Anthropic only; overrides the level's budget
|
||||
|
||||
# Optional skills (instruction directories). See doc/skills.md.
|
||||
# [skills]
|
||||
@@ -172,6 +177,20 @@ max_context_tokens = 200000
|
||||
|
||||
Per-provider **`timeout`** is passed to the HTTP session for chat/completions, Responses, and `GET /models`. A single number sets one timeout; `{ connect, read }` splits TCP/TLS setup vs idle wait between response bytes (SSE can run longer than `read` while chunks keep arriving). Tool/process timeouts (`run_argv`, PTY, policy confirm) are separate.
|
||||
|
||||
**Thinking strength** (`reasoning_effort`) can be set on `[agent]`, on a provider, or on a single model: the most specific layer wins, per field, so a model may override only the effort or only the budget. `""` sends nothing and lets the provider apply its own default. Each API surface maps the value onto its own wire field — chat completions send `reasoning_effort`, Responses send `reasoning.effort`, and Anthropic has no effort level at all, so it sends `thinking` with the token budget that level maps onto (`minimal` 1024 … `xhigh`/`max` 32768; `none` disables thinking). `max` is DeepSeek's own top level on chat completions and clamps to `xhigh` everywhere else. `thinking_budget_tokens` overrides that mapping; either way the request stays valid: the budget never drops below 1024, and when it would swallow the output limit `max_tokens` grows so the answer still has room. DeepSeek chat keeps `thinking.type = "enabled"` unless the effort is `none`.
|
||||
|
||||
```toml
|
||||
[providers.deepseek]
|
||||
reasoning_effort = "high" # provider default
|
||||
|
||||
[providers.deepseek.models]
|
||||
"deepseek-v4-pro" = { text = true, reasoning_effort = "max" } # per-model wins
|
||||
|
||||
[providers.anthropic]
|
||||
reasoning_effort = "medium"
|
||||
thinking_budget_tokens = 16000 # explicit Anthropic thinking budget
|
||||
```
|
||||
|
||||
Third-party **plugins**: install a package that declares `project.entry-points."plyngent.tools"` (and later other groups), then allowlist the entry-point name under **`[plugins].enable`**. Details: [doc/plugins.md](doc/plugins.md).
|
||||
|
||||
**Skills** (instruction directories with a `SKILL.md`): discovered under `[skills].paths`, our own user skills directory, and other harnesses' skill directories (`~/.claude/skills`, plus `<workspace>/.claude/skills` when `include_project_roots`). The catalog — name, source, description — is folded into the system prompt when tools are on; the model reads a `SKILL.md` with `skill_read`, searches with `skill_search`, and creates or updates skills with `skill_create` / `skill_edit` behind a timed `once`/`session`/`no` grant (`[skills].allow_write = true` for a standing one; the path denylist still wins and `--yes`/`/yolo` never skips it). Reading needs no grant: skill roots are read-only roots, so `read_file` / `regex_files` see them too while writes stay denied. Details: [doc/skills.md](doc/skills.md).
|
||||
|
||||
@@ -96,6 +96,20 @@ max_context_tokens = 200000
|
||||
# directive_reminder_tokens = 100000
|
||||
# directive_reminder_text = ""
|
||||
|
||||
# Thinking strength, resolved as [agent] < provider < model (the most specific
|
||||
# layer wins, per field). "" = send nothing and keep the provider's own default.
|
||||
# none | minimal | low | medium | high | xhigh | max
|
||||
# ``max`` is DeepSeek's own top level on chat completions and clamps to ``xhigh``
|
||||
# everywhere else. The agent maps the value onto each API surface: chat
|
||||
# completions send ``reasoning_effort``, Responses send ``reasoning.effort``, and
|
||||
# Anthropic — which has no effort level — sends ``thinking`` with the token
|
||||
# budget the level maps onto (minimal 1024 … xhigh/max 32768, none = disabled).
|
||||
# reasoning_effort = "medium"
|
||||
# Explicit Anthropic thinking budget in tokens, overriding that mapping; the
|
||||
# request stays valid either way (floor 1024, and ``max_tokens`` grows when the
|
||||
# budget would leave no room for the answer).
|
||||
# thinking_budget_tokens = 8000
|
||||
|
||||
# Optional: custom compact/summarisation prompts (empty = built-in defaults).
|
||||
# compact_system_prompt = ""
|
||||
# compact_user_prefix = "Summarize the following conversation:\n\n{transcript}"
|
||||
@@ -120,6 +134,9 @@ access_key_or_token = "sk-replace-me"
|
||||
# preset = "openai"
|
||||
# access_key_or_token = "sk-replace-me"
|
||||
# url = "https://api.openai.com/v1"
|
||||
# # Thinking strength for every model of this provider ("" = inherit [agent];
|
||||
# # a model entry may override it). Values as in the [agent] section above.
|
||||
# reasoning_effort = "medium"
|
||||
# # Hosted/provider-side tools (OpenAI runs them; not local ToolRegistry).
|
||||
# # Default if omitted: [{ type = "web_search" }]. Use [] to disable.
|
||||
# # Examples: web_search, file_search (+ vector_store_ids), code_interpreter, mcp.
|
||||
@@ -135,6 +152,8 @@ access_key_or_token = "sk-replace-me"
|
||||
# # `preset = "openai"` means /responses; `openai-compatible` means /chat/completions;
|
||||
# # `anthropic` means native /messages.
|
||||
# # "claude-sonnet" = { text = true, preset = "anthropic", url = "https://api.anthropic.com/v1" }
|
||||
# # A model entry may also set its own thinking strength, e.g. skip thinking for
|
||||
# # a cheap model: "gpt-5.4-nano" = { text = true, reasoning_effort = "none" }
|
||||
|
||||
# --- Anthropic (Messages API) ---
|
||||
# [providers.anthropic]
|
||||
@@ -142,10 +161,14 @@ access_key_or_token = "sk-replace-me"
|
||||
# access_key_or_token = "sk-ant-replace-me"
|
||||
# # url defaults to https://api.anthropic.com/v1 when empty
|
||||
# timeout = { connect = 10, read = 600 }
|
||||
# # Anthropic has no effort level: an effort becomes a thinking token budget
|
||||
# # (see the [agent] section above). Set the budget explicitly to pin it:
|
||||
# reasoning_effort = "high"
|
||||
# thinking_budget_tokens = 16000
|
||||
#
|
||||
# [providers.anthropic.models]
|
||||
# "claude-sonnet-4-20250514" = { text = true }
|
||||
# "claude-opus-4-20250514" = { text = true }
|
||||
# "claude-opus-4-20250514" = { text = true, reasoning_effort = "xhigh" }
|
||||
|
||||
# --- DeepSeek (OpenAI-compatible / Responses / Anthropic conventions) ---
|
||||
# Default models if [providers.*.models] is omitted: deepseek-v4-flash, deepseek-v4-pro.
|
||||
@@ -164,7 +187,9 @@ access_key_or_token = "sk-replace-me"
|
||||
# convention = "responses"
|
||||
# # Legacy location (parsed but ignored; prefer the typed field above):
|
||||
# # extras = { convention = "openai" }
|
||||
# # Thinking strength for this provider; per-model entries below may override it.
|
||||
# # reasoning_effort = "high"
|
||||
#
|
||||
# [providers.deepseek.models]
|
||||
# "deepseek-v4-flash" = { text = true, convention = "responses" }
|
||||
# "deepseek-v4-pro" = { text = true }
|
||||
# "deepseek-v4-pro" = { text = true, reasoning_effort = "max" }
|
||||
|
||||
@@ -37,6 +37,9 @@ _MINIMAL_CONFIG = """\
|
||||
# confirm_destructive = true
|
||||
# path_denylist = ["/secrets/", ".ssh/"]
|
||||
# max_context_tokens = 200000
|
||||
# # Thinking strength: "" (provider default) | none | minimal | low | medium |
|
||||
# # high | xhigh | max; providers and models may override it.
|
||||
# # reasoning_effort = "medium"
|
||||
#
|
||||
# # Optional compact prompts (empty = use built-in defaults).
|
||||
# # compact_system_prompt = ""
|
||||
|
||||
Reference in New Issue
Block a user