mirror of
https://github.com/NCBM/plyngent.git
synced 2026-11-07 07:17:54 +08:00
docs: describe the adaptive-thinking retry
This commit is contained in:
@@ -78,7 +78,7 @@ Async SQLAlchemy + aiosqlite. `MemoryStore`: schema init (+ lightweight SQLite `
|
||||
|
||||
- **Kind-based dispatch**: clients expose `kind` (`chat_completions` | `responses` | `messages`). Agent history stays chat-completions-shaped; transport adapters return synthetic `ChatCompletionResponse` / stream chunks.
|
||||
- **`responses_bridge` / `responses_dispatch`**: chat ↔ OpenAI Responses; stream text/reasoning deltas (`response.reasoning_text.done` repeats what its fragments spelled out, so the loop streams each part once and still keeps the whole text in the round's message); tool calls + usage on `response.completed`; a request carrying `tools` returns every round's reasoning as a `reasoning` item of `reasoning_text` parts — "必须完整回传 `reasoning_content` … 即使该轮模型未实际进行工具调用" (a missing item and an empty part are both a 400, so a round that kept none sends a blank line) — and a round's `function_call` items stay contiguous ahead of their outputs (`call, output, call, output` was a 400: the second call reads as a round of its own without reasoning); reasoning is read back from the item's `content` parts.
|
||||
- **`messages_bridge` / `messages_dispatch`**: chat ↔ Anthropic Messages; system fold; tool_use/tool_result; stream text + tool argument fragments; `thinking` from the request's `reasoning_effort` (level → token budget, floor 1024, `max_tokens` grown when the budget would leave no room) plus the client's configured `thinking_budget_tokens`; a server tool (`server_tool_use` + its `web_search_tool_result`) is the surface's own business — it streams `input_json_delta` fragments like a call but nothing local asked for it, so only a `tool_use` block becomes a tool call and nothing is executed; `thinking` and the two search blocks are kept as `AssistantChatMessage.provider_blocks` (the thinking text also lands in `reasoning_content`, so it streams as reasoning) and go back with the round, which is also why a round that kept only those blocks is not an empty completion; a chat-completions request drops the field (`compat.coerce_chat_messages`).
|
||||
- **`messages_bridge` / `messages_dispatch`**: chat ↔ Anthropic Messages; system fold; tool_use/tool_result; stream text + tool argument fragments; `thinking` from the request's `reasoning_effort` (level → token budget, floor 1024, `max_tokens` grown when the budget would leave no room) plus the client's configured `thinking_budget_tokens`; a model that refuses that shape (from Claude 4.7 on, `thinking.type.enabled` is a 400 asking for adaptive thinking + `output_config.effort`) gets one retry as `thinking.type = "adaptive"` with the level in `output_config.effort`, and the model is remembered on the client so only its first request pays the round trip (a refusal before the stream yields anything is the only one retried — a later failure has already been seen); a server tool (`server_tool_use` + its `web_search_tool_result`) is the surface's own business — it streams `input_json_delta` fragments like a call but nothing local asked for it, so only a `tool_use` block becomes a tool call and nothing is executed; `thinking` and the two search blocks are kept as `AssistantChatMessage.provider_blocks` (the thinking text also lands in `reasoning_content`, so it streams as reasoning) and go back with the round, which is also why a round that kept only those blocks is not an empty completion; a chat-completions request drops the field (`compat.coerce_chat_messages`).
|
||||
- **Provider-side tools**: `provider_tools` on the OpenAI **and** DeepSeek presets (list of dicts; `[{type="web_search"}]` when omitted; `[]` disables) threaded via `ChatAgent`/`run_chat_loop` into Responses and Messages; never executed by `ToolRegistry`. The Messages surface declares its own server tool instead — a bare `web_search` becomes `{type="web_search_20250305", name="web_search"}` (the search then runs inside the response, and its `encrypted_content` results go back verbatim with the round — `AssistantChatMessage.provider_blocks`) — and a DeepSeek provider on a surface that ignores hosted tools (`config.routing.local_web_search`) gets the local `web_search` tool from `tools/net/search.py` instead.
|
||||
- **`@tool` / `ToolRegistry`**: decorator infers JSON Schema from type hints; execute tools by name. Before running, `execute` answers a call that omits a required argument with the accepted names (`_missing_required_error`) — the handler's bare `TypeError` never reaches the model, and no confirm prompt fires for a call that cannot run. Shape mistakes inside a *present* argument are the tool's own job (see the `chat` bullet). `@tool(writes=("path",))` names a handler's file arguments: one batch applies calls that name the same file one at a time, in call order — the lock is taken before the confirm prompt, so a pending prompt cannot be overtaken — while different files keep running in parallel.
|
||||
- **`run_chat_loop`**: multi-round tool loop; default **streaming** text deltas + stream tool-call merge; parallel tools; tool-result char budget; soft context compact on request (**API-calibrated** after first usage when available); cooperative cancel points; optional `on_limit`; resolved `reasoning_effort` on every request (see Config).
|
||||
|
||||
@@ -175,7 +175,7 @@ max_context_tokens = 200000
|
||||
|
||||
Per-provider **`timeout`** is passed to the HTTP session for chat/completions, Responses, and `GET /models`. A single number sets one timeout; `{ connect, read }` splits TCP/TLS setup vs idle wait between response bytes (SSE can run longer than `read` while chunks keep arriving). Tool/process timeouts (`run_argv`, PTY, policy confirm) are separate.
|
||||
|
||||
**Thinking strength** (`reasoning_effort`) can be set on `[agent]`, on a provider, or on a single model: the most specific layer wins, per field, so a model may override only the effort or only the budget. `""` sends nothing and lets the provider apply its own default. Each API surface maps the value onto its own wire field — chat completions send `reasoning_effort`, Responses send `reasoning.effort`, and Anthropic has no effort level at all, so it sends `thinking` with the token budget that level maps onto (`minimal` 1024 … `xhigh`/`max` 32768; `none` disables thinking). `max` is DeepSeek's own top level on chat completions and clamps to `xhigh` everywhere else. `thinking_budget_tokens` overrides that mapping; either way the request stays valid: the budget never drops below 1024, and when it would swallow the output limit `max_tokens` grows so the answer still has room. DeepSeek chat keeps `thinking.type = "enabled"` unless the effort is `none`.
|
||||
**Thinking strength** (`reasoning_effort`) can be set on `[agent]`, on a provider, or on a single model: the most specific layer wins, per field, so a model may override only the effort or only the budget. `""` sends nothing and lets the provider apply its own default. Each API surface maps the value onto its own wire field — chat completions send `reasoning_effort`, Responses send `reasoning.effort`, and Anthropic has no effort level at all, so it sends `thinking` with the token budget that level maps onto (`minimal` 1024 … `xhigh`/`max` 32768; `none` disables thinking); a model that no longer takes a budget answers 400 and wants adaptive thinking instead (from Claude 4.7 on), so the request is retried once as `thinking.type = "adaptive"` with the level in `output_config.effort`, and that model is remembered for the session — `thinking_budget_tokens` has no adaptive equivalent and is then ignored. `max` is DeepSeek's own top level on chat completions and clamps to `xhigh` everywhere else. `thinking_budget_tokens` overrides that mapping; either way the request stays valid: the budget never drops below 1024, and when it would swallow the output limit `max_tokens` grows so the answer still has room. DeepSeek chat keeps `thinking.type = "enabled"` unless the effort is `none`.
|
||||
|
||||
```toml
|
||||
[providers.deepseek]
|
||||
|
||||
@@ -105,7 +105,8 @@ max_context_tokens = 200000
|
||||
# reasoning_effort = "medium"
|
||||
# Explicit Anthropic thinking budget in tokens, overriding that mapping; the
|
||||
# request stays valid either way (floor 1024, and ``max_tokens`` grows when the
|
||||
# budget would leave no room for the answer).
|
||||
# budget would leave no room for the answer). A model that only takes adaptive
|
||||
# thinking ignores it (see the anthropic provider below).
|
||||
# thinking_budget_tokens = 8000
|
||||
|
||||
# Optional: custom compact/summarisation prompts (empty = built-in defaults).
|
||||
@@ -160,7 +161,9 @@ access_key_or_token = "sk-replace-me"
|
||||
# # url defaults to https://api.anthropic.com/v1 when empty
|
||||
# timeout = { connect = 10, read = 600 }
|
||||
# # Anthropic has no effort level: an effort becomes a thinking token budget
|
||||
# # (see the [agent] section above). Set the budget explicitly to pin it:
|
||||
# # (see the [agent] section above). A model that no longer takes one (from
|
||||
# # Claude 4.7 on) is retried with adaptive thinking, whose depth is the effort
|
||||
# # level itself. Set the budget explicitly to pin it:
|
||||
# reasoning_effort = "high"
|
||||
# thinking_budget_tokens = 16000
|
||||
#
|
||||
|
||||
Reference in New Issue
Block a user