From 556eca66db833510561a4b1e22001b2541d2ed50 Mon Sep 17 00:00:00 2001 From: worldmozara Date: Sat, 1 Aug 2026 00:34:48 +0800 Subject: [PATCH] docs: document openai responses and anthropic messages wiring --- CLAUDE.md | 22 +++++++++++++--------- README.md | 15 ++++++++++++--- doc/architecture.md | 14 ++++++++------ doc/plyngent.example.toml | 13 ++++++++++++- 4 files changed, 45 insertions(+), 19 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 2041cd3..d1bedae 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,15 +39,18 @@ All protocol models use `msgspec.Struct` — not dataclasses, not Pydantic. Opti ``` lmproto/openai_compatible/ ← Base: model, config, client lmproto/openai/ ← OpenAI platform: Responses + chat (extends base) +lmproto/anthropic/ ← Anthropic Messages: model, config, client lmproto/deepseek/openai_compat/ ← Extends base via inheritance + extra fields ``` - **`openai_compatible/model.py`** — tagged chat messages (`SystemChatMessage`, `UserChatMessage`, …), tools, request/response, streaming chunks. -- **`openai_compatible/client.py`** — `BaseOpenAIClient` / `OpenAICompatibleClient` via `niquests` async + SSE; `chat_completions`, `models` only (`OpenAIClient` is a compat alias). +- **`openai_compatible/client.py`** — `BaseOpenAIClient` / `OpenAICompatibleClient` via `niquests` async + SSE; `chat_completions`, `models` only. - **`openai_compatible/config.py`** — `OpenAIConfig` (token + base URL). - **`openai/model.py`** — OpenAI Responses API (`ResponsesCreateParam`, `Response`, function_call items, stream events). -- **`openai/client.py`** — platform `OpenAIClient`: chat completions + `responses` / `get_response` / `delete_response`. -- **DeepSeek** — `DeepseekOpenAIClient`; models add `reasoning_content`, `prefix`, `ThinkingOptions`. Config default model ids: `deepseek-v4-flash`, `deepseek-v4-pro`. +- **`openai/client.py`** — platform `OpenAIClient` (`kind="responses"`): chat completions + `responses` / `get_response` / `delete_response`. +- **`anthropic/model.py`** — Anthropic Messages request/response + SSE event models. +- **`anthropic/client.py`** — `AnthropicClient` (`kind="messages"`): `POST /messages` + `GET /models`. +- **DeepSeek** — `DeepseekOpenAIClient` (`kind="chat_completions"`); models add `reasoning_content`, `prefix`, `ThinkingOptions`. Config default model ids: `deepseek-v4-flash`, `deepseek-v4-pro`. ### Config (`config/`) @@ -55,7 +58,7 @@ TOML load/store (`ConfigStore`): `[providers]` tagged union presets, `[database] ### Runtime (`runtime/`) -`create_client(provider)` maps config `Provider` → protocol client. `provider_to_openai_config` normalizes `timeout` via `normalize_http_timeout` into `OpenAIConfig.timeout` for `niquests.AsyncSession`. OpenAI → `lmproto.openai.OpenAIClient` (Responses-capable); openai-compatible / deepseek(openai convention) → chat-completions clients; anthropic and deepseek anthropic convention raise `ProviderNotSupportedError`. +`create_client(provider)` maps config `Provider` → protocol client by effective preset (model-level `preset`/`url` overrides apply). `normalize_http_timeout` feeds OpenAI-compatible configs and Anthropic. Mapping: `openai` → `OpenAIClient` (agent uses Responses), `openai-compatible` / `deepseek` → chat-completions clients, `anthropic` → `AnthropicClient`. ### Memory (`memory/`) @@ -63,13 +66,14 @@ Async SQLAlchemy + aiosqlite. `MemoryStore`: schema init (+ lightweight SQLite ` ### Agent (`agent/`) -- **`ChatClient`** Protocol for `chat_completions` (agent history stays chat-shaped). -- **OpenAI Responses integration**: `ResponsesChatClient` adapts platform `OpenAIClient.responses` to `ChatClient`; selected automatically for `OpenAIProvider` in `create_client`. Compat/DeepSeek stay on chat completions. -- **Provider-side tools**: `OpenAIProvider.provider_tools` (list of dicts; default `[{type="web_search"}]` when omitted; `[]` disables) merged into Responses `tools` alongside local function tools; never executed by `ToolRegistry`. +- **Kind-based dispatch**: clients expose `kind` (`chat_completions` | `responses` | `messages`). Agent history stays chat-completions-shaped; transport adapters return synthetic `ChatCompletionResponse` / stream chunks. +- **`responses_bridge` / `responses_dispatch`**: chat ↔ OpenAI Responses; stream text/reasoning deltas; tool calls + usage on `response.completed`. +- **`messages_bridge` / `messages_dispatch`**: chat ↔ Anthropic Messages; system fold; tool_use/tool_result; stream text + tool argument fragments. +- **Provider-side tools**: `OpenAIProvider.provider_tools` (list of dicts; default `[{type="web_search"}]` when omitted; `[]` disables) threaded via `ChatAgent`/`run_chat_loop` into Responses only; never executed by `ToolRegistry`. - **`@tool` / `ToolRegistry`**: decorator infers JSON Schema from type hints; execute tools by name. - **`run_chat_loop`**: multi-round tool loop; default **streaming** text deltas + stream tool-call merge; parallel tools; tool-result char budget; soft context compact on request (**API-calibrated** after first usage when available); cooperative cancel points; optional `on_limit`. -- **`ChatAgent`**: optional `MemoryStore` (user message persisted immediately; **completed tool batches checkpointed** mid-turn; unfinished assistant suffix rolled back on failure); `stream`; system prompt; `retry()` continues incomplete turns (user-only **or** after committed tools — does not re-run those tools). -- **`/compact`**: soft-compact tool dumps → model summary (no tools) → **new** session seeded with summary message. Soft-compact keeps the last **12** tool results full-size by default (`DEFAULT_RECENT_TOOL_RESULTS`); older tool payloads shrink first. +- **`ChatAgent`**: optional `MemoryStore` (user message persisted immediately; **completed tool batches checkpointed** mid-turn; unfinished assistant suffix rolled back on failure); `stream`; system prompt; `provider_tools`; `retry()` continues incomplete turns (user-only **or** after committed tools — does not re-run those tools). +- **`/compact`**: soft-compact tool dumps → model summary (no tools) via the same kind dispatch → **new** session seeded with summary message. Soft-compact keeps the last **12** tool results full-size by default (`DEFAULT_RECENT_TOOL_RESULTS`); older tool payloads shrink first. - Events: text_delta, **reasoning_delta**, assistant_message, tool_call/result, max_rounds, **error** (`retryable`/`source`), **cancelled** (`reason`), **usage** (`TokenUsage`). - Usage: API `usage` from completions (stream with `include_usage`); **char≈token fallback** (~4 chars/token) when omitted; **context size** = last request ``prompt_tokens`` (API preferred); `last_turn_usage` / `session_usage` are **billed sums** (tool rounds re-send history); CLI end-of-turn + `/status`. - Config ``[agent]``: `system_prompt` (persona; default `DEFAULT_SYSTEM_PROMPT`; `""` omits persona), `tool_directives` (tool playbook; default `DEFAULT_TOOL_DIRECTIVES`; `""` omits playbook; both empty → no system), `max_tool_result_chars`, `parallel_tools`, `confirm_destructive`, `path_denylist`, `max_context_tokens` (default 200k est. tokens). diff --git a/README.md b/README.md index 936ff68..e5da484 100644 --- a/README.md +++ b/README.md @@ -157,7 +157,16 @@ Per-provider **`timeout`** is passed to the HTTP session for chat/completions, R Third-party **plugins**: install a package that declares `project.entry-points."plyngent.tools"` (and later other groups), then allowlist the entry-point name under **`[plugins].enable`**. Details: [doc/plugins.md](doc/plugins.md). -Supported provider presets today: `openai` (default if `preset` is omitted; default models `gpt-5.4` / `gpt-5.4-mini` / `gpt-5.4-nano` when `models` is omitted), `openai-compatible`, `deepseek` (OpenAI convention; default models `deepseek-v4-flash` and `deepseek-v4-pro` if `models` is omitted). Anthropic presets are modeled in config but not wired in the runtime client yet. +Supported provider presets today: + +| Preset | API used by the agent | Notes | +|--------|------------------------|--------| +| `openai` (default if `preset` omitted) | OpenAI **Responses** (`POST /responses`) | Default models `gpt-5.4` / `gpt-5.4-mini` / `gpt-5.4-nano` when `models` is omitted; optional `provider_tools` (default `web_search`) | +| `openai-compatible` | Chat Completions | Generic hosts (vLLM, LiteLLM, proxies); requires `url` | +| `deepseek` | Chat Completions (DeepSeek) | Default models `deepseek-v4-flash` / `deepseek-v4-pro` when `models` is omitted | +| `anthropic` | Anthropic **Messages** (`POST /messages`) | Native tools/streaming; set `access_key_or_token` (API key) | + +Model-level `preset` / `url` overrides on a catalog entry can route a single provider name to a different API (e.g. gateway + Anthropic model). If `[database]` is omitted (or SQLite `url` is unset/empty), chat uses a durable file under the user data dir (e.g. `~/.local/share/plyngent/chat.db` on Linux). Set `url = ":memory:"` for a true in-memory SQLite (CLI warns; no file; useful for tests). @@ -274,8 +283,8 @@ plyngent --log-level INFO chat ... See [doc/architecture.md](doc/architecture.md) and [CLAUDE.md](CLAUDE.md) for developers. -- **`lmproto/`** — OpenAI-compatible (+ DeepSeek) msgspec models and async SSE clients -- **`agent/`** — tool loop, streaming, usage, compact +- **`lmproto/`** — OpenAI-compatible, OpenAI Responses, Anthropic Messages, DeepSeek msgspec models and async SSE clients +- **`agent/`** — kind-based tool loop (`chat_completions` / `responses` / `messages`), streaming, usage, compact - **`memory/`** — async SQLAlchemy sessions/messages - **`tools/`** — workspace tools - **`cli/`** — Click entry + slash registry (`awaitlet` bridges sync Click to async work) diff --git a/doc/architecture.md b/doc/architecture.md index ba60b69..abf61b8 100644 --- a/doc/architecture.md +++ b/doc/architecture.md @@ -8,13 +8,12 @@ - config: Configurations for specified providers. - client: Clients for accessing existent specified services. - server: Servers for accepting other clients. - - openai - - openai_compatible - - anthropic - - ollama + - openai_compatible: Chat Completions base (``kind=chat_completions``). + - openai: Platform Responses + chat (``kind=responses``). + - anthropic: Messages API (``kind=messages``). - deepseek - openai_compat - - anthropic + - anthropic (convention extras; runtime uses openai_compat today) - utils: Common utilities for code architecture. - components: Utilities for class composition. - memory: Storage controlling for sessions and messages. @@ -22,7 +21,10 @@ - config: Plyngent configuration center (TOML), including ``[plugins]`` and ``[networking]`` (e.g. fetch SSRF Fake-IP CIDR exemptions). Model entries may override provider ``preset`` / ``url`` for mixed API routing. - - agent: Tool loop, streaming, usage, compact; `@tool` / tags / registry. + - runtime: ``create_client`` maps effective provider preset → protocol client. + - agent: Kind-based tool loop, streaming, usage, compact; bridges: + ``responses_bridge``/``responses_dispatch``, ``messages_bridge``/``messages_dispatch``; + ``@tool`` / tags / registry. History stays chat-completions-shaped. - tools: Workspace file/process/VCS/chat/todo/net tools; catalog; plugins; instance/session context and views. - prompting: Shared ask/choose/form for CLI and tools. diff --git a/doc/plyngent.example.toml b/doc/plyngent.example.toml index 7763861..513eb40 100644 --- a/doc/plyngent.example.toml +++ b/doc/plyngent.example.toml @@ -89,9 +89,20 @@ access_key_or_token = "sk-replace-me" # "gpt-4o-mini" = { text = true } # # Model-level preset/url overrides let one provider route models via different APIs. # # `preset = "openai"` means /responses; `openai-compatible` means /chat/completions; -# # `anthropic` means native /messages (support planned). +# # `anthropic` means native /messages. # # "claude-sonnet" = { text = true, preset = "anthropic", url = "https://api.anthropic.com/v1" } +# --- Anthropic (Messages API) --- +# [providers.anthropic] +# preset = "anthropic" +# access_key_or_token = "sk-ant-replace-me" +# # url defaults to https://api.anthropic.com/v1 when empty +# timeout = { connect = 10, read = 600 } +# +# [providers.anthropic.models] +# "claude-sonnet-4-20250514" = { text = true } +# "claude-opus-4-20250514" = { text = true } + # --- DeepSeek (OpenAI-compatible convention) --- # Default models if [providers.*.models] is omitted: deepseek-v4-flash, deepseek-v4-pro. # [providers.deepseek]