mirror of
https://github.com/NCBM/plyngent.git
synced 2026-11-07 23:47:54 +08:00
docs: document openai responses and anthropic messages wiring
This commit is contained in:
@@ -39,15 +39,18 @@ All protocol models use `msgspec.Struct` — not dataclasses, not Pydantic. Opti
|
|||||||
```
|
```
|
||||||
lmproto/openai_compatible/ ← Base: model, config, client
|
lmproto/openai_compatible/ ← Base: model, config, client
|
||||||
lmproto/openai/ ← OpenAI platform: Responses + chat (extends base)
|
lmproto/openai/ ← OpenAI platform: Responses + chat (extends base)
|
||||||
|
lmproto/anthropic/ ← Anthropic Messages: model, config, client
|
||||||
lmproto/deepseek/openai_compat/ ← Extends base via inheritance + extra fields
|
lmproto/deepseek/openai_compat/ ← Extends base via inheritance + extra fields
|
||||||
```
|
```
|
||||||
|
|
||||||
- **`openai_compatible/model.py`** — tagged chat messages (`SystemChatMessage`, `UserChatMessage`, …), tools, request/response, streaming chunks.
|
- **`openai_compatible/model.py`** — tagged chat messages (`SystemChatMessage`, `UserChatMessage`, …), tools, request/response, streaming chunks.
|
||||||
- **`openai_compatible/client.py`** — `BaseOpenAIClient` / `OpenAICompatibleClient` via `niquests` async + SSE; `chat_completions`, `models` only (`OpenAIClient` is a compat alias).
|
- **`openai_compatible/client.py`** — `BaseOpenAIClient` / `OpenAICompatibleClient` via `niquests` async + SSE; `chat_completions`, `models` only.
|
||||||
- **`openai_compatible/config.py`** — `OpenAIConfig` (token + base URL).
|
- **`openai_compatible/config.py`** — `OpenAIConfig` (token + base URL).
|
||||||
- **`openai/model.py`** — OpenAI Responses API (`ResponsesCreateParam`, `Response`, function_call items, stream events).
|
- **`openai/model.py`** — OpenAI Responses API (`ResponsesCreateParam`, `Response`, function_call items, stream events).
|
||||||
- **`openai/client.py`** — platform `OpenAIClient`: chat completions + `responses` / `get_response` / `delete_response`.
|
- **`openai/client.py`** — platform `OpenAIClient` (`kind="responses"`): chat completions + `responses` / `get_response` / `delete_response`.
|
||||||
- **DeepSeek** — `DeepseekOpenAIClient`; models add `reasoning_content`, `prefix`, `ThinkingOptions`. Config default model ids: `deepseek-v4-flash`, `deepseek-v4-pro`.
|
- **`anthropic/model.py`** — Anthropic Messages request/response + SSE event models.
|
||||||
|
- **`anthropic/client.py`** — `AnthropicClient` (`kind="messages"`): `POST /messages` + `GET /models`.
|
||||||
|
- **DeepSeek** — `DeepseekOpenAIClient` (`kind="chat_completions"`); models add `reasoning_content`, `prefix`, `ThinkingOptions`. Config default model ids: `deepseek-v4-flash`, `deepseek-v4-pro`.
|
||||||
|
|
||||||
### Config (`config/`)
|
### Config (`config/`)
|
||||||
|
|
||||||
@@ -55,7 +58,7 @@ TOML load/store (`ConfigStore`): `[providers]` tagged union presets, `[database]
|
|||||||
|
|
||||||
### Runtime (`runtime/`)
|
### Runtime (`runtime/`)
|
||||||
|
|
||||||
`create_client(provider)` maps config `Provider` → protocol client. `provider_to_openai_config` normalizes `timeout` via `normalize_http_timeout` into `OpenAIConfig.timeout` for `niquests.AsyncSession`. OpenAI → `lmproto.openai.OpenAIClient` (Responses-capable); openai-compatible / deepseek(openai convention) → chat-completions clients; anthropic and deepseek anthropic convention raise `ProviderNotSupportedError`.
|
`create_client(provider)` maps config `Provider` → protocol client by effective preset (model-level `preset`/`url` overrides apply). `normalize_http_timeout` feeds OpenAI-compatible configs and Anthropic. Mapping: `openai` → `OpenAIClient` (agent uses Responses), `openai-compatible` / `deepseek` → chat-completions clients, `anthropic` → `AnthropicClient`.
|
||||||
|
|
||||||
### Memory (`memory/`)
|
### Memory (`memory/`)
|
||||||
|
|
||||||
@@ -63,13 +66,14 @@ Async SQLAlchemy + aiosqlite. `MemoryStore`: schema init (+ lightweight SQLite `
|
|||||||
|
|
||||||
### Agent (`agent/`)
|
### Agent (`agent/`)
|
||||||
|
|
||||||
- **`ChatClient`** Protocol for `chat_completions` (agent history stays chat-shaped).
|
- **Kind-based dispatch**: clients expose `kind` (`chat_completions` | `responses` | `messages`). Agent history stays chat-completions-shaped; transport adapters return synthetic `ChatCompletionResponse` / stream chunks.
|
||||||
- **OpenAI Responses integration**: `ResponsesChatClient` adapts platform `OpenAIClient.responses` to `ChatClient`; selected automatically for `OpenAIProvider` in `create_client`. Compat/DeepSeek stay on chat completions.
|
- **`responses_bridge` / `responses_dispatch`**: chat ↔ OpenAI Responses; stream text/reasoning deltas; tool calls + usage on `response.completed`.
|
||||||
- **Provider-side tools**: `OpenAIProvider.provider_tools` (list of dicts; default `[{type="web_search"}]` when omitted; `[]` disables) merged into Responses `tools` alongside local function tools; never executed by `ToolRegistry`.
|
- **`messages_bridge` / `messages_dispatch`**: chat ↔ Anthropic Messages; system fold; tool_use/tool_result; stream text + tool argument fragments.
|
||||||
|
- **Provider-side tools**: `OpenAIProvider.provider_tools` (list of dicts; default `[{type="web_search"}]` when omitted; `[]` disables) threaded via `ChatAgent`/`run_chat_loop` into Responses only; never executed by `ToolRegistry`.
|
||||||
- **`@tool` / `ToolRegistry`**: decorator infers JSON Schema from type hints; execute tools by name.
|
- **`@tool` / `ToolRegistry`**: decorator infers JSON Schema from type hints; execute tools by name.
|
||||||
- **`run_chat_loop`**: multi-round tool loop; default **streaming** text deltas + stream tool-call merge; parallel tools; tool-result char budget; soft context compact on request (**API-calibrated** after first usage when available); cooperative cancel points; optional `on_limit`.
|
- **`run_chat_loop`**: multi-round tool loop; default **streaming** text deltas + stream tool-call merge; parallel tools; tool-result char budget; soft context compact on request (**API-calibrated** after first usage when available); cooperative cancel points; optional `on_limit`.
|
||||||
- **`ChatAgent`**: optional `MemoryStore` (user message persisted immediately; **completed tool batches checkpointed** mid-turn; unfinished assistant suffix rolled back on failure); `stream`; system prompt; `retry()` continues incomplete turns (user-only **or** after committed tools — does not re-run those tools).
|
- **`ChatAgent`**: optional `MemoryStore` (user message persisted immediately; **completed tool batches checkpointed** mid-turn; unfinished assistant suffix rolled back on failure); `stream`; system prompt; `provider_tools`; `retry()` continues incomplete turns (user-only **or** after committed tools — does not re-run those tools).
|
||||||
- **`/compact`**: soft-compact tool dumps → model summary (no tools) → **new** session seeded with summary message. Soft-compact keeps the last **12** tool results full-size by default (`DEFAULT_RECENT_TOOL_RESULTS`); older tool payloads shrink first.
|
- **`/compact`**: soft-compact tool dumps → model summary (no tools) via the same kind dispatch → **new** session seeded with summary message. Soft-compact keeps the last **12** tool results full-size by default (`DEFAULT_RECENT_TOOL_RESULTS`); older tool payloads shrink first.
|
||||||
- Events: text_delta, **reasoning_delta**, assistant_message, tool_call/result, max_rounds, **error** (`retryable`/`source`), **cancelled** (`reason`), **usage** (`TokenUsage`).
|
- Events: text_delta, **reasoning_delta**, assistant_message, tool_call/result, max_rounds, **error** (`retryable`/`source`), **cancelled** (`reason`), **usage** (`TokenUsage`).
|
||||||
- Usage: API `usage` from completions (stream with `include_usage`); **char≈token fallback** (~4 chars/token) when omitted; **context size** = last request ``prompt_tokens`` (API preferred); `last_turn_usage` / `session_usage` are **billed sums** (tool rounds re-send history); CLI end-of-turn + `/status`.
|
- Usage: API `usage` from completions (stream with `include_usage`); **char≈token fallback** (~4 chars/token) when omitted; **context size** = last request ``prompt_tokens`` (API preferred); `last_turn_usage` / `session_usage` are **billed sums** (tool rounds re-send history); CLI end-of-turn + `/status`.
|
||||||
- Config ``[agent]``: `system_prompt` (persona; default `DEFAULT_SYSTEM_PROMPT`; `""` omits persona), `tool_directives` (tool playbook; default `DEFAULT_TOOL_DIRECTIVES`; `""` omits playbook; both empty → no system), `max_tool_result_chars`, `parallel_tools`, `confirm_destructive`, `path_denylist`, `max_context_tokens` (default 200k est. tokens).
|
- Config ``[agent]``: `system_prompt` (persona; default `DEFAULT_SYSTEM_PROMPT`; `""` omits persona), `tool_directives` (tool playbook; default `DEFAULT_TOOL_DIRECTIVES`; `""` omits playbook; both empty → no system), `max_tool_result_chars`, `parallel_tools`, `confirm_destructive`, `path_denylist`, `max_context_tokens` (default 200k est. tokens).
|
||||||
|
|||||||
@@ -157,7 +157,16 @@ Per-provider **`timeout`** is passed to the HTTP session for chat/completions, R
|
|||||||
|
|
||||||
Third-party **plugins**: install a package that declares `project.entry-points."plyngent.tools"` (and later other groups), then allowlist the entry-point name under **`[plugins].enable`**. Details: [doc/plugins.md](doc/plugins.md).
|
Third-party **plugins**: install a package that declares `project.entry-points."plyngent.tools"` (and later other groups), then allowlist the entry-point name under **`[plugins].enable`**. Details: [doc/plugins.md](doc/plugins.md).
|
||||||
|
|
||||||
Supported provider presets today: `openai` (default if `preset` is omitted; default models `gpt-5.4` / `gpt-5.4-mini` / `gpt-5.4-nano` when `models` is omitted), `openai-compatible`, `deepseek` (OpenAI convention; default models `deepseek-v4-flash` and `deepseek-v4-pro` if `models` is omitted). Anthropic presets are modeled in config but not wired in the runtime client yet.
|
Supported provider presets today:
|
||||||
|
|
||||||
|
| Preset | API used by the agent | Notes |
|
||||||
|
|--------|------------------------|--------|
|
||||||
|
| `openai` (default if `preset` omitted) | OpenAI **Responses** (`POST /responses`) | Default models `gpt-5.4` / `gpt-5.4-mini` / `gpt-5.4-nano` when `models` is omitted; optional `provider_tools` (default `web_search`) |
|
||||||
|
| `openai-compatible` | Chat Completions | Generic hosts (vLLM, LiteLLM, proxies); requires `url` |
|
||||||
|
| `deepseek` | Chat Completions (DeepSeek) | Default models `deepseek-v4-flash` / `deepseek-v4-pro` when `models` is omitted |
|
||||||
|
| `anthropic` | Anthropic **Messages** (`POST /messages`) | Native tools/streaming; set `access_key_or_token` (API key) |
|
||||||
|
|
||||||
|
Model-level `preset` / `url` overrides on a catalog entry can route a single provider name to a different API (e.g. gateway + Anthropic model).
|
||||||
|
|
||||||
If `[database]` is omitted (or SQLite `url` is unset/empty), chat uses a durable file under the user data dir (e.g. `~/.local/share/plyngent/chat.db` on Linux). Set `url = ":memory:"` for a true in-memory SQLite (CLI warns; no file; useful for tests).
|
If `[database]` is omitted (or SQLite `url` is unset/empty), chat uses a durable file under the user data dir (e.g. `~/.local/share/plyngent/chat.db` on Linux). Set `url = ":memory:"` for a true in-memory SQLite (CLI warns; no file; useful for tests).
|
||||||
|
|
||||||
@@ -274,8 +283,8 @@ plyngent --log-level INFO chat ...
|
|||||||
|
|
||||||
See [doc/architecture.md](doc/architecture.md) and [CLAUDE.md](CLAUDE.md) for developers.
|
See [doc/architecture.md](doc/architecture.md) and [CLAUDE.md](CLAUDE.md) for developers.
|
||||||
|
|
||||||
- **`lmproto/`** — OpenAI-compatible (+ DeepSeek) msgspec models and async SSE clients
|
- **`lmproto/`** — OpenAI-compatible, OpenAI Responses, Anthropic Messages, DeepSeek msgspec models and async SSE clients
|
||||||
- **`agent/`** — tool loop, streaming, usage, compact
|
- **`agent/`** — kind-based tool loop (`chat_completions` / `responses` / `messages`), streaming, usage, compact
|
||||||
- **`memory/`** — async SQLAlchemy sessions/messages
|
- **`memory/`** — async SQLAlchemy sessions/messages
|
||||||
- **`tools/`** — workspace tools
|
- **`tools/`** — workspace tools
|
||||||
- **`cli/`** — Click entry + slash registry (`awaitlet` bridges sync Click to async work)
|
- **`cli/`** — Click entry + slash registry (`awaitlet` bridges sync Click to async work)
|
||||||
|
|||||||
+8
-6
@@ -8,13 +8,12 @@
|
|||||||
- config: Configurations for specified providers.
|
- config: Configurations for specified providers.
|
||||||
- client: Clients for accessing existent specified services.
|
- client: Clients for accessing existent specified services.
|
||||||
- server: Servers for accepting other clients.
|
- server: Servers for accepting other clients.
|
||||||
- openai
|
- openai_compatible: Chat Completions base (``kind=chat_completions``).
|
||||||
- openai_compatible
|
- openai: Platform Responses + chat (``kind=responses``).
|
||||||
- anthropic
|
- anthropic: Messages API (``kind=messages``).
|
||||||
- ollama
|
|
||||||
- deepseek
|
- deepseek
|
||||||
- openai_compat
|
- openai_compat
|
||||||
- anthropic
|
- anthropic (convention extras; runtime uses openai_compat today)
|
||||||
- utils: Common utilities for code architecture.
|
- utils: Common utilities for code architecture.
|
||||||
- components: Utilities for class composition.
|
- components: Utilities for class composition.
|
||||||
- memory: Storage controlling for sessions and messages.
|
- memory: Storage controlling for sessions and messages.
|
||||||
@@ -22,7 +21,10 @@
|
|||||||
- config: Plyngent configuration center (TOML), including ``[plugins]`` and
|
- config: Plyngent configuration center (TOML), including ``[plugins]`` and
|
||||||
``[networking]`` (e.g. fetch SSRF Fake-IP CIDR exemptions). Model entries
|
``[networking]`` (e.g. fetch SSRF Fake-IP CIDR exemptions). Model entries
|
||||||
may override provider ``preset`` / ``url`` for mixed API routing.
|
may override provider ``preset`` / ``url`` for mixed API routing.
|
||||||
- agent: Tool loop, streaming, usage, compact; `@tool` / tags / registry.
|
- runtime: ``create_client`` maps effective provider preset → protocol client.
|
||||||
|
- agent: Kind-based tool loop, streaming, usage, compact; bridges:
|
||||||
|
``responses_bridge``/``responses_dispatch``, ``messages_bridge``/``messages_dispatch``;
|
||||||
|
``@tool`` / tags / registry. History stays chat-completions-shaped.
|
||||||
- tools: Workspace file/process/VCS/chat/todo/net tools; catalog; plugins;
|
- tools: Workspace file/process/VCS/chat/todo/net tools; catalog; plugins;
|
||||||
instance/session context and views.
|
instance/session context and views.
|
||||||
- prompting: Shared ask/choose/form for CLI and tools.
|
- prompting: Shared ask/choose/form for CLI and tools.
|
||||||
|
|||||||
@@ -89,9 +89,20 @@ access_key_or_token = "sk-replace-me"
|
|||||||
# "gpt-4o-mini" = { text = true }
|
# "gpt-4o-mini" = { text = true }
|
||||||
# # Model-level preset/url overrides let one provider route models via different APIs.
|
# # Model-level preset/url overrides let one provider route models via different APIs.
|
||||||
# # `preset = "openai"` means /responses; `openai-compatible` means /chat/completions;
|
# # `preset = "openai"` means /responses; `openai-compatible` means /chat/completions;
|
||||||
# # `anthropic` means native /messages (support planned).
|
# # `anthropic` means native /messages.
|
||||||
# # "claude-sonnet" = { text = true, preset = "anthropic", url = "https://api.anthropic.com/v1" }
|
# # "claude-sonnet" = { text = true, preset = "anthropic", url = "https://api.anthropic.com/v1" }
|
||||||
|
|
||||||
|
# --- Anthropic (Messages API) ---
|
||||||
|
# [providers.anthropic]
|
||||||
|
# preset = "anthropic"
|
||||||
|
# access_key_or_token = "sk-ant-replace-me"
|
||||||
|
# # url defaults to https://api.anthropic.com/v1 when empty
|
||||||
|
# timeout = { connect = 10, read = 600 }
|
||||||
|
#
|
||||||
|
# [providers.anthropic.models]
|
||||||
|
# "claude-sonnet-4-20250514" = { text = true }
|
||||||
|
# "claude-opus-4-20250514" = { text = true }
|
||||||
|
|
||||||
# --- DeepSeek (OpenAI-compatible convention) ---
|
# --- DeepSeek (OpenAI-compatible convention) ---
|
||||||
# Default models if [providers.*.models] is omitted: deepseek-v4-flash, deepseek-v4-pro.
|
# Default models if [providers.*.models] is omitted: deepseek-v4-flash, deepseek-v4-pro.
|
||||||
# [providers.deepseek]
|
# [providers.deepseek]
|
||||||
|
|||||||
Reference in New Issue
Block a user