31 Commits

Author SHA1 Message Date
NCBM ea8e64e913 release: plyngent 0.4.0 2026-08-11 21:18:01 +08:00
NCBM 1973ef93bb cli: keep Ctrl+C a plain KeyboardInterrupt under asyncio.run
asyncio.run (Runner) replaces SIGINT with its own handler when the
current one is signal.default_int_handler: the first Ctrl+C silently
cancels the main task (invisible at the REPL prompt, leaving the task
marked cancelled) and the second raises KeyboardInterrupt, so a Ctrl+D
exit after a cancel was cancelled mid-memory.close() (SQLAlchemy
CancelledError traceback) and the CLI died with "Aborted!".

Install a distinct KeyboardInterrupt-raising handler before asyncio.run
so Runner leaves SIGINT alone, and have run_cancellable restore the
previous handler after each turn (remove_signal_handler leaves SIG_DFL,
which would kill the process on a stray Ctrl+C between turns).
2026-08-11 21:13:17 +08:00
NCBM a197a67142 cli: keep REPL alive on stray Ctrl+C outside turn calls
After a turn the run_cancellable SIGINT handler is removed, so a Ctrl+C
landing between turns (echoing user text, expire_yolo_once, gaps between
statements, or re-entering _echo_interrupted) raised KeyboardInterrupt at
an arbitrary bytecode and escaped run_repl, killing the whole REPL. Wrap
the loop body in one try/except KeyboardInterrupt that re-prompts, and
make _echo_interrupted itself suppress a further interrupt.
2026-08-11 20:54:13 +08:00
NCBM 28a191cd79 core/agent: full reasoning_text.done replaces streamed fragments 2026-08-11 20:08:52 +08:00
NCBM ec6edbcba4 core/config: tell model to clear todo stack when done 2026-08-11 19:42:10 +08:00
NCBM bd1053a3bc core/config: document default truncate_token everywhere 2026-08-11 19:42:10 +08:00
NCBM b5a30190f6 core/tools: default read_file truncation with token (no hint when uncut) 2026-08-11 19:42:10 +08:00
NCBM 26472776fb core/tools: token-ify per-stream caps and resume memory remainders 2026-08-11 19:41:54 +08:00
NCBM 6fdfb8398e core/agent: token-ify generic tool-result cap and compact shrinks 2026-08-11 19:41:54 +08:00
NCBM ec1ae59a28 core/tools: unified truncate marker with memory remainder store 2026-08-11 19:41:54 +08:00
NCBM b5950bdb57 core/agent: drop get_truncated hint from token-less generic truncation 2026-08-11 18:21:45 +08:00
NCBM c2735dada3 core/config: document read_file L-range and prettier summaries 2026-08-11 15:38:44 +08:00
NCBM 65349b3c07 cli: prettier tool call summaries for read_file and todo tools 2026-08-11 15:38:07 +08:00
NCBM e4ed89b391 core/tools: distinguish missing files from non-files on read 2026-08-11 15:37:56 +08:00
NCBM 25e6ec3d44 core/tools: show 1-based line range on read_file results 2026-08-11 15:33:12 +08:00
NCBM 3fbf2dc388 core/agent: fix deepseek responses reasoning_content for tool calls 2026-08-11 15:27:01 +08:00
NCBM 9640f86b89 core/tools: make truncate tokens gap-free (advance by returned length) 2026-08-11 15:16:55 +08:00
NCBM 4c60e9b1b3 docs: document truncate tokens and get_truncated 2026-08-11 15:03:00 +08:00
NCBM 6458a4ad31 core/tools: embed truncate tokens in read_file (max_chars) 2026-08-11 15:02:37 +08:00
NCBM 00db7cdc7f core/tools: add get_truncated tool for chained resume 2026-08-11 14:59:39 +08:00
NCBM c96f669bcb core/tools: embed truncate tokens in fetch (offset param) 2026-08-11 14:55:55 +08:00
NCBM 8cd456171c core/agent: drop increase-limit hint in favor of truncate tokens 2026-08-11 14:53:49 +08:00
NCBM cb137c1dee core/tools: add truncate token module (chainable cursors) 2026-08-11 14:53:15 +08:00
NCBM fa0e68d594 docs: document wait tool 2026-08-11 14:47:53 +08:00
NCBM d3a5200e0f core/tools: add wait tool (Enter to disturb) 2026-08-11 14:46:58 +08:00
NCBM fe7b75aad8 core/prompting: add line input with timeout 2026-08-11 14:44:34 +08:00
NCBM 010173cfc7 docs: use run_argv tool names in docs 2026-08-04 21:41:39 +08:00
NCBM f4c020b761 core/config: point default playbook and reminder at run_argv tools 2026-08-04 21:40:55 +08:00
NCBM df8f6bf959 core/tools: rename run_command_batch to run_argv_batch (steps/argv keys) 2026-08-04 21:40:25 +08:00
NCBM d698fbf06f core/tools: rename run_command to run_argv (argv param) 2026-08-04 21:38:20 +08:00
NCBM 23f7b4e299 core/tools: clear argv error when run_command gets a string 2026-08-04 20:35:22 +08:00
52 changed files with 1755 additions and 314 deletions
+3 -3
View File
@@ -85,13 +85,13 @@ Async SQLAlchemy + aiosqlite. `MemoryStore`: schema init (+ lightweight SQLite `
Module-level `@tool` handlers. Call `set_workspace_root()` before use. Module-level `@tool` handlers. Call `set_workspace_root()` before use.
- **`workspace`**: path resolve under primary root **or** temporary allowlist roots; path substring denylist; command basename denylist (CLI: timed human allow override, default deny on timeout; **not** skipped by YOLO); clearer deny messages. - **`workspace`**: path resolve under primary root **or** temporary allowlist roots; path substring denylist; command basename denylist (CLI: timed human allow override, default deny on timeout; **not** skipped by YOLO); clearer deny messages.
- **`file`**: `read_file` (`with_lineno`), `write_file`, `listdir`, `tree` (VCS + default noise dirs + optional `skip_dirs` / denylist walk), `glob_paths`, `grep_files` (regex, skip VCS/binary), `edit_replace` (`max_replaces`, reports remaining matches), `edit_lineno` (1-based range; only lines read via `with_lineno`; read state resets on any file write/edit/copy/move/delete), `copy_path` / `move_path` / `delete_path`, `new_temporary_workspace` (system temp allowlist; cleanup on chat exit; no session rebind). - **`file`**: `read_file` (`with_lineno`, `max_chars` default 32k; results start with a 1-based `L{begin}-{end}` range), `write_file`, `listdir`, `tree` (VCS + default noise dirs + optional `skip_dirs` / denylist walk), `glob_paths`, `grep_files` (regex, skip VCS/binary), `edit_replace` (`max_replaces`, reports remaining matches), `edit_lineno` (1-based range; only lines read via `with_lineno`; read state resets on any file write/edit/copy/move/delete), `copy_path` / `move_path` / `delete_path`, `new_temporary_workspace` (system temp allowlist; cleanup on chat exit; no session rebind), `get_truncated` (resume any truncated result via its `truncate_token=...`).
- **`process`**: `run_command` (argv, no shell, timeout, optional stdin/env); `run_command_batch` (serial steps, `pipe_out` on provider, `mix_stderr`, `stop_on_error`); PTY `open_pty` / `read_pty` / `write_pty` (literal) / `write_pty_keys` (escapes) / `ask_into_pty` (human→PTY only; `secret` no-echo; answer never in tool result) / `close_pty` (POSIX: `pty`+`fork`; Windows: ConPTY via `pywinpty` env marker dep). - **`process`**: `run_argv` (argv, no shell, timeout, optional stdin/env); `run_argv_batch` (serial steps, `pipe_out` on provider, `mix_stderr`, `stop_on_error`); PTY `open_pty` / `read_pty` / `write_pty` (literal) / `write_pty_keys` (escapes) / `ask_into_pty` (human→PTY only; `secret` no-echo; answer never in tool result) / `close_pty` (POSIX: `pty`+`fork`; Windows: ConPTY via `pywinpty` env marker dep).
- PTY: backend in `pty_backend.py`; structured status (`alive`/`exit_code`/`data`); `read_pty(..., until=)` (**CSI sanitized** so host TTY is never reset); keys via `write_pty_keys` only (`\xHH`, `ctrl+x`, `key=esc`); secrets via `ask_into_pty` (not `write_pty` data); session limit/idle TTL/output budget; close terminate→kill. - PTY: backend in `pty_backend.py`; structured status (`alive`/`exit_code`/`data`); `read_pty(..., until=)` (**CSI sanitized** so host TTY is never reset); keys via `write_pty_keys` only (`\xHH`, `ctrl+x`, `key=esc`); secrets via `ask_into_pty` (not `write_pty` data); session limit/idle TTL/output budget; close terminate→kill.
- CLI limit hooks: interactive confirm to raise tool-loop rounds, PTY session cap, or PTY output budget. - CLI limit hooks: interactive confirm to raise tool-loop rounds, PTY session cap, or PTY output budget.
- Destructive confirms: `classify_danger` + `ToolRegistry(on_confirm=…)`; CLI default deny; config `confirm_destructive` / `path_denylist`. Soft confirm also for shells/REPLs and `python|bash -c` one-liners (argv + code preview); deny may include a user comment for the model. Session YOLO: `/yolo on|off|once` and `--yes` (skip soft confirms; hard denylists unchanged; `once` expires after the next user turn). - Destructive confirms: `classify_danger` + `ToolRegistry(on_confirm=…)`; CLI default deny; config `confirm_destructive` / `path_denylist`. Soft confirm also for shells/REPLs and `python|bash -c` one-liners (argv + code preview); deny may include a user comment for the model. Session YOLO: `/yolo on|off|once` and `--yes` (skip soft confirms; hard denylists unchanged; `once` expires after the next user turn).
- **`vcs`**: read-only VCS tools (`vcs_kind` / `vcs_status` / `vcs_diff` / `vcs_log` / `vcs_branch`) via `VcsBackend` protocol; **git** implemented; detectors are pluggable for other systems. - **`vcs`**: read-only VCS tools (`vcs_kind` / `vcs_status` / `vcs_diff` / `vcs_log` / `vcs_branch`) via `VcsBackend` protocol; **git** implemented; detectors are pluggable for other systems.
- **`chat`**: human prompts as tools — `ask_user_line` / `ask_user_choice` / `ask_user_form` (shared `prompting` core). - **`chat`**: human prompts as tools — `ask_user_line` / `ask_user_choice` / `ask_user_form` (shared `prompting` core); `wait` (line prompt with timeout; Enter disturbs).
- **`todo`**: LIFO stack of **task groups** (not a queue of tasks) — `todo_push` creates one group of siblings; `todo_pop` removes the whole top group; `todo_update` by id; DFS breakdown push[T1,T2]→push[T1.1…]→pop→push[T2.1…]; stored on session row; open items = unfinished work (`[TODO OPEN WORK]`); all-terminal non-empty = hygiene (`[TODO HYGIENE]`); nag channel via `[agent] todo_nag_strategy` (`developer`/`user` = prose nag; `synthetic_tool` = forged `todo_list` call + **real** `stack.render()` result + ToolCall/Result events; `none`). - **`todo`**: LIFO stack of **task groups** (not a queue of tasks) — `todo_push` creates one group of siblings; `todo_pop` removes the whole top group; `todo_update` by id; DFS breakdown push[T1,T2]→push[T1.1…]→pop→push[T2.1…]; stored on session row; open items = unfinished work (`[TODO OPEN WORK]`); all-terminal non-empty = hygiene (`[TODO HYGIENE]`); nag channel via `[agent] todo_nag_strategy` (`developer`/`user` = prose nag; `synthetic_tool` = forged `todo_list` call + **real** `stack.render()` result + ToolCall/Result events; `none`).
- **`net` / `fetch`**: HTTP GET/POST/PUT/DELETE via niquests (isolated session, not the LLM client). Manual redirects with per-hop SSRF checks (`asyncio` `getaddrinfo`). Private/loopback hosts need instance policy grant (CLI timed confirm; **not** YOLO). Public cleartext HTTP and mutating methods soft-confirm (`YOLO`/`TRUSTABLE`). HTTPS→HTTP redirects blocked unless `allow_http_downgrade`. `user_agent` tool arg (or headers) is never replaced by the default when provided. Caps: body bytes/chars, request body size. - **`net` / `fetch`**: HTTP GET/POST/PUT/DELETE via niquests (isolated session, not the LLM client). Manual redirects with per-hop SSRF checks (`asyncio` `getaddrinfo`). Private/loopback hosts need instance policy grant (CLI timed confirm; **not** YOLO). Public cleartext HTTP and mutating methods soft-confirm (`YOLO`/`TRUSTABLE`). HTTPS→HTTP redirects blocked unless `allow_http_downgrade`. `user_agent` tool arg (or headers) is never replaced by the default when provided. Caps: body bytes/chars, request body size.
- **`DEFAULT_TOOLS` / catalog**: file + process + vcs + chat + todo + net (`fetch`) for local surface. - **`DEFAULT_TOOLS` / catalog**: file + process + vcs + chat + todo + net (`fetch`) for local surface.
+2 -2
View File
@@ -153,7 +153,7 @@ max_context_tokens = 200000
# enable = ["acme"] # enable = ["acme"]
``` ```
Per-provider **`timeout`** is passed to the HTTP session for chat/completions, Responses, and `GET /models`. A single number sets one timeout; `{ connect, read }` splits TCP/TLS setup vs idle wait between response bytes (SSE can run longer than `read` while chunks keep arriving). Tool/process timeouts (`run_command`, PTY, policy confirm) are separate. Per-provider **`timeout`** is passed to the HTTP session for chat/completions, Responses, and `GET /models`. A single number sets one timeout; `{ connect, read }` splits TCP/TLS setup vs idle wait between response bytes (SSE can run longer than `read` while chunks keep arriving). Tool/process timeouts (`run_argv`, PTY, policy confirm) are separate.
Third-party **plugins**: install a package that declares `project.entry-points."plyngent.tools"` (and later other groups), then allowlist the entry-point name under **`[plugins].enable`**. Details: [doc/plugins.md](doc/plugins.md). Third-party **plugins**: install a package that declares `project.entry-points."plyngent.tools"` (and later other groups), then allowlist the entry-point name under **`[plugins].enable`**. Details: [doc/plugins.md](doc/plugins.md).
@@ -270,7 +270,7 @@ User messages are saved immediately. On API error or Ctrl+C, partial assistant/t
## Tools (when enabled) ## Tools (when enabled)
Default registry: file ops (including `tree` with default noise-dir skips), `run_command` / PTY (POSIX openpty; Windows ConPTY via pywinpty), read-only VCS (git), HTTP `fetch` (GET/POST/PUT/DELETE via niquests; private/loopback hosts need a human policy grant, not YOLO), human prompts (`ask_user_line` / `ask_user_choice` / `ask_user_form`), and todo stack tools (`todo_list` / `todo_push` / `todo_pop` / `todo_update` / `todo_clear`). Default registry: file ops (including `tree` with default noise-dir skips), `run_argv` / `run_argv_batch` / PTY (POSIX openpty; Windows ConPTY via pywinpty), read-only VCS (git), HTTP `fetch` (GET/POST/PUT/DELETE via niquests; private/loopback hosts need a human policy grant, not YOLO), human prompts (`ask_user_line` / `ask_user_choice` / `ask_user_form`), `wait` (line prompt with timeout; Enter disturbs), `get_truncated` (resume any truncated result via its `truncate_token=...`), and todo stack tools (`todo_list` / `todo_push` / `todo_pop` / `todo_update` / `todo_clear`).
Safety defaults: Safety defaults:
+1 -1
View File
@@ -159,7 +159,7 @@ Listed tools show **tags** and **catalog source** (`builtin` vs `plugin:…`).
## Policy checklist for authors ## Policy checklist for authors
1. Prefer **LOCAL** for host FS, shell, PTY, or arbitrary network side effects. 1. Prefer **LOCAL** for host FS, shell, PTY, or arbitrary network side effects.
2. Do not shadow builtin names (`read_file`, `run_command`, todo tools, …). 2. Do not shadow builtin names (`read_file`, `run_argv`, todo tools, …).
3. Do not rely on process-global workspace/todo alone when a host binds 3. Do not rely on process-global workspace/todo alone when a host binds
instance/session context — prefer the same patterns as builtins instance/session context — prefer the same patterns as builtins
(`get_workspace_root` already prefers instance policy; session state via (`get_workspace_root` already prefers instance policy; session state via
Generated
+54 -54
View File
@@ -115,51 +115,51 @@ files = [
[[package]] [[package]]
name = "greenlet" name = "greenlet"
version = "3.5.4" version = "3.5.5"
requires_python = ">=3.10" requires_python = ">=3.10"
summary = "Lightweight in-process concurrent programming" summary = "Lightweight in-process concurrent programming"
groups = ["default"] groups = ["default"]
files = [ files = [
{file = "greenlet-3.5.4-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:7e8afa5eac028f8140ceafe5ceec66e6aa127ddcb21452d2a564dcd2900b5f22"}, {file = "greenlet-3.5.5-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:d4a389a852e392a6366058651a20fa5ba40d979865aa81bea2ccbdc44805070d"},
{file = "greenlet-3.5.4-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:73b37afe369021423ea53dd3123e04bffa7e93ac64429b9f50835b2e4fcae7cf"}, {file = "greenlet-3.5.5-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:70b157cd319873e8b544ddc2de158f55bbd0a9b0218c8ce9332039801518e328"},
{file = "greenlet-3.5.4-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3ef964f56dfcb6f9bbef2a190d9126795eac408716aeae47b5e7c73c32aafca9"}, {file = "greenlet-3.5.5-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8bdfd1424abcf26832961e766570cae79efdb9599d709088c9cb6ef82b194926"},
{file = "greenlet-3.5.4-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:cef589bc65fae02d10bca2ac341191c5b33acc2967892ebf4fcbd10eabb7a74c"}, {file = "greenlet-3.5.5-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d98ef6f92e67c6dbf299dbfd8facc1b0d2d9cedf91e325e73b3d0373fe4309d8"},
{file = "greenlet-3.5.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c53ff01a5c53a40f2c16820ebc56d7c61a77f5fbe009dadd96292d5682f80f8"}, {file = "greenlet-3.5.5-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:182de51c6b572a705f2fafaab2e783bcf7d2760940229dfe73086cbae037af3e"},
{file = "greenlet-3.5.4-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:dfc41ae893d9ceaf22c824f2153a88b30651b20e8758c2cd9ac143f23640563c"}, {file = "greenlet-3.5.5-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:159df1942d88e8f784cbb38d6f18bdb365cd11319cfbb3e89623de2b97892d53"},
{file = "greenlet-3.5.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ecca4d80d55a01ad6b23b33262662956149fbb7b2c6be2910f1705921958cbf3"}, {file = "greenlet-3.5.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:8fec3f165dfe332e490c3247c0f6c23b0bfc45f06496ad7f00ddb00e3d35e4dc"},
{file = "greenlet-3.5.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2ffbc533e0eaf8e80d8471411646ab88fe58f641d508c0b02b24494479f4d9ec"}, {file = "greenlet-3.5.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:c6ce25fee6cabc8bf22cb8b52e642cbb821be5b9aec8094d07ff03378141b8e9"},
{file = "greenlet-3.5.4-cp314-cp314-win_amd64.whl", hash = "sha256:305f69e6c4523d7f6979ed001cff4e5853c063e5da04880296603aa0227e544c"}, {file = "greenlet-3.5.5-cp314-cp314-win_amd64.whl", hash = "sha256:7dffc5c859fe6059974df1e37d7923d654a83e2ae18fdd616994270e001115e1"},
{file = "greenlet-3.5.4-cp314-cp314-win_arm64.whl", hash = "sha256:f260930bbbbcf9caee661211235a5111c86dfe5832fdf6ae4570da1e0995320f"}, {file = "greenlet-3.5.5-cp314-cp314-win_arm64.whl", hash = "sha256:5e2afcfc4d4305dd715809b03da5cbe437c8984f61d8917751eb5fe4aefa3e07"},
{file = "greenlet-3.5.4-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:41ddab54e4b238f4a6c323f39b4e59e176affd5a94d461a9fb7583dac74240a3"}, {file = "greenlet-3.5.5-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:0e5a7de979d764aea1f5b6e95cf92b5b37741b9823702041f34b126e7f690277"},
{file = "greenlet-3.5.4-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b3dabe3e2809013052c68bdf0b7fa5f5f2859c43a80803131ad61af9cabd7867"}, {file = "greenlet-3.5.5-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fef01bd457f11fc158b130ca0027a3c365693280e8e231b65bdaf57999f39f5b"},
{file = "greenlet-3.5.4-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:27d3f00718634d4520a3a150154ac5da36f257869d41321953375b90bfbbc72c"}, {file = "greenlet-3.5.5-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5173a72310725a74afc82c164f0e52cb8ad0de62f2bb623f24f6c0cc07d80272"},
{file = "greenlet-3.5.4-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:39169a11d87a6a263afda3e9a27d1df16d0f919d40a4837cc73986c9884c0dd8"}, {file = "greenlet-3.5.5-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5e9ec2e7c98e895fcea0c5cc57b2606cf86ece6d0a56578f3eb225e2af4f0387"},
{file = "greenlet-3.5.4-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cbd60b5763c6543c1827e48faaf14ea9bfbad245f52b1a4d76a2a2d8884c6c66"}, {file = "greenlet-3.5.5-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:44f08341873200ba8a60a8bc14ace3d91f1754f7fa7bc66157714a8cd420a476"},
{file = "greenlet-3.5.4-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:bd3d1145f603b2db19feb9078c2e6855eb7c67e15580c010ed815cee519b86fd"}, {file = "greenlet-3.5.5-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:102817506f6090b5176c746a82603341a549b40e5c3d5b72a4c672228a918c41"},
{file = "greenlet-3.5.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:f00f910f0e7b35416c63b23ad78b769aeccfc1775f712b43c4ee525624a2eef7"}, {file = "greenlet-3.5.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d246c0db9a2513cd45f019ba178ea4d4d4705bd210ee465e2c15d76a1ab13874"},
{file = "greenlet-3.5.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:91c26423753b92caf41ab3f98fd547d7374d4d9fc2d85be041886c1579d9255e"}, {file = "greenlet-3.5.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:72507285b5caa1d17904a3f7c322ca780823a54170a0e04ec3f37bcc60d4db71"},
{file = "greenlet-3.5.4-cp314-cp314t-win_amd64.whl", hash = "sha256:ee032b91fd8ec29ec6c4cea2b8c561b178435134bd0752c7334b94e9c736c132"}, {file = "greenlet-3.5.5-cp314-cp314t-win_amd64.whl", hash = "sha256:7805655781fb8f28a55d05fe57ed61f5f10f1892fb587673e3bb5264f28041f0"},
{file = "greenlet-3.5.4-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:178111881dd7a6c946471fda85485ec796e1043c2b939f694b096e2ecf986809"}, {file = "greenlet-3.5.5-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:c0db80fcd5b8aece93f66c64f78a786bbb6b96c5fe63ef5a5a4581ecf8bab206"},
{file = "greenlet-3.5.4-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d92df08dd65fede97fc37aad36c2e9dcda3b31c467f8e0c2c096456cb818e927"}, {file = "greenlet-3.5.5-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b241c32f912ada659808d68e308c568baf577eebf757d15471472de0c18cfad"},
{file = "greenlet-3.5.4-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:99e8f8c4ebc4fd80aa26c1280ae9ad43a0976e786349703a181cf0bae60413e5"}, {file = "greenlet-3.5.5-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ef6a08349401d8eaf3cb12688ac8557de95788556b8631ef17555a4a173022c0"},
{file = "greenlet-3.5.4-cp315-cp315-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:1f17e362d78e37559e0506c5a7d066bdd45073c36a0127a543e8a0df27242ff3"}, {file = "greenlet-3.5.5-cp315-cp315-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:37faa97daccb6d9f4c2141ce3118d023c3c5506864a7d8bdf726f665018c1f76"},
{file = "greenlet-3.5.4-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:394de08dad5ffcb1f50c2159d93e398d9d2da3ed437645eaa54771fa720db9f0"}, {file = "greenlet-3.5.5-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f2e3d061b8e13aec2f0441689b3c71b244a20e5d274a52cb0f7e31bd1d139552"},
{file = "greenlet-3.5.4-cp315-cp315-manylinux_2_39_riscv64.whl", hash = "sha256:cd320d998cbaa032932830448e39abf3c6a12901295e386e8114db926e10cffb"}, {file = "greenlet-3.5.5-cp315-cp315-manylinux_2_39_riscv64.whl", hash = "sha256:b18007dc2473a7942fd157366b55f01da6fed7ce85318591005b419e0a439474"},
{file = "greenlet-3.5.4-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:c883d61f2282d72c767a14936641b3efcbde9d82f1080712aaea0b1d3126cb88"}, {file = "greenlet-3.5.5-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:9ab5f5b93655e77fe0d6c2dfd22b5eac751bb1f876d8ec21761b7c1fb9266007"},
{file = "greenlet-3.5.4-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:2a924f15d17957e252a810acefcb5942f5ca712298e8b6fcaed9a307d357522c"}, {file = "greenlet-3.5.5-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:f0e5a21bd4452a88cf032fc43c4a5b307ab1380eacb63b5988f9c0317885e773"},
{file = "greenlet-3.5.4-cp315-cp315-win_amd64.whl", hash = "sha256:ed17e5f3420360d5b459de8462efb52060399a5326a613d4cde31cef63ef95da"}, {file = "greenlet-3.5.5-cp315-cp315-win_amd64.whl", hash = "sha256:469dbb0a78625642f4a626cfd0c6e8bccc0385b5e49189b6308bbe849ec88a8e"},
{file = "greenlet-3.5.4-cp315-cp315-win_arm64.whl", hash = "sha256:f908898d6fa484ce4b6f447ce70ea99b52c503fee419e53cf74d60a16bc9e667"}, {file = "greenlet-3.5.5-cp315-cp315-win_arm64.whl", hash = "sha256:2d57406c3efd32d7a81e17a674314e8bd00792cdab49ea3228a49aa1bfb2e769"},
{file = "greenlet-3.5.4-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:1833637f17d5e7472548a48575c394fe39f1b1890d676d162d86593610f44d8c"}, {file = "greenlet-3.5.5-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:68184dfcf50ccaa8e864770fe0633a7e27250ea9329f8192ef47ee9ecfd78e1c"},
{file = "greenlet-3.5.4-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:12cda9122e03341f1cb6b8207a19d7a9d375e52f1b4e9243918375f40fd7b4b9"}, {file = "greenlet-3.5.5-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9ec0dc0e59dc9c61af5c47348365ccbbd7addfafe0a93b00336ff3da2907bdc6"},
{file = "greenlet-3.5.4-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d83ae0e32d14957ab7170785a20f582635c8474deab1bfbb552b17e769a6ce25"}, {file = "greenlet-3.5.5-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e604f58e35833fc46ef20302bcb314dddbfd3fcf33a4f936216d51dd678d63ae"},
{file = "greenlet-3.5.4-cp315-cp315t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:123aa379c962ed5fe90a880327e0c3066124ac64ec99e12a238be9fd8eb3db3d"}, {file = "greenlet-3.5.5-cp315-cp315t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:2888a3a38bc5ee5bb6c438372197152e815837e4fab7ed7a1f86ef18ffd58ad1"},
{file = "greenlet-3.5.4-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9f1467de1bb767f75db0aa34c195e3a496d8d1278c796e70c24ce205d3e99cde"}, {file = "greenlet-3.5.5-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:40239b5384f96da3963585cc6d7eaa9b56f8ae67e8d92cc82dd9e202fc847de3"},
{file = "greenlet-3.5.4-cp315-cp315t-manylinux_2_39_riscv64.whl", hash = "sha256:adf2244d7f69409925a8f22ed22cc5f93cdfe5c9dc87ff3476be2c2aaae61a05"}, {file = "greenlet-3.5.5-cp315-cp315t-manylinux_2_39_riscv64.whl", hash = "sha256:1e8d9391fe77f15649589a907cef972dbbd6352ef7ff7dc0492f658c0c26495f"},
{file = "greenlet-3.5.4-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:0fa53040b78b578120eecdc0265e3f1051487cc425d11a2b7c761daadf4feaa8"}, {file = "greenlet-3.5.5-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:a1eaccf5c3a1d3e46dead602c72e6836731e8e245c9de6a27764567b6b62d4c0"},
{file = "greenlet-3.5.4-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:60e0bc961d367df506660e9ac0177a76bc6d81305300704b0977d1634f76efe2"}, {file = "greenlet-3.5.5-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:19e4e026fe20691f333b8eb1a3bc9625eceba8c3f9d62ec5a6f8581afbc6b5a5"},
{file = "greenlet-3.5.4-cp315-cp315t-win_amd64.whl", hash = "sha256:f680e549edb3eaf21eea4e7fe101e15ec180c74b7879ab46adc080f22d4015d2"}, {file = "greenlet-3.5.5-cp315-cp315t-win_amd64.whl", hash = "sha256:712aee154f648bde84634654bb38bb78c69ac640c37a45c9effed800735049d8"},
{file = "greenlet-3.5.4-cp315-cp315t-win_arm64.whl", hash = "sha256:08fc36de8442d5c3e95b044550dbea9bf144d31ec0cc58e36fb241cb6ef6a994"}, {file = "greenlet-3.5.5-cp315-cp315t-win_arm64.whl", hash = "sha256:7f049911ee81a16a03c33d5450d8d5867d27f596ca5fb201b86f4524e874468b"},
{file = "greenlet-3.5.4.tar.gz", hash = "sha256:0232ae1de90a8e07867bb127d7a6ba2301e859145489f25cda8a6096dabe1d20"}, {file = "greenlet-3.5.5.tar.gz", hash = "sha256:adb4bae02e91a8e863e48b177e4014bdcac8a6b5e047ea1df687a61534b85e6c"},
] ]
[[package]] [[package]]
@@ -315,24 +315,24 @@ files = [
[[package]] [[package]]
name = "packaging" name = "packaging"
version = "26.2" version = "26.3"
requires_python = ">=3.8" requires_python = ">=3.9"
summary = "Core utilities for Python packages" summary = "Core utilities for Python packages"
groups = ["dev"] groups = ["dev"]
files = [ files = [
{file = "packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e"}, {file = "packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c"},
{file = "packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661"}, {file = "packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79"},
] ]
[[package]] [[package]]
name = "platformdirs" name = "platformdirs"
version = "4.11.0" version = "4.11.2"
requires_python = ">=3.10" requires_python = ">=3.10"
summary = "A small Python package for determining appropriate platform-specific dirs, e.g. a `user data dir`." summary = "A small Python package for determining appropriate platform-specific dirs, e.g. a `user data dir`."
groups = ["default"] groups = ["default"]
files = [ files = [
{file = "platformdirs-4.11.0-py3-none-any.whl", hash = "sha256:360ccded2b7fce0af0ff80cc8f5942a1c5d99b0e856033acb030bfc634709e74"}, {file = "platformdirs-4.11.2-py3-none-any.whl", hash = "sha256:7f89089b6ea71bda7962953edcf784b2e2d9d285b40ad88be2bb75c6e9d82ab4"},
{file = "platformdirs-4.11.0.tar.gz", hash = "sha256:0555d18370482847566ffabcaa53ad7c6c1c29f195989ae1ed634a05f76ea1e0"}, {file = "platformdirs-4.11.2.tar.gz", hash = "sha256:3a2ae5fca3520a01ab1be8b45613537f52ddf5b5f6f53d88233892dfbf0cd82d"},
] ]
[[package]] [[package]]
@@ -549,7 +549,7 @@ files = [
[[package]] [[package]]
name = "urllib3-future" name = "urllib3-future"
version = "2.23.900" version = "2.24.900"
requires_python = ">=3.7" requires_python = ">=3.7"
summary = "urllib3.future is a powerful HTTP 1.1, 2, and 3 client with both sync and async interfaces" summary = "urllib3.future is a powerful HTTP 1.1, 2, and 3 client with both sync and async interfaces"
groups = ["default"] groups = ["default"]
@@ -559,18 +559,18 @@ dependencies = [
"qh3<2.0.0,>=1.5.4; (platform_system == \"Darwin\" or platform_system == \"Windows\" or platform_system == \"Linux\") and platform_python_implementation == \"PyPy\" and (platform_machine == \"x86_64\" or platform_machine == \"s390x\" or platform_machine == \"armv7l\" or platform_machine == \"ppc64le\" or platform_machine == \"ppc64\" or platform_machine == \"AMD64\" or platform_machine == \"aarch64\" or platform_machine == \"arm64\" or platform_machine == \"ARM64\" or platform_machine == \"x86\" or platform_machine == \"i686\" or platform_machine == \"riscv64\" or platform_machine == \"riscv64gc\") and python_version < \"3.12\" or (platform_system == \"Darwin\" or platform_system == \"Windows\" or platform_system == \"Linux\") and python_full_version > \"3.7.10\" and (platform_machine == \"x86_64\" or platform_machine == \"s390x\" or platform_machine == \"armv7l\" or platform_machine == \"ppc64le\" or platform_machine == \"ppc64\" or platform_machine == \"AMD64\" or platform_machine == \"aarch64\" or platform_machine == \"arm64\" or platform_machine == \"ARM64\" or platform_machine == \"x86\" or platform_machine == \"i686\" or platform_machine == \"riscv64\" or platform_machine == \"riscv64gc\") and platform_python_implementation == \"CPython\"", "qh3<2.0.0,>=1.5.4; (platform_system == \"Darwin\" or platform_system == \"Windows\" or platform_system == \"Linux\") and platform_python_implementation == \"PyPy\" and (platform_machine == \"x86_64\" or platform_machine == \"s390x\" or platform_machine == \"armv7l\" or platform_machine == \"ppc64le\" or platform_machine == \"ppc64\" or platform_machine == \"AMD64\" or platform_machine == \"aarch64\" or platform_machine == \"arm64\" or platform_machine == \"ARM64\" or platform_machine == \"x86\" or platform_machine == \"i686\" or platform_machine == \"riscv64\" or platform_machine == \"riscv64gc\") and python_version < \"3.12\" or (platform_system == \"Darwin\" or platform_system == \"Windows\" or platform_system == \"Linux\") and python_full_version > \"3.7.10\" and (platform_machine == \"x86_64\" or platform_machine == \"s390x\" or platform_machine == \"armv7l\" or platform_machine == \"ppc64le\" or platform_machine == \"ppc64\" or platform_machine == \"AMD64\" or platform_machine == \"aarch64\" or platform_machine == \"arm64\" or platform_machine == \"ARM64\" or platform_machine == \"x86\" or platform_machine == \"i686\" or platform_machine == \"riscv64\" or platform_machine == \"riscv64gc\") and platform_python_implementation == \"CPython\"",
] ]
files = [ files = [
{file = "urllib3_future-2.23.900-py3-none-any.whl", hash = "sha256:4303dc9a5d8c4c810f3fcd3b13bd62d69c3a4f63bb7e62089410907c34aeaf6b"}, {file = "urllib3_future-2.24.900-py3-none-any.whl", hash = "sha256:a4fe06b59b0c3ce2fffa821c3de72eaf59b1e61619becc255dee18919ee25260"},
{file = "urllib3_future-2.23.900.tar.gz", hash = "sha256:f488f22983f96cd8c733640b47e6a32def462d33c59bcc4bea6d97a8568b70c8"}, {file = "urllib3_future-2.24.900.tar.gz", hash = "sha256:05c2e9d09293ab29a154fed8f5b60644c7cdaffe4ae2587a07b1c7f00f23a7fb"},
] ]
[[package]] [[package]]
name = "wassima" name = "wassima"
version = "2.1.2" version = "2.1.3"
requires_python = ">=3.7" requires_python = ">=3.7"
summary = "Access your OS root certificates with utmost ease" summary = "Access your OS root certificates with utmost ease"
groups = ["default"] groups = ["default"]
marker = "sys_platform != \"emscripten\"" marker = "sys_platform != \"emscripten\""
files = [ files = [
{file = "wassima-2.1.2-py3-none-any.whl", hash = "sha256:9a5779e8ade39960973c6cc17e095cc998d6345ab3dbdd1ec2d7fde8c356fb82"}, {file = "wassima-2.1.3-py3-none-any.whl", hash = "sha256:f8251c23720dd092117a554b3012756caee3feed6d0122c2458a2aca5d7d92c5"},
{file = "wassima-2.1.2.tar.gz", hash = "sha256:f74b5441151728c54118ece6d747cfe92b2c595a3d062a1955f42a2bc894cd10"}, {file = "wassima-2.1.3.tar.gz", hash = "sha256:fcd6c38be0f909c393da35cb2a993101fcdcff673b8fa8d5da228f73b630d0d0"},
] ]
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "plyngent" name = "plyngent"
version = "0.3.0" version = "0.4.0"
description = "LLM chat and agent toolkit" description = "LLM chat and agent toolkit"
authors = [ authors = [
{ name = "HivertMoZara", email = "worldmozara@163.com" }, { name = "HivertMoZara", email = "worldmozara@163.com" },
+10 -15
View File
@@ -31,28 +31,23 @@ DEFAULT_CONTEXT_MAX_CHARS = DEFAULT_CONTEXT_MAX_TOKENS * 4
def truncate_tool_result( def truncate_tool_result(
text: str, text: str,
max_chars: int = DEFAULT_TOOL_RESULT_MAX_CHARS, max_chars: int = DEFAULT_TOOL_RESULT_MAX_CHARS,
*,
hint: bool = True,
) -> str: ) -> str:
"""Cap tool output so huge dumps do not flood model context. """Cap tool output so huge dumps do not flood model context.
With ``hint=True`` the suffix tells the model the result was cut and that This is the generic agent-loop cap for arbitrary tool results. It embeds a
it may ask the user to raise the config limit (``max_tool_result_chars``). memory truncate token via :func:`truncate_generic` so the model can always
``hint=False`` keeps the terse marker for internal shrinks whose cap is not resume the remainder with ``get_truncated`` — resumable or not (remainders
a user-facing config knob (e.g. request-time compacting). live in the in-memory store for the process lifetime). The compact marker
``[Truncated (N chars max.; M omitted). truncate_token=...]`` is emitted
only when output is actually cut.
""" """
if max_chars < 1: if max_chars < 1:
return text return text
if len(text) <= max_chars: if len(text) <= max_chars:
return text return text
omitted = len(text) - max_chars from plyngent.tools.truncate_token import truncate_generic
if hint:
return ( return truncate_generic(text, max_chars)
f"{text[:max_chars]}\n"
f"...[Truncated ({max_chars} chars max.; {omitted} chars omitted). "
f"You may ask for user to increase the limit via configuration file.]"
)
return f"{text[:max_chars]}\n...[truncated {omitted} characters]"
def estimate_message_chars(message: AnyChatMessage) -> int: def estimate_message_chars(message: AnyChatMessage) -> int:
@@ -120,7 +115,7 @@ def _shrink_tool(message: ToolChatMessage, max_chars: int) -> ToolChatMessage:
return message return message
return msgspec.structs.replace( return msgspec.structs.replace(
message, message,
content=truncate_tool_result(message.content, max_chars, hint=False), content=truncate_tool_result(message.content, max_chars),
) )
+7 -3
View File
@@ -17,13 +17,17 @@ DEFAULT_DIRECTIVE_REMINDER_TOKENS = 100_000
DEFAULT_DIRECTIVE_REMINDER_TEXT = """\ DEFAULT_DIRECTIVE_REMINDER_TEXT = """\
Tool playbook still applies (see system). Hard constraints: Tool playbook still applies (see system). Hard constraints:
- Prefer file tools over shell; several `run_command` calls may run in parallel; \ - Prefer file tools over shell; several `run_argv` calls may run in parallel; \
use `run_command_batch` for ordered pipelines. use `run_argv_batch` for ordered pipelines.
- `edit_replace`: fix match / `max_replaces`; `edit_lineno` edits only lines read via \ - `edit_replace`: fix match / `max_replaces`; `edit_lineno` edits only lines read via \
`read_file(with_lineno=true)`, and line numbers reset after any file write. `read_file(with_lineno=true)`, and line numbers reset after any file write.
- `read_file` results start with `L{begin}-{end}` (1-based lines; offset is 0-based).
- Truncated results carry a `truncate_token=...`; continue with `get_truncated`.
- Prefer `fetch` for HTTP(S); private/LAN hosts need human policy allow (not YOLO). - Prefer `fetch` for HTTP(S); private/LAN hosts need human policy allow (not YOLO).
- PTY secrets only via `ask_into_pty`; denylists and confirms still apply. - PTY secrets only via `ask_into_pty`; denylists and confirms still apply.
- Todo stack: open items mean unfinished work. - `wait` pauses; pressing Enter disturbs early (optional reason).
- Todo stack: open items mean unfinished work; the stack does not auto-clear — \
call `todo_clear` (or pop finished TOP groups) when the whole plan is done.
""" """
_BAND_MARKER = re.compile( _BAND_MARKER = re.compile(
+14 -1
View File
@@ -16,6 +16,7 @@ from plyngent.lmproto.openai_compatible.model import (
ChatCompletionChunk, ChatCompletionChunk,
ChatCompletionResponse, ChatCompletionResponse,
ChatCompletionsParam, ChatCompletionsParam,
DeltaMessage,
StreamOptions, StreamOptions,
StreamToolCallDelta, StreamToolCallDelta,
ToolChatMessage, ToolChatMessage,
@@ -254,6 +255,18 @@ async def _non_stream_round(
) )
def _append_reasoning_delta(
reasoning_parts: list[str],
delta: DeltaMessage,
) -> list[str]:
"""Accumulate a reasoning delta; a ``full`` (``.done``) event replaces fragments."""
if isinstance(delta.reasoning_content, str) and delta.reasoning_content:
if delta.reasoning_full is True:
return [delta.reasoning_content]
return [*reasoning_parts, delta.reasoning_content]
return reasoning_parts
async def _stream_round( async def _stream_round(
client: AnyLLMClient, client: AnyLLMClient,
param: ChatCompletionsParam, param: ChatCompletionsParam,
@@ -295,7 +308,7 @@ async def _stream_round(
saw_terminal = True saw_terminal = True
delta = choice.delta delta = choice.delta
if isinstance(delta.reasoning_content, str) and delta.reasoning_content: if isinstance(delta.reasoning_content, str) and delta.reasoning_content:
reasoning_parts.append(delta.reasoning_content) reasoning_parts = _append_reasoning_delta(reasoning_parts, delta)
yield ReasoningDeltaEvent(content=delta.reasoning_content) yield ReasoningDeltaEvent(content=delta.reasoning_content)
if isinstance(delta.content, str) and delta.content: if isinstance(delta.content, str) and delta.content:
content_parts.append(delta.content) content_parts.append(delta.content)
+31 -14
View File
@@ -171,7 +171,7 @@ def response_to_assistant_message(response: Response) -> AssistantChatMessage:
) )
for call in calls for call in calls
] ]
reasoning = _reasoning_summary_text(response) reasoning = reasoning_summary_text(response)
return AssistantChatMessage( return AssistantChatMessage(
content=text or None, content=text or None,
tool_calls=tool_calls, tool_calls=tool_calls,
@@ -179,23 +179,40 @@ def response_to_assistant_message(response: Response) -> AssistantChatMessage:
) )
def _reasoning_summary_text(response: Response) -> str: def _reasoning_text_blocks(raw: object) -> list[str]:
"""Collect text from a reasoning block list (``summary``/``content``)."""
parts: list[str] = []
if not isinstance(raw, list):
return parts
for block_obj in cast("list[object]", raw):
if not isinstance(block_obj, dict):
continue
block_map = cast("dict[str, object]", block_obj)
if block_map.get("type") in {"summary_text", "output_text", "reasoning_content"}:
text = block_map.get("text")
if isinstance(text, str) and text:
parts.append(text)
return parts
def reasoning_summary_text(response: Response) -> str:
"""Concatenate reasoning text from the response.
OpenAI puts ``reasoning`` items in ``output`` with a ``summary`` block list.
DeepSeek (Responses convention) additionally returns the full chain-of-thought
in the top-level ``response.reasoning`` (``content``/``summary`` block lists);
prefer it when present so the two sources are not duplicated.
"""
reasoning = response.reasoning
if isinstance(reasoning, dict):
parts = _reasoning_text_blocks(reasoning.get("content")) + _reasoning_text_blocks(reasoning.get("summary"))
if parts:
return "".join(parts)
parts: list[str] = [] parts: list[str] = []
for raw in response.output: for raw in response.output:
if raw.get("type") != "reasoning": if raw.get("type") != "reasoning":
continue continue
summary = raw.get("summary") parts.extend(_reasoning_text_blocks(raw.get("summary")))
if not isinstance(summary, list):
continue
summary_items = cast("list[object]", summary)
for block_obj in summary_items:
if not isinstance(block_obj, dict):
continue
block_map = cast("dict[str, object]", block_obj)
if block_map.get("type") in {"summary_text", "output_text"}:
text = block_map.get("text")
if isinstance(text, str) and text:
parts.append(text)
return "".join(parts) return "".join(parts)
+53 -11
View File
@@ -15,6 +15,7 @@ from msgspec import UNSET
from plyngent.agent.responses_bridge import ( from plyngent.agent.responses_bridge import (
chat_param_to_responses_kwargs, chat_param_to_responses_kwargs,
reasoning_summary_text,
response_to_assistant_message, response_to_assistant_message,
response_to_chat_completion, response_to_chat_completion,
responses_status_to_finish_reason, responses_status_to_finish_reason,
@@ -45,7 +46,16 @@ if TYPE_CHECKING:
) )
# Responses stream event types that carry reasoning text deltas. # Responses stream event types that carry reasoning text deltas.
_REASONING_DELTA_TYPES = {"response.reasoning_summary_text.delta", "response.reasoning_text.delta"} # OpenAI streams ``response.reasoning_summary_text.delta``; DeepSeek streams
# ``response.reasoning_text.delta`` fragments and emits the FULL chain-of-thought
# once via ``response.reasoning_text.done``. Fragments append; a ``.done`` event
# carries the complete text and must replace the accumulated fragments (otherwise
# the CoT is concatenated twice).
_REASONING_DELTA_TYPES = {
"response.reasoning_summary_text.delta",
"response.reasoning_text.delta",
}
_REASONING_FULL_TYPES = {"response.reasoning_text.done"}
# Terminal events that carry the full Response object (status, tool calls, usage). # Terminal events that carry the full Response object (status, tool calls, usage).
# ``response.failed`` also carries a Response but is surfaced as an error. # ``response.failed`` also carries a Response but is surfaced as an error.
_FINAL_EVENT_TYPES = {"response.completed", "response.incomplete"} _FINAL_EVENT_TYPES = {"response.completed", "response.incomplete"}
@@ -90,6 +100,34 @@ def _decode_final_event(event: ResponseStreamEvent) -> ResponseModel | None:
return None return None
def _stream_reasoning_content(event: ResponseStreamEvent) -> tuple[str | None, bool]:
"""Return (reasoning_text, is_full) from a reasoning stream event.
``.delta`` events carry incremental fragments (``is_full=False``); ``.done``
events carry the full chain-of-thought (``is_full=True``) and must replace
the accumulated fragments.
"""
if event.type not in _REASONING_DELTA_TYPES and event.type not in _REASONING_FULL_TYPES:
return None, False
content = event.delta if isinstance(event.delta, str) else event.text
if not isinstance(content, str) or not content:
return None, False
return content, event.type in _REASONING_FULL_TYPES
def _terminal_chunks(final: ResponseModel, *, model: str) -> list[ChatCompletionChunk]:
"""Build the post-stream chunks (tool calls, finish_reason, usage)."""
assistant = response_to_assistant_message(final)
has_tools = assistant.tool_calls is not UNSET and bool(assistant.tool_calls)
finish = responses_status_to_finish_reason(final, has_tool_calls=has_tools)
chunks: list[ChatCompletionChunk] = list(tool_call_chunks_from_response(final, model=model))
chunks.append(finish_reason_chunk(model=model, finish_reason=finish))
usage = usage_chunk_from_response(final, model=model)
if usage is not None:
chunks.append(usage)
return chunks
async def _stream_as_chat_chunks( async def _stream_as_chat_chunks(
client: OpenAIClient, client: OpenAIClient,
create: ResponsesCreateParam, create: ResponsesCreateParam,
@@ -108,6 +146,7 @@ async def _stream_as_chat_chunks(
""" """
stream = await client.responses(create, stream=True) stream = await client.responses(create, stream=True)
final: ResponseModel | None = None final: ResponseModel | None = None
saw_reasoning = False
async for event in stream: async for event in stream:
etype = event.type etype = event.type
if etype in {"error", "response.failed"}: if etype in {"error", "response.failed"}:
@@ -118,8 +157,10 @@ async def _stream_as_chat_chunks(
if etype == "response.output_text.delta" and isinstance(event.delta, str) and event.delta: if etype == "response.output_text.delta" and isinstance(event.delta, str) and event.delta:
yield text_delta_chunk(model=model, content=event.delta) yield text_delta_chunk(model=model, content=event.delta)
continue continue
if etype in _REASONING_DELTA_TYPES and isinstance(event.delta, str) and event.delta: reasoning, reasoning_full = _stream_reasoning_content(event)
yield reasoning_delta_chunk(model=model, content=event.delta) if reasoning is not None:
saw_reasoning = True
yield reasoning_delta_chunk(model=model, content=reasoning, full=reasoning_full)
continue continue
if final is None: if final is None:
@@ -127,15 +168,16 @@ async def _stream_as_chat_chunks(
# The loop will treat empty + no finish_reason as a glitch. # The loop will treat empty + no finish_reason as a glitch.
return return
assistant = response_to_assistant_message(final) if not saw_reasoning:
has_tools = assistant.tool_calls is not UNSET and bool(assistant.tool_calls) # DeepSeek sometimes delivers reasoning only on the terminal response
finish = responses_status_to_finish_reason(final, has_tool_calls=has_tools) # (top-level ``reasoning``); surface it so tool-call turns keep their
for chunk in tool_call_chunks_from_response(final, model=model): # reasoning_content when the assistant message goes back to the API.
terminal_reasoning = reasoning_summary_text(final)
if terminal_reasoning:
yield reasoning_delta_chunk(model=model, content=terminal_reasoning)
for chunk in _terminal_chunks(final, model=model):
yield chunk yield chunk
yield finish_reason_chunk(model=model, finish_reason=finish)
usage = usage_chunk_from_response(final, model=model)
if usage is not None:
yield usage
async def dispatch_responses( async def dispatch_responses(
+8 -2
View File
@@ -58,7 +58,13 @@ def text_delta_chunk(*, model: str, content: str, created: int = 0) -> ChatCompl
) )
def reasoning_delta_chunk(*, model: str, content: str, created: int = 0) -> ChatCompletionChunk: def reasoning_delta_chunk(
*,
model: str,
content: str,
created: int = 0,
full: bool = False,
) -> ChatCompletionChunk:
return ChatCompletionChunk( return ChatCompletionChunk(
id="bridge-stream", id="bridge-stream",
object="chat.completion.chunk", object="chat.completion.chunk",
@@ -67,7 +73,7 @@ def reasoning_delta_chunk(*, model: str, content: str, created: int = 0) -> Chat
choices=[ choices=[
ChunkChoice( ChunkChoice(
index=0, index=0,
delta=DeltaMessage(reasoning_content=content), delta=DeltaMessage(reasoning_content=content, reasoning_full=full or UNSET),
) )
], ],
) )
+8
View File
@@ -17,6 +17,7 @@ from plyngent.cli.editor import (
resolve_config_path, resolve_config_path,
) )
from plyngent.cli.exit_codes import EXIT_CANCELLED, EXIT_OK, EXIT_TURN_FAILED from plyngent.cli.exit_codes import EXIT_CANCELLED, EXIT_OK, EXIT_TURN_FAILED
from plyngent.cli.interrupt import install_keyboard_interrupt_sigint
from plyngent.cli.limits import install_cli_limit_hooks from plyngent.cli.limits import install_cli_limit_hooks
from plyngent.cli.repl import run_repl from plyngent.cli.repl import run_repl
from plyngent.cli.retry import run_user_text_with_retries from plyngent.cli.retry import run_user_text_with_retries
@@ -484,6 +485,13 @@ def chat_cmd(
raise click.ClickException(msg) raise click.ClickException(msg)
root = workspace if workspace is not None else Path.cwd() root = workspace if workspace is not None else Path.cwd()
# asyncio.run installs its own SIGINT handler when the current one is the
# default: the first Ctrl+C then silently cancels the main task (invisible
# at the REPL prompt) and the second raises KeyboardInterrupt, which also
# cancels memory.close() on a Ctrl+D exit ("Aborted!"). Pre-install a plain
# KeyboardInterrupt-raising handler so Ctrl+C stays a normal KeyboardInterrupt
# that the REPL / turn-retry / oneshot code handles deliberately.
install_keyboard_interrupt_sigint()
code = asyncio.run( code = asyncio.run(
_run_chat( _run_chat(
config_path=config_path, config_path=config_path,
+65 -4
View File
@@ -1,10 +1,11 @@
from __future__ import annotations from __future__ import annotations
import contextlib import contextlib
import json
import os import os
import sys import sys
from contextvars import ContextVar from contextvars import ContextVar
from typing import TYPE_CHECKING, Literal from typing import TYPE_CHECKING, Literal, cast
import click import click
@@ -28,6 +29,10 @@ if TYPE_CHECKING:
_TOOL_RESULT_PREVIEW = 120 _TOOL_RESULT_PREVIEW = 120
_TOOL_ARGS_PREVIEW = 80 _TOOL_ARGS_PREVIEW = 80
# Tool calls that render as a single pretty summary line instead of
# ``[tool]`` / ``[tool result]``.
_PRETTY_TOOLS = frozenset({"read_file", "todo_push", "todo_update"})
# Process/session display flags (set from ReplState / slash). # Process/session display flags (set from ReplState / slash).
_verbose_tool_results: ContextVar[bool] = ContextVar("verbose_tool_results", default=False) _verbose_tool_results: ContextVar[bool] = ContextVar("verbose_tool_results", default=False)
_markdown_enabled: ContextVar[bool] = ContextVar("markdown_enabled", default=True) _markdown_enabled: ContextVar[bool] = ContextVar("markdown_enabled", default=True)
@@ -69,6 +74,49 @@ def _preview(text: str, limit: int) -> str:
return text[:limit] + "…" return text[:limit] + "…"
def _json_str_arg(args_json: str, key: str) -> str | None:
"""Extract a string argument from a tool-call arguments JSON blob."""
try:
raw = json.loads(args_json)
except ValueError:
return None
if isinstance(raw, dict):
value = cast("dict[str, object]", raw).get(key)
if isinstance(value, str):
return value
return None
def _read_file_pretty(args_json: str, result: str) -> str:
"""One-line summary for a ``read_file`` call: ``* Read 'path' L1-4 (done)``."""
path = _json_str_arg(args_json, "path") or "?"
if result.startswith("error: file not found"):
return f"* Read '{path}' (file not found)"
if result.startswith("error: not a file"):
return f"* Read '{path}' (not a file)"
if result.startswith("error:"):
return f"* Read '{path}' (error)"
first = result.splitlines()[0] if result.splitlines() else ""
if first.startswith("L") and "-" in first:
return f"* Read '{path}' {first} (done)"
return f"* Read '{path}' (done)"
def _todo_pretty(name: str, result: str) -> str:
"""Summary for todo push/update: header + the rendered todo stack."""
label = "Todo Push" if name == "todo_push" else "Todo Update"
return f"* {label}:\n{result}"
def _pretty_tool_result(name: str, args_json: str, result: str) -> str | None:
"""Pretty summary for a known tool call + result; None keeps the old style."""
if name == "read_file":
return _read_file_pretty(args_json, result)
if name in {"todo_push", "todo_update"}:
return _todo_pretty(name, result)
return None
def _echo_stream(text: str) -> None: def _echo_stream(text: str) -> None:
"""Write without newline and flush so assistant text appears token-by-token.""" """Write without newline and flush so assistant text appears token-by-token."""
click.echo(text, nl=False) click.echo(text, nl=False)
@@ -144,6 +192,9 @@ async def render_events( # noqa: C901, PLR0912, PLR0915
assistant_buf: list[str] = [] assistant_buf: list[str] = []
printed_reasoning = False printed_reasoning = False
printed_assistant = False printed_assistant = False
# Tool calls buffer so prettified tools can render once their result lands
# (the summary line needs the status/range). FIFO matches loop event order.
pending_tools: list[tuple[str, str, bool]] = []
def flush_assistant() -> None: def flush_assistant() -> None:
nonlocal source, assistant_buf, printed_assistant nonlocal source, assistant_buf, printed_assistant
@@ -192,14 +243,24 @@ async def render_events( # noqa: C901, PLR0912, PLR0915
flush_assistant() flush_assistant()
call = event.tool_call call = event.tool_call
if isinstance(call, AssistantFunctionToolCall): if isinstance(call, AssistantFunctionToolCall):
args = _preview(call.function.arguments, _TOOL_ARGS_PREVIEW) name = call.function.name
click.secho(f"\n[tool] {call.function.name}({args})", fg="yellow") args = call.function.arguments
pretty_tool = name in _PRETTY_TOOLS
pending_tools.append((name, args, pretty_tool))
if not pretty_tool:
preview = _preview(args, _TOOL_ARGS_PREVIEW)
click.secho(f"\n[tool] {name}({preview})", fg="yellow")
else: else:
pending_tools.append(("custom", call.id, False))
click.secho(f"\n[tool] custom id={call.id}", fg="yellow") click.secho(f"\n[tool] custom id={call.id}", fg="yellow")
elif isinstance(event, ToolResultEvent): elif isinstance(event, ToolResultEvent):
flush_assistant() flush_assistant()
content = event.message.content content = event.message.content
if show_full: name, args, pretty_tool = pending_tools.pop(0) if pending_tools else ("", "", False)
pretty_line = _pretty_tool_result(name, args, content) if pretty_tool else None
if pretty_line is not None:
click.secho(f"\n{pretty_line}", fg="yellow")
elif show_full:
click.secho(f"[tool ok]\n{content}", fg="magenta") click.secho(f"[tool ok]\n{content}", fg="magenta")
else: else:
preview = _preview(content, _TOOL_RESULT_PREVIEW) preview = _preview(content, _TOOL_RESULT_PREVIEW)
+27
View File
@@ -29,6 +29,33 @@ def set_sigint_reinstall(callback: Callable[[], None] | None) -> None:
_reinstall_holder[0] = callback _reinstall_holder[0] = callback
def raise_keyboard_interrupt_sigint(signum: int, frame: FrameType | None) -> None:
"""SIGINT handler: raise KeyboardInterrupt (same effect as the default).
A distinct function object — deliberately *not*
``signal.default_int_handler`` — so that :func:`asyncio.run`'s Runner does
not recognise it and install its own silent-cancel handler on top.
"""
del signum, frame
raise KeyboardInterrupt
def install_keyboard_interrupt_sigint() -> None:
"""Install :func:`raise_keyboard_interrupt_sigint` as the SIGINT handler.
``asyncio.run`` (Runner) replaces the SIGINT handler with its own when the
current one *is* ``signal.default_int_handler``: the first Ctrl+C then
silently cancels the main task (invisible at the REPL prompt, leaving the
task marked cancelled) and only the second raises KeyboardInterrupt. A
later Ctrl+D (EOF) is then cancelled mid-``memory.close()`` and the CLI
exits with ``Aborted!``. Installing a plain KeyboardInterrupt-raising
handler first keeps every Ctrl+C a normal KeyboardInterrupt that the CLI
(prompt, turn-retry, oneshot) already handles deliberately.
"""
with contextlib.suppress(ValueError):
_ = signal.signal(signal.SIGINT, raise_keyboard_interrupt_sigint)
@contextmanager @contextmanager
def pause_task_cancel_for_prompt() -> Generator[None]: def pause_task_cancel_for_prompt() -> Generator[None]:
"""Disable turn-task cancel during blocking TTY prompts (confirm, etc.). """Disable turn-task cancel during blocking TTY prompts (confirm, etc.).
+12 -9
View File
@@ -1,5 +1,6 @@
from __future__ import annotations from __future__ import annotations
import contextlib
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
import click import click
@@ -23,6 +24,10 @@ def _echo_user(text: str) -> None:
def _echo_interrupted() -> None: def _echo_interrupted() -> None:
# A further Ctrl+C during this output must stay benign (same rule as
# retry._echo_cancel_lines): after a turn the SIGINT handler is removed, so
# the next Ctrl+C raises KeyboardInterrupt at an arbitrary bytecode.
with contextlib.suppress(KeyboardInterrupt):
click.echo() click.echo()
click.secho("interrupted", fg="yellow") click.secho("interrupted", fg="yellow")
click.echo() click.echo()
@@ -49,15 +54,15 @@ async def run_repl(state: ReplState) -> None:
click.echo() click.echo()
break break
# Between turns the turn-task SIGINT handler is removed (run_cancellable),
# so a stray Ctrl+C raises KeyboardInterrupt at whatever bytecode is
# executing — echoing user text, expire_yolo_once, gaps between
# statements. Swallow it and re-prompt instead of exiting the REPL.
try:
if entry is None: if entry is None:
continue continue
if entry.startswith("/"): if entry.startswith("/"):
try:
cont = await handle_slash(state, entry) cont = await handle_slash(state, entry)
except KeyboardInterrupt:
_echo_interrupted()
continue
if not cont: if not cont:
break break
if state.pending_user_text is not None: if state.pending_user_text is not None:
@@ -66,8 +71,6 @@ async def run_repl(state: ReplState) -> None:
_echo_user(text) _echo_user(text)
try: try:
_ = await run_user_text_with_retries(state.agent, text) _ = await run_user_text_with_retries(state.agent, text)
except KeyboardInterrupt:
_echo_interrupted()
finally: finally:
state.expire_yolo_once() state.expire_yolo_once()
continue continue
@@ -75,7 +78,7 @@ async def run_repl(state: ReplState) -> None:
_echo_user(entry) _echo_user(entry)
try: try:
_ = await run_user_text_with_retries(state.agent, entry) _ = await run_user_text_with_retries(state.agent, entry)
except KeyboardInterrupt:
_echo_interrupted()
finally: finally:
state.expire_yolo_once() state.expire_yolo_once()
except KeyboardInterrupt:
_echo_interrupted()
+7
View File
@@ -64,6 +64,11 @@ async def run_cancellable[T](coro: Coroutine[object, object, T]) -> T:
task: asyncio.Task[T] = asyncio.create_task(coro) task: asyncio.Task[T] = asyncio.create_task(coro)
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
installed = False installed = False
# What to restore once the turn-cancel handler is removed. remove_signal_handler
# leaves SIG_DFL, which would terminate the process on a stray Ctrl+C between
# turns; restore the handler from before (the CLI installs a
# KeyboardInterrupt-raising one so the REPL can catch it).
previous_sigint = signal.getsignal(signal.SIGINT)
def _on_sigint() -> None: def _on_sigint() -> None:
# allow_task_cancel() uses a process-level depth counter (not ContextVar) # allow_task_cancel() uses a process-level depth counter (not ContextVar)
@@ -98,6 +103,8 @@ async def run_cancellable[T](coro: Coroutine[object, object, T]) -> T:
if installed: if installed:
with contextlib.suppress(NotImplementedError, RuntimeError, ValueError): with contextlib.suppress(NotImplementedError, RuntimeError, ValueError):
_ = loop.remove_signal_handler(signal.SIGINT) _ = loop.remove_signal_handler(signal.SIGINT)
with contextlib.suppress(ValueError):
_ = signal.signal(signal.SIGINT, previous_sigint)
# Only cancel if still running (e.g. KeyboardInterrupt path above). # Only cancel if still running (e.g. KeyboardInterrupt path above).
# Do not cancel a finished task — that would mask success. # Do not cancel a finished task — that would mask success.
if not task.done(): if not task.done():
+9 -2
View File
@@ -32,10 +32,14 @@ raise `max_replaces` or narrow `old_string`.
- `edit_lineno` edits only lines you read via `read_file(path, with_lineno=true)`; \ - `edit_lineno` edits only lines you read via `read_file(path, with_lineno=true)`; \
re-read with line numbers after any `edit_*` / `write_file` / copy/move/delete, \ re-read with line numbers after any `edit_*` / `write_file` / copy/move/delete, \
since line numbers go stale. since line numbers go stale.
- `read_file` results start with a 1-based line range `L{begin}-{end}` (`offset` \
is 0-based); `with_lineno` shows per-line numbers instead.
- Truncated results (any tool) carry a `truncate_token=...`; use `get_truncated` \
with it to keep reading (chains through truncations).
### Commands ### Commands
- Prefer `run_command` / `run_command_batch` (argv, no shell) over `bash -c` or similar. - Prefer `run_argv` / `run_argv_batch` (argv lists, no shell) over `bash -c` or similar.
- Several `run_command` calls in one step may run in parallel; use `run_command_batch` \ - Several `run_argv` calls in one step may run in parallel; use `run_argv_batch` \
for ordered pipelines (`pipe_out` / `mix_stderr` as needed). for ordered pipelines (`pipe_out` / `mix_stderr` as needed).
- Prefer `vcs_*` for status/diff/log/branch when enough. - Prefer `vcs_*` for status/diff/log/branch when enough.
@@ -49,6 +53,7 @@ otherwise a small default is used. Never rely on shell to spoof identity.
### Humans & safety ### Humans & safety
- Prefer `ask_user_line` / `ask_user_choice` / `ask_user_form` over waiting for the next free-form turn. - Prefer `ask_user_line` / `ask_user_choice` / `ask_user_form` over waiting for the next free-form turn.
- Use `wait` to pause; pressing Enter during the wait disturbs it (reason optional; turn continues immediately).
- Overwrites, deletes, shells, and risky ops may require human confirm; hard denylists are not skipped by YOLO. - Overwrites, deletes, shells, and risky ops may require human confirm; hard denylists are not skipped by YOLO.
- Temporary scratch: `new_temporary_workspace`. - Temporary scratch: `new_temporary_workspace`.
@@ -60,6 +65,8 @@ otherwise a small default is used. Never rely on shell to spoof identity.
### Todos ### Todos
- Use the todo stack for multi-step work (LIFO groups: push related items, finish/update, \ - Use the todo stack for multi-step work (LIFO groups: push related items, finish/update, \
pop the group). Open items mean unfinished work. pop the group). Open items mean unfinished work.
- The stack does **not** auto-clear. When every task is done (all items done/cancelled), \
call `todo_clear` (or pop finished TOP groups) so no hygiene noise is left behind.
""" """
@@ -239,6 +239,9 @@ class DeltaMessage(Struct):
# Providers often send content=null / reasoning_content=null on partial chunks. # Providers often send content=null / reasoning_content=null on partial chunks.
content: str | None | Unset = UNSET content: str | None | Unset = UNSET
reasoning_content: str | None | Unset = UNSET reasoning_content: str | None | Unset = UNSET
# True when reasoning_content is the FULL chain-of-thought (e.g. DeepSeek's
# ``response.reasoning_text.done``), replacing any streamed fragments.
reasoning_full: bool | Unset = UNSET
tool_calls: list[StreamToolCallDelta] | Unset = UNSET tool_calls: list[StreamToolCallDelta] | Unset = UNSET
+84 -1
View File
@@ -2,10 +2,12 @@ from __future__ import annotations
import asyncio import asyncio
import contextlib import contextlib
import select
import sys import sys
import time
from contextlib import AbstractContextManager from contextlib import AbstractContextManager
from dataclasses import dataclass from dataclasses import dataclass
from typing import TYPE_CHECKING, Protocol, runtime_checkable from typing import TYPE_CHECKING, Protocol, TextIO, cast, runtime_checkable
import click import click
@@ -98,6 +100,87 @@ def _readline_input(prompt: str, *, completions: Sequence[str] | None = None) ->
readline.set_completer_delims(previous_delims) readline.set_completer_delims(previous_delims)
type PollFn = Callable[
[list[object], list[object], list[object], float],
tuple[list[object], list[object], list[object]],
]
def _read_line_with_timeout_posix(
prompt: str,
timeout: float,
*,
stream: TextIO,
poll: PollFn,
) -> str | None:
"""Blocking line read on POSIX: poll stdin, then read one line on data.
Returns the line (without trailing newline), ``""`` for an empty Enter, or
``None`` when the timeout elapses before any line is complete (EOF too).
"""
_ = sys.stdout.write(prompt)
_ = sys.stdout.flush()
deadline = time.monotonic() + timeout
while True:
remaining = deadline - time.monotonic()
if remaining <= 0:
return None
ready, _, _ = poll([stream], [], [], min(remaining, 0.1))
if not ready:
continue
line = stream.readline()
if line == "":
return None # EOF
return line.rstrip("\n")
def _read_line_with_timeout_windows(prompt: str, timeout: float) -> str | None:
"""Blocking line read on Windows via msvcrt char polling (best-effort)."""
import msvcrt
_ = sys.stdout.write(prompt)
_ = sys.stdout.flush()
deadline = time.monotonic() + timeout
buf: list[str] = []
while time.monotonic() < deadline:
if msvcrt.kbhit():
ch = msvcrt.getwch()
if ch in ("\r", "\n"):
_ = sys.stdout.write("\n")
_ = sys.stdout.flush()
return "".join(buf)
if ch == "\x03":
raise KeyboardInterrupt
if ch in ("\x08", "\x7f") and buf: # backspace / DEL
_ = buf.pop()
_ = sys.stdout.write("\b \b")
_ = sys.stdout.flush()
continue
buf.append(ch)
_ = sys.stdout.write(ch)
_ = sys.stdout.flush()
else:
time.sleep(0.05)
_ = sys.stdout.write("\n")
_ = sys.stdout.flush()
return None
def read_line_with_timeout(prompt: str, timeout: float) -> str | None:
"""Read one line, blocking at most ``timeout`` seconds.
Returns the typed line (without trailing newline), ``""`` for an empty
Enter, or ``None`` when the timeout elapses first (EOF too). Ctrl+C
propagates as :class:`KeyboardInterrupt`. POSIX polls stdin via ``select``;
Windows uses ``msvcrt`` char polling (best-effort).
"""
if timeout < 0:
timeout = 0.0
if sys.platform == "win32":
return _read_line_with_timeout_windows(prompt, timeout)
return _read_line_with_timeout_posix(prompt, timeout, stream=sys.stdin, poll=cast("PollFn", select.select))
class ClickPromptBackend: class ClickPromptBackend:
"""Click/TTY backend for interactive prompts (readline + Tab when available).""" """Click/TTY backend for interactive prompts (readline + Tab when available)."""
+3 -2
View File
@@ -8,6 +8,7 @@ from .chat import CHAT_TOOLS as CHAT_TOOLS
from .chat import ask_user as ask_user from .chat import ask_user as ask_user
from .chat import choose_user as choose_user from .chat import choose_user as choose_user
from .chat import form_user as form_user from .chat import form_user as form_user
from .chat import wait as wait
from .context import InstanceState as InstanceState from .context import InstanceState as InstanceState
from .context import SessionState as SessionState from .context import SessionState as SessionState
from .context import bind_instance as bind_instance from .context import bind_instance as bind_instance
@@ -41,8 +42,8 @@ from .process import ask_into_pty as ask_into_pty
from .process import close_pty as close_pty from .process import close_pty as close_pty
from .process import open_pty as open_pty from .process import open_pty as open_pty
from .process import read_pty as read_pty from .process import read_pty as read_pty
from .process import run_command as run_command from .process import run_argv as run_argv
from .process import run_command_batch as run_command_batch from .process import run_argv_batch as run_argv_batch
from .process import write_pty as write_pty from .process import write_pty as write_pty
from .process import write_pty_keys as write_pty_keys from .process import write_pty_keys as write_pty_keys
from .temp_workspace import cleanup_temporary_workspaces as cleanup_temporary_workspaces from .temp_workspace import cleanup_temporary_workspaces as cleanup_temporary_workspaces
+2 -1
View File
@@ -1,5 +1,6 @@
from .ask import ask_user as ask_user from .ask import ask_user as ask_user
from .choose import choose_user as choose_user from .choose import choose_user as choose_user
from .form import form_user as form_user from .form import form_user as form_user
from .wait import wait as wait
CHAT_TOOLS = [ask_user, choose_user, form_user] CHAT_TOOLS = [ask_user, choose_user, form_user, wait]
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
import asyncio
from plyngent.agent import ToolTag, tool
from plyngent.prompting import get_prompt_backend, read_line_with_timeout, run_prompt_async
WAIT_PROMPT = "Waiting for {duration}s. Press Enter to disturb. Feel free to ship any reason before pressing Enter. "
@tool(name="wait", tags=ToolTag.LOCAL)
async def wait(duration: int, *, reason: str | None = None) -> str:
"""Wait ``duration`` seconds before continuing.
Interactive sessions show a line prompt: pressing Enter (optionally after
typing a reason) "disturbs" the wait so the turn continues immediately.
Non-interactive runs simply sleep the full duration.
"""
if duration < 0:
return "error: duration must not be negative"
if duration == 0:
return "waited 0s"
backend = get_prompt_backend()
if not backend.is_interactive():
await asyncio.sleep(duration)
return f"waited {duration}s"
prompt = WAIT_PROMPT.format(duration=duration)
if reason:
prompt = (
f"Waiting for {duration}s (reason: {reason}). Press Enter to disturb. "
"Feel free to ship any reason before pressing Enter. "
)
line = await run_prompt_async(read_line_with_timeout, prompt, float(duration))
if line is None:
return f"waited {duration}s"
text = line.strip()
if text:
return f"disturbed by user: {text}"
return "disturbed by user (no reason)"
+20 -20
View File
@@ -59,8 +59,8 @@ def _basename(argv0: str) -> str:
return name.lower() return name.lower()
def _as_argv(args: Mapping[str, object]) -> list[str] | None: def _as_argv(args: Mapping[str, object], *, key: str = "command") -> list[str] | None:
command = args.get("command") command = args.get(key)
if not isinstance(command, list) or not command: if not isinstance(command, list) or not command:
return None return None
out: list[str] = [] out: list[str] = []
@@ -110,7 +110,7 @@ def _shell_or_dash_c_reason(argv: Sequence[str], *, via: str) -> str | None:
"""Confirm interactive shells and ``*-c`` one-liners so the user can inspect argv. """Confirm interactive shells and ``*-c`` one-liners so the user can inspect argv.
Multi-line reason (shown inside the CLI confirm box). ``via`` is a short Multi-line reason (shown inside the CLI confirm box). ``via`` is a short
label such as ``run_command`` or ``open_pty``. label such as ``run_argv`` or ``open_pty``.
For ``-c`` scripts, argv shows ``$(command)`` instead of inlining the body; For ``-c`` scripts, argv shows ``$(command)`` instead of inlining the body;
the full script is printed below untruncated, with every line indented. the full script is printed below untruncated, with every line indented.
@@ -181,31 +181,31 @@ def _delete_path_reason(args: Mapping[str, object]) -> str | None:
return f"delete path {path!r}{extra}" return f"delete path {path!r}{extra}"
def _run_command_reason(args: Mapping[str, object]) -> str | None: def _run_argv_reason(args: Mapping[str, object]) -> str | None:
argv = _as_argv(args) argv = _as_argv(args, key="argv")
if argv is None: if argv is None:
return None return None
return _shell_or_dash_c_reason(argv, via="run_command") return _shell_or_dash_c_reason(argv, via="run_argv")
def _batch_step_argv(item: object) -> list[str] | None: def _batch_step_argv(item: object) -> list[str] | None:
if not isinstance(item, dict): if not isinstance(item, dict):
return None return None
step = cast("dict[str, object]", item) step = cast("dict[str, object]", item)
command = step.get("command") argv = step.get("argv")
if not isinstance(command, list) or not command: if not isinstance(argv, list) or not argv:
return None return None
argv: list[str] = [] parts: list[str] = []
for part in cast("list[object]", command): for part in cast("list[object]", argv):
if not isinstance(part, str): if not isinstance(part, str):
return None return None
argv.append(part) parts.append(part)
return argv or None return parts or None
def _run_command_batch_reason(args: Mapping[str, object]) -> str | None: def _run_argv_batch_reason(args: Mapping[str, object]) -> str | None:
"""One confirm for the whole batch if any step is shell/REPL/-c.""" """One confirm for the whole batch if any step is shell/REPL/-c."""
raw = args.get("commands") raw = args.get("steps")
if isinstance(raw, str): if isinstance(raw, str):
try: try:
loaded: object = json.loads(raw) loaded: object = json.loads(raw)
@@ -218,11 +218,11 @@ def _run_command_batch_reason(args: Mapping[str, object]) -> str | None:
reason reason
for index, item in enumerate(cast("list[object]", raw)) for index, item in enumerate(cast("list[object]", raw))
if (argv := _batch_step_argv(item)) is not None if (argv := _batch_step_argv(item)) is not None
and (reason := _shell_or_dash_c_reason(argv, via=f"run_command_batch[{index}]")) is not None and (reason := _shell_or_dash_c_reason(argv, via=f"run_argv_batch[{index}]")) is not None
] ]
if not risky: if not risky:
return None return None
header = f"run_command_batch: {len(risky)} risky step(s) (review before allow)" header = f"run_argv_batch: {len(risky)} risky step(s) (review before allow)"
return header + "\n" + "\n".join(risky) return header + "\n" + "\n".join(risky)
@@ -262,10 +262,10 @@ def classify_danger(name: str, args: Mapping[str, object]) -> str | None: # noq
return _copy_path_reason(args) return _copy_path_reason(args)
if name == "write_file": if name == "write_file":
return _write_file_reason(args) return _write_file_reason(args)
if name == "run_command": if name == "run_argv":
return _run_command_reason(args) return _run_argv_reason(args)
if name == "run_command_batch": if name == "run_argv_batch":
return _run_command_batch_reason(args) return _run_argv_batch_reason(args)
if name == "open_pty": if name == "open_pty":
return _open_pty_reason(args) return _open_pty_reason(args)
if name == "fetch": if name == "fetch":
+2
View File
@@ -5,6 +5,7 @@ from .edit_replace import edit_replace as edit_replace
from .fs_ops import copy_path as copy_path from .fs_ops import copy_path as copy_path
from .fs_ops import delete_path as delete_path from .fs_ops import delete_path as delete_path
from .fs_ops import move_path as move_path from .fs_ops import move_path as move_path
from .get_truncated import get_truncated as get_truncated
from .glob_paths import glob_paths as glob_paths from .glob_paths import glob_paths as glob_paths
from .grep_files import grep_files as grep_files from .grep_files import grep_files as grep_files
from .listdir import listdir as listdir from .listdir import listdir as listdir
@@ -25,4 +26,5 @@ FILE_TOOLS = [
move_path, move_path,
delete_path, delete_path,
new_temporary_workspace, new_temporary_workspace,
get_truncated,
] ]
+68
View File
@@ -0,0 +1,68 @@
"""``get_truncated`` tool: resume truncated tool results via a truncate token."""
from __future__ import annotations
from typing import cast
from plyngent.agent import ToolTag, tool
from plyngent.agent.budget import DEFAULT_TOOL_RESULT_MAX_CHARS
from plyngent.tools.file.read import line_range_label, read_raw_text
from plyngent.tools.net.fetch import fetch
from plyngent.tools.truncate_token import (
decode_truncate_token,
get_remainder,
truncate_with_token,
)
@tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE)
async def get_truncated(token: str, *, max_chars: int = DEFAULT_TOOL_RESULT_MAX_CHARS) -> str:
"""Fetch the next chunk of a truncated result using its ``truncate_token``.
Pass the token shown at the end of a truncated ``read_file``, ``fetch``,
``run_command``, or earlier ``get_truncated`` result to keep reading without
re-requesting the whole source or raising limits. Truncated output carries a
fresh token, so chunks chain indefinitely. File chunks start with a 1-based
``L{begin}-{end}`` line range like ``read_file``; memory chunks (generic
tool output) are served from a short-lived in-memory store that is forgotten
when the agent exits.
"""
parsed = decode_truncate_token(token)
if parsed is None:
return "error: invalid truncate token"
if parsed.kind == "http":
return await fetch.handler(parsed.location, offset=parsed.offset, max_chars=parsed.limit)
if parsed.kind == "memory":
remainder = get_remainder(parsed.location)
if remainder is None:
return "error: truncate token expired (no longer in memory); re-run the tool"
start = parsed.offset
segment = remainder[start : start + max_chars]
chunk, _ = truncate_with_token(
segment,
max_chars,
kind="memory",
location=parsed.location,
offset=start,
limit=max_chars,
total_len=len(remainder),
)
return chunk
text, err = read_raw_text(parsed.location)
if err:
return err
text = cast("str", text)
start = parsed.offset
segment = text[start : start + max_chars]
chunk, _ = truncate_with_token(
segment,
max_chars,
kind="file",
location=parsed.location,
offset=start,
limit=max_chars,
total_len=len(text),
)
content_len = len(chunk.split("\n[Truncated", 1)[0]) if "\n[Truncated" in chunk else len(chunk)
header = line_range_label(text, start, start + content_len)
return f"{header}\n{chunk}"
+70 -9
View File
@@ -1,6 +1,10 @@
from __future__ import annotations from __future__ import annotations
from typing import cast
from plyngent.agent import ToolTag, mark_lineno_read, tool from plyngent.agent import ToolTag, mark_lineno_read, tool
from plyngent.agent.budget import DEFAULT_TOOL_RESULT_MAX_CHARS
from plyngent.tools.truncate_token import truncate_with_token
from plyngent.tools.workspace import resolve_path from plyngent.tools.workspace import resolve_path
_LINENO_WIDTH = 6 _LINENO_WIDTH = 6
@@ -17,6 +21,34 @@ def _format_with_lineno(lines: list[str], *, start_lineno: int) -> str:
return "".join(out) return "".join(out)
def line_range_label(text: str, start_char: int, end_char: int) -> str:
"""1-based inclusive line range covering ``text[start_char:end_char]``.
Char offsets do not map 1:1 to lines; this converts a contiguous char range
into the 1-based file line numbers it spans (offset is 0-based, so line 1
starts at char 0).
"""
begin = text.count("\n", 0, start_char) + 1
last = max(start_char, end_char - 1)
end = text.count("\n", 0, last) + 1
return f"L{begin}-{end}"
def read_raw_text(path: str) -> tuple[str | None, str]:
"""(text, error) for a workspace file read; ``error`` is ``""`` on success.
Shared by ``read_file`` and ``get_truncated`` so truncate-token char offsets
always refer to the raw file text (never the ``L{begin}-{end}`` header).
Missing paths and directories are distinguished for the caller.
"""
target = resolve_path(path)
if not target.exists():
return None, f"error: file not found: {path}"
if not target.is_file():
return None, f"error: not a file: {path}"
return target.read_text(encoding="utf-8", errors="replace"), ""
@tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE) @tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE)
async def read_file( async def read_file(
path: str, path: str,
@@ -24,18 +56,25 @@ async def read_file(
offset: int = 0, offset: int = 0,
limit: int | None = None, limit: int | None = None,
with_lineno: bool = False, with_lineno: bool = False,
max_chars: int = DEFAULT_TOOL_RESULT_MAX_CHARS,
) -> str: ) -> str:
"""Read a text file under the workspace. """Read a text file under the workspace.
``offset`` is 0-based line start; ``limit`` is max lines (None = rest of file). ``offset`` is 0-based line start (0 = first line); ``limit`` is max lines
When ``with_lineno`` is true, each line is prefixed with its 1-based file line (None = rest of file). When ``with_lineno`` is true, each line is prefixed
number (`` N|…``), matching ``edit_lineno`` numbering, and those lines are with its 1-based file line number (`` N|…``), matching ``edit_lineno``
marked readable for ``edit_lineno`` this turn. numbering, and those lines are marked readable for ``edit_lineno`` this turn.
``max_chars`` caps the returned slice (default: the agent's tool-result cap,
``DEFAULT_TOOL_RESULT_MAX_CHARS``); a ``truncate_token`` is appended when cut
so ``get_truncated`` can continue reading the rest without a new request.
Pass ``max_chars < 1`` to read unbounded.
""" """
target = resolve_path(path) target = resolve_path(path)
if not target.is_file(): text, err = read_raw_text(path)
return f"error: not a file: {path}" if err:
text = target.read_text(encoding="utf-8", errors="replace") return err
text = cast("str", text)
lines = text.splitlines(keepends=True) lines = text.splitlines(keepends=True)
if offset < 0: if offset < 0:
return "error: offset must be >= 0" return "error: offset must be >= 0"
@@ -46,5 +85,27 @@ async def read_file(
slice_lines = lines[start:end] slice_lines = lines[start:end]
if with_lineno: if with_lineno:
mark_lineno_read(str(target), set(range(start + 1, end + 1))) mark_lineno_read(str(target), set(range(start + 1, end + 1)))
return _format_with_lineno(slice_lines, start_lineno=start + 1) body = _format_with_lineno(slice_lines, start_lineno=start + 1)
return "".join(slice_lines) else:
body = "".join(slice_lines)
if max_chars >= 1:
char_start = len("".join(lines[:start]))
body, _ = truncate_with_token(
body,
max_chars,
kind="file",
location=path, # arg form: short token, resolves to the same file
offset=char_start,
limit=max_chars,
total_len=len(text),
# read_file bounds the read via offset/limit; only mark when the
# returned slice itself was cut by max_chars (not just "file continues").
emit_more_after=False,
)
if not body:
return ""
if with_lineno:
return body # per-line numbers already show the range
begin = start + 1
end_line = start + len(slice_lines)
return f"L{begin}-{end_line}\n{body}"
+17 -7
View File
@@ -16,6 +16,7 @@ from plyngent.tools.net.policy import (
parse_fetch_url, parse_fetch_url,
soft_confirm_reason, soft_confirm_reason,
) )
from plyngent.tools.truncate_token import truncate_with_token
def format_fetch_result( def format_fetch_result(
@@ -32,13 +33,20 @@ def format_fetch_result(
warnings: list[str], warnings: list[str],
body_kind: str, body_kind: str,
max_chars: int, max_chars: int,
offset: int = 0,
) -> str: ) -> str:
text = body_text start = max(0, offset)
char_truncated = False body = body_text[start:]
if max_chars >= 1 and len(text) > max_chars: body, token = truncate_with_token(
omitted = len(text) - max_chars body,
text = text[:max_chars] + f"\n...[truncated {omitted} characters]" max_chars,
char_truncated = True kind="http",
location=final_url,
offset=start,
limit=max_chars,
total_len=len(body_text),
)
char_truncated = token is not None
warn_line = "; ".join(warnings) if warnings else "" warn_line = "; ".join(warnings) if warnings else ""
parts = [ parts = [
f"status={status}", f"status={status}",
@@ -54,7 +62,7 @@ def format_fetch_result(
if warn_line: if warn_line:
parts.append(f"warnings={warn_line}") parts.append(f"warnings={warn_line}")
parts.append("--- body ---") parts.append("--- body ---")
parts.append(text) parts.append(body)
return "\n".join(parts) return "\n".join(parts)
@@ -72,6 +80,7 @@ async def fetch(
max_redirects: int = DEFAULT_MAX_REDIRECTS, max_redirects: int = DEFAULT_MAX_REDIRECTS,
follow_redirects: bool = True, follow_redirects: bool = True,
allow_http_downgrade: bool = False, allow_http_downgrade: bool = False,
offset: int = 0,
) -> str: ) -> str:
"""HTTP request (GET/POST/PUT/DELETE); return status, metadata, and truncated body. """HTTP request (GET/POST/PUT/DELETE); return status, metadata, and truncated body.
@@ -122,6 +131,7 @@ async def fetch(
warnings=result.warnings, warnings=result.warnings,
body_kind=result.body_kind, body_kind=result.body_kind,
max_chars=max_chars, max_chars=max_chars,
offset=offset,
) )
+4 -4
View File
@@ -4,14 +4,14 @@ from .open_pty import open_pty as open_pty
from .pty_terminal import decode_write_data as decode_write_data from .pty_terminal import decode_write_data as decode_write_data
from .pty_terminal import sanitize_pty_output_for_tool as sanitize_pty_output_for_tool from .pty_terminal import sanitize_pty_output_for_tool as sanitize_pty_output_for_tool
from .read_pty import read_pty as read_pty from .read_pty import read_pty as read_pty
from .run_command import run_command as run_command from .run_argv import run_argv as run_argv
from .run_command_batch import run_command_batch as run_command_batch from .run_argv_batch import run_argv_batch as run_argv_batch
from .write_pty import write_pty as write_pty from .write_pty import write_pty as write_pty
from .write_pty_keys import write_pty_keys as write_pty_keys from .write_pty_keys import write_pty_keys as write_pty_keys
PROCESS_TOOLS = [ PROCESS_TOOLS = [
run_command, run_argv,
run_command_batch, run_argv_batch,
open_pty, open_pty,
read_pty, read_pty,
write_pty, write_pty,
+13 -3
View File
@@ -22,9 +22,19 @@ DEFAULT_MAX_BATCH_STEPS = 20
def truncate_output(text: str, label: str, max_chars: int = DEFAULT_MAX_OUTPUT_CHARS) -> str: def truncate_output(text: str, label: str, max_chars: int = DEFAULT_MAX_OUTPUT_CHARS) -> str:
if len(text) <= max_chars: """Cap a per-stream output block; embed a truncate token when cut.
``label`` (stdout/stderr/batch) is kept for call-site clarity but the
marker itself is the uniform ``[Truncated (N chars max.; M omitted).
truncate_token=...]`` (no label), matching every other truncation site.
Returns ``text`` unchanged when it fits.
"""
del label
if max_chars < 1 or len(text) <= max_chars:
return text return text
return text[:max_chars] + f"\n...[{label} truncated]" from plyngent.tools.truncate_token import truncate_generic
return truncate_generic(text, max_chars)
def format_command_result( def format_command_result(
@@ -83,7 +93,7 @@ async def execute_argv(
timeout_seconds: float = DEFAULT_TIMEOUT_SECONDS, timeout_seconds: float = DEFAULT_TIMEOUT_SECONDS,
mix_stderr: bool = False, mix_stderr: bool = False,
) -> CommandStepResult: ) -> CommandStepResult:
"""Run one argv command; no shell. Used by run_command and run_command_batch.""" """Run one argv command; no shell. Used by run_argv and run_argv_batch."""
if not command: if not command:
msg = "command argv must not be empty" msg = "command argv must not be empty"
raise WorkspaceError(msg) raise WorkspaceError(msg)
@@ -1,7 +1,7 @@
from __future__ import annotations from __future__ import annotations
from plyngent.agent import ToolTag, tool from plyngent.agent import ToolTag, tool
from plyngent.tools.workspace import WorkspaceError from plyngent.tools.workspace import WorkspaceError, argv_shape_error
from .command_exec import ( from .command_exec import (
DEFAULT_TIMEOUT_SECONDS, DEFAULT_TIMEOUT_SECONDS,
@@ -11,18 +11,21 @@ from .command_exec import (
@tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE | ToolTag.YOLO) @tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE | ToolTag.YOLO)
async def run_command( async def run_argv(
command: list[str], argv: list[str],
*, *,
cwd: str = ".", cwd: str = ".",
timeout_seconds: float = DEFAULT_TIMEOUT_SECONDS, timeout_seconds: float = DEFAULT_TIMEOUT_SECONDS,
stdin: str | None = None, stdin: str | None = None,
env: dict[str, str] | None = None, env: dict[str, str] | None = None,
) -> str: ) -> str:
"""Run ``command`` (argv, no shell) under the workspace; capture stdout/stderr.""" """Run ``argv`` (argv, no shell) under the workspace; capture stdout/stderr."""
shape_error = argv_shape_error(argv)
if shape_error is not None:
return f"error: `run_argv` {shape_error}"
try: try:
result = await execute_argv( result = await execute_argv(
command, argv,
cwd=cwd, cwd=cwd,
env=env, env=env,
stdin=stdin, stdin=stdin,
@@ -50,23 +50,23 @@ def _as_str_dict(value: object) -> dict[str, str] | None:
def _parse_step(raw: object, index: int) -> dict[str, Any]: def _parse_step(raw: object, index: int) -> dict[str, Any]:
if not isinstance(raw, dict): if not isinstance(raw, dict):
msg = f"commands[{index}] must be an object" msg = f"steps[{index}] must be an object"
raise WorkspaceError(msg) raise WorkspaceError(msg)
data = cast("dict[str, object]", raw) data = cast("dict[str, object]", raw)
command = data.get("command") argv = data.get("argv")
if not isinstance(command, list) or not command: if not isinstance(argv, list) or not argv:
msg = f"commands[{index}].command must be a non-empty argv list" msg = f"steps[{index}].argv must be a non-empty argv list"
raise WorkspaceError(msg) raise WorkspaceError(msg)
command_parts = cast("list[object]", command) argv_parts = cast("list[object]", argv)
argv = [part for part in command_parts if isinstance(part, str)] argv_str = [part for part in argv_parts if isinstance(part, str)]
if len(argv) != len(command_parts): if len(argv_str) != len(argv_parts):
msg = f"commands[{index}].command must be a list of strings" msg = f"steps[{index}].argv must be a list of strings"
raise WorkspaceError(msg) raise WorkspaceError(msg)
cwd = data.get("cwd") cwd = data.get("cwd")
stdin = data.get("stdin") stdin = data.get("stdin")
stop = data.get("stop_on_error", None) stop = data.get("stop_on_error", None)
return { return {
"command": argv, "argv": argv_str,
"cwd": str(cwd) if isinstance(cwd, str) and cwd else None, "cwd": str(cwd) if isinstance(cwd, str) and cwd else None,
"env": _as_str_dict(data.get("env")), "env": _as_str_dict(data.get("env")),
"stdin": None if stdin is None else str(stdin), "stdin": None if stdin is None else str(stdin),
@@ -77,18 +77,18 @@ def _parse_step(raw: object, index: int) -> dict[str, Any]:
} }
def _normalize_commands(commands: list[dict[str, object]] | str) -> list[object]: def _normalize_steps(steps: list[dict[str, object]] | str) -> list[object]:
if isinstance(commands, str): if isinstance(steps, str):
try: try:
loaded: object = json.loads(commands) loaded: object = json.loads(steps)
except json.JSONDecodeError as exc: except json.JSONDecodeError as exc:
msg = f"commands must be a JSON array: {exc}" msg = f"steps must be a JSON array: {exc}"
raise WorkspaceError(msg) from exc raise WorkspaceError(msg) from exc
if not isinstance(loaded, list): if not isinstance(loaded, list):
msg = "commands must be a JSON array of step objects" msg = "steps must be a JSON array of step objects"
raise WorkspaceError(msg) raise WorkspaceError(msg)
return cast("list[object]", loaded) return cast("list[object]", loaded)
return cast("list[object]", commands) return cast("list[object]", steps)
def _format_step(index: int, result: CommandStepResult, *, pipe_out: bool) -> str: def _format_step(index: int, result: CommandStepResult, *, pipe_out: bool) -> str:
@@ -131,7 +131,7 @@ async def _run_batch_steps(
for index, step in enumerate(steps): for index, step in enumerate(steps):
stdin_text = prev_capture if prev_piped else step["stdin"] stdin_text = prev_capture if prev_piped else step["stdin"]
result = await execute_argv( result = await execute_argv(
step["command"], step["argv"],
cwd=step["cwd"] if step["cwd"] is not None else cwd, cwd=step["cwd"] if step["cwd"] is not None else cwd,
env=_merge_env(env, step["env"]), env=_merge_env(env, step["env"]),
stdin=stdin_text, stdin=stdin_text,
@@ -155,8 +155,8 @@ async def _run_batch_steps(
@tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE | ToolTag.YOLO) @tool(tags=ToolTag.LOCAL | ToolTag.INSTANCE_STATE | ToolTag.YOLO)
async def run_command_batch( async def run_argv_batch(
commands: list[dict[str, object]] | str, steps: list[dict[str, object]] | str,
*, *,
cwd: str = ".", cwd: str = ".",
env: dict[str, str] | None = None, env: dict[str, str] | None = None,
@@ -164,10 +164,10 @@ async def run_command_batch(
) -> str: ) -> str:
"""Run a serial batch of argv commands (no shell). """Run a serial batch of argv commands (no shell).
Each element of ``commands`` is an object:: Each element of ``steps`` is an object::
{ {
"command": ["git", "status"], # required argv "argv": ["git", "status"], # required argv
"cwd": ".", # optional, else batch cwd "cwd": ".", # optional, else batch cwd
"env": {"FOO": "1"}, # optional overlay "env": {"FOO": "1"}, # optional overlay
"stdin": "...", # optional; unused if previous pipe_out "stdin": "...", # optional; unused if previous pipe_out
@@ -181,15 +181,15 @@ async def run_command_batch(
Default ``stop_on_error`` is true. Max 20 steps; total output budget 32k chars. Default ``stop_on_error`` is true. Max 20 steps; total output budget 32k chars.
""" """
try: try:
raw_steps = _normalize_commands(commands) raw_steps = _normalize_steps(steps)
if not raw_steps: if not raw_steps:
return "error: commands must not be empty" return "error: steps must not be empty"
if len(raw_steps) > DEFAULT_MAX_BATCH_STEPS: if len(raw_steps) > DEFAULT_MAX_BATCH_STEPS:
return f"error: at most {DEFAULT_MAX_BATCH_STEPS} commands per batch" return f"error: at most {DEFAULT_MAX_BATCH_STEPS} steps per batch"
steps = [_parse_step(item, i) for i, item in enumerate(raw_steps)] steps = [_parse_step(item, i) for i, item in enumerate(raw_steps)]
if steps[-1]["pipe_out"]: if steps[-1]["pipe_out"]:
return "error: last command has pipe_out=true but there is no next step" return "error: last step has pipe_out=true but there is no next step"
results, stopped_early = await _run_batch_steps(steps, cwd=cwd, env=env, stop_on_error=stop_on_error) results, stopped_early = await _run_batch_steps(steps, cwd=cwd, env=env, stop_on_error=stop_on_error)
body = "\n".join( body = "\n".join(
+154
View File
@@ -0,0 +1,154 @@
"""Truncate-token cursors for resuming truncated tool results.
Every truncation site (the agent loop's generic tool-result cap,
``run_command`` / ``run_command_batch`` per-stream caps, ``read_file``,
``fetch``, and request-time compact shrinks) appends the same compact marker
when output is cut short::
[Truncated (32000 chars max.; 500 omitted). truncate_token=...]
The model passes the token to ``get_truncated`` to read the next chunk without
re-requesting the whole source or raising limits. Tokens chain: each truncated
chunk carries a fresh token, so ``get_truncated`` keeps resuming until the
source is exhausted (then no marker is emitted).
Resumable sources (``file`` / ``http``) are re-read by ``location`` +
``offset``. Arbitrary tool output (run_command stdout, todo renders, …) has no
resumable source, so its remainder lives in a short-lived in-memory store
(``kind="memory"``) that is forgotten when the agent process exits — the model
must re-run the tool after that.
"""
from __future__ import annotations
import base64
import uuid
from typing import Literal
import msgspec
type TruncateKind = Literal["file", "http", "memory"]
class TruncateToken(msgspec.Struct, frozen=True):
"""Opaque cursor into a truncated source.
``offset``/``limit`` are in characters (http bodies, memory remainders) or
lines (files is line-based via ``read_file``); ``get_truncated`` interprets
by ``kind``. For ``memory``, ``location`` is a key into the in-memory
remainder store and ``offset``/``limit`` index characters of that stored
remainder.
"""
kind: TruncateKind
location: str
offset: int
limit: int
# In-memory remainders for generic (non-resumable) tool output. Process-lifetime
# only: forgotten when the agent exits. Single-user CLI → one process per agent.
_MEMORY_REMAINDERS: dict[str, str] = {}
def store_remainder(text: str) -> str:
"""Store truncated-away output; return its opaque store key."""
key = f"mem-{uuid.uuid4().hex[:12]}"
_MEMORY_REMAINDERS[key] = text
return key
def get_remainder(key: str) -> str | None:
"""Fetch a stored remainder by key; None when expired/unknown."""
return _MEMORY_REMAINDERS.get(key)
def encode_truncate_token(token: TruncateToken) -> str:
"""Base64url-encode a token so it is safe inside tool-result text."""
payload = msgspec.json.encode(token)
return base64.urlsafe_b64encode(payload).decode("ascii")
def decode_truncate_token(text: str) -> TruncateToken | None:
"""Decode a token; None when malformed (model garbage or tampering)."""
try:
raw = base64.urlsafe_b64decode(text.encode("ascii"))
token = msgspec.json.decode(raw, type=TruncateToken)
except ValueError, msgspec.DecodeError:
return None
if token.offset < 0 or token.limit < 1:
return None
return token
def truncation_marker(max_chars: int, omitted: int, token: TruncateToken) -> str:
"""The single truncation hint: ``[Truncated (N chars max.; M omitted). truncate_token=...]``."""
return f"\n[Truncated ({max_chars} chars max.; {omitted} omitted). truncate_token={encode_truncate_token(token)}]"
def truncate_with_token(
text: str,
max_chars: int,
*,
kind: TruncateKind,
location: str,
offset: int,
limit: int,
total_len: int,
emit_more_after: bool = True,
) -> tuple[str, TruncateToken | None]:
"""Bound ``text`` (a slice of the full source starting at ``offset``) to ``max_chars``.
Returns ``(bounded_text, next_token)``. A token is embedded whenever the
returned chunk is cut (overflow) or, with ``emit_more_after`` (default),
whenever more source content remains after the chunk — so ``get_truncated``
chains through truncations. The token advances by the length actually
returned (no content is skipped), and the marker is budgeted so the total
returned length stays near ``max_chars`` (the agent loop's own cap never
re-cuts it and never strips the token). ``omitted`` in the marker counts
source characters not yet delivered. When nothing was cut and
(``emit_more_after=False`` or the source ends), ``text`` is returned
unchanged with no marker.
"""
if max_chars < 1:
return text, None
if len(text) > max_chars:
# First pass sizes the marker with an offset+max_chars token; re-encode
# with the real cut so the cursor lands exactly where content stops.
first = TruncateToken(kind=kind, location=location, offset=offset + max_chars, limit=limit)
omitted = total_len - (offset + max_chars)
cut = max_chars - len(truncation_marker(max_chars, omitted, first))
if cut < 1:
return text[:max_chars] + truncation_marker(max_chars, omitted, first), first
token = TruncateToken(kind=kind, location=location, offset=offset + cut, limit=limit)
return text[:cut] + truncation_marker(max_chars, omitted, token), token
if not emit_more_after:
return text, None
more_after = offset + len(text) < total_len
if not more_after:
return text, None
omitted = total_len - (offset + len(text))
token = TruncateToken(kind=kind, location=location, offset=offset + len(text), limit=limit)
return text + truncation_marker(max_chars, omitted, token), token
def truncate_generic(text: str, max_chars: int) -> str:
"""Cap arbitrary tool output; embed a memory truncate token (chainable).
The full remainder is kept in the in-memory store so ``get_truncated`` can
resume it. Returns ``text`` unchanged (no marker) when it fits within
``max_chars``.
"""
if max_chars < 1 or len(text) <= max_chars:
return text
key = store_remainder(text)
bounded, _ = truncate_with_token(
text,
max_chars,
kind="memory",
location=key,
offset=0,
limit=max_chars,
total_len=len(text),
)
return bounded
+22 -1
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING from typing import TYPE_CHECKING, cast
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable, Sequence from collections.abc import Callable, Sequence
@@ -268,6 +268,23 @@ def resolve_path(path: str | Path) -> Path:
return resolved return resolved
def argv_shape_error(argv: object) -> str | None:
"""Return a model-facing error when ``argv`` is not a non-empty argv list.
Models occasionally pass a shell-style string or a JSON-encoded array as a
string; duck-typing would otherwise let it reach ``create_subprocess_exec``
and splat into single characters. Declared as ``object`` so the isinstance
checks are not flagged as unnecessary.
"""
if isinstance(argv, str):
return 'only accepts an array of args, not a string; pass e.g. ["ls", "-la"]'
if not isinstance(argv, list):
return f"only accepts an array of args, not a {type(argv).__name__}"
if not all(isinstance(part, str) for part in cast("list[object]", argv)):
return "only accepts an array of string args"
return None
def check_command_allowed(argv: list[str]) -> None: def check_command_allowed(argv: list[str]) -> None:
"""Raise if argv is empty or the executable basename is denylisted. """Raise if argv is empty or the executable basename is denylisted.
@@ -275,6 +292,10 @@ def check_command_allowed(argv: list[str]) -> None:
installed: the human is asked (with a timeout; default deny). Session grants installed: the human is asked (with a timeout; default deny). Session grants
skip re-prompting for the same basename. Independent of YOLO soft-confirm. skip re-prompting for the same basename. Independent of YOLO soft-confirm.
""" """
shape_error = argv_shape_error(argv)
if shape_error is not None:
msg = f"command {shape_error}"
raise WorkspaceError(msg)
if not argv: if not argv:
msg = "command argv must not be empty" msg = "command argv must not be empty"
raise WorkspaceError(msg) raise WorkspaceError(msg)
+6 -8
View File
@@ -11,19 +11,17 @@ def test_truncate_tool_result_short() -> None:
def test_truncate_tool_result_long() -> None: def test_truncate_tool_result_long() -> None:
text = "a" * 50 text = "a" * 50
out = truncate_tool_result(text, 20) out = truncate_tool_result(text, 20)
assert out.startswith("a" * 20)
assert "Truncated" in out assert "Truncated" in out
assert "20" in out assert "20" in out
assert "30" in out assert "30" in out
assert "configuration file" in out # Generic tool results now embed a memory token too (resumable via get_truncated).
assert "truncate_token=" in out
assert out.startswith(("a" * 10, "a" * 20)) # marker is budgeted
def test_truncate_tool_result_no_hint() -> None: def test_truncate_tool_result_fits_no_marker() -> None:
text = "a" * 50 assert truncate_tool_result("tiny", 100) == "tiny"
out = truncate_tool_result(text, 20, hint=False) assert "Truncated" not in truncate_tool_result("tiny", 100)
assert out.startswith("a" * 20)
assert "truncated" in out
assert "configuration file" not in out
def test_estimate_message_chars() -> None: def test_estimate_message_chars() -> None:
+10 -9
View File
@@ -40,19 +40,18 @@ def test_truncate_tool_result_short() -> None:
def test_truncate_tool_result_long() -> None: def test_truncate_tool_result_long() -> None:
text = "a" * 50 text = "a" * 50
out = truncate_tool_result(text, 20) out = truncate_tool_result(text, 20)
assert out.startswith("a" * 20)
assert "Truncated" in out assert "Truncated" in out
assert "20" in out assert "20" in out
assert "30" in out assert "30" in out
assert "configuration file" in out # Generic tool results now embed a memory token too (resumable via get_truncated).
assert "truncate_token=" in out
def test_truncate_tool_result_no_hint() -> None: def test_truncate_tool_result_no_hint() -> None:
text = "a" * 50 text = "a" * 50
out = truncate_tool_result(text, 20, hint=False) out = truncate_tool_result(text, 20)
assert out.startswith("a" * 20) assert "Truncated" in out
assert "truncated" in out assert "truncate_token=" in out
assert "configuration file" not in out
def test_compact_shrinks_old_tool_results() -> None: def test_compact_shrinks_old_tool_results() -> None:
@@ -92,7 +91,9 @@ def test_compact_shrinks_old_tool_results() -> None:
) )
assert isinstance(compacted[2], ToolChatMessage) assert isinstance(compacted[2], ToolChatMessage)
assert len(compacted[2].content) < original_len assert len(compacted[2].content) < original_len
assert "truncated" in compacted[2].content assert "Truncated" in compacted[2].content
# Compact shrinks carry a token too (resumable via get_truncated).
assert "truncate_token=" in compacted[2].content
# Full history unchanged # Full history unchanged
assert isinstance(messages[2], ToolChatMessage) assert isinstance(messages[2], ToolChatMessage)
assert len(messages[2].content) == original_len assert len(messages[2].content) == original_len
@@ -132,7 +133,7 @@ def test_compact_uses_api_calibration() -> None:
# Raw est under budget → no compact # Raw est under budget → no compact
no_api = compact_messages_for_request(messages, max_tokens=est + 100) no_api = compact_messages_for_request(messages, max_tokens=est + 100)
assert isinstance(no_api[1], ToolChatMessage) assert isinstance(no_api[1], ToolChatMessage)
assert "truncated" not in no_api[1].content assert "Truncated" not in no_api[1].content
# Calibrate so measured size is 5x → over a mid budget → shrink old tool # Calibrate so measured size is 5x → over a mid budget → shrink old tool
compacted = compact_messages_for_request( compacted = compact_messages_for_request(
messages, messages,
@@ -145,7 +146,7 @@ def test_compact_uses_api_calibration() -> None:
assert isinstance(compacted[1], ToolChatMessage) assert isinstance(compacted[1], ToolChatMessage)
old = messages[1] old = messages[1]
assert isinstance(old, ToolChatMessage) assert isinstance(old, ToolChatMessage)
assert "truncated" in compacted[1].content or len(compacted[1].content) < len(old.content) assert "Truncated" in compacted[1].content or len(compacted[1].content) < len(old.content)
def _response(message: AssistantChatMessage) -> ChatCompletionResponse: def _response(message: AssistantChatMessage) -> ChatCompletionResponse:
+53
View File
@@ -1,8 +1,13 @@
from __future__ import annotations
from typing import Any
import msgspec import msgspec
from plyngent.agent.responses_bridge import ( from plyngent.agent.responses_bridge import (
chat_messages_to_responses_input, chat_messages_to_responses_input,
chat_param_to_responses_kwargs, chat_param_to_responses_kwargs,
reasoning_summary_text,
response_to_assistant_message, response_to_assistant_message,
response_to_chat_completion, response_to_chat_completion,
responses_status_to_finish_reason, responses_status_to_finish_reason,
@@ -145,6 +150,54 @@ def test_response_to_assistant_with_tools() -> None:
assert completion.choices[0].finish_reason == "tool_calls" assert completion.choices[0].finish_reason == "tool_calls"
def _reasoning_response(
*,
output_reasoning: bool = False,
top_level: dict[str, Any] | None = None,
) -> Response:
output: list[dict[str, Any]] = []
if output_reasoning:
output.append(
{
"id": "reason_1",
"type": "reasoning",
"summary": [{"type": "summary_text", "text": "openai-style reasoning"}],
}
)
return Response(
id="resp_r",
created_at=1,
model="deepseek-v4-flash",
output=output,
reasoning=top_level,
)
def test_reasoning_summary_text_deepseek_top_level() -> None:
"""DeepSeek Responses returns reasoning in the top-level ``reasoning`` field."""
response = _reasoning_response(top_level={"content": [{"type": "reasoning_content", "text": "think step by step"}]})
assert reasoning_summary_text(response) == "think step by step"
assistant = response_to_assistant_message(response)
assert assistant.reasoning_content == "think step by step"
def test_reasoning_summary_text_prefers_top_level() -> None:
response = _reasoning_response(
output_reasoning=True,
top_level={"summary": [{"type": "summary_text", "text": "top-level reasoning"}]},
)
assert reasoning_summary_text(response) == "top-level reasoning"
def test_reasoning_summary_text_falls_back_to_output_items() -> None:
response = _reasoning_response(output_reasoning=True)
assert reasoning_summary_text(response) == "openai-style reasoning"
def test_reasoning_summary_text_empty_when_absent() -> None:
assert reasoning_summary_text(_reasoning_response()) == ""
def test_responses_status_to_finish_reason_incomplete() -> None: def test_responses_status_to_finish_reason_incomplete() -> None:
body = { body = {
"id": "resp_i", "id": "resp_i",
+75 -1
View File
@@ -177,13 +177,87 @@ async def test_dispatch_responses_stream_text_and_usage() -> None:
) )
stream = await dispatch_responses(cast("Any", client), param, stream=True) stream = await dispatch_responses(cast("Any", client), param, stream=True)
chunks = [chunk async for chunk in cast("Any", stream)] chunks = [chunk async for chunk in cast("Any", stream)]
# text deltas + finish + usage
texts = [c.choices[0].delta.content for c in chunks if c.choices and isinstance(c.choices[0].delta.content, str)] texts = [c.choices[0].delta.content for c in chunks if c.choices and isinstance(c.choices[0].delta.content, str)]
assert texts == ["stream", "ed"] assert texts == ["stream", "ed"]
assert any(c.choices and c.choices[0].finish_reason == "stop" for c in chunks) assert any(c.choices and c.choices[0].finish_reason == "stop" for c in chunks)
assert any(token_usage_present(c) for c in chunks) assert any(token_usage_present(c) for c in chunks)
async def _streamed_reasoning(
events: list[ResponseStreamEvent],
) -> str:
"""Run one Responses stream and join streamed ``reasoning_content`` deltas.
Mirrors the agent loop: ``.done`` (``reasoning_full``) replaces accumulated
fragments instead of appending.
"""
import msgspec
final = _completed_response(text="ok")
final.reasoning = {"content": [{"type": "reasoning_content", "text": "terminal reasoning"}]}
events.append(ResponseStreamEvent(type="response.completed", response=msgspec.to_builtins(final)))
client = ScriptedResponsesClient(stream_events=[events])
param = ChatCompletionsParam(model="gpt-test", messages=[UserChatMessage(content="hi")])
stream = await dispatch_responses(cast("Any", client), param, stream=True)
chunks = [chunk async for chunk in cast("Any", stream)]
parts: list[str] = []
for chunk in chunks:
if not chunk.choices:
continue
delta = chunk.choices[0].delta
if isinstance(delta.reasoning_content, str) and delta.reasoning_content:
if delta.reasoning_full is True:
parts = [delta.reasoning_content]
else:
parts.append(delta.reasoning_content)
return "".join(parts)
async def test_dispatch_responses_stream_reasoning_text_done() -> None:
"""DeepSeek ``response.reasoning_text.done`` carries the full reasoning text."""
reasoning = await _streamed_reasoning(
[ResponseStreamEvent(type="response.reasoning_text.done", delta="full chain of thought")]
)
assert reasoning == "full chain of thought"
async def test_dispatch_responses_stream_reasoning_deltas() -> None:
reasoning = await _streamed_reasoning(
[
ResponseStreamEvent(type="response.reasoning_text.delta", delta="step one "),
ResponseStreamEvent(type="response.reasoning_text.delta", delta="step two"),
]
)
assert reasoning == "step one step two"
async def test_dispatch_responses_stream_reasoning_full_replaces_fragments() -> None:
"""A ``.done`` event with the full CoT must replace streamed fragments, not
append — otherwise the chain-of-thought is concatenated twice."""
reasoning = await _streamed_reasoning(
[
ResponseStreamEvent(type="response.reasoning_text.delta", delta="partial "),
ResponseStreamEvent(type="response.reasoning_text.delta", delta="thinking"),
ResponseStreamEvent(type="response.reasoning_text.done", delta="full chain of thought"),
]
)
assert reasoning == "full chain of thought"
async def test_dispatch_responses_stream_terminal_only_reasoning() -> None:
"""DeepSeek sometimes delivers reasoning only on the terminal response."""
reasoning = await _streamed_reasoning([])
assert reasoning == "terminal reasoning"
async def test_dispatch_responses_stream_reasoning_no_duplicate() -> None:
"""Streamed deltas win; the terminal reasoning is not re-emitted."""
reasoning = await _streamed_reasoning(
[ResponseStreamEvent(type="response.reasoning_text.delta", delta="streamed only")]
)
assert reasoning == "streamed only"
def token_usage_present(chunk: object) -> bool: def token_usage_present(chunk: object) -> bool:
usage = getattr(chunk, "usage", UNSET) usage = getattr(chunk, "usage", UNSET)
return usage is not UNSET and usage is not None return usage is not UNSET and usage is not None
+74 -3
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from plyngent.agent import ReasoningDeltaEvent, TextDeltaEvent, ToolResultEvent from plyngent.agent import ReasoningDeltaEvent, TextDeltaEvent, ToolCallEvent, ToolResultEvent
from plyngent.cli.display import ( from plyngent.cli.display import (
get_markdown_enabled, get_markdown_enabled,
markdown_render_available, markdown_render_available,
@@ -11,7 +11,11 @@ from plyngent.cli.display import (
set_markdown_enabled, set_markdown_enabled,
set_verbose_tool_results, set_verbose_tool_results,
) )
from plyngent.lmproto.openai_compatible.model import ToolChatMessage from plyngent.lmproto.openai_compatible.model import (
AssistantFunctionTool,
AssistantFunctionToolCall,
ToolChatMessage,
)
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import AsyncIterator from collections.abc import AsyncIterator
@@ -61,7 +65,7 @@ async def test_flush_markdown_on_source_change(
call = AssistantFunctionToolCall( call = AssistantFunctionToolCall(
id="1", id="1",
function=AssistantFunctionTool(name="read_file", arguments="{}"), function=AssistantFunctionTool(name="run_argv", arguments='{"argv": ["echo", "x"]}'),
) )
await render_events( await render_events(
_aiter( _aiter(
@@ -78,6 +82,73 @@ async def test_flush_markdown_on_source_change(
assert "after" in out assert "after" in out
def _read_call(args_json: str) -> ToolCallEvent:
return ToolCallEvent(
tool_call=AssistantFunctionToolCall(
id="1",
function=AssistantFunctionTool(name="read_file", arguments=args_json),
)
)
def _result(content: str) -> ToolResultEvent:
return ToolResultEvent(message=ToolChatMessage(content=content, tool_call_id="1"))
async def test_pretty_read_file_done(capsys: pytest.CaptureFixture[str]) -> None:
await render_events(_aiter([_read_call('{"path": "a.txt"}'), _result("L1-4\none\ntwo\n")]))
out = capsys.readouterr().out
assert "* Read 'a.txt' L1-4 (done)" in out
assert "[tool]" not in out
assert "[tool ok]" not in out
async def test_pretty_read_file_statuses(capsys: pytest.CaptureFixture[str]) -> None:
cases = [
("error: file not found: a.txt", "(file not found)"),
("error: not a file: a.txt", "(not a file)"),
("error: something else", "(error)"),
("no header content", "(done)"),
]
for content, status in cases:
await render_events(_aiter([_read_call('{"path": "a.txt"}'), _result(content)]))
out = capsys.readouterr().out
assert f"* Read 'a.txt' {status}" in out, f"status {status} not rendered for {content!r}"
async def test_pretty_todo_push(capsys: pytest.CaptureFixture[str]) -> None:
call = ToolCallEvent(
tool_call=AssistantFunctionToolCall(
id="2",
function=AssistantFunctionTool(name="todo_push", arguments='{"titles": ["T1"]}'),
)
)
result = (
"pushed group (depth=1) items=[a1]\n"
"(LIFO of groups: depth=1; TOP group = current breakdown level)\n"
"group d=0 TOP:\n"
" [ ] a1: T1"
)
await render_events(_aiter([call, _result(result)]))
out = capsys.readouterr().out
assert "* Todo Push:" in out
assert " [ ] a1: T1" in out
assert "[tool]" not in out
async def test_non_pretty_tool_keeps_old_style(capsys: pytest.CaptureFixture[str]) -> None:
call = ToolCallEvent(
tool_call=AssistantFunctionToolCall(
id="3",
function=AssistantFunctionTool(name="run_argv", arguments='{"argv": ["ls"]}'),
)
)
await render_events(_aiter([call, _result("exit_code=0")]))
out = capsys.readouterr().out
assert "[tool] run_argv(" in out
assert "[tool ok]" in out
async def test_tool_result_preview_vs_verbose(capsys: pytest.CaptureFixture[str]) -> None: async def test_tool_result_preview_vs_verbose(capsys: pytest.CaptureFixture[str]) -> None:
long = "x" * 200 long = "x" * 200
msg = ToolChatMessage(content=long, tool_call_id="1") msg = ToolChatMessage(content=long, tool_call_id="1")
+70
View File
@@ -8,7 +8,9 @@ import pytest
from plyngent.cli.interrupt import ( from plyngent.cli.interrupt import (
allow_task_cancel, allow_task_cancel,
install_keyboard_interrupt_sigint,
pause_task_cancel_for_prompt, pause_task_cancel_for_prompt,
raise_keyboard_interrupt_sigint,
run_in_prompt_thread, run_in_prompt_thread,
set_sigint_reinstall, set_sigint_reinstall,
) )
@@ -113,3 +115,71 @@ async def test_sigint_cancels_after_prompt_pause() -> None:
assert cancelled.is_set() assert cancelled.is_set()
# Cleanup any leftover reinstall hook from run_cancellable # Cleanup any leftover reinstall hook from run_cancellable
set_sigint_reinstall(None) set_sigint_reinstall(None)
def test_install_keyboard_interrupt_sigint() -> None:
"""Installs a non-default SIGINT handler that raises KeyboardInterrupt.
The handler must be a distinct function object: ``asyncio.run`` only skips
installing its own silent-cancel handler when the current handler is not
``signal.default_int_handler``.
"""
install_keyboard_interrupt_sigint()
try:
handler = signal.getsignal(signal.SIGINT)
assert handler is not signal.default_int_handler
assert handler is not signal.SIG_DFL
assert callable(handler)
with pytest.raises(KeyboardInterrupt):
handler(signal.SIGINT, None) # type: ignore[call-arg]
finally:
signal.signal(signal.SIGINT, signal.default_int_handler)
def test_asyncio_run_keeps_keyboard_interrupt_sigint() -> None:
"""Regression: Runner must not replace our handler with its own.
Previously the first Ctrl+C under ``asyncio.run`` silently cancelled the
main task (invisible at the REPL prompt) and the second raised
KeyboardInterrupt, which also cancelled ``memory.close()`` on a Ctrl+D exit.
With a non-default handler installed first, Runner leaves SIGINT alone.
"""
import asyncio
install_keyboard_interrupt_sigint()
try:
async def quick() -> int:
return 7
assert asyncio.run(quick()) == 7
assert signal.getsignal(signal.SIGINT) is raise_keyboard_interrupt_sigint
finally:
signal.signal(signal.SIGINT, signal.default_int_handler)
async def test_run_cancellable_restores_previous_sigint() -> None:
"""Regression: after a turn SIGINT is restored, not left as SIG_DFL.
``loop.remove_signal_handler`` leaves SIG_DFL, which would terminate the
process on a stray Ctrl+C between turns instead of letting the REPL catch
a KeyboardInterrupt.
"""
loop = asyncio.get_running_loop()
try:
loop.add_signal_handler(signal.SIGINT, lambda: None)
loop.remove_signal_handler(signal.SIGINT)
except NotImplementedError, RuntimeError, ValueError:
pytest.skip("asyncio signal handlers not available on this platform")
previous = signal.getsignal(signal.SIGINT)
async def quick() -> None:
return None
try:
await run_cancellable(quick())
assert signal.getsignal(signal.SIGINT) is previous
assert signal.getsignal(signal.SIGINT) is not signal.SIG_DFL
finally:
signal.signal(signal.SIGINT, signal.default_int_handler)
+3 -3
View File
@@ -30,10 +30,10 @@ def test_prompt_confirm_tool_default_deny() -> None:
def test_format_tool_confirm_box_multiline() -> None: def test_format_tool_confirm_box_multiline() -> None:
reason = "run_command: python3 -c (review code before allow)\nargv:\n python3 -c print(1)\n-c code:\n print(1)" reason = "run_argv: python3 -c (review code before allow)\nargv:\n python3 -c print(1)\n-c code:\n print(1)"
box = format_tool_confirm_box("run_command", reason) box = format_tool_confirm_box("run_argv", reason)
assert "┌" in box and "└" in box assert "┌" in box and "└" in box
assert "confirm · tool 'run_command'" in box assert "confirm · tool 'run_argv'" in box
assert "python3 -c" in box assert "python3 -c" in box
assert "print(1)" in box assert "print(1)" in box
# Distinct lines, not one jammed row # Distinct lines, not one jammed row
+40
View File
@@ -593,3 +593,43 @@ async def test_repl_ctrl_c_from_turn_returns_to_prompt(
await run_repl(state) # must return normally (EOF), never raise KeyboardInterrupt await run_repl(state) # must return normally (EOF), never raise KeyboardInterrupt
assert calls["reads"] == 2 assert calls["reads"] == 2
async def test_repl_second_ctrl_c_after_cancel_returns_to_prompt(
state: ReplState,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""A second Ctrl+C landing outside the turn call (e.g. while echoing the
next user input, right after a cancel) must return to the prompt, not exit."""
from plyngent.cli import repl as repl_mod
from plyngent.cli.repl import run_repl
reads = ["first", "second", "third"]
calls: dict[str, int] = {"reads": 0, "echoes": 0}
def fake_read_entry() -> str:
calls["reads"] += 1
if calls["reads"] > len(reads):
raise EOFError
return reads[calls["reads"] - 1]
async def cancel_turn(_agent: object, _text: str) -> bool:
raise KeyboardInterrupt # first Ctrl+C: cancels every turn
def echo_user(text: str) -> None:
calls["echoes"] += 1
if calls["echoes"] == 2:
# second Ctrl+C lands while echoing user text — outside the turn
# call, previously unguarded (used to exit the whole REPL).
raise KeyboardInterrupt
monkeypatch.setattr(repl_mod, "setup_readline", lambda _state: None)
monkeypatch.setattr(repl_mod, "read_repl_entry", fake_read_entry)
monkeypatch.setattr(repl_mod, "run_user_text_with_retries", cancel_turn)
monkeypatch.setattr(repl_mod, "_echo_user", echo_user)
await run_repl(state) # must return normally (EOF), never raise KeyboardInterrupt
assert calls["reads"] == 4 # three entries + EOF
assert calls["echoes"] == 3 # the interrupting echo still counted, loop survived
assert "interrupted" in capsys.readouterr().out
+36
View File
@@ -1,5 +1,7 @@
from __future__ import annotations from __future__ import annotations
import io
import pytest import pytest
from plyngent.prompting import ( from plyngent.prompting import (
@@ -7,14 +9,48 @@ from plyngent.prompting import (
FormField, FormField,
NonInteractiveBackend, NonInteractiveBackend,
NonInteractiveError, NonInteractiveError,
_read_line_with_timeout_posix,
ask, ask,
choose, choose,
confirm, confirm,
form, form,
read_line_with_timeout,
temporary_backend, temporary_backend,
) )
def _ready_poll(r, w, x, t):
return (r, w, x)
def _never_poll(r, w, x, t):
return ([], [], [])
def test_read_line_with_timeout_reads_line() -> None:
stream = io.StringIO("hello\n")
assert _read_line_with_timeout_posix("p> ", 5.0, stream=stream, poll=_ready_poll) == "hello"
def test_read_line_with_timeout_empty_enter() -> None:
stream = io.StringIO("\n")
assert _read_line_with_timeout_posix("p> ", 5.0, stream=stream, poll=_ready_poll) == ""
def test_read_line_with_timeout_timeout_returns_none() -> None:
stream = io.StringIO()
assert _read_line_with_timeout_posix("p> ", 0.01, stream=stream, poll=_never_poll) is None
def test_read_line_with_timeout_eof_returns_none() -> None:
stream = io.StringIO("")
assert _read_line_with_timeout_posix("p> ", 5.0, stream=stream, poll=_ready_poll) is None
def test_read_line_with_timeout_negative_timeout() -> None:
assert read_line_with_timeout("p> ", -1) is None
class ScriptedBackend: class ScriptedBackend:
"""Deterministic backend for unit tests.""" """Deterministic backend for unit tests."""
+61 -2
View File
@@ -1,12 +1,15 @@
from __future__ import annotations from __future__ import annotations
import importlib
import json import json
from plyngent.agent import ToolRegistry from plyngent.agent import ToolRegistry
from plyngent.prompting import NonInteractiveBackend, temporary_backend from plyngent.prompting import NonInteractiveBackend, temporary_backend
from plyngent.tools.chat import CHAT_TOOLS, ask_user, choose_user, form_user from plyngent.tools.chat import CHAT_TOOLS, ask_user, choose_user, form_user, wait
from tests.test_prompting import ScriptedBackend from tests.test_prompting import ScriptedBackend
wait_module = importlib.import_module("plyngent.tools.chat.wait")
async def test_ask_user_tool() -> None: async def test_ask_user_tool() -> None:
backend = ScriptedBackend(["42"]) backend = ScriptedBackend(["42"])
@@ -77,7 +80,7 @@ async def test_form_user_tool() -> None:
async def test_chat_tools_in_default_list() -> None: async def test_chat_tools_in_default_list() -> None:
names = {t.name for t in CHAT_TOOLS} names = {t.name for t in CHAT_TOOLS}
assert names == {"ask_user_line", "ask_user_choice", "ask_user_form"} assert names == {"ask_user_line", "ask_user_choice", "ask_user_form", "wait"}
async def test_ask_user_non_interactive_error() -> None: async def test_ask_user_non_interactive_error() -> None:
@@ -85,3 +88,59 @@ async def test_ask_user_non_interactive_error() -> None:
registry = ToolRegistry([ask_user]) registry = ToolRegistry([ask_user])
out = await registry.execute("ask_user_line", '{"question": "hi"}') out = await registry.execute("ask_user_line", '{"question": "hi"}')
assert out.startswith("error:") assert out.startswith("error:")
async def test_wait_tool_zero_seconds(monkeypatch) -> None:
with temporary_backend(NonInteractiveBackend()):
registry = ToolRegistry([wait])
out = await registry.execute("wait", '{"duration": 0}')
assert out == "waited 0s"
async def test_wait_tool_negative_duration() -> None:
with temporary_backend(NonInteractiveBackend()):
registry = ToolRegistry([wait])
out = await registry.execute("wait", '{"duration": -1}')
assert out.startswith("error:")
async def test_wait_tool_noninteractive_sleeps(monkeypatch) -> None:
slept: list[float] = []
async def fake_sleep(seconds: float) -> None:
slept.append(seconds)
monkeypatch.setattr(wait_module.asyncio, "sleep", fake_sleep)
with temporary_backend(NonInteractiveBackend()):
registry = ToolRegistry([wait])
out = await registry.execute("wait", '{"duration": 5}')
assert out == "waited 5s"
assert slept == [5]
async def test_wait_tool_disturbed_with_reason(monkeypatch) -> None:
monkeypatch.setattr(
wait_module,
"read_line_with_timeout",
lambda prompt, timeout: "tests are slow",
)
with temporary_backend(ScriptedBackend([])):
registry = ToolRegistry([wait])
out = await registry.execute("wait", '{"duration": 3}')
assert out == "disturbed by user: tests are slow"
async def test_wait_tool_disturbed_empty_enter(monkeypatch) -> None:
monkeypatch.setattr(wait_module, "read_line_with_timeout", lambda prompt, timeout: "")
with temporary_backend(ScriptedBackend([])):
registry = ToolRegistry([wait])
out = await registry.execute("wait", '{"duration": 3}')
assert out == "disturbed by user (no reason)"
async def test_wait_tool_times_out(monkeypatch) -> None:
monkeypatch.setattr(wait_module, "read_line_with_timeout", lambda prompt, timeout: None)
with temporary_backend(ScriptedBackend([])):
registry = ToolRegistry([wait])
out = await registry.execute("wait", '{"duration": 3}')
assert out == "waited 3s"
+15 -15
View File
@@ -43,46 +43,46 @@ def test_classify_write_overwrite_only(tmp_path: Path) -> None:
def test_classify_safe_tools() -> None: def test_classify_safe_tools() -> None:
assert classify_danger("read_file", {"path": "a"}) is None assert classify_danger("read_file", {"path": "a"}) is None
assert classify_danger("listdir", {"path": "."}) is None assert classify_danger("listdir", {"path": "."}) is None
assert classify_danger("run_command", {"command": ["echo", "hi"]}) is None assert classify_danger("run_argv", {"argv": ["echo", "hi"]}) is None
assert classify_danger("open_pty", {"command": ["true"]}) is None assert classify_danger("open_pty", {"command": ["true"]}) is None
assert classify_danger("run_command", {"command": ["ls", "-la"]}) is None assert classify_danger("run_argv", {"argv": ["ls", "-la"]}) is None
def test_classify_run_command_batch_risky() -> None: def test_classify_run_argv_batch_risky() -> None:
reason = classify_danger( reason = classify_danger(
"run_command_batch", "run_argv_batch",
{ {
"commands": [ "steps": [
{"command": ["echo", "ok"]}, {"argv": ["echo", "ok"]},
{"command": ["bash", "-c", "echo risky"]}, {"argv": ["bash", "-c", "echo risky"]},
] ]
}, },
) )
assert reason is not None assert reason is not None
assert "run_command_batch" in reason assert "run_argv_batch" in reason
assert "risky" in reason or "bash" in reason assert "risky" in reason or "bash" in reason
assert ( assert (
classify_danger( classify_danger(
"run_command_batch", "run_argv_batch",
{"commands": [{"command": ["echo", "ok"]}]}, {"steps": [{"argv": ["echo", "ok"]}]},
) )
is None is None
) )
def test_classify_shell_and_dash_c() -> None: def test_classify_shell_and_dash_c() -> None:
r = classify_danger("run_command", {"command": ["bash", "-c", "rm -rf /"]}) r = classify_danger("run_argv", {"argv": ["bash", "-c", "rm -rf /"]})
assert r is not None assert r is not None
assert "bash -c" in r assert "bash -c" in r
assert "rm -rf" in r assert "rm -rf" in r
r2 = classify_danger("run_command", {"command": ["python3", "-c", "print(1)"]}) r2 = classify_danger("run_argv", {"argv": ["python3", "-c", "print(1)"]})
assert r2 is not None assert r2 is not None
assert "python" in r2 assert "python" in r2
assert "-c" in r2 assert "-c" in r2
assert "print(1)" in r2 assert "print(1)" in r2
r_py = classify_danger("run_command", {"command": ["python", "-c", "import os"]}) r_py = classify_danger("run_argv", {"argv": ["python", "-c", "import os"]})
assert r_py is not None and "python -c" in r_py assert r_py is not None and "python -c" in r_py
r3 = classify_danger("open_pty", {"command": ["bash"]}) r3 = classify_danger("open_pty", {"command": ["bash"]})
@@ -118,8 +118,8 @@ async def test_confirm_deny_with_comment() -> None:
def test_shell_confirm_formats_command_placeholder() -> None: def test_shell_confirm_formats_command_placeholder() -> None:
script = chr(10).join(["line1", "line2", "line3"]) script = chr(10).join(["line1", "line2", "line3"])
reason = classify_danger( reason = classify_danger(
"run_command", "run_argv",
{"command": ["bash", "-c", script]}, {"argv": ["bash", "-c", script]},
) )
assert reason is not None assert reason is not None
assert "$(command)" in reason assert "$(command)" in reason
+4 -4
View File
@@ -19,7 +19,7 @@ def test_edit_lineno_replace_middle(workspace: object) -> None:
_ = _read_lineno("a.txt") _ = _read_lineno("a.txt")
out = call_sync(edit_lineno, "a.txt", 2, 3, "TWO\nTHREE\n") out = call_sync(edit_lineno, "a.txt", 2, 3, "TWO\nTHREE\n")
assert "replaced lines 2-3" in out assert "replaced lines 2-3" in out
assert call_sync(read_file, "a.txt") == "one\nTWO\nTHREE\nfour\n" assert call_sync(read_file, "a.txt") == "L1-4\none\nTWO\nTHREE\nfour\n"
def test_edit_lineno_requires_all_lines_read(workspace: object) -> None: def test_edit_lineno_requires_all_lines_read(workspace: object) -> None:
@@ -49,7 +49,7 @@ def test_edit_lineno_append_requires_last_line_read(workspace: object) -> None:
_ = call_sync(read_file, "ap.txt", offset=2, limit=1, with_lineno=True) _ = call_sync(read_file, "ap.txt", offset=2, limit=1, with_lineno=True)
out = call_sync(edit_lineno, "ap.txt", 4, 4, "d\n") out = call_sync(edit_lineno, "ap.txt", 4, 4, "d\n")
assert "appended" in out assert "appended" in out
assert call_sync(read_file, "ap.txt") == "a\nb\nc\nd\n" assert call_sync(read_file, "ap.txt") == "L1-4\na\nb\nc\nd\n"
def test_edit_lineno_delete_range(workspace: object) -> None: def test_edit_lineno_delete_range(workspace: object) -> None:
@@ -58,7 +58,7 @@ def test_edit_lineno_delete_range(workspace: object) -> None:
_ = _read_lineno("b.txt") _ = _read_lineno("b.txt")
out = call_sync(edit_lineno, "b.txt", 2, 2, "") out = call_sync(edit_lineno, "b.txt", 2, 2, "")
assert "replaced" in out assert "replaced" in out
assert call_sync(read_file, "b.txt") == "a\nc\n" assert call_sync(read_file, "b.txt") == "L1-2\na\nc\n"
def test_edit_lineno_append(workspace: object) -> None: def test_edit_lineno_append(workspace: object) -> None:
@@ -67,7 +67,7 @@ def test_edit_lineno_append(workspace: object) -> None:
_ = _read_lineno("c.txt") _ = _read_lineno("c.txt")
out = call_sync(edit_lineno, "c.txt", 2, 2, "more\n") out = call_sync(edit_lineno, "c.txt", 2, 2, "more\n")
assert "appended" in out assert "appended" in out
assert call_sync(read_file, "c.txt") == "only\nmore\n" assert call_sync(read_file, "c.txt") == "L1-2\nonly\nmore\n"
def test_edit_lineno_invalid(workspace: object) -> None: def test_edit_lineno_invalid(workspace: object) -> None:
+35
View File
@@ -20,6 +20,7 @@ from plyngent.tools.net.policy import (
parse_fetch_url, parse_fetch_url,
soft_confirm_reason, soft_confirm_reason,
) )
from plyngent.tools.truncate_token import decode_truncate_token
from tests.test_tools.helpers import call_async from tests.test_tools.helpers import call_async
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -237,6 +238,40 @@ async def test_fetch_truncation_and_binary(workspace: Path, http_server: str) ->
assert "binary body omitted" in out_bin assert "binary body omitted" in out_bin
async def test_fetch_char_truncation_embeds_token(workspace: Path, http_server: str) -> None:
del workspace
port = int(http_server.rsplit(":", 1)[1])
grant_private_host("127.0.0.1", port)
out = await call_async(fetch, f"{http_server}/big", max_chars=1000)
assert "truncated=true" in out
body = out.split("--- body ---", 1)[-1]
assert "truncate_token=" in body
token_line = next(ln for ln in body.splitlines() if "truncate_token=" in ln)
token_str = token_line.split("truncate_token=", 1)[1].rstrip("]")
token = decode_truncate_token(token_str)
assert token is not None
assert token.kind == "http"
assert token.limit == 1000
assert 0 < token.offset < 1000
# Continuation resumes exactly where the chunk stopped (no gap).
out2 = await call_async(fetch, f"{http_server}/big", max_chars=1000, offset=token.offset)
body2 = out2.split("--- body ---", 1)[-1].strip()
assert body2.startswith("x")
async def test_fetch_offset_skips_body(workspace: Path, http_server: str) -> None:
del workspace
port = int(http_server.rsplit(":", 1)[1])
grant_private_host("127.0.0.1", port)
# /big body is 5000 chars; offset 4900 leaves 100 chars, no token needed.
out = await call_async(fetch, f"{http_server}/big", max_chars=200, offset=4900)
body = out.split("--- body ---", 1)[-1]
assert body.strip() == "x" * 100
assert "truncate_token=" not in out
async def test_fetch_private_denied_without_grant(workspace: Path, http_server: str) -> None: async def test_fetch_private_denied_without_grant(workspace: Path, http_server: str) -> None:
del workspace del workspace
# No grant, no policy hook → hard deny (even though server is up). # No grant, no policy hook → hard deny (even though server is up).
+64 -6
View File
@@ -1,26 +1,77 @@
from __future__ import annotations from __future__ import annotations
import importlib
from pathlib import Path
from plyngent.tools.file import edit_replace, listdir, read_file, write_file from plyngent.tools.file import edit_replace, listdir, read_file, write_file
from tests.test_tools.helpers import call_sync from tests.test_tools.helpers import call_sync
get_truncated_module = importlib.import_module("plyngent.tools.file.get_truncated")
def _extract_token(out: str) -> str | None:
for line in out.splitlines():
if "truncate_token=" in line:
return line.split("truncate_token=", 1)[1].rstrip("]")
return None
def test_write_read_listdir_edit(workspace: object) -> None: def test_write_read_listdir_edit(workspace: object) -> None:
del workspace del workspace
assert "wrote" in call_sync(write_file, "notes/a.txt", "hello world") assert "wrote" in call_sync(write_file, "notes/a.txt", "hello world")
assert call_sync(read_file, "notes/a.txt") == "hello world" assert call_sync(read_file, "notes/a.txt") == "L1-1\nhello world"
listing = call_sync(listdir, "notes") listing = call_sync(listdir, "notes")
assert "a.txt" in listing assert "a.txt" in listing
assert "file" in listing assert "file" in listing
result = call_sync(edit_replace, "notes/a.txt", "world", "there") result = call_sync(edit_replace, "notes/a.txt", "world", "there")
assert "replaced" in result assert "replaced" in result
assert call_sync(read_file, "notes/a.txt") == "hello there" assert call_sync(read_file, "notes/a.txt") == "L1-1\nhello there"
def test_read_file_max_chars_embeds_token(workspace: Path) -> None:
(workspace / "big.txt").write_text("a" * 1000, encoding="utf-8")
out = call_sync(read_file, "big.txt", max_chars=200)
header, _, rest = out.partition("\n")
assert header.startswith("L1-") # 1-based range of the raw slice read
content, _, marker = rest.partition("\n[Truncated")
assert content == "a" * len(content)
assert len(content) < 200
assert "truncate_token=" in marker
token = _extract_token(out)
assert token is not None
parsed = get_truncated_module.decode_truncate_token(token)
assert parsed is not None
assert parsed.kind == "file"
assert parsed.offset == len(content) # cursor lands exactly where content stops
def test_read_file_default_truncates_at_cap(workspace: Path) -> None:
"""Truncation is the default: a large read embeds a token without max_chars."""
(workspace / "huge.txt").write_text("\n".join(f"line-{i}" for i in range(10_000)), encoding="utf-8")
out = call_sync(read_file, "huge.txt")
header, _, rest = out.partition("\n")
assert header == "L1-10000" # the requested slice (whole file)
assert "truncate_token=" in rest
# The slice is capped near the default tool-result cap.
assert len(rest) <= 32_000
# No marker when the file is small (no truncation occurred).
small = call_sync(read_file, "huge.txt", max_chars=0)
assert "Truncated" not in small
assert small.startswith("L1-10000\n")
def test_read_file_max_chars_small_file_untouched(workspace: Path) -> None:
(workspace / "small.txt").write_text("tiny", encoding="utf-8")
out = call_sync(read_file, "small.txt", max_chars=200)
assert out == "L1-1\ntiny"
assert "truncate_token=" not in out
def test_edit_replace_first_only(workspace: object) -> None: def test_edit_replace_first_only(workspace: object) -> None:
del workspace del workspace
_ = call_sync(write_file, "t.txt", "aa aa") _ = call_sync(write_file, "t.txt", "aa aa")
result = call_sync(edit_replace, "t.txt", "aa", "bb") result = call_sync(edit_replace, "t.txt", "aa", "bb")
assert call_sync(read_file, "t.txt") == "bb aa" assert call_sync(read_file, "t.txt") == "L1-1\nbb aa"
assert "1 of 2" in result or "1 of 2 matches" in result assert "1 of 2" in result or "1 of 2 matches" in result
assert "remain" in result assert "remain" in result
@@ -29,7 +80,7 @@ def test_edit_replace_max_replaces(workspace: object) -> None:
del workspace del workspace
_ = call_sync(write_file, "t.txt", "aa aa aa") _ = call_sync(write_file, "t.txt", "aa aa aa")
result = call_sync(edit_replace, "t.txt", "aa", "bb", max_replaces=2) result = call_sync(edit_replace, "t.txt", "aa", "bb", max_replaces=2)
assert call_sync(read_file, "t.txt") == "bb bb aa" assert call_sync(read_file, "t.txt") == "L1-1\nbb bb aa"
assert "2 of 3" in result assert "2 of 3" in result
assert "1 remain" in result assert "1 remain" in result
@@ -38,7 +89,7 @@ def test_edit_replace_all_matches(workspace: object) -> None:
del workspace del workspace
_ = call_sync(write_file, "t.txt", "aa aa") _ = call_sync(write_file, "t.txt", "aa aa")
result = call_sync(edit_replace, "t.txt", "aa", "bb", max_replaces=10) result = call_sync(edit_replace, "t.txt", "aa", "bb", max_replaces=10)
assert call_sync(read_file, "t.txt") == "bb bb" assert call_sync(read_file, "t.txt") == "L1-1\nbb bb"
assert "all 2 matches" in result assert "all 2 matches" in result
@@ -57,7 +108,14 @@ def test_edit_missing_old_string(workspace: object) -> None:
def test_read_offset_limit(workspace: object) -> None: def test_read_offset_limit(workspace: object) -> None:
del workspace del workspace
_ = call_sync(write_file, "lines.txt", "a\nb\nc\nd\n") _ = call_sync(write_file, "lines.txt", "a\nb\nc\nd\n")
assert call_sync(read_file, "lines.txt", offset=1, limit=2) == "b\nc\n" # offset is 0-based: offset=1 skips line 1, so the read starts at 1-based line 2.
assert call_sync(read_file, "lines.txt", offset=1, limit=2) == "L2-3\nb\nc\n"
def test_read_file_missing_and_directory(workspace: Path) -> None:
(workspace / "sub").mkdir()
assert call_sync(read_file, "nope.txt") == "error: file not found: nope.txt"
assert call_sync(read_file, "sub") == "error: not a file: sub"
def test_read_with_lineno(workspace: object) -> None: def test_read_with_lineno(workspace: object) -> None:
+6 -6
View File
@@ -11,7 +11,7 @@ def test_copy_file(workspace: object) -> None:
_ = call_sync(write_file, "a.txt", "hello") _ = call_sync(write_file, "a.txt", "hello")
out = call_sync(copy_path, "a.txt", "b.txt") out = call_sync(copy_path, "a.txt", "b.txt")
assert "copied" in out assert "copied" in out
assert call_sync(read_file, "b.txt") == "hello" assert call_sync(read_file, "b.txt") == "L1-1\nhello"
assert (workspace / "a.txt").is_file() assert (workspace / "a.txt").is_file()
@@ -21,8 +21,8 @@ def test_copy_directory(workspace: object) -> None:
_ = call_sync(write_file, "src/nested/y.txt", "y") _ = call_sync(write_file, "src/nested/y.txt", "y")
out = call_sync(copy_path, "src", "dst") out = call_sync(copy_path, "src", "dst")
assert "directory" in out assert "directory" in out
assert call_sync(read_file, "dst/x.txt") == "x" assert call_sync(read_file, "dst/x.txt") == "L1-1\nx"
assert call_sync(read_file, "dst/nested/y.txt") == "y" assert call_sync(read_file, "dst/nested/y.txt") == "L1-1\ny"
def test_copy_overwrite(workspace: object) -> None: def test_copy_overwrite(workspace: object) -> None:
@@ -32,7 +32,7 @@ def test_copy_overwrite(workspace: object) -> None:
assert "exists" in call_sync(copy_path, "b.txt", "a.txt") assert "exists" in call_sync(copy_path, "b.txt", "a.txt")
out = call_sync(copy_path, "b.txt", "a.txt", overwrite=True) out = call_sync(copy_path, "b.txt", "a.txt", overwrite=True)
assert "copied" in out assert "copied" in out
assert call_sync(read_file, "a.txt") == "new" assert call_sync(read_file, "a.txt") == "L1-1\nnew"
def test_move_file(workspace: object) -> None: def test_move_file(workspace: object) -> None:
@@ -41,7 +41,7 @@ def test_move_file(workspace: object) -> None:
out = call_sync(move_path, "old.txt", "renamed.txt") out = call_sync(move_path, "old.txt", "renamed.txt")
assert "moved" in out assert "moved" in out
assert not (workspace / "old.txt").exists() assert not (workspace / "old.txt").exists()
assert call_sync(read_file, "renamed.txt") == "data" assert call_sync(read_file, "renamed.txt") == "L1-1\ndata"
def test_move_directory(workspace: object) -> None: def test_move_directory(workspace: object) -> None:
@@ -50,7 +50,7 @@ def test_move_directory(workspace: object) -> None:
out = call_sync(move_path, "d", "e") out = call_sync(move_path, "d", "e")
assert "moved" in out assert "moved" in out
assert not (workspace / "d").exists() assert not (workspace / "d").exists()
assert call_sync(read_file, "e/a.txt") == "a" assert call_sync(read_file, "e/a.txt") == "L1-1\na"
def test_delete_file(workspace: object) -> None: def test_delete_file(workspace: object) -> None:
+103
View File
@@ -0,0 +1,103 @@
from __future__ import annotations
import importlib
from pathlib import Path
from plyngent.tools.truncate_token import TruncateToken, decode_truncate_token, encode_truncate_token
from tests.test_tools.helpers import call_async
get_truncated_module = importlib.import_module("plyngent.tools.file.get_truncated")
get_truncated = get_truncated_module.get_truncated
def _extract_token(out: str) -> str | None:
for line in out.splitlines():
if "truncate_token=" in line:
return line.split("truncate_token=", 1)[1].rstrip("]")
return None
async def test_get_truncated_invalid_token() -> None:
out = await call_async(get_truncated, "garbage-token")
assert out.startswith("error:")
assert "invalid" in out
async def test_get_truncated_file_chains_without_gap(workspace: Path) -> None:
(workspace / "big.txt").write_text("a" * 1000, encoding="utf-8")
token = encode_truncate_token(TruncateToken(kind="file", location="big.txt", offset=0, limit=200))
total_read = 0
for _ in range(20):
out = await call_async(get_truncated, token, max_chars=200)
body, _, _ = out.partition("\n[Truncated")
header, _, content = body.partition("\n")
assert header.startswith("L") and "-" in header
assert content == "a" * len(content)
total_read += len(content)
next_token = _extract_token(out)
if next_token is None:
break
parsed = decode_truncate_token(next_token)
assert parsed is not None
assert parsed.offset == total_read # no gap, no overlap
token = next_token
assert total_read == 1000
async def test_get_truncated_file_missing(workspace: Path) -> None:
token = encode_truncate_token(TruncateToken(kind="file", location="nope.txt", offset=0, limit=200))
out = await call_async(get_truncated, token, max_chars=200)
assert out.startswith("error:")
async def test_get_truncated_http_passthrough(monkeypatch) -> None:
calls: list[tuple[str, int, int]] = []
async def fake_fetch(url: str, *, offset: int, max_chars: int) -> str:
calls.append((url, offset, max_chars))
return f"body-from-{offset}"
monkeypatch.setattr(get_truncated_module.fetch, "handler", fake_fetch)
token = encode_truncate_token(TruncateToken(kind="http", location="https://example.com/x", offset=500, limit=400))
out = await call_async(get_truncated, token, max_chars=200)
assert out == "body-from-500"
assert calls == [("https://example.com/x", 500, 400)]
async def test_get_truncated_memory_chains() -> None:
from plyngent.tools.truncate_token import truncate_generic
bounded = truncate_generic("m" * 1000, 200)
assert "truncate_token=" in bounded
token = bounded.split("truncate_token=", 1)[1].rstrip("]")
parsed = decode_truncate_token(token)
assert parsed is not None
assert parsed.kind == "memory"
cursor = parsed.offset # chars already delivered by truncate_generic
delivered = cursor
for _ in range(20):
out = await call_async(get_truncated, token, max_chars=200)
content, _, _ = out.partition("\n[Truncated")
assert content == "m" * len(content)
delivered += len(content)
next_token = _extract_token(out)
if next_token is None:
break # source exhausted → no token, no marker
parsed = decode_truncate_token(next_token)
assert parsed is not None
assert parsed.kind == "memory"
# No gap: the next cursor is exactly where this chunk ended.
assert parsed.offset == delivered
token = next_token
assert delivered == 1000
async def test_get_truncated_memory_expired() -> None:
from plyngent.tools.truncate_token import _MEMORY_REMAINDERS, store_remainder
key = store_remainder("x" * 500)
del _MEMORY_REMAINDERS[key]
token = encode_truncate_token(TruncateToken(kind="memory", location=key, offset=0, limit=200))
out = await call_async(get_truncated, token, max_chars=200)
assert out.startswith("error:")
assert "expired" in out
+82 -35
View File
@@ -9,8 +9,8 @@ from plyngent.tools.process import (
close_pty, close_pty,
open_pty, open_pty,
read_pty, read_pty,
run_command, run_argv,
run_command_batch, run_argv_batch,
write_pty, write_pty,
write_pty_keys, write_pty_keys,
) )
@@ -41,20 +41,50 @@ def _py(code: str) -> list[str]:
return [sys.executable, "-c", code] return [sys.executable, "-c", code]
async def test_run_command_echo(workspace: object) -> None: async def test_run_argv_echo(workspace: object) -> None:
del workspace del workspace
out = await call_async(run_command, _py("print('hi')")) out = await call_async(run_argv, _py("print('hi')"))
assert "exit_code=0" in out assert "exit_code=0" in out
assert "hi" in out assert "hi" in out
async def test_run_command_batch_serial(workspace: object) -> None: async def test_run_argv_string_command_error(workspace: object) -> None:
del workspace
out = await call_async(run_argv, "ls -la")
assert "error:" in out
assert "only accepts an array" in out
assert "executable not found" not in out
async def test_run_argv_stringified_json_array_error(workspace: object) -> None:
del workspace
out = await call_async(run_argv, '["ls", "-la"]')
assert "error:" in out
assert "only accepts an array" in out
assert "executable not found" not in out
async def test_run_argv_non_string_args_error(workspace: object) -> None:
del workspace
out = await call_async(run_argv, ["ls", 1])
assert "error:" in out
assert "array of string args" in out
async def test_open_pty_string_command_error(workspace: object) -> None:
del workspace
out = await call_async(open_pty, "echo hi")
assert "error:" in out
assert "only accepts an array" in out
async def test_run_argv_batch_serial(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command_batch, run_argv_batch,
[ [
{"command": _py("print('a')")}, {"argv": _py("print('a')")},
{"command": _py("print('b')")}, {"argv": _py("print('b')")},
], ],
) )
assert "steps=2" in out assert "steps=2" in out
@@ -65,13 +95,13 @@ async def test_run_command_batch_serial(workspace: object) -> None:
assert "a" in out and "b" in out assert "a" in out and "b" in out
async def test_run_command_batch_stop_on_error(workspace: object) -> None: async def test_run_argv_batch_stop_on_error(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command_batch, run_argv_batch,
[ [
{"command": _py("import sys; sys.exit(2)")}, {"argv": _py("import sys; sys.exit(2)")},
{"command": _py("print('should-not-run')")}, {"argv": _py("print('should-not-run')")},
], ],
stop_on_error=True, stop_on_error=True,
) )
@@ -80,17 +110,17 @@ async def test_run_command_batch_stop_on_error(workspace: object) -> None:
assert "should-not-run" not in out assert "should-not-run" not in out
async def test_run_command_batch_pipe_out(workspace: object) -> None: async def test_run_argv_batch_pipe_out(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command_batch, run_argv_batch,
[ [
{ {
"command": _py("print('piped-payload', end='')"), "argv": _py("print('piped-payload', end='')"),
"pipe_out": True, "pipe_out": True,
}, },
{ {
"command": _py("import sys; print(sys.stdin.read())"), "argv": _py("import sys; print(sys.stdin.read())"),
}, },
], ],
) )
@@ -99,27 +129,27 @@ async def test_run_command_batch_pipe_out(workspace: object) -> None:
assert "pipe_out=true" in out assert "pipe_out=true" in out
async def test_run_command_batch_last_pipe_out_error(workspace: object) -> None: async def test_run_argv_batch_last_pipe_out_error(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command_batch, run_argv_batch,
[{"command": _py("print(1)"), "pipe_out": True}], [{"argv": _py("print(1)"), "pipe_out": True}],
) )
assert "error:" in out assert "error:" in out
assert "pipe_out" in out assert "pipe_out" in out
async def test_run_command_batch_mix_stderr(workspace: object) -> None: async def test_run_argv_batch_mix_stderr(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command_batch, run_argv_batch,
[ [
{ {
"command": _py("import sys; sys.stdout.write('out'); sys.stderr.write('err')"), "argv": _py("import sys; sys.stdout.write('out'); sys.stderr.write('err')"),
"mix_stderr": True, "mix_stderr": True,
"pipe_out": True, "pipe_out": True,
}, },
{"command": _py("import sys; print(repr(sys.stdin.read()))")}, {"argv": _py("import sys; print(repr(sys.stdin.read()))")},
], ],
) )
assert "ran=2" in out assert "ran=2" in out
@@ -127,32 +157,32 @@ async def test_run_command_batch_mix_stderr(workspace: object) -> None:
assert "out" in out and "err" in out assert "out" in out and "err" in out
async def test_run_command_denied(workspace: object) -> None: async def test_run_argv_denied(workspace: object) -> None:
del workspace del workspace
out = await call_async(run_command, ["rm", "-rf", "."]) out = await call_async(run_argv, ["rm", "-rf", "."])
assert "denied" in out assert "denied" in out
async def test_run_command_cwd(workspace: object) -> None: async def test_run_argv_cwd(workspace: object) -> None:
assert isinstance(workspace, Path) assert isinstance(workspace, Path)
sub = workspace / "sub" sub = workspace / "sub"
sub.mkdir() sub.mkdir()
_ = (sub / "f.txt").write_text("z", encoding="utf-8") _ = (sub / "f.txt").write_text("z", encoding="utf-8")
out = await call_async(run_command, _py("import os; print('\\n'.join(os.listdir()))"), cwd="sub") out = await call_async(run_argv, _py("import os; print('\\n'.join(os.listdir()))"), cwd="sub")
assert "f.txt" in out assert "f.txt" in out
async def test_run_command_timeout(workspace: object) -> None: async def test_run_argv_timeout(workspace: object) -> None:
del workspace del workspace
out = await call_async(run_command, _py("import time; time.sleep(5)"), timeout_seconds=0.2) out = await call_async(run_argv, _py("import time; time.sleep(5)"), timeout_seconds=0.2)
assert "timed_out=true" in out assert "timed_out=true" in out
async def test_run_command_timeout_keeps_partial_output(workspace: object) -> None: async def test_run_argv_timeout_keeps_partial_output(workspace: object) -> None:
del workspace del workspace
# Print then sleep past the timeout so communicate has partial stdout after kill. # Print then sleep past the timeout so communicate has partial stdout after kill.
out = await call_async( out = await call_async(
run_command, run_argv,
_py("import sys, time; sys.stdout.write('partial-out'); sys.stdout.flush(); time.sleep(5)"), _py("import sys, time; sys.stdout.write('partial-out'); sys.stdout.flush(); time.sleep(5)"),
timeout_seconds=0.3, timeout_seconds=0.3,
) )
@@ -160,10 +190,27 @@ async def test_run_command_timeout_keeps_partial_output(workspace: object) -> No
assert "partial-out" in out assert "partial-out" in out
async def test_run_command_stdin(workspace: object) -> None: async def test_run_argv_stdout_cap_embeds_token(workspace: object) -> None:
del workspace
# Per-stream cap default is 32000 chars: a bigger stdout embeds a token.
out = await call_async(run_argv, _py("print('x' * 40000)"))
assert "truncate_token=" in out
token_line = next(ln for ln in out.splitlines() if "truncate_token=" in ln)
assert "[Truncated (32000 chars max.;" in token_line
assert "[Truncated" in out
async def test_run_argv_small_stdout_no_marker(workspace: object) -> None:
del workspace
out = await call_async(run_argv, _py("print('tiny')"))
assert "Truncated" not in out
assert "truncate_token=" not in out
async def test_run_argv_stdin(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command, run_argv,
_py("import sys; print(sys.stdin.read(), end='')"), _py("import sys; print(sys.stdin.read(), end='')"),
stdin="hello-stdin\n", stdin="hello-stdin\n",
) )
@@ -171,10 +218,10 @@ async def test_run_command_stdin(workspace: object) -> None:
assert "hello-stdin" in out assert "hello-stdin" in out
async def test_run_command_env(workspace: object) -> None: async def test_run_argv_env(workspace: object) -> None:
del workspace del workspace
out = await call_async( out = await call_async(
run_command, run_argv,
_py("import os; print(os.environ.get('PLYNGENT_TEST_VAR', ''), end='')"), _py("import os; print(os.environ.get('PLYNGENT_TEST_VAR', ''), end='')"),
env={"PLYNGENT_TEST_VAR": "from-env"}, env={"PLYNGENT_TEST_VAR": "from-env"},
) )
+2 -2
View File
@@ -33,10 +33,10 @@ def test_new_temporary_workspace_allowlist(workspace: object) -> None:
target = temp / "scratch.txt" target = temp / "scratch.txt"
_ = target.write_text("hello-temp", encoding="utf-8") _ = target.write_text("hello-temp", encoding="utf-8")
assert resolve_path(str(target)) == target.resolve() assert resolve_path(str(target)) == target.resolve()
assert call_sync(read_file, str(target)) == "hello-temp" assert call_sync(read_file, str(target)) == "L1-1\nhello-temp"
_ = call_sync(write_file, "project.txt", "proj") _ = call_sync(write_file, "project.txt", "proj")
assert call_sync(read_file, "project.txt") == "proj" assert call_sync(read_file, "project.txt") == "L1-1\nproj"
# Sibling under system temp that we did not allowlist still fails. # Sibling under system temp that we did not allowlist still fails.
outside = temp.parent / "not-ours-should-fail" outside = temp.parent / "not-ours-should-fail"
+150
View File
@@ -0,0 +1,150 @@
from __future__ import annotations
from plyngent.tools.truncate_token import (
TruncateToken,
decode_truncate_token,
encode_truncate_token,
get_remainder,
truncate_generic,
truncate_with_token,
truncation_marker,
)
def test_token_roundtrip() -> None:
token = TruncateToken(kind="file", location="src/a.py", offset=100, limit=32_000)
encoded = encode_truncate_token(token)
assert decode_truncate_token(encoded) == token
def test_decode_garbage_returns_none() -> None:
assert decode_truncate_token("not-a-token") is None
assert decode_truncate_token("") is None
assert decode_truncate_token("!!!") is None
def test_decode_rejects_invalid_fields() -> None:
bad = TruncateToken(kind="file", location="x", offset=-1, limit=10)
assert decode_truncate_token(encode_truncate_token(bad)) is None
bad_limit = TruncateToken(kind="http", location="u", offset=0, limit=0)
assert decode_truncate_token(encode_truncate_token(bad_limit)) is None
def test_marker_format() -> None:
token = TruncateToken(kind="file", location="a.txt", offset=100, limit=32_000)
marker = truncation_marker(32_000, 500, token)
assert marker.startswith("\n[Truncated (32000 chars max.; 500 omitted). truncate_token=")
assert marker.endswith("]")
assert encode_truncate_token(token) in marker
def test_truncate_with_token_short_text_untouched() -> None:
out, token = truncate_with_token(
"hello",
32_000,
kind="file",
location="a.txt",
offset=0,
limit=32_000,
total_len=5,
)
assert out == "hello"
assert token is None
assert "Truncated" not in out # no hint when no truncation
def test_truncate_with_token_cuts_and_advances() -> None:
text = "x" * 500
out, token = truncate_with_token(
text,
200,
kind="file",
location="a.txt",
offset=0,
limit=200,
total_len=500,
)
assert token is not None
content, _, _ = out.partition("\n[Truncated")
assert content.startswith("x")
assert "[Truncated (200 chars max.; 300 omitted). truncate_token=" in out
assert token.offset == len(content) # cursor lands exactly where content stops
assert len(out) <= 200 # marker fits inside the char budget
def test_truncate_with_token_more_after_nonoverflow() -> None:
# Segment fits the chunk, but the source continues: still emit a token.
text = "y" * 30
out, token = truncate_with_token(
text,
40,
kind="file",
location="a.txt",
offset=0,
limit=40,
total_len=200,
)
assert token is not None
assert token.offset == 30 # advanced by returned length only
assert "[Truncated (40 chars max.; 170 omitted). truncate_token=" in out
assert out.startswith("y" * 30)
def test_truncate_with_token_end_reached_no_token() -> None:
text = "z" * 30
out, token = truncate_with_token(
text,
40,
kind="http",
location="https://example.com/x",
offset=0,
limit=40,
total_len=30,
)
assert token is None
assert out == text
assert "Truncated" not in out # no hint when the source ends
def test_truncate_with_token_nonzero_start() -> None:
text = "y" * 500
out, token = truncate_with_token(
text,
200,
kind="http",
location="https://example.com/x",
offset=150,
limit=200,
total_len=700,
)
assert token is not None
assert token.kind == "http"
content, _, _ = out.partition("\n[Truncated")
assert token.offset == 150 + len(content)
assert "[Truncated (200 chars max.; 350 omitted). truncate_token=" in out
assert len(out) <= 200
def test_truncate_generic_short_untouched() -> None:
assert truncate_generic("tiny", 200) == "tiny"
def test_truncate_generic_embeds_memory_token() -> None:
text = "a" * 500
out = truncate_generic(text, 200)
assert len(out) <= 200
assert "[Truncated (200 chars max." in out
token_str = out.split("truncate_token=", 1)[1].rstrip("]")
token = decode_truncate_token(token_str)
assert token is not None
assert token.kind == "memory"
# Remainder is the full original text; token offset points into it.
remainder = get_remainder(token.location)
assert remainder == text
assert token.offset < len(text)
def test_truncate_generic_no_marker_when_fits() -> None:
out = truncate_generic("a" * 50, 200)
assert out == "a" * 50
assert "Truncated" not in out