Commit Graph

318 Commits

Author SHA1 Message Date
NCBM e72d2359d7 core/skills: read the filesystem live, drop the cache and reload 2026-10-09 21:40:38 +08:00
NCBM 1f04889449 cli/skills: open the skill directory instead of printing SKILL.md 2026-10-09 21:40:27 +08:00
NCBM 6807055b9f cli/slash: remove the /clear command 2026-10-09 21:15:48 +08:00
NCBM f89c8dec31 lmproto/openai_compatible: drop provider blocks on a chat wire
`provider_blocks` means something only to the surface that produced them (Anthropic `thinking` / server-tool blocks). A session that switched surface would otherwise send those dicts to a chat-completions provider as an assistant-message key it does not know; the request normalization drops the field, on a copy, so the history keeps it.
2026-10-09 20:09:36 +08:00
NCBM 7a54642ecf core/agent: rebuild a streamed round's provider blocks
Streaming is the default path: `thinking` fragments, a `server_tool_use`'s `partial_json` query and a `web_search_tool_result`'s entries arrive as separate SSE events, so the blocks the round has to hand back exist only as fragments. The dispatch rebuilds each one from its start payload (field for field, under the stream's own names) plus the fragments that follow, and passes it to the loop as a chunk; the loop copies it onto the round's message, where a round that kept only those blocks counts as payload rather than an empty completion.

The thinking text streams as reasoning as it arrives — the search query and the signature go back with the round only.
2026-10-09 20:09:29 +08:00
NCBM 2fdbc3ce5d core/agent: keep a Messages round's provider blocks on the assistant message
DeepSeek's Anthropic compat answers a hosted search inside the turn: the response carries the `thinking` that picked the query, the `server_tool_use`, and the `web_search_tool_result`. The response mapping kept text and tool_use only, so the round lost its evidence — and the `thinking` signature, which only the provider can verify, could not go back at all. `AssistantChatMessage.provider_blocks` carries those blocks as raw JSON now (`thinking` text *and* signature, the query, the result entries with an `encrypted_content` the API rejects when trimmed) and the assistant turn hands them back first, ahead of its text and tool_use blocks; `thinking` also lands in `reasoning_content`, so the chain of thought streams as reasoning like every other surface's.

`AnthropicRawContentBlock` names the provider fields a streamed block carries, and `AnthropicServerToolUseContent` keeps the `caller` metadata the wire sends.
2026-10-09 20:09:20 +08:00
NCBM 6c4c8b9328 core/tools: edit_lineno rejections name the read_file call 2026-10-09 19:46:16 +08:00
NCBM bf8203ae37 core/agent: stream a repeated reasoning text only once
DeepSeek's Responses stream sends ``response.reasoning_text.delta`` fragments and then ``response.reasoning_text.done`` carrying the whole chain of thought those fragments spelled out. The loop folded a full delta into the round — it replaces the fragments, and that is what the assistant message hands back to the API — but streamed every delta verbatim, so the chain of thought was printed twice: once as it arrived and again as the ``.done`` text.

A full delta now streams only a tail the fragments did not cover, and nothing at all when it merely repeats them; a round whose fragments never arrived still streams the full text once. The message keeps the complete text either way.
2026-10-07 00:26:38 +08:00
NCBM 4b5f67f5f3 cli/display: render web_search as a pretty line
``web_search`` is composed by the CLI host rather than the builtin catalog, so it had no ``_PRETTY_LINES`` entry and fell back to the ``[tool] …`` / ``[tool ok] …`` whole lines. It now leads with ``* Web Search '<query>' `` (the query as a 60-character preview, like the ask and skill subjects) and lands on the hit count — or on ``(no results)`` when every source came up empty, an emptiness the tool reports as ``error: the search found nothing`` and which is therefore not shown as a failure. A real error still prints in red with its message.

The guard test that keeps a builtin tool from shipping without a renderer now checks this host-composed tool too, since a catalog walk cannot see it.
2026-10-07 00:20:41 +08:00
NCBM 2bb6e06171 core/tools: search a list of sources, DeepSeek's index then Bing
``search_sources`` (a provider list, ``["deepseek", "bing"]`` by default,
``[]`` to not offer the tool at all) names what the local ``web_search`` tool
searches with, in fallback order: ``search_in_order`` asks each source in turn
and keeps the first answer, so a source that errors — a rate limit, a rejected
key — or finds nothing hands over to the next instead of failing the tool. The
handler reports the failures only when every source came up empty.

The Bing leg answers with its RSS results (``?format=rss``), which need no key
and carry a description — the snippet DeepSeek's own index does not hand out —
and it goes through the ``fetch`` client, so the SSRF and host-grant policy
applies to it too (a host that binds tool instance state, as the CLI does).
2026-10-07 00:09:46 +08:00
NCBM e877df9d94 config: default DeepSeek to web_search and honour its provider_tools
``provider_tools`` reached the effective provider only for the OpenAI preset,
so a DeepSeek ``provider_tools`` in TOML was dropped on the way — and the preset
defaulted to none, leaving search unavailable for every convention.

DeepSeek now defaults to ``[{type="web_search"}]`` like OpenAI does (``[]``
turns it off) and its list reaches the agent: the Anthropic convention runs the
hosted search inside the response, the Responses and chat conventions get the
local ``web_search`` tool (see ``config.routing.local_web_search``).
2026-10-06 23:59:01 +08:00
NCBM a0d39bba6d core/tools: search through a local tool where a hosted one is ignored
Only DeepSeek's Anthropic-compatible surface runs a hosted ``web_search``: the
Responses and chat-completions surfaces answer "I can't browse the web" with
``{"type": "web_search"}`` and with the server-tool shape alike, so a DeepSeek
provider that talks them had no search at all.

``config.routing.local_web_search`` decides per provider: a DeepSeek provider
whose ``provider_tools`` ask for ``web_search`` and whose convention is not
``anthropic`` gets the local tool (``tools/net/search.py``), which asks the
search-capable endpoint behind the scenes and hands the model one ``title —
url`` line per hit — read a page with ``fetch``, the bodies stay server-side.
DeepSeek's preset asks for no hosted tool by default, so this stays opt-in.
2026-10-06 23:45:35 +08:00
NCBM e2ee937d8f lmproto: search through DeepSeek's Anthropic-compatible endpoint
DeepSeek runs ``web_search`` inside a turn — the request declares the server
tool, the response carries a ``server_tool_use`` per query and the
``web_search_tool_result`` it produced — so a search is one ``/messages`` call
whose hits are read out of the content blocks. The turn stops as soon as it has
searched (a live probe with 64 output tokens spent all of them before the search
and returned nothing, 512 answers with the hits).

Hits carry a title, a URL and an ``encrypted_content`` only the index can read,
so the hit list is all a caller gets: the page bodies stay server-side.
2026-10-06 23:45:29 +08:00
NCBM 21f59ea33c core/agent: count the prompt a search turn really sent
The compat opens with the question alone (``message_start`` reports 165 tokens)
and puts what the search then pulled in on ``message_delta`` (12468); merging
preferred the earlier value, so the CLI's context gauge and the auto-compact
threshold ran ~12k short of the request. The larger of the two wins now (output
tokens are the delta's cumulative count either way).
2026-10-06 23:36:35 +08:00
NCBM 45246bf949 lmproto: keep the thinking block's type on the wire
``omit_defaults`` dropped the block's defaulted discriminator, and a request
that carried thinking was answered with 422 "thinking: missing field ``type``" —
which is what the compat sent for any configured ``reasoning_effort``. The field
has no default now, so it always encodes (both call sites pass it).
2026-10-06 23:36:29 +08:00
NCBM 1fa41c9fde core/agent: run a hosted search as the surface's own tool
DeepSeek's Anthropic compat runs a hosted ``web_search`` itself and answers with
the blocks that search produced — ``thinking``, ``server_tool_use`` (the query)
and ``web_search_tool_result`` — so the content union that knew only text and
tool_use could not decode a live search turn. It carries them now, ``tools``
accepts the opaque server-tool dicts the surface declares, and
``web_search_tool_result.content`` stays loose on purpose: its
``web_search_result`` entries hold an ``encrypted_content`` the API demands back
unchanged (a trimmed result is answered with 422 "missing field
``encrypted_content``").

``provider_tools`` reached only the Responses surface, and the Anthropic one
wants its own definition of the same intent: a bare ``web_search`` becomes
``{type="web_search_20250305", name="web_search"}`` (the server tool Claude
declares and DeepSeek's compat reads, keeping a ``max_uses`` cap or any other
key), merged after the local function tools.

Such a tool is the server's: it streams ``input_json_delta`` fragments like a
call, and reading them as one asked the registry for a tool the server had
already run. Block types are now tracked, so only a ``tool_use`` block becomes a
tool call; the search blocks and ``thinking`` are skipped by the response
mapping (the answer the model wrote already used them) and never executed.
2026-10-06 23:35:46 +08:00
NCBM 0da6e8da8d lmproto: give every DeepSeek assistant round its reasoning field
Thinking mode checks the field per round, not per call: an answer whose message
carried no `reasoning_content` came back 400 with the wording a call gets ("the
`reasoning_content` … must be passed back to the API"), while an empty string
satisfies the check. The injection now touches every assistant message instead
of only the ones carrying `tool_calls`.
2026-10-06 23:23:24 +08:00
NCBM c06fb57e72 core/agent: return each round's reasoning in the Responses input
A request carrying `tools` must return every round's chain-of-thought — "必须完整
回传 `reasoning_content` … 即使该轮模型未实际进行工具调用" — and the API merges a
`reasoning` item into the assistant message its round forms: a missing item and
an empty `reasoning_text` are both answered with "The `reasoning_text` in the
thinking mode must be passed back to the API". Each round now goes back as its
reasoning, its text, then its calls, with a blank line standing in when the
round kept none (a `rollback_tail` carrier, a forged todo call, the seed a
compacted session starts with) — never words of the bridge's own for a round
that may genuinely have reasoned nothing. A round's calls stay contiguous ahead
of their outputs: an interleaved batch reads as a second round without
reasoning and is rejected the same way, and an output recorded after another
item moves up beside its own call.

The reasoning is read back from the item's `content` parts, the only part the
API documents for `reasoning` items (`summary` and `encrypted_content` are not
supported).
2026-10-06 23:23:18 +08:00
NCBM 932b4f0a7f core/agent: keep a rolled-back turn's reasoning with its calls
A failed turn keeps the calls that already ran as a bare pairing carrier, and
clipping the chain-of-thought off that carrier sent the call back to the API
with no reasoning at all — thinking mode refuses a call whose reasoning never
came back. The carrier keeps the reasoning and drops only the text.
2026-10-06 23:23:12 +08:00
NCBM 135c28a622 core/tools: review a wrapper option that carries a command line
`env -S "rm -rf /"` ran a command nothing looked at: the option belongs to the
wrapper, so the scan skipped it and the denylist found no program to block while
the soft confirm had no interpreter to review. Collect those values — `env
-S`/`--split-string` and `su -c`/`--command`/`--session-command`, jammed
`--opt=value` and `-Svalue` forms included — and confirm the call the way an
interpreter's `-c` one-liner is confirmed, printing the text under its own
`env -S` label. Recursing into the text for a hard denylist is left out for now:
the review asks the human, who sees the command.
2026-10-06 21:54:18 +08:00
NCBM 2b3b2073a0 core/tools: only value-taking options may swallow the program
The wrapper option table listed flags as if they consumed the next token, so
the program behind them was skipped: `exec -c git -c init.defaultBranch=main
init` resolved to `init` and the policy denied it as a denylisted basename,
while `exec -c rm -rf /`, `command -p rm -rf /` and `command -v rm` hid `rm`
from the denylist completely and `exec -c python -c …` lost the interpreter
review. Drop the entries that take no value — `exec -c`/`-l`, the whole
`command` entry, `pkexec --disable-internal-agent`, `su -p`, and `xargs -e`
(its argument is optional, and the jammed form needs no entry) — so the
resolved program is the one that really runs.
2026-10-06 21:50:21 +08:00
NCBM 768b310a0b cli/display: hold a batch's later prefixes until the open row is done
A round that asked for several calls printed the first one as an abandoned
half-line: the next prefix closed the still-open row (a bare newline), the
result then printed whole and repeated that prefix, and the rest of the batch
was suppressed. The model edits files in batches, so the duplicate row showed
up there most. Queue the later prefixes instead: each one opens its own row
once the row before it has taken its detail, which keeps one finished row per
call in call order and leaves the bytes identical to the non-interactive path.
Only output that genuinely takes the line — a prompt box, streamed text, an
error — still ends the open prefix as its own line. The cursor is still never
moved.
2026-10-06 21:38:03 +08:00
NCBM ec5340bdce config: rename the DeepSeek flash model to deepseek-flash
DeepSeek dropped the `v4` prefix from its flash model, and the old id stays
valid only as a compatibility alias. Seed the default catalog with the current
name so a provider that omits `models` no longer offers a deprecated id, and
follow the rename in the wire fixtures and the minimal config template. An
explicit `deepseek-v4-flash` in TOML still passes through unchanged.
2026-10-06 21:22:35 +08:00
NCBM 60bc5957a7 cli: resolve thinking settings for the chat session 2026-10-06 21:11:28 +08:00
NCBM fb8ef88d9f core/agent: send the resolved reasoning effort with each request 2026-10-06 21:11:28 +08:00
NCBM 445aba7e5f lmproto: carry thinking options on the wire 2026-10-06 21:11:28 +08:00
NCBM ffd4dbbe26 config: resolve thinking strength across agent, provider, and model layers 2026-10-06 21:11:28 +08:00
NCBM 7cd5fcc576 cli/display: merge a batch of todo calls into one line
A round that updated its todo list printed one header plus a full stack
render per call, so the same list appeared several times. When a batch only
touches todo tools, print one `* Todo x<N>: Push, Update` line and the last
result, whose text already ends with the final stack. A batch holding any
other tool keeps the per-call lines, and a todo call never prints a prefix
at call time (it would dangle when the batch is merged).
2026-10-06 20:51:07 +08:00
NCBM 7d9b979604 core/agent: record every tool result before yielding them
A caller that goes away mid-batch (cancel, display error) left the results of
tools that had already run out of history, so the retry ran them again. Append
the collected results first, then emit the events.
2026-10-06 20:33:30 +08:00
NCBM cecd5e5ae3 core/agent: keep executed tool results when a failed turn is rolled back
A tool batch counted as committed only once every call had its result, so a
batch cut short was dropped whole and the retry ran tools that had already
had effects. Keep the calls that did report a result: their assistant row
stays as the pairing carrier (a request must answer every tool_calls id),
without the text of the unfinished turn, and the calls without a result are
absent so the retry re-issues them instead of assuming they ran. The kept
rows go to the store right away, and a failed rollback write is not allowed
to mask the failure being rolled back.
2026-10-06 20:32:09 +08:00
NCBM c7680491c9 core/tools: denylist power-management subcommands and privileged programs 2026-10-06 19:55:14 +08:00
NCBM 7862a555bc core/tools: review interpreters and detached runners past wrappers 2026-10-06 19:44:29 +08:00
NCBM 9501efb3d7 core/tools: block a denylisted program behind env wrappers and run launchers 2026-10-06 19:42:06 +08:00
NCBM 6453a8ac89 core/tools: resolve the program behind env wrappers and run launchers 2026-10-06 19:41:17 +08:00
NCBM 25b8ef3e58 core/tools: resume fetch bodies from memory instead of refetching 2026-10-06 19:14:38 +08:00
NCBM 1068ea0025 prompting: end an open tool line before writing a prompt 2026-10-06 19:06:22 +08:00
NCBM a81b138bf8 cli/display: close an interrupted pretty line instead of erasing it 2026-10-06 19:01:42 +08:00
NCBM 7127190b9a cli/display: label regex_files lines with Search 2026-09-22 23:22:29 +08:00
NCBM de03e5932b core/tools: rename grep_files to regex_files (regex pattern) 2026-09-22 23:22:13 +08:00
NCBM 9b365403d7 cli/history: print a row's reasoning before its answer
A reasoning trace was appended after the body it produced, which read as if the
answer had come first and the thinking had followed it. It now leads the row,
under its own `reasoning:` label with a blank line before the answer - the order
the live stream uses. The trace only prints where it already did: a turn's ends
(the default view) and -vv; previews stay silent.
2026-09-22 23:05:59 +08:00
NCBM 20fc493c47 tools/skills: rebase search hits from either path form
A skill root can live inside the workspace (the project-scope .claude/skills),
and grep reports those hits relative to the workspace root while outside ones
come back absolute. The attribution only stripped an absolute skill prefix, so
an in-workspace root produced alpha/alpha/SKILL.md - and, worse, a path that no
longer started with the skill name when the root itself was the workspace.
Normalizing the hit to an absolute path before relativizing to the skill covers
both, and the hit line is now matched with a pattern instead of a colon split so
a Windows drive letter cannot split it wrongly.

Found by running the REPL against a workspace-local skill root; the tool tests
only covered roots outside the workspace, so they get that case now.
2026-09-22 22:59:57 +08:00
NCBM 90174340d7 cli/skills: expose skills to the session
ReplState builds the store from [skills] against the current workspace and
publishes its roots as static read-only roots, so read_file/grep_files see a
skill directory while writing stays denied; the roots are rebuilt when the
workspace changes, since the project-scope directories hang off it. Discovered
skills are folded into the system prompt the way MCP instructions are, with the
catalog cap and its 'use skill_search' hint, and only when tools are on and
injection is not switched off; [skills].enabled = false drops the tools from the
registry entirely.

The write policy is installed with the CLI confirm: once, session, or no, timed
with a default of no and independent of YOLO, because a skill directory is not a
grant. A library host that installs no hook keeps the deny default. /skills
lists, searches, reads, and reloads; /status counts what was found.
2026-09-22 22:55:24 +08:00
NCBM c7441d4a43 tools/skills: create and modify skills behind a grant
skill_create writes a SKILL.md (name, description, body) plus whatever bundled
files the model lists, and skill_edit changes one file of an existing skill
either by literal replacement, by appending, or by rewriting it. Both resolve
every path inside the skill directory first, so a bad argument is answered
before anything is written and before the human is asked anything; a rewritten
SKILL.md is re-parsed so a skill that lost its description (or its frontmatter)
says so instead of failing quietly. The store is reloaded after a write, which
makes a new skill visible to the very next skill_list.

Writing is a host policy, not a directory-access grant: the denylist wins
first, then a session grant, then [skills].allow_write, then the host's confirm
hook — asked once per skill directory, so one answer covers its files. A host
without a hook denies, which keeps the library default safe; the confirm can
also answer 'session' to cover the rest of the chat.
2026-09-22 22:46:21 +08:00
NCBM 1d05da62c2 tools/skills: list, read, and search skills
skill_list shows what was discovered (name, source root, directory, description)
together with the copies a name collision shadowed and the notes from repaired
frontmatter, so a third-party skill never disappears quietly. skill_read
addresses a skill by name and defaults to its SKILL.md, reusing read_file for
ranges, line numbers, and the continuation token; a bundled file is resolved
inside the skill directory only, symlinks included. skill_search greps every
visible skill (or one) and rebases each hit on its skill, which makes a match
attributable without absolute paths.

Reads need no grant: the host registers the roots as static read-only roots on
the workspace policy, so read_file/grep_files see skill directories too while
write access stays denied — skill writes will go through their own confirm.
Grep used to drop every file outside the primary workspace root, which also hid
hits under a granted directory; those now come back as absolute paths.
2026-09-22 22:42:45 +08:00
NCBM a9f58428bf skills: discover roots and parse SKILL.md frontmatter
A skill is a directory with a SKILL.md; roots are ordered by priority (an
explicit path, then our own user directory, then the directories other
harnesses install into, with the project-scope one behind its own switch). The
first copy of a name wins and the ones it shadows stay listed for reporting
instead of vanishing.

The frontmatter reader covers the subset those files actually use — scalars,
booleans, ints, inline and block lists, | and > blocks, folded plain scalars —
so no YAML dependency is needed; a nested mapping is skipped and a structurally
broken block degrades to "no metadata" with a note, keeping a third-party skill
listable rather than dropping it. Metadata is cached against the directory list
and its SKILL.md timestamps, so a listing is cheap yet notices an edit.
2026-09-22 22:37:42 +08:00
NCBM bbda658eab skills/config: add the [skills] section
Skills are discovered from user-level directories by default: an explicit
``paths`` list wins, then the built-in sources (our own user config dir, and
other harnesses' skill dirs) with the project-scope dirs behind their own
switch. A name found twice shadows the lower-priority copy without hiding it.

``inject_catalog`` / ``max_catalog_skills`` bound what reaches the system
prompt, ``enabled = false`` registers nothing, and ``allow_write`` is the
standing grant for editing skills (default: ask every time). A malformed
section degrades to the defaults, like [agent] and [mcp].
2026-09-22 22:26:50 +08:00
NCBM d6c4aa0d4f cli/display: prettify the remaining tools
PTY, human-prompt, temp-dir, truncate, and the rest of the VCS/todo family still
printed raw "[tool] name(args)" lines while every other tool had a syntax. Each
one now has a prefix that appears as the call starts and an outcome parsed from
its own result shape: session ids and alive/exit state for PTY, answered/
selected/count for the ask family, granted or denied for access requests, the
new path for a temp dir, changed files with line counts for vcs_diff, and the
rendered stack for the remaining todo commands. ask_into_pty reports only that
the answer was written.

Resolution is no longer a dict lookup: namespaced MCP tools are built from
their ``mcp__<server>__<tool>`` name so a newly connected server gets the same
line without registration, and the resolved renderer is carried from call to
result instead of being looked up twice. Plugin tools and custom calls keep the
whole-line fallback. The shared key=value parser and the ranged-read detail are
reused by run_argv, fetch, read_file, and get_truncated rather than copied.

Tests cover every added line (including the error paths), the MCP naming rule,
and an invariant that each builtin tool resolves to a renderer.
2026-09-22 22:25:34 +08:00
NCBM 221d5d5b68 cli/display: keep the line above an erased tool prefix
Erasing the in-flight prefix reused a helper that cleared one row more than
requested, so a parallel tool batch also erased the line above the prefix -
the streamed reasoning text that sits there when a model reasons and then
calls tools. The helper now clears exactly the requested rows and takes
explicit control of where the cursor lands; the parallel fallback erases its
own line and lets the leading newline of the whole-line write reuse it.

Row counts are computed from physical terminal rows instead of logical
newlines, so an assistant segment wider than the terminal is fully erased
before the markdown re-render rather than leaving a stray copy behind.

Tests: a byte-level regression for reasoning before a parallel batch, unit
coverage for the row-counting and erase primitives, and a wrapped-line flush.
2026-09-22 22:13:24 +08:00
NCBM 836ac8d6f8 cli/history: never collapse a turn's ends
The mixed policy previewed the user row, so a long prompt came back truncated
even though it is half of what the turn is about. Both ends - the human's
message and the model's answer - now print in full at every verbosity level;
--preview remains the explicit one-line skimmer that collapses them too.

Tests cover the ends staying full in the default and -v views (and the rounds
between them staying one-liners) plus a long-prompt regression.
2026-09-22 21:15:07 +08:00
NCBM 76e5b8ac0f cli/history: group /history by turn
/history counted raw message rows, so /history 1 printed whatever row happened
to be last (usually a tool result in a tool-heavy session) and "one row means
full body" was a proxy for "show me that one thing".

A turn is what a human reviews: /history now prints the last turn's two ends
(the user message and the model's answer, markdown for the latter), -v prints
every row of the selected turn(s), -vv prints full bodies plus the local system
row, and --preview collapses every row to one line. N addresses turn N,
last [N] the last N turns, --message N a single row.

Numbering is a display ordinal over conversation rows (local-only rows print as
local) and deliberately never a database row number - seq is an implementation
detail that shifts with /clear. Grouping and rendering live in the new
cli/transcript.py.
2026-09-22 21:01:49 +08:00