core/skills: read the filesystem live, drop the cache and reload

This commit is contained in:
2026-10-09 21:40:38 +08:00
parent 1f04889449
commit e72d2359d7
8 changed files with 29 additions and 63 deletions
+3 -3
View File
@@ -68,7 +68,7 @@ MCP (`runtime/mcp_client.py`): `McpManager` owns one `McpServerConnection` per `
### Skills (`skills/`)
Discovery and parsing, independent of the agent: `build_roots(config, workspace=…, user_config_dir=…, home=…)` resolves `[skills]` into ordered `SkillRoot`s (`config` explicit paths → `plyngent` user dir → `claude` user/project dirs), `SkillStore` scans each root for directories with a `SKILL.md` (a root may itself be one), parses the frontmatter subset (`frontmatter.py`), and caches the records against the directory list and its `SKILL.md` timestamps. `skills` returns the visible records sorted by name (first root wins), `shadowed()` the losing copies, `find(name)` exact-then-case-insensitive, `default_write_root()` our own directory (or the first root), `reload()` forces a rescan; `resolve_inside` / `resolve_skill_file` keep every read and write inside a skill directory (absolute paths and symlink escapes are refused). Records carry `issues` (unusable declared name, no description, unparseable frontmatter) so a repaired third-party skill stays visible instead of being silently fixed.
Discovery and parsing, independent of the agent: `build_roots(config, workspace=…, user_config_dir=…, home=…)` resolves `[skills]` into ordered `SkillRoot`s (`config` explicit paths → `plyngent` user dir → `claude` user/project dirs), `SkillStore` scans each root for directories with a `SKILL.md` (a root may itself be one) and parses the frontmatter subset (`frontmatter.py`) on **every** access — nothing is cached, so an edit or a new skill made outside the chat is simply there. `skills` returns the visible records sorted by name (first root wins), `shadowed()` the losing copies, `find(name)` exact-then-case-insensitive, `default_write_root()` our own directory (or the first root); `resolve_inside` / `resolve_skill_file` keep every read and write inside a skill directory (absolute paths and symlink escapes are refused). Records carry `issues` (unusable declared name, no description, unparseable frontmatter) so a repaired third-party skill stays visible instead of being silently fixed.
### Memory (`memory/`)
@@ -105,7 +105,7 @@ Module-level `@tool` handlers. Call `set_workspace_root()` before use.
- **`net` / `fetch`**: HTTP GET/POST/PUT/DELETE via niquests (isolated session, not the LLM client). Manual redirects with per-hop SSRF checks (`asyncio` `getaddrinfo`). Private/loopback hosts need instance policy grant (CLI timed confirm; **not** YOLO). Public cleartext HTTP and mutating methods soft-confirm (`YOLO`/`TRUSTABLE`). HTTPS→HTTP redirects blocked unless `allow_http_downgrade`. `user_agent` tool arg (or headers) is never replaced by the default when provided. Caps: body bytes/chars, request body size.
- **`net` / `web_search`** (`tools/net/search.py`, `tools/net/bing.py`): the hosted search a DeepSeek provider cannot use, as a local tool. `build_web_search_tool(backends)` closes over the sources named by `provider.search_sources` and asks them **in fallback order** (`search_in_order`): `DeepseekSearchBackend` (one `/messages` turn with the server tool, `lmproto/deepseek/anthropic/search.py`) and/or `BingSearchBackend` (Bing's RSS results, snippets included, fetched through the same client as `fetch`). The handler returns one `title — url` entry per hit (plus the snippet when the source has one); a source that raises or finds nothing is skipped and reported only if every source fails. Registered by the CLI host only when `config.routing.local_web_search` says so: a DeepSeek provider whose `provider_tools` ask for `web_search`, on a surface that ignores hosted tools (`convention` other than `anthropic`), with a non-empty `search_sources`. Tags LOCAL|READ_ONLY.
- **`mcp`** (`tools/mcp.py`): register one server's advertised tools into the catalog as namespaced `mcp__<server>__<tool>` definitions with `ToolSource(kind="mcp", plugin_id=server)`; tags LOCAL (+ READ_ONLY when the server config sets `read_only`) — never YOLO/TRUSTABLE. `register_mcp_tools` purges prior MCP entries then re-registers every connected server (hosts call it on tool-registry rebuild); connections must already be started by `McpManager`.
- **`skills`** (`tools/skills.py`): discovery, reading, and writing of skill directories (`plyngent.skills` owns roots, the `SKILL.md` frontmatter subset, the mtime-cached store, and the inside-the-skill path rule). `skill_list` (names, source root, directory, description, shadowed copies, frontmatter notes), `skill_read` (a `SKILL.md` by name, or one bundled file — reuses `read_file` for ranges/line numbers/`truncate_token`), `skill_search` (regex over every visible skill via `regex_files`, hits rebased on `<skill>/<file>`), `skill_create` (new `SKILL.md` + optional `files`), `skill_edit` (replace / `append` / rewrite one file; a rewritten `SKILL.md` is re-parsed and notes a lost description). Reads need no grant (the host publishes the roots as `WorkspacePolicy.static_read`, so `read_file`/`regex_files` see them while writes stay denied); writes go through **`SkillWritePolicy`** on the instance — denylist first, then `SessionState.skill_write_roots` (a `session` answer), then `[skills].allow_write`, then the host's confirm hook (CLI `prompt_skill_write_confirm`: `o`nce/`s`ession/`n`o, timed, default deny, **not** skipped by YOLO); a host with no hook denies. Grants are per **skill directory**; `[skills].enabled = false` drops the tools from the registry (`SKILL_TOOL_NAMES`).
- **`skills`** (`tools/skills.py`): discovery, reading, and writing of skill directories (`plyngent.skills` owns roots, the `SKILL.md` frontmatter subset, the uncached store, and the inside-the-skill path rule). `skill_list` (names, source root, directory, description, shadowed copies, frontmatter notes), `skill_read` (a `SKILL.md` by name, or one bundled file — reuses `read_file` for ranges/line numbers/`truncate_token`), `skill_search` (regex over every visible skill via `regex_files`, hits rebased on `<skill>/<file>`), `skill_create` (new `SKILL.md` + optional `files`), `skill_edit` (replace / `append` / rewrite one file; a rewritten `SKILL.md` is re-parsed and notes a lost description). Reads need no grant (the host publishes the roots as `WorkspacePolicy.static_read`, so `read_file`/`regex_files` see them while writes stay denied); writes go through **`SkillWritePolicy`** on the instance — denylist first, then `SessionState.skill_write_roots` (a `session` answer), then `[skills].allow_write`, then the host's confirm hook (CLI `prompt_skill_write_confirm`: `o`nce/`s`ession/`n`o, timed, default deny, **not** skipped by YOLO); a host with no hook denies. Grants are per **skill directory**; `[skills].enabled = false` drops the tools from the registry (`SKILL_TOOL_NAMES`).
- **`DEFAULT_TOOLS` / catalog**: access + file + process + vcs + chat + todo + skills + net (`fetch`) for local surface. `ToolTag.READ_ONLY` marks non-mutating tools (`read_file`, `get_truncated`, `listdir`, `glob_paths`, `regex_files`, `tree`, `todo_list`, `wait`, `ask_user_*`, `vcs_*`, `skill_list`/`skill_read`/`skill_search`, `fetch`); `ToolRegistry.clone(read_only_only=True)` selects just those for safe side turns / planning, and `read_only_context()` (in `tools/context.py`) makes `fetch` accept GET only.
### Prompting (`prompting.py`)
@@ -117,7 +117,7 @@ Shared interactive I/O: `ask` / `choose` / `form` / `confirm` with pluggable bac
Click app + readline REPL. Entry: `plyngent` / `python -m plyngent`.
- **`plyngent chat`**: provider/model (flags or interactive; Tab via readline in `prompting`); sessions store `provider_name`/`model` and restore on resume; SQLite via `[database]` (file DB under user data when url unset; explicit `url = ":memory:"` kept + warn); workspace-bound; resume latest for cwd/`--workspace` by default (`--new` / `--session`). One-shot: `-p/--prompt` and non-TTY stdin; exit codes 0/1/2/3; `--yes` (YOLO on), `--auto-continue` (auto-raise limits), `--stream/--no-stream`, `--quiet`. Root `--log-level`. Enabled `[mcp]` servers start before the REPL (`_start_mcp_manager`, per-server failures warn on stderr) and are closed on exit (`McpManager.aclose`); each server spawns in its own session/process group (`start_new_session` / `CREATE_NEW_PROCESS_GROUP`), so a terminal Ctrl+C does not kill it.
- Slash: Click group in `cli/slash.py` + `awaitlet` for async work; Tab completer from registry + ParamType `shell_complete`. `/history` is turn-oriented (`cli/transcript.py`): a turn runs from a user row to the next one, rows carry **display ordinals** (never DB row numbers; restart with the session), the turn's ends (user + final answer) always print in full, `-v` expands every row (one-liners), `-vv` prints full bodies + local rows, `--preview` collapses every row, and `N`/`last [N]` address turns while `--message N` addresses one row. Multiline `"""` … `"""`; `/edit` via `$VISUAL`/`$EDITOR` (blocking only). `/yolo on|off|once` for soft destructive confirms. `/model --persist` / `/models --persist` write model catalog entries into TOML. `/todos` for human show/push/pop/clear of the todo stack. `/skills` (`list` default) prints the discovered skills with their roots, shadowed copies, and frontmatter notes; `/skills search PATTERN` runs the matching tool handler (the slash path binds the tool context itself); `/skills open NAME` reveals the skill's directory in the system file manager; `/skills reload` rescans the roots and rebuilds tools. `/mcp` (`list` default) prints per-server status/tool counts + an `instructions` preview when the server sent usage guidance, + last stderr lines; `/mcp reconnect` re-reads config, restarts every enabled server, and rebuilds the tool registry. `/grants` lists directory-access grants (`revoke <index|all>`; config grants point at the TOML); `/status` includes the grant count and the visible skill count (or `off` when `[skills].enabled` is false).
- Slash: Click group in `cli/slash.py` + `awaitlet` for async work; Tab completer from registry + ParamType `shell_complete`. `/history` is turn-oriented (`cli/transcript.py`): a turn runs from a user row to the next one, rows carry **display ordinals** (never DB row numbers; restart with the session), the turn's ends (user + final answer) always print in full, `-v` expands every row (one-liners), `-vv` prints full bodies + local rows, `--preview` collapses every row, and `N`/`last [N]` address turns while `--message N` addresses one row. Multiline `"""` … `"""`; `/edit` via `$VISUAL`/`$EDITOR` (blocking only). `/yolo on|off|once` for soft destructive confirms. `/model --persist` / `/models --persist` write model catalog entries into TOML. `/todos` for human show/push/pop/clear of the todo stack. `/skills` (`list` default) prints the discovered skills with their roots, shadowed copies, and frontmatter notes; `/skills search PATTERN` runs the matching tool handler (the slash path binds the tool context itself); `/skills open NAME` reveals the skill's directory in the system file manager. `/mcp` (`list` default) prints per-server status/tool counts + an `instructions` preview when the server sent usage guidance, + last stderr lines; `/mcp reconnect` re-reads config, restarts every enabled server, and rebuilds the tool registry. `/grants` lists directory-access grants (`revoke <index|all>`; config grants point at the TOML); `/status` includes the grant count and the visible skill count (or `off` when `[skills].enabled` is false).
- `/btw` side questions: forked session; main transcript/DB untouched. `--tools=read` (default) exposes only `READ_ONLY` tools under `read_only_context()` (fetch=GET) with the main session view; `--tools=no` off; `--tools=full` whole-registry clone with fresh session. Tools off: default falls back to no tools; explicit `read`/`full` errors. The model gets an `aside` notice as its last message before the question (unsaved exchange + tool scope), shown to the user as `[notice] aside: …`.
- Explicit `/resume` or `--session` from another workspace prompts: **keep** / **update** / **abort**. A session resumed by a **freshly started process** (`--session` / default latest) also gets a `resume` notice about what a restart drops (PTY sessions, running commands, temp workspaces) — resuming inside a live process pushes nothing.
- Host notices (`agent/notices.py`): system-level messages appended as trailing `developer` rows, never system-prompt edits — `Notice.to_message()` for the model, `.summary` for the user via `cli.display.echo_notice` (`--quiet` hides the user line only). Durable hosts persist them (`ChatAgent.push_notice`); a side agent without memory keeps its notice local. Trailing developer messages never end a turn, so `/retry` still sees an interrupted turn behind a notice or checkpoint.
+1 -1
View File
@@ -303,7 +303,7 @@ Type `/help` in the REPL for the live list. Common ones:
| `/models` | List config + remote `GET /models` (always re-fetches) |
| `/models --persist` | Merge remote catalog into TOML for this provider |
| `/todos` | Todo/task stack: list, push, pop, done, clear |
| `/skills [list\|search\|open\|reload]` | List discovered skills, regex-search them, open one's directory in the system file manager, or rescan the roots |
| `/skills [list\|search\|open]` | List discovered skills, regex-search them, or open one's directory in the system file manager |
| `/grants` | Directory-access grants: list, or `revoke <index\|all>` |
| `/config` | Edit `plyngent.toml` ($VISUAL/$EDITOR or system open); reload after blocking editor |
| `/quit` | Leave the REPL |
+2 -8
View File
@@ -1267,16 +1267,15 @@ def _open_skill_directory(store: SkillStore, name: str) -> None:
@slash.command("skills")
@click.argument("action", required=False, type=click.Choice(["list", "search", "open", "reload"]))
@click.argument("action", required=False, type=click.Choice(["list", "search", "open"]))
@click.argument("target", required=False)
@click.pass_obj
def skills_slash_cmd(state: ReplState, action: str | None, target: str | None) -> None:
"""List skills, search them, open one's directory, or rescan the roots.
"""List skills, search them, or open one's directory.
``/skills`` or ``/skills list`` — discovered skills with their source root.
``/skills search PATTERN`` — regex over every skill's contents.
``/skills open NAME`` — reveal the skill's directory in the file manager.
``/skills reload`` — rescan the roots after an external edit.
"""
from plyngent.tools.context import bind_tool_context
from plyngent.tools.skills import SKILL_TOOL_NAMES, format_skill_table, skill_search
@@ -1285,11 +1284,6 @@ def skills_slash_cmd(state: ReplState, action: str | None, target: str | None) -
click.echo("skills: disabled (set [skills].enabled = true)")
return
act = (action or "list").lower()
if act == "reload":
state.reload_skills()
_slash_rebuild_tools_if_on(state)
click.echo(f"skills reloaded: {len(state.skills.skills)} visible; tools rebuilt")
return
if act == "open":
if not target:
msg = "/skills open requires a skill name"
-5
View File
@@ -343,11 +343,6 @@ class ReplState:
self.instance_state.skills = self.skills
_ = set_static_read_roots([root.path for root in roots], instance=self.instance_state)
def reload_skills(self) -> None:
"""Rescan the roots (after an external edit or a config change)."""
if self.skills is not None:
self.skills.reload()
def _skills_text(self) -> str:
"""Fold the discovered skills into the system prompt (like MCP instructions).
+16 -36
View File
@@ -3,9 +3,8 @@
A skill is a directory holding a ``SKILL.md``. Roots are ordered by priority
(explicit ``[skills].paths`` first, then the built-in sources), so the first
skill with a given name wins and the copies it shadows stay visible for
reporting rather than disappearing. Parsed metadata is cached against the
directory list and its ``SKILL.md`` timestamps, which makes a listing cheap
while still noticing an edit made outside the chat.
reporting rather than disappearing. Nothing is cached: every lookup reads the
filesystem, so an edit or a new skill made outside the chat is simply there.
"""
from __future__ import annotations
@@ -119,11 +118,10 @@ def _expand(token: str, *, home: Path | None) -> Path:
@dataclass(frozen=True, slots=True)
class _Entry:
"""One candidate skill directory plus the timestamp that invalidates the cache."""
"""One candidate skill directory under its root."""
root: SkillRoot
directory: Path
mtime_ns: int
def validate_skill_name(name: str) -> str | None:
@@ -166,14 +164,16 @@ def resolve_skill_file(skill: Skill, relative: str) -> Path | None:
class SkillStore:
"""Skills discovered under *roots*, highest-priority root first."""
"""Skills discovered under *roots*, highest-priority root first.
Each access re-reads the filesystem: the roots are a fixed list, the skills
under them are not.
"""
_roots: tuple[SkillRoot, ...]
_cache: tuple[tuple[tuple[str, int], ...], list[Skill]] | None
def __init__(self, roots: Sequence[SkillRoot]) -> None:
self._roots = tuple(roots)
self._cache = None
@property
def roots(self) -> tuple[SkillRoot, ...]:
@@ -182,20 +182,20 @@ class SkillStore:
@property
def all_skills(self) -> list[Skill]:
"""Every discovered skill, shadowed copies included (priority order)."""
return list(self._load())
return list(self._discover())
@property
def skills(self) -> list[Skill]:
"""Visible skills by name, sorted for display (shadows removed)."""
by_name: dict[str, Skill] = {}
for skill in self._load():
for skill in self._discover():
_ = by_name.setdefault(skill.name, skill)
return [by_name[name] for name in sorted(by_name)]
def shadowed(self) -> dict[str, list[Skill]]:
"""Skill names found in more than one root → the losing copies in order."""
found: dict[str, list[Skill]] = {}
for skill in self._load():
for skill in self._discover():
found.setdefault(skill.name, []).append(skill)
return {name: skills for name, skills in sorted(found.items()) if len(skills) > 1}
@@ -204,7 +204,7 @@ class SkillStore:
token = name.strip()
wanted = token.casefold()
fallback: Skill | None = None
for skill in self._load():
for skill in self._discover():
if skill.name == token:
return skill
if fallback is None and skill.name.casefold() == wanted:
@@ -218,25 +218,12 @@ class SkillStore:
return root
return self._roots[0] if self._roots else None
def reload(self) -> None:
"""Drop the cache so the next access rescans the filesystem."""
self._cache = None
def _load(self) -> list[Skill]:
entries = self._scan()
signature = tuple((str(entry.directory), entry.mtime_ns) for entry in entries)
if self._cache is not None and self._cache[0] == signature:
return self._cache[1]
skills = [skill for entry in entries if (skill := _read_skill(entry)) is not None]
self._cache = (signature, skills)
return skills
def _discover(self) -> list[Skill]:
"""Read every candidate ``SKILL.md`` now — nothing is cached."""
return [skill for entry in self._scan() if (skill := _read_skill(entry)) is not None]
def _scan(self) -> list[_Entry]:
return [
_Entry(root=root, directory=directory, mtime_ns=_skill_mtime(directory))
for root in self._roots
for directory in _skill_dirs(root.path)
]
return [_Entry(root=root, directory=directory) for root in self._roots for directory in _skill_dirs(root.path)]
def _skill_dirs(root: Path) -> list[Path]:
@@ -252,13 +239,6 @@ def _skill_dirs(root: Path) -> list[Path]:
return found
def _skill_mtime(directory: Path) -> int:
try:
return (directory / SKILL_FILE).stat().st_mtime_ns
except OSError:
return -1
def _read_skill(entry: _Entry) -> Skill | None:
"""Read one ``SKILL.md`` head and derive the record; ``None`` when unreadable."""
issues: list[str] = []
-2
View File
@@ -465,7 +465,6 @@ async def skill_create(
_write_file(path, content)
except OSError as exc:
return f"error: failed to create skill {token!r}: {exc}"
store.reload()
return f"created skill {token!r} at {target}{_skill_notes(target / SKILL_FILE)}"
@@ -571,7 +570,6 @@ async def skill_edit(
_write_file(target, updated)
except OSError as exc:
return f"error: failed to write {relative} of skill {skill.name!r}: {exc}"
store.reload()
notes = _skill_notes(target) if target.name == SKILL_FILE else ""
return f"updated skill {skill.name!r} ({detail}){notes}"
+2 -3
View File
@@ -161,7 +161,7 @@ async def test_slash_skills_open_reports_an_unknown_skill(
assert "unknown skill 'nope'" in capsys.readouterr().out
async def test_slash_skills_reload_picks_up_a_new_skill(
async def test_slash_skills_sees_a_skill_added_while_chatting(
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
) -> None:
@@ -172,8 +172,7 @@ async def test_slash_skills_reload_picks_up_a_new_skill(
assert await handle_slash(state, "/skills") is True
assert "skills: 0" in capsys.readouterr().out
_ = _write_skill(root, "added-later")
assert await handle_slash(state, "/skills reload") is True
assert "skills reloaded: 1 visible" in capsys.readouterr().out
# No reload: the next listing reads the filesystem.
assert await handle_slash(state, "/skills") is True
assert "added-later" in capsys.readouterr().out
finally:
+5 -5
View File
@@ -144,21 +144,21 @@ def test_broken_frontmatter_still_yields_a_listable_skill(tmp_path: Path) -> Non
assert any("frontmatter not parsed" in issue for issue in skill.issues)
def test_cache_picks_up_an_edit_and_reload_forces_a_rescan(tmp_path: Path) -> None:
def test_every_lookup_reads_the_filesystem(tmp_path: Path) -> None:
"""No cache: an edit or a skill added outside the chat is simply there."""
root = tmp_path / "skills"
directory = _write_skill(root, "cached", description="Before")
directory = _write_skill(root, "live", description="Before")
store = SkillStore(build_roots(SkillsConfig(discover=["plyngent"]), workspace=tmp_path, user_config_dir=tmp_path))
assert store.skills[0].description == "Before"
_ = (directory / SKILL_FILE).write_text(
"---\nname: cached\ndescription: After\n---\n\nBody.\n",
"---\nname: live\ndescription: After\n---\n\nBody.\n",
encoding="utf-8",
)
assert store.skills[0].description == "After"
_ = _write_skill(root, "added")
store.reload()
assert [s.name for s in store.skills] == ["added", "cached"]
assert [s.name for s in store.skills] == ["added", "live"]
def test_default_write_root_prefers_our_own_directory(tmp_path: Path) -> None: