Commit Graph

17 Commits

Author SHA1 Message Date
Ashvin ba43c73d2a fix(agent): confine glob literal lookups to the search root (#5010)
GlobTool resolves its search root through _resolve_search_root (which
confines it to the workspace or default allowlist), but the literal
fast-path joined the model-supplied pattern onto that root without
re-confining it. os.path.join lets an absolute pattern or one containing
../ escape the root, and normpath collapsed the .. segments, so glob
returned the absolute path of arbitrary host files once they existed --
an existence/path oracle that bypasses the confinement read_file,
write_file, grep, and ls all enforce.

Keep the literal lookup inside the root via a commonpath containment
check; an escaping literal falls through to the os.walk matcher, which
only ever yields paths under the root. Wildcard matching was already
confined.
2026-06-30 17:49:53 +01:00
Michael 3d75fad52f fix(security): apply sensitive-file deny-list to grep tool (#5011) (#5013)
The grep tool bypassed the sensitive-file deny-list that read_file,
write_file, and edit_file all respect. Two code paths fixed:

1. ripgrep path: adds --glob exclusion patterns for each entry in
   _SENSITIVE_FILE_PATTERNS (id_rsa, known_hosts, authorized_keys, etc.)
2. Pure-Python os.walk fallback: checks _is_sensitive_path() before
   opening each file, skipping files that match the deny-list

Fixes #5011

Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
2026-06-30 17:44:45 +01:00
botinate 69b9bb0869 fix(agent): execute fenced tool calls with inline args and route bare email tool names (#3681)
* fix(agent): execute fenced tool calls with inline args and bare email tool names

Two bugs made local (Ollama) models unable to use email tools, leaving
raw fences like ```list_email_accounts {}``` in the chat:

1. _TOOL_BLOCK_RE required a newline right after the fence tag, so a
   tool call with args on the same line ("```list_email_accounts {}")
   never matched and was never executed. The fence now matches with
   optional spaces/newline after the tag.

2. Even when parsed, bare email tool names had no dispatch branch in
   tool_execution.py and fell through to "Unknown tool type". They now
   route to the email MCP server as mcp__email__<name>, matching how
   function_call_to_tool_block already maps them for native callers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): block all bare email tool names for non-admins; harden fence-tag regex

Review follow-up on #3681 (thanks @vgalin):

1. Routing bare email names made 10 of the 14 email tools executable by
   non-admin owners — is_public_blocked_tool() runs on the bare name
   before dispatch, and NON_ADMIN_BLOCKED_TOOLS only listed 4. Define the
   full email tool set once (BUILTIN_EMAIL_TOOLS in tool_security.py) and
   derive the blocklist, the fence tags (TOOL_TAGS), the bare-name
   dispatch, and the native-call mapping from it so they can't drift.
   This also fixes 4 tools (search_emails, draft_email, draft_email_reply,
   ai_draft_email_reply) that were missing from the old tool_schemas copy
   and therefore unreachable even for native function-calling models.

2. The relaxed fence regex from the previous commit could prefix-match
   longer fence tags: ```python3 parsed as tool "python" with content
   "3\nprint(...)" and executed as code. Add a (?![\w-]) boundary after
   the tag.

Tests: test_public_agent_policy_blocks_sensitive_tools now covers all 14
bare email names + the mcp__email__ form; new tests/test_fenced_inline_args.py
pins inline-args parsing, the python3/hyphenated-tag non-matches, and
strip/parse display mirroring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(security): gate bare and mcp-qualified email names together; stop executing Markdown info strings

Review follow-up on #3681 (thanks @RaresKeY):

1. P1: execute_tool_block() checked disabled_tools / the turn ToolPolicy
   only against the incoming block name, then the bare-email branch
   qualified it to mcp__email__<name> and called the MCP manager. Plan
   mode and the MCP settings toggle write the QUALIFIED name into the
   denylist, so a bare fence like ```list_emails``` sailed past a
   mcp__email__list_emails entry. Both gates now match on both
   spellings (bare <-> mcp__email__-qualified), in either direction.

2. P2: the relaxed fence regex accepted arbitrary same-line text after
   a recognized tag, which made ordinary Markdown info strings
   executable: ```python title="example.py" ran as a python tool call.
   Same-line content now only counts as tool input when it starts with
   { or [ (JSON args); anything else leaves the fence as display text,
   and strip_tool_blocks mirrors that (the fence stays visible).

Tests: disabled-tools alias regression (qualified entry blocks bare
name and vice versa, never reaching the MCP manager), ToolPolicy alias
regression, python/bash title="..." non-execution + display retention,
and inline JSON-array args still parsing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(security): reject brace-style fence metadata; cover the full email set in the friendly toggle

Review follow-up round 3 on #3681 (thanks @RaresKeY):

1. Brace-style fence metadata no longer executes. The previous narrowing
   still treated any same-line {/[ after a recognized tag as tool input,
   so ```bash {title="setup"} ran as a bash call. The fence header is now
   captured separately and judged by one predicate shared between
   parse_tool_blocks and strip_tool_blocks (_fenced_tool_call), so the
   execute and display decisions can't disagree: same-line content only
   counts as inline args when the tag is NOT a code tag (bash/python
   never take same-line args — that text is Markdown fence attributes)
   AND the inline text (plus any continuation lines) parses as standalone
   JSON. ```bash {title="setup"}, ```python {"title":"example.py"} and
   ```list_emails {title="x"} all stay visible and inert.

2. The friendly `disable_tool email` toggle covered 3 of the 14 email
   tools (mcp__email__{list_emails,read_email,send_email}); the other
   bare aliases this PR routes stayed executable after an operator
   disabled email. The alias now derives from BUILTIN_EMAIL_TOOLS in
   BOTH spellings — bare (function-schema hiding, bare-fence dispatch)
   and mcp__email__* (MCP schema hiding, qualified runtime blocks) —
   so the toggle and the runtime gate can't drift apart.

Tests: brace/bracket metadata regressions for parse and strip symmetry
(code tags, invalid-JSON inline on a JSON tool, multi-line inline JSON
still parsing), and disable_tool/enable_tool email covering all 14 names
in both spellings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(email): close remaining email-tool registry drift; classify every email tool for plan mode

Deep self-review follow-up on #3681. Three review rounds each found another
hand-maintained copy of the email tool list that had drifted; this commit
hunts down ALL remaining copies and pins them to BUILTIN_EMAIL_TOOLS.

The same 5 tools (search_emails, draft_email, draft_email_reply,
ai_draft_email_reply, download_attachment) were missing from every
advertising surface, so they were dispatchable but never offered:

- FUNCTION_TOOL_SCHEMAS: native function-calling models never saw them
  (the round-1 fix covered dispatch only); schemas added, mirroring the
  email server's inputSchema definitions.
- TOOL_SECTIONS: fenced-block models were never told about them; prompt
  sections added.
- tool_index: absent from the RAG embedding registry (never retrievable),
  the email keyword hints, and the scheduled assistant's always-available
  set — the latter two now derive from BUILTIN_EMAIL_TOOLS.
- agent_loop._DOMAIN_TOOL_MAP["email"], tool_policy._COMMON_TOOL_NAMES,
  the assistant tool-selector UI groups (assistant.js), and the default
  Assistant crew seed (task_scheduler) now derive from / cover the set.

Plan mode now classifies every email tool explicitly:

- list_email_accounts and search_emails join PLAN_MODE_READONLY_TOOLS.
  Without this, list_email_accounts sat in the plan-mode bare denylist
  (schema-derived) while its qualified form passed the MCP read-only
  filter — and the round-2 bare/qualified alias gate would have blocked
  the qualified call too, regressing read-only email discovery in plan
  mode.
- draft_email, draft_email_reply, ai_draft_email_reply, and
  download_attachment join the fail-closed mutator backstop (drafts
  create documents; download_attachment writes to disk).

Tests: tests/test_email_registry_sync.py pins every registry (including
the email server source and assistant.js) to BUILTIN_EMAIL_TOOLS and
asserts the plan-mode partition, so the next email tool can't drift; a
parse/strip mirror grid covers 192 fence shapes (tag x header x body)
asserting executed <=> stripped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: move the email alias rule into tool_security; extract the assistant seed constant

Code-quality pass over the PR's own changes:

- The bare<->qualified email aliasing rule lived inline in the generic
  dispatcher (_execute_tool_block_impl). It is policy knowledge, so it
  moves next to BUILTIN_EMAIL_TOOLS as email_tool_policy_names(); the
  dispatcher just consumes it, and the rule gets its own unit test
  (including the mcp__email__<not-a-tool> and mcp__other__ non-alias
  cases).

- The default Assistant's enabled_tools list was an inline literal
  inside the CrewMember seed, and its registry-sync test asserted a
  source-code substring. Extracted to DEFAULT_ASSISTANT_ENABLED_TOOLS
  so the test imports and checks the actual value.

- _fenced_tool_call return type tightened to Optional[Tuple[str, str]].

No behavior change; suite green (3295 passed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* revert: move the email registry consolidation to a follow-up PR

Per review feedback on scope, this PR stays narrow: fenced inline-args
parsing, bare email tool routing, and the directly required safety
gates. This commit reverts the registry/advertising consolidation from
db29046 and 016ce47 (native schemas, prompt sections, RAG description
index + keyword hints, assistant always-available set, guide-only
known-names union, frontend tool-selector groups, default assistant
seed, and their sync tests) — all of that moves to a dedicated
follow-up PR together with the _EMAIL_TOOL_HINTS finding.

Kept here because the narrow scope needs them:
- email_tool_policy_names() in tool_security + its use in the
  execute_tool_block gates and its unit test (refactor of this PR's own
  round-2 alias fix),
- list_email_accounts in PLAN_MODE_READONLY_TOOLS (the alias gate works
  both ways, and the schema-derived plan-mode bare denylist would
  otherwise block the qualified read-only call too),
- the parse/strip mirror grid test (parser scope),
- the narrow registry sync tests (email server <-> BUILTIN_EMAIL_TOOLS
  match, fence-tag coverage, non-admin blocklist coverage).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(email): execute empty email fences with empty args; reject non-object JSON args

Two gaps found by replaying captured local-model traffic against the
narrowed branch:

1. ```list_email_accounts``` with NO body — a shape gemma really emits
   for no-arg tools — was silently dropped (parse skips empty content),
   so the model concluded email was broken: the original #337 symptom
   through a different door. Empty fences whose tag is a built-in email
   tool now dispatch with {} args and the tool's own validation answers
   (e.g. an empty send_email returns "to is required" instead of
   silence). Empty bash/python/other fences keep skipping, and strip
   stays mirrored (the fence was executed, so it is removed).

2. The fence parser accepts JSON arrays as inline args, but the email
   dispatch parsed only objects — an array silently became {} args.
   Non-object JSON now returns a correctable "arguments must be a JSON
   object" error before reaching the MCP server (same class as #3966).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(security): classify all email tools for plan mode statically; reject invalid email JSON bodies

Review follow-up round 5 on #3681 (thanks @RaresKeY):

1. This PR makes every BUILTIN_EMAIL_TOOLS name fence-taggable, so each
   one must be explicitly classified for plan mode — the draft tools and
   download_attachment were in neither the read-only allowlist nor the
   static denylist, leaving their bare-alias plan-mode safety dependent
   on the MCP read-only inventory being present and current.
   search_emails joins PLAN_MODE_READONLY_TOOLS (explicit, not
   allowed-by-omission); draft_email, draft_email_reply,
   ai_draft_email_reply, and download_attachment join the fail-closed
   _PLAN_MODE_KNOWN_MUTATORS backstop. (Moved back from the #4053 split:
   the partition is directly required for this PR to merge
   independently.)

2. The classic tag/body fence form reaches execution unvalidated (only
   INLINE args are JSON-checked by the parser), so a body like
   {account: "work"} silently became {} args and read the DEFAULT
   mailbox instead of the intended one. JSON-looking bodies that fail to
   parse now return a correctable "not valid JSON" error before reaching
   the MCP server.

Tests: a partition invariant (every email tool is explicitly read-only
or plan-mode-denied), a mutating-alias probe that uses only the static
denylist with a fake MCP manager (no inventory layer), and the
body-form invalid-JSON regression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tool-dispatch): decode inline JSON args for legacy MCP tools; reject all non-object email bodies

Review follow-up round 6 on #3681 (thanks @RaresKeY) — both pre-existing
on this branch, surfaced by the relaxed inline-args parser:

1. The relaxed parser accepts inline JSON for every non-code tag, but
   the legacy line-based arg builders (web_search/web_fetch/read_file/
   write_file/generate_image/manage_memory) wrapped the whole JSON
   string as the query/url/path/prompt — so `web_search {"query": "x"}`
   executed as a search for the literal string `{"query": "x"}`.
   _build_mcp_args now uses a fenced JSON object directly when it carries
   the tool's primary arg key (query/url/path/prompt/action). Keyed off
   membership so it can't drift; an object without the primary key (e.g.
   a freeform JSON query, or bare object content for write_file) falls
   through to the line parser unchanged. Also fixes the same corruption
   for the classic newline-JSON form.

2. The bare-email dispatch only rejected bodies starting with { or [, so
   a non-empty non-JSON body like `account: work` still fell through to
   {} args and silently read the DEFAULT mailbox. Now ANY non-empty body
   must decode to a JSON object or it returns a correctable error; only a
   truly empty body keeps the no-arg path (```list_email_accounts```).

Tests: inline-JSON arg decoding for the five legacy tools plus the
freeform and missing-primary-key fallbacks; the email body rejection
extended to cover the brace-looking and bare `key: value` shapes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tool-dispatch): drop dead manage_memory JSON-decode entry; pin the live-path invariant

Self-audit catch on the round-6 fix. manage_memory was added to
_MCP_JSON_PRIMARY_KEYS, but _build_mcp_args is only reached via
_call_mcp_tool, which only runs for _MCP_TOOL_MAP tools — and
manage_memory isn't one (its tag routes through dispatch_ai_tool ->
do_manage_memory, which line-parses). So the round-6 decode for
manage_memory was dead code: the unit test exercising _build_mcp_args
passed while a real `manage_memory {"action": ...}` fence still parsed
the whole JSON blob as the action.

Remove the dead entry and add test_mcp_json_primary_keys_are_all_live,
which asserts every JSON-primary tool is in _MCP_TOOL_MAP so a dead
decode can't be added again. The same inline-JSON corruption for
manage_memory and the other tools that route through positional
dispatchers (create_session, ui_control, send_to_session, search_chats,
the document tools, etc.) is pre-existing (dev corrupts their newline
JSON form too) and tracked separately; the proper fix there is to route
fenced JSON through function_call_to_tool_block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tool-dispatch): decode inline JSON in WriteFileTool (its live path); round-6 fix was on the dead MCP path

Self-audit: round 6 claimed to fix inline JSON args for write_file via
_build_mcp_args, but there is no filesystem MCP server, so write_file
always runs through _direct_fallback -> WriteFileTool, never through
_build_mcp_args. WriteFileTool — unlike its siblings ReadFileTool /
WebSearchTool / WebFetchTool, which all decode JSON — took lines[0] as
the path, so `write_file {"path": "/tmp/x", "content": "y"}` wrote to a
file literally named with the JSON blob. The round-6 _build_mcp_args
entry decoded correctly but on a path that never executes (same class
as the manage_memory dead entry), and the round-6 unit test passed on
that dead path.

WriteFileTool now decodes a JSON object carrying "path" (matching
ReadFileTool directly above it), and the comment on _MCP_JSON_PRIMARY_KEYS
records that only generate_image has a live MCP server today — the other
entries are defense-in-depth for the MCP path; the live fix for each
server-less tool is in its handler.

Test: test_write_file_inline_json_args drives the LIVE path
(execute_tool_block with no MCP) and asserts the intended path is used —
verified to fail without the handler fix. web_search/web_fetch/read_file
were already correct (their handlers decode); write_file was the gap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(strip-fence): derive the live-strip TOOL_TAGS from the real set

Semantic conflict from the dev merge that textual auto-merge didn't flag:
dev added test_live_strip_email_tool_fences.py whose _tool_tags() helper
source-scrapes only the TOOL_TAGS literal `{...}`, which worked on dev
because the email tool names were listed inline there. This branch makes
TOOL_TAGS the single source — `{...} | BUILTIN_EMAIL_TOOLS` — so the email
names are no longer in the literal and the scraper missed them, leaving the
email-fence strip assertions failing even though TOOL_TAGS does contain them
at runtime.

Import the real TOOL_TAGS instead of scraping source, so the test mirrors
exactly what GET /api/tools serves (sorted(TOOL_TAGS)) and the live
EXEC_FENCE_RE derives from — robust to however the set is composed. The
source-level frontend/route guards in the same file are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: botinate <285686135+botinate@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 16:50:32 +01:00
tanmayraut45 c01c09559a fix(ai): offload model resolution from async paths
Wrap blocking _resolve_model calls in asyncio.to_thread across async model interaction paths so endpoint/model resolution does not stall the event loop. Preserve owner-scoped resolution and add focused regression coverage.
2026-06-28 00:48:35 +01:00
muhamed hamed 3e7af8634f fix: improve uploaded document retrieval and deep research reuse (#4784)
* fix: improve uploaded document retrieval and deep research reuse

* test: add coverage for upload manifest and document pagination

* chore: rerun CI

* fix: restore _insert_before_latest_user helper

* fix(agent_loop): restore missing upload context helper
2026-06-27 19:24:17 +01:00
Dewangga Abdullah 6d429a49b9 refactor(tools): register update_plan tool and support dynamic execution (#4069)
* refactor(tools): register update_plan tool and support dynamic execution

* refactor: move interaction tools to registry and fix tuple unpacking error

* docs: add HACK comment for circular dependency workaround

Signed-off-by: dewanggaabdullah <255674162+dewanggaabdullah@users.noreply.github.com>

* refactor(tools): use docstring for better code style

Signed-off-by: dewanggaabdullah <255674162+dewanggaabdullah@users.noreply.github.com>

* fix(tools & file): restore file tool_registry & unknown tool fallback and fix dynamic handlers unpacking

Signed-off-by: dewanggaabdullah <255674162+dewanggaabdullah@users.noreply.github.com>

---------

Signed-off-by: dewanggaabdullah <255674162+dewanggaabdullah@users.noreply.github.com>
2026-06-27 17:36:10 +02:00
Kenny Van de Maele 5ce2056521 refactor(tools): migrate config/integration admin tools to the registry (#4742)
Part of #3629 (the `admin_tools.py` bullet). Moves the config/integration admin
tools off the legacy elif dispatch chain in tool_implementations.py onto the
agent_tools registry:

  manage_endpoints, manage_mcp, manage_webhooks, manage_tokens, manage_settings

The do_* implementations (and manage_mcp's command-allowlist / RCE guard:
_validate_mcp_command, _mcp_allowed_commands, and the _MCP_* constants) move
verbatim into the new src/agent_tools/admin_tools.py. They register through a
single ADMIN_TOOL_HANDLERS map that TOOL_HANDLERS.update()s, and the five elif
branches plus their imports are dropped from tool_execution.py, so these tools
now flow through _direct_fallback like the other migrated clusters. The names
are re-exported from src.agent_tools for back-compat.

Dedup:
  - _parse_tool_args was duplicated in tool_implementations.py and
    document_tools.py. It now lives once in src.tool_utils (which imports nothing
    from the project beyond src.constants, so this introduces no cycle) and both
    call sites import it from there. The orphaned `import json` in document_tools
    is removed with it.
  - The five tools share one _owner_adapter(fn) factory that threads ctx["owner"]
    into the owner-taking do_* signature, instead of five near-identical wrappers.

Tests: new tests/test_admin_tools_registry.py pins the registration, the
re-export back-compat, the owner-threading adapter, and the single-source
_parse_tool_args (across admin_tools and document_tools). Existing MCP /
settings / webhook suites are repointed at the new module.
2026-06-24 09:29:10 +02:00
pewdiepie-archdaemon 75f04bc088 Merge origin/dev into main 2026-06-21 11:08:50 +00:00
pewdiepie-archdaemon c504214925 Cookbook model workflow fixes 2026-06-21 11:02:35 +00:00
Kenny Van de Maele ed18192a8e refactor(tools): move session tools to the agent_tools registry (#4454)
Moves create_session, list_sessions, send_to_session and manage_session out of
ai_interaction.py into src/agent_tools/session_tools.py (the do_ prefix
dropped) and registers them in TOOL_HANDLERS, so dispatch flows through the
registry instead of the dispatch_ai_tool elif in tool_execution.py. Same
pattern as the model-interaction move.

The bodies move verbatim; each fetches the runtime-set session manager via a
get_session_manager() shim, and reuses _resolve_model / AI_CHAT_TIMEOUT from
ai_interaction. manage_session's internal 'list' alias is repointed from the
old do_list_sessions to the moved list_sessions. stream_ai_tool (dead, no
callers) and do_pipeline stay put. dispatch_ai_tool loses its four now-unused
branches.

Tests: test_session_tools_registry covers registration, owner threading, the
manage_session->list_sessions delegation, graceful no-manager handling, and
registry dispatch. Verified end-to-end against a live SessionManager.
2026-06-19 11:55:22 +02:00
Kenny Van de Maele cdae9879f2 feat(agent): add manage_bg_jobs tool to inspect and kill background bash jobs (#4577)
Detached bash jobs (#!bg) could be launched and auto-reported on completion,
but the agent had no way to act on a running one: no on-demand output read and
no kill (it blocked until the 1h max-runtime). bg_jobs had the pieces
(_read_output, list_for_session, internal _kill) but none was exposed.

Adds:
- bg_jobs.kill(job_id): tears down the process tree, marks the job killed, and
  sets followed_up so the monitor does not also auto-continue a deliberate kill.
- manage_bg_jobs registry tool with actions list / output / kill, scoped to the
  chat that launched the job (cross-session access reads as not found).
- Wiring: TOOL_HANDLERS/TAGS, function schema, RAG index + keyword hints, parser
  name map, dispatch (threads session_id via _direct_fallback). Gated like bash
  (NON_ADMIN_BLOCKED_TOOLS; plan-mode mutator).
- agent_loop: background-job intent regex maps to the files domain (and the tool
  joins _DOMAIN_TOOL_MAP[files]) so short commands like 'kill that job' are not
  dropped by the low-signal gate that skips tool retrieval.
- bg launch message tells the model to call manage_bg_jobs itself for check/stop
  rather than printing raw tool syntax to the user.

Tests: tests/test_bg_job_tools.py (kill semantics, per-chat scoping, actions,
and the intent classifier).
2026-06-19 00:28:22 -07:00
Michael 39a802bea2 fix(tools): prune skipped dirs before descending in glob tool (#4538)
* fix(tools): prune skipped dirs before descending in glob tool

GlobTool used pathlib.Path.rglob which descends into every directory
(including node_modules, .git, dist, etc.) and filters AFTER the walk.
On repos with large junk directories this causes the glob tool to hang
for minutes.

Replace rglob with os.walk that prunes _CODENAV_SKIP_DIRS before
descending — matching the approach GrepTool already uses. Also add a
fast path for literal patterns (no wildcards → direct path lookup).

Fixes #4493

* fix(tools): use regex glob matching to fix * semantics and literal fallback

Replace fnmatch with _glob_to_regex so that * stays within a single
path segment (matching pathlib/rglob semantics) and **/ spans zero or
more directories.  Literal patterns now fall through to os.walk when
the direct path lookup misses, so e.g. 'foo.py' still finds files at
any depth.

Add tests for:
- bare literal matching in subdirectories
- multi-segment single-star patterns (sub/*.txt)
- * not crossing / boundaries
- ** matching at arbitrary depth

Closes #4493

---------

Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
2026-06-18 22:02:29 +02:00
Kenny Van de Maele 56ba144875 refactor(tools): move model-interaction tools to the agent_tools registry (#4445)
Moves chat_with_model, ask_teacher and list_models out of ai_interaction.py
into src/agent_tools/model_interaction_tools.py (the do_ prefix dropped) and
registers them in TOOL_HANDLERS, so dispatch flows through the registry instead
of the dispatch_ai_tool elif in tool_execution.py.

The implementations are relocated, not wrapped. ai_interaction.py keeps only
the shared helpers they reuse (_resolve_model, AI_CHAT_TIMEOUT), still used by
the not-yet-migrated session/pipeline tools. dispatch_ai_tool loses its three
now-unused branches.

Also removes the dead do_second_opinion: it was already off the live tool
surface (no tag/schema/parsing/dispatch; tool_index.py notes it was removed),
so the function and its stale frontend catalog entries (admin.js, assistant.js)
are deleted.

Tests: owner-scope test points at the new list_models location and drops the
moved tools from the dispatch_ai_tool parametrize; a new
test_model_interaction_registry covers registration, owner threading, and
registry dispatch.
2026-06-18 05:56:37 +00:00
Kenny Van de Maele 074a1e6eff fix(search): add download budgets to web_fetch with truncation notice and hard ceiling (#3955)
* fix(search): add download budgets to web_fetch with truncation notice and hard ceiling

MAX_OUTPUT_CHARS only trims what the agent sees; fetch_webpage_content
buffered and cached the entire response body first, so a large or hostile
URL could pull arbitrarily many bytes into memory and the content cache.

The fetch is now a capped streaming GET (SSRF redirect guard unchanged):
a soft default budget (WEB_FETCH_SOFT_MAX_BYTES, 2 MB), a per-call
override via full/max_bytes on the web_fetch tool, and a hard ceiling
(WEB_FETCH_HARD_MAX_BYTES, 20 MB) that the override can never exceed.
When Content-Length already declares a body over the ceiling the fetch
is refused before any body bytes are buffered. Truncated results carry
truncated/fetched_bytes/total_bytes, the tool output leads with a
partial-content notice telling the model how to re-fetch with full=true,
and the tool schema documents the flag. A truncated PDF is reported as
a budget error since a cut PDF is unparseable. The effective cap is part
of the content-cache key so a truncated fetch is never served to a
full-budget request.

Existing tests that faked httpx.get or the old _get_public_url signature
are adapted to the streaming interface; behavior pins are unchanged.

Fixes #3812

* fix(search): close compressed-body cap bypass and protect the partial notice

Addresses RaresKeY's review on #3955:

- Force Accept-Encoding: identity for the capped fetch. With gzip/deflate the
  wire bytes (and Content-Length) can be a fraction of the decoded body, so a
  tiny compressed response could pass the hard-cap preflight and then expand
  past the ceiling in a single decoded chunk before the streamed cap could
  slice it. Identity makes Content-Length the true body size and keeps each
  streamed chunk bounded by the network read, so the hard ceiling actually
  bounds memory.
- Lead web_fetch output with the partial-content notice and cap the page
  title. The notice is the user-facing contract for partial fetches, but the
  title is untrusted, uncapped page content; placed ahead of the notice a giant
  title could push it past MAX_OUTPUT_CHARS and drop it. The notice now leads
  and the title is capped as a second guard.

Adds regressions: the fetch advertises identity encoding, and a truncated
result with an oversized title still surfaces the partial notice.

* fix(search): reject compressed responses that ignore the identity request

Requesting Accept-Encoding: identity is not enough on its own: a server can
ignore it and still return Content-Encoding: gzip, and httpx.iter_bytes would
decode that, so a tiny compressed body could balloon into one decoded chunk
far past the hard cap before the streamed loop slices it (and Content-Length,
the compressed wire length, makes the preflight and size metadata unreliable).

Refuse a non-identity Content-Encoding before reading the body. Adds a
regression where the server ignores the identity request and returns gzip;
the fetch is refused before any body is decoded.
2026-06-15 17:38:09 +00:00
Kenny Van de Maele 620fdd0859 feat(agent): confine agent file/shell tools to a selectable workspace (#3665)
* feat(agent): workspace confinement via context-local binding + get_workspace tool

Bind the per-turn workspace once in execute_tool_block; the shared path
resolvers (_resolve_tool_path / _resolve_search_root) and the subprocess cwd
helper (agent_cwd) read it, so file tools + bash/python are confined centrally
and a new tool that uses the shared helpers cannot accidentally bypass it.

Adds the admin-gated /api/workspace/browse picker, a workspace pill + directory
modal (reusing existing modal/button CSS), the /workspace slash command, and a
get_workspace tool (replaces a system-prompt block). Confinement is OS-agnostic
(realpath/normcase/commonpath) and docker-safe (container paths, no host
assumptions). Reopens #2023.

* ux(workspace): clarify workspace is not a sandbox

Picker modal note + pill tooltip + get_workspace tool/output wording now state
plainly: read_file/write_file/edit_file/grep/glob/ls are confined to the folder,
but bash/python only start there (cwd) and are not sandboxed. Modal note reuses
the existing .muted class.

* fix(agent): treat an active workspace as file-work intent

A vague low-signal message (e.g. "look at the local project") matches no
domain keywords, so tool retrieval is skipped and only always-available tools
are offered — leaving the agent with no file access even though a workspace is
set. When a workspace is active, include the file/code tools (incl.
get_workspace) on low-signal turns so the agent can act on the folder.

Also requires the tool index (ChromaDB) to be reachable for normal retrieval;
that is an environment dependency, not part of this change.

* ux(workspace): hide pill + overflow entry in chat mode

Workspace only scopes the agent's file/shell tools, so the pill and the
overflow 'Workspace' entry are agent-only now — hidden in chat mode like the
bash toggle. Mode read from the DOM in syncWorkspaceIndicator; applyMode() is
called from the agent/chat setMode handler.

* prompt(tools): steer bash/python to defer to the dedicated file tools

bash/python schema descriptions (what native-tool-calling models read) were
bare and gave no steer, so models would do file ops via the shell (e.g. writing
SVG/HTML, which then dumps raw markup into the tool preview). Tell bash/python
in the schema + tool-index + prompt section to prefer read_file/write_file/
edit_file/grep/glob/ls and only be used for what those do not cover.

* prompt(tools): keep bash/python deferral generic (no hardcoded tool names)

Reference 'a dedicated tool' rather than listing read_file/write_file/grep/etc.
by name, so the guidance does not go stale if those tools are renamed.

* style(workspace): drop em-dashes from added code comments/strings

* ux(workspace): terser non-sandbox note in picker (no tool-name list)

* ux(workspace): mirror terse non-sandbox wording in pill tooltip

* chore: untrack local venv symlink (run-only, not part of the feature)

* prompt(workspace): keep get_workspace text generic (no hardcoded tool names)

* fix(agent): low-signal + workspace surfaces only read-only file tools

Intersect the files tool group with PLAN_MODE_READONLY_TOOLS so a vague message
in a workspace exposes read_file/grep/glob/ls/get_workspace for exploration, but
not write_file/edit_file/bash/python -- those wait for a request that actually
calls for them (RAG retrieval still adds them on a real ask).

* feat(workspace): cap browse listing at 500 dirs with a truncated hint

Mirror the filesystem_tools._CODENAV_MAX_HITS pattern with a module-local
_MAX_BROWSE_DIRS so a directory with thousands of children does not dump every
row into the picker; the response carries a truncated flag and the modal tells
the user to type a path to jump in.

* chore: untrack local venv symlink (run-only artifact)

* fix(workspace): vet the workspace root against the sensitive-path deny list at bind time

The in-workspace resolver deny-lists sensitive paths inside the workspace,
but the empty-path search root is the workspace itself, so a workspace of
~/.ssh could be listed via ls with no path. vet_workspace() (public, in
tool_execution next to the resolvers) rejects non-directories and sensitive
roots before the path is ever bound; chat_routes uses it instead of its
inline isdir check.

* fix(workspace): reject filesystem roots and stop showing rejected workspaces as active

Review findings from #3665:

P2: vet_workspace accepted / (and would accept drive/UNC roots), which makes
every absolute path 'inside' the workspace and collapses confinement into
host-wide file access. A root is its own dirname, so reject when
dirname(resolved) == resolved; the browse response now carries a selectable
flag and the picker disables 'Use this folder' on unselectable dirs.

P3: /workspace set stored any string client-side and the chat route silently
dropped rejected values, so the pill could claim a confinement that was not
in effect. New admin-gated /api/workspace/vet validates manual paths before
they persist (canonical path returned), and when a posted workspace is
rejected at send time the stream emits workspace_rejected so the client
clears the stored value and toasts instead of continuing silently.

* fix(workspace): check caller privilege before vetting the posted workspace

Review finding: /api/chat_stream called vet_workspace() on the posted value
for every caller and emitted workspace_rejected on failure, so a non-admin
who can chat but cannot use file/shell tools could distinguish existing
directories from missing/file/sensitive/root paths by whether the event
appeared. The resolution now lives in _resolve_request_workspace, which
drops the submitted value uniformly for non-admin callers, with no vetting
and no event, before the path ever touches the filesystem. Admin and
single-user behavior is unchanged. Test pins that valid and invalid paths
are indistinguishable for a non-admin and that vet_workspace is never
invoked for them.
2026-06-11 18:17:54 +02:00
Yeoh Ing Ji 3e49658204 refactor(tools): extract document tools to handle registry (#3666)
* feat(tools): add document management tool handlers to the agent_tools module

* feat(tools): extraced document tools for create, update, edit, suggest, and manage from tool_implementations.py

* feat(tests): refactor document tool tests to use TOOL_HANDLERS and document_tools

* refactor(tools): add document tool dispatcher and updated tool calling path

* refactor(tools): remove duplicated document management functions

* refactor(tools): removing unused functions and adding new import paths

* refactor(tools): update document tool execute methods to use context dictionary

* refactor(tests): update import paths for document tools in test files

* refactor(tests): update owner parameter format in document management tests

* refactor(tests): update import path for _owned_document_query

* feat(tools): add document management tool handlers to the agent_tools module

* feat(tools): extraced document tools for create, update, edit, suggest, and manage from tool_implementations.py

* feat(tests): refactor document tool tests to use TOOL_HANDLERS and document_tools

* refactor(tools): add document tool dispatcher and updated tool calling path

* refactor(tools): remove duplicated document management functions

* refactor(tools): removing unused functions and adding new import paths

* refactor(tools): update document tool execute methods to use context dictionary

* refactor(tests): update import paths for document tools in test files

* refactor(tests): update owner parameter format in document management tests

* refactor(tests): update import path for _owned_document_query

* refactor: update import paths for document tools

* fix(tests): correct source path for document ID test
2026-06-10 10:41:52 +02:00
Maanas c1674fc2aa refactor(tools): migrate execution logic to src/agent_tools/ package with handler registry (#3435)
* refactor(tools): implement strict cohesive class coordinator pattern per #2917

* test: update edit_file tests to use EditFileTool class

* fix(tools): restore tool_policy param and security backstop in coordinator

* refactor(tools): migrate domain tools to agent_tools package per #2917

* test: update test imports for new agent_tools package

* fix: resolve circular import between tool_execution and agent_tools

* fix: remove leftover git conflict markers

* fix(tools): resolve pytest failure and document _apply method

* fix(tools): clean up whitespace and remove dead _tool_python helper

---------

Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
2026-06-09 14:35:36 +01:00