Commit Graph

1888 Commits

Author SHA1 Message Date
pewdiepie-archdaemon b5ec40d505 Merge remote-tracking branch 'origin/dev'
# Conflicts:
#	routes/contacts_routes.py
2026-07-07 00:51:34 +00:00
pewdiepie-archdaemon 017903de61 Checkpoint Odysseus local update 2026-07-07 00:50:07 +00:00
Boody 35f867c959 Merge pull request #4983 from michaelxer/fix-setup-link-4926-20260628
fix(docs): correct broken backup-restore link in setup.md
2026-07-06 03:18:54 +03:00
RaresKeY 2826dcfc33 fix(tasks): gate cookbook serve task execution (#5235) 2026-07-05 13:19:04 +01:00
RaresKeY 3592285db7 fix(email): enforce MCP account owner scope (#5234) 2026-07-05 13:13:56 +01:00
Ashvin c8169ad7a9 fix(security): scope owner-less email accounts to a mailbox match in route guards (#5238)
The HTTP email route guard `_assert_owns_account` and the explicit-account_id
path in `_get_email_config` gated cross-tenant access with
`if row.owner and row.owner != owner` -- which skips the check entirely when the
account row is owner-less (owner NULL or ""). `email_accounts` is the one
owner-scoped table left out of the legacy-owner migration backfill
(core/database.py), so such rows persist on multi-user deploys: an account
configured while auth was disabled, or an imported legacy row. Any authenticated
user could then pass that account's id to read/send/update-credentials/delete
another tenant's mailbox and read its decrypted IMAP/SMTP creds.

Both sibling paths already enforce the intended contract -- the same-file
`_owner_or_matching_legacy_account` fallback and the MCP `_account_visible_to_owner`
gate (whose comment says it mirrors "the HTTP email route fallback") only expose
an owner-less account when its own mailbox (imap_user / from_address) is the
caller's. Factor that row-level predicate into `_account_visible_to_owner` and
use it in both guards, so owner-less accounts are visible only on a mailbox
match. Owned accounts, the legacy-claim path, and single-user mode (owner == "")
are unchanged.

Complements #5234 (which fixes the same class on the MCP tool layer); this is
the HTTP route layer it does not touch.
2026-07-05 12:50:32 +01:00
Tal.Yuan 5acd0ceae9 refactor(routes): move contacts domain into routes/contacts/ subpackage (#5227)
Slice 2e of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves contacts_routes.py into
routes/contacts/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.

The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, and history #5090 slices) so that
`import routes.contacts_routes`, `from routes.contacts_routes import X`,
`importlib.import_module(...)`, the string-targeted
`monkeypatch.setattr("routes.contacts_routes.SETTINGS_FILE", ...)` used by
test_carddav_password_encryption.py, and the `import ... as cr` +
`setattr(cr, ...)` pattern in test_contacts_add_null_name.py all operate on
the same module object the application uses. This also keeps the mutable
module state `_contact_cache` identical across import paths.

The canonical module does NOT depend on the shim — routes/contacts/
contacts_routes.py imports only from core/, src/, and stdlib (zero internal
routes/ coupling). The inbound edge from routes/email_helpers.py (imports
_fetch_contacts) keeps working through the shim.

Zero source-introspection landmines — no test reads this file by path.

Adds tests/test_contacts_routes_shim.py to pin the sys.modules shim contract
(same-object + string-targeted monkeypatch reach-through).

Verified: compileall clean; full suite 4485 passed, 3 skipped.
2026-07-05 03:58:34 +02:00
Boody 9dc0d661cf Merge pull request #5222 from RaresKeY/fix/chat-web-search-deny-20260704
fix(chat): honor explicit web search denial
2026-07-05 04:04:04 +03:00
Boody a05221571c Merge pull request #5181 from harshit-ojha0324/fix/webhook-trailing-slash
fix(integrations): don't append a trailing slash when api_call path is '/'
2026-07-05 03:44:19 +03:00
Odysseus Review Oracle 264da65186 fix(chat): honor explicit web search denial 2026-07-04 23:33:43 +00:00
Harshit Ojha a50e30c28b test(integrations): drop redundant trailing-slash assertion
The exact-equality assert on the line above (requested_url == WEBHOOK_BASE)
already implies the URL has no trailing slash, so the endswith check adds
nothing.
2026-07-04 17:35:52 -04:00
Ocean Bennett d8d98caa78 fix(security): sanitize email rich body render path (#5212) 2026-07-04 23:21:18 +02:00
Boody 440d99d02c Merge pull request #5166 from QlikChrister/fix/tool-rag-timeout-keyword-fallback
fix(agent): fall back to keyword tool selection when retrieval times out
2026-07-04 23:16:52 +03:00
Boody 7d481b250c Merge pull request #5204 from Ohualtex/fix/search-query-unicode-entity-names
fix(search): extract non-ASCII capitalized names in _extract_entities
2026-07-04 22:38:25 +03:00
Alexandre Teixeira e3750fcdcb fix(security): make research path lookup CodeQL-friendly (#5129)
* fix(security): make research path lookup CodeQL-friendly

* fix(security): avoid duplicate research path scans

* fix(research): preserve active completed spinoff query
2026-07-04 20:17:45 +02:00
Ohualtex 439285e1b7 fix(search): extract non-ASCII capitalized names in _extract_entities
_extract_entities used the ASCII-only class [A-Z][a-zA-Z]+ to pull name
entities from a query, so non-ASCII names were dropped ("İstanbul",
"Zürich" yielded nothing) or shredded ("São Paulo" -> only "Paulo"),
degrading query enhancement for non-English/accented searches. Match
Unicode words and keep the alphabetic, uppercase-initial ones; ASCII
behaviour (the word boundary already excludes camelCase mid-word
capitals) is unchanged.
2026-07-04 20:42:06 +03:00
Wes Huber 897e6950af fix(security): apply the webhook SSRF guard to the reminder ntfy sender (#5142)
The webhook branch of dispatch_reminder validates its target with
check_outbound_url before posting; the ntfy branch posted to the
integration's user-configured base_url with no check, so a base_url
pointing at the metadata range (169.254.169.254) was fetched
server-side — with the integration's Authorization header attached —
every time a reminder fired.

Run the same check (and honor the same REMINDER_WEBHOOK_BLOCK_PRIVATE_IPS
knob) before the post, surfacing rejections in ntfy_error exactly like
the webhook branch does. LAN ntfy servers keep working by default,
matching the project's local-first policy.

Fixes #5141

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:37 +01:00
Wes Huber 6114ef0d6d fix(security): pin webhook delivery to the SSRF-validated IP (DNS rebinding) (#5147)
validate_webhook_url resolves the host to accept/reject, but the delivery
connect (httpx.AsyncClient.post) re-resolved independently — a DNS record
flipping between the two lookups (rebinding) could slip an internal IP
(127.0.0.1 / 169.254.169.254 / LAN) past the check and receive the signed
payload. The module docstring already flagged this as only a "partial
defense".

Resolve + validate once via _validated_public_ips, then pin the delivery
TCP connect to that approved IP with an async _PinnedAsyncTransport built
on the public httpcore/httpx APIs (mirrors the sync search-fetch pin from
#704). The URL, Host header, and TLS SNI are unchanged, so certificate
validation and vhost routing still target the original hostname; only the
socket destination is pinned.

Delivery now uses a per-request pinned client instead of one shared client,
so close() is a no-op kept for API compatibility. Adds end-to-end tests that
drive the real transport against loopback servers, proving the connect
follows the pin rather than re-resolving the URL host.

Fixes #5146

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:03:38 +01:00
Wes Huber 3dd031c139 fix(security): validate integration api_call URLs with the outbound SSRF guard (#5145)
execute_api_call — reachable by the LLM through the api_call agent
tool — joined the integration's user-configured base_url with an
LLM-controlled path and requested it with no IP validation, so a
base_url (or a hostname resolving) into the metadata range
(169.254.169.254) was fetched server-side with the integration's auth
headers attached.

Run check_outbound_url on the joined URL before connecting, matching
the gallery endpoint, embeddings, CardDAV, and reminder webhook
surfaces. Link-local/metadata is always rejected;
INTEGRATION_API_BLOCK_PRIVATE_IPS=true also blocks RFC-1918/loopback.
Private stays allowed by default because LAN integrations
(Home Assistant, Miniflux, ntfy) are the primary use case.

The truncation-test helpers stub the guard open because their
api.example.com fixture host does not resolve and the guard fails
closed on DNS errors.

Fixes #5143

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 16:58:14 +01:00
Ashvin d3faa00aaa fix(security): match grep's rg sensitive-file exclusions case-insensitively (#5189)
The grep tool's ripgrep fast-path excluded deny-listed key files with
`--glob "!*<pat>*"` for each entry in _SENSITIVE_FILE_PATTERNS. ripgrep's
--glob is case-sensitive, so on a case-insensitive filesystem (Windows,
default macOS) a key stored under a case variant of its name (ID_RSA,
Known_Hosts, Authorized_Keys) is the same file on disk but slips past the
lowercase exclusion, and ripgrep returns its contents. Those names are
non-dotfiles, so ripgrep's default hidden-file skipping does not cover them
either. The Python fallback already blocks them via the case-folded
_is_sensitive_path (#5097), so the two paths disagreed.

Switch the sensitive-pattern exclusions to --iglob so they match
case-insensitively, mirroring _is_sensitive_path. Add a regression test
that seeds ID_RSA and Known_Hosts and asserts grep returns ordinary
matches but not the key contents.
2026-07-04 16:52:25 +01:00
Alexandre Teixeira a3bbe37923 Merge pull request #5195 from ashvinctrl/fix/send-to-session-null-owner
fix(security): scope send_to_session to an exact session owner
2026-07-04 16:47:10 +01:00
badgerbees 5c16d39e91 fix(calendar): honor list_events date range aliases (#3283)
* fix(calendar): honor list_events date range aliases

* fix(calendar): reject partially resolved loose range queries
2026-07-04 14:44:46 +02:00
Tal.Yuan 6f6cb6ea88 refactor(routes): move history domain into routes/history/ subpackage (#5090)
Slice 2d of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves history_routes.py into
routes/history/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.

The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, and memory #5007 slices) so that `import
routes.history_routes`, `from routes.history_routes import X`,
`importlib.import_module(...)`, and the `import ... as history_routes` +
`monkeypatch.setattr(history_routes, ...)` pattern used by
test_history_compact_tool_calls.py / test_fork_session_metadata.py all
operate on the same module object the application uses.

The canonical module does NOT depend on the shim — routes/history/
history_routes.py imports only from core/, src/, and routes.session_routes
(a sibling route module whose old import path stays valid via its own shim
when session is migrated later).

Three source-introspection test sites repointed to the new canonical path:
- test_history_db_fallback_hidden.py
- test_history_order_by_timestamp_regression.py
- test_model_helper_owner_scope.py

Adds tests/test_history_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).

Verified: compileall clean; full suite 4351 passed, 3 skipped.
2026-07-04 13:36:35 +02:00
ashvinctrl 43ead1a0eb fix(security): scope send_to_session to an exact session owner
send_to_session let an authenticated caller reach a null-owner session.
The owner gate was `if owner and sess.owner and sess.owner != owner`, so a
target whose owner is None (legacy rows, or a session created while auth
was off) skipped the check and was read/written by any authenticated user.
list_sessions (get_sessions_for_user) and manage_session already exclude
null-owner sessions from an authenticated caller via an exact owner match,
so this path was the lone inconsistency — the same class of gap the
calendar owner=None fix closed.

Require an exact owner match: `if owner and sess.owner != owner`. Auth-off
(no owner) is unchanged, an exact-owner match still passes, and both
another user's session and a null-owner session are now not-found. Adds a
regression test that an authenticated caller cannot read the transcript of
or write into a null-owner session while single-user access still works.
2026-07-04 14:13:04 +05:30
harshit-ojha0324 d3ab478ef1 fix(integrations): don't append a trailing slash when api_call path is '/'
_join_integration_url built urljoin(base + '/', '') for a bare '/'
path — the minimum execute_api_call accepts — so every request against
a POST-to-base integration went to base_url + '/'. Discord webhook
URLs 404 ('Unknown Webhook') on the trailing-slash variant, which made
the integration look broken even though the stored base URL was
correct.

Resolve a bare '/' (or empty) path to the base URL itself and keep all
other paths joining exactly as before, including deliberate trailing
slashes inside non-empty paths (linkding /api/tags/, Home Assistant
/api/). The reminder webhook sender and the discord_webhook
connectivity test already posted to the bare base URL; execute_api_call
was the remaining path that re-added the slash.

Fixes #5138
2026-07-03 18:26:36 -04:00
Alexandre Teixeira 1f6dc80525 ci: add focused test guidance signal (#4982)
* ci: add focused test guidance signal

* ci: diff focused guidance from merge base
2026-07-03 21:17:28 +02:00
Alexandre Teixeira 0b3338c69d test: split service health tests (#4972)
* test: split service health tests

* test(service-health): preserve focus selector
2026-07-03 20:50:49 +02:00
Christer Hantilson b7df800e94 fix(agent): fall back to keyword tool selection when retrieval times out
The retrieval-timeout branch hard-coded ALWAYS_AVAILABLE, silently skipping
the deterministic keyword hints whenever the embedding backend was slow
(e.g. a remote endpoint cold-loading its model). Queries that named email
or calendar outright lost those tools and the model concluded the
integrations did not exist. Let the timeout fall through to the existing
keyword fallback instead — same baseline, plus the hints.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:10:46 +02:00
Abdul Fatah Jamro ff7164b9ec fix: resolve RAG manager search signature TypeError (#4994)
* fix: resolve RAG manager search signature TypeError and adjust similarity threshold

* fix: revert similarity threshold change to keep PR focused

* test(rag): remove trailing whitespace

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-03 15:07:16 +01:00
Tanmay Garg 7f43678a24 fix(tools): handle non-dict JSON values in _parse_tool_args (closes #5043) (#5064)
When an LLM generates a valid JSON string that parses to a native non-dict
type (like a list, int, or string), _parse_tool_args previously returned
that object. Callers expecting a dictionary would then crash with
AttributeError or KeyError when attempting to look up action keys.

- Update _parse_tool_args in src/tool_utils.py to explicitly type-check
  the parsed JSON object and return {} for non-dict objects.
- Add test coverage in tests/test_admin_tools_registry.py for lists,
  ints, and strings.
2026-07-03 13:07:44 +01:00
pewdiepie-archdaemon 5f6e6a2c4a Hide untagged reasoning dumps in chat 2026-07-03 03:59:42 +00:00
pewdiepie-archdaemon cf85c42195 Parse local function_model tool wrappers 2026-07-03 03:54:59 +00:00
pewdiepie-archdaemon 79716d717a Keep open document context for section edits 2026-07-03 03:04:22 +00:00
pewdiepie-archdaemon d360401808 Route structured writing requests to documents 2026-07-03 02:51:42 +00:00
pewdiepie-archdaemon c89258e4a6 Open documents from native tool outputs 2026-07-03 02:20:23 +00:00
pewdiepie-archdaemon 5777cf2d00 Add AI edit command box to gallery editor 2026-07-03 02:13:45 +00:00
pewdiepie-archdaemon 0fc98c4a17 Add bulk email attachment downloads 2026-07-03 01:15:40 +00:00
pewdiepie-archdaemon 9718f7874b Fix stale streams and cookbook task controls 2026-07-03 00:45:43 +00:00
Moniz 8c943226f8 fix(mobile): stack the model-comparison grid into one column on phones (#4979)
The comparison grid hard-codes 2-4 equal columns with no phone breakpoint, so at
390px two models get ~178px columns and four get ~88px columns. Each column is a
full scrolling chat, so content is unreadably over-wrapped and clipped. On
phones (<=768px), stack the panes into a single scrollable column. Desktop is
unaffected.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 17:28:23 +02:00
holden093 0dc98ec9b9 fix(ui): prevent race condition in default chat model dropdown init (#5024)
Setting epSel.value triggered an async change event whose handler
called refreshModels('') — wiping the correct model selection that
refreshModels(settings.default_model) had just applied moments earlier.
The dropdown silently fell back to the alphabetically-first model
(deepseek-v4-flash instead of qwen-3.6-35B-A3B).

Moved the change listener registration to after the settings block
so the async change event fires before any listener exists. The
utility and teacher sections already followed this pattern.
2026-07-02 17:05:55 +02:00
Ernest Hysa 5e9b415bd9 fix(search): pin httpx connection to resolved IP to block DNS rebinding (#704)
* fix(search): pin DNS-validated fetch connections

Rebase the DNS-rebinding SSRF fix onto current dev after search content moved behind the services.search.content canonical module.

Integrate the pinned httpcore NetworkBackend/BaseTransport approach with the current size-capped Client.stream fetch path, preserving Host/SNI semantics while forcing TCP connect to the already validated resolved IP.

Keep src.search.content as the compatibility wrapper and preserve existing OG-image http(s) behavior; this avoids reintroducing the unrelated scope changes that previously blocked review.

Add the explicit httpcore>=1.0,<2.0 requirement used by the public httpcore NetworkBackend and ConnectionPool APIs.

* test(search): restore and rebase DNS rebinding regressions

Keep the current security regression coverage that the stale PR branch had deleted, including auth-disabled localhost bypass and Ollama cookbook hardening tests.

Carry forward the DNS-rebinding coverage for private resolve blocking, pinned TCP connect behavior, Host header preservation, redirect revalidation, and the BaseTransport/public-httpcore static guard.

Update redirect tests to mock the current Client.stream-based capped fetch path rather than the older httpx.stream/get path.

* test(search): adapt size-cap fetch tests to pinned client stream

The DNS-rebinding repair moved _get_public_url from the module-level httpx.stream shortcut to httpx.Client(...).stream(...) so the fetch can use the pinned transport.

Keep the existing size-cap test fakes by routing Client.stream through the monkeypatched httpx.stream only when a test has installed that fake; otherwise fall back to a real Client.

This fixes the CI failures in tests/test_web_fetch_size_caps.py without touching unrelated upload-handler atomicity behavior, which is already flaky on clean origin/dev.

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-02 13:08:06 +01:00
Alexandre Teixeira b1f9f67d9d fix(security): confine research file paths (#4986) 2026-07-02 11:58:35 +01:00
Afonso Coutinho 88191d17fb fix: auto-spam move/delete targets the wrong message (seqnum vs UID) (#1874) 2026-07-02 10:40:19 +01:00
Ashvin dff91efb10 fix(agent): skip deny-listed sensitive files in glob (#5094) 2026-07-02 10:28:33 +01:00
Ashvin b26ebbda95 fix(security): match the sensitive-file deny-list case-insensitively (#5097) 2026-07-02 10:11:51 +01:00
lekt8 260f432332 fix(session): use utcnow_naive across session routes (#1116) (#5003)
Replace remaining datetime.utcnow() call sites in session CRUD, incognito
purge cutoff, and webhook payloads with core.database.utcnow_naive.
2026-07-02 11:04:22 +02:00
Mazen Tamer Salah e157f1e63d fix(cookbook): stop Ollama runner from executing the install one-liner (#3926)
The generated bash runner printed the missing-ollama hint with the install
one-liner wrapped in backticks inside a double-quoted echo. Backticks in
double quotes are command substitution, so on any serve target without
ollama the script downloaded and ran the system-wide installer (including
remote SSH hosts) instead of printing the hint. _validate_serve_cmd rejects
backticks in user-supplied commands for exactly this reason; the app's own
generated script never goes through that validator.

Move the hint into OLLAMA_MISSING_HINT in cookbook_helpers (no substitution
tokens) and emit it single-quoted via _bash_squote. Tests assert the hint
has no expansion tokens, that no generated echo line carries backticks
inside double quotes, and that bash prints the line literally.

Fixes #3816
2026-07-02 10:01:57 +01:00
pewdiepie-archdaemon 265f0911d5 Support mobile enter for queued agent prompts 2026-07-01 14:42:52 +00:00
pewdiepie-archdaemon dc3530b8fa Show fallback model in picker 2026-07-01 13:53:51 +00:00
pewdiepie-archdaemon 2918739489 Fix merged test regressions 2026-07-01 11:12:55 +00:00