_scheduled_poll_once selected rows WHERE status='pending' and only wrote
status='sent'/'failed' after the SMTP send and IMAP append completed -
no atomic claim in between. Two overlapping callers (the in-process 30s
poller and an externally cron/systemd-driven 'odysseus-mail
poll-scheduled', or the CLI run manually) can both SELECT the same
pending row before either UPDATEs it, and both send it. _start_poller's
own docstring already names this exact risk ('avoid two copies of
_scheduled_poll_once racing on the same SQLite') but nothing in the code
enforced it - it was advisory only.
Add an atomic per-row claim: UPDATE ... SET status='sending' WHERE
id=? AND status='pending', proceeding only when rowcount == 1. The
loser of the race sees rowcount == 0 and skips the row instead of
sending a duplicate.
Adds a regression test that drives two real threads through the real
_scheduled_poll_once against a shared SQLite file, synchronized with a
barrier and a widened send-path window, and asserts exactly one send
fires. Reverting the fix makes the test fail reliably (5/5 runs); with
the fix it passes reliably (5/5 runs).
Fixes#5109
_store_email_flag and _move_email_message (used by the archive / delete / move /
mark-read endpoints) had an else branch that, when _uid_exists returned False,
ran conn.store(uid, ...) / conn.copy(uid, ...) followed by a folder-wide
conn.expunge(). But imaplib's plain store()/copy() take a message SEQUENCE
NUMBER, not a UID, so the op landed on whichever message occupied sequence
position == the UID value, and the expunge then permanently removed it. A stale
cached UID (or a server whose UID probe misbehaves) therefore deleted an
unrelated email instead of reporting 'not found'.
There is no valid case where treating a UID as a sequence number is correct, so
drop the fallback: when the UID isn't present, return False — callers already
surface 'Email not found'. Only the UID command path remains.
Sibling of #1874 (which fixes the auto-spam poller's _imap_move in
email_helpers.py); this covers the user-facing endpoints in email_routes.py.
Part of #2124.
The edit/delete/pause/run actions of do_manage_tasks gated ownership with
`if owner and task.owner and task.owner != owner`. The middle term made the
check a no-op whenever task.owner was null/empty — the state a scheduled task
sits in when it was created in no-login mode (or via the localhost middleware
bypass) before the periodic legacy-owner sweep reassigns it to the admin user.
Any authenticated user's agent could then edit, delete, pause, or run another
tenant's owner-less task; edit+run lets an attacker rewrite the task prompt and
execute it in the scheduler's agent context.
The sibling `list` action already scopes with an exact `owner == owner` filter,
so the mutators were strictly more permissive than the reader. Drop the middle
term so the guard fails closed on owner-less rows for authenticated callers,
matching `list` and the calendar/notes/gallery/session null-owner gates. Auth
disabled (owner falsy) and same-owner access are unchanged.
request_flags derives (agent, vision) and does last.get("role") after only
a truthy check. A client can send a bare-string message element
("messages": ["hi"]), and the vision loop right below already guards each
element with isinstance — so the .get() on a non-dict last element is an
oversight that raises AttributeError on every Copilot-proxied request with
such a body.
Use isinstance(last, dict) to match the loop's own guard.
Fixes#5273
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
build_user_content derived the data-URL subtype from the file extension
only (image_format = ext[1:]). An extensionless upload (e.g. a pasted
screenshot) has ext == "", producing "data:image/;base64,..." with an
empty subtype (invalid per RFC 2046) that vision/audio endpoints reject,
silently dropping the attachment. Fall back to the resolved MIME subtype
when the extension is missing; present extensions are unchanged.
Align regression tests with the current Odysseus behavior after merging origin/dev into local main.
- keep phone/name-only contacts valid and cover null email without crashes
- pin explicit web-search false form submission in chat.js
- update Cookbook dependency/download completion tests for combined live + persisted output
- expose SGLang OS package repair hints from backend diagnosis
- treat MLX and MLX-community repos as servable on Apple Metal while keeping CUDA behavior unchanged
- keep desktop new-chat coverage on the shared preferred-model helper
- remove a hardcoded crop overlay portal z-index literal
- include the local agent-loop cleanup that removes the old manage_notes reminder repair shim
Verified with: docker run --rm -v /home/pewds/odysseus-cookbook-fresh:/app -w /app odysseus-cookbook-fresh-odysseus python3 -m pytest -q (4515 passed, 4 skipped).
The HTTP email route guard `_assert_owns_account` and the explicit-account_id
path in `_get_email_config` gated cross-tenant access with
`if row.owner and row.owner != owner` -- which skips the check entirely when the
account row is owner-less (owner NULL or ""). `email_accounts` is the one
owner-scoped table left out of the legacy-owner migration backfill
(core/database.py), so such rows persist on multi-user deploys: an account
configured while auth was disabled, or an imported legacy row. Any authenticated
user could then pass that account's id to read/send/update-credentials/delete
another tenant's mailbox and read its decrypted IMAP/SMTP creds.
Both sibling paths already enforce the intended contract -- the same-file
`_owner_or_matching_legacy_account` fallback and the MCP `_account_visible_to_owner`
gate (whose comment says it mirrors "the HTTP email route fallback") only expose
an owner-less account when its own mailbox (imap_user / from_address) is the
caller's. Factor that row-level predicate into `_account_visible_to_owner` and
use it in both guards, so owner-less accounts are visible only on a mailbox
match. Owned accounts, the legacy-claim path, and single-user mode (owner == "")
are unchanged.
Complements #5234 (which fixes the same class on the MCP tool layer); this is
the HTTP route layer it does not touch.
Slice 2e of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves contacts_routes.py into
routes/contacts/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, and history #5090 slices) so that
`import routes.contacts_routes`, `from routes.contacts_routes import X`,
`importlib.import_module(...)`, the string-targeted
`monkeypatch.setattr("routes.contacts_routes.SETTINGS_FILE", ...)` used by
test_carddav_password_encryption.py, and the `import ... as cr` +
`setattr(cr, ...)` pattern in test_contacts_add_null_name.py all operate on
the same module object the application uses. This also keeps the mutable
module state `_contact_cache` identical across import paths.
The canonical module does NOT depend on the shim — routes/contacts/
contacts_routes.py imports only from core/, src/, and stdlib (zero internal
routes/ coupling). The inbound edge from routes/email_helpers.py (imports
_fetch_contacts) keeps working through the shim.
Zero source-introspection landmines — no test reads this file by path.
Adds tests/test_contacts_routes_shim.py to pin the sys.modules shim contract
(same-object + string-targeted monkeypatch reach-through).
Verified: compileall clean; full suite 4485 passed, 3 skipped.
The exact-equality assert on the line above (requested_url == WEBHOOK_BASE)
already implies the URL has no trailing slash, so the endswith check adds
nothing.
_extract_entities used the ASCII-only class [A-Z][a-zA-Z]+ to pull name
entities from a query, so non-ASCII names were dropped ("İstanbul",
"Zürich" yielded nothing) or shredded ("São Paulo" -> only "Paulo"),
degrading query enhancement for non-English/accented searches. Match
Unicode words and keep the alphabetic, uppercase-initial ones; ASCII
behaviour (the word boundary already excludes camelCase mid-word
capitals) is unchanged.
The webhook branch of dispatch_reminder validates its target with
check_outbound_url before posting; the ntfy branch posted to the
integration's user-configured base_url with no check, so a base_url
pointing at the metadata range (169.254.169.254) was fetched
server-side — with the integration's Authorization header attached —
every time a reminder fired.
Run the same check (and honor the same REMINDER_WEBHOOK_BLOCK_PRIVATE_IPS
knob) before the post, surfacing rejections in ntfy_error exactly like
the webhook branch does. LAN ntfy servers keep working by default,
matching the project's local-first policy.
Fixes#5141
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
validate_webhook_url resolves the host to accept/reject, but the delivery
connect (httpx.AsyncClient.post) re-resolved independently — a DNS record
flipping between the two lookups (rebinding) could slip an internal IP
(127.0.0.1 / 169.254.169.254 / LAN) past the check and receive the signed
payload. The module docstring already flagged this as only a "partial
defense".
Resolve + validate once via _validated_public_ips, then pin the delivery
TCP connect to that approved IP with an async _PinnedAsyncTransport built
on the public httpcore/httpx APIs (mirrors the sync search-fetch pin from
#704). The URL, Host header, and TLS SNI are unchanged, so certificate
validation and vhost routing still target the original hostname; only the
socket destination is pinned.
Delivery now uses a per-request pinned client instead of one shared client,
so close() is a no-op kept for API compatibility. Adds end-to-end tests that
drive the real transport against loopback servers, proving the connect
follows the pin rather than re-resolving the URL host.
Fixes#5146
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
execute_api_call — reachable by the LLM through the api_call agent
tool — joined the integration's user-configured base_url with an
LLM-controlled path and requested it with no IP validation, so a
base_url (or a hostname resolving) into the metadata range
(169.254.169.254) was fetched server-side with the integration's auth
headers attached.
Run check_outbound_url on the joined URL before connecting, matching
the gallery endpoint, embeddings, CardDAV, and reminder webhook
surfaces. Link-local/metadata is always rejected;
INTEGRATION_API_BLOCK_PRIVATE_IPS=true also blocks RFC-1918/loopback.
Private stays allowed by default because LAN integrations
(Home Assistant, Miniflux, ntfy) are the primary use case.
The truncation-test helpers stub the guard open because their
api.example.com fixture host does not resolve and the guard fails
closed on DNS errors.
Fixes#5143
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The grep tool's ripgrep fast-path excluded deny-listed key files with
`--glob "!*<pat>*"` for each entry in _SENSITIVE_FILE_PATTERNS. ripgrep's
--glob is case-sensitive, so on a case-insensitive filesystem (Windows,
default macOS) a key stored under a case variant of its name (ID_RSA,
Known_Hosts, Authorized_Keys) is the same file on disk but slips past the
lowercase exclusion, and ripgrep returns its contents. Those names are
non-dotfiles, so ripgrep's default hidden-file skipping does not cover them
either. The Python fallback already blocks them via the case-folded
_is_sensitive_path (#5097), so the two paths disagreed.
Switch the sensitive-pattern exclusions to --iglob so they match
case-insensitively, mirroring _is_sensitive_path. Add a regression test
that seeds ID_RSA and Known_Hosts and asserts grep returns ordinary
matches but not the key contents.
Slice 2d of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves history_routes.py into
routes/history/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, and memory #5007 slices) so that `import
routes.history_routes`, `from routes.history_routes import X`,
`importlib.import_module(...)`, and the `import ... as history_routes` +
`monkeypatch.setattr(history_routes, ...)` pattern used by
test_history_compact_tool_calls.py / test_fork_session_metadata.py all
operate on the same module object the application uses.
The canonical module does NOT depend on the shim — routes/history/
history_routes.py imports only from core/, src/, and routes.session_routes
(a sibling route module whose old import path stays valid via its own shim
when session is migrated later).
Three source-introspection test sites repointed to the new canonical path:
- test_history_db_fallback_hidden.py
- test_history_order_by_timestamp_regression.py
- test_model_helper_owner_scope.py
Adds tests/test_history_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).
Verified: compileall clean; full suite 4351 passed, 3 skipped.
send_to_session let an authenticated caller reach a null-owner session.
The owner gate was `if owner and sess.owner and sess.owner != owner`, so a
target whose owner is None (legacy rows, or a session created while auth
was off) skipped the check and was read/written by any authenticated user.
list_sessions (get_sessions_for_user) and manage_session already exclude
null-owner sessions from an authenticated caller via an exact owner match,
so this path was the lone inconsistency — the same class of gap the
calendar owner=None fix closed.
Require an exact owner match: `if owner and sess.owner != owner`. Auth-off
(no owner) is unchanged, an exact-owner match still passes, and both
another user's session and a null-owner session are now not-found. Adds a
regression test that an authenticated caller cannot read the transcript of
or write into a null-owner session while single-user access still works.
_join_integration_url built urljoin(base + '/', '') for a bare '/'
path — the minimum execute_api_call accepts — so every request against
a POST-to-base integration went to base_url + '/'. Discord webhook
URLs 404 ('Unknown Webhook') on the trailing-slash variant, which made
the integration look broken even though the stored base URL was
correct.
Resolve a bare '/' (or empty) path to the base URL itself and keep all
other paths joining exactly as before, including deliberate trailing
slashes inside non-empty paths (linkding /api/tags/, Home Assistant
/api/). The reminder webhook sender and the discord_webhook
connectivity test already posted to the bare base URL; execute_api_call
was the remaining path that re-added the slash.
Fixes#5138
When an LLM generates a valid JSON string that parses to a native non-dict
type (like a list, int, or string), _parse_tool_args previously returned
that object. Callers expecting a dictionary would then crash with
AttributeError or KeyError when attempting to look up action keys.
- Update _parse_tool_args in src/tool_utils.py to explicitly type-check
the parsed JSON object and return {} for non-dict objects.
- Add test coverage in tests/test_admin_tools_registry.py for lists,
ints, and strings.
* fix(search): pin DNS-validated fetch connections
Rebase the DNS-rebinding SSRF fix onto current dev after search content moved behind the services.search.content canonical module.
Integrate the pinned httpcore NetworkBackend/BaseTransport approach with the current size-capped Client.stream fetch path, preserving Host/SNI semantics while forcing TCP connect to the already validated resolved IP.
Keep src.search.content as the compatibility wrapper and preserve existing OG-image http(s) behavior; this avoids reintroducing the unrelated scope changes that previously blocked review.
Add the explicit httpcore>=1.0,<2.0 requirement used by the public httpcore NetworkBackend and ConnectionPool APIs.
* test(search): restore and rebase DNS rebinding regressions
Keep the current security regression coverage that the stale PR branch had deleted, including auth-disabled localhost bypass and Ollama cookbook hardening tests.
Carry forward the DNS-rebinding coverage for private resolve blocking, pinned TCP connect behavior, Host header preservation, redirect revalidation, and the BaseTransport/public-httpcore static guard.
Update redirect tests to mock the current Client.stream-based capped fetch path rather than the older httpx.stream/get path.
* test(search): adapt size-cap fetch tests to pinned client stream
The DNS-rebinding repair moved _get_public_url from the module-level httpx.stream shortcut to httpx.Client(...).stream(...) so the fetch can use the pinned transport.
Keep the existing size-cap test fakes by routing Client.stream through the monkeypatched httpx.stream only when a test has installed that fake; otherwise fall back to a real Client.
This fixes the CI failures in tests/test_web_fetch_size_caps.py without touching unrelated upload-handler atomicity behavior, which is already flaky on clean origin/dev.
---------
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
The generated bash runner printed the missing-ollama hint with the install
one-liner wrapped in backticks inside a double-quoted echo. Backticks in
double quotes are command substitution, so on any serve target without
ollama the script downloaded and ran the system-wide installer (including
remote SSH hosts) instead of printing the hint. _validate_serve_cmd rejects
backticks in user-supplied commands for exactly this reason; the app's own
generated script never goes through that validator.
Move the hint into OLLAMA_MISSING_HINT in cookbook_helpers (no substitution
tokens) and emit it single-quoted via _bash_squote. Tests assert the hint
has no expansion tokens, that no generated echo line carries backticks
inside double quotes, and that bash prints the line literally.
Fixes#3816
jagadish-zentiti