19 Commits

Author SHA1 Message Date
Wes Huber 1f8687abeb fix(calendar): trust operator CA bundle in CalDAV test_connection (#4796)
* fix(calendar): trust operator CA bundle in CalDAV test_connection

The pre-flight test used httpx with trust_env=False, which ignored
SSL_CERT_FILE/REQUESTS_CA_BUNDLE. Self-signed CalDAV servers that
the real sync accepts (via caldav lib → requests → honors bundle)
were rejected by the test with CERTIFICATE_VERIFY_FAILED.

Build an explicit SSL context that loads the operator's CA bundle
and clears VERIFY_X509_STRICT (which rejects certs without a
keyUsage extension — common in self-signed setups). SSRF guards
(follow_redirects=False, trust_env=False) are preserved.

Fixes #4795
Fixes #4779

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(calendar): add regression tests and edge case handling for SSL context

Per review: add route-level regression tests covering SSL_CERT_FILE
precedence, VERIFY_X509_STRICT clearing, missing bundle graceful
fallback, and empty env var handling. Also log a warning when the
configured CA bundle path doesn't exist instead of silently falling
back to system CAs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(calendar): rewrite SSL tests to exercise route handler directly

Addresses review feedback: tests now use FastAPI TestClient to hit the
actual test_connection route, capturing the verify= kwarg passed to
httpx.AsyncClient. This ensures the route's SSL context construction
is covered, not a test-side duplicate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: retrigger CI (redirect hardening test is a CI-env flake, passes locally)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tests): remove module-level sys.modules stubs that leaked into other tests

The collection-time MagicMock stub of `caldav` replaced the real library
for every later test in the same process — test_caldav_redirect_hardening's
DAVClient became a mock that never sent the PROPFIND, failing its
must-reach-the-public-server assertion in CI. conftest already pre-imports
the real sqlalchemy/core.database, and the route's lazy imports are patched
per-request, so the stub block was both harmful and unnecessary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(calendar): verify exact CA bundle precedence

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-11 13:25:16 +01:00
tanmayraut45 e6d9d68729 CalDAV: close the DAVClient on sync and write-back paths (#4793)
_sync_blocking (src/caldav_sync.py) and _writeback_blocking
(src/caldav_writeback.py) each open their own caldav.DAVClient via
_build_dav_client, but never close it. The client owns an HTTP session
with a pooled connection; without a close() that connection is held until
process exit.

Previously the fix added explicit client.close() calls before each early
return and at the end of the DB finally block. This still leaked the
client when SessionLocal() raised before the DB try/finally was entered.

Now _sync_blocking wraps the entire post-construction path in an outer
try/finally that calls client.close() unconditionally, covering:
  - AuthorizationError / NotFoundError early return
  - URL-fallback failure early return
  - no-calendars early return
  - normal return after sync
  - SessionLocal() construction failure (new regression coverage)

_writeback_blocking already used a try/finally (unchanged).

- src/caldav_sync.py: replace scattered client.close() calls with a
  single outer try/finally block around the discovery + DB sync path
- tests/test_caldav_client_cleanup.py: add CalendarDeletedEvent to the
  database stub; add regression test for SessionLocal() failure path

Closes #4593
2026-07-11 13:03:24 +01:00
red person 6efc07c5fc Skip vanished backup list entries (#2006) 2026-07-11 05:26:42 +01:00
jagadish-zentiti bc771fbc1e fix(mcp): guard DbTokenStorage against non-dict oauth_tokens JSON (#5107)
_load() returned whatever json.loads() produced without checking it was a
dict; _update() did the same before assigning data[key] = value. If the
oauth_tokens column ever held a JSON array or primitive (DB corruption,
manual edit, migration drift), _load()'s callers crashed with
AttributeError on .get(), and _update() crashed with TypeError trying to
item-assign into a list/string/int.

Validate the parsed value is a dict in both methods, falling back to {}
otherwise - same recovery behavior already used elsewhere in the codebase
for this exact JSON-blob-is-not-a-dict shape (_parse_tool_args,
_is_sensitive_path's siblings).

Adds 3 regression tests for _load, get_tokens, and _update against a
non-dict oauth_tokens value.

Fixes #5082
2026-07-11 05:26:23 +01:00
jagadish-zentiti 21c7bf802e fix(email): atomically claim scheduled emails before sending (#5110)
_scheduled_poll_once selected rows WHERE status='pending' and only wrote
status='sent'/'failed' after the SMTP send and IMAP append completed -
no atomic claim in between. Two overlapping callers (the in-process 30s
poller and an externally cron/systemd-driven 'odysseus-mail
poll-scheduled', or the CLI run manually) can both SELECT the same
pending row before either UPDATEs it, and both send it. _start_poller's
own docstring already names this exact risk ('avoid two copies of
_scheduled_poll_once racing on the same SQLite') but nothing in the code
enforced it - it was advisory only.

Add an atomic per-row claim: UPDATE ... SET status='sending' WHERE
id=? AND status='pending', proceeding only when rowcount == 1. The
loser of the race sees rowcount == 0 and skips the row instead of
sending a duplicate.

Adds a regression test that drives two real threads through the real
_scheduled_poll_once against a shared SQLite file, synchronized with a
barrier and a widened send-path window, and asserts exactly one send
fires. Reverting the fix makes the test fail reliably (5/5 runs); with
the fix it passes reliably (5/5 runs).

Fixes #5109
2026-07-11 04:23:36 +01:00
L1 e0fd68160f fix(email): never fall back to sequence-number IMAP ops for move/flag (#2732)
_store_email_flag and _move_email_message (used by the archive / delete / move /
mark-read endpoints) had an else branch that, when _uid_exists returned False,
ran conn.store(uid, ...) / conn.copy(uid, ...) followed by a folder-wide
conn.expunge(). But imaplib's plain store()/copy() take a message SEQUENCE
NUMBER, not a UID, so the op landed on whichever message occupied sequence
position == the UID value, and the expunge then permanently removed it. A stale
cached UID (or a server whose UID probe misbehaves) therefore deleted an
unrelated email instead of reporting 'not found'.

There is no valid case where treating a UID as a sequence number is correct, so
drop the fallback: when the UID isn't present, return False — callers already
surface 'Email not found'. Only the UID command path remains.

Sibling of #1874 (which fixes the auto-spam poller's _imap_move in
email_helpers.py); this covers the user-facing endpoints in email_routes.py.
Part of #2124.
2026-07-11 03:27:28 +01:00
Afonso Coutinho def3483032 fix: TTS available crashes on non-string tts_provider (#2034) 2026-07-11 03:19:51 +01:00
Afonso Coutinho 667f9e4cae fix: _matchesCombo crashes on non-string keybind from server (#2049) 2026-07-11 03:15:19 +01:00
Afonso Coutinho 7b25b6dbdc fix: odysseus-memory cmd_add crashes on non-dict existing memory row (#2091) 2026-07-11 03:05:17 +01:00
Afonso Coutinho 7a1c7395c0 fix: hwfit params_b/is_prequantized crash on non-string catalog fields (#2094) 2026-07-11 03:00:28 +01:00
Ashvin a8d215a390 fix(tasks): scope manage_tasks mutations to an exact task owner (#5264)
The edit/delete/pause/run actions of do_manage_tasks gated ownership with
`if owner and task.owner and task.owner != owner`. The middle term made the
check a no-op whenever task.owner was null/empty — the state a scheduled task
sits in when it was created in no-login mode (or via the localhost middleware
bypass) before the periodic legacy-owner sweep reassigns it to the admin user.
Any authenticated user's agent could then edit, delete, pause, or run another
tenant's owner-less task; edit+run lets an attacker rewrite the task prompt and
execute it in the scheduler's agent context.

The sibling `list` action already scopes with an exact `owner == owner` filter,
so the mutators were strictly more permissive than the reader. Drop the middle
term so the guard fails closed on owner-less rows for authenticated callers,
matching `list` and the calendar/notes/gallery/session null-owner gates. Auth
disabled (owner falsy) and same-owner access are unchanged.
2026-07-11 01:45:14 +01:00
Am-GJ 4901a96591 fix(reminders): sanitize ntfy Title header to ASCII (#5208)
* fix(reminders): sanitize ntfy Title header to ASCII

The ntfy notification Title header was set directly from the note title.
HTTP headers must be ASCII, so a title containing emoji or other
non-ASCII characters caused httpx to raise UnicodeEncodeError, which
was swallowed by the surrounding try/except — so the reminder silently
failed and no notification was ever sent.

Sanitize the title with encode('ascii', 'replace') before placing it
into the header, replacing unsupported characters with '?'. This is
standard practice for HTTP header values. The note body is unaffected
(it is sent as request content, not a header) and continues to support
full UTF-8.

* fix(reminders): also truncate ntfy title to 200 chars for header safety

* style: compact ntfy header comment

---------

Co-authored-by: Am-GJ <Am-GJ@users.noreply.github.com>
Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
2026-07-10 22:50:00 +02:00
Wes Huber 807e92c3bc docs: remove completed troubleshooting cookbook task from ROADMAP (#4906)
The self-host troubleshooting cookbook has been implemented in
docs/setup.md under "Common self-host traps" (PR #4834).

Fixes #4900

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-10 22:29:20 +02:00
Boody d88c8cbacf Merge pull request #5313 from RaresKeY/fix/chat-web-search-explicit-deny
fix(chat): require explicit web search enable
2026-07-09 01:56:32 +03:00
Wes Huber a35384e68f fix(copilot): guard request_flags against a non-dict last message (#5274)
request_flags derives (agent, vision) and does last.get("role") after only
a truthy check. A client can send a bare-string message element
("messages": ["hi"]), and the vision loop right below already guards each
element with isinstance — so the .get() on a non-dict last element is an
oversight that raises AttributeError on every Copilot-proxied request with
such a body.

Use isinstance(last, dict) to match the loop's own guard.

Fixes #5273

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:57:23 +02:00
Miraç Duran 3064819e3d fix(chat): give extensionless image/audio uploads a valid MIME subtype (#5205)
build_user_content derived the data-URL subtype from the file extension
only (image_format = ext[1:]). An extensionless upload (e.g. a pasted
screenshot) has ext == "", producing "data:image/;base64,..." with an
empty subtype (invalid per RFC 2046) that vision/audio endpoints reject,
silently dropping the attachment. Fall back to the resolved MIME subtype
when the extension is missing; present extensions are unchanged.
2026-07-08 21:04:15 +02:00
RaresKeY 54f1d015b5 fix(chat): require explicit web search enable 2026-07-08 17:44:28 +00:00
Steve Holloway 2d8177035b fix(chat): restore missing _explicit_web_intent definition (#5290)
chat_stream() references `_explicit_web_intent` in three places
(disabled-tools gating, global-disabled web allowance, and the
per-turn tool filter) but the assignment was dropped during a
branch merge. Every chat request raised

    NameError: name '_explicit_web_intent' is not defined

at routes/chat_routes.py, surfacing to the client as a bare
"Internal Server Error" before any LLM call was made — chat was
fully broken on dev and main.

Restore the original definition, computed from the already-derived
tool intent, immediately before its first use:

    _explicit_web_intent = bool(_tool_intent and _tool_intent.category == "web")

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 19:14:43 +02:00
PewDiePie c67deaa60a Merge pull request #5283 from pewdiepie-archdaemon/sync-main-into-dev-20260707
chore: sync tested main into dev
2026-07-07 11:32:53 +09:00
35 changed files with 1582 additions and 304 deletions
-1
View File
@@ -12,7 +12,6 @@ the codebase, you are probably right to stay away.
and WSL all need coverage.
- Integration audit: do integrations even work? Confirm what works, what needs setup docs, and what should be removed or hidden.
- Self-host troubleshooting cookbook. Document the weird 30-second fixes that otherwise become 30-minute searches: Dovecot cleartext auth for local stacks, ntfy Android Instant Delivery for non-ntfy.sh servers, clipboard limits on plain-HTTP Tailscale URLs, Radicale collection URLs, and similar traps.
- Cookbook reliability on other computers. This is probably the area most likely to need work across different machines, GPUs, drivers, shells, and Python environments.
- Cookbook SGLang support across platforms. Make sure SGLang setup/serve works
predictably on Linux, Windows/WSL, macOS where possible, Docker, and common
+18 -1
View File
@@ -913,7 +913,24 @@ def setup_calendar_routes() -> APIRouter:
'</d:prop></d:propfind>'
)
try:
async with httpx.AsyncClient(timeout=8.0, follow_redirects=False, trust_env=False) as cx:
# Build an SSL context that trusts the operator's custom CA bundle
# (SSL_CERT_FILE / REQUESTS_CA_BUNDLE) so self-signed CalDAV servers
# pass the pre-flight the same way they pass the real sync.
# trust_env=False is kept to block proxy/auth env leakage; the CA
# bundle is loaded explicitly instead.
import ssl as _ssl
_ssl_ctx = _ssl.create_default_context()
# Disable VERIFY_X509_STRICT so certs without a keyUsage extension
# (common in self-signed setups) are accepted, matching the
# requests/urllib3 behavior used by the CalDAV sync path.
_ssl_ctx.verify_flags &= ~_ssl.VERIFY_X509_STRICT
_ca_bundle = _os.environ.get("SSL_CERT_FILE") or _os.environ.get("REQUESTS_CA_BUNDLE")
if _ca_bundle:
if _os.path.isfile(_ca_bundle):
_ssl_ctx.load_verify_locations(_ca_bundle)
else:
logger.warning("CalDAV test: CA bundle %s not found, using system CAs", _ca_bundle)
async with httpx.AsyncClient(timeout=8.0, follow_redirects=False, trust_env=False, verify=_ssl_ctx) as cx:
r = await cx.request(
"PROPFIND", url,
auth=(user, pw),
+24 -33
View File
@@ -42,7 +42,12 @@ from routes.chat_helpers import (
_enforce_chat_privileges,
)
from src.action_intents import ToolIntent, classify_tool_intent as _classify_tool_intent
from src.tool_policy import build_effective_tool_policy
from src.tool_policy import (
WEB_TOOL_NAMES,
build_effective_tool_policy,
is_web_search_explicitly_denied,
web_search_enabled_for_turn,
)
logger = logging.getLogger(__name__)
@@ -583,10 +588,7 @@ def setup_chat_routes(
# below). Skill extraction should only learn from real agent sessions,
# not chats we quietly promoted for a notes/calendar intent.
user_requested_agent = (chat_mode == "agent")
_search_enabled = (
str(allow_web_search).lower() == "true"
or str(use_web).lower() == "true"
)
_search_enabled = web_search_enabled_for_turn(allow_web_search, use_web)
# Intent auto-escalation: if the user is clearly asking the assistant
# to create a todo, reminder, or calendar event, promote chat → agent
# for this turn so the LLM has access to manage_notes / manage_calendar.
@@ -870,23 +872,20 @@ def setup_chat_routes(
# Build disabled-tools set from frontend toggles + user privileges
disabled_tools = set()
# Only disable bash/web_search when the caller *explicitly* set them
# to a falsy value. When unset (None), defer to per-user privilege
# checks below — this lets admins with can_use_bash=True use bash
# by default without having to send allow_bash in every request.
# Only disable bash when the caller *explicitly* set it to a falsy
# value. When unset (None), defer to per-user privilege checks below.
# Web search is per-turn opt-in: either the chat pre-search setting
# (`use_web=true`) or agent web toggle (`allow_web_search=true`) must
# explicitly enable it.
if allow_bash is not None and str(allow_bash).lower() != "true":
disabled_tools.add("bash")
if (
allow_web_search is not None
and str(allow_web_search).lower() != "true"
):
disabled_tools.add("web_search")
disabled_tools.add("web_fetch")
_explicit_web_intent = bool(_tool_intent and _tool_intent.category == "web")
if is_web_search_explicitly_denied(allow_web_search) or not _search_enabled:
disabled_tools.update(WEB_TOOL_NAMES)
if _explicit_web_intent:
# A direct lookup/search request should not drift into personal
# tools or shell fallbacks. We still keep web_search/web_fetch
# available even when the frontend toggle is stale/falsy because
# the user's words are the stronger signal.
# tools or shell fallbacks. It can only use web_search/web_fetch
# when the request's explicit web setting enabled them.
disabled_tools.update({
"bash", "python",
"search_chats", "manage_skills", "manage_memory",
@@ -896,11 +895,12 @@ def setup_chat_routes(
"manage_notes", "manage_calendar", "manage_tasks",
"api_call", "builtin_browser",
})
disabled_tools.discard("web_search")
disabled_tools.discard("web_fetch")
if _search_enabled:
disabled_tools.difference_update(WEB_TOOL_NAMES)
else:
disabled_tools.update(WEB_TOOL_NAMES)
elif _search_enabled:
disabled_tools.discard("web_search")
disabled_tools.discard("web_fetch")
disabled_tools.difference_update(WEB_TOOL_NAMES)
# Nobody/incognito mode: deny tools that would expose the user's
# persistent memory, past chats, or other identity-linked data.
@@ -951,13 +951,6 @@ def setup_chat_routes(
from src.settings import get_setting
_global_disabled = get_setting("disabled_tools", [])
if _global_disabled and isinstance(_global_disabled, list):
explicit_web_allowed = (
_explicit_web_intent
or (allow_web_search is not None and str(allow_web_search).lower() == "true")
)
if explicit_web_allowed:
disabled_tools.update(t for t in _global_disabled if t not in {"web_search", "web_fetch"})
else:
disabled_tools.update(_global_disabled)
# Light auto-escalation: the user is in chat mode and just expressed a
@@ -1410,10 +1403,8 @@ def setup_chat_routes(
_max_rounds = max(1, min(_max_rounds, 200))
_forced_tools = None
if _explicit_web_intent:
_forced_tools = {"web_search", "web_fetch"}
elif _search_enabled:
_forced_tools = {"web_search", "web_fetch"}
if _search_enabled:
_forced_tools = set(WEB_TOOL_NAMES)
async for chunk in stream_agent_loop(
sess.endpoint_url,
+22
View File
@@ -1068,6 +1068,28 @@ def _scheduled_poll_once() -> dict:
for r in rows:
sid = r[0]
try:
# Atomically claim this row before doing any work. Two
# pollers can race here (the in-process asyncio task and an
# externally cron-driven `odysseus-mail poll-scheduled`, or
# an admin running the CLI manually alongside the in-process
# one despite the ODYSSEUS_INPROCESS_POLLERS=0 guidance) -
# both can SELECT the same 'pending' row before either has
# updated its status. The UPDATE...WHERE status='pending' is
# the atomicity boundary: only the poller whose UPDATE
# actually changes a row (rowcount == 1) proceeds to send;
# a loser sees rowcount == 0 and skips it instead of sending
# a duplicate.
claim_conn = sqlite3.connect(SCHEDULED_DB)
claim_cur = claim_conn.execute(
"UPDATE scheduled_emails SET status='sending' WHERE id=? AND status='pending'",
(sid,),
)
claim_conn.commit()
claimed = claim_cur.rowcount == 1
claim_conn.close()
if not claimed:
continue
attachments = json.loads(r[8] or "[]")
row_account_id = r[9] if len(r) > 9 else None
odysseus_kind = r[10] if len(r) > 10 else "scheduled"
+12 -9
View File
@@ -923,17 +923,25 @@ def _resolve_send_config(account_id: str | None = None, owner: str = "") -> dict
def _store_email_flag(conn, uid: str, flag: str, add: bool = True) -> bool:
# imaplib's plain store() takes a message SEQUENCE NUMBER, not a UID, so the
# old `else` fallback flagged whichever message happened to occupy sequence
# position == the UID value. When the UID isn't present, fail safe (callers
# surface "Email not found") rather than touch an unrelated message.
if not _uid_exists(conn, uid):
return False
op = "+FLAGS" if add else "-FLAGS"
if _uid_exists(conn, uid):
status, _ = conn.uid("STORE", _uid_bytes(uid), op, flag)
else:
status, _ = conn.store(_uid_bytes(uid), op, flag)
return status == "OK"
def _move_email_message(conn, uid: str, dest: str, role: str = "") -> bool:
dest = _resolve_mail_folder(conn, dest, role or _folder_role_from_name(dest))
if _uid_exists(conn, uid):
# copy()/store() are SEQUENCE-NUMBER commands; using them with a UID (the old
# `else` branch) copied + \Deleted-flagged the wrong message and then
# expunge() permanently removed it. There is no valid case where treating a
# UID as a sequence number is correct, so fail safe when the UID is absent.
if not _uid_exists(conn, uid):
return False
status, _ = conn.uid("MOVE", _uid_bytes(uid), _q(dest))
if status == "OK":
return True
@@ -941,11 +949,6 @@ def _move_email_message(conn, uid: str, dest: str, role: str = "") -> bool:
if status != "OK":
return False
status, _ = conn.uid("STORE", _uid_bytes(uid), "+FLAGS", "\\Deleted")
else:
status, _ = conn.copy(_uid_bytes(uid), _q(dest))
if status != "OK":
return False
status, _ = conn.store(_uid_bytes(uid), "+FLAGS", "\\Deleted")
if status == "OK":
conn.expunge()
return True
+3 -1
View File
@@ -479,7 +479,9 @@ async def dispatch_reminder(
base = intg["base_url"].rstrip("/")
topic = settings.get("reminder_ntfy_topic") or "reminders"
ntfy_body = synthesis or note_body or title
hdrs = {"Title": title or "Reminder", "Priority": "high", "Tags": "bell"}
# ntfy Title is an ASCII HTTP header; sanitize Unicode and cap its length.
_clean_title = (title or "Reminder").encode("ascii", "replace").decode("ascii")[:200]
hdrs = {"Title": _clean_title, "Priority": "high", "Tags": "bell"}
api_key = intg.get("api_key", "")
if api_key:
hdrs["Authorization"] = f"Bearer {api_key}"
+21 -7
View File
@@ -133,16 +133,30 @@ def cmd_list(args):
emit([], args)
return
entries = []
for p in sorted(_BACKUP_DIR.iterdir(), key=lambda x: x.stat().st_mtime, reverse=True):
for p in _BACKUP_DIR.iterdir():
entry = _backup_entry(p)
if entry is not None:
entries.append(entry)
entries.sort(key=lambda entry: entry["_mtime"], reverse=True)
for entry in entries:
entry.pop("_mtime", None)
emit(entries, args)
def _backup_entry(p):
try:
if not p.is_file():
continue
entries.append({
return None
st = p.stat()
except OSError:
return None
return {
"path": str(p),
"name": p.name,
"bytes": p.stat().st_size,
"modified": datetime.fromtimestamp(p.stat().st_mtime).isoformat(),
})
emit(entries, args)
"bytes": st.st_size,
"modified": datetime.fromtimestamp(st.st_mtime).isoformat(),
"_mtime": st.st_mtime,
}
def cmd_verify(args):
+1 -1
View File
@@ -90,7 +90,7 @@ def cmd_add(args):
# add_entry doesn't save by default — the call in chat does it
# after dedup checks. Persist here so a one-shot CLI add sticks.
all_entries = _manager().load_all()
if not any(e.get("id") == entry.get("id") for e in all_entries):
if not any(isinstance(e, dict) and e.get("id") == entry.get("id") for e in all_entries):
all_entries.append(entry)
_manager().save(all_entries)
emit(entry, args)
+2 -2
View File
@@ -145,7 +145,7 @@ def is_prequantized(model):
or re.search(r"(^|[-_/])fp8($|[-_/\s])", text) is not None
or (not (model.get("is_gguf") or model.get("gguf_sources")) and re.search(r"(^|[-_/])(?:int)?8bit($|[-_/\s])", text) is not None)
or any(x in text for x in ("awq", "gptq", "mlx"))
or any(q.startswith(p) for p in PREQUANTIZED_PREFIXES)
or any(isinstance(q, str) and q.startswith(p) for p in PREQUANTIZED_PREFIXES)
)
@@ -155,7 +155,7 @@ def params_b(model):
return raw / 1_000_000_000.0
pc = model.get("parameter_count", "")
if pc:
if isinstance(pc, str) and pc:
pc = pc.strip().upper()
m = re.match(r"^([\d.]+)\s*([BKMGT]?)$", pc)
if m:
+1 -1
View File
@@ -68,7 +68,7 @@ class TTSService:
if provider == "local":
kokoro = self._get_kokoro()
return kokoro is not None and kokoro.available
if provider.startswith("endpoint:"):
if isinstance(provider, str) and provider.startswith("endpoint:"):
return True # assume reachable; errors surface at synthesis time
return False
+10 -11
View File
@@ -24,7 +24,7 @@ from src.model_context import estimate_tokens
from src.settings import get_setting
from src.prompt_security import untrusted_context_message
from src.tool_security import blocked_tools_for_owner, plan_mode_disabled_tools
from src.tool_policy import GUIDE_ONLY_DIRECTIVE, ToolPolicy
from src.tool_policy import GUIDE_ONLY_DIRECTIVE, WEB_TOOL_NAMES, ToolPolicy
from src.tool_utils import _truncate, get_mcp_manager
from src.agent_tools import (
parse_tool_blocks,
@@ -321,7 +321,7 @@ _DOMAIN_RULES = {
}
_DOMAIN_TOOL_MAP = {
"web": {"web_search", "web_fetch"},
"web": set(WEB_TOOL_NAMES),
"documents": {"create_document", "edit_document", "update_document", "suggest_document", "manage_documents"},
"email": {"list_email_accounts", "list_emails", "read_email", "send_email", "reply_to_email", "bulk_email", "archive_email", "delete_email", "mark_email_read", "resolve_contact", "manage_contact"},
"cookbook": {"download_model", "serve_model", "serve_preset", "list_serve_presets", "list_served_models", "stop_served_model", "tail_serve_output", "list_downloads", "cancel_download", "search_hf_models", "list_cached_models", "list_cookbook_servers", "adopt_served_model"},
@@ -2847,13 +2847,12 @@ async def stream_agent_loop(
if "email" in (_intent.get("domains") or set()):
_relevant_tools.add("ui_control")
if "web" in (_intent.get("domains") or set()):
_relevant_tools.update({"web_search", "web_fetch"})
_removed_web_blocks = sorted({"web_search", "web_fetch"} & disabled_tools)
if _removed_web_blocks:
disabled_tools.difference_update({"web_search", "web_fetch"})
_relevant_tools.update(WEB_TOOL_NAMES)
_blocked_web_tools = sorted(WEB_TOOL_NAMES & disabled_tools)
if _blocked_web_tools:
logger.info(
"[agent-intent] web turn forced search tools enabled; removed disabled=%s",
_removed_web_blocks,
"[agent-intent] web domain selected but search tools remain disabled=%s",
_blocked_web_tools,
)
if "ui" in (_intent.get("domains") or set()):
_relevant_tools.add("ui_control")
@@ -2887,9 +2886,9 @@ async def stream_agent_loop(
_relevant_tools = set(ALWAYS_AVAILABLE)
_relevant_tools.update({"read_file", "grep", "ls", "manage_documents"})
# Per-request forced tools are stronger than retrieval. Search toggles and
# explicit lookup turns must make web tools visible even when tool RAG
# misses them; route-level disabled_tools decides what else is allowed.
# Per-request forced tools are stronger than retrieval. Explicit search
# settings make web tools visible even when tool RAG misses them;
# route-level disabled_tools decides what remains allowed.
if not guide_only and forced_tools:
forced_set = {t for t in forced_tools if t not in disabled_tools}
if _relevant_tools is None:
+9 -7
View File
@@ -280,7 +280,7 @@ def _sync_blocking(owner: str, url: str, username: str, password: str, account_i
result = {"calendars": 0, "events": 0, "deleted": 0, "errors": []}
client = _build_dav_client(url, username, password)
try:
# Discovery: try principal → calendars first; if the server doesn't
# support discovery (or the URL points directly at a calendar), fall
# back to treating the URL as a single calendar.
@@ -290,26 +290,26 @@ def _sync_blocking(owner: str, url: str, username: str, password: str, account_i
calendars = principal.calendars()
except (AuthorizationError, NotFoundError) as e:
result["errors"].append(f"Discovery failed: {e}")
return result
return result # outer finally will call client.close()
except Exception as e:
logger.info(f"CalDAV principal discovery failed, trying URL as calendar: {e}")
try:
calendars = [_open_url_as_calendar(client, url)]
except Exception as e2:
result["errors"].append(f"Could not open URL as calendar: {e2}")
return result
return result # outer finally will call client.close()
if not calendars:
try:
calendars = [_open_url_as_calendar(client, url)]
except Exception as e:
result["errors"].append(f"No calendars and URL fallback failed: {e}")
return result
return result # outer finally will call client.close()
start = datetime.utcnow() - timedelta(days=_LOOKBACK_DAYS)
end = datetime.utcnow() + timedelta(days=_LOOKAHEAD_DAYS)
db = SessionLocal()
db = SessionLocal() # if this raises, outer finally still calls client.close()
try:
for remote_cal in calendars:
try:
@@ -357,7 +357,7 @@ def _sync_blocking(owner: str, url: str, username: str, password: str, account_i
seen_uids = set()
# Track events added to the session but not yet committed so
# duplicate UIDs within the same batch are updated, not re-inserted
# (which would violate the UNIQUE constraint on commit).
# (which would violates the UNIQUE constraint on commit).
pending: dict = {}
parse_failed = False
try:
@@ -485,9 +485,11 @@ def _sync_blocking(owner: str, url: str, username: str, password: str, account_i
result["errors"].append(str(e)[:200])
db.rollback()
finally:
db.close()
db.close() # NOT client.close() here anymore
return result
finally:
client.close() # always called
def _event_payload(ev) -> dict:
+3
View File
@@ -192,11 +192,14 @@ def _writeback_blocking(local_cal_id, ev, delete, url, username, password,
# Redirects disabled here too: the write-back path opens its own DAVClient,
# so it needs the same SSRF-via-redirect protection as the pull path.
client = _build_dav_client(url, username, password)
try:
calendars = _discover_calendars(client)
if not calendars:
return {"ok": False, "error": "no remote calendars discovered"}
return push_event(calendars, local_cal_id, ev, delete=delete,
owner=owner, account_id=account_id)
finally:
client.close()
def _persist_writeback_result(owner: str, calendar_id: str, uid: str, result: dict, *, delete: bool) -> None:
+4 -1
View File
@@ -230,7 +230,10 @@ def request_flags(messages) -> tuple:
"""
msgs = messages or []
last = msgs[-1] if msgs else None
agent = bool(last) and last.get("role") != "user"
# A message element can be a non-dict (clients send `"messages": ["hi"]`);
# the vision loop below already guards each element with isinstance, so do
# the same here rather than call .get() on a bare string.
agent = isinstance(last, dict) and last.get("role") != "user"
vision = False
for m in msgs:
content = m.get("content") if isinstance(m, dict) else None
+5 -2
View File
@@ -440,7 +440,10 @@ def build_user_content(
try:
with open(path, "rb") as image_file:
encoded_string = base64.b64encode(image_file.read()).decode("utf-8")
image_format = ext[1:]
# Extensionless uploads (e.g. a pasted screenshot) have no ext,
# so fall back to the resolved MIME subtype rather than emitting
# an invalid "data:image/;base64," with an empty subtype.
image_format = ext[1:] or (mime.split("/", 1)[1] if mime.startswith("image/") else "png")
content.append({
"type": "image_url",
"image_url": {"url": f"data:image/{image_format};base64,{encoded_string}"},
@@ -456,7 +459,7 @@ def build_user_content(
try:
with open(path, "rb") as audio_file:
encoded_string = base64.b64encode(audio_file.read()).decode("utf-8")
audio_format = ext[1:]
audio_format = ext[1:] or (mime.split("/", 1)[1] if mime.startswith("audio/") else "mpeg")
content.append({
"type": "audio",
"audio": {"url": f"data:audio/{audio_format};base64,{encoded_string}"},
+5 -1
View File
@@ -96,7 +96,9 @@ class DbTokenStorage:
try:
srv = db.query(McpServer).filter(McpServer.id == self.server_id).first()
if srv and srv.oauth_tokens:
return json.loads(srv.oauth_tokens)
parsed = json.loads(srv.oauth_tokens)
if isinstance(parsed, dict):
return parsed
finally:
db.close()
return {}
@@ -111,6 +113,8 @@ class DbTokenStorage:
if srv is None:
return
data = json.loads(srv.oauth_tokens) if srv.oauth_tokens else {}
if not isinstance(data, dict):
data = {}
data[key] = value
srv.oauth_tokens = json.dumps(data)
db.commit()
+33
View File
@@ -16,6 +16,39 @@ GUIDE_ONLY_DIRECTIVE = (
"output they will produce locally."
)
WEB_TOOL_NAMES = frozenset({"web_search", "web_fetch"})
def tool_toggle_enabled(value: object) -> bool:
"""Return true only for explicit true-like tool toggle values."""
return str(value).lower() == "true"
def tool_toggle_explicitly_denied(value: object) -> bool:
"""Return true when a caller explicitly supplied a non-true toggle value."""
return value is not None and not tool_toggle_enabled(value)
def is_web_search_explicitly_denied(allow_web_search: object) -> bool:
"""Whether the web-search agent toggle was explicitly set to false."""
return tool_toggle_explicitly_denied(allow_web_search)
def web_search_enabled_for_turn(allow_web_search: object, use_web: object = None) -> bool:
"""Return true only when this request explicitly enables web search.
Agent mode sends ``allow_web_search``; chat-mode pre-search sends
``use_web``. If both are present, an explicit ``allow_web_search=false``
wins so a stale or conflicting intent path cannot re-enable web tools.
"""
if is_web_search_explicitly_denied(allow_web_search):
return False
return tool_toggle_enabled(allow_web_search) or tool_toggle_enabled(use_web)
_COMMON_TOOL_NAMES = {
"api_call",
+8 -4
View File
@@ -356,7 +356,11 @@ async def do_manage_tasks(content: str, owner: Optional[str] = None) -> Dict:
task = db.query(ScheduledTask).filter(ScheduledTask.id == task_id).first()
if not task:
return {"error": f"Task {task_id} not found", "exit_code": 1}
if owner and task.owner and task.owner != owner:
# Strict ownership: the old `task.owner and task.owner != owner`
# skipped the check on an owner-less task (created in no-login mode
# or before the legacy-owner sweep), letting any authenticated user
# reach it. `list` already scopes to an exact owner match.
if owner and task.owner != owner:
return {"error": "Access denied", "exit_code": 1}
changed = []
@@ -402,7 +406,7 @@ async def do_manage_tasks(content: str, owner: Optional[str] = None) -> Dict:
task = db.query(ScheduledTask).filter(ScheduledTask.id == task_id).first()
if not task:
return {"error": f"Task {task_id} not found", "exit_code": 1}
if owner and task.owner and task.owner != owner:
if owner and task.owner != owner:
return {"error": "Access denied", "exit_code": 1}
name = task.name
db.delete(task)
@@ -416,7 +420,7 @@ async def do_manage_tasks(content: str, owner: Optional[str] = None) -> Dict:
task = db.query(ScheduledTask).filter(ScheduledTask.id == task_id).first()
if not task:
return {"error": f"Task {task_id} not found", "exit_code": 1}
if owner and task.owner and task.owner != owner:
if owner and task.owner != owner:
return {"error": "Access denied", "exit_code": 1}
if action == "pause":
@@ -437,7 +441,7 @@ async def do_manage_tasks(content: str, owner: Optional[str] = None) -> Dict:
task = db.query(ScheduledTask).filter(ScheduledTask.id == task_id).first()
if not task:
return {"error": f"Task {task_id} not found", "exit_code": 1}
if owner and task.owner and task.owner != owner:
if owner and task.owner != owner:
return {"error": "Access denied", "exit_code": 1}
from src.event_bus import get_task_scheduler
+1 -1
View File
@@ -16,7 +16,7 @@ const _defaultKeybinds = {
};
export function _matchesCombo(e, combo, isMac = IS_MAC) {
if (!combo) return false;
if (typeof combo !== 'string' || !combo) return false;
// Drop AltGr keystrokes so typing characters on non-US layouts can't fire a
// Ctrl+Alt shortcut — e.g. the destructive delete_session. See platform.js.
if (isAltGrEvent(e, isMac)) return false;
+40
View File
@@ -25,6 +25,46 @@ def _verify_args(path: Path):
return SimpleNamespace(path=str(path), pretty=False)
def test_backup_entry_skips_files_that_disappear():
backup = _load_backup_cli()
class Vanished:
name = "gone.tar.gz"
def is_file(self):
return True
def stat(self):
raise FileNotFoundError("gone")
def __str__(self):
return "backups/gone.tar.gz"
assert backup._backup_entry(Vanished()) is None
def test_backup_list_sorts_by_captured_mtime(monkeypatch):
backup = _load_backup_cli()
first = SimpleNamespace(name="older.tar.gz")
second = SimpleNamespace(name="newer.tar.gz")
monkeypatch.setattr(backup, "_BACKUP_DIR", SimpleNamespace(
is_dir=lambda: True,
iterdir=lambda: [first, second],
))
monkeypatch.setattr(backup, "_backup_entry", lambda p: {
"name": p.name,
"modified": "2026-10-25T01:45:00" if p is first else "2026-10-25T01:15:00",
"_mtime": 100 if p is first else 200,
})
seen = []
monkeypatch.setattr(backup, "emit", lambda payload, args: seen.append(payload))
backup.cmd_list(SimpleNamespace(pretty=False))
assert [entry["name"] for entry in seen[0]] == ["newer.tar.gz", "older.tar.gz"]
assert all("_mtime" not in entry for entry in seen[0])
def test_snapshot_rejects_output_inside_data_dir(tmp_path, monkeypatch):
backup = _load_backup_cli()
repo = tmp_path / "repo"
+145
View File
@@ -0,0 +1,145 @@
"""Issue #4593 — the CalDAV DAVClient must be closed on every path.
`_sync_blocking` (src/caldav_sync.py) and `_writeback_blocking`
(src/caldav_writeback.py) each open their own DAVClient. The client holds an
HTTP session with pooled connections; if it is never closed those connections
leak for the lifetime of the process. These tests pin that the client is
closed on the discovery early-returns, the normal return, and the
write-back paths, using a fake client so no network or `caldav` install is
needed.
"""
import sys
import types
import pytest
from unittest.mock import MagicMock
def _stub_sync_deps(monkeypatch):
"""Make `_sync_blocking`'s lazy imports resolve without a real caldav/db."""
err_mod = types.ModuleType("caldav.lib.error")
class AuthorizationError(Exception):
pass
class NotFoundError(Exception):
pass
err_mod.AuthorizationError = AuthorizationError
err_mod.NotFoundError = NotFoundError
monkeypatch.setitem(sys.modules, "caldav", types.ModuleType("caldav"))
monkeypatch.setitem(sys.modules, "caldav.lib", types.ModuleType("caldav.lib"))
monkeypatch.setitem(sys.modules, "caldav.lib.error", err_mod)
db_mod = types.ModuleType("core.database")
db_mod.CalendarCal = MagicMock()
db_mod.CalendarEvent = MagicMock()
db_mod.CalendarDeletedEvent = MagicMock()
db_mod.SessionLocal = MagicMock()
if "core" not in sys.modules:
monkeypatch.setitem(sys.modules, "core", types.ModuleType("core"))
monkeypatch.setitem(sys.modules, "core.database", db_mod)
# Stub routes.calendar_routes so the lazy import of _ensure_positive_duration
# inside _sync_blocking doesn't drag in dateutil / FastAPI / SQLAlchemy.
routes_mod = types.ModuleType("routes")
cal_routes_mod = types.ModuleType("routes.calendar_routes")
cal_routes_mod._ensure_positive_duration = lambda start, end, all_day: end
if "routes" not in sys.modules:
monkeypatch.setitem(sys.modules, "routes", routes_mod)
monkeypatch.setitem(sys.modules, "routes.calendar_routes", cal_routes_mod)
return AuthorizationError
def test_sync_closes_client_on_discovery_auth_failure(monkeypatch):
import src.caldav_sync as sync
AuthorizationError = _stub_sync_deps(monkeypatch)
client = MagicMock()
client.principal.side_effect = AuthorizationError("bad credentials")
monkeypatch.setattr(sync, "_build_dav_client", lambda *a, **k: client)
result = sync._sync_blocking("alice", "https://dav.example.com/", "u", "p")
client.close.assert_called_once()
assert any("Discovery failed" in e for e in result["errors"])
def test_sync_closes_client_when_url_fallback_fails(monkeypatch):
import src.caldav_sync as sync
_stub_sync_deps(monkeypatch)
client = MagicMock()
# principal() raises a generic error -> the URL-as-calendar fallback is
# tried; make that fail too so the function hits the early return.
client.principal.side_effect = RuntimeError("no principal endpoint")
monkeypatch.setattr(sync, "_build_dav_client", lambda *a, **k: client)
monkeypatch.setattr(
sync, "_open_url_as_calendar",
MagicMock(side_effect=RuntimeError("not a calendar")),
)
result = sync._sync_blocking("alice", "https://dav.example.com/", "u", "p")
client.close.assert_called_once()
assert result["errors"]
def test_writeback_closes_client_when_no_calendars(monkeypatch):
import src.caldav_sync as sync
import src.caldav_writeback as wb
client = MagicMock()
monkeypatch.setattr(sync, "_build_dav_client", lambda *a, **k: client)
monkeypatch.setattr(wb, "_discover_calendars", lambda c: [])
result = wb._writeback_blocking(
"caldav-1", {"uid": "evt-1"}, False, "https://dav.example.com/", "u", "p"
)
client.close.assert_called_once()
assert result["ok"] is False
def test_writeback_closes_client_on_success(monkeypatch):
import src.caldav_sync as sync
import src.caldav_writeback as wb
client = MagicMock()
monkeypatch.setattr(sync, "_build_dav_client", lambda *a, **k: client)
monkeypatch.setattr(wb, "_discover_calendars", lambda c: [MagicMock()])
monkeypatch.setattr(wb, "push_event", lambda *a, **k: {"ok": True})
result = wb._writeback_blocking(
"caldav-1", {"uid": "evt-1"}, False, "https://dav.example.com/", "u", "p"
)
client.close.assert_called_once()
assert result["ok"] is True
def test_sync_closes_client_when_session_local_raises(monkeypatch):
import src.caldav_sync as sync
AuthorizationError = _stub_sync_deps(monkeypatch)
# Give principal() a working response so discovery passes
mock_principal = MagicMock()
mock_cal = MagicMock()
mock_cal.url = "https://dav.example.com/alice/home/"
mock_principal.calendars.return_value = [mock_cal]
client = MagicMock()
client.principal.return_value = mock_principal
monkeypatch.setattr(sync, "_build_dav_client", lambda *a, **k: client)
# Make SessionLocal blow up before any DB work
import sys
sys.modules["core.database"].SessionLocal.side_effect = RuntimeError("DB unavailable")
with pytest.raises(RuntimeError, match="DB unavailable"):
sync._sync_blocking("alice", "https://dav.example.com/", "u", "p")
client.close.assert_called_once()
@@ -93,6 +93,10 @@ class _FakeClient:
def calendar(self, url=None):
return _FakeCalendar(url)
def close(self):
# Mirror the real DAVClient: sync now closes the client on every path.
self.closed = True
def _install_fake_caldav(monkeypatch):
fake = types.ModuleType("caldav")
+194
View File
@@ -0,0 +1,194 @@
"""Regression: CalDAV test_connection must trust the operator's CA bundle.
The pre-flight used httpx with trust_env=False, which ignored
SSL_CERT_FILE/REQUESTS_CA_BUNDLE. Self-signed CalDAV servers that the
real sync accepts (via caldav lib -> requests -> honors bundle) were
rejected by the test with CERTIFICATE_VERIFY_FAILED.
These tests exercise the *route handler* directly (via ASGI TestClient)
and capture the verify= kwarg passed to httpx.AsyncClient, ensuring the
route code not a test-side duplicate builds the SSL context correctly.
"""
import os
import ssl
import sys
from unittest.mock import MagicMock, patch
import httpx
import pytest
# No module-level sys.modules stubbing here: conftest pre-imports the real
# sqlalchemy/core.database, and stubbing extras (e.g. caldav) at collection
# time leaks MagicMocks into later tests in the same process — it made
# test_caldav_redirect_hardening's real DAVClient a mock that never sent
# the PROPFIND. The route's lazy imports are patched per-request instead.
def _fake_response(status_code=207, headers=None):
resp = MagicMock()
resp.status_code = status_code
resp.headers = headers or {}
return resp
@pytest.fixture()
def client():
from fastapi import FastAPI
from fastapi.testclient import TestClient
from routes.calendar_routes import setup_calendar_routes
with patch("routes.calendar_routes._require_user", return_value="test-owner"):
router = setup_calendar_routes()
app = FastAPI()
app.include_router(router)
return TestClient(app)
def _make_fake_async_client(captured):
"""Return a fake httpx.AsyncClient class that captures constructor kwargs."""
class FakeAsyncClient:
def __init__(self, **kwargs):
captured.update(kwargs)
async def __aenter__(self):
return self
async def __aexit__(self, *a):
pass
async def request(self, *a, **kw):
return _fake_response(207)
return FakeAsyncClient
def _post_test(client, captured, env=None):
"""POST /api/calendar/test with credentials in body so no DB lookup needed.
Patches httpx.AsyncClient at the real module level so the route's
``import httpx; httpx.AsyncClient(...)`` picks up the fake class.
Also stubs validate_caldav_url (lazy-imported from src.caldav_sync).
"""
fake_cls = _make_fake_async_client(captured)
# Stub the caldav_sync module so the lazy `from src.caldav_sync import validate_caldav_url`
# inside the route body resolves to a pass-through.
caldav_sync_stub = MagicMock()
caldav_sync_stub.validate_caldav_url = lambda u: u
ctx_managers = [
patch.object(httpx, "AsyncClient", fake_cls),
patch.dict(sys.modules, {"src.caldav_sync": caldav_sync_stub}),
patch("routes.calendar_routes._require_user", return_value="test-owner"),
]
if env is not None:
ctx_managers.append(patch.dict(os.environ, env))
# Enter all context managers
for cm in ctx_managers:
cm.__enter__()
try:
return client.post(
"/api/calendar/test",
json={"url": "https://cal.example.com", "username": "u", "password": "p"},
)
finally:
for cm in reversed(ctx_managers):
cm.__exit__(None, None, None)
# ---------------------------------------------------------------------------
# Route-level tests
# ---------------------------------------------------------------------------
def test_route_passes_ssl_context_with_correct_flags(client):
"""The route must pass an ssl.SSLContext to httpx.AsyncClient(verify=...)
with trust_env=False, follow_redirects=False, and VERIFY_X509_STRICT cleared."""
captured = {}
resp = _post_test(client, captured)
assert resp.status_code == 200
assert isinstance(captured.get("verify"), ssl.SSLContext), (
f"verify= should be an ssl.SSLContext, got {type(captured.get('verify'))}"
)
assert captured.get("trust_env") is False
assert captured.get("follow_redirects") is False
ctx = captured["verify"]
assert not (ctx.verify_flags & ssl.VERIFY_X509_STRICT), (
"VERIFY_X509_STRICT must be cleared for self-signed CA compat"
)
def test_route_ssl_cert_file_takes_precedence(client, tmp_path):
"""SSL_CERT_FILE is the exact bundle loaded when both variables are set."""
bundle_a = tmp_path / "ssl-cert-file.pem"
bundle_b = tmp_path / "requests-ca-bundle.pem"
bundle_a.write_text("ssl-cert-file", encoding="utf-8")
bundle_b.write_text("requests-ca-bundle", encoding="utf-8")
loaded = []
class FakeSSLContext:
def __init__(self):
self.verify_flags = ssl.VERIFY_X509_STRICT
def load_verify_locations(self, cafile=None, capath=None, cadata=None):
loaded.append(
{
"cafile": cafile,
"capath": capath,
"cadata": cadata,
}
)
ssl_context = FakeSSLContext()
captured = {}
env = {
"SSL_CERT_FILE": str(bundle_a),
"REQUESTS_CA_BUNDLE": str(bundle_b),
}
with patch.object(
ssl,
"create_default_context",
return_value=ssl_context,
):
resp = _post_test(client, captured, env=env)
assert resp.status_code == 200
assert resp.json() == {"ok": True}
assert loaded == [
{
"cafile": str(bundle_a),
"capath": None,
"cadata": None,
}
]
assert captured.get("verify") is ssl_context
assert captured.get("trust_env") is False
assert captured.get("follow_redirects") is False
assert not (
ssl_context.verify_flags & ssl.VERIFY_X509_STRICT
)
def test_route_missing_bundle_does_not_crash(client):
"""A nonexistent CA bundle path must not crash -- fall back to system CAs."""
captured = {}
resp = _post_test(client, captured, env={"SSL_CERT_FILE": "/nonexistent/ca-bundle.pem"})
assert resp.status_code == 200
ctx = captured["verify"]
assert isinstance(ctx, ssl.SSLContext)
assert not (ctx.verify_flags & ssl.VERIFY_X509_STRICT)
def test_route_empty_env_vars_use_system_defaults(client):
"""Empty SSL_CERT_FILE and REQUESTS_CA_BUNDLE should not crash."""
captured = {}
resp = _post_test(client, captured, env={"SSL_CERT_FILE": "", "REQUESTS_CA_BUNDLE": ""})
assert resp.status_code == 200
ctx = captured["verify"]
assert isinstance(ctx, ssl.SSLContext)
assert not (ctx.verify_flags & ssl.VERIFY_X509_STRICT)
+83 -23
View File
@@ -1,12 +1,11 @@
"""Issue #3229 — allow_bash / allow_web_search must work for JSON API callers
and admin users must get bash enabled by default.
"""Issue #3229 and explicit web-toggle regressions.
Bug: allow_bash and allow_web_search were only read from form_data, so JSON
API callers (Content-Type: application/json) always had bash disabled.
Fix: (1) Read from JSON body as fallback.
(2) Only add bash/web_search to disabled_tools when explicitly set to a
falsy value; when unset (None), defer to per-user privilege checks.
(2) Keep bash on the privilege fallback when unset.
(3) Require an explicit per-turn web setting before exposing web tools.
"""
import ast
@@ -15,6 +14,11 @@ from pathlib import Path
import pytest
from src.action_intents import classify_tool_intent
from src.tool_policy import (
WEB_TOOL_NAMES,
is_web_search_explicitly_denied,
web_search_enabled_for_turn,
)
_CHAT_ROUTES = Path(__file__).resolve().parent.parent / "routes" / "chat_routes.py"
@@ -76,8 +80,7 @@ def test_allow_web_search_reads_from_body_as_fallback():
def test_disabled_tools_respects_missing_vs_explicit_toggles():
"""When allow_bash is not set (None), bash must NOT be unconditionally
added to disabled_tools. The per-user privilege check handles it.
"""Bash still defers to privileges, but web is an explicit per-turn opt-in.
"""
source = _CHAT_ROUTES.read_text(encoding="utf-8")
@@ -88,11 +91,14 @@ def test_disabled_tools_respects_missing_vs_explicit_toggles():
assert "allow_bash is not None" in source, (
"disabled_tools check must guard against allow_bash being None"
)
assert "allow_web_search is not None" in source, (
"disabled_tools check must guard against allow_web_search being None"
assert "web_search_enabled_for_turn(allow_web_search, use_web)" in source, (
"web tools must be gated through the explicit per-turn web setting"
)
assert "and not _explicit_web_intent" not in source, (
"explicit allow_web_search=false must not be overridden by prompt web intent"
assert "disabled_tools.update(WEB_TOOL_NAMES)" in source, (
"disabled_tools must add web_search/web_fetch when web is not explicitly enabled"
)
assert "_forced_tools = set(WEB_TOOL_NAMES)" in source, (
"web tools should only be forced visible from the explicit web setting"
)
@@ -102,9 +108,11 @@ def test_disabled_tools_respects_missing_vs_explicit_toggles():
def _build_disabled_tools(
allow_bash=None,
allow_web_search=None,
use_web=None,
can_use_bash=True,
can_use_browser=True,
explicit_web_intent=False,
global_disabled=None,
):
"""Replicate the disabled-tools logic from chat_stream for unit testing.
@@ -112,21 +120,36 @@ def _build_disabled_tools(
"""
disabled_tools = set()
# Issue #3229 fix: only disable when explicitly set to a falsy value.
# Issue #3229 fix: only disable bash when explicitly set to a falsy value.
if allow_bash is not None and str(allow_bash).lower() != "true":
disabled_tools.add("bash")
if (
allow_web_search is not None
and str(allow_web_search).lower() != "true"
):
disabled_tools.add("web_search")
disabled_tools.add("web_fetch")
search_enabled = web_search_enabled_for_turn(allow_web_search, use_web)
if is_web_search_explicitly_denied(allow_web_search) or not search_enabled:
disabled_tools.update(WEB_TOOL_NAMES)
if explicit_web_intent:
disabled_tools.update({
"bash", "python",
"search_chats", "manage_skills", "manage_memory",
"read_file", "write_file", "edit_file",
"create_document", "edit_document", "update_document",
"send_email", "reply_to_email",
"manage_notes", "manage_calendar", "manage_tasks",
"api_call", "builtin_browser",
})
if search_enabled:
disabled_tools.difference_update(WEB_TOOL_NAMES)
else:
disabled_tools.update(WEB_TOOL_NAMES)
elif search_enabled:
disabled_tools.difference_update(WEB_TOOL_NAMES)
# Enforce per-user privileges
if not can_use_bash:
disabled_tools.update({"bash", "python", "read_file", "write_file"})
if not can_use_browser:
disabled_tools.add("builtin_browser")
if global_disabled and isinstance(global_disabled, list):
disabled_tools.update(global_disabled)
return disabled_tools
@@ -157,6 +180,20 @@ def test_json_body_allow_web_search_false_disables_web():
assert "web_fetch" in disabled
def test_chat_mode_use_web_true_enables_web():
"""Chat pre-search sends use_web=true as the explicit web setting."""
disabled = _build_disabled_tools(use_web="true")
assert "web_search" not in disabled
assert "web_fetch" not in disabled
def test_allow_web_search_false_wins_over_use_web_true():
"""The agent web toggle hard-denies web even if another path says use_web=true."""
disabled = _build_disabled_tools(allow_web_search="false", use_web="true")
assert "web_search" in disabled
assert "web_fetch" in disabled
@pytest.mark.parametrize(
"message",
[
@@ -180,6 +217,21 @@ def test_explicit_false_disables_web_despite_prompt_web_intent(message):
assert "web_fetch" in disabled
def test_prompt_web_intent_does_not_enable_web_without_setting():
"""Prompt-derived web intent alone must not expose web tools."""
intent = classify_tool_intent("look up the latest docs")
assert intent is not None
assert intent.category == "web"
disabled = _build_disabled_tools(
allow_web_search=None,
use_web=None,
explicit_web_intent=True,
)
assert "web_search" in disabled
assert "web_fetch" in disabled
def test_admin_user_gets_bash_enabled_by_default():
"""When allow_bash is not set and user has can_use_bash privilege,
bash must NOT be disabled.
@@ -188,13 +240,11 @@ def test_admin_user_gets_bash_enabled_by_default():
assert "bash" not in disabled
def test_admin_user_gets_web_search_enabled_by_default():
"""When allow_web_search is not set and user has normal privileges,
web_search must NOT be disabled.
"""
def test_web_search_disabled_by_default_without_explicit_turn_setting():
"""Missing web settings must not expose web tools by default."""
disabled = _build_disabled_tools(allow_web_search=None)
assert "web_search" not in disabled
assert "web_fetch" not in disabled
assert "web_search" in disabled
assert "web_fetch" in disabled
def test_non_privileged_user_without_explicit_flag_still_disabled():
@@ -213,6 +263,16 @@ def test_non_privileged_user_explicit_true_overridden_by_privilege():
assert "bash" in disabled
def test_global_disabled_web_wins_over_explicit_web_enable():
"""Admin-level disabled tools are still a hard deny."""
disabled = _build_disabled_tools(
allow_web_search="true",
global_disabled=["web_search", "web_fetch"],
)
assert "web_search" in disabled
assert "web_fetch" in disabled
def test_form_data_none_body_true_works():
"""Simulates: form_data has no allow_bash, body has allow_bash=true.
After the fallback (`form_data.get(...) or body.get(...)`), allow_bash
+12
View File
@@ -89,6 +89,18 @@ def test_request_flags_vision():
assert vision is True
def test_request_flags_non_dict_last_message_does_not_crash():
# A client can send a bare-string (non-dict) last element; before the
# isinstance guard this raised AttributeError on last.get("role").
assert copilot.request_flags(["hi"]) == (False, False)
assert copilot.request_flags([{"role": "user"}, "trailing"]) == (False, False)
def test_request_flags_empty_and_none():
assert copilot.request_flags([]) == (False, False)
assert copilot.request_flags(None) == (False, False)
def test_apply_request_headers_mutates():
h = {"X-GitHub-Api-Version": "v"}
copilot.apply_request_headers(h, [{"role": "tool", "content": "x"}])
@@ -0,0 +1,74 @@
"""Regression: extensionless image/audio uploads must get a valid MIME subtype.
The data-URL subtype was derived only from the stored file's extension
(`image_format = ext[1:]`). A pasted screenshot or any file whose stored id
carries no extension yields `ext == ""`, so the emitted URL was
`data:image/;base64,...` an empty MIME subtype (invalid per RFC 2046) that
vision/audio endpoints reject, silently dropping the attachment. When the
extension is missing, fall back to the resolved MIME subtype. Extensions that
are present are unchanged.
"""
class _Handler:
def __init__(self, uploads, image=False, audio=False):
self.uploads = uploads
self._image = image
self._audio = audio
def resolve_upload(self, fid, owner=None):
return self.uploads.get(fid)
def _inside_upload_dir(self, path):
return True
def is_image_file(self, name, mime):
return self._image and (mime or "").startswith("image/")
def is_audio_file(self, name, mime):
return self._audio and (mime or "").startswith("audio/")
def is_document_file(self, name, mime):
return False
def _blocks(content, block_type):
return [b for b in content if isinstance(b, dict) and b.get("type") == block_type]
def test_extensionless_image_uses_mime_subtype(tmp_path):
import src.document_processor as dp
p = tmp_path / ("a" * 32) # bare id, no extension
p.write_bytes(b"\x89PNG\r\n\x1a\nfake")
uploads = {"img": {"path": str(p), "name": "screenshot", "mime": "image/png"}}
content = dp.build_user_content("look", ["img"], str(tmp_path), _Handler(uploads, image=True), owner="t")
imgs = _blocks(content, "image_url")
assert imgs, content
assert imgs[0]["image_url"]["url"].startswith("data:image/png;base64,")
def test_extensionless_audio_uses_mime_subtype(tmp_path):
import src.document_processor as dp
p = tmp_path / ("b" * 32)
p.write_bytes(b"fakeaudio")
uploads = {"aud": {"path": str(p), "name": "recording", "mime": "audio/mpeg"}}
content = dp.build_user_content("listen", ["aud"], str(tmp_path), _Handler(uploads, audio=True), owner="t")
auds = _blocks(content, "audio")
assert auds, content
assert auds[0]["audio"]["url"].startswith("data:audio/mpeg;base64,")
def test_extension_present_is_unchanged(tmp_path):
import src.document_processor as dp
p = tmp_path / "pic.png"
p.write_bytes(b"\x89PNG\r\n\x1a\n")
uploads = {"img": {"path": str(p), "name": "pic.png", "mime": "image/png"}}
content = dp.build_user_content("look", ["img"], str(tmp_path), _Handler(uploads, image=True), owner="t")
imgs = _blocks(content, "image_url")
assert imgs[0]["image_url"]["url"].startswith("data:image/png;base64,")
+88
View File
@@ -0,0 +1,88 @@
"""Email move/flag must never fall back to sequence-number IMAP ops (#1874 sibling).
`imaplib`'s plain `store()` / `copy()` operate on message SEQUENCE NUMBERS, not
UIDs. `_store_email_flag` / `_move_email_message` (used by the archive / delete /
move / mark endpoints) had an `else` fallback that, when `_uid_exists` returned
False, ran `conn.store(uid, )` / `conn.copy(uid, )` + `conn.expunge()` i.e.
it flagged/copied whichever message occupied sequence position == the UID value
and then permanently expunged it. A stale cached UID (or a server whose UID
probe misbehaves) therefore deleted an unrelated email.
The fix fails safe: when the UID isn't present, return False (callers surface
"Email not found") and never touch a message by sequence number.
This is distinct from #1874, which fixes the auto-spam poller's `_imap_move` in
`routes/email_helpers.py`; this covers the user-facing endpoints in
`routes/email_routes.py`.
"""
import pytest
from routes import email_routes
from routes.email_routes import _store_email_flag, _move_email_message
class _FakeConn:
"""Records IMAP calls. `uid_present` controls the FETCH-UID probe result.
The sequence-number commands (store/copy/expunge) raise if ever called
the whole point of the fix is that they must not be reached.
"""
def __init__(self, uid_present, uid_move_ok=True):
self.uid_present = uid_present
self.uid_move_ok = uid_move_ok
self.uid_calls = []
self.seqno_calls = []
def uid(self, command, *args):
self.uid_calls.append((command.upper(), args))
cmd = command.upper()
if cmd == "FETCH":
return ("OK", [b"1 (UID 5031)"] if self.uid_present else [])
if cmd == "MOVE":
return ("OK" if self.uid_move_ok else "NO", [b""])
if cmd in ("COPY", "STORE"):
return ("OK", [b""])
return ("OK", [b""])
# Sequence-number APIs — must never be used with a UID.
def store(self, *a):
self.seqno_calls.append(("store", a)); return ("OK", [b""])
def copy(self, *a):
self.seqno_calls.append(("copy", a)); return ("OK", [b""])
def expunge(self, *a):
self.seqno_calls.append(("expunge", a)); return ("OK", [b""])
@pytest.fixture(autouse=True)
def _no_folder_resolution(monkeypatch):
# _move_email_message resolves the destination folder via the connection;
# short-circuit it so the test focuses on the UID-vs-seqno behaviour.
monkeypatch.setattr(email_routes, "_resolve_mail_folder", lambda conn, dest, role="": dest)
def test_store_flag_missing_uid_fails_safe():
conn = _FakeConn(uid_present=False)
assert _store_email_flag(conn, "5031", "\\Deleted", add=True) is False
assert conn.seqno_calls == [] # never touched a message by sequence number
def test_move_missing_uid_fails_safe():
conn = _FakeConn(uid_present=False)
assert _move_email_message(conn, "5031", "Trash", role="trash") is False
assert conn.seqno_calls == [] # no copy/store/expunge on a phantom seqno
def test_store_flag_present_uid_uses_uid_store():
conn = _FakeConn(uid_present=True)
assert _store_email_flag(conn, "5031", "\\Seen", add=True) is True
assert any(c[0] == "STORE" for c in conn.uid_calls)
assert conn.seqno_calls == []
def test_move_present_uid_uses_uid_move():
conn = _FakeConn(uid_present=True, uid_move_ok=True)
assert _move_email_message(conn, "5031", "Archive", role="archive") is True
assert any(c[0] == "MOVE" for c in conn.uid_calls)
assert conn.seqno_calls == []
@@ -0,0 +1,31 @@
"""Harden hwfit model-catalog parsing against non-string field values.
`params_b` and `is_prequantized` read free-form fields straight off the HF
catalog JSON. `parameter_count` is normally a string like "7B" and
`quantization` a string like "FP8", but a catalog row can carry a non-string
(e.g. an integer parameter_count, or a null/number quantization). The code
called `pc.strip()` / `q.startswith(...)` directly, so one such row raised
AttributeError and aborted the whole ranking pass (params_b/is_prequantized
run for every model). Non-strings are now treated as unknown.
"""
from services.hwfit.models import params_b, is_prequantized
def test_params_b_nonstring_count_does_not_raise():
assert params_b({"parameter_count": 7}) == 0.0
assert params_b({"parameter_count": ["7B"]}) == 0.0
def test_params_b_valid_count_still_parses():
assert params_b({"parameter_count": "7B"}) == 7.0
assert params_b({"parameters_raw": 7_000_000_000}) == 7.0
def test_is_prequantized_nonstring_quantization_does_not_raise():
assert is_prequantized({"quantization": 8}) is False
assert is_prequantized({"name": "plain-model", "quantization": 123}) is False
def test_is_prequantized_still_detects_real_markers():
assert is_prequantized({"name": "some-model-awq"}) is True
assert is_prequantized({"quantization": "FP8-Mixed"}) is True
+139
View File
@@ -0,0 +1,139 @@
"""manage_tasks mutations must fail closed on owner-less / cross-owner tasks.
The edit/delete/pause/run actions of ``do_manage_tasks`` previously gated with
``if owner and task.owner and task.owner != owner``. The middle term made the
check a no-op whenever the task had no owner the state a scheduled task is in
when it was created in no-login mode (or via the localhost middleware bypass)
before the periodic legacy-owner sweep reassigns it to the admin user. So any
authenticated user's agent could edit, delete, pause, or *run* another tenant's
owner-less task. The sibling ``list`` action already scopes with an exact
``ScheduledTask.owner == owner`` filter, so the mutators were strictly more
permissive than the reader.
"""
import json
import tempfile
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.pool import NullPool
from tests.helpers.import_state import clear_fake_database_modules
clear_fake_database_modules()
import core.database as cdb
from core.database import ScheduledTask
from src.tools.system import do_manage_tasks
_TMPDB = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
_ENGINE = create_engine(
f"sqlite:///{_TMPDB.name}",
connect_args={"check_same_thread": False},
poolclass=NullPool,
)
cdb.Base.metadata.create_all(_ENGINE)
_TS = sessionmaker(bind=_ENGINE, autoflush=False, autocommit=False)
# do_manage_tasks does `from core.database import SessionLocal` at call time,
# so patching the module attribute is enough to point it at the temp DB.
cdb.SessionLocal = _TS
def _seed(task_id, owner):
db = _TS()
try:
db.add(ScheduledTask(
id=task_id, owner=owner, name=task_id, prompt="original",
task_type="llm", trigger_type="webhook", status="active",
output_target="session",
))
db.commit()
finally:
db.close()
def _get(task_id):
db = _TS()
try:
return db.query(ScheduledTask).filter(ScheduledTask.id == task_id).first()
finally:
db.close()
@pytest.mark.asyncio
async def test_edit_denied_on_ownerless_task_for_authenticated_user():
_seed("ownerless-edit", None)
out = await do_manage_tasks(
json.dumps({"action": "edit", "task_id": "ownerless-edit", "prompt": "pwned"}),
owner="alice",
)
assert out["exit_code"] == 1 and out["error"] == "Access denied"
assert _get("ownerless-edit").prompt == "original"
@pytest.mark.asyncio
async def test_delete_denied_on_ownerless_task_for_authenticated_user():
_seed("ownerless-del", None)
out = await do_manage_tasks(
json.dumps({"action": "delete", "task_id": "ownerless-del"}),
owner="alice",
)
assert out["exit_code"] == 1 and out["error"] == "Access denied"
assert _get("ownerless-del") is not None
@pytest.mark.asyncio
async def test_pause_denied_on_ownerless_task_for_authenticated_user():
_seed("ownerless-pause", None)
out = await do_manage_tasks(
json.dumps({"action": "pause", "task_id": "ownerless-pause"}),
owner="alice",
)
assert out["exit_code"] == 1 and out["error"] == "Access denied"
assert _get("ownerless-pause").status == "active"
@pytest.mark.asyncio
async def test_run_denied_on_ownerless_task_for_authenticated_user():
_seed("ownerless-run", None)
out = await do_manage_tasks(
json.dumps({"action": "run", "task_id": "ownerless-run"}),
owner="alice",
)
assert out["exit_code"] == 1 and out["error"] == "Access denied"
@pytest.mark.asyncio
async def test_edit_denied_on_other_owners_task():
_seed("bob-task", "bob")
out = await do_manage_tasks(
json.dumps({"action": "edit", "task_id": "bob-task", "prompt": "pwned"}),
owner="alice",
)
assert out["exit_code"] == 1 and out["error"] == "Access denied"
assert _get("bob-task").prompt == "original"
@pytest.mark.asyncio
async def test_edit_allowed_for_matching_owner():
_seed("alice-task", "alice")
out = await do_manage_tasks(
json.dumps({"action": "edit", "task_id": "alice-task", "prompt": "updated"}),
owner="alice",
)
assert out["exit_code"] == 0
assert _get("alice-task").prompt == "updated"
@pytest.mark.asyncio
async def test_edit_allowed_in_no_login_mode():
# owner is None when auth is disabled — single-user mode keeps full access
# to shared (owner-less) tasks, exactly as `list` returns them unfiltered.
_seed("shared-task", None)
out = await do_manage_tasks(
json.dumps({"action": "edit", "task_id": "shared-task", "prompt": "updated"}),
owner=None,
)
assert out["exit_code"] == 0
assert _get("shared-task").prompt == "updated"
+47
View File
@@ -0,0 +1,47 @@
"""Pin _matchesCombo (static/js/keyboard-shortcuts.js) against a non-string
keybind. Driven through `node --input-type=module` (same approach as
tests/test_markdown_table_row_js.py); skips when `node` is missing.
Regression: keybinds are merged from the server response of
`/api/auth/settings` (`{ ..._defaultKeybinds, ...s.keybinds }`). A corrupt
or malformed `keybinds` value (e.g. a number instead of "ctrl+k") reached
`combo.split('+')` and threw "combo.split is not a function", breaking the
whole keydown handler. The guard treats any non-string combo as "no match".
"""
import json
import shutil
import subprocess
from pathlib import Path
import pytest
_REPO = Path(__file__).resolve().parent.parent
_MOD = _REPO / "static" / "js" / "keyboard-shortcuts.js"
_HAS_NODE = shutil.which("node") is not None
_EVENT = "{key:'k',ctrlKey:false,altKey:false,shiftKey:false,metaKey:false}"
def _match(combo_js):
js = f"""
import {{ _matchesCombo }} from '{_MOD.as_posix()}';
console.log(JSON.stringify(_matchesCombo({_EVENT}, {combo_js})));
"""
proc = subprocess.run(
["node", "--input-type=module"],
input=js, capture_output=True, text=True, cwd=str(_REPO), timeout=30,
)
assert proc.returncode == 0, proc.stderr
return json.loads(proc.stdout.strip())
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_non_string_combo_is_no_match():
assert _match("123") is False
assert _match("{}") is False
assert _match("null") is False
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_matching_combo_still_fires():
assert _match("'k'") is True
+51
View File
@@ -1,4 +1,5 @@
import asyncio
import json
from src import mcp_oauth
@@ -79,3 +80,53 @@ def test_db_token_storage_round_trip():
t = asyncio.run(go())
assert t.access_token == "abc"
assert srv.oauth_tokens is not None # persisted as JSON
def _fake_storage(oauth_tokens):
class FakeSrv:
pass
srv = FakeSrv()
srv.oauth_tokens = oauth_tokens
class FakeQuery:
def filter(self, *a):
return self
def first(self):
return srv
class FakeSession:
def query(self, *a):
return FakeQuery()
def commit(self):
pass
def close(self):
pass
return srv, mcp_oauth.DbTokenStorage("srv-1", session_factory=lambda: FakeSession())
def test_load_falls_back_to_empty_dict_for_non_dict_json():
# A corrupted/migrated oauth_tokens column holding a JSON array, not an
# object, must not crash _load()'s callers with AttributeError.
_srv, storage = _fake_storage('["stale", "data"]')
assert storage._load() == {}
def test_get_tokens_returns_none_for_non_dict_oauth_tokens():
_srv, storage = _fake_storage("42")
async def go():
return await storage.get_tokens()
assert asyncio.run(go()) is None
def test_update_recovers_from_non_dict_oauth_tokens():
# _update() must not raise TypeError trying to item-assign into a list.
srv, storage = _fake_storage('["stale", "data"]')
storage._update("tokens", {"access_token": "new"})
assert json.loads(srv.oauth_tokens) == {"tokens": {"access_token": "new"}}
+46
View File
@@ -0,0 +1,46 @@
"""cmd_add (scripts/odysseus-memory) must tolerate a non-dict row in the
existing store. Every other command funnels load_all() through
`_memory_entries()` (which drops non-dicts), but cmd_add iterated the raw
list in its dedup check: `any(e.get("id") == ... for e in all_entries)`
crashed with AttributeError on a corrupt/hand-edited memory.json row that
is not a dict. The isinstance check short-circuits before `.get`.
"""
import importlib.machinery
import importlib.util
import sys
import types
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock
ROOT = Path(__file__).resolve().parents[1]
def _load_cli(monkeypatch):
svc = types.ModuleType("services.memory.memory")
svc.MemoryManager = MagicMock()
monkeypatch.setitem(sys.modules, "services.memory.memory", svc)
path = ROOT / "scripts" / "odysseus-memory"
loader = importlib.machinery.SourceFileLoader("odysseus_memory_cli_add", str(path))
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
def test_cmd_add_tolerates_non_dict_existing_row(monkeypatch):
cli = _load_cli(monkeypatch)
cli._mgr = MagicMock()
cli._mgr.add_entry.return_value = {"id": "m2", "text": "new"}
cli._mgr.load_all.return_value = [
{"id": "m1", "text": "existing"},
"corrupt-row",
None,
]
emitted = []
monkeypatch.setattr(cli, "emit", lambda value, args: emitted.append(value))
cli.cmd_add(SimpleNamespace(text="new", category="fact", owner=None))
assert emitted == [{"id": "m2", "text": "new"}]
cli._mgr.save.assert_called_once()
+116
View File
@@ -0,0 +1,116 @@
"""Regression: two concurrent callers of `_scheduled_poll_once` (the
in-process 30s poller and the `odysseus-mail poll-scheduled` CLI, which the
project's own docstrings warn can race on the same SQLite when
ODYSSEUS_INPROCESS_POLLERS is left enabled alongside an external cron/systemd
driver) must not both send the same scheduled email.
The old code selected pending rows, then only updated their status to 'sent'
*after* the SMTP send completed - two overlapping calls can both SELECT the
same 'pending' row before either UPDATEs it, so both send it. The fix adds
an atomic claim step (`UPDATE ... SET status='sending' WHERE status='pending'`)
before any work happens; only the caller whose UPDATE actually changes a row
proceeds, the other sees rowcount == 0 and skips it.
This test drives two real threads through the real `_scheduled_poll_once`
against a shared SQLite file, synchronized with a barrier so both reach the
SELECT at (as close to) the same moment as possible, and asserts the send
callback fired exactly once.
"""
import sqlite3
import threading
import time
def test_concurrent_pollers_do_not_double_send(tmp_path, monkeypatch):
import routes.email_helpers as email_helpers
import routes.email_pollers as email_pollers
db_path = tmp_path / "scheduled_emails.db"
monkeypatch.setattr(email_helpers, "SCHEDULED_DB", db_path)
monkeypatch.setattr(email_pollers, "SCHEDULED_DB", db_path)
email_helpers._init_scheduled_db()
conn = sqlite3.connect(db_path)
conn.execute(
"""
INSERT INTO scheduled_emails
(id, to_addr, subject, body, attachments, send_at, created_at, status, account_id, owner)
VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)
""",
(
"sched-race-1",
"recipient@example.com",
"Subject",
"Body",
"[]",
"2000-01-01T00:00:00",
"1999-12-31T00:00:00",
"acct-alice",
"alice",
),
)
conn.commit()
conn.close()
send_calls = []
send_lock = threading.Lock()
barrier = threading.Barrier(2)
def fake_get_email_config(account_id=None, owner=""):
return {
"from_address": "alice@example.com",
"smtp_host": "smtp.example.com",
"smtp_user": "alice@example.com",
"smtp_password": "secret",
}
def fake_send_smtp_message(*args, **kwargs):
# Widen the window between the claim and the actual send so a
# buggy (unclaimed) second poller has every opportunity to also
# get past its SELECT and attempt to send.
time.sleep(0.05)
with send_lock:
send_calls.append(threading.get_ident())
class FakeImap:
def __init__(self, account_id=None, owner=""):
pass
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def append(self, folder, flags, date_time, message):
pass
monkeypatch.setattr(email_pollers, "_get_email_config", fake_get_email_config)
monkeypatch.setattr(email_pollers, "_send_smtp_message", fake_send_smtp_message)
monkeypatch.setattr(email_pollers, "_imap", FakeImap)
monkeypatch.setattr(email_pollers, "_detect_sent_folder", lambda imap: "Sent")
monkeypatch.setattr(email_pollers, "_cleanup_compose_uploads", lambda attachments: None)
results = []
def _run():
barrier.wait()
results.append(email_pollers._scheduled_poll_once())
threads = [threading.Thread(target=_run) for _ in range(2)]
for t in threads:
t.start()
for t in threads:
t.join(timeout=5)
assert len(send_calls) == 1, (
f"expected exactly one send for the racing pollers, got {len(send_calls)}: "
"the second poller must lose the atomic claim and skip the row"
)
conn = sqlite3.connect(db_path)
status = conn.execute(
"SELECT status FROM scheduled_emails WHERE id=?", ("sched-race-1",)
).fetchone()[0]
conn.close()
assert status == "sent"
+116 -1
View File
@@ -6,7 +6,12 @@ from types import SimpleNamespace
import src.agent_loop as al
from src.agent_tools import ToolBlock
from src.tool_execution import execute_tool_block
from src.tool_policy import build_effective_tool_policy, detect_guide_only_turn
from src.tool_policy import (
WEB_TOOL_NAMES,
build_effective_tool_policy,
detect_guide_only_turn,
web_search_enabled_for_turn,
)
def _collect(gen):
@@ -76,6 +81,116 @@ def test_normal_policy_preserves_existing_disabled_tools():
assert not policy.blocks("bash")
def test_web_search_enabled_for_turn_requires_explicit_enable():
assert web_search_enabled_for_turn(None, None) is False
assert web_search_enabled_for_turn("true", None) is True
assert web_search_enabled_for_turn(None, "true") is True
assert web_search_enabled_for_turn(True, None) is True
assert web_search_enabled_for_turn("false", "true") is False
assert web_search_enabled_for_turn(False, "true") is False
def _schema_names(tools):
return {
tool.get("function", {}).get("name") or tool.get("name")
for tool in (tools or [])
}
def test_agent_loop_web_intent_preserves_disabled_web_tools(monkeypatch):
_patch_loop_basics(monkeypatch)
sent_tools = []
async def _fake_stream(_candidates, messages, **kwargs):
sent_tools.append(kwargs.get("tools"))
yield _delta_chunk("ok")
yield "data: [DONE]\n\n"
monkeypatch.setattr(al, "stream_llm_with_fallback", _fake_stream, raising=False)
_collect(
al.stream_agent_loop(
"https://api.openai.com/v1",
"gpt-test",
[{"role": "user", "content": "please look up the latest CVEs"}],
max_rounds=1,
relevant_tools=set(),
disabled_tools=set(WEB_TOOL_NAMES),
)
)
assert sent_tools
assert WEB_TOOL_NAMES.isdisjoint(_schema_names(sent_tools[0]))
def test_agent_loop_forced_web_tools_filtered_by_disabled_tools(monkeypatch):
_patch_loop_basics(monkeypatch)
sent_tools = []
async def _fake_stream(_candidates, messages, **kwargs):
sent_tools.append(kwargs.get("tools"))
yield _delta_chunk("ok")
yield "data: [DONE]\n\n"
monkeypatch.setattr(al, "stream_llm_with_fallback", _fake_stream, raising=False)
_collect(
al.stream_agent_loop(
"https://api.openai.com/v1",
"gpt-test",
[{"role": "user", "content": "latest Kubernetes release"}],
max_rounds=1,
relevant_tools=set(),
forced_tools=set(WEB_TOOL_NAMES),
disabled_tools=set(WEB_TOOL_NAMES),
)
)
assert sent_tools
assert WEB_TOOL_NAMES.isdisjoint(_schema_names(sent_tools[0]))
def test_agent_loop_policy_blocks_disabled_web_tool_call_before_execution(monkeypatch):
_patch_loop_basics(monkeypatch)
called = False
async def _fake_exec(*args, **kwargs):
nonlocal called
called = True
return ("web_search", {"output": "ran", "exit_code": 0})
async def _fake_stream(_candidates, messages, **kwargs):
yield _delta_chunk('```web_search\n{"query":"current CVEs"}\n```')
yield "data: [DONE]\n\n"
monkeypatch.setattr(al, "execute_tool_block", _fake_exec, raising=False)
monkeypatch.setattr(al, "stream_llm_with_fallback", _fake_stream, raising=False)
policy = build_effective_tool_policy(
disabled_tools=WEB_TOOL_NAMES,
last_user_message="please look up the latest CVEs",
)
chunks = _collect(
al.stream_agent_loop(
"http://local.test/v1",
"local-model",
[{"role": "user", "content": "please look up the latest CVEs"}],
max_rounds=1,
relevant_tools={"web_search"},
disabled_tools=set(policy.all_disabled_names()),
tool_policy=policy,
)
)
events = _events(chunks)
blocked = [event for event in events if event.get("type") == "tool_output"]
assert called is False
assert not any(event.get("type") == "tool_start" for event in events)
assert blocked
assert blocked[0]["tool"] == "web_search"
assert blocked[0]["exit_code"] == 1
def test_executor_policy_backstop_blocks_tools():
policy = build_effective_tool_policy(last_user_message="Do not use tools.")
desc, result = asyncio.run(
@@ -0,0 +1,17 @@
from services.tts.tts_service import TTSService
def test_available_tolerates_non_string_provider(tmp_path):
"""A hand-edited/corrupt data/settings.json can store a non-string
tts_provider (e.g. null or a number). available reads it and calls
provider.startswith("endpoint:"), which raised AttributeError on a
non-str. It must instead fall through and report unavailable."""
service = TTSService(cache_dir=str(tmp_path))
service._load_settings = lambda: {
"tts_enabled": True,
"tts_provider": 123,
"tts_model": "tts-1",
"tts_voice": "alloy",
"tts_speed": "1",
}
assert service.available is False