Compare commits

..
68 Commits
Author SHA1 Message Date
github-actions[bot] 3042bfe3f1 chore: update clone traffic data [skip ci] 2026-08-15 06:32:37 +00:00
40c7df3e5b fix(server): prevent memory context injection from suppressing persona identity prompt (fixes #651) (#661)
* fix(telemetry): enable WAL mode and batching in TelemetryStore (fixes #560)

* fix(telemetry): flush batched writes before reads and under stale batches

Serialize all SQLite access under the store lock, flush pending batches
before queries, and add a stale-batch flush interval so external readers
like TelemetryAggregator see committed rows. Also apply secure_create and
batch_size validation from review feedback.

* fix(telemetry): background flusher so idle batches become visible; harden close()

The stale-batch check only ran inside record calls, so a partial batch
written just before traffic stopped stayed invisible to readers on other
connections (TelemetryAggregator, the leaderboard pipeline) until the next
write arrived - potentially forever on an idle server. A daemon flusher
thread now guarantees pending rows land within flush_interval_seconds;
passing 0 disables it (and time-based flushing) for deterministic tests.

Also: document the visibility contract on the class docstring, make
close() idempotent (a second close previously raised ProgrammingError from
commit-on-closed-connection), and fix the batching test to actually close
its raw sqlite3 connections (the "with conn" form is a transaction scope,
not a close) plus pin the new background-flush and double-close behavior.

* fix(server): prevent memory context injection from suppressing persona identity prompt (fixes #651)

---------

Co-authored-by: Elliot Slusky <elliot@slusky.com>
Co-authored-by: Arush Wadhawan <soulsniper@Arushs-MacBook-Pro.local>
2026-08-14 18:34:19 -07:00
Elliot Slusky da841e5282 fix(connectors): sync new Apple Notes (#746)
Fixes #719.
2026-08-14 18:20:16 -07:00
Elliot Slusky 548d9e04fe fix(digest): prevent persona fact leakage (#742) 2026-08-14 18:19:54 -07:00
8d90e3dff1 fix: reorder HeuristicRouter rules so low-complexity check precedes math check (#671)
* fix: reorder HeuristicRouter rules so low-complexity check precedes math check

Trivial arithmetic like "what is 2+2?" was escalating to the largest
available model just because it matched the math keyword, since the
math rule ran before the low-complexity rule. Math queries above the
low-complexity threshold still escalate correctly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: make low-complexity math routing effective

---------

Co-authored-by: Ari <ari.silva@paipe.co>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 18:18:33 -07:00
b375d7cf09 fix(server): wire WS event bridge to the bus channels actually publish to (#673)
* fix(server): wire WS event bridge to the bus channels actually publish to

include_all_routes() built the /v1/agents/events WebSocket router from
get_event_bus() — a global singleton nothing in `jarvis serve` publishes
to. The real bus lives at app.state.bus (set in server/app.py and handed
to every channel/agent). Events published on it silently never reached
any connected browser client, since the WS router was subscribed to a
different, disconnected EventBus instance entirely.

* fix(server): keep managed agent events on app bus

---------

Co-authored-by: Ari <ari.silva@paipe.co>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 18:18:11 -07:00
c25c649048 fix: strip openrouter/ prefix before forwarding to OpenRouter API (#672)
* fix: strip openrouter/ prefix before forwarding to OpenRouter API

cloud_router.get_provider() correctly detects LiteLLM-style
"openrouter/anthropic/claude-haiku-4.5" strings as the openrouter
provider, but was forwarding them verbatim, so the redundant prefix
reached OpenRouter's API and the request failed.

* fix: preserve native OpenRouter model IDs

---------

Co-authored-by: Ari <ari.silva@paipe.co>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 18:01:53 -07:00
Haotian Zheng f2df968aa5 fix(memory): replace chunks when re-indexing sources (#675) 2026-08-14 17:41:40 -07:00
Matéo H. PetelandElliot Slusky 8145597052 feat(scan): add application data-boundary diagnostic mode (#621)
* feat(security): add data boundary audit scanner

* feat(cli): add jarvis scan data-boundaries command

* test: cover data boundary audit and scan CLI

* docs(user-guide): document data boundary scan

* chore(scripts): add data boundary scan patch helper

* chore: remove patch script and harden PR verification

Drop the one-off apply patch helper, dedupe security user-guide copy, and add a CLI integration test ensuring scan --data-boundaries skips update checks.

* test/docs: verify update-check skip and trim security guide

Add integration test for scan --data-boundaries update-check suppression and replace duplicated three-layer section with a short link.

* test(scan): tighten data-boundary update-check regression

* test(scan): fix data-boundary test spacing

* test(scan): harden data-boundary tests against config defaults

* fix(scan): preserve symlink paths in data-boundary store checks

* fix(scan): address maintainer review on data-boundary audit

* test(scan): assert required tool names independently of audit constants

* fix(scan): classify outbound and remaining local-access tools

* fix(scan): sync data-boundary audit with current-main surfaces

Classify scan_chunks and browser_axtree, report knowledge.db and credentials.toml without reading contents, cover TOOL_CREDENTIALS env keys, and separate external egress from cloud API-key claims after the upstream merge.

* fix(scan): harden data-boundary audit follow-up

---------

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 17:39:36 -07:00
aec96f9d5d fix(cli): pass prior conversation turns to agent.run() in chat REPL (#744)
* fix(cli): pass prior conversation turns to agent.run() in chat REPL

Why: jarvis chat built an AgentContext seeded only with an optional
memory-injected fact, never with the turn-by-turn conversation history
already tracked in `history`. Every agent-backed chat turn after the
first ran with no memory of what was said before it.

- src/openjarvis/cli/chat_cmd.py: always build AgentContext for
  agent-backed turns, seeded from prior non-system history messages,
  still layering the memory-fact message on top when present
- tests/cli/test_chat_cmd.py: regression test asserting the second
  turn's AgentContext carries the first turn's user/assistant messages

* fix(cli): keep memory context before chat history

---------

Co-authored-by: Ari <ari.silva@paipe.co>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 17:25:23 -07:00
Elliot Slusky c2e1c375aa Merge pull request #679 from gilbert-barajas/fix/serve-persona-prompt-builder
fix: served/SDK agents silently lose their persona (SOUL.md never reaches the model over HTTP)
2026-08-14 17:16:19 -07:00
Elliot Slusky ef28e5f84b fix: complete persona wiring across agent entry points 2026-08-14 17:11:25 -07:00
Ari f2483e7bf3 fix(frontend): move MessageBubble hooks above the early return
The three useMemo calls ran after the isUser early return, so they
were skipped entirely for user messages, violating React's rules of
hooks. Caught by the react-hooks/rules-of-hooks lint rule added last
commit. Pure reordering, no logic change; tsc and the full vitest
suite (38/38) still pass.
2026-08-14 17:05:00 -07:00
Elliot Slusky 156d41d2f9 Merge remote-tracking branch 'origin/main' into elliot/pr679-fix 2026-08-14 17:02:52 -07:00
Elliot Slusky 4b7bb936ff fix(desktop): complete remote server support
Allow plaintext user-configured API hosts in the macOS webview and authenticate remote agent event WebSockets with the configured API key. Add regression coverage for both paths.
2026-08-14 16:29:59 -07:00
Haotian Zheng 3c68a17ac5 fix(desktop): allow remote API connections 2026-08-14 16:29:59 -07:00
Elliot Slusky 0d32784ed6 fix(agents): harden structured tool input normalization 2026-08-14 16:29:01 -07:00
Haotian Zheng 20a7424883 fix(agents): normalize structured string tool inputs 2026-08-14 16:29:01 -07:00
github-actions[bot] a97c64c67b chore: update clone traffic data [skip ci] 2026-08-14 07:19:51 +00:00
Jon Saad-Falcon 64333651d1 Delete tools/pearl-reference-oracle directory 2026-08-13 19:27:37 -07:00
Elliot Slusky 9bee016c82 fix(server): preserve grounded agent stream content (#736)
* fix(server): preserve grounded agent stream content

* test(server): cover active grounded stream path

* fix(server): retain agent stream bridge
2026-08-13 18:15:36 -07:00
Elliot Slusky c9942961ad fix(evals): make TauBench dependency explicit (#739)
* fix(evals): make TauBench dependency explicit

* fix(evals): verify TauBench install provenance
2026-08-13 18:14:55 -07:00
Elliot Slusky c3a7ffebff fix(memory): recall auto-captured facts across sessions (#740)
* fix(memory): recall auto-captured facts

* fix(memory): harden recalled context injection

* fix(memory): preserve mixed context history

* fix(agents): preserve caller system context
2026-08-13 17:07:10 -07:00
Elliot Slusky b3c57468ae fix(cli): honor enabled tools when serving (#737)
* fix(cli): honor enabled tools when serving

* fix(server): preserve tools for streaming agents
2026-08-13 17:03:51 -07:00
Elliot Slusky ff69797135 fix(web): normalize tool call arguments (#738)
* fix(web): normalize tool call arguments

* fix(web): preserve chats when repair writeback fails
2026-08-13 17:03:10 -07:00
github-actions[bot] 465dba4b3f chore: update clone traffic data [skip ci] 2026-08-13 07:22:13 +00:00
github-actions[bot] 4f857b0abb chore: update clone traffic data [skip ci] 2026-08-12 07:20:09 +00:00
Jon Saad-FalconandClaude Opus 5 20aa08ef04 ci(desktop): preflight Apple notarization credentials before build (#724)
Notarization is the last thing tauri-action does, so any credential or
account-state fault surfaced ~10 minutes into the macOS job -- after the
Rust toolchain, npm install, two Ollama sidecar downloads and a universal
cargo build -- as one opaque line:

  failed to bundle project: failed codesign application: failed to
  notarize app: Error: HTTP status code: 403. ...

That message conflates three unrelated causes, and the signing step
succeeds in all of them, so the log actively misleads: the certificate is
clearly valid right up until the failure.

Add a read-only `notarytool history` call immediately after checkout. It
submits nothing and exercises the identical auth path, so all three
failures reach us in ~2s with the specific cause and fix named:

  401 invalid credentials  -> APPLE_PASSWORD is not an app-specific
                              password, or was minted under a different
                              Apple ID than APPLE_ID
  403 inaccessible team    -> APPLE_ID is not a member of APPLE_TEAM_ID
  403 required agreement   -> the Program License Agreement lapsed; only
                              the Account Holder can accept it

xcrun is preinstalled on macOS runners, hence placement before the
toolchain steps rather than beside "Configure Apple signing".

Skips cleanly when APPLE_CERTIFICATE is unset (unsigned builds never
notarize), mirroring the existing signing step, and errors when a
certificate is present but notarization secrets are missing -- previously
that combination signed successfully and then failed at the very end.
Transient network faults retry 3x; credential errors are deterministic
and exit on the first definitive answer.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 17:49:52 -07:00
github-actions[bot] 9a63561db8 chore: update clone traffic data [skip ci] 2026-08-11 07:02:06 +00:00
Elliot Slusky 7959285ad8 fix(cli): preserve extras during self-update (#718)
* fix(cli): preserve extras during self-update

* style(cli): format install detection
2026-08-10 17:22:29 -07:00
goatoush 26e1741059 fix(desktop): keep background streams out of active chat (#654) 2026-08-10 17:15:33 -07:00
kelliott-cloudandElliot Slusky 2ed885eb11 fix: never auto-select embed-only models for chat (#659)
* fix: never auto-select embed-only models for chat

Ollama lists nomic-embed-text alongside chat models. Auto-picking
models[0] / recommending the only available id selected the embedder
and every generation failed with HTTP 400 "does not support chat".

- Filter embed-only ids out of GET /v1/models (chat picker)
- Exclude them from /v1/recommended-model; return empty when none left
- Frontend setModels prefers chat models and clears a bad embed selection
- Regression tests for mixed, embed-only, and classifier cases

* fix: harden chat model capability filtering

---------

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-10 17:06:48 -07:00
07fcf35276 fix: use a signal-free liveness probe for the daemon on Windows (#681)
* fix: use a signal-free liveness probe for the daemon on Windows

`_read_pid()` probed the recorded pid with `os.kill(pid, 0)`. That is a
POSIX idiom: on Windows signal 0 is `CTRL_C_EVENT`, so the call routes to
`GenerateConsoleCtrlEvent` rather than testing for existence, and raises
`OSError` (WinError 87, "The parameter is incorrect") for any pid that is
not a live console process-group leader — which includes both dead pids
and the detached server `jarvis start` creates.

That single call produced three symptoms. `jarvis status` propagated the
error and crashed. `_read_pid`'s `except OSError` swallowed it for a
running server, so `status` and `stop` reported "not running" and deleted
a live pid file. And because the probe *sends* a console control event
rather than merely asking, running `status` against the daemon could
terminate it.

Add `_pid_alive()`, which opens a process handle and checks it on Windows
and keeps the signal-0 probe on POSIX, and use it for both liveness
checks. `SIGKILL` in the stop path is now reached on Windows for the
first time, so guard it — it is POSIX-only, and `SIGTERM` already maps to
`TerminateProcess` there.

The existing round-trip test mocked `os.kill` to succeed, which is why
this passed CI on Linux while failing on every Windows run. Point it at
the new seam and add `TestPidLiveness`, which exercises real pids so the
platform behaviour is actually covered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* style: format daemon tests with CI Ruff

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-10 16:56:53 -07:00
Haotian Zheng 4efdb07dae fix(cli): use UTF-8 for config files 2026-08-10 16:44:17 -07:00
Elliot Slusky 7feda3dad3 fix(cloud): isolate Gemini stream metadata 2026-08-10 16:32:14 -07:00
Elliot Slusky f08ad574d3 fix(cloud): handle parallel Gemini tool calls 2026-08-10 16:32:14 -07:00
Dustin Zander 1bfc25a860 fix: preserve Gemini tool calls while streaming 2026-08-10 16:32:14 -07:00
Elliot Slusky 063dd8ea75 fix: unify managed-agent tool resolution (#705)
* fix: unify managed-agent tool resolution

* fix: harden managed-agent tool lifecycle
2026-08-10 15:09:41 -07:00
Elliot Slusky 6af9317556 fix: route configured LiteLLM models by engine ownership (#714) 2026-08-10 12:51:16 -07:00
Elliot Slusky 410562409d fix: make websocket bridge race cancellation-safe 2026-08-10 11:04:12 -07:00
Ari 9498adc7c4 fix: close ws_bridge send loop on client disconnect
Previously the ws_bridge send loop blocked forever on queue.get(),
never learning the client left. When the server service stopped,
uvicorn waited for open WebSocket tasks until systemd SIGKILLed
after TimeoutStopSec=90s. Now each iteration races recv+send; a
completed receive means the client disconnected => break the loop.
2026-08-10 11:04:12 -07:00
github-actions[bot] ebf370595d chore: update clone traffic data [skip ci] 2026-08-10 07:26:44 +00:00
Elliot Slusky bcdbf13d02 test(tools): cover eager deep-research registration 2026-08-09 22:59:45 -07:00
Ari 3dc621618f fix(tools): eager-import scan_chunks and knowledge_sql at package load
Every other built-in tool is imported here specifically to fire its
@ToolRegistry.register() decorator at package-load time; these two
were missing, so their test_registered tests only passed when some
unrelated test (via agent_manager_routes.py, channels_cmd.py, or
deep_research_setup_cmd.py) happened to import the module first in
the same process. Under pytest-xdist that's worker-distribution
dependent, so adding an unrelated test file could flip either test
from pass to fail.
2026-08-09 22:59:45 -07:00
github-actions[bot] fd0b60fefc chore: update clone traffic data [skip ci] 2026-08-09 06:51:50 +00:00
github-actions[bot] 95a9857984 chore: update clone traffic data [skip ci] 2026-08-08 06:46:50 +00:00
github-actions[bot] f9c89308fc chore: update clone traffic data [skip ci] 2026-08-07 07:10:58 +00:00
Elliot Slusky 65d08e9d94 Fix proactive cron reconciliation and notifications 2026-08-06 13:40:00 -07:00
Loma 45717780fa Fix proactive agent cron duplicating on every server restart
register_cron() ran unconditionally on every 'jarvis serve' startup and
create_task() persists to scheduler.db, so each restart added another
copy of the daily proactive cron. On a real install 68 duplicates
accumulated; when due they fired back-to-back and monopolized the
single-slot local inference queue, stalling interactive chat.

register_cron() is now idempotent: an existing active task is reused and
surplus duplicates are cancelled. Also stop the notification channel
from calling connect() — a second getUpdates poll loop on the same bot
token makes Telegram return Conflict and kills the main listener.
2026-08-06 13:40:00 -07:00
goatoush 9da7c30880 Fixing text wrap for agent creator tool selector (#655) 2026-08-06 11:23:27 -07:00
github-actions[bot] 98e791f258 chore: update clone traffic data [skip ci] 2026-08-06 08:31:13 +00:00
Elliot Slusky b9e0928aef fix(telemetry): record cloud inference cost (#704) 2026-08-05 20:51:06 -07:00
Elliot Slusky 652a522e50 fix(config): deserialize skill source tables (#703) 2026-08-05 20:50:42 -07:00
github-actions[bot] ce1a9ce133 chore: update clone traffic data [skip ci] 2026-08-05 08:31:05 +00:00
github-actions[bot] ae45a4f67c chore: update clone traffic data [skip ci] 2026-08-04 08:31:51 +00:00
github-actions[bot] 697eed23d4 chore: update clone traffic data [skip ci] 2026-08-03 09:38:25 +00:00
github-actions[bot] 100595f8aa chore: update clone traffic data [skip ci] 2026-08-02 08:20:07 +00:00
github-actions[bot] dd03a55028 chore: update clone traffic data [skip ci] 2026-08-01 08:15:52 +00:00
github-actions[bot] a72218f99f chore: update clone traffic data [skip ci] 2026-07-31 08:45:27 +00:00
github-actions[bot] eaa76032d5 chore: update clone traffic data [skip ci] 2026-07-30 08:21:39 +00:00
CurryrajandClaude Opus 5 ed01ab8c8d fix: proxy WebSocket upgrades to the API in dev (#692)
The Vite dev proxy forwards `/v1` as plain HTTP with no `ws: true`, so the
WebSocket upgrade for `/v1/agents/events` is never proxied. The socket does not
open, does not error, and does not close — it just sits silent — so every live
agent view is empty under `npm run dev` while working in a production build,
where the frontend is served from the same origin as the API.

That covers the per-agent live trace on the Agents page, which subscribes via
`useAgentEvents` (`frontend/src/lib/useAgentEvents.ts`).

The silence is what makes it costly: with no error to see, it reads as "no
events are being emitted" rather than "the transport never connected", so the
search starts on the server side.

Verified on Windows 11 with `jarvis start` running: before, a
`new WebSocket('ws://localhost:5173/v1/agents/events')` from the dev page never
fired open, error or close within 6s. After, it opens, and a real agent tick
delivers 8 events (agent_tick_start, inference_start/end, tool_call_start/end,
agent_tick_end).

`changeOrigin` is set alongside so the upgrade request carries the target's
host, which some setups require when the API is not on localhost.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:04:41 -07:00
CurryrajandClaude Opus 5 a65f663d2e fix: stop SessionStore treating ":memory:" as a file path (#684)
``SessionStore.__init__`` passed its ``db_path`` straight to
``secure_create()``, which touches the path and chmods it. ``:memory:`` is
a SQLite sentinel, not a filename, so this tried to create a file literally
named ``:memory:``.

On Windows ``:`` is illegal in a filename, so construction raised
``OSError: [Errno 22] Invalid argument: ':memory:'`` and the two
``tests/server/test_channel_bridge_deep_research.py`` tests failed there.
Elsewhere it succeeds and is merely wrong: it leaves a stray ``:memory:``
file in the working directory, and because ``Path(":memory:").parent`` is
``.``, ``secure_mkdir`` chmods the working directory itself to 0o700.

``KnowledgeStore``, ``TelemetryStore`` and ``TraceStore`` already guard this
exact case; ``SessionStore`` was the one store missing the check. Apply the
same guard, with the same comment.

Adds two regression tests: one that an in-memory store is usable, one that
constructing it creates no file. The second fails on every platform without
the fix, so the bug cannot silently return on Linux or macOS.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 10:44:18 -07:00
Elliot Slusky c6382f2473 fix web search setup in browser quickstart (#691) 2026-07-29 08:38:04 -07:00
github-actions[bot] 2922a2b154 chore: update clone traffic data [skip ci] 2026-07-29 08:35:42 +00:00
Elliot Slusky 403dec8e98 Merge pull request #687 from Curryraj/fix/windows-daemon-detach
fix: detach the daemon from its console on Windows
2026-07-28 23:34:01 -07:00
Elliot Slusky 2c7cf6118c test: make Windows daemon test cross-platform 2026-07-28 23:27:25 -07:00
Jaiydaan RajandClaude Opus 5 9fc5b875d1 fix: detach the daemon from its console on Windows
``jarvis start`` spawned the server with ``start_new_session=True``. That is
POSIX-only — CPython's Windows ``_execute_child`` names the parameter
``unused_start_new_session`` and ignores it — so on Windows the server
inherited the launching console instead of detaching from it.

Closing that console, or logging off, therefore delivered CTRL_CLOSE_EVENT
to the server. Observed in the wild as the daemon dying overnight, with

    forrtl: error (200): program aborting due to window-CLOSE event

in server.log (the Fortran runtime under NumPy handles the event and
aborts). ``jarvis start`` looked like it worked: it printed a PID, wrote the
pid file and exited 0, and the server ran for as long as the console stayed
open. Registered as a log-on scheduled task, this means the machine comes
back up with no backend.

Pass DETACHED_PROCESS on Windows so the child gets no console at all, plus
CREATE_NEW_PROCESS_GROUP so a Ctrl-C in the parent console cannot reach it.
POSIX keeps start_new_session.

Verified by attaching to each spawned process with AttachConsole():
start_new_session=True attaches successfully (the child shares a console);
DETACHED_PROCESS fails with ERROR_INVALID_HANDLE (no console exists).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 13:17:06 +08:00
Gilbert Barajas 6e40d87eb5 fix: wire persona prompt_builder into the serve + SDK agent paths
cli/serve.py and sdk.py constructed their agents without passing
prompt_builder, so an agent reached over HTTP (jarvis serve) or via the
SDK silently lost its SOUL.md / MEMORY.md / USER.md persona while the
same agent via `jarvis ask` / `jarvis chat` kept it. cli/ask.py,
cli/chat_cmd.py, and the managed-agent executor already wired the
builder; these two entry points never did.

Found deploying a personal assistant: the server answered as a generic
assistant that explicitly denied being the persona, with SOUL.md
sitting correctly on disk the whole time. No error, no warning.

Mirrors the existing inspect.signature-guarded wiring from ask.py, so
agents whose __init__ doesn't accept the kwarg (e.g. OrchestratorAgent)
opt out automatically and keep their own system-prompt machinery.

Adds a serve-path regression test (tests/cli/test_serve_persona.py)
that fails without the fix: agent._prompt_builder is None on the
unpatched serve path, so the persona files never reach the model.
2026-07-27 00:50:56 -05:00
154 changed files with 12673 additions and 1426 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"label": "Git Clones",
"message": "175,911",
"message": "191,195",
"color": "green",
"namedLogo": "git"
}
+21 -3
View File
@@ -1,6 +1,6 @@
{
"total_clones": 175911,
"last_updated": "2026-07-28T08:29:18Z",
"total_clones": 191195,
"last_updated": "2026-08-15T06:32:36Z",
"daily": {
"2026-03-27": 2189,
"2026-03-28": 1874,
@@ -124,6 +124,24 @@
"2026-07-24": 1118,
"2026-07-25": 928,
"2026-07-26": 740,
"2026-07-27": 799
"2026-07-27": 799,
"2026-07-28": 665,
"2026-07-29": 745,
"2026-07-30": 591,
"2026-07-31": 783,
"2026-08-01": 567,
"2026-08-02": 1248,
"2026-08-03": 724,
"2026-08-04": 708,
"2026-08-05": 647,
"2026-08-06": 604,
"2026-08-07": 624,
"2026-08-08": 706,
"2026-08-09": 1076,
"2026-08-10": 1060,
"2026-08-11": 2182,
"2026-08-12": 641,
"2026-08-13": 770,
"2026-08-14": 943
}
}
+98
View File
@@ -121,6 +121,104 @@ jobs:
# latest release tag (#526).
fetch-depth: 0
# Validate Apple credentials BEFORE the expensive work. Notarization is
# the very last thing `tauri-action` does, so a bad credential or a
# lapsed account agreement previously surfaced ~10 minutes in — after the
# Rust toolchain, npm install, two Ollama sidecar downloads and a
# universal cargo build — as a single opaque line:
#
# failed to bundle project: failed codesign application: failed to
# notarize app: Error: HTTP status code: 403. ...
#
# `notarytool history` is a read-only call (it submits nothing) that
# exercises the identical auth path, so every credential/account failure
# mode reaches us here first, in seconds, with the specific cause named.
# `xcrun` is preinstalled on macOS runners, hence placement before the
# toolchain steps rather than next to "Configure Apple signing".
- name: Preflight Apple notarization credentials
if: matrix.platform == 'macos-14'
env:
CERT: ${{ secrets.APPLE_CERTIFICATE }}
A_ID: ${{ secrets.APPLE_ID }}
A_PASS: ${{ secrets.APPLE_PASSWORD }}
A_TEAM: ${{ secrets.APPLE_TEAM_ID }}
shell: bash
run: |
set -uo pipefail
# Mirror the skip logic in "Configure Apple signing": without a
# certificate the build is unsigned and never notarizes, so there is
# nothing to preflight. Tag builds still hard-fail there.
if [ -z "$CERT" ]; then
echo "No Apple certificate configured; skipping notarization preflight."
exit 0
fi
missing=""
[ -z "$A_ID" ] && missing="$missing APPLE_ID"
[ -z "$A_PASS" ] && missing="$missing APPLE_PASSWORD"
[ -z "$A_TEAM" ] && missing="$missing APPLE_TEAM_ID"
if [ -n "$missing" ]; then
echo "::error::APPLE_CERTIFICATE is set but notarization secrets are missing:$missing"
echo "::error::Signing would succeed and notarization would then fail. Set them or clear APPLE_CERTIFICATE."
exit 1
fi
# Retry only to absorb transient network faults. Credential and
# account errors are deterministic, so we classify and exit on the
# first definitive answer rather than retrying into the same wall.
attempt=1
while [ "$attempt" -le 3 ]; do
out=$(xcrun notarytool history \
--apple-id "$A_ID" \
--team-id "$A_TEAM" \
--password "$A_PASS" \
--output-format json 2>&1)
rc=$?
if [ $rc -eq 0 ]; then
echo "Apple notarization preflight OK — credentials valid, team reachable, agreements in effect."
exit 0
fi
case "$out" in
*"Invalid credentials"*|*"401"*)
echo "::error::Apple notarization preflight failed: invalid credentials (HTTP 401)."
echo "::error::APPLE_PASSWORD must be an app-specific password from appleid.apple.com,"
echo "::error::generated while signed in as the SAME Apple ID as APPLE_ID. A regular"
echo "::error::Apple ID password will not work, and a password minted under a different"
echo "::error::Apple ID authenticates as that other account."
exit 1
;;
*"Invalid or inaccessible developer team ID"*)
echo "::error::Apple notarization preflight failed: APPLE_ID is not a member of team APPLE_TEAM_ID (HTTP 403)."
echo "::error::The Team ID must match the signing certificate. Read it from the cert's"
echo "::error::subject, where it appears as: Developer ID Application: NAME (TEAMID)."
echo "::error::If you belong to several teams, confirm APPLE_ID is a member of this one."
exit 1
;;
*"required agreement"*|*"agreement"*)
echo "::error::Apple notarization preflight failed: the team has no in-effect agreement (HTTP 403)."
echo "::error::Apple reissues the Developer Program License Agreement periodically and"
echo "::error::notarization is refused until it is accepted. ONLY THE ACCOUNT HOLDER can"
echo "::error::accept it — team Admins cannot. Sign in to the account that owns this team:"
echo "::error:: 1. https://developer.apple.com/account -> review any pending agreement"
echo "::error:: 2. App Store Connect -> Business -> accept anything pending there too"
echo "::error::Certificates stay valid while this is outstanding, so signing still works."
exit 1
;;
esac
echo "Preflight attempt ${attempt}/3 failed with a non-credential error."
echo "$out" | tail -5
attempt=$((attempt + 1))
[ "$attempt" -le 3 ] && sleep 10
done
echo "::error::Apple notarization preflight failed after 3 attempts. Last output:"
echo "$out" | tail -20
exit 1
- name: Install system dependencies (Linux)
if: matrix.platform == 'ubuntu-22.04'
run: |
@@ -1,8 +1,6 @@
You are Jarvis — the local AI assistant. You are loyal, efficient, dry-witted, and genuinely care about the person you serve. You have a warm British sensibility: polite but never obsequious, witty but never frivolous.
PERSONALITY:
- You anticipate needs before being asked
- You deliver bad news with constructive dry wit: "Your rebuttals appear to have slipped past their deadline, sir. I'd suggest making them your first order of business — before anyone notices."
- Your humor is understated — a raised eyebrow in voice form
- You are calm under pressure and never flustered
- You treat the briefing as a conversation with someone you respect, not a status report
@@ -12,19 +10,7 @@ ADDRESS:
- Use it 2-3 times per briefing: once in greeting, once mid-briefing, once in closing
- Never every sentence — that would be a parody, not Jarvis
EMAIL TRIAGE:
- Important emails are from REAL PEOPLE (not automated senders, newsletters, or marketing)
- Prioritize emails that need a REPLY or DECISION, or contain a DEADLINE
- Skip promotional, automated, and notification emails entirely
- For important emails, mention the sender name and what they need
MESSAGE TRIAGE (iMessage, Slack, etc.):
- Highlight messages from key people and threads needing a reply
- Briefly acknowledge casual threads so the user knows you checked: "Your group chat has been lively but nothing requiring a response"
- Skip reactions, emoji-only messages, and automated notifications
CONSTRAINTS:
- ONLY report facts present in the provided data. Never invent.
- NEVER describe actions you are taking (adjusting lights, ordering food, queuing playlists, etc.)
- No markdown formatting, no emojis, no bullet points, no headers — this is spoken aloud
- If a data source is disconnected or errored, skip it silently — do not mention connection issues
+13
View File
@@ -135,6 +135,19 @@ cd OpenJarvis
This launches the backend API server and a React frontend at [http://localhost:5173](http://localhost:5173).
You get a ChatGPT-like interface with streaming responses, tool use, energy monitoring, and a telemetry dashboard — all running locally on your hardware.
Web search is available through the built-in DuckDuckGo fallback. To use
Tavily, add `TAVILY_API_KEY` under **Settings → Tools → Web Search** after the
app starts, or export it before starting quickstart:
```bash
export TAVILY_API_KEY="tvly-..."
./scripts/quickstart.sh
```
The script does not automatically source `.env` files. Run `source .env`
first if that is where you keep the key. Stop any existing OpenJarvis server
before restarting so it inherits the updated environment.
To stop all services, press ++ctrl+c++ in the terminal.
!!! tip "Environment variable"
@@ -422,12 +422,17 @@ We recommend creating **one Slack app** that handles both. The App Manifest belo
2. Apple Notes is detected automatically when Full Disk Access is granted
OpenJarvis searches an indexed snapshot rather than querying Notes.app live.
After creating notes, open **Data Sources** and click **Re-sync** on Apple Notes
before searching for the new content.
### Troubleshooting
| Issue | Solution |
|-------|----------|
| "Not connected" despite Full Disk Access | Restart your terminal app after granting access |
| Notes content is garbled | Some very old notes may have encoding issues. Most notes should be clean. |
| New notes are missing | In **Data Sources**, click **Re-sync** on Apple Notes to refresh the index. |
| Missing notes | Only notes stored locally or in iCloud are indexed. Notes in third-party accounts (Gmail, Exchange) may not appear. |
---
+174
View File
@@ -0,0 +1,174 @@
# Data-boundary scan
`jarvis scan --data-boundaries` reports application-level data boundaries in the
current OpenJarvis configuration. It complements the existing host/environment
scan, which checks OS posture such as disk encryption, cloud-sync agents, remote
access tools, and exposed engine ports.
The data-boundary scan is a configuration diagnostic. It is not a vulnerability
scanner, a legal privacy assessment, a network monitor, or an OAuth-scope audit.
## Run the scan
```bash
jarvis scan --data-boundaries
jarvis scan --data-boundaries --json
jarvis scan --data-boundaries --json --show-paths
jarvis scan --data-boundaries --strict
```
`--strict` exits with status code `1` when the report contains either a `fail`
or a `warn` finding. Use it when CI or pre-demo checks need to enforce a
conservative local-only posture.
Without `--strict`, the command always exits `0` even when fail or warn findings
are present. This is useful for exploratory review.
On a fresh `jarvis init` configuration, common warn findings include
`server.host = "0.0.0.0"` and `telemetry.enabled = true`. Running
`jarvis scan --data-boundaries --strict` after init therefore exits `1` until
those defaults are tightened.
Absolute paths and connector file basenames are redacted by default so JSON
reports can be pasted into issues without revealing local usernames, mount
points, or account labels. Use `--show-paths` only for local debugging.
## What it checks
The scan inspects configuration values, environment-variable presence, and the
existence of known local runtime files. It does not read private content from
memory databases, trace databases, connector credentials, prompt files, logs, or
OAuth token files.
The current checks cover:
- cloud-capable model provider, engine, and default model settings
- local memory context injection combined with cloud-capable inference
- traces, telemetry, learning, training, and spec-search settings
- automatic memory service (`tools.storage.enabled` / `[memory].enabled`)
- deep research engine and model settings
- security bypass flags when cloud inference is configured
- unset `security.profile` (informational)
- web search, browser, local file, shell, code, knowledge chunk scanning, and MCP tool surfaces
- local knowledge.db composition with cloud-capable Deep Research targets
- server binding and unauthenticated A2A exposure
- channel enablement, channel credential fields, and channel credential env vars
- skills, skill auto-sync, digest sources, and cloud speech/TTS backends such as Cartesia
- local stores such as `knowledge.db`, `credentials.toml`, `memory.db`, `traces.db`,
`telemetry.db`, `scheduler.db`, embeddings, skill index, `.vault_key`, and memory files
- connector credential files under `connectors/*.json`, without reading them
- API-key and other runtime credential environment variables (presence only)
- a scope note for frontend credential storage when cloud/API-key surfaces exist
Configured database paths (for example `traces.db_path` or `memory.db_path`)
are resolved from config when set, not only the default locations under the
OpenJarvis home directory.
Static Deep Research targeting uses configuration only (no request overrides):
`deep_research.engine` or `engine.default`, and `deep_research.model` or
`server.model` or `intelligence.default_model`.
Model identifiers that contain vendor names (for example `deepseek-r1` or
`openai/gpt-oss`) are not treated as cloud-bound when their effective engine is
explicitly local, such as Ollama.
## Status levels
| Status | Meaning |
| --- | --- |
| `fail` | A configuration composition is likely incompatible with strict local-only use. |
| `warn` | A configured surface may send data outside the local runtime or persist sensitive data. |
| `info` | A relevant setting or local store exists, with no immediate fail or warn condition. |
The command reports potential data paths. It does not prove that a path has been
used during a specific run.
JSON output includes `"schema_version": 1` for stable downstream parsing.
## Strict local-only checklist
For a conservative local-only setup, review these settings:
```toml
[analytics]
enabled = false
[traces]
enabled = false
[telemetry]
enabled = false
[agent]
context_from_memory = false
[intelligence]
provider = ""
preferred_engine = ""
default_model = "" # local model name only
[engine]
default = "ollama" # or another local engine
[tools]
enabled = ""
[tools.storage]
enabled = false
[tools.mcp]
enabled = false
servers = ""
[channel]
enabled = false
[learning]
enabled = false
auto_update = false
training_enabled = false
[learning.spec_search]
enabled = false
[server]
host = "127.0.0.1"
[security]
profile = "personal"
[a2a]
enabled = false
```
Also unset cloud and channel credentials from the process environment when they
are not needed.
## Scope and non-goals
The scan intentionally avoids reading private data. In particular, it does not:
- read connector JSON contents or OAuth scopes
- inspect browser `localStorage` or Tauri secure storage
- inspect frontend credential storage directly
- inspect installed skill source code
- intercept runtime network traffic
- classify provider retention or training policies
- prove that a configured path was used at runtime
Frontend credential storage is tracked separately from this CLI diagnostic. If a
cloud/API-key surface is present, the scan emits an informational scope note so
users know that browser/Tauri credential storage must be reviewed separately.
## Configuration resolution
The scan follows the same explicit configuration override used by the runtime:
if `OPENJARVIS_CONFIG` is set, that file is audited. Otherwise the scan uses
the default OpenJarvis config path under the resolved OpenJarvis home. If the
home directory cannot be resolved, the command reports a `config-root-error`
finding instead of crashing.
## See also
- [Security](security.md) — three-layer security model (host scan, config scan, BoundaryGuard)
- [Configuration](../getting-started/configuration.md) — full config reference
+10
View File
@@ -31,6 +31,16 @@ uv sync --extra dev --extra eval-wandb # Weights & Biases run tracking
uv sync --extra dev --extra eval-sheets # Google Sheets results export
```
TauBench additionally requires Python 3.12 or newer and the upstream `tau2`
package. Install the pinned revision explicitly before running that benchmark:
```bash
uv pip install "tau2 @ git+https://github.com/sierra-research/tau2-bench.git@fc0055dc4e0a316c3f83133267fbd6faaa770992"
```
OpenJarvis does not install third-party packages automatically when an
evaluation is imported or run.
!!! note "Python version requirement"
Python 3.10 requires the `tomli` package for TOML config parsing. `openjarvis` declares it as a conditional dependency, so it is installed automatically.
+21
View File
@@ -2,6 +2,20 @@
OpenJarvis includes a security layer that scans prompts and model outputs for secrets, personally identifiable information (PII), and sensitive file paths. The system is designed to be composable: scanners run as a pipeline, and the `GuardrailsEngine` wrapper drops in front of any inference backend without changing how the rest of your code works.
## Three layers of security review
OpenJarvis separates host posture, application data boundaries, and runtime prompt guardrails:
| Layer | Command / component | What it checks |
| --- | --- | --- |
| Host scan | `jarvis scan` | Disk encryption, cloud-sync agents, exposed engine ports, remote-access tools |
| Data-boundary scan | `jarvis scan --data-boundaries` | Configured inference, memory, traces, channels, tools, and local stores |
| Runtime guardrails | `GuardrailsEngine` / BoundaryGuard | Secrets, PII, and file-policy violations in live prompts and outputs |
Use the host scan before storing sensitive data on the machine. Use the data-boundary scan to verify whether your `config.toml` is local-only, cloud-capable, or mixed. Use BoundaryGuard during inference when you need live redaction or blocking.
See [Data Boundary Scan](data-boundary-scan.md) for the application config diagnostic and [BoundaryGuard](#guardrailsengine) below for runtime scanning.
---
## Overview
@@ -446,8 +460,15 @@ guarded = GuardrailsEngine(
---
## Data boundary scan
See [Data Boundary Scan](data-boundary-scan.md) for the application config diagnostic (`jarvis scan --data-boundaries`).
---
## See Also
- [Data Boundary Scan](data-boundary-scan.md) — application config and local-store diagnostic (`jarvis scan --data-boundaries`)
- [Architecture: Security](../architecture/security.md) — pipeline design, event flow, and file policy integration
- [API Reference: Security](../api-reference/openjarvis/security/index.md) — full class and function signatures
- [Tools](tools.md) — how `FileReadTool` uses file policy
+14
View File
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSAppTransportSecurity</key>
<dict>
<!-- The desktop API URL is user-configurable, so it cannot be represented
by Tauri's single, build-time exceptionDomain setting. Keep this
exception scoped to WKWebView; native URLSession traffic retains ATS. -->
<key>NSAllowsArbitraryLoadsInWebContent</key>
<true/>
</dict>
</dict>
</plist>
+1 -2
View File
@@ -24,7 +24,7 @@
}
],
"security": {
"csp": "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; connect-src 'self' http://localhost:* http://127.0.0.1:* ws://localhost:* ws://127.0.0.1:*; img-src 'self' data: blob:"
"csp": "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; connect-src 'self' http: https: ws: wss:; img-src 'self' data: blob:"
}
},
"bundle": {
@@ -45,7 +45,6 @@
"macOS": {
"entitlements": "Entitlements.plist",
"minimumSystemVersion": "10.15",
"exceptionDomain": "",
"frameworks": [],
"providerShortName": null,
"signingIdentity": "-"
-2
View File
@@ -31,7 +31,6 @@ export default function App() {
const prevModelRef = useRef<string>('');
const setModels = useAppStore((s) => s.setModels);
const setModelsLoading = useAppStore((s) => s.setModelsLoading);
const setSelectedModel = useAppStore((s) => s.setSelectedModel);
const selectedModel = useAppStore((s) => s.selectedModel);
const setServerInfo = useAppStore((s) => s.setServerInfo);
const setSavings = useAppStore((s) => s.setSavings);
@@ -70,7 +69,6 @@ export default function App() {
fetchModels()
.then((m) => {
setModels(m);
if (!selectedModel && m.length > 0) setSelectedModel(m[0].id);
})
.catch(() => setModels([]))
.finally(() => setModelsLoading(false));
+9 -6
View File
@@ -15,6 +15,7 @@ function getGreeting(): string {
}
export function ChatArea() {
const activeId = useAppStore((s) => s.activeId);
const messages = useAppStore((s) => s.messages);
const streamState = useAppStore((s) => s.streamState);
const systemPanelOpen = useAppStore((s) => s.systemPanelOpen);
@@ -24,6 +25,8 @@ export function ChatArea() {
const shouldAutoScroll = useRef(true);
const wasStreaming = useRef(false);
const lastScrollTop = useRef(0);
const isCurrentChatStreaming = streamState.isStreaming && streamState.conversationId === activeId;
const currentStreamContent = isCurrentChatStreaming ? streamState.content : '';
// Check if any data sources are connected
const [hasConnectedSources, setHasConnectedSources] = useState<boolean | null>(null);
@@ -38,14 +41,14 @@ export function ChatArea() {
useEffect(() => {
// Sending a message always pins the view to the bottom, even if the
// user had scrolled up to read earlier messages.
if (streamState.isStreaming && !wasStreaming.current) {
if (isCurrentChatStreaming && !wasStreaming.current) {
shouldAutoScroll.current = true;
}
wasStreaming.current = streamState.isStreaming;
wasStreaming.current = isCurrentChatStreaming;
if (shouldAutoScroll.current && listRef.current) {
listRef.current.scrollTop = listRef.current.scrollHeight;
}
}, [messages, streamState.content, streamState.isStreaming]);
}, [messages, currentStreamContent, isCurrentChatStreaming]);
const handleScroll = () => {
if (!listRef.current) return;
@@ -66,7 +69,7 @@ export function ChatArea() {
}
};
const isEmpty = messages.length === 0 && !streamState.isStreaming;
const isEmpty = messages.length === 0 && !isCurrentChatStreaming;
const PanelIcon = systemPanelOpen ? PanelRightClose : PanelRightOpen;
@@ -174,12 +177,12 @@ export function ChatArea() {
<MessageBubble
key={msg.id}
message={msg}
isLive={isLastAssistant && streamState.isStreaming}
isLive={isLastAssistant && isCurrentChatStreaming}
/>
);
})}
{(() => {
if (!streamState.isStreaming || streamState.content !== '') return null;
if (!isCurrentChatStreaming || streamState.content !== '') return null;
// For research messages the ResearchTimeline handles its own
// pre-content loading state — suppress the generic dots.
const last = messages[messages.length - 1];
+11 -5
View File
@@ -5,6 +5,7 @@ import { useAppStore, generateId } from '../../lib/store';
import { streamChat, streamResearch } from '../../lib/sse';
import { fetchSavings, getBase } from '../../lib/api';
import { listConnectors, getSyncStatus } from '../../lib/connectors-api';
import { serializeToolCallArguments } from '../../lib/tool-call';
import { MicButton } from './MicButton';
import { useSpeech } from '../../hooks/useSpeech';
import type {
@@ -96,6 +97,7 @@ export function InputArea() {
const deepResearch = useAppStore((s) => s.deepResearch);
const setDeepResearch = useAppStore((s) => s.setDeepResearch);
const corpusSync = useResearchCorpusSync(deepResearch);
const isCurrentChatStreaming = streamState.isStreaming && streamState.conversationId === activeId;
const {
state: speechState,
@@ -226,6 +228,7 @@ export function InputArea() {
let ttftMs: number | undefined;
setStreamState({
conversationId: convId,
isStreaming: true,
phase: deepResearch ? 'Researching...' : 'Generating...',
elapsedMs: 0,
@@ -387,7 +390,7 @@ export function InputArea() {
const tc: ToolCallInfo = {
id: generateId(),
tool: data.tool,
arguments: data.arguments || '',
arguments: serializeToolCallArguments(data.arguments),
status: 'running',
};
toolCalls.push(tc);
@@ -398,7 +401,7 @@ export function InputArea() {
updateLastAssistant(convId, accumulatedContent, [...toolCalls]);
useAppStore.getState().addLogEntry({
timestamp: Date.now(), level: 'info', category: 'tool',
message: `Calling ${data.tool}(${data.arguments || ''})`,
message: `Calling ${data.tool}(${serializeToolCallArguments(data.arguments)})`,
});
} catch {}
} else if (eventName === 'tool_call_end') {
@@ -466,7 +469,10 @@ export function InputArea() {
}
const totalMs = Date.now() - startTime;
const _CLOUD_PREFIXES = ['gpt-', 'o1-', 'o3-', 'o4-', 'claude-', 'gemini-', 'openrouter/', 'MiniMax-', 'chatgpt-'];
const engineLabel = _CLOUD_PREFIXES.some(p => selectedModel.startsWith(p)) ? 'cloud' : 'ollama';
const selectedOwner = useAppStore.getState().models.find((m) => m.id === selectedModel)?.owned_by;
const engineLabel = selectedOwner === 'litellm'
? 'litellm'
: _CLOUD_PREFIXES.some(p => selectedModel.startsWith(p)) ? 'cloud' : 'ollama';
const telemetry: MessageTelemetry = {
engine: engineLabel,
model_id: selectedModel,
@@ -599,7 +605,7 @@ export function InputArea() {
style={{ color: 'var(--color-text)', maxHeight: '200px' }}
disabled={streamState.isStreaming || modelLoading}
/>
{streamState.isStreaming ? (
{isCurrentChatStreaming ? (
<button
onClick={stopStreaming}
className="p-2 rounded-xl transition-colors shrink-0 cursor-pointer"
@@ -618,7 +624,7 @@ export function InputArea() {
/>
<button
onClick={sendMessage}
disabled={!input.trim() || modelLoading || !selectedModel}
disabled={streamState.isStreaming || !input.trim() || modelLoading || !selectedModel}
title={selectedModel ? 'Send message' : 'Pick a model first (⌘K)'}
className="p-2 rounded-xl transition-colors shrink-0 cursor-pointer disabled:opacity-30 disabled:cursor-default"
style={{
+18 -18
View File
@@ -103,6 +103,24 @@ function CopyMessageButton({ content }: { content: string }) {
export function MessageBubble({ message, isLive = false }: Props) {
const isUser = message.role === 'user';
const cleanContent = useMemo(() => stripThinkTags(message.content), [message.content]);
// Build a ref→source lookup once per render. Memoized so the rehype plugin
// identity stays stable until the source list actually changes.
const sourcesMap = useMemo(() => {
const m = new Map<number, NonNullable<ChatMessage['researchSources']>[number]>();
for (const s of message.researchSources ?? []) {
if (typeof s.ref === 'number') m.set(s.ref, s);
}
return m;
}, [message.researchSources]);
const rehypePlugins = useMemo(() => {
const base: any[] = [[rehypeHighlight, { detect: true }], rehypeKatex];
if (sourcesMap.size > 0) base.push([rehypeCitations, { sources: sourcesMap }]);
return base;
}, [sourcesMap]);
if (isUser) {
return (
<div className="flex justify-end mb-4">
@@ -122,24 +140,6 @@ export function MessageBubble({ message, isLive = false }: Props) {
);
}
const cleanContent = useMemo(() => stripThinkTags(message.content), [message.content]);
// Build a ref→source lookup once per render. Memoized so the rehype plugin
// identity stays stable until the source list actually changes.
const sourcesMap = useMemo(() => {
const m = new Map<number, NonNullable<ChatMessage['researchSources']>[number]>();
for (const s of message.researchSources ?? []) {
if (typeof s.ref === 'number') m.set(s.ref, s);
}
return m;
}, [message.researchSources]);
const rehypePlugins = useMemo(() => {
const base: any[] = [[rehypeHighlight, { detect: true }], rehypeKatex];
if (sourcesMap.size > 0) base.push([rehypeCitations, { sources: sourcesMap }]);
return base;
}, [sourcesMap]);
return (
<div className="group mb-6">
{/* Deep Research timeline (steps + status) */}
@@ -1,6 +1,7 @@
import { useState } from 'react';
import { ChevronDown, ChevronRight, Loader2, CheckCircle2, XCircle } from 'lucide-react';
import type { ToolCallInfo } from '../../types';
import { serializeToolCallArguments } from '../../lib/tool-call';
interface Props {
toolCall: ToolCallInfo;
@@ -35,7 +36,10 @@ export function ToolCallCard({ toolCall }: Props) {
const [expanded, setExpanded] = useState(false);
const config = statusConfig[toolCall.status];
const StatusIcon = config.icon;
const preview = previewArgs(toolCall.arguments);
// Persisted conversations may contain the pre-fix object payload despite
// the TypeScript contract, so normalize again at the final render boundary.
const argumentsText = serializeToolCallArguments(toolCall.arguments);
const preview = previewArgs(argumentsText);
return (
<div
@@ -95,7 +99,7 @@ export function ToolCallCard({ toolCall }: Props) {
className="px-2.5 pb-2 pt-0.5"
style={{ borderTop: '1px solid var(--color-border-subtle, var(--color-border))' }}
>
{toolCall.arguments && (
{argumentsText && (
<div className="mt-1.5">
<div
style={{
@@ -120,7 +124,7 @@ export function ToolCallCard({ toolCall }: Props) {
wordBreak: 'break-all',
}}
>
{formatJson(toolCall.arguments)}
{formatJson(argumentsText)}
</pre>
</div>
)}
+23 -16
View File
@@ -143,7 +143,7 @@ export function CommandPalette() {
}
}, [pullSuccess]);
const handleSelect = async (modelId: string) => {
const handleSelect = async (modelId: string, owner?: string) => {
const previousModel = selectedModel;
setSelectedModel(modelId);
setCommandPaletteOpen(false);
@@ -153,7 +153,7 @@ export function CommandPalette() {
setModelLoading(true);
addLogEntry({ timestamp: Date.now(), level: 'info', category: 'model', message: `Switching to ${modelId}...` });
try {
await preloadModel(modelId);
await preloadModel(modelId, owner);
addLogEntry({ timestamp: Date.now(), level: 'info', category: 'model', message: `${modelId} loaded` });
} catch (e: any) {
addLogEntry({ timestamp: Date.now(), level: 'error', category: 'model', message: `Failed to load ${modelId}: ${e.message}` });
@@ -255,7 +255,8 @@ export function CommandPalette() {
setSelectedIdx((i) => Math.max(i - 1, 0));
} else if (e.key === 'Enter' && tab === 'installed' && filtered.length > 0) {
e.preventDefault();
handleSelect((filtered[selectedIdx] as any).id);
const model = filtered[selectedIdx] as (typeof models)[number];
handleSelect(model.id, model.owned_by);
}
};
@@ -365,11 +366,15 @@ export function CommandPalette() {
onMouseEnter={() => setSelectedIdx(idx)}
>
<button
onClick={() => handleSelect(model.id)}
onClick={() => handleSelect(model.id, model.owned_by)}
className="flex items-center gap-3 flex-1 min-w-0 text-left cursor-pointer"
style={{ background: 'none', border: 'none', padding: 0 }}
>
<Cpu size={16} style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text-tertiary)' }} />
{model.owned_by === 'litellm' ? (
<Cloud size={16} style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text-tertiary)' }} />
) : (
<Cpu size={16} style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text-tertiary)' }} />
)}
<div className="flex-1 min-w-0">
<div className="text-sm truncate" style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text)', fontWeight: isActive ? 500 : 400 }}>
{model.id}
@@ -381,17 +386,19 @@ export function CommandPalette() {
</span>
)}
</button>
<button
onClick={() => handleDelete(model.id)}
disabled={isDeleting}
className="p-1 rounded transition-colors cursor-pointer"
style={{ color: 'var(--color-text-tertiary)', opacity: 0 }}
title="Delete model"
onMouseEnter={(e) => { e.currentTarget.style.opacity = '1'; e.currentTarget.style.color = 'var(--color-error)'; }}
onMouseLeave={(e) => { e.currentTarget.style.opacity = '0'; e.currentTarget.style.color = 'var(--color-text-tertiary)'; }}
>
{isDeleting ? <Loader2 size={14} className="animate-spin" /> : <Trash2 size={14} />}
</button>
{model.owned_by !== 'litellm' && (
<button
onClick={() => handleDelete(model.id)}
disabled={isDeleting}
className="p-1 rounded transition-colors cursor-pointer"
style={{ color: 'var(--color-text-tertiary)', opacity: 0 }}
title="Delete model"
onMouseEnter={(e) => { e.currentTarget.style.opacity = '1'; e.currentTarget.style.color = 'var(--color-error)'; }}
onMouseLeave={(e) => { e.currentTarget.style.opacity = '0'; e.currentTarget.style.color = 'var(--color-text-tertiary)'; }}
>
{isDeleting ? <Loader2 size={14} className="animate-spin" /> : <Trash2 size={14} />}
</button>
)}
</div>
);
})
+6 -3
View File
@@ -7,6 +7,7 @@ import {
type SetupStatus,
} from '../lib/api';
import { useAppStore } from '../lib/store';
import { isEmbedOnlyModel } from '../lib/model-capabilities';
const STEPS = [
{ key: 'ollama_ready', label: 'Inference Engine', icon: Cpu, detail: 'Starting Ollama...' },
@@ -91,12 +92,14 @@ export function SetupScreen({ onReady }: { onReady: () => void }) {
fetchRecommendedModel().catch(() => ({ model: '', reason: '' })),
]);
const store = useAppStore.getState();
const hadSelection = !!store.selectedModel;
store.setModels(models);
store.setModelsLoading(false);
const recommended = rec.model && models.some((m) => m.id === rec.model)
const chatModels = models.filter((m) => !isEmbedOnlyModel(m.id));
const recommended = rec.model && chatModels.some((m) => m.id === rec.model)
? rec.model
: models[0]?.id || '';
if (recommended && !store.selectedModel) {
: chatModels[0]?.id || '';
if (recommended && !hadSelection) {
store.setSelectedModel(recommended);
}
} catch {
@@ -22,6 +22,9 @@ export function ConversationList({ searchQuery }: Props) {
const navigate = useNavigate();
const conversations = useAppStore((s) => s.conversations);
const activeId = useAppStore((s) => s.activeId);
const streamingConversationId = useAppStore((s) =>
s.streamState.isStreaming ? s.streamState.conversationId : null,
);
const selectConversation = useAppStore((s) => s.selectConversation);
const deleteConversation = useAppStore((s) => s.deleteConversation);
@@ -43,6 +46,7 @@ export function ConversationList({ searchQuery }: Props) {
<div className="flex flex-col gap-0.5 py-1">
{filtered.map((conv) => {
const isActive = conv.id === activeId;
const isStreaming = conv.id === streamingConversationId;
return (
<div
key={conv.id}
@@ -82,11 +86,18 @@ export function ConversationList({ searchQuery }: Props) {
e.stopPropagation();
deleteConversation(conv.id);
}}
className="p-1.5 mr-1 rounded opacity-0 group-hover:opacity-100 transition-opacity cursor-pointer"
disabled={isStreaming}
className="p-1.5 mr-1 rounded opacity-0 group-hover:opacity-100 transition-opacity cursor-pointer disabled:cursor-not-allowed disabled:opacity-30"
style={{ color: 'var(--color-text-tertiary)' }}
onMouseEnter={(e) => (e.currentTarget.style.color = 'var(--color-error)')}
onMouseEnter={(e) => {
if (!isStreaming) e.currentTarget.style.color = 'var(--color-error)';
}}
onMouseLeave={(e) => (e.currentTarget.style.color = 'var(--color-text-tertiary)')}
title="Delete conversation"
title={
isStreaming
? 'Stop generating before deleting this conversation'
: 'Delete conversation'
}
>
<Trash2 size={14} />
</button>
+50
View File
@@ -6,6 +6,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
// authHeaders) that source the key and build the header.
const SETTINGS_KEY = 'openjarvis-settings';
const fetchMock = vi.fn<typeof fetch>();
// Minimal in-memory localStorage stub so the helpers can run under node
// (no jsdom dependency).
@@ -28,6 +29,8 @@ class MemoryStorage {
beforeEach(() => {
vi.resetModules();
vi.stubEnv('VITE_SUPABASE_ANON_KEY', 'test-anon-key');
fetchMock.mockReset();
globalThis.fetch = fetchMock;
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
@@ -86,3 +89,50 @@ describe('authHeaders', () => {
});
});
});
describe('tool credentials', () => {
it('reads credential status from the local server', async () => {
fetchMock.mockResolvedValue(
new Response(JSON.stringify({ TAVILY_API_KEY: true }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
}),
);
const { fetchToolCredentialStatus } = await freshApi();
await expect(fetchToolCredentialStatus('web_search')).resolves.toEqual({
TAVILY_API_KEY: true,
});
expect(fetchMock).toHaveBeenCalledWith(
'/v1/tools/web_search/credentials/status',
{ headers: {} },
);
});
it('saves a tool credential through the local server', async () => {
fetchMock.mockResolvedValue(new Response('{}', { status: 200 }));
const { saveToolCredentials } = await freshApi();
await saveToolCredentials('web_search', {
TAVILY_API_KEY: 'tvly-test',
});
expect(fetchMock).toHaveBeenCalledWith('/v1/tools/web_search/credentials', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ TAVILY_API_KEY: 'tvly-test' }),
});
});
it('deletes a tool credential through the local server', async () => {
fetchMock.mockResolvedValue(new Response('{}', { status: 200 }));
const { deleteToolCredential } = await freshApi();
await deleteToolCredential('web_search', 'TAVILY_API_KEY');
expect(fetchMock).toHaveBeenCalledWith(
'/v1/tools/web_search/credentials/TAVILY_API_KEY',
{ method: 'DELETE', headers: {} },
);
});
});
+23 -3
View File
@@ -1,5 +1,6 @@
import type { ModelInfo, SavingsData, ServerInfo } from '../types';
import { SUPABASE_ANON_KEY, SUPABASE_URL } from './supabase';
import { serializeToolCallArguments } from './tool-call';
// ---------------------------------------------------------------------------
// Supabase config
@@ -218,9 +219,9 @@ export async function deleteModel(modelName: string): Promise<void> {
const _CLOUD_PREFIXES = ['gpt-', 'o1-', 'o3-', 'o4-', 'claude-', 'gemini-', 'openrouter/'];
export async function preloadModel(modelName: string): Promise<void> {
export async function preloadModel(modelName: string, owner?: string): Promise<void> {
// Cloud models don't need Ollama preloading
if (_CLOUD_PREFIXES.some(p => modelName.startsWith(p))) {
if (owner === 'litellm' || _CLOUD_PREFIXES.some(p => modelName.startsWith(p))) {
return;
}
// Trigger Ollama to load the model into memory (empty prompt, no generation).
@@ -741,7 +742,7 @@ export async function sendAgentMessage(
const parsed = JSON.parse(data);
callbacks?.onToolCallStart?.({
tool: parsed.tool,
arguments: parsed.arguments ?? '',
arguments: serializeToolCallArguments(parsed.arguments),
});
} catch {
/* skip */
@@ -885,6 +886,25 @@ export async function saveToolCredentials(
if (!res.ok) throw new Error(`Failed: ${res.status}`);
}
export async function fetchToolCredentialStatus(
toolName: string,
): Promise<Record<string, boolean>> {
const res = await apiFetch(`/v1/tools/${toolName}/credentials/status`);
if (!res.ok) throw new Error(`Failed: ${res.status}`);
return await res.json();
}
export async function deleteToolCredential(
toolName: string,
keyName: string,
): Promise<void> {
const res = await apiFetch(
`/v1/tools/${encodeURIComponent(toolName)}/credentials/${encodeURIComponent(keyName)}`,
{ method: 'DELETE' },
);
if (!res.ok) throw new Error(`Failed: ${res.status}`);
}
export interface AgentTraceDetail {
id: string;
agent: string;
@@ -0,0 +1,19 @@
import { describe, expect, it } from 'vitest';
import { isEmbedOnlyModel } from './model-capabilities';
describe('isEmbedOnlyModel', () => {
it.each([
'nomic-embed-text',
'mxbai-embed-large',
'text-embedding-3-small',
'all-minilm:latest',
'hf.co/BAAI/bge-m3:latest',
])('classifies %s as embedding-only', (modelId) => {
expect(isEmbedOnlyModel(modelId)).toBe(true);
});
it.each(['qwen3.5:4b', 'codegemma:7b'])('keeps %s available for chat', (modelId) => {
expect(isEmbedOnlyModel(modelId)).toBe(false);
});
});
+22
View File
@@ -0,0 +1,22 @@
const EMBEDDING_MODEL_PREFIXES = [
'all-minilm',
'bge-',
'bge_',
'e5-',
'e5_',
'gte-',
'gte_',
'jina-embeddings',
'nomic-bert',
'sentence-transformers',
];
export function isEmbedOnlyModel(modelId: string): boolean {
const name = (modelId || '').trim().toLowerCase();
const leaf = name.slice(name.lastIndexOf('/') + 1).split(':')[0];
return (
leaf.includes('embed') ||
leaf.includes('minilm') ||
EMBEDDING_MODEL_PREFIXES.some((prefix) => leaf.startsWith(prefix))
);
}
+64
View File
@@ -0,0 +1,64 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { ModelInfo } from '../types';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
}
const model = (id: string): ModelInfo => ({
id,
object: 'model',
created: 0,
owned_by: 'openjarvis',
});
beforeEach(() => {
vi.resetModules();
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
describe('setModels', () => {
it('does not select an embedding-only model', async () => {
const { useAppStore } = await import('./store');
useAppStore.getState().setModels([model('nomic-embed-text')]);
expect(useAppStore.getState().selectedModel).toBe('');
});
it('clears a missing selection when no chat fallback exists', async () => {
const { useAppStore } = await import('./store');
useAppStore.getState().setSelectedModel('deleted-chat-model');
useAppStore.getState().setModels([model('nomic-embed-text')]);
expect(useAppStore.getState().selectedModel).toBe('');
});
it('replaces an embedding selection with an available chat model', async () => {
const { useAppStore } = await import('./store');
useAppStore.getState().setSelectedModel('all-minilm:latest');
useAppStore.getState().setModels([
model('all-minilm:latest'),
model('qwen3.5:4b'),
]);
expect(useAppStore.getState().selectedModel).toBe('qwen3.5:4b');
});
});
@@ -0,0 +1,82 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
removeItem(key: string): void {
this.store.delete(key);
}
}
beforeEach(() => {
vi.resetModules();
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
async function freshStore() {
return (await import('./store')).useAppStore;
}
describe('conversation stream ownership', () => {
it('persists background stream updates without replacing the active messages', async () => {
const store = await freshStore();
const sourceId = store.getState().createConversation('test-model');
store.getState().addMessage(sourceId, {
id: 'assistant',
role: 'assistant',
content: '',
timestamp: 1,
});
const activeId = store.getState().createConversation('test-model');
store.getState().setStreamState({
conversationId: sourceId,
isStreaming: true,
content: 'streamed response',
});
store.getState().updateLastAssistant(sourceId, 'streamed response');
expect(store.getState().activeId).toBe(activeId);
expect(store.getState().messages).toEqual([]);
store.getState().selectConversation(sourceId);
expect(store.getState().messages).toHaveLength(1);
expect(store.getState().messages[0].content).toBe('streamed response');
});
it('keeps the stream-owning conversation until generation stops', async () => {
const store = await freshStore();
const sourceId = store.getState().createConversation('test-model');
const activeId = store.getState().createConversation('test-model');
store.getState().setStreamState({
conversationId: sourceId,
isStreaming: true,
});
store.getState().deleteConversation(sourceId);
expect(
store.getState().conversations.map((conversation) => conversation.id),
).toContain(sourceId);
expect(store.getState().activeId).toBe(activeId);
store.getState().resetStream();
store.getState().deleteConversation(sourceId);
expect(
store.getState().conversations.map((conversation) => conversation.id),
).not.toContain(sourceId);
});
});
@@ -0,0 +1,122 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const CONVERSATIONS_KEY = 'openjarvis-conversations';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
removeItem(key: string): void {
this.store.delete(key);
}
}
beforeEach(() => {
vi.resetModules();
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
describe('persisted tool calls', () => {
it('repairs parsed argument objects while loading conversations', async () => {
localStorage.setItem(
CONVERSATIONS_KEY,
JSON.stringify({
version: 1,
activeId: 'conversation-1',
conversations: {
'conversation-1': {
id: 'conversation-1',
title: 'Broken chat',
createdAt: 1,
updatedAt: 1,
model: 'test-model',
messages: [
{
id: 'assistant-1',
role: 'assistant',
content: '',
timestamp: 1,
toolCalls: [
{
id: 'call-1',
tool: 'web_search',
arguments: { query: 'python' },
status: 'success',
},
],
},
],
},
},
}),
);
const { useAppStore } = await import('./store');
expect(useAppStore.getState().messages[0].toolCalls?.[0].arguments).toBe(
'{"query":"python"}',
);
const repaired = JSON.parse(localStorage.getItem(CONVERSATIONS_KEY) ?? '{}');
expect(
repaired.conversations['conversation-1'].messages[0].toolCalls[0].arguments,
).toBe('{"query":"python"}');
});
it('keeps repaired conversations in memory when writeback fails', async () => {
localStorage.setItem(
CONVERSATIONS_KEY,
JSON.stringify({
version: 1,
activeId: 'conversation-1',
conversations: {
'conversation-1': {
id: 'conversation-1',
title: 'Readable chat',
createdAt: 1,
updatedAt: 1,
model: 'test-model',
messages: [
{
id: 'assistant-1',
role: 'assistant',
content: '',
timestamp: 1,
toolCalls: [
{
id: 'call-1',
tool: 'web_search',
arguments: { query: 'python' },
status: 'success',
},
],
},
],
},
},
}),
);
vi.spyOn(localStorage, 'setItem').mockImplementation(() => {
throw new DOMException('Storage quota exceeded', 'QuotaExceededError');
});
const { useAppStore } = await import('./store');
expect(useAppStore.getState().messages).toHaveLength(1);
expect(useAppStore.getState().messages[0].toolCalls?.[0].arguments).toBe(
'{"query":"python"}',
);
});
});
+71 -13
View File
@@ -15,6 +15,8 @@ import type {
TokenUsage,
} from '../types';
import type { ManagedAgent } from './api';
import { isEmbedOnlyModel } from './model-capabilities';
import { serializeToolCallArguments } from './tool-call';
export interface CachedConnector {
connector_id: string;
@@ -54,7 +56,30 @@ function loadConversations(): ConversationStore {
const raw = localStorage.getItem(CONVERSATIONS_KEY);
if (!raw) return { version: 1, conversations: {}, activeId: null };
const parsed = JSON.parse(raw);
if (parsed.version === 1) return parsed;
if (parsed.version === 1) {
let repaired = false;
for (const conversation of Object.values(parsed.conversations ?? {}) as Conversation[]) {
for (const message of conversation.messages ?? []) {
for (const toolCall of message.toolCalls ?? []) {
const argumentsText = serializeToolCallArguments(toolCall.arguments);
if (argumentsText !== toolCall.arguments) {
toolCall.arguments = argumentsText;
repaired = true;
}
}
}
}
if (repaired) {
try {
localStorage.setItem(CONVERSATIONS_KEY, JSON.stringify(parsed));
} catch {
// Keep the repaired conversations usable in memory when storage is
// read-only or full. A failed best-effort writeback must not make
// otherwise readable conversation history disappear from the UI.
}
}
return parsed;
}
return { version: 1, conversations: {}, activeId: null };
} catch {
return { version: 1, conversations: {}, activeId: null };
@@ -110,6 +135,7 @@ function saveSettings(settings: Settings): void {
// ── Store ─────────────────────────────────────────────────────────────
const INITIAL_STREAM: StreamState = {
conversationId: null,
isStreaming: false,
phase: '',
elapsedMs: 0,
@@ -351,6 +377,9 @@ export const useAppStore = create<AppState>((set, get) => {
},
deleteConversation: (id: string) => {
const streamState = get().streamState;
if (streamState.isStreaming && streamState.conversationId === id) return;
const store = loadConversations();
delete store.conversations[id];
if (store.activeId === id) {
@@ -393,12 +422,14 @@ export const useAppStore = create<AppState>((set, get) => {
(message.content.length > 50 ? '...' : '');
}
saveConversations(store);
set({
messages: [...conv.messages],
conversations: Object.values(store.conversations).sort(
(a, b) => b.updatedAt - a.updatedAt,
),
});
const conversations = Object.values(store.conversations).sort(
(a, b) => b.updatedAt - a.updatedAt,
);
if (get().activeId === conversationId) {
set({ messages: [...conv.messages], conversations });
} else {
set({ conversations });
}
},
updateLastAssistant: (
@@ -425,7 +456,9 @@ export const useAppStore = create<AppState>((set, get) => {
if (researchSources) lastMsg.researchSources = researchSources;
conv.updatedAt = Date.now();
saveConversations(store);
set({ messages: [...conv.messages] });
if (get().activeId === conversationId) {
set({ messages: [...conv.messages] });
}
}
},
@@ -444,11 +477,36 @@ export const useAppStore = create<AppState>((set, get) => {
// ── Models & server ────────────────────────────────────────────
setModels: (models: ModelInfo[]) =>
set((state) =>
!state.selectedModel && models.length > 0
? { models, selectedModel: models[0].id }
: { models },
),
set((state) => {
// Ollama returns embed-only models (e.g. nomic-embed-text) in the
// same list as chat models. Auto-picking models[0] selected the
// embedder and every chat failed with HTTP 400 "does not support
// chat". Prefer a real chat model for selection / fallback.
const chatModels = models.filter((m) => !isEmbedOnlyModel(m.id));
const preferred =
(state.settings.defaultModel &&
chatModels.some((m) => m.id === state.settings.defaultModel) &&
state.settings.defaultModel) ||
chatModels[0]?.id ||
models.find((m) => !isEmbedOnlyModel(m.id))?.id ||
'';
const currentIsBad =
!!state.selectedModel && isEmbedOnlyModel(state.selectedModel);
const currentMissing =
!!state.selectedModel &&
!models.some((m) => m.id === state.selectedModel);
if (!state.selectedModel || currentIsBad || currentMissing) {
// Prefer a real chat model. If none exist, clear a bad/missing
// selection rather than keeping an embed-only id that 400s on chat.
return {
models,
selectedModel: preferred,
};
}
return { models };
}),
setModelsLoading: (loading: boolean) => set({ modelsLoading: loading }),
setSelectedModel: (model: string) => set({ selectedModel: model }),
setServerInfo: (info: ServerInfo | null) => set({ serverInfo: info }),
+22
View File
@@ -0,0 +1,22 @@
import { describe, expect, it } from 'vitest';
import { serializeToolCallArguments } from './tool-call';
describe('serializeToolCallArguments', () => {
it('preserves JSON strings', () => {
expect(serializeToolCallArguments('{"query":"python"}')).toBe(
'{"query":"python"}',
);
});
it('serializes parsed argument objects', () => {
expect(serializeToolCallArguments({ query: 'python' })).toBe(
'{"query":"python"}',
);
});
it('uses an empty string for missing arguments', () => {
expect(serializeToolCallArguments(null)).toBe('');
expect(serializeToolCallArguments(undefined)).toBe('');
});
});
+11
View File
@@ -0,0 +1,11 @@
/** Convert tool-call arguments from API or persisted data into display-safe text. */
export function serializeToolCallArguments(value: unknown): string {
if (typeof value === 'string') return value;
if (value == null) return '';
try {
return JSON.stringify(value) ?? String(value);
} catch {
return String(value);
}
}
+66
View File
@@ -0,0 +1,66 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { buildWsUrl } from './useAgentEvents';
const SETTINGS_KEY = 'openjarvis-settings';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
}
beforeEach(() => {
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
describe('buildWsUrl', () => {
it('authenticates agent events with the configured API key', () => {
localStorage.setItem(
SETTINGS_KEY,
JSON.stringify({
apiUrl: 'https://jarvis.example.com:8443',
apiKey: 'secret+/=',
}),
);
const url = new URL(buildWsUrl('agent/one'));
expect(url.origin).toBe('wss://jarvis.example.com:8443');
expect(url.pathname).toBe('/v1/agents/events');
expect(url.searchParams.get('agent_id')).toBe('agent/one');
expect(url.searchParams.get('token')).toBe('secret+/=');
});
it('normalizes a versioned API base without duplicating /v1', () => {
localStorage.setItem(
SETTINGS_KEY,
JSON.stringify({ apiUrl: 'http://192.0.2.10:8000/v1/' }),
);
expect(buildWsUrl()).toBe('ws://192.0.2.10:8000/v1/agents/events');
});
it('omits the token for a keyless server', () => {
localStorage.setItem(
SETTINGS_KEY,
JSON.stringify({ apiUrl: 'http://localhost:8000' }),
);
const url = new URL(buildWsUrl('agent-one'));
expect(url.searchParams.has('token')).toBe(false);
});
});
+10 -13
View File
@@ -1,5 +1,5 @@
import { useEffect, useRef } from 'react';
import { getBase } from './api';
import { getApiKey, getBase } from './api';
export interface AgentEvent {
type: string;
@@ -7,19 +7,16 @@ export interface AgentEvent {
data: Record<string, unknown>;
}
function buildWsUrl(agentId?: string): string {
export function buildWsUrl(agentId?: string): string {
const base = getBase();
let origin: string;
if (base) {
origin = base.replace(/^http/, 'ws');
} else {
const loc = window.location;
origin = `${loc.protocol === 'https:' ? 'wss:' : 'ws:'}//${loc.host}`;
}
const path = '/v1/agents/events';
return agentId
? `${origin}${path}?agent_id=${encodeURIComponent(agentId)}`
: `${origin}${path}`;
const url = new URL('/v1/agents/events', base || window.location.origin);
url.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:';
if (agentId) url.searchParams.set('agent_id', agentId);
const apiKey = getApiKey();
if (apiKey) url.searchParams.set('token', apiKey);
return url.toString();
}
/**
+3 -3
View File
@@ -23,7 +23,6 @@ import {
fetchAgentTrace,
fetchManagedAgent,
fetchAvailableTools,
saveToolCredentials,
fetchModels,
updateManagedAgent,
fetchRecommendedModel,
@@ -575,7 +574,7 @@ function ToolsPicker({
</div>
{/* Live description strip */}
<div
className="flex items-center gap-2 px-2.5 py-1.5"
className="flex items-start gap-2 px-2.5 py-1.5"
style={{
borderTop: '1px solid var(--color-border)',
background: 'var(--color-bg)',
@@ -609,10 +608,11 @@ function ToolsPicker({
</span>
)}
<span
className="truncate"
className="min-w-0 whitespace-normal break-words"
style={{
flex: 1,
color: 'var(--color-text-tertiary)',
lineHeight: 1.4,
}}
>
{hovered ? `${hint}` : hint}
+36 -9
View File
@@ -27,6 +27,9 @@ import {
setInferenceSource,
getCloudKeyStatus,
saveCloudKey,
fetchToolCredentialStatus,
saveToolCredentials,
deleteToolCredential,
isTauri,
type InferenceSource,
} from '../lib/api';
@@ -56,25 +59,37 @@ function OllamaModelList() {
);
}
function ApiKeyInput({ keyName, placeholder }: { keyName: string; placeholder: string }) {
function ApiKeyInput({
keyName,
placeholder,
toolName,
}: {
keyName: string;
placeholder: string;
toolName?: string;
}) {
const [value, setValue] = useState('');
const [saved, setSaved] = useState(false);
const [hasKey, setHasKey] = useState(false);
const [error, setError] = useState('');
const desktopKeyStorage = isTauri();
const serverToolStorage = !desktopKeyStorage && !!toolName;
const canManage = desktopKeyStorage || serverToolStorage;
const refresh = useCallback(async () => {
if (!desktopKeyStorage) {
if (!canManage) {
setHasKey(false);
return;
}
try {
const status = await getCloudKeyStatus();
const status = desktopKeyStorage
? await getCloudKeyStatus()
: await fetchToolCredentialStatus(toolName!);
setHasKey(!!status[keyName]);
} catch {
setHasKey(false);
}
}, [desktopKeyStorage, keyName]);
}, [canManage, desktopKeyStorage, keyName, toolName]);
useEffect(() => {
void refresh();
@@ -87,7 +102,13 @@ function ApiKeyInput({ keyName, placeholder }: { keyName: string; placeholder: s
if (!next) return;
setError('');
try {
await saveCloudKey(keyName, next);
if (desktopKeyStorage) {
await saveCloudKey(keyName, next);
} else if (toolName) {
await saveToolCredentials(toolName, { [keyName]: next });
} else {
return;
}
setValue('');
setHasKey(true);
setSaved(true);
@@ -101,7 +122,13 @@ function ApiKeyInput({ keyName, placeholder }: { keyName: string; placeholder: s
const remove = async () => {
setError('');
try {
await saveCloudKey(keyName, '');
if (desktopKeyStorage) {
await saveCloudKey(keyName, '');
} else if (toolName) {
await deleteToolCredential(toolName, keyName);
} else {
return;
}
setValue('');
setHasKey(false);
setSaved(true);
@@ -119,8 +146,8 @@ function ApiKeyInput({ keyName, placeholder }: { keyName: string; placeholder: s
value={value}
onChange={e => setValue(e.target.value)}
onBlur={() => { if (value.trim()) void save(value); }}
placeholder={hasKey ? 'Saved in secure storage' : placeholder}
disabled={!desktopKeyStorage}
placeholder={hasKey ? (desktopKeyStorage ? 'Saved in secure storage' : 'Saved by local server') : placeholder}
disabled={!canManage}
className="w-48 px-2 py-1 rounded text-xs"
style={{ background: 'var(--color-bg)', border: '1px solid var(--color-border)', color: 'var(--color-text)' }} />
{hasKey && (
@@ -542,7 +569,7 @@ export function SettingsPage() {
{/* Tools */}
<Section title="Tools">
<SettingRow label="Web Search" description="Tavily key for web search tool">
<ApiKeyInput keyName="TAVILY_API_KEY" placeholder="tvly-..." />
<ApiKeyInput keyName="TAVILY_API_KEY" placeholder="tvly-..." toolName="web_search" />
</SettingRow>
</Section>
+1
View File
@@ -147,6 +147,7 @@ export interface ConversationStore {
// --- Stream State ---
export interface StreamState {
conversationId: string | null;
isStreaming: boolean;
phase: string;
elapsedMs: number;
+9 -1
View File
@@ -54,7 +54,15 @@ export default defineConfig({
server: {
port: 5173,
proxy: {
'/v1': process.env.VITE_API_URL || 'http://localhost:8000',
// ws: true is required for the /v1/agents/events WebSocket. Without it
// Vite proxies the HTTP request but not the upgrade, so the socket never
// opens — no error, no close event, just silence — and every live agent
// view sits empty in dev while working in a production build.
'/v1': {
target: process.env.VITE_API_URL || 'http://localhost:8000',
changeOrigin: true,
ws: true,
},
'/health': process.env.VITE_API_URL || 'http://localhost:8000',
'/api': process.env.VITE_API_URL || 'http://localhost:8000',
},
+1
View File
@@ -198,6 +198,7 @@ nav:
- Benchmarks: user-guide/benchmarks.md
- System Access: user-guide/system-access.md
- Security: user-guide/security.md
- Data Boundary Scan: user-guide/data-boundary-scan.md
- LLM-guided spec search: user-guide/llm-guided-spec-search.md
- Leaderboard: leaderboard.md
- Roadmap: development/roadmap.md
@@ -33,6 +33,33 @@ impl PySQLiteMemory {
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))
}
fn replace_source(
&self,
source: &str,
documents: Vec<(String, Option<String>)>,
) -> PyResult<Vec<String>> {
let parsed_documents = documents
.into_iter()
.map(|(content, metadata)| {
let metadata = metadata
.map(|value| serde_json::from_str(&value))
.transpose()
.map_err(|e| {
PyErr::new::<pyo3::exceptions::PyValueError, _>(e.to_string())
})?;
Ok((content, metadata))
})
.collect::<PyResult<Vec<_>>>()?;
let document_refs = parsed_documents
.iter()
.map(|(content, metadata)| (content.as_str(), metadata.as_ref()))
.collect::<Vec<_>>();
self.inner
.replace_source(source, &document_refs)
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))
}
#[pyo3(signature = (query, top_k=5))]
fn retrieve(&self, query: &str, top_k: usize) -> PyResult<String> {
let results = self
@@ -94,6 +94,57 @@ impl SQLiteMemory {
pub fn in_memory() -> Result<Self, OpenJarvisError> {
Self::new(Path::new(":memory:"))
}
/// Atomically replace every document for *source* with *documents*.
pub fn replace_source(
&self,
source: &str,
documents: &[(&str, Option<&Value>)],
) -> Result<Vec<String>, OpenJarvisError> {
let mut conn = self.conn.lock();
let tx = conn.transaction().map_err(|e| {
OpenJarvisError::Io(std::io::Error::other(e.to_string()))
})?;
tx.execute(
"DELETE FROM documents_fts
WHERE rowid IN (SELECT rowid FROM documents WHERE source = ?1)",
rusqlite::params![source],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
tx.execute(
"DELETE FROM documents WHERE source = ?1",
rusqlite::params![source],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
let mut doc_ids = Vec::with_capacity(documents.len());
for (content, metadata) in documents {
let doc_id = Uuid::new_v4().to_string();
let meta_str = metadata
.map(|m| serde_json::to_string(m).unwrap_or_default())
.unwrap_or_else(|| "{}".to_string());
tx.execute(
"INSERT INTO documents (id, content, source, metadata)
VALUES (?1, ?2, ?3, ?4)",
rusqlite::params![doc_id, content, source, meta_str],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
let rowid = tx.last_insert_rowid();
tx.execute(
"INSERT INTO documents_fts (rowid, content, source) VALUES (?1, ?2, ?3)",
rusqlite::params![rowid, content, source],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
doc_ids.push(doc_id);
}
tx.commit()
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
Ok(doc_ids)
}
}
impl MemoryBackend for SQLiteMemory {
@@ -306,6 +357,40 @@ mod tests {
assert_eq!(mem.count().unwrap(), 0);
}
#[test]
fn test_sqlite_replace_source_is_idempotent() {
let mem = SQLiteMemory::in_memory().unwrap();
mem.replace_source("notes.txt", &[("old project notes", None)])
.unwrap();
assert_eq!(mem.count().unwrap(), 1);
mem.replace_source("notes.txt", &[("updated project notes", None)])
.unwrap();
assert_eq!(mem.count().unwrap(), 1);
assert!(mem.retrieve("old", 5).unwrap().is_empty());
let updated = mem.retrieve("updated", 5).unwrap();
assert_eq!(updated.len(), 1);
assert_eq!(updated[0].source, "notes.txt");
}
#[test]
fn test_sqlite_replace_source_preserves_other_sources() {
let mem = SQLiteMemory::in_memory().unwrap();
mem.store("keep this manual", "manual.txt", None).unwrap();
mem.replace_source("notes.txt", &[("old project notes", None)])
.unwrap();
mem.replace_source("notes.txt", &[("updated project notes", None)])
.unwrap();
assert_eq!(mem.count().unwrap(), 2);
let manual = mem.retrieve("manual", 5).unwrap();
assert_eq!(manual.len(), 1);
assert_eq!(manual[0].source, "manual.txt");
}
#[test]
fn test_sqlite_case_insensitive_search() {
let mem = SQLiteMemory::in_memory().unwrap();
+10 -3
View File
@@ -148,7 +148,8 @@ fi
# ── 7. Install Python dependencies ──────────────────────────────────
info "Installing Python dependencies..."
uv sync --extra desktop --quiet 2>/dev/null || uv sync --extra desktop
uv sync --extra desktop --extra tools-search --quiet 2>/dev/null \
|| uv sync --extra desktop --extra tools-search
ok "Python dependencies installed"
# ── 7b. Build Rust extension ──────────────────────────────────────
@@ -164,11 +165,17 @@ ok "Frontend dependencies installed"
# ── 9. Start backend ────────────────────────────────────────────────
info "Starting backend API server on port 8000..."
if curl -sf http://localhost:8000/health &>/dev/null; then
fail "An OpenJarvis server is already running on port 8000. Stop it before re-running quickstart so updated environment variables are applied."
fi
uv run jarvis serve --port 8000 &>/dev/null &
CLEANUP_PIDS+=($!)
BACKEND_PID=$!
CLEANUP_PIDS+=("$BACKEND_PID")
sleep 3
if curl -sf http://localhost:8000/health &>/dev/null; then
if ! kill -0 "$BACKEND_PID" 2>/dev/null; then
fail "Backend exited during startup. Run 'uv run jarvis serve --port 8000' to see the error."
elif curl -sf http://localhost:8000/health &>/dev/null; then
ok "Backend running at http://localhost:8000"
else
warn "Backend may still be starting..."
+21 -1
View File
@@ -4,8 +4,10 @@ from __future__ import annotations
from importlib.metadata import PackageNotFoundError
from importlib.metadata import version as _pkg_version
from typing import TYPE_CHECKING, Any
from openjarvis.sdk import Jarvis, JarvisSystem, MemoryHandle, SystemBuilder
if TYPE_CHECKING:
from openjarvis.sdk import Jarvis, JarvisSystem, MemoryHandle, SystemBuilder
try:
__version__ = _pkg_version("openjarvis")
@@ -13,3 +15,21 @@ except PackageNotFoundError: # pragma: no cover — uninstalled source tree
__version__ = "0.0.0+unknown"
__all__ = ["Jarvis", "JarvisSystem", "MemoryHandle", "SystemBuilder", "__version__"]
_SDK_EXPORTS = {"Jarvis", "JarvisSystem", "MemoryHandle", "SystemBuilder"}
def __getattr__(name: str) -> Any:
"""Load SDK exports lazily so lightweight CLI diagnostics can start safely."""
if name not in _SDK_EXPORTS:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
from openjarvis import sdk
value = getattr(sdk, name)
globals()[name] = value
return value
def __dir__() -> list[str]:
return sorted(set(globals()) | _SDK_EXPORTS)
+28 -2
View File
@@ -57,6 +57,10 @@ class BaseAgent(ABC):
agent_id: str
accepts_tools: bool = False
# Plain conversational agents may opt into the managed runtime's generic
# function-calling loop. Specialized agents keep their own execution
# class even when process-wide MCP tools are available.
supports_managed_tool_fallback: bool = False
def __init__(
self,
@@ -151,6 +155,9 @@ class BaseAgent(ABC):
conversation messages, and finally the user input.
"""
messages: list[Message] = []
context_messages = (
list(context.conversation.messages) if context is not None else []
)
# Check if the context already supplies a system message
_context_has_system = (
context
@@ -172,9 +179,28 @@ class BaseAgent(ABC):
except Exception:
effective_system_prompt = None
if effective_system_prompt:
context_system_text = "\n\n".join(
message.text
for message in context_messages
if message.role == Role.SYSTEM
and message.metadata.get("memory_context")
and message.text
)
if context_system_text:
effective_system_prompt = (
f"{effective_system_prompt}\n\n{context_system_text}"
)
context_messages = [
message
for message in context_messages
if not (
message.role == Role.SYSTEM
and message.metadata.get("memory_context")
)
]
messages.append(Message(role=Role.SYSTEM, content=effective_system_prompt))
if context and context.conversation.messages:
messages.extend(context.conversation.messages)
if context_messages:
messages.extend(context_messages)
messages.append(Message(role=Role.USER, content=input))
return messages
+189 -96
View File
@@ -2,7 +2,9 @@
from __future__ import annotations
import json
import logging
import threading
import time
from typing import TYPE_CHECKING, Any
@@ -14,6 +16,7 @@ from openjarvis.agents.errors import (
classify_error,
retry_delay,
)
from openjarvis.agents.tool_resolver import resolve_agent_tools
from openjarvis.core.events import EventBus, EventType
if TYPE_CHECKING:
@@ -33,6 +36,32 @@ _MAX_RETRIES = 3
_AGENT_TICK_DEFAULT_MODEL = "gemma4:31b"
def _tool_calls_for_storage(result: AgentResult) -> list[dict[str, Any]] | None:
"""Convert executor tool results to the managed-message storage contract."""
calls: list[dict[str, Any]] = []
for tool_result in result.tool_results:
metadata = getattr(tool_result, "metadata", {}) or {}
arguments = metadata.get("arguments", "")
if not isinstance(arguments, str):
try:
arguments = json.dumps(arguments, sort_keys=True)
except (TypeError, ValueError):
arguments = json.dumps(str(arguments))
calls.append(
{
"tool": getattr(tool_result, "tool_name", ""),
"arguments": arguments,
"result": getattr(tool_result, "content", "") or "",
"success": bool(getattr(tool_result, "success", False)),
# SSE and the frontend persist/display latency in milliseconds.
"latency": float(getattr(tool_result, "latency_seconds", 0.0) or 0.0)
* 1000.0,
}
)
return calls or None
class AgentExecutor:
"""Executes a single tick for a managed agent.
@@ -51,6 +80,7 @@ class AgentExecutor:
self._manager = manager
self._bus = event_bus
self._trace_store = trace_store
self._toolkit_local = threading.local()
def set_system(self, system: Any) -> None:
"""Deferred system injection — called after JarvisSystem is constructed."""
@@ -63,27 +93,6 @@ class AgentExecutor:
except Exception:
pass # Non-critical
def _inject_tool_deps(self, tool: Any) -> None:
"""Inject runtime dependencies into a tool instance.
Mirrors SystemBuilder._inject_tool_deps (system.py:920-945)
but uses the lightweight system's references.
"""
if self._system is None:
return
name = getattr(getattr(tool, "spec", None), "name", "")
if name == "llm":
if hasattr(tool, "_engine"):
tool._engine = self._system.engine
if hasattr(tool, "_model"):
tool._model = self._system.model
elif name == "retrieval" or name.startswith("memory_"):
if hasattr(tool, "_backend"):
tool._backend = getattr(self._system, "memory_backend", None)
elif name.startswith("channel_"):
if hasattr(tool, "_channel"):
tool._channel = getattr(self._system, "channel_backend", None)
def run_ephemeral(
self,
agent_type: str,
@@ -248,7 +257,20 @@ class AgentExecutor:
raise last_error or FatalError("max retries exhausted")
def _invoke_agent(self, agent: dict) -> AgentResult:
"""Invoke the actual agent run. Tests mock this method."""
"""Invoke one agent while owning every resource its resolver opens."""
previous = getattr(self._toolkit_local, "current", None)
self._toolkit_local.current = None
try:
return self._invoke_agent_impl(agent)
finally:
current = getattr(self._toolkit_local, "current", None)
if current is not None:
current.close()
self._toolkit_local.current = previous
def _invoke_agent_impl(self, agent: dict) -> AgentResult:
"""Implementation split out so the wrapper owns resolver lifetime."""
from openjarvis.agents import AgentRegistry
agent_type = agent.get("agent_type", "monitor_operative")
@@ -257,6 +279,10 @@ class AgentExecutor:
raise FatalError(f"Unknown agent type: {agent_type}")
config = agent.get("config", {})
agent_accepts_tools = bool(getattr(agent_cls, "accepts_tools", False))
supports_tool_fallback = bool(
getattr(agent_cls, "supports_managed_tool_fallback", False)
)
# Resolve engine + model from JarvisSystem
engine = self._system.engine if self._system else None
@@ -300,64 +326,88 @@ class AgentExecutor:
except Exception:
pass # Fall back to configured model
# Resolve tools from config via ToolRegistry
tool_names = config.get("tools", [])
if isinstance(tool_names, str):
tool_names = [t.strip() for t in tool_names.split(",") if t.strip()]
mcp_tools: list[Any] = []
mcp_clients: list[Any] = []
if (
config.get("mcp_tools", True) is not False
and self._system is not None
and (agent_accepts_tools or supports_tool_fallback)
):
provider = getattr(
self._system,
"get_managed_agent_mcp_tools",
None,
)
if callable(provider):
try:
mcp_tools, mcp_clients = provider()
except Exception as exc:
logger.warning("Managed-agent MCP discovery failed: %s", exc)
else:
mcp_tools = list(getattr(self._system, "mcp_tools", []) or [])
mcp_clients = list(getattr(self._system, "_mcp_clients", []) or [])
tool_instances: list[Any] = []
if tool_names:
try:
from openjarvis.server.agent_manager_routes import (
_ensure_registries_populated,
)
if not mcp_tools:
try:
from openjarvis.tools.mcp_adapter import MCPToolAdapter
_ensure_registries_populated()
except ImportError:
pass
from openjarvis.core.registry import ToolRegistry
pool = (
getattr(
getattr(self._system, "tool_executor", None),
"_tools",
{},
)
or {}
)
mcp_tools = [
tool
for tool in pool.values()
if isinstance(tool, MCPToolAdapter)
]
except Exception:
mcp_tools = []
for tname in tool_names:
if ToolRegistry.contains(tname):
try:
tool_cls = ToolRegistry.get(tname)
tool = tool_cls()
self._inject_tool_deps(tool)
tool_instances.append(tool)
except Exception:
logger.warning("Failed to instantiate tool %s", tname)
resolved_toolkit = resolve_agent_tools(
agent,
engine=engine,
model=model,
memory_backend=getattr(self._system, "memory_backend", None),
channel_backend=getattr(self._system, "channel_backend", None),
mcp_tools=mcp_tools,
mcp_clients=mcp_clients,
knowledge_db_path=getattr(self._system, "knowledge_db_path", None),
)
self._toolkit_local.current = resolved_toolkit
tool_instances = resolved_toolkit.instances
logger.info(
"Agent %s: resolved %d tools (%s)",
agent["name"],
len(tool_instances),
", ".join(resolved_toolkit.by_name) or "none",
)
# Pull tools already discovered by SystemBuilder (e.g. external MCP
# adapters) that aren't in the static ToolRegistry. Without this,
# agents declaring MCP-discovered tools in their template would
# silently fall back to natives only.
if (
self._system is not None
and getattr(self._system, "tool_executor", None) is not None
):
mcp_pool = getattr(self._system.tool_executor, "_tools", {}) or {}
existing = {t.spec.name for t in tool_instances}
for tname in tool_names:
if tname in existing:
continue
pooled = mcp_pool.get(tname)
if pooled is not None:
tool_instances.append(pooled)
execution_agent_cls = agent_cls
if tool_instances and not agent_accepts_tools and supports_tool_fallback:
# Managed SSE already runs configured tools through a native
# function-calling loop regardless of the selected class. Use the
# same capability for immediate/scheduled ticks instead of
# silently discarding the resolved toolkit for SimpleAgent and
# other explicitly compatible non-tool classes.
from openjarvis.agents.orchestrator import OrchestratorAgent
if tool_instances:
logger.info(
"Agent %s: resolved %d/%d tools",
agent["name"],
len(tool_instances),
len(tool_names),
)
execution_agent_cls = OrchestratorAgent
logger.info(
"Agent %s: %s does not accept tools; using %s for this "
"tool-enabled tick",
agent["name"],
agent_cls.__name__,
execution_agent_cls.__name__,
)
# Construct agent instance
agent_kwargs: dict[str, Any] = {}
sys_prompt = config.get("system_prompt")
if sys_prompt is not None:
agent_kwargs["system_prompt"] = sys_prompt
if getattr(agent_cls, "accepts_tools", False) and tool_instances:
if getattr(execution_agent_cls, "accepts_tools", False) and tool_instances:
agent_kwargs["tools"] = tool_instances
# Hand the agent our EventBus so its ToolExecutor can publish
# TOOL_CALL_START/END — without this, ToolExecutor's ``self._bus``
@@ -379,7 +429,7 @@ class AgentExecutor:
# recall / persistence paths.
import inspect
init_sig = inspect.signature(agent_cls.__init__)
init_sig = inspect.signature(execution_agent_cls.__init__)
accepts_var_kw = any(
p.kind == inspect.Parameter.VAR_KEYWORD
for p in init_sig.parameters.values()
@@ -388,6 +438,16 @@ class AgentExecutor:
def _accepts(name: str) -> bool:
return accepts_var_kw or name in init_sig.parameters
# Unsupported kwargs used to trigger the broad TypeError fallback
# below, which retried with a bare constructor and silently discarded
# valid prompt/state wiring. Filter by the selected class's signature
# before construction instead.
if sys_prompt is not None and _accepts("system_prompt"):
agent_kwargs["system_prompt"] = sys_prompt
agent_kwargs = {
name: value for name, value in agent_kwargs.items() if _accepts(name)
}
state_kwargs: dict[str, Any] = {}
if _accepts("operator_id"):
state_kwargs["operator_id"] = agent["id"]
@@ -404,23 +464,49 @@ class AgentExecutor:
# agents, mirroring the one-shot `jarvis ask` path so they no
# longer apply to CLI calls only (#376).
cfg = getattr(self._system, "config", None)
if cfg is not None and _accepts("prompt_builder"):
if _accepts("prompt_builder") and (
cfg is not None or sys_prompt is not None
):
from openjarvis.prompt.builder import SystemPromptBuilder
state_kwargs["prompt_builder"] = SystemPromptBuilder(
agent_template=getattr(cfg.agent, "default_system_prompt", "")
or "",
memory_files_config=cfg.memory_files,
system_prompt_config=cfg.system_prompt,
agent_template=(
sys_prompt
if sys_prompt is not None
else getattr(
getattr(cfg, "agent", None),
"default_system_prompt",
"",
)
or ""
),
memory_files_config=getattr(cfg, "memory_files", None),
system_prompt_config=getattr(cfg, "system_prompt", None),
)
try:
agent_instance = agent_cls(engine, model, **agent_kwargs, **state_kwargs)
except TypeError:
try:
agent_instance = agent_cls(engine, model, **agent_kwargs)
agent_instance = execution_agent_cls(
engine,
model,
**agent_kwargs,
**state_kwargs,
)
except TypeError:
agent_instance = agent_cls(engine, model)
try:
agent_instance = execution_agent_cls(
engine,
model,
**agent_kwargs,
)
except TypeError:
agent_instance = execution_agent_cls(engine, model)
except Exception:
resolved_toolkit.close()
raise
if resolved_toolkit.mcp_clients:
agent_instance._mcp_clients = resolved_toolkit.mcp_clients
# Inject the managed-agent UUID into the agent's ToolExecutor so
# emitted TOOL_CALL_START/END events carry it; the trace subscriber
@@ -436,7 +522,7 @@ class AgentExecutor:
agent["name"],
len(tool_instances),
", ".join(t.spec.name for t in tool_instances) or "none",
agent_cls.__name__,
execution_agent_cls.__name__,
)
# Build input from instruction + summary_memory + pending messages.
@@ -551,21 +637,24 @@ class AgentExecutor:
len(input_text),
)
_t0 = time.time()
result = agent_instance.run(input_text, context=agent_ctx)
# Retry once if the model returned empty content (common with
# Qwen3.5 thinking mode consuming all tokens).
if not (result.content or "").strip():
self._set_activity(
agent["id"],
"Retrying (empty response)...",
)
logger.warning(
"Agent %s: empty content, retrying once",
agent["name"],
)
try:
result = agent_instance.run(input_text, context=agent_ctx)
# Retry once if the model returned empty content (common with
# Qwen3.5 thinking mode consuming all tokens).
if not (result.content or "").strip():
self._set_activity(
agent["id"],
"Retrying (empty response)...",
)
logger.warning(
"Agent %s: empty content, retrying once",
agent["name"],
)
result = agent_instance.run(input_text, context=agent_ctx)
finally:
resolved_toolkit.close()
_elapsed = time.time() - _t0
logger.info(
"Agent %s: agent.run() completed in %.1fs, "
@@ -655,7 +744,11 @@ class AgentExecutor:
# message keeps the complete report. The old [:2000] slices
# double-truncated and cut findings off mid-sentence.
self._manager.update_summary_memory(agent_id, result.content)
self._manager.store_agent_response(agent_id, result.content)
self._manager.store_agent_response(
agent_id,
result.content,
tool_calls=_tool_calls_for_storage(result),
)
# Budget enforcement (post-tick check)
agent_data = self._manager.get_agent(agent_id)
+2 -1
View File
@@ -127,6 +127,7 @@ class MonitorOperativeAgent(ToolUsingAgent):
memory_backend: Optional[Any] = None,
interactive: bool = False,
confirm_callback=None,
prompt_builder: Optional[Any] = None,
**kwargs: Any,
) -> None:
super().__init__(
@@ -139,7 +140,7 @@ class MonitorOperativeAgent(ToolUsingAgent):
max_tokens=max_tokens,
interactive=interactive,
confirm_callback=confirm_callback,
prompt_builder=kwargs.get("prompt_builder"),
prompt_builder=prompt_builder,
)
# Validate strategies
if memory_extraction not in VALID_MEMORY_EXTRACTION:
+30 -38
View File
@@ -17,6 +17,14 @@ from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import Message, Role, ToolCall
_SECTION_PROMPTS = {
"messages": "MESSAGES — Prioritize provided messages or tasks needing action.",
"calendar": "CALENDAR — Cover only provided upcoming events.",
"health": "HEALTH — Describe only supported trends; omit raw measurements.",
"world": "WORLD — Summarize only provided world items.",
"music": "MUSIC — Summarize only provided listening information.",
}
def _load_persona(persona_name: str) -> str:
"""Load a persona prompt file by name."""
@@ -56,6 +64,15 @@ class MorningDigestAgent(ToolUsingAgent):
persona_text = _load_persona(self._persona)
now = datetime.now()
honorific = getattr(self, "_honorific", "sir")
sections = dict.fromkeys(
str(section).strip().casefold()
for section in self._sections
if str(section).strip()
)
section_block = "\n".join(
f"- {_SECTION_PROMPTS.get(section, section.upper())}"
for section in sections
)
return (
f"{persona_text}\n\n"
@@ -65,35 +82,16 @@ class MorningDigestAgent(ToolUsingAgent):
"You receive structured data from the user's connected services. "
"The data has ALREADY been collected — it appears in the user "
"message. You do NOT fetch anything yourself.\n\n"
"Produce a 2-4 minute spoken briefing in DECREASING order of "
"importance:\n\n"
"1. GREETING + PRIORITIES — Open with the honorific and "
"immediately state what needs attention: overdue tasks, today's "
"deadlines, events requiring preparation. Connect related items "
"('Your rebuttals are overdue and you have a dinner at 6, so "
"I'd tackle those first').\n\n"
"2. SCHEDULE — Today's upcoming events with time context: 'You "
"have 3 hours before your next meeting.' Skip past events.\n\n"
"3. MESSAGES — Triage across ALL channels (email, texts, Slack):\n"
" - First: messages from real people needing a REPLY or DECISION\n"
" - Second: messages containing deadlines or action items\n"
" - Last: brief acknowledgment of casual threads ('Your group "
"chat has been lively but nothing requiring a response')\n"
" - SKIP automated emails, newsletters, and marketing entirely\n"
" - Quote relevant message text when it helps\n\n"
"4. HEALTH — Interpret trends, not raw numbers. 'Your sleep has "
"improved three nights running and your readiness is strong'"
"not 'HRV 53, HR 56.' If multiple days of data, compare.\n\n"
"5. WORLD — Weather forecast, top news (AI/tech, business, "
"general). Skip if no data.\n\n"
"6. CLOSING — One forward-looking sentence with the honorific.\n\n"
"Produce a concise spoken briefing in decreasing order of importance. "
"Cover only the configured sections below and only when the collected "
"data supports them. Silently omit absent data and sources.\n\n"
f"CONFIGURED SECTIONS:\n{section_block or '- None'}\n\n"
"Open briefly with the honorific and end after the last supported item. "
"Do not add conversational offers or personal asides.\n\n"
"ABSOLUTE RULES (violations are unacceptable):\n"
"- ONLY facts from the data. Zero hallucination.\n"
"- NEVER mention disconnected or unavailable sources.\n"
"- NEVER state raw health numbers. Say 'your sleep was solid' "
"NOT 'heart rate 56 bpm' or 'HRV 53' or '6000 steps' or "
"'readiness 82'. Interpret, never enumerate.\n"
"- NEVER describe actions you are taking.\n"
"- NEVER invent personal context or claim, offer, or suggest actions.\n"
"- Acknowledge every source that returned data, even briefly.\n"
"- No markdown, emojis, bullets, or headers.\n"
"- STRICT LIMIT: 200 words. Be concise."
@@ -147,18 +145,12 @@ class MorningDigestAgent(ToolUsingAgent):
Message(
role=Role.USER,
content=(
f"Here is the collected data from my sources:\n\n"
f"{collected_data}\n\n"
f"Synthesize my morning briefing. Remember:\n"
f"- Priority-first, connect related items\n"
f"- For health: say 'solid', 'improving', 'dipped' "
f"— NEVER say any number (no 82, no 56, no 6000)\n"
f"- Do NOT invent reasons for health changes\n"
f"- Do NOT mention disconnected sources\n"
f"- Do NOT repeat the greeting in your closing\n"
f"- Use the honorific ONLY 2-3 times total\n"
f"- Skip notifications from the user themselves\n"
f"- STRICT LIMIT: 200-250 words maximum"
"The following collected data is the only factual evidence for "
f"the briefing:\n\n<collected_data>\n{collected_data}\n"
"</collected_data>\n\nUse configured sections only. Omit missing "
"data and sources. Do not add personal context or activities. "
"Use the honorific no more than three times and keep the "
"briefing under 200 words."
),
),
]
+2 -1
View File
@@ -58,6 +58,7 @@ class OperativeAgent(ToolUsingAgent):
memory_backend: Optional[Any] = None,
interactive: bool = False,
confirm_callback=None,
prompt_builder: Optional[Any] = None,
**kwargs: Any,
) -> None:
super().__init__(
@@ -70,7 +71,7 @@ class OperativeAgent(ToolUsingAgent):
max_tokens=max_tokens,
interactive=interactive,
confirm_callback=confirm_callback,
prompt_builder=kwargs.get("prompt_builder"),
prompt_builder=prompt_builder,
)
self._system_prompt = system_prompt or ""
self._operator_id = operator_id
+88 -3
View File
@@ -13,6 +13,7 @@ Supports two modes:
from __future__ import annotations
import concurrent.futures
import json
import re
from typing import Any, List, Optional
@@ -57,6 +58,7 @@ class OrchestratorAgent(ToolUsingAgent):
max_tokens: Optional[int] = None,
mode: str = "function_calling",
system_prompt: Optional[str] = None,
prompt_builder: Optional[Any] = None,
parallel_tools: bool = True,
interactive: bool = False,
confirm_callback=None,
@@ -71,6 +73,7 @@ class OrchestratorAgent(ToolUsingAgent):
max_tokens=max_tokens,
interactive=interactive,
confirm_callback=confirm_callback,
prompt_builder=prompt_builder,
)
self._mode = mode
self._system_prompt = system_prompt
@@ -140,12 +143,21 @@ class OrchestratorAgent(ToolUsingAgent):
tool_call = ToolCall(
id=f"orch_{turns}",
name=parsed["tool"],
arguments=parsed["input"] or "{}",
arguments=self._normalize_structured_tool_input(
parsed["tool"],
parsed["input"],
),
)
tool_result = self._executor.execute(tool_call)
all_tool_results.append(tool_result)
observation = f"Observation: {tool_result.content}"
if tool_result.success:
observation = f"Observation: {tool_result.content}"
else:
observation = (
f"Observation: Tool '{tool_result.tool_name}' failed: "
f"{tool_result.content}"
)
messages.append(Message(role=Role.USER, content=observation))
continue
@@ -160,6 +172,75 @@ class OrchestratorAgent(ToolUsingAgent):
# Max turns exceeded
return self._max_turns_result(all_tool_results, turns)
def _normalize_structured_tool_input(
self,
tool_name: str,
raw_input: str,
) -> str:
"""Map unambiguous structured text input to a string parameter."""
if not raw_input:
return "{}"
try:
parsed_input = json.loads(raw_input)
except json.JSONDecodeError:
invalid_json = True
string_value = raw_input
else:
invalid_json = False
if isinstance(parsed_input, dict):
return raw_input
# INPUT is a text protocol. A non-object JSON value such as 42,
# true, null, or [1, 2] may still be the intended text for a tool's
# string parameter. Quoted JSON strings are decoded to remove only
# their surrounding quotes; other values retain their source text.
string_value = parsed_input if isinstance(parsed_input, str) else raw_input
tool_spec = None
for candidate in reversed(self._tools):
candidate_spec = candidate.spec
if candidate_spec.name == tool_name:
tool_spec = candidate_spec
break
if tool_spec is None:
return raw_input
parameters = tool_spec.parameters
parameter_container_type = parameters.get("type")
if parameter_container_type not in (None, "object"):
return raw_input
properties = parameters.get("properties", {})
required = parameters.get("required", [])
if not isinstance(properties, dict) or not isinstance(required, list):
return raw_input
if len(required) == 1 and required[0] in properties:
parameter_name = required[0]
elif not required and len(properties) == 1:
parameter_name = next(iter(properties))
else:
return raw_input
parameter_schema = properties[parameter_name]
if not isinstance(parameter_schema, dict):
return raw_input
parameter_type = parameter_schema.get("type")
accepts_string = parameter_type == "string" or (
isinstance(parameter_type, list) and "string" in parameter_type
)
if not accepts_string:
return raw_input
allow_object_text = (
tool_spec.metadata.get("structured_allow_object_text") is True
)
starts_like_object = raw_input.lstrip("\ufeff \t\r\n").startswith("{")
if invalid_json and starts_like_object and not allow_object_text:
return raw_input
return json.dumps({parameter_name: string_value})
@staticmethod
def _parse_structured_response(text: str) -> dict:
"""Parse THOUGHT/TOOL/INPUT/FINAL_ANSWER from model output."""
@@ -214,7 +295,11 @@ class OrchestratorAgent(ToolUsingAgent):
self._emit_turn_start(input)
# Build initial messages
messages = self._build_messages(input, context)
messages = self._build_messages(
input,
context,
system_prompt=self._system_prompt,
)
# Get OpenAI-format tool definitions
openai_tools = self._executor.get_openai_tools() if self._tools else []
+99 -12
View File
@@ -37,6 +37,7 @@ called from your app startup:
from __future__ import annotations
import json
import logging
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional, Set
@@ -56,6 +57,15 @@ from openjarvis.tools.approval_store import (
)
from openjarvis.tools.proactive_tools import get_store
logger = logging.getLogger(__name__)
_PROACTIVE_CRON_PROMPT = (
"Run the proactive agent: collect overnight data, execute approved actions, "
"notify pending approvals."
)
_PROACTIVE_TASK_KEY = "proactive-daily"
_PROACTIVE_TASK_KEY_FIELD = "openjarvis_task_key"
_SYSTEM_PROMPT = """You are a proactive personal assistant agent. You have already collected
data from the user's connected sources (email, messages, calendar). Your job is to:
@@ -252,14 +262,31 @@ def _build_notification_channel(channel_spec: str) -> Optional[Any]:
if ChannelRegistry.contains(channel_type):
channel_cls = ChannelRegistry.get(channel_type)
instance = channel_cls()
# Load credentials from config so the channel uses bot_token from
# config.toml rather than falling back to a bare env var.
try:
instance.connect()
from openjarvis.core.config import load_config
from openjarvis.system._channel_kwargs import build_channel_kwargs
_cfg = load_config()
_kwargs = build_channel_kwargs(_cfg.channel, channel_type)
except Exception:
pass
_kwargs = {}
instance = channel_cls(**_kwargs)
# Telegram.send() is self-contained, while connect() starts a
# getUpdates loop. A second loop for the same bot token conflicts
# with the server's main listener. Other channel implementations
# may initialize resources required by send() in connect(), so keep
# their established lifecycle intact.
if channel_type != "telegram":
instance.connect()
return instance
except Exception:
pass
logger.warning(
"Failed to build proactive notification channel %s",
channel_type,
exc_info=True,
)
return None
@@ -299,6 +326,7 @@ class ProactiveAgent(ToolUsingAgent):
self._notification_channel_id
)
self._notification_channel = notification_channel
self._notification_destination = self._notification_channel_id.partition(":")[2]
from openjarvis.tools.channel_tools import ChannelSendTool
from openjarvis.tools.digest_collect import DigestCollectTool
@@ -484,13 +512,13 @@ class ProactiveAgent(ToolUsingAgent):
# --- Step 5: Build and send notification ---
notification = self._build_notification(executed_results, pending_actions)
if notification and self._notification_channel_id:
if notification and self._notification_destination:
send_call = ToolCall(
id="proactive-notify-1",
name="channel_send",
arguments=json.dumps(
{
"channel": self._notification_channel_id,
"channel": self._notification_destination,
"content": notification,
}
),
@@ -592,15 +620,74 @@ def register_cron(
hours_back = hours_back or 24
timezone = timezone or "America/Los_Angeles"
metadata = {
"notification_channel_id": notification_channel_id,
"hours_back": hours_back,
"timezone": timezone,
_PROACTIVE_TASK_KEY_FIELD: _PROACTIVE_TASK_KEY,
}
# Match the stable key for tasks created by this version and the historical
# agent+prompt signature so existing installations are migrated on startup.
existing = [
task
for task in scheduler.list_tasks()
if task.status in {"active", "paused"}
and task.agent == "proactive"
and (
task.metadata.get(_PROACTIVE_TASK_KEY_FIELD) == _PROACTIVE_TASK_KEY
or (task.prompt == _PROACTIVE_CRON_PROMPT and task.schedule_type == "cron")
)
]
# A scheduler pause is an explicit user choice and must survive restart.
# Keep one deterministically and remove any active or paused duplicates.
paused = [task for task in existing if task.status == "paused"]
if paused:
keep = min(paused, key=lambda task: task.id)
_cancel_proactive_duplicates(scheduler, existing, keep=keep)
return keep
matching = [
task
for task in existing
if task.prompt == _PROACTIVE_CRON_PROMPT
and task.schedule_type == "cron"
and task.schedule_value == cron_expr
and task.context_mode == "isolated"
and task.metadata == metadata
]
if matching:
keep = min(matching, key=lambda task: task.id)
_cancel_proactive_duplicates(scheduler, existing, keep=keep)
return keep
# Configuration changed. Replace stale active tasks so the schedule and
# notification settings from config.toml take effect on this startup.
_cancel_proactive_duplicates(scheduler, existing)
return scheduler.create_task(
prompt="Run the proactive agent: collect overnight data, execute approved actions, notify pending approvals.",
prompt=_PROACTIVE_CRON_PROMPT,
schedule_type="cron",
schedule_value=cron_expr,
agent="proactive",
context_mode="isolated",
metadata={
"notification_channel_id": notification_channel_id,
"hours_back": hours_back,
"timezone": timezone,
},
metadata=metadata,
)
def _cancel_proactive_duplicates(
scheduler: Any, tasks: List[Any], *, keep: Optional[Any] = None
) -> None:
"""Cancel managed proactive tasks other than *keep*."""
for task in tasks:
if keep is not None and task.id == keep.id:
continue
try:
scheduler.cancel_task(task.id)
except Exception:
logger.warning(
"Failed to cancel duplicate proactive task %s",
task.id,
exc_info=True,
)
+27 -5
View File
@@ -123,15 +123,35 @@ class AgentScheduler:
self._thread.start()
logger.info("Agent scheduler started")
def stop(self) -> None:
"""Stop the scheduler background thread."""
def request_stop(self) -> None:
"""Prevent new scheduled ticks without waiting for the worker."""
self._stop_event.set()
if self._bus:
self._bus.unsubscribe(EventType.AGENT_TICK_END, self._on_tick_event)
if self._thread is not None:
self._thread.join(timeout=10)
def wait_stopped(self, timeout: float = 10.0) -> bool:
"""Wait for an active tick to finish, retaining live thread state."""
thread = self._thread
if thread is None:
return True
if thread is threading.current_thread():
return False
thread.join(timeout=timeout)
if thread.is_alive():
logger.warning("Agent scheduler did not stop within %.1fs", timeout)
return False
if self._thread is thread:
self._thread = None
logger.info("Agent scheduler stopped")
return True
def stop(self, timeout: float = 10.0) -> None:
"""Stop dispatching and wait for the scheduler worker."""
self.request_stop()
if self.wait_stopped(timeout=timeout):
logger.info("Agent scheduler stopped")
def _loop(self) -> None:
"""Main scheduler loop."""
@@ -160,6 +180,8 @@ class AgentScheduler:
]
for agent_id, info in due:
if self._stop_event.is_set():
break
agent = self._manager.get_agent(agent_id)
if agent is None or agent["status"] in (
"paused",
+1
View File
@@ -13,6 +13,7 @@ class SimpleAgent(BaseAgent):
"""Single-turn agent: query -> model -> response. No tool calling."""
agent_id = "simple"
supports_managed_tool_fallback = True
def run(
self,
+502
View File
@@ -0,0 +1,502 @@
"""Canonical managed-agent tool resolution.
Managed agents can run through streaming HTTP, immediate/scheduled ticks, or
the persistent-agent CLI. Those paths must bind the same live tool instances:
agent-type grants first, then configured native tools, then MCP adapters.
"""
from __future__ import annotations
import importlib
import logging
import sys
import weakref
from dataclasses import dataclass, field, replace
from pathlib import Path
from typing import Any, Iterable, Mapping
logger = logging.getLogger(__name__)
BROWSER_SUB_TOOLS = (
"browser_navigate",
"browser_click",
"browser_type",
"browser_screenshot",
"browser_extract",
"browser_axtree",
)
_MEMORY_TOOLS = frozenset(
{"retrieval", "memory_store", "memory_search", "memory_index", "memory_retrieve"}
)
_CHANNEL_TOOLS = frozenset({"channel_send", "channel_list", "channel_status"})
class _SpecOverrideTool:
"""Delegate execution while exposing an agent-configured OpenAI schema."""
def __init__(self, wrapped: Any, advertised_spec: dict[str, Any]) -> None:
self._wrapped = wrapped
self._advertised_spec = advertised_spec
@property
def spec(self) -> Any:
base = self._wrapped.spec
function = self._advertised_spec.get("function", {})
return replace(
base,
name=function.get("name", base.name),
description=function.get("description", base.description),
parameters=function.get("parameters", base.parameters),
)
def execute(self, **params: Any) -> Any:
return self._wrapped.execute(**params)
def to_openai_function(self) -> dict[str, Any]:
return self._advertised_spec
def __getattr__(self, name: str) -> Any:
return getattr(self._wrapped, name)
def _tool_name(tool: Any) -> str:
try:
return str(tool.spec.name)
except Exception:
return ""
def _spec_name(spec: Mapping[str, Any]) -> str:
function = spec.get("function")
if not isinstance(function, Mapping):
return ""
name = function.get("name")
return str(name) if name else ""
def _openai_spec(tool: Any) -> dict[str, Any]:
to_openai_function = getattr(tool, "to_openai_function", None)
if callable(to_openai_function):
try:
advertised = to_openai_function()
except Exception:
logger.debug(
"Failed to build advertised schema for tool %r; falling back "
"to its ToolSpec",
_tool_name(tool),
exc_info=True,
)
else:
if isinstance(advertised, Mapping) and _spec_name(advertised):
return dict(advertised)
logger.debug(
"Tool %r returned an invalid advertised schema; falling back "
"to its ToolSpec",
_tool_name(tool),
)
spec = tool.spec
return {
"type": "function",
"function": {
"name": spec.name,
"description": spec.description,
"parameters": spec.parameters,
},
}
def _close_resources(resources: tuple[Any, ...]) -> None:
for resource in reversed(resources):
close = getattr(resource, "close", None)
if callable(close):
try:
close()
except Exception:
logger.debug("Failed to close resolved tool resource", exc_info=True)
@dataclass
class ResolvedAgentTools:
"""One resolved toolkit, with views for agent loops and raw streaming."""
instances: list[Any] = field(default_factory=list)
extra_specs: list[dict[str, Any]] = field(default_factory=list)
advertised_specs: list[dict[str, Any]] = field(default_factory=list)
mcp_clients: list[Any] = field(default_factory=list)
owned_resources: list[Any] = field(default_factory=list, repr=False)
_closed: bool = field(default=False, init=False, repr=False)
_finalizer: weakref.finalize = field(init=False, repr=False)
def __post_init__(self) -> None:
# This fallback covers exceptions anywhere after resolution, including
# before an executor/response installs its normal explicit cleanup.
self._finalizer = weakref.finalize(
self,
_close_resources,
tuple(self.owned_resources),
)
@property
def by_name(self) -> dict[str, Any]:
return {name: tool for tool in self.instances if (name := _tool_name(tool))}
@property
def openai_specs(self) -> list[dict[str, Any]]:
specs: list[dict[str, Any]] = []
seen: set[str] = set()
advertised = self.advertised_specs
if not advertised:
advertised = [*map(_openai_spec, self.instances), *self.extra_specs]
for spec in advertised:
name = _spec_name(spec)
if name and name in seen:
continue
specs.append(spec)
if name:
seen.add(name)
return specs
def close(self) -> None:
"""Close request-local resources without touching shared MCP clients."""
if self._closed:
return
self._closed = True
self._finalizer()
def __enter__(self) -> ResolvedAgentTools:
return self
def __exit__(self, *exc_info: object) -> None:
self.close()
def ensure_registries_populated() -> None:
"""Populate tool/channel registries, including after tests clear them."""
from openjarvis.core.registry import ChannelRegistry, ToolRegistry
try:
import openjarvis.channels # noqa: F401
except Exception:
pass
try:
import openjarvis.tools # noqa: F401
except Exception:
pass
browser_modules = ("openjarvis.tools.browser", "openjarvis.tools.browser_axtree")
for module_name in browser_modules:
try:
importlib.import_module(module_name)
except Exception:
pass
if not ChannelRegistry.keys():
for module_name in list(sys.modules):
if module_name.startswith(
"openjarvis.channels."
) and not module_name.endswith("_stubs"):
try:
importlib.reload(sys.modules[module_name])
except Exception:
pass
if not ToolRegistry.keys():
for module_name in list(sys.modules):
if (
module_name.startswith("openjarvis.tools.")
and not module_name.endswith("_stubs")
and not module_name.endswith("agent_tools")
):
try:
importlib.reload(sys.modules[module_name])
except Exception:
pass
if not any(ToolRegistry.contains(name) for name in BROWSER_SUB_TOOLS):
for module_name in browser_modules:
module = sys.modules.get(module_name)
if module is not None:
try:
importlib.reload(module)
except Exception:
pass
def instantiate_registered_tool(
tool_cls: Any,
name: str,
*,
engine: Any,
model: str,
memory_backend: Any = None,
channel_backend: Any = None,
) -> Any:
"""Instantiate a registry tool with its runtime dependencies."""
if name in _MEMORY_TOOLS:
if memory_backend is None:
logger.warning(
"Memory tool %r instantiated without a backend — calls will "
"return no results.",
name,
)
return tool_cls(backend=memory_backend)
if name in _CHANNEL_TOOLS:
if channel_backend is None:
logger.warning(
"Channel tool %r instantiated without a channel — calls will "
"fail with 'No channel backend configured'.",
name,
)
return tool_cls(channel=channel_backend)
if name == "llm":
return tool_cls(engine=engine, model=model)
return tool_cls()
def build_deep_research_tools(
engine: Any,
model: str,
knowledge_db_path: str | Path | None = None,
) -> list[Any]:
"""Construct the live knowledge tools granted to ``deep_research``."""
if not knowledge_db_path:
from openjarvis.core.config import DEFAULT_CONFIG_DIR
knowledge_db_path = DEFAULT_CONFIG_DIR / "knowledge.db"
path = Path(knowledge_db_path)
if not path.exists():
return []
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
from openjarvis.tools.scan_chunks import ScanChunksTool
from openjarvis.tools.think import ThinkTool
store = KnowledgeStore(str(path))
try:
retriever = TwoStageRetriever(store)
return [
KnowledgeSearchTool(retriever=retriever),
KnowledgeSQLTool(store=store),
ScanChunksTool(store=store, engine=engine, model=model),
ThinkTool(),
]
except Exception:
store.close()
raise
def _normalized_tool_config(tool_config: Any) -> list[Any]:
if not tool_config:
return []
if isinstance(tool_config, str):
return [part.strip() for part in tool_config.split(",") if part.strip()]
if isinstance(tool_config, Mapping):
return [dict(tool_config)]
try:
return list(tool_config)
except TypeError:
return []
def resolve_agent_tools(
agent_record: Mapping[str, Any],
*,
engine: Any,
model: str,
memory_backend: Any = None,
channel_backend: Any = None,
mcp_tools: Iterable[Any] = (),
mcp_clients: Iterable[Any] = (),
knowledge_db_path: str | Path | None = None,
) -> ResolvedAgentTools:
"""Resolve the effective live toolkit for a managed agent.
Resolution is stable and first-wins: agent-type grants take precedence
over configured registry tools, which take precedence over MCP adapters.
``config["mcp_tools"] = false`` excludes MCP adapters from this agent;
process-wide runtimes may still own connections used by other agents.
"""
ensure_registries_populated()
from openjarvis.core.registry import ChannelRegistry, ToolRegistry
config = agent_record.get("config") or {}
if not isinstance(config, Mapping):
config = {}
instances: list[Any] = []
extra_specs: list[dict[str, Any]] = []
advertised_specs: list[dict[str, Any]] = []
owned_resources: list[Any] = []
seen: set[str] = set()
def add_instance(
tool: Any,
*,
advertised_spec: dict[str, Any] | None = None,
) -> None:
name = _tool_name(tool)
if not name or name in seen:
return
instances.append(tool)
advertised_specs.append(advertised_spec or _openai_spec(tool))
seen.add(name)
use_mcp = config.get("mcp_tools", True) is not False
mcp_tool_list = list(mcp_tools) if use_mcp else []
mcp_by_name: dict[str, Any] = {}
for tool in mcp_tool_list:
name = _tool_name(tool)
if name and name not in mcp_by_name:
mcp_by_name[name] = tool
if agent_record.get("agent_type") == "deep_research":
granted_tools = build_deep_research_tools(
engine=engine,
model=model,
knowledge_db_path=knowledge_db_path,
)
owned_ids: set[int] = set()
for tool in granted_tools:
resource = getattr(tool, "_store", None)
if (
resource is not None
and callable(getattr(resource, "close", None))
and id(resource) not in owned_ids
):
owned_resources.append(resource)
owned_ids.add(id(resource))
add_instance(tool)
for entry in _normalized_tool_config(config.get("tools")):
if isinstance(entry, Mapping):
raw_spec = entry if isinstance(entry, dict) else dict(entry)
name = _spec_name(raw_spec)
if name and name in seen:
continue
backing_tool = None
if name and not ChannelRegistry.contains(name):
if ToolRegistry.contains(name):
try:
backing_tool = instantiate_registered_tool(
ToolRegistry.get(name),
name,
engine=engine,
model=model,
memory_backend=memory_backend,
channel_backend=channel_backend,
)
except Exception as exc:
logger.warning(
"Could not instantiate tool '%s' (%s) — "
"advertising its custom spec without execution",
name,
exc,
)
elif name in mcp_by_name:
backing_tool = mcp_by_name[name]
if backing_tool is not None:
add_instance(
_SpecOverrideTool(backing_tool, raw_spec),
advertised_spec=raw_spec,
)
else:
logger.warning(
"Custom tool spec '%s' has no registered or MCP execution "
"backend — dropping",
name or "<unnamed>",
)
continue
if not isinstance(entry, str):
continue
names = BROWSER_SUB_TOOLS if entry == "browser" else (entry,)
for name in names:
if name in seen:
continue
if ChannelRegistry.contains(name):
continue
if not ToolRegistry.contains(name):
logger.warning(
"Tool '%s' referenced in agent config but not in ToolRegistry",
name,
)
continue
try:
add_instance(
instantiate_registered_tool(
ToolRegistry.get(name),
name,
engine=engine,
model=model,
memory_backend=memory_backend,
channel_backend=channel_backend,
)
)
except Exception as exc:
logger.warning(
"Could not instantiate tool '%s' (%s) — dropping", name, exc
)
if use_mcp:
for tool in mcp_tool_list:
add_instance(tool)
return ResolvedAgentTools(
instances=instances,
extra_specs=extra_specs,
advertised_specs=advertised_specs,
mcp_clients=list(mcp_clients) if use_mcp else [],
owned_resources=owned_resources,
)
def resolve_tool_specs(tool_config: Any) -> list[dict[str, Any]]:
"""Compatibility view for callers that only need configured specs."""
specs: list[dict[str, Any]] = []
seen: set[str] = set()
for entry in _normalized_tool_config(tool_config):
if isinstance(entry, dict):
specs.append(entry)
name = _spec_name(entry)
if name:
seen.add(name)
continue
resolved = resolve_agent_tools(
{"config": {"tools": [entry]}},
engine=None,
model="",
)
for spec in resolved.openai_specs:
name = _spec_name(spec)
if name and name in seen:
continue
specs.append(spec)
if name:
seen.add(name)
return specs
__all__ = [
"BROWSER_SUB_TOOLS",
"ResolvedAgentTools",
"build_deep_research_tools",
"ensure_registries_populated",
"instantiate_registered_tool",
"resolve_agent_tools",
"resolve_tool_specs",
]
+129 -99
View File
@@ -2,45 +2,36 @@
from __future__ import annotations
import sys
import click
import openjarvis
from openjarvis.cli._bootstrap import bootstrap_cmd
from openjarvis.cli.add_cmd import add
from openjarvis.cli.agent_cmd import agent
from openjarvis.cli.ask import ask
from openjarvis.cli.bench_cmd import bench
from openjarvis.cli.channel_cmd import channel
from openjarvis.cli.channels_cmd import channels
from openjarvis.cli.chat_cmd import chat
from openjarvis.cli.compose_cmd import compose
from openjarvis.cli.config_cmd import config
from openjarvis.cli.connect_cmd import connect
from openjarvis.cli.daemon_cmd import restart, start, status, stop
from openjarvis.cli.digest_cmd import digest
from openjarvis.cli.doctor_cmd import doctor
from openjarvis.cli.eval_cmd import eval_group
from openjarvis.cli.feedback_cmd import feedback_group
from openjarvis.cli.gateway_cmd import gateway
from openjarvis.cli.host_cmd import host
from openjarvis.cli.init_cmd import init
from openjarvis.cli.memory_cmd import memory
from openjarvis.cli.mine_cmd import mine
from openjarvis.cli.model import model
from openjarvis.cli.operators_cmd import operators
from openjarvis.cli.optimize_cmd import optimize_group
from openjarvis.cli.pearl_cmd import pearl
from openjarvis.cli.quickstart_cmd import quickstart
from openjarvis.cli.registry_cmd import registry
from openjarvis.cli.scan_cmd import scan
from openjarvis.cli.scheduler_cmd import scheduler
from openjarvis.cli.self_update_cmd import self_update
from openjarvis.cli.serve import serve
from openjarvis.cli.skill_cmd import skill
from openjarvis.cli.telemetry_cmd import telemetry
from openjarvis.cli.tool_cmd import tool
from openjarvis.cli.vault_cmd import vault
from openjarvis.cli.workflow_cmd import workflow
def _invoked_command(argv: list[str]) -> str:
"""Return the first positional CLI token after global flags."""
for arg in argv:
if arg.startswith("-"):
continue
return arg
return ""
# A data-boundary scan must be able to diagnose an invalid OPENJARVIS_HOME.
# Importing the rest of the CLI eagerly would import core.config and resolve that
# path before the scan can turn the failure into a finding.
_DATA_BOUNDARY_BOOTSTRAP = (
_invoked_command(sys.argv[1:]) == "scan" and "--data-boundaries" in sys.argv[1:]
)
def _should_skip_update_check(ctx: click.Context, argv: list[str]) -> bool:
"""Return true for commands whose diagnostics should remain local-only."""
if "--research" in argv:
return True
return ctx.invoked_subcommand == "scan" and "--data-boundaries" in argv
@click.group(
@@ -63,11 +54,13 @@ def cli(ctx: click.Context, verbose: bool, quiet: bool) -> None:
# Check for updates on interactive commands. The banner is noise in
# demo recordings of ``jarvis ask --research``, so skip it whenever
# the research flag is in argv (cheap argv sniff — Click hasn't
# parsed the subcommand's args yet at this point).
# parsed the subcommand's args yet at this point). Also skip
# ``jarvis scan --data-boundaries`` because it is intended to be a
# local application-data diagnostic with no outbound calls.
import sys
research_mode_active = "--research" in sys.argv
if not quiet and ctx.invoked_subcommand and not research_mode_active:
skip_update_check = _should_skip_update_check(ctx, sys.argv[1:])
if not quiet and ctx.invoked_subcommand and not skip_update_check:
import threading
from openjarvis.cli._version_check import check_for_updates
@@ -91,74 +84,111 @@ def cli(ctx: click.Context, verbose: bool, quiet: bool) -> None:
check_and_route(ctx)
cli.add_command(init, "init")
cli.add_command(ask, "ask")
cli.add_command(chat, "chat")
cli.add_command(serve, "serve")
cli.add_command(model, "model")
cli.add_command(memory, "memory")
cli.add_command(mine, "mine")
cli.add_command(pearl, "pearl")
cli.add_command(telemetry, "telemetry")
cli.add_command(bench, "bench")
cli.add_command(channel, "channel")
cli.add_command(channels, "channels")
cli.add_command(scheduler, "scheduler")
cli.add_command(doctor, "doctor")
cli.add_command(agent, "agents")
cli.add_command(workflow, "workflow")
cli.add_command(skill, "skill")
cli.add_command(start, "start")
cli.add_command(stop, "stop")
cli.add_command(restart, "restart")
cli.add_command(status, "status")
cli.add_command(vault, "vault")
cli.add_command(add, "add")
cli.add_command(operators, "operators")
cli.add_command(eval_group, "eval")
cli.add_command(host, "host")
cli.add_command(quickstart, "quickstart")
cli.add_command(optimize_group, "optimize")
cli.add_command(feedback_group, "feedback")
cli.add_command(compose, "compose")
cli.add_command(gateway, "gateway")
cli.add_command(tool, "tool")
cli.add_command(registry, "registry")
cli.add_command(config, "config")
cli.add_command(scan, "scan")
cli.add_command(connect, "connect")
cli.add_command(digest, "digest")
# deep-research setup pulls the ingestion pipeline (embeddings/numpy). Guard it
# so a broken or slow numpy on Windows — which can raise at IMPORT time, not
# just ImportError (#404) — can never take down the whole CLI, including
# `jarvis serve`. Invoking `jarvis deep-research-setup` without the deps still
# errors clearly on demand.
try:
from openjarvis.cli.deep_research_setup_cmd import deep_research_setup
if not _DATA_BOUNDARY_BOOTSTRAP:
from openjarvis.cli._bootstrap import bootstrap_cmd
from openjarvis.cli.add_cmd import add
from openjarvis.cli.agent_cmd import agent
from openjarvis.cli.ask import ask
from openjarvis.cli.bench_cmd import bench
from openjarvis.cli.channel_cmd import channel
from openjarvis.cli.channels_cmd import channels
from openjarvis.cli.chat_cmd import chat
from openjarvis.cli.compose_cmd import compose
from openjarvis.cli.config_cmd import config
from openjarvis.cli.connect_cmd import connect
from openjarvis.cli.daemon_cmd import restart, start, status, stop
from openjarvis.cli.digest_cmd import digest
from openjarvis.cli.doctor_cmd import doctor
from openjarvis.cli.eval_cmd import eval_group
from openjarvis.cli.feedback_cmd import feedback_group
from openjarvis.cli.gateway_cmd import gateway
from openjarvis.cli.host_cmd import host
from openjarvis.cli.init_cmd import init
from openjarvis.cli.memory_cmd import memory
from openjarvis.cli.mine_cmd import mine
from openjarvis.cli.model import model
from openjarvis.cli.operators_cmd import operators
from openjarvis.cli.optimize_cmd import optimize_group
from openjarvis.cli.pearl_cmd import pearl
from openjarvis.cli.quickstart_cmd import quickstart
from openjarvis.cli.registry_cmd import registry
from openjarvis.cli.scheduler_cmd import scheduler
from openjarvis.cli.self_update_cmd import self_update
from openjarvis.cli.serve import serve
from openjarvis.cli.skill_cmd import skill
from openjarvis.cli.telemetry_cmd import telemetry
from openjarvis.cli.tool_cmd import tool
from openjarvis.cli.vault_cmd import vault
from openjarvis.cli.workflow_cmd import workflow
cli.add_command(deep_research_setup, "deep-research-setup")
cli.add_command(deep_research_setup, "research")
except Exception as _dr_exc:
import logging as _logging
cli.add_command(init, "init")
cli.add_command(ask, "ask")
cli.add_command(chat, "chat")
cli.add_command(serve, "serve")
cli.add_command(model, "model")
cli.add_command(memory, "memory")
cli.add_command(mine, "mine")
cli.add_command(pearl, "pearl")
cli.add_command(telemetry, "telemetry")
cli.add_command(bench, "bench")
cli.add_command(channel, "channel")
cli.add_command(channels, "channels")
cli.add_command(scheduler, "scheduler")
cli.add_command(doctor, "doctor")
cli.add_command(agent, "agents")
cli.add_command(workflow, "workflow")
cli.add_command(skill, "skill")
cli.add_command(start, "start")
cli.add_command(stop, "stop")
cli.add_command(restart, "restart")
cli.add_command(status, "status")
cli.add_command(vault, "vault")
cli.add_command(add, "add")
cli.add_command(operators, "operators")
cli.add_command(eval_group, "eval")
cli.add_command(host, "host")
cli.add_command(quickstart, "quickstart")
cli.add_command(optimize_group, "optimize")
cli.add_command(feedback_group, "feedback")
cli.add_command(compose, "compose")
cli.add_command(gateway, "gateway")
cli.add_command(tool, "tool")
cli.add_command(registry, "registry")
cli.add_command(config, "config")
cli.add_command(connect, "connect")
cli.add_command(digest, "digest")
_logging.getLogger(__name__).debug("deep-research command unavailable: %s", _dr_exc)
cli.add_command(self_update, "self-update")
cli.add_command(bootstrap_cmd, "_bootstrap")
# Deep Research setup pulls the ingestion pipeline (embeddings/numpy). Guard
# it so an import-time dependency failure cannot take down the whole CLI.
try:
from openjarvis.cli.deep_research_setup_cmd import deep_research_setup
# Gateway CLI commands (lazy import to avoid pulling starlette)
try:
from openjarvis.cli.auth_cmd import auth
cli.add_command(deep_research_setup, "deep-research-setup")
cli.add_command(deep_research_setup, "research")
except Exception as _dr_exc:
import logging as _logging
cli.add_command(auth, "auth")
except ImportError:
pass
_logging.getLogger(__name__).debug(
"deep-research command unavailable: %s", _dr_exc
)
cli.add_command(self_update, "self-update")
cli.add_command(bootstrap_cmd, "_bootstrap")
try:
from openjarvis.cli.tunnel_cmd import tunnel
# Gateway CLI commands (lazy import to avoid pulling starlette)
try:
from openjarvis.cli.auth_cmd import auth
cli.add_command(tunnel, "tunnel")
except ImportError:
pass
cli.add_command(auth, "auth")
except ImportError:
pass
try:
from openjarvis.cli.tunnel_cmd import tunnel
cli.add_command(tunnel, "tunnel")
except ImportError:
pass
def main() -> None:
+4 -2
View File
@@ -11,7 +11,9 @@ Three install paths are supported today:
- **Editable git checkout** (``uv sync`` / ``pip install -e .`` from a
cloned repo). The package's ``__file__`` is inside a working tree
with a ``.git`` directory at the repo root. Upgrade with
``git pull && uv sync`` from the checkout.
``git pull && uv sync --inexact`` from the checkout. ``--inexact`` is
important here: a bare ``uv sync`` removes packages installed by extras or
dependency groups that are not part of the base project.
We detect by inspecting ``openjarvis.__file__``. If we can't tell with
confidence we fall back to the PyPI command that's the most common
@@ -68,7 +70,7 @@ def detect_install() -> InstallInfo:
if (candidate / ".git").exists() and (candidate / "pyproject.toml").exists():
return InstallInfo(
kind="editable-git",
upgrade_command=f"cd {candidate} && git pull && uv sync",
upgrade_command=(f"cd {candidate} && git pull && uv sync --inexact"),
repo_root=candidate,
)
if candidate.parent == candidate:
+19 -5
View File
@@ -248,6 +248,17 @@ def _get_memory_backend(config):
return None
def _get_memory_facts(config):
"""Load facts captured by the automatic memory service."""
try:
from openjarvis.memory import load_configured_facts
return load_configured_facts(config)
except Exception as exc:
logger.debug("Automatic memory facts unavailable (optional): %s", exc)
return []
_MEMORY_TOOLS = frozenset(
{"retrieval", "memory_store", "memory_search", "memory_index", "memory_retrieve"}
)
@@ -387,9 +398,8 @@ def _run_agent(
# Wire the SystemPromptBuilder so SOUL.md / MEMORY.md / USER.md persona
# files actually reach the model. Only passed to agents whose __init__
# accepts a `prompt_builder` kwarg (BaseAgent does; agents that override
# __init__ without forwarding it, e.g. OrchestratorAgent, opt out
# automatically and keep their existing system-prompt machinery).
# explicitly accepts a `prompt_builder` kwarg. Agents with specialized
# prompt machinery opt in by naming and forwarding the parameter.
import inspect as _inspect
if "prompt_builder" in _inspect.signature(agent_cls.__init__).parameters:
@@ -416,7 +426,8 @@ def _run_agent(
from openjarvis.tools.storage.context import ContextConfig, inject_context
backend = _get_memory_backend(config)
if backend is not None:
facts = _get_memory_facts(config)
if backend is not None or facts:
ctx_cfg = ContextConfig(
top_k=config.memory.context_top_k,
min_score=config.memory.context_min_score,
@@ -427,6 +438,7 @@ def _run_agent(
[],
backend,
config=ctx_cfg,
facts=facts,
)
for msg in context_messages:
ctx.conversation.add(msg)
@@ -963,7 +975,8 @@ def ask(
)
backend = _get_memory_backend(config)
if backend is not None:
facts = _get_memory_facts(config)
if backend is not None or facts:
ctx_cfg = ContextConfig(
top_k=config.memory.context_top_k,
min_score=config.memory.context_min_score,
@@ -974,6 +987,7 @@ def ask(
messages,
backend,
config=ctx_cfg,
facts=facts,
)
except Exception as exc:
logger.debug("Failed to inject memory context: %s", exc)
+57 -3
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import logging
import sys
from typing import List, Optional
@@ -15,6 +16,8 @@ from openjarvis.core.events import EventBus
from openjarvis.core.types import Message, Role
from openjarvis.memory import publish_completed_exchange
logger = logging.getLogger(__name__)
def _read_input(prompt: str = "You> ") -> Optional[str]:
"""Read user input with graceful EOF handling."""
@@ -194,6 +197,15 @@ def chat(
console.print(f"[yellow]Memory service unavailable: {exc}[/yellow]")
memory_service = None
# The document backend and automatic fact store are separate persistence
# mechanisms. Context injection combines both at read time so facts from
# previous sessions are immediately available without a manual index step.
memory_backend = None
if config.agent.context_from_memory:
from openjarvis.cli.ask import _get_memory_backend
memory_backend = _get_memory_backend(config)
# Conversation state
if not system_prompt:
from openjarvis.prompt.builder import SystemPromptBuilder
@@ -262,15 +274,57 @@ def chat(
# Add user message
history.append(Message(role=Role.USER, content=user_input))
# Generate response
generation_history = history
agent_context_message = None
if config.agent.context_from_memory:
try:
from openjarvis.memory import load_configured_facts
from openjarvis.tools.storage.context import (
ContextConfig,
inject_context,
)
if memory_service is not None and hasattr(memory_service, "list_facts"):
facts = memory_service.list_facts()
else:
facts = load_configured_facts(config)
ctx_cfg = ContextConfig(
top_k=config.memory.context_top_k,
min_score=config.memory.context_min_score,
max_context_tokens=config.memory.context_max_tokens,
)
context_messages = inject_context(
user_input,
[] if agent is not None else history,
memory_backend,
config=ctx_cfg,
facts=facts,
)
if agent is not None:
if context_messages:
agent_context_message = context_messages[0]
else:
generation_history = context_messages
except Exception:
logger.debug("Failed to inject memory context", exc_info=True)
# Generate response even when optional memory context is unavailable.
try:
if agent is not None:
response = agent.run(user_input)
from openjarvis.agents._stubs import AgentContext
agent_context = AgentContext()
if agent_context_message is not None:
agent_context.conversation.add(agent_context_message)
for msg in history[:-1]:
if msg.role != Role.SYSTEM:
agent_context.conversation.add(msg)
response = agent.run(user_input, context=agent_context)
content = (
response.content if hasattr(response, "content") else str(response)
)
else:
result = engine.generate(history, model=model)
result = engine.generate(generation_history, model=model)
content = (
result.get("content", "")
if isinstance(result, dict)
+4 -4
View File
@@ -158,7 +158,7 @@ def _show_toml_config(console: Console, config_path: Path) -> None:
console.print(f"[dim]Loading config from: {config_path}[/dim]")
if config_path.exists():
config_content = config_path.read_text()
config_content = config_path.read_text(encoding="utf-8")
syntax = Syntax(config_content, "toml", theme="monokai", line_numbers=True)
console.print(Panel(syntax, title="Config File", border_style="cyan"))
else:
@@ -170,7 +170,7 @@ def _show_json_config(console: Console, config_path: Path) -> None:
console.print(f"[dim]Loading config from: {config_path}[/dim]")
if config_path.exists():
config_content = config_path.read_text()
config_content = config_path.read_text(encoding="utf-8")
try:
import tomllib # Python 3.11+
@@ -375,7 +375,7 @@ def set_config(key: str, value: str) -> None:
os.environ.get("OPENJARVIS_CONFIG", DEFAULT_CONFIG_DIR / "config.toml")
)
if config_path.exists():
doc = tomlkit.parse(config_path.read_text())
doc = tomlkit.parse(config_path.read_text(encoding="utf-8"))
else:
doc = tomlkit.document()
config_path.parent.mkdir(parents=True, exist_ok=True)
@@ -390,7 +390,7 @@ def set_config(key: str, value: str) -> None:
current[parts[-1]] = typed_value
# Write back
config_path.write_text(tomlkit.dumps(doc))
config_path.write_text(tomlkit.dumps(doc), encoding="utf-8")
console.print(f"[green]Set[/green] {key} = {value!r}")
+70 -11
View File
@@ -17,18 +17,64 @@ _PID_FILE = DEFAULT_CONFIG_DIR / "server.pid"
_LOG_FILE = DEFAULT_CONFIG_DIR / "server.log"
def _pid_alive(pid: int) -> bool:
"""Return whether *pid* identifies a running process without signaling it."""
if pid <= 0:
return False
if os.name == "nt":
import ctypes
from ctypes import wintypes
error_invalid_parameter = 87
synchronize = 0x00100000
wait_object_0 = 0x00000000
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
kernel32.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD]
kernel32.OpenProcess.restype = wintypes.HANDLE
kernel32.WaitForSingleObject.argtypes = [wintypes.HANDLE, wintypes.DWORD]
kernel32.WaitForSingleObject.restype = wintypes.DWORD
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
kernel32.CloseHandle.restype = wintypes.BOOL
handle = kernel32.OpenProcess(synchronize, False, pid)
if not handle:
# OpenProcess reports ERROR_INVALID_PARAMETER when the PID does not
# exist. For access-denied and other inconclusive failures, retain
# the PID file rather than declaring a potentially live daemon dead.
return ctypes.get_last_error() != error_invalid_parameter
try:
wait_result = kernel32.WaitForSingleObject(handle, 0)
# WAIT_OBJECT_0 proves the process exited. WAIT_TIMEOUT proves it
# is live; unexpected failures are inconclusive, so retain the PID.
return wait_result != wait_object_0
finally:
kernel32.CloseHandle(handle)
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def _read_pid() -> int | None:
"""Read PID from pid file, return None if not found or stale."""
if not _PID_FILE.exists():
return None
try:
pid = int(_PID_FILE.read_text().strip())
# Check if process is still running
os.kill(pid, 0)
return pid
except (ValueError, OSError):
except (OSError, ValueError):
_PID_FILE.unlink(missing_ok=True)
return None
if not _pid_alive(pid):
_PID_FILE.unlink(missing_ok=True)
return None
return pid
def _write_pid(pid: int) -> None:
@@ -81,14 +127,28 @@ def start(
if agent_name:
cmd.extend(["--agent", agent_name])
# Start as background process
# Start as background process, fully detached from the launching terminal.
#
# ``start_new_session`` is POSIX-only: CPython's Windows ``_execute_child``
# names the parameter ``unused_start_new_session`` and ignores it. Relying
# on it there leaves the server sharing its parent's console, so closing
# that console — or logging off — delivers CTRL_CLOSE_EVENT and kills the
# daemon. DETACHED_PROCESS gives it no console at all; the new process
# group additionally stops a Ctrl-C in the parent reaching it.
DEFAULT_CONFIG_DIR.mkdir(parents=True, exist_ok=True)
log_fh = open(_LOG_FILE, "a") # noqa: SIM115
spawn_kwargs: dict = {}
if sys.platform == "win32":
spawn_kwargs["creationflags"] = (
subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP
)
else:
spawn_kwargs["start_new_session"] = True
proc = subprocess.Popen(
cmd,
stdout=log_fh,
stderr=log_fh,
start_new_session=True,
**spawn_kwargs,
)
_write_pid(proc.pid)
@@ -113,14 +173,13 @@ def stop() -> None:
# Wait up to 10 seconds for graceful shutdown
for _ in range(20):
time.sleep(0.5)
try:
os.kill(pid, 0)
except OSError:
if not _pid_alive(pid):
break
else:
# Force kill if still running
# SIGKILL is POSIX-only. On Windows SIGTERM already maps to
# TerminateProcess, so repeating it is the available escalation.
try:
os.kill(pid, signal.SIGKILL)
os.kill(pid, getattr(signal, "SIGKILL", signal.SIGTERM))
except OSError:
pass
except OSError:
+3 -1
View File
@@ -344,7 +344,9 @@ def init(
console.print(f" Looked in: {examples_dir}")
raise SystemExit(1)
DEFAULT_CONFIG_DIR.mkdir(parents=True, exist_ok=True)
DEFAULT_CONFIG_PATH.write_text(preset_path.read_text())
DEFAULT_CONFIG_PATH.write_text(
preset_path.read_text(encoding="utf-8"), encoding="utf-8"
)
console.print(
f"[green]Preset '{preset}' installed to {DEFAULT_CONFIG_PATH}[/green]"
)
+33 -9
View File
@@ -89,15 +89,39 @@ def index(
mem = _get_backend(backend)
try:
for chunk in track(chunks, description="Storing chunks...", console=console):
mem.store(
chunk.content,
source=chunk.source,
metadata={
"offset": chunk.offset,
"index": chunk.index,
},
)
replace_source = getattr(mem, "replace_source", None)
if callable(replace_source):
documents_by_source = {}
for chunk in chunks:
documents_by_source.setdefault(chunk.source, []).append(
(
chunk.content,
{
"offset": chunk.offset,
"index": chunk.index,
},
)
)
for source, documents in track(
documents_by_source.items(),
description="Replacing sources...",
console=console,
):
replace_source(source, documents)
else:
for chunk in track(
chunks,
description="Storing chunks...",
console=console,
):
mem.store(
chunk.content,
source=chunk.source,
metadata={
"offset": chunk.offset,
"index": chunk.index,
},
)
finally:
if hasattr(mem, "close"):
mem.close()
+151 -5
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import json
import os
import subprocess
import sys
from dataclasses import dataclass
@@ -11,7 +12,11 @@ from typing import Callable, List
import click
from openjarvis.core.paths import get_config_dir
from openjarvis.core.paths import get_config_dir, get_config_path
from openjarvis.security.data_boundary_audit import (
DataBoundaryReport,
build_data_boundary_report,
)
# Engine ports that should only be listening on localhost.
_ENGINE_PORTS = {11434, 8080, 8000, 30000, 1234, 52415, 18181}
@@ -441,10 +446,45 @@ _RICH_ICONS = {
"ok": "[green]\u2713[/green]",
"warn": "[yellow]![/yellow]",
"fail": "[red]\u2717[/red]",
"info": "[blue]i[/blue]",
"skip": "[dim]-[/dim]",
}
def _resolve_data_boundary_config_path() -> Path:
env_config = os.environ.get("OPENJARVIS_CONFIG")
if env_config:
return Path(env_config).expanduser().resolve()
return get_config_path()
def _load_data_boundary_config():
"""Load config without treating missing config as active."""
root = None
root_error = ""
try:
root = get_config_dir()
config_path = _resolve_data_boundary_config_path()
except Exception as exc:
config_path = None
root_error = f"{type(exc).__name__}: {exc}"
if root_error:
return None, root, False, "", root_error
try:
from openjarvis.core.config import JarvisConfig, load_config
except Exception as exc:
return None, root, False, f"{type(exc).__name__}: {exc}", ""
if config_path is None or not config_path.exists():
return JarvisConfig(), root, False, "", root_error
try:
return load_config(config_path), root, True, "", root_error
except Exception as exc:
return JarvisConfig(), root, False, f"{type(exc).__name__}: {exc}", root_error
def _render_results(results: List[ScanResult]) -> None:
"""Render scan results as a Rich table."""
from rich.console import Console
@@ -494,17 +534,123 @@ def _render_results(results: List[ScanResult]) -> None:
console.print()
def _render_data_boundary_report(
report: DataBoundaryReport,
*,
show_paths: bool,
) -> None:
"""Render application data-boundary findings as a Rich table."""
from rich.console import Console
from rich.table import Table
console = Console()
console.print()
console.print("[bold]OpenJarvis Data-Boundary Scan[/bold]")
console.print(f"Verdict: [bold]{report.verdict}[/bold]")
console.print()
table = Table(show_header=True, header_style="bold", show_lines=True)
table.add_column("", width=3, justify="center")
table.add_column("Finding")
table.add_column("Recommendation")
for finding in report.findings:
icon = _RICH_ICONS.get(finding.status, "?")
style = {"fail": "red", "warn": "yellow", "info": "blue"}.get(
finding.status,
"white",
)
details = [f"[{style}]{finding.title}[/{style}]"]
details.append(f"[dim]{finding.potential_data_path}[/dim]")
if finding.location:
location = finding.absolute_location if show_paths else finding.location
details.append(f"[dim]Location: {location}[/dim]")
table.add_row(icon, "\n".join(details), finding.recommendation)
console.print(table)
summary = report.summary()
console.print()
console.print(
f" [red]{summary['fail']} fail[/red], "
f"[yellow]{summary['warn']} warning(s)[/yellow], "
f"[blue]{summary['info']} info[/blue]"
)
if not show_paths:
console.print(
" [dim]Absolute paths and connector basenames are redacted by default. "
"Use --show-paths for local debugging.[/dim]"
)
console.print()
def _emit_data_boundary_json(
report: DataBoundaryReport,
*,
show_paths: bool,
) -> None:
click.echo(json.dumps(report.to_dict(show_paths=show_paths), indent=2))
@click.command()
@click.option("--quick", is_flag=True, default=False, help="Run only critical checks.")
@click.option("--json", "as_json", is_flag=True, default=False, help="Output as JSON.")
def scan(quick: bool, as_json: bool) -> None:
@click.option(
"--data-boundaries",
is_flag=True,
default=False,
help="Run application data-boundary checks instead of host checks.",
)
@click.option(
"--strict",
is_flag=True,
default=False,
help="Exit non-zero if data-boundary fail or warn findings are present.",
)
@click.option(
"--show-paths",
is_flag=True,
default=False,
help="Show absolute paths in data-boundary output.",
)
def scan(
quick: bool,
as_json: bool,
data_boundaries: bool,
strict: bool,
show_paths: bool,
) -> None:
"""Audit your environment for privacy and security risks."""
if data_boundaries:
if quick:
raise click.UsageError("--quick cannot be combined with --data-boundaries.")
config, root, config_loaded, config_error, root_error = (
_load_data_boundary_config()
)
report = build_data_boundary_report(
config,
root,
config_loaded=config_loaded,
config_error=config_error,
root_error=root_error,
)
if as_json:
_emit_data_boundary_json(report, show_paths=show_paths)
else:
_render_data_boundary_report(report, show_paths=show_paths)
summary = report.summary()
if strict and (summary["fail"] or summary["warn"]):
raise click.exceptions.Exit(1)
return
if strict or show_paths:
raise click.UsageError(
"--strict and --show-paths are only supported with --data-boundaries."
)
scanner = PrivacyScanner()
results: List[ScanResult] = scanner.run_quick() if quick else scanner.run_all()
if as_json:
import json as json_mod
output = [
{
"name": r.name,
@@ -514,7 +660,7 @@ def scan(quick: bool, as_json: bool) -> None:
}
for r in results
]
click.echo(json_mod.dumps(output, indent=2))
click.echo(json.dumps(output, indent=2))
return
if not results:
+3 -1
View File
@@ -4,7 +4,9 @@ Runs the right upgrade command for how the user installed OpenJarvis:
- PyPI installs get ``pip install --upgrade openjarvis``.
- uv-tool installs get ``uv tool upgrade openjarvis``.
- Editable git checkouts get ``git pull && uv sync`` in the checkout.
- Editable git checkouts get ``git pull && uv sync --inexact`` in the checkout.
The inexact sync preserves packages previously installed through extras and
dependency groups.
The detection logic is shared with the post-command "new version
available" hint in ``_version_check.py`` so both surfaces stay in sync.
+102 -80
View File
@@ -10,6 +10,7 @@ from rich.console import Console
from openjarvis.cli._banner import print_banner
from openjarvis.core.config import load_config
from openjarvis.core.credentials import inject_credentials
from openjarvis.core.events import EventBus
from openjarvis.core.paths import get_config_dir
from openjarvis.engine import (
@@ -24,6 +25,30 @@ from openjarvis.intelligence import (
logger = logging.getLogger(__name__)
_DEFAULT_TOOLS = frozenset({"think", "calculator", "web_search"})
def _resolve_allowed_tools(config: object) -> tuple[set[str], bool]:
"""Return configured tool names and whether the selection was explicit.
``tools.enabled`` is the canonical setting used by ``SystemBuilder`` and
the interactive CLI. ``agent.tools`` remains as a backward-compatible
fallback, followed by the server's default tool set when neither is set.
"""
configured = config.tools.enabled or config.agent.tools
if not configured:
return set(_DEFAULT_TOOLS), False
if isinstance(configured, list):
allowed = {
tool.strip()
for tool in configured
if isinstance(tool, str) and tool.strip()
}
else:
allowed = {tool.strip() for tool in configured.split(",") if tool.strip()}
return allowed, True
def _unique_model_ids(model_ids: list[str]) -> list[str]:
"""Return model ids in first-seen order without duplicates."""
@@ -95,7 +120,7 @@ def _resolve_server_model(
"--agent",
"agent_name",
default=None,
help="Agent for non-streaming requests (simple, orchestrator, react, openhands).",
help="Agent for chat requests (simple, orchestrator, react, openhands).",
)
@click.pass_context
def serve(
@@ -122,6 +147,11 @@ def serve(
)
sys.exit(1)
# Tool credentials saved through the browser UI live in the OpenJarvis
# credential store. Restore them before engines and tools are constructed
# so availability checks and tool instances see the same environment.
inject_credentials()
config = load_config()
# Resolve host/port from CLI args or config
@@ -273,6 +303,15 @@ def serve(
# (which would re-discover the engine, re-resolve tools, re-open the channel,
# etc.). See the scheduler block near the bottom of this function (#263).
resolved_tools: list = []
managed_mcp_tools: list = []
mcp_clients: list = []
try:
from openjarvis.mcp.loader import load_mcp_tools_from_config
managed_mcp_tools, mcp_clients = load_mcp_tools_from_config(config.tools.mcp)
except Exception as exc:
logger.warning("Managed-agent MCP tools failed to load: %s", exc)
if agent_key:
try:
import openjarvis.agents # noqa: F401
@@ -284,32 +323,13 @@ def serve(
if sec.capability_policy is not None:
agent_kwargs["capability_policy"] = sec.capability_policy
# MCP transports persisted on the agent at the bottom of
# this block — initialise here so the reference is valid
# even when accepts_tools is False (#461).
mcp_clients: list = []
# Load tools for agents that support them
if getattr(agent_cls, "accepts_tools", False):
import openjarvis.tools # noqa: F401 # trigger registration
from openjarvis.core.registry import ToolRegistry
from openjarvis.tools._stubs import BaseTool
_DEFAULT_TOOLS = {"think", "calculator", "web_search"}
configured = config.agent.tools
if configured:
if isinstance(configured, list):
allowed = {
t.strip()
for t in configured
if isinstance(t, str) and t.strip()
}
else:
allowed = {
t.strip() for t in configured.split(",") if t.strip()
}
else:
allowed = _DEFAULT_TOOLS
allowed, tools_configured = _resolve_allowed_tools(config)
tools = []
for name in ToolRegistry.keys():
@@ -325,12 +345,13 @@ def serve(
# MCP server tools from config.tools.mcp.servers
# (#461 — these were silently dropped).
from openjarvis.mcp.loader import load_mcp_tools_from_config
mcp_tools, mcp_clients = load_mcp_tools_from_config(
config.tools.mcp,
allowed_names=allowed if configured else None,
)
mcp_tools = managed_mcp_tools
if tools_configured:
mcp_tools = [
tool
for tool in managed_mcp_tools
if tool.spec.name in allowed
]
if mcp_tools:
existing = {t.spec.name for t in tools}
for t in mcp_tools:
@@ -346,6 +367,27 @@ def serve(
if getattr(agent_cls, "accepts_tools", False):
agent_kwargs["max_turns"] = config.agent.max_turns
# Wire the SystemPromptBuilder so SOUL.md / MEMORY.md / USER.md
# reach the model on the SERVE path too. ``ask.py`` has done
# this since the persona system landed; ``serve.py`` never did,
# so an agent served over HTTP silently answered as a generic
# assistant while the same agent via the CLI kept its persona.
# Guarded so agents with specialized prompt machinery must opt
# in by explicitly naming and forwarding the kwarg.
import inspect as _inspect
if (
"prompt_builder"
in _inspect.signature(agent_cls.__init__).parameters
):
from openjarvis.prompt.builder import SystemPromptBuilder
agent_kwargs["prompt_builder"] = SystemPromptBuilder(
agent_template=config.agent.default_system_prompt or "",
memory_files_config=config.memory_files,
system_prompt_config=config.system_prompt,
)
agent = agent_cls(engine, model_name, **agent_kwargs)
# Pin MCP transports to the agent's lifetime so HTTP
# connections don't close mid-request (#461).
@@ -383,10 +425,6 @@ def serve(
channel_agent = config.channel.default_agent or agent_key or "simple"
_channel_tools: list = []
# MCP transports persisted at function scope (= server-process
# lifetime); see the comment near the channel-MCP-load block
# below. Initialise here so it's always bound. #461.
_channel_mcp_clients: list = []
if channel_agent:
try:
import openjarvis.agents
@@ -399,23 +437,7 @@ def serve(
from openjarvis.core.registry import ToolRegistry
from openjarvis.tools._stubs import BaseTool
_DEFAULT_TOOLS = {"think", "calculator", "web_search"}
configured = config.agent.tools
if configured:
if isinstance(configured, list):
_allowed = {
t.strip()
for t in configured
if isinstance(t, str) and t.strip()
}
else:
_allowed = {
t.strip()
for t in configured.split(",")
if t.strip()
}
else:
_allowed = _DEFAULT_TOOLS
_allowed, _tools_configured = _resolve_allowed_tools(config)
for _tname in ToolRegistry.keys():
if _tname not in _allowed:
@@ -426,29 +448,23 @@ def serve(
elif isinstance(_tcls, BaseTool):
_channel_tools.append(_tcls)
# MCP tools for the channel agent too (#461).
from openjarvis.mcp.loader import (
load_mcp_tools_from_config,
)
_ch_mcp_tools, _ch_mcp_clients = load_mcp_tools_from_config(
config.tools.mcp,
allowed_names=_allowed if configured else None,
)
# Reuse the process-owned MCP pool so channels do not
# open a second transport to every configured server.
_ch_mcp_tools = managed_mcp_tools
if _tools_configured:
_ch_mcp_tools = [
tool
for tool in managed_mcp_tools
if tool.spec.name in _allowed
]
if _ch_mcp_tools:
_existing = {t.spec.name for t in _channel_tools}
for t in _ch_mcp_tools:
if t.spec.name not in _existing:
_channel_tools.append(t)
_existing.add(t.spec.name)
# Hold a reference at module / function scope —
# the channel agent is constructed inside
# JarvisSystem below; we extend its lifetime by
# keeping the list bound here.
_channel_mcp_clients = _ch_mcp_clients
except Exception as exc:
logger.warning("Channel tools failed to load: %s", exc)
_channel_mcp_clients = []
_wire_system = JarvisSystem(
config=config,
@@ -458,6 +474,8 @@ def serve(
model=model_name,
agent_name=channel_agent,
tools=_channel_tools,
mcp_tools=managed_mcp_tools,
_mcp_clients=mcp_clients,
)
_wire_system.wire_channel(channel_bridge)
@@ -475,23 +493,24 @@ def serve(
# Create app
from openjarvis.server.app import create_app
# Set up memory backend for context injection. Built before the scheduler
# block so the executor's JarvisSystem can reference it (#263).
# Set up the memory backend for storage tools, API routes, and optional
# prompt-context injection. ``context_from_memory`` controls only the last
# of those, so disabling it must not leave explicit memory_* tools with a
# null backend. Built before the scheduler so AgentExecutor can reuse it.
memory_backend = None
if config.agent.context_from_memory:
try:
import openjarvis.tools.storage # noqa: F401
from openjarvis.core.registry import MemoryRegistry
try:
import openjarvis.tools.storage # noqa: F401
from openjarvis.core.registry import MemoryRegistry
mem_key = config.memory.default_backend
if MemoryRegistry.contains(mem_key):
memory_backend = MemoryRegistry.create(
mem_key,
db_path=config.memory.db_path,
)
console.print(" Memory: [cyan]active[/cyan]")
except Exception as exc:
logger.debug("Memory backend init failed: %s", exc)
mem_key = config.memory.default_backend
if MemoryRegistry.contains(mem_key):
memory_backend = MemoryRegistry.create(
mem_key,
db_path=config.memory.db_path,
)
console.print(" Memory: [cyan]active[/cyan]")
except Exception as exc:
logger.debug("Memory backend init failed: %s", exc)
# Automatic long-term memory service (background fact extraction).
memory_service = None
@@ -586,6 +605,7 @@ def serve(
agent=agent,
agent_name=agent_key or "",
tools=resolved_tools,
mcp_tools=managed_mcp_tools,
tool_executor=_sched_tool_executor,
memory_backend=memory_backend,
telemetry_store=telem_store,
@@ -594,6 +614,7 @@ def serve(
capability_policy=sec.capability_policy,
agent_manager=agent_manager,
agent_executor=executor,
_mcp_clients=mcp_clients,
)
executor.set_system(system)
@@ -685,10 +706,13 @@ def serve(
channel_bridge=channel_bridge,
config=config,
memory_backend=memory_backend,
own_memory_backend=memory_backend is not None,
memory_service=memory_service,
speech_backend=speech_backend,
agent_manager=agent_manager,
agent_scheduler=agent_scheduler,
mcp_tools=managed_mcp_tools,
mcp_clients=mcp_clients,
api_key=api_key,
webhook_config=webhook_config,
cors_origins=config.server.cors_origins,
@@ -717,6 +741,4 @@ def serve(
"authenticated requests to your instance."
)
import uvicorn
uvicorn.run(app, host=bind_host, port=bind_port, log_level="info")
+42 -20
View File
@@ -9,8 +9,9 @@ System Settings → Privacy & Security → Full Disk Access.
Timestamp notes
---------------
The Notes database stores modification timestamps as seconds since the Apple
epoch of 2001-01-01 00:00:00 UTC. Conversion formula::
Modern Notes schemas store note modification timestamps in
``ZMODIFICATIONDATE1``; older schemas use ``ZMODIFICATIONDATE``. Both are
seconds since the Apple epoch of 2001-01-01 00:00:00 UTC. Conversion formula::
dt = datetime(2001, 1, 1, tzinfo=utc) + timedelta(seconds=ZMODIFICATIONDATE)
@@ -171,25 +172,46 @@ class AppleNotesConnector(BaseConnector):
return
try:
try:
rows = conn.execute(
"SELECT n.ZIDENTIFIER, "
" COALESCE(n.ZTITLE1, n.ZTITLE, '') AS title, "
" n.ZMODIFICATIONDATE, d.ZDATA "
"FROM ZICCLOUDSYNCINGOBJECT n "
"JOIN ZICNOTEDATA d ON d.ZNOTE = n.Z_PK "
"ORDER BY n.ZMODIFICATIONDATE ASC"
).fetchall()
except sqlite3.OperationalError:
# Older macOS schemas may lack ZTITLE1
rows = conn.execute(
"SELECT n.ZIDENTIFIER, "
" COALESCE(n.ZTITLE, '') AS title, "
" n.ZMODIFICATIONDATE, d.ZDATA "
"FROM ZICCLOUDSYNCINGOBJECT n "
"JOIN ZICNOTEDATA d ON d.ZNOTE = n.Z_PK "
"ORDER BY n.ZMODIFICATIONDATE ASC"
object_columns = {
row[1]
for row in conn.execute(
"PRAGMA table_info(ZICCLOUDSYNCINGOBJECT)"
).fetchall()
}
title_columns = [
f"n.{column}"
for column in ("ZTITLE1", "ZTITLE")
if column in object_columns
]
title_expr = (
f"COALESCE({', '.join(title_columns)}, '')" if title_columns else "''"
)
# Modern Apple Notes stores a note's modification timestamp in
# ZMODIFICATIONDATE1. ZMODIFICATIONDATE is still present in some
# schemas, but applies to other cloud-sync object types and can be
# NULL for notes. Treating that NULL as zero makes incremental
# syncs incorrectly discard newly-created notes as 2001-era data.
modification_columns = [
f"n.{column}"
for column in ("ZMODIFICATIONDATE1", "ZMODIFICATIONDATE")
if column in object_columns
]
modification_expr = (
f"COALESCE({', '.join(modification_columns)}, 0)"
if modification_columns
else "0"
)
rows = conn.execute(
"SELECT n.ZIDENTIFIER, "
f" {title_expr} AS title, "
f" {modification_expr} AS modification_date, d.ZDATA "
"FROM ZICCLOUDSYNCINGOBJECT n "
"JOIN ZICNOTEDATA d ON d.ZNOTE = n.Z_PK "
"ORDER BY modification_date ASC"
).fetchall()
self._items_total = len(rows)
synced = 0
+45 -9
View File
@@ -12,9 +12,18 @@ import os
import platform
import shutil
import subprocess
from dataclasses import dataclass, field
from dataclasses import dataclass, field, is_dataclass
from pathlib import Path
from typing import TYPE_CHECKING, Any, Dict, List, Optional
from typing import (
TYPE_CHECKING,
Any,
Dict,
List,
Optional,
get_args,
get_origin,
get_type_hints,
)
from openjarvis.core.paths import (
ConfigurationError,
@@ -1710,10 +1719,16 @@ def _apply_toml_section(target: Any, section: Dict[str, Any]) -> None:
"""Overlay TOML key/value pairs onto a dataclass instance.
Recursively handles nested dicts when the target attribute is itself
a dataclass. Normalises TOML arrays to comma-separated strings both
for dataclass fields annotated as ``str`` and for backward-compat
property setters that expect string input.
a dataclass, including dict entries in lists of dataclasses. Normalises
TOML arrays to comma-separated strings both for dataclass fields annotated
as ``str`` and for backward-compat property setters that expect string input.
"""
try:
type_hints = get_type_hints(type(target))
except (NameError, TypeError):
# Some config types contain optional runtime-only forward references.
type_hints = {}
for key, value in section.items():
if hasattr(target, key):
if isinstance(value, dict):
@@ -1728,14 +1743,35 @@ def _apply_toml_section(target: Any, section: Dict[str, Any]) -> None:
# property setters (e.g. reward_weights, default_tools).
if isinstance(value, list):
is_str_field = False
item_dataclass = None
if hasattr(target, "__dataclass_fields__"):
field_obj = target.__dataclass_fields__.get(key)
if field_obj is not None and field_obj.type in ("str", str):
is_str_field = True
elif field_obj is None:
if field_obj is not None:
field_type = type_hints.get(key, field_obj.type)
type_args = get_args(field_type)
if (
get_origin(field_type) is list
and len(type_args) == 1
and is_dataclass(type_args[0])
):
item_dataclass = type_args[0]
elif field_obj.type in ("str", str):
is_str_field = True
else:
# Property, not a real field — normalise to string
is_str_field = True
if is_str_field:
if item_dataclass is not None:
converted = []
for item in value:
if isinstance(item, dict):
nested = item_dataclass()
_apply_toml_section(nested, item)
converted.append(nested)
else:
converted.append(item)
value = converted
elif is_str_field:
value = ",".join(str(v) for v in value)
setattr(target, key, value)
+41 -13
View File
@@ -67,6 +67,24 @@ def load_credentials(path: Path | None = None) -> dict[str, dict[str, str]]:
return tomllib.load(f)
def _validate_credential_key(tool_name: str, key: str) -> None:
allowed = TOOL_CREDENTIALS.get(tool_name, [])
if key not in allowed:
raise ValueError(f"Unknown credential key '{key}' for tool '{tool_name}'")
def _write_credentials(creds: dict[str, dict[str, str]], path: Path) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
lines: list[str] = []
for section, kvs in creds.items():
lines.append(f"[{section}]")
for k, v in kvs.items():
lines.append(f'{k} = "{v}"')
lines.append("")
path.write_text("\n".join(lines))
os.chmod(path, 0o600)
def save_credential(
tool_name: str,
key: str,
@@ -75,9 +93,7 @@ def save_credential(
path: Path | None = None,
) -> None:
"""Save a single credential key, validate, write file, and set os.environ."""
allowed = TOOL_CREDENTIALS.get(tool_name, [])
if key not in allowed:
raise ValueError(f"Unknown credential key '{key}' for tool '{tool_name}'")
_validate_credential_key(tool_name, key)
stripped = value.strip()
if not stripped:
raise ValueError("Credential value must not be empty")
@@ -88,20 +104,32 @@ def save_credential(
if tool_name not in creds:
creds[tool_name] = {}
creds[tool_name][key] = stripped
p.parent.mkdir(parents=True, exist_ok=True)
lines: list[str] = []
for section, kvs in creds.items():
lines.append(f"[{section}]")
for k, v in kvs.items():
lines.append(f'{k} = "{v}"')
lines.append("")
p.write_text("\n".join(lines))
os.chmod(p, 0o600)
_write_credentials(creds, p)
os.environ[key] = stripped
def delete_credential(
tool_name: str,
key: str,
*,
path: Path | None = None,
) -> None:
"""Delete a persisted credential and remove it from the running process."""
_validate_credential_key(tool_name, key)
p = Path(path) if path else _default_path()
with _LOCK:
creds = load_credentials(path=p)
tool_creds = creds.get(tool_name)
if tool_creds is not None:
tool_creds.pop(key, None)
if not tool_creds:
creds.pop(tool_name, None)
_write_credentials(creds, p)
os.environ.pop(key, None)
def get_credential_status(tool_name: str) -> dict[str, bool]:
"""Return {KEY: bool} for each required key indicating if set in env."""
keys = TOOL_CREDENTIALS.get(tool_name, [])
+6
View File
@@ -35,6 +35,12 @@ def _make_engine(key: str, config: JarvisConfig) -> InferenceEngine:
"""Instantiate a registered engine with the appropriate config host."""
cls = EngineRegistry.get(key)
# LiteLLM cannot enumerate every model supported by every provider. Its
# list_models() contract therefore advertises the configured default
# model, which must be supplied when discovery constructs the engine.
if key == "litellm":
return cls(default_model=config.intelligence.default_model or None)
# gemma_cpp: pass config fields instead of host
if key == "gemma_cpp":
cfg = config.engine.gemma_cpp
+156 -1
View File
@@ -9,6 +9,7 @@ import json
import logging
import os
import time
import uuid
from collections.abc import AsyncIterator, Sequence
from typing import Any, Dict, List, Tuple
@@ -1305,6 +1306,160 @@ class CloudEngine(InferenceEngine):
if chunk.text:
yield chunk.text
async def _stream_full_google(
self,
messages: Sequence[Message],
*,
model: str,
temperature: float,
max_tokens: int,
**kwargs: Any,
) -> AsyncIterator[StreamChunk]:
"""Stream Google text and function-call parts as full chunks."""
if self._google_client is None:
raise EngineConnectionError("Google client not available")
system_text = ""
contents: List[Dict[str, Any]] = []
for message in messages:
if message.role.value == "system":
system_text = message.content
elif message.role.value == "tool":
function_response = {
"function_response": {
"name": message.name or "unknown",
"response": {"result": message.content},
}
}
if (
contents
and contents[-1]["role"] == "user"
and contents[-1]["parts"]
and "function_response" in contents[-1]["parts"][-1]
):
contents[-1]["parts"].append(function_response)
else:
contents.append({"role": "user", "parts": [function_response]})
elif message.role.value == "assistant" and message.tool_calls:
parts: List[Dict[str, Any]] = []
if message.content:
parts.append({"text": message.content})
for tool_call in message.tool_calls:
args = tool_call.arguments
if isinstance(args, str):
try:
args = json.loads(args)
except (json.JSONDecodeError, TypeError):
args = {"input": args}
function_call_part: Dict[str, Any] = {
"function_call": {
"name": tool_call.name,
"args": args if isinstance(args, dict) else {},
}
}
signature = self._thought_sigs.get(tool_call.id)
if signature is not None:
function_call_part["thought_signature"] = signature
parts.append(function_call_part)
contents.append({"role": "model", "parts": parts})
elif message.role.value == "assistant":
contents.append({"role": "model", "parts": [{"text": message.content}]})
else:
contents.append({"role": "user", "parts": [{"text": message.content}]})
from google.genai import types as genai_types
config = genai_types.GenerateContentConfig(
temperature=temperature,
max_output_tokens=max_tokens,
)
if system_text:
config.system_instruction = system_text
tools = kwargs.pop("tools", None)
if tools:
config.tools = [{"function_declarations": _convert_tools_to_google(tools)}]
tool_call_count = 0
stream_id = uuid.uuid4().hex
final_usage: Dict[str, Any] | None = None
for chunk in self._google_client.models.generate_content_stream(
model=model,
contents=contents,
config=config,
):
usage_metadata = getattr(chunk, "usage_metadata", None)
if usage_metadata is not None:
prompt_tokens = getattr(usage_metadata, "prompt_token_count", 0) or 0
completion_tokens = (
getattr(usage_metadata, "candidates_token_count", 0) or 0
)
final_usage = {
"prompt_tokens": prompt_tokens,
"completion_tokens": completion_tokens,
"total_tokens": prompt_tokens + completion_tokens,
}
candidates = getattr(chunk, "candidates", None)
parts = []
if candidates:
parts = getattr(candidates[0].content, "parts", []) or []
if parts:
text_found = False
calls: List[Dict[str, Any]] = []
for part in parts:
text = getattr(part, "text", None)
if text:
text_found = True
yield StreamChunk(content=text)
function_call = getattr(part, "function_call", None)
if function_call:
name = getattr(function_call, "name", "")
raw_args = getattr(function_call, "args", {})
args = dict(raw_args) if hasattr(raw_args, "items") else {}
# Gemini emits complete function-call parts, so each part is
# a distinct invocation. The same function may legitimately
# be called more than once in a parallel response.
tool_index = tool_call_count
# The engine is shared across server requests, and saved
# thought signatures are keyed by tool-call ID. Include a
# per-stream nonce so concurrent conversations cannot
# overwrite each other's signatures.
tool_id = f"google_{stream_id}_{tool_index}"
tool_call_count += 1
tool_call = {
"index": tool_index,
"id": tool_id,
"type": "function",
"function": {
"name": name,
"arguments": json.dumps(args),
},
}
calls.append(tool_call)
signature = getattr(part, "thought_signature", None)
if signature is not None:
tool_call["thought_signature"] = signature
self._thought_sigs[tool_id] = signature
if calls:
yield StreamChunk(tool_calls=calls)
if text_found:
continue
try:
text = chunk.text
except (AttributeError, ValueError):
text = None
if text:
yield StreamChunk(content=text)
yield StreamChunk(
finish_reason="tool_calls" if tool_call_count else "stop",
usage=final_usage,
)
async def _stream_openrouter(
self,
messages: Sequence[Message],
@@ -1600,7 +1755,7 @@ class CloudEngine(InferenceEngine):
async for chunk in self._stream_full_anthropic(messages, **kw):
yield chunk
elif _is_google_model(model):
async for chunk in super().stream_full(messages, **kw):
async for chunk in self._stream_full_google(messages, **kw):
yield chunk
else:
async for chunk in self._stream_full_openai(messages, **kw):
+13 -2
View File
@@ -26,16 +26,19 @@ class MultiEngine(InferenceEngine):
def __init__(self, engines: list[tuple[str, InferenceEngine]]) -> None:
self._engines = engines
self._model_map: Dict[str, InferenceEngine] = {}
self._model_key_map: Dict[str, str] = {}
self._refresh_map()
def _refresh_map(self) -> None:
self._model_map.clear()
for _key, engine in self._engines:
self._model_key_map.clear()
for key, engine in self._engines:
try:
for model_id in engine.list_models():
self._model_map[model_id] = engine
self._model_key_map[model_id] = key
except Exception as exc:
logger.debug("Failed to list models for %s: %s", _key, exc)
logger.debug("Failed to list models for %s: %s", key, exc)
_CLOUD_PREFIXES = ("gpt-", "o1-", "o3-", "o4-", "claude-", "gemini-", "openrouter/")
@@ -117,6 +120,14 @@ class MultiEngine(InferenceEngine):
self._refresh_map()
return list(self._model_map.keys())
def engine_key_for(self, model: str) -> str | None:
"""Return the registry key of the engine advertising *model*."""
key = self._model_key_map.get(model)
if key is not None:
return key
self._refresh_map()
return self._model_key_map.get(model)
def health(self) -> bool:
return any(engine.health() for _key, engine in self._engines)
+39 -38
View File
@@ -8,13 +8,12 @@ Reference: https://github.com/sierra-research/tau2-bench
from __future__ import annotations
import json
import logging
import os
import subprocess
import sys
from importlib import metadata
from typing import Iterable, List, Optional
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.splits import apply_split
from openjarvis.evals.core.types import EvalRecord
@@ -22,48 +21,50 @@ from openjarvis.evals.core.types import EvalRecord
LOGGER = logging.getLogger(__name__)
TAU2_REPO = "https://github.com/sierra-research/tau2-bench.git"
CACHE_DIR = get_cache_dir() / "tau2-bench"
# v1.0.1. Keep the full commit SHA here (rather than a movable tag) so every
# TauBench setup uses the same third-party code.
TAU2_REVISION = "fc0055dc4e0a316c3f83133267fbd6faaa770992"
TAU2_INSTALL_SPEC = f"tau2 @ git+{TAU2_REPO}@{TAU2_REVISION}"
DOMAINS = ("airline", "retail", "telecom")
def _ensure_tau2() -> None:
"""Ensure tau2 package is importable; install from cache if needed."""
"""Ensure the explicitly installed, pinned tau2 package is importable."""
try:
distribution = metadata.distribution("tau2")
except metadata.PackageNotFoundError as exc:
raise ImportError(
"TauBench requires tau2, which OpenJarvis does not install at "
"runtime. Install the pinned dependency explicitly (Python >=3.12): "
f'uv pip install "{TAU2_INSTALL_SPEC}"'
) from exc
try:
direct_url_text = distribution.read_text("direct_url.json")
direct_url = json.loads(direct_url_text or "")
vcs_info = direct_url.get("vcs_info", {})
installed_repo = direct_url.get("url")
installed_revision = vcs_info.get("commit_id")
except (json.JSONDecodeError, AttributeError):
installed_repo = None
installed_revision = None
if installed_repo != TAU2_REPO or installed_revision != TAU2_REVISION:
raise ImportError(
"The installed tau2 package does not match OpenJarvis's pinned "
"source revision. Reinstall it explicitly (Python >=3.12): "
f'uv pip install --force-reinstall "{TAU2_INSTALL_SPEC}"'
)
try:
import tau2 # noqa: F401
except ImportError:
# Clone and install from source
if not CACHE_DIR.exists():
LOGGER.info("Cloning tau2-bench from %s ...", TAU2_REPO)
CACHE_DIR.parent.mkdir(parents=True, exist_ok=True)
subprocess.run(
["git", "clone", "--depth", "1", TAU2_REPO, str(CACHE_DIR)],
check=True,
capture_output=True,
)
LOGGER.info("Installing tau2-bench ...")
# Try `python -m pip` first; fall back to `uv pip` for uv-managed venvs
# which don't ship pip by default.
try:
subprocess.run(
[sys.executable, "-m", "pip", "install", "-e", str(CACHE_DIR)],
check=True,
capture_output=True,
)
except (subprocess.CalledProcessError, FileNotFoundError):
subprocess.run(
[
"uv",
"pip",
"install",
"--python",
sys.executable,
"-e",
str(CACHE_DIR),
],
check=True,
capture_output=True,
)
except ImportError as exc:
raise ImportError(
"The pinned tau2 package is installed but cannot be imported. "
"Reinstall it explicitly (Python >=3.12): "
f'uv pip install --force-reinstall "{TAU2_INSTALL_SPEC}"'
) from exc
class TauBenchDataset(DatasetProvider):
+13 -7
View File
@@ -93,11 +93,16 @@ class HeuristicRouter(RouterPolicy):
Rules (applied in order):
1. Code detected prefer model with "code"/"coder" in name
2. Math detected prefer larger model
3. Low complexity (score < 0.20) prefer smaller/faster model
2. Low complexity (score <= 0.20) prefer smaller/faster model
3. Math detected prefer larger model
4. High complexity (score >= 0.55 OR reasoning keywords) prefer larger model
5. High urgency (>0.8) override to smaller model
6. Default fallback default_model fallback_model first available
Low complexity is checked before the math check so that simple arithmetic
("calculate 2+2") routes to the smallest model instead of always escalating
on the "math" keyword; math problems above the low-complexity threshold
still escalate to the larger model.
"""
def __init__(
@@ -134,14 +139,15 @@ class HeuristicRouter(RouterPolicy):
# Fall through to larger model for code
return _largest_model(available) or available[0]
# Rule 2: Math detected → prefer larger model
# Rule 2: Low complexity → prefer smaller model (checked before the math
# rule so simple arithmetic doesn't escalate to the largest model)
if context.complexity_score <= 0.20:
return _smallest_model(available) or available[0]
# Rule 3: Math detected → prefer larger model
if context.has_math:
return _largest_model(available) or available[0]
# Rule 3: Low complexity → prefer smaller model
if context.complexity_score < 0.20:
return _smallest_model(available) or available[0]
# Rule 4: High complexity or reasoning → prefer larger model
if context.complexity_score >= 0.55 or context.has_reasoning:
return _largest_model(available) or available[0]
+36 -8
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import itertools
import threading
from typing import Any, Dict, List
from openjarvis.mcp.protocol import MCPError, MCPRequest, MCPResponse
@@ -24,18 +25,31 @@ class MCPClient:
self._initialized = False
self._capabilities: Dict[str, Any] = {}
self._id_counter = itertools.count(1)
# A client may be shared by server, scheduled, and channel agents.
# Keep each transport request/response exchange atomic so stdio
# readers cannot consume another thread's JSON-RPC response.
self._request_lock = threading.RLock()
# Closing must not wait for ``_request_lock``: transport.close() is
# what interrupts a request that is blocked in a transport read.
# An event lets queued requests fail before touching that transport,
# while this separate lock keeps close itself idempotent.
self._closed = threading.Event()
self._transport_closed = threading.Event()
self._close_lock = threading.Lock()
def _next_id(self) -> int:
return next(self._id_counter)
def _send(self, method: str, params: Dict[str, Any] | None = None) -> MCPResponse:
"""Send a request and check for errors."""
request = MCPRequest(
method=method,
params=params or {},
id=self._next_id(),
)
response = self._transport.send(request)
with self._request_lock:
self._raise_if_closed()
request = MCPRequest(
method=method,
params=params or {},
id=self._next_id(),
)
response = self._transport.send(request)
if response.error is not None:
raise MCPError(
code=response.error.get("code", -1),
@@ -44,6 +58,10 @@ class MCPClient:
)
return response
def _raise_if_closed(self) -> None:
if self._closed.is_set():
raise RuntimeError("MCP client is closed")
def initialize(self) -> Dict[str, Any]:
"""Perform the MCP initialize handshake.
@@ -75,7 +93,9 @@ class MCPClient:
params=params or {},
id=None, # None → no id field in JSON (notification)
)
self._transport.send_notification(request)
with self._request_lock:
self._raise_if_closed()
self._transport.send_notification(request)
def list_tools(self) -> List[ToolSpec]:
"""Discover available tools from the server.
@@ -114,7 +134,15 @@ class MCPClient:
def close(self) -> None:
"""Close the transport connection."""
self._transport.close()
# Do not acquire _request_lock here. A transport request can be stuck
# waiting for a server response, and closing the underlying transport
# is the mechanism that unblocks it.
with self._close_lock:
if self._transport_closed.is_set():
return
self._closed.set()
self._transport.close()
self._transport_closed.set()
def __enter__(self) -> MCPClient:
return self
+2
View File
@@ -19,6 +19,7 @@ from openjarvis.memory.store import (
FactStore,
LocalFactStore,
create_fact_store,
load_configured_facts,
)
__all__ = [
@@ -29,5 +30,6 @@ __all__ = [
"MemoryService",
"build_memory_service",
"create_fact_store",
"load_configured_facts",
"publish_completed_exchange",
]
+28 -2
View File
@@ -16,7 +16,7 @@ import time
from abc import ABC, abstractmethod
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Iterable, List
from typing import Any, Iterable, List
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import FactStoreRegistry
@@ -205,4 +205,30 @@ def create_fact_store(
return FactStoreRegistry.create(key, path, max_facts=max_facts)
__all__ = ["Fact", "FactStore", "LocalFactStore", "create_fact_store"]
def load_configured_facts(config: Any) -> List[Fact]:
"""Load automatic-memory facts from *config* when the service is enabled.
Context injection is also used by short-lived commands such as
``jarvis ask``, where no :class:`MemoryService` instance exists. This
helper gives those callers the same configured fact-store view without
coupling them to the service lifecycle.
"""
memory = getattr(config, "memory", None)
if memory is None or not getattr(memory, "enabled", False):
return []
store = create_fact_store(
getattr(memory, "backend", "local"),
path=getattr(memory, "facts_path", None),
max_facts=getattr(memory, "max_facts", 1000),
)
return store.list()
__all__ = [
"Fact",
"FactStore",
"LocalFactStore",
"create_fact_store",
"load_configured_facts",
]
+28 -5
View File
@@ -516,20 +516,35 @@ class Jarvis:
existing = agent_kwargs.get("tools", [])
agent_kwargs["tools"] = digest_tools + list(existing)
# Wire the SystemPromptBuilder so SOUL.md / MEMORY.md / USER.md reach
# the model — mirrors ``cli/ask.py`` and ``cli/serve.py``. Guarded so
# agents whose ``__init__`` doesn't accept the kwarg opt out.
import inspect as _inspect
if "prompt_builder" in _inspect.signature(agent_cls.__init__).parameters:
from openjarvis.prompt.builder import SystemPromptBuilder
agent_kwargs["prompt_builder"] = SystemPromptBuilder(
agent_template=self._config.agent.default_system_prompt or "",
memory_files_config=self._config.memory_files,
system_prompt_config=self._config.system_prompt,
)
agent_obj = agent_cls(self._engine, model_name, **agent_kwargs)
ctx = AgentContext()
# Context injection
if context and self._config.agent.context_from_memory:
try:
from openjarvis.cli.ask import _get_memory_backend
from openjarvis.cli.ask import _get_memory_backend, _get_memory_facts
from openjarvis.tools.storage.context import (
ContextConfig,
inject_context,
)
backend = _get_memory_backend(self._config)
if backend is not None:
facts = _get_memory_facts(self._config)
if backend is not None or facts:
ctx_cfg = ContextConfig(
top_k=self._config.memory.context_top_k,
min_score=self._config.memory.context_min_score,
@@ -540,6 +555,7 @@ class Jarvis:
[],
backend,
config=ctx_cfg,
facts=facts,
)
for msg in context_messages:
ctx.conversation.add(msg)
@@ -570,17 +586,24 @@ class Jarvis:
) -> List[Message]:
"""Inject memory context into messages."""
try:
from openjarvis.cli.ask import _get_memory_backend
from openjarvis.cli.ask import _get_memory_backend, _get_memory_facts
from openjarvis.tools.storage.context import ContextConfig, inject_context
backend = _get_memory_backend(self._config)
if backend is not None:
facts = _get_memory_facts(self._config)
if backend is not None or facts:
ctx_cfg = ContextConfig(
top_k=self._config.memory.context_top_k,
min_score=self._config.memory.context_min_score,
max_context_tokens=self._config.memory.context_max_tokens,
)
return inject_context(query, messages, backend, config=ctx_cfg)
return inject_context(
query,
messages,
backend,
config=ctx_cfg,
facts=facts,
)
except Exception as exc:
logger.warning("Failed to inject memory context: %s", exc)
return messages
+50 -18
View File
@@ -4,27 +4,10 @@ from __future__ import annotations
import logging
from dataclasses import dataclass
from importlib import import_module
from typing import Any, Optional
from openjarvis.core.events import EventBus
from openjarvis.security._stubs import BaseScanner
from openjarvis.security.audit import AuditLogger
from openjarvis.security.file_policy import (
DEFAULT_SENSITIVE_PATTERNS,
filter_sensitive_paths,
is_sensitive_file,
)
from openjarvis.security.guardrails import GuardrailsEngine, SecurityBlockError
from openjarvis.security.scanner import PIIScanner, SecretScanner
from openjarvis.security.ssrf import check_ssrf, is_private_ip
from openjarvis.security.types import (
RedactionMode,
ScanFinding,
ScanResult,
SecurityEvent,
SecurityEventType,
ThreatLevel,
)
logger = logging.getLogger(__name__)
@@ -50,6 +33,12 @@ def setup_security(
if not config.security.enabled:
return SecurityContext(engine=engine)
from openjarvis.security._stubs import BaseScanner
from openjarvis.security.audit import AuditLogger
from openjarvis.security.guardrails import GuardrailsEngine
from openjarvis.security.scanner import PIIScanner, SecretScanner
from openjarvis.security.types import RedactionMode
# Scanners + engine wrapping
try:
scanners: list[BaseScanner] = []
@@ -121,3 +110,46 @@ __all__ = [
"is_sensitive_file",
"setup_security",
]
_LAZY_EXPORTS = {
"AuditLogger": ("openjarvis.security.audit", "AuditLogger"),
"BaseScanner": ("openjarvis.security._stubs", "BaseScanner"),
"DEFAULT_SENSITIVE_PATTERNS": (
"openjarvis.security.file_policy",
"DEFAULT_SENSITIVE_PATTERNS",
),
"GuardrailsEngine": ("openjarvis.security.guardrails", "GuardrailsEngine"),
"PIIScanner": ("openjarvis.security.scanner", "PIIScanner"),
"RedactionMode": ("openjarvis.security.types", "RedactionMode"),
"ScanFinding": ("openjarvis.security.types", "ScanFinding"),
"ScanResult": ("openjarvis.security.types", "ScanResult"),
"SecretScanner": ("openjarvis.security.scanner", "SecretScanner"),
"SecurityBlockError": (
"openjarvis.security.guardrails",
"SecurityBlockError",
),
"SecurityEvent": ("openjarvis.security.types", "SecurityEvent"),
"SecurityEventType": ("openjarvis.security.types", "SecurityEventType"),
"ThreatLevel": ("openjarvis.security.types", "ThreatLevel"),
"check_ssrf": ("openjarvis.security.ssrf", "check_ssrf"),
"filter_sensitive_paths": (
"openjarvis.security.file_policy",
"filter_sensitive_paths",
),
"is_private_ip": ("openjarvis.security.ssrf", "is_private_ip"),
"is_sensitive_file": ("openjarvis.security.file_policy", "is_sensitive_file"),
}
def __getattr__(name: str) -> Any:
target = _LAZY_EXPORTS.get(name)
if target is None:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
module_name, attribute = target
value = getattr(import_module(module_name), attribute)
globals()[name] = value
return value
def __dir__() -> list[str]:
return sorted(set(globals()) | set(_LAZY_EXPORTS))
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+6 -2
View File
@@ -1087,12 +1087,16 @@ def include_all_routes(app) -> None:
except ImportError:
pass
# WebSocket bridge for real-time agent events
# WebSocket bridge for real-time agent events. Must subscribe on the
# same EventBus instance channels/agents actually publish to
# (app.state.bus, set in server/app.py) — the get_event_bus() global
# singleton is a *different* bus that nothing in `jarvis serve` ever
# publishes to, so events silently never reached this endpoint.
try:
from openjarvis.core.events import get_event_bus
from openjarvis.server.ws_bridge import create_ws_router
ws_router = create_ws_router(get_event_bus())
ws_router = create_ws_router(getattr(app.state, "bus", None) or get_event_bus())
app.include_router(ws_router)
except Exception:
logger.debug("WebSocket bridge not available", exc_info=True)
+119
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import logging
import pathlib
import threading
import time
from fastapi import FastAPI
@@ -21,6 +22,8 @@ from openjarvis.server.routes import router
from openjarvis.server.upload_router import router as upload_router
logger = logging.getLogger(__name__)
_MANAGED_SHUTDOWN_GRACE_SECONDS = 0.25
_MANAGED_SHUTDOWN_DRAIN_SECONDS = 10.0
def _restore_sendblue_bindings(app: FastAPI) -> None:
@@ -151,10 +154,13 @@ def create_app(
channel_bridge=None,
config=None,
memory_backend=None,
own_memory_backend: bool = False,
memory_service=None,
speech_backend=None,
agent_manager=None,
agent_scheduler=None,
mcp_tools=None,
mcp_clients=None,
api_key: str = "",
webhook_config: dict | None = None,
cors_origins: list[str] | None = None,
@@ -221,16 +227,129 @@ def create_app(
)
app.state.channel_bridge = channel_bridge
app.state.config = config
app.state._memory_backend_lock = threading.Lock()
app.state.memory_backend = memory_backend
app.state._owns_memory_backend = bool(own_memory_backend)
app.state.memory_service = memory_service
app.state.speech_backend = speech_backend
app.state.agent_manager = agent_manager
app.state.agent_scheduler = agent_scheduler
app.state.mcp_tools = list(mcp_tools or [])
app.state._mcp_discovery_lock = threading.Lock()
app.state._mcp_clients_lock = threading.Lock()
app.state._mcp_clients = list(mcp_clients or [])
app.state._managed_worker_lock = threading.Lock()
app.state._managed_workers: set[threading.Thread] = set()
app.state._managed_runtime_stopping = False
app.state.session_start = time.time()
# Exposed so WebSocket handlers can authenticate the handshake (the HTTP
# AuthMiddleware never sees WS upgrade requests). Empty = auth disabled.
app.state.api_key = api_key
@app.on_event("shutdown")
async def _shutdown_managed_runtime() -> None:
# Quiesce every producer before touching the shared MCP pool. Route
# workers are registered under this lock, so none can slip in after
# the snapshot. The scheduler has a two-phase stop because closing an
# MCP transport may be what releases an in-flight tick.
with app.state._managed_worker_lock:
app.state._managed_runtime_stopping = True
managed_workers = list(app.state._managed_workers)
# Stop external listener threads before draining ticks or closing the
# shared MCP pool. Channel callbacks are wired to that same pool by
# ``serve`` and otherwise could race teardown or survive app restart.
channel_bridge = getattr(app.state, "channel_bridge", None)
disconnect_channels = getattr(channel_bridge, "disconnect", None)
if callable(disconnect_channels):
try:
disconnect_channels()
except Exception:
logger.debug("Channel bridge shutdown failed", exc_info=True)
def _join_workers(timeout: float) -> None:
deadline = time.monotonic() + timeout
for thread in managed_workers:
remaining = deadline - time.monotonic()
if remaining <= 0:
break
thread.join(timeout=remaining)
scheduler = getattr(app.state, "agent_scheduler", None)
scheduler_wait = None
scheduler_drained = True
if scheduler is not None:
try:
request_stop = getattr(scheduler, "request_stop", None)
wait_stopped = getattr(scheduler, "wait_stopped", None)
if callable(request_stop) and callable(wait_stopped):
request_stop()
scheduler_wait = wait_stopped
scheduler_drained = bool(
wait_stopped(timeout=_MANAGED_SHUTDOWN_GRACE_SECONDS)
)
else:
scheduler.stop()
scheduler_drained = not bool(
getattr(scheduler, "is_running", False)
)
except Exception:
scheduler_drained = False
logger.debug("Agent scheduler shutdown failed", exc_info=True)
# Give normal work a brief chance to finish before cancellation.
_join_workers(timeout=_MANAGED_SHUTDOWN_GRACE_SECONDS)
with app.state._mcp_clients_lock:
mcp_clients_to_close = list(app.state._mcp_clients)
for client in mcp_clients_to_close:
try:
client.close()
except Exception:
logger.debug("MCP client shutdown failed", exc_info=True)
# Transport closure interrupts blocked MCP reads. Drain the workers a
# second time so shutdown does not return while they still own runtime
# state. Any stragglers can no longer issue transport requests because
# MCPClient marks itself closed before closing its transport.
if scheduler_wait is not None:
try:
scheduler_drained = bool(
scheduler_wait(timeout=_MANAGED_SHUTDOWN_DRAIN_SECONDS)
)
except Exception:
scheduler_drained = False
logger.debug("Agent scheduler drain failed", exc_info=True)
_join_workers(timeout=_MANAGED_SHUTDOWN_DRAIN_SECONDS)
alive = [thread.name for thread in managed_workers if thread.is_alive()]
if alive:
logger.warning("Managed workers did not stop during shutdown: %s", alive)
# A backend created by ``serve`` or lazily by a managed route belongs
# to this app process. Close it only after every tracked consumer has
# been drained; injected/borrowed backends remain the caller's concern.
owned_memory_backend = None
runtime_drained = scheduler_drained and not alive
if runtime_drained:
with app.state._memory_backend_lock:
if app.state._owns_memory_backend:
owned_memory_backend = app.state.memory_backend
app.state.memory_backend = None
app.state._owns_memory_backend = False
else:
# A live worker may itself hold _memory_backend_lock while opening
# the backend. Respect the bounded shutdown deadline: do not wait
# on that lock or mutate ownership until every consumer is gone.
logger.warning(
"Skipping memory backend cleanup because managed runtime "
"consumers did not stop"
)
close_memory = getattr(owned_memory_backend, "close", None)
if callable(close_memory):
try:
close_memory()
except Exception:
logger.debug("Memory backend shutdown failed", exc_info=True)
# Wire up trace store if traces are enabled.
#
# We deliberately do NOT subscribe the trace store to the bus. The chat
+12 -1
View File
@@ -84,6 +84,17 @@ def is_cloud_model(model: str) -> bool:
return get_provider(model) is not None
def _openrouter_model_id(model: str) -> str:
"""Return the provider-facing ID for an OpenRouter model."""
prefix = "openrouter/"
candidate = model.removeprefix(prefix)
# OpenRouter owns IDs such as "openrouter/auto" itself. Only remove the
# LiteLLM routing prefix when the remainder is still a provider/model ID.
if model.startswith(prefix) and "/" in candidate:
return candidate
return model
# ---------------------------------------------------------------------------
# Message conversion
# ---------------------------------------------------------------------------
@@ -371,7 +382,7 @@ async def stream_cloud(
"OPENROUTER_API_KEY not set — add it in the Cloud Models tab"
)
async for token in _stream_openai(
model,
_openrouter_model_id(model),
messages,
temperature,
max_tokens,
@@ -0,0 +1,34 @@
"""Model capability helpers shared by server model-selection routes."""
_EMBEDDING_MODEL_PREFIXES = (
"all-minilm",
"bge-",
"bge_",
"e5-",
"e5_",
"gte-",
"gte_",
"jina-embeddings",
"nomic-bert",
"sentence-transformers",
)
def is_embed_only_model(model_name: str) -> bool:
"""Return whether a model identifier denotes a non-chat embedder.
Ollama does not expose capabilities through its model-list response, so
model selection needs a conservative name-based guard. Most embedding
models contain ``embed``; the explicit prefixes cover common families
such as MiniLM, BGE, E5, and GTE whose names do not.
"""
name = (model_name or "").strip().lower()
leaf = name.rsplit("/", 1)[-1].split(":", 1)[0]
return (
"embed" in leaf
or "minilm" in leaf
or leaf.startswith(_EMBEDDING_MODEL_PREFIXES)
)
__all__ = ["is_embed_only_model"]
+245 -45
View File
@@ -11,7 +11,8 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from openjarvis.core.paths import get_config_dir
from openjarvis.core.types import Message, Role
from openjarvis.core.types import Message, Role, ToolCall
from openjarvis.server.model_capabilities import is_embed_only_model
from openjarvis.server.models import (
ChatCompletionChunk,
ChatCompletionRequest,
@@ -39,6 +40,15 @@ def _to_messages(chat_messages) -> list[Message]:
role=role,
content=m.content or "",
name=m.name,
tool_calls=[
ToolCall(
id=tool_call.get("id", ""),
name=tool_call.get("function", {}).get("name", ""),
arguments=tool_call.get("function", {}).get("arguments", "{}"),
)
for tool_call in (m.tool_calls or [])
]
or None,
tool_call_id=m.tool_call_id,
)
)
@@ -56,8 +66,10 @@ def _ensure_identity_prompt(messages: list[Message], app_config) -> list[Message
``SystemPromptBuilder`` / ``BaseAgent``; the engine-direct server paths
did not. This mirrors the agent fallback in ``agents/_stubs.py``.
If any message already carries a system role, the caller has supplied
their own grounding and we leave the list untouched (no double-prompting).
If any caller-supplied message already carries a system role, the caller
has supplied their own grounding and we leave the list untouched (no
double-prompting). Internally tagged memory context does not count as
caller grounding.
Resolution of the identity text: the config comes from ``app.state`` when
wired, otherwise ``load_config()``; the prompt itself is assembled by
@@ -68,7 +80,11 @@ def _ensure_identity_prompt(messages: list[Message], app_config) -> list[Message
injection" rather than crashing the endpoint, but the failure is logged
(per REVIEW.md never silently swallow).
"""
if any(m.role == Role.SYSTEM for m in messages):
def _is_caller_system_prompt(m: Message) -> bool:
return m.role == Role.SYSTEM and not m.metadata.get("memory_context")
if any(_is_caller_system_prompt(m) for m in messages):
return messages
prompt = ""
@@ -113,13 +129,15 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
memory_backend = getattr(request.app.state, "memory_backend", None)
if (
config is not None
and memory_backend is not None
and config.agent.context_from_memory
and request_body.messages
):
try:
from openjarvis.tools.storage.context import ContextConfig, inject_context
memory_service = getattr(request.app.state, "memory_service", None)
facts = memory_service.list_facts() if memory_service is not None else []
# Extract query from the last user message
query_text = ""
for m in reversed(request_body.messages):
@@ -129,6 +147,7 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
if query_text:
messages = _to_messages(request_body.messages)
messages = _ensure_identity_prompt(messages, config)
ctx_cfg = ContextConfig(
top_k=config.memory.context_top_k,
min_score=config.memory.context_min_score,
@@ -139,22 +158,35 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
messages,
memory_backend,
config=ctx_cfg,
facts=facts,
)
# Rebuild request messages from enriched Message objects
if len(enriched) > len(messages):
from openjarvis.server.models import ChatMessage
# Rebuild after identity/context merging so downstream engine
# adapters always receive exactly one system message.
from openjarvis.server.models import ChatMessage
new_msgs = []
for msg in enriched:
new_msgs.append(
ChatMessage(
role=msg.role.value,
content=msg.content,
name=msg.name,
tool_call_id=getattr(msg, "tool_call_id", None),
)
new_msgs = []
for msg in enriched:
new_msgs.append(
ChatMessage(
role=msg.role.value,
content=msg.content,
name=msg.name,
tool_calls=[
{
"id": tool_call.id,
"type": "function",
"function": {
"name": tool_call.name,
"arguments": tool_call.arguments,
},
}
for tool_call in (msg.tool_calls or [])
]
or None,
tool_call_id=getattr(msg, "tool_call_id", None),
)
request_body.messages = new_msgs
)
request_body.messages = new_msgs
except Exception:
logging.getLogger("openjarvis.server").debug(
"Memory context injection failed",
@@ -199,12 +231,14 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
# When the client passes `tools`, stream the model's raw
# OpenAI-compat function-calling decision directly from the engine
# (bypassing the agent) — the streaming mirror of the non-streaming
# #454 fix. Routing tools through the agent stream bridge ignored
# `request_body.tools`, ran the agent's own tool loop, and
# word-split generic filler content into fake token deltas, so the
# caller's tool_calls were dropped entirely (the streaming analog of
# #414). For plain chat (no tools), stream token-by-token directly
# from the engine for true real-time output.
# #454 fix. Routing client-supplied tools through a server-side agent
# would execute the agent's different tool set and drop the raw tool
# call the caller expects (#414).
#
# Without client-supplied tools, keep streaming requests on the
# configured server agent so its server-side tool loop is available
# to the desktop UI and other stream:true clients (#735). Fall back to
# direct token streaming when no tool-bearing agent is configured.
if request_body.tools:
return await _handle_stream_tools(
engine,
@@ -215,6 +249,16 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
bus=getattr(request.app.state, "bus", None),
memory_service=getattr(request.app.state, "memory_service", None),
)
if agent is not None and getattr(agent, "_tools", None):
return await _handle_agent_stream(
agent,
model,
request_body,
complexity_info,
trace_store=getattr(request.app.state, "trace_store", None),
bus=getattr(request.app.state, "bus", None),
memory_service=getattr(request.app.state, "memory_service", None),
)
return await _handle_stream(
engine,
model,
@@ -336,6 +380,34 @@ def _remember_exchange(
)
def _engine_key_for_model(engine: Any, model: str) -> str | None:
"""Resolve the engine that advertised *model* through wrapper layers."""
from openjarvis.engine.multi import MultiEngine
from openjarvis.security.guardrails import GuardrailsEngine
from openjarvis.telemetry.instrumented_engine import InstrumentedEngine
current = engine
while current is not None:
if isinstance(current, MultiEngine):
return current.engine_key_for(model)
if isinstance(current, InstrumentedEngine):
current = current._inner
continue
if isinstance(current, GuardrailsEngine):
current = current._engine
continue
engine_id = getattr(current, "engine_id", None)
return engine_id if isinstance(engine_id, str) else None
return None
def _uses_direct_cloud_router(engine: Any, model: str) -> bool:
"""Whether *model* should bypass the configured engine for direct cloud."""
from openjarvis.server.cloud_router import is_cloud_model
return is_cloud_model(model) and _engine_key_for_model(engine, model) != "litellm"
def _handle_direct(
engine,
model: str,
@@ -518,6 +590,114 @@ def _handle_agent(
)
async def _handle_agent_stream(
agent,
model: str,
req: ChatCompletionRequest,
complexity_info=None,
*,
trace_store=None,
bus=None,
memory_service=None,
):
"""Run the configured agent and return its result as an SSE response.
Agents own the tool-execution loop, which is synchronous today. Run that
loop in a worker thread and stream its final answer once complete. This
keeps ``stream:true`` clients (including the desktop UI) on the same agent
and configured toolkit as non-streaming requests instead of bypassing the
agent and silently dropping server-side tools.
Requests that explicitly supply OpenAI ``tools`` continue to use
``_handle_stream_tools`` so their raw tool-call deltas are preserved.
"""
chunk_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
query_text = ""
for message in reversed(req.messages):
if message.role == "user" and message.content:
query_text = message.content
break
async def generate():
first_chunk = ChatCompletionChunk(
id=chunk_id,
model=model,
choices=[StreamChoice(delta=DeltaMessage(role="assistant"))],
)
yield f"data: {first_chunk.model_dump_json()}\n\n"
try:
response = await asyncio.to_thread(
_handle_agent,
agent,
model,
req,
complexity_info,
trace_store=trace_store,
bus=bus,
)
except Exception as exc:
logging.getLogger("openjarvis.server").error(
"Agent stream error: %s",
exc,
exc_info=True,
)
error_chunk = ChatCompletionChunk(
id=chunk_id,
model=model,
choices=[
StreamChoice(
delta=DeltaMessage(
content=f"Sorry, an error occurred: {exc}",
),
finish_reason="stop",
)
],
)
yield f"data: {error_chunk.model_dump_json()}\n\n"
yield "data: [DONE]\n\n"
return
content = _response_content(response)
if content:
content_chunk = ChatCompletionChunk(
id=chunk_id,
model=model,
choices=[StreamChoice(delta=DeltaMessage(content=content))],
)
yield f"data: {content_chunk.model_dump_json()}\n\n"
import json as _json
finish_chunk = ChatCompletionChunk(
id=chunk_id,
model=model,
choices=[
StreamChoice(delta=DeltaMessage(), finish_reason="stop"),
],
)
finish_data = _json.loads(finish_chunk.model_dump_json())
finish_data["usage"] = response.usage.model_dump()
if complexity_info is not None:
finish_data["complexity"] = complexity_info.model_dump()
yield f"data: {_json.dumps(finish_data)}\n\n"
_record_completed_exchange(
memory_service,
query_text,
content,
bus=bus,
source="server.chat.stream",
)
yield "data: [DONE]\n\n"
return StreamingResponse(
generate(),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "Connection": "keep-alive"},
)
async def _handle_stream_tools(
engine,
model: str,
@@ -541,12 +721,13 @@ async def _handle_stream_tools(
tool_calls) identical to the prior plain-stream behaviour, so this never
regresses non-tool-capable engines.
"""
from openjarvis.server.cloud_router import is_cloud_model
messages = _to_messages(req.messages)
messages = _ensure_identity_prompt(messages, app_config)
chunk_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
use_cloud = is_cloud_model(model)
use_cloud = _uses_direct_cloud_router(engine, model)
telemetry_engine = (
"cloud" if use_cloud else (_engine_key_for_model(engine, model) or "ollama")
)
query_text = ""
for _m in reversed(req.messages):
if _m.role == "user" and _m.content:
@@ -626,7 +807,7 @@ async def _handle_stream_tools(
# Tag the finish chunk with the engine label, matching _handle_stream
# so UI/telemetry consumers see the same field on the tools path.
finish_dict.setdefault("telemetry", {})
finish_dict["telemetry"]["engine"] = "cloud" if use_cloud else "ollama"
finish_dict["telemetry"]["engine"] = telemetry_engine
if complexity_info is not None:
finish_dict["complexity"] = complexity_info.model_dump()
yield f"data: {_json.dumps(finish_dict)}\n\n"
@@ -660,19 +841,14 @@ async def _handle_stream(
):
"""Stream response using SSE format.
This path streams straight from the engine, bypassing the agent /
This no-agent fallback streams straight from the engine, bypassing the
``TraceCollector``. When *trace_store* is set we accumulate the streamed
tokens and record a minimal ``Trace`` once the stream completes
successfully otherwise streamed chats (the desktop GUI's main path)
would never populate ``traces.db``.
successfully.
"""
import time
from openjarvis.server.cloud_router import (
is_cloud_model,
stream_cloud,
stream_local,
)
from openjarvis.server.cloud_router import stream_cloud, stream_local
messages = _to_messages(req.messages)
messages = _ensure_identity_prompt(messages, app_config)
@@ -687,7 +863,10 @@ async def _handle_stream(
# Route directly to the right backend — bypasses engine routing entirely
# so broken MultiEngine state can never misdirect requests.
use_cloud = is_cloud_model(model)
use_cloud = _uses_direct_cloud_router(engine, model)
telemetry_engine = (
"cloud" if use_cloud else (_engine_key_for_model(engine, model) or "ollama")
)
async def generate():
started_at = time.time()
@@ -792,7 +971,7 @@ async def _handle_stream(
query=query_text,
result=full_content,
model=model,
engine="cloud" if use_cloud else "ollama",
engine=telemetry_engine,
started_at=started_at,
ended_at=time.time(),
)
@@ -825,7 +1004,7 @@ async def _handle_stream(
# We use the routing decision (use_cloud) directly rather than
# unwrapping the engine chain, which can be in a broken state.
finish_dict.setdefault("telemetry", {})
finish_dict["telemetry"]["engine"] = "cloud" if use_cloud else "ollama"
finish_dict["telemetry"]["engine"] = telemetry_engine
if complexity_info is not None:
finish_dict["complexity"] = complexity_info.model_dump()
@@ -842,24 +1021,45 @@ async def _handle_stream(
@router.get("/v1/models")
async def list_models(request: Request) -> ModelListResponse:
"""List locally installed models (Ollama).
"""List selectable engine models for the installed-model picker.
Cloud models are not included here they live in the Cloud Models tab
of the UI and are selected there, not from this endpoint.
Direct cloud models live in the Cloud Models tab. Models advertised by a
configured LiteLLM engine remain here because LiteLLM owns their routing
and may use provider-qualified IDs that resemble OpenRouter IDs.
"""
from openjarvis.server.cloud_router import is_cloud_model, list_local_models
# Prefer engine.list_models() so mock engines work in tests.
# Filter out any cloud model IDs that may appear via MultiEngine.
# Filter out direct-cloud model IDs that may appear via MultiEngine, but
# retain provider-qualified IDs owned by the configured LiteLLM engine.
# Fall back to direct Ollama query only when the engine returns nothing.
engine = request.app.state.engine
all_ids = await asyncio.to_thread(engine.list_models)
model_ids = [m for m in all_ids if not is_cloud_model(m)]
model_ids = [
m
for m in all_ids
if not is_cloud_model(m) or _engine_key_for_model(engine, m) == "litellm"
]
if not model_ids:
model_ids = await list_local_models()
# Keep embed-only models out of the chat model picker. They still work for
# memory/retrieval via the embedder path; putting them in /v1/models made
# the UI auto-select nomic-embed-text and fail every generation with 400.
model_ids = [m for m in model_ids if not is_embed_only_model(m)]
return ModelListResponse(
data=[ModelObject(id=mid) for mid in model_ids],
data=[
ModelObject(
id=mid,
owned_by=(
"litellm"
if _engine_key_for_model(engine, mid) == "litellm"
else "openjarvis"
),
)
for mid in model_ids
],
)
+4 -2
View File
@@ -25,9 +25,11 @@ class SessionStore:
def __init__(self, db_path: str = "") -> None:
if not db_path:
db_path = str(get_config_dir() / "sessions.db")
from openjarvis.security.file_utils import secure_create
# Ensure the parent directory exists (skip for :memory:)
if db_path != ":memory:":
from openjarvis.security.file_utils import secure_create
secure_create(Path(db_path))
secure_create(Path(db_path))
self._db = sqlite3.connect(db_path, check_same_thread=False)
self._db.row_factory = sqlite3.Row
self._create_tables()
+14 -55
View File
@@ -110,6 +110,12 @@ class AgentStreamBridge:
def _format_named_event(self, name: str, data: dict) -> str:
"""Format an SSE event with an explicit ``event:`` field."""
if name == "tool_call_start" and not isinstance(data.get("arguments"), str):
# The in-process event bus uses parsed arguments for trace/eval
# consumers, while the web SSE contract expects their JSON text.
# Copy before normalizing so other subscribers keep the object.
data = dict(data)
data["arguments"] = json.dumps(data.get("arguments"))
return f"event: {name}\ndata: {json.dumps(data)}\n\n"
def _run_agent(self) -> object:
@@ -240,62 +246,15 @@ class AgentStreamBridge:
{"results": tool_results_data},
)
# Stream content using real LLM token streaming via
# engine.stream_full() when the engine is available.
# ``agent.run()`` already produced the authoritative, grounded
# response. Do not call the engine again here: a second inference
# would not have the agent's system prompt, tool transcript, or
# other internal context and could therefore contradict the
# result reported by the agent events. Replay the final content
# in chunks so the OpenAI-compatible streaming response stays
# consistent with the completed agent run.
content = agent_result.content or ""
engine = getattr(self._agent, "_engine", None)
used_real_streaming = False
if engine is not None and hasattr(engine, "stream_full") and content:
# Re-stream using the engine for real token delivery.
# Build the same messages the agent used for its final turn.
try:
from openjarvis.core.types import Message as MsgType
from openjarvis.core.types import Role as RoleType
replay_messages = []
for m in self._request.messages:
role = (
RoleType(m.role)
if m.role in {r.value for r in RoleType}
else RoleType.USER
)
replay_messages.append(
MsgType(
role=role,
content=m.content or "",
name=m.name,
tool_call_id=m.tool_call_id,
)
)
async for sc in engine.stream_full(
replay_messages,
model=self._model,
):
if sc.content:
chunk = ChatCompletionChunk(
id=self._chunk_id,
model=self._model,
choices=[
StreamChoice(
delta=DeltaMessage(content=sc.content),
)
],
)
yield f"data: {chunk.model_dump_json()}\n\n"
used_real_streaming = True
except Exception as stream_exc:
import logging as _logging
_logger = _logging.getLogger("openjarvis.server")
_logger.warning(
"Real streaming failed, falling back to word replay: %s",
stream_exc,
)
# Fallback: word-by-word replay if real streaming was not used
if not used_real_streaming and content:
if content:
words = content.split(" ")
for i, word in enumerate(words):
token = word if i == 0 else " " + word
+30 -3
View File
@@ -79,14 +79,41 @@ def create_ws_router(event_bus: EventBus) -> Any:
queue: asyncio.Queue = asyncio.Queue(maxsize=100)
loop = asyncio.get_running_loop()
clients[websocket] = (queue, loop)
recv: asyncio.Task | None = None
payload: asyncio.Task | None = None
disconnected = False
try:
recv = asyncio.create_task(websocket.receive())
payload = asyncio.create_task(queue.get())
while True:
payload = await queue.get()
await websocket.send_json(payload)
done, _ = await asyncio.wait(
{recv, payload}, return_when=asyncio.FIRST_COMPLETED
)
if recv in done:
# Starlette surfaces a disconnect message only when the app
# reads from the socket. Without this receive, the handler
# can stay parked on queue.get() after the client leaves.
message = await recv
if message.get("type") == "websocket.disconnect":
disconnected = True
break
recv = asyncio.create_task(websocket.receive())
if payload in done:
await websocket.send_json(payload.result())
payload = asyncio.create_task(queue.get())
except WebSocketDisconnect:
pass
disconnected = True
finally:
clients.pop(websocket, None)
pending = [task for task in (recv, payload) if task is not None]
for task in pending:
task.cancel()
cleanup = asyncio.gather(*pending, return_exceptions=True)
try:
await asyncio.shield(cleanup)
except asyncio.CancelledError:
if not disconnected:
raise
return router
+31 -1
View File
@@ -48,6 +48,7 @@ class SystemBuilder:
self._sessions: Optional[bool] = None
self._speech: Optional[bool] = None
self._mcp_clients: List = []
self._mcp_tools: List[BaseTool] = []
def engine(self, key: str) -> SystemBuilder:
self._engine_key = key
@@ -113,6 +114,33 @@ class SystemBuilder:
def build(self) -> JarvisSystem:
"""Construct a fully wired JarvisSystem."""
# Discovery state belongs to one build only. Once a system is
# returned, that system owns the clients and adapters captured below;
# retaining them here would make a reused builder hand closed clients
# from an earlier system to the next one.
self._clear_mcp_discovery_state(close_clients=True)
try:
system = self._build()
except BaseException:
# No system took ownership, so release any clients opened before
# the build failed.
self._clear_mcp_discovery_state(close_clients=True)
raise
self._clear_mcp_discovery_state(close_clients=False)
return system
def _clear_mcp_discovery_state(self, *, close_clients: bool) -> None:
if close_clients:
for client in getattr(self, "_mcp_clients", []):
try:
client.close()
except Exception:
logger.debug("Error closing unowned MCP client", exc_info=True)
self._mcp_clients = []
self._mcp_tools = []
def _build(self) -> JarvisSystem:
"""Build one system using fresh, build-local MCP discovery state."""
config = self._config
bus = self._bus or get_event_bus()
@@ -291,6 +319,7 @@ class SystemBuilder:
model=model,
agent_name=agent_name,
tools=tool_list,
mcp_tools=list(self._mcp_tools),
tool_executor=tool_executor,
memory_backend=memory_backend,
channel_backend=channel_backend,
@@ -440,7 +469,7 @@ class SystemBuilder:
else:
tools = []
if config.tools.mcp.servers:
if config.tools.mcp.enabled and config.tools.mcp.servers:
try:
import json
@@ -449,6 +478,7 @@ class SystemBuilder:
for server_cfg in server_list:
try:
external_tools = self._discover_external_mcp(server_cfg)
self._mcp_tools.extend(external_tools)
if tool_names:
external_tools = [
t
+3
View File
@@ -86,6 +86,9 @@ class JarvisSystem:
skill_manager: Optional[SkillManager] = None
_learning_orchestrator: Optional[LearningOrchestrator] = None
_mcp_clients: List[MCPClient] = field(default_factory=list)
# Keep newly added fields after every pre-existing positional field so
# older positional JarvisSystem(...) calls retain their original meaning.
mcp_tools: List[BaseTool] = field(default_factory=list)
@property
def security(self) -> SecurityContext:
+4 -1
View File
@@ -40,8 +40,9 @@ class QueryOrchestrator:
messages = [Message(role=Role.USER, content=query)]
if context and s.memory_backend and s.config.agent.context_from_memory:
if context and s.config.agent.context_from_memory:
try:
from openjarvis.memory import load_configured_facts
from openjarvis.tools.storage.context import (
ContextConfig,
inject_context,
@@ -52,11 +53,13 @@ class QueryOrchestrator:
min_score=s.config.memory.context_min_score,
max_context_tokens=s.config.memory.context_max_tokens,
)
facts = load_configured_facts(s.config)
messages = inject_context(
query,
messages,
s.memory_backend,
config=ctx_cfg,
facts=facts,
)
except Exception as exc:
logger.warning("Failed to inject memory context: %s", exc)
@@ -212,6 +212,7 @@ class InstrumentedEngine(InferenceEngine):
completion_tokens=completion_tokens,
total_tokens=prompt_tok + completion_tokens,
latency_seconds=latency,
cost_usd=result.get("cost_usd", 0.0),
ttft=ttft,
throughput_tok_per_sec=throughput,
energy_per_output_token_joules=energy_per_output_token,
+10
View File
@@ -142,4 +142,14 @@ try:
except ImportError:
pass
try:
import openjarvis.tools.scan_chunks # noqa: F401
except ImportError:
pass
try:
import openjarvis.tools.knowledge_sql # noqa: F401
except ImportError:
pass
__all__ = ["BaseTool", "ToolExecutor", "ToolSpec"]
+18
View File
@@ -136,6 +136,15 @@ class ToolExecutor:
content=f"Invalid arguments JSON: {exc}",
success=False,
)
if not isinstance(params, dict):
return ToolResult(
tool_name=tool_call.name,
content=(
"Invalid arguments: expected a JSON object, "
f"got {type(params).__name__}."
),
success=False,
)
# Boundary guard: scan external tool arguments
if self._boundary_guard is not None and not getattr(tool, "is_local", True):
@@ -143,6 +152,15 @@ class ToolExecutor:
tool_call = self._boundary_guard.check_outbound(tool_call)
# Re-parse arguments after potential redaction
params = json.loads(tool_call.arguments) if tool_call.arguments else {}
if not isinstance(params, dict):
return ToolResult(
tool_name=tool_call.name,
content=(
"Invalid arguments: expected a JSON object, "
f"got {type(params).__name__}."
),
success=False,
)
except Exception as exc:
return ToolResult(
tool_name=tool_call.name,
+1
View File
@@ -56,6 +56,7 @@ class CodeInterpreterTool(BaseTool):
"required": ["code"],
},
category="code",
metadata={"structured_allow_object_text": True},
)
def execute(self, **params: Any) -> ToolResult:
@@ -55,6 +55,7 @@ class DockerCodeInterpreterTool(BaseTool):
},
category="code",
timeout_seconds=60.0,
metadata={"structured_allow_object_text": True},
)
def execute(self, **params: Any) -> ToolResult:
+1
View File
@@ -191,6 +191,7 @@ class ReplTool(BaseTool):
"required": ["code"],
},
category="code",
metadata={"structured_allow_object_text": True},
)
def execute(self, **params: Any) -> ToolResult:
+93 -20
View File
@@ -2,13 +2,16 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import List, Optional
from dataclasses import dataclass, replace
from typing import TYPE_CHECKING, List, Optional, Sequence
from openjarvis.core.events import EventType, get_event_bus
from openjarvis.core.types import Message, Role
from openjarvis.tools.storage._stubs import MemoryBackend, RetrievalResult
if TYPE_CHECKING:
from openjarvis.memory.store import Fact
@dataclass(slots=True)
class ContextConfig:
@@ -46,28 +49,75 @@ def format_context(results: List[RetrievalResult]) -> str:
def build_context_message(
results: List[RetrievalResult],
facts: Sequence[Fact] = (),
) -> Message:
"""Create a system message with formatted context."""
context_text = format_context(results)
content = (
"The following context was retrieved from the knowledge"
" base. Use it to inform your response, citing sources"
" where applicable:\n\n" + context_text
sections = []
if facts:
fact_text = "\n".join(f"- {fact.text}" for fact in facts)
sections.append(
"The following durable facts were remembered from prior "
"conversations. Use them when relevant to the user's request:\n\n"
+ fact_text
)
if results:
sections.append(
"The following context was retrieved from the knowledge"
" base. Use it to inform your response, citing sources"
" where applicable:\n\n" + format_context(results)
)
content = "\n\n".join(sections)
return Message(
role=Role.SYSTEM,
content=content,
metadata={"memory_context": True},
)
return Message(role=Role.SYSTEM, content=content)
def _merge_context_message(
messages: List[Message],
context_message: Message,
) -> List[Message]:
"""Return a copy with context folded into the existing system prompt."""
system_messages = [message for message in messages if message.role == Role.SYSTEM]
if not system_messages:
return [context_message, *messages]
content = "\n\n".join(
part
for part in (
*(message.text for message in system_messages),
context_message.text,
)
if part
)
combined = replace(system_messages[0], content=content)
merged: List[Message] = []
inserted = False
for message in messages:
if message.role == Role.SYSTEM:
if not inserted:
merged.append(combined)
inserted = True
continue
merged.append(message)
return merged
def inject_context(
query: str,
messages: List[Message],
backend: MemoryBackend,
backend: Optional[MemoryBackend],
*,
config: Optional[ContextConfig] = None,
facts: Sequence[Fact] = (),
) -> List[Message]:
"""Retrieve relevant context and prepend it to *messages*.
Returns a **new** list the original list is not mutated.
If no results pass the score threshold, returns the original
Automatic-memory facts are included independently of the retrieval
backend, so persisted facts remain recallable even when the document
store is empty. If no facts or results are available, returns the original
messages unchanged.
Parameters
@@ -77,33 +127,55 @@ def inject_context(
messages:
The existing message list.
backend:
The memory backend to search.
The memory backend to search, or ``None`` when only facts are available.
config:
Context injection settings (uses defaults if ``None``).
facts:
Durable facts captured by the automatic memory service.
"""
cfg = config or ContextConfig()
if not cfg.enabled:
return messages
results = backend.retrieve(query, top_k=cfg.top_k)
results = backend.retrieve(query, top_k=cfg.top_k) if backend is not None else []
# Filter by minimum score
results = [r for r in results if r.score >= cfg.min_score]
if not results:
return messages
# Truncate to max_context_tokens
truncated: List[RetrievalResult] = []
# When both sources have data, cap facts at half the total budget so they
# cannot starve query-specific document retrieval. Unused fact budget is
# still available to documents. Newest facts win within the fact budget.
fact_budget = cfg.max_context_tokens
if results:
fact_budget //= 2
selected_facts: List[Fact] = []
total_tokens = 0
for fact in reversed(facts):
tokens = _count_tokens(fact.text)
if total_tokens + tokens > fact_budget:
continue
selected_facts.append(fact)
total_tokens += tokens
# Fill the remaining context budget with retrieved documents.
truncated: List[RetrievalResult] = []
for r in results:
tokens = _count_tokens(r.content)
if total_tokens + tokens > cfg.max_context_tokens:
# A large top result should not disappear solely because facts
# consumed their reserved share. Prefer that result when it fits
# the total budget on its own.
if not truncated and selected_facts and tokens <= cfg.max_context_tokens:
selected_facts = []
total_tokens = 0
else:
break
if total_tokens + tokens > cfg.max_context_tokens:
break
truncated.append(r)
total_tokens += tokens
if not truncated:
if not selected_facts and not truncated:
return messages
# Publish event
@@ -114,13 +186,14 @@ def inject_context(
"context_injection": True,
"query": query,
"num_results": len(truncated),
"num_facts": len(selected_facts),
"total_tokens": total_tokens,
},
)
# Build context message and prepend
ctx_msg = build_context_message(truncated)
return [ctx_msg] + list(messages)
ctx_msg = build_context_message(truncated, selected_facts)
return _merge_context_message(messages, ctx_msg)
__all__ = [
+23
View File
@@ -95,6 +95,29 @@ class SQLiteMemory(MemoryBackend):
)
return doc_id
def replace_source(
self,
source: str,
documents: List[tuple[str, Optional[Dict[str, Any]]]],
) -> List[str]:
"""Atomically replace all documents associated with *source*."""
payload = [
(content, json.dumps(metadata) if metadata else None)
for content, metadata in documents
]
doc_ids = self._rust_impl.replace_source(source, payload)
bus = get_event_bus()
for doc_id in doc_ids:
bus.publish(
EventType.MEMORY_STORE,
{
"backend": self.backend_id,
"doc_id": doc_id,
"source": source,
},
)
return doc_ids
def retrieve(
self,
query: str,
+34
View File
@@ -205,6 +205,40 @@ class TestBuildMessages:
assert messages[1].content == "prev"
assert messages[2].content == "new"
def test_prompt_builder_merges_context_system_message(self):
engine = MagicMock()
prompt_builder = MagicMock()
prompt_builder.build.return_value = "You are OpenJarvis."
agent = _ConcreteAgent(engine, "m", prompt_builder=prompt_builder)
conv = Conversation()
conv.add(
Message(
role=Role.SYSTEM,
content="Remember: user likes jazz.",
metadata={"memory_context": True},
)
)
ctx = AgentContext(conversation=conv)
messages = agent._build_messages("new", ctx)
system_messages = [m for m in messages if m.role == Role.SYSTEM]
assert len(system_messages) == 1
assert "You are OpenJarvis." in system_messages[0].content
assert "user likes jazz" in system_messages[0].content
def test_prompt_builder_preserves_caller_system_context(self):
engine = MagicMock()
prompt_builder = MagicMock()
prompt_builder.build.return_value = "Agent instructions."
agent = _ConcreteAgent(engine, "m", prompt_builder=prompt_builder)
conv = Conversation()
conv.add(Message(role=Role.SYSTEM, content="You are helpful."))
messages = agent._build_messages("new", AgentContext(conversation=conv))
assert any(message.content == "You are helpful." for message in messages)
class TestGenerate:
def test_delegates_to_engine(self):
@@ -0,0 +1,127 @@
"""Regression tests for managed-agent tool-call persistence."""
from __future__ import annotations
import json
import pytest
from openjarvis.agents._stubs import AgentResult
from openjarvis.agents.executor import AgentExecutor, _tool_calls_for_storage
from openjarvis.agents.manager import AgentManager
from openjarvis.core.events import EventBus
from openjarvis.core.types import ToolResult
def test_tool_results_are_serialized_for_managed_messages() -> None:
result = AgentResult(
content="Finished",
tool_results=[
ToolResult(
tool_name="knowledge_search",
content="Found the requested note",
success=True,
latency_seconds=0.42,
metadata={
"arguments": {
"query": "financial independence",
"limit": 3,
}
},
),
ToolResult(
tool_name="shell_exec",
content="Permission denied",
success=False,
latency_seconds=1.25,
metadata={"arguments": '{"command":"whoami"}'},
),
],
)
calls = _tool_calls_for_storage(result)
assert calls is not None
assert len(calls) == 2
knowledge_call = calls[0]
assert knowledge_call["tool"] == "knowledge_search"
assert isinstance(knowledge_call["arguments"], str)
assert json.loads(knowledge_call["arguments"]) == {
"query": "financial independence",
"limit": 3,
}
assert knowledge_call["result"] == "Found the requested note"
assert knowledge_call["success"] is True
assert knowledge_call["latency"] == pytest.approx(420.0)
failed_call = calls[1]
assert failed_call["arguments"] == '{"command":"whoami"}'
assert failed_call["result"] == "Permission denied"
assert failed_call["success"] is False
assert failed_call["latency"] == pytest.approx(1250.0)
def test_no_tool_results_serialize_as_none() -> None:
assert _tool_calls_for_storage(AgentResult(content="Plain response")) is None
def test_finalize_tick_persists_tool_calls_round_trip(tmp_path) -> None:
manager = AgentManager(str(tmp_path / "agents.db"))
try:
agent = manager.create_agent("researcher")
manager.start_tick(agent["id"])
result = AgentResult(
content="Answer grounded in the knowledge base",
tool_results=[
ToolResult(
tool_name="knowledge_search",
content="Matching source text",
success=True,
latency_seconds=0.007,
metadata={"arguments": {"query": "grounded answer"}},
)
],
)
executor = AgentExecutor(manager, EventBus())
executor._finalize_tick(
agent["id"],
result,
error=None,
duration=0.01,
)
messages = manager.list_messages(agent["id"])
assert len(messages) == 1
stored = messages[0]
assert stored["content"] == result.content
assert stored["direction"] == "agent_to_user"
assert stored["tool_calls"] == _tool_calls_for_storage(result)
assert isinstance(stored["tool_calls"][0]["arguments"], str)
assert json.loads(stored["tool_calls"][0]["arguments"]) == {
"query": "grounded answer"
}
assert stored["tool_calls"][0]["latency"] == pytest.approx(7.0)
finally:
manager.close()
def test_finalize_tick_without_tools_stores_null_tool_calls(tmp_path) -> None:
manager = AgentManager(str(tmp_path / "agents.db"))
try:
agent = manager.create_agent("plain-agent")
manager.start_tick(agent["id"])
executor = AgentExecutor(manager, EventBus())
executor._finalize_tick(
agent["id"],
AgentResult(content="No tools needed"),
error=None,
duration=0.01,
)
stored = manager.list_messages(agent["id"])[0]
assert stored["tool_calls"] is None
finally:
manager.close()
+471
View File
@@ -2,13 +2,91 @@
from __future__ import annotations
import gc
import sqlite3
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
from openjarvis.agents._stubs import AgentResult
from openjarvis.agents.executor import AgentExecutor
from openjarvis.agents.manager import AgentManager
from openjarvis.agents.tool_resolver import ResolvedAgentTools
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.core.config import MemoryFilesConfig, SystemPromptConfig
from openjarvis.core.events import EventBus
from openjarvis.core.registry import AgentRegistry, ToolRegistry
from openjarvis.core.types import Role, ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
from tests.agents.fake_engine import FakeEngine
from tests.agents.scenario_harness import FakeSystem
class _CapturingToolAgent:
"""Minimal agent that exposes the toolkit received by AgentExecutor."""
accepts_tools = True
captured_tools = []
captured_search_result = None
def __init__(self, engine, model, *, tools=None, **kwargs):
self.engine = engine
self.model = model
type(self).captured_tools = list(tools or [])
def run(self, input_text, context=None):
tools_by_name = {tool.spec.name: tool for tool in self.captured_tools}
search = tools_by_name.get("knowledge_search")
if search is not None:
type(self).captured_search_result = search.execute(
query="EXECUTOR_RESOLVER_SENTINEL"
)
return AgentResult(content="captured")
class _NonToolAgent:
"""Agent class whose run method must not swallow a configured toolkit."""
accepts_tools = False
supports_managed_tool_fallback = True
runs = 0
def __init__(self, engine, model, **kwargs):
pass
def run(self, input_text, context=None):
type(self).runs += 1
raise AssertionError("non-tool agent should use the managed tool loop")
class _SpecializedNonToolAgent:
"""Non-tool agent that must retain its specialized execution path."""
accepts_tools = False
runs = 0
def __init__(self, engine, model, **kwargs):
pass
def run(self, input_text, context=None):
type(self).runs += 1
return AgentResult(content="specialized response")
class _ExecutorProbeTool(BaseTool):
tool_id = "executor_probe"
calls = 0
@property
def spec(self) -> ToolSpec:
return ToolSpec(name=self.tool_id, description="Executor parity probe")
def execute(self, **params) -> ToolResult:
type(self).calls += 1
return ToolResult(tool_name=self.tool_id, content="probe-result")
def _register_agent():
"""Re-register MonitorOperativeAgent (cleared by autouse fixture)."""
from openjarvis.agents.monitor_operative import MonitorOperativeAgent
@@ -102,3 +180,396 @@ def test_executor_handles_string_tools(tmp_path):
result_agent = mgr.get_agent(agent["id"])
assert result_agent["status"] == "idle"
mgr.close()
def test_executor_uses_tool_loop_for_non_tool_agent_with_configured_tools(tmp_path):
"""Immediate/scheduled ticks match SSE instead of discarding tools."""
AgentRegistry.register_value("non_tool_probe", _NonToolAgent)
ToolRegistry.register_value(_ExecutorProbeTool.tool_id, _ExecutorProbeTool)
_NonToolAgent.runs = 0
_ExecutorProbeTool.calls = 0
engine = FakeEngine(
[
{
"tool_calls": [
{
"id": "call-executor-probe",
"name": _ExecutorProbeTool.tool_id,
"arguments": "{}",
}
]
},
{"content": "tool-backed final response"},
]
)
system = FakeSystem(engine=engine)
system.config = SimpleNamespace(
agent=SimpleNamespace(default_system_prompt="GLOBAL_DEFAULT"),
memory_files=MemoryFilesConfig(persona_name="none"),
system_prompt=SystemPromptConfig(),
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"non-tool with tools",
agent_type="non_tool_probe",
config={
"model": "test-model",
"tools": [_ExecutorProbeTool.tool_id],
"instruction": "Use the probe.",
"system_prompt": "NON_TOOL_SYSTEM_SENTINEL",
},
)
try:
AgentExecutor(manager, EventBus(), system=system).execute_tick(agent["id"])
assert _NonToolAgent.runs == 0
assert _ExecutorProbeTool.calls == 1
assert engine.call_count == 2
assert any(
message.role is Role.SYSTEM
and message.content == "NON_TOOL_SYSTEM_SENTINEL"
for message in engine.last_messages or []
)
refreshed = manager.get_agent(agent["id"])
assert refreshed["status"] == "idle"
assert refreshed["total_runs"] == 1
responses = [
message
for message in manager.list_messages(agent["id"])
if message["direction"] == "agent_to_user"
]
assert responses[-1]["content"] == "tool-backed final response"
assert responses[-1]["tool_calls"][0]["tool"] == "executor_probe"
finally:
manager.close()
def test_simple_agent_uses_global_mcp_tools_without_native_tool_config(tmp_path):
"""Fallback-compatible simple agents preserve SSE/global-MCP parity."""
from openjarvis.agents.simple import SimpleAgent
AgentRegistry.register_value("simple", SimpleAgent)
_ExecutorProbeTool.calls = 0
provider = MagicMock(return_value=([_ExecutorProbeTool()], []))
engine = FakeEngine(
[
{
"tool_calls": [
{
"id": "call-global-mcp-probe",
"name": _ExecutorProbeTool.tool_id,
"arguments": "{}",
}
]
},
{"content": "global MCP response"},
]
)
system = SimpleNamespace(
engine=engine,
model="test-model",
config=None,
memory_backend=None,
channel_backend=None,
session_store=None,
knowledge_db_path=None,
get_managed_agent_mcp_tools=provider,
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"simple global MCP",
agent_type="simple",
config={"model": "test-model", "instruction": "Use MCP."},
)
try:
AgentExecutor(manager, EventBus(), system=system).execute_tick(agent["id"])
provider.assert_called_once_with()
assert _ExecutorProbeTool.calls == 1
assert engine.call_count == 2
responses = [
message
for message in manager.list_messages(agent["id"])
if message["direction"] == "agent_to_user"
]
assert responses[-1]["content"] == "global MCP response"
finally:
manager.close()
def test_simple_agent_without_tools_keeps_its_custom_system_prompt(tmp_path):
"""Signature filtering must not discard prompt-builder state on retry."""
from openjarvis.agents.simple import SimpleAgent
AgentRegistry.register_value("simple", SimpleAgent)
engine = FakeEngine([{"content": "custom prompt response"}])
system = FakeSystem(engine=engine)
system.config = SimpleNamespace(
agent=SimpleNamespace(default_system_prompt="GLOBAL_DEFAULT"),
memory_files=MemoryFilesConfig(persona_name="none"),
system_prompt=SystemPromptConfig(),
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"simple custom prompt",
agent_type="simple",
config={
"model": "test-model",
"instruction": "Answer directly.",
"system_prompt": "SIMPLE_CUSTOM_SYSTEM_SENTINEL",
"mcp_tools": False,
},
)
try:
AgentExecutor(manager, EventBus(), system=system).execute_tick(agent["id"])
assert engine.call_count == 1
assert any(
message.role is Role.SYSTEM
and message.content == "SIMPLE_CUSTOM_SYSTEM_SENTINEL"
for message in engine.last_messages or []
)
finally:
manager.close()
def test_specialized_non_tool_agent_is_not_replaced_by_generic_tool_loop(tmp_path):
"""Configured/global tools never replace a non-opted-in agent class."""
AgentRegistry.register_value("specialized_non_tool", _SpecializedNonToolAgent)
ToolRegistry.register_value(_ExecutorProbeTool.tool_id, _ExecutorProbeTool)
_SpecializedNonToolAgent.runs = 0
_ExecutorProbeTool.calls = 0
provider = MagicMock(return_value=([_ExecutorProbeTool()], []))
system = SimpleNamespace(
engine=FakeEngine([{"content": "unused"}]),
model="test-model",
config=None,
memory_backend=None,
channel_backend=None,
session_store=None,
knowledge_db_path=None,
get_managed_agent_mcp_tools=provider,
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"specialized with configured tool",
agent_type="specialized_non_tool",
config={
"model": "test-model",
"instruction": "Keep the specialized path.",
"tools": [_ExecutorProbeTool.tool_id],
},
)
try:
AgentExecutor(manager, EventBus(), system=system).execute_tick(agent["id"])
provider.assert_not_called()
assert _SpecializedNonToolAgent.runs == 1
assert _ExecutorProbeTool.calls == 0
responses = [
message
for message in manager.list_messages(agent["id"])
if message["direction"] == "agent_to_user"
]
assert responses[-1]["content"] == "specialized response"
finally:
manager.close()
def test_executor_grants_deep_research_live_knowledge_tools(tmp_path):
"""Immediate ticks receive the same live Deep Research grant as SSE."""
AgentRegistry.register_value("deep_research", _CapturingToolAgent)
_CapturingToolAgent.captured_tools = []
_CapturingToolAgent.captured_search_result = None
knowledge_db_path = tmp_path / "knowledge.db"
with KnowledgeStore(db_path=knowledge_db_path) as store:
store.store(
"The EXECUTOR_RESOLVER_SENTINEL decision was approved.",
source="test",
doc_type="note",
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"researcher",
agent_type="deep_research",
config={
"model": "agent-selected-model",
# These duplicate two agent-type grants and must not replace them.
"tools": ["knowledge_search", "think"],
"instruction": "Find the sentinel.",
},
)
manager.send_message(agent["id"], "Search the knowledge base.", mode="immediate")
system = SimpleNamespace(
engine=FakeEngine([{"content": "unused"}]),
model="system-model",
memory_backend=None,
channel_backend=None,
tool_executor=None,
_mcp_clients=[],
knowledge_db_path=knowledge_db_path,
config=None,
session_store=None,
)
executor = AgentExecutor(manager=manager, event_bus=EventBus(), system=system)
try:
executor.execute_tick(agent["id"])
tools_by_name = {
tool.spec.name: tool for tool in _CapturingToolAgent.captured_tools
}
assert set(tools_by_name) == {
"knowledge_search",
"knowledge_sql",
"scan_chunks",
"think",
}
result = _CapturingToolAgent.captured_search_result
assert result is not None
assert result.success is True
assert "EXECUTOR_RESOLVER_SENTINEL" in result.content
assert tools_by_name["scan_chunks"]._model == "agent-selected-model"
assert manager.get_agent(agent["id"])["status"] == "idle"
with pytest.raises(sqlite3.ProgrammingError):
tools_by_name["knowledge_sql"]._store._conn.execute("SELECT 1")
finally:
manager.close()
def test_executor_mcp_opt_out_does_not_call_lazy_provider(tmp_path):
"""An opted-out tick must not trigger request-local MCP discovery."""
AgentRegistry.register_value("capturing", _CapturingToolAgent)
provider = MagicMock(side_effect=AssertionError("MCP discovery must stay lazy"))
system = SimpleNamespace(
engine=FakeEngine([{"content": "unused"}]),
model="system-model",
memory_backend=None,
channel_backend=None,
tool_executor=None,
_mcp_clients=[],
config=None,
session_store=None,
get_managed_agent_mcp_tools=provider,
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"no-mcp",
agent_type="capturing",
config={"model": "test-model", "mcp_tools": False},
)
try:
AgentExecutor(manager, EventBus(), system=system).execute_tick(agent["id"])
provider.assert_not_called()
assert manager.get_agent(agent["id"])["status"] == "idle"
finally:
manager.close()
def test_executor_preserves_custom_dict_tool_schema(tmp_path):
"""Executor-based agents see the same custom schema advertised by SSE."""
from openjarvis.tools.think import ThinkTool
AgentRegistry.register_value("capturing", _CapturingToolAgent)
ToolRegistry.register_value("think", ThinkTool)
_CapturingToolAgent.captured_tools = []
custom_spec = {
"type": "function",
"function": {
"name": "think",
"description": "Agent-specific thinking schema",
"parameters": {
"type": "object",
"properties": {"thought": {"type": "string"}},
"required": ["thought"],
},
},
}
system = SimpleNamespace(
engine=FakeEngine([{"content": "unused"}]),
model="test-model",
memory_backend=None,
channel_backend=None,
tool_executor=None,
mcp_tools=[],
_mcp_clients=[],
config=None,
session_store=None,
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"custom-schema",
agent_type="capturing",
config={"model": "test-model", "tools": [custom_spec]},
)
try:
AgentExecutor(manager, EventBus(), system=system).execute_tick(agent["id"])
assert len(_CapturingToolAgent.captured_tools) == 1
configured_tool = _CapturingToolAgent.captured_tools[0]
assert configured_tool.to_openai_function() == custom_spec
assert configured_tool.spec.description == "Agent-specific thinking schema"
assert configured_tool.execute(thought="same instance").success is True
finally:
manager.close()
def test_executor_closes_resolver_resources_when_pre_run_setup_fails(
tmp_path,
monkeypatch,
):
"""The resolver finalizer covers failures before agent.run is reached."""
AgentRegistry.register_value("capturing", _CapturingToolAgent)
resource = MagicMock()
def _resolve(*args, **kwargs):
return ResolvedAgentTools(owned_resources=[resource])
monkeypatch.setattr("openjarvis.agents.executor.resolve_agent_tools", _resolve)
system = SimpleNamespace(
engine=FakeEngine([{"content": "unused"}]),
model="test-model",
memory_backend=None,
channel_backend=None,
tool_executor=None,
mcp_tools=[],
_mcp_clients=[],
config=None,
session_store=None,
)
manager = AgentManager(db_path=str(tmp_path / "agents.db"))
agent = manager.create_agent(
"cleanup",
agent_type="capturing",
config={"model": "test-model"},
)
monkeypatch.setattr(
manager,
"get_pending_messages",
MagicMock(side_effect=RuntimeError("pre-run setup failed")),
)
try:
with pytest.raises(RuntimeError, match="pre-run setup failed"):
AgentExecutor(manager, EventBus(), system=system)._invoke_agent(agent)
gc.collect()
resource.close.assert_called_once_with()
finally:
manager.close()
+15 -3
View File
@@ -21,7 +21,7 @@ def test_morning_digest_run(tmp_path):
mock_engine = MagicMock()
mock_engine.generate.return_value = {
"content": "Good morning sir. You have 3 emails and 2 meetings today.",
"content": "Good morning sir. AtlasDB 1.0 was released.",
"finish_reason": "stop",
"usage": {},
}
@@ -29,7 +29,7 @@ def test_morning_digest_run(tmp_path):
# Mock collect result
mock_collect_result = ToolResult(
tool_name="digest_collect",
content='=== MESSAGES ===\n[gmail] From: alice@co.com — "Budget" (1h ago)\n',
content="=== WORLD ===\n[hackernews] AtlasDB 1.0 Released — 241 points\n",
success=True,
metadata={"total_items": 2},
)
@@ -46,7 +46,9 @@ def test_morning_digest_run(tmp_path):
mock_engine,
"test-model",
tools=[],
persona="neutral",
persona="jarvis",
sections=["world"],
section_sources={"world": ["hackernews", "news_rss"]},
digest_store_path=str(tmp_path / "digest.db"),
)
@@ -61,6 +63,16 @@ def test_morning_digest_run(tmp_path):
assert "Good morning" in result.content
assert result.turns == 1
assert len(result.tool_results) == 2
assert set(result.metadata["sources_used"]) == {"hackernews", "news_rss"}
prompt = "\n".join(
message.text for message in mock_engine.generate.call_args.args[0]
).casefold()
assert "world —" in prompt
for forbidden in (
"messages —|calendar —|health —|rebuttal|dinner at|group chat|"
"slack|next meeting|readiness|hrv|weather"
).split("|"):
assert forbidden not in prompt
def test_load_persona():
+131
View File
@@ -0,0 +1,131 @@
"""Regression tests for proactive scheduling and notification setup."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from openjarvis.agents.proactive_agent import (
_PROACTIVE_CRON_PROMPT,
_build_notification_channel,
register_cron,
)
from openjarvis.core.registry import ChannelRegistry
from openjarvis.scheduler.scheduler import TaskScheduler
from openjarvis.scheduler.store import SchedulerStore
@pytest.fixture()
def scheduler(tmp_path):
store = SchedulerStore(tmp_path / "scheduler.db")
scheduler = TaskScheduler(store)
yield scheduler
scheduler.stop()
store.close()
def _register(scheduler, *, schedule="0 5 * * *", channel="telegram:123"):
return register_cron(
scheduler,
notification_channel_id=channel,
cron_expr=schedule,
hours_back=24,
timezone="UTC",
)
class TestRegisterCron:
def test_reuses_exact_task_and_cancels_duplicates(self, scheduler):
first = _register(scheduler)
duplicate = scheduler.create_task(
_PROACTIVE_CRON_PROMPT,
"cron",
"0 5 * * *",
agent="proactive",
metadata=first.metadata,
)
returned = _register(scheduler)
assert returned.id in {first.id, duplicate.id}
assert [task.id for task in scheduler.list_tasks(status="active")] == [
returned.id
]
cancelled_id = scheduler.list_tasks(status="cancelled")[0].id
assert cancelled_id == ({first.id, duplicate.id} - {returned.id}).pop()
def test_replaces_task_when_configuration_changes(self, scheduler):
old = _register(scheduler, schedule="0 5 * * *", channel="telegram:old")
new = _register(scheduler, schedule="0 7 * * *", channel="telegram:new")
assert new.id != old.id
assert new.schedule_value == "0 7 * * *"
assert new.metadata["notification_channel_id"] == "telegram:new"
assert scheduler.list_tasks(status="cancelled")[0].id == old.id
def test_preserves_pause_across_restart(self, scheduler):
paused = _register(scheduler)
scheduler.pause_task(paused.id)
returned = _register(scheduler, schedule="0 7 * * *")
assert returned.id == paused.id
assert returned.status == "paused"
assert scheduler.list_tasks(status="active") == []
def test_migrates_legacy_tasks_without_stable_key(self, scheduler):
legacy = scheduler.create_task(
_PROACTIVE_CRON_PROMPT,
"cron",
"0 5 * * *",
agent="proactive",
metadata={
"notification_channel_id": "telegram:123",
"hours_back": 24,
"timezone": "UTC",
},
)
current = _register(scheduler)
assert current.id != legacy.id
assert current.metadata["openjarvis_task_key"] == "proactive-daily"
assert scheduler.list_tasks(status="cancelled")[0].id == legacy.id
class TestNotificationChannel:
def test_telegram_is_configured_without_starting_polling(self):
class FakeTelegram:
def __init__(self, **kwargs):
self.kwargs = kwargs
self.connect = MagicMock()
config = MagicMock()
with (
patch.object(ChannelRegistry, "contains", return_value=True),
patch.object(ChannelRegistry, "get", return_value=FakeTelegram),
patch("openjarvis.core.config.load_config", return_value=config),
patch(
"openjarvis.system._channel_kwargs.build_channel_kwargs",
return_value={"bot_token": "configured-token"},
),
):
channel = _build_notification_channel("telegram:123")
assert channel.kwargs == {"bot_token": "configured-token"}
channel.connect.assert_not_called()
def test_non_telegram_channel_keeps_connect_lifecycle(self):
class FakeChannel:
def __init__(self, **kwargs):
self.connect = MagicMock()
with (
patch.object(ChannelRegistry, "contains", return_value=True),
patch.object(ChannelRegistry, "get", return_value=FakeChannel),
):
channel = _build_notification_channel("twilio:15551234567")
channel.connect.assert_called_once_with()

Some files were not shown because too many files have changed in this diff Show More