Compare commits

..
Author SHA1 Message Date
Jon Saad-FalconandClaude Opus 5 20aa08ef04 ci(desktop): preflight Apple notarization credentials before build (#724)
Notarization is the last thing tauri-action does, so any credential or
account-state fault surfaced ~10 minutes into the macOS job -- after the
Rust toolchain, npm install, two Ollama sidecar downloads and a universal
cargo build -- as one opaque line:

  failed to bundle project: failed codesign application: failed to
  notarize app: Error: HTTP status code: 403. ...

That message conflates three unrelated causes, and the signing step
succeeds in all of them, so the log actively misleads: the certificate is
clearly valid right up until the failure.

Add a read-only `notarytool history` call immediately after checkout. It
submits nothing and exercises the identical auth path, so all three
failures reach us in ~2s with the specific cause and fix named:

  401 invalid credentials  -> APPLE_PASSWORD is not an app-specific
                              password, or was minted under a different
                              Apple ID than APPLE_ID
  403 inaccessible team    -> APPLE_ID is not a member of APPLE_TEAM_ID
  403 required agreement   -> the Program License Agreement lapsed; only
                              the Account Holder can accept it

xcrun is preinstalled on macOS runners, hence placement before the
toolchain steps rather than beside "Configure Apple signing".

Skips cleanly when APPLE_CERTIFICATE is unset (unsigned builds never
notarize), mirroring the existing signing step, and errors when a
certificate is present but notarization secrets are missing -- previously
that combination signed successfully and then failed at the very end.
Transient network faults retry 3x; credential errors are deterministic
and exit on the first definitive answer.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 17:49:52 -07:00
github-actions[bot] 9a63561db8 chore: update clone traffic data [skip ci] 2026-08-11 07:02:06 +00:00
Elliot Slusky 7959285ad8 fix(cli): preserve extras during self-update (#718)
* fix(cli): preserve extras during self-update

* style(cli): format install detection
2026-08-10 17:22:29 -07:00
goatoush 26e1741059 fix(desktop): keep background streams out of active chat (#654) 2026-08-10 17:15:33 -07:00
kelliott-cloudandElliot Slusky 2ed885eb11 fix: never auto-select embed-only models for chat (#659)
* fix: never auto-select embed-only models for chat

Ollama lists nomic-embed-text alongside chat models. Auto-picking
models[0] / recommending the only available id selected the embedder
and every generation failed with HTTP 400 "does not support chat".

- Filter embed-only ids out of GET /v1/models (chat picker)
- Exclude them from /v1/recommended-model; return empty when none left
- Frontend setModels prefers chat models and clears a bad embed selection
- Regression tests for mixed, embed-only, and classifier cases

* fix: harden chat model capability filtering

---------

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-10 17:06:48 -07:00
07fcf35276 fix: use a signal-free liveness probe for the daemon on Windows (#681)
* fix: use a signal-free liveness probe for the daemon on Windows

`_read_pid()` probed the recorded pid with `os.kill(pid, 0)`. That is a
POSIX idiom: on Windows signal 0 is `CTRL_C_EVENT`, so the call routes to
`GenerateConsoleCtrlEvent` rather than testing for existence, and raises
`OSError` (WinError 87, "The parameter is incorrect") for any pid that is
not a live console process-group leader — which includes both dead pids
and the detached server `jarvis start` creates.

That single call produced three symptoms. `jarvis status` propagated the
error and crashed. `_read_pid`'s `except OSError` swallowed it for a
running server, so `status` and `stop` reported "not running" and deleted
a live pid file. And because the probe *sends* a console control event
rather than merely asking, running `status` against the daemon could
terminate it.

Add `_pid_alive()`, which opens a process handle and checks it on Windows
and keeps the signal-0 probe on POSIX, and use it for both liveness
checks. `SIGKILL` in the stop path is now reached on Windows for the
first time, so guard it — it is POSIX-only, and `SIGTERM` already maps to
`TerminateProcess` there.

The existing round-trip test mocked `os.kill` to succeed, which is why
this passed CI on Linux while failing on every Windows run. Point it at
the new seam and add `TestPidLiveness`, which exercises real pids so the
platform behaviour is actually covered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* style: format daemon tests with CI Ruff

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-10 16:56:53 -07:00
Haotian Zheng 4efdb07dae fix(cli): use UTF-8 for config files 2026-08-10 16:44:17 -07:00
Elliot Slusky 7feda3dad3 fix(cloud): isolate Gemini stream metadata 2026-08-10 16:32:14 -07:00
Elliot Slusky f08ad574d3 fix(cloud): handle parallel Gemini tool calls 2026-08-10 16:32:14 -07:00
Dustin Zander 1bfc25a860 fix: preserve Gemini tool calls while streaming 2026-08-10 16:32:14 -07:00
Elliot Slusky 063dd8ea75 fix: unify managed-agent tool resolution (#705)
* fix: unify managed-agent tool resolution

* fix: harden managed-agent tool lifecycle
2026-08-10 15:09:41 -07:00
Elliot Slusky 6af9317556 fix: route configured LiteLLM models by engine ownership (#714) 2026-08-10 12:51:16 -07:00
Elliot Slusky 410562409d fix: make websocket bridge race cancellation-safe 2026-08-10 11:04:12 -07:00
Ari 9498adc7c4 fix: close ws_bridge send loop on client disconnect
Previously the ws_bridge send loop blocked forever on queue.get(),
never learning the client left. When the server service stopped,
uvicorn waited for open WebSocket tasks until systemd SIGKILLed
after TimeoutStopSec=90s. Now each iteration races recv+send; a
completed receive means the client disconnected => break the loop.
2026-08-10 11:04:12 -07:00
github-actions[bot] ebf370595d chore: update clone traffic data [skip ci] 2026-08-10 07:26:44 +00:00
Elliot Slusky bcdbf13d02 test(tools): cover eager deep-research registration 2026-08-09 22:59:45 -07:00
Ari 3dc621618f fix(tools): eager-import scan_chunks and knowledge_sql at package load
Every other built-in tool is imported here specifically to fire its
@ToolRegistry.register() decorator at package-load time; these two
were missing, so their test_registered tests only passed when some
unrelated test (via agent_manager_routes.py, channels_cmd.py, or
deep_research_setup_cmd.py) happened to import the module first in
the same process. Under pytest-xdist that's worker-distribution
dependent, so adding an unrelated test file could flip either test
from pass to fail.
2026-08-09 22:59:45 -07:00
github-actions[bot] fd0b60fefc chore: update clone traffic data [skip ci] 2026-08-09 06:51:50 +00:00
github-actions[bot] 95a9857984 chore: update clone traffic data [skip ci] 2026-08-08 06:46:50 +00:00
github-actions[bot] f9c89308fc chore: update clone traffic data [skip ci] 2026-08-07 07:10:58 +00:00
Elliot Slusky 65d08e9d94 Fix proactive cron reconciliation and notifications 2026-08-06 13:40:00 -07:00
Loma 45717780fa Fix proactive agent cron duplicating on every server restart
register_cron() ran unconditionally on every 'jarvis serve' startup and
create_task() persists to scheduler.db, so each restart added another
copy of the daily proactive cron. On a real install 68 duplicates
accumulated; when due they fired back-to-back and monopolized the
single-slot local inference queue, stalling interactive chat.

register_cron() is now idempotent: an existing active task is reused and
surplus duplicates are cancelled. Also stop the notification channel
from calling connect() — a second getUpdates poll loop on the same bot
token makes Telegram return Conflict and kills the main listener.
2026-08-06 13:40:00 -07:00
goatoush 9da7c30880 Fixing text wrap for agent creator tool selector (#655) 2026-08-06 11:23:27 -07:00
github-actions[bot] 98e791f258 chore: update clone traffic data [skip ci] 2026-08-06 08:31:13 +00:00
Elliot Slusky b9e0928aef fix(telemetry): record cloud inference cost (#704) 2026-08-05 20:51:06 -07:00
Elliot Slusky 652a522e50 fix(config): deserialize skill source tables (#703) 2026-08-05 20:50:42 -07:00
github-actions[bot] ce1a9ce133 chore: update clone traffic data [skip ci] 2026-08-05 08:31:05 +00:00
github-actions[bot] ae45a4f67c chore: update clone traffic data [skip ci] 2026-08-04 08:31:51 +00:00
github-actions[bot] 697eed23d4 chore: update clone traffic data [skip ci] 2026-08-03 09:38:25 +00:00
github-actions[bot] 100595f8aa chore: update clone traffic data [skip ci] 2026-08-02 08:20:07 +00:00
github-actions[bot] dd03a55028 chore: update clone traffic data [skip ci] 2026-08-01 08:15:52 +00:00
github-actions[bot] a72218f99f chore: update clone traffic data [skip ci] 2026-07-31 08:45:27 +00:00
github-actions[bot] eaa76032d5 chore: update clone traffic data [skip ci] 2026-07-30 08:21:39 +00:00
CurryrajandClaude Opus 5 ed01ab8c8d fix: proxy WebSocket upgrades to the API in dev (#692)
The Vite dev proxy forwards `/v1` as plain HTTP with no `ws: true`, so the
WebSocket upgrade for `/v1/agents/events` is never proxied. The socket does not
open, does not error, and does not close — it just sits silent — so every live
agent view is empty under `npm run dev` while working in a production build,
where the frontend is served from the same origin as the API.

That covers the per-agent live trace on the Agents page, which subscribes via
`useAgentEvents` (`frontend/src/lib/useAgentEvents.ts`).

The silence is what makes it costly: with no error to see, it reads as "no
events are being emitted" rather than "the transport never connected", so the
search starts on the server side.

Verified on Windows 11 with `jarvis start` running: before, a
`new WebSocket('ws://localhost:5173/v1/agents/events')` from the dev page never
fired open, error or close within 6s. After, it opens, and a real agent tick
delivers 8 events (agent_tick_start, inference_start/end, tool_call_start/end,
agent_tick_end).

`changeOrigin` is set alongside so the upgrade request carries the target's
host, which some setups require when the API is not on localhost.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:04:41 -07:00
CurryrajandClaude Opus 5 a65f663d2e fix: stop SessionStore treating ":memory:" as a file path (#684)
``SessionStore.__init__`` passed its ``db_path`` straight to
``secure_create()``, which touches the path and chmods it. ``:memory:`` is
a SQLite sentinel, not a filename, so this tried to create a file literally
named ``:memory:``.

On Windows ``:`` is illegal in a filename, so construction raised
``OSError: [Errno 22] Invalid argument: ':memory:'`` and the two
``tests/server/test_channel_bridge_deep_research.py`` tests failed there.
Elsewhere it succeeds and is merely wrong: it leaves a stray ``:memory:``
file in the working directory, and because ``Path(":memory:").parent`` is
``.``, ``secure_mkdir`` chmods the working directory itself to 0o700.

``KnowledgeStore``, ``TelemetryStore`` and ``TraceStore`` already guard this
exact case; ``SessionStore`` was the one store missing the check. Apply the
same guard, with the same comment.

Adds two regression tests: one that an in-memory store is usable, one that
constructing it creates no file. The second fails on every platform without
the fix, so the bug cannot silently return on Linux or macOS.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 10:44:18 -07:00
Elliot Slusky c6382f2473 fix web search setup in browser quickstart (#691) 2026-07-29 08:38:04 -07:00
github-actions[bot] 2922a2b154 chore: update clone traffic data [skip ci] 2026-07-29 08:35:42 +00:00
Elliot Slusky 403dec8e98 Merge pull request #687 from Curryraj/fix/windows-daemon-detach
fix: detach the daemon from its console on Windows
2026-07-28 23:34:01 -07:00
Elliot Slusky 2c7cf6118c test: make Windows daemon test cross-platform 2026-07-28 23:27:25 -07:00
CurryrajandClaude Opus 5 2bdd860b54 test: tolerate a 503 memory backend in the route-wiring tests (#685)
``TestMemoryRoutes.test_search`` and ``test_stats`` assert the status code
is in ``(200, 500)``. That list dates from the initial commit; #527 later
made the memory routes raise 503 when the native ``openjarvis_rust``
extension is missing, so both tests now fail on any checkout where the
extension has not been built — which is every contributor who has not run
``maturin develop``.

The failure is spurious: these two tests only check that the routes are
wired up, and their own comment ("May fail if SQLite not set up, that's
ok") says an unavailable backend is tolerated. 503 is exactly that case,
and it is already asserted deliberately in ``TestMemoryRustMissing``
directly below.

Add 503 to the tolerated set via a named constant, so the reason is stated
once rather than repeated as a bare literal.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 23:17:25 -07:00
Robby Manihani 81f1ffbb4f docs: document how to give an agent OS-level access (#689)
Someone asked in Discord how to give Jarvis access to their whole
machine and hit a confusing failure. With no ~/.openjarvis/config.toml,
tools.enabled and agent.tools both come back empty, and SystemBuilder
builds the agent with no tools at all. It reads like a permissions
problem but it's just missing config, and nothing in the docs points
you anywhere useful.

Adds docs/user-guide/system-access.md, covering the empty tool list as
the usual cause, what shell_exec and the file tools actually reach,
which entry points prompt for confirmation and which quietly
auto-approve, Full Disk Access on macOS and which process needs it, and
the fact that there's no computer use at all, so Accessibility and
Screen Recording grants buy you nothing on their own.

Also adds a full-system-access.toml example to copy from.

Fixes two config tables that describe security.enforce_tool_confirmation
as requiring confirmation before tools run. The loader accepts the key
but nothing on the execution path reads it, so anyone setting it gets
assurance they don't actually have.
2026-07-28 20:11:27 -07:00
github-actions[bot] 93fc7b9e77 chore: update clone traffic data [skip ci] 2026-07-28 08:29:18 +00:00
Jaiydaan RajandClaude Opus 5 9fc5b875d1 fix: detach the daemon from its console on Windows
``jarvis start`` spawned the server with ``start_new_session=True``. That is
POSIX-only — CPython's Windows ``_execute_child`` names the parameter
``unused_start_new_session`` and ignores it — so on Windows the server
inherited the launching console instead of detaching from it.

Closing that console, or logging off, therefore delivered CTRL_CLOSE_EVENT
to the server. Observed in the wild as the daemon dying overnight, with

    forrtl: error (200): program aborting due to window-CLOSE event

in server.log (the Fortran runtime under NumPy handles the event and
aborts). ``jarvis start`` looked like it worked: it printed a PID, wrote the
pid file and exited 0, and the server ran for as long as the console stayed
open. Registered as a log-on scheduled task, this means the machine comes
back up with no backend.

Pass DETACHED_PROCESS on Windows so the child gets no console at all, plus
CREATE_NEW_PROCESS_GROUP so a Ctrl-C in the parent console cannot reach it.
POSIX keeps start_new_session.

Verified by attaching to each spawned process with AttachConsole():
start_new_session=True attaches successfully (the child shares a console);
DETACHED_PROCESS fails with ERROR_INVALID_HANDLE (no console exists).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 13:17:06 +08:00
github-actions[bot] 08279e6b99 chore: update clone traffic data [skip ci] 2026-07-27 09:46:21 +00:00
github-actions[bot] a7c31e89b4 chore: update clone traffic data [skip ci] 2026-07-26 08:22:36 +00:00
github-actions[bot] 04014c658a chore: update clone traffic data [skip ci] 2026-07-25 08:02:21 +00:00
github-actions[bot] c1238d3e7e chore: update clone traffic data [skip ci] 2026-07-24 08:23:24 +00:00
github-actions[bot] 687e80a55a chore: update clone traffic data [skip ci] 2026-07-23 08:25:07 +00:00
github-actions[bot] b90fd01af2 chore: update clone traffic data [skip ci] 2026-07-22 08:24:05 +00:00
github-actions[bot] bbe7df7d33 chore: update clone traffic data [skip ci] 2026-07-21 08:23:44 +00:00
Arush WadhawanandElliot Slusky 9685b9b78f fix(telemetry): enable WAL and batch writes in TelemetryStore to avoid SQLITE_BUSY under concurrency (#597)
TelemetryStore opened SQLite without WAL, so concurrent readers (server, aggregator, dashboard) hitting the database under inference load raised SQLITE_BUSY, and every insert committed immediately, paying fsync on each record.

- Enable PRAGMA journal_mode=WAL with synchronous=NORMAL and busy_timeout=5000, matching TraceStore.
- Batch inserts in memory under a lock and flush via executemany() when a batch reaches batch_size (default 50), when a batch goes stale, on any read through the store, and on close().
- Run a background flusher thread (default 5s interval) so a partial batch written just before traffic stops still becomes visible to other connections; close() stops the thread with an ordering that prevents touching a closed connection.
- Tests cover batching deferral, read-triggered flushes, stale-batch flushes, and close() behavior.

Fixes #560

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-07-20 14:11:10 -07:00
goatoush aa2d127de4 fix(frontend): remove jitter when scrolling up during chat autoscroll (#646)
The chat area re-armed autoscroll whenever the user was within 100px of the bottom, so scrolling up during a streaming response fought the incoming content ticks and produced jitter.

Autoscroll now disengages on any upward scroll (direction-based, no distance threshold), re-engages when scrolled back within 2px of the bottom (tolerating sub-pixel rounding at fractional zoom levels, where the at-bottom residual can reach 1px), and ignores sub-1px upward movement so macOS elastic-bounce settling does not disengage it. Sending a message pins the view to the bottom even if the user had scrolled up to read earlier messages.
2026-07-20 13:45:12 -07:00
github-actions[bot] 87f6238338 chore: update clone traffic data [skip ci] 2026-07-20 08:53:13 +00:00
github-actions[bot] 452bcc38cf chore: update clone traffic data [skip ci] 2026-07-19 08:12:04 +00:00
goatoush b35a4c8113 fix(desktop): preserve active chat when switching models (#648)
Switching models from the command palette called createConversation() on every change, creating a persisted empty "New chat" entry and pulling the user out of their active conversation. Because updateLastAssistant writes the visible messages array without checking the active conversation, a mid-stream switch could also clobber the new chat's view with the old conversation's messages.

Remove the conversation-creation side effect. Model switching now preserves the active chat (matching the pull-completion and delete-fallback paths, which already switched silently); the next request uses the newly selected model with the current conversation context. Preloading, loading state, and logging are unchanged.
2026-07-18 12:51:38 -07:00
github-actions[bot] f001e3b0ca chore: update clone traffic data [skip ci] 2026-07-18 07:44:44 +00:00
Cesar Schneider b6dba93ae5 fix: make pytest suite hermetic against local dev-machine state (#647)
Two test-isolation fixes: (1) an autouse conftest fixture sets OPENJARVIS_NO_UPDATE_CHECK=1 so the CLI's PyPI update-check banner (stderr, merged into CliRunner output) can never pollute JSON/CSV-parsing CLI tests on local runs; CI was already covered by CI=true. (2) test_dense.py's Ollama skip-guard now queries /api/tags and requires nomic-embed-text to be pulled instead of a bare TCP connect, so machines running Ollama without the embed model skip instead of erroring. The probe normalizes all documented OLLAMA_HOST forms (full URL, host:port, bare host) and the Ollama-backed tests construct DenseMemory against that same endpoint rather than the embedder's hard-coded localhost default, with unit tests covering the probe. Related: #645.
2026-07-17 13:21:36 -07:00
github-actions[bot] 3000116d18 chore: update clone traffic data [skip ci] 2026-07-17 08:05:34 +00:00
Elliot Slusky 9db21d37ef style: apply Ruff formatting to recently merged tests (#644)
Fix the Ruff formatter check on main by formatting tests changed in #639 and #640 with the repository's pinned Ruff 0.15.1. No behavior change.
2026-07-16 18:13:36 -07:00
Elliot Slusky 95480363b7 style(skills): wrap importer manifest parse call (#643)
Fix the Ruff E501 failure on main introduced during the #639 fix-up. The call was 89 characters against the repository's 88-character limit. No behavior change.
2026-07-16 18:07:44 -07:00
CurryrajandElliot Slusky 4419b76412 fix: catch ImportError in git tools, fall back to CLI when Rust ext missing (#636)
get_rust_module() was called outside the try block in GitStatusTool/GitDiffTool/GitLogTool.execute(), so on installs without the compiled openjarvis-rust extension (e.g. plain pip installs, where openjarvis-rust is a uv-only group since #624) the ImportError escaped uncaught instead of degrading. Move the call inside try and fall back to the git CLI via the existing _run_git helper on ImportError, matching the fallback git_log already had. Adds regression tests covering the fallback path for all three tools.

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-07-16 17:59:25 -07:00
Jon Saad-FalconandClaude Opus 4.8 99bbc2054a Add arXiv badge to README (#642)
Add a red arXiv badge linking to the OpenJarvis paper (2605.17172) as
the first item in the header badge row, matching the style used on the
Intelligence-Per-Watt repo.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 17:42:20 -07:00
Jon Saad-FalconandClaude Opus 4.8 d5d8fddc94 Fix leaderboard savings lookup after provider-key rename (#635)
PR #634 renamed the Anthropic cost-comparison provider key
`claude-opus-4.6` -> `claude-fable-5` but missed one consumer:
App.tsx looks up the Anthropic entry by that key to compute the
`dollar_savings` value submitted to the leaderboard. After the rename
`per_provider.find(p => p.provider === 'claude-opus-4.6')` returned
undefined, so this path silently submitted dollar_savings = 0.

Point the lookup at the new key.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 17:33:38 -07:00
jaaaaorr06 cadb3e2ae6 feat(skills): enforce capability/trust-tier checks at install and run time (#639)
Wires the previously-dead skills/security.py checks into two places. SkillImporter.import_skill() classifies trust tier before writing to disk, refuses unreviewed skills requesting dangerous capabilities unless confirmed (confirm_dangerous=True, or --yes-dangerous on skill install/sync), and persists tier/capabilities to the .source sidecar. SkillExecutor.run() gains opt-in capability enforcement: allowed_capabilities=None (the default, used by all existing call sites) means no policy; passing a set blocks skills whose required_capabilities are not covered before any step runs. Bulk sync reports refused skills instead of silently skipping them. Both enforcement points are covered by tests.
2026-07-16 17:23:27 -07:00
Trevor Willard 7dc904c1b2 fix: wire persona files into default chat endpoint, fix FTS5 apostrophe crash (#637)
Two fixes: (1) /v1/chat/completions now builds its injected system prompt via SystemPromptBuilder, so SOUL.md/MEMORY.md/USER.md persona files apply to the OpenAI-compatible endpoint exactly as they do on the managed-agent path; injection still only happens when the client omits a system message. (2) FTS5 query tokens are now split on any non-alphanumeric character, so apostrophes (user's) and quotes can no longer reach the MATCH string unescaped; all-punctuation queries return empty instead of erroring.
2026-07-16 17:10:51 -07:00
ScottyAmr d9725fbb6a fix(speech): close temp file before transcribing to fix Windows EACCES in faster-whisper backend (#638)
faster-whisper's transcribe() was handed the path of a still-open NamedTemporaryFile; on Windows the open handle is exclusive, so PyAV's reopen failed with EACCES and local STT was broken. Switch to delete=False, close before transcribing, and unlink in a finally. The write is wrapped in the file's context manager so the handle closes even if write() raises, and unlink failures are logged at debug.
2026-07-16 17:09:45 -07:00
Syed Osama Ali ShahandElliot Slusky 23f04264f9 fix(knowledge_sql): match write keywords on word boundaries (allow valid SELECTs) (#640)
* fix(knowledge_sql): match write keywords on word boundaries

The read-only guard rejected a query if any of DROP/DELETE/INSERT/UPDATE/
ALTER/CREATE/ATTACH appeared as a bare substring of the uppercased text. That
wrongly blocks valid SELECTs whose column/alias/literal merely contains one --
e.g. "deleted_at" (DELETE), "created_at" (CREATE), "updated_content" (UPDATE).
The knowledge_chunks table actually has deleted_at/created_at columns and the
store's own retrieval filters on "WHERE deleted_at IS NULL", so realistic
read queries were refused. Match on word boundaries with a compiled regex,
mirroring the sibling tool db_query.py. Add a regression test.

* fix(knowledge_sql): ignore string literals in keyword scan, broaden error handling

- Strip single-quoted literals before the forbidden-keyword scan so
  SELECTs whose data merely mentions a write keyword (e.g. LIKE
  '%delete%') are not rejected.
- Catch sqlite3.Error instead of only OperationalError so multi-
  statement strings return a failed ToolResult instead of raising.
- Document created_at/deleted_at in the tool's schema description.

---------

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-07-16 16:36:31 -07:00
github-actions[bot] 8b59eb87e0 chore: update clone traffic data [skip ci] 2026-07-16 08:08:14 +00:00
github-actions[bot] 2e68e227b7 chore: update clone traffic data [skip ci] 2026-07-15 08:04:26 +00:00
github-actions[bot] fc98614437 chore: update clone traffic data [skip ci] 2026-07-14 07:59:08 +00:00
github-actions[bot] b1c5aba6fd chore: update clone traffic data [skip ci] 2026-07-13 09:20:55 +00:00
Jon Saad-FalconandClaude Opus 4.8 6240c59ca3 Update cost-comparison models: GPT-5.6 Sol, Claude Fable 5 (#634)
Refresh the cost-comparison / savings surfaces to current frontier cloud
pricing (per 1M tokens):

- OpenAI:    GPT-5.3 ($2/$10)        -> GPT-5.6 Sol ($5/$30)
- Anthropic: Claude Opus 4.6 ($5/$25) -> Claude Fable 5 ($10/$50)
- Google:    Gemini 3.1 Pro ($2/$12)  -> unchanged

Internal provider keys are renamed in lockstep (gpt-5.3 -> gpt-5.6-sol,
claude-opus-4.6 -> claude-fable-5) across the canonical CLOUD_PRICING
dict, the two server-rendered HTML pages, and the frontend color/label
maps so backend, dashboard, and UI stay consistent. Energy/FLOPs
metadata is carried over unchanged. Model catalog and eval configs are
untouched (real model/benchmark entries, not the cost comparison).

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 14:48:22 -07:00
github-actions[bot] 9f3c7fd086 chore: update clone traffic data [skip ci] 2026-07-12 08:12:51 +00:00
github-actions[bot] 657c8dd26b chore: update clone traffic data [skip ci] 2026-07-11 07:51:15 +00:00
github-actions[bot] 4ef296e9d0 chore: update clone traffic data [skip ci] 2026-07-10 09:30:29 +00:00
github-actions[bot] d5d06ca0e5 chore: update clone traffic data [skip ci] 2026-07-09 09:39:27 +00:00
github-actions[bot] 213ee4ff7e chore: update clone traffic data [skip ci] 2026-07-08 08:23:57 +00:00
Jon Saad-FalconandClaude Opus 4.8 215ab76e5f docs: point the Project Site link to openjarvis.stanford.edu (#628)
The project's canonical site moved from the Scaling Intelligence Lab blog
(scalingintelligence.stanford.edu/blogs/openjarvis/) to
https://openjarvis.stanford.edu/. Update every reference to that URL:

- README: the "Project" badge and the "Project Site" link
- docs/index.md: the research write-up link
- desktop Settings: the "Project site" link (SettingsPage.tsx)
- the Twitter-bot operator prompt

The bare Scaling Intelligence Lab homepage links (the lab itself, not the
project site) are intentionally left unchanged, as are the github.io
documentation and installer URLs.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 13:27:32 -07:00
github-actions[bot] 0812ee0701 chore: update clone traffic data [skip ci] 2026-07-07 09:36:44 +00:00
talyaseenandClaude Opus 4.8 0b140110d8 Make OpenAI-compat and Ollama engine streaming truly async (#626)
The OpenAI-compat and Ollama engines exposed stream()/stream_full() as async def but iterated a synchronous httpx.Client.iter_lines() internally, blocking the single event loop on every inter-token read (serializing concurrent chats; one wedged upstream read froze the whole API). Convert both to a shared AsyncHTTPEngineMixin using httpx.AsyncClient + aiter_lines() with a per-event-loop pooled client and the configured timeout applied; map mid-stream transport errors (RemoteProtocolError/ReadError) to EngineConnectionError via a deliberately narrow set that keeps CancelledError/GeneratorExit propagating; handle non-2xx explicitly (incl. 3xx and a typed EngineContextLengthError for context-window overflow 400s); switch litellm streaming to acompletion; and offload the blocking non-streaming handlers and websocket generate() to asyncio.to_thread. No public API change. Strong MockTransport-based tests, including a pin that the async path never touches the sync client. Complements #618.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 14:10:22 -07:00
github-actions[bot] 2623f9e0f4 chore: update clone traffic data [skip ci] 2026-07-06 10:14:22 +00:00
github-actions[bot] d454c41500 chore: update clone traffic data [skip ci] 2026-07-05 08:49:16 +00:00
github-actions[bot] e3fb816d12 chore: update clone traffic data [skip ci] 2026-07-04 08:33:11 +00:00
github-actions[bot] 3486f27357 chore: update clone traffic data [skip ci] 2026-07-03 08:58:35 +00:00
Elliot SluskyandClaude Opus 4.8 928776a71c ci: enforce ruff format in CI, add Makefile matching the CI test lane (#625)
CI's lint job ran ruff check but never ruff format --check, letting format drift land silently (79 files had drifted from the pinned ruff 0.15.1). Add the ruff format --check step to ci.yml, reformat the 79 drifted files with the pinned ruff (mechanical only — verified AST-identical to before across all files, no logic changes), and add a Makefile whose test target mirrors the actual CI lane.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 14:49:18 -07:00
github-actions[bot] 2c2a4b6ae4 chore: update clone traffic data [skip ci] 2026-07-02 07:10:21 +00:00
26bc7efb09 fix(packaging): make openjarvis-rust a uv-only group to unblock pip install openjarvis[desktop] (#624)
* fix(packaging): make openjarvis-rust a uv-only dependency group (unblock pip[desktop])

#615 added openjarvis-rust to the published `desktop` extra so
`uv sync --extra desktop` builds the native PyO3 extension for the desktop
app. But openjarvis-rust is not on PyPI and `[tool.uv.sources]` is stripped
from published wheel metadata, so `pip install openjarvis[desktop]` from PyPI
failed at install trying to resolve openjarvis-rust from PyPI (#584).

Move openjarvis-rust into a uv `desktop-native` dependency group (PEP 735 —
excluded from wheel metadata) and sync it in the desktop app via
`uv sync --group desktop-native`. The extension is still built from the local
path source; only the published metadata changes.

Verified: the built wheel no longer lists openjarvis-rust in any Requires-Dist
(nowhere in the metadata), and uv.lock still resolves it from the local path
source under the group. Adds tests/deployment/test_packaging.py to guard the
split (not in the published extra, present in the group, path source, and the
desktop app syncs the group).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(packaging): sync native group in install paths

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-07-01 13:02:13 -07:00
github-actions[bot] f3954e087a chore: update clone traffic data [skip ci] 2026-07-01 07:33:10 +00:00
Elliot SluskyandClaude Opus 4.8 d865b4bed4 Fix blocking async server handlers (#618)
Closes #219. Replace synchronous httpx calls in async SendBlue and model-management handlers with awaited httpx.AsyncClient (context-managed close); run Whisper transcription and engine.list_models via asyncio.to_thread so they don't block the event loop; and harden TelemetryStore/aggregator SQLite for concurrency (WAL, synchronous=NORMAL, busy_timeout=5000, plus a write-serializing lock on the shared connection). Adds async-usage assertions and a real 8-thread concurrent-write test. Related: #570 (async httpx, different issue #559).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:37:20 -07:00
Elliot SluskyandClaude Opus 4.8 c686517cc7 Fix upcoming Google Calendar event retrieval (#617)
Closes #388. Parse Google Calendar all-day events from start.date instead of stamping them with the current time; treat generic next/upcoming calendar-event queries as gcalendar timeline requests that return nearest-future events first (UTC-normalized, instant-aware comparison that handles tz offsets and all-day events); and update the research planner guidance to route such queries with sources=[gcalendar] + a today-onward time_range. Real in-memory KnowledgeStore integration tests cover ordering, tz normalization, all-day inclusion, and source narrowing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 14:18:09 -07:00
Elliot SluskyandClaude Opus 4.8 904133cb25 fix(research): respect configured engine for Deep Research (#616)
Fixes #575. Web Deep Research was hardcoded to OllamaEngine + DEFAULT_PLANNER_MODEL, ignoring the user's configured/active engine and model. Resolve the planner from [deep_research] override -> live app chat engine + selected model -> config defaults -> legacy Ollama, pass the chat picker's model from the frontend into /api/research, record the actual planner engine in telemetry, and refuse to silently fall back to a different engine (raise an actionable error instead). Adds config support and focused tests for resolution and the route. Related: #576 (duplicate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 13:46:18 -07:00
Elliot SluskyandClaude Opus 4.8 299dee1f40 fix(desktop): verify Rust extension before server startup (#615)
Fixes #505. Make the desktop backend resilient to a missing/unbuilt openjarvis_rust extension: declare openjarvis-rust as a uv-managed desktop path dependency so 'uv sync --extra desktop' owns the PyO3 build (instead of pruning an undeclared package); add ~/.cargo/bin to the subprocess PATH and fail early with Rust / Windows Build Tools guidance when the toolchain is missing; verify 'import openjarvis_rust' before starting jarvis serve; and add a TCP bind preflight for port 8000 to catch non-HTTP listeners the /health probe can't classify. Includes the uv.lock entry for the new path dependency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 13:24:24 -07:00
github-actions[bot] 44ff286005 chore: update clone traffic data [skip ci] 2026-06-30 07:21:15 +00:00
Gilbert Barajas be51eb8684 docs(user-guide): document SOUL/MEMORY/USER.md persona files (#604) (#610)
* docs(user-guide): document SOUL/MEMORY/USER.md persona files (#604)

The persistent-memory showcase links to the User Guide: Agents page for how
SOUL.md / MEMORY.md / USER.md are loaded at conversation start, but that page
never covered them (site search for the filenames returns nothing).

Add a "Persistent Persona" section to user-guide/agents.md: the three files and
what each holds, where they live (config dir + [memory_files]), how they load
(after the agent template, cached per conversation, per-section truncation),
named personas (--persona / personas/<name>/), and editing by hand or via the
memory_manage / user_profile_manage tools. Cross-links the distinct retrieval
memory backend to resolve the reporter's confusion.

Closes #604

* docs(user-guide): clarify memory_manage/user_profile_manage target the default MEMORY.md/USER.md
2026-06-29 17:38:22 -07:00
Elliot Slusky 420908401c fix(engine): count tool call payloads in token estimates (#614)
Closes #608. Make Message.content officially Optional (str | None) with a Message.text accessor that treats None as empty, and count tool-call IDs/names/arguments, tool-result IDs, and reasoning/thinking metadata in estimate_prompt_tokens — all are replayed into later prompt turns, so they belong in the estimate. Extends estimator and message-type regression tests.
2026-06-29 17:34:10 -07:00
Elliot Slusky b70be55681 fix(openhands): handle none content in token estimates (#612)
Closes #607. Assistant tool-call turns can carry content=None, which crashed token estimation (len(m.content)) and think-tag stripping. Normalize with 'content or ""', route native OpenHands truncation through the shared estimate_prompt_tokens, and add tests for the estimator, the truncation helper, and an end-to-end tool-call run with None content.
2026-06-29 16:51:52 -07:00
Elliot Slusky a0187e40e6 Fix desktop startup fallback to installed Ollama models (#611)
Addresses #605. Prefer an already-installed Ollama model before attempting a startup download (matching the requested tag, else a preferred non-embedding installed model); fall back through installed -> FALLBACK_MODEL -> error, reusing installed models at each failure point; persist the resolved model only for first-run/default so an explicit user choice is never overwritten. Refactors the model logic into testable helpers with unit coverage.
2026-06-29 16:51:49 -07:00
Jon Saad-FalconandClaude Opus 4.8 d32f20f9b3 fix(desktop): align @tauri-apps npm packages with the 2.11 Rust crate (#613)
Desktop release builds failed on all platforms with 'Found version mismatched Tauri packages' because @tauri-apps/api and @tauri-apps/cli were pinned at 2.10.1 while the tauri Rust crate resolved to 2.11.3. Bump both npm packages to the 2.11 line (api 2.11.1, cli 2.11.4) so they share the crate's major.minor. Plugins were already aligned.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 16:51:46 -07:00
github-actions[bot] 0b552cbcb5 chore: update clone traffic data [skip ci] 2026-06-29 07:46:59 +00:00
github-actions[bot] 19fd3c8d2b chore: update clone traffic data [skip ci] 2026-06-28 07:24:04 +00:00
Jon Saad-FalconandClaude Opus 4.8 1fa80d8ecd fix(docs): wire the savings leaderboard's Supabase anon key into the docs build (#596)
The docs-site leaderboard (docs/javascripts/leaderboard.js) reads the public
Supabase anon key from window.OPENJARVIS_SUPABASE_ANON_KEY, but nothing set it,
so the published leaderboard always rendered "Leaderboard not configured yet".

Add a generated config file (leaderboard-config.js) loaded before
leaderboard.js that supplies the global, and inject its value at docs-build
time from the existing VITE_SUPABASE_ANON_KEY repo secret. The committed
default is empty, so local `mkdocs build` and fork PRs (no secret) degrade
gracefully. The anon key is public by design (Supabase RLS protects the data).

- docs/javascripts/leaderboard-config.js: empty-default global declaration.
- mkdocs.yml: load leaderboard-config.js before leaderboard.js.
- docs.yml: write the config from the secret (read via env, JSON-encoded into a
  JS string literal to avoid injection) before `mkdocs build`.
- tests/deployment/test_docs_leaderboard.py: guard the wiring + load order.

Verified with a local `mkdocs build`: the generated config ships in site/ and
loads before leaderboard.js.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:49:18 -07:00
github-actions[bot] b3f90691bf chore: update clone traffic data [skip ci] 2026-06-27 07:07:41 +00:00
github-actions[bot] 4ebf0839e7 chore: update clone traffic data [skip ci] 2026-06-26 07:21:03 +00:00
github-actions[bot] b1e93d4ed0 chore: update clone traffic data [skip ci] 2026-06-25 07:14:52 +00:00
Elliot SluskyandClaude Opus 4.8 eb2b612c7c fix(memory): address service follow-ups (#591)
Closes #582. Route fact-store construction through a new FactStoreRegistry (local backend registered by default); align the default facts path with get_config_dir(); wire completed chat exchanges (streamed and non-streamed) through the EventBus so the memory service captures them consistently; reload the local fact store from disk before operations so external clears don't resurrect stale facts; make the affected config/persona/memory/CLI/route tests hermetic; and refresh uv.lock with the current resolver (locks pytest-xdist + transitive deps, drops py3.14 artifacts since the project constrains Python <3.14).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 19:58:49 -07:00
Elliot SluskyandClaude Opus 4.8 560ec860df fix(docker): build native Rust extension into images (#590)
Build and install the mandatory openjarvis_rust wheel in the CPU, NVIDIA, ROCm, and sandbox Docker images. Rust 1.88 (matching the workspace MSRV / rust-toolchain.toml) and maturin are installed only in the builder stage, the module's import is verified during the build, and maturin is removed before the runtime artifacts are copied so build tooling never ships. The frontend leaderboard anon key is an optional empty-by-default build arg (post-#589), so default images cleanly disable the leaderboard. Adds static deployment coverage for the native build path. Closes #584.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 15:27:17 -07:00
Jon Saad-FalconandClaude Opus 4.8 e7c46c1985 fix(frontend): make Supabase anon key optional to unblock PyPI publishing (#589)
PyPI publishing had been broken since v1.0.3.dev851: #587 made VITE_SUPABASE_ANON_KEY a hard build-time requirement, but no such secret exists, so the frontend build aborted every publish run before the PyPI upload. Decouple package buildability from the leaderboard credential: a missing anon key now disables the savings leaderboard at runtime instead of failing the build, and auto-enables when the secret is provided. Verified: npm run build with the key unset succeeds; tsc + vitest pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:15:06 -07:00
github-actions[bot] 00d1e39b6d chore: update clone traffic data [skip ci] 2026-06-24 07:14:44 +00:00
Elliot SluskyandClaude Opus 4.8 843375d6ef Fix Supabase frontend build env for release builds (#588)
Follow-up to #587. Pass VITE_SUPABASE_ANON_KEY into the frontend builds of both release paths: the PyPI publish workflow (wheel-bundled frontend) and the desktop tauri-action build (npm run build:tauri -> vite build). Kept strict: a missing/empty secret fails the release by design rather than shipping a placeholder key. Requires the VITE_SUPABASE_ANON_KEY repo secret to be set for releases to succeed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:32:13 -07:00
Elliot SluskyandClaude Opus 4.8 8d33cb58fa Fix secure cloud key storage and Supabase key config (#587)
Route desktop cloud-key saves/status through the OS credential store (keyring with per-platform native backends: apple-native / windows-native / sync-secret-service), migrate the legacy plaintext ~/.openjarvis/cloud-keys.env into it, remove browser localStorage persistence of provider keys, and push key updates to the running server via /v1/cloud/reload (legacy env-file fallback retained). Remove hardcoded Supabase anon JWTs from frontend/docs source and make VITE_SUPABASE_ANON_KEY a required build var. Adds libdbus-1-dev to the Linux desktop build and a CI build var. Closes #220.

NOTE (post-merge follow-ups, not covered by CI): add the VITE_SUPABASE_ANON_KEY repo secret with the rotated key (release/docs builds otherwise use a placeholder), rotate the previously-committed Supabase anon key, and run a desktop save->restart->read smoke test to confirm keychain persistence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:11:32 -07:00
github-actions[bot] e4c4bcbae3 chore: update clone traffic data [skip ci] 2026-06-23 07:18:13 +00:00
Jon Saad-Falcon 993c24c8b9 test(server): make TestTraceRecording hermetic (env-independent) (#583)
TestTraceRecording relied on the ambient ~/.openjarvis/config.toml leaving traces.enabled at its default, so it failed on any machine with traces disabled locally (passing in CI only because the runner has no config file). Pass an explicit traces-enabled config with a tmp db_path so the tests are environment-independent and parallel-safe under pytest -n auto. Relates to #582.
2026-06-22 19:12:10 -07:00
Elliot Slusky 5bc8d3a2f6 Harden Docker and systemd deployment configs (#581)
Pin all base images and ollama to fixed versions + @sha256 digests (no floating :latest), run Docker images as an unprivileged openjarvis user (uid 10001), replace the curl|bash NodeSource install with a digest-pinned multi-stage copy, install from the committed uv.lock via uv export --frozen --no-dev (hash-verified, --no-deps), and add systemd sandboxing (NoNewPrivileges, ProtectSystem=strict, PrivateTmp, kernel/SUID protections). Closes #228, #563, #564, #565, #566, #567.
2026-06-22 19:12:07 -07:00
Elliot Slusky 433d10db5e feat(memory): native persistent memory service integrated into core (#579)
Adds the openjarvis.memory package (LocalFactStore, FactExtractor, background MemoryService), starts/stops it in the jarvis serve and jarvis chat lifecycle, feeds completed non-streaming exchanges to it, adds [memory] config support, and adds jarvis memory list/clear CLI commands. Extraction runs on a background thread and degrades to a no-op on any failure (BrokenPipe, timeouts, unparseable output) so it can never block a reply or crash the host. Disabled by default. Closes #393, #571, #572, #573.
2026-06-22 13:59:02 -07:00
Elliot Slusky 9b7b3681f6 ci: parallelize the test suite and cut install/coverage overhead (#580)
Run pytest with -n auto (pytest-xdist) and COVERAGE_CORE=sysmon, enable the uv cache, and switch test output to -q. Cuts the test job from ~40min to ~4min without changing what's tested or the 60% coverage gate.
2026-06-22 13:58:46 -07:00
Jon Saad-FalconandClaude Opus 4.8 6dbe5461bb fix(engine): drop Qwen3 control-token tool calls from Ollama responses (#578)
Qwen3 treats /think and /no_think as soft-switch control tokens. On small
models a multi-line prompt makes the model emit one as the sole tool argument
(e.g. {"command": "/no_think"}); OpenJarvis forwards Ollama's native tool_calls
verbatim, so the operative agent executes garbage. Filter control-token-only
tool calls in both the non-streaming generate() and streaming _run_stream()
paths, keeping legitimate calls like {"command": "date"}.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 11:40:25 -07:00
github-actions[bot] a65592fecb chore: update clone traffic data [skip ci] 2026-06-22 08:05:29 +00:00
github-actions[bot] 0513fbdb84 chore: update clone traffic data [skip ci] 2026-06-21 07:41:30 +00:00
Elliot Slusky d4eb6308b1 Fix desktop speech dependency setup (#574) 2026-06-20 17:08:24 -07:00
github-actions[bot] 2853a0001d chore: update clone traffic data [skip ci] 2026-06-20 07:23:14 +00:00
github-actions[bot] 3c99481975 chore: update clone traffic data [skip ci] 2026-06-19 07:55:20 +00:00
Elliot Slusky 4bf39af9bd Add provider-aware search support to hybrid orchestration agents (#558) 2026-06-18 13:40:06 -07:00
github-actions[bot] 0a3e812751 chore: update clone traffic data [skip ci] 2026-06-18 07:44:28 +00:00
github-actions[bot] eb46febad5 chore: update clone traffic data [skip ci] 2026-06-17 07:54:02 +00:00
github-actions[bot] 81482b45d4 chore: update clone traffic data [skip ci] 2026-06-16 08:00:49 +00:00
Jon Saad-FalconandClaude Opus 4.8 3e2f4bcdb4 feat(core): consolidate all state under a single env-aware home directory (#462) (#549)
Previously `core/config.py` defined `DEFAULT_CONFIG_DIR = Path.home() /
".openjarvis"` as a by-value module constant imported into ~45 modules, and 34
modules hardcoded `Path.home() / ".openjarvis"` directly. The installer honored
`OPENJARVIS_HOME` but the Python runtime ignored it, producing a split-brain
layout (some modules honored the override, the core config dir did not). Eval
dataset caches also scattered into `~/.cache/<benchmark>`.

This introduces a single env-aware resolver in `openjarvis/core/paths.py` and
routes every state/config/cache path through it. OpenJarvis now keeps ALL of
its state under ONE root, resolved in priority order:

  1. $OPENJARVIS_HOME
  2. $XDG_DATA_HOME/openjarvis   (single nested dir, when XDG_DATA_HOME is set)
  3. ~/.openjarvis               (default — unchanged, so existing installs are
                                  untouched and no data migration is required)

Implementation:
- New `core/paths.py`: get_config_dir / get_config_path / get_data_dir /
  get_cache_dir, with a source-tree rejection guard (fails loudly per
  REVIEW.md if the root resolves inside the repo).
- `core/config.py`: DEFAULT_CONFIG_DIR / DEFAULT_CONFIG_PATH are now resolved
  via the env-aware resolver at import (real attributes, so existing
  monkeypatch.setattr-based tests keep working). All dataclass field defaults
  that pointed at ~/.openjarvis converted to default_factory so they honor the
  override at instantiation.
- Routed all 34 hardcoders plus several string-literal escapees the original
  audit missed: prompt_loader / description_loader (were OPENJARVIS_HOME-only,
  no XDG), swebench_harness cache, tools/{memory,skill,user_profile}_manage
  defaults, server trace.db fallbacks, doctor_cmd hints.
- spec_search storage/paths now delegates to the unified resolver (gains XDG);
  its ConfigurationError is aliased to the core one.
- Eval dataset caches moved from ~/.cache/<name> to <root>/cache/<name>
  (~/.cache/huggingface left alone — it is HF's own cache).
- Docs + installer comment + `jarvis config path` to show resolved dirs.

Read-only macOS connectors and OS service files (LaunchAgents/systemd) are
intentionally left untouched.

Fixes #462

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 09:32:02 -07:00
github-actions[bot] a35b21195f chore: update clone traffic data [skip ci] 2026-06-15 08:03:30 +00:00
Jon Saad-FalconandClaude Opus 4.8 f9d1bc8c27 fix(connectors): complete Google OAuth and register Drive in Data Sources (fixes #512) (#548)
Pasting a Google Client ID / Secret never completed OAuth: Drive (and its
Google siblings) accepted the credentials, showed no error, opened no browser,
and never appeared in Data Sources. Root cause is three coupled defects, all
reproduced at the unit level against main with a FastAPI TestClient (no Google
creds, network-free):

(A/B) POST /connect routed a `client_id:client_secret` pair into the
  connector's handle_callback, which spawned a daemon thread that popped a
  browser and ran its own localhost:8789 callback server. That thread fails
  silently in the bundled desktop context (`except Exception: pass`), so the
  connector never gained an access_token; /connect returned status "pending"
  and the UI's 20x2s poll timed out with no error.
  Fix: in POST /connect, an OAuth `client_id:client_secret` pair now persists
  the client credentials to every Google credential file and returns an
  `oauth_required` directive pointing at the in-process server flow, instead of
  the silent background thread. The Google connectors' handle_callback no longer
  spawns the browser thread for the pair case — it only persists the creds; the
  server's /oauth/start -> /oauth/callback owns the consent round-trip.

(C) The would-be-correct server flow was itself broken: under
  `from __future__ import annotations` plus a `Request` import local to the
  router factory, FastAPI could not resolve the stringized `request: Request`
  annotation. /oauth/start returned HTTP 422 (request mis-bound as a query
  param) and /oauth/callback injected None -> AttributeError on
  `request.base_url`. Fix: import `Request` at module scope and make the
  callback's `request` a required injected dependency.

A malformed/blank client pair now raises HTTP 400 with the provider setup URL
instead of a perpetual silent "pending" (REVIEW.md silent-failure discipline).

Frontend: DataSourcesPage now opens the server OAuth window when /connect
returns `oauth_required`, then polls until connected; connect errors surface the
backend detail; the Drive setup steps document the "Web application" OAuth
client + server-callback redirect URI the in-process flow requires.

Tests (run on the main venv, hermetic — no ~/.openjarvis pollution):
- test_oauth_flow.py: the three handle_callback tests now assert NO browser is
  opened and only client creds are persisted (was: assert background flow ran).
- test_connectors_router_oauth.py (new): reproduces + fixes all three defects via
  TestClient with mocked token exchange; parametrized over gdrive/gcalendar/
  gcontacts/gmail/google_tasks to prove the shared OAuth path is fixed for every
  sibling and that a single consent writes the access_token to all six Google
  credential files and flips is_connected() to True.
Full tests/connectors suite: 355 passed.

Relationship to PR #510: #510 rewrites all of these files (account-scoped
retrieval) but still carries all three defects. This fix is intentionally scoped
to the OAuth path and does not modify oauth.py, to minimize collision. A
maintainer can either merge this and rebase #510 on top, or port these changes
into #510. See PR body for details.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:55:22 -07:00
Aditya MalikandJon Saad-Falcon dfa908c358 Adopt hatch-vcs dynamic versioning (reworks autotag, pypi-publish, desktop) (#538)
* build: adopt hatch-vcs dynamic versioning (#526)

Replace the static `version = "1.0.2"` with `dynamic = ["version"]` and
derive the version from git tags via hatch-vcs, so source and editable
checkouts report their true git describe version (e.g.
1.0.3.dev110+g<sha>) instead of a stale constant.

Config notes:
- Exclude .dev/.rc/desktop-* tags from derivation. setuptools_scm cannot
  bump custom .devN tags, so the base is taken from the latest plain
  release tag (vX.Y.Z) and the dev distance from commit count.
- Add fallback_version so builds without a git checkout (shallow CI
  clones, Docker COPY src/, source-zip installs) resolve to a sentinel
  instead of hard-failing. CI release builds inject the exact version via
  SETUPTOOLS_SCM_PRETEND_VERSION.

* ci(autotag): derive dev base from the latest release tag (#526)

pyproject no longer carries a static version, so read the base from the
latest plain release tag (vX.Y.Z) reachable from HEAD instead of grepping
pyproject. .dev/.rc/desktop-* tags are excluded so they cannot be mistaken
for the release base. The computed tag (vX.Y.Z.devN) is unchanged.

* ci(pypi-publish): pin build version from tag, drop sed injection (#526)

With dynamic versioning there is no static line to sed. Pin the exact
build version from the pushed tag via SETUPTOOLS_SCM_PRETEND_VERSION so
the published version equals the tag. This is required, not cosmetic: a
naive hatch-vcs build emits 1.0.3.devN+g<sha>, and PyPI rejects local
version segments on upload.

Also add a dry_run input that targets TestPyPI instead of PyPI, for
validating the release path without a production upload.

* ci(desktop): derive dispatch-fallback version from release tag (#526)

The workflow_dispatch fallback grepped the now-removed static pyproject
version. Derive its base from the latest release tag instead (matching
autotag), and give the build-and-release checkout full history and tags
so the derivation works.

---------

Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
2026-06-14 19:35:17 -07:00
Jon Saad-FalconandClaude Opus 4.8 8ef1ab1928 fix(ci): gate secret-bearing Claude workflows by author association (fixes #218) (#547)
The Claude automation workflows (claude-issues.yml, claude-review.yml)
trigger on public, attacker-controllable events (issues, issue_comment,
pull_request_review_comment) and grant the job secrets.ANTHROPIC_API_KEY
plus a write-scoped GITHUB_TOKEN with NO author-association gate.

Because issues / issue_comment / pull_request_review_comment always run in
the base-repo context with full secret access (unlike fork pull_request,
from which GitHub withholds secrets), any external GitHub user could fire
these jobs — draining the API budget and, via contents:write +
pull-requests:write, creating branches/PRs.

Fix:
- Add an author-association gate to every human-triggered, secret-bearing
  if: clause, restricting to OWNER / MEMBER / COLLABORATOR. Uses the correct
  event payload field per trigger: github.event.issue.author_association for
  the `issues` event, github.event.comment.author_association for
  issue_comment and pull_request_review_comment. workflow_dispatch stays
  trusted (requires repo write to invoke).
- Drop unused id-token: write from both workflows (claude-code-action@v1 is
  passed github_token directly, so OIDC is unused).
- Reduce claude-issues.yml timeout-minutes 60 -> 15.

desktop.yml and take-assign.yml are intentionally NOT touched: independently
verified as not exploitable for ANTHROPIC_API_KEY (desktop.yml's only
pull_request job uses no secrets and the trigger is plain pull_request, not
pull_request_target; take-assign.yml uses only GITHUB_TOKEN with issues:write
and no checkout/no Anthropic key). claude-review.yml's stale pull_request
auto-trigger was already removed in 3f2f46e4.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:31:51 -07:00
Jon Saad-FalconandClaude Opus 4.8 28e75cb513 fix(server): inject OpenJarvis identity system prompt on the desktop chat path (fixes #540) (#546)
The OpenAI-compatible POST /v1/chat/completions endpoint — the desktop UI's
chat backend — never injected OpenJarvis's agent.default_system_prompt when the
client omits a system message. The frontend (Chat/InputArea.tsx) posts only
user/assistant turns, so the model answered from its training identity
("I'm Claude", "I am Qwen", ...). The CLI paths ground identity via
SystemPromptBuilder / BaseAgent; the engine-direct server handlers did not.

Fix:
- Add _ensure_identity_prompt(messages, app_config) in server/routes.py: returns
  messages unchanged when any has role==SYSTEM, else prepends a SYSTEM message
  with the resolved identity prompt (app.state.config.agent.default_system_prompt,
  else load_config()), wrapped in try/except that debug-logs on failure (no crash,
  no silent swallow per REVIEW.md).
- Apply it after _to_messages() in all three engine-direct handlers:
  _handle_stream, _handle_stream_tools, and _handle_direct; thread app.state.config
  through. _handle_agent is left untouched (BaseAgent already injects the default).
- Harden AgentConfig.default_system_prompt so distilled models stop claiming to be
  Claude/ChatGPT/Gemini and self-identify as OpenJarvis.

Tests (tests/server/test_routes.py, tests/core/test_config.py): identity prompt IS
prepended when no system message is present (stream / direct / tools paths) and is
NOT duplicated when the client supplies one; config wording anchors "OpenJarvis"
and "not Claude". Verified fail-on-unfixed against main (3 inject tests + config
wording test fail there).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 18:38:05 -07:00
4b9948250b feat(engine): add DeepSeek as a first-class cloud provider + fix #335 over-permissive cloud fallback (#545)
* feat(engine): add DeepSeek as a first-class cloud provider

Adds DEEPSEEK_API_KEY support to the cloud engine, wiring DeepSeek's
OpenAI-compatible API (api.deepseek.com/v1) alongside the existing
MiniMax, OpenRouter, Anthropic, and Google providers.

- Add _DEEPSEEK_MODELS list (deepseek-v4-flash, deepseek-v4-pro)
- Add _is_deepseek_model() routing predicate
- Init self._deepseek_client from DEEPSEEK_API_KEY in _init_clients()
- Add _generate_deepseek() and _stream_deepseek() methods
- Wire DeepSeek into generate(), stream(), _stream_full_openai(),
  list_models(), and health()
- Add approximate pricing entries for both models

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(engine): strict cloud model routing + deepseek can_serve branch

Builds on the DeepSeek provider (PR #504) with two routing-correctness
fixes to CloudEngine._client_for_model:

1. Add the missing DeepSeek branch so can_serve('deepseek-*') agrees with
   list_models()/health() when only DEEPSEEK_API_KEY is set (mirrors the
   minimax branch). Without it the engine advertised deepseek models via
   list_models() but refused to serve them (the #532 can_serve contract).

2. Fix #335: _client_for_model previously fell through to the OpenAI client
   for ANY unrecognized model name, so an OpenAI key (even a dummy
   sk-dummy... one) made can_serve('qwen3.5:0.8b') return True. With the
   local engine transiently down (classic post-Windows-restart Ollama not
   yet up), model-aware get_engine then mis-selected the cloud engine for a
   local model and died with "OpenAI client not available". Add a positive
   _is_openai_model predicate (gpt-/chatgpt-/o1/o3/o4 + _OPENAI_MODELS) and
   return None for unrecognized names, so can_serve declines them. generate()
   and stream() keep their OpenAI fall-through, preserving loud failure for an
   explicitly-requested unknown cloud model.

Tests: DeepSeek detection/pricing/health/list_models/generate-routing/
can_serve and a #335 regression (can_serve rejects local names with an
OpenAI key; unknown model not served even with all clients set; end-to-end
get_engine does not misroute a local model with a dummy OpenAI key).

Fixes #335

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Jen Huls <me@jenhuls.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 18:21:56 -07:00
79e23719d4 feat(vision): add image + screen capture input for vision models (#486)
OpenJarvis can run vision-capable local models (gemma3, qwen2.5-vl), but the
CLI had no way to send them a picture -- the Ollama engine only serialized
text. This adds end-to-end image input.

What's new
- `jarvis ask -i/--image <file>` attaches one or more images to the query.
- `jarvis ask -S/--screen` captures the primary monitor (dependency-free on
  Windows via .NET; mss/Pillow fallback elsewhere).
- Vision auto-routes to direct-to-engine mode; with an explicit --agent it
  warns rather than silently dropping the image.
- Privacy guard: warns before sending an image to a non-local engine,
  keeping OpenJarvis local-first by default.
- Context-window default raised 8k -> 16k (JARVIS_NUM_CTX) so an image plus
  a conversation fit.

Implementation
- Message.images carries base64 data; messages_to_dicts() forwards it to
  Ollama's /api/chat "images" field. Text-only messages are unchanged.
- GuardrailsEngine preserves images when it rewrites a flagged message.

Tests (tests/test_vision.py, 6/6 pass, ruff-clean)
- payload forwarding, text path untouched, num_ctx override, guardrail
  image preservation.

Verified on AMD RX 9070 XT (Ollama/Vulkan, 100% GPU) with gemma3:4b:
solid-color image, file image, and live screen capture all described.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
2026-06-14 18:19:00 -07:00
SANJAYandsanjayravit 7ba334b5f0 fix(gui): inject bearer token into streaming chat/research routes (#499)
Ensures that the desktop GUI correctly sends the Authorization header
when an API key is configured. This resolves the 'Failed to get response'
bug on Windows systems with enabled authentication.

Ref: #266

Co-authored-by: sanjayravit <sanjay@example.com>
2026-06-14 17:32:43 -07:00
github-actions[bot] 48a2627c9a chore: update clone traffic data [skip ci] 2026-06-14 07:38:47 +00:00
github-actions[bot] 8625f4f95f chore: update clone traffic data [skip ci] 2026-06-13 07:21:38 +00:00
github-actions[bot] cf08f164c0 chore: update clone traffic data [skip ci] 2026-06-12 07:41:03 +00:00
Jon Saad-FalconandClaude Opus 4.8 b21463aab6 fix(evals): harden terminalbench-native harness against tmux death and setup hangs (#536)
Two failure classes hit by a downstream team:

1) tmux/task-env death (TerminalBenchTaskEnv.__enter__, terminalbench_env.py):
   create_session ran inside the spin_up_terminal generator-CM with no
   exception safety — a tmux failure leaked the docker compose project
   (down deferred to GC, never if the env was retained) and surfaced as an
   opaque mid-run death. Now: exception-safe __enter__ with an idempotent
   _teardown(), a tmux/asciinema preflight in the container BEFORE the
   agent loop (TaskEnvironmentError naming the task image + remedy), and
   fail-that-task-cleanly semantics — the failure is recorded as a harness
   error (QueryTrace.error_kind="harness_error"), the container is downed,
   and the run continues.

2) OpenHands in-container SETUP hang misattributed as a model result:
   harness.run() had no bound (terminal-bench runs installed-agent setup
   with max_timeout_sec=inf inside the per-trial agent budget), and the
   summary conversion read the nonexistent results.trial_results attr and
   hardcoded errors=0, folding zero-model-request trials into resolve-rate
   as model misses. Now: global_agent_timeout_sec / global_timeout_multiplier
   are threaded config -> backend -> Harness kwargs (default 1800 s bound on
   SETUP+RUN; configurable per [run]/[[benchmarks]] TOML), and
   summarize_benchmark_results() classifies harness/infra failures out of
   the accuracy denominator keyed on token usage (zero/missing tokens +
   unresolved = the agent never contacted the model), NOT failure_mode —
   terminal-bench 0.2.18 leaves failure_mode UNSET on success AND on
   genuine misses, so a failure_mode-based check would misflag every real
   model miss. Genuine misses (tokens>0, is_resolved=false) stay in the
   denominator.

QueryTrace gains error/error_kind (wired through to_dict/from_dict so the
fields actually reach traces.jsonl; backward-compatible loads), the
AgenticRunner flags zero-model-contact traces and TaskEnvironmentError as
harness errors, and export/summary/console output exclude harness errors
from resolve-rate while reporting them loudly.

terminal-bench stays an undeclared dep on purpose: it requires Python
>=3.12 while this project supports >=3.10,<3.14, so an unmarked extra
would break uv lock. pyproject/uv.lock untouched.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 14:30:10 -07:00
Jon Saad-FalconandClaude Opus 4.8 0cac61d3bb docs(evals): rewrite evaluations guide to match the real CLI surface; add openjarvis-eval alias (#534)
docs/user-guide/evaluations.md documented a standalone "openjarvis-evals"
package, a "uv sync --extra eval" install, and an "openjarvis-eval" console
script — a layout from commit bd493832 that was never an ancestor of main.
Rewrite the page against the real surface (jarvis eval / python -m
openjarvis.evals), document all 40 registered benchmark keys and 4 backends,
fix the judge-model default, correct run-all semantics, and split the option
reference into the jarvis-eval subset and the module CLI's research-only
options.

Add the one-line [project.scripts] alias
openjarvis-eval = "openjarvis.evals.cli:main" (the click group the module
CLI already dispatches to) so the long-documented command name works again.

The page was a complete orphan: add it (and the equally orphaned
benchmarks.md) to the mkdocs nav and link it from docs/index.md.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 13:50:17 -07:00
Jon Saad-FalconandClaude Opus 4.8 50993dfa4d fix(evals): honor --base-url/--api-key for first-party eval backends (#535)
`jarvis eval run --base-url ... --api-key ...` was silently dropped for
jarvis-direct/jarvis-agent (_build_backend only forwarded the flags to
hermes/openclaw) and ignored by terminalbench-native, which hardcoded
api_base="http://localhost:8000/v1". Worse, with --base-url set the
engine-discovery fallback silently substituted ANY healthy local engine
(observed: requested vllm + healthy endpoint at --base-url, got
OllamaEngine@localhost:11434 — the requested URL was never contacted).

Changes:
- _OpenAICompatibleEngine gains an api_key param (Bearer Authorization
  header on the httpx client; {ENGINE_ID}_API_KEY env fallback with
  hyphen-sanitized names; no header when unset).
- New non-registered OpenAICompatEngine + normalize_openai_base_url()
  (strips a single literal trailing "/v1" so request paths don't double).
- SystemBuilder.engine_instance() injects a pre-built engine; build()
  health-checks it and fails loudly naming the host instead of falling
  back to discovery. Discovery substitution after an explicit -e key now
  logs a warning.
- JarvisDirectBackend/JarvisAgentBackend accept base_url/api_key; on
  base_url they pin an OpenAICompatEngine to that endpoint with a
  fail-fast pre-flight (actionable error naming the URL and probe).
- _build_backend forwards base_url/api_key to first-party backends on
  the CLI path; _run_terminalbench_native receives --base-url as
  api_base (single /v1 suffix) and exports OPENAI_API_KEY around the
  in-process harness run (terminus-2 routes via LiteLLM).
- Suite TOML [backend.external] stays scoped to hermes/openclaw
  (suite_mode=True in the suite drivers) — first-party suite semantics
  are explicitly deferred. The config-host path is untouched.
- Help text updated on both CLI surfaces; KNOWN_BACKENDS now lists
  hermes/openclaw/terminalbench-native.

Fixes the eval-CLI endpoint gap reported by the downstream team.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 13:49:37 -07:00
github-actions[bot] 527f84f960 chore: update clone traffic data [skip ci] 2026-06-11 07:44:18 +00:00
8eaeb3a754 fix(windows): desktop backend spawn + model-aware engine selection (#533)
* fix(windows): desktop backend spawn (#531) + model-aware engine selection (#532)

Two runtime bugs found during end-to-end testing on a clean Windows 11
24H2 Azure VM.

#531 - Desktop "Failed to get response": run_jarvis_command spawned the
backend with .output(), which waits for the process to exit. `jarvis
serve` never exits, so the Tauri command hung forever (the Start button
never resolved); and it ran `uv run jarvis` with no cwd, so in a packaged
install -- where the cwd isn't the checkout -- `jarvis` wasn't found and
the server never started. Now: run from find_project_root(), and for
`serve` spawn detached (.spawn()), drain stderr, and poll /health for
readiness (mirrors start_backend); short commands keep .output().

The server layer itself was verified healthy on Windows (/health and
/v1/chat/completions both 200, localhost included) -- the fault was the
Tauri spawn path.

#532 - "OpenAI client not available" after reboot: when the local engine
is down, get_engine's fallback selected CloudEngine because health() is
True if ANY provider client exists -- without checking the resolved
model's provider has a client. A user with e.g. OPENROUTER_API_KEY and a
gpt-* model then hit the OpenAI path with no client. Add
CloudEngine.can_serve(model) (checks the specific provider client via the
same routing generate()/stream() use) + a default can_serve->True on the
base engine, and make get_engine model-aware so it skips an engine that
can't serve the model -- the user falls through to the helpful "no engine
available / start ollama" message instead.

Tests: engine discovery/cloud/model-matrix + cli serve/ask suites pass
(the one ask_e2e failure is a pre-existing version-banner flake, fails
identically on main). The Tauri crate couldn't be compiled locally (no
GTK/webkit sys-libs in this env); relies on CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(engine): cover model-aware engine selection + CloudEngine.can_serve (#532)

#533 added a `model` arg to get_engine and a can_serve() gate but shipped no
tests. Add them:
- get_engine skips a healthy engine that can't serve the requested model
  (the cloud-fallback-for-unservable-model case behind #532),
- model=None preserves the legacy model-agnostic selection,
- CloudEngine.can_serve gates on the per-provider client (gpt->OpenAI,
  claude->Anthropic, ...), verified empirically.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
2026-06-10 19:52:36 -07:00
Robby ManihaniandClaude Opus 4.8 90b7d0cb9b fix(windows): installer encoding + Ollama readiness loop (#523)
Two bugs found during end-to-end testing on a clean Windows 11 24H2
Azure VM (closes #522). Both are dodged by the canonical `irm | iex`
one-liner but hit by the documented `-OutFile` fallback and any
non-interactive run.

1. Encoding. install.ps1 was UTF-8 without a BOM and contained em-dashes
   plus a box-drawing banner. Windows PowerShell 5.1 decodes BOM-less
   files with the legacy ANSI/OEM code page, mis-decoding the multi-byte
   sequences and desyncing the parser into cascading here-string parse
   errors. Converted the file to pure ASCII (em-dashes -> hyphens, banner
   -> ASCII art) so it parses no matter how it's read.

2. Ollama readiness loop. With $ErrorActionPreference='Stop', the probe
   `& $ollamaExe list 2>&1 | Out-Null` turned the daemon-not-up stderr
   into a terminating NativeCommandError, aborting the install on the
   first iteration and making the loop's own Start-Process serve retry +
   Write-Warn2 fallback dead code. Wrapped the probe in try/catch so it
   falls through to the self-start path as intended.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 19:15:57 -07:00
d7053c35d5 security: harden network-exposed surface (#509)
* security: harden network-exposed surface

Hardening for the network-reachable attack surface, prioritizing fixes
that are strong but do not change working local/loopback defaults.

- auth_middleware: constant-time API key comparison (secrets.compare_digest)
  for the HTTP path, and gate /metrics behind auth so operational counters
  are not readable unauthenticated. /health stays open.
- webhook_routes: fail closed when a channel's secret/token is unset. Twilio,
  BlueBubbles, WhatsApp (verify + inbound), and SendBlue now reject (403)
  instead of processing unsigned/unauthenticated input. Constant-time
  comparisons for BlueBubbles/SendBlue/WhatsApp verify token.
- http_request: follow redirects manually and re-run the SSRF check on every
  hop (capped at 5) so an allowed public URL cannot 30x-redirect to an
  internal/metadata address.
- api_routes /v1/memory/index: restrict indexing to OPENJARVIS_WORKSPACE roots
  when configured and refuse sensitive files (.env, keys, credentials).
- config.toml: default [server] host to 127.0.0.1 (loopback) with a comment
  on how to safely expose to a LAN (0.0.0.0 + API key).

Tests: new fail-closed webhook tests, /metrics auth tests, and SSRF
redirect block/follow tests; updated SendBlue tests for the new
secret-required behavior. Affected suites pass (95 tests), ruff clean.

* fix(http): keep SSRF redirect-following patchable via httpx.request

The manual redirect-following loop used a private httpx.Client, which
bypassed the `http_request.httpx.request` mock seam that consumers' tests
rely on (e.g. the twitter-bot GitHub-issue tests escaped to the real
network and 401'd). Issue each hop via module-level httpx.request with
follow_redirects=False instead — same per-hop SSRF re-check, restored
testability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 15:32:28 -07:00
03c5ec3e40 fix(chat): wire SystemPromptBuilder so persona files load in jarvis chat (fixes #458) (#497)
* fix(chat): wire SystemPromptBuilder so persona files load in jarvis chat (fixes #458)

* fix(chat): make `--persona none` actually disable persona files

This PR exposes `--persona none`, but SystemPromptBuilder._load_file read
empty paths as "." (Path("") -> ".") and raised IsADirectoryError, so the
documented opt-out crashed. Guard empty path_str so the "none" opt-out
(which _resolve_persona maps to empty file paths) cleanly injects no
persona. Adds an end-to-end regression test (building with persona
"none" must not raise). Also merges current main (branch was stale).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:10:13 -07:00
Jon Saad-FalconandClaude Opus 4.8 4218258486 perf(serve): build the system once — drop duplicate SystemBuilder.build() (#263) (#529)
`jarvis serve` constructed every heavy component inline (engine discovery +
instrumentation, telemetry, memory, agent manager, per-agent tools) and then
called `SystemBuilder(config).build()` a second time inside the scheduler
block purely to feed `AgentExecutor.set_system()`. That second build
re-discovered and re-connected the engine, re-instrumented it, re-resolved
tools, re-opened the configured channel and re-created the agent manager —
~30-40s of fully redundant startup work (the headline remaining cost in #263
after engine probes were parallelised and the version check moved off the hot
path in #470).

Fix: assemble the executor's `JarvisSystem` from the components already built
inline instead of rebuilding from scratch. `AgentExecutor` only reads
`engine`, `model`, `config`, `memory_backend`, `tool_executor`,
`session_store` and `channel_backend` off the system; all are wired here. The
memory backend is now constructed just before the scheduler block (it was
built later) so the executor's system can reference it, and the primary
agent's resolved tool list is reused to build the scheduler's `ToolExecutor`
(preserving the MCP-discovered-tool pool the executor reads via
`tool_executor._tools`). `skill_manager` / the learning orchestrator are only
consumed by the orchestrator's `system.ask()` path, which the executor never
invokes, so they are intentionally omitted.

Tests: new `tests/cli/test_serve_single_build.py` patches
`SystemBuilder.build` and asserts it is never called during `jarvis serve`
startup, and that the executor still receives a system exposing
`tool_executor` / `session_store` / `memory_backend` (plus engine/model/config).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:10:10 -07:00
Jon Saad-FalconandClaude Opus 4.8 176ed3029b fix(channels): unify send() destination/reply contract (Discord #515/#516) (#528)
The channel `send()` contract was inconsistent across adapters. Almost
every adapter (Discord, Slack, email, WhatsApp, ...) treats the first
positional `channel` arg as the real DESTINATION id and `conversation_id`
as an optional reply/thread reference. Telegram alone treated
`conversation_id` as the destination (`chat_id = conversation_id or
channel`).

`JarvisSystem._on_channel_message` hard-coded the Telegram-shaped mapping
for ALL channels: `send(cm.channel, reply, conversation_id=cm.conversation_id)`.
Since inbound `ChannelMessage`s carry the channel TYPE label in `.channel`
("discord") and the real destination id in `.conversation_id`, this sent
the literal "discord" as the Discord channel id (HTTP 400
NUMBER_TYPE_COERCE, #515) and passed the channel id as a Discord
`message_reference` (MESSAGE_REFERENCE_UNKNOWN_MESSAGE, #516).

Fix: define and document ONE canonical contract on `BaseChannel.send` —
positional `channel` = destination id, `conversation_id` = inbound message
id used as a reply reference — and dispatch it from `_on_channel_message`
as `send(cm.conversation_id, reply, conversation_id=cm.message_id)`,
matching the already-fixed `ChannelAgent` path (#495/#459). Telegram's
`send()` is brought into line (destination = `channel`, with a
`reply_to_message_id` reply ref and a legacy `conversation_id`-only
fallback) so it keeps working unchanged.

The DiscordChannel `_gateway_loop` ChannelMessage shape locked by #495 is
untouched; `tests/agents/test_channel_agent.py` passes unchanged. The
stale `test_serve_channel_wiring.py` assertions (which encoded the old
buggy mapping from #94) are updated, and regression tests are added for
Discord (real channel id + correct message_reference), Telegram (chat id
+ reply ref + legacy fallback), and per-channel dispatch in
`_on_channel_message`.

Fixes #515
Fixes #516

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:10:06 -07:00
Jon Saad-FalconandClaude Opus 4.8 590addae4c test(engine): expect EngineConnectionError for vllm 404 (fix main CI) (#530)
#463 made the OpenAI-compatible engine wrap upstream HTTP errors (incl.
404) in EngineConnectionError with an actionable message, but
test_invalid_model_404 still asserted the raw httpx.HTTPStatusError, so it
broke on main once #463 landed (#463 was a stale fork PR with no CI, so it
wasn't caught pre-merge). Expect EngineConnectionError now, asserting the
httpx.HTTPStatusError is preserved as the chained cause. Whole tests/engine
suite is green again.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:09:37 -07:00
Jon Saad-FalconandClaude Opus 4.8 bc9fa6b3c8 fix(memory): surface clear error when openjarvis_rust missing instead of silent no-op (#527)
Memory tools degraded silently and misleadingly when the mandatory
`openjarvis_rust` extension was absent from the *serving* venv:

- `POST /v1/memory/store` returned HTTP 200 `{"status":"stored","note":
  "no backend available"}` and stored nothing (silent data loss).
- `POST /v1/memory/index` returned a generic "No memory backend available",
  and the desktop frontend discarded the server `detail` and threw a blanket
  "Failed to index path", blaming the path instead of the real cause.
- `GET /v1/memory/config` reported `backend_type: sqlite` even though no
  backend could be constructed.

Root cause: `SQLiteMemory.__init__` calls `get_rust_module()` (which raises
ImportError by design — the Rust ext is mandatory, no Python fallback), and
`_get_memory_backend` swallowed that ImportError and returned `None`,
conflating "native extension missing" (a hard install error) with "memory
intentionally disabled" (benign). A chunking floor also silently dropped whole
short documents, and the installer never verified the extension imported from
the serving venv before writing its success marker.

Fix (no fake Python fallback — the Rust ext stays mandatory by design):

- Add `MemoryBackendUnavailable` + `RUST_MISSING_HINT` in tools/storage/_stubs.
  `SQLiteMemory.__init__` translates the bridge ImportError into this clear,
  actionable error ("run `uv run maturin develop ...`").
- `_get_memory_backend` distinguishes the two cases: a missing native ext
  raises HTTP 503 with the actionable hint; a benign unconfigured backend
  still returns `None` (graceful path preserved for search/stats).
- `/store` now returns 503 instead of a 200 silent no-op.
- `/config` reports `available: false` + `detail` instead of falsely claiming
  a healthy `backend_type`.
- `/index` adds a `note` when `chunks_indexed == 0` so "indexed" never
  silently means "stored nothing".
- chunk_text no longer drops an entire short document below `min_chunk_size`
  (the floor only discards tiny *trailing* fragments now).
- Frontend `storeMemory`/`indexMemoryPath` surface the server `detail` instead
  of blanket strings; `MemoryConfig` gains optional `available`/`detail`.
  (Left the pre-existing `backend` vs `backend_type` mismatch untouched.)
- build-extension.sh verifies `import openjarvis_rust` succeeds in the serving
  venv before writing the `extension-built` marker.

Regression tests: tests/server/test_api_routes.py::TestMemoryRustMissing mocks
`get_rust_module` to raise ImportError and asserts /store (503, not 200 no-op),
/index (actionable detail, not "Failed to index path"), and /config
(available:false) all surface the clear error; tests/memory/test_chunking.py
asserts short-only docs are kept while tiny trailing fragments are still
filtered.

Fixes #502

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 13:25:10 -07:00
376 changed files with 23754 additions and 6907 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"label": "Git Clones",
"message": "107,695",
"message": "186,659",
"color": "green",
"namedLogo": "git"
}
+66 -3
View File
@@ -1,6 +1,6 @@
{
"total_clones": 107695,
"last_updated": "2026-06-10T07:32:26Z",
"total_clones": 186659,
"last_updated": "2026-08-11T07:02:05Z",
"daily": {
"2026-03-27": 2189,
"2026-03-28": 1874,
@@ -75,6 +75,69 @@
"2026-06-05": 2127,
"2026-06-06": 2204,
"2026-06-07": 1174,
"2026-06-08": 2369
"2026-06-08": 2369,
"2026-06-09": 1361,
"2026-06-10": 1310,
"2026-06-11": 2564,
"2026-06-12": 1313,
"2026-06-13": 2804,
"2026-06-14": 1543,
"2026-06-15": 1379,
"2026-06-16": 1317,
"2026-06-17": 1170,
"2026-06-18": 1408,
"2026-06-19": 1350,
"2026-06-20": 1437,
"2026-06-21": 1426,
"2026-06-22": 1350,
"2026-06-23": 1468,
"2026-06-24": 1635,
"2026-06-25": 1640,
"2026-06-26": 1338,
"2026-06-27": 1338,
"2026-06-28": 1028,
"2026-06-29": 765,
"2026-06-30": 951,
"2026-07-01": 1134,
"2026-07-02": 593,
"2026-07-03": 537,
"2026-07-04": 411,
"2026-07-05": 485,
"2026-07-06": 555,
"2026-07-07": 905,
"2026-07-08": 1171,
"2026-07-09": 1857,
"2026-07-10": 1181,
"2026-07-11": 2185,
"2026-07-12": 1917,
"2026-07-13": 2102,
"2026-07-14": 2337,
"2026-07-15": 2362,
"2026-07-16": 2497,
"2026-07-17": 1773,
"2026-07-18": 1542,
"2026-07-19": 1445,
"2026-07-20": 1481,
"2026-07-21": 1528,
"2026-07-22": 1529,
"2026-07-23": 1209,
"2026-07-24": 1118,
"2026-07-25": 928,
"2026-07-26": 740,
"2026-07-27": 799,
"2026-07-28": 665,
"2026-07-29": 745,
"2026-07-30": 591,
"2026-07-31": 783,
"2026-08-01": 567,
"2026-08-02": 1248,
"2026-08-03": 724,
"2026-08-04": 708,
"2026-08-05": 647,
"2026-08-06": 604,
"2026-08-07": 624,
"2026-08-08": 706,
"2026-08-09": 1076,
"2026-08-10": 1060
}
}
+10 -4
View File
@@ -21,14 +21,20 @@ jobs:
id: version
run: |
set -euo pipefail
# Base version is the next patch above whatever is in pyproject.toml.
# Base version is the next patch above the latest plain release tag
# (vX.Y.Z) reachable from HEAD. pyproject.toml no longer carries a
# static version (#526 switched it to hatch-vcs), so the release tag
# is the source of truth. `.devN`/`.rcN`/`desktop-*` tags are excluded
# so they can't be mistaken for the release base.
# Any future manual `X.Y.Z` release will outrank every `X.Y.Z.devN`
# autotag — PEP 440 sorts dev releases strictly below the final.
BASE=$(grep -E '^version = "' pyproject.toml | head -1 | sed -E 's/^version = "([^"]+)"/\1/')
if [[ -z "$BASE" ]]; then
echo "::error::Could not parse version from pyproject.toml"
LATEST_RELEASE=$(git tag --list 'v[0-9]*' --merged HEAD \
| grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)
if [[ -z "$LATEST_RELEASE" ]]; then
echo "::error::No release tag (vX.Y.Z) reachable from HEAD"
exit 1
fi
BASE="${LATEST_RELEASE#v}"
MAJOR=$(echo "$BASE" | cut -d. -f1)
MINOR=$(echo "$BASE" | cut -d. -f2)
PATCH=$(echo "$BASE" | cut -d. -f3 | sed -E 's/[^0-9].*$//')
+15 -1
View File
@@ -23,6 +23,8 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v8.0.0
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev --extra framework-comparison --extra server
@@ -30,6 +32,9 @@ jobs:
- name: Ruff check
run: uv run ruff check src/ tests/
- name: Ruff format check
run: uv run ruff format --check src/ tests/
test:
runs-on: ubuntu-latest
steps:
@@ -55,6 +60,8 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v8.0.0
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev --extra framework-comparison --extra server
@@ -63,8 +70,13 @@ jobs:
run: uv run maturin develop --manifest-path rust/crates/openjarvis-python/Cargo.toml
- name: Run tests
# COVERAGE_CORE=sysmon uses CPython 3.12's sys.monitoring backend,
# which is dramatically cheaper than the default C trace function.
# -n auto fans the suite out across all runner cores via pytest-xdist.
env:
COVERAGE_CORE: sysmon
run: |
uv run pytest tests/ -v --tb=short -m "not live and not cloud and not hub" \
uv run pytest tests/ -n auto -q --tb=short -m "not live and not cloud and not hub" \
--cov=openjarvis \
--cov-report=term-missing \
--cov-report=xml \
@@ -107,6 +119,8 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v8.0.0
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev --extra server
+13 -2
View File
@@ -11,22 +11,33 @@ concurrency:
group: claude-issues-${{ github.event.issue.number || github.run_id }}
cancel-in-progress: true
# Least-privilege: only what the issue-fixer job actually needs.
# id-token (OIDC) is intentionally omitted — claude-code-action@v1 is passed
# github_token directly, so OIDC is unused here.
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
jobs:
fix:
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 15
# Security gate: this job reaches secrets.ANTHROPIC_API_KEY and holds a
# write-scoped GITHUB_TOKEN. `issues` / `issue_comment` are public,
# attacker-controllable events that run in the base-repo context with full
# secret access, so the human-triggered paths are restricted to actors with
# write-level association (OWNER / MEMBER / COLLABORATOR). This blocks
# external / first-time contributors from draining the API budget or
# creating branches/PRs, while leaving maintainer use unaffected.
if: |
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issues' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.issue.author_association) &&
(contains(github.event.issue.labels.*.name, 'bug') ||
contains(github.event.issue.labels.*.name, 'autofix'))) ||
(github.event_name == 'issue_comment' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
!github.event.issue.pull_request &&
contains(github.event.comment.body, '@claude') &&
github.actor != 'claude[bot]')
+11 -1
View File
@@ -11,23 +11,33 @@ concurrency:
group: claude-review-${{ github.event.pull_request.number || github.event.issue.number || github.run_id }}
cancel-in-progress: true
# Least-privilege: PR review only needs to post comments on the PR.
# id-token (OIDC) is omitted — claude-code-action@v1 is passed github_token
# directly, so OIDC is unused here.
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
jobs:
review:
runs-on: ubuntu-latest
timeout-minutes: 30
# Security gate: this job reaches secrets.ANTHROPIC_API_KEY. Both
# issue_comment and pull_request_review_comment are public,
# attacker-controllable events that run in the base-repo context with full
# secret access, so the @claude paths are restricted to actors with
# write-level association (OWNER / MEMBER / COLLABORATOR). External /
# first-time contributors cannot trigger the key; maintainers are unaffected.
if: |
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '@claude') &&
github.actor != 'claude[bot]') ||
(github.event_name == 'pull_request_review_comment' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
contains(github.event.comment.body, '@claude') &&
github.actor != 'claude[bot]')
steps:
+121 -3
View File
@@ -40,7 +40,8 @@ jobs:
libappindicator3-dev \
librsvg2-dev \
patchelf \
libxdo-dev
libxdo-dev \
libdbus-1-dev
- name: Setup Node.js
uses: actions/setup-node@v6
@@ -114,6 +115,109 @@ jobs:
steps:
- uses: actions/checkout@v6
with:
# Full history + tags so the workflow_dispatch fallback in
# "Determine release info" can derive the dev version from the
# latest release tag (#526).
fetch-depth: 0
# Validate Apple credentials BEFORE the expensive work. Notarization is
# the very last thing `tauri-action` does, so a bad credential or a
# lapsed account agreement previously surfaced ~10 minutes in — after the
# Rust toolchain, npm install, two Ollama sidecar downloads and a
# universal cargo build — as a single opaque line:
#
# failed to bundle project: failed codesign application: failed to
# notarize app: Error: HTTP status code: 403. ...
#
# `notarytool history` is a read-only call (it submits nothing) that
# exercises the identical auth path, so every credential/account failure
# mode reaches us here first, in seconds, with the specific cause named.
# `xcrun` is preinstalled on macOS runners, hence placement before the
# toolchain steps rather than next to "Configure Apple signing".
- name: Preflight Apple notarization credentials
if: matrix.platform == 'macos-14'
env:
CERT: ${{ secrets.APPLE_CERTIFICATE }}
A_ID: ${{ secrets.APPLE_ID }}
A_PASS: ${{ secrets.APPLE_PASSWORD }}
A_TEAM: ${{ secrets.APPLE_TEAM_ID }}
shell: bash
run: |
set -uo pipefail
# Mirror the skip logic in "Configure Apple signing": without a
# certificate the build is unsigned and never notarizes, so there is
# nothing to preflight. Tag builds still hard-fail there.
if [ -z "$CERT" ]; then
echo "No Apple certificate configured; skipping notarization preflight."
exit 0
fi
missing=""
[ -z "$A_ID" ] && missing="$missing APPLE_ID"
[ -z "$A_PASS" ] && missing="$missing APPLE_PASSWORD"
[ -z "$A_TEAM" ] && missing="$missing APPLE_TEAM_ID"
if [ -n "$missing" ]; then
echo "::error::APPLE_CERTIFICATE is set but notarization secrets are missing:$missing"
echo "::error::Signing would succeed and notarization would then fail. Set them or clear APPLE_CERTIFICATE."
exit 1
fi
# Retry only to absorb transient network faults. Credential and
# account errors are deterministic, so we classify and exit on the
# first definitive answer rather than retrying into the same wall.
attempt=1
while [ "$attempt" -le 3 ]; do
out=$(xcrun notarytool history \
--apple-id "$A_ID" \
--team-id "$A_TEAM" \
--password "$A_PASS" \
--output-format json 2>&1)
rc=$?
if [ $rc -eq 0 ]; then
echo "Apple notarization preflight OK — credentials valid, team reachable, agreements in effect."
exit 0
fi
case "$out" in
*"Invalid credentials"*|*"401"*)
echo "::error::Apple notarization preflight failed: invalid credentials (HTTP 401)."
echo "::error::APPLE_PASSWORD must be an app-specific password from appleid.apple.com,"
echo "::error::generated while signed in as the SAME Apple ID as APPLE_ID. A regular"
echo "::error::Apple ID password will not work, and a password minted under a different"
echo "::error::Apple ID authenticates as that other account."
exit 1
;;
*"Invalid or inaccessible developer team ID"*)
echo "::error::Apple notarization preflight failed: APPLE_ID is not a member of team APPLE_TEAM_ID (HTTP 403)."
echo "::error::The Team ID must match the signing certificate. Read it from the cert's"
echo "::error::subject, where it appears as: Developer ID Application: NAME (TEAMID)."
echo "::error::If you belong to several teams, confirm APPLE_ID is a member of this one."
exit 1
;;
*"required agreement"*|*"agreement"*)
echo "::error::Apple notarization preflight failed: the team has no in-effect agreement (HTTP 403)."
echo "::error::Apple reissues the Developer Program License Agreement periodically and"
echo "::error::notarization is refused until it is accepted. ONLY THE ACCOUNT HOLDER can"
echo "::error::accept it — team Admins cannot. Sign in to the account that owns this team:"
echo "::error:: 1. https://developer.apple.com/account -> review any pending agreement"
echo "::error:: 2. App Store Connect -> Business -> accept anything pending there too"
echo "::error::Certificates stay valid while this is outstanding, so signing still works."
exit 1
;;
esac
echo "Preflight attempt ${attempt}/3 failed with a non-credential error."
echo "$out" | tail -5
attempt=$((attempt + 1))
[ "$attempt" -le 3 ] && sleep 10
done
echo "::error::Apple notarization preflight failed after 3 attempts. Last output:"
echo "$out" | tail -20
exit 1
- name: Install system dependencies (Linux)
if: matrix.platform == 'ubuntu-22.04'
@@ -125,7 +229,8 @@ jobs:
libappindicator3-dev \
librsvg2-dev \
patchelf \
libxdo-dev
libxdo-dev \
libdbus-1-dev
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
@@ -183,7 +288,16 @@ jobs:
# workflow_dispatch fallback (manual UI dispatch without --ref).
# Derive a PEP 440 dev version aligned with autotag.yml so we
# don't burn the X.Y.Z release-version namespace.
BASE=$(grep -E '^version = "' pyproject.toml | head -1 | sed -E 's/^version = "([^"]+)"/\1/')
# pyproject.toml no longer carries a static version (#526), so the
# base comes from the latest plain release tag (vX.Y.Z), matching
# autotag.yml. .dev/.rc/desktop-* tags are excluded.
LATEST_RELEASE=$(git tag --list 'v[0-9]*' --merged HEAD \
| grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)
if [[ -z "$LATEST_RELEASE" ]]; then
echo "::error::No release tag (vX.Y.Z) reachable from HEAD"
exit 1
fi
BASE="${LATEST_RELEASE#v}"
MAJOR=$(echo "$BASE" | cut -d. -f1)
MINOR=$(echo "$BASE" | cut -d. -f2)
PATCH=$(echo "$BASE" | cut -d. -f3 | sed -E 's/[^0-9].*$//')
@@ -238,6 +352,10 @@ jobs:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
TAURI_CONFIG: '{"version":"${{ steps.release-info.outputs.tauri_version }}","bundle":{"externalBin":["binaries/ollama"]}}'
# tauri-action runs beforeBuildCommand (npm run build:tauri -> vite
# build), which requires this at build time (#587). Strict for
# releases: a missing/empty secret fails the build by design.
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
with:
projectPath: frontend
tauriScript: npx tauri
+20
View File
@@ -41,6 +41,26 @@ jobs:
- name: Install dependencies
run: uv sync --extra docs
# Inject the public Supabase anon key so the savings leaderboard works on
# the published docs site. Missing/empty (e.g. fork PRs) leaves the
# leaderboard gracefully disabled. The key is read from env (not inlined)
# and JSON-encoded into a JS string literal to avoid any injection.
- name: Inject leaderboard Supabase anon key
env:
OPENJARVIS_LEADERBOARD_ANON: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
run: |
python3 - <<'PY'
import json, os, pathlib
key = os.environ.get("OPENJARVIS_LEADERBOARD_ANON", "")
pathlib.Path("docs/javascripts/leaderboard-config.js").write_text(
"// Generated at docs-build time from the VITE_SUPABASE_ANON_KEY secret.\n"
"window.OPENJARVIS_SUPABASE_ANON_KEY = " + json.dumps(key) + ";\n",
encoding="utf-8",
)
print("leaderboard anon key:", "set" if key else "empty (leaderboard disabled)")
PY
- name: Build documentation
run: uv run mkdocs build
+5
View File
@@ -35,3 +35,8 @@ jobs:
- run: npm ci
- run: npx tsc --noEmit
- run: npm run build
env:
# Optional: when the secret is unset the build still succeeds and the
# leaderboard is disabled (see src/lib/supabase.ts). No placeholder,
# so a keyless CI build doesn't bake in a bogus anon key.
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
+32 -11
View File
@@ -12,6 +12,11 @@ on:
description: 'Tag to publish (e.g. v1.0.2.dev500). Overrides github.ref.'
required: false
type: string
dry_run:
description: 'Dry run: build + validate, then publish to TestPyPI instead of PyPI (no production upload).'
required: false
default: false
type: boolean
permissions:
contents: read
@@ -50,6 +55,8 @@ jobs:
cache-dependency-path: frontend/package-lock.json
- name: Build frontend and bundle into package
env:
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
run: |
set -euo pipefail
cd frontend
@@ -67,27 +74,41 @@ jobs:
exit 1
}
- name: Set version from tag
- name: Resolve build version from tag
env:
REF: ${{ steps.ref.outputs.ref }}
run: |
set -euo pipefail
# Strip leading "v" if present (e.g. v1.0.2.dev500 -> 1.0.2.dev500)
# Strip leading "v" (e.g. v1.0.3.dev825 -> 1.0.3.dev825).
VERSION="${REF#v}"
if [[ -z "$VERSION" ]]; then
echo "::error::Could not resolve version from ref '$REF'"
if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+ ]]; then
echo "::error::ref '$REF' is not a version tag (expected vX.Y.Z[.devN]); pass -f tag=vX.Y.Z"
exit 1
fi
sed -i "s/^version = \".*\"/version = \"${VERSION}\"/" pyproject.toml
# Sanity check the substitution actually took
grep -q "^version = \"${VERSION}\"" pyproject.toml || {
echo "::error::sed failed to update pyproject.toml version"
exit 1
}
echo "Building version $VERSION"
# pyproject.toml is now dynamic = ["version"] via hatch-vcs (#526), so
# there is no static line to sed. setuptools_scm cannot bump custom
# `.devN` tags, so we pin the exact build version explicitly — the
# published version always equals the pushed tag.
echo "SETUPTOOLS_SCM_PRETEND_VERSION=${VERSION}" >> "$GITHUB_ENV"
echo "Building version ${VERSION}"
- name: Build package
run: uv build
- name: Publish to TestPyPI (dry run)
if: ${{ inputs.dry_run }}
env:
UV_PUBLISH_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
run: |
set -euo pipefail
if [[ -z "${UV_PUBLISH_TOKEN:-}" ]]; then
echo "::warning::TEST_PYPI_API_TOKEN is not set — skipping the TestPyPI upload."
echo "Build + twine check passed, which validated version derivation and packaging end to end."
echo "To exercise a real upload, add a TEST_PYPI_API_TOKEN secret (or a TestPyPI trusted publisher)."
exit 0
fi
uv publish --publish-url https://test.pypi.org/legacy/
- name: Publish to PyPI
if: ${{ !inputs.dry_run }}
run: uv publish
+13
View File
@@ -8,6 +8,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Added
**Vision input for `jarvis ask`** — attach images to a query with
`-i`/`--image` (repeatable) or capture the current screen with
`-S`/`--screen`, for vision-capable models such as `gemma3:4b`. Images flow
through `Message.images` into Ollama's `/api/chat` `images` field; text-only
requests are unaffected. A privacy guard warns before any image is sent to a
non-local engine, and the security guardrail now preserves images when it
sanitizes a flagged prompt. Screen capture uses the built-in Windows .NET
stack with `mss`/`Pillow` fallbacks on other platforms. Adds the
`JARVIS_NUM_CTX` environment variable to tune the Ollama context window
(default `16384`).
## [1.0.2] - 2026-05-24
A patch release that fixes a packaging bug which broke the v1.0.1
+19
View File
@@ -0,0 +1,19 @@
.PHONY: setup build test lint format
# Mirrors .github/workflows/ci.yml so `make test` matches CI locally.
setup:
uv sync --extra dev --extra framework-comparison --extra server
build:
uv run maturin develop --manifest-path rust/crates/openjarvis-python/Cargo.toml
test: build
uv run pytest tests/ -n auto -q --tb=short -m "not live and not cloud and not hub"
lint:
uv run ruff check src/ tests/
uv run ruff format --check src/ tests/
format:
uv run ruff format src/ tests/
+3 -2
View File
@@ -4,7 +4,8 @@
<p><i>Personal AI, On Personal Devices.</i></p>
<p>
<a href="https://scalingintelligence.stanford.edu/blogs/openjarvis/"><img src="https://img.shields.io/badge/project-OpenJarvis-blue" alt="Project"></a>
<a href="https://arxiv.org/abs/2605.17172"><img src="https://img.shields.io/badge/arXiv-2605.17172-b31b1b.svg" alt="arXiv"></a>
<a href="https://openjarvis.stanford.edu/"><img src="https://img.shields.io/badge/project-OpenJarvis-blue" alt="Project"></a>
<a href="https://open-jarvis.github.io/OpenJarvis/"><img src="https://img.shields.io/badge/docs-mkdocs-blue" alt="Docs"></a>
<img src="https://img.shields.io/badge/python-%3E%3D3.10-blue" alt="Python">
<img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License">
@@ -23,7 +24,7 @@
> **[Documentation](https://open-jarvis.github.io/OpenJarvis/)**
>
> **[Project Site](https://scalingintelligence.stanford.edu/blogs/openjarvis/)**
> **[Project Site](https://openjarvis.stanford.edu/)**
>
> **[Paper](https://arxiv.org/abs/2605.17172)**
>
+6 -1
View File
@@ -106,6 +106,11 @@ enabled = true # Record traces for analysis
db_path = "~/.openjarvis/traces.db"
[server]
host = "0.0.0.0"
# Bind to loopback by default so the API is not exposed to the local network.
# To serve other devices on your LAN, set host = "0.0.0.0" AND set an API key
# (OPENJARVIS_API_KEY / `jarvis auth generate-key`) — startup refuses a
# non-loopback bind without a key. The "server" security profile also flips
# this to 0.0.0.0 intentionally.
host = "127.0.0.1"
port = 8000
agent = "native_openhands"
@@ -0,0 +1,39 @@
# Full system access: unrestricted shell and filesystem
# Copy to ~/.openjarvis/config.toml
#
# WARNING: shell_exec runs arbitrary commands as your user. No command
# allowlist, no denylist, no working-directory restriction. file_read and
# file_write aren't restricted to any directory either. Only enable what you
# actually want the agent to have. tools.enabled is the whole permission grant;
# there's no second allowlist to configure.
#
# On macOS, this config alone does not reach TCC-protected data (Messages,
# Mail, Photos, Safari). That requires Full Disk Access granted to the process
# hosting the backend. See docs/user-guide/system-access.md.
#
# Usage:
# jarvis ask "What's using the most disk space in my home directory?"
# jarvis chat # prompts before each shell_exec call
[engine]
default = "ollama"
[intelligence]
default_model = "qwen3.5:9b"
[agent]
default_agent = "orchestrator"
max_turns = 10
[tools]
enabled = [
"shell_exec",
"file_read",
"file_write",
"apply_patch",
"code_interpreter",
"git_status",
"git_diff",
"think",
"calculator",
]
+56 -7
View File
@@ -1,34 +1,83 @@
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers — reproducible builds and
# safe rollbacks (#563).
# Stage 1: Build frontend SPA
FROM node:22-slim AS frontend
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS frontend
# Public Supabase anon key for the savings leaderboard; empty by default so
# the image's leaderboard stays disabled (#589). Pass --build-arg to enable.
ARG OPENJARVIS_LEADERBOARD_PUBLIC_ANON=
WORKDIR /frontend
COPY frontend/package.json frontend/package-lock.json* ./
RUN npm ci --ignore-scripts 2>/dev/null || npm install
COPY frontend/ .
RUN npm run build
RUN VITE_SUPABASE_ANON_KEY="${OPENJARVIS_LEADERBOARD_PUBLIC_ANON}" npm run build
# Stage 2: Build Python package
FROM python:3.12-slim-bookworm AS builder
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf AS builder
RUN apt-get update && \
apt-get install -y --no-install-recommends build-essential ca-certificates curl && \
rm -rf /var/lib/apt/lists/*
ENV PATH="/root/.cargo/bin:${PATH}"
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --profile minimal --default-toolchain none && \
rustup toolchain install 1.88 --profile minimal && \
rustup default 1.88
WORKDIR /app
COPY pyproject.toml README.md ./
# Install dependencies from the committed lockfile (#567). `uv export --frozen`
# reads uv.lock as-is (no re-resolution) and emits a fully pinned, hash-verified
# requirements set; `--no-deps` then installs exactly that set. This is a
# separate layer from the source copy so dependency installs stay cached when
# only application code changes.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt && \
uv pip install --system --no-deps "maturin>=1.12.6,<2"
# Copy the source and the non-src force-include paths (see pyproject
# [tool.hatch.build.targets.wheel.force-include]) before building the project.
COPY src/ src/
COPY rust/ rust/
COPY scripts/install scripts/install
COPY deploy/windows deploy/windows
# Copy built frontend into the server static directory
COPY --from=frontend /src/openjarvis/server/static src/openjarvis/server/static/
RUN pip install --no-cache-dir uv && \
uv pip install --system ".[server]"
# Install the project itself without re-resolving dependencies.
RUN uv pip install --system --no-deps . && \
maturin build --release \
--manifest-path rust/crates/openjarvis-python/Cargo.toml \
--interpreter python3 \
--out /tmp/openjarvis-rust-wheel && \
uv pip install --system --no-deps /tmp/openjarvis-rust-wheel/*.whl && \
python3 -c "import openjarvis_rust; print('openjarvis_rust ok')" && \
python3 -m pip uninstall -y maturin && \
rm -rf /tmp/openjarvis-rust-wheel rust
# Stage 3: Runtime
FROM python:3.12-slim-bookworm
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf
COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
WORKDIR /app
# Run as an unprivileged user — the server needs no root privileges, so dropping
# them limits the blast radius of a compromise (#565). The app writes only to
# $HOME (config/cache/state), which is owned by this user.
RUN groupadd --system --gid 10001 openjarvis && \
useradd --system --uid 10001 --gid openjarvis \
--create-home --home-dir /home/openjarvis openjarvis
ENV HOME=/home/openjarvis
USER openjarvis
EXPOSE 8000
ENTRYPOINT ["jarvis"]
+53 -8
View File
@@ -1,32 +1,69 @@
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers — reproducible builds and
# safe rollbacks (#563).
# Stage 1: Build frontend SPA
FROM node:22-slim AS frontend
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS frontend
# Public Supabase anon key for the savings leaderboard; empty by default so
# the image's leaderboard stays disabled (#589). Pass --build-arg to enable.
ARG OPENJARVIS_LEADERBOARD_PUBLIC_ANON=
WORKDIR /frontend
COPY frontend/package.json frontend/package-lock.json* ./
RUN npm ci --ignore-scripts 2>/dev/null || npm install
COPY frontend/ .
RUN npm run build
RUN VITE_SUPABASE_ANON_KEY="${OPENJARVIS_LEADERBOARD_PUBLIC_ANON}" npm run build
# Stage 2: Build Python package (NVIDIA CUDA 12.4)
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04 AS builder
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04@sha256:af8bd179ed3bf69d4b63b19a763662a6141f0f62ef099283f68d0b14b4bab0e3 AS builder
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip python3-venv && \
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
curl \
python3 \
python3-dev \
python3-pip \
python3-venv && \
rm -rf /var/lib/apt/lists/*
ENV PATH="/root/.cargo/bin:${PATH}"
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --profile minimal --default-toolchain none && \
rustup toolchain install 1.88 --profile minimal && \
rustup default 1.88
WORKDIR /app
COPY pyproject.toml README.md ./
# Install dependencies from the committed lockfile (#567). See deploy/docker/Dockerfile
# for the rationale behind the frozen export + --no-deps install.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt && \
uv pip install --system --no-deps "maturin>=1.12.6,<2"
COPY src/ src/
COPY rust/ rust/
COPY scripts/install scripts/install
COPY deploy/windows deploy/windows
COPY --from=frontend /src/openjarvis/server/static src/openjarvis/server/static/
RUN pip install --no-cache-dir uv && \
uv pip install --system ".[server]"
RUN uv pip install --system --no-deps . && \
maturin build --release \
--manifest-path rust/crates/openjarvis-python/Cargo.toml \
--interpreter python3 \
--out /tmp/openjarvis-rust-wheel && \
uv pip install --system --no-deps /tmp/openjarvis-rust-wheel/*.whl && \
python3 -c "import openjarvis_rust; print('openjarvis_rust ok')" && \
python3 -m pip uninstall -y maturin && \
rm -rf /tmp/openjarvis-rust-wheel rust
# Stage 3: Runtime
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04@sha256:af8bd179ed3bf69d4b63b19a763662a6141f0f62ef099283f68d0b14b4bab0e3
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip && \
@@ -36,6 +73,14 @@ COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
WORKDIR /app
# Run as an unprivileged user (#565). NVIDIA device nodes (/dev/nvidia*) are
# world-accessible, so GPU workloads do not require root.
RUN groupadd --system --gid 10001 openjarvis && \
useradd --system --uid 10001 --gid openjarvis \
--create-home --home-dir /home/openjarvis openjarvis
ENV HOME=/home/openjarvis
USER openjarvis
EXPOSE 8000
ENTRYPOINT ["jarvis"]
+57 -8
View File
@@ -1,32 +1,69 @@
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers — reproducible builds and
# safe rollbacks (#563).
# Stage 1: Build frontend SPA
FROM node:22-slim AS frontend
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS frontend
# Public Supabase anon key for the savings leaderboard; empty by default so
# the image's leaderboard stays disabled (#589). Pass --build-arg to enable.
ARG OPENJARVIS_LEADERBOARD_PUBLIC_ANON=
WORKDIR /frontend
COPY frontend/package.json frontend/package-lock.json* ./
RUN npm ci --ignore-scripts 2>/dev/null || npm install
COPY frontend/ .
RUN npm run build
RUN VITE_SUPABASE_ANON_KEY="${OPENJARVIS_LEADERBOARD_PUBLIC_ANON}" npm run build
# Stage 2: Build Python package (AMD ROCm 7.2)
FROM rocm/dev-ubuntu-22.04:7.2 AS builder
FROM rocm/dev-ubuntu-22.04:7.2@sha256:05af5f04a06b04676d4c7438997d0deadaeb7478961ad621376e199bf3aeb644 AS builder
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip python3-venv && \
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
curl \
python3 \
python3-dev \
python3-pip \
python3-venv && \
rm -rf /var/lib/apt/lists/*
ENV PATH="/root/.cargo/bin:${PATH}"
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --profile minimal --default-toolchain none && \
rustup toolchain install 1.88 --profile minimal && \
rustup default 1.88
WORKDIR /app
COPY pyproject.toml README.md ./
# Install dependencies from the committed lockfile (#567). See deploy/docker/Dockerfile
# for the rationale behind the frozen export + --no-deps install.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt && \
uv pip install --system --no-deps "maturin>=1.12.6,<2"
COPY src/ src/
COPY rust/ rust/
COPY scripts/install scripts/install
COPY deploy/windows deploy/windows
COPY --from=frontend /src/openjarvis/server/static src/openjarvis/server/static/
RUN pip install --no-cache-dir uv && \
uv pip install --system ".[server]"
RUN uv pip install --system --no-deps . && \
maturin build --release \
--manifest-path rust/crates/openjarvis-python/Cargo.toml \
--interpreter python3 \
--out /tmp/openjarvis-rust-wheel && \
uv pip install --system --no-deps /tmp/openjarvis-rust-wheel/*.whl && \
python3 -c "import openjarvis_rust; print('openjarvis_rust ok')" && \
python3 -m pip uninstall -y maturin && \
rm -rf /tmp/openjarvis-rust-wheel rust
# Stage 3: Runtime
FROM rocm/dev-ubuntu-22.04:7.2
FROM rocm/dev-ubuntu-22.04:7.2@sha256:05af5f04a06b04676d4c7438997d0deadaeb7478961ad621376e199bf3aeb644
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip && \
@@ -36,6 +73,18 @@ COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
WORKDIR /app
# Run as an unprivileged user (#565). ROCm GPU access is gated by the `video` and
# `render` groups (see group_add in docker-compose.gpu.rocm.yml), so the user is
# added to both; root is not required.
RUN groupadd --system --gid 10001 openjarvis && \
useradd --system --uid 10001 --gid openjarvis \
--create-home --home-dir /home/openjarvis openjarvis && \
(getent group video >/dev/null || groupadd --system video) && \
(getent group render >/dev/null || groupadd --system render) && \
usermod -aG video,render openjarvis
ENV HOME=/home/openjarvis
USER openjarvis
EXPOSE 8000
ENTRYPOINT ["jarvis"]
+61 -7
View File
@@ -1,15 +1,69 @@
FROM python:3.12-slim
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers (#563).
# Install Node.js 22
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates && \
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
apt-get install -y nodejs && \
# Node.js is sourced from the official, digest-pinned image rather than piping a
# remote setup script into bash (`curl ... | bash -`), which performed no
# checksum or signature verification of the downloaded installer (#566). The
# image digest is the integrity check, and the copy is architecture-agnostic.
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS node
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf AS builder
RUN apt-get update && \
apt-get install -y --no-install-recommends build-essential ca-certificates curl && \
rm -rf /var/lib/apt/lists/*
ENV PATH="/root/.cargo/bin:${PATH}"
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --profile minimal --default-toolchain none && \
rustup toolchain install 1.88 --profile minimal && \
rustup default 1.88
WORKDIR /app
# Install dependencies from the committed lockfile (#567): `uv export --frozen`
# reads uv.lock as-is and emits a pinned, hash-verified set installed with
# --no-deps (no re-resolution). Copied first so this layer caches independently
# of application source.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt && \
uv pip install --system --no-deps "maturin>=1.12.6,<2"
COPY . .
RUN pip install --no-cache-dir ".[server]"
# Install the project itself without re-resolving dependencies.
RUN uv pip install --system --no-deps . && \
maturin build --release \
--manifest-path rust/crates/openjarvis-python/Cargo.toml \
--interpreter python3 \
--out /tmp/openjarvis-rust-wheel && \
uv pip install --system --no-deps /tmp/openjarvis-rust-wheel/*.whl && \
python3 -c "import openjarvis_rust; print('openjarvis_rust ok')" && \
python3 -m pip uninstall -y maturin && \
rm -rf /tmp/openjarvis-rust-wheel rust/target
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf
# libstdc++6 + ca-certificates are the only runtime requirements of the Node
# binary copied below (the python slim image already provides libc/libgcc).
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates libstdc++6 && \
rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
# Transplant the Node.js runtime from the official image. Both images are Debian
# bookworm, so the glibc/libstdc++ ABI matches.
COPY --from=node /usr/local/bin/node /usr/local/bin/node
COPY --from=node /usr/local/lib/node_modules /usr/local/lib/node_modules
RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm && \
ln -sf /usr/local/lib/node_modules/npm/bin/npx-cli.js /usr/local/bin/npx
WORKDIR /app
LABEL openjarvis-sandbox=true
+3 -1
View File
@@ -18,7 +18,9 @@ services:
capabilities: [gpu]
ollama:
image: ollama/ollama:latest
# Pinned to a fixed version + digest for reproducible deployments (#563);
# must match the tag in docker-compose.yml.
image: ollama/ollama:0.30.10@sha256:bfc9c6d53cc6989aa5131a6fde6b162b2802d4d337657f3253b5f69579bddeee
environment:
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
+3 -1
View File
@@ -18,7 +18,9 @@ services:
restart: unless-stopped
ollama:
image: ollama/ollama:latest
# Pinned to a fixed version + digest for reproducible deployments and
# predictable rollbacks (#563). Bump deliberately, not implicitly via :latest.
image: ollama/ollama:0.30.10@sha256:bfc9c6d53cc6989aa5131a6fde6b162b2802d4d337657f3253b5f69579bddeee
ports:
- "11434:11434"
volumes:
+20
View File
@@ -14,7 +14,27 @@ Environment=HOME=/opt/openjarvis
# OPENJARVIS_API_KEY=<key> (generate one: `jarvis auth generate-key`)
# It is not prefixed with "-", so the unit fails to start if the file is
# missing — preventing an accidentally unauthenticated public server.
# Keep secrets here (mode 0600, owned by root) rather than inline Environment=
# lines, which leak into `systemctl show` and the journal.
EnvironmentFile=/etc/openjarvis/env
# --- Sandboxing / hardening (#564) ---
# Conservative set: tightens the unit without blocking the server's normal I/O
# or local GPU inference. ProtectSystem=strict makes the whole filesystem
# read-only except ReadWritePaths, so $HOME (config/cache/state under
# /opt/openjarvis) stays writable.
NoNewPrivileges=true
ProtectSystem=strict
ReadWritePaths=/opt/openjarvis
ProtectHome=true
PrivateTmp=true
ProtectControlGroups=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
RestrictRealtime=true
RestrictSUIDSGID=true
LockPersonality=true
[Install]
WantedBy=multi-user.target
+3 -3
View File
@@ -20,8 +20,8 @@ What it does:
4. Installs `uv` (https://astral.sh/uv) if absent.
5. Clones the OpenJarvis repository to `%LOCALAPPDATA%\OpenJarvis`
(override with `$env:OPENJARVIS_HOME`).
6. Runs `uv sync --extra server` so the FastAPI server entry point is
importable.
6. Runs `uv sync --extra desktop --group desktop-native` so the FastAPI server,
speech backend, and native extension are importable.
7. Optionally prompts to register a scheduled task that auto-starts the
server at logon.
@@ -105,7 +105,7 @@ To pull the latest:
```powershell
cd "$env:LOCALAPPDATA\OpenJarvis\src"
git pull --ff-only
uv sync --extra server
uv sync --extra desktop --group desktop-native
```
Or re-run the installer with `-Force`:
+35 -30
View File
@@ -5,19 +5,19 @@
.DESCRIPTION
Phase-1 of the native-Windows-support RFC (#298). Mirrors the
behavior of scripts/install/install.sh (the curl-pipe-bash installer
for Linux/WSL2/macOS) but for native Windows PowerShell no WSL,
for Linux/WSL2/macOS) but for native Windows PowerShell - no WSL,
no Docker, no MSYS2.
Steps:
1. Refuse non-Windows / Windows < 10.
2. Check Python 3.10 3.13 on PATH (3.14 has no numpy wheels yet,
2. Check Python 3.10 - 3.13 on PATH (3.14 has no numpy wheels yet,
see #432).
3. Check git on PATH.
4. Install uv (https://astral.sh/uv) if absent.
5. Clone the OpenJarvis repository to $env:LOCALAPPDATA\OpenJarvis
(override with $env:OPENJARVIS_HOME).
6. Run `uv sync --extra server` so the FastAPI server entry point
is importable.
6. Run `uv sync --extra desktop --group desktop-native` so the FastAPI
server, speech backend, and native extension are importable.
7. Optionally register the scheduled-task service (see
deploy/windows/jarvis-service.ps1).
@@ -65,7 +65,7 @@ if (-not $Service -and $env:OPENJARVIS_SERVICE) { $Service = $true
if (-not $Force -and $env:OPENJARVIS_FORCE) { $Force = $true }
# ---------------------------------------------------------------------------
# Output helpers coloured but plain enough for Constrained Language Mode.
# Output helpers - coloured but plain enough for Constrained Language Mode.
# ---------------------------------------------------------------------------
function Write-Info ($msg) { Write-Host "[info] $msg" -ForegroundColor Cyan }
@@ -77,13 +77,13 @@ function Write-Fail ($msg) {
}
# ---------------------------------------------------------------------------
# Shared helpers winget bootstrap + PATH refresh
# Shared helpers - winget bootstrap + PATH refresh
# ---------------------------------------------------------------------------
# Pull the latest Machine + User PATH from the registry into the current
# PowerShell session. Tools installed by `winget install` (Python, git,
# Ollama, etc.) update the User PATH, but the running process inherits
# the parent shell's environment so without this refresh the just-
# the parent shell's environment - so without this refresh the just-
# installed tool stays invisible to subsequent `Get-Command` calls.
#
# CRITICAL: registry PATH entries can be REG_EXPAND_SZ (with literal
@@ -157,7 +157,7 @@ function Get-PythonCommand {
Write-Info "Checking Python (3.10 - 3.13)..."
$pythonExe = Get-PythonCommand
if (-not $pythonExe) {
Write-Info "Python not on PATH attempting auto-install via winget..."
Write-Info "Python not on PATH - attempting auto-install via winget..."
$pythonExe = Install-WithWinget -WingetId 'Python.Python.3.13' -CommandName 'python'
if (-not $pythonExe) {
Write-Fail @"
@@ -196,7 +196,7 @@ Write-Ok "Python $pyMajor.$pyMinor ($pythonExe)"
Write-Info "Checking git..."
$gitExe = (Get-Command git -ErrorAction SilentlyContinue).Source
if (-not $gitExe) {
Write-Info "git not on PATH attempting auto-install via winget..."
Write-Info "git not on PATH - attempting auto-install via winget..."
$gitExe = Install-WithWinget -WingetId 'Git.Git' -CommandName 'git'
if (-not $gitExe) {
Write-Fail @"
@@ -227,7 +227,7 @@ if (-not $uvExe) {
}
# The astral installer puts uv at %USERPROFILE%\.local\bin\uv.exe and
# adds that dir to the User PATH. The current process's PATH isn't
# refreshed automatically prepend the install dir so the rest of
# refreshed automatically - prepend the install dir so the rest of
# this script picks it up.
$uvDir = Join-Path $env:USERPROFILE '.local\bin'
if (Test-Path (Join-Path $uvDir 'uv.exe')) {
@@ -279,13 +279,13 @@ if (Test-Path (Join-Path $srcDir '.git')) {
}
# ---------------------------------------------------------------------------
# 6. uv sync --extra server
# 6. uv sync --extra desktop --group desktop-native
# ---------------------------------------------------------------------------
Write-Info "Running 'uv sync --extra server' in $srcDir (this can take a few minutes)..."
Write-Info "Running 'uv sync --extra desktop --group desktop-native' in $srcDir (this can take a few minutes)..."
Push-Location $srcDir
try {
& $uvExe sync --extra server
& $uvExe sync --extra desktop --group desktop-native
if ($LASTEXITCODE -ne 0) {
Write-Fail "uv sync failed with exit code $LASTEXITCODE. Check the output above."
}
@@ -295,13 +295,13 @@ try {
Write-Ok "Dependencies installed"
# ---------------------------------------------------------------------------
# 7. Ollama install + start + wait for daemon
# 7. Ollama - install + start + wait for daemon
# ---------------------------------------------------------------------------
Write-Info "Checking Ollama..."
$ollamaExe = (Get-Command ollama -ErrorAction SilentlyContinue).Source
if (-not $ollamaExe) {
Write-Info " Ollama not on PATH downloading the official installer (~150 MB)..."
Write-Info " Ollama not on PATH - downloading the official installer (~150 MB)..."
$ollamaSetup = Join-Path $env:TEMP 'OllamaSetup.exe'
# SilentlyContinue is load-bearing in PS 5.1: the default progress
# bar renderer slows Invoke-WebRequest down 30x on large downloads
@@ -340,13 +340,18 @@ Write-Ok "Ollama ($ollamaExe)"
Write-Info "Waiting for Ollama daemon..."
$ollamaReady = $false
for ($i = 0; $i -lt 60; $i++) {
& $ollamaExe list 2>&1 | Out-Null
# 'ollama list' writes to stderr until the daemon is reachable; under
# $ErrorActionPreference='Stop' the 2>&1 merge surfaces that as a
# terminating NativeCommandError that would abort the whole install on
# the very first probe. Swallow it and rely on $LASTEXITCODE so the
# Start-Process serve fallback below actually runs (issue #522).
try { & $ollamaExe list 2>&1 | Out-Null } catch { }
if ($LASTEXITCODE -eq 0) {
$ollamaReady = $true
break
}
if ($i -eq 5) {
# Daemon clearly isn't auto-running start it ourselves. Ollama
# Daemon clearly isn't auto-running - start it ourselves. Ollama
# for Windows uses the tray app `ollama app.exe`; falling back to
# `ollama serve` works headless.
Start-Process -FilePath $ollamaExe -ArgumentList 'serve' -WindowStyle Hidden -ErrorAction SilentlyContinue
@@ -354,11 +359,11 @@ for ($i = 0; $i -lt 60; $i++) {
Start-Sleep -Seconds 1
}
if (-not $ollamaReady) {
Write-Warn2 "Ollama daemon didn't become ready in 60s. Continuing bg-orchestrator will retry later."
Write-Warn2 "Ollama daemon didn't become ready in 60s. Continuing - bg-orchestrator will retry later."
}
# ---------------------------------------------------------------------------
# 8. Pull a starter model (qwen3.5:2b ~1.5 GB)
# 8. Pull a starter model (qwen3.5:2b - ~1.5 GB)
# ---------------------------------------------------------------------------
$modelPullOk = $false
@@ -372,11 +377,11 @@ if ($ollamaReady) {
Write-Warn2 "ollama pull failed; the bg-orchestrator will retry once Ollama is reachable."
}
} else {
Write-Warn2 "Skipping model pull daemon wasn't ready."
Write-Warn2 "Skipping model pull - daemon wasn't ready."
}
# ---------------------------------------------------------------------------
# 9. jarvis.cmd shim so bare `jarvis` works in any new PowerShell
# 9. jarvis.cmd shim - so bare `jarvis` works in any new PowerShell
# ---------------------------------------------------------------------------
$binDir = Join-Path $installRoot 'bin'
@@ -387,7 +392,7 @@ if (-not (Test-Path $binDir)) {
}
# %~dp0 in a .cmd file resolves to the directory containing the script,
# so the shim is self-locating moving %LOCALAPPDATA%\OpenJarvis won't
# so the shim is self-locating - moving %LOCALAPPDATA%\OpenJarvis won't
# break it as long as the user moves the whole tree. `uv` is resolved
# from PATH at runtime (astral installer adds it to User PATH); avoids
# pinning to the install-time uv.exe path which can shift on uv updates.
@@ -400,7 +405,7 @@ uv run --project "%SRC%" jarvis %*
Set-Content -Path $shimPath -Value $shimContent -Encoding ASCII
# Add %LOCALAPPDATA%\OpenJarvis\bin to User PATH if it isn't already
# there. The current process won't see it until restart handled in the
# there. The current process won't see it until restart - handled in the
# final banner.
#
# Compare against the EXPANDED form: a previous install may have written
@@ -430,7 +435,7 @@ Write-Ok "jarvis shim installed at $shimPath"
$serviceScript = Join-Path $srcDir 'deploy\windows\jarvis-service.ps1'
$shouldInstallService = $false
# Pre-check admin if the user wants the service Register-ScheduledTask
# Pre-check admin if the user wants the service - Register-ScheduledTask
# requires elevation. We do this before the prompt so we don't ask "do
# you want the service?" only to fail with Access Denied after they say
# yes.
@@ -439,7 +444,7 @@ $isAdmin = ([Security.Principal.WindowsPrincipal] `
).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
if ($Service -and -not $isAdmin) {
Write-Fail "-Service was requested, but this PowerShell is not elevated. Register-ScheduledTask needs admin rights re-run from an elevated PowerShell, or drop -Service."
Write-Fail "-Service was requested, but this PowerShell is not elevated. Register-ScheduledTask needs admin rights - re-run from an elevated PowerShell, or drop -Service."
}
if ($Service) {
$shouldInstallService = $true
@@ -448,7 +453,7 @@ if ($Service) {
} elseif (-not $isAdmin) {
# Default to skip-with-explanation when we can't elevate, rather
# than prompting and then failing at Register-ScheduledTask.
Write-Warn2 "Skipping scheduled-task setup this PowerShell is not elevated."
Write-Warn2 "Skipping scheduled-task setup - this PowerShell is not elevated."
Write-Warn2 " Register-ScheduledTask requires admin. To install the service later:"
Write-Warn2 " Right-click PowerShell -> Run as administrator, then run:"
Write-Warn2 " powershell -ExecutionPolicy Bypass -File `"$serviceScript`" install"
@@ -464,7 +469,7 @@ if ($Service) {
$reply = Read-Host "Register OpenJarvis as a Windows scheduled task (auto-start at logon, loopback only)? [y/N]"
$shouldInstallService = ($reply -match '^[yY]')
} else {
Write-Warn2 "Non-interactive install skipping scheduled-task setup."
Write-Warn2 "Non-interactive install - skipping scheduled-task setup."
Write-Warn2 "To register the service later, run (from an elevated PowerShell):"
Write-Warn2 " powershell -ExecutionPolicy Bypass -File `"$serviceScript`" install"
}
@@ -487,9 +492,9 @@ if ($shouldInstallService) {
# ---------------------------------------------------------------------------
Write-Host ""
Write-Host " ┌──────────────────────────────────┐" -ForegroundColor Green
Write-Host " OpenJarvis install complete " -ForegroundColor Green
Write-Host " └──────────────────────────────────┘" -ForegroundColor Green
Write-Host " +----------------------------------+" -ForegroundColor Green
Write-Host " | OpenJarvis install complete |" -ForegroundColor Green
Write-Host " +----------------------------------+" -ForegroundColor Green
Write-Host ""
Write-Host " Repo: $srcDir"
+1 -1
View File
@@ -109,7 +109,7 @@ If you prefer to run each step yourself:
```bash
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync --extra server
uv sync --extra desktop
cd frontend && npm install && cd ..
```
+41 -1
View File
@@ -17,6 +17,46 @@ The configuration file lives at:
OpenJarvis creates the `~/.openjarvis/` directory and populates it with a default config when you run `jarvis init`.
## Relocating the OpenJarvis directory
OpenJarvis keeps **all** of its state — config, databases, caches, logs,
credentials, skills, recipes, connectors — under a **single root** so it never
clutters your home directory beyond one folder. By default that root is
`~/.openjarvis`, but you can move it.
The root is resolved in priority order:
1. **`$OPENJARVIS_HOME`** — explicit override. Honored by both the installer
and the Python runtime.
2. **`$XDG_DATA_HOME/openjarvis`** — used when `$XDG_DATA_HOME` is set (a single
`openjarvis` directory nested under it, per the XDG Base Directory spec).
3. **`~/.openjarvis`** — the default. With no environment variables set, the
resolved path is exactly this, so existing installs are untouched.
```bash
# Relocate the whole install + runtime tree at install time:
OPENJARVIS_HOME=~/apps/openjarvis curl -fsSL https://open-jarvis.github.io/OpenJarvis/install.sh | bash
# Or for a single run / your shell profile:
export OPENJARVIS_HOME=~/apps/openjarvis
```
Confirm where your data lives with:
```bash
jarvis config path
```
!!! note "Migration"
Because the default is unchanged, **no data migration is required** for
existing installs. If you set `OPENJARVIS_HOME` (or `XDG_DATA_HOME`) on a
machine that already has data in `~/.openjarvis`, OpenJarvis will look in
the new location and not see your old data — move it yourself if you want
to keep it: `mv ~/.openjarvis "$OPENJARVIS_HOME"`.
`$OPENJARVIS_CONFIG` still points at an explicit `config.toml` file
independently of the root, if you need to override just the config file path.
## Generating Configuration
### First-Time Setup
@@ -564,7 +604,7 @@ enforce_tool_confirmation = true
| `scan_output` | bool | `true` | Whether to scan model output. |
| `secret_scanner` | bool | `true` | Enable secret detection (API keys, tokens, passwords). |
| `pii_scanner` | bool | `true` | Enable PII detection (emails, SSNs, credit cards). |
| `enforce_tool_confirmation` | bool | `true` | Require confirmation before executing tools. |
| `enforce_tool_confirmation` | bool | `true` | Accepted but **not currently enforced**. Whether you get prompts depends on the entry point. See [System Access](../user-guide/system-access.md#confirmation-behaviour). |
!!! tip "Choosing a security mode"
Use `"warn"` during development to see what would be flagged without disrupting output.
+3 -2
View File
@@ -41,7 +41,7 @@ If you prefer to run each step yourself:
```bash
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync --extra server
uv sync --extra desktop
uv run maturin develop -m rust/crates/openjarvis-python/Cargo.toml
cd frontend && npm install && cd ..
```
@@ -278,6 +278,7 @@ OpenJarvis uses optional extras to keep the base installation lightweight.
| Extra | Install Command | Description |
|-------|----------------|-------------|
| `desktop` | `uv sync --extra desktop` | Desktop/API server plus local speech input |
| `server` | `uv sync --extra server` | OpenAI-compatible API server (`jarvis serve`) |
| `dev` | `uv sync --extra dev` | Development and testing tools |
| `docs` | `uv sync --extra docs` | Documentation build tools |
@@ -285,7 +286,7 @@ OpenJarvis uses optional extras to keep the base installation lightweight.
Combine extras:
```bash
uv sync --extra server --extra memory-faiss --extra inference-cloud
uv sync --extra desktop --extra memory-faiss --extra inference-cloud
```
## Setting Up an Inference Backend
+13
View File
@@ -135,6 +135,19 @@ cd OpenJarvis
This launches the backend API server and a React frontend at [http://localhost:5173](http://localhost:5173).
You get a ChatGPT-like interface with streaming responses, tool use, energy monitoring, and a telemetry dashboard — all running locally on your hardware.
Web search is available through the built-in DuckDuckGo fallback. To use
Tavily, add `TAVILY_API_KEY` under **Settings → Tools → Web Search** after the
app starts, or export it before starting quickstart:
```bash
export TAVILY_API_KEY="tvly-..."
./scripts/quickstart.sh
```
The script does not automatically source `.env` files. Run `source .env`
first if that is where you keep the key. Stop any existing OpenJarvis server
before restarting so it inherits the updated environment.
To stop all services, press ++ctrl+c++ in the terminal.
!!! tip "Environment variable"
+2 -2
View File
@@ -8,7 +8,7 @@ avoid a Linux VM; WSL2 remains the smoother experience for most users.
## What you get
- A PowerShell installer that probes prerequisites, installs `uv`,
clones the repo, and runs `uv sync --extra server`.
clones the repo, and runs `uv sync --extra desktop --group desktop-native`.
- An optional Windows scheduled-task service equivalent to the systemd
unit and launchd plist.
- Loopback default — the service binds `127.0.0.1` so no API key is
@@ -38,7 +38,7 @@ The installer will:
4. Install `uv` if absent (via the official `astral.sh/uv` PowerShell
installer).
5. Clone the repo to `%LOCALAPPDATA%\OpenJarvis\src`.
6. Run `uv sync --extra server`.
6. Run `uv sync --extra desktop --group desktop-native`.
7. Prompt to register the scheduled-task service (skip with
`-SkipService`).
+2 -2
View File
@@ -183,7 +183,7 @@ OpenJarvis is built around five composable layers. Each has a clean interface an
---
CLI, Python SDK, and guides for [Morning Digest](user-guide/morning-digest.md), [Deep Research](user-guide/deep-research.md), [Code Assistant](user-guide/code-assistant.md), [Scheduled Monitor](user-guide/scheduled-monitor.md), [Simple Chat](user-guide/chat-simple.md), agents, memory, tools, and telemetry.
CLI, Python SDK, and guides for [Morning Digest](user-guide/morning-digest.md), [Deep Research](user-guide/deep-research.md), [Code Assistant](user-guide/code-assistant.md), [Scheduled Monitor](user-guide/scheduled-monitor.md), [Simple Chat](user-guide/chat-simple.md), [Evaluations](user-guide/evaluations.md), agents, memory, tools, and telemetry.
- **[Architecture](architecture/overview.md)**
@@ -215,7 +215,7 @@ OpenJarvis is built around five composable layers. Each has a clean interface an
OpenJarvis is part of [Intelligence Per Watt](https://www.intelligence-per-watt.ai/), a research initiative studying the efficiency of on-device AI systems. Developed at [Hazy Research](https://hazyresearch.stanford.edu/) and the [Scaling Intelligence Lab](https://scalingintelligence.stanford.edu/) at [Stanford SAIL](https://ai.stanford.edu/).
Read the [blog post](https://scalingintelligence.stanford.edu/blogs/openjarvis/) for the full research motivation, architecture details, and experimental results.
Read the [blog post](https://openjarvis.stanford.edu/) for the full research motivation, architecture details, and experimental results.
## Citation
+12
View File
@@ -0,0 +1,12 @@
// Public Supabase config for the savings leaderboard.
//
// This file is loaded *before* leaderboard.js and supplies the anon key it
// reads from `window.OPENJARVIS_SUPABASE_ANON_KEY`. The key is injected at
// docs-build time from the VITE_SUPABASE_ANON_KEY repo secret (see
// .github/workflows/docs.yml). It is intentionally empty here so that local
// `mkdocs build` and fork pull requests — which have no secret — render the
// graceful "Leaderboard not configured yet" message instead of failing.
//
// The anon key is public by design: Supabase Row-Level Security protects the
// data, so shipping it in the public docs bundle is expected.
window.OPENJARVIS_SUPABASE_ANON_KEY = "";
+3 -3
View File
@@ -1,9 +1,9 @@
(function () {
"use strict";
var SUPABASE_URL = "https://mtbtgpwzrbostweaanpr.supabase.co";
var SUPABASE_ANON_KEY =
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Im10YnRncHd6cmJvc3R3ZWFhbnByIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NzMxODk0OTQsImV4cCI6MjA4ODc2NTQ5NH0._xMlqCfljtXpwPj54H-ghxfLFO-jiq4W2WhpU8vVL1c";
var SUPABASE_URL =
window.OPENJARVIS_SUPABASE_URL || "https://mtbtgpwzrbostweaanpr.supabase.co";
var SUPABASE_ANON_KEY = window.OPENJARVIS_SUPABASE_ANON_KEY || "";
var PAGE_SIZE = 50;
var allRows = [];
+1 -1
View File
@@ -55,5 +55,5 @@ See how the OpenJarvis community saves money, energy, and compute by running AI
<div id="leaderboard-pagination" class="lb-pagination"></div>
<p style="font-size:12px;opacity:0.6;margin-top:12px">
*Dollar savings estimated vs. Claude Opus 4.6 API pricing ($5/1M input, $25/1M output tokens). Assumes local open-source models produce roughly the same number of tokens per request as cloud models.
*Dollar savings estimated vs. Claude Fable 5 API pricing ($10/1M input, $50/1M output tokens). Assumes local open-source models produce roughly the same number of tokens per request as cloud models.
</p>
+65
View File
@@ -19,6 +19,71 @@ Agents are the agentic logic layer of OpenJarvis. They determine how a query is
---
## Persistent Persona: SOUL.md, MEMORY.md, USER.md
Every agent's system prompt is assembled at conversation start by the `SystemPromptBuilder`, which injects up to three optional Markdown files -- the **persistent persona**. They are plain text you own and edit, loaded at the start of each conversation. There is no vector database or embedding cache behind them.
| File | What it holds | Example line |
|------|---------------|--------------|
| `SOUL.md` | How the agent should behave -- tone, length, what to push back on | `Be concise. Challenge weak assumptions.` |
| `MEMORY.md` | Facts about you, your projects, your preferences | `I deploy to Postgres, never MySQL.` |
| `USER.md` | Who you are -- role, team, context | `Backend engineer at Acme, on the payments team.` |
This persona is distinct from the retrieval [memory backend](memory.md): the persona is always-on Markdown context loaded into the prompt, while the memory backend is searchable long-term storage the agent queries on demand.
### Where they live
By default the files are read from the config directory:
```
~/.openjarvis/SOUL.md
~/.openjarvis/MEMORY.md
~/.openjarvis/USER.md
```
(The config directory honors `$OPENJARVIS_HOME` / `$XDG_DATA_HOME` when set.) The paths are configurable under `[memory_files]`:
```toml
[memory_files]
soul_path = "~/.openjarvis/SOUL.md"
memory_path = "~/.openjarvis/MEMORY.md"
user_path = "~/.openjarvis/USER.md"
persona_name = "" # optional named persona -- see below
```
### How they're loaded
At the start of each conversation, `SystemPromptBuilder` reads each file as UTF-8 and adds its contents as a section of the system prompt, after the agent template and before the skill catalog:
- **All three are optional.** A missing or empty file is skipped, so any subset works and an install with no persona files behaves exactly as before.
- **Edits apply to the next conversation.** The files are read once when a conversation's prompt is built, so there is no restart or re-indexing -- edit or delete a line and it takes effect the next time you start a conversation.
- **Each section is length-capped.** Files are truncated to a per-section character budget so a large `MEMORY.md` cannot crowd out the rest of the prompt.
### Named personas
A single install can answer as different personas without changing global config. A named persona lives in its own directory:
```
~/.openjarvis/personas/<name>/SOUL.md
~/.openjarvis/personas/<name>/MEMORY.md
~/.openjarvis/personas/<name>/USER.md
```
Select one per invocation, or opt out entirely:
```bash
jarvis ask --persona work "summarize my open PRs"
jarvis ask --persona none "what is 2 + 2?" # inject no persona
```
Set `persona_name` under `[memory_files]` to make a named persona the default. `persona_name = "none"` (equivalently `--persona none`) disables persona injection for that run.
### Editing them
`SOUL.md`, `MEMORY.md`, and `USER.md` are plain Markdown -- open them in any editor. `MEMORY.md` and `USER.md` can also be updated by the agent itself through the `memory_manage` and `user_profile_manage` tools when those are enabled, so the agent can record a new fact mid-conversation. These tools always target the default `MEMORY.md` and `USER.md` (under `~/.openjarvis/`), never a named persona's copies -- edit those by hand.
---
## BaseAgent ABC
All agents extend the abstract `BaseAgent` class.
+35
View File
@@ -66,6 +66,8 @@ jarvis ask "What is the capital of France?"
| `--no-context` | flag | off | Disable memory context injection |
| `-a`, `--agent AGENT` | string | none | Agent to use (`simple`, `orchestrator`) |
| `--tools TOOLS` | string | none | Comma-separated tool names to enable |
| `-i`, `--image PATH` | path | none | Image file for a vision model (e.g. `gemma3:4b`); repeatable |
| `-S`, `--screen` | flag | off | Capture the current screen and send it to the vision model |
### Direct Mode vs Agent Mode
@@ -105,6 +107,39 @@ jarvis ask --no-context "Tell me about Python"
jarvis ask --max-tokens 2048 "Write a detailed essay about AI"
```
### Vision Input
Vision-capable models (such as `gemma3:4b`) can read images alongside your
text prompt. Attach one or more image files with `-i`/`--image`, or capture
the current screen with `-S`/`--screen`:
```bash
# Ask about a local image
jarvis ask -i screenshot.png "What is shown in this image?"
# Send multiple images (the flag is repeatable)
jarvis ask -i chart-a.png -i chart-b.png "Compare these two charts"
# Capture the current screen and ask about it
jarvis ask --screen "Summarize what's on my screen"
```
Vision runs in **direct mode** only. If you also pass `--agent`, the image is
ignored and a note is printed — re-run with `--agent ""` to force direct mode.
The Ollama context window can be tuned for large images or long prompts with
the `JARVIS_NUM_CTX` environment variable (default `16384`):
```bash
JARVIS_NUM_CTX=8192 jarvis ask --screen "What's on my screen?"
```
!!! note "Keep vision on-device"
Images are sensitive. OpenJarvis prints a privacy warning before sending
an image to a non-local engine, so a screenshot never leaves your machine
unnoticed. Use a local engine (e.g. `ollama` with `gemma3:4b`) to keep
vision fully local.
### JSON Output Format
When using `--json` in **direct mode**, the output includes:
+191 -55
View File
@@ -1,14 +1,14 @@
# Evaluations
The OpenJarvis evaluation framework (`openjarvis-evals`) measures model **correctness and accuracy** on academic datasets. It is a separate package from the main OpenJarvis library and is designed specifically for research workflows where you need reproducible, dataset-driven quality assessments.
The OpenJarvis evaluation framework (`openjarvis.evals`) measures model **correctness and accuracy** on academic datasets. It ships inside the main `openjarvis` package (at `src/openjarvis/evals/`) and is designed specifically for research workflows where you need reproducible, dataset-driven quality assessments.
!!! info "Evals vs. Benchmarks"
OpenJarvis has two distinct measurement systems that complement each other:
| System | Package | Measures | Entry Point |
|--------|---------|----------|-------------|
| **Evaluations** | `openjarvis-evals` | Correctness on academic datasets (accuracy, pass rate) | `openjarvis-eval` |
| **Benchmarks** | `openjarvis` | Engine performance (latency, throughput) | `jarvis bench` |
| System | Module | Measures | Entry Point |
|--------|--------|----------|-------------|
| **Evaluations** | `openjarvis.evals` | Correctness on academic datasets (accuracy, pass rate) | `jarvis eval` |
| **Benchmarks** | `openjarvis.bench` | Engine performance (latency, throughput) | `jarvis bench` |
Use evaluations to answer "does this model get the right answer?" and benchmarks to answer "how fast does this model respond?". See the [Benchmarks guide](benchmarks.md) for the performance measurement system.
@@ -18,22 +18,38 @@ The OpenJarvis evaluation framework (`openjarvis-evals`) measures model **correc
## Installation
The evaluation framework is a standalone package in the `evals/` directory. Install it alongside OpenJarvis:
The evaluation framework is part of the main `openjarvis` package — no separate install or extra is required. The standard dev setup is enough:
```bash
uv sync --extra eval
uv sync --extra dev
```
This installs the `openjarvis-eval` CLI entry point and all required dependencies (`datasets`, `huggingface-hub`, `tqdm`, `rich`).
The framework's core dependencies (`click`, `datasets`, `rich`) are base dependencies of `openjarvis`. Two optional extras enable experiment tracking integrations:
```bash
uv sync --extra dev --extra eval-wandb # Weights & Biases run tracking
uv sync --extra dev --extra eval-sheets # Google Sheets results export
```
!!! note "Python version requirement"
Python 3.10 requires the `tomli` package for TOML config parsing. The `evals/pyproject.toml` includes this as a conditional dependency, so it is installed automatically.
Python 3.10 requires the `tomli` package for TOML config parsing. `openjarvis` declares it as a conditional dependency, so it is installed automatically.
## Entry Points
Two equivalent entry points expose the framework:
| Command | Surface |
|---------|---------|
| `jarvis eval {list,run,compare,report}` | Canonical CLI. `run` covers the common options; `compare` and `report` post-process result files. |
| `python -m openjarvis.evals {list,run,run-all,summarize,reparse-judge}` | Full research surface, including judge configuration, the agentic runner, and episode mode. |
The `openjarvis-eval` console script is an alias for `python -m openjarvis.evals` — same commands, same options. This guide uses `jarvis eval` wherever its option set suffices and the module form for research-only options.
---
## Datasets
The framework ships with **30+ datasets** covering academic reasoning, agentic tasks, retrieval, conversation quality, and practical use-case benchmarks. Datasets are grouped by category below.
The framework ships with **40 registered benchmarks** covering academic reasoning, agentic tasks, coding, retrieval, conversation quality, and practical use-case benchmarks. Datasets are grouped by category below; `uv run python -m openjarvis.evals list` prints the authoritative registry.
### Use-Case Benchmarks
@@ -64,6 +80,7 @@ These benchmarks measure reasoning and knowledge on established academic dataset
| **MATH-500** | `math500` | reasoning | Competition-level math problems |
| **NaturalReasoning** | `natural-reasoning` | reasoning | Natural language reasoning |
| **HLE** | `hle` | reasoning | Humanity's Last Exam hard challenges |
| **LiveResearchBench** | `liveresearchbench` | reasoning | Recent research comprehension (Salesforce) |
| **SimpleQA** | `simpleqa` | chat | Short-form factual question answering |
| **IPW** | `ipw` | chat | Intelligence Per Watt mixed benchmark |
@@ -79,6 +96,11 @@ These benchmarks test multi-step agent capabilities including tool use, code gen
| **TerminalBench** | `terminalbench` | agentic | Terminal-based task completion |
| **TerminalBench Native** | `terminalbench-native` | agentic | TerminalBench with native Docker execution |
| **TerminalBench V2.1** | `terminalbench-v2.1` | agentic | TB v2.1 Harbor-style Docker tasks |
| **PinchBench** | `pinchbench` | agentic | Real-world agent tasks |
| **TauBench** | `taubench` | agentic | Multi-turn customer service |
| **DeepResearchBench** | `liveresearch` | agentic | Deep research report generation |
| **DeepResearchBench (alias)** | `deepresearch` | agentic | Same benchmark as `liveresearch` |
| **ToolCall-15** | `toolcall15` | agentic | Tool calling benchmark |
| **LifelongAgent** | `lifelong-agent` | agentic | Sequential task learning across sessions |
| **PaperArena** | `paperarena` | agentic | Scientific paper analysis |
| **DeepPlanning** | `deepplanning` | agentic | Shopping constraint planning |
@@ -87,6 +109,14 @@ These benchmarks test multi-step agent capabilities including tool use, code gen
| **WebChoreArena** | `webchorearena` | agentic | Web chore tasks |
| **WorkArena** | `workarena` | agentic | WorkArena++ enterprise workflows |
Both `liveresearch` and `deepresearch` are registered keys for the DeepResearchBench report-generation benchmark.
### Coding Benchmarks
| Dataset | Key | Category | Description |
|---------|-----|----------|-------------|
| **LiveCodeBench** | `livecodebench` | coding | Competitive programming |
### Retrieval Benchmarks
| Dataset | Key | Category | Description |
@@ -123,7 +153,7 @@ The framework includes two pre-built configs for evaluating models on the five c
### Cloud models
```bash
uv run python -m openjarvis.evals --config src/openjarvis/evals/configs/use_case_v2_cloud.toml
uv run jarvis eval run --config src/openjarvis/evals/configs/use_case_v2_cloud.toml
```
This config evaluates **6 cloud models** (Claude Opus 4.6, Claude Haiku 4.5, Gemini 3.1 Pro, Gemini 3.1 Flash Lite, GPT-5.4, GPT-5 Mini) against all 5 use-case benchmarks with 30 samples each, producing a 6x5 = 30-run matrix. Results are written to `results/use-cases-v2-cloud/`.
@@ -131,7 +161,7 @@ This config evaluates **6 cloud models** (Claude Opus 4.6, Claude Haiku 4.5, Gem
### Local models
```bash
uv run python -m openjarvis.evals --config src/openjarvis/evals/configs/use_case_v2_local.toml
uv run jarvis eval run --config src/openjarvis/evals/configs/use_case_v2_local.toml
```
This config evaluates **5 local models** via Ollama (Qwen3.5 122B-A10B, GPT-OSS 120B, GLM4, Qwen3.5 35B-A3B, GLM-4.7-Flash) against the same 5 benchmarks, producing a 5x5 = 25-run matrix. Uses 2 workers (suitable for single-GPU setups). Results are written to `results/use-cases-v2-local/`.
@@ -143,15 +173,22 @@ This config evaluates **5 local models** via Ollama (Qwen3.5 122B-A10B, GPT-OSS
## Inference Backends
Every evaluation run routes model calls through one of two backends:
Every evaluation run routes model calls through one of four backends:
| Backend | Key | Description |
|---------|-----|-------------|
| **jarvis-direct** | `jarvis-direct` | Engine-level inference via `SystemBuilder`. Works for local (Ollama, vLLM, llama.cpp) and cloud models. |
| **jarvis-agent** | `jarvis-agent` | Agent-level inference with tool calling. Uses `JarvisSystem.ask()` with the specified agent and tools. |
| **hermes** | `hermes` | Real Hermes Agent (Nous Research) via subprocess. Requires `--base-url` and `--api-key`. |
| **openclaw** | `openclaw` | Real OpenClaw via Node subprocess. Requires `--base-url` and `--api-key`. |
Use `jarvis-direct` for most evaluations. Use `jarvis-agent` when the benchmark requires tool use — for example, GAIA tasks that reference files that must be read with `file_read`, or arithmetic tasks that benefit from `calculator`.
The `hermes` and `openclaw` backends shell out to external agent frameworks and need an OpenAI-compatible endpoint for their model calls: pass `--base-url`/`--api-key`, set the `JARVIS_BACKEND_BASE_URL`/`JARVIS_BACKEND_API_KEY` environment variables, or add a `[backend.external]` section to your config (see [Config Reference](#backendexternal)).
!!! note "TerminalBench Native"
`jarvis eval run --backend` additionally accepts `terminalbench-native`, a Docker-based execution backend used by the TerminalBench Native benchmark.
---
## CLI Usage
@@ -159,73 +196,106 @@ Use `jarvis-direct` for most evaluations. Use `jarvis-agent` when the benchmark
### List available benchmarks and backends
```bash
openjarvis-eval list
uv run python -m openjarvis.evals list
```
Output:
Abridged output (40 benchmarks, 4 backends):
```
Benchmarks:
supergpqa [reasoning ] SuperGPQA multiple-choice
gaia [agentic ] GAIA agentic benchmark
frames [rag ] FRAMES multi-hop RAG
wildchat [chat ] WildChat conversation quality
Backends:
jarvis-direct Engine-level inference (local or cloud)
jarvis-agent Agent-level inference with tool calling
Available Benchmarks
┌──────────────────────┬───────────┬───────────────────────────────────┐
│ Name │ Category │ Description │
├──────────────────────┼───────────┼───────────────────────────────────┤
│ supergpqa │ reasoning │ SuperGPQA multiple-choice │
│ gpqa │ reasoning │ GPQA graduate-level MCQ │
│ ... │ ... │ ... │
│ livecodebench │ coding │ LiveCodeBench competitive progr. │
│ toolcall15 │ agentic │ ToolCall-15 tool calling benchmark│
└──────────────────────┴───────────┴───────────────────────────────────┘
Available Backends
┌───────────────┬──────────────────────────────────────────────────┐
│ jarvis-direct │ Engine-level inference (local or cloud) │
│ jarvis-agent │ Agent-level inference with tool calling │
│ hermes │ Real Hermes Agent (Nous Research) via subprocess │
│ openclaw │ Real OpenClaw via Node subprocess │
└───────────────┴──────────────────────────────────────────────────┘
```
`jarvis eval list` prints a similar table but currently shows a curated subset of the registry; the module form above is the authoritative listing.
### Run a single benchmark
```bash
# Evaluate qwen3:8b on SuperGPQA (engine-level, 10 samples default)
openjarvis-eval run -b supergpqa -m qwen3:8b
# Evaluate qwen3:8b on SuperGPQA (engine-level, 10 samples)
uv run jarvis eval run -b supergpqa -m qwen3:8b -n 10
# Evaluate GPT-4o on GAIA using the agent backend with tools
openjarvis-eval run -b gaia -m gpt-4o --backend jarvis-agent \
# Evaluate GPT-5 Mini on GAIA using the agent backend with tools
uv run jarvis eval run -b gaia -m gpt-5-mini --backend jarvis-agent \
--agent orchestrator --tools calculator,file_read -n 50
# Run FRAMES with vLLM engine, write output to a file
openjarvis-eval run -b frames -m llama3:70b -e vllm \
# Run FRAMES with the vLLM engine, write output to a file
uv run jarvis eval run -b frames -m llama3:70b -e vllm \
-o results/frames_llama70b.jsonl
# Run WildChat with a higher temperature for chat quality
openjarvis-eval run -b wildchat -m qwen3:8b --temperature 0.7 -n 100
uv run jarvis eval run -b wildchat -m qwen3:8b --temperature 0.7 -n 100
```
#### Full option reference
#### `jarvis eval run` option reference
| Option | Short | Type | Default | Description |
|--------|-------|------|---------|-------------|
| `--config` | `-c` | path | — | TOML config file; when provided, `-b` and `-m` are not required |
| `--benchmark` | `-b` | choice | required* | `supergpqa`, `gaia`, `frames`, or `wildchat` |
| `--backend` | | choice | `jarvis-direct` | `jarvis-direct` or `jarvis-agent` |
| `--model` | `-m` | str | required* | Model identifier (e.g., `qwen3:8b`, `gpt-4o`) |
| `--engine` | `-e` | str | auto | Engine key (`ollama`, `vllm`, `cloud`, ...) |
| `--agent` | | str | `orchestrator` | Agent name for `jarvis-agent` backend |
| `--tools` | | str | `""` | Comma-separated tool names (e.g., `calculator,file_read`) |
| `--benchmark` | `-b` | str | required* | Any registered benchmark key (see `... list`) |
| `--model` | `-m` | str | required* | Model identifier (e.g., `qwen3:8b`, `gpt-5-mini`) |
| `--max-samples` | `-n` | int | all | Limit the number of samples evaluated |
| `--max-workers` | `-w` | int | `4` | Parallel evaluation workers |
| `--judge-model` | | str | `gpt-4o` | LLM used for judge-based scoring |
| `--output` | `-o` | path | auto-generated | Output JSONL file path |
| `--backend` | | choice | `jarvis-direct` | `jarvis-direct`, `jarvis-agent`, `hermes`, `openclaw`, or `terminalbench-native` |
| `--base-url` | | str | — | OpenAI-compatible endpoint URL (env: `JARVIS_BACKEND_BASE_URL`) |
| `--api-key` | | str | — | API key for the endpoint (env: `JARVIS_BACKEND_API_KEY`) |
| `--agent` | | str | — | Agent name for `jarvis-agent` backend (e.g., `orchestrator`) |
| `--engine` | `-e` | str | auto | Engine key (`ollama`, `vllm`, `cloud`, ...) |
| `--tools` | | str | `""` | Comma-separated tool names (e.g., `calculator,file_read`) |
| `--telemetry/--no-telemetry` | | flag | off | Enable telemetry collection during eval |
| `--gpu-metrics/--no-gpu-metrics` | | flag | off | Enable GPU metric polling |
| `--seed` | | int | `42` | Random seed for dataset shuffling |
| `--split` | | str | dataset default | Override the dataset split |
| `--temperature` | | float | `0.0` | Generation temperature |
| `--max-tokens` | | int | `2048` | Maximum output tokens |
| `--model-filter` | | str | — | Filter models by name substring (multi-model configs) |
| `--output` | `-o` | path | auto-generated | Output JSONL file path |
| `--wandb-project` / `--wandb-entity` / `--wandb-tags` / `--wandb-group` | | str | `""` | Weights & Biases tracking (requires `eval-wandb` extra) |
| `--sheets-id` / `--sheets-worksheet` / `--sheets-creds` | | str | `""` | Google Sheets export (requires `eval-sheets` extra) |
| `--verbose` | `-v` | flag | off | Enable debug logging |
*Required when `--config` is not provided.
#### Research-only options (`python -m openjarvis.evals run`)
The module CLI accepts everything above plus research-grade options that `jarvis eval run` does not expose:
| Option | Short | Type | Default | Description |
|--------|-------|------|---------|-------------|
| `--max-workers` | `-w` | int | `4` | Parallel evaluation workers |
| `--judge-model` | | str | `gpt-5-mini-2025-08-07` | LLM used for judge-based scoring (see `--help` for the current default) |
| `--judge-engine` | | str | `cloud` | Engine key for the LLM judge; use `vllm` to judge locally |
| `--split` | | str | dataset default | Override the dataset split |
| `--compact` | | flag | off | Dense single-table output |
| `--trace-detail` | | flag | off | Full per-step trace listing |
| `--agentic` | | flag | off | Use `AgenticRunner` for multi-turn agent execution |
| `--episode-mode` | | flag | off | Sequential episode processing with lifelong learning (required for `lifelong-agent` and similar benchmarks) |
| `--concurrency` | | int | `1` | Parallel query execution (AgenticRunner only) |
| `--query-timeout` | | float | — | Per-query wall-clock timeout in seconds (AgenticRunner only) |
Note: the module CLI's `--backend` choice covers `jarvis-direct`, `jarvis-agent`, `hermes`, and `openclaw`; `terminalbench-native` as a backend is available via `jarvis eval run` and TOML configs.
### Run all benchmarks at once
The `run-all` command evaluates a single model against all four benchmarks sequentially and writes results to an output directory:
The `run-all` command (module CLI only) evaluates a single model against **every registered benchmark** sequentially and writes results to an output directory:
```bash
openjarvis-eval run-all -m qwen3:8b
uv run python -m openjarvis.evals run-all -m qwen3:8b
# With options
openjarvis-eval run-all -m gpt-4o -n 100 --output-dir results/gpt4o/
uv run python -m openjarvis.evals run-all -m gpt-5-mini -n 100 --output-dir results/gpt5mini/
```
Output files are written as `{output_dir}/{benchmark}_{model-slug}.jsonl`. The model slug replaces `/` and `:` with `-`, so `qwen3:8b` becomes `qwen3-8b`.
@@ -235,7 +305,7 @@ Output files are written as `{output_dir}/{benchmark}_{model-slug}.jsonl`. The m
After a run, inspect a JSONL results file:
```bash
openjarvis-eval summarize results/supergpqa_qwen3-8b.jsonl
uv run python -m openjarvis.evals summarize results/supergpqa_qwen3-8b.jsonl
```
Output:
@@ -251,6 +321,55 @@ Accuracy: 0.7222
Errors: 2
```
The module CLI also provides `reparse-judge`, which re-parses stored judge output in a results file and recovers records whose judge verdicts initially failed to parse — useful after improving the judge-output parser without re-running inference.
### Compare and report
`jarvis eval` adds two post-processing commands for result files:
```bash
# Side-by-side metric comparison across runs
uv run jarvis eval compare results/supergpqa_qwen3-8b.jsonl results/supergpqa_gpt-5-mini.jsonl
# Detailed report (accuracy, latency, cost, per-subject breakdown) for one run
uv run jarvis eval report results/supergpqa_qwen3-8b.jsonl
```
---
## Evaluating an Already-Running Endpoint
If you already have an OpenAI-compatible server running — `jarvis serve`, vLLM, SGLang, llama.cpp's server, or a hosted endpoint — point an eval directly at it with `--base-url` and `--api-key`:
```bash
# A vLLM server is already serving Qwen/Qwen3-8B on a GPU node:
# vllm serve Qwen/Qwen3-8B --port 8000
uv run jarvis eval run -b supergpqa -m Qwen/Qwen3-8B \
--base-url http://gpu-node:8000/v1 \
--api-key local-key \
-n 50
```
The `-m` value must match a model id the server reports at `GET /v1/models`. Both flags fall back to the `JARVIS_BACKEND_BASE_URL` and `JARVIS_BACKEND_API_KEY` environment variables, so CI jobs can set them once:
```bash
export JARVIS_BACKEND_BASE_URL=http://gpu-node:8000/v1
export JARVIS_BACKEND_API_KEY=local-key
uv run jarvis eval run -b gaia -m Qwen/Qwen3-8B --backend jarvis-agent -n 25
```
For the external `hermes` and `openclaw` backends these values are **required** (the foreign frameworks need an endpoint to send model calls to).
!!! tip "Engine-level alternative for vLLM"
The vLLM engine also honors the `VLLM_HOST` environment variable (default `http://localhost:8000`):
```bash
VLLM_HOST=http://gpu-node:8000 uv run python -m openjarvis.evals run \
-b supergpqa -m Qwen/Qwen3-8B -e vllm -n 50
```
`VLLM_HOST` is process-global — if the candidate and the judge both use the `vllm` engine, they share the same endpoint. Prefer `--base-url` when you need them separate.
---
## TOML Config System
@@ -260,7 +379,7 @@ For research workflows that compare multiple models across multiple benchmarks,
### Running from a config
```bash
openjarvis-eval run --config src/openjarvis/evals/configs/full-suite.toml
uv run jarvis eval run --config src/openjarvis/evals/configs/full-suite.toml
```
When `--config` is provided, the `-b`/`--benchmark` and `-m`/`--model` options are not required. All settings come from the config file. The CLI expands the matrix, prints a progress table, and writes results to the configured `output_dir`.
@@ -269,7 +388,7 @@ When `--config` is provided, the `-b`/`--benchmark` and `-m`/`--model` options a
A config file has six sections: `[meta]`, `[defaults]`, `[judge]`, `[run]`, `[[models]]`, and `[[benchmarks]]`. Only `[[models]]` and `[[benchmarks]]` are required — all other sections are optional and fall back to built-in defaults.
```toml title="evals/configs/full-suite.toml"
```toml title="src/openjarvis/evals/configs/full-suite.toml"
# Suite-level metadata (optional)
[meta]
name = "full-suite-v1"
@@ -353,7 +472,7 @@ For example, `temperature` is resolved as: use `[defaults].temperature` (0.0), t
A config requires only one `[[models]]` and one `[[benchmarks]]` entry:
```toml title="evals/configs/minimal.toml"
```toml title="src/openjarvis/evals/configs/minimal.toml"
[[models]]
name = "qwen3:8b"
@@ -365,7 +484,7 @@ This runs SuperGPQA against qwen3:8b with all default settings. Use this as a st
### Single-run config with full options
```toml title="evals/configs/single-run.toml"
```toml title="src/openjarvis/evals/configs/single-run.toml"
[meta]
name = "single-run-example"
description = "Evaluate SuperGPQA with a single model and full configuration"
@@ -425,7 +544,8 @@ Configuration for the LLM used as a judge in GAIA, FRAMES, and WildChat scoring.
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `model` | str | `"gpt-4o"` | Judge model identifier |
| `model` | str | `"gpt-5-mini-2025-08-07"` | Judge model identifier |
| `engine` | str | `None` | Engine key for the judge (e.g., `"vllm"` to judge locally; defaults to cloud) |
| `provider` | str | `None` | Provider override (e.g., `"openai"`) |
| `temperature` | float | `0.0` | Judge sampling temperature |
| `max_tokens` | int | `1024` | Maximum judge output tokens |
@@ -444,6 +564,20 @@ Execution settings that apply to the entire suite.
| `seed` | int | `42` | Random seed for dataset shuffling |
| `telemetry` | bool | `false` | Enable GPU telemetry capture (energy, power, utilization, throughput) |
| `gpu_metrics` | bool | `false` | Enable GPU metric polling via `pynvml` (requires `pynvml` or `nvidia-ml-py`) |
| `warmup_samples` | int | `0` | Untimed warmup samples before measurement |
| `energy_vendor` | str | `""` | GPU energy vendor override |
| `max_turns` | int | `None` | Maximum agent turns per query |
| `wandb_project` / `wandb_entity` / `wandb_tags` / `wandb_group` | str | `""` | Weights & Biases tracking |
| `sheets_spreadsheet_id` / `sheets_worksheet` / `sheets_credentials_path` | str | `""` / `"Results"` / `""` | Google Sheets export |
### `[backend.external]`
Endpoint settings for the `hermes` and `openclaw` backends. Environment variables override TOML values.
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `base_url` | str | `None` | OpenAI-compatible endpoint URL (env: `JARVIS_BACKEND_BASE_URL`) |
| `api_key` | str | `None` | API key for the endpoint (env: `JARVIS_BACKEND_API_KEY`) |
### `[[models]]`
@@ -451,7 +585,7 @@ One block per model. The `name` field is required.
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `name` | str | required | Model identifier (e.g., `"qwen3:8b"`, `"gpt-4o"`) |
| `name` | str | required | Model identifier (e.g., `"qwen3:8b"`, `"gpt-5-mini"`) |
| `engine` | str | `None` | Engine key to use (`"ollama"`, `"vllm"`, `"cloud"`, ...) |
| `provider` | str | `None` | Provider override for cloud models (e.g., `"openai"`) |
| `temperature` | float | `None` | Override `[defaults].temperature` for this model |
@@ -468,10 +602,12 @@ One block per benchmark. The `name` field is required.
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `name` | str | required | Benchmark key: `supergpqa`, `gaia`, `frames`, or `wildchat` |
| `backend` | str | `"jarvis-direct"` | Inference backend: `jarvis-direct` or `jarvis-agent` |
| `name` | str | required | Any registered benchmark key (see `uv run python -m openjarvis.evals list`) |
| `backend` | str | `"jarvis-direct"` | `jarvis-direct`, `jarvis-agent`, `hermes`, `openclaw`, or `terminalbench-native` |
| `max_samples` | int | `None` | Limit number of samples; `None` evaluates the full dataset |
| `split` | str | `None` | Override the default dataset split |
| `subset` | str | `None` | Dataset subset/variant (benchmark-specific) |
| `record_ids` | list[str] | `None` | Evaluate only these record ids |
| `agent` | str | `None` | Agent name for `jarvis-agent` backend (e.g., `"orchestrator"`) |
| `tools` | list[str] | `[]` | Tool names for `jarvis-agent` backend |
| `judge_model` | str | `None` | Override `[judge].model` for this benchmark only |
@@ -647,7 +783,7 @@ The `EvalRunner` processes samples concurrently using a `ThreadPoolExecutor`. Re
```bash
# Use more workers for faster evaluation (if the engine supports concurrent requests)
openjarvis-eval run -b supergpqa -m qwen3:8b -w 8 -n 500
uv run python -m openjarvis.evals run -b supergpqa -m qwen3:8b -w 8 -n 500
```
!!! warning "Worker count and engine load"
+1 -1
View File
@@ -392,7 +392,7 @@ enforce_tool_confirmation = true
| `secret_scanner` | `bool` | `true` | Run `SecretScanner` on all text |
| `pii_scanner` | `bool` | `true` | Run `PIIScanner` on all text |
| `audit_log_path` | `str` | `~/.openjarvis/audit.db` | Path to the SQLite audit log |
| `enforce_tool_confirmation` | `bool` | `true` | Require explicit confirmation before tool execution |
| `enforce_tool_confirmation` | `bool` | `true` | Accepted by the loader but **not currently enforced**. See [System Access](system-access.md#confirmation-behaviour) for when prompts actually happen |
!!! tip "Start with warn, tighten later"
`mode = "warn"` is a good starting point. It lets you observe what patterns are being triggered without disrupting normal usage. Switch to `"redact"` once you are satisfied that the scanner isn't producing too many false positives for your workload.
+198
View File
@@ -0,0 +1,198 @@
# System Access
How to give an agent access to the machine it runs on, and where the real
limits are.
!!! warning
`shell_exec` runs arbitrary commands as your user. There is no command
allowlist, no denylist, and no sandbox unless you turn one on. An agent
holding this tool can do anything you can do from a terminal.
---
## Start here: you probably have no tools enabled
If the agent tells you it can't run commands or read files, that's usually not
a permissions problem. It means no tools were enabled in the first place.
Tools come from `tools.enabled`, falling back to `agent.tools`. Both default to
empty, and an empty value builds the agent with **zero tools**. Nothing is
enabled by default.
First check whether you have a config file at all:
```bash
cat ~/.openjarvis/config.toml
```
If it isn't there, that's your answer. Create it:
```toml
[engine]
default = "ollama"
[intelligence]
default_model = "qwen3.5:9b"
[agent]
default_agent = "orchestrator"
[tools]
enabled = ["shell_exec", "file_read", "file_write", "think"]
```
There's a fuller version at
`configs/openjarvis/examples/full-system-access.toml`.
Then confirm the list actually resolved:
```bash
python -c "from openjarvis.core.config import load_config; print(load_config().tools.enabled)"
```
---
## What the tools reach
| Tool | Scope |
|------|-------|
| `shell_exec` | Any command, as your user. 30s default timeout, 300s max, output capped at 100 KB per stream. |
| `file_read` | Any readable path. 1 MB cap. |
| `file_write` | Any writable path. 10 MB cap, can create parent directories. |
| `apply_patch` | Applies unified diffs to any path. |
| `code_interpreter` | Python in a subprocess, behind a coarse pattern blocklist. |
`file_read` and `file_write` take an `allowed_dirs` argument that limits them to
a set of directories, but no config key populates it. When it's empty every path
is allowed. If you want a filesystem jail today, use the container sandbox
instead of relying on these tools to enforce one.
### Sensitive filenames
`file_read` and `file_write` refuse names matching a short glob list: `.env`,
`*.pem`, `id_rsa`, `credentials.*` and a dozen or so others. It matches on the
filename only, not the path or the contents, and only those two tools consult
it. `shell_exec`, `apply_patch` and `code_interpreter` skip it entirely, so
`cat ~/.ssh/id_rsa` through `shell_exec` works fine. Treat it as protection
against fat fingers, not as a security boundary.
---
## Confirmation behaviour
`shell_exec`, `git_commit` and `agent_kill` are marked `requires_confirmation`.
What that translates to depends entirely on how you launched the agent:
| Entry point | Behaviour |
|-------------|-----------|
| `jarvis chat` | Prompts before each call. |
| `jarvis ask` | Auto-approves. |
| `jarvis agent ask` | Auto-approves. Pass `--no-yes` if you want prompts. |
| HTTP server, desktop app | Auto-approves. Tools you added to an agent's toolkit count as pre-approved. |
| Embedded via `SystemBuilder` | No callback is wired, so these tools fail closed. |
That last row catches people out. If `shell_exec` returns "requires
confirmation but no confirmation callback is available", you're constructing the
agent yourself and need to pass a `confirm_callback`.
!!! note "`enforce_tool_confirmation` doesn't do anything"
The config loader accepts `security.enforce_tool_confirmation`, but nothing
on the tool execution path reads it. Setting it won't change confirmation
behaviour anywhere. Use the table above instead.
---
## macOS: Full Disk Access
On macOS the operating system is the real boundary, not the config. Shell
access and ordinary file access start working as soon as you enable the tools.
TCC-protected data does not: Messages, Mail, Photos, Safari history, Contacts
and Calendar all stay locked, and no config key will change that.
Grant Full Disk Access to whichever process hosts the backend. Child processes
inherit it:
| How you run OpenJarvis | Grant access to |
|------------------------|-----------------|
| CLI (`jarvis ask`, `jarvis chat`) | Your terminal (Terminal, iTerm, Warp) |
| Desktop app | `OpenJarvis.app`, which spawns `jarvis serve` beneath it |
| launchd (`deploy/launchd/com.openjarvis.plist`) | The `jarvis` binary, as its own entry |
System Settings, then Privacy & Security, then Full Disk Access, then **+**.
A launchd daemon gets its own TCC context, so granting access to Terminal does
nothing for it. Add `/usr/local/bin/jarvis` separately.
To check whether the grant took:
```bash
head -c 16 ~/Library/Messages/chat.db >/dev/null 2>&1 \
&& echo "granted" || echo "denied"
```
Restart the host process after you change the setting.
### Driving Mac apps
AppleScript works through `shell_exec`:
```
osascript -e 'tell application "Music" to play'
```
macOS asks for Automation permission once per target app, the first time you
touch it.
---
## What you can't do
There's no computer use. OpenJarvis can't see your screen, move the pointer or
send keystrokes. No tool for it is registered and no input automation library
appears anywhere in the codebase, so granting Accessibility or Screen Recording
buys you nothing on its own.
The `click` and `type` actions you'll find are Playwright, scoped to a browser
page rather than the desktop.
Some of this is reachable through `shell_exec` if you bring the tooling
yourself. `screencapture` will take screenshots once you've granted Screen
Recording, and something like `cliclick` will move the pointer. That gets you
scripted actions. It doesn't get you an agent that looks at the screen and
works out where to click.
---
## Narrowing access
Access widens and narrows through `tools.enabled`. Drop entries to take
capabilities away. That list is the whole grant.
Two stronger isolation options exist. Both are off by default:
```toml
[sandbox]
enabled = true # run tools inside a container
runtime = "docker"
[security.capabilities]
enabled = true # RBAC over declared tool capabilities
policy_path = "~/.openjarvis/policy.yaml"
```
!!! note "Capabilities are open by default even once enabled"
`CapabilityPolicy` is built with `default_deny=False` and no config key
exposes that flag, so an agent with no explicit policy entry gets every
capability. Write entries for every agent you mean to restrict.
For anything untrusted, reach for `docker_shell_exec` and
`code_interpreter_docker` rather than the host-side versions.
---
## See also
- [Security](security.md) for scanners, the audit log and guardrails
- [Tools](tools.md) for the full registry
- [Code Assistant](code-assistant.md) for a narrower shell-enabled setup
- [External MCP Servers](mcp-external-servers.md) for capabilities OpenJarvis doesn't ship
+67 -52
View File
@@ -11,7 +11,7 @@
"@base-ui/react": "^1.3.0",
"@fontsource-variable/geist": "^5.2.8",
"@tailwindcss/vite": "^4.2.1",
"@tauri-apps/api": "^2",
"@tauri-apps/api": "^2.11.1",
"@tauri-apps/plugin-autostart": "^2",
"@tauri-apps/plugin-dialog": "^2.7.0",
"@tauri-apps/plugin-global-shortcut": "^2",
@@ -42,7 +42,7 @@
"zustand": "^5.0.11"
},
"devDependencies": {
"@tauri-apps/cli": "^2",
"@tauri-apps/cli": "^2.11.4",
"@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0",
"@vitejs/plugin-react": "^4.3.4",
@@ -3720,9 +3720,9 @@
}
},
"node_modules/@tauri-apps/api": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/api/-/api-2.10.1.tgz",
"integrity": "sha512-hKL/jWf293UDSUN09rR69hrToyIXBb8CjGaWC7gfinvnQrBVvnLr08FeFi38gxtugAVyVcTa5/FD/Xnkb1siBw==",
"version": "2.11.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/api/-/api-2.11.1.tgz",
"integrity": "sha512-M2FPuYND2m+wh5hfW9ZpSdxMPdEJovPBWwoHJmwUpysTYNHaOkVFN419m/K0LIgjb/7KU2vBgsUepJWugQCvAA==",
"license": "Apache-2.0 OR MIT",
"funding": {
"type": "opencollective",
@@ -3730,9 +3730,9 @@
}
},
"node_modules/@tauri-apps/cli": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli/-/cli-2.10.1.tgz",
"integrity": "sha512-jQNGF/5quwORdZSSLtTluyKQ+o6SMa/AUICfhf4egCGFdMHqWssApVgYSbg+jmrZoc8e1DscNvjTnXtlHLS11g==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli/-/cli-2.11.4.tgz",
"integrity": "sha512-R8xGtMpwyetawSqm9kYOuMmEqkhUbvcUy8n0aNXIxollKBLESUu5f4Fx+64hgASYm1H+jSWq6jCW6zqTnH6hqQ==",
"dev": true,
"license": "Apache-2.0 OR MIT",
"bin": {
@@ -3746,23 +3746,23 @@
"url": "https://opencollective.com/tauri"
},
"optionalDependencies": {
"@tauri-apps/cli-darwin-arm64": "2.10.1",
"@tauri-apps/cli-darwin-x64": "2.10.1",
"@tauri-apps/cli-linux-arm-gnueabihf": "2.10.1",
"@tauri-apps/cli-linux-arm64-gnu": "2.10.1",
"@tauri-apps/cli-linux-arm64-musl": "2.10.1",
"@tauri-apps/cli-linux-riscv64-gnu": "2.10.1",
"@tauri-apps/cli-linux-x64-gnu": "2.10.1",
"@tauri-apps/cli-linux-x64-musl": "2.10.1",
"@tauri-apps/cli-win32-arm64-msvc": "2.10.1",
"@tauri-apps/cli-win32-ia32-msvc": "2.10.1",
"@tauri-apps/cli-win32-x64-msvc": "2.10.1"
"@tauri-apps/cli-darwin-arm64": "2.11.4",
"@tauri-apps/cli-darwin-x64": "2.11.4",
"@tauri-apps/cli-linux-arm-gnueabihf": "2.11.4",
"@tauri-apps/cli-linux-arm64-gnu": "2.11.4",
"@tauri-apps/cli-linux-arm64-musl": "2.11.4",
"@tauri-apps/cli-linux-riscv64-gnu": "2.11.4",
"@tauri-apps/cli-linux-x64-gnu": "2.11.4",
"@tauri-apps/cli-linux-x64-musl": "2.11.4",
"@tauri-apps/cli-win32-arm64-msvc": "2.11.4",
"@tauri-apps/cli-win32-ia32-msvc": "2.11.4",
"@tauri-apps/cli-win32-x64-msvc": "2.11.4"
}
},
"node_modules/@tauri-apps/cli-darwin-arm64": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-arm64/-/cli-darwin-arm64-2.10.1.tgz",
"integrity": "sha512-Z2OjCXiZ+fbYZy7PmP3WRnOpM9+Fy+oonKDEmUE6MwN4IGaYqgceTjwHucc/kEEYZos5GICve35f7ZiizgqEnQ==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-arm64/-/cli-darwin-arm64-2.11.4.tgz",
"integrity": "sha512-1ryOF3ZhpZ/nemHV5zVwBQBz9jDGKmKPvWPADOhc83ig0P4bMc2iER4NbC6r9sjeIZ6RVQ4g3RZIYvezhcl4TQ==",
"cpu": [
"arm64"
],
@@ -3777,9 +3777,9 @@
}
},
"node_modules/@tauri-apps/cli-darwin-x64": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-x64/-/cli-darwin-x64-2.10.1.tgz",
"integrity": "sha512-V/irQVvjPMGOTQqNj55PnQPVuH4VJP8vZCN7ajnj+ZS8Kom1tEM2hR3qbbIRoS3dBKs5mbG8yg1WC+97dq17Pw==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-x64/-/cli-darwin-x64-2.11.4.tgz",
"integrity": "sha512-uFsGQAAfuyz1k/yGLmkWfkBlgKAqZfxqlHmLWx81QU27RJWfmbNHCIq8T8w1e+VClleIuZUjpHWfoE4E3DLo3A==",
"cpu": [
"x64"
],
@@ -3794,9 +3794,9 @@
}
},
"node_modules/@tauri-apps/cli-linux-arm-gnueabihf": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm-gnueabihf/-/cli-linux-arm-gnueabihf-2.10.1.tgz",
"integrity": "sha512-Hyzwsb4VnCWKGfTw+wSt15Z2pLw2f0JdFBfq2vHBOBhvg7oi6uhKiF87hmbXOBXUZaGkyRDkCHsdzJcIfoJC2w==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm-gnueabihf/-/cli-linux-arm-gnueabihf-2.11.4.tgz",
"integrity": "sha512-IaHZn5CdBL21oUmjiVOS1ctw6Ip1O0pjp70FwOWmYz1myWe0SY96ZIj2FYf7pT0m8bI2h/hrs5ZbEXXh44/MkQ==",
"cpu": [
"arm"
],
@@ -3811,13 +3811,16 @@
}
},
"node_modules/@tauri-apps/cli-linux-arm64-gnu": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-gnu/-/cli-linux-arm64-gnu-2.10.1.tgz",
"integrity": "sha512-OyOYs2t5GkBIvyWjA1+h4CZxTcdz1OZPCWAPz5DYEfB0cnWHERTnQ/SLayQzncrT0kwRoSfSz9KxenkyJoTelA==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-gnu/-/cli-linux-arm64-gnu-2.11.4.tgz",
"integrity": "sha512-N41/ukTRVe6XSuUTESuFdGeOW2i7k62tK+6gHK5Kd5/q5RPvvi19GaWAVPPb9u95HSGmTChSolBfzynUsssFaA==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
@@ -3828,13 +3831,16 @@
}
},
"node_modules/@tauri-apps/cli-linux-arm64-musl": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.10.1.tgz",
"integrity": "sha512-MIj78PDDGjkg3NqGptDOGgfXks7SYJwhiMh8SBoZS+vfdz7yP5jN18bNaLnDhsVIPARcAhE1TlsZe/8Yxo2zqg==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.11.4.tgz",
"integrity": "sha512-v277UnT/fB64xAfSroL5N3Km3tLmvATWqJJw/wRI+g6o+HkeD0slyE7gOhNs1MbjE41R7bQOTxMVoL3aomUJmw==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
@@ -3845,13 +3851,16 @@
}
},
"node_modules/@tauri-apps/cli-linux-riscv64-gnu": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-riscv64-gnu/-/cli-linux-riscv64-gnu-2.10.1.tgz",
"integrity": "sha512-X0lvOVUg8PCVaoEtEAnpxmnkwlE1gcMDTqfhbefICKDnOTJ5Est3qL0SrWxizDackIOKBcvtpejrSiVpuJI1kw==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-riscv64-gnu/-/cli-linux-riscv64-gnu-2.11.4.tgz",
"integrity": "sha512-qqgNkQ2u1yZHxjhxsZaxUtRDW8dIqIYm33rx/mzwQv0SfY9x1B+iraj8vWeFiXjjSVVhEMepXSOts1TqPzvXNQ==",
"cpu": [
"riscv64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
@@ -3862,13 +3871,16 @@
}
},
"node_modules/@tauri-apps/cli-linux-x64-gnu": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-gnu/-/cli-linux-x64-gnu-2.10.1.tgz",
"integrity": "sha512-2/12bEzsJS9fAKybxgicCDFxYD1WEI9kO+tlDwX5znWG2GwMBaiWcmhGlZ8fi+DMe9CXlcVarMTYc0L3REIRxw==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-gnu/-/cli-linux-x64-gnu-2.11.4.tgz",
"integrity": "sha512-2VRNWl84FOH0m2giiDkO2h0QXlcMJeX+zJDpI5kDIQAx6s+geF3v48F4DXfJez4GS/FdoDGnPnw1C2iYGbQ7bQ==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
@@ -3879,13 +3891,16 @@
}
},
"node_modules/@tauri-apps/cli-linux-x64-musl": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-musl/-/cli-linux-x64-musl-2.10.1.tgz",
"integrity": "sha512-Y8J0ZzswPz50UcGOFuXGEMrxbjwKSPgXftx5qnkuMs2rmwQB5ssvLb6tn54wDSYxe7S6vlLob9vt0VKuNOaCIQ==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-musl/-/cli-linux-x64-musl-2.11.4.tgz",
"integrity": "sha512-o9GyhYor/nc7xarmwDE3ka2szuW3uuZzXjHWh64Q8YX5AtSgxdQkFWzrY4O8KiGtVNvFBI14H3Q49Qj5TOIP/A==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
@@ -3896,9 +3911,9 @@
}
},
"node_modules/@tauri-apps/cli-win32-arm64-msvc": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-arm64-msvc/-/cli-win32-arm64-msvc-2.10.1.tgz",
"integrity": "sha512-iSt5B86jHYAPJa/IlYw++SXtFPGnWtFJriHn7X0NFBVunF6zu9+/zOn8OgqIWSl8RgzhLGXQEEtGBdR4wzpVgg==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-arm64-msvc/-/cli-win32-arm64-msvc-2.11.4.tgz",
"integrity": "sha512-ld5Ehb598m0VkYyylRPNeCFsBe/km0jxis6KgMpl3IGY6I/i1RwQXO05I1AsXUXO2WC6AvB/Lw4qTf/asiuEiQ==",
"cpu": [
"arm64"
],
@@ -3913,9 +3928,9 @@
}
},
"node_modules/@tauri-apps/cli-win32-ia32-msvc": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-ia32-msvc/-/cli-win32-ia32-msvc-2.10.1.tgz",
"integrity": "sha512-gXyxgEzsFegmnWywYU5pEBURkcFN/Oo45EAwvZrHMh+zUSEAvO5E8TXsgPADYm31d1u7OQU3O3HsYfVBf2moHw==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-ia32-msvc/-/cli-win32-ia32-msvc-2.11.4.tgz",
"integrity": "sha512-12Hxi0XX/H5VFxO/bGgHkFWhml9VMgEOu9CidjeCeTNQ1l6fpUlbiGgSP7CLI3PFtW9/FfbeHieZ+kyWK5H7CA==",
"cpu": [
"ia32"
],
@@ -3930,9 +3945,9 @@
}
},
"node_modules/@tauri-apps/cli-win32-x64-msvc": {
"version": "2.10.1",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-x64-msvc/-/cli-win32-x64-msvc-2.10.1.tgz",
"integrity": "sha512-6Cn7YpPFwzChy0ERz6djKEmUehWrYlM+xTaNzGPgZocw3BD7OfwfWHKVWxXzdjEW2KfKkHddfdxK1XXTYqBRLg==",
"version": "2.11.4",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-x64-msvc/-/cli-win32-x64-msvc-2.11.4.tgz",
"integrity": "sha512-+vDiqBIU5dMISg/wNvX3sF+ZHfgJGJ5T0AcO+EHNXV9GGAG+P5fzodlDXD3QdKCRgZxMoCm5PPvj3BqLNjBthw==",
"cpu": [
"x64"
],
+2 -2
View File
@@ -18,7 +18,7 @@
"@base-ui/react": "^1.3.0",
"@fontsource-variable/geist": "^5.2.8",
"@tailwindcss/vite": "^4.2.1",
"@tauri-apps/api": "^2",
"@tauri-apps/api": "^2.11.1",
"@tauri-apps/plugin-autostart": "^2",
"@tauri-apps/plugin-dialog": "^2.7.0",
"@tauri-apps/plugin-global-shortcut": "^2",
@@ -49,7 +49,7 @@
"zustand": "^5.0.11"
},
"devDependencies": {
"@tauri-apps/cli": "^2",
"@tauri-apps/cli": "^2.11.4",
"@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0",
"@vitejs/plugin-react": "^4.3.4",
+1108 -1007
View File
File diff suppressed because it is too large Load Diff
+13
View File
@@ -24,9 +24,22 @@ serde_json = "1"
reqwest = { version = "0.12", features = ["json", "multipart"] }
tokio = { version = "1", features = ["full"] }
# Cloud API keys are stored in the OS credential store via `keyring`. keyring v3
# enables NO backend by default — without an explicit per-platform feature it
# silently falls back to a non-persistent in-memory mock, so keys would not
# survive an app restart. Each desktop target opts into its native store.
[target.'cfg(target_os = "macos")'.dependencies]
objc = "0.2"
dispatch = "0.2"
keyring = { version = "3", features = ["apple-native"] }
[target.'cfg(target_os = "windows")'.dependencies]
keyring = { version = "3", features = ["windows-native"] }
[target.'cfg(target_os = "linux")'.dependencies]
# Blocking Secret Service backend (no internal async runtime, so it is safe to
# call from the tokio-driven Tauri commands). Needs libdbus-1-dev at build time.
keyring = { version = "3", features = ["sync-secret-service", "crypto-rust"] }
[features]
default = ["custom-protocol"]
+678 -113
View File
@@ -8,8 +8,10 @@ use tokio::sync::Mutex;
const OLLAMA_PORT: u16 = 11434;
const JARVIS_PORT: u16 = 8000;
const DESKTOP_UV_SYNC_COMMAND: &str =
"uv sync --extra desktop --extra inference-cloud --extra inference-google --group desktop-native";
/// Small, fast model pulled at startup so the app opens quickly.
/// Small, fast model used when startup needs a default Ollama tag.
const STARTUP_MODEL: &str = "qwen3.5:4b";
/// Tiny fallback model if even the startup model can't be pulled.
@@ -104,7 +106,7 @@ fn default_local_model(ram_gb: f64) -> &'static str {
struct BootPlan {
/// Whether to start and wait for the bundled Ollama.
launch_ollama: bool,
/// The single Ollama model to pull (None for custom endpoints).
/// The preferred Ollama model (None for custom endpoints).
model_to_pull: Option<String>,
/// Optional `(engine_key, bare_host)` override for a custom endpoint,
/// e.g. `("lmstudio", "http://localhost:1234")`. Written into
@@ -608,6 +610,69 @@ async fn wait_for_jarvis_health(
}
async fn ollama_has_model(model: &str) -> bool {
let models = ollama_model_names().await;
matching_installed_model(&models, model).is_some()
}
fn parse_ollama_model_names(body: &serde_json::Value) -> Vec<String> {
body.get("models")
.and_then(|m| m.as_array())
.map(|models| {
models
.iter()
.filter_map(|m| {
m.get("name")
.or_else(|| m.get("model"))
.and_then(|n| n.as_str())
})
.filter(|name| !name.trim().is_empty())
.map(|name| name.to_string())
.collect()
})
.unwrap_or_default()
}
fn model_names_match(installed: &str, requested: &str) -> bool {
installed == requested
|| installed.strip_suffix(":latest") == Some(requested)
|| requested.strip_suffix(":latest") == Some(installed)
}
fn matching_installed_model(models: &[String], requested: &str) -> Option<String> {
models
.iter()
.find(|model| model_names_match(model, requested))
.cloned()
}
fn model_name_looks_embedding_only(model: &str) -> bool {
let name = model.to_ascii_lowercase();
["embed", "embedding", "rerank", "minilm", "bge-", "bge_", "e5-", "e5_"]
.iter()
.any(|marker| name.contains(marker))
}
fn preferred_installed_model(models: &[String]) -> Option<String> {
models
.iter()
.find(|model| !model.trim().is_empty() && !model_name_looks_embedding_only(model))
.or_else(|| models.iter().find(|model| !model.trim().is_empty()))
.cloned()
}
fn startup_installed_model(requested_model: &str, installed_models: &[String]) -> Option<String> {
matching_installed_model(installed_models, requested_model)
.or_else(|| preferred_installed_model(installed_models))
}
fn should_persist_resolved_model(cfg: &InferenceConfig) -> bool {
cfg.model
.as_deref()
.map(|model| model.trim().is_empty())
.unwrap_or(true)
}
async fn ollama_model_names() -> Vec<String> {
let url = format!("http://127.0.0.1:{}/api/tags", OLLAMA_PORT);
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(5))
@@ -615,21 +680,10 @@ async fn ollama_has_model(model: &str) -> bool {
.unwrap();
if let Ok(resp) = client.get(&url).send().await {
if let Ok(body) = resp.json::<serde_json::Value>().await {
if let Some(models) = body.get("models").and_then(|m| m.as_array()) {
return models.iter().any(|m| {
m.get("name")
.and_then(|n| n.as_str())
.map(|n| {
n == model
|| n.strip_suffix(":latest") == Some(model)
|| model.strip_suffix(":latest") == Some(n)
})
.unwrap_or(false)
});
}
return parse_ollama_model_names(&body);
}
}
false
Vec::new()
}
async fn pull_model(model: &str) -> Result<(), String> {
@@ -679,13 +733,21 @@ fn format_uv_sync_failure(
let code = exit_code
.map(|c| c.to_string())
.unwrap_or_else(|| "unknown".to_string());
let tail = uv_sync_stderr_tail(stderr, 800);
let rust_hint = if looks_like_rust_extension_build_error(stderr) {
format!("\n\n{}", rust_toolchain_install_hint())
} else {
String::new()
};
format!(
"`uv sync` failed in {} (exit {}). Last output:\n\n{}\n\n\
Try opening a terminal in that directory and running \
`uv sync --extra server` manually for the full output.",
`{}` manually for the full output.{}",
root.display(),
code,
uv_sync_stderr_tail(stderr, 800),
tail,
DESKTOP_UV_SYNC_COMMAND,
rust_hint,
)
}
@@ -734,6 +796,122 @@ fn format_uv_sync_spawn_error(root: &std::path::Path, uv_bin: &str, err: &str) -
)
}
fn rust_toolchain_install_hint() -> &'static str {
"The desktop app needs the Rust toolchain to build `openjarvis_rust`. \
Install Rust from https://rustup.rs. On Windows, also install Visual Studio \
Build Tools with the C++ workload, then relaunch."
}
fn looks_like_rust_extension_build_error(stderr: &str) -> bool {
let lower = stderr.to_ascii_lowercase();
[
"openjarvis-rust",
"openjarvis_rust",
"maturin",
"cargo",
"rustc",
"link.exe",
"visual studio",
]
.iter()
.any(|marker| lower.contains(marker))
}
fn format_missing_rust_toolchain() -> String {
format!(
"Could not find Rust's `cargo` command. {}\n\n\
If Rust is already installed, close and relaunch the desktop app so \
PATH includes `~/.cargo/bin`.",
rust_toolchain_install_hint(),
)
}
fn format_extension_import_failure(root: &std::path::Path, stderr: &str) -> String {
let tail = uv_sync_stderr_tail(stderr, 4000);
format!(
"`openjarvis_rust` is still not importable after building. Last output:\n\n{}\n\n\
Run these manually for the full build log:\n\n\
cd {}\n\
{}\n\
uv run python -c \"import openjarvis_rust\"",
if tail.is_empty() {
"(no stderr output)"
} else {
&tail
},
root.display(),
DESKTOP_UV_SYNC_COMMAND,
)
}
fn add_cargo_bin_to_path(cmd: &mut tokio::process::Command) {
let mut paths: Vec<std::path::PathBuf> = std::env::var_os("PATH")
.map(|path| std::env::split_paths(&path).collect())
.unwrap_or_default();
paths.insert(
0,
std::path::PathBuf::from(home_dir())
.join(".cargo")
.join("bin"),
);
if let Ok(joined) = std::env::join_paths(paths) {
cmd.env("PATH", joined);
}
}
async fn verify_openjarvis_rust_extension(
root: &std::path::Path,
uv_bin: &str,
) -> Result<(), String> {
let mut cmd = tokio::process::Command::new(uv_bin);
cmd.args(["run", "python", "-c", "import openjarvis_rust"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::piped())
.current_dir(root);
prepare_subprocess_for_appimage(&mut cmd);
add_cargo_bin_to_path(&mut cmd);
match cmd.output().await {
Ok(out) if out.status.success() => Ok(()),
Ok(out) => {
let stderr = String::from_utf8_lossy(&out.stderr);
Err(format_extension_import_failure(root, &stderr))
}
Err(e) => Err(format!(
"Could not verify `openjarvis_rust`: {}. Verify uv is installed at `{}`.",
e, uv_bin
)),
}
}
fn port_owner_hint() -> String {
if cfg!(target_os = "windows") {
format!("netstat -ano | findstr :{}", JARVIS_PORT)
} else {
format!("lsof -i :{}", JARVIS_PORT)
}
}
fn format_port_unavailable(port: u16, reason: &str) -> String {
format!(
"Port {} is not available: {}. Stop the process using that port or \
change the OpenJarvis port, then relaunch.\n\nTo identify it:\n {}",
port,
reason,
port_owner_hint(),
)
}
fn check_jarvis_port_available() -> Result<(), String> {
match std::net::TcpListener::bind(("127.0.0.1", JARVIS_PORT)) {
Ok(listener) => {
drop(listener);
Ok(())
}
Err(err) => Err(format_port_unavailable(JARVIS_PORT, &err.to_string())),
}
}
// ---------------------------------------------------------------------------
// Backend boot sequence (runs in background after app launch)
// ---------------------------------------------------------------------------
@@ -751,7 +929,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
.into();
}
// For the Ollama path, the model pull may fall back to FALLBACK_MODEL; we
// For the Ollama path, model resolution may fall back to FALLBACK_MODEL; we
// record what is actually available here so the serve command below uses
// it instead of the originally-planned tag. None on the custom path.
let mut serve_model_override: Option<String> = None;
@@ -798,8 +976,8 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
s.detail = "Inference engine ready.".into();
}
// Phase 2: Pull the single default model (see default_local_model /
// boot_plan). We deliberately do NOT pull any others.
// Phase 2: Resolve one model to serve. Prefer an installed model on
// first run so startup does not depend on a download succeeding.
let model = plan
.model_to_pull
.clone()
@@ -810,41 +988,63 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
s.detail = format!("Checking for {}...", model);
}
if !ollama_has_model(&model).await {
let installed_models = ollama_model_names().await;
let resolved_model = if let Some(installed) = startup_installed_model(&model, &installed_models) {
installed
} else {
{
let mut s = status.lock().await;
s.detail = format!("Downloading {}... (this may take a minute)", model);
}
if let Err(e) = pull_model(&model).await {
// If the chosen model fails, try the tiny fallback
eprintln!("Warning: failed to pull {}: {}", model, e);
if !ollama_has_model(FALLBACK_MODEL).await {
{
let mut s = status.lock().await;
s.detail = format!("Downloading {}...", FALLBACK_MODEL);
}
if let Err(e2) = pull_model(FALLBACK_MODEL).await {
let mut s = status.lock().await;
s.error = Some(format!("Failed to download model: {}", e2));
return;
match pull_model(&model).await {
Ok(()) => model.clone(),
Err(e) => {
eprintln!("Warning: failed to pull {}: {}", model, e);
// If a local model appeared while pulling, use it instead of
// making startup depend on another network pull.
if let Some(installed) = preferred_installed_model(&ollama_model_names().await) {
installed
} else if ollama_has_model(FALLBACK_MODEL).await {
FALLBACK_MODEL.to_string()
} else {
{
let mut s = status.lock().await;
s.detail = format!("Downloading {}...", FALLBACK_MODEL);
}
if let Err(e2) = pull_model(FALLBACK_MODEL).await {
if let Some(installed) =
preferred_installed_model(&ollama_model_names().await)
{
installed
} else {
let mut s = status.lock().await;
s.error = Some(format!("Failed to download model: {}", e2));
return;
}
} else {
FALLBACK_MODEL.to_string()
}
}
}
}
};
if resolved_model != model {
let mut s = status.lock().await;
s.detail = format!("Using installed model {}.", resolved_model);
}
// The pull may have fallen back to FALLBACK_MODEL; serve and persist
// whatever is actually available now, not the originally-planned tag.
let resolved_model = if ollama_has_model(&model).await {
model
} else {
FALLBACK_MODEL.to_string()
};
serve_model_override = Some(resolved_model.clone());
// Persist the resolved model so Settings shows it and future boots reuse it.
let mut persisted = cfg.clone();
persisted.model = Some(resolved_model);
let _ = write_inference_config(&persisted);
// Persist only first-run/default resolution. If the user explicitly
// configured a model, do not overwrite that choice with a temporary
// fallback selected just to keep startup nonfatal.
if should_persist_resolved_model(&cfg) {
let mut persisted = cfg.clone();
persisted.model = Some(resolved_model);
let _ = write_inference_config(&persisted);
}
{
let mut s = status.lock().await;
@@ -1097,11 +1297,6 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
// Something else (a different web server, a stale process,
// a 4xx-returning instance) is on our port. Don't kill it —
// give the user actionable info instead.
let lsof_hint = if cfg!(target_os = "windows") {
format!("netstat -ano | findstr :{}", JARVIS_PORT)
} else {
format!("lsof -i :{}", JARVIS_PORT)
};
let mut s = status.lock().await;
s.error = Some(format!(
"Port {} is already in use by another service (it answered \
@@ -1109,7 +1304,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
OpenJarvis port, then relaunch.\n\nTo identify it:\n {}",
JARVIS_PORT,
resp.status(),
lsof_hint,
port_owner_hint(),
));
return;
}
@@ -1119,8 +1314,21 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
}
}
if let Err(err) = check_jarvis_port_available() {
let mut s = status.lock().await;
s.error = Some(err);
return;
}
let root = project_root.as_ref().unwrap();
let cargo_bin = resolve_bin("cargo");
if !std::path::Path::new(&cargo_bin).exists() && cargo_bin == "cargo" {
let mut s = status.lock().await;
s.error = Some(format_missing_rust_toolchain());
return;
}
// Install dependencies automatically (handles fresh clones).
//
// Previously we ran `uv sync` with both stdout AND stderr piped to
@@ -1143,15 +1351,19 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
sync_cmd
.args([
"sync",
"--extra", "server",
"--extra", "desktop",
"--extra", "inference-cloud",
"--extra", "inference-google",
// openjarvis_rust lives in a uv dependency group (not the published
// `desktop` extra) so pip installs from PyPI don't require it (#584).
"--group", "desktop-native",
])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::piped())
.current_dir(root);
// Avoid LD_LIBRARY_PATH leak when running inside an AppImage (#455).
prepare_subprocess_for_appimage(&mut sync_cmd);
add_cargo_bin_to_path(&mut sync_cmd);
let sync_output = sync_cmd.output().await;
match sync_output {
Ok(out) if !out.status.success() => {
@@ -1168,6 +1380,16 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
Ok(_) => {} // success — fall through
}
{
let mut s = status.lock().await;
s.detail = "Verifying Rust extension (openjarvis_rust)...".into();
}
if let Err(err) = verify_openjarvis_rust_extension(root, &uv_bin).await {
let mut s = status.lock().await;
s.error = Some(err);
return;
}
{
let mut s = status.lock().await;
s.detail = format!("Starting API server from {}...", root.display());
@@ -1204,7 +1426,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
// additions aren't accidentally stripped.
prepare_subprocess_for_appimage(&mut cmd);
// Inject cloud API keys from ~/.openjarvis/cloud-keys.env
// Inject cloud API keys from secure desktop storage.
for (key, value) in read_cloud_keys() {
cmd.env(&key, &value);
}
@@ -1540,19 +1762,89 @@ async fn fetch_models(api_url: String) -> Result<serde_json::Value, String> {
#[tauri::command]
async fn run_jarvis_command(args: Vec<String>) -> Result<String, String> {
let mut cmd_args = vec!["run".to_string(), "jarvis".to_string()];
cmd_args.extend(args);
let uv_bin = resolve_bin("uv");
let output = tokio::process::Command::new(&uv_bin)
.args(&cmd_args)
.output()
.await
.map_err(|e| format!("Failed to launch jarvis: {}", e))?;
if output.status.success() {
Ok(String::from_utf8_lossy(&output.stdout).to_string())
} else {
Err(String::from_utf8_lossy(&output.stderr).to_string())
let mut cmd_args = vec!["run".to_string(), "jarvis".to_string()];
cmd_args.extend(args.iter().cloned());
let mut cmd = tokio::process::Command::new(&uv_bin);
cmd.args(&cmd_args);
// Run from the project root so `uv run jarvis` resolves the OpenJarvis
// project regardless of the app's launch cwd. In a packaged install the
// cwd isn't the checkout, so without this `jarvis` isn't found and the
// backend never starts — the UI then shows "Failed to get response"
// (see #531).
if let Some(ref root) = find_project_root() {
cmd.current_dir(root);
}
let is_serve = args.first().map(|a| a.as_str() == "serve").unwrap_or(false);
if !is_serve {
// Short-lived command (e.g. `stop`, `status`): wait for it and return
// its captured output.
let output = cmd
.output()
.await
.map_err(|e| format!("Failed to launch jarvis: {}", e))?;
return if output.status.success() {
Ok(String::from_utf8_lossy(&output.stdout).to_string())
} else {
Err(String::from_utf8_lossy(&output.stderr).to_string())
};
}
// `jarvis serve` is a long-running server that never exits. The old code
// used `.output()`, which waits for the process to exit and so hung this
// command forever — the "Start" button never resolved (#531). Spawn it
// detached instead, drain stderr (a full 4 KB Windows pipe can otherwise
// stall the child mid-startup, #309), and poll /health for readiness.
cmd.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::piped());
let mut child = cmd
.spawn()
.map_err(|e| format!("Failed to launch jarvis serve: {}", e))?;
let tail: StderrTail = Arc::new(Mutex::new(Vec::new()));
if let Some(stderr) = child.stderr.take() {
spawn_jarvis_stderr_drainer(stderr, tail.clone());
}
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(2))
.build()
.map_err(|e| format!("Failed to build HTTP client: {}", e))?;
let url = format!("http://127.0.0.1:{}/health", JARVIS_PORT);
let deadline = tokio::time::Instant::now() + Duration::from_secs(120);
loop {
// Surface an early crash (bad venv, missing Rust ext, etc.) right away
// instead of waiting out the full readiness timeout.
if let Ok(Some(status)) = child.try_wait() {
let stderr = String::from_utf8_lossy(tail.lock().await.as_slice()).into_owned();
return Err(format!(
"jarvis serve exited (code {:?}) before becoming healthy:\n{}",
status.code(),
stderr.trim()
));
}
if let Ok(resp) = client.get(&url).send().await {
if resp.status().is_success() {
// Leave the server running (the Child is detached on drop —
// kill_on_drop defaults to false); `stop` tears it down.
return Ok(format!(
"jarvis serve is ready on http://127.0.0.1:{}",
JARVIS_PORT
));
}
}
if tokio::time::Instant::now() >= deadline {
return Err(format!(
"jarvis serve did not become healthy on port {} within 120s.",
JARVIS_PORT
));
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
}
@@ -1594,11 +1886,29 @@ async fn transcribe_audio(
.send()
.await
.map_err(|e| format!("Connection failed: {}", e))?;
let body: serde_json::Value = resp
.json()
let status = resp.status();
let body = resp
.text()
.await
.map_err(|e| format!("Invalid response: {}", e))?;
Ok(body)
if !status.is_success() {
let detail = serde_json::from_str::<serde_json::Value>(&body)
.ok()
.and_then(|value| {
value
.get("detail")
.and_then(|detail| detail.as_str())
.map(str::to_string)
})
.filter(|detail| !detail.is_empty())
.unwrap_or(body);
return Err(format!(
"Transcription failed ({}): {}",
status.as_u16(),
detail
));
}
serde_json::from_str(&body).map_err(|e| format!("Invalid response: {}", e))
}
/// Submit savings to Supabase leaderboard.
@@ -1632,17 +1942,111 @@ async fn submit_savings(
// Cloud API key management
// ---------------------------------------------------------------------------
/// Path to the cloud keys file (~/.openjarvis/cloud-keys.env).
fn cloud_keys_path() -> std::path::PathBuf {
const SECURE_KEY_SERVICE: &str = "OpenJarvis Cloud Keys";
const MANAGED_CLOUD_KEY_NAMES: &[&str] = &[
"OPENAI_API_KEY",
"ANTHROPIC_API_KEY",
"GEMINI_API_KEY",
"GOOGLE_API_KEY",
"OPENROUTER_API_KEY",
"MINIMAX_API_KEY",
"TAVILY_API_KEY",
];
/// Legacy path used by older desktop builds. New saves never write here.
fn legacy_cloud_keys_path() -> std::path::PathBuf {
let home = home_dir();
std::path::PathBuf::from(home)
.join(".openjarvis")
.join("cloud-keys.env")
}
/// Read cloud keys from disk and return as key=value pairs.
fn read_cloud_keys() -> Vec<(String, String)> {
let path = cloud_keys_path();
fn validate_cloud_key_name(key_name: &str) -> Result<(), String> {
let valid = !key_name.is_empty()
&& key_name.len() <= 128
&& key_name.ends_with("_API_KEY")
&& key_name
.chars()
.all(|ch| ch.is_ascii_uppercase() || ch.is_ascii_digit() || ch == '_');
if valid {
Ok(())
} else {
Err(format!("Invalid API key name: {}", key_name))
}
}
fn engine_api_key_name(engine: &str) -> String {
let normalized: String = engine
.chars()
.map(|ch| {
if ch.is_ascii_alphanumeric() {
ch.to_ascii_uppercase()
} else {
'_'
}
})
.collect();
let trimmed = normalized.trim_matches('_');
let engine_name = if trimmed.is_empty() {
CUSTOM_FALLBACK_ENGINE.to_ascii_uppercase()
} else {
trimmed.to_string()
};
format!("{}_API_KEY", engine_name)
}
fn managed_cloud_key_names() -> Vec<String> {
let mut names: Vec<String> = MANAGED_CLOUD_KEY_NAMES
.iter()
.map(|name| (*name).to_string())
.collect();
let cfg = read_inference_config();
if matches!(&cfg.kind, SourceKind::Custom) {
let engine = cfg.engine.unwrap_or_else(|| CUSTOM_FALLBACK_ENGINE.to_string());
let key_name = engine_api_key_name(&engine);
if validate_cloud_key_name(&key_name).is_ok() {
names.push(key_name);
}
}
names.sort();
names.dedup();
names
}
fn secure_store_get(key_name: &str) -> Result<Option<String>, String> {
validate_cloud_key_name(key_name)?;
let entry = keyring::Entry::new(SECURE_KEY_SERVICE, key_name)
.map_err(|err| format!("Failed to open secure key storage for {}: {}", key_name, err))?;
match entry.get_password() {
Ok(value) => Ok(Some(value)),
Err(keyring::Error::NoEntry) => Ok(None),
Err(err) => Err(format!("Failed to read {} from secure key storage: {}", key_name, err)),
}
}
fn secure_store_set(key_name: &str, key_value: &str) -> Result<(), String> {
validate_cloud_key_name(key_name)?;
let entry = keyring::Entry::new(SECURE_KEY_SERVICE, key_name)
.map_err(|err| format!("Failed to open secure key storage for {}: {}", key_name, err))?;
if key_value.is_empty() {
return match entry.delete_credential() {
Ok(()) => Ok(()),
Err(keyring::Error::NoEntry) => Ok(()),
Err(err) => Err(format!(
"Failed to remove {} from secure key storage: {}",
key_name, err
)),
};
}
entry
.set_password(key_value)
.map_err(|err| format!("Failed to save {} in secure key storage: {}", key_name, err))
}
fn read_legacy_cloud_keys() -> Vec<(String, String)> {
let path = legacy_cloud_keys_path();
let mut keys = Vec::new();
if let Ok(contents) = std::fs::read_to_string(&path) {
for line in contents.lines() {
@@ -1658,47 +2062,68 @@ fn read_cloud_keys() -> Vec<(String, String)> {
keys
}
/// Save a single cloud API key to the keys file.
#[tauri::command]
async fn save_cloud_key(key_name: String, key_value: String) -> Result<(), String> {
let path = cloud_keys_path();
// Ensure directory exists
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
fn migrate_legacy_cloud_keys() {
let path = legacy_cloud_keys_path();
if !path.exists() {
return;
}
// Read existing keys, update/add the one being saved
let mut keys: Vec<(String, String)> = read_cloud_keys()
let legacy_keys = read_legacy_cloud_keys();
if legacy_keys.is_empty() {
let _ = std::fs::remove_file(&path);
return;
}
let mut migrated_all = true;
for (key, value) in legacy_keys {
if value.is_empty() {
continue;
}
if secure_store_set(&key, &value).is_err() {
migrated_all = false;
}
}
if migrated_all {
let _ = std::fs::remove_file(path);
}
}
/// Read cloud keys from secure desktop storage and return key=value pairs.
fn read_cloud_keys() -> Vec<(String, String)> {
migrate_legacy_cloud_keys();
managed_cloud_key_names()
.into_iter()
.filter(|(k, _)| k != &key_name)
.collect();
if !key_value.is_empty() {
keys.push((key_name, key_value));
}
.filter_map(|key| match secure_store_get(&key) {
Ok(Some(value)) if !value.is_empty() => Some((key, value)),
_ => None,
})
.collect()
}
// Write back
let content: String = keys
.iter()
.map(|(k, v)| format!("{}={}", k, v))
.collect::<Vec<_>>()
.join("\n");
std::fs::write(&path, content + "\n").map_err(|e| format!("Failed to save key: {}", e))?;
// Set permissions to owner-only (chmod 600)
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600));
}
// Tell the running server to hot-reload its cloud engine so the user
// doesn't need to restart the app after entering an API key.
async fn reload_cloud_keys(keys: Vec<(String, String)>) {
let reload_url = format!("http://127.0.0.1:{}/v1/cloud/reload", JARVIS_PORT);
let key_map: serde_json::Map<String, serde_json::Value> = keys
.into_iter()
.map(|(key, value)| (key, serde_json::Value::String(value)))
.collect();
let _ = reqwest::Client::new()
.post(&reload_url)
.json(&serde_json::json!({ "keys": key_map }))
.timeout(std::time::Duration::from_secs(10))
.send()
.await;
}
/// Save a single cloud API key to secure desktop storage.
#[tauri::command]
async fn save_cloud_key(key_name: String, key_value: String) -> Result<(), String> {
let key_value = key_value.trim().to_string();
secure_store_set(&key_name, &key_value)?;
// Tell the running server to hot-reload its cloud engine so the user
// doesn't need to restart the app after entering an API key.
reload_cloud_keys(vec![(key_name, key_value)]).await;
Ok(())
}
@@ -1706,10 +2131,13 @@ async fn save_cloud_key(key_name: String, key_value: String) -> Result<(), Strin
/// Get which cloud providers have keys configured (without exposing values).
#[tauri::command]
async fn get_cloud_key_status() -> Result<serde_json::Value, String> {
let keys = read_cloud_keys();
let status: Vec<serde_json::Value> = keys
.iter()
.map(|(k, v)| serde_json::json!({ "key": k, "set": !v.is_empty() }))
migrate_legacy_cloud_keys();
let status: Vec<serde_json::Value> = managed_cloud_key_names()
.into_iter()
.map(|key| {
let set = matches!(secure_store_get(&key), Ok(Some(value)) if !value.is_empty());
serde_json::json!({ "key": key, "set": set })
})
.collect();
Ok(serde_json::json!(status))
}
@@ -1721,8 +2149,8 @@ async fn get_inference_source() -> Result<InferenceConfig, String> {
}
/// Persist the chosen inference source. `host` is normalized to a bare base
/// URL. For custom endpoints, an optional API key is stored in cloud-keys.env
/// under `<ENGINE>_API_KEY`. Applies on next app launch.
/// URL. For custom endpoints, an optional API key is stored in secure desktop
/// storage under `<ENGINE>_API_KEY`. Applies on next app launch.
#[tauri::command]
async fn set_inference_source(
kind: String,
@@ -1754,7 +2182,7 @@ async fn set_inference_source(
.engine
.clone()
.unwrap_or_else(|| CUSTOM_FALLBACK_ENGINE.to_string());
let key_name = format!("{}_API_KEY", engine.to_ascii_uppercase());
let key_name = engine_api_key_name(&engine);
// Save the key before persisting the config: if the key can't be
// written, surface it and DON'T record a custom source whose
// credential is missing (which would fail confusingly at runtime).
@@ -2440,9 +2868,12 @@ pub fn run() {
#[cfg(test)]
mod tests {
use super::{
boot_plan, default_local_model, format_uv_sync_failure, format_uv_sync_spawn_error,
normalize_host, parse_inference_config, upsert_engine_host, uv_sync_stderr_tail,
InferenceConfig, SourceKind,
boot_plan, default_local_model, format_extension_import_failure,
format_missing_rust_toolchain, format_port_unavailable, format_uv_sync_failure,
format_uv_sync_spawn_error, matching_installed_model, model_names_match, normalize_host,
parse_inference_config, parse_ollama_model_names, preferred_installed_model,
should_persist_resolved_model, startup_installed_model, upsert_engine_host,
uv_sync_stderr_tail, InferenceConfig, SourceKind, DESKTOP_UV_SYNC_COMMAND,
};
use std::path::Path;
@@ -2486,7 +2917,7 @@ mod tests {
assert!(msg.contains("exit 2"));
assert!(msg.contains("/home/u/.openjarvis/src"));
assert!(msg.contains("failed to resolve numpy==2.1.3"));
assert!(msg.contains("uv sync --extra server")); // actionable next step
assert!(msg.contains(DESKTOP_UV_SYNC_COMMAND)); // actionable next step
}
#[test]
@@ -2509,6 +2940,49 @@ mod tests {
assert!(msg.contains("No such file or directory"));
}
#[test]
fn missing_rust_toolchain_message_names_cargo_and_installer() {
let msg = format_missing_rust_toolchain();
assert!(msg.contains("cargo"));
assert!(msg.contains("https://rustup.rs"));
assert!(msg.contains("openjarvis_rust"));
assert!(msg.contains("Visual Studio Build Tools"));
}
#[test]
fn uv_sync_rust_failure_mentions_toolchain() {
let msg = format_uv_sync_failure(
Path::new("C:\\Users\\me\\OpenJarvis"),
Some(1),
"maturin failed: linker `link.exe` not found while building openjarvis-rust",
);
assert!(msg.contains("exit 1"));
assert!(msg.contains("link.exe"));
assert!(msg.contains("https://rustup.rs"));
assert!(msg.contains("Visual Studio Build Tools"));
}
#[test]
fn extension_import_failure_names_verification_command() {
let msg = format_extension_import_failure(
Path::new("C:\\Users\\me\\OpenJarvis"),
"ModuleNotFoundError: No module named 'openjarvis_rust'",
);
assert!(msg.contains("openjarvis_rust"));
assert!(msg.contains(DESKTOP_UV_SYNC_COMMAND));
assert!(msg.contains("uv run python -c \"import openjarvis_rust\""));
assert!(msg.contains("ModuleNotFoundError"));
}
#[test]
fn port_unavailable_message_names_port_and_owner_hint() {
let msg = format_port_unavailable(8000, "address already in use");
assert!(msg.contains("Port 8000 is not available"));
assert!(msg.contains("address already in use"));
assert!(msg.contains("To identify it"));
assert!(msg.contains("8000"));
}
#[test]
fn default_local_model_picks_second_largest_that_fits() {
// QWEN35_MODELS min_ram ladder: 4,6,8,12,24,32,96 GB
@@ -2524,6 +2998,97 @@ mod tests {
assert_eq!(default_local_model(1.0), super::FALLBACK_MODEL);
}
#[test]
fn parse_ollama_model_names_reads_nonempty_names() {
let body = serde_json::json!({
"models": [
{"name": "llama3.2:latest"},
{"name": ""},
{"name": "qwen3.5:4b"},
{"model": "mistral:latest"}
]
});
assert_eq!(
parse_ollama_model_names(&body),
vec![
"llama3.2:latest".to_string(),
"qwen3.5:4b".to_string(),
"mistral:latest".to_string()
]
);
}
#[test]
fn model_names_match_treats_latest_as_optional() {
assert!(model_names_match("llama3.2:latest", "llama3.2"));
assert!(model_names_match("llama3.2", "llama3.2:latest"));
assert!(model_names_match("qwen3.5:4b", "qwen3.5:4b"));
assert!(!model_names_match("llama3.2:latest", "qwen3.5:4b"));
}
#[test]
fn installed_model_helpers_pick_matching_or_first_model() {
let models = vec!["llama3.2:latest".to_string(), "qwen3.5:4b".to_string()];
assert_eq!(
matching_installed_model(&models, "llama3.2"),
Some("llama3.2:latest".to_string())
);
assert_eq!(
preferred_installed_model(&models),
Some("llama3.2:latest".to_string())
);
}
#[test]
fn preferred_installed_model_skips_embedding_names_when_chat_model_exists() {
let models = vec![
"nomic-embed-text:latest".to_string(),
"llama3.2:latest".to_string(),
];
assert_eq!(
preferred_installed_model(&models),
Some("llama3.2:latest".to_string())
);
}
#[test]
fn startup_installed_model_uses_existing_model_for_defaults() {
let models = vec!["llama3.2:latest".to_string()];
assert_eq!(
startup_installed_model("qwen3.5:4b", &models),
Some("llama3.2:latest".to_string())
);
}
#[test]
fn startup_installed_model_uses_existing_model_when_configured_model_missing() {
let models = vec!["llama3.2:latest".to_string()];
assert_eq!(
startup_installed_model("qwen3.5:4b", &models),
Some("llama3.2:latest".to_string())
);
}
#[test]
fn resolved_model_is_only_persisted_when_no_model_was_configured() {
let default_cfg = InferenceConfig { kind: SourceKind::Ollama, ..Default::default() };
assert!(should_persist_resolved_model(&default_cfg));
let empty_cfg = InferenceConfig {
kind: SourceKind::Ollama,
model: Some(" ".into()),
..Default::default()
};
assert!(should_persist_resolved_model(&empty_cfg));
let user_cfg = InferenceConfig {
kind: SourceKind::Ollama,
model: Some("qwen3.5:9b".into()),
..Default::default()
};
assert!(!should_persist_resolved_model(&user_cfg));
}
#[test]
fn parse_defaults_to_ollama_when_file_missing_or_garbage() {
assert!(matches!(parse_inference_config("").kind, SourceKind::Ollama));
+1 -3
View File
@@ -31,7 +31,6 @@ export default function App() {
const prevModelRef = useRef<string>('');
const setModels = useAppStore((s) => s.setModels);
const setModelsLoading = useAppStore((s) => s.setModelsLoading);
const setSelectedModel = useAppStore((s) => s.setSelectedModel);
const selectedModel = useAppStore((s) => s.selectedModel);
const setServerInfo = useAppStore((s) => s.setServerInfo);
const setSavings = useAppStore((s) => s.setSavings);
@@ -70,7 +69,6 @@ export default function App() {
fetchModels()
.then((m) => {
setModels(m);
if (!selectedModel && m.length > 0) setSelectedModel(m[0].id);
})
.catch(() => setModels([]))
.finally(() => setModelsLoading(false));
@@ -89,7 +87,7 @@ export default function App() {
setSavings(data);
if (optInEnabled && optInDisplayName && data) {
const claudeEntry = data.per_provider.find(
(p) => p.provider === 'claude-opus-4.6',
(p) => p.provider === 'claude-fable-5',
);
const dollarSavings = claudeEntry ? claudeEntry.total_cost : 0;
const energySaved = data.per_provider.reduce(
+29 -5
View File
@@ -15,6 +15,7 @@ function getGreeting(): string {
}
export function ChatArea() {
const activeId = useAppStore((s) => s.activeId);
const messages = useAppStore((s) => s.messages);
const streamState = useAppStore((s) => s.streamState);
const systemPanelOpen = useAppStore((s) => s.systemPanelOpen);
@@ -22,6 +23,10 @@ export function ChatArea() {
const navigate = useNavigate();
const listRef = useRef<HTMLDivElement>(null);
const shouldAutoScroll = useRef(true);
const wasStreaming = useRef(false);
const lastScrollTop = useRef(0);
const isCurrentChatStreaming = streamState.isStreaming && streamState.conversationId === activeId;
const currentStreamContent = isCurrentChatStreaming ? streamState.content : '';
// Check if any data sources are connected
const [hasConnectedSources, setHasConnectedSources] = useState<boolean | null>(null);
@@ -34,18 +39,37 @@ export function ChatArea() {
}, []);
useEffect(() => {
// Sending a message always pins the view to the bottom, even if the
// user had scrolled up to read earlier messages.
if (isCurrentChatStreaming && !wasStreaming.current) {
shouldAutoScroll.current = true;
}
wasStreaming.current = isCurrentChatStreaming;
if (shouldAutoScroll.current && listRef.current) {
listRef.current.scrollTop = listRef.current.scrollHeight;
}
}, [messages, streamState.content]);
}, [messages, currentStreamContent, isCurrentChatStreaming]);
const handleScroll = () => {
if (!listRef.current) return;
const { scrollTop, scrollHeight, clientHeight } = listRef.current;
shouldAutoScroll.current = scrollHeight - scrollTop - clientHeight < 100;
const distance = scrollHeight - scrollTop - clientHeight;
const scrolledUp = scrollTop < lastScrollTop.current;
lastScrollTop.current = scrollTop;
if (scrolledUp && distance >= 1) {
// Any upward scroll away from the bottom stops autoscroll immediately,
// so streaming content never fights the user (no jitter). Sub-1px
// upward movement (elastic bounce settling at the bottom) is ignored.
shouldAutoScroll.current = false;
} else if (!scrolledUp) {
// Re-engage when scrolled back to the bottom. < 2 rather than < 1:
// at fractional zoom levels the at-bottom residual can reach 1px,
// which would otherwise leave autoscroll permanently disengaged.
shouldAutoScroll.current = distance < 2;
}
};
const isEmpty = messages.length === 0 && !streamState.isStreaming;
const isEmpty = messages.length === 0 && !isCurrentChatStreaming;
const PanelIcon = systemPanelOpen ? PanelRightClose : PanelRightOpen;
@@ -153,12 +177,12 @@ export function ChatArea() {
<MessageBubble
key={msg.id}
message={msg}
isLive={isLastAssistant && streamState.isStreaming}
isLive={isLastAssistant && isCurrentChatStreaming}
/>
);
})}
{(() => {
if (!streamState.isStreaming || streamState.content !== '') return null;
if (!isCurrentChatStreaming || streamState.content !== '') return null;
// For research messages the ResearchTimeline handles its own
// pre-content loading state — suppress the generic dots.
const last = messages[messages.length - 1];
+26 -5
View File
@@ -96,8 +96,15 @@ export function InputArea() {
const deepResearch = useAppStore((s) => s.deepResearch);
const setDeepResearch = useAppStore((s) => s.setDeepResearch);
const corpusSync = useResearchCorpusSync(deepResearch);
const isCurrentChatStreaming = streamState.isStreaming && streamState.conversationId === activeId;
const { state: speechState, available: speechAvailable, startRecording, stopRecording } = useSpeech();
const {
state: speechState,
error: speechError,
available: speechAvailable,
startRecording,
stopRecording,
} = useSpeech();
// Abort in-flight stream when the user switches models mid-generation.
// This prevents errors from trying to continue a stream with a stale model.
@@ -122,6 +129,12 @@ export function InputArea() {
: streamState.isStreaming ? 'streaming'
: undefined;
useEffect(() => {
if (speechError) {
toast.error(speechError, { duration: 8000 });
}
}, [speechError]);
const handleMicClick = useCallback(async () => {
if (speechState === 'recording') {
try {
@@ -214,6 +227,7 @@ export function InputArea() {
let ttftMs: number | undefined;
setStreamState({
conversationId: convId,
isStreaming: true,
phase: deepResearch ? 'Researching...' : 'Generating...',
elapsedMs: 0,
@@ -231,7 +245,11 @@ export function InputArea() {
try {
if (deepResearch) {
for await (const ev of streamResearch(content, controller.signal)) {
for await (const ev of streamResearch(
content,
selectedModel,
controller.signal,
)) {
if (ev.type === 'search_call') {
const trace: ResearchSearchTrace = {
id: generateId(),
@@ -450,7 +468,10 @@ export function InputArea() {
}
const totalMs = Date.now() - startTime;
const _CLOUD_PREFIXES = ['gpt-', 'o1-', 'o3-', 'o4-', 'claude-', 'gemini-', 'openrouter/', 'MiniMax-', 'chatgpt-'];
const engineLabel = _CLOUD_PREFIXES.some(p => selectedModel.startsWith(p)) ? 'cloud' : 'ollama';
const selectedOwner = useAppStore.getState().models.find((m) => m.id === selectedModel)?.owned_by;
const engineLabel = selectedOwner === 'litellm'
? 'litellm'
: _CLOUD_PREFIXES.some(p => selectedModel.startsWith(p)) ? 'cloud' : 'ollama';
const telemetry: MessageTelemetry = {
engine: engineLabel,
model_id: selectedModel,
@@ -583,7 +604,7 @@ export function InputArea() {
style={{ color: 'var(--color-text)', maxHeight: '200px' }}
disabled={streamState.isStreaming || modelLoading}
/>
{streamState.isStreaming ? (
{isCurrentChatStreaming ? (
<button
onClick={stopStreaming}
className="p-2 rounded-xl transition-colors shrink-0 cursor-pointer"
@@ -602,7 +623,7 @@ export function InputArea() {
/>
<button
onClick={sendMessage}
disabled={!input.trim() || modelLoading || !selectedModel}
disabled={streamState.isStreaming || !input.trim() || modelLoading || !selectedModel}
title={selectedModel ? 'Send message' : 'Pick a model first (⌘K)'}
className="p-2 rounded-xl transition-colors shrink-0 cursor-pointer disabled:opacity-30 disabled:cursor-default"
style={{
+2 -2
View File
@@ -29,8 +29,8 @@ interface TelemetryStats {
}
const CLOUD_PRICING = [
{ name: 'GPT-5.3', input: 2.00, output: 10.00, primary: true },
{ name: 'Claude Opus 4.6', input: 5.00, output: 25.00, primary: false },
{ name: 'GPT-5.6 Sol', input: 5.00, output: 30.00, primary: true },
{ name: 'Claude Fable 5', input: 10.00, output: 50.00, primary: false },
{ name: 'Gemini 3.1 Pro', input: 2.00, output: 12.00, primary: false },
];
+98 -66
View File
@@ -1,7 +1,15 @@
import { useState, useRef, useEffect } from 'react';
import { useState, useRef, useEffect, useCallback } from 'react';
import { Search, Cpu, X, Download, Loader2, Trash2, Check, Cloud, Key, Eye, EyeOff } from 'lucide-react';
import { useAppStore } from '../lib/store';
import { pullModel, deleteModel, fetchModels, preloadModel, isTauri } from '../lib/api';
import {
pullModel,
deleteModel,
fetchModels,
preloadModel,
isTauri,
getCloudKeyStatus,
saveCloudKey,
} from '../lib/api';
/** Popular models that users can download from the catalogue. */
const CATALOGUE_MODELS = [
@@ -23,7 +31,6 @@ const CATALOGUE_MODELS = [
interface CloudProvider {
name: string;
envKey: string;
storageKey: string;
models: Array<{ id: string; desc: string }>;
}
@@ -31,7 +38,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'OpenAI',
envKey: 'OPENAI_API_KEY',
storageKey: 'openjarvis-openai-key',
models: [
{ id: 'gpt-4o', desc: 'GPT-4o — fast, multimodal' },
{ id: 'gpt-4o-mini', desc: 'GPT-4o Mini — cheap, fast' },
@@ -41,7 +47,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'Anthropic',
envKey: 'ANTHROPIC_API_KEY',
storageKey: 'openjarvis-anthropic-key',
models: [
{ id: 'claude-sonnet-4-6', desc: 'Claude Sonnet 4.6 — balanced' },
{ id: 'claude-opus-4-6', desc: 'Claude Opus 4.6 — most capable' },
@@ -51,7 +56,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'Google',
envKey: 'GEMINI_API_KEY',
storageKey: 'openjarvis-gemini-key',
models: [
{ id: 'gemini-2.5-pro', desc: 'Gemini 2.5 Pro — flagship' },
{ id: 'gemini-2.5-flash', desc: 'Gemini 2.5 Flash — fast' },
@@ -61,7 +65,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'OpenRouter',
envKey: 'OPENROUTER_API_KEY',
storageKey: 'openjarvis-openrouter-key',
models: [
{ id: 'openrouter/auto', desc: 'Auto — best model for the task' },
{ id: 'openrouter/anthropic/claude-sonnet-4', desc: 'Claude Sonnet 4 via OpenRouter' },
@@ -70,16 +73,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
},
];
function getStoredKey(storageKey: string): string {
try { return localStorage.getItem(storageKey) || ''; } catch { return ''; }
}
function setStoredKey(storageKey: string, value: string): void {
try {
if (value) localStorage.setItem(storageKey, value);
else localStorage.removeItem(storageKey);
} catch {}
}
type Tab = 'installed' | 'catalogue' | 'cloud';
export function CommandPalette() {
@@ -92,11 +85,10 @@ export function CommandPalette() {
const [deleting, setDeleting] = useState<string | null>(null);
const [customModel, setCustomModel] = useState('');
const [showKeys, setShowKeys] = useState<Record<string, boolean>>({});
const [apiKeys, setApiKeys] = useState<Record<string, string>>(() => {
const keys: Record<string, string> = {};
for (const p of CLOUD_PROVIDERS) keys[p.storageKey] = getStoredKey(p.storageKey);
return keys;
});
const [apiKeys, setApiKeys] = useState<Record<string, string>>({});
const [cloudKeyStatus, setCloudKeyStatus] = useState<Record<string, boolean>>({});
const [cloudKeyError, setCloudKeyError] = useState<string | null>(null);
const [savingKey, setSavingKey] = useState<string | null>(null);
const inputRef = useRef<HTMLInputElement>(null);
const models = useAppStore((s) => s.models);
@@ -106,6 +98,20 @@ export function CommandPalette() {
const setCommandPaletteOpen = useAppStore((s) => s.setCommandPaletteOpen);
const installedIds = new Set(models.map((m) => m.id));
const desktopKeyStorage = isTauri();
const refreshCloudKeyStatus = useCallback(async () => {
if (!desktopKeyStorage) {
setCloudKeyStatus({});
return;
}
try {
setCloudKeyStatus(await getCloudKeyStatus());
setCloudKeyError(null);
} catch (e: any) {
setCloudKeyError(e?.message || 'Failed to read cloud key status');
}
}, [desktopKeyStorage]);
const filtered = tab === 'installed'
? (query
@@ -122,6 +128,10 @@ export function CommandPalette() {
inputRef.current?.focus();
}, []);
useEffect(() => {
void refreshCloudKeyStatus();
}, [refreshCloudKeyStatus]);
useEffect(() => {
setSelectedIdx(0);
}, [query, tab]);
@@ -133,18 +143,17 @@ export function CommandPalette() {
}
}, [pullSuccess]);
const handleSelect = async (modelId: string) => {
const handleSelect = async (modelId: string, owner?: string) => {
const previousModel = selectedModel;
setSelectedModel(modelId);
setCommandPaletteOpen(false);
if (modelId !== previousModel) {
const { createConversation, setModelLoading, addLogEntry } = useAppStore.getState();
createConversation(modelId);
const { setModelLoading, addLogEntry } = useAppStore.getState();
setModelLoading(true);
addLogEntry({ timestamp: Date.now(), level: 'info', category: 'model', message: `Switching to ${modelId}...` });
try {
await preloadModel(modelId);
await preloadModel(modelId, owner);
addLogEntry({ timestamp: Date.now(), level: 'info', category: 'model', message: `${modelId} loaded` });
} catch (e: any) {
addLogEntry({ timestamp: Date.now(), level: 'error', category: 'model', message: `Failed to load ${modelId}: ${e.message}` });
@@ -210,24 +219,29 @@ export function CommandPalette() {
};
const handleSaveKey = async (provider: CloudProvider, value: string) => {
setStoredKey(provider.storageKey, value);
setApiKeys((prev) => ({ ...prev, [provider.storageKey]: value }));
const keyValue = value.trim();
setSavingKey(provider.envKey);
setCloudKeyError(null);
// Also save to Tauri backend so the server process picks up the key
if (isTauri()) {
try {
const { invoke } = await import('@tauri-apps/api/core');
await invoke('save_cloud_key', { keyName: provider.envKey, keyValue: value });
} catch {}
try {
await saveCloudKey(provider.envKey, keyValue);
setApiKeys((prev) => ({ ...prev, [provider.envKey]: '' }));
await refreshCloudKeyStatus();
useAppStore.getState().addLogEntry({
timestamp: Date.now(), level: 'info', category: 'model',
message: `${provider.name} API key ${keyValue ? 'saved' : 'removed'}. Refreshing model list...`,
});
await refreshModels();
} catch (e: any) {
setCloudKeyError(e?.message || `Failed to save ${provider.name} API key`);
} finally {
setSavingKey(null);
}
};
useAppStore.getState().addLogEntry({
timestamp: Date.now(), level: 'info', category: 'model',
message: `${provider.name} API key ${value ? 'saved' : 'removed'}. Refreshing model list…`,
});
// Refresh the model list so cloud models appear immediately.
await refreshModels();
const handleKeyBlur = (provider: CloudProvider) => {
const draft = apiKeys[provider.envKey] || '';
if (draft.trim()) void handleSaveKey(provider, draft);
};
const handleKeyDown = (e: React.KeyboardEvent) => {
@@ -241,7 +255,8 @@ export function CommandPalette() {
setSelectedIdx((i) => Math.max(i - 1, 0));
} else if (e.key === 'Enter' && tab === 'installed' && filtered.length > 0) {
e.preventDefault();
handleSelect((filtered[selectedIdx] as any).id);
const model = filtered[selectedIdx] as (typeof models)[number];
handleSelect(model.id, model.owned_by);
}
};
@@ -323,6 +338,11 @@ export function CommandPalette() {
<Check size={12} /> Downloaded {pullSuccess} successfully
</div>
)}
{tab === 'cloud' && cloudKeyError && (
<div className="px-4 py-2 text-xs" style={{ color: 'var(--color-error)', background: 'rgba(220,38,38,0.05)' }}>
{cloudKeyError}
</div>
)}
{/* Results */}
<div className="max-h-[400px] overflow-y-auto py-2">
@@ -346,11 +366,15 @@ export function CommandPalette() {
onMouseEnter={() => setSelectedIdx(idx)}
>
<button
onClick={() => handleSelect(model.id)}
onClick={() => handleSelect(model.id, model.owned_by)}
className="flex items-center gap-3 flex-1 min-w-0 text-left cursor-pointer"
style={{ background: 'none', border: 'none', padding: 0 }}
>
<Cpu size={16} style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text-tertiary)' }} />
{model.owned_by === 'litellm' ? (
<Cloud size={16} style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text-tertiary)' }} />
) : (
<Cpu size={16} style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text-tertiary)' }} />
)}
<div className="flex-1 min-w-0">
<div className="text-sm truncate" style={{ color: isActive ? 'var(--color-accent)' : 'var(--color-text)', fontWeight: isActive ? 500 : 400 }}>
{model.id}
@@ -362,17 +386,19 @@ export function CommandPalette() {
</span>
)}
</button>
<button
onClick={() => handleDelete(model.id)}
disabled={isDeleting}
className="p-1 rounded transition-colors cursor-pointer"
style={{ color: 'var(--color-text-tertiary)', opacity: 0 }}
title="Delete model"
onMouseEnter={(e) => { e.currentTarget.style.opacity = '1'; e.currentTarget.style.color = 'var(--color-error)'; }}
onMouseLeave={(e) => { e.currentTarget.style.opacity = '0'; e.currentTarget.style.color = 'var(--color-text-tertiary)'; }}
>
{isDeleting ? <Loader2 size={14} className="animate-spin" /> : <Trash2 size={14} />}
</button>
{model.owned_by !== 'litellm' && (
<button
onClick={() => handleDelete(model.id)}
disabled={isDeleting}
className="p-1 rounded transition-colors cursor-pointer"
style={{ color: 'var(--color-text-tertiary)', opacity: 0 }}
title="Delete model"
onMouseEnter={(e) => { e.currentTarget.style.opacity = '1'; e.currentTarget.style.color = 'var(--color-error)'; }}
onMouseLeave={(e) => { e.currentTarget.style.opacity = '0'; e.currentTarget.style.color = 'var(--color-text-tertiary)'; }}
>
{isDeleting ? <Loader2 size={14} className="animate-spin" /> : <Trash2 size={14} />}
</button>
)}
</div>
);
})
@@ -431,13 +457,17 @@ export function CommandPalette() {
/* ── Cloud Models tab ── */
<div className="px-4 py-2">
<div className="text-[11px] mb-3" style={{ color: 'var(--color-text-tertiary)' }}>
Add your API keys to use cloud models. Keys are stored locally on your device only.
{desktopKeyStorage
? 'Add your API keys to use cloud models. Keys are stored in secure desktop storage.'
: 'Configure cloud provider keys in the server environment to use cloud models.'}
</div>
{CLOUD_PROVIDERS.map((provider) => {
const key = apiKeys[provider.storageKey] || '';
const hasKey = !!key;
const isVisible = showKeys[provider.storageKey];
const key = apiKeys[provider.envKey] || '';
const hasSavedKey = !!cloudKeyStatus[provider.envKey];
const hasKey = hasSavedKey || !!key.trim();
const isVisible = showKeys[provider.envKey];
const isSaving = savingKey === provider.envKey;
return (
<div key={provider.name} className="mb-4">
@@ -458,26 +488,28 @@ export function CommandPalette() {
<input
type={isVisible ? 'text' : 'password'}
value={key}
onChange={(e) => setApiKeys((prev) => ({ ...prev, [provider.storageKey]: e.target.value }))}
onBlur={() => handleSaveKey(provider, apiKeys[provider.storageKey] || '')}
placeholder={`${provider.envKey}`}
onChange={(e) => setApiKeys((prev) => ({ ...prev, [provider.envKey]: e.target.value }))}
onBlur={() => handleKeyBlur(provider)}
placeholder={hasSavedKey ? 'Saved in secure storage' : provider.envKey}
disabled={!desktopKeyStorage || isSaving}
className="flex-1 text-xs px-2 py-1.5 bg-transparent outline-none font-mono"
style={{ color: 'var(--color-text)' }}
/>
<button
onClick={() => setShowKeys((prev) => ({ ...prev, [provider.storageKey]: !prev[provider.storageKey] }))}
onClick={() => setShowKeys((prev) => ({ ...prev, [provider.envKey]: !prev[provider.envKey] }))}
className="px-2 cursor-pointer" style={{ color: 'var(--color-text-tertiary)' }}
>
{isVisible ? <EyeOff size={12} /> : <Eye size={12} />}
</button>
</div>
{hasKey && (
{hasSavedKey && (
<button
onClick={() => handleSaveKey(provider, '')}
disabled={isSaving}
className="px-2 py-1 rounded-lg text-[10px] cursor-pointer"
style={{ color: 'var(--color-error)', border: '1px solid var(--color-error)' }}
style={{ color: 'var(--color-error)', border: '1px solid var(--color-error)', opacity: isSaving ? 0.5 : 1 }}
>
Remove
{isSaving ? 'Saving' : 'Remove'}
</button>
)}
</div>
@@ -2,8 +2,8 @@ import { DollarSign, TrendingDown, Cloud, HardDrive } from 'lucide-react';
import { useAppStore } from '../../lib/store';
const CLOUD_PRICING = [
{ name: 'GPT-5.3', input: 2.00, output: 10.00 },
{ name: 'Claude Opus 4.6', input: 5.00, output: 25.00 },
{ name: 'GPT-5.6 Sol', input: 5.00, output: 30.00 },
{ name: 'Claude Fable 5', input: 10.00, output: 50.00 },
{ name: 'Gemini 3.1 Pro', input: 2.00, output: 12.00 },
];
@@ -1,6 +1,7 @@
import { useState, useEffect, useCallback } from 'react';
import type React from 'react';
import { invoke } from '@tauri-apps/api/core';
import { LEADERBOARD_ENABLED, SUPABASE_ANON_KEY, SUPABASE_URL } from '../../lib/supabase';
// ---------------------------------------------------------------------------
// Types
@@ -221,8 +222,8 @@ const styles: Record<string, React.CSSProperties> = {
};
const PROVIDER_COLORS: Record<string, string> = {
'gpt-5.3': colors.green,
'claude-opus-4.6': colors.yellow,
'gpt-5.6-sol': colors.green,
'claude-fable-5': colors.yellow,
'gemini-3.1-pro': colors.accent,
};
@@ -279,9 +280,6 @@ function getOrCreateAnonId(): string {
return id;
}
const SUPABASE_URL = 'https://mtbtgpwzrbostweaanpr.supabase.co';
const SUPABASE_KEY = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Im10YnRncHd6cmJvc3R3ZWFhbnByIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NzMxODk0OTQsImV4cCI6MjA4ODc2NTQ5NH0._xMlqCfljtXpwPj54H-ghxfLFO-jiq4W2WhpU8vVL1c';
const REFRESH_INTERVAL_MS = 5000;
export function SavingsDashboard({ apiUrl }: { apiUrl: string }) {
@@ -318,15 +316,16 @@ export function SavingsDashboard({ apiUrl }: { apiUrl: string }) {
return () => clearInterval(timer);
}, [fetchData]);
// Share savings to Supabase when opted in and data changes
// Share savings to Supabase when opted in and data changes. Skipped entirely
// when no anon key was built in (leaderboard disabled).
useEffect(() => {
if (!optInEnabled || !displayName || !data) return;
if (!LEADERBOARD_ENABLED || !optInEnabled || !displayName || !data) return;
const dollarSavings = data.per_provider.reduce((s, p) => s + p.total_cost, 0);
const energySaved = data.per_provider.reduce((s, p) => s + (p.energy_wh || 0), 0);
const flopsSaved = data.per_provider.reduce((s, p) => s + (p.flops || 0), 0);
invoke('submit_savings', {
supabaseUrl: SUPABASE_URL,
supabaseKey: SUPABASE_KEY,
supabaseKey: SUPABASE_ANON_KEY,
payload: {
anon_id: anonId,
display_name: displayName,
+6 -3
View File
@@ -7,6 +7,7 @@ import {
type SetupStatus,
} from '../lib/api';
import { useAppStore } from '../lib/store';
import { isEmbedOnlyModel } from '../lib/model-capabilities';
const STEPS = [
{ key: 'ollama_ready', label: 'Inference Engine', icon: Cpu, detail: 'Starting Ollama...' },
@@ -91,12 +92,14 @@ export function SetupScreen({ onReady }: { onReady: () => void }) {
fetchRecommendedModel().catch(() => ({ model: '', reason: '' })),
]);
const store = useAppStore.getState();
const hadSelection = !!store.selectedModel;
store.setModels(models);
store.setModelsLoading(false);
const recommended = rec.model && models.some((m) => m.id === rec.model)
const chatModels = models.filter((m) => !isEmbedOnlyModel(m.id));
const recommended = rec.model && chatModels.some((m) => m.id === rec.model)
? rec.model
: models[0]?.id || '';
if (recommended && !store.selectedModel) {
: chatModels[0]?.id || '';
if (recommended && !hadSelection) {
store.setSelectedModel(recommended);
}
} catch {
@@ -22,6 +22,9 @@ export function ConversationList({ searchQuery }: Props) {
const navigate = useNavigate();
const conversations = useAppStore((s) => s.conversations);
const activeId = useAppStore((s) => s.activeId);
const streamingConversationId = useAppStore((s) =>
s.streamState.isStreaming ? s.streamState.conversationId : null,
);
const selectConversation = useAppStore((s) => s.selectConversation);
const deleteConversation = useAppStore((s) => s.deleteConversation);
@@ -43,6 +46,7 @@ export function ConversationList({ searchQuery }: Props) {
<div className="flex flex-col gap-0.5 py-1">
{filtered.map((conv) => {
const isActive = conv.id === activeId;
const isStreaming = conv.id === streamingConversationId;
return (
<div
key={conv.id}
@@ -82,11 +86,18 @@ export function ConversationList({ searchQuery }: Props) {
e.stopPropagation();
deleteConversation(conv.id);
}}
className="p-1.5 mr-1 rounded opacity-0 group-hover:opacity-100 transition-opacity cursor-pointer"
disabled={isStreaming}
className="p-1.5 mr-1 rounded opacity-0 group-hover:opacity-100 transition-opacity cursor-pointer disabled:cursor-not-allowed disabled:opacity-30"
style={{ color: 'var(--color-text-tertiary)' }}
onMouseEnter={(e) => (e.currentTarget.style.color = 'var(--color-error)')}
onMouseEnter={(e) => {
if (!isStreaming) e.currentTarget.style.color = 'var(--color-error)';
}}
onMouseLeave={(e) => (e.currentTarget.style.color = 'var(--color-text-tertiary)')}
title="Delete conversation"
title={
isStreaming
? 'Stop generating before deleting this conversation'
: 'Delete conversation'
}
>
<Trash2 size={14} />
</button>
+54 -1
View File
@@ -1,4 +1,4 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
// Regression for #266: the frontend must send the local API key as a Bearer
// token on /v1 + /api requests, or `jarvis serve` with a key configured 401s
@@ -6,6 +6,7 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest';
// authHeaders) that source the key and build the header.
const SETTINGS_KEY = 'openjarvis-settings';
const fetchMock = vi.fn<typeof fetch>();
// Minimal in-memory localStorage stub so the helpers can run under node
// (no jsdom dependency).
@@ -26,11 +27,16 @@ class MemoryStorage {
}
beforeEach(() => {
vi.resetModules();
vi.stubEnv('VITE_SUPABASE_ANON_KEY', 'test-anon-key');
fetchMock.mockReset();
globalThis.fetch = fetchMock;
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
vi.unstubAllEnvs();
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
@@ -83,3 +89,50 @@ describe('authHeaders', () => {
});
});
});
describe('tool credentials', () => {
it('reads credential status from the local server', async () => {
fetchMock.mockResolvedValue(
new Response(JSON.stringify({ TAVILY_API_KEY: true }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
}),
);
const { fetchToolCredentialStatus } = await freshApi();
await expect(fetchToolCredentialStatus('web_search')).resolves.toEqual({
TAVILY_API_KEY: true,
});
expect(fetchMock).toHaveBeenCalledWith(
'/v1/tools/web_search/credentials/status',
{ headers: {} },
);
});
it('saves a tool credential through the local server', async () => {
fetchMock.mockResolvedValue(new Response('{}', { status: 200 }));
const { saveToolCredentials } = await freshApi();
await saveToolCredentials('web_search', {
TAVILY_API_KEY: 'tvly-test',
});
expect(fetchMock).toHaveBeenCalledWith('/v1/tools/web_search/credentials', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ TAVILY_API_KEY: 'tvly-test' }),
});
});
it('deletes a tool credential through the local server', async () => {
fetchMock.mockResolvedValue(new Response('{}', { status: 200 }));
const { deleteToolCredential } = await freshApi();
await deleteToolCredential('web_search', 'TAVILY_API_KEY');
expect(fetchMock).toHaveBeenCalledWith(
'/v1/tools/web_search/credentials/TAVILY_API_KEY',
{ method: 'DELETE', headers: {} },
);
});
});
+83 -12
View File
@@ -1,12 +1,10 @@
import type { ModelInfo, SavingsData, ServerInfo } from '../types';
import { SUPABASE_ANON_KEY, SUPABASE_URL } from './supabase';
// ---------------------------------------------------------------------------
// Supabase config — safe to embed (RLS protects writes)
// Supabase config
// ---------------------------------------------------------------------------
const SUPABASE_URL = import.meta.env.VITE_SUPABASE_URL || 'https://mtbtgpwzrbostweaanpr.supabase.co';
const SUPABASE_ANON_KEY = import.meta.env.VITE_SUPABASE_ANON_KEY || 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Im10YnRncHd6cmJvc3R3ZWFhbnByIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NzMxODk0OTQsImV4cCI6MjA4ODc2NTQ5NH0._xMlqCfljtXpwPj54H-ghxfLFO-jiq4W2WhpU8vVL1c';
declare global {
interface Window {
__TAURI_INTERNALS__?: unknown;
@@ -15,6 +13,31 @@ declare global {
export const isTauri = () => typeof window !== 'undefined' && !!window.__TAURI_INTERNALS__;
export type CloudKeyStatus = Record<string, boolean>;
export async function getCloudKeyStatus(): Promise<CloudKeyStatus> {
if (!isTauri()) return {};
try {
const { invoke } = await import('@tauri-apps/api/core');
const rows = await invoke<Array<{ key: string; set: boolean }>>('get_cloud_key_status');
return Object.fromEntries(rows.map((row) => [row.key, row.set]));
} catch (e: any) {
throw new Error(e?.message ?? e ?? 'Failed to read cloud key status');
}
}
export async function saveCloudKey(keyName: string, keyValue: string): Promise<void> {
if (!isTauri()) {
throw new Error('Cloud API keys can be saved in the desktop app only.');
}
try {
const { invoke } = await import('@tauri-apps/api/core');
await invoke('save_cloud_key', { keyName, keyValue });
} catch (e: any) {
throw new Error(e?.message ?? e ?? 'Failed to save cloud key');
}
}
// Cached API base URL fetched from the Tauri backend at startup.
// This avoids hardcoding the port — the Rust backend is the single
// source of truth for JARVIS_PORT.
@@ -195,9 +218,9 @@ export async function deleteModel(modelName: string): Promise<void> {
const _CLOUD_PREFIXES = ['gpt-', 'o1-', 'o3-', 'o4-', 'claude-', 'gemini-', 'openrouter/'];
export async function preloadModel(modelName: string): Promise<void> {
export async function preloadModel(modelName: string, owner?: string): Promise<void> {
// Cloud models don't need Ollama preloading
if (_CLOUD_PREFIXES.some(p => modelName.startsWith(p))) {
if (owner === 'litellm' || _CLOUD_PREFIXES.some(p => modelName.startsWith(p))) {
return;
}
// Trigger Ollama to load the model into memory (empty prompt, no generation).
@@ -317,8 +340,9 @@ export async function transcribeAudio(audioBlob: Blob, filename = 'recording.web
audioData: Array.from(new Uint8Array(buffer)),
filename,
});
} catch {
// Fall through to fetch
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
throw new Error(msg || 'Transcription failed');
}
}
const formData = new FormData();
@@ -327,7 +351,16 @@ export async function transcribeAudio(audioBlob: Blob, filename = 'recording.web
method: 'POST',
body: formData,
});
if (!res.ok) throw new Error(`Transcription failed: ${res.status}`);
if (!res.ok) {
let detail = "";
try {
const body = await res.json();
detail = typeof body.detail === 'string' ? body.detail : "";
} catch {
// Keep the status-only message below when the body is not JSON.
}
throw new Error(detail || `Transcription failed: ${res.status}`);
}
return res.json();
}
@@ -852,6 +885,25 @@ export async function saveToolCredentials(
if (!res.ok) throw new Error(`Failed: ${res.status}`);
}
export async function fetchToolCredentialStatus(
toolName: string,
): Promise<Record<string, boolean>> {
const res = await apiFetch(`/v1/tools/${toolName}/credentials/status`);
if (!res.ok) throw new Error(`Failed: ${res.status}`);
return await res.json();
}
export async function deleteToolCredential(
toolName: string,
keyName: string,
): Promise<void> {
const res = await apiFetch(
`/v1/tools/${encodeURIComponent(toolName)}/credentials/${encodeURIComponent(keyName)}`,
{ method: 'DELETE' },
);
if (!res.ok) throw new Error(`Failed: ${res.status}`);
}
export interface AgentTraceDetail {
id: string;
agent: string;
@@ -948,12 +1000,31 @@ export interface MemoryStats {
export interface MemoryConfig {
backend: string;
// Set by the server when the native `openjarvis_rust` extension is missing,
// so the UI can show the real cause instead of a healthy-looking config.
available?: boolean;
detail?: string | null;
context_from_memory: boolean;
context_top_k: number;
context_min_score: number;
context_max_tokens: number;
}
/**
* Extract the server's `detail` message from a failed JSON response so the UI
* surfaces the real cause (e.g. "openjarvis_rust extension is not installed")
* instead of a blanket fallback string (#502).
*/
async function memoryErrorDetail(res: Response, fallback: string): Promise<string> {
try {
const data = await res.json();
if (data && typeof data.detail === 'string' && data.detail) return data.detail;
} catch {
// Non-JSON body — fall through to the generic message below.
}
return fallback;
}
export async function getMemoryStats(): Promise<MemoryStats> {
const res = await apiFetch(`/v1/memory/stats`);
if (!res.ok) throw new Error('Failed to fetch memory stats');
@@ -977,16 +1048,16 @@ export async function storeMemory(content: string, metadata?: Record<string, unk
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ content, metadata }),
});
if (!res.ok) throw new Error('Failed to store memory');
if (!res.ok) throw new Error(await memoryErrorDetail(res, 'Failed to store memory'));
}
export async function indexMemoryPath(path: string): Promise<{ chunks_indexed: number }> {
export async function indexMemoryPath(path: string): Promise<{ chunks_indexed: number; note?: string }> {
const res = await apiFetch(`/v1/memory/index`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ path }),
});
if (!res.ok) throw new Error('Failed to index path');
if (!res.ok) throw new Error(await memoryErrorDetail(res, 'Failed to index path'));
return res.json();
}
+34 -3
View File
@@ -1,5 +1,5 @@
import { getBase } from './api';
import type { ConnectorInfo, SyncStatus, ConnectRequest } from '../types/connectors';
import type { ConnectorInfo, SyncStatus, ConnectRequest, ConnectResponse } from '../types/connectors';
// ---------------------------------------------------------------------------
// Connectors API
@@ -18,16 +18,47 @@ export async function getConnector(id: string): Promise<ConnectorInfo> {
return res.json();
}
export async function connectSource(id: string, req: ConnectRequest): Promise<ConnectorInfo> {
export async function connectSource(id: string, req: ConnectRequest): Promise<ConnectResponse> {
const res = await fetch(`${getBase()}/v1/connectors/${encodeURIComponent(id)}/connect`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(req),
});
if (!res.ok) throw new Error(`Failed to connect ${id}: ${res.status}`);
if (!res.ok) {
// Surface the backend's actionable detail (e.g. malformed Client ID /
// Secret) instead of a bare status code so the UI can render it.
const err = await res.json().catch(() => ({ detail: res.statusText }));
throw new Error(err.detail || `Failed to connect ${id}: ${res.status}`);
}
return res.json();
}
/** Open the server-side OAuth consent flow in a popup and resolve once the
* connector reports connected (or reject on timeout). Reused for any OAuth
* connector whose /connect returned `oauth_required` (issue #512). */
export function startServerOAuth(id: string, oauthStartPath?: string): Promise<void> {
const path = oauthStartPath || `/v1/connectors/${encodeURIComponent(id)}/oauth/start`;
window.open(`${getBase()}${path}`, '_blank', 'width=600,height=700');
return new Promise((resolve, reject) => {
const interval = setInterval(async () => {
try {
const info = await getConnector(id);
if (info.connected) {
clearInterval(interval);
clearTimeout(timer);
resolve();
}
} catch {
// ignore transient polling errors
}
}, 2000);
const timer = setTimeout(() => {
clearInterval(interval);
reject(new Error('Authorization timed out — please try again.'));
}, 180000);
});
}
export async function disconnectSource(id: string): Promise<void> {
const res = await fetch(`${getBase()}/v1/connectors/${encodeURIComponent(id)}/disconnect`, {
method: 'POST',
@@ -0,0 +1,19 @@
import { describe, expect, it } from 'vitest';
import { isEmbedOnlyModel } from './model-capabilities';
describe('isEmbedOnlyModel', () => {
it.each([
'nomic-embed-text',
'mxbai-embed-large',
'text-embedding-3-small',
'all-minilm:latest',
'hf.co/BAAI/bge-m3:latest',
])('classifies %s as embedding-only', (modelId) => {
expect(isEmbedOnlyModel(modelId)).toBe(true);
});
it.each(['qwen3.5:4b', 'codegemma:7b'])('keeps %s available for chat', (modelId) => {
expect(isEmbedOnlyModel(modelId)).toBe(false);
});
});
+22
View File
@@ -0,0 +1,22 @@
const EMBEDDING_MODEL_PREFIXES = [
'all-minilm',
'bge-',
'bge_',
'e5-',
'e5_',
'gte-',
'gte_',
'jina-embeddings',
'nomic-bert',
'sentence-transformers',
];
export function isEmbedOnlyModel(modelId: string): boolean {
const name = (modelId || '').trim().toLowerCase();
const leaf = name.slice(name.lastIndexOf('/') + 1).split(':')[0];
return (
leaf.includes('embed') ||
leaf.includes('minilm') ||
EMBEDDING_MODEL_PREFIXES.some((prefix) => leaf.startsWith(prefix))
);
}
+5 -4
View File
@@ -1,5 +1,5 @@
import type { ResearchEvent, SSEEvent } from '../types';
import { getBase } from './api';
import { getBase, authHeaders } from './api';
export interface ChatRequest {
model: string;
@@ -16,7 +16,7 @@ export async function* streamChat(
const base = getBase();
const response = await fetch(`${base}/v1/chat/completions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
headers: authHeaders({ 'Content-Type': 'application/json' }),
body: JSON.stringify(request),
signal,
});
@@ -60,6 +60,7 @@ export async function* streamChat(
export async function* streamResearch(
query: string,
model?: string,
signal?: AbortSignal,
): AsyncGenerator<ResearchEvent> {
// /api/research is mounted at the server root — strip any trailing /v1
@@ -67,8 +68,8 @@ export async function* streamResearch(
const base = getBase().replace(/\/v1\/?$/, '');
const response = await fetch(`${base}/api/research`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query }),
headers: authHeaders({ 'Content-Type': 'application/json' }),
body: JSON.stringify({ query, ...(model ? { model } : {}) }),
signal,
});
+64
View File
@@ -0,0 +1,64 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { ModelInfo } from '../types';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
}
const model = (id: string): ModelInfo => ({
id,
object: 'model',
created: 0,
owned_by: 'openjarvis',
});
beforeEach(() => {
vi.resetModules();
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
describe('setModels', () => {
it('does not select an embedding-only model', async () => {
const { useAppStore } = await import('./store');
useAppStore.getState().setModels([model('nomic-embed-text')]);
expect(useAppStore.getState().selectedModel).toBe('');
});
it('clears a missing selection when no chat fallback exists', async () => {
const { useAppStore } = await import('./store');
useAppStore.getState().setSelectedModel('deleted-chat-model');
useAppStore.getState().setModels([model('nomic-embed-text')]);
expect(useAppStore.getState().selectedModel).toBe('');
});
it('replaces an embedding selection with an available chat model', async () => {
const { useAppStore } = await import('./store');
useAppStore.getState().setSelectedModel('all-minilm:latest');
useAppStore.getState().setModels([
model('all-minilm:latest'),
model('qwen3.5:4b'),
]);
expect(useAppStore.getState().selectedModel).toBe('qwen3.5:4b');
});
});
@@ -0,0 +1,82 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
removeItem(key: string): void {
this.store.delete(key);
}
}
beforeEach(() => {
vi.resetModules();
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
async function freshStore() {
return (await import('./store')).useAppStore;
}
describe('conversation stream ownership', () => {
it('persists background stream updates without replacing the active messages', async () => {
const store = await freshStore();
const sourceId = store.getState().createConversation('test-model');
store.getState().addMessage(sourceId, {
id: 'assistant',
role: 'assistant',
content: '',
timestamp: 1,
});
const activeId = store.getState().createConversation('test-model');
store.getState().setStreamState({
conversationId: sourceId,
isStreaming: true,
content: 'streamed response',
});
store.getState().updateLastAssistant(sourceId, 'streamed response');
expect(store.getState().activeId).toBe(activeId);
expect(store.getState().messages).toEqual([]);
store.getState().selectConversation(sourceId);
expect(store.getState().messages).toHaveLength(1);
expect(store.getState().messages[0].content).toBe('streamed response');
});
it('keeps the stream-owning conversation until generation stops', async () => {
const store = await freshStore();
const sourceId = store.getState().createConversation('test-model');
const activeId = store.getState().createConversation('test-model');
store.getState().setStreamState({
conversationId: sourceId,
isStreaming: true,
});
store.getState().deleteConversation(sourceId);
expect(
store.getState().conversations.map((conversation) => conversation.id),
).toContain(sourceId);
expect(store.getState().activeId).toBe(activeId);
store.getState().resetStream();
store.getState().deleteConversation(sourceId);
expect(
store.getState().conversations.map((conversation) => conversation.id),
).not.toContain(sourceId);
});
});
+46 -12
View File
@@ -15,6 +15,7 @@ import type {
TokenUsage,
} from '../types';
import type { ManagedAgent } from './api';
import { isEmbedOnlyModel } from './model-capabilities';
export interface CachedConnector {
connector_id: string;
@@ -110,6 +111,7 @@ function saveSettings(settings: Settings): void {
// ── Store ─────────────────────────────────────────────────────────────
const INITIAL_STREAM: StreamState = {
conversationId: null,
isStreaming: false,
phase: '',
elapsedMs: 0,
@@ -351,6 +353,9 @@ export const useAppStore = create<AppState>((set, get) => {
},
deleteConversation: (id: string) => {
const streamState = get().streamState;
if (streamState.isStreaming && streamState.conversationId === id) return;
const store = loadConversations();
delete store.conversations[id];
if (store.activeId === id) {
@@ -393,12 +398,14 @@ export const useAppStore = create<AppState>((set, get) => {
(message.content.length > 50 ? '...' : '');
}
saveConversations(store);
set({
messages: [...conv.messages],
conversations: Object.values(store.conversations).sort(
(a, b) => b.updatedAt - a.updatedAt,
),
});
const conversations = Object.values(store.conversations).sort(
(a, b) => b.updatedAt - a.updatedAt,
);
if (get().activeId === conversationId) {
set({ messages: [...conv.messages], conversations });
} else {
set({ conversations });
}
},
updateLastAssistant: (
@@ -425,7 +432,9 @@ export const useAppStore = create<AppState>((set, get) => {
if (researchSources) lastMsg.researchSources = researchSources;
conv.updatedAt = Date.now();
saveConversations(store);
set({ messages: [...conv.messages] });
if (get().activeId === conversationId) {
set({ messages: [...conv.messages] });
}
}
},
@@ -444,11 +453,36 @@ export const useAppStore = create<AppState>((set, get) => {
// ── Models & server ────────────────────────────────────────────
setModels: (models: ModelInfo[]) =>
set((state) =>
!state.selectedModel && models.length > 0
? { models, selectedModel: models[0].id }
: { models },
),
set((state) => {
// Ollama returns embed-only models (e.g. nomic-embed-text) in the
// same list as chat models. Auto-picking models[0] selected the
// embedder and every chat failed with HTTP 400 "does not support
// chat". Prefer a real chat model for selection / fallback.
const chatModels = models.filter((m) => !isEmbedOnlyModel(m.id));
const preferred =
(state.settings.defaultModel &&
chatModels.some((m) => m.id === state.settings.defaultModel) &&
state.settings.defaultModel) ||
chatModels[0]?.id ||
models.find((m) => !isEmbedOnlyModel(m.id))?.id ||
'';
const currentIsBad =
!!state.selectedModel && isEmbedOnlyModel(state.selectedModel);
const currentMissing =
!!state.selectedModel &&
!models.some((m) => m.id === state.selectedModel);
if (!state.selectedModel || currentIsBad || currentMissing) {
// Prefer a real chat model. If none exist, clear a bad/missing
// selection rather than keeping an embed-only id that 400s on chat.
return {
models,
selectedModel: preferred,
};
}
return { models };
}),
setModelsLoading: (loading: boolean) => set({ modelsLoading: loading }),
setSelectedModel: (model: string) => set({ selectedModel: model }),
setServerInfo: (info: ServerInfo | null) => set({ serverInfo: info }),
+11
View File
@@ -0,0 +1,11 @@
export const SUPABASE_URL =
import.meta.env.VITE_SUPABASE_URL || 'https://mtbtgpwzrbostweaanpr.supabase.co';
// The Supabase anon key is optional at build time. When it is unset the public
// savings leaderboard is disabled rather than failing the build — this keeps
// the `openjarvis` package and desktop app buildable without coupling
// publishability to a leaderboard credential. Set VITE_SUPABASE_ANON_KEY at
// build time (from a repo secret) to enable the leaderboard.
export const SUPABASE_ANON_KEY = import.meta.env.VITE_SUPABASE_ANON_KEY ?? '';
export const LEADERBOARD_ENABLED = SUPABASE_ANON_KEY.length > 0;
+5 -5
View File
@@ -23,7 +23,6 @@ import {
fetchAgentTrace,
fetchManagedAgent,
fetchAvailableTools,
saveToolCredentials,
fetchModels,
updateManagedAgent,
fetchRecommendedModel,
@@ -575,7 +574,7 @@ function ToolsPicker({
</div>
{/* Live description strip */}
<div
className="flex items-center gap-2 px-2.5 py-1.5"
className="flex items-start gap-2 px-2.5 py-1.5"
style={{
borderTop: '1px solid var(--color-border)',
background: 'var(--color-bg)',
@@ -609,10 +608,11 @@ function ToolsPicker({
</span>
)}
<span
className="truncate"
className="min-w-0 whitespace-normal break-words"
style={{
flex: 1,
color: 'var(--color-text-tertiary)',
lineHeight: 1.4,
}}
>
{hovered ? `${hint}` : hint}
@@ -3740,8 +3740,8 @@ export function AgentsPage() {
const paramsB = paramMatch ? parseFloat(paramMatch[1]) : 9;
const flops = 2 * paramsB * 1e9 * (inTok + outTok);
const providers = [
{ label: 'GPT-5.3', inPer1M: 2.0, outPer1M: 10.0 },
{ label: 'Claude Opus 4.6', inPer1M: 5.0, outPer1M: 25.0 },
{ label: 'GPT-5.6 Sol', inPer1M: 5.0, outPer1M: 30.0 },
{ label: 'Claude Fable 5', inPer1M: 10.0, outPer1M: 50.0 },
{ label: 'Gemini 3.1 Pro', inPer1M: 2.0, outPer1M: 12.0 },
];
const energyWh = (inTok + outTok) / 1000 * 0.4;
+14 -2
View File
@@ -24,7 +24,7 @@ import {
import type { LucideIcon } from 'lucide-react';
import { SOURCE_CATALOG } from '../types/connectors';
import type { ConnectRequest } from '../types/connectors';
import { listConnectors, connectSource, disconnectSource, getSyncStatus, triggerSync } from '../lib/connectors-api';
import { listConnectors, connectSource, disconnectSource, getSyncStatus, triggerSync, startServerOAuth } from '../lib/connectors-api';
import type { SyncStatus } from '../types/connectors';
// ---------------------------------------------------------------------------
@@ -673,7 +673,19 @@ function DataSourcesSection() {
setConnectStage('Connecting...');
setConnectError('');
try {
await connectSource(id, req);
const resp = await connectSource(id, req);
// OAuth connectors (Google Drive/Calendar/Contacts/Gmail/Tasks): pasting
// a Client ID / Secret only registers the app credentials. The backend
// returns `oauth_required` with the path to the in-process consent flow,
// which is the only path that actually mints an access token. Open it now
// and wait for the callback to flip the connector to connected. Without
// this the connector would stay "pending" forever — the exact #512 bug.
if (resp.status === 'oauth_required') {
setConnectStage('Opening Google sign-in...');
await startServerOAuth(id, resp.oauth_start);
}
setConnectStage('Connected! Starting sync...');
// Wait for connector to show as connected
+1 -1
View File
@@ -417,7 +417,7 @@ function SelfHostedView() {
<p className="text-xs" style={{ color: 'var(--color-text-secondary)' }}>
Launch the API server to get the full UI in your browser:
</p>
<CodeBlock code={"git clone https://github.com/open-jarvis/OpenJarvis.git\ncd OpenJarvis\nuv sync --extra server\njarvis serve --port 8000"} />
<CodeBlock code={"git clone https://github.com/open-jarvis/OpenJarvis.git\ncd OpenJarvis\nuv sync --extra desktop\njarvis serve --port 8000"} />
<p className="text-xs" style={{ color: 'var(--color-text-tertiary)' }}>
The chat, dashboard, energy profiling, and cost comparison all run
locally on your machine.
+143 -25
View File
@@ -19,9 +19,24 @@ import {
RefreshCw,
} from 'lucide-react';
import { useAppStore, type ThemeMode } from '../lib/store';
import { checkHealth, fetchSpeechHealth, getMemoryStats, getInferenceSource, setInferenceSource, type InferenceSource } from '../lib/api';
import {
checkHealth,
fetchSpeechHealth,
getMemoryStats,
getInferenceSource,
setInferenceSource,
getCloudKeyStatus,
saveCloudKey,
fetchToolCredentialStatus,
saveToolCredentials,
deleteToolCredential,
isTauri,
type InferenceSource,
} from '../lib/api';
import { isAutoUpdateDisabled, setAutoUpdateDisabled } from '../components/Desktop/UpdateChecker';
const CLOUD_KEY_STATUS_CHANGED = 'openjarvis-cloud-key-status-changed';
function OllamaModelList() {
const [models, setModels] = useState<Array<{ name: string; size: number }>>([]);
useEffect(() => {
@@ -44,32 +59,135 @@ function OllamaModelList() {
);
}
function ApiKeyInput({ storageKey, placeholder }: { storageKey: string; placeholder: string }) {
const [value, setValue] = useState(() => {
try { return localStorage.getItem(storageKey) || ''; } catch { return ''; }
});
function ApiKeyInput({
keyName,
placeholder,
toolName,
}: {
keyName: string;
placeholder: string;
toolName?: string;
}) {
const [value, setValue] = useState('');
const [saved, setSaved] = useState(false);
const save = (v: string) => {
setValue(v);
try { if (v) localStorage.setItem(storageKey, v); else localStorage.removeItem(storageKey); } catch {}
setSaved(true);
setTimeout(() => setSaved(false), 2000);
const [hasKey, setHasKey] = useState(false);
const [error, setError] = useState('');
const desktopKeyStorage = isTauri();
const serverToolStorage = !desktopKeyStorage && !!toolName;
const canManage = desktopKeyStorage || serverToolStorage;
const refresh = useCallback(async () => {
if (!canManage) {
setHasKey(false);
return;
}
try {
const status = desktopKeyStorage
? await getCloudKeyStatus()
: await fetchToolCredentialStatus(toolName!);
setHasKey(!!status[keyName]);
} catch {
setHasKey(false);
}
}, [canManage, desktopKeyStorage, keyName, toolName]);
useEffect(() => {
void refresh();
window.addEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
return () => window.removeEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
}, [refresh]);
const save = async (v: string) => {
const next = v.trim();
if (!next) return;
setError('');
try {
if (desktopKeyStorage) {
await saveCloudKey(keyName, next);
} else if (toolName) {
await saveToolCredentials(toolName, { [keyName]: next });
} else {
return;
}
setValue('');
setHasKey(true);
setSaved(true);
window.dispatchEvent(new Event(CLOUD_KEY_STATUS_CHANGED));
setTimeout(() => setSaved(false), 2000);
} catch (e: any) {
setError(e?.message || 'Failed to save API key');
}
};
const remove = async () => {
setError('');
try {
if (desktopKeyStorage) {
await saveCloudKey(keyName, '');
} else if (toolName) {
await deleteToolCredential(toolName, keyName);
} else {
return;
}
setValue('');
setHasKey(false);
setSaved(true);
window.dispatchEvent(new Event(CLOUD_KEY_STATUS_CHANGED));
setTimeout(() => setSaved(false), 2000);
} catch (e: any) {
setError(e?.message || 'Failed to remove API key');
}
};
return (
<div className="flex items-center gap-2">
<input type="password" value={value} onChange={e => save(e.target.value)} placeholder={placeholder}
<input
type="password"
value={value}
onChange={e => setValue(e.target.value)}
onBlur={() => { if (value.trim()) void save(value); }}
placeholder={hasKey ? (desktopKeyStorage ? 'Saved in secure storage' : 'Saved by local server') : placeholder}
disabled={!canManage}
className="w-48 px-2 py-1 rounded text-xs"
style={{ background: 'var(--color-bg)', border: '1px solid var(--color-border)', color: 'var(--color-text)' }} />
{hasKey && (
<button
onClick={() => void remove()}
className="px-2 py-1 rounded text-[10px] cursor-pointer"
style={{ color: 'var(--color-error)', border: '1px solid var(--color-error)' }}
>
Remove
</button>
)}
{saved && <span className="text-[10px]" style={{ color: 'var(--color-success)' }}>Saved</span>}
{error && <span className="text-[10px]" style={{ color: 'var(--color-error)' }}>{error}</span>}
</div>
);
}
function CloudProviderStatus({ label, storageKey }: { label: string; storageKey: string }) {
function CloudProviderStatus({ label, keyName }: { label: string; keyName: string }) {
const [hasKey, setHasKey] = useState(false);
const desktopKeyStorage = isTauri();
const refresh = useCallback(async () => {
if (!desktopKeyStorage) {
setHasKey(false);
return;
}
try {
const status = await getCloudKeyStatus();
setHasKey(!!status[keyName]);
} catch {
setHasKey(false);
}
}, [desktopKeyStorage, keyName]);
useEffect(() => {
try { setHasKey(!!localStorage.getItem(storageKey)); } catch { setHasKey(false); }
}, [storageKey]);
void refresh();
window.addEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
return () => window.removeEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
}, [refresh]);
return (
<span className="flex items-center gap-1 text-xs" style={{ color: 'var(--color-text-secondary)' }}>
<span style={{
@@ -424,10 +542,10 @@ export function SettingsPage() {
</div>
<SettingRow label="Cloud providers" description="Green dot means API key is configured">
<div className="flex flex-wrap gap-3">
<CloudProviderStatus label="OpenAI" storageKey="openjarvis-openai-key" />
<CloudProviderStatus label="Anthropic" storageKey="openjarvis-anthropic-key" />
<CloudProviderStatus label="Google" storageKey="openjarvis-gemini-key" />
<CloudProviderStatus label="OpenRouter" storageKey="openjarvis-openrouter-key" />
<CloudProviderStatus label="OpenAI" keyName="OPENAI_API_KEY" />
<CloudProviderStatus label="Anthropic" keyName="ANTHROPIC_API_KEY" />
<CloudProviderStatus label="Google" keyName="GEMINI_API_KEY" />
<CloudProviderStatus label="OpenRouter" keyName="OPENROUTER_API_KEY" />
</div>
</SettingRow>
</Section>
@@ -435,23 +553,23 @@ export function SettingsPage() {
{/* API Keys */}
<Section title="API Keys">
<SettingRow label="OpenAI" description="GPT-4, GPT-3.5, etc.">
<ApiKeyInput storageKey="openjarvis-openai-key" placeholder="sk-..." />
<ApiKeyInput keyName="OPENAI_API_KEY" placeholder="sk-..." />
</SettingRow>
<SettingRow label="Anthropic" description="Claude models">
<ApiKeyInput storageKey="openjarvis-anthropic-key" placeholder="sk-ant-..." />
<ApiKeyInput keyName="ANTHROPIC_API_KEY" placeholder="sk-ant-..." />
</SettingRow>
<SettingRow label="Google" description="Gemini models">
<ApiKeyInput storageKey="openjarvis-gemini-key" placeholder="AI..." />
<ApiKeyInput keyName="GEMINI_API_KEY" placeholder="AI..." />
</SettingRow>
<SettingRow label="OpenRouter" description="Multi-provider routing">
<ApiKeyInput storageKey="openjarvis-openrouter-key" placeholder="sk-or-..." />
<ApiKeyInput keyName="OPENROUTER_API_KEY" placeholder="sk-or-..." />
</SettingRow>
</Section>
{/* Tools */}
<Section title="Tools">
<SettingRow label="Web Search" description="SerpAPI or Tavily key for web search tool">
<ApiKeyInput storageKey="openjarvis-search-key" placeholder="API key..." />
<SettingRow label="Web Search" description="Tavily key for web search tool">
<ApiKeyInput keyName="TAVILY_API_KEY" placeholder="tvly-..." toolName="web_search" />
</SettingRow>
</Section>
@@ -714,7 +832,7 @@ export function SettingsPage() {
</p>
<div className="flex gap-3 mt-3 text-xs">
<a
href="https://scalingintelligence.stanford.edu/blogs/openjarvis/"
href="https://openjarvis.stanford.edu/"
target="_blank"
rel="noopener noreferrer"
style={{ color: 'var(--color-accent)' }}
+15 -2
View File
@@ -55,6 +55,19 @@ export interface ConnectRequest {
password?: string;
}
/** Response from POST /v1/connectors/{id}/connect.
* For OAuth connectors, pasting a Client ID / Secret pair only registers the
* app credentials; the backend returns `status: "oauth_required"` plus an
* `oauth_start` path the UI must open to run the browser consent flow that
* actually mints an access token (see issue #512). */
export interface ConnectResponse {
connector_id: string;
connected: boolean;
status: "connected" | "pending" | "oauth_required" | "disconnected";
oauth_start?: string;
sync_status?: string | null;
}
export type WizardStep = "pick" | "connect" | "ingest" | "ready";
// Backward-compatible alias
@@ -257,12 +270,12 @@ export const SOURCE_CATALOG: ConnectorMeta[] = [
urlLabel: 'Enable Drive API',
},
{
label: 'Create OAuth credentials: go to Credentials (link below) → click "+ Create Credentials" → choose "OAuth client ID" → Application type: "Desktop app" → click "Create"',
label: 'Create OAuth credentials: go to Credentials (link below) → click "+ Create Credentials" → choose "OAuth client ID" → Application type: "Web application". Under "Authorized redirect URIs" add this server\'s callback (e.g. http://localhost:1313/v1/connectors/gdrive/oauth/callback — match the host/port your OpenJarvis server is bound to) → click "Create".',
url: 'https://console.cloud.google.com/apis/credentials',
urlLabel: 'Open Credentials',
},
{
label: 'A dialog will show your Client ID and Client Secret. Copy both and paste them below. (If you miss it, click the download icon next to your OAuth client to see them again)',
label: 'A dialog will show your Client ID and Client Secret. Copy both and paste them below, then click Connect — a Google sign-in window opens to finish authorization. (If you miss the dialog, click the download icon next to your OAuth client to see them again.)',
},
],
inputFields: [
+1
View File
@@ -147,6 +147,7 @@ export interface ConversationStore {
// --- Stream State ---
export interface StreamState {
conversationId: string | null;
isStreaming: boolean;
phase: string;
elapsedMs: number;
+3 -1
View File
@@ -1,7 +1,9 @@
/// <reference types="vite/client" />
interface ImportMetaEnv {
readonly VITE_API_URL: string;
readonly VITE_API_URL?: string;
readonly VITE_SUPABASE_URL?: string;
readonly VITE_SUPABASE_ANON_KEY?: string;
}
interface ImportMeta {
+12 -1
View File
@@ -4,6 +4,9 @@ import react from '@vitejs/plugin-react';
import tailwindcss from '@tailwindcss/vite';
import { VitePWA } from 'vite-plugin-pwa';
// VITE_SUPABASE_ANON_KEY is intentionally NOT required here: a missing key
// disables the savings leaderboard at runtime (see src/lib/supabase.ts) rather
// than failing the build, so the package/app stays publishable without it.
export default defineConfig({
resolve: {
alias: {
@@ -51,7 +54,15 @@ export default defineConfig({
server: {
port: 5173,
proxy: {
'/v1': process.env.VITE_API_URL || 'http://localhost:8000',
// ws: true is required for the /v1/agents/events WebSocket. Without it
// Vite proxies the HTTP request but not the upgrade, so the socket never
// opens — no error, no close event, just silence — and every live agent
// view sits empty in dev while working in a production build.
'/v1': {
target: process.env.VITE_API_URL || 'http://localhost:8000',
changeOrigin: true,
ws: true,
},
'/health': process.env.VITE_API_URL || 'http://localhost:8000',
'/api': process.env.VITE_API_URL || 'http://localhost:8000',
},
+4
View File
@@ -127,6 +127,7 @@ markdown_extensions:
- pymdownx.tilde
extra_javascript:
- javascripts/leaderboard-config.js
- javascripts/leaderboard.js
- https://cdn.jsdelivr.net/npm/@docsearch/js@3
- javascripts/docsearch-init.js
@@ -193,6 +194,9 @@ nav:
- External MCP Servers: user-guide/mcp-external-servers.md
- Scheduler: user-guide/scheduler.md
- Telemetry: user-guide/telemetry.md
- Evaluations: user-guide/evaluations.md
- Benchmarks: user-guide/benchmarks.md
- System Access: user-guide/system-access.md
- Security: user-guide/security.md
- LLM-guided spec search: user-guide/llm-guided-spec-search.md
- Leaderboard: leaderboard.md
+47 -2
View File
@@ -1,10 +1,10 @@
[build-system]
requires = ["hatchling"]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"
[project]
name = "OpenJarvis"
version = "1.0.2"
dynamic = ["version"]
description = "OpenJarvis — modular AI assistant backend with composable intelligence primitives"
readme = "README.md"
# Upper bound: numpy 2.2.x (pinned transitively via datasets/pandas) ships no
@@ -48,6 +48,7 @@ dev = [
"pytest>=8",
"pytest-asyncio>=0.24",
"pytest-cov>=5",
"pytest-xdist>=3",
"respx>=0.22",
"ruff>=0.4",
"pre-commit>=3.0",
@@ -84,6 +85,13 @@ server = [
"pydantic>=2.0",
"python-multipart>=0.0.9",
]
desktop = [
"fastapi>=0.110",
"uvicorn>=0.30",
"pydantic>=2.0",
"python-multipart>=0.0.9",
"faster-whisper>=1.0",
]
openhands = ["openhands-sdk>=1.0; python_version >= '3.12'"]
gpu-metrics = ["pynvml>=12.0"]
energy-amd = ["amdsmi>=6.1"]
@@ -152,6 +160,34 @@ Issues = "https://github.com/open-jarvis/OpenJarvis/issues"
[project.scripts]
jarvis = "openjarvis.cli:main"
openjarvis-eval = "openjarvis.evals.cli:main"
# Version is derived from git tags by hatch-vcs (see #526). For source/editable
# checkouts this yields the true `git describe` version (e.g. 1.0.3.dev109+g<sha>)
# rather than a stale static string. CI release builds override this with
# SETUPTOOLS_SCM_PRETEND_VERSION so the published version equals the pushed tag.
#
# setuptools_scm cannot bump custom `.devN` tags (only `.dev0`), so the autotag
# `vX.Y.Z.devN` tags are deliberately EXCLUDED from version derivation here; the
# base is taken from the latest plain release tag (vX.Y.Z) and the dev distance
# is computed from commit count since that release.
[tool.hatch.version]
source = "vcs"
[tool.hatch.version.raw-options]
tag_regex = '^v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$'
git_describe_command = [
"git", "describe", "--dirty", "--tags", "--long",
"--match", "v[0-9]*", "--exclude", "*dev*", "--exclude", "*rc*", "--exclude", "desktop-*",
]
# Builds without a git checkout (e.g. the `COPY src/ src/` Docker stages, which
# never include .git) can't run `git describe`. Without a fallback that would
# hard-fail the build. Mirror the runtime sentinel in src/openjarvis/__init__.py.
# Such builds can inject the real version via SETUPTOOLS_SCM_PRETEND_VERSION.
fallback_version = "0.0.0+unknown"
[tool.uv.sources]
openjarvis-rust = { path = "rust/crates/openjarvis-python" }
[tool.hatch.build.targets.wheel]
packages = ["src/openjarvis"]
@@ -202,3 +238,12 @@ select = ["E", "F", "I", "W"]
dev = [
"maturin>=1.12.6",
]
# openjarvis_rust is the native PyO3 extension, built from the local Rust
# workspace. It lives in a uv dependency group (PEP 735) — not the published
# `desktop` extra — so `uv sync --group desktop-native` builds it from source
# for the desktop app, while `pip install openjarvis[desktop]` from PyPI does
# NOT try to resolve openjarvis-rust from PyPI, where it isn't published
# (dependency groups are excluded from wheel metadata). See #584 / #615.
desktop-native = [
"openjarvis-rust",
]
@@ -144,16 +144,21 @@ impl MemoryBackend for SQLiteMemory {
) -> Result<Vec<RetrievalResult>, OpenJarvisError> {
let conn = self.conn.lock();
// Split on any non-alphanumeric character (not just whitespace) so
// internal punctuation — apostrophes in particular ("user's") — never
// reaches the FTS5 MATCH string. FTS5's query grammar treats an
// unescaped `'` as a string delimiter, so passing a raw token like
// `user's` through silently fails to parse and yields zero rows with
// no visible error. Splitting fully avoids needing to escape anything.
let words: Vec<String> = query
.split_whitespace()
.map(|w| w.trim_matches(|c: char| "?.,!;:'\"()[]{}/ ".contains(c)).to_string())
.split(|c: char| !c.is_alphanumeric())
.map(|w| w.to_string())
.filter(|w| !w.is_empty())
.collect();
let fts_query = if words.len() == 1 {
words[0].clone()
} else {
words.join(" OR ")
};
if words.is_empty() {
return Ok(Vec::new());
}
let fts_query = words.join(" OR ");
let mut stmt = conn
.prepare(
@@ -320,6 +325,27 @@ mod tests {
assert_eq!(mixed.len(), 2, "mixed-case query should find both documents");
}
#[test]
fn test_sqlite_apostrophe_in_query() {
let mem = SQLiteMemory::in_memory().unwrap();
mem.store("The user's name is Trev.", "identity", None).unwrap();
// A query containing an internal apostrophe must not break FTS5's
// MATCH syntax (an unescaped `'` is a string delimiter in FTS5's
// query grammar), which previously caused this to silently return
// zero results instead of matching or erroring.
let multi_word = mem.retrieve("what is the user's name", 5).unwrap();
assert!(
!multi_word.is_empty(),
"query with an internal apostrophe should not silently return zero results"
);
// Bare single-word possessive: exercises the (former) single-word
// bypass path that skipped the OR-join entirely.
let bare = mem.retrieve("user's", 5).unwrap();
assert!(!bare.is_empty(), "single-word possessive query should still match");
}
#[test]
fn test_sqlite_scores_are_positive() {
let mem = SQLiteMemory::in_memory().unwrap();
+16
View File
@@ -28,6 +28,22 @@ fi
cd "$SRC_DIR"
if uv run maturin develop -m "$MANIFEST" >>"$LOG" 2>&1; then
# Verify the extension actually imports from THIS venv before declaring
# success. `maturin develop` can report success while installing the .so
# into a different venv than the one that runs the server, which leaves
# memory silently broken at runtime (#502). Only the import check below
# proves the serving venv can load it.
if ! uv run python -c "import openjarvis_rust" >>"$LOG" 2>&1; then
rc=$?
{
echo "build-extension.sh: maturin succeeded but 'import openjarvis_rust'"
echo "failed in the serving venv ($SRC_DIR/.venv) — the extension was"
echo "not installed where the server runs. (exit=$rc)"
tail -n 50 "$LOG" 2>/dev/null || true
} > "$FAILED"
rm -f "$BUILT"
exit "$rc"
fi
tmp="$BUILT.tmp"
date -u +"%Y-%m-%dT%H:%M:%SZ" > "$tmp"
mv "$tmp" "$BUILT"
+8
View File
@@ -210,6 +210,14 @@ if ! command -v python3 >/dev/null 2>&1; then
fi
# ---- env ----
# OpenJarvis keeps ALL of its state (install tree + runtime data, configs,
# databases, caches, logs) under a single root so it never clutters $HOME
# beyond one directory. Relocate it by exporting OPENJARVIS_HOME before
# running the installer, e.g.:
# OPENJARVIS_HOME=~/apps/openjarvis curl ... | bash
# The Python runtime honors the same override (and, when OPENJARVIS_HOME is
# unset, $XDG_DATA_HOME/openjarvis if XDG_DATA_HOME is set). With nothing set
# the root is ~/.openjarvis, so existing installs are untouched.
OPENJARVIS_HOME="${OPENJARVIS_HOME:-$HOME/.openjarvis}"
OPENJARVIS_REPO_URL="${OPENJARVIS_REPO_URL:-https://github.com/open-jarvis/OpenJarvis.git}"
SRC_DIR="$OPENJARVIS_HOME/src"
+10 -3
View File
@@ -148,7 +148,8 @@ fi
# ── 7. Install Python dependencies ──────────────────────────────────
info "Installing Python dependencies..."
uv sync --extra server --quiet 2>/dev/null || uv sync --extra server
uv sync --extra desktop --extra tools-search --quiet 2>/dev/null \
|| uv sync --extra desktop --extra tools-search
ok "Python dependencies installed"
# ── 7b. Build Rust extension ──────────────────────────────────────
@@ -164,11 +165,17 @@ ok "Frontend dependencies installed"
# ── 9. Start backend ────────────────────────────────────────────────
info "Starting backend API server on port 8000..."
if curl -sf http://localhost:8000/health &>/dev/null; then
fail "An OpenJarvis server is already running on port 8000. Stop it before re-running quickstart so updated environment variables are applied."
fi
uv run jarvis serve --port 8000 &>/dev/null &
CLEANUP_PIDS+=($!)
BACKEND_PID=$!
CLEANUP_PIDS+=("$BACKEND_PID")
sleep 3
if curl -sf http://localhost:8000/health &>/dev/null; then
if ! kill -0 "$BACKEND_PID" 2>/dev/null; then
fail "Backend exited during startup. Run 'uv run jarvis serve --port 8000' to see the error."
elif curl -sf http://localhost:8000/health &>/dev/null; then
ok "Backend running at http://localhost:8000"
else
warn "Backend may still be starting..."
+4
View File
@@ -57,6 +57,10 @@ class BaseAgent(ABC):
agent_id: str
accepts_tools: bool = False
# Plain conversational agents may opt into the managed runtime's generic
# function-calling loop. Specialized agents keep their own execution
# class even when process-wide MCP tools are available.
supports_managed_tool_fallback: bool = False
def __init__(
self,
+2 -1
View File
@@ -19,6 +19,7 @@ from typing import Any, List, Optional
from openjarvis.agents._stubs import AgentContext, AgentResult, BaseAgent
from openjarvis.core.events import EventBus
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import ToolResult
from openjarvis.engine._stubs import InferenceEngine
@@ -103,7 +104,7 @@ class ClaudeCodeAgent(BaseAgent):
"Install it from https://nodejs.org/ or via your package manager."
)
dest = Path.home() / ".openjarvis" / "claude_code_runner"
dest = get_config_dir() / "claude_code_runner"
dest.mkdir(parents=True, exist_ok=True)
# Copy runner files if missing or outdated
+3 -1
View File
@@ -9,6 +9,8 @@ from datetime import datetime
from pathlib import Path
from typing import Dict, List, Optional
from openjarvis.core.paths import get_config_dir
@dataclass
class DigestArtifact:
@@ -30,7 +32,7 @@ class DigestStore:
def __init__(self, db_path: str = "") -> None:
if not db_path:
db_path = str(Path.home() / ".openjarvis" / "digest.db")
db_path = str(get_config_dir() / "digest.db")
self._db_path = db_path
self._conn = sqlite3.connect(db_path, check_same_thread=False)
self._conn.execute("PRAGMA journal_mode=WAL")
+15
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
from openjarvis.engine._base import looks_like_context_length_error
class AgentTickError(Exception):
"""Base class for agent tick errors."""
@@ -64,6 +66,14 @@ def classify_error(exc: Exception) -> AgentTickError:
msg = str(exc).lower()
# A context-window overflow is deterministic: retrying the identical
# over-length request can never succeed, so fail fast instead of burning
# the retry budget on it.
if getattr(exc, "is_context_length_error", False) or (
looks_like_context_length_error(msg)
):
return FatalError(str(exc))
# Check fatal patterns first (more specific)
if isinstance(exc, PermissionError):
return FatalError(str(exc))
@@ -90,6 +100,11 @@ def retry_delay(attempt: int) -> int:
def suggest_action(error: AgentTickError) -> str:
"""Return a human-readable suggested action for the given error."""
msg = str(error).lower()
if looks_like_context_length_error(msg):
return (
"Conversation too long for the model's context window \u2014 "
"start a new chat or shorten the conversation"
)
if any(p in msg for p in ("rate limit", "rate_limit", "429", "too many requests")):
return "Rate limited \u2014 agent will auto-retry on next tick"
if any(p in msg for p in ("timeout", "timed out", "connection", "unavailable")):
+192 -109
View File
@@ -2,7 +2,9 @@
from __future__ import annotations
import json
import logging
import threading
import time
from typing import TYPE_CHECKING, Any
@@ -14,6 +16,7 @@ from openjarvis.agents.errors import (
classify_error,
retry_delay,
)
from openjarvis.agents.tool_resolver import resolve_agent_tools
from openjarvis.core.events import EventBus, EventType
if TYPE_CHECKING:
@@ -33,6 +36,32 @@ _MAX_RETRIES = 3
_AGENT_TICK_DEFAULT_MODEL = "gemma4:31b"
def _tool_calls_for_storage(result: AgentResult) -> list[dict[str, Any]] | None:
"""Convert executor tool results to the managed-message storage contract."""
calls: list[dict[str, Any]] = []
for tool_result in result.tool_results:
metadata = getattr(tool_result, "metadata", {}) or {}
arguments = metadata.get("arguments", "")
if not isinstance(arguments, str):
try:
arguments = json.dumps(arguments, sort_keys=True)
except (TypeError, ValueError):
arguments = json.dumps(str(arguments))
calls.append(
{
"tool": getattr(tool_result, "tool_name", ""),
"arguments": arguments,
"result": getattr(tool_result, "content", "") or "",
"success": bool(getattr(tool_result, "success", False)),
# SSE and the frontend persist/display latency in milliseconds.
"latency": float(getattr(tool_result, "latency_seconds", 0.0) or 0.0)
* 1000.0,
}
)
return calls or None
class AgentExecutor:
"""Executes a single tick for a managed agent.
@@ -51,6 +80,7 @@ class AgentExecutor:
self._manager = manager
self._bus = event_bus
self._trace_store = trace_store
self._toolkit_local = threading.local()
def set_system(self, system: Any) -> None:
"""Deferred system injection — called after JarvisSystem is constructed."""
@@ -63,27 +93,6 @@ class AgentExecutor:
except Exception:
pass # Non-critical
def _inject_tool_deps(self, tool: Any) -> None:
"""Inject runtime dependencies into a tool instance.
Mirrors SystemBuilder._inject_tool_deps (system.py:920-945)
but uses the lightweight system's references.
"""
if self._system is None:
return
name = getattr(getattr(tool, "spec", None), "name", "")
if name == "llm":
if hasattr(tool, "_engine"):
tool._engine = self._system.engine
if hasattr(tool, "_model"):
tool._model = self._system.model
elif name == "retrieval" or name.startswith("memory_"):
if hasattr(tool, "_backend"):
tool._backend = getattr(self._system, "memory_backend", None)
elif name.startswith("channel_"):
if hasattr(tool, "_channel"):
tool._channel = getattr(self._system, "channel_backend", None)
def run_ephemeral(
self,
agent_type: str,
@@ -102,9 +111,7 @@ class AgentExecutor:
)
return agent.run(input_text)
def execute_tick(
self, agent_id: str, *, lock_already_held: bool = False
) -> None:
def execute_tick(self, agent_id: str, *, lock_already_held: bool = False) -> None:
"""Run one tick for the given agent.
1. Acquire concurrency guard (start_tick)
@@ -126,9 +133,7 @@ class AgentExecutor:
self._manager.start_tick(agent_id)
self._set_activity(agent_id, "Preparing tick...")
except ValueError:
logger.warning(
"Agent %s already running, skipping tick", agent_id
)
logger.warning("Agent %s already running, skipping tick", agent_id)
return
agent = self._manager.get_agent(agent_id)
@@ -252,7 +257,20 @@ class AgentExecutor:
raise last_error or FatalError("max retries exhausted")
def _invoke_agent(self, agent: dict) -> AgentResult:
"""Invoke the actual agent run. Tests mock this method."""
"""Invoke one agent while owning every resource its resolver opens."""
previous = getattr(self._toolkit_local, "current", None)
self._toolkit_local.current = None
try:
return self._invoke_agent_impl(agent)
finally:
current = getattr(self._toolkit_local, "current", None)
if current is not None:
current.close()
self._toolkit_local.current = previous
def _invoke_agent_impl(self, agent: dict) -> AgentResult:
"""Implementation split out so the wrapper owns resolver lifetime."""
from openjarvis.agents import AgentRegistry
agent_type = agent.get("agent_type", "monitor_operative")
@@ -261,6 +279,10 @@ class AgentExecutor:
raise FatalError(f"Unknown agent type: {agent_type}")
config = agent.get("config", {})
agent_accepts_tools = bool(getattr(agent_cls, "accepts_tools", False))
supports_tool_fallback = bool(
getattr(agent_cls, "supports_managed_tool_fallback", False)
)
# Resolve engine + model from JarvisSystem
engine = self._system.engine if self._system else None
@@ -304,64 +326,88 @@ class AgentExecutor:
except Exception:
pass # Fall back to configured model
# Resolve tools from config via ToolRegistry
tool_names = config.get("tools", [])
if isinstance(tool_names, str):
tool_names = [t.strip() for t in tool_names.split(",") if t.strip()]
mcp_tools: list[Any] = []
mcp_clients: list[Any] = []
if (
config.get("mcp_tools", True) is not False
and self._system is not None
and (agent_accepts_tools or supports_tool_fallback)
):
provider = getattr(
self._system,
"get_managed_agent_mcp_tools",
None,
)
if callable(provider):
try:
mcp_tools, mcp_clients = provider()
except Exception as exc:
logger.warning("Managed-agent MCP discovery failed: %s", exc)
else:
mcp_tools = list(getattr(self._system, "mcp_tools", []) or [])
mcp_clients = list(getattr(self._system, "_mcp_clients", []) or [])
tool_instances: list[Any] = []
if tool_names:
try:
from openjarvis.server.agent_manager_routes import (
_ensure_registries_populated,
)
if not mcp_tools:
try:
from openjarvis.tools.mcp_adapter import MCPToolAdapter
_ensure_registries_populated()
except ImportError:
pass
from openjarvis.core.registry import ToolRegistry
pool = (
getattr(
getattr(self._system, "tool_executor", None),
"_tools",
{},
)
or {}
)
mcp_tools = [
tool
for tool in pool.values()
if isinstance(tool, MCPToolAdapter)
]
except Exception:
mcp_tools = []
for tname in tool_names:
if ToolRegistry.contains(tname):
try:
tool_cls = ToolRegistry.get(tname)
tool = tool_cls()
self._inject_tool_deps(tool)
tool_instances.append(tool)
except Exception:
logger.warning("Failed to instantiate tool %s", tname)
resolved_toolkit = resolve_agent_tools(
agent,
engine=engine,
model=model,
memory_backend=getattr(self._system, "memory_backend", None),
channel_backend=getattr(self._system, "channel_backend", None),
mcp_tools=mcp_tools,
mcp_clients=mcp_clients,
knowledge_db_path=getattr(self._system, "knowledge_db_path", None),
)
self._toolkit_local.current = resolved_toolkit
tool_instances = resolved_toolkit.instances
logger.info(
"Agent %s: resolved %d tools (%s)",
agent["name"],
len(tool_instances),
", ".join(resolved_toolkit.by_name) or "none",
)
# Pull tools already discovered by SystemBuilder (e.g. external MCP
# adapters) that aren't in the static ToolRegistry. Without this,
# agents declaring MCP-discovered tools in their template would
# silently fall back to natives only.
if (
self._system is not None
and getattr(self._system, "tool_executor", None) is not None
):
mcp_pool = getattr(self._system.tool_executor, "_tools", {}) or {}
existing = {t.spec.name for t in tool_instances}
for tname in tool_names:
if tname in existing:
continue
pooled = mcp_pool.get(tname)
if pooled is not None:
tool_instances.append(pooled)
execution_agent_cls = agent_cls
if tool_instances and not agent_accepts_tools and supports_tool_fallback:
# Managed SSE already runs configured tools through a native
# function-calling loop regardless of the selected class. Use the
# same capability for immediate/scheduled ticks instead of
# silently discarding the resolved toolkit for SimpleAgent and
# other explicitly compatible non-tool classes.
from openjarvis.agents.orchestrator import OrchestratorAgent
if tool_instances:
logger.info(
"Agent %s: resolved %d/%d tools",
agent["name"],
len(tool_instances),
len(tool_names),
)
execution_agent_cls = OrchestratorAgent
logger.info(
"Agent %s: %s does not accept tools; using %s for this "
"tool-enabled tick",
agent["name"],
agent_cls.__name__,
execution_agent_cls.__name__,
)
# Construct agent instance
agent_kwargs: dict[str, Any] = {}
sys_prompt = config.get("system_prompt")
if sys_prompt is not None:
agent_kwargs["system_prompt"] = sys_prompt
if getattr(agent_cls, "accepts_tools", False) and tool_instances:
if getattr(execution_agent_cls, "accepts_tools", False) and tool_instances:
agent_kwargs["tools"] = tool_instances
# Hand the agent our EventBus so its ToolExecutor can publish
# TOOL_CALL_START/END — without this, ToolExecutor's ``self._bus``
@@ -383,7 +429,7 @@ class AgentExecutor:
# recall / persistence paths.
import inspect
init_sig = inspect.signature(agent_cls.__init__)
init_sig = inspect.signature(execution_agent_cls.__init__)
accepts_var_kw = any(
p.kind == inspect.Parameter.VAR_KEYWORD
for p in init_sig.parameters.values()
@@ -392,6 +438,16 @@ class AgentExecutor:
def _accepts(name: str) -> bool:
return accepts_var_kw or name in init_sig.parameters
# Unsupported kwargs used to trigger the broad TypeError fallback
# below, which retried with a bare constructor and silently discarded
# valid prompt/state wiring. Filter by the selected class's signature
# before construction instead.
if sys_prompt is not None and _accepts("system_prompt"):
agent_kwargs["system_prompt"] = sys_prompt
agent_kwargs = {
name: value for name, value in agent_kwargs.items() if _accepts(name)
}
state_kwargs: dict[str, Any] = {}
if _accepts("operator_id"):
state_kwargs["operator_id"] = agent["id"]
@@ -408,27 +464,49 @@ class AgentExecutor:
# agents, mirroring the one-shot `jarvis ask` path so they no
# longer apply to CLI calls only (#376).
cfg = getattr(self._system, "config", None)
if cfg is not None and _accepts("prompt_builder"):
if _accepts("prompt_builder") and (
cfg is not None or sys_prompt is not None
):
from openjarvis.prompt.builder import SystemPromptBuilder
state_kwargs["prompt_builder"] = SystemPromptBuilder(
agent_template=getattr(
cfg.agent, "default_system_prompt", ""
)
or "",
memory_files_config=cfg.memory_files,
system_prompt_config=cfg.system_prompt,
agent_template=(
sys_prompt
if sys_prompt is not None
else getattr(
getattr(cfg, "agent", None),
"default_system_prompt",
"",
)
or ""
),
memory_files_config=getattr(cfg, "memory_files", None),
system_prompt_config=getattr(cfg, "system_prompt", None),
)
try:
agent_instance = agent_cls(
engine, model, **agent_kwargs, **state_kwargs
)
except TypeError:
try:
agent_instance = agent_cls(engine, model, **agent_kwargs)
agent_instance = execution_agent_cls(
engine,
model,
**agent_kwargs,
**state_kwargs,
)
except TypeError:
agent_instance = agent_cls(engine, model)
try:
agent_instance = execution_agent_cls(
engine,
model,
**agent_kwargs,
)
except TypeError:
agent_instance = execution_agent_cls(engine, model)
except Exception:
resolved_toolkit.close()
raise
if resolved_toolkit.mcp_clients:
agent_instance._mcp_clients = resolved_toolkit.mcp_clients
# Inject the managed-agent UUID into the agent's ToolExecutor so
# emitted TOOL_CALL_START/END events carry it; the trace subscriber
@@ -444,7 +522,7 @@ class AgentExecutor:
agent["name"],
len(tool_instances),
", ".join(t.spec.name for t in tool_instances) or "none",
agent_cls.__name__,
execution_agent_cls.__name__,
)
# Build input from instruction + summary_memory + pending messages.
@@ -474,9 +552,7 @@ class AgentExecutor:
tick_note = f"Previous tick: {first_sentence}"
if instruction:
input_text = (
f"Current date: {today}\n\nStanding instruction: {instruction}"
)
input_text = f"Current date: {today}\n\nStanding instruction: {instruction}"
if tick_note:
input_text += f"\n\n{tick_note}"
else:
@@ -561,21 +637,24 @@ class AgentExecutor:
len(input_text),
)
_t0 = time.time()
result = agent_instance.run(input_text, context=agent_ctx)
# Retry once if the model returned empty content (common with
# Qwen3.5 thinking mode consuming all tokens).
if not (result.content or "").strip():
self._set_activity(
agent["id"],
"Retrying (empty response)...",
)
logger.warning(
"Agent %s: empty content, retrying once",
agent["name"],
)
try:
result = agent_instance.run(input_text, context=agent_ctx)
# Retry once if the model returned empty content (common with
# Qwen3.5 thinking mode consuming all tokens).
if not (result.content or "").strip():
self._set_activity(
agent["id"],
"Retrying (empty response)...",
)
logger.warning(
"Agent %s: empty content, retrying once",
agent["name"],
)
result = agent_instance.run(input_text, context=agent_ctx)
finally:
resolved_toolkit.close()
_elapsed = time.time() - _t0
logger.info(
"Agent %s: agent.run() completed in %.1fs, "
@@ -665,7 +744,11 @@ class AgentExecutor:
# message keeps the complete report. The old [:2000] slices
# double-truncated and cut findings off mid-sentence.
self._manager.update_summary_memory(agent_id, result.content)
self._manager.store_agent_response(agent_id, result.content)
self._manager.store_agent_response(
agent_id,
result.content,
tool_calls=_tool_calls_for_storage(result),
)
# Budget enforcement (post-tick check)
agent_data = self._manager.get_agent(agent_id)
+283 -188
View File
@@ -73,12 +73,19 @@ WEB_SEARCH_COST_PER_CALL = 0.01
# $0.01/call number — kept as a separate constant so it can drift.
OPENAI_WEB_SEARCH_COST_PER_CALL = 0.01
# Gemini Google-Search grounding: billed at $35 per 1000 grounded
# *requests* (2025-12 public list price for the Grounding-with-Google-Search
# tool, charged once per request that uses the tool regardless of how many
# internal queries it issues). We charge per grounded request, not per
# `web_search_queries` entry.
GEMINI_SEARCH_COST_PER_CALL = 0.035
# Gemini 3 Google-Search grounding: billed at $14 per 1000 search queries.
# `_call_gemini_agent` reports the model's `web_search_queries`, so this is
# charged per query, not per outer generate_content request.
GEMINI_SEARCH_COST_PER_CALL = 0.014
# Tavily Search, advanced depth: 2 API credits per search request at $0.008
# per credit on the public pay-as-you-go plan. WebSearchTool captures actual
# credits when Tavily returns usage metadata; this is the fallback estimate.
TAVILY_SEARCH_COST_PER_CREDIT = 0.008
TAVILY_ADVANCED_SEARCH_CREDITS = 2
TAVILY_SEARCH_COST_PER_CALL = (
TAVILY_SEARCH_COST_PER_CREDIT * TAVILY_ADVANCED_SEARCH_CREDITS
)
ANTHROPIC_WEB_SEARCH_TOOL = {
"type": "web_search_20250305",
@@ -101,6 +108,40 @@ def build_web_search_tool(max_uses: int = 8) -> Dict[str, Any]:
}
def tavily_search_context(
query: str,
*,
max_results: int = 5,
) -> Dict[str, Any]:
"""Run OpenJarvis WebSearchTool and return accounting-friendly metadata."""
from openjarvis.tools.web_search import WebSearchTool
tool = WebSearchTool(max_results=max_results)
res = tool.execute(query=query, max_results=max_results)
meta = dict(res.metadata or {})
engine = str(meta.get("engine") or "unknown")
credits = 0
cost_usd = 0.0
if engine == "tavily":
try:
credits = int(meta.get("credits") or TAVILY_ADVANCED_SEARCH_CREDITS)
except (TypeError, ValueError):
credits = TAVILY_ADVANCED_SEARCH_CREDITS
cost_usd = credits * TAVILY_SEARCH_COST_PER_CREDIT
text = res.content or ""
if not res.success and not text:
text = "(no search results)"
return {
"text": text,
"success": bool(res.success),
"engine": engine,
"credits": credits,
"cost_usd": cost_usd,
"n_searches": 1 if (query or "").strip() else 0,
"error": None if res.success else text,
}
def web_search_cfg(method_cfg: Optional[Dict[str, Any]]) -> Tuple[bool, int]:
"""Parse ``method_cfg.web_search = { enabled, max_uses }``.
@@ -262,7 +303,9 @@ def _openrouter_limiter() -> _OpenRouterLimiter:
if _OPENROUTER_LIMITER is None:
with _OPENROUTER_LIMITER_LOCK:
if _OPENROUTER_LIMITER is None:
max_concurrent = int(os.environ.get("OJ_OPENROUTER_MAX_CONCURRENT", "20") or 20)
max_concurrent = int(
os.environ.get("OJ_OPENROUTER_MAX_CONCURRENT", "20") or 20
)
rpm = int(os.environ.get("OJ_OPENROUTER_RPM", "60") or 60)
_OPENROUTER_LIMITER = _OpenRouterLimiter(max_concurrent, rpm)
return _OPENROUTER_LIMITER
@@ -278,8 +321,14 @@ def _serialize_block(block: Any) -> Dict[str, Any]:
"""
out: Dict[str, Any] = {"type": getattr(block, "type", type(block).__name__)}
for attr in (
"id", "name", "input", "text", "thinking", "signature",
"tool_use_id", "content",
"id",
"name",
"input",
"text",
"thinking",
"signature",
"tool_use_id",
"content",
):
if hasattr(block, attr):
val = getattr(block, attr)
@@ -300,14 +349,16 @@ def _serialize_openai_tool_calls(tool_calls: Any) -> List[Dict[str, Any]]:
return out
for tc in tool_calls:
fn = getattr(tc, "function", None)
out.append({
"id": getattr(tc, "id", None),
"type": getattr(tc, "type", "function"),
"function": {
"name": getattr(fn, "name", None) if fn else None,
"arguments": getattr(fn, "arguments", None) if fn else None,
},
})
out.append(
{
"id": getattr(tc, "id", None),
"type": getattr(tc, "type", "function"),
"function": {
"name": getattr(fn, "name", None) if fn else None,
"arguments": getattr(fn, "arguments", None) if fn else None,
},
}
)
return out
@@ -432,32 +483,46 @@ class LocalCloudAgent(BaseAgent):
srv = getattr(msg.usage, "server_tool_use", None)
n_searches = getattr(srv, "web_search_requests", 0) if srv else 0
content_blocks = [_serialize_block(b) for b in msg.content]
tool_use_blocks = [b for b in content_blocks if b.get("type") in (
"tool_use", "server_tool_use",
)]
tool_result_blocks = [b for b in content_blocks if b.get("type") in (
"web_search_tool_result", "tool_result",
)]
_record_event({
"kind": "anthropic",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"content_blocks": content_blocks,
"tool_calls": tool_use_blocks,
"tool_results": tool_result_blocks,
"tokens_in": msg.usage.input_tokens,
"tokens_out": msg.usage.output_tokens,
"n_web_searches": n_searches,
"tools_declared": tools,
"tool_choice": tool_choice,
"output_config": output_config,
"stop_reason": getattr(msg, "stop_reason", None),
"latency_s": latency,
"ts": time.time(),
})
tool_use_blocks = [
b
for b in content_blocks
if b.get("type")
in (
"tool_use",
"server_tool_use",
)
]
tool_result_blocks = [
b
for b in content_blocks
if b.get("type")
in (
"web_search_tool_result",
"tool_result",
)
]
_record_event(
{
"kind": "anthropic",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"content_blocks": content_blocks,
"tool_calls": tool_use_blocks,
"tool_results": tool_result_blocks,
"tokens_in": msg.usage.input_tokens,
"tokens_out": msg.usage.output_tokens,
"n_web_searches": n_searches,
"tools_declared": tools,
"tool_choice": tool_choice,
"output_config": output_config,
"stop_reason": getattr(msg, "stop_reason", None),
"latency_s": latency,
"ts": time.time(),
}
)
return text, msg.usage.input_tokens, msg.usage.output_tokens, n_searches
@staticmethod
@@ -509,24 +574,26 @@ class LocalCloudAgent(BaseAgent):
u = resp.usage
p = getattr(u, "prompt_tokens", 0) if u else 0
c = getattr(u, "completion_tokens", 0) if u else 0
_record_event({
"kind": "openai",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tool_calls": tool_calls,
"reasoning_content": reasoning,
"tokens_in": p,
"tokens_out": c,
"response_format": response_format,
"tools_declared": tools,
"tool_choice": tool_choice,
"finish_reason": getattr(choice, "finish_reason", None),
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "openai",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tool_calls": tool_calls,
"reasoning_content": reasoning,
"tokens_in": p,
"tokens_out": c,
"response_format": response_format,
"tools_declared": tools,
"tool_choice": tool_choice,
"finish_reason": getattr(choice, "finish_reason", None),
"latency_s": latency,
"ts": time.time(),
}
)
return text, p, c
@staticmethod
@@ -573,12 +640,10 @@ class LocalCloudAgent(BaseAgent):
from openai import OpenAI
if model.startswith("openrouter/"):
model = model[len("openrouter/"):]
model = model[len("openrouter/") :]
api_key = os.environ.get("OPENROUTER_API_KEY")
if not api_key:
raise RuntimeError(
"OPENROUTER_API_KEY is not set; cannot call OpenRouter."
)
raise RuntimeError("OPENROUTER_API_KEY is not set; cannot call OpenRouter.")
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=api_key,
@@ -617,21 +682,23 @@ class LocalCloudAgent(BaseAgent):
u = resp.usage
p = getattr(u, "prompt_tokens", 0) if u else 0
c = getattr(u, "completion_tokens", 0) if u else 0
_record_event({
"kind": "openrouter",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tool_calls": tool_calls,
"reasoning_content": reasoning,
"tokens_in": p,
"tokens_out": c,
"finish_reason": getattr(choice, "finish_reason", None),
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "openrouter",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tool_calls": tool_calls,
"reasoning_content": reasoning,
"tokens_in": p,
"tokens_out": c,
"finish_reason": getattr(choice, "finish_reason", None),
"latency_s": latency,
"ts": time.time(),
}
)
return text, p, c
@staticmethod
@@ -660,7 +727,9 @@ class LocalCloudAgent(BaseAgent):
from google import genai
from google.genai import types
client = genai.Client(http_options=types.HttpOptions(timeout=int(timeout * 1000)))
client = genai.Client(
http_options=types.HttpOptions(timeout=int(timeout * 1000))
)
cfg = types.GenerateContentConfig(
temperature=temperature,
max_output_tokens=max_tokens,
@@ -687,21 +756,23 @@ class LocalCloudAgent(BaseAgent):
finish_reason = str(resp.candidates[0].finish_reason)
except Exception:
pass
_record_event({
"kind": "gemini",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tokens_in": p,
"tokens_out": c,
"temperature": temperature,
"max_tokens": max_tokens,
"finish_reason": finish_reason,
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "gemini",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tokens_in": p,
"tokens_out": c,
"temperature": temperature,
"max_tokens": max_tokens,
"finish_reason": finish_reason,
"latency_s": latency,
"ts": time.time(),
}
)
return text, p, c
@staticmethod
@@ -756,27 +827,29 @@ class LocalCloudAgent(BaseAgent):
u = resp.usage
p = getattr(u, "prompt_tokens", 0) if u else 0
c = getattr(u, "completion_tokens", 0) if u else 0
_record_event({
"kind": "vllm",
"role": trace_role,
"model": model,
"endpoint": endpoint,
"system": system,
"user": user,
"response": text,
"tool_calls": tool_calls,
"reasoning_content": reasoning,
"tokens_in": p,
"tokens_out": c,
"temperature": temperature,
"max_tokens": max_tokens,
"enable_thinking": enable_thinking,
"tools_declared": tools,
"tool_choice": tool_choice,
"finish_reason": getattr(choice, "finish_reason", None),
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "vllm",
"role": trace_role,
"model": model,
"endpoint": endpoint,
"system": system,
"user": user,
"response": text,
"tool_calls": tool_calls,
"reasoning_content": reasoning,
"tokens_in": p,
"tokens_out": c,
"temperature": temperature,
"max_tokens": max_tokens,
"enable_thinking": enable_thinking,
"tools_declared": tools,
"tool_choice": tool_choice,
"finish_reason": getattr(choice, "finish_reason", None),
"latency_s": latency,
"ts": time.time(),
}
)
return text, p, c
@staticmethod
@@ -843,33 +916,37 @@ class LocalCloudAgent(BaseAgent):
n_searches = getattr(srv, "web_search_requests", 0) if srv else 0
content_blocks = [_serialize_block(b) for b in msg.content]
tool_use_blocks = [
b for b in content_blocks
b
for b in content_blocks
if b.get("type") in ("tool_use", "server_tool_use")
]
tool_result_blocks = [
b for b in content_blocks
b
for b in content_blocks
if b.get("type") in ("web_search_tool_result", "tool_result")
]
stop_reason = getattr(msg, "stop_reason", None)
_record_event({
"kind": "anthropic",
"role": trace_role,
"model": model,
"system": system if turn == 0 else None,
"user": user if turn == 0 else None,
"turn": turn,
"response": text,
"content_blocks": content_blocks,
"tool_calls": tool_use_blocks,
"tool_results": tool_result_blocks,
"tokens_in": msg.usage.input_tokens,
"tokens_out": msg.usage.output_tokens,
"n_web_searches": n_searches,
"tools_declared": tools,
"stop_reason": stop_reason,
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "anthropic",
"role": trace_role,
"model": model,
"system": system if turn == 0 else None,
"user": user if turn == 0 else None,
"turn": turn,
"response": text,
"content_blocks": content_blocks,
"tool_calls": tool_use_blocks,
"tool_results": tool_result_blocks,
"tokens_in": msg.usage.input_tokens,
"tokens_out": msg.usage.output_tokens,
"n_web_searches": n_searches,
"tools_declared": tools,
"stop_reason": stop_reason,
"latency_s": latency,
"ts": time.time(),
}
)
p_total += msg.usage.input_tokens
c_total += msg.usage.output_tokens
n_searches_total += n_searches
@@ -879,9 +956,7 @@ class LocalCloudAgent(BaseAgent):
# here — break and let the caller (or future loop variant)
# handle it. Only ``server_tool_use`` blocks (web_search)
# are auto-continued by Anthropic itself.
client_tool_use = any(
b.get("type") == "tool_use" for b in content_blocks
)
client_tool_use = any(b.get("type") == "tool_use" for b in content_blocks)
if client_tool_use:
break
if stop_reason == "end_turn" or stop_reason is None:
@@ -890,10 +965,12 @@ class LocalCloudAgent(BaseAgent):
# (server side) — Anthropic returned mid-thought. Append the
# assistant turn and ask it to continue.
messages.append({"role": "assistant", "content": msg.content})
messages.append({
"role": "user",
"content": "Continue.",
})
messages.append(
{
"role": "user",
"content": "Continue.",
}
)
return last_text, p_total, c_total, n_searches_total, turns
@staticmethod
@@ -967,8 +1044,12 @@ class LocalCloudAgent(BaseAgent):
continue
raise
if resp is None:
raise last_exc if last_exc is not None else RuntimeError(
"openai responses.create failed for all web_search tool names"
raise (
last_exc
if last_exc is not None
else RuntimeError(
"openai responses.create failed for all web_search tool names"
)
)
_bump_cloud_calls()
latency = time.time() - t0
@@ -992,30 +1073,35 @@ class LocalCloudAgent(BaseAgent):
text = "".join(chunks)
n_searches = sum(
1 for item in output_items
if getattr(item, "type", None) in (
"web_search_call", "web_search_tool_call",
1
for item in output_items
if getattr(item, "type", None)
in (
"web_search_call",
"web_search_tool_call",
)
)
u = getattr(resp, "usage", None)
p = int(getattr(u, "input_tokens", 0) or 0) if u else 0
c = int(getattr(u, "output_tokens", 0) or 0) if u else 0
_record_event({
"kind": "openai_agent",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"output_items": _jsonable(output_items),
"tokens_in": p,
"tokens_out": c,
"n_web_searches": n_searches,
"tools_declared": [{"type": used_tool_name}],
"stop_reason": getattr(resp, "status", None),
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "openai_agent",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"output_items": _jsonable(output_items),
"tokens_in": p,
"tokens_out": c,
"n_web_searches": n_searches,
"tools_declared": [{"type": used_tool_name}],
"stop_reason": getattr(resp, "status", None),
"latency_s": latency,
"ts": time.time(),
}
)
return text, p, c, n_searches, 1
@staticmethod
@@ -1088,23 +1174,25 @@ class LocalCloudAgent(BaseAgent):
n_searches = len(web_search_queries)
except Exception: # noqa: BLE001
pass
_record_event({
"kind": "gemini_agent",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tokens_in": p,
"tokens_out": c,
"n_web_searches": n_searches,
"web_search_queries": web_search_queries,
"temperature": temperature,
"max_tokens": max_tokens,
"finish_reason": finish_reason,
"latency_s": latency,
"ts": time.time(),
})
_record_event(
{
"kind": "gemini_agent",
"role": trace_role,
"model": model,
"system": system,
"user": user,
"response": text,
"tokens_in": p,
"tokens_out": c,
"n_web_searches": n_searches,
"web_search_queries": web_search_queries,
"temperature": temperature,
"max_tokens": max_tokens,
"finish_reason": finish_reason,
"latency_s": latency,
"ts": time.time(),
}
)
return text, p, c, n_searches, 1
def _call_cloud(
@@ -1228,8 +1316,13 @@ class LocalCloudAgent(BaseAgent):
# Persist the trace before the trace state is closed (and even on
# hard failure, so we get a record of what we did before it broke).
self._write_trace_log(
context, input, answer, meta if "meta" in locals() else {},
events, soft_reason, exc_obj,
context,
input,
answer,
meta if "meta" in locals() else {},
events,
soft_reason,
exc_obj,
)
_close_trace()
_close_call_counts()
@@ -1285,9 +1378,7 @@ class LocalCloudAgent(BaseAgent):
"metadata": meta,
"events": events,
"soft_error": soft_reason,
"error": (
f"{type(exc).__name__}: {exc}" if exc is not None else None
),
"error": (f"{type(exc).__name__}: {exc}" if exc is not None else None),
}
(out_dir / f"{task_id}.json").write_text(
json.dumps(blob, indent=2, default=str)
@@ -1322,6 +1413,9 @@ __all__ = [
"LocalCloudAgent",
"NO_TEMP_PREFIXES",
"OPENAI_WEB_SEARCH_COST_PER_CALL",
"TAVILY_ADVANCED_SEARCH_CREDITS",
"TAVILY_SEARCH_COST_PER_CALL",
"TAVILY_SEARCH_COST_PER_CREDIT",
"WEB_SEARCH_COST_PER_CALL",
"_bump_cloud_calls",
"_bump_local_calls",
@@ -1329,5 +1423,6 @@ __all__ = [
"estimate_cost",
"is_gpt5_family",
"supports_temperature",
"tavily_search_context",
"web_search_cfg",
]
+1
View File
@@ -96,6 +96,7 @@ class EnergyCollector:
return self
try:
import pynvml # type: ignore[import-not-found]
pynvml.nvmlInit()
total = pynvml.nvmlDeviceGetCount()
self.gpu_indices = _resolve_gpu_indices(total)
+21 -16
View File
@@ -174,12 +174,15 @@ def _is_retryable(exc: BaseException) -> bool:
import openai
except ImportError:
return False
if isinstance(exc, (
openai.RateLimitError,
openai.APITimeoutError,
openai.APIConnectionError,
openai.InternalServerError,
)):
if isinstance(
exc,
(
openai.RateLimitError,
openai.APITimeoutError,
openai.APIConnectionError,
openai.InternalServerError,
),
):
return True
if isinstance(exc, openai.APIStatusError):
status = getattr(exc, "status_code", None)
@@ -196,7 +199,7 @@ def _sleep_for(attempt: int, exc: BaseException) -> float:
# Respect a server-provided hint, but clamp to our cap so a
# pathological header can't stall the run for hours.
return min(_RETRY_CAP, hinted) + random.uniform(0, 0.5)
base = min(_RETRY_CAP, _RETRY_BASE * (2 ** attempt))
base = min(_RETRY_CAP, _RETRY_BASE * (2**attempt))
# Full jitter — better tail behavior than equal jitter when many
# workers wake at the same moment.
return random.uniform(0.0, base)
@@ -237,16 +240,19 @@ def _wrap_create(orig: Callable[..., Any]) -> Callable[..., Any]:
import openai
except ImportError:
raise
if not isinstance(exc, (
openai.APIConnectionError,
openai.APITimeoutError,
openai.InternalServerError,
)):
if not isinstance(
exc,
(
openai.APIConnectionError,
openai.APITimeoutError,
openai.InternalServerError,
),
):
raise
local_last_exc = exc
if attempt >= 2:
break
time.sleep(2 ** attempt)
time.sleep(2**attempt)
assert local_last_exc is not None
raise local_last_exc
@@ -266,6 +272,7 @@ def _wrap_create(orig: Callable[..., Any]) -> Callable[..., Any]:
# stay parseable in the runner log.
try:
import sys
print(
f"[openai-retry] attempt {attempt + 1}/{_MAX_RETRIES} "
f"{type(exc).__name__}: {str(exc)[:120]}"
@@ -333,9 +340,7 @@ def patch_openai_globally() -> None:
from openai.resources.chat import completions as _comp_mod_async
cls = getattr(_comp_mod_async, "AsyncCompletions", None)
if cls is not None and not getattr(
cls.create, "_hybrid_patched", False
):
if cls is not None and not getattr(cls.create, "_hybrid_patched", False):
# Async wrapper is structurally different — only patch
# the bumped defaults via __init__; full retry loop on
# async would need an async wrapper. Leave that for the
+20 -17
View File
@@ -11,24 +11,27 @@ from __future__ import annotations
# USD per million tokens, (input, output). Local models = 0.
PRICES: dict[str, tuple[float, float]] = {
"claude-opus-4-7": (5.00, 25.0),
"claude-sonnet-4-6": (3.00, 15.0),
"claude-haiku-4-5": (1.00, 5.00),
"claude-haiku-4-5-20251001": (1.00, 5.00),
"gpt-5": (1.25, 10.0),
"gpt-5-mini": (0.25, 2.00),
"gpt-5-mini-2025-08-07": (0.25, 2.00),
"gpt-4o": (0.15, 0.60),
# Gemini Developer API prices (USD per 1M tokens), 2025-12 list price.
# 2.5 Pro uses tiered pricing (>200K context = $2.50/$15); we charge the
# low-context tier since GAIA / SWE-bench prompts stay well under 200K.
"gemini-2.5-pro": (1.25, 10.0),
"gemini-2.5-flash": (0.30, 2.50),
"gemini-2.5-flash-lite": (0.10, 0.40),
"claude-opus-4-7": (5.00, 25.0),
"claude-sonnet-4-6": (3.00, 15.0),
"claude-haiku-4-5": (1.00, 5.00),
"claude-haiku-4-5-20251001": (1.00, 5.00),
"gpt-5.5": (5.00, 30.0),
"gpt-5": (1.25, 10.0),
"gpt-5-mini": (0.25, 2.00),
"gpt-5-mini-2025-08-07": (0.25, 2.00),
"gpt-4o": (0.15, 0.60),
# Gemini Developer API prices (USD per 1M tokens). Pro models use tiered
# pricing above 200K prompt tokens; GAIA prompts stay under that tier, so
# charge the low-context standard rate.
"gemini-3.1-pro-preview": (2.00, 12.0),
"gemini-3.1-pro-preview-customtools": (2.00, 12.0),
"gemini-2.5-pro": (1.25, 10.0),
"gemini-2.5-flash": (0.30, 2.50),
"gemini-2.5-flash-lite": (0.10, 0.40),
# OpenRouter slugs (used by toolorchestra paper-match pool).
# Prices are OpenRouter list (USD/1M tokens), 2026-05 snapshot.
"qwen/qwen-2.5-coder-32b-instruct": (0.08, 0.18),
"qwen/qwen3-32b": (0.10, 0.30),
"qwen/qwen-2.5-coder-32b-instruct": (0.08, 0.18),
"qwen/qwen3-32b": (0.10, 0.30),
"meta-llama/llama-3.3-70b-instruct": (0.13, 0.39),
}
@@ -61,7 +64,7 @@ def is_reasoning_model(model: str) -> bool:
before emitting visible answer text. At max_tokens=4096 these silently
truncate with empty answers on GAIA (26/100 GPT-5, 18/100 Gemini Pro)."""
m = (model or "").lower()
return is_gpt5_family(model) or "gemini-2.5-pro" in m
return is_gpt5_family(model) or "gemini-2.5-pro" in m or "gemini-3.1-pro" in m
def default_max_output_tokens(model: str) -> int:
+75 -27
View File
@@ -34,6 +34,7 @@ from openjarvis.agents.hybrid._base import (
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
build_web_search_tool,
tavily_search_context,
web_search_cfg,
)
from openjarvis.agents.hybrid.mini_swe_agent import (
@@ -79,9 +80,7 @@ def _resolve_local_model(endpoint: str, registry_model: str) -> str:
a model id (e.g. ``Qwen3.5-9B``) that's different from what's loaded.
"""
try:
with urllib.request.urlopen(
endpoint.rstrip("/") + "/models", timeout=5
) as r:
with urllib.request.urlopen(endpoint.rstrip("/") + "/models", timeout=5) as r:
data = json.loads(r.read())
served = [m["id"] for m in data.get("data", [])]
except Exception:
@@ -135,7 +134,12 @@ class AdvisorsAgent(LocalCloudAgent):
advisor_temperature = float(cfg.get("advisor_temperature", 0.2))
ws_enabled, ws_max_uses = web_search_cfg(cfg)
if ws_enabled and self._cloud_endpoint not in _SEARCH_CAPABLE_ENDPOINTS:
search_backend = str(cfg.get("search_backend", "provider")).lower()
if (
ws_enabled
and search_backend != "tavily"
and self._cloud_endpoint not in _SEARCH_CAPABLE_ENDPOINTS
):
raise ValueError(
f"web_search.enabled=true but cloud_endpoint={self._cloud_endpoint!r}; "
"server-side web_search is wired for anthropic / openai / gemini "
@@ -146,19 +150,24 @@ class AdvisorsAgent(LocalCloudAgent):
use_ws = ws_enabled
gaia_max_turns = int(cfg.get("gaia_max_turns", 8))
n_searches_total = 0
search_cost_total = 0.0
# 1. Initial executor pass — advisor (Qwen) doesn't get tools;
# only the cloud executor passes do. With web_search on, dispatch
# to the search-capable agent loop for the configured provider.
if use_ws:
initial_resp, e1_in, e1_out, n_s1, e1_turns = self._executor_search(
user=f"Question:\n{question}",
system=EXECUTOR_INITIAL_SYS,
max_tokens=executor_max_tokens,
ws_max_uses=ws_max_uses,
max_turns=gaia_max_turns,
(initial_resp, e1_in, e1_out, n_s1, e1_turns, e1_search_cost) = (
self._executor_search(
user=f"Question:\n{question}",
system=EXECUTOR_INITIAL_SYS,
max_tokens=executor_max_tokens,
ws_max_uses=ws_max_uses,
max_turns=gaia_max_turns,
query=question,
)
)
n_searches_total += n_s1
search_cost_total += e1_search_cost
else:
initial_resp, e1_in, e1_out = self._call_cloud(
user=f"Question:\n{question}",
@@ -176,7 +185,8 @@ class AdvisorsAgent(LocalCloudAgent):
)
local_model = _resolve_local_model(self._local_endpoint, self._local_model)
advisor_prompt = ADVISOR_TEMPLATE.format(
question=question, initial_response=initial_resp,
question=question,
initial_response=initial_resp,
)
advisor_text, adv_in, adv_out = self._call_vllm(
local_model,
@@ -196,14 +206,18 @@ class AdvisorsAgent(LocalCloudAgent):
f"answer-format rules."
)
if use_ws:
final_answer, e2_in, e2_out, n_s2, e2_turns = self._executor_search(
user=final_user,
system=EXECUTOR_FINAL_SYS,
max_tokens=executor_max_tokens,
ws_max_uses=ws_max_uses,
max_turns=gaia_max_turns,
(final_answer, e2_in, e2_out, n_s2, e2_turns, e2_search_cost) = (
self._executor_search(
user=final_user,
system=EXECUTOR_FINAL_SYS,
max_tokens=executor_max_tokens,
ws_max_uses=ws_max_uses,
max_turns=gaia_max_turns,
query=question,
)
)
n_searches_total += n_s2
search_cost_total += e2_search_cost
else:
final_answer, e2_in, e2_out = self._call_cloud(
user=final_user,
@@ -216,7 +230,10 @@ class AdvisorsAgent(LocalCloudAgent):
tokens_local = adv_in + adv_out
tokens_cloud = e1_in + e1_out + e2_in + e2_out
cost = self.cost_usd(self._cloud_model, e1_in + e2_in, e1_out + e2_out)
cost += n_searches_total * _search_cost_per_call(self._cloud_endpoint)
if search_backend == "tavily":
cost += search_cost_total
else:
cost += n_searches_total * _search_cost_per_call(self._cloud_endpoint)
meta: Dict[str, Any] = {
"tokens_local": tokens_local,
@@ -233,7 +250,9 @@ class AdvisorsAgent(LocalCloudAgent):
"initial_response": initial_resp,
"advisor_feedback": advisor_text,
"web_search_enabled": use_ws,
"search_backend": search_backend,
"n_web_searches": n_searches_total,
"search_cost_usd": search_cost_total,
"note": "inference-only advisor (untrained); lower bound on the technique.",
},
}
@@ -251,16 +270,40 @@ class AdvisorsAgent(LocalCloudAgent):
max_tokens: int,
ws_max_uses: int,
max_turns: int,
) -> Tuple[str, int, int, int, int]:
query: Optional[str] = None,
) -> Tuple[str, int, int, int, int, float]:
"""Run a search-capable executor pass for the configured cloud.
Dispatches by ``self._cloud_endpoint`` to the matching ``_base``
agent loop. Returns the shared 5-tuple ``(text, p_tok, c_tok,
n_searches, turns)``. The endpoint is assumed already validated
against ``_SEARCH_CAPABLE_ENDPOINTS`` by the caller.
agent loop, or through Tavily when ``method_cfg.search_backend`` is
``"tavily"``. Returns ``(text, p_tok, c_tok, n_searches, turns,
search_cost_usd)``.
"""
if str(self._cfg.get("search_backend", "provider")).lower() == "tavily":
res = tavily_search_context(
query or user,
max_results=int(self._cfg.get("tavily_max_results", 5)),
)
grounded_user = (
f"Web search results:\n{res['text']}\n\n"
f"Using the search results above, answer this request:\n{user}"
)
text, p, c = self._call_cloud(
user=grounded_user,
system=system,
max_tokens=max_tokens,
temperature=0.0,
)
return (
text,
p,
c,
int(res["n_searches"]),
1,
float(res["cost_usd"]),
)
if self._cloud_endpoint == "anthropic":
return self._call_anthropic_agent(
text, p, c, n_searches, turns = self._call_anthropic_agent(
self._cloud_model,
user=user,
system=system,
@@ -269,8 +312,9 @@ class AdvisorsAgent(LocalCloudAgent):
tools=[build_web_search_tool(ws_max_uses)],
max_turns=max_turns,
)
return text, p, c, n_searches, turns, 0.0
if self._cloud_endpoint == "openai":
return self._call_openai_agent(
text, p, c, n_searches, turns = self._call_openai_agent(
self._cloud_model,
user=user,
system=system,
@@ -278,8 +322,9 @@ class AdvisorsAgent(LocalCloudAgent):
temperature=0.0,
max_turns=max_turns,
)
return text, p, c, n_searches, turns, 0.0
if self._cloud_endpoint == "gemini":
return self._call_gemini_agent(
text, p, c, n_searches, turns = self._call_gemini_agent(
self._cloud_model,
user=user,
system=system,
@@ -287,6 +332,7 @@ class AdvisorsAgent(LocalCloudAgent):
temperature=0.0,
max_turns=max_turns,
)
return text, p, c, n_searches, turns, 0.0
# Genuinely unsupported (openrouter / vllm / unknown). The caller
# guard should have caught this; raise defensively.
raise ValueError(
@@ -379,8 +425,10 @@ class AdvisorsAgent(LocalCloudAgent):
tokens_local = adv_in + adv_out
tokens_cloud = (
initial_out["tokens_in"] + initial_out["tokens_out"]
+ final_out["tokens_in"] + final_out["tokens_out"]
initial_out["tokens_in"]
+ initial_out["tokens_out"]
+ final_out["tokens_in"]
+ final_out["tokens_out"]
)
cost = initial_out["cost_usd"] + final_out["cost_usd"]
meta: Dict[str, Any] = {
+174 -117
View File
@@ -73,6 +73,7 @@ ARCHON_SWE_RANKER_SYS = (
# ---------- Stubs for Archon's eager-imported heavy deps we don't need ----------
def _stub_archon_imports() -> None:
"""``utils.py`` imports groq/google/litellm/dotenv at module load. Stub
the ones we don't use so the import chain doesn't fail when those
@@ -97,6 +98,7 @@ def _add_archon_to_path() -> None:
# ---------- Anthropic patch for Opus 4.7 ----------
def _patch_anthropic_for_opus() -> None:
from anthropic.resources.messages import messages as _msgs_mod
@@ -129,8 +131,10 @@ def _tally() -> Dict[str, int]:
counts = getattr(_TALLY_LOCAL, "counts", None)
if counts is None:
counts = {
"cloud_prompt": 0, "cloud_completion": 0,
"local_prompt": 0, "local_completion": 0,
"cloud_prompt": 0,
"cloud_completion": 0,
"local_prompt": 0,
"local_completion": 0,
"n_web_searches": 0,
}
_TALLY_LOCAL.counts = counts
@@ -141,8 +145,10 @@ def _tally() -> Dict[str, int]:
def _reset_tally() -> None:
_TALLY_LOCAL.counts = {
"cloud_prompt": 0, "cloud_completion": 0,
"local_prompt": 0, "local_completion": 0,
"cloud_prompt": 0,
"cloud_completion": 0,
"local_prompt": 0,
"local_completion": 0,
"n_web_searches": 0,
}
@@ -169,6 +175,7 @@ def _make_local_generator(local_endpoint: str, local_model: str):
def local_gen(model, messages, max_tokens=2048, temperature=0.7, **_kw): # type: ignore[no-untyped-def]
import time as _time
t0 = _time.time()
try:
resp = client.chat.completions.create(
@@ -179,31 +186,35 @@ def _make_local_generator(local_endpoint: str, local_model: str):
)
_bump_local_calls()
except Exception as e:
_record_event({
"kind": "archon_local_gen_error",
"model": local_model,
"messages": messages,
"error": f"{type(e).__name__}: {e}",
"ts": _time.time(),
})
_record_event(
{
"kind": "archon_local_gen_error",
"model": local_model,
"messages": messages,
"error": f"{type(e).__name__}: {e}",
"ts": _time.time(),
}
)
return f"[local-vllm error: {e!r}]"
u = resp.usage
if u:
_tally()["local_prompt"] += getattr(u, "prompt_tokens", 0) or 0
_tally()["local_completion"] += getattr(u, "completion_tokens", 0) or 0
text = (resp.choices[0].message.content or "").strip()
_record_event({
"kind": "archon_local_gen",
"model": local_model,
"messages": messages,
"response": text,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"temperature": temperature,
"max_tokens": max_tokens,
"latency_s": _time.time() - t0,
"ts": _time.time(),
})
_record_event(
{
"kind": "archon_local_gen",
"model": local_model,
"messages": messages,
"response": text,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"temperature": temperature,
"max_tokens": max_tokens,
"latency_s": _time.time() - t0,
"ts": _time.time(),
}
)
return text
return local_gen
@@ -218,10 +229,13 @@ def _wrap_archon_cloud_generators() -> None:
def gen_openai(model, messages, max_tokens=2048, temperature=0.7, **_kw): # type: ignore[no-untyped-def]
import time as _time
client = _OAI()
kwargs: Dict[str, Any] = dict(
model=model, messages=messages,
max_tokens=max_tokens, temperature=temperature,
model=model,
messages=messages,
max_tokens=max_tokens,
temperature=temperature,
)
# GPT-5/o1/o3 reject non-default temperature and use max_completion_tokens.
if model.startswith(("gpt-5", "o1", "o3")):
@@ -236,20 +250,23 @@ def _wrap_archon_cloud_generators() -> None:
_tally()["cloud_prompt"] += getattr(u, "prompt_tokens", 0) or 0
_tally()["cloud_completion"] += getattr(u, "completion_tokens", 0) or 0
text = (resp.choices[0].message.content or "").strip()
_record_event({
"kind": "archon_cloud_openai",
"model": model,
"messages": messages,
"response": text,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"latency_s": _time.time() - t0,
"ts": _time.time(),
})
_record_event(
{
"kind": "archon_cloud_openai",
"model": model,
"messages": messages,
"response": text,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"latency_s": _time.time() - t0,
"ts": _time.time(),
}
)
return text
def gen_anthropic(model, messages, max_tokens=2048, temperature=0.7, **_kw): # type: ignore[no-untyped-def]
import time as _time
client = _anth.Anthropic(timeout=600.0)
system = ""
msgs = []
@@ -259,7 +276,10 @@ def _wrap_archon_cloud_generators() -> None:
else:
msgs.append(m)
kwargs: Dict[str, Any] = dict(
model=model, system=system, messages=msgs, max_tokens=max_tokens,
model=model,
system=system,
messages=msgs,
max_tokens=max_tokens,
)
if not model.startswith(NO_TEMP_PREFIXES):
kwargs["temperature"] = temperature
@@ -277,24 +297,27 @@ def _wrap_archon_cloud_generators() -> None:
srv = getattr(u, "server_tool_use", None) if u else None
n_searches = getattr(srv, "web_search_requests", 0) if srv else 0
_tally()["n_web_searches"] += int(n_searches)
_record_event({
"kind": "archon_cloud_anthropic",
"model": model,
"system": system,
"messages": msgs,
"response": text.strip(),
"tokens_in": getattr(u, "input_tokens", 0) if u else 0,
"tokens_out": getattr(u, "output_tokens", 0) if u else 0,
"n_web_searches": int(n_searches),
"tools_declared": kwargs.get("tools"),
"latency_s": _time.time() - t0,
"ts": _time.time(),
})
_record_event(
{
"kind": "archon_cloud_anthropic",
"model": model,
"system": system,
"messages": msgs,
"response": text.strip(),
"tokens_in": getattr(u, "input_tokens", 0) if u else 0,
"tokens_out": getattr(u, "output_tokens", 0) if u else 0,
"n_web_searches": int(n_searches),
"tools_declared": kwargs.get("tools"),
"latency_s": _time.time() - t0,
"ts": _time.time(),
}
)
return text.strip()
from archon.completions.components.Generator import (
GENERATE_MAP as _GMAP, # type: ignore[import-not-found]
)
_GMAP["OpenAI_API"] = gen_openai
_GMAP["Anthropic_API"] = gen_anthropic
@@ -330,7 +353,9 @@ def _patch_archon_prompts() -> None:
orig = _p.make_fuser_prompt
def patched(conv, references, critiques=None, length_control=False): # type: ignore[no-untyped-def]
base = orig(conv, references, critiques=critiques, length_control=length_control)
base = orig(
conv, references, critiques=critiques, length_control=length_control
)
return base + _FUSER_FORMAT_REMINDER
patched._hybrid_format_patched = True # type: ignore[attr-defined]
@@ -339,6 +364,7 @@ def _patch_archon_prompts() -> None:
from archon.completions.components import (
Fuser as _F, # type: ignore[import-not-found]
)
_F.make_fuser_prompt = patched
@@ -354,6 +380,7 @@ def _apply_patches_once() -> None:
_patch_anthropic_for_opus()
# Trigger Archon imports so GENERATE_MAP exists.
import archon.completions.components.Generator # type: ignore[import-not-found] # noqa: F401
_wrap_archon_cloud_generators()
_patch_archon_prompts()
_PATCHES_APPLIED = True
@@ -361,49 +388,62 @@ def _apply_patches_once() -> None:
# ---------- Architecture presets ----------
def _presets():
return {
"ensemble_rank_fuse": lambda K, local_model, ranker_model, fuser_model, max_tokens, temperature: [
[{
"type": "generator",
"model": local_model,
"model_type": "vllm_local",
"top_k": 1,
"temperature": temperature,
"max_tokens": max_tokens,
"samples": K,
}],
[{
"type": "ranker",
"model": ranker_model,
"model_type": "Anthropic_API" if ranker_model.startswith("claude") else "OpenAI_API",
"top_k": min(K, 5),
"temperature": 0.0,
"max_tokens": max_tokens,
}],
[{
"type": "fuser",
"model": fuser_model,
"model_type": "Anthropic_API" if fuser_model.startswith("claude") else "OpenAI_API",
"temperature": 0.0,
"max_tokens": max_tokens,
"samples": 1,
}],
[
{
"type": "generator",
"model": local_model,
"model_type": "vllm_local",
"top_k": 1,
"temperature": temperature,
"max_tokens": max_tokens,
"samples": K,
}
],
[
{
"type": "ranker",
"model": ranker_model,
"model_type": "Anthropic_API"
if ranker_model.startswith("claude")
else "OpenAI_API",
"top_k": min(K, 5),
"temperature": 0.0,
"max_tokens": max_tokens,
}
],
[
{
"type": "fuser",
"model": fuser_model,
"model_type": "Anthropic_API"
if fuser_model.startswith("claude")
else "OpenAI_API",
"temperature": 0.0,
"max_tokens": max_tokens,
"samples": 1,
}
],
],
# ``single_local`` honors the cfg ``max_tokens`` (passed positionally
# like ``ensemble_rank_fuse``). Previously it hard-coded 2048, which
# cut Qwen off mid-reasoning before it could emit the GAIA
# ``FINAL ANSWER:`` line — the scorer then had nothing to extract.
"single_local": lambda K, local_model, ranker_model, fuser_model, max_tokens, temperature: [
[{
"type": "generator",
"model": local_model,
"model_type": "vllm_local",
"top_k": 1,
"temperature": 0.0,
"max_tokens": max_tokens,
"samples": 1,
}],
[
{
"type": "generator",
"model": local_model,
"model_type": "vllm_local",
"top_k": 1,
"temperature": 0.0,
"max_tokens": max_tokens,
"samples": 1,
}
],
],
}
@@ -464,7 +504,12 @@ class ArchonAgent(LocalCloudAgent):
)
layers = presets[arch](
K, self._local_model, ranker_model, fuser_model, max_tokens, temperature,
K,
self._local_model,
ranker_model,
fuser_model,
max_tokens,
temperature,
)
archon_cfg = {"name": f"hybrid-archon-{arch}", "layers": layers}
@@ -480,10 +525,12 @@ class ArchonAgent(LocalCloudAgent):
archon = Archon(archon_cfg)
try:
answer = archon.generate([
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": input},
])
answer = archon.generate(
[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": input},
]
)
except Exception:
# Re-raise so the base ``run()`` / runner's ``_run_one_inner``
# records this in the row's ``error`` field instead of stashing
@@ -518,10 +565,10 @@ class ArchonAgent(LocalCloudAgent):
"tool_calls": int(n_searches),
"traces": {
"architecture": arch,
"n_samples": K,
"n_samples": K,
"ranker_model": ranker_model,
"fuser_model": fuser_model,
"local_model": self._local_model,
"fuser_model": fuser_model,
"local_model": self._local_model,
"tokens_breakdown": dict(_tally()),
"web_search_enabled": ws_enabled,
"n_web_searches": n_searches,
@@ -568,26 +615,30 @@ class ArchonAgent(LocalCloudAgent):
turn_max_tokens=turn_max_tokens,
trace_prefix=f"archon_gen{k}",
)
candidates.append({
"idx": k,
"summary": out["final_summary"],
"patch": out["patch"],
"framed": out["answer"],
"tokens_in": out["tokens_in"],
"tokens_out": out["tokens_out"],
"turns": out["turns"],
})
candidates.append(
{
"idx": k,
"summary": out["final_summary"],
"patch": out["patch"],
"framed": out["answer"],
"tokens_in": out["tokens_in"],
"tokens_out": out["tokens_out"],
"turns": out["turns"],
}
)
total_tokens_local += out["tokens_in"] + out["tokens_out"]
self.record_trace_event({
"kind": "archon_swe_candidate",
"idx": k,
"patch_chars": len(out["patch"]),
"summary": out["final_summary"],
})
self.record_trace_event(
{
"kind": "archon_swe_candidate",
"idx": k,
"patch_chars": len(out["patch"]),
"summary": out["final_summary"],
}
)
# Ranker: cloud picks the best candidate.
ranker_user = (
f"Issue:\n{task.get('problem_statement','')}\n\n"
f"Issue:\n{task.get('problem_statement', '')}\n\n"
f"K = {K} candidate patches:\n\n"
+ "\n\n".join(
f"=== Candidate {c['idx']} ===\nSummary: {c['summary']}\n"
@@ -615,13 +666,15 @@ class ArchonAgent(LocalCloudAgent):
chosen_idx = 0
chosen = candidates[chosen_idx]
self.record_trace_event({
"kind": "archon_swe_rank",
"chosen_idx": chosen_idx,
"ranker_raw": ranker_text,
"tokens_in": r_in,
"tokens_out": r_out,
})
self.record_trace_event(
{
"kind": "archon_swe_rank",
"chosen_idx": chosen_idx,
"ranker_raw": ranker_text,
"tokens_in": r_in,
"tokens_out": r_out,
}
)
meta = {
"tokens_local": total_tokens_local,
@@ -635,8 +688,12 @@ class ArchonAgent(LocalCloudAgent):
"swe_mode": True,
"K": K,
"candidates": [
{"idx": c["idx"], "summary": c["summary"],
"patch_chars": len(c["patch"]), "turns": c["turns"]}
{
"idx": c["idx"],
"summary": c["summary"],
"patch_chars": len(c["patch"]),
"turns": c["turns"],
}
for c in candidates
],
"chosen_idx": chosen_idx,
+22 -8
View File
@@ -82,7 +82,11 @@ class BaselineCloudAgent(LocalCloudAgent):
max_turns=int(cfg.get("swe_max_turns", 30)),
bash_timeout=int(cfg.get("swe_bash_timeout_s", 120)),
output_cap=int(cfg.get("swe_output_cap", 10_000)),
turn_max_tokens=int(cfg.get("cloud_max_tokens", default_max_output_tokens(self._cloud_model))),
turn_max_tokens=int(
cfg.get(
"cloud_max_tokens", default_max_output_tokens(self._cloud_model)
)
),
trace_prefix="baseline_cloud",
)
meta = {
@@ -112,7 +116,11 @@ class BaselineCloudAgent(LocalCloudAgent):
text, p_tok, c_tok, n_searches, turns = self._call_anthropic_agent(
self._cloud_model,
user=input,
max_tokens=int(cfg.get("cloud_max_tokens", default_max_output_tokens(self._cloud_model))),
max_tokens=int(
cfg.get(
"cloud_max_tokens", default_max_output_tokens(self._cloud_model)
)
),
temperature=0.0,
tools=[build_web_search_tool(ws_max_uses)],
max_turns=gaia_max_turns,
@@ -145,17 +153,23 @@ class BaselineCloudAgent(LocalCloudAgent):
# wired here. Skip cleanly rather than fake one. Cells that
# want web_search must run on Anthropic until those backends
# are wired.
self.record_trace_event({
"kind": "web_search_skipped",
"reason": "non_anthropic_endpoint",
"endpoint": self._cloud_endpoint,
})
self.record_trace_event(
{
"kind": "web_search_skipped",
"reason": "non_anthropic_endpoint",
"endpoint": self._cloud_endpoint,
}
)
# One-shot direct cloud call. GAIA only — SWE goes through the
# mini-SWE-agent loop above (now supports anthropic/openai/gemini).
text, p_tok, c_tok = self._call_cloud(
user=input,
max_tokens=int(cfg.get("cloud_max_tokens", default_max_output_tokens(self._cloud_model))),
max_tokens=int(
cfg.get(
"cloud_max_tokens", default_max_output_tokens(self._cloud_model)
)
),
temperature=0.0,
)
meta = {
+283 -199
View File
@@ -46,6 +46,7 @@ from openjarvis.agents.hybrid._base import (
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
build_web_search_tool,
tavily_search_context,
web_search_cfg,
)
from openjarvis.agents.hybrid._prices import (
@@ -88,19 +89,20 @@ CONDUCTOR_STRICTER = (
"Your previous response was not valid JSON or was missing required fields. "
"Reply with ONLY a single JSON object — no prose, no code fences, no commentary "
"— containing exactly the three keys model_id (list[int]), subtasks (list[str]), "
"and access_list (list[list[int] or \"all\"]) of equal length, at most 5 entries, "
'and access_list (list[list[int] or "all"]) of equal length, at most 5 entries, '
"and access_list[0] must be [] (an empty list)."
)
# ---------- Plan parsing ----------
def _strip_fences(s: str) -> str:
s = s.strip()
if s.startswith("```"):
first_nl = s.find("\n")
if first_nl != -1:
s = s[first_nl + 1:]
s = s[first_nl + 1 :]
if s.endswith("```"):
s = s[:-3]
s = s.strip()
@@ -118,9 +120,7 @@ def _try_literal(s: str):
"""Fallback for the paper's literal Python-list output style."""
out = {}
for key in ("model_id", "subtasks", "access_list"):
m = re.search(
rf"{key}\s*=\s*(\[[^\]]*\](?:\s*\+\s*\[[^\]]*\])*)", s, re.DOTALL
)
m = re.search(rf"{key}\s*=\s*(\[[^\]]*\](?:\s*\+\s*\[[^\]]*\])*)", s, re.DOTALL)
if not m:
return None
try:
@@ -153,7 +153,7 @@ def _validate_plan(plan: Any, n_workers: int) -> Optional[str]:
if a == "all":
continue
if not isinstance(a, list):
return f"access_list[{i}] must be list or \"all\""
return f'access_list[{i}] must be list or "all"'
for j in a:
if not isinstance(j, int) or not (0 <= j < i):
return f"access_list[{i}] has bad ref {j!r}"
@@ -173,17 +173,18 @@ def _parse_plan(text: str, n_workers: int):
# ---------- Worker pool ----------
def _vllm_alive(base_url: str) -> bool:
try:
with urllib.request.urlopen(
base_url.rstrip("/") + "/models", timeout=3
) as r:
with urllib.request.urlopen(base_url.rstrip("/") + "/models", timeout=3) as r:
return r.status == 200
except Exception:
return False
def _default_pool(local_model: Optional[str], local_endpoint: Optional[str]) -> List[Dict[str, Any]]:
def _default_pool(
local_model: Optional[str], local_endpoint: Optional[str]
) -> List[Dict[str, Any]]:
"""Default worker pool — faithful to the Sakana Conductor paper (arXiv 2512.04388).
The paper composes a heterogeneous 7-worker pool spanning three frontier
@@ -205,98 +206,112 @@ def _default_pool(local_model: Optional[str], local_endpoint: Optional[str]) ->
del local_model, local_endpoint # paper default carries no local worker
pool: List[Dict[str, Any]] = []
if not os.environ.get("OJ_CONDUCTOR_DISABLE_GEMINI"):
pool.append({
"id": len(pool),
"name": "gemini-pro",
"endpoint": "gemini",
"model": "gemini-2.5-pro",
"description": (
"Google Gemini 2.5 Pro. Frontier multimodal reasoner with a "
"very large context window. Strong at long-document synthesis, "
"multi-hop factual reasoning, and tasks that benefit from "
"wide retrieval. Slower and pricier than mid-tier workers."
),
})
pool.append(
{
"id": len(pool),
"name": "gemini-pro",
"endpoint": "gemini",
"model": "gemini-2.5-pro",
"description": (
"Google Gemini 2.5 Pro. Frontier multimodal reasoner with a "
"very large context window. Strong at long-document synthesis, "
"multi-hop factual reasoning, and tasks that benefit from "
"wide retrieval. Slower and pricier than mid-tier workers."
),
}
)
if not os.environ.get("OJ_CONDUCTOR_DISABLE_ANTHROPIC"):
pool.append({
"id": len(pool),
"name": "claude-sonnet-4",
"endpoint": "anthropic",
"model": "claude-sonnet-4-6",
"description": (
"Anthropic Claude Sonnet 4. Strong general-purpose reasoner "
"with careful instruction following and reliable formatting. "
"Good default for code, structured writing, and decisive "
"steps where accuracy matters more than raw throughput."
),
})
pool.append(
{
"id": len(pool),
"name": "claude-sonnet-4",
"endpoint": "anthropic",
"model": "claude-sonnet-4-6",
"description": (
"Anthropic Claude Sonnet 4. Strong general-purpose reasoner "
"with careful instruction following and reliable formatting. "
"Good default for code, structured writing, and decisive "
"steps where accuracy matters more than raw throughput."
),
}
)
if not os.environ.get("OJ_CONDUCTOR_DISABLE_OPENAI"):
pool.append({
"id": len(pool),
"name": "gpt-5",
"endpoint": "openai",
"model": "gpt-5",
"description": (
"OpenAI GPT-5. Frontier-tier broad-knowledge model. Best for "
"open-domain factual recall, creative generation, and "
"ambiguous questions where coverage matters. Expensive; use "
"for steps where breadth of world knowledge is the bottleneck."
),
})
pool.append(
{
"id": len(pool),
"name": "gpt-5",
"endpoint": "openai",
"model": "gpt-5",
"description": (
"OpenAI GPT-5. Frontier-tier broad-knowledge model. Best for "
"open-domain factual recall, creative generation, and "
"ambiguous questions where coverage matters. Expensive; use "
"for steps where breadth of world knowledge is the bottleneck."
),
}
)
if not os.environ.get("OJ_CONDUCTOR_DISABLE_OPENROUTER"):
pool.append({
"id": len(pool),
"name": "deepseek-r1-distill-qwen-32b",
"endpoint": "openrouter",
"model": "deepseek/deepseek-r1-distill-qwen-32b",
"description": (
"DeepSeek R1 distilled into Qwen-32B (open weights via "
"OpenRouter). Specialized for chain-of-thought math, logic, "
"and competitive-programming-style problems. Verbose; "
"produces extensive reasoning traces before the final answer."
),
})
pool.append({
"id": len(pool),
"name": "gemma3-27b-it",
"endpoint": "openrouter",
"model": "google/gemma-3-27b-it",
"description": (
"Google Gemma 3 27B Instruct (open weights via OpenRouter). "
"Mid-size instruction-tuned model. Cheap and fast; solid at "
"concise summarization, extraction, and short-form Q&A on "
"given context. Weaker than the frontier workers on multi-step "
"reasoning."
),
})
pool.append({
"id": len(pool),
"name": "qwen3-32b",
"endpoint": "openrouter",
"model": "qwen/qwen3-32b",
"description": (
"Qwen3-32B in non-thinking mode (open weights via OpenRouter). "
"Fast general-purpose dialogue and instruction following. "
"Use when the step is straightforward generation, "
"summarization, or formatting — does NOT spend tokens on "
"internal reasoning."
),
})
pool.append({
"id": len(pool),
"name": "qwen3-32b-thinking",
"endpoint": "openrouter",
"model": "qwen/qwen3-32b",
"extra_body": {"reasoning": {"effort": "medium"}},
"description": (
"Qwen3-32B with reasoning enabled (open weights via "
"OpenRouter). Same backbone as 'qwen3-32b' but spends tokens "
"on an internal chain of thought before answering. Stronger "
"on math, code, and multi-step logic; slower and consumes "
"more completion tokens. Prefer this for hard reasoning "
"steps; prefer the non-thinking variant for plain dialogue."
),
})
pool.append(
{
"id": len(pool),
"name": "deepseek-r1-distill-qwen-32b",
"endpoint": "openrouter",
"model": "deepseek/deepseek-r1-distill-qwen-32b",
"description": (
"DeepSeek R1 distilled into Qwen-32B (open weights via "
"OpenRouter). Specialized for chain-of-thought math, logic, "
"and competitive-programming-style problems. Verbose; "
"produces extensive reasoning traces before the final answer."
),
}
)
pool.append(
{
"id": len(pool),
"name": "gemma3-27b-it",
"endpoint": "openrouter",
"model": "google/gemma-3-27b-it",
"description": (
"Google Gemma 3 27B Instruct (open weights via OpenRouter). "
"Mid-size instruction-tuned model. Cheap and fast; solid at "
"concise summarization, extraction, and short-form Q&A on "
"given context. Weaker than the frontier workers on multi-step "
"reasoning."
),
}
)
pool.append(
{
"id": len(pool),
"name": "qwen3-32b",
"endpoint": "openrouter",
"model": "qwen/qwen3-32b",
"description": (
"Qwen3-32B in non-thinking mode (open weights via OpenRouter). "
"Fast general-purpose dialogue and instruction following. "
"Use when the step is straightforward generation, "
"summarization, or formatting — does NOT spend tokens on "
"internal reasoning."
),
}
)
pool.append(
{
"id": len(pool),
"name": "qwen3-32b-thinking",
"endpoint": "openrouter",
"model": "qwen/qwen3-32b",
"extra_body": {"reasoning": {"effort": "medium"}},
"description": (
"Qwen3-32B with reasoning enabled (open weights via "
"OpenRouter). Same backbone as 'qwen3-32b' but spends tokens "
"on an internal chain of thought before answering. Stronger "
"on math, code, and multi-step logic; slower and consumes "
"more completion tokens. Prefer this for hard reasoning "
"steps; prefer the non-thinking variant for plain dialogue."
),
}
)
# Reassign ids contiguously in case env-gates skipped some entries.
for new_id, entry in enumerate(pool):
entry["id"] = new_id
@@ -362,16 +377,17 @@ def _resolve_worker_pool(
f"Invalid worker_pool entry [{wid_repr}]: 'id' must be an int"
)
if wid in seen_ids:
raise ValueError(
f"Invalid worker_pool entry [{wid}]: duplicate id"
)
raise ValueError(f"Invalid worker_pool entry [{wid}]: duplicate id")
seen_ids.add(wid)
if not entry.get("name") or not isinstance(entry["name"], str):
raise ValueError(
f"Invalid worker_pool entry [{wid}]: 'name' must be a non-empty string"
)
endpoint = entry.get("endpoint") or entry.get("type")
if not isinstance(endpoint, str) or endpoint.lower() not in _CONDUCTOR_VALID_ENDPOINTS:
if (
not isinstance(endpoint, str)
or endpoint.lower() not in _CONDUCTOR_VALID_ENDPOINTS
):
raise ValueError(
f"Invalid worker_pool entry [{wid}]: 'endpoint' must be one of "
f"{_CONDUCTOR_VALID_ENDPOINTS} (got {endpoint!r})"
@@ -456,12 +472,18 @@ def _format_worker_pool(workers: List[Dict[str, Any]]) -> str:
)
def _search_capable_indices(workers: List[Dict[str, Any]]) -> List[int]:
def _search_capable_indices(
workers: List[Dict[str, Any]],
*,
search_backend: str = "provider",
) -> List[int]:
"""Indices of workers whose endpoint can run server-side web search."""
if search_backend == "tavily":
return [w["id"] for w in workers]
return [
w["id"] for w in workers
if (w.get("endpoint") or "openai").lower()
in _SEARCH_CAPABLE_WORKER_ENDPOINTS
w["id"]
for w in workers
if (w.get("endpoint") or "openai").lower() in _SEARCH_CAPABLE_WORKER_ENDPOINTS
]
@@ -470,6 +492,7 @@ def _build_conductor_prompt(
workers: List[Dict[str, Any]],
*,
web_search_enabled: bool = False,
search_backend: str = "provider",
) -> str:
"""Build the planner prompt.
@@ -485,12 +508,18 @@ def _build_conductor_prompt(
)
if not web_search_enabled:
return base
capable = _search_capable_indices(workers)
capable = _search_capable_indices(workers, search_backend=search_backend)
if capable:
cap_str = ", ".join(str(i) for i in capable)
if search_backend == "tavily":
capability = (
"External Tavily search results will be prepended to worker prompts"
)
else:
capability = "Only these model indices can perform live web search"
constraint = (
"\n\nWEB SEARCH CONSTRAINT:\n"
f"Only these model indices can perform live web search: [{cap_str}]. "
f"{capability}: [{cap_str}]. "
"Any step that needs to look up facts, current events, or other "
"information not reliably known from memory MUST be routed to one "
"of those indices. Steps routed to any other model can only use "
@@ -550,8 +579,8 @@ def _call_worker(
*,
web_search_tool: Optional[Dict[str, Any]] = None,
web_search_max_uses: int = 8,
) -> Tuple[str, int, int, bool, int]:
"""Returns (text, p_tok, c_tok, is_local, n_web_searches).
) -> Tuple[str, int, int, bool, int, float]:
"""Returns (text, p_tok, c_tok, is_local, n_web_searches, extra_cost).
``web_search_tool``: a truthy marker that web_search is enabled for
this run. When set AND the worker endpoint is search-capable
@@ -565,6 +594,22 @@ def _call_worker(
max_tok = int(cfg.get("worker_max_tokens", 4096))
temp = float(cfg.get("worker_temperature", 0.2))
use_ws = web_search_tool is not None
search_backend = str(cfg.get("search_backend", "provider")).lower()
extra_cost = 0.0
if use_ws and search_backend == "tavily":
res = tavily_search_context(
prompt,
max_results=int(cfg.get("tavily_max_results", 5)),
)
prompt = (
f"Web search results:\n{res['text']}\n\n"
f"Using the search results above, answer this request:\n{prompt}"
)
extra_cost = float(res["cost_usd"])
use_ws = False
tavily_searches = int(res["n_searches"])
else:
tavily_searches = 0
if ep == "vllm":
text, p, c = LocalCloudAgent._call_vllm(
@@ -575,7 +620,7 @@ def _call_worker(
temperature=temp,
enable_thinking=False,
)
return text, p, c, True, 0
return text, p, c, True, tavily_searches, extra_cost
if ep == "openai":
if use_ws:
text, p, c, n_searches, _ = LocalCloudAgent._call_openai_agent(
@@ -584,14 +629,14 @@ def _call_worker(
max_tokens=max_tok,
temperature=(1.0 if is_gpt5_family(worker["model"]) else temp),
)
return text, p, c, False, n_searches
return text, p, c, False, n_searches, 0.0
text, p, c = LocalCloudAgent._call_openai(
worker["model"],
user=prompt,
max_tokens=max_tok,
temperature=(1.0 if is_gpt5_family(worker["model"]) else temp),
)
return text, p, c, False, 0
return text, p, c, False, tavily_searches, extra_cost
if ep == "openrouter":
# OpenRouter is OpenAI-compatible; the helper handles the
# base_url + OPENROUTER_API_KEY plumbing. No server-side web
@@ -607,7 +652,7 @@ def _call_worker(
temperature=temp,
extra_body=extra_body if isinstance(extra_body, dict) else None,
)
return text, p, c, False, 0
return text, p, c, False, tavily_searches, extra_cost
if ep == "anthropic":
eff_temp = temp if supports_temperature(worker["model"]) else 0.0
anthropic_kwargs: Dict[str, Any] = dict(
@@ -620,7 +665,7 @@ def _call_worker(
text, p, c, n_searches = LocalCloudAgent._call_anthropic(
worker["model"], **anthropic_kwargs
)
return text, p, c, False, n_searches
return text, p, c, False, n_searches or tavily_searches, extra_cost
if ep == "gemini":
# Gemini Developer API via google-genai. With web_search on, route
# through the Google-Search-grounded agent loop; otherwise plain
@@ -632,14 +677,14 @@ def _call_worker(
max_tokens=max_tok,
temperature=temp,
)
return text, p, c, False, n_searches
return text, p, c, False, n_searches, 0.0
text, p, c = LocalCloudAgent._call_gemini(
worker["model"],
user=prompt,
max_tokens=max_tok,
temperature=temp,
)
return text, p, c, False, 0
return text, p, c, False, tavily_searches, extra_cost
raise ValueError(f"unsupported worker endpoint: {ep!r}")
@@ -659,12 +704,18 @@ def _swe_worker_step(
ep = (worker.get("endpoint") or "openai").lower()
if ep == "vllm":
backbone, model, endpoint, is_local = (
"local", worker["model"], worker.get("base_url"), True,
"local",
worker["model"],
worker.get("base_url"),
True,
)
cloud_endpoint = "anthropic" # unused on the local path
elif ep == "anthropic":
backbone, model, endpoint, is_local = (
"cloud", worker["model"], None, False,
"cloud",
worker["model"],
None,
False,
)
cloud_endpoint = "anthropic"
else:
@@ -672,7 +723,7 @@ def _swe_worker_step(
# backbones today (the loop's tool-call format is Anthropic- or
# OpenAI-via-vllm-shaped only). Fall back to one-shot for those —
# SWE-bench-wise they were already weak; this preserves behavior.
text, p, c, is_local, n_searches = _call_worker(worker, prompt, cfg)
text, p, c, is_local, n_searches, _extra = _call_worker(worker, prompt, cfg)
return text, p, c, is_local, n_searches, 0
out = run_swe_agent_loop(
task,
@@ -690,7 +741,11 @@ def _swe_worker_step(
)
return (
out["final_summary"] or out["answer"],
out["tokens_in"], out["tokens_out"], is_local, 0, int(out["turns"]),
out["tokens_in"],
out["tokens_out"],
is_local,
0,
int(out["turns"]),
)
@@ -753,13 +808,17 @@ class ConductorAgent(LocalCloudAgent):
and bool(task_meta_early.get("base_commit"))
)
ws_enabled, ws_max_uses = web_search_cfg(cfg)
search_backend = str(cfg.get("search_backend", "provider")).lower()
planner_ws = ws_enabled and not swe_mode_early
# 1. Plan — when web_search is on (GAIA), the prompt names which
# worker indices can actually search, so the planner routes
# research steps to a search-capable worker.
user = _build_conductor_prompt(
question, workers, web_search_enabled=planner_ws,
question,
workers,
web_search_enabled=planner_ws,
search_backend=search_backend,
)
plan_text, p_in, p_out = self._call_cloud(
user=user,
@@ -787,21 +846,22 @@ class ConductorAgent(LocalCloudAgent):
if plan is None:
fallback_used = True
plan = {
"model_id": [len(workers) - 1],
"subtasks": [question],
"model_id": [len(workers) - 1],
"subtasks": [question],
"access_list": [[]],
}
self.record_trace_event({
"kind": "conductor_plan",
"plan": plan,
"fallback_used": fallback_used,
"parse_attempts": parse_attempts,
"workers": [
{k: v for k, v in w.items() if k != "api_key"}
for w in workers
],
})
self.record_trace_event(
{
"kind": "conductor_plan",
"plan": plan,
"fallback_used": fallback_used,
"parse_attempts": parse_attempts,
"workers": [
{k: v for k, v in w.items() if k != "api_key"} for w in workers
],
}
)
# 2. Execute
# If we're on a SWE-bench task AND cfg["swe_use_agent_loop"] is on,
@@ -833,16 +893,17 @@ class ConductorAgent(LocalCloudAgent):
# memory. Fail loud instead of degrading silently.
# ``ws_enabled`` / ``ws_max_uses`` computed up front for the planner
# constraint — reuse them here.
if ws_enabled and not swe_mode:
if ws_enabled and search_backend != "tavily" and not swe_mode:
search_workers = [
w for w in workers
w
for w in workers
if (w.get("endpoint") or "openai").lower()
in _SEARCH_CAPABLE_WORKER_ENDPOINTS
]
if not search_workers:
endpoints = sorted({
(w.get("endpoint") or "openai").lower() for w in workers
})
endpoints = sorted(
{(w.get("endpoint") or "openai").lower() for w in workers}
)
raise ValueError(
f"web_search.enabled=true but the worker pool has no "
f"search-capable worker (endpoints present: {endpoints}); "
@@ -853,39 +914,43 @@ class ConductorAgent(LocalCloudAgent):
)
# ``ws_tool`` doubles as the enable marker passed to `_call_worker`
# (truthy => route search-capable workers through their agent loop).
ws_tool = (
build_web_search_tool(ws_max_uses) if ws_enabled else None
)
ws_tool = build_web_search_tool(ws_max_uses) if ws_enabled else None
try:
if swe_mode:
shared_workdir = Path(tempfile.mkdtemp(
prefix=f"conductor-swe-{task_meta.get('task_id','x')}-"
))
shared_workdir = Path(
tempfile.mkdtemp(
prefix=f"conductor-swe-{task_meta.get('task_id', 'x')}-"
)
)
_clone_repo(task_meta["repo"], task_meta["base_commit"], shared_workdir)
self.record_trace_event({
"kind": "conductor_swe_workdir",
"workdir": str(shared_workdir),
"repo": task_meta["repo"],
"base_commit": task_meta["base_commit"],
})
self.record_trace_event(
{
"kind": "conductor_swe_workdir",
"workdir": str(shared_workdir),
"repo": task_meta["repo"],
"base_commit": task_meta["base_commit"],
}
)
for i, (mid, subtask, access) in enumerate(
zip(plan["model_id"], plan["subtasks"], plan["access_list"])
):
worker = workers[mid]
prompt = _build_step_prompt(question, subtask, steps, access)
self.record_trace_event({
"kind": "conductor_step_dispatch",
"step_idx": i,
"worker_id": mid,
"worker_name": worker["name"],
"worker_model": worker["model"],
"subtask": subtask,
"access": access,
"prompt": prompt,
"swe_mode": swe_mode,
})
self.record_trace_event(
{
"kind": "conductor_step_dispatch",
"step_idx": i,
"worker_id": mid,
"worker_name": worker["name"],
"worker_model": worker["model"],
"subtask": subtask,
"access": access,
"prompt": prompt,
"swe_mode": swe_mode,
}
)
worker_ep = (worker.get("endpoint") or "openai").lower()
# Post-hoc routing check: if web_search is on but the
@@ -894,35 +959,49 @@ class ConductorAgent(LocalCloudAgent):
# may legitimately not need search; see Task-3 planner
# constraint that tries to prevent this upfront).
if (
ws_enabled and not swe_mode
ws_enabled
and search_backend != "tavily"
and not swe_mode
and worker_ep not in _SEARCH_CAPABLE_WORKER_ENDPOINTS
):
self.record_trace_event({
"kind": "conductor_search_routing_warning",
"step_idx": i,
"worker_id": mid,
"worker_name": worker["name"],
"worker_endpoint": worker_ep,
"warning": (
f"web_search enabled but step {i} routed to "
f"search-incapable worker {worker['name']!r} "
f"(endpoint {worker_ep!r}); this step cannot "
"ground and may answer blind."
),
})
self.record_trace_event(
{
"kind": "conductor_search_routing_warning",
"step_idx": i,
"worker_id": mid,
"worker_name": worker["name"],
"worker_endpoint": worker_ep,
"warning": (
f"web_search enabled but step {i} routed to "
f"search-incapable worker {worker['name']!r} "
f"(endpoint {worker_ep!r}); this step cannot "
"ground and may answer blind."
),
}
)
extra_cost = 0.0
if swe_mode:
text, w_in, w_out, is_local, n_searches, bash_turns = (
_swe_worker_step(
worker, task_meta, prompt, cfg, shared_workdir, i,
worker,
task_meta,
prompt,
cfg,
shared_workdir,
i,
)
)
tool_calls += bash_turns
else:
text, w_in, w_out, is_local, n_searches = _call_worker(
worker, prompt, cfg,
web_search_tool=ws_tool,
web_search_max_uses=ws_max_uses,
(text, w_in, w_out, is_local, n_searches, extra_cost) = (
_call_worker(
worker,
prompt,
cfg,
web_search_tool=ws_tool,
web_search_max_uses=ws_max_uses,
)
)
if is_local:
@@ -930,20 +1009,25 @@ class ConductorAgent(LocalCloudAgent):
else:
tokens_cloud += w_in + w_out
cost += self.cost_usd(worker["model"], w_in, w_out)
cost += n_searches * _worker_search_cost_per_call(worker_ep)
if search_backend != "tavily":
cost += n_searches * _worker_search_cost_per_call(worker_ep)
if search_backend == "tavily":
cost += extra_cost
n_web_searches_total += n_searches
tool_calls += n_searches
steps.append({
"step_idx": i,
"model_id": mid,
"worker_name": worker["name"],
"worker_model": worker["model"],
"subtask": subtask,
"access": access,
"output": text,
"tokens_in": w_in,
"tokens_out": w_out,
})
steps.append(
{
"step_idx": i,
"model_id": mid,
"worker_name": worker["name"],
"worker_model": worker["model"],
"subtask": subtask,
"access": access,
"output": text,
"tokens_in": w_in,
"tokens_out": w_out,
}
)
final_answer = text
# For SWE mode, the authoritative patch is whatever lives in
@@ -954,7 +1038,8 @@ class ConductorAgent(LocalCloudAgent):
if patch.strip():
final_answer = (
f"{final_answer}\n\n```diff\n{patch}```"
if final_answer else f"```diff\n{patch}```"
if final_answer
else f"```diff\n{patch}```"
)
finally:
if shared_workdir is not None:
@@ -965,8 +1050,7 @@ class ConductorAgent(LocalCloudAgent):
tokens_cloud += conductor_p_in + conductor_p_out
traces = [
(s["step_idx"], s["model_id"], s["subtask"], s["output"])
for s in steps
(s["step_idx"], s["model_id"], s["subtask"], s["output"]) for s in steps
]
meta = {
@@ -981,11 +1065,11 @@ class ConductorAgent(LocalCloudAgent):
"plan": plan,
"fallback_used": fallback_used,
"web_search_enabled": ws_enabled,
"search_backend": search_backend,
"n_web_searches": n_web_searches_total,
"parse_attempts": parse_attempts,
"workers": [
{k: v for k, v in w.items() if k != "api_key"}
for w in workers
{k: v for k, v in w.items() if k != "api_key"} for w in workers
],
},
}
+409 -257
View File
@@ -172,11 +172,16 @@ def _clone_repo(repo: str, base_commit: str, dest: Path) -> None:
url = f"https://github.com/{repo}.git"
subprocess.run(
["git", "clone", "--quiet", url, str(dest)],
check=True, timeout=300, capture_output=True,
check=True,
timeout=300,
capture_output=True,
)
subprocess.run(
["git", "checkout", "--quiet", base_commit],
cwd=str(dest), check=True, timeout=120, capture_output=True,
cwd=str(dest),
check=True,
timeout=120,
capture_output=True,
)
@@ -265,10 +270,14 @@ def _run_bash(
stderr = _decode_bash_output(stderr_b, exit_code)
truncated = False
if len(stdout) > output_cap:
stdout = stdout[:output_cap] + f"\n…[+{len(stdout) - output_cap} chars truncated]"
stdout = (
stdout[:output_cap] + f"\n…[+{len(stdout) - output_cap} chars truncated]"
)
truncated = True
if len(stderr) > output_cap:
stderr = stderr[:output_cap] + f"\n…[+{len(stderr) - output_cap} chars truncated]"
stderr = (
stderr[:output_cap] + f"\n…[+{len(stderr) - output_cap} chars truncated]"
)
truncated = True
return {
"stdout": stdout,
@@ -297,7 +306,10 @@ def _extract_diff(workdir: Path) -> str:
"""``git diff`` against the base commit — the final SWE-bench patch."""
proc = subprocess.run(
["git", "diff", "--no-color"],
cwd=str(workdir), capture_output=True, text=True, timeout=60,
cwd=str(workdir),
capture_output=True,
text=True,
timeout=60,
)
return proc.stdout
@@ -320,10 +332,11 @@ def _anthropic_assistant_block(block: Any) -> Dict[str, Any]:
# ---------- Reusable agent-loop entry point ----------
def run_swe_agent_loop(
task: Dict[str, Any],
*,
backbone: str, # "cloud" or "local"
backbone: str, # "cloud" or "local"
backbone_model: str,
cloud_endpoint: str = "anthropic",
local_endpoint: Optional[str] = None,
@@ -387,32 +400,33 @@ def run_swe_agent_loop(
own_workdir = workdir is None
if own_workdir:
workdir = Path(tempfile.mkdtemp(
prefix=f"mini-swe-{task.get('task_id','x')}-"
))
workdir = Path(tempfile.mkdtemp(prefix=f"mini-swe-{task.get('task_id', 'x')}-"))
try:
_clone_repo(repo, base_commit, workdir)
except Exception:
shutil.rmtree(workdir, ignore_errors=True)
raise
_record_event({
"kind": f"{trace_prefix}_setup",
"repo": repo,
"base_commit": base_commit,
"workdir": str(workdir),
"owns_workdir": own_workdir,
"backbone": backbone,
"backbone_model": backbone_model,
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_setup",
"repo": repo,
"base_commit": base_commit,
"workdir": str(workdir),
"owns_workdir": own_workdir,
"backbone": backbone,
"backbone_model": backbone_model,
"ts": time.time(),
}
)
user_prompt = initial_prompt or task.get("problem_statement") or ""
try:
if backbone == "cloud":
result = _loop_cloud(
user_prompt, workdir,
user_prompt,
workdir,
model=backbone_model,
cloud_endpoint=cloud_endpoint,
max_turns=max_turns,
@@ -423,9 +437,12 @@ def run_swe_agent_loop(
)
elif backbone == "local":
if not local_endpoint:
raise ValueError("run_swe_agent_loop(backbone='local') needs local_endpoint")
raise ValueError(
"run_swe_agent_loop(backbone='local') needs local_endpoint"
)
result = _loop_local(
user_prompt, workdir,
user_prompt,
workdir,
model=backbone_model,
endpoint=local_endpoint,
max_turns=max_turns,
@@ -440,7 +457,7 @@ def run_swe_agent_loop(
raise ValueError(f"unsupported backbone: {backbone!r}")
patch = _extract_diff(workdir)
framed = (result["final_summary"] or "[mini-swe-agent produced no summary text]")
framed = result["final_summary"] or "[mini-swe-agent produced no summary text]"
if patch.strip():
framed = f"{framed}\n\n```diff\n{patch}```"
@@ -450,11 +467,16 @@ def run_swe_agent_loop(
"final_summary": result["final_summary"],
"tokens_in": result["tokens_in"],
"tokens_out": result["tokens_out"],
"tokens_local": result["tokens_in"] + result["tokens_out"] if backbone == "local" else 0,
"tokens_cloud": result["tokens_in"] + result["tokens_out"] if backbone == "cloud" else 0,
"tokens_local": result["tokens_in"] + result["tokens_out"]
if backbone == "local"
else 0,
"tokens_cloud": result["tokens_in"] + result["tokens_out"]
if backbone == "cloud"
else 0,
"cost_usd": (
estimate_cost(backbone_model, result["tokens_in"], result["tokens_out"])
if backbone == "cloud" else 0.0
if backbone == "cloud"
else 0.0
),
"turns": result["turns"],
"max_turns_hit": result["max_turns_hit"],
@@ -467,6 +489,7 @@ def run_swe_agent_loop(
# ---------- Cloud loop (dispatcher → per-endpoint multi-turn tool loops) ----------
def _loop_cloud(
problem: str,
workdir: Path,
@@ -485,24 +508,36 @@ def _loop_cloud(
to unblock the 8 SWE cells that were stuck on Anthropic-only support."""
if cloud_endpoint == "anthropic":
return _loop_cloud_anthropic(
problem, workdir,
model=model, max_turns=max_turns,
bash_timeout=bash_timeout, output_cap=output_cap,
turn_max_tokens=turn_max_tokens, trace_prefix=trace_prefix,
problem,
workdir,
model=model,
max_turns=max_turns,
bash_timeout=bash_timeout,
output_cap=output_cap,
turn_max_tokens=turn_max_tokens,
trace_prefix=trace_prefix,
)
if cloud_endpoint == "openai":
return _loop_cloud_openai(
problem, workdir,
model=model, max_turns=max_turns,
bash_timeout=bash_timeout, output_cap=output_cap,
turn_max_tokens=turn_max_tokens, trace_prefix=trace_prefix,
problem,
workdir,
model=model,
max_turns=max_turns,
bash_timeout=bash_timeout,
output_cap=output_cap,
turn_max_tokens=turn_max_tokens,
trace_prefix=trace_prefix,
)
if cloud_endpoint == "gemini":
return _loop_cloud_gemini(
problem, workdir,
model=model, max_turns=max_turns,
bash_timeout=bash_timeout, output_cap=output_cap,
turn_max_tokens=turn_max_tokens, trace_prefix=trace_prefix,
problem,
workdir,
model=model,
max_turns=max_turns,
bash_timeout=bash_timeout,
output_cap=output_cap,
turn_max_tokens=turn_max_tokens,
trace_prefix=trace_prefix,
)
raise ValueError(
f"mini-SWE-agent cloud backbone unsupported endpoint: {cloud_endpoint!r}"
@@ -521,6 +556,7 @@ def _loop_cloud_anthropic(
trace_prefix: str,
) -> Dict[str, Any]:
import anthropic
client = anthropic.Anthropic(timeout=600.0, max_retries=5)
messages: List[Dict[str, Any]] = [{"role": "user", "content": problem}]
@@ -553,30 +589,39 @@ def _loop_cloud_anthropic(
btype = getattr(block, "type", None)
if btype == "tool_use":
tool_uses.append((block.id, block.name, dict(block.input or {})))
content_blocks.append({
"type": "tool_use", "id": block.id, "name": block.name,
"input": dict(block.input or {}),
})
content_blocks.append(
{
"type": "tool_use",
"id": block.id,
"name": block.name,
"input": dict(block.input or {}),
}
)
elif hasattr(block, "text"):
text_parts.append(block.text)
content_blocks.append({"type": "text", "text": block.text})
else:
content_blocks.append({"type": btype or "unknown"})
_record_event({
"kind": f"{trace_prefix}_turn",
"turn": turn,
"stop_reason": msg.stop_reason,
"tokens_in": msg.usage.input_tokens,
"tokens_out": msg.usage.output_tokens,
"latency_s": latency,
"content_blocks": content_blocks,
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_turn",
"turn": turn,
"stop_reason": msg.stop_reason,
"tokens_in": msg.usage.input_tokens,
"tokens_out": msg.usage.output_tokens,
"latency_s": latency,
"content_blocks": content_blocks,
"ts": time.time(),
}
)
messages.append({"role": "assistant", "content": [
_anthropic_assistant_block(b) for b in msg.content
]})
messages.append(
{
"role": "assistant",
"content": [_anthropic_assistant_block(b) for b in msg.content],
}
)
if not tool_uses:
final_text = "\n".join(text_parts).strip()
@@ -586,28 +631,40 @@ def _loop_cloud_anthropic(
for tu_id, tu_name, tu_input in tool_uses:
if tu_name != "bash":
obs = f"unknown tool: {tu_name!r}"
_record_event({
"kind": f"{trace_prefix}_unknown_tool",
"turn": turn, "name": tu_name, "input": tu_input,
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_unknown_tool",
"turn": turn,
"name": tu_name,
"input": tu_input,
"ts": time.time(),
}
)
else:
command = str(tu_input.get("command", ""))
result = _run_bash(
command, workdir,
timeout=bash_timeout, output_cap=output_cap,
command,
workdir,
timeout=bash_timeout,
output_cap=output_cap,
)
_record_event(
{
"kind": f"{trace_prefix}_bash",
"turn": turn,
"command": command,
**result,
"ts": time.time(),
}
)
_record_event({
"kind": f"{trace_prefix}_bash",
"turn": turn, "command": command,
**result, "ts": time.time(),
})
obs = _format_observation(result)
tool_result_blocks.append({
"type": "tool_result",
"tool_use_id": tu_id,
"content": obs,
})
tool_result_blocks.append(
{
"type": "tool_result",
"tool_use_id": tu_id,
"content": obs,
}
)
messages.append({"role": "user", "content": tool_result_blocks})
return {
@@ -621,6 +678,7 @@ def _loop_cloud_anthropic(
# ---------- Cloud loop (OpenAI multi-turn with function tools) ----------
def _loop_cloud_openai(
problem: str,
workdir: Path,
@@ -646,6 +704,7 @@ def _loop_cloud_openai(
``_loop_local`` behavior).
"""
from openai import OpenAI
client = OpenAI(timeout=600.0)
messages: List[Dict[str, Any]] = [
@@ -681,21 +740,27 @@ def _loop_cloud_openai(
tool_calls = list(getattr(message, "tool_calls", None) or [])
text = message.content or ""
_record_event({
"kind": f"{trace_prefix}_turn",
"turn": turn,
"endpoint": "openai",
"finish_reason": choice.finish_reason,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"latency_s": latency,
"text": text,
"tool_calls": [
{"id": tc.id, "name": tc.function.name, "arguments": tc.function.arguments}
for tc in tool_calls
],
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_turn",
"turn": turn,
"endpoint": "openai",
"finish_reason": choice.finish_reason,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"latency_s": latency,
"text": text,
"tool_calls": [
{
"id": tc.id,
"name": tc.function.name,
"arguments": tc.function.arguments,
}
for tc in tool_calls
],
"ts": time.time(),
}
)
# Append the assistant turn (including any tool_calls) so the
# follow-up tool messages have the right call ids to reference.
@@ -711,7 +776,8 @@ def _loop_cloud_openai(
if tool_calls:
assistant_msg["tool_calls"] = [
{
"id": tc.id, "type": "function",
"id": tc.id,
"type": "function",
"function": {
"name": tc.function.name,
"arguments": tc.function.arguments,
@@ -734,21 +800,26 @@ def _loop_cloud_openai(
and not text.strip()
and turn < max_turns
):
messages.append({
"role": "user",
"content": (
"Your previous response was truncated by the token limit "
"before producing a tool call or final summary. Retry: "
"either issue ONE bash tool call (short command, no large "
"output) or send a brief one-line final summary with no "
"tool calls to end the loop."
),
})
_record_event({
"kind": f"{trace_prefix}_recover",
"turn": turn, "reason": "length_truncation_no_tool_call",
"ts": time.time(),
})
messages.append(
{
"role": "user",
"content": (
"Your previous response was truncated by the token limit "
"before producing a tool call or final summary. Retry: "
"either issue ONE bash tool call (short command, no large "
"output) or send a brief one-line final summary with no "
"tool calls to end the loop."
),
}
)
_record_event(
{
"kind": f"{trace_prefix}_recover",
"turn": turn,
"reason": "length_truncation_no_tool_call",
"ts": time.time(),
}
)
continue
# No tool call → the model is done. Same termination rule as
# the Anthropic branch.
@@ -762,28 +833,40 @@ def _loop_cloud_openai(
args = {}
if tc.function.name != "bash":
obs = f"unknown tool: {tc.function.name!r}"
_record_event({
"kind": f"{trace_prefix}_unknown_tool",
"turn": turn, "name": tc.function.name, "input": args,
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_unknown_tool",
"turn": turn,
"name": tc.function.name,
"input": args,
"ts": time.time(),
}
)
else:
command = str(args.get("command", ""))
result = _run_bash(
command, workdir,
timeout=bash_timeout, output_cap=output_cap,
command,
workdir,
timeout=bash_timeout,
output_cap=output_cap,
)
_record_event(
{
"kind": f"{trace_prefix}_bash",
"turn": turn,
"command": command,
**result,
"ts": time.time(),
}
)
_record_event({
"kind": f"{trace_prefix}_bash",
"turn": turn, "command": command,
**result, "ts": time.time(),
})
obs = _format_observation(result)
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": obs,
})
messages.append(
{
"role": "tool",
"tool_call_id": tc.id,
"content": obs,
}
)
return {
"tokens_in": tokens_in,
@@ -796,6 +879,7 @@ def _loop_cloud_openai(
# ---------- Cloud loop (Gemini multi-turn with function tools) ----------
def _loop_cloud_gemini(
problem: str,
workdir: Path,
@@ -831,13 +915,15 @@ def _loop_cloud_gemini(
from google.genai import types
client = genai.Client(http_options=types.HttpOptions(timeout=600_000))
bash_tool = types.Tool(function_declarations=[
types.FunctionDeclaration(
name="bash",
description=BASH_TOOL_ANTHROPIC["description"],
parameters=BASH_TOOL_GEMINI_PARAMETERS,
),
])
bash_tool = types.Tool(
function_declarations=[
types.FunctionDeclaration(
name="bash",
description=BASH_TOOL_ANTHROPIC["description"],
parameters=BASH_TOOL_GEMINI_PARAMETERS,
),
]
)
contents: List[types.Content] = [
types.Content(role="user", parts=[types.Part(text=problem)]),
@@ -859,7 +945,9 @@ def _loop_cloud_gemini(
)
t0 = time.time()
resp = client.models.generate_content(
model=model, contents=contents, config=cfg,
model=model,
contents=contents,
config=cfg,
)
_bump_cloud_calls()
latency = time.time() - t0
@@ -896,21 +984,22 @@ def _loop_cloud_gemini(
except Exception:
pass
_record_event({
"kind": f"{trace_prefix}_turn",
"turn": turn,
"endpoint": "gemini",
"finish_reason": finish_reason,
"tokens_in": p,
"tokens_out": c,
"latency_s": latency,
"text": "\n".join(text_parts),
"tool_calls": [
{"name": name, "arguments": args}
for name, args in function_calls
],
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_turn",
"turn": turn,
"endpoint": "gemini",
"finish_reason": finish_reason,
"tokens_in": p,
"tokens_out": c,
"latency_s": latency,
"text": "\n".join(text_parts),
"tool_calls": [
{"name": name, "arguments": args} for name, args in function_calls
],
"ts": time.time(),
}
)
# Append the model's content as-is so the next turn sees its own
# prior function_call parts (Gemini requires this for the
@@ -934,28 +1023,37 @@ def _loop_cloud_gemini(
# treat genuine ``STOP`` with text as a final answer.
fr_str = str(finish_reason or "")
empty_text = not any(t.strip() for t in text_parts)
recoverable = empty_text and turn < max_turns and (
"MALFORMED_FUNCTION_CALL" in fr_str
or "MAX_TOKENS" in fr_str
recoverable = (
empty_text
and turn < max_turns
and ("MALFORMED_FUNCTION_CALL" in fr_str or "MAX_TOKENS" in fr_str)
)
if recoverable:
contents.append(types.Content(
role="user",
parts=[types.Part(text=(
"Your previous response had no parsable function call "
"and no final text (finish_reason="
f"{fr_str}). Retry: either issue ONE well-formed "
"`bash` function call (short command, valid JSON-ish "
"args) or send a brief final text message with no "
"function call to end the loop."
))],
))
_record_event({
"kind": f"{trace_prefix}_recover",
"turn": turn,
"reason": f"empty_response_{fr_str}",
"ts": time.time(),
})
contents.append(
types.Content(
role="user",
parts=[
types.Part(
text=(
"Your previous response had no parsable function call "
"and no final text (finish_reason="
f"{fr_str}). Retry: either issue ONE well-formed "
"`bash` function call (short command, valid JSON-ish "
"args) or send a brief final text message with no "
"function call to end the loop."
)
)
],
)
)
_record_event(
{
"kind": f"{trace_prefix}_recover",
"turn": turn,
"reason": f"empty_response_{fr_str}",
"ts": time.time(),
}
)
continue
final_text = "\n".join(text_parts).strip()
break
@@ -964,26 +1062,39 @@ def _loop_cloud_gemini(
for name, args in function_calls:
if name != "bash":
obs = f"unknown tool: {name!r}"
_record_event({
"kind": f"{trace_prefix}_unknown_tool",
"turn": turn, "name": name, "input": args,
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_unknown_tool",
"turn": turn,
"name": name,
"input": args,
"ts": time.time(),
}
)
else:
command = str(args.get("command", ""))
result = _run_bash(
command, workdir,
timeout=bash_timeout, output_cap=output_cap,
command,
workdir,
timeout=bash_timeout,
output_cap=output_cap,
)
_record_event(
{
"kind": f"{trace_prefix}_bash",
"turn": turn,
"command": command,
**result,
"ts": time.time(),
}
)
_record_event({
"kind": f"{trace_prefix}_bash",
"turn": turn, "command": command,
**result, "ts": time.time(),
})
obs = _format_observation(result)
response_parts.append(types.Part.from_function_response(
name=name, response={"output": obs},
))
response_parts.append(
types.Part.from_function_response(
name=name,
response={"output": obs},
)
)
contents.append(types.Content(role="user", parts=response_parts))
return {
@@ -1013,10 +1124,14 @@ def _get_tiktoken_enc() -> Any:
return _TIKTOKEN_ENC
try:
import tiktoken
_TIKTOKEN_ENC = tiktoken.get_encoding("cl100k_base")
except Exception as exc:
if not _TIKTOKEN_WARNED:
print(f"[mini_swe_agent] tiktoken unavailable ({exc!r}); falling back to len(s)//4", flush=True)
print(
f"[mini_swe_agent] tiktoken unavailable ({exc!r}); falling back to len(s)//4",
flush=True,
)
_TIKTOKEN_WARNED = True
_TIKTOKEN_ENC = False
return _TIKTOKEN_ENC
@@ -1038,7 +1153,7 @@ def _estimate_prompt_tokens(messages: List[Dict[str, Any]]) -> int:
s = "\n".join(parts)
else:
s = ""
for tc in (m.get("tool_calls") or []):
for tc in m.get("tool_calls") or []:
try:
s += "\n" + (tc["function"]["arguments"] or "")
s += "\n" + (tc["function"].get("name") or "")
@@ -1119,7 +1234,7 @@ def _compact_local_messages(
before_tokens = _estimate_prompt_tokens(messages)
new_messages: List[Dict[str, Any]] = list(messages)
n_tool_elided = 0
for (s, e) in old_turns:
for s, e in old_turns:
for k in range(s, e):
m = new_messages[k]
if m.get("role") != "tool":
@@ -1140,17 +1255,19 @@ def _compact_local_messages(
n_tool_elided += 1
after_stage1_tokens = _estimate_prompt_tokens(new_messages)
_record_event({
"kind": f"{trace_prefix}_compact",
"stage": "1",
"msgs_before": len(messages),
"msgs_after": len(new_messages),
"before_tokens": before_tokens,
"after_tokens": after_stage1_tokens,
"n_tool_elided": n_tool_elided,
"n_turns_folded": 0,
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_compact",
"stage": "1",
"msgs_before": len(messages),
"msgs_after": len(new_messages),
"before_tokens": before_tokens,
"after_tokens": after_stage1_tokens,
"n_tool_elided": n_tool_elided,
"n_turns_folded": 0,
"ts": time.time(),
}
)
if after_stage1_tokens <= compact_at_tokens:
return new_messages
@@ -1163,12 +1280,21 @@ def _compact_local_messages(
summary_input = [
{"role": "system", "content": _COMPACT_PROMPT},
{"role": "user", "content": json.dumps(
[{"role": m.get("role"),
"content": m.get("content") if isinstance(m.get("content"), str) else str(m.get("content"))[:4000]}
for m in middle],
default=str,
)[:60_000]},
{
"role": "user",
"content": json.dumps(
[
{
"role": m.get("role"),
"content": m.get("content")
if isinstance(m.get("content"), str)
else str(m.get("content"))[:4000],
}
for m in middle
],
default=str,
)[:60_000],
},
]
summary = ""
try:
@@ -1194,18 +1320,20 @@ def _compact_local_messages(
}
folded = [system_msg, initial_user, synthetic, *tail]
after_stage2_tokens = _estimate_prompt_tokens(folded)
_record_event({
"kind": f"{trace_prefix}_compact",
"stage": "2",
"msgs_before": len(new_messages),
"msgs_after": len(folded),
"before_tokens": after_stage1_tokens,
"after_tokens": after_stage2_tokens,
"n_tool_elided": n_tool_elided,
"n_turns_folded": n_turns_folded,
"summary_chars": len(summary),
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_compact",
"stage": "2",
"msgs_before": len(new_messages),
"msgs_after": len(folded),
"before_tokens": after_stage1_tokens,
"after_tokens": after_stage2_tokens,
"n_tool_elided": n_tool_elided,
"n_turns_folded": n_turns_folded,
"summary_chars": len(summary),
"ts": time.time(),
}
)
return folded
@@ -1231,6 +1359,7 @@ def _loop_local(
# but still saw 28k-input 400s on the n=100 SWE sweep (the keep window
# alone routinely exceeded the budget once bash outputs piled up).
from openai import OpenAI
client = OpenAI(base_url=endpoint, api_key="EMPTY", timeout=600.0)
messages: List[Dict[str, Any]] = [
@@ -1243,10 +1372,16 @@ def _loop_local(
turns = 0
for turn in range(1, max_turns + 1):
turns = turn
if compact_at_tokens > 0 and _estimate_prompt_tokens(messages) > compact_at_tokens:
if (
compact_at_tokens > 0
and _estimate_prompt_tokens(messages) > compact_at_tokens
):
messages = _compact_local_messages(
messages, client=client, model=model,
keep_last=compact_keep_last, trace_prefix=trace_prefix,
messages,
client=client,
model=model,
keep_last=compact_keep_last,
trace_prefix=trace_prefix,
compact_at_tokens=compact_at_tokens,
)
t0 = time.time()
@@ -1268,23 +1403,27 @@ def _loop_local(
# server walled the call. Compact aggressively (keep_last=1)
# and retry once. Re-raise on anything else or on a second
# failure — the runner records the row as errored.
from openjarvis.engine._base import looks_like_context_length_error
msg = str(exc)
is_ctx = (
"maximum context length" in msg
or "context length" in msg.lower() and "exceed" in msg.lower()
)
is_ctx = looks_like_context_length_error(msg)
if not is_ctx:
raise
_record_event({
"kind": f"{trace_prefix}_emergency_compact",
"turn": turn,
"error": msg[:300],
"tokens_before": _estimate_prompt_tokens(messages),
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_emergency_compact",
"turn": turn,
"error": msg[:300],
"tokens_before": _estimate_prompt_tokens(messages),
"ts": time.time(),
}
)
messages = _compact_local_messages(
messages, client=client, model=model,
keep_last=1, trace_prefix=trace_prefix,
messages,
client=client,
model=model,
keep_last=1,
trace_prefix=trace_prefix,
compact_at_tokens=max(8_000, compact_at_tokens // 2),
)
resp = client.chat.completions.create(
@@ -1306,20 +1445,26 @@ def _loop_local(
tool_calls = list(getattr(message, "tool_calls", None) or [])
text = message.content or ""
_record_event({
"kind": f"{trace_prefix}_turn",
"turn": turn,
"finish_reason": choice.finish_reason,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"latency_s": latency,
"text": text,
"tool_calls": [
{"id": tc.id, "name": tc.function.name, "arguments": tc.function.arguments}
for tc in tool_calls
],
"ts": time.time(),
})
_record_event(
{
"kind": f"{trace_prefix}_turn",
"turn": turn,
"finish_reason": choice.finish_reason,
"tokens_in": getattr(u, "prompt_tokens", 0) if u else 0,
"tokens_out": getattr(u, "completion_tokens", 0) if u else 0,
"latency_s": latency,
"text": text,
"tool_calls": [
{
"id": tc.id,
"name": tc.function.name,
"arguments": tc.function.arguments,
}
for tc in tool_calls
],
"ts": time.time(),
}
)
# Match the OpenAI cloud branch: content="" (not None) when only
# tool_calls are present; omit ``tool_calls`` entirely when there
@@ -1333,7 +1478,8 @@ def _loop_local(
if tool_calls:
assistant_local_msg["tool_calls"] = [
{
"id": tc.id, "type": "function",
"id": tc.id,
"type": "function",
"function": {
"name": tc.function.name,
"arguments": tc.function.arguments,
@@ -1357,20 +1503,28 @@ def _loop_local(
else:
command = str(args.get("command", ""))
result = _run_bash(
command, workdir,
timeout=bash_timeout, output_cap=output_cap,
command,
workdir,
timeout=bash_timeout,
output_cap=output_cap,
)
_record_event(
{
"kind": f"{trace_prefix}_bash",
"turn": turn,
"command": command,
**result,
"ts": time.time(),
}
)
_record_event({
"kind": f"{trace_prefix}_bash",
"turn": turn, "command": command,
**result, "ts": time.time(),
})
obs = _format_observation(result)
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": obs,
})
messages.append(
{
"role": "tool",
"tool_call_id": tc.id,
"content": obs,
}
)
return {
"tokens_in": tokens_in,
@@ -1383,6 +1537,7 @@ def _loop_local(
# ---------- Standalone agent ----------
@AgentRegistry.register("mini_swe_agent")
class MiniSWEAgent(LocalCloudAgent):
"""Single-model bash-loop agent for SWE-bench-shaped tasks.
@@ -1410,10 +1565,7 @@ class MiniSWEAgent(LocalCloudAgent):
task = context.metadata.get("task") or {}
backbone = cfg.get("backbone", "cloud")
model = (
self._cloud_model if backbone == "cloud"
else (self._local_model or "")
)
model = self._cloud_model if backbone == "cloud" else (self._local_model or "")
out = run_swe_agent_loop(
task,
+88 -47
View File
@@ -48,12 +48,13 @@ from openjarvis.agents.hybrid._base import (
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
build_web_search_tool,
tavily_search_context,
web_search_cfg,
)
from openjarvis.agents.hybrid._openai_retry import (
patch_openai_globally as _patch_openai_globally,
)
from openjarvis.agents.hybrid._prices import NO_TEMP_PREFIXES
from openjarvis.agents.hybrid._prices import NO_TEMP_PREFIXES, default_max_output_tokens
from openjarvis.agents.hybrid.mini_swe_agent import run_swe_agent_loop
from openjarvis.core.registry import AgentRegistry
@@ -86,7 +87,7 @@ MINIONS_FIRST_TURN_SCHEMA = {
"type": "object",
"properties": {
"reasoning": {"type": "string"},
"message": {"type": "string"},
"message": {"type": "string"},
},
"required": ["reasoning", "message"],
"additionalProperties": False,
@@ -103,7 +104,7 @@ MINIONS_CONVERSATION_SCHEMA = {
"type": "object",
"properties": {
"decision": {"const": "request_additional_info"},
"message": {"type": "string"},
"message": {"type": "string"},
},
"required": ["decision", "message"],
"additionalProperties": False,
@@ -112,7 +113,7 @@ MINIONS_CONVERSATION_SCHEMA = {
"type": "object",
"properties": {
"decision": {"const": "provide_final_answer"},
"answer": {"type": "string"},
"answer": {"type": "string"},
},
"required": ["decision", "answer"],
"additionalProperties": False,
@@ -125,8 +126,8 @@ MINIONS_CONVERSATION_SCHEMA = {
# Markers from Minions's supervisor prompts (prompts/minion.py). Any one
# being present in the call's messages/system is a strong Minions signal.
MINIONS_PROMPT_MARKERS = (
"small language model that has read", # SUPERVISOR_INITIAL_PROMPT
"provide_final_answer", # SUPERVISOR_CONVERSATION_PROMPT
"small language model that has read", # SUPERVISOR_INITIAL_PROMPT
"provide_final_answer", # SUPERVISOR_CONVERSATION_PROMPT
"request_additional_info",
)
@@ -170,6 +171,7 @@ def _stub_missing_imports() -> None:
"""
try:
import mistralai
if not hasattr(mistralai, "Mistral"):
mistralai.Mistral = type("Mistral", (), {}) # type: ignore[attr-defined]
except ImportError:
@@ -214,12 +216,12 @@ def _patch_anthropic_globally() -> None:
model = kwargs.get("model", "")
if model.startswith(NO_TEMP_PREFIXES):
kwargs.pop("temperature", None)
if (
"output_config" not in kwargs
and _looks_like_minions_call(kwargs)
if "output_config" not in kwargs and _looks_like_minions_call(
kwargs
):
kwargs["output_config"] = _minions_turn_schema(kwargs)
return orig(self, **kwargs)
patched._hybrid_patched = True # type: ignore[attr-defined]
return patched
@@ -357,11 +359,14 @@ def _apply_patches_once() -> None:
# ---------- Pre-fetch helper (GAIA only) ----------
def _prefetch_context(
question: str,
cloud_endpoint: str,
cloud_model: str,
max_uses: int = 8,
search_backend: str = "provider",
tavily_max_results: int = 5,
) -> Dict[str, Any]:
"""Use Anthropic web_search to fetch real source material the worker can read.
@@ -373,8 +378,27 @@ def _prefetch_context(
and zeros the protocol still runs.
"""
out: Dict[str, Any] = {
"text": "", "tokens": 0, "cost_usd": 0.0, "n_searches": 0,
"text": "",
"tokens": 0,
"cost_usd": 0.0,
"n_searches": 0,
}
if search_backend == "tavily":
try:
res = tavily_search_context(question, max_results=tavily_max_results)
out.update(
text=res["text"],
cost_usd=float(res["cost_usd"]),
n_searches=int(res["n_searches"]),
tokens=0,
engine=res.get("engine"),
credits=res.get("credits"),
)
if res.get("error"):
out["error"] = res["error"]
except Exception as e:
out["error"] = f"{type(e).__name__}: {e}"
return out
if cloud_endpoint != "anthropic" or not (question or "").strip():
return out
try:
@@ -393,10 +417,12 @@ def _prefetch_context(
tool_choice={"type": "any"},
)
from openjarvis.agents.hybrid._prices import cost as _cost_usd
out.update(
text=text,
tokens=p + c,
cost_usd=_cost_usd(cloud_model, p, c) + n_searches * WEB_SEARCH_COST_PER_CALL,
cost_usd=_cost_usd(cloud_model, p, c)
+ n_searches * WEB_SEARCH_COST_PER_CALL,
n_searches=n_searches,
)
except Exception as e:
@@ -404,9 +430,7 @@ def _prefetch_context(
return out
def _context_for(
task: Optional[Dict[str, Any]], prefetched: str = ""
) -> List[str]:
def _context_for(task: Optional[Dict[str, Any]], prefetched: str = "") -> List[str]:
"""Minions wants a context list."""
bits: List[str] = []
task = task or {}
@@ -421,6 +445,7 @@ def _context_for(
# ---------- Main agent ----------
@AgentRegistry.register("minions")
class MinionsAgent(LocalCloudAgent):
"""HazyResearch Minions supervisor/worker protocol. See module docstring."""
@@ -432,6 +457,7 @@ class MinionsAgent(LocalCloudAgent):
# 400/529, KeyError on missing schema fields.
try:
import anthropic
if isinstance(exc, anthropic.BadRequestError):
return f"{type(exc).__name__}: {str(exc)[:120]}"
except Exception:
@@ -498,18 +524,21 @@ class MinionsAgent(LocalCloudAgent):
max_tokens=cfg.get("worker_max_tokens", 4096),
local=True,
)
cloud_max_tokens = int(
cfg.get("cloud_max_tokens") or default_max_output_tokens(self._cloud_model)
)
if self._cloud_endpoint == "openai":
cloud_client = OpenAIClient(
model_name=self._cloud_model,
temperature=0.0,
max_tokens=4096,
max_tokens=cloud_max_tokens,
)
elif self._cloud_endpoint == "anthropic":
# Temperature stripping is handled by the global patch above for Opus 4.7+.
cloud_client = AnthropicClient(
model_name=self._cloud_model,
temperature=0.0,
max_tokens=4096,
max_tokens=cloud_max_tokens,
)
elif self._cloud_endpoint == "gemini":
# The vendored Minion library already special-cases GeminiClient
@@ -520,7 +549,7 @@ class MinionsAgent(LocalCloudAgent):
cloud_client = GeminiClient(
model_name=self._cloud_model,
temperature=0.0,
max_tokens=4096,
max_tokens=cloud_max_tokens,
)
else:
raise ValueError(f"unsupported cloud endpoint: {self._cloud_endpoint!r}")
@@ -543,9 +572,14 @@ class MinionsAgent(LocalCloudAgent):
# - enabled = false → prefetch OFF
# - enabled = true → prefetch ON (honors max_uses)
prefetch: Dict[str, Any] = {
"text": "", "tokens": 0, "cost_usd": 0.0, "n_searches": 0,
"text": "",
"tokens": 0,
"cost_usd": 0.0,
"n_searches": 0,
}
ws_block = cfg.get("web_search") if isinstance(cfg.get("web_search"), dict) else None
ws_block = (
cfg.get("web_search") if isinstance(cfg.get("web_search"), dict) else None
)
ws_enabled, ws_max_uses = web_search_cfg(cfg)
# If the cell explicitly set web_search.enabled = false, honor that.
# If it set web_search.enabled = true, honor max_uses. If it didn't
@@ -560,17 +594,21 @@ class MinionsAgent(LocalCloudAgent):
self._cloud_endpoint,
self._cloud_model,
max_uses=ws_max_uses,
search_backend=str(cfg.get("search_backend", "provider")).lower(),
tavily_max_results=int(cfg.get("tavily_max_results", 5)),
)
if prefetch.get("text"):
self.record_trace_event({
"kind": "minions_prefetch",
"n_searches": prefetch["n_searches"],
"tokens": prefetch["tokens"],
"cost_usd": prefetch["cost_usd"],
"text": prefetch["text"],
"error": prefetch.get("error"),
})
self.record_trace_event(
{
"kind": "minions_prefetch",
"n_searches": prefetch["n_searches"],
"tokens": prefetch["tokens"],
"cost_usd": prefetch["cost_usd"],
"text": prefetch["text"],
"error": prefetch.get("error"),
}
)
out = protocol(
task=input, # full formatted prompt (with bench instruction)
@@ -582,15 +620,17 @@ class MinionsAgent(LocalCloudAgent):
# The Minions library doesn't go through our SDK helpers, so the
# auto-trace missed every turn. Record the protocol output directly —
# supervisor_messages + worker_messages contain the full conversation.
self.record_trace_event({
"kind": "minions_protocol",
"mode": mode,
"supervisor_messages": out.get("supervisor_messages"),
"worker_messages": out.get("worker_messages"),
"timing": out.get("timing"),
"log_file": out.get("log_file"),
"final_answer": out.get("final_answer", ""),
})
self.record_trace_event(
{
"kind": "minions_protocol",
"mode": mode,
"supervisor_messages": out.get("supervisor_messages"),
"worker_messages": out.get("worker_messages"),
"timing": out.get("timing"),
"log_file": out.get("log_file"),
"final_answer": out.get("final_answer", ""),
}
)
local_usage = out.get("local_usage")
remote_usage = out.get("remote_usage")
@@ -625,7 +665,6 @@ class MinionsAgent(LocalCloudAgent):
}
return out.get("final_answer", ""), meta
# ------------------------------------------------------------------
# SWE-bench variant
# ------------------------------------------------------------------
@@ -643,21 +682,23 @@ class MinionsAgent(LocalCloudAgent):
# 1. Cloud supervisor writes a high-level plan (no tools).
plan_text, p_in, p_out = self._call_cloud(
user=(
f"Issue:\n{task.get('problem_statement','')}\n\n"
f"Repo: {task.get('repo','')}\n"
f"Base commit: {task.get('base_commit','')}\n\n"
f"{task.get('hints_text','')}"
f"Issue:\n{task.get('problem_statement', '')}\n\n"
f"Repo: {task.get('repo', '')}\n"
f"Base commit: {task.get('base_commit', '')}\n\n"
f"{task.get('hints_text', '')}"
),
system=MINIONS_SWE_PLANNER_SYS,
max_tokens=int(cfg.get("supervisor_max_tokens", 1024)),
temperature=0.0,
)
self.record_trace_event({
"kind": "minions_swe_plan",
"plan": plan_text,
"tokens_in": p_in,
"tokens_out": p_out,
})
self.record_trace_event(
{
"kind": "minions_swe_plan",
"plan": plan_text,
"tokens_in": p_in,
"tokens_out": p_out,
}
)
supervisor_cost = self.cost_usd(self._cloud_model, p_in, p_out)
# 2. Local worker runs mini-SWE-agent with the plan as context.
+85 -48
View File
@@ -38,13 +38,14 @@ except ModuleNotFoundError:
from openjarvis.agents._stubs import AgentContext, AgentResult
from openjarvis.agents.hybrid._energy import EnergyCollector
from openjarvis.agents.hybrid._prompts import format_prompt as _format_prompt
from openjarvis.core.paths import get_config_dir
PACKAGE_DIR = Path(__file__).parent
DEFAULT_REGISTRY_DIR = PACKAGE_DIR / "registry"
DEFAULT_EXPERIMENTS_DIR = Path(
os.environ.get(
"OPENJARVIS_HYBRID_EXPERIMENTS_DIR",
Path.home() / ".openjarvis" / "experiments" / "hybrid",
get_config_dir() / "experiments" / "hybrid",
)
)
DEFAULT_SUBSETS_DIR = DEFAULT_EXPERIMENTS_DIR / "subsets"
@@ -122,6 +123,7 @@ def load_registry(registry_dir: Optional[Path] = None) -> Dict[str, Dict[str, An
# ---------- Bench dispatch ----------
def _load_gaia_tasks(n: Optional[int]) -> List[Dict[str, Any]]:
"""GAIA validation. Each task is a dict with `task_id` + `question`."""
from openjarvis.evals.datasets.gaia import GAIADataset
@@ -137,12 +139,14 @@ def _load_gaia_tasks(n: Optional[int]) -> List[Dict[str, Any]]:
# id round-trip.
md = rec.metadata or {}
task_id = md.get("task_id") or rec.record_id
out.append({
"task_id": task_id,
"question": md.get("question", rec.problem),
"reference": rec.reference,
"metadata": dict(md),
})
out.append(
{
"task_id": task_id,
"question": md.get("question", rec.problem),
"reference": rec.reference,
"metadata": dict(md),
}
)
return out
@@ -155,19 +159,21 @@ def _load_swebench_tasks(n: Optional[int]) -> List[Dict[str, Any]]:
out: List[Dict[str, Any]] = []
for rec in ds.iter_records():
md = rec.metadata or {}
out.append({
"task_id": md.get("instance_id", rec.record_id),
"repo": md.get("repo", ""),
"base_commit": md.get("base_commit", ""),
"problem_statement": md.get("problem_statement", rec.problem),
"hints_text": md.get("hints_text", ""),
"test_patch": md.get("test_patch", ""),
"FAIL_TO_PASS": md.get("FAIL_TO_PASS", []),
"PASS_TO_PASS": md.get("PASS_TO_PASS", []),
"version": md.get("version"),
"reference": rec.reference,
"metadata": dict(md),
})
out.append(
{
"task_id": md.get("instance_id", rec.record_id),
"repo": md.get("repo", ""),
"base_commit": md.get("base_commit", ""),
"problem_statement": md.get("problem_statement", rec.problem),
"hints_text": md.get("hints_text", ""),
"test_patch": md.get("test_patch", ""),
"FAIL_TO_PASS": md.get("FAIL_TO_PASS", []),
"PASS_TO_PASS": md.get("PASS_TO_PASS", []),
"version": md.get("version"),
"reference": rec.reference,
"metadata": dict(md),
}
)
return out
@@ -206,7 +212,9 @@ def _load_subset_file(subset_path: str) -> Dict[str, Any]:
f"subset {p.name} has no 'task_ids' field; got keys {list(data.keys())}"
)
return data
raise ValueError(f"subset {p.name} must be a list or dict; got {type(data).__name__}")
raise ValueError(
f"subset {p.name} must be a list or dict; got {type(data).__name__}"
)
def _apply_subset(
@@ -332,7 +340,11 @@ def _score_swebench(
patch = extract_patch(answer)
if patch is None:
return {"success": False, "score": 0.0, "details": {"reason": "no_patch_extracted"}}
return {
"success": False,
"score": 0.0,
"details": {"reason": "no_patch_extracted"},
}
record = EvalRecord(
record_id=task["task_id"],
@@ -368,6 +380,7 @@ def score(
# ---------- Cell run ----------
def _cell_dir(cell_name: str, root: Path) -> Path:
d = root / cell_name
d.mkdir(parents=True, exist_ok=True)
@@ -440,8 +453,10 @@ def _error_row(task: Dict[str, Any], t0: float, error: str) -> Dict[str, Any]:
return {
"task_id": task["task_id"],
"answer": "",
"tokens_local": 0, "tokens_cloud": 0,
"cost_usd": 0.0, "latency_s": time.time() - t0,
"tokens_local": 0,
"tokens_cloud": 0,
"cost_usd": 0.0,
"latency_s": time.time() - t0,
"web_search_uses": 0,
"tool_calls": 0,
"n_cloud_calls": 0,
@@ -456,11 +471,13 @@ def _run_one_inner(
) -> Dict[str, Any]:
"""Run the agent on one task. Returns a hybrid-shape row."""
prompt = _format_prompt(task)
ctx = AgentContext(metadata={
"task": task,
"task_id": task["task_id"],
"log_dir": log_dir,
})
ctx = AgentContext(
metadata={
"task": task,
"task_id": task["task_id"],
"log_dir": log_dir,
}
)
t0 = time.time()
try:
result: AgentResult = agent.run(prompt, ctx)
@@ -534,11 +551,12 @@ def _run_one(
if worker.is_alive():
print(
f"[timeout] task={task['task_id']} exceeded "
f"{task_timeout_s/60:.1f}m — abandoning worker, recording error row",
f"{task_timeout_s / 60:.1f}m — abandoning worker, recording error row",
flush=True,
)
return _error_row(
task, t0,
task,
t0,
f"TaskTimeout: task exceeded the {task_timeout_s:.0f}s hybrid "
"per-task wall-clock cap (likely a hung network or Modal-harness "
"call); worker thread abandoned, task left for resume.",
@@ -558,7 +576,7 @@ def _heartbeat(done: int, total: int, row: Dict[str, Any], t_start: float) -> No
print(
f"[{done}/{total}] {ok} task={row['task_id']} score={sc_str} "
f"local={row['tokens_local']} cloud={row['tokens_cloud']} "
f"${row['cost_usd']:.3f} {row['latency_s']:.1f}s eta={eta/60:.1f}m",
f"${row['cost_usd']:.3f} {row['latency_s']:.1f}s eta={eta / 60:.1f}m",
flush=True,
)
@@ -644,9 +662,9 @@ def _write_summary(
summary_path.write_text(json.dumps(summary, indent=2))
print(
f"[summary] {cell_name}: n={n_done}/{cell['n']} err={n_err} "
f"acc={acc:.3f} cost=${total_cost:.2f} time={wall/60:.1f}m "
f"energy={energy_j/1000:.1f}kJ "
f"(session +{elapsed/60:.1f}m +{energy_j_session/1000:.1f}kJ, "
f"acc={acc:.3f} cost=${total_cost:.2f} time={wall / 60:.1f}m "
f"energy={energy_j / 1000:.1f}kJ "
f"(session +{elapsed / 60:.1f}m +{energy_j_session / 1000:.1f}kJ, "
f"processed={n_processed})",
flush=True,
)
@@ -664,8 +682,11 @@ def run_cell(
out_dir = _cell_dir(cell_name, out_root)
with _cell_lock(out_dir, cell_name):
_run_cell_locked(
cell_name, cell, out_dir,
do_score=do_score, resume=resume,
cell_name,
cell,
out_dir,
do_score=do_score,
resume=resume,
)
@@ -727,7 +748,7 @@ def _run_cell_locked(
mcfg = cell.get("method_cfg") or {}
task_timeout_s = float(mcfg.get("task_timeout_s", DEFAULT_TASK_TIMEOUT_S))
if task_timeout_s > 0:
print(f"[task-timeout] {task_timeout_s/60:.1f}m per task", flush=True)
print(f"[task-timeout] {task_timeout_s / 60:.1f}m per task", flush=True)
agent = _build_agent(cell)
@@ -739,18 +760,25 @@ def _run_cell_locked(
def _process(task: Dict[str, Any]) -> None:
row = _run_one(
agent, cell["bench"], task, log_dir,
agent,
cell["bench"],
task,
log_dir,
task_timeout_s=task_timeout_s,
)
scored: Optional[Dict[str, Any]] = None
if do_score and row.get("error") is None:
try:
scored = score(
cell["bench"], task, row["answer"], cell_name=cell_name,
cell["bench"],
task,
row["answer"],
cell_name=cell_name,
)
except Exception as e:
scored = {
"success": False, "score": 0.0,
"success": False,
"score": 0.0,
"details": {"score_error": str(e)},
}
full_row = {**row, "score": scored}
@@ -789,9 +817,7 @@ def _run_cell_locked(
while not watchdog_stop.wait(60.0):
try:
cur = (
results_path.stat().st_mtime
if results_path.exists()
else last_seen
results_path.stat().st_mtime if results_path.exists() else last_seen
)
except Exception:
cur = last_seen
@@ -830,7 +856,11 @@ def _run_cell_locked(
watchdog_stop.set()
_write_summary(
out_dir, cell_name, cell, tasks, t_start,
out_dir,
cell_name,
cell,
tasks,
t_start,
n_processed=len(pending),
energy_j_session=energy.energy_j_total,
)
@@ -838,6 +868,7 @@ def _run_cell_locked(
# ---------- CLI ----------
def main(argv: Optional[List[str]] = None) -> int:
p = argparse.ArgumentParser(
prog="python -m openjarvis.agents.hybrid.runner",
@@ -855,13 +886,18 @@ def main(argv: Optional[List[str]] = None) -> int:
help="Override experiments output root.",
)
p.add_argument("--no-score", action="store_true", help="Skip scoring.")
p.add_argument("--no-resume", action="store_true", help="Don't resume from results.jsonl.")
p.add_argument(
"--no-resume", action="store_true", help="Don't resume from results.jsonl."
)
args = p.parse_args(argv)
reg_dir = Path(args.registry_dir) if args.registry_dir else None
cells = load_registry(reg_dir)
if not cells:
print(f"[error] no cells found in {reg_dir or DEFAULT_REGISTRY_DIR}", file=sys.stderr)
print(
f"[error] no cells found in {reg_dir or DEFAULT_REGISTRY_DIR}",
file=sys.stderr,
)
return 2
if args.cell not in cells:
print(
@@ -871,7 +907,8 @@ def main(argv: Optional[List[str]] = None) -> int:
return 2
root = Path(args.root) if args.root else None
run_cell(
args.cell, cells[args.cell],
args.cell,
cells[args.cell],
do_score=not args.no_score,
resume=not args.no_resume,
root=root,
+109 -56
View File
@@ -47,33 +47,33 @@ from openjarvis.core.registry import AgentRegistry
# would seed before any oracle update.
SKILL_CATALOG: Dict[str, str] = {
"factual_recall": "Recall named entities, dates, places, well-known facts from training data without external lookup.",
"factual_recall": "Recall named entities, dates, places, well-known facts from training data without external lookup.",
"multi_step_reasoning": "Chain several inference steps together (e.g. compose dates, traverse relationships, decompose then aggregate).",
"arithmetic": "Exact numeric computation on values already given in the question.",
"web_grounding": "Question needs information likely NOT in a small model's parametric memory (rare facts, recent events, niche sources).",
"arithmetic": "Exact numeric computation on values already given in the question.",
"web_grounding": "Question needs information likely NOT in a small model's parametric memory (rare facts, recent events, niche sources).",
"long_text_extraction": "Read a long supplied document/context and extract a specific piece.",
"format_compliance": "Strict output formatting (e.g. GAIA's `FINAL ANSWER: <answer>` rule, comma-separated lists with no units).",
"code_or_logic": "Write or trace code, or apply logical/symbolic constraints precisely.",
"format_compliance": "Strict output formatting (e.g. GAIA's `FINAL ANSWER: <answer>` rule, comma-separated lists with no units).",
"code_or_logic": "Write or trace code, or apply logical/symbolic constraints precisely.",
}
DEFAULT_AGENT_COMPETENCE: Dict[str, Dict[str, float]] = {
"local-qwen-27b": {
"factual_recall": 0.25,
"factual_recall": 0.25,
"multi_step_reasoning": 0.30,
"arithmetic": 0.55,
"web_grounding": 0.10,
"arithmetic": 0.55,
"web_grounding": 0.10,
"long_text_extraction": 0.55,
"format_compliance": 0.65,
"code_or_logic": 0.45,
"format_compliance": 0.65,
"code_or_logic": 0.45,
},
"cloud-opus-4-7": {
"factual_recall": 0.85,
"factual_recall": 0.85,
"multi_step_reasoning": 0.88,
"arithmetic": 0.85,
"web_grounding": 0.70,
"arithmetic": 0.85,
"web_grounding": 0.70,
"long_text_extraction": 0.90,
"format_compliance": 0.92,
"code_or_logic": 0.90,
"format_compliance": 0.92,
"code_or_logic": 0.90,
},
}
@@ -149,6 +149,17 @@ def _build_router_schema(agent_ids: List[str]) -> Dict[str, Any]:
}
def _openai_response_format(schema: Dict[str, Any]) -> Dict[str, Any]:
return {
"type": "json_schema",
"json_schema": {
"name": "skillorchestra_route",
"schema": schema["format"]["schema"],
"strict": True,
},
}
def _parse_router_json(text: str) -> Dict[str, Any]:
s = (text or "").strip()
try:
@@ -178,9 +189,7 @@ def _score_agents(
lam = 0.5
scores: Dict[str, Dict[str, float]] = {}
for aid, comps in competence.items():
comp = sum(
skill_weights.get(sid, 0.0) * comps[sid] for sid in SKILL_CATALOG
)
comp = sum(skill_weights.get(sid, 0.0) * comps[sid] for sid in SKILL_CATALOG)
cost_pen = lam * cost.get(aid, 0.0)
scores[aid] = {
"competence": comp,
@@ -196,6 +205,70 @@ class SkillOrchestraAgent(LocalCloudAgent):
agent_id = "skillorchestra"
def _route_call(
self,
*,
question: str,
router_sys: str,
router_schema: Dict[str, Any],
router_max: int,
) -> Tuple[str, int, int]:
user = f"Question:\n{question}"
if self._cloud_endpoint == "anthropic":
kwargs: Dict[str, Any] = {
"user": user,
"system": router_sys,
"max_tokens": router_max,
"output_config": router_schema,
}
if supports_temperature(self._cloud_model):
kwargs["temperature"] = 0.0
text, r_in, r_out, _ = self._call_anthropic(
self._cloud_model,
**kwargs,
)
return text, r_in, r_out
if self._cloud_endpoint == "openai":
return self._call_openai(
self._cloud_model,
user=user,
system=router_sys,
max_tokens=router_max,
temperature=0.0,
response_format=_openai_response_format(router_schema),
)
if self._cloud_endpoint == "gemini":
return self._call_gemini(
self._cloud_model,
user=user,
system=router_sys,
max_tokens=router_max,
temperature=0.0,
)
raise ValueError(
f"SkillOrchestra router unsupported cloud_endpoint={self._cloud_endpoint!r}"
)
def _executor_call(
self,
*,
question: str,
max_tokens: int,
) -> Tuple[str, int, int]:
if self._cloud_endpoint == "anthropic":
text, w_in, w_out, _ = self._call_anthropic(
self._cloud_model,
user=question,
max_tokens=max_tokens,
temperature=0.0,
)
return text, w_in, w_out
return self._call_cloud(
user=question,
max_tokens=max_tokens,
temperature=0.0,
)
def _is_soft_failure(self, exc: BaseException) -> Optional[str]:
# Empty/unbalanced router JSON — treat as soft failure to match the
# hybrid adapter's behavior (matches `err=1` rows in the n=30 cell).
@@ -220,33 +293,13 @@ class SkillOrchestraAgent(LocalCloudAgent):
router_sys = _build_router_sys(competence, cost)
router_schema = _build_router_schema(agent_ids)
# 1. Route — Anthropic only (output_config schema is Anthropic-specific
# in the hybrid adapter). If you need OpenAI routing, swap the prompt
# to JSON-mode and bypass output_config.
if self._cloud_endpoint != "anthropic":
raise ValueError(
"SkillOrchestra router requires cloud_endpoint='anthropic'; "
f"got {self._cloud_endpoint!r}"
)
router_max = int(cfg.get("router_max_tokens", 1024))
# Strip temperature for Opus 4.7+; Anthropic's output_config does the schema.
if supports_temperature(self._cloud_model):
router_text, r_in, r_out, _ = self._call_anthropic(
self._cloud_model,
user=f"Question:\n{question}",
system=router_sys,
max_tokens=router_max,
temperature=0.0,
output_config=router_schema,
)
else:
router_text, r_in, r_out, _ = self._call_anthropic(
self._cloud_model,
user=f"Question:\n{question}",
system=router_sys,
max_tokens=router_max,
output_config=router_schema,
)
router_text, r_in, r_out = self._route_call(
question=question,
router_sys=router_sys,
router_schema=router_schema,
router_max=router_max,
)
decision = _parse_router_json(router_text)
skill_weights: Dict[str, float] = decision.get("skill_weights") or {}
@@ -258,14 +311,16 @@ class SkillOrchestraAgent(LocalCloudAgent):
if chosen not in competence:
chosen = max(scored, key=lambda a: scored[a]["final_score"])
self.record_trace_event({
"kind": "skillorchestra_route",
"chosen_agent": chosen,
"skill_weights": skill_weights,
"agent_scores": scored,
"reasoning": decision.get("reasoning", ""),
"router_raw": router_text,
})
self.record_trace_event(
{
"kind": "skillorchestra_route",
"chosen_agent": chosen,
"skill_weights": skill_weights,
"agent_scores": scored,
"reasoning": decision.get("reasoning", ""),
"router_raw": router_text,
}
)
tokens_local = 0
tokens_cloud = r_in + r_out
@@ -329,11 +384,9 @@ class SkillOrchestraAgent(LocalCloudAgent):
tokens_cloud += out["tokens_in"] + out["tokens_out"]
run_cost += out["cost_usd"]
else:
ans, w_in, w_out, _ = self._call_anthropic(
self._cloud_model,
user=question,
ans, w_in, w_out = self._executor_call(
question=question,
max_tokens=int(cfg.get("cloud_max_tokens", 4096)),
temperature=0.0,
)
tokens_cloud += w_in + w_out
run_cost += self.cost_usd(self._cloud_model, w_in, w_out)
@@ -41,8 +41,12 @@ from .orchestrator import run_orchestrator
from .stage_router import StageSkillHandbook
_VALID_STRATEGIES = {
"none", "router_decides", "analyze_model_decide",
"weighted_avg", "weakest_skill", "strongest_skill",
"none",
"router_decides",
"analyze_model_decide",
"weighted_avg",
"weakest_skill",
"strongest_skill",
}
@@ -146,7 +150,11 @@ class SkillOrchestraAgent(LocalCloudAgent):
strategy = "none"
return run_orchestrator(
self, input, cfg=cfg, handbook=handbook, strategy=strategy,
self,
input,
cfg=cfg,
handbook=handbook,
strategy=strategy,
)
@@ -31,7 +31,14 @@ from .stage_router import (
get_routing_strategy,
parse_skill_analysis,
)
from .tools import anthropic_tools, openai_tools, run_answer, run_code, run_search
from .tools import (
anthropic_tools,
gemini_tools,
openai_tools,
run_answer,
run_code,
run_search,
)
# tool name -> routing stage (stage_router uses "reasoning" for code).
_TOOL_STAGE = {
@@ -51,6 +58,7 @@ _STAGE_DEFAULT_ALIAS = {
# Orchestrator decision step (raw SDK — needs tool_use blocks back)
# ---------------------------------------------------------------------------
def _orchestrate_step(
agent: Any,
*,
@@ -115,24 +123,63 @@ def _orchestrate_step(
u = resp.usage
p = getattr(u, "prompt_tokens", 0) if u else 0
c = getattr(u, "completion_tokens", 0) if u else 0
elif endpoint == "gemini":
from google import genai
from google.genai import types
client = genai.Client(http_options=types.HttpOptions(timeout=600_000))
cfg = types.GenerateContentConfig(
temperature=1.0,
max_output_tokens=max_tokens,
tools=[types.Tool(function_declarations=gemini_tools())],
)
resp = client.models.generate_content(
model=model,
contents=user,
config=cfg,
)
text = (resp.text or "") if hasattr(resp, "text") else ""
tool_calls = []
try:
parts = resp.candidates[0].content.parts or []
except Exception: # noqa: BLE001
parts = []
for part in parts:
fc = getattr(part, "function_call", None)
if fc is None:
continue
name = getattr(fc, "name", None)
if not isinstance(name, str) or not name:
continue
args = getattr(fc, "args", None) or {}
try:
args = dict(args)
except Exception: # noqa: BLE001
args = {}
tool_calls.append({"name": name, "input": args})
um = getattr(resp, "usage_metadata", None)
p = int(getattr(um, "prompt_token_count", 0) or 0) if um else 0
c = int(getattr(um, "candidates_token_count", 0) or 0) if um else 0
else:
raise ValueError(
f"orchestrator endpoint {endpoint!r} unsupported — route the "
"orchestrator through anthropic/openai (set method_cfg."
"orchestrator through anthropic/openai/gemini (set method_cfg."
"orchestrator_endpoint)."
)
cost = agent.cost_usd(model, p, c)
agent.record_trace_event({
"kind": "skillorchestra_orchestrate",
"model": model,
"endpoint": endpoint,
"prompt": user,
"response": text,
"tool_calls": tool_calls,
"tokens_in": p,
"tokens_out": c,
})
agent.record_trace_event(
{
"kind": "skillorchestra_orchestrate",
"model": model,
"endpoint": endpoint,
"prompt": user,
"response": text,
"tool_calls": tool_calls,
"tokens_in": p,
"tokens_out": c,
}
)
return text, tool_calls, p, c, cost
@@ -140,6 +187,7 @@ def _orchestrate_step(
# Context assembly — eval_frames.py:1305-1351
# ---------------------------------------------------------------------------
def _build_context(
doc_list: List[Tuple[str, str]],
code_list: List[Tuple[str, str]],
@@ -177,6 +225,7 @@ def _build_context(
# Main loop — eval_frames.py:run_single
# ---------------------------------------------------------------------------
def run_orchestrator(
agent: Any,
problem: str,
@@ -192,18 +241,22 @@ def run_orchestrator(
code_timeout = int(cfg.get("code_timeout_s", 60))
answer_max_tokens = int(cfg.get("answer_max_tokens", 40000))
ws_max_uses = int(cfg.get("web_search_max_uses", 5))
search_backend = str(cfg.get("search_backend", "provider")).lower()
tavily_max_results = int(cfg.get("tavily_max_results", 5))
# The orchestrator model: a fixed model per run (the original's
# MODEL_NAME). Defaults to the cell's cloud model when that endpoint
# supports tool calls, else Opus. ``router_model`` / ``router_endpoint``
# are accepted as back-compat aliases (pre-restructure cfg key names).
orch_endpoint = (cfg.get("orchestrator_endpoint")
or cfg.get("router_endpoint")
or agent._cloud_endpoint).lower()
orch_model = (cfg.get("orchestrator_model")
or cfg.get("router_model")
or agent._cloud_model)
if orch_endpoint not in ("anthropic", "openai"):
orch_endpoint = (
cfg.get("orchestrator_endpoint")
or cfg.get("router_endpoint")
or agent._cloud_endpoint
).lower()
orch_model = (
cfg.get("orchestrator_model") or cfg.get("router_model") or agent._cloud_model
)
if orch_endpoint not in ("anthropic", "openai", "gemini"):
orch_endpoint, orch_model = "anthropic", "claude-opus-4-7"
orch_max_tokens = int(cfg.get("orchestrator_max_tokens", 4096))
@@ -233,7 +286,9 @@ def run_orchestrator(
if handbook is not None and strategy != "none":
sa = parse_skill_analysis(orch_text)
rr = get_routing_strategy(strategy, handbook).select_model(
stage, sa, tool_call_model=tool_alias,
stage,
sa,
tool_call_model=tool_alias,
)
return rr.model_alias
return tool_alias or _STAGE_DEFAULT_ALIAS[stage]
@@ -242,7 +297,10 @@ def run_orchestrator(
used_rounds = step + 1
is_last = step == max_rounds - 1
context_str = _build_context(
doc_list, code_list, attempt_list, char_cap=char_cap,
doc_list,
code_list,
attempt_list,
char_cap=char_cap,
)
if handbook is not None and strategy != "none":
@@ -253,14 +311,14 @@ def run_orchestrator(
handbook=handbook,
)
else:
user = (
f"Problem: {problem}\n\n{context_str}\n\n"
"Choose an appropriate tool."
)
user = f"Problem: {problem}\n\n{context_str}\n\nChoose an appropriate tool."
text, tcalls, p, c, ocost = _orchestrate_step(
agent, user=user, model=orch_model,
endpoint=orch_endpoint, max_tokens=orch_max_tokens,
agent,
user=user,
model=orch_model,
endpoint=orch_endpoint,
max_tokens=orch_max_tokens,
)
tokens_cloud += p + c
cost_usd += ocost
@@ -289,23 +347,31 @@ def run_orchestrator(
tool_alias = (tc.get("input") or {}).get("model")
stage = _TOOL_STAGE.get(tool, "answer")
chosen_alias = _route(stage, tool_alias, text)
spec: ModelSpec = pool.get(chosen_alias) or pool[
_STAGE_DEFAULT_ALIAS[stage]
]
route_log.append({
"step": step,
"tool": tool,
"orchestrator_alias": tool_alias,
"routed_alias": chosen_alias,
"routed_model": spec.model,
"is_local": spec.is_local,
})
spec: ModelSpec = (
pool.get(chosen_alias) or pool[_STAGE_DEFAULT_ALIAS[stage]]
)
route_log.append(
{
"step": step,
"tool": tool,
"orchestrator_alias": tool_alias,
"routed_alias": chosen_alias,
"routed_model": spec.model,
"is_local": spec.is_local,
}
)
tool_calls_n += 1
if tool == "search":
res = run_search(
agent, spec, context_str=context_str, problem=problem,
retriever_url=retriever_url, web_search_max_uses=ws_max_uses,
agent,
spec,
context_str=context_str,
problem=problem,
retriever_url=retriever_url,
web_search_max_uses=ws_max_uses,
search_backend=search_backend,
tavily_max_results=tavily_max_results,
)
docs = res["search_results_data"]
joined = "\n---\n".join(d for d in docs if d)[:char_cap]
@@ -313,13 +379,19 @@ def run_orchestrator(
web_uses += res.get("web_search_uses", 0)
elif tool in ("enhance_reasoning", "code"):
res = run_code(
agent, spec, context_str=context_str, problem=problem,
agent,
spec,
context_str=context_str,
problem=problem,
bash_timeout_s=code_timeout,
)
code_list.append((res["generated_code"], res["exec_result"]))
else: # answer
res = run_answer(
agent, spec, context_str=context_str, problem=problem,
agent,
spec,
context_str=context_str,
problem=problem,
max_tokens=answer_max_tokens,
)
final_pred = res["pred"]
@@ -335,12 +407,14 @@ def run_orchestrator(
if finish:
break
agent.record_trace_event({
"kind": "skillorchestra_route_log",
"strategy": strategy,
"rounds_used": used_rounds,
"routes": route_log,
})
agent.record_trace_event(
{
"kind": "skillorchestra_route_log",
"strategy": strategy,
"rounds_used": used_rounds,
"routes": route_log,
}
)
meta = {
"tokens_local": tokens_local,
@@ -32,8 +32,14 @@ from typing import Any, Dict, List, Optional, Tuple
STAGE_ALIASES: Dict[str, List[str]] = {
"search": ["search-1", "search-2", "search-3"],
"reasoning": ["reasoner-1", "reasoner-2", "reasoner-3"],
"answer": ["answer-1", "answer-2", "answer-3", "answer-4",
"answer-math-1", "answer-math-2"],
"answer": [
"answer-1",
"answer-2",
"answer-3",
"answer-4",
"answer-math-1",
"answer-math-2",
],
}
# Every alias the orchestrator can emit, flat.
@@ -42,9 +48,13 @@ ALL_ALIASES: List[str] = [a for aliases in STAGE_ALIASES.values() for a in alias
# Default tier: which aliases collapse onto the cloud model vs the local
# model. Dearer ``-1``/``-2`` (+ answer-math-1) -> cloud; cheaper -> local.
_CLOUD_ALIASES = {
"search-1", "search-2",
"reasoner-1", "reasoner-2",
"answer-1", "answer-2", "answer-math-1",
"search-1",
"search-2",
"reasoner-1",
"reasoner-2",
"answer-1",
"answer-2",
"answer-math-1",
}
@@ -54,8 +64,8 @@ class ModelSpec:
alias: str
model: str
endpoint: str # "anthropic" | "openai" | "gemini" | "http://..."
kind: str # "cloud" | "local"
endpoint: str # "anthropic" | "openai" | "gemini" | "http://..."
kind: str # "cloud" | "local"
@property
def is_local(self) -> bool:
@@ -87,7 +97,10 @@ def build_pool(
pool[alias] = ModelSpec(alias, cloud_model, cloud_endpoint, "cloud")
else:
pool[alias] = ModelSpec(
alias, local_model, local_endpoint, "local" # type: ignore[arg-type]
alias,
local_model,
local_endpoint,
"local", # type: ignore[arg-type]
)
for alias, spec in (overrides or {}).items():
@@ -133,18 +146,30 @@ def call_alias(
ep = spec.endpoint.lower()
if ep == "anthropic":
text, p, c, _ = agent._call_anthropic(
spec.model, user=user, system=system,
max_tokens=max_tokens, temperature=temperature, trace_role="cloud",
spec.model,
user=user,
system=system,
max_tokens=max_tokens,
temperature=temperature,
trace_role="cloud",
)
elif ep == "openai":
text, p, c = agent._call_openai(
spec.model, user=user, system=system,
max_tokens=max_tokens, temperature=temperature, trace_role="cloud",
spec.model,
user=user,
system=system,
max_tokens=max_tokens,
temperature=temperature,
trace_role="cloud",
)
elif ep == "gemini":
text, p, c = agent._call_gemini(
spec.model, user=user, system=system,
max_tokens=max_tokens, temperature=temperature, trace_role="cloud",
spec.model,
user=user,
system=system,
max_tokens=max_tokens,
temperature=temperature,
trace_role="cloud",
)
else:
raise ValueError(f"unsupported pool endpoint: {spec.endpoint!r}")
@@ -94,7 +94,12 @@ class StageSkillHandbook:
"answer": {},
}
self.model_profiles: Dict[str, ModelProfile] = {}
self.usage_patterns: Dict[str, Any] = {"stages": {}, "guidelines": {}, "models": {}, "raw": {}}
self.usage_patterns: Dict[str, Any] = {
"stages": {},
"guidelines": {},
"models": {},
"raw": {},
}
self.routing_insights: List[str] = []
self.learning_history: List[Dict[str, Any]] = []
self.version = "1.0.0"
@@ -102,7 +107,10 @@ class StageSkillHandbook:
self.updated_at = ""
def get_model_skill_scores(self) -> Dict[str, Dict[str, float]]:
return {alias: profile.skill_scores for alias, profile in self.model_profiles.items()}
return {
alias: profile.skill_scores
for alias, profile in self.model_profiles.items()
}
def get_models_for_stage(self, stage: str) -> List[ModelProfile]:
return [p for p in self.model_profiles.values() if p.stage == stage]
@@ -135,19 +143,29 @@ class StageSkillHandbook:
def format_model_performance(self, stage: str) -> str:
profiles = self.get_models_for_stage(stage)
valid_prefixes = {"search": ["search-"], "code": ["reasoner-", "code-"], "answer": ["answer-"]}
valid_prefixes = {
"search": ["search-"],
"code": ["reasoner-", "code-"],
"answer": ["answer-"],
}
prefixes = valid_prefixes.get(stage, [])
lines = []
for p in profiles:
if not any(p.model_alias.startswith(prefix) for prefix in prefixes):
continue
has_data = (p.skill_scores and len(p.skill_scores) > 0) or p.strengths or p.weaknesses
has_data = (
(p.skill_scores and len(p.skill_scores) > 0)
or p.strengths
or p.weaknesses
)
if p.total_attempts > 0 or has_data:
lines.append(f"\n### {p.model_alias} ({p.actual_model})")
if p.total_attempts > 0:
rate = p.total_successes / p.total_attempts
lines.append(f"Overall: {rate:.0%} success ({p.total_successes}/{p.total_attempts})")
lines.append(
f"Overall: {rate:.0%} success ({p.total_successes}/{p.total_attempts})"
)
else:
lines.append("Overall: 0% overall")
if p.skill_scores:
@@ -157,7 +175,9 @@ class StageSkillHandbook:
for sid, s in p.skill_scores.items()
if (sid.split(".")[0] if "." in sid else sid) in ("code", stage)
}
for skill_id, score in sorted(stage_skill_scores.items(), key=lambda x: x[1], reverse=True):
for skill_id, score in sorted(
stage_skill_scores.items(), key=lambda x: x[1], reverse=True
):
lines.append(f" - {skill_id}: {score:.0%}")
if p.strengths:
lines.append(f"Strengths: {', '.join(p.strengths[:3])}")
@@ -220,7 +240,9 @@ def parse_skill_analysis(output: str) -> Optional[SkillAnalysis]:
try:
data = json.loads(match.group(1).strip())
required_skills = [
SkillWeight(skill_id=s.get("skill_id", ""), percentage=float(s.get("percentage", 0)))
SkillWeight(
skill_id=s.get("skill_id", ""), percentage=float(s.get("percentage", 0))
)
for s in data.get("required_skills", [])
]
return SkillAnalysis(
@@ -272,7 +294,14 @@ class RoutingStrategy:
if stage == "reasoning":
return ["reasoner-1", "reasoner-2", "reasoner-3"]
if stage == "answer":
return ["answer-1", "answer-2", "answer-3", "answer-4", "answer-math-1", "answer-math-2"]
return [
"answer-1",
"answer-2",
"answer-3",
"answer-4",
"answer-math-1",
"answer-math-2",
]
return []
def select_model(
@@ -292,9 +321,17 @@ class RouterDecidesStrategy(RoutingStrategy):
tool_call_model: Optional[str] = None,
) -> ModelRoutingResult:
if tool_call_model:
return ModelRoutingResult(tool_call_model, "router_decides_from_tool_call", 1.0)
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "router_decides_fallback", 0.5)
return ModelRoutingResult(
tool_call_model, "router_decides_from_tool_call", 1.0
)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "router_decides_fallback", 0.5
)
class AnalyzeModelDecideStrategy(RoutingStrategy):
@@ -305,17 +342,33 @@ class AnalyzeModelDecideStrategy(RoutingStrategy):
tool_call_model: Optional[str] = None,
) -> ModelRoutingResult:
if tool_call_model:
return ModelRoutingResult(tool_call_model, "analyze_model_decide_with_skill_analysis", 1.0)
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "analyze_model_decide_fallback", 0.5)
return ModelRoutingResult(
tool_call_model, "analyze_model_decide_with_skill_analysis", 1.0
)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "analyze_model_decide_fallback", 0.5
)
class WeightedAverageStrategy(RoutingStrategy):
COST_TIERS = {
"search-3": 1, "search-2": 2, "search-1": 3,
"reasoner-3": 1, "reasoner-2": 2, "reasoner-1": 3,
"answer-math-2": 1, "answer-4": 1, "answer-3": 2,
"answer-math-1": 2, "answer-2": 3, "answer-1": 4,
"search-3": 1,
"search-2": 2,
"search-1": 3,
"reasoner-3": 1,
"reasoner-2": 2,
"reasoner-1": 3,
"answer-math-2": 1,
"answer-4": 1,
"answer-3": 2,
"answer-math-1": 2,
"answer-2": 3,
"answer-1": 4,
}
def select_model(
@@ -326,9 +379,17 @@ class WeightedAverageStrategy(RoutingStrategy):
) -> ModelRoutingResult:
if not skill_analysis or not skill_analysis.required_skills:
if tool_call_model:
return ModelRoutingResult(tool_call_model, "weighted_avg_no_skills_use_tool_call", 0.7)
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "weighted_avg_no_skills_fallback", 0.5)
return ModelRoutingResult(
tool_call_model, "weighted_avg_no_skills_use_tool_call", 0.7
)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "weighted_avg_no_skills_fallback", 0.5
)
models = self._get_models_for_stage(stage)
model_scores = {}
@@ -341,15 +402,25 @@ class WeightedAverageStrategy(RoutingStrategy):
score = scores.get(sid, 0.0)
weighted_sum += weight * score
total_weight += weight
model_scores[model] = weighted_sum / total_weight if total_weight > 0 else 0.5
model_scores[model] = (
weighted_sum / total_weight if total_weight > 0 else 0.5
)
if not model_scores:
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "weighted_avg_no_model_scores", 0.5)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "weighted_avg_no_model_scores", 0.5
)
max_score = max(model_scores.values())
best = [m for m, s in model_scores.items() if abs(s - max_score) < 0.001]
best.sort(key=lambda m: self.COST_TIERS.get(m, 999))
return ModelRoutingResult(best[0], "weighted_avg_from_skill_analysis", max_score, model_scores)
return ModelRoutingResult(
best[0], "weighted_avg_from_skill_analysis", max_score, model_scores
)
class WeakestSkillStrategy(RoutingStrategy):
@@ -361,18 +432,36 @@ class WeakestSkillStrategy(RoutingStrategy):
) -> ModelRoutingResult:
if not skill_analysis or not skill_analysis.required_skills:
if tool_call_model:
return ModelRoutingResult(tool_call_model, "weakest_skill_no_skills_use_tool_call", 0.7)
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "weakest_skill_no_skills_fallback", 0.5)
return ModelRoutingResult(
tool_call_model, "weakest_skill_no_skills_use_tool_call", 0.7
)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "weakest_skill_no_skills_fallback", 0.5
)
weakest = min(skill_analysis.required_skills, key=lambda s: s.percentage)
sid = self._find_skill_id(stage, weakest.skill_id) or weakest.skill_id
models = self._get_models_for_stage(stage)
model_scores = {m: self._model_skill_scores.get(m, {}).get(sid, 0.5) for m in models}
model_scores = {
m: self._model_skill_scores.get(m, {}).get(sid, 0.5) for m in models
}
if not model_scores:
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "weakest_skill_no_model_scores", 0.5)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "weakest_skill_no_model_scores", 0.5
)
best = max(model_scores, key=model_scores.get)
return ModelRoutingResult(best, f"weakest_skill_{weakest.skill_id}", model_scores[best], model_scores)
return ModelRoutingResult(
best, f"weakest_skill_{weakest.skill_id}", model_scores[best], model_scores
)
class StrongestSkillStrategy(RoutingStrategy):
@@ -384,18 +473,41 @@ class StrongestSkillStrategy(RoutingStrategy):
) -> ModelRoutingResult:
if not skill_analysis or not skill_analysis.required_skills:
if tool_call_model:
return ModelRoutingResult(tool_call_model, "strongest_skill_no_skills_use_tool_call", 0.7)
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "strongest_skill_no_skills_fallback", 0.5)
return ModelRoutingResult(
tool_call_model, "strongest_skill_no_skills_use_tool_call", 0.7
)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"),
"strongest_skill_no_skills_fallback",
0.5,
)
strongest = max(skill_analysis.required_skills, key=lambda s: s.percentage)
sid = self._find_skill_id(stage, strongest.skill_id) or strongest.skill_id
models = self._get_models_for_stage(stage)
model_scores = {m: self._model_skill_scores.get(m, {}).get(sid, 0.5) for m in models}
model_scores = {
m: self._model_skill_scores.get(m, {}).get(sid, 0.5) for m in models
}
if not model_scores:
defaults = {"search": "search-1", "reasoning": "reasoner-1", "answer": "answer-1"}
return ModelRoutingResult(defaults.get(stage, "answer-1"), "strongest_skill_no_model_scores", 0.5)
defaults = {
"search": "search-1",
"reasoning": "reasoner-1",
"answer": "answer-1",
}
return ModelRoutingResult(
defaults.get(stage, "answer-1"), "strongest_skill_no_model_scores", 0.5
)
best = max(model_scores, key=model_scores.get)
return ModelRoutingResult(best, f"strongest_skill_{strongest.skill_id}", model_scores[best], model_scores)
return ModelRoutingResult(
best,
f"strongest_skill_{strongest.skill_id}",
model_scores[best],
model_scores,
)
ROUTING_STRATEGIES = {
@@ -27,6 +27,7 @@ from .._base import (
OPENAI_WEB_SEARCH_COST_PER_CALL,
WEB_SEARCH_COST_PER_CALL,
build_web_search_tool,
tavily_search_context,
)
from .pool import ModelSpec, call_alias
@@ -40,8 +41,7 @@ _SEARCH_CAPABLE_ENDPOINTS = ("anthropic", "openai", "gemini")
_SEARCH_DESC = "Search for missing information."
_CODE_DESC = (
"Write and execute Python code to compute intermediate results for "
"the problem."
"Write and execute Python code to compute intermediate results for the problem."
)
_ANSWER_DESC = (
"Extract the final answer when you have gathered enough information "
@@ -51,8 +51,14 @@ _ANSWER_DESC = (
_ENUMS = {
"search": ["search-1", "search-2", "search-3"],
"enhance_reasoning": ["reasoner-1", "reasoner-2", "reasoner-3"],
"answer": ["answer-1", "answer-2", "answer-3", "answer-4",
"answer-math-1", "answer-math-2"],
"answer": [
"answer-1",
"answer-2",
"answer-3",
"answer-4",
"answer-math-1",
"answer-math-2",
],
}
@@ -75,15 +81,17 @@ def anthropic_tools() -> List[Dict[str, Any]]:
("enhance_reasoning", _CODE_DESC),
("answer", _ANSWER_DESC),
):
out.append({
"name": name,
"description": desc,
"input_schema": {
"type": "object",
"properties": {"model": _model_prop(name)},
"required": ["model"],
},
})
out.append(
{
"name": name,
"description": desc,
"input_schema": {
"type": "object",
"properties": {"model": _model_prop(name)},
"required": ["model"],
},
}
)
return out
@@ -95,9 +103,33 @@ def openai_tools() -> List[Dict[str, Any]]:
("enhance_reasoning", _CODE_DESC),
("answer", _ANSWER_DESC),
):
out.append({
"type": "function",
"function": {
out.append(
{
"type": "function",
"function": {
"name": name,
"description": desc,
"parameters": {
"type": "object",
"properties": {"model": _model_prop(name)},
"required": ["model"],
},
},
}
)
return out
def gemini_tools() -> List[Dict[str, Any]]:
"""The 3 orchestrator tools in Gemini function-declaration shape."""
out = []
for name, desc in (
("search", _SEARCH_DESC),
("enhance_reasoning", _CODE_DESC),
("answer", _ANSWER_DESC),
):
out.append(
{
"name": name,
"description": desc,
"parameters": {
@@ -105,8 +137,8 @@ def openai_tools() -> List[Dict[str, Any]]:
"properties": {"model": _model_prop(name)},
"required": ["model"],
},
},
})
}
)
return out
@@ -114,6 +146,7 @@ def openai_tools() -> List[Dict[str, Any]]:
# enhance_reasoning / code — eval_frames.py:659-812
# ---------------------------------------------------------------------------
def run_code(
agent: Any,
spec: ModelSpec,
@@ -129,7 +162,8 @@ def run_code(
rather than raising the orchestrator learns the model can't code.
"""
prompt = (
context_str.strip() + "\n\n"
context_str.strip()
+ "\n\n"
+ f"Question: {problem}\nInstead of directly answering the question, "
"please write additional python code that will give intermidiate "
"results after execution. Wrap the code within ```python and ```. "
@@ -137,7 +171,11 @@ def run_code(
"initialization."
)
text, p, c, cost = call_alias(
agent, spec, user=prompt, max_tokens=8000, temperature=1.0,
agent,
spec,
user=prompt,
max_tokens=8000,
temperature=1.0,
)
generated_code = ""
if "```python" in text:
@@ -176,6 +214,7 @@ def run_code(
# answer — eval_frames.py:814-997
# ---------------------------------------------------------------------------
def run_answer(
agent: Any,
spec: ModelSpec,
@@ -198,11 +237,15 @@ def run_answer(
boxed = False
if "qwen3" in model_l and "235" not in model_l:
system = "Please reason step by step, and put your final answer within \\boxed{}."
system = (
"Please reason step by step, and put your final answer within \\boxed{}."
)
user = base
boxed = True
elif "qwen2.5-math" in model_l or "qwen-2.5-math" in model_l:
system = "Please reason step by step, and put your final answer within \\boxed{}."
system = (
"Please reason step by step, and put your final answer within \\boxed{}."
)
user = base
boxed = True
else:
@@ -215,8 +258,12 @@ def run_answer(
)
text, p, c, cost = call_alias(
agent, spec, user=user, system=system,
max_tokens=max_tokens, temperature=1.0,
agent,
spec,
user=user,
system=system,
max_tokens=max_tokens,
temperature=1.0,
)
pred = ""
@@ -247,6 +294,7 @@ def run_answer(
# search — eval_frames.py:999-1096
# ---------------------------------------------------------------------------
def run_search(
agent: Any,
spec: ModelSpec,
@@ -256,6 +304,8 @@ def run_search(
retriever_url: Optional[str] = None,
topk: int = 150,
web_search_max_uses: int = 5,
search_backend: str = "provider",
tavily_max_results: int = 5,
) -> Dict[str, Any]:
"""Write a search query with ``spec``, then retrieve documents.
@@ -265,13 +315,18 @@ def run_search(
OpenJarvis substitution for the missing FAISS wiki index).
"""
prompt = (
context_str.strip() + "\n\n"
context_str.strip()
+ "\n\n"
+ f"Question: {problem}\nInstead of directly answering the question, "
"please think hard and write a concise query to search Wikipedia. "
"Wrap the query within <query> and </query>."
)
text, p, c, cost = call_alias(
agent, spec, user=prompt, max_tokens=8000, temperature=1.0,
agent,
spec,
user=prompt,
max_tokens=8000,
temperature=1.0,
)
if "<query>" in text:
query = text.split("<query>")[-1].split("</query>")[0].strip()
@@ -283,7 +338,12 @@ def run_search(
contents: List[str] = []
search_uses = 0
if retriever_url:
if search_backend == "tavily":
res = tavily_search_context(query, max_results=tavily_max_results)
contents.append(res["text"])
search_uses = int(res["n_searches"])
cost += float(res["cost_usd"])
elif retriever_url:
# Faithful path — the original FAISS retriever service.
import requests
@@ -294,7 +354,9 @@ def run_search(
}
try:
results = requests.post(
f"{retriever_url.rstrip('/')}/retrieve", json=payload, timeout=120,
f"{retriever_url.rstrip('/')}/retrieve",
json=payload,
timeout=120,
).json()
for r in results[0]:
doc = r.get("document", {})
@@ -20,6 +20,7 @@ from typing import Any, Dict, List, Optional
# BetaCompetence
# ---------------------------------------------------------------------------
@dataclass
class BetaCompetence:
"""Bayesian competence estimate for an agent on a specific skill.
@@ -80,6 +81,7 @@ class BetaCompetence:
# CostStats
# ---------------------------------------------------------------------------
@dataclass
class CostStats:
"""Execution cost statistics for an agent under a specific mode.
@@ -110,11 +112,17 @@ class CostStats:
"""Incremental running-average update."""
n = self.total_executions
self.avg_prompt_tokens = (self.avg_prompt_tokens * n + prompt_tokens) / (n + 1)
self.avg_completion_tokens = (self.avg_completion_tokens * n + completion_tokens) / (n + 1)
self.avg_completion_tokens = (
self.avg_completion_tokens * n + completion_tokens
) / (n + 1)
self.avg_latency_s = (self.avg_latency_s * n + latency_s) / (n + 1)
self.avg_cost_usd = (self.avg_cost_usd * n + cost_usd) / (n + 1)
self.avg_completion_cost_usd = (self.avg_completion_cost_usd * n + completion_cost_usd) / (n + 1)
self.avg_prompt_cost_usd = (self.avg_prompt_cost_usd * n + prompt_cost_usd) / (n + 1)
self.avg_completion_cost_usd = (
self.avg_completion_cost_usd * n + completion_cost_usd
) / (n + 1)
self.avg_prompt_cost_usd = (self.avg_prompt_cost_usd * n + prompt_cost_usd) / (
n + 1
)
self.total_executions = n + 1
def to_dict(self) -> Dict[str, Any]:
@@ -137,6 +145,7 @@ class CostStats:
# RoutingInsight
# ---------------------------------------------------------------------------
@dataclass
class RoutingInsight:
"""A single routing insight learned from execution traces"""
@@ -165,6 +174,7 @@ class RoutingInsight:
# ModeMetadata
# ---------------------------------------------------------------------------
@dataclass
class ModeMetadata:
"""Mode-level routing metadata."""
@@ -197,6 +207,7 @@ class ModeMetadata:
# Skill
# ---------------------------------------------------------------------------
@dataclass
class SkillProvenance:
"""Tracks how and why a skill was discovered."""
@@ -231,7 +242,7 @@ class Skill:
indicators: List[str] = field(default_factory=list)
examples: List[str] = field(default_factory=list)
mode: str = ""
parent_skill_id: Optional[str] = None # for hierarchical skills
parent_skill_id: Optional[str] = None # for hierarchical skills
provenance: SkillProvenance = field(default_factory=SkillProvenance)
def to_dict(self) -> Dict[str, Any]:
@@ -273,6 +284,7 @@ class Skill:
# AgentProfile
# ---------------------------------------------------------------------------
@dataclass
class AgentProfile:
"""Agent profile for skill-aware orchestration."""
@@ -309,9 +321,7 @@ class AgentProfile:
"""Update competence estimate for a skill."""
self.get_competence_dist(skill_id).update(success)
def weighted_competence(
self, skill_weights: Dict[str, float]
) -> float:
def weighted_competence(self, skill_weights: Dict[str, float]) -> float:
"""Compute weighted competence: sum w_{t,sigma} * alpha/(alpha+beta)."""
if not skill_weights:
return 0.5
@@ -334,9 +344,7 @@ class AgentProfile:
]
return sum(scores) / len(scores) if scores else 0.0
def category_competence_for_skills(
self, active_skill_ids: List[str]
) -> float:
def category_competence_for_skills(self, active_skill_ids: List[str]) -> float:
"""Category-level competence for hierarchical tie-breaking.
Extracts parent categories from active_skill_ids (e.g. 'entertainment_knowledge'
@@ -349,7 +357,9 @@ class AgentProfile:
categories.add(cat)
if not categories:
return 0.0
return sum(self.category_competence(cat) for cat in categories) / len(categories)
return sum(self.category_competence(cat) for cat in categories) / len(
categories
)
@property
def overall_success_rate(self) -> float:
@@ -390,8 +400,12 @@ class AgentProfile:
"skill_scores": skill_scores,
"skill_attempts": skill_attempts,
"skill_successes": skill_successes,
"total_attempts": self.total_attempts if self.total_attempts > 0 else skill_total_attempts,
"total_successes": self.total_successes if self.total_attempts > 0 else skill_total_successes,
"total_attempts": self.total_attempts
if self.total_attempts > 0
else skill_total_attempts,
"total_successes": self.total_successes
if self.total_attempts > 0
else skill_total_successes,
"cost_stats": self.cost_stats.to_dict(),
"routing_signals": self.routing_signals,
"strengths": self.strengths,
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More