Compare commits

...
Author SHA1 Message Date
Jon Saad-FalconandClaude Opus 4.8 6dbe5461bb fix(engine): drop Qwen3 control-token tool calls from Ollama responses (#578)
Qwen3 treats /think and /no_think as soft-switch control tokens. On small
models a multi-line prompt makes the model emit one as the sole tool argument
(e.g. {"command": "/no_think"}); OpenJarvis forwards Ollama's native tool_calls
verbatim, so the operative agent executes garbage. Filter control-token-only
tool calls in both the non-streaming generate() and streaming _run_stream()
paths, keeping legitimate calls like {"command": "date"}.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 11:40:25 -07:00
github-actions[bot] a65592fecb chore: update clone traffic data [skip ci] 2026-06-22 08:05:29 +00:00
github-actions[bot] 0513fbdb84 chore: update clone traffic data [skip ci] 2026-06-21 07:41:30 +00:00
Elliot Slusky d4eb6308b1 Fix desktop speech dependency setup (#574) 2026-06-20 17:08:24 -07:00
github-actions[bot] 2853a0001d chore: update clone traffic data [skip ci] 2026-06-20 07:23:14 +00:00
github-actions[bot] 3c99481975 chore: update clone traffic data [skip ci] 2026-06-19 07:55:20 +00:00
Elliot Slusky 4bf39af9bd Add provider-aware search support to hybrid orchestration agents (#558) 2026-06-18 13:40:06 -07:00
github-actions[bot] 0a3e812751 chore: update clone traffic data [skip ci] 2026-06-18 07:44:28 +00:00
github-actions[bot] eb46febad5 chore: update clone traffic data [skip ci] 2026-06-17 07:54:02 +00:00
github-actions[bot] 81482b45d4 chore: update clone traffic data [skip ci] 2026-06-16 08:00:49 +00:00
Jon Saad-FalconandClaude Opus 4.8 3e2f4bcdb4 feat(core): consolidate all state under a single env-aware home directory (#462) (#549)
Previously `core/config.py` defined `DEFAULT_CONFIG_DIR = Path.home() /
".openjarvis"` as a by-value module constant imported into ~45 modules, and 34
modules hardcoded `Path.home() / ".openjarvis"` directly. The installer honored
`OPENJARVIS_HOME` but the Python runtime ignored it, producing a split-brain
layout (some modules honored the override, the core config dir did not). Eval
dataset caches also scattered into `~/.cache/<benchmark>`.

This introduces a single env-aware resolver in `openjarvis/core/paths.py` and
routes every state/config/cache path through it. OpenJarvis now keeps ALL of
its state under ONE root, resolved in priority order:

  1. $OPENJARVIS_HOME
  2. $XDG_DATA_HOME/openjarvis   (single nested dir, when XDG_DATA_HOME is set)
  3. ~/.openjarvis               (default — unchanged, so existing installs are
                                  untouched and no data migration is required)

Implementation:
- New `core/paths.py`: get_config_dir / get_config_path / get_data_dir /
  get_cache_dir, with a source-tree rejection guard (fails loudly per
  REVIEW.md if the root resolves inside the repo).
- `core/config.py`: DEFAULT_CONFIG_DIR / DEFAULT_CONFIG_PATH are now resolved
  via the env-aware resolver at import (real attributes, so existing
  monkeypatch.setattr-based tests keep working). All dataclass field defaults
  that pointed at ~/.openjarvis converted to default_factory so they honor the
  override at instantiation.
- Routed all 34 hardcoders plus several string-literal escapees the original
  audit missed: prompt_loader / description_loader (were OPENJARVIS_HOME-only,
  no XDG), swebench_harness cache, tools/{memory,skill,user_profile}_manage
  defaults, server trace.db fallbacks, doctor_cmd hints.
- spec_search storage/paths now delegates to the unified resolver (gains XDG);
  its ConfigurationError is aliased to the core one.
- Eval dataset caches moved from ~/.cache/<name> to <root>/cache/<name>
  (~/.cache/huggingface left alone — it is HF's own cache).
- Docs + installer comment + `jarvis config path` to show resolved dirs.

Read-only macOS connectors and OS service files (LaunchAgents/systemd) are
intentionally left untouched.

Fixes #462

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 09:32:02 -07:00
github-actions[bot] a35b21195f chore: update clone traffic data [skip ci] 2026-06-15 08:03:30 +00:00
Jon Saad-FalconandClaude Opus 4.8 f9d1bc8c27 fix(connectors): complete Google OAuth and register Drive in Data Sources (fixes #512) (#548)
Pasting a Google Client ID / Secret never completed OAuth: Drive (and its
Google siblings) accepted the credentials, showed no error, opened no browser,
and never appeared in Data Sources. Root cause is three coupled defects, all
reproduced at the unit level against main with a FastAPI TestClient (no Google
creds, network-free):

(A/B) POST /connect routed a `client_id:client_secret` pair into the
  connector's handle_callback, which spawned a daemon thread that popped a
  browser and ran its own localhost:8789 callback server. That thread fails
  silently in the bundled desktop context (`except Exception: pass`), so the
  connector never gained an access_token; /connect returned status "pending"
  and the UI's 20x2s poll timed out with no error.
  Fix: in POST /connect, an OAuth `client_id:client_secret` pair now persists
  the client credentials to every Google credential file and returns an
  `oauth_required` directive pointing at the in-process server flow, instead of
  the silent background thread. The Google connectors' handle_callback no longer
  spawns the browser thread for the pair case — it only persists the creds; the
  server's /oauth/start -> /oauth/callback owns the consent round-trip.

(C) The would-be-correct server flow was itself broken: under
  `from __future__ import annotations` plus a `Request` import local to the
  router factory, FastAPI could not resolve the stringized `request: Request`
  annotation. /oauth/start returned HTTP 422 (request mis-bound as a query
  param) and /oauth/callback injected None -> AttributeError on
  `request.base_url`. Fix: import `Request` at module scope and make the
  callback's `request` a required injected dependency.

A malformed/blank client pair now raises HTTP 400 with the provider setup URL
instead of a perpetual silent "pending" (REVIEW.md silent-failure discipline).

Frontend: DataSourcesPage now opens the server OAuth window when /connect
returns `oauth_required`, then polls until connected; connect errors surface the
backend detail; the Drive setup steps document the "Web application" OAuth
client + server-callback redirect URI the in-process flow requires.

Tests (run on the main venv, hermetic — no ~/.openjarvis pollution):
- test_oauth_flow.py: the three handle_callback tests now assert NO browser is
  opened and only client creds are persisted (was: assert background flow ran).
- test_connectors_router_oauth.py (new): reproduces + fixes all three defects via
  TestClient with mocked token exchange; parametrized over gdrive/gcalendar/
  gcontacts/gmail/google_tasks to prove the shared OAuth path is fixed for every
  sibling and that a single consent writes the access_token to all six Google
  credential files and flips is_connected() to True.
Full tests/connectors suite: 355 passed.

Relationship to PR #510: #510 rewrites all of these files (account-scoped
retrieval) but still carries all three defects. This fix is intentionally scoped
to the OAuth path and does not modify oauth.py, to minimize collision. A
maintainer can either merge this and rebase #510 on top, or port these changes
into #510. See PR body for details.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:55:22 -07:00
Aditya MalikandJon Saad-Falcon dfa908c358 Adopt hatch-vcs dynamic versioning (reworks autotag, pypi-publish, desktop) (#538)
* build: adopt hatch-vcs dynamic versioning (#526)

Replace the static `version = "1.0.2"` with `dynamic = ["version"]` and
derive the version from git tags via hatch-vcs, so source and editable
checkouts report their true git describe version (e.g.
1.0.3.dev110+g<sha>) instead of a stale constant.

Config notes:
- Exclude .dev/.rc/desktop-* tags from derivation. setuptools_scm cannot
  bump custom .devN tags, so the base is taken from the latest plain
  release tag (vX.Y.Z) and the dev distance from commit count.
- Add fallback_version so builds without a git checkout (shallow CI
  clones, Docker COPY src/, source-zip installs) resolve to a sentinel
  instead of hard-failing. CI release builds inject the exact version via
  SETUPTOOLS_SCM_PRETEND_VERSION.

* ci(autotag): derive dev base from the latest release tag (#526)

pyproject no longer carries a static version, so read the base from the
latest plain release tag (vX.Y.Z) reachable from HEAD instead of grepping
pyproject. .dev/.rc/desktop-* tags are excluded so they cannot be mistaken
for the release base. The computed tag (vX.Y.Z.devN) is unchanged.

* ci(pypi-publish): pin build version from tag, drop sed injection (#526)

With dynamic versioning there is no static line to sed. Pin the exact
build version from the pushed tag via SETUPTOOLS_SCM_PRETEND_VERSION so
the published version equals the tag. This is required, not cosmetic: a
naive hatch-vcs build emits 1.0.3.devN+g<sha>, and PyPI rejects local
version segments on upload.

Also add a dry_run input that targets TestPyPI instead of PyPI, for
validating the release path without a production upload.

* ci(desktop): derive dispatch-fallback version from release tag (#526)

The workflow_dispatch fallback grepped the now-removed static pyproject
version. Derive its base from the latest release tag instead (matching
autotag), and give the build-and-release checkout full history and tags
so the derivation works.

---------

Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
2026-06-14 19:35:17 -07:00
Jon Saad-FalconandClaude Opus 4.8 8ef1ab1928 fix(ci): gate secret-bearing Claude workflows by author association (fixes #218) (#547)
The Claude automation workflows (claude-issues.yml, claude-review.yml)
trigger on public, attacker-controllable events (issues, issue_comment,
pull_request_review_comment) and grant the job secrets.ANTHROPIC_API_KEY
plus a write-scoped GITHUB_TOKEN with NO author-association gate.

Because issues / issue_comment / pull_request_review_comment always run in
the base-repo context with full secret access (unlike fork pull_request,
from which GitHub withholds secrets), any external GitHub user could fire
these jobs — draining the API budget and, via contents:write +
pull-requests:write, creating branches/PRs.

Fix:
- Add an author-association gate to every human-triggered, secret-bearing
  if: clause, restricting to OWNER / MEMBER / COLLABORATOR. Uses the correct
  event payload field per trigger: github.event.issue.author_association for
  the `issues` event, github.event.comment.author_association for
  issue_comment and pull_request_review_comment. workflow_dispatch stays
  trusted (requires repo write to invoke).
- Drop unused id-token: write from both workflows (claude-code-action@v1 is
  passed github_token directly, so OIDC is unused).
- Reduce claude-issues.yml timeout-minutes 60 -> 15.

desktop.yml and take-assign.yml are intentionally NOT touched: independently
verified as not exploitable for ANTHROPIC_API_KEY (desktop.yml's only
pull_request job uses no secrets and the trigger is plain pull_request, not
pull_request_target; take-assign.yml uses only GITHUB_TOKEN with issues:write
and no checkout/no Anthropic key). claude-review.yml's stale pull_request
auto-trigger was already removed in 3f2f46e4.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:31:51 -07:00
Jon Saad-FalconandClaude Opus 4.8 28e75cb513 fix(server): inject OpenJarvis identity system prompt on the desktop chat path (fixes #540) (#546)
The OpenAI-compatible POST /v1/chat/completions endpoint — the desktop UI's
chat backend — never injected OpenJarvis's agent.default_system_prompt when the
client omits a system message. The frontend (Chat/InputArea.tsx) posts only
user/assistant turns, so the model answered from its training identity
("I'm Claude", "I am Qwen", ...). The CLI paths ground identity via
SystemPromptBuilder / BaseAgent; the engine-direct server handlers did not.

Fix:
- Add _ensure_identity_prompt(messages, app_config) in server/routes.py: returns
  messages unchanged when any has role==SYSTEM, else prepends a SYSTEM message
  with the resolved identity prompt (app.state.config.agent.default_system_prompt,
  else load_config()), wrapped in try/except that debug-logs on failure (no crash,
  no silent swallow per REVIEW.md).
- Apply it after _to_messages() in all three engine-direct handlers:
  _handle_stream, _handle_stream_tools, and _handle_direct; thread app.state.config
  through. _handle_agent is left untouched (BaseAgent already injects the default).
- Harden AgentConfig.default_system_prompt so distilled models stop claiming to be
  Claude/ChatGPT/Gemini and self-identify as OpenJarvis.

Tests (tests/server/test_routes.py, tests/core/test_config.py): identity prompt IS
prepended when no system message is present (stream / direct / tools paths) and is
NOT duplicated when the client supplies one; config wording anchors "OpenJarvis"
and "not Claude". Verified fail-on-unfixed against main (3 inject tests + config
wording test fail there).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 18:38:05 -07:00
4b9948250b feat(engine): add DeepSeek as a first-class cloud provider + fix #335 over-permissive cloud fallback (#545)
* feat(engine): add DeepSeek as a first-class cloud provider

Adds DEEPSEEK_API_KEY support to the cloud engine, wiring DeepSeek's
OpenAI-compatible API (api.deepseek.com/v1) alongside the existing
MiniMax, OpenRouter, Anthropic, and Google providers.

- Add _DEEPSEEK_MODELS list (deepseek-v4-flash, deepseek-v4-pro)
- Add _is_deepseek_model() routing predicate
- Init self._deepseek_client from DEEPSEEK_API_KEY in _init_clients()
- Add _generate_deepseek() and _stream_deepseek() methods
- Wire DeepSeek into generate(), stream(), _stream_full_openai(),
  list_models(), and health()
- Add approximate pricing entries for both models

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(engine): strict cloud model routing + deepseek can_serve branch

Builds on the DeepSeek provider (PR #504) with two routing-correctness
fixes to CloudEngine._client_for_model:

1. Add the missing DeepSeek branch so can_serve('deepseek-*') agrees with
   list_models()/health() when only DEEPSEEK_API_KEY is set (mirrors the
   minimax branch). Without it the engine advertised deepseek models via
   list_models() but refused to serve them (the #532 can_serve contract).

2. Fix #335: _client_for_model previously fell through to the OpenAI client
   for ANY unrecognized model name, so an OpenAI key (even a dummy
   sk-dummy... one) made can_serve('qwen3.5:0.8b') return True. With the
   local engine transiently down (classic post-Windows-restart Ollama not
   yet up), model-aware get_engine then mis-selected the cloud engine for a
   local model and died with "OpenAI client not available". Add a positive
   _is_openai_model predicate (gpt-/chatgpt-/o1/o3/o4 + _OPENAI_MODELS) and
   return None for unrecognized names, so can_serve declines them. generate()
   and stream() keep their OpenAI fall-through, preserving loud failure for an
   explicitly-requested unknown cloud model.

Tests: DeepSeek detection/pricing/health/list_models/generate-routing/
can_serve and a #335 regression (can_serve rejects local names with an
OpenAI key; unknown model not served even with all clients set; end-to-end
get_engine does not misroute a local model with a dummy OpenAI key).

Fixes #335

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Jen Huls <me@jenhuls.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 18:21:56 -07:00
107 changed files with 3042 additions and 528 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"label": "Git Clones",
"message": "117,047",
"message": "128,077",
"color": "green",
"namedLogo": "git"
}
+11 -3
View File
@@ -1,6 +1,6 @@
{
"total_clones": 117047,
"last_updated": "2026-06-14T07:38:47Z",
"total_clones": 128077,
"last_updated": "2026-06-22T08:05:29Z",
"daily": {
"2026-03-27": 2189,
"2026-03-28": 1874,
@@ -80,6 +80,14 @@
"2026-06-10": 1310,
"2026-06-11": 2564,
"2026-06-12": 1313,
"2026-06-13": 2804
"2026-06-13": 2804,
"2026-06-14": 1543,
"2026-06-15": 1379,
"2026-06-16": 1317,
"2026-06-17": 1170,
"2026-06-18": 1408,
"2026-06-19": 1350,
"2026-06-20": 1437,
"2026-06-21": 1426
}
}
+10 -4
View File
@@ -21,14 +21,20 @@ jobs:
id: version
run: |
set -euo pipefail
# Base version is the next patch above whatever is in pyproject.toml.
# Base version is the next patch above the latest plain release tag
# (vX.Y.Z) reachable from HEAD. pyproject.toml no longer carries a
# static version (#526 switched it to hatch-vcs), so the release tag
# is the source of truth. `.devN`/`.rcN`/`desktop-*` tags are excluded
# so they can't be mistaken for the release base.
# Any future manual `X.Y.Z` release will outrank every `X.Y.Z.devN`
# autotag — PEP 440 sorts dev releases strictly below the final.
BASE=$(grep -E '^version = "' pyproject.toml | head -1 | sed -E 's/^version = "([^"]+)"/\1/')
if [[ -z "$BASE" ]]; then
echo "::error::Could not parse version from pyproject.toml"
LATEST_RELEASE=$(git tag --list 'v[0-9]*' --merged HEAD \
| grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)
if [[ -z "$LATEST_RELEASE" ]]; then
echo "::error::No release tag (vX.Y.Z) reachable from HEAD"
exit 1
fi
BASE="${LATEST_RELEASE#v}"
MAJOR=$(echo "$BASE" | cut -d. -f1)
MINOR=$(echo "$BASE" | cut -d. -f2)
PATCH=$(echo "$BASE" | cut -d. -f3 | sed -E 's/[^0-9].*$//')
+13 -2
View File
@@ -11,22 +11,33 @@ concurrency:
group: claude-issues-${{ github.event.issue.number || github.run_id }}
cancel-in-progress: true
# Least-privilege: only what the issue-fixer job actually needs.
# id-token (OIDC) is intentionally omitted — claude-code-action@v1 is passed
# github_token directly, so OIDC is unused here.
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
jobs:
fix:
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 15
# Security gate: this job reaches secrets.ANTHROPIC_API_KEY and holds a
# write-scoped GITHUB_TOKEN. `issues` / `issue_comment` are public,
# attacker-controllable events that run in the base-repo context with full
# secret access, so the human-triggered paths are restricted to actors with
# write-level association (OWNER / MEMBER / COLLABORATOR). This blocks
# external / first-time contributors from draining the API budget or
# creating branches/PRs, while leaving maintainer use unaffected.
if: |
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issues' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.issue.author_association) &&
(contains(github.event.issue.labels.*.name, 'bug') ||
contains(github.event.issue.labels.*.name, 'autofix'))) ||
(github.event_name == 'issue_comment' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
!github.event.issue.pull_request &&
contains(github.event.comment.body, '@claude') &&
github.actor != 'claude[bot]')
+11 -1
View File
@@ -11,23 +11,33 @@ concurrency:
group: claude-review-${{ github.event.pull_request.number || github.event.issue.number || github.run_id }}
cancel-in-progress: true
# Least-privilege: PR review only needs to post comments on the PR.
# id-token (OIDC) is omitted — claude-code-action@v1 is passed github_token
# directly, so OIDC is unused here.
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
jobs:
review:
runs-on: ubuntu-latest
timeout-minutes: 30
# Security gate: this job reaches secrets.ANTHROPIC_API_KEY. Both
# issue_comment and pull_request_review_comment are public,
# attacker-controllable events that run in the base-repo context with full
# secret access, so the @claude paths are restricted to actors with
# write-level association (OWNER / MEMBER / COLLABORATOR). External /
# first-time contributors cannot trigger the key; maintainers are unaffected.
if: |
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '@claude') &&
github.actor != 'claude[bot]') ||
(github.event_name == 'pull_request_review_comment' &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
contains(github.event.comment.body, '@claude') &&
github.actor != 'claude[bot]')
steps:
+15 -1
View File
@@ -114,6 +114,11 @@ jobs:
steps:
- uses: actions/checkout@v6
with:
# Full history + tags so the workflow_dispatch fallback in
# "Determine release info" can derive the dev version from the
# latest release tag (#526).
fetch-depth: 0
- name: Install system dependencies (Linux)
if: matrix.platform == 'ubuntu-22.04'
@@ -183,7 +188,16 @@ jobs:
# workflow_dispatch fallback (manual UI dispatch without --ref).
# Derive a PEP 440 dev version aligned with autotag.yml so we
# don't burn the X.Y.Z release-version namespace.
BASE=$(grep -E '^version = "' pyproject.toml | head -1 | sed -E 's/^version = "([^"]+)"/\1/')
# pyproject.toml no longer carries a static version (#526), so the
# base comes from the latest plain release tag (vX.Y.Z), matching
# autotag.yml. .dev/.rc/desktop-* tags are excluded.
LATEST_RELEASE=$(git tag --list 'v[0-9]*' --merged HEAD \
| grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)
if [[ -z "$LATEST_RELEASE" ]]; then
echo "::error::No release tag (vX.Y.Z) reachable from HEAD"
exit 1
fi
BASE="${LATEST_RELEASE#v}"
MAJOR=$(echo "$BASE" | cut -d. -f1)
MINOR=$(echo "$BASE" | cut -d. -f2)
PATCH=$(echo "$BASE" | cut -d. -f3 | sed -E 's/[^0-9].*$//')
+30 -11
View File
@@ -12,6 +12,11 @@ on:
description: 'Tag to publish (e.g. v1.0.2.dev500). Overrides github.ref.'
required: false
type: string
dry_run:
description: 'Dry run: build + validate, then publish to TestPyPI instead of PyPI (no production upload).'
required: false
default: false
type: boolean
permissions:
contents: read
@@ -67,27 +72,41 @@ jobs:
exit 1
}
- name: Set version from tag
- name: Resolve build version from tag
env:
REF: ${{ steps.ref.outputs.ref }}
run: |
set -euo pipefail
# Strip leading "v" if present (e.g. v1.0.2.dev500 -> 1.0.2.dev500)
# Strip leading "v" (e.g. v1.0.3.dev825 -> 1.0.3.dev825).
VERSION="${REF#v}"
if [[ -z "$VERSION" ]]; then
echo "::error::Could not resolve version from ref '$REF'"
if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+ ]]; then
echo "::error::ref '$REF' is not a version tag (expected vX.Y.Z[.devN]); pass -f tag=vX.Y.Z"
exit 1
fi
sed -i "s/^version = \".*\"/version = \"${VERSION}\"/" pyproject.toml
# Sanity check the substitution actually took
grep -q "^version = \"${VERSION}\"" pyproject.toml || {
echo "::error::sed failed to update pyproject.toml version"
exit 1
}
echo "Building version $VERSION"
# pyproject.toml is now dynamic = ["version"] via hatch-vcs (#526), so
# there is no static line to sed. setuptools_scm cannot bump custom
# `.devN` tags, so we pin the exact build version explicitly — the
# published version always equals the pushed tag.
echo "SETUPTOOLS_SCM_PRETEND_VERSION=${VERSION}" >> "$GITHUB_ENV"
echo "Building version ${VERSION}"
- name: Build package
run: uv build
- name: Publish to TestPyPI (dry run)
if: ${{ inputs.dry_run }}
env:
UV_PUBLISH_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
run: |
set -euo pipefail
if [[ -z "${UV_PUBLISH_TOKEN:-}" ]]; then
echo "::warning::TEST_PYPI_API_TOKEN is not set — skipping the TestPyPI upload."
echo "Build + twine check passed, which validated version derivation and packaging end to end."
echo "To exercise a real upload, add a TEST_PYPI_API_TOKEN secret (or a TestPyPI trusted publisher)."
exit 0
fi
uv publish --publish-url https://test.pypi.org/legacy/
- name: Publish to PyPI
if: ${{ !inputs.dry_run }}
run: uv publish
+2 -2
View File
@@ -20,7 +20,7 @@ What it does:
4. Installs `uv` (https://astral.sh/uv) if absent.
5. Clones the OpenJarvis repository to `%LOCALAPPDATA%\OpenJarvis`
(override with `$env:OPENJARVIS_HOME`).
6. Runs `uv sync --extra server` so the FastAPI server entry point is
6. Runs `uv sync --extra desktop` so the FastAPI server and speech backend are
importable.
7. Optionally prompts to register a scheduled task that auto-starts the
server at logon.
@@ -105,7 +105,7 @@ To pull the latest:
```powershell
cd "$env:LOCALAPPDATA\OpenJarvis\src"
git pull --ff-only
uv sync --extra server
uv sync --extra desktop
```
Or re-run the installer with `-Force`:
+5 -5
View File
@@ -16,8 +16,8 @@
4. Install uv (https://astral.sh/uv) if absent.
5. Clone the OpenJarvis repository to $env:LOCALAPPDATA\OpenJarvis
(override with $env:OPENJARVIS_HOME).
6. Run `uv sync --extra server` so the FastAPI server entry point
is importable.
6. Run `uv sync --extra desktop` so the FastAPI server and speech
backend are importable.
7. Optionally register the scheduled-task service (see
deploy/windows/jarvis-service.ps1).
@@ -279,13 +279,13 @@ if (Test-Path (Join-Path $srcDir '.git')) {
}
# ---------------------------------------------------------------------------
# 6. uv sync --extra server
# 6. uv sync --extra desktop
# ---------------------------------------------------------------------------
Write-Info "Running 'uv sync --extra server' in $srcDir (this can take a few minutes)..."
Write-Info "Running 'uv sync --extra desktop' in $srcDir (this can take a few minutes)..."
Push-Location $srcDir
try {
& $uvExe sync --extra server
& $uvExe sync --extra desktop
if ($LASTEXITCODE -ne 0) {
Write-Fail "uv sync failed with exit code $LASTEXITCODE. Check the output above."
}
+1 -1
View File
@@ -109,7 +109,7 @@ If you prefer to run each step yourself:
```bash
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync --extra server
uv sync --extra desktop
cd frontend && npm install && cd ..
```
+40
View File
@@ -17,6 +17,46 @@ The configuration file lives at:
OpenJarvis creates the `~/.openjarvis/` directory and populates it with a default config when you run `jarvis init`.
## Relocating the OpenJarvis directory
OpenJarvis keeps **all** of its state — config, databases, caches, logs,
credentials, skills, recipes, connectors — under a **single root** so it never
clutters your home directory beyond one folder. By default that root is
`~/.openjarvis`, but you can move it.
The root is resolved in priority order:
1. **`$OPENJARVIS_HOME`** — explicit override. Honored by both the installer
and the Python runtime.
2. **`$XDG_DATA_HOME/openjarvis`** — used when `$XDG_DATA_HOME` is set (a single
`openjarvis` directory nested under it, per the XDG Base Directory spec).
3. **`~/.openjarvis`** — the default. With no environment variables set, the
resolved path is exactly this, so existing installs are untouched.
```bash
# Relocate the whole install + runtime tree at install time:
OPENJARVIS_HOME=~/apps/openjarvis curl -fsSL https://open-jarvis.github.io/OpenJarvis/install.sh | bash
# Or for a single run / your shell profile:
export OPENJARVIS_HOME=~/apps/openjarvis
```
Confirm where your data lives with:
```bash
jarvis config path
```
!!! note "Migration"
Because the default is unchanged, **no data migration is required** for
existing installs. If you set `OPENJARVIS_HOME` (or `XDG_DATA_HOME`) on a
machine that already has data in `~/.openjarvis`, OpenJarvis will look in
the new location and not see your old data — move it yourself if you want
to keep it: `mv ~/.openjarvis "$OPENJARVIS_HOME"`.
`$OPENJARVIS_CONFIG` still points at an explicit `config.toml` file
independently of the root, if you need to override just the config file path.
## Generating Configuration
### First-Time Setup
+3 -2
View File
@@ -41,7 +41,7 @@ If you prefer to run each step yourself:
```bash
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync --extra server
uv sync --extra desktop
uv run maturin develop -m rust/crates/openjarvis-python/Cargo.toml
cd frontend && npm install && cd ..
```
@@ -278,6 +278,7 @@ OpenJarvis uses optional extras to keep the base installation lightweight.
| Extra | Install Command | Description |
|-------|----------------|-------------|
| `desktop` | `uv sync --extra desktop` | Desktop/API server plus local speech input |
| `server` | `uv sync --extra server` | OpenAI-compatible API server (`jarvis serve`) |
| `dev` | `uv sync --extra dev` | Development and testing tools |
| `docs` | `uv sync --extra docs` | Documentation build tools |
@@ -285,7 +286,7 @@ OpenJarvis uses optional extras to keep the base installation lightweight.
Combine extras:
```bash
uv sync --extra server --extra memory-faiss --extra inference-cloud
uv sync --extra desktop --extra memory-faiss --extra inference-cloud
```
## Setting Up an Inference Backend
+2 -2
View File
@@ -8,7 +8,7 @@ avoid a Linux VM; WSL2 remains the smoother experience for most users.
## What you get
- A PowerShell installer that probes prerequisites, installs `uv`,
clones the repo, and runs `uv sync --extra server`.
clones the repo, and runs `uv sync --extra desktop`.
- An optional Windows scheduled-task service equivalent to the systemd
unit and launchd plist.
- Loopback default — the service binds `127.0.0.1` so no API key is
@@ -38,7 +38,7 @@ The installer will:
4. Install `uv` if absent (via the official `astral.sh/uv` PowerShell
installer).
5. Clone the repo to `%LOCALAPPDATA%\OpenJarvis\src`.
6. Run `uv sync --extra server`.
6. Run `uv sync --extra desktop`.
7. Prompt to register the scheduled-task service (skip with
`-SkipService`).
+24 -6
View File
@@ -682,7 +682,7 @@ fn format_uv_sync_failure(
format!(
"`uv sync` failed in {} (exit {}). Last output:\n\n{}\n\n\
Try opening a terminal in that directory and running \
`uv sync --extra server` manually for the full output.",
`uv sync --extra desktop` manually for the full output.",
root.display(),
code,
uv_sync_stderr_tail(stderr, 800),
@@ -1143,7 +1143,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
sync_cmd
.args([
"sync",
"--extra", "server",
"--extra", "desktop",
"--extra", "inference-cloud",
"--extra", "inference-google",
])
@@ -1664,11 +1664,29 @@ async fn transcribe_audio(
.send()
.await
.map_err(|e| format!("Connection failed: {}", e))?;
let body: serde_json::Value = resp
.json()
let status = resp.status();
let body = resp
.text()
.await
.map_err(|e| format!("Invalid response: {}", e))?;
Ok(body)
if !status.is_success() {
let detail = serde_json::from_str::<serde_json::Value>(&body)
.ok()
.and_then(|value| {
value
.get("detail")
.and_then(|detail| detail.as_str())
.map(str::to_string)
})
.filter(|detail| !detail.is_empty())
.unwrap_or(body);
return Err(format!(
"Transcription failed ({}): {}",
status.as_u16(),
detail
));
}
serde_json::from_str(&body).map_err(|e| format!("Invalid response: {}", e))
}
/// Submit savings to Supabase leaderboard.
@@ -2556,7 +2574,7 @@ mod tests {
assert!(msg.contains("exit 2"));
assert!(msg.contains("/home/u/.openjarvis/src"));
assert!(msg.contains("failed to resolve numpy==2.1.3"));
assert!(msg.contains("uv sync --extra server")); // actionable next step
assert!(msg.contains("uv sync --extra desktop")); // actionable next step
}
#[test]
+13 -1
View File
@@ -97,7 +97,13 @@ export function InputArea() {
const setDeepResearch = useAppStore((s) => s.setDeepResearch);
const corpusSync = useResearchCorpusSync(deepResearch);
const { state: speechState, available: speechAvailable, startRecording, stopRecording } = useSpeech();
const {
state: speechState,
error: speechError,
available: speechAvailable,
startRecording,
stopRecording,
} = useSpeech();
// Abort in-flight stream when the user switches models mid-generation.
// This prevents errors from trying to continue a stream with a stale model.
@@ -122,6 +128,12 @@ export function InputArea() {
: streamState.isStreaming ? 'streaming'
: undefined;
useEffect(() => {
if (speechError) {
toast.error(speechError, { duration: 8000 });
}
}, [speechError]);
const handleMicClick = useCallback(async () => {
if (speechState === 'recording') {
try {
+13 -3
View File
@@ -317,8 +317,9 @@ export async function transcribeAudio(audioBlob: Blob, filename = 'recording.web
audioData: Array.from(new Uint8Array(buffer)),
filename,
});
} catch {
// Fall through to fetch
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
throw new Error(msg || 'Transcription failed');
}
}
const formData = new FormData();
@@ -327,7 +328,16 @@ export async function transcribeAudio(audioBlob: Blob, filename = 'recording.web
method: 'POST',
body: formData,
});
if (!res.ok) throw new Error(`Transcription failed: ${res.status}`);
if (!res.ok) {
let detail = "";
try {
const body = await res.json();
detail = typeof body.detail === 'string' ? body.detail : "";
} catch {
// Keep the status-only message below when the body is not JSON.
}
throw new Error(detail || `Transcription failed: ${res.status}`);
}
return res.json();
}
+34 -3
View File
@@ -1,5 +1,5 @@
import { getBase } from './api';
import type { ConnectorInfo, SyncStatus, ConnectRequest } from '../types/connectors';
import type { ConnectorInfo, SyncStatus, ConnectRequest, ConnectResponse } from '../types/connectors';
// ---------------------------------------------------------------------------
// Connectors API
@@ -18,16 +18,47 @@ export async function getConnector(id: string): Promise<ConnectorInfo> {
return res.json();
}
export async function connectSource(id: string, req: ConnectRequest): Promise<ConnectorInfo> {
export async function connectSource(id: string, req: ConnectRequest): Promise<ConnectResponse> {
const res = await fetch(`${getBase()}/v1/connectors/${encodeURIComponent(id)}/connect`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(req),
});
if (!res.ok) throw new Error(`Failed to connect ${id}: ${res.status}`);
if (!res.ok) {
// Surface the backend's actionable detail (e.g. malformed Client ID /
// Secret) instead of a bare status code so the UI can render it.
const err = await res.json().catch(() => ({ detail: res.statusText }));
throw new Error(err.detail || `Failed to connect ${id}: ${res.status}`);
}
return res.json();
}
/** Open the server-side OAuth consent flow in a popup and resolve once the
* connector reports connected (or reject on timeout). Reused for any OAuth
* connector whose /connect returned `oauth_required` (issue #512). */
export function startServerOAuth(id: string, oauthStartPath?: string): Promise<void> {
const path = oauthStartPath || `/v1/connectors/${encodeURIComponent(id)}/oauth/start`;
window.open(`${getBase()}${path}`, '_blank', 'width=600,height=700');
return new Promise((resolve, reject) => {
const interval = setInterval(async () => {
try {
const info = await getConnector(id);
if (info.connected) {
clearInterval(interval);
clearTimeout(timer);
resolve();
}
} catch {
// ignore transient polling errors
}
}, 2000);
const timer = setTimeout(() => {
clearInterval(interval);
reject(new Error('Authorization timed out — please try again.'));
}, 180000);
});
}
export async function disconnectSource(id: string): Promise<void> {
const res = await fetch(`${getBase()}/v1/connectors/${encodeURIComponent(id)}/disconnect`, {
method: 'POST',
+14 -2
View File
@@ -24,7 +24,7 @@ import {
import type { LucideIcon } from 'lucide-react';
import { SOURCE_CATALOG } from '../types/connectors';
import type { ConnectRequest } from '../types/connectors';
import { listConnectors, connectSource, disconnectSource, getSyncStatus, triggerSync } from '../lib/connectors-api';
import { listConnectors, connectSource, disconnectSource, getSyncStatus, triggerSync, startServerOAuth } from '../lib/connectors-api';
import type { SyncStatus } from '../types/connectors';
// ---------------------------------------------------------------------------
@@ -673,7 +673,19 @@ function DataSourcesSection() {
setConnectStage('Connecting...');
setConnectError('');
try {
await connectSource(id, req);
const resp = await connectSource(id, req);
// OAuth connectors (Google Drive/Calendar/Contacts/Gmail/Tasks): pasting
// a Client ID / Secret only registers the app credentials. The backend
// returns `oauth_required` with the path to the in-process consent flow,
// which is the only path that actually mints an access token. Open it now
// and wait for the callback to flip the connector to connected. Without
// this the connector would stay "pending" forever — the exact #512 bug.
if (resp.status === 'oauth_required') {
setConnectStage('Opening Google sign-in...');
await startServerOAuth(id, resp.oauth_start);
}
setConnectStage('Connected! Starting sync...');
// Wait for connector to show as connected
+1 -1
View File
@@ -417,7 +417,7 @@ function SelfHostedView() {
<p className="text-xs" style={{ color: 'var(--color-text-secondary)' }}>
Launch the API server to get the full UI in your browser:
</p>
<CodeBlock code={"git clone https://github.com/open-jarvis/OpenJarvis.git\ncd OpenJarvis\nuv sync --extra server\njarvis serve --port 8000"} />
<CodeBlock code={"git clone https://github.com/open-jarvis/OpenJarvis.git\ncd OpenJarvis\nuv sync --extra desktop\njarvis serve --port 8000"} />
<p className="text-xs" style={{ color: 'var(--color-text-tertiary)' }}>
The chat, dashboard, energy profiling, and cost comparison all run
locally on your machine.
+15 -2
View File
@@ -55,6 +55,19 @@ export interface ConnectRequest {
password?: string;
}
/** Response from POST /v1/connectors/{id}/connect.
* For OAuth connectors, pasting a Client ID / Secret pair only registers the
* app credentials; the backend returns `status: "oauth_required"` plus an
* `oauth_start` path the UI must open to run the browser consent flow that
* actually mints an access token (see issue #512). */
export interface ConnectResponse {
connector_id: string;
connected: boolean;
status: "connected" | "pending" | "oauth_required" | "disconnected";
oauth_start?: string;
sync_status?: string | null;
}
export type WizardStep = "pick" | "connect" | "ingest" | "ready";
// Backward-compatible alias
@@ -257,12 +270,12 @@ export const SOURCE_CATALOG: ConnectorMeta[] = [
urlLabel: 'Enable Drive API',
},
{
label: 'Create OAuth credentials: go to Credentials (link below) → click "+ Create Credentials" → choose "OAuth client ID" → Application type: "Desktop app" → click "Create"',
label: 'Create OAuth credentials: go to Credentials (link below) → click "+ Create Credentials" → choose "OAuth client ID" → Application type: "Web application". Under "Authorized redirect URIs" add this server\'s callback (e.g. http://localhost:1313/v1/connectors/gdrive/oauth/callback — match the host/port your OpenJarvis server is bound to) → click "Create".',
url: 'https://console.cloud.google.com/apis/credentials',
urlLabel: 'Open Credentials',
},
{
label: 'A dialog will show your Client ID and Client Secret. Copy both and paste them below. (If you miss it, click the download icon next to your OAuth client to see them again)',
label: 'A dialog will show your Client ID and Client Secret. Copy both and paste them below, then click Connect — a Google sign-in window opens to finish authorization. (If you miss the dialog, click the download icon next to your OAuth client to see them again.)',
},
],
inputFields: [
+33 -2
View File
@@ -1,10 +1,10 @@
[build-system]
requires = ["hatchling"]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"
[project]
name = "OpenJarvis"
version = "1.0.2"
dynamic = ["version"]
description = "OpenJarvis — modular AI assistant backend with composable intelligence primitives"
readme = "README.md"
# Upper bound: numpy 2.2.x (pinned transitively via datasets/pandas) ships no
@@ -84,6 +84,13 @@ server = [
"pydantic>=2.0",
"python-multipart>=0.0.9",
]
desktop = [
"fastapi>=0.110",
"uvicorn>=0.30",
"pydantic>=2.0",
"python-multipart>=0.0.9",
"faster-whisper>=1.0",
]
openhands = ["openhands-sdk>=1.0; python_version >= '3.12'"]
gpu-metrics = ["pynvml>=12.0"]
energy-amd = ["amdsmi>=6.1"]
@@ -154,6 +161,30 @@ Issues = "https://github.com/open-jarvis/OpenJarvis/issues"
jarvis = "openjarvis.cli:main"
openjarvis-eval = "openjarvis.evals.cli:main"
# Version is derived from git tags by hatch-vcs (see #526). For source/editable
# checkouts this yields the true `git describe` version (e.g. 1.0.3.dev109+g<sha>)
# rather than a stale static string. CI release builds override this with
# SETUPTOOLS_SCM_PRETEND_VERSION so the published version equals the pushed tag.
#
# setuptools_scm cannot bump custom `.devN` tags (only `.dev0`), so the autotag
# `vX.Y.Z.devN` tags are deliberately EXCLUDED from version derivation here; the
# base is taken from the latest plain release tag (vX.Y.Z) and the dev distance
# is computed from commit count since that release.
[tool.hatch.version]
source = "vcs"
[tool.hatch.version.raw-options]
tag_regex = '^v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$'
git_describe_command = [
"git", "describe", "--dirty", "--tags", "--long",
"--match", "v[0-9]*", "--exclude", "*dev*", "--exclude", "*rc*", "--exclude", "desktop-*",
]
# Builds without a git checkout (e.g. the `COPY src/ src/` Docker stages, which
# never include .git) can't run `git describe`. Without a fallback that would
# hard-fail the build. Mirror the runtime sentinel in src/openjarvis/__init__.py.
# Such builds can inject the real version via SETUPTOOLS_SCM_PRETEND_VERSION.
fallback_version = "0.0.0+unknown"
[tool.hatch.build.targets.wheel]
packages = ["src/openjarvis"]
+8
View File
@@ -210,6 +210,14 @@ if ! command -v python3 >/dev/null 2>&1; then
fi
# ---- env ----
# OpenJarvis keeps ALL of its state (install tree + runtime data, configs,
# databases, caches, logs) under a single root so it never clutters $HOME
# beyond one directory. Relocate it by exporting OPENJARVIS_HOME before
# running the installer, e.g.:
# OPENJARVIS_HOME=~/apps/openjarvis curl ... | bash
# The Python runtime honors the same override (and, when OPENJARVIS_HOME is
# unset, $XDG_DATA_HOME/openjarvis if XDG_DATA_HOME is set). With nothing set
# the root is ~/.openjarvis, so existing installs are untouched.
OPENJARVIS_HOME="${OPENJARVIS_HOME:-$HOME/.openjarvis}"
OPENJARVIS_REPO_URL="${OPENJARVIS_REPO_URL:-https://github.com/open-jarvis/OpenJarvis.git}"
SRC_DIR="$OPENJARVIS_HOME/src"
+1 -1
View File
@@ -148,7 +148,7 @@ fi
# ── 7. Install Python dependencies ──────────────────────────────────
info "Installing Python dependencies..."
uv sync --extra server --quiet 2>/dev/null || uv sync --extra server
uv sync --extra desktop --quiet 2>/dev/null || uv sync --extra desktop
ok "Python dependencies installed"
# ── 7b. Build Rust extension ──────────────────────────────────────
+2 -1
View File
@@ -19,6 +19,7 @@ from typing import Any, List, Optional
from openjarvis.agents._stubs import AgentContext, AgentResult, BaseAgent
from openjarvis.core.events import EventBus
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import ToolResult
from openjarvis.engine._stubs import InferenceEngine
@@ -103,7 +104,7 @@ class ClaudeCodeAgent(BaseAgent):
"Install it from https://nodejs.org/ or via your package manager."
)
dest = Path.home() / ".openjarvis" / "claude_code_runner"
dest = get_config_dir() / "claude_code_runner"
dest.mkdir(parents=True, exist_ok=True)
# Copy runner files if missing or outdated
+3 -1
View File
@@ -9,6 +9,8 @@ from datetime import datetime
from pathlib import Path
from typing import Dict, List, Optional
from openjarvis.core.paths import get_config_dir
@dataclass
class DigestArtifact:
@@ -30,7 +32,7 @@ class DigestStore:
def __init__(self, db_path: str = "") -> None:
if not db_path:
db_path = str(Path.home() / ".openjarvis" / "digest.db")
db_path = str(get_config_dir() / "digest.db")
self._db_path = db_path
self._conn = sqlite3.connect(db_path, check_same_thread=False)
self._conn.execute("PRAGMA journal_mode=WAL")
+51 -6
View File
@@ -73,12 +73,19 @@ WEB_SEARCH_COST_PER_CALL = 0.01
# $0.01/call number — kept as a separate constant so it can drift.
OPENAI_WEB_SEARCH_COST_PER_CALL = 0.01
# Gemini Google-Search grounding: billed at $35 per 1000 grounded
# *requests* (2025-12 public list price for the Grounding-with-Google-Search
# tool, charged once per request that uses the tool regardless of how many
# internal queries it issues). We charge per grounded request, not per
# `web_search_queries` entry.
GEMINI_SEARCH_COST_PER_CALL = 0.035
# Gemini 3 Google-Search grounding: billed at $14 per 1000 search queries.
# `_call_gemini_agent` reports the model's `web_search_queries`, so this is
# charged per query, not per outer generate_content request.
GEMINI_SEARCH_COST_PER_CALL = 0.014
# Tavily Search, advanced depth: 2 API credits per search request at $0.008
# per credit on the public pay-as-you-go plan. WebSearchTool captures actual
# credits when Tavily returns usage metadata; this is the fallback estimate.
TAVILY_SEARCH_COST_PER_CREDIT = 0.008
TAVILY_ADVANCED_SEARCH_CREDITS = 2
TAVILY_SEARCH_COST_PER_CALL = (
TAVILY_SEARCH_COST_PER_CREDIT * TAVILY_ADVANCED_SEARCH_CREDITS
)
ANTHROPIC_WEB_SEARCH_TOOL = {
"type": "web_search_20250305",
@@ -101,6 +108,40 @@ def build_web_search_tool(max_uses: int = 8) -> Dict[str, Any]:
}
def tavily_search_context(
query: str,
*,
max_results: int = 5,
) -> Dict[str, Any]:
"""Run OpenJarvis WebSearchTool and return accounting-friendly metadata."""
from openjarvis.tools.web_search import WebSearchTool
tool = WebSearchTool(max_results=max_results)
res = tool.execute(query=query, max_results=max_results)
meta = dict(res.metadata or {})
engine = str(meta.get("engine") or "unknown")
credits = 0
cost_usd = 0.0
if engine == "tavily":
try:
credits = int(meta.get("credits") or TAVILY_ADVANCED_SEARCH_CREDITS)
except (TypeError, ValueError):
credits = TAVILY_ADVANCED_SEARCH_CREDITS
cost_usd = credits * TAVILY_SEARCH_COST_PER_CREDIT
text = res.content or ""
if not res.success and not text:
text = "(no search results)"
return {
"text": text,
"success": bool(res.success),
"engine": engine,
"credits": credits,
"cost_usd": cost_usd,
"n_searches": 1 if (query or "").strip() else 0,
"error": None if res.success else text,
}
def web_search_cfg(method_cfg: Optional[Dict[str, Any]]) -> Tuple[bool, int]:
"""Parse ``method_cfg.web_search = { enabled, max_uses }``.
@@ -1322,6 +1363,9 @@ __all__ = [
"LocalCloudAgent",
"NO_TEMP_PREFIXES",
"OPENAI_WEB_SEARCH_COST_PER_CALL",
"TAVILY_ADVANCED_SEARCH_CREDITS",
"TAVILY_SEARCH_COST_PER_CALL",
"TAVILY_SEARCH_COST_PER_CREDIT",
"WEB_SEARCH_COST_PER_CALL",
"_bump_cloud_calls",
"_bump_local_calls",
@@ -1329,5 +1373,6 @@ __all__ = [
"estimate_cost",
"is_gpt5_family",
"supports_temperature",
"tavily_search_context",
"web_search_cfg",
]
+11 -4
View File
@@ -15,13 +15,16 @@ PRICES: dict[str, tuple[float, float]] = {
"claude-sonnet-4-6": (3.00, 15.0),
"claude-haiku-4-5": (1.00, 5.00),
"claude-haiku-4-5-20251001": (1.00, 5.00),
"gpt-5.5": (5.00, 30.0),
"gpt-5": (1.25, 10.0),
"gpt-5-mini": (0.25, 2.00),
"gpt-5-mini-2025-08-07": (0.25, 2.00),
"gpt-4o": (0.15, 0.60),
# Gemini Developer API prices (USD per 1M tokens), 2025-12 list price.
# 2.5 Pro uses tiered pricing (>200K context = $2.50/$15); we charge the
# low-context tier since GAIA / SWE-bench prompts stay well under 200K.
# Gemini Developer API prices (USD per 1M tokens). Pro models use tiered
# pricing above 200K prompt tokens; GAIA prompts stay under that tier, so
# charge the low-context standard rate.
"gemini-3.1-pro-preview": (2.00, 12.0),
"gemini-3.1-pro-preview-customtools": (2.00, 12.0),
"gemini-2.5-pro": (1.25, 10.0),
"gemini-2.5-flash": (0.30, 2.50),
"gemini-2.5-flash-lite": (0.10, 0.40),
@@ -61,7 +64,11 @@ def is_reasoning_model(model: str) -> bool:
before emitting visible answer text. At max_tokens=4096 these silently
truncate with empty answers on GAIA (26/100 GPT-5, 18/100 Gemini Pro)."""
m = (model or "").lower()
return is_gpt5_family(model) or "gemini-2.5-pro" in m
return (
is_gpt5_family(model)
or "gemini-2.5-pro" in m
or "gemini-3.1-pro" in m
)
def default_max_output_tokens(model: str) -> int:
+56 -11
View File
@@ -34,6 +34,7 @@ from openjarvis.agents.hybrid._base import (
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
build_web_search_tool,
tavily_search_context,
web_search_cfg,
)
from openjarvis.agents.hybrid.mini_swe_agent import (
@@ -135,7 +136,12 @@ class AdvisorsAgent(LocalCloudAgent):
advisor_temperature = float(cfg.get("advisor_temperature", 0.2))
ws_enabled, ws_max_uses = web_search_cfg(cfg)
if ws_enabled and self._cloud_endpoint not in _SEARCH_CAPABLE_ENDPOINTS:
search_backend = str(cfg.get("search_backend", "provider")).lower()
if (
ws_enabled
and search_backend != "tavily"
and self._cloud_endpoint not in _SEARCH_CAPABLE_ENDPOINTS
):
raise ValueError(
f"web_search.enabled=true but cloud_endpoint={self._cloud_endpoint!r}; "
"server-side web_search is wired for anthropic / openai / gemini "
@@ -146,19 +152,23 @@ class AdvisorsAgent(LocalCloudAgent):
use_ws = ws_enabled
gaia_max_turns = int(cfg.get("gaia_max_turns", 8))
n_searches_total = 0
search_cost_total = 0.0
# 1. Initial executor pass — advisor (Qwen) doesn't get tools;
# only the cloud executor passes do. With web_search on, dispatch
# to the search-capable agent loop for the configured provider.
if use_ws:
initial_resp, e1_in, e1_out, n_s1, e1_turns = self._executor_search(
(initial_resp, e1_in, e1_out, n_s1, e1_turns,
e1_search_cost) = self._executor_search(
user=f"Question:\n{question}",
system=EXECUTOR_INITIAL_SYS,
max_tokens=executor_max_tokens,
ws_max_uses=ws_max_uses,
max_turns=gaia_max_turns,
query=question,
)
n_searches_total += n_s1
search_cost_total += e1_search_cost
else:
initial_resp, e1_in, e1_out = self._call_cloud(
user=f"Question:\n{question}",
@@ -196,14 +206,17 @@ class AdvisorsAgent(LocalCloudAgent):
f"answer-format rules."
)
if use_ws:
final_answer, e2_in, e2_out, n_s2, e2_turns = self._executor_search(
(final_answer, e2_in, e2_out, n_s2, e2_turns,
e2_search_cost) = self._executor_search(
user=final_user,
system=EXECUTOR_FINAL_SYS,
max_tokens=executor_max_tokens,
ws_max_uses=ws_max_uses,
max_turns=gaia_max_turns,
query=question,
)
n_searches_total += n_s2
search_cost_total += e2_search_cost
else:
final_answer, e2_in, e2_out = self._call_cloud(
user=final_user,
@@ -216,7 +229,10 @@ class AdvisorsAgent(LocalCloudAgent):
tokens_local = adv_in + adv_out
tokens_cloud = e1_in + e1_out + e2_in + e2_out
cost = self.cost_usd(self._cloud_model, e1_in + e2_in, e1_out + e2_out)
cost += n_searches_total * _search_cost_per_call(self._cloud_endpoint)
if search_backend == "tavily":
cost += search_cost_total
else:
cost += n_searches_total * _search_cost_per_call(self._cloud_endpoint)
meta: Dict[str, Any] = {
"tokens_local": tokens_local,
@@ -233,7 +249,9 @@ class AdvisorsAgent(LocalCloudAgent):
"initial_response": initial_resp,
"advisor_feedback": advisor_text,
"web_search_enabled": use_ws,
"search_backend": search_backend,
"n_web_searches": n_searches_total,
"search_cost_usd": search_cost_total,
"note": "inference-only advisor (untrained); lower bound on the technique.",
},
}
@@ -251,16 +269,40 @@ class AdvisorsAgent(LocalCloudAgent):
max_tokens: int,
ws_max_uses: int,
max_turns: int,
) -> Tuple[str, int, int, int, int]:
query: Optional[str] = None,
) -> Tuple[str, int, int, int, int, float]:
"""Run a search-capable executor pass for the configured cloud.
Dispatches by ``self._cloud_endpoint`` to the matching ``_base``
agent loop. Returns the shared 5-tuple ``(text, p_tok, c_tok,
n_searches, turns)``. The endpoint is assumed already validated
against ``_SEARCH_CAPABLE_ENDPOINTS`` by the caller.
agent loop, or through Tavily when ``method_cfg.search_backend`` is
``"tavily"``. Returns ``(text, p_tok, c_tok, n_searches, turns,
search_cost_usd)``.
"""
if str(self._cfg.get("search_backend", "provider")).lower() == "tavily":
res = tavily_search_context(
query or user,
max_results=int(self._cfg.get("tavily_max_results", 5)),
)
grounded_user = (
f"Web search results:\n{res['text']}\n\n"
f"Using the search results above, answer this request:\n{user}"
)
text, p, c = self._call_cloud(
user=grounded_user,
system=system,
max_tokens=max_tokens,
temperature=0.0,
)
return (
text,
p,
c,
int(res["n_searches"]),
1,
float(res["cost_usd"]),
)
if self._cloud_endpoint == "anthropic":
return self._call_anthropic_agent(
text, p, c, n_searches, turns = self._call_anthropic_agent(
self._cloud_model,
user=user,
system=system,
@@ -269,8 +311,9 @@ class AdvisorsAgent(LocalCloudAgent):
tools=[build_web_search_tool(ws_max_uses)],
max_turns=max_turns,
)
return text, p, c, n_searches, turns, 0.0
if self._cloud_endpoint == "openai":
return self._call_openai_agent(
text, p, c, n_searches, turns = self._call_openai_agent(
self._cloud_model,
user=user,
system=system,
@@ -278,8 +321,9 @@ class AdvisorsAgent(LocalCloudAgent):
temperature=0.0,
max_turns=max_turns,
)
return text, p, c, n_searches, turns, 0.0
if self._cloud_endpoint == "gemini":
return self._call_gemini_agent(
text, p, c, n_searches, turns = self._call_gemini_agent(
self._cloud_model,
user=user,
system=system,
@@ -287,6 +331,7 @@ class AdvisorsAgent(LocalCloudAgent):
temperature=0.0,
max_turns=max_turns,
)
return text, p, c, n_searches, turns, 0.0
# Genuinely unsupported (openrouter / vllm / unknown). The caller
# guard should have caught this; raise defensively.
raise ValueError(
+57 -18
View File
@@ -46,6 +46,7 @@ from openjarvis.agents.hybrid._base import (
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
build_web_search_tool,
tavily_search_context,
web_search_cfg,
)
from openjarvis.agents.hybrid._prices import (
@@ -456,8 +457,14 @@ def _format_worker_pool(workers: List[Dict[str, Any]]) -> str:
)
def _search_capable_indices(workers: List[Dict[str, Any]]) -> List[int]:
def _search_capable_indices(
workers: List[Dict[str, Any]],
*,
search_backend: str = "provider",
) -> List[int]:
"""Indices of workers whose endpoint can run server-side web search."""
if search_backend == "tavily":
return [w["id"] for w in workers]
return [
w["id"] for w in workers
if (w.get("endpoint") or "openai").lower()
@@ -470,6 +477,7 @@ def _build_conductor_prompt(
workers: List[Dict[str, Any]],
*,
web_search_enabled: bool = False,
search_backend: str = "provider",
) -> str:
"""Build the planner prompt.
@@ -485,12 +493,16 @@ def _build_conductor_prompt(
)
if not web_search_enabled:
return base
capable = _search_capable_indices(workers)
capable = _search_capable_indices(workers, search_backend=search_backend)
if capable:
cap_str = ", ".join(str(i) for i in capable)
if search_backend == "tavily":
capability = "External Tavily search results will be prepended to worker prompts"
else:
capability = "Only these model indices can perform live web search"
constraint = (
"\n\nWEB SEARCH CONSTRAINT:\n"
f"Only these model indices can perform live web search: [{cap_str}]. "
f"{capability}: [{cap_str}]. "
"Any step that needs to look up facts, current events, or other "
"information not reliably known from memory MUST be routed to one "
"of those indices. Steps routed to any other model can only use "
@@ -550,8 +562,8 @@ def _call_worker(
*,
web_search_tool: Optional[Dict[str, Any]] = None,
web_search_max_uses: int = 8,
) -> Tuple[str, int, int, bool, int]:
"""Returns (text, p_tok, c_tok, is_local, n_web_searches).
) -> Tuple[str, int, int, bool, int, float]:
"""Returns (text, p_tok, c_tok, is_local, n_web_searches, extra_cost).
``web_search_tool``: a truthy marker that web_search is enabled for
this run. When set AND the worker endpoint is search-capable
@@ -565,6 +577,22 @@ def _call_worker(
max_tok = int(cfg.get("worker_max_tokens", 4096))
temp = float(cfg.get("worker_temperature", 0.2))
use_ws = web_search_tool is not None
search_backend = str(cfg.get("search_backend", "provider")).lower()
extra_cost = 0.0
if use_ws and search_backend == "tavily":
res = tavily_search_context(
prompt,
max_results=int(cfg.get("tavily_max_results", 5)),
)
prompt = (
f"Web search results:\n{res['text']}\n\n"
f"Using the search results above, answer this request:\n{prompt}"
)
extra_cost = float(res["cost_usd"])
use_ws = False
tavily_searches = int(res["n_searches"])
else:
tavily_searches = 0
if ep == "vllm":
text, p, c = LocalCloudAgent._call_vllm(
@@ -575,7 +603,7 @@ def _call_worker(
temperature=temp,
enable_thinking=False,
)
return text, p, c, True, 0
return text, p, c, True, tavily_searches, extra_cost
if ep == "openai":
if use_ws:
text, p, c, n_searches, _ = LocalCloudAgent._call_openai_agent(
@@ -584,14 +612,14 @@ def _call_worker(
max_tokens=max_tok,
temperature=(1.0 if is_gpt5_family(worker["model"]) else temp),
)
return text, p, c, False, n_searches
return text, p, c, False, n_searches, 0.0
text, p, c = LocalCloudAgent._call_openai(
worker["model"],
user=prompt,
max_tokens=max_tok,
temperature=(1.0 if is_gpt5_family(worker["model"]) else temp),
)
return text, p, c, False, 0
return text, p, c, False, tavily_searches, extra_cost
if ep == "openrouter":
# OpenRouter is OpenAI-compatible; the helper handles the
# base_url + OPENROUTER_API_KEY plumbing. No server-side web
@@ -607,7 +635,7 @@ def _call_worker(
temperature=temp,
extra_body=extra_body if isinstance(extra_body, dict) else None,
)
return text, p, c, False, 0
return text, p, c, False, tavily_searches, extra_cost
if ep == "anthropic":
eff_temp = temp if supports_temperature(worker["model"]) else 0.0
anthropic_kwargs: Dict[str, Any] = dict(
@@ -620,7 +648,7 @@ def _call_worker(
text, p, c, n_searches = LocalCloudAgent._call_anthropic(
worker["model"], **anthropic_kwargs
)
return text, p, c, False, n_searches
return text, p, c, False, n_searches or tavily_searches, extra_cost
if ep == "gemini":
# Gemini Developer API via google-genai. With web_search on, route
# through the Google-Search-grounded agent loop; otherwise plain
@@ -632,14 +660,14 @@ def _call_worker(
max_tokens=max_tok,
temperature=temp,
)
return text, p, c, False, n_searches
return text, p, c, False, n_searches, 0.0
text, p, c = LocalCloudAgent._call_gemini(
worker["model"],
user=prompt,
max_tokens=max_tok,
temperature=temp,
)
return text, p, c, False, 0
return text, p, c, False, tavily_searches, extra_cost
raise ValueError(f"unsupported worker endpoint: {ep!r}")
@@ -672,7 +700,7 @@ def _swe_worker_step(
# backbones today (the loop's tool-call format is Anthropic- or
# OpenAI-via-vllm-shaped only). Fall back to one-shot for those —
# SWE-bench-wise they were already weak; this preserves behavior.
text, p, c, is_local, n_searches = _call_worker(worker, prompt, cfg)
text, p, c, is_local, n_searches, _extra = _call_worker(worker, prompt, cfg)
return text, p, c, is_local, n_searches, 0
out = run_swe_agent_loop(
task,
@@ -753,13 +781,17 @@ class ConductorAgent(LocalCloudAgent):
and bool(task_meta_early.get("base_commit"))
)
ws_enabled, ws_max_uses = web_search_cfg(cfg)
search_backend = str(cfg.get("search_backend", "provider")).lower()
planner_ws = ws_enabled and not swe_mode_early
# 1. Plan — when web_search is on (GAIA), the prompt names which
# worker indices can actually search, so the planner routes
# research steps to a search-capable worker.
user = _build_conductor_prompt(
question, workers, web_search_enabled=planner_ws,
question,
workers,
web_search_enabled=planner_ws,
search_backend=search_backend,
)
plan_text, p_in, p_out = self._call_cloud(
user=user,
@@ -833,7 +865,7 @@ class ConductorAgent(LocalCloudAgent):
# memory. Fail loud instead of degrading silently.
# ``ws_enabled`` / ``ws_max_uses`` computed up front for the planner
# constraint — reuse them here.
if ws_enabled and not swe_mode:
if ws_enabled and search_backend != "tavily" and not swe_mode:
search_workers = [
w for w in workers
if (w.get("endpoint") or "openai").lower()
@@ -894,7 +926,7 @@ class ConductorAgent(LocalCloudAgent):
# may legitimately not need search; see Task-3 planner
# constraint that tries to prevent this upfront).
if (
ws_enabled and not swe_mode
ws_enabled and search_backend != "tavily" and not swe_mode
and worker_ep not in _SEARCH_CAPABLE_WORKER_ENDPOINTS
):
self.record_trace_event({
@@ -911,6 +943,7 @@ class ConductorAgent(LocalCloudAgent):
),
})
extra_cost = 0.0
if swe_mode:
text, w_in, w_out, is_local, n_searches, bash_turns = (
_swe_worker_step(
@@ -919,7 +952,9 @@ class ConductorAgent(LocalCloudAgent):
)
tool_calls += bash_turns
else:
text, w_in, w_out, is_local, n_searches = _call_worker(
(
text, w_in, w_out, is_local, n_searches, extra_cost
) = _call_worker(
worker, prompt, cfg,
web_search_tool=ws_tool,
web_search_max_uses=ws_max_uses,
@@ -930,7 +965,10 @@ class ConductorAgent(LocalCloudAgent):
else:
tokens_cloud += w_in + w_out
cost += self.cost_usd(worker["model"], w_in, w_out)
cost += n_searches * _worker_search_cost_per_call(worker_ep)
if search_backend != "tavily":
cost += n_searches * _worker_search_cost_per_call(worker_ep)
if search_backend == "tavily":
cost += extra_cost
n_web_searches_total += n_searches
tool_calls += n_searches
steps.append({
@@ -981,6 +1019,7 @@ class ConductorAgent(LocalCloudAgent):
"plan": plan,
"fallback_used": fallback_used,
"web_search_enabled": ws_enabled,
"search_backend": search_backend,
"n_web_searches": n_web_searches_total,
"parse_attempts": parse_attempts,
"workers": [
+29 -4
View File
@@ -48,12 +48,13 @@ from openjarvis.agents.hybrid._base import (
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
build_web_search_tool,
tavily_search_context,
web_search_cfg,
)
from openjarvis.agents.hybrid._openai_retry import (
patch_openai_globally as _patch_openai_globally,
)
from openjarvis.agents.hybrid._prices import NO_TEMP_PREFIXES
from openjarvis.agents.hybrid._prices import NO_TEMP_PREFIXES, default_max_output_tokens
from openjarvis.agents.hybrid.mini_swe_agent import run_swe_agent_loop
from openjarvis.core.registry import AgentRegistry
@@ -362,6 +363,8 @@ def _prefetch_context(
cloud_endpoint: str,
cloud_model: str,
max_uses: int = 8,
search_backend: str = "provider",
tavily_max_results: int = 5,
) -> Dict[str, Any]:
"""Use Anthropic web_search to fetch real source material the worker can read.
@@ -375,6 +378,22 @@ def _prefetch_context(
out: Dict[str, Any] = {
"text": "", "tokens": 0, "cost_usd": 0.0, "n_searches": 0,
}
if search_backend == "tavily":
try:
res = tavily_search_context(question, max_results=tavily_max_results)
out.update(
text=res["text"],
cost_usd=float(res["cost_usd"]),
n_searches=int(res["n_searches"]),
tokens=0,
engine=res.get("engine"),
credits=res.get("credits"),
)
if res.get("error"):
out["error"] = res["error"]
except Exception as e:
out["error"] = f"{type(e).__name__}: {e}"
return out
if cloud_endpoint != "anthropic" or not (question or "").strip():
return out
try:
@@ -498,18 +517,22 @@ class MinionsAgent(LocalCloudAgent):
max_tokens=cfg.get("worker_max_tokens", 4096),
local=True,
)
cloud_max_tokens = int(
cfg.get("cloud_max_tokens")
or default_max_output_tokens(self._cloud_model)
)
if self._cloud_endpoint == "openai":
cloud_client = OpenAIClient(
model_name=self._cloud_model,
temperature=0.0,
max_tokens=4096,
max_tokens=cloud_max_tokens,
)
elif self._cloud_endpoint == "anthropic":
# Temperature stripping is handled by the global patch above for Opus 4.7+.
cloud_client = AnthropicClient(
model_name=self._cloud_model,
temperature=0.0,
max_tokens=4096,
max_tokens=cloud_max_tokens,
)
elif self._cloud_endpoint == "gemini":
# The vendored Minion library already special-cases GeminiClient
@@ -520,7 +543,7 @@ class MinionsAgent(LocalCloudAgent):
cloud_client = GeminiClient(
model_name=self._cloud_model,
temperature=0.0,
max_tokens=4096,
max_tokens=cloud_max_tokens,
)
else:
raise ValueError(f"unsupported cloud endpoint: {self._cloud_endpoint!r}")
@@ -560,6 +583,8 @@ class MinionsAgent(LocalCloudAgent):
self._cloud_endpoint,
self._cloud_model,
max_uses=ws_max_uses,
search_backend=str(cfg.get("search_backend", "provider")).lower(),
tavily_max_results=int(cfg.get("tavily_max_results", 5)),
)
if prefetch.get("text"):
+85 -48
View File
@@ -38,13 +38,14 @@ except ModuleNotFoundError:
from openjarvis.agents._stubs import AgentContext, AgentResult
from openjarvis.agents.hybrid._energy import EnergyCollector
from openjarvis.agents.hybrid._prompts import format_prompt as _format_prompt
from openjarvis.core.paths import get_config_dir
PACKAGE_DIR = Path(__file__).parent
DEFAULT_REGISTRY_DIR = PACKAGE_DIR / "registry"
DEFAULT_EXPERIMENTS_DIR = Path(
os.environ.get(
"OPENJARVIS_HYBRID_EXPERIMENTS_DIR",
Path.home() / ".openjarvis" / "experiments" / "hybrid",
get_config_dir() / "experiments" / "hybrid",
)
)
DEFAULT_SUBSETS_DIR = DEFAULT_EXPERIMENTS_DIR / "subsets"
@@ -122,6 +123,7 @@ def load_registry(registry_dir: Optional[Path] = None) -> Dict[str, Dict[str, An
# ---------- Bench dispatch ----------
def _load_gaia_tasks(n: Optional[int]) -> List[Dict[str, Any]]:
"""GAIA validation. Each task is a dict with `task_id` + `question`."""
from openjarvis.evals.datasets.gaia import GAIADataset
@@ -137,12 +139,14 @@ def _load_gaia_tasks(n: Optional[int]) -> List[Dict[str, Any]]:
# id round-trip.
md = rec.metadata or {}
task_id = md.get("task_id") or rec.record_id
out.append({
"task_id": task_id,
"question": md.get("question", rec.problem),
"reference": rec.reference,
"metadata": dict(md),
})
out.append(
{
"task_id": task_id,
"question": md.get("question", rec.problem),
"reference": rec.reference,
"metadata": dict(md),
}
)
return out
@@ -155,19 +159,21 @@ def _load_swebench_tasks(n: Optional[int]) -> List[Dict[str, Any]]:
out: List[Dict[str, Any]] = []
for rec in ds.iter_records():
md = rec.metadata or {}
out.append({
"task_id": md.get("instance_id", rec.record_id),
"repo": md.get("repo", ""),
"base_commit": md.get("base_commit", ""),
"problem_statement": md.get("problem_statement", rec.problem),
"hints_text": md.get("hints_text", ""),
"test_patch": md.get("test_patch", ""),
"FAIL_TO_PASS": md.get("FAIL_TO_PASS", []),
"PASS_TO_PASS": md.get("PASS_TO_PASS", []),
"version": md.get("version"),
"reference": rec.reference,
"metadata": dict(md),
})
out.append(
{
"task_id": md.get("instance_id", rec.record_id),
"repo": md.get("repo", ""),
"base_commit": md.get("base_commit", ""),
"problem_statement": md.get("problem_statement", rec.problem),
"hints_text": md.get("hints_text", ""),
"test_patch": md.get("test_patch", ""),
"FAIL_TO_PASS": md.get("FAIL_TO_PASS", []),
"PASS_TO_PASS": md.get("PASS_TO_PASS", []),
"version": md.get("version"),
"reference": rec.reference,
"metadata": dict(md),
}
)
return out
@@ -206,7 +212,9 @@ def _load_subset_file(subset_path: str) -> Dict[str, Any]:
f"subset {p.name} has no 'task_ids' field; got keys {list(data.keys())}"
)
return data
raise ValueError(f"subset {p.name} must be a list or dict; got {type(data).__name__}")
raise ValueError(
f"subset {p.name} must be a list or dict; got {type(data).__name__}"
)
def _apply_subset(
@@ -332,7 +340,11 @@ def _score_swebench(
patch = extract_patch(answer)
if patch is None:
return {"success": False, "score": 0.0, "details": {"reason": "no_patch_extracted"}}
return {
"success": False,
"score": 0.0,
"details": {"reason": "no_patch_extracted"},
}
record = EvalRecord(
record_id=task["task_id"],
@@ -368,6 +380,7 @@ def score(
# ---------- Cell run ----------
def _cell_dir(cell_name: str, root: Path) -> Path:
d = root / cell_name
d.mkdir(parents=True, exist_ok=True)
@@ -440,8 +453,10 @@ def _error_row(task: Dict[str, Any], t0: float, error: str) -> Dict[str, Any]:
return {
"task_id": task["task_id"],
"answer": "",
"tokens_local": 0, "tokens_cloud": 0,
"cost_usd": 0.0, "latency_s": time.time() - t0,
"tokens_local": 0,
"tokens_cloud": 0,
"cost_usd": 0.0,
"latency_s": time.time() - t0,
"web_search_uses": 0,
"tool_calls": 0,
"n_cloud_calls": 0,
@@ -456,11 +471,13 @@ def _run_one_inner(
) -> Dict[str, Any]:
"""Run the agent on one task. Returns a hybrid-shape row."""
prompt = _format_prompt(task)
ctx = AgentContext(metadata={
"task": task,
"task_id": task["task_id"],
"log_dir": log_dir,
})
ctx = AgentContext(
metadata={
"task": task,
"task_id": task["task_id"],
"log_dir": log_dir,
}
)
t0 = time.time()
try:
result: AgentResult = agent.run(prompt, ctx)
@@ -534,11 +551,12 @@ def _run_one(
if worker.is_alive():
print(
f"[timeout] task={task['task_id']} exceeded "
f"{task_timeout_s/60:.1f}m — abandoning worker, recording error row",
f"{task_timeout_s / 60:.1f}m — abandoning worker, recording error row",
flush=True,
)
return _error_row(
task, t0,
task,
t0,
f"TaskTimeout: task exceeded the {task_timeout_s:.0f}s hybrid "
"per-task wall-clock cap (likely a hung network or Modal-harness "
"call); worker thread abandoned, task left for resume.",
@@ -558,7 +576,7 @@ def _heartbeat(done: int, total: int, row: Dict[str, Any], t_start: float) -> No
print(
f"[{done}/{total}] {ok} task={row['task_id']} score={sc_str} "
f"local={row['tokens_local']} cloud={row['tokens_cloud']} "
f"${row['cost_usd']:.3f} {row['latency_s']:.1f}s eta={eta/60:.1f}m",
f"${row['cost_usd']:.3f} {row['latency_s']:.1f}s eta={eta / 60:.1f}m",
flush=True,
)
@@ -644,9 +662,9 @@ def _write_summary(
summary_path.write_text(json.dumps(summary, indent=2))
print(
f"[summary] {cell_name}: n={n_done}/{cell['n']} err={n_err} "
f"acc={acc:.3f} cost=${total_cost:.2f} time={wall/60:.1f}m "
f"energy={energy_j/1000:.1f}kJ "
f"(session +{elapsed/60:.1f}m +{energy_j_session/1000:.1f}kJ, "
f"acc={acc:.3f} cost=${total_cost:.2f} time={wall / 60:.1f}m "
f"energy={energy_j / 1000:.1f}kJ "
f"(session +{elapsed / 60:.1f}m +{energy_j_session / 1000:.1f}kJ, "
f"processed={n_processed})",
flush=True,
)
@@ -664,8 +682,11 @@ def run_cell(
out_dir = _cell_dir(cell_name, out_root)
with _cell_lock(out_dir, cell_name):
_run_cell_locked(
cell_name, cell, out_dir,
do_score=do_score, resume=resume,
cell_name,
cell,
out_dir,
do_score=do_score,
resume=resume,
)
@@ -727,7 +748,7 @@ def _run_cell_locked(
mcfg = cell.get("method_cfg") or {}
task_timeout_s = float(mcfg.get("task_timeout_s", DEFAULT_TASK_TIMEOUT_S))
if task_timeout_s > 0:
print(f"[task-timeout] {task_timeout_s/60:.1f}m per task", flush=True)
print(f"[task-timeout] {task_timeout_s / 60:.1f}m per task", flush=True)
agent = _build_agent(cell)
@@ -739,18 +760,25 @@ def _run_cell_locked(
def _process(task: Dict[str, Any]) -> None:
row = _run_one(
agent, cell["bench"], task, log_dir,
agent,
cell["bench"],
task,
log_dir,
task_timeout_s=task_timeout_s,
)
scored: Optional[Dict[str, Any]] = None
if do_score and row.get("error") is None:
try:
scored = score(
cell["bench"], task, row["answer"], cell_name=cell_name,
cell["bench"],
task,
row["answer"],
cell_name=cell_name,
)
except Exception as e:
scored = {
"success": False, "score": 0.0,
"success": False,
"score": 0.0,
"details": {"score_error": str(e)},
}
full_row = {**row, "score": scored}
@@ -789,9 +817,7 @@ def _run_cell_locked(
while not watchdog_stop.wait(60.0):
try:
cur = (
results_path.stat().st_mtime
if results_path.exists()
else last_seen
results_path.stat().st_mtime if results_path.exists() else last_seen
)
except Exception:
cur = last_seen
@@ -830,7 +856,11 @@ def _run_cell_locked(
watchdog_stop.set()
_write_summary(
out_dir, cell_name, cell, tasks, t_start,
out_dir,
cell_name,
cell,
tasks,
t_start,
n_processed=len(pending),
energy_j_session=energy.energy_j_total,
)
@@ -838,6 +868,7 @@ def _run_cell_locked(
# ---------- CLI ----------
def main(argv: Optional[List[str]] = None) -> int:
p = argparse.ArgumentParser(
prog="python -m openjarvis.agents.hybrid.runner",
@@ -855,13 +886,18 @@ def main(argv: Optional[List[str]] = None) -> int:
help="Override experiments output root.",
)
p.add_argument("--no-score", action="store_true", help="Skip scoring.")
p.add_argument("--no-resume", action="store_true", help="Don't resume from results.jsonl.")
p.add_argument(
"--no-resume", action="store_true", help="Don't resume from results.jsonl."
)
args = p.parse_args(argv)
reg_dir = Path(args.registry_dir) if args.registry_dir else None
cells = load_registry(reg_dir)
if not cells:
print(f"[error] no cells found in {reg_dir or DEFAULT_REGISTRY_DIR}", file=sys.stderr)
print(
f"[error] no cells found in {reg_dir or DEFAULT_REGISTRY_DIR}",
file=sys.stderr,
)
return 2
if args.cell not in cells:
print(
@@ -871,7 +907,8 @@ def main(argv: Optional[List[str]] = None) -> int:
return 2
root = Path(args.root) if args.root else None
run_cell(
args.cell, cells[args.cell],
args.cell,
cells[args.cell],
do_score=not args.no_score,
resume=not args.no_resume,
root=root,
+83 -30
View File
@@ -149,6 +149,17 @@ def _build_router_schema(agent_ids: List[str]) -> Dict[str, Any]:
}
def _openai_response_format(schema: Dict[str, Any]) -> Dict[str, Any]:
return {
"type": "json_schema",
"json_schema": {
"name": "skillorchestra_route",
"schema": schema["format"]["schema"],
"strict": True,
},
}
def _parse_router_json(text: str) -> Dict[str, Any]:
s = (text or "").strip()
try:
@@ -196,6 +207,70 @@ class SkillOrchestraAgent(LocalCloudAgent):
agent_id = "skillorchestra"
def _route_call(
self,
*,
question: str,
router_sys: str,
router_schema: Dict[str, Any],
router_max: int,
) -> Tuple[str, int, int]:
user = f"Question:\n{question}"
if self._cloud_endpoint == "anthropic":
kwargs: Dict[str, Any] = {
"user": user,
"system": router_sys,
"max_tokens": router_max,
"output_config": router_schema,
}
if supports_temperature(self._cloud_model):
kwargs["temperature"] = 0.0
text, r_in, r_out, _ = self._call_anthropic(
self._cloud_model,
**kwargs,
)
return text, r_in, r_out
if self._cloud_endpoint == "openai":
return self._call_openai(
self._cloud_model,
user=user,
system=router_sys,
max_tokens=router_max,
temperature=0.0,
response_format=_openai_response_format(router_schema),
)
if self._cloud_endpoint == "gemini":
return self._call_gemini(
self._cloud_model,
user=user,
system=router_sys,
max_tokens=router_max,
temperature=0.0,
)
raise ValueError(
f"SkillOrchestra router unsupported cloud_endpoint={self._cloud_endpoint!r}"
)
def _executor_call(
self,
*,
question: str,
max_tokens: int,
) -> Tuple[str, int, int]:
if self._cloud_endpoint == "anthropic":
text, w_in, w_out, _ = self._call_anthropic(
self._cloud_model,
user=question,
max_tokens=max_tokens,
temperature=0.0,
)
return text, w_in, w_out
return self._call_cloud(
user=question,
max_tokens=max_tokens,
temperature=0.0,
)
def _is_soft_failure(self, exc: BaseException) -> Optional[str]:
# Empty/unbalanced router JSON — treat as soft failure to match the
# hybrid adapter's behavior (matches `err=1` rows in the n=30 cell).
@@ -220,33 +295,13 @@ class SkillOrchestraAgent(LocalCloudAgent):
router_sys = _build_router_sys(competence, cost)
router_schema = _build_router_schema(agent_ids)
# 1. Route — Anthropic only (output_config schema is Anthropic-specific
# in the hybrid adapter). If you need OpenAI routing, swap the prompt
# to JSON-mode and bypass output_config.
if self._cloud_endpoint != "anthropic":
raise ValueError(
"SkillOrchestra router requires cloud_endpoint='anthropic'; "
f"got {self._cloud_endpoint!r}"
)
router_max = int(cfg.get("router_max_tokens", 1024))
# Strip temperature for Opus 4.7+; Anthropic's output_config does the schema.
if supports_temperature(self._cloud_model):
router_text, r_in, r_out, _ = self._call_anthropic(
self._cloud_model,
user=f"Question:\n{question}",
system=router_sys,
max_tokens=router_max,
temperature=0.0,
output_config=router_schema,
)
else:
router_text, r_in, r_out, _ = self._call_anthropic(
self._cloud_model,
user=f"Question:\n{question}",
system=router_sys,
max_tokens=router_max,
output_config=router_schema,
)
router_text, r_in, r_out = self._route_call(
question=question,
router_sys=router_sys,
router_schema=router_schema,
router_max=router_max,
)
decision = _parse_router_json(router_text)
skill_weights: Dict[str, float] = decision.get("skill_weights") or {}
@@ -329,11 +384,9 @@ class SkillOrchestraAgent(LocalCloudAgent):
tokens_cloud += out["tokens_in"] + out["tokens_out"]
run_cost += out["cost_usd"]
else:
ans, w_in, w_out, _ = self._call_anthropic(
self._cloud_model,
user=question,
ans, w_in, w_out = self._executor_call(
question=question,
max_tokens=int(cfg.get("cloud_max_tokens", 4096)),
temperature=0.0,
)
tokens_cloud += w_in + w_out
run_cost += self.cost_usd(self._cloud_model, w_in, w_out)
@@ -31,7 +31,14 @@ from .stage_router import (
get_routing_strategy,
parse_skill_analysis,
)
from .tools import anthropic_tools, openai_tools, run_answer, run_code, run_search
from .tools import (
anthropic_tools,
gemini_tools,
openai_tools,
run_answer,
run_code,
run_search,
)
# tool name -> routing stage (stage_router uses "reasoning" for code).
_TOOL_STAGE = {
@@ -115,10 +122,49 @@ def _orchestrate_step(
u = resp.usage
p = getattr(u, "prompt_tokens", 0) if u else 0
c = getattr(u, "completion_tokens", 0) if u else 0
elif endpoint == "gemini":
from google import genai
from google.genai import types
client = genai.Client(
http_options=types.HttpOptions(timeout=600_000)
)
cfg = types.GenerateContentConfig(
temperature=1.0,
max_output_tokens=max_tokens,
tools=[types.Tool(function_declarations=gemini_tools())],
)
resp = client.models.generate_content(
model=model,
contents=user,
config=cfg,
)
text = (resp.text or "") if hasattr(resp, "text") else ""
tool_calls = []
try:
parts = resp.candidates[0].content.parts or []
except Exception: # noqa: BLE001
parts = []
for part in parts:
fc = getattr(part, "function_call", None)
if fc is None:
continue
name = getattr(fc, "name", None)
if not isinstance(name, str) or not name:
continue
args = getattr(fc, "args", None) or {}
try:
args = dict(args)
except Exception: # noqa: BLE001
args = {}
tool_calls.append({"name": name, "input": args})
um = getattr(resp, "usage_metadata", None)
p = int(getattr(um, "prompt_token_count", 0) or 0) if um else 0
c = int(getattr(um, "candidates_token_count", 0) or 0) if um else 0
else:
raise ValueError(
f"orchestrator endpoint {endpoint!r} unsupported — route the "
"orchestrator through anthropic/openai (set method_cfg."
"orchestrator through anthropic/openai/gemini (set method_cfg."
"orchestrator_endpoint)."
)
@@ -192,6 +238,8 @@ def run_orchestrator(
code_timeout = int(cfg.get("code_timeout_s", 60))
answer_max_tokens = int(cfg.get("answer_max_tokens", 40000))
ws_max_uses = int(cfg.get("web_search_max_uses", 5))
search_backend = str(cfg.get("search_backend", "provider")).lower()
tavily_max_results = int(cfg.get("tavily_max_results", 5))
# The orchestrator model: a fixed model per run (the original's
# MODEL_NAME). Defaults to the cell's cloud model when that endpoint
@@ -203,7 +251,7 @@ def run_orchestrator(
orch_model = (cfg.get("orchestrator_model")
or cfg.get("router_model")
or agent._cloud_model)
if orch_endpoint not in ("anthropic", "openai"):
if orch_endpoint not in ("anthropic", "openai", "gemini"):
orch_endpoint, orch_model = "anthropic", "claude-opus-4-7"
orch_max_tokens = int(cfg.get("orchestrator_max_tokens", 4096))
@@ -306,6 +354,8 @@ def run_orchestrator(
res = run_search(
agent, spec, context_str=context_str, problem=problem,
retriever_url=retriever_url, web_search_max_uses=ws_max_uses,
search_backend=search_backend,
tavily_max_results=tavily_max_results,
)
docs = res["search_results_data"]
joined = "\n---\n".join(d for d in docs if d)[:char_cap]
@@ -27,6 +27,7 @@ from .._base import (
OPENAI_WEB_SEARCH_COST_PER_CALL,
WEB_SEARCH_COST_PER_CALL,
build_web_search_tool,
tavily_search_context,
)
from .pool import ModelSpec, call_alias
@@ -110,6 +111,26 @@ def openai_tools() -> List[Dict[str, Any]]:
return out
def gemini_tools() -> List[Dict[str, Any]]:
"""The 3 orchestrator tools in Gemini function-declaration shape."""
out = []
for name, desc in (
("search", _SEARCH_DESC),
("enhance_reasoning", _CODE_DESC),
("answer", _ANSWER_DESC),
):
out.append({
"name": name,
"description": desc,
"parameters": {
"type": "object",
"properties": {"model": _model_prop(name)},
"required": ["model"],
},
})
return out
# ---------------------------------------------------------------------------
# enhance_reasoning / code — eval_frames.py:659-812
# ---------------------------------------------------------------------------
@@ -256,6 +277,8 @@ def run_search(
retriever_url: Optional[str] = None,
topk: int = 150,
web_search_max_uses: int = 5,
search_backend: str = "provider",
tavily_max_results: int = 5,
) -> Dict[str, Any]:
"""Write a search query with ``spec``, then retrieve documents.
@@ -283,7 +306,12 @@ def run_search(
contents: List[str] = []
search_uses = 0
if retriever_url:
if search_backend == "tavily":
res = tavily_search_context(query, max_results=tavily_max_results)
contents.append(res["text"])
search_uses = int(res["n_searches"])
cost += float(res["cost_usd"])
elif retriever_url:
# Faithful path — the original FAISS retriever service.
import requests
+96 -34
View File
@@ -16,7 +16,8 @@ Two modes, gated by ``method_cfg.orchestrator_mode``:
(``answer-1``, ``reasoner-2``, ``search-3``, ) is mapped to a real
backend through ``EXPERT_MODEL_MAPPING`` by default the frontier
Anthropic worker for `*-1` slots, gpt-5-mini for `*-2`, local Qwen
for `*-3`. Search routes to the Anthropic server-side web_search.
for `*-3`. Search routes to the configured provider's server-side
web-search helper when available.
We do NOT reproduce the upstream Tavily / FAISS-wiki retriever, the
code-interpreter sandbox, or the multi-vLLM mix (Llama-3.3-70B,
@@ -43,8 +44,8 @@ Prompted-mode pipeline:
prompt; fallback to strongest worker on parse failure.
Workers come from ``cfg["workers"]`` or a sensible default pool (local
Qwen if vLLM up, plus a web-search tool via Anthropic, Opus 4.7,
gpt-5-mini).
Qwen if vLLM up, plus provider-native web search, the configured frontier
cloud model, and gpt-5-mini).
"""
from __future__ import annotations
@@ -59,8 +60,11 @@ from typing import Any, Dict, List, Optional, Tuple
from openjarvis.agents._stubs import AgentContext
from openjarvis.agents.hybrid._base import (
ANTHROPIC_WEB_SEARCH_TOOL,
GEMINI_SEARCH_COST_PER_CALL,
OPENAI_WEB_SEARCH_COST_PER_CALL,
WEB_SEARCH_COST_PER_CALL,
LocalCloudAgent,
tavily_search_context,
)
from openjarvis.agents.hybrid._prices import (
PRICES,
@@ -197,10 +201,23 @@ def _expert_for(slot: str, local_model: Optional[str],
cost tier for mid OpenAI calls)
- `*-3` (local tier) -> local vLLM (`local_model`)
- `answer-math-*` -> same tiers as the numeric suffix
- `search-*` -> always the Anthropic web_search tool (the
upstream uses Tavily; we have web_search)
- `search-*` -> provider-native web search when the cloud
endpoint supports it; otherwise Anthropic
"""
if slot.startswith("search"):
ep = (cloud_endpoint or "anthropic").lower()
if ep == "openai":
return {
"name": f"search:{slot}",
"type": "openai-web-search",
"model": cloud_model,
}
if ep == "gemini":
return {
"name": f"search:{slot}",
"type": "gemini-web-search",
"model": cloud_model,
}
return {
"name": f"search:{slot}",
"type": "anthropic-web-search",
@@ -340,21 +357,18 @@ def _paper_expert_for(
# ---- Tavily + Modal helpers -------------------------------------------------
def _call_tavily_search(query: str, max_results: int = 5) -> Tuple[str, int, int]:
"""One-shot Tavily search. Returns (text, p_tok=0, c_tok=0).
def _call_tavily_search(
query: str,
max_results: int = 5,
) -> Tuple[str, int, int, float, int]:
"""One-shot Tavily search. Returns (text, p_tok=0, c_tok=0, cost, uses).
Token counts are reported as zero (no LLM was billed); the OpenJarvis
accounting layer separately tallies tool-call counts. Falls back to
DuckDuckGo if Tavily is unreachable (see ``WebSearchTool``).
"""
from openjarvis.tools.web_search import WebSearchTool
tool = WebSearchTool(max_results=max_results)
res = tool.execute(query=query, max_results=max_results)
text = res.content or ""
if not res.success and not text:
text = "(no results)"
return text, 0, 0
res = tavily_search_context(query, max_results=max_results)
return res["text"], 0, 0, float(res["cost_usd"]), int(res["n_searches"])
_MODAL_APP_NAME = "openjarvis-toolorchestra-sandbox"
@@ -674,13 +688,25 @@ def _default_pool(
"concise extraction, formatting, arithmetic on given data."
),
})
if ep == "openai":
search_type = "openai-web-search"
search_model = cloud_model
search_desc = "OpenAI hosted web search on the configured frontier model."
elif ep == "gemini":
search_type = "gemini-web-search"
search_model = cloud_model
search_desc = "Gemini Google Search grounding on the configured frontier model."
else:
search_type = "anthropic-web-search"
search_model = _DEFAULT_WEB_SEARCH_MODEL
search_desc = "Anthropic server-side web_search."
pool.append({
"id": len(pool),
"name": "web-search",
"type": "anthropic-web-search",
"model": "claude-haiku-4-5",
"type": search_type,
"model": search_model,
"description": (
"Anthropic server-side web_search. Use for facts that need a lookup "
f"{search_desc} Use for facts that need a lookup "
"(recent events, rare names/dates, niche sources). Returns a digest."
),
})
@@ -717,8 +743,13 @@ def _default_pool(
# `modal-python` — One-shot Python exec in a fresh Modal Sandbox (the
# paper's "Python sandbox" inside `enhance_reasoning`).
_TOOLORCH_VALID_TYPES = (
"vllm", "openai", "anthropic", "anthropic-web-search", "gemini",
"tavily-search", "openrouter", "modal-python",
"vllm", "openai", "anthropic", "anthropic-web-search",
"openai-web-search", "gemini", "gemini-web-search", "tavily-search",
"openrouter", "modal-python",
)
_TOOLORCH_SEARCH_TYPES = (
"anthropic-web-search", "openai-web-search", "gemini-web-search",
"tavily-search",
)
# Default model used when an `anthropic-web-search` entry omits `model`.
@@ -739,10 +770,12 @@ def _resolve_worker_pool(
the override is absent.
Each user-supplied entry must be a dict with keys ``id``, ``name``,
``type``, and (for non-search types) ``model``. ``type`` must be one
of ``vllm`` / ``openai`` / ``anthropic`` / ``anthropic-web-search``.
``anthropic-web-search`` entries may omit ``model`` it defaults to
``claude-haiku-4-5``.
``type``, and (for non-search types) ``model``. Search worker types are
``anthropic-web-search``, ``openai-web-search``, ``gemini-web-search``,
and ``tavily-search``. ``anthropic-web-search`` entries may omit
``model`` it defaults to ``claude-haiku-4-5``. OpenAI and Gemini
search workers default to the configured cloud model. Tavily does not
require a model.
Substitution: ``model = "$local"`` (or ``"<local>"``) resolves to
``local_model``; ``model = "$cloud"`` / ``"<cloud>"`` to ``cloud_model``.
@@ -804,14 +837,24 @@ def _resolve_worker_pool(
elif isinstance(model, str) and model in ("$cloud", "<cloud>"):
model = cloud_model
entry["model"] = model
if wtype == "anthropic-web-search":
if wtype in _TOOLORCH_SEARCH_TYPES:
if model in (None, ""):
model = _DEFAULT_WEB_SEARCH_MODEL
if wtype == "anthropic-web-search":
model = _DEFAULT_WEB_SEARCH_MODEL
elif wtype in ("openai-web-search", "gemini-web-search"):
model = cloud_model
else:
model = wtype
entry["model"] = model
elif not isinstance(model, str):
raise ValueError(
f"Invalid worker_pool entry [{wid}]: 'model' must be a string when set"
)
if wtype in ("openai-web-search", "gemini-web-search") and model not in PRICES:
raise ValueError(
f"Invalid worker_pool entry [{wid}]: model {model!r} "
f"is not in PRICES (known: {sorted(PRICES)})"
)
# Search workers don't satisfy the "needs a solver" requirement.
else:
if not isinstance(model, str) or not model:
@@ -843,7 +886,7 @@ def _resolve_worker_pool(
if not has_non_search:
raise ValueError(
"Invalid worker_pool entry [-]: worker_pool must contain at least "
"one non-search worker (vllm / openai / anthropic)"
"one non-search worker (vllm / openai / anthropic / gemini)"
)
return resolved
@@ -910,13 +953,31 @@ def _call_worker(
)
extra = n_searches * WEB_SEARCH_COST_PER_CALL
return text, p, c, False, extra, n_searches
if wtype == "openai-web-search":
eff_temp = 1.0 if is_gpt5_family(worker["model"]) else temp
text, p, c, n_searches, _ = LocalCloudAgent._call_openai_agent(
worker["model"],
user=prompt,
max_tokens=max(max_tok, 16384) if is_gpt5_family(worker["model"]) else max_tok,
temperature=eff_temp,
)
extra = n_searches * OPENAI_WEB_SEARCH_COST_PER_CALL
return text, p, c, False, extra, n_searches
if wtype == "gemini-web-search":
text, p, c, n_searches, _ = LocalCloudAgent._call_gemini_agent(
worker["model"],
user=prompt,
max_tokens=max_tok,
temperature=temp,
)
extra = n_searches * GEMINI_SEARCH_COST_PER_CALL
return text, p, c, False, extra, n_searches
if wtype == "tavily-search":
# Tavily costs are flat per call; charge `WEB_SEARCH_COST_PER_CALL`
# for parity with the Anthropic web-search worker. One call = one
# "n_search" for accounting.
max_results = int(cfg.get("tavily_max_results", 5))
text, p, c = _call_tavily_search(str(prompt), max_results=max_results)
return text, p, c, False, WEB_SEARCH_COST_PER_CALL, 1
text, p, c, extra, n_searches = _call_tavily_search(
str(prompt), max_results=max_results,
)
return text, p, c, False, extra, n_searches
if wtype == "openrouter":
text, p, c = LocalCloudAgent._call_openrouter(
worker["model"],
@@ -951,7 +1012,7 @@ def _swe_call_worker(
caller can surface ``tool_calls`` per row. Fallbacks to one-shot
workers return 0 bash turns (no agent loop ran)."""
wtype = worker.get("type", "openai")
if wtype == "anthropic-web-search":
if wtype in _TOOLORCH_SEARCH_TYPES:
# Search workers stay one-shot.
text, p, c, is_local, extra, n_searches = _call_worker(worker, prompt, cfg)
return text, p, c, is_local, extra, n_searches, 0
@@ -1197,7 +1258,8 @@ class ToolOrchestraAgent(LocalCloudAgent):
# Search workers are excluded — they answer fact-lookup
# questions, not synthesis.
non_search = [
w for w in workers if w.get("type") != "anthropic-web-search"
w for w in workers
if w.get("type") not in _TOOLORCH_SEARCH_TYPES
] or workers
worker = max(
non_search,
+3 -2
View File
@@ -11,10 +11,11 @@ import logging
import sqlite3
import time
import uuid
from pathlib import Path
from typing import Any, Dict, List, Optional
from uuid import uuid4
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
_CREATE_AGENTS = """\
@@ -537,7 +538,7 @@ class AgentManager:
pass
# User templates
user_dir = Path("~/.openjarvis/templates").expanduser()
user_dir = get_config_dir() / "templates"
if user_dir.is_dir():
for f in user_dir.glob("*.toml"):
try:
+3 -2
View File
@@ -13,6 +13,7 @@ from typing import Any, List, Optional
from openjarvis.agents._stubs import AgentContext, AgentResult, ToolUsingAgent
from openjarvis.agents.digest_store import DigestArtifact, DigestStore
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import Message, Role, ToolCall
@@ -21,7 +22,7 @@ def _load_persona(persona_name: str) -> str:
"""Load a persona prompt file by name."""
search_paths = [
Path("configs/openjarvis/prompts/personas") / f"{persona_name}.md",
Path.home() / ".openjarvis" / "prompts" / "personas" / f"{persona_name}.md",
get_config_dir() / "prompts" / "personas" / f"{persona_name}.md",
]
for p in search_paths:
if p.exists():
@@ -202,7 +203,7 @@ class MorningDigestAgent(ToolUsingAgent):
tts_text = re.sub(r"\*{1,2}([^*]+)\*{1,2}", r"\1", tts_text)
tts_text = tts_text.strip()
output_dir = str(Path.home() / ".openjarvis" / "digests")
output_dir = str(get_config_dir() / "digests")
tts_call = ToolCall(
id="digest-tts-1",
name="text_to_speech",
+3 -2
View File
@@ -43,6 +43,7 @@ from typing import Any, Dict, List, Optional, Set
from openjarvis.agents._stubs import AgentContext, AgentResult, ToolUsingAgent
from openjarvis.core.config import load_config
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import Message, Role, ToolCall
from openjarvis.tools.approval_store import (
@@ -342,8 +343,8 @@ class ProactiveAgent(ToolUsingAgent):
return self._approval_store
def _build_system_prompt(self) -> str:
user_md = _load_md_file(Path.home() / ".openjarvis" / "USER.md")
memory_md = _load_md_file(Path.home() / ".openjarvis" / "MEMORY.md")
user_md = _load_md_file(get_config_dir() / "USER.md")
memory_md = _load_md_file(get_config_dir() / "MEMORY.md")
now = datetime.now()
context_block = ""
if user_md or memory_md:
+4 -3
View File
@@ -15,16 +15,17 @@ from __future__ import annotations
import json
import logging
import os
from pathlib import Path
from typing import Any
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
def _openjarvis_home() -> Path:
"""Resolve $OPENJARVIS_HOME, defaulting to ~/.openjarvis."""
return Path(os.environ.get("OPENJARVIS_HOME", "~/.openjarvis")).expanduser()
"""Resolve the OpenJarvis root, honoring OPENJARVIS_HOME / XDG_DATA_HOME."""
return get_config_dir()
def load_system_prompt_override(agent_name: str) -> str | None:
+3 -1
View File
@@ -18,11 +18,13 @@ import time
from pathlib import Path
from typing import Any, Dict, List
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
_DEFAULT_DB_PATH = str(Path.home() / "Library" / "Messages" / "chat.db")
_POLL_INTERVAL = 5
_PID_FILE = str(Path.home() / ".openjarvis" / "imessage-agent.pid")
_PID_FILE = str(get_config_dir() / "imessage-agent.pid")
def poll_new_messages(
+3 -1
View File
@@ -14,9 +14,11 @@ import sys
from pathlib import Path
from typing import Any
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
_PID_FILE = str(Path.home() / ".openjarvis" / "slack-daemon.pid")
_PID_FILE = str(get_config_dir() / "slack-daemon.pid")
def _to_slack_fmt(text: str) -> str:
+2 -1
View File
@@ -22,6 +22,7 @@ from openjarvis.channels._stubs import (
ChannelStatus,
)
from openjarvis.core.events import EventBus, EventType
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import ChannelRegistry
logger = logging.getLogger(__name__)
@@ -38,7 +39,7 @@ if not _BRIDGE_SRC.exists():
)
# Default runtime directory (npm install + auth state).
_DEFAULT_RUNTIME_DIR = Path.home() / ".openjarvis" / "whatsapp_baileys_bridge"
_DEFAULT_RUNTIME_DIR = get_config_dir() / "whatsapp_baileys_bridge"
@ChannelRegistry.register("whatsapp_baileys")
+4 -2
View File
@@ -9,9 +9,11 @@ import sys
import time
from pathlib import Path
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
_CACHE_PATH = Path("~/.openjarvis/version-check.json").expanduser()
_CACHE_PATH = get_config_dir() / "version-check.json"
_CACHE_TTL = 86400 # 24 hours
_PYPI_API = "https://pypi.org/pypi/openjarvis/json"
@@ -21,7 +23,7 @@ def _config_path() -> Path:
override = os.environ.get("OPENJARVIS_CONFIG")
if override:
return Path(override).expanduser()
return Path("~/.openjarvis/config.toml").expanduser()
return get_config_dir() / "config.toml"
# Commands that surface the "new version available" nudge. We deliberately
+9 -11
View File
@@ -12,15 +12,12 @@ from rich.table import Table
def _get_manager():
"""Get or create the AgentManager singleton."""
from pathlib import Path
from openjarvis.agents.manager import AgentManager
from openjarvis.core.config import load_config
from openjarvis.core.paths import get_config_dir
config = load_config()
db_path = config.agent_manager.db_path or str(
Path("~/.openjarvis/agents.db").expanduser()
)
db_path = config.agent_manager.db_path or str(get_config_dir() / "agents.db")
return AgentManager(db_path=db_path)
@@ -273,6 +270,7 @@ def search(agent_id: str, query: str, limit: int) -> None:
console = Console(stderr=True)
try:
from openjarvis.core.config import load_config
from openjarvis.core.paths import get_config_dir
from openjarvis.traces.store import TraceStore
config = load_config()
@@ -281,7 +279,7 @@ def search(agent_id: str, query: str, limit: int) -> None:
if not agent:
console.print(f"[red]Agent not found: {agent_id}[/red]")
return
store = TraceStore(config.traces.db_path or "~/.openjarvis/traces.db")
store = TraceStore(config.traces.db_path or str(get_config_dir() / "traces.db"))
results = store.search(query, agent=agent["name"], limit=limit)
if not results:
console.print("[dim]No results.[/dim]")
@@ -545,8 +543,7 @@ def run_agent(agent_id):
updated = manager.get_agent(agent_id)
runs = updated.get("total_runs", 0)
console.print(
f"[green]✓[/green] Tick complete. "
f"Status: {updated['status']}, runs: {runs}"
f"[green]✓[/green] Tick complete. Status: {updated['status']}, runs: {runs}"
)
# Print the agent's actual output. summary_memory holds the latest tick's
@@ -662,6 +659,7 @@ def trace(agent_id, run_number, limit):
import datetime
from openjarvis.core.config import load_config
from openjarvis.core.paths import get_config_dir
from openjarvis.traces.store import TraceStore
manager = _get_manager()
@@ -671,7 +669,7 @@ def trace(agent_id, run_number, limit):
raise SystemExit(1)
config = load_config()
store = TraceStore(config.traces.db_path or "~/.openjarvis/traces.db")
store = TraceStore(config.traces.db_path or str(get_config_dir() / "traces.db"))
traces = store.list_traces(agent=agent_id, limit=limit)
if not traces:
@@ -841,8 +839,8 @@ def ask(agent_id, message, auto_approve):
if auto_approve:
executor._confirm_callback = lambda _prompt: True
else:
executor._confirm_callback = (
lambda prompt: click.confirm(f"\n{prompt}", default=False)
executor._confirm_callback = lambda prompt: click.confirm(
f"\n{prompt}", default=False
)
# Run the tick with a live trace rather than blocking in silence — the
# message we just queued is consumed as this tick's input, so the user
+30
View File
@@ -276,6 +276,36 @@ def hardware() -> None:
config.add_command(show_group, "show")
@config.command("path")
def show_path() -> None:
"""Print the resolved OpenJarvis directories (home, config, cache).
All OpenJarvis state lives under a single root, resolved in priority
order: ``$OPENJARVIS_HOME`` > ``$XDG_DATA_HOME/openjarvis`` >
``~/.openjarvis``. Use this to confirm where your data is stored after
setting an override.
"""
from openjarvis.core.paths import get_cache_dir, get_config_dir, get_config_path
console = Console(stderr=True)
home = get_config_dir()
override = (
"OPENJARVIS_HOME"
if os.environ.get("OPENJARVIS_HOME")
else "XDG_DATA_HOME"
if os.environ.get("XDG_DATA_HOME")
else "default (~/.openjarvis)"
)
table = Table(show_header=True, header_style="bold")
table.add_column("Directory")
table.add_column("Path", style="cyan")
table.add_row("Home (root)", str(home))
table.add_row("Config file", str(get_config_path()))
table.add_row("Cache", str(get_cache_dir()))
console.print(table)
console.print(f"[dim]Resolved via: {override}[/dim]")
def _probe_engine_host(url: str, console: Console) -> None:
"""Probe an engine host URL and print reachability status."""
try:
+47 -3
View File
@@ -223,6 +223,47 @@ def _check_optional_deps() -> List[CheckResult]:
return results
def _check_speech_backend() -> CheckResult:
"""Check whether the configured speech backend can load."""
try:
from openjarvis.speech._discovery import get_speech_backend
config = _get_config()
backend = get_speech_backend(config)
if backend is None:
return CheckResult(
"Speech backend",
"warn",
"Not configured",
details="Install desktop dependencies with `uv sync --extra desktop`.",
)
if backend.health():
return CheckResult(
"Speech backend",
"ok",
f"{backend.backend_id} ready",
)
details = None
last_error = getattr(backend, "last_error", None)
if callable(last_error):
details = last_error()
return CheckResult(
"Speech backend",
"warn",
f"{backend.backend_id} unavailable",
details=details
or "Install desktop dependencies with `uv sync --extra desktop`.",
)
except Exception as exc:
return CheckResult(
"Speech backend",
"warn",
f"Could not check: {exc}",
)
def _check_security_profile() -> CheckResult:
"""Check if a security profile is configured."""
try:
@@ -306,6 +347,7 @@ def _run_all_checks() -> List[CheckResult]:
checks.extend(_check_models())
checks.append(_check_default_model())
checks.extend(_check_optional_deps())
checks.append(_check_speech_backend())
checks.append(_check_nodejs())
checks.append(_check_security_profile())
return checks
@@ -354,7 +396,9 @@ def doctor(as_json: bool) -> None:
# Background tasks section
from openjarvis.cli._bg_state import get_status
from openjarvis.core.paths import get_config_dir
scripts_dir = get_config_dir() / ".scripts"
console.print("[bold]Background tasks[/bold]")
bg = get_status()
bg_failed = False
@@ -364,8 +408,8 @@ def doctor(as_json: bool) -> None:
elif bg.rust_extension == "failed":
console.print(f" [red]✗[/red] Rust extension: failed — {bg.rust_error[:80]}")
console.print(
" retry: ~/.openjarvis/.scripts/install-rust.sh && "
"~/.openjarvis/.scripts/build-extension.sh"
f" retry: {scripts_dir}/install-rust.sh && "
f"{scripts_dir}/build-extension.sh"
)
bg_failed = True
else:
@@ -380,7 +424,7 @@ def doctor(as_json: bool) -> None:
console.print(f" [green]✓[/green] {model_id}: ready")
elif state == "failed":
console.print(f" [red]✗[/red] {model_id}: failed")
console.print(f" retry: ~/.openjarvis/.scripts/pull-model.sh {model_id}")
console.print(f" retry: {scripts_dir}/pull-model.sh {model_id}")
bg_failed = True
else:
console.print(f" [yellow]…[/yellow] {model_id}: downloading")
+2 -1
View File
@@ -7,6 +7,7 @@ from logging.handlers import RotatingFileHandler
from pathlib import Path
from typing import Optional, Union
from openjarvis.core.paths import get_config_dir
from openjarvis.security.credential_stripper import CredentialStripper
_stripper = CredentialStripper()
@@ -68,7 +69,7 @@ def setup_logging(
if log_file is None:
from openjarvis.security.file_utils import secure_mkdir
log_dir = Path.home() / ".openjarvis"
log_dir = get_config_dir()
secure_mkdir(log_dir)
log_file = log_dir / "cli.log"
file_handler = RotatingFileHandler(
+3 -1
View File
@@ -11,6 +11,8 @@ from typing import Callable, List
import click
from openjarvis.core.paths import get_config_dir
# Engine ports that should only be listening on localhost.
_ENGINE_PORTS = {11434, 8080, 8000, 30000, 1234, 52415, 18181}
@@ -132,7 +134,7 @@ class PrivacyScanner:
def check_icloud_sync(self) -> ScanResult:
"""Check whether ~/.openjarvis is inside iCloud Drive sync scope."""
try:
config_path = Path("~/.openjarvis").expanduser().resolve()
config_path = get_config_dir().resolve()
icloud_path = Path("~/Library/Mobile Documents/").expanduser().resolve()
if str(config_path).startswith(str(icloud_path)):
return ScanResult(
+3 -8
View File
@@ -11,6 +11,7 @@ from rich.console import Console
from openjarvis.cli._banner import print_banner
from openjarvis.core.config import load_config
from openjarvis.core.events import EventBus
from openjarvis.core.paths import get_config_dir
from openjarvis.engine import (
discover_engines,
discover_models,
@@ -496,13 +497,9 @@ def serve(
agent_manager = None
if config.agent_manager.enabled:
try:
from pathlib import Path
from openjarvis.agents.manager import AgentManager
am_db = config.agent_manager.db_path or str(
Path("~/.openjarvis/agents.db").expanduser()
)
am_db = config.agent_manager.db_path or str(get_config_dir() / "agents.db")
# The server owns the scheduler and is the authoritative tick
# runner — on boot it holds no locks, so it (and only it) sweeps
# any zombie running→idle left by a previous crash.
@@ -607,9 +604,7 @@ def serve(
try:
import tomllib
_cfg_path = str(
__import__("pathlib").Path.home() / ".openjarvis" / "config.toml"
)
_cfg_path = str(get_config_dir() / "config.toml")
with open(_cfg_path, "rb") as _f:
_raw = tomllib.load(_f)
api_key = _raw.get("server", {}).get("auth", {}).get("api_key", "")
+8 -5
View File
@@ -11,6 +11,7 @@ from rich.table import Table
from openjarvis.core.config import load_config
from openjarvis.core.events import EventBus
from openjarvis.core.paths import get_config_dir
from openjarvis.skills.manager import SkillManager
@@ -28,12 +29,12 @@ def _get_trace_store():
def _get_discovered_dir() -> Path:
"""Return the directory where discovered skill manifests are written."""
return Path("~/.openjarvis/skills/discovered/").expanduser()
return get_config_dir() / "skills" / "discovered"
def _get_overlay_dir() -> Path:
"""Return the directory where optimization overlays are stored."""
return Path("~/.openjarvis/learning/skills/").expanduser()
return get_config_dir() / "learning" / "skills"
def _get_skill_paths() -> List[Path]:
@@ -41,7 +42,7 @@ def _get_skill_paths() -> List[Path]:
workspace = Path("./skills")
if workspace.exists():
paths.append(workspace)
user_dir = Path("~/.openjarvis/skills/").expanduser()
user_dir = get_config_dir() / "skills"
paths.append(user_dir)
return paths
@@ -174,8 +175,10 @@ def _get_resolver(source: str, url: str = ""):
from openjarvis.skills.sources.github import GitHubResolver
cache = _Path(
"~/.openjarvis/skill-cache/github/" + url.rstrip("/").rsplit("/", 1)[-1]
).expanduser()
str(get_config_dir() / "skill-cache" / "github")
+ "/"
+ url.rstrip("/").rsplit("/", 1)[-1]
)
return GitHubResolver(cache_root=cache, repo_url=url)
raise click.BadParameter(f"Unknown source: {source!r}")
+10 -19
View File
@@ -20,7 +20,6 @@ from openjarvis.connectors.oauth import (
delete_tokens,
load_tokens,
resolve_google_credentials,
run_oauth_flow,
save_tokens,
)
from openjarvis.core.config import DEFAULT_CONFIG_DIR
@@ -290,12 +289,18 @@ class GCalendarConnector(BaseConnector):
"""Handle the OAuth callback.
If *code* looks like a ``client_id:client_secret`` pair (containing
``.apps.googleusercontent.com``), store the credentials and trigger
the full browser-based OAuth flow. Otherwise treat it as a raw
token / auth code.
``.apps.googleusercontent.com``), persist the client credentials only.
The browser consent + codetoken exchange is owned by the in-process
server flow (``/v1/connectors/{id}/oauth/start`` ``/oauth/callback``),
which writes the real ``access_token`` to every Google credential file.
The previous daemon-thread browser flow (its own ``localhost:8789``
callback server) failed silently in the bundled desktop context and is
intentionally removed here (issue #512).
Any other *code* is treated as a raw token / auth code.
"""
code = code.strip()
# If user pastes client_id:client_secret, store and run OAuth flow
if ":" in code and ".apps.googleusercontent.com" in code:
client_id, client_secret = code.split(":", 1)
save_tokens(
@@ -305,20 +310,6 @@ class GCalendarConnector(BaseConnector):
"client_secret": client_secret.strip(),
},
)
import threading
def _run() -> None:
try:
run_oauth_flow(
client_id=client_id.strip(),
client_secret=client_secret.strip(),
scopes=GOOGLE_ALL_SCOPES,
credentials_path=self._credentials_path,
)
except Exception: # noqa: BLE001
pass
threading.Thread(target=_run, daemon=True).start()
else:
# Raw token or auth code
save_tokens(self._credentials_path, {"token": code})
+10 -19
View File
@@ -20,7 +20,6 @@ from openjarvis.connectors.oauth import (
delete_tokens,
load_tokens,
resolve_google_credentials,
run_oauth_flow,
save_tokens,
)
from openjarvis.core.config import DEFAULT_CONFIG_DIR
@@ -195,12 +194,18 @@ class GContactsConnector(BaseConnector):
"""Handle the OAuth callback.
If *code* looks like a ``client_id:client_secret`` pair (containing
``.apps.googleusercontent.com``), store the credentials and trigger
the full browser-based OAuth flow. Otherwise treat it as a raw
token / auth code.
``.apps.googleusercontent.com``), persist the client credentials only.
The browser consent + codetoken exchange is owned by the in-process
server flow (``/v1/connectors/{id}/oauth/start`` ``/oauth/callback``),
which writes the real ``access_token`` to every Google credential file.
The previous daemon-thread browser flow (its own ``localhost:8789``
callback server) failed silently in the bundled desktop context and is
intentionally removed here (issue #512).
Any other *code* is treated as a raw token / auth code.
"""
code = code.strip()
# If user pastes client_id:client_secret, store and run OAuth flow
if ":" in code and ".apps.googleusercontent.com" in code:
client_id, client_secret = code.split(":", 1)
save_tokens(
@@ -210,20 +215,6 @@ class GContactsConnector(BaseConnector):
"client_secret": client_secret.strip(),
},
)
import threading
def _run() -> None:
try:
run_oauth_flow(
client_id=client_id.strip(),
client_secret=client_secret.strip(),
scopes=GOOGLE_ALL_SCOPES,
credentials_path=self._credentials_path,
)
except Exception: # noqa: BLE001
pass
threading.Thread(target=_run, daemon=True).start()
else:
# Raw token or auth code
save_tokens(self._credentials_path, {"token": code})
+15 -21
View File
@@ -20,7 +20,6 @@ from openjarvis.connectors.oauth import (
delete_tokens,
load_tokens,
resolve_google_credentials,
run_oauth_flow,
save_tokens,
)
from openjarvis.core.config import DEFAULT_CONFIG_DIR
@@ -178,15 +177,25 @@ class GDriveConnector(BaseConnector):
"""Handle the OAuth callback.
If *code* looks like a ``client_id:client_secret`` pair (containing
``.apps.googleusercontent.com``), store the credentials and trigger
the full browser-based OAuth flow. Otherwise treat it as a raw
token / auth code.
``.apps.googleusercontent.com``), persist the client credentials only.
The actual browser consent + codetoken exchange is owned by the
in-process server flow (``/v1/connectors/{id}/oauth/start``
``/oauth/callback``), which writes the real ``access_token`` to every
Google credential file.
Previously this spawned a daemon thread that popped a browser and ran
its own ``localhost:8789`` callback server; that thread failed silently
in the bundled desktop context, so the connector never gained an access
token and never appeared in Data Sources (issue #512). The background
flow is intentionally removed here.
Any other *code* is treated as a raw token / auth code.
"""
code = code.strip()
# If user pastes client_id:client_secret, store and run OAuth flow
# A pasted client_id:client_secret pair is the app registration, not a
# completed credential — persist it and let the server flow finish auth.
if ":" in code and ".apps.googleusercontent.com" in code:
client_id, client_secret = code.split(":", 1)
# Save credentials immediately
save_tokens(
self._credentials_path,
{
@@ -194,21 +203,6 @@ class GDriveConnector(BaseConnector):
"client_secret": client_secret.strip(),
},
)
# Run OAuth flow in background thread to avoid blocking
import threading
def _run() -> None:
try:
run_oauth_flow(
client_id=client_id.strip(),
client_secret=client_secret.strip(),
scopes=GOOGLE_ALL_SCOPES,
credentials_path=self._credentials_path,
)
except Exception: # noqa: BLE001
pass
threading.Thread(target=_run, daemon=True).start()
else:
# Raw token or auth code
save_tokens(self._credentials_path, {"token": code})
+59 -23
View File
@@ -16,6 +16,14 @@ from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Dict, List, Optional
from openjarvis.core.paths import (
ConfigurationError,
get_cache_dir,
get_config_dir,
get_config_path,
get_data_dir,
)
if TYPE_CHECKING:
# Only used by type-checkers (mypy/pyright) for the ``JarvisConfig.mining``
# field annotation. The runtime import is deferred inside
@@ -33,15 +41,24 @@ except ModuleNotFoundError:
# Hardware dataclasses
# ---------------------------------------------------------------------------
DEFAULT_CONFIG_DIR = Path.home() / ".openjarvis"
DEFAULT_CONFIG_PATH = DEFAULT_CONFIG_DIR / "config.toml"
# Legacy names, kept for the ~45 modules that import them. They are resolved
# once at import via the env-aware resolver in ``openjarvis.core.paths`` (the
# install-script model: ``OPENJARVIS_HOME`` / ``XDG_DATA_HOME`` are set before
# the process starts). They are real module attributes — not computed lazily —
# so existing tests can ``monkeypatch.setattr`` them and so dataclass-instance
# defaults stay consistent. Code that must react to a mid-process env change
# (or wants the override regardless of import order) should call
# ``get_config_dir()`` / ``get_config_path()`` directly; the dataclass field
# defaults below already do this via ``default_factory``.
DEFAULT_CONFIG_DIR = get_config_dir()
DEFAULT_CONFIG_PATH = get_config_path()
def _ensure_config_dir() -> Path:
"""Ensure the config directory exists with restrictive permissions."""
from openjarvis.security.file_utils import secure_mkdir
return secure_mkdir(DEFAULT_CONFIG_DIR)
return secure_mkdir(get_config_dir())
@dataclass(slots=True)
@@ -742,7 +759,9 @@ class SkillsLearningConfig:
optimizer: str = "dspy" # "dspy" or "gepa"
min_traces_per_skill: int = 20
optimization_interval_seconds: int = 86400
overlay_dir: str = "~/.openjarvis/learning/skills/"
overlay_dir: str = field(
default_factory=lambda: str(get_config_dir() / "learning" / "skills")
)
@dataclass(slots=True)
@@ -894,7 +913,7 @@ class StorageConfig:
"""Storage (memory) backend settings."""
default_backend: str = "sqlite"
db_path: str = str(DEFAULT_CONFIG_DIR / "memory.db")
db_path: str = field(default_factory=lambda: str(get_config_dir() / "memory.db"))
context_top_k: int = 5
context_min_score: float = 0.0
context_max_tokens: int = 2048
@@ -946,8 +965,10 @@ class AgentConfig:
system_prompt_path: str = "" # path to system prompt file (.txt, .md)
context_from_memory: bool = True # inject relevant memory context into prompts
default_system_prompt: str = (
"You are a helpful AI assistant running locally on the user's own "
"hardware through OpenJarvis. You are not a cloud service. Respond "
"You are OpenJarvis, a helpful AI assistant running locally on the "
"user's own hardware. You are not a cloud service, and you are not "
"Claude, ChatGPT, Gemini, or any other branded assistant. If asked "
"who or what you are, identify yourself as OpenJarvis. Respond "
"helpfully, concisely, and accurately."
)
@@ -996,7 +1017,7 @@ class TelemetryConfig:
"""Telemetry persistence settings."""
enabled: bool = True
db_path: str = str(DEFAULT_CONFIG_DIR / "telemetry.db")
db_path: str = field(default_factory=lambda: str(get_config_dir() / "telemetry.db"))
gpu_metrics: bool = False
gpu_poll_interval_ms: int = 50
energy_vendor: str = "" # auto-detect or force "nvidia"/"amd"/"apple"/"cpu_rapl"
@@ -1021,7 +1042,7 @@ class AnalyticsConfig:
enabled: bool = True
host: str = "https://34.231.106.201.sslip.io"
key: str = "phc_ysKu72QaxzYNmDpHFcesD2ZZAe68zkdWJEKoYYkc5e3n"
anon_id_path: str = str(DEFAULT_CONFIG_DIR / "anon_id")
anon_id_path: str = field(default_factory=lambda: str(get_config_dir() / "anon_id"))
flush_interval_seconds: int = 30
flush_at_size: int = 100
@@ -1031,7 +1052,7 @@ class TracesConfig:
"""Trace system settings."""
enabled: bool = True
db_path: str = str(DEFAULT_CONFIG_DIR / "traces.db")
db_path: str = field(default_factory=lambda: str(get_config_dir() / "traces.db"))
@dataclass(slots=True)
@@ -1233,7 +1254,9 @@ class SecurityConfig:
mode: str = "redact" # "redact" | "warn" | "block"
secret_scanner: bool = True
pii_scanner: bool = True
audit_log_path: str = str(DEFAULT_CONFIG_DIR / "audit.db")
audit_log_path: str = field(
default_factory=lambda: str(get_config_dir() / "audit.db")
)
enforce_tool_confirmation: bool = True
merkle_audit: bool = True
signing_key_path: str = ""
@@ -1244,7 +1267,9 @@ class SecurityConfig:
local_engine_bypass: bool = False
local_tool_bypass: bool = False
profile: str = ""
vault_key_path: str = str(DEFAULT_CONFIG_DIR / ".vault_key")
vault_key_path: str = field(
default_factory=lambda: str(get_config_dir() / ".vault_key")
)
capabilities: CapabilitiesConfig = field(default_factory=CapabilitiesConfig)
@@ -1365,7 +1390,7 @@ class SessionConfig:
enabled: bool = False
max_age_hours: float = 24.0
consolidation_threshold: int = 100
db_path: str = str(DEFAULT_CONFIG_DIR / "sessions.db")
db_path: str = field(default_factory=lambda: str(get_config_dir() / "sessions.db"))
@dataclass(slots=True)
@@ -1383,7 +1408,9 @@ class OperatorsConfig:
"""Operator lifecycle settings."""
enabled: bool = False
manifests_dir: str = "~/.openjarvis/operators"
manifests_dir: str = field(
default_factory=lambda: str(get_config_dir() / "operators")
)
auto_activate: str = "" # Comma-separated operator IDs
@@ -1409,7 +1436,7 @@ class OptimizeConfig:
benchmark: str = ""
max_samples: int = 50
judge_model: str = "gpt-5-mini-2025-08-07"
db_path: str = str(DEFAULT_CONFIG_DIR / "optimize.db")
db_path: str = field(default_factory=lambda: str(get_config_dir() / "optimize.db"))
@dataclass(slots=True)
@@ -1417,18 +1444,20 @@ class AgentManagerConfig:
"""Persistent agent manager settings."""
enabled: bool = True
db_path: str = str(DEFAULT_CONFIG_DIR / "agents.db")
db_path: str = field(default_factory=lambda: str(get_config_dir() / "agents.db"))
@dataclass(slots=True)
class MemoryFilesConfig:
"""Persistent memory-file paths and nudge settings."""
soul_path: str = "~/.openjarvis/SOUL.md"
memory_path: str = "~/.openjarvis/MEMORY.md"
user_path: str = "~/.openjarvis/USER.md"
soul_path: str = field(default_factory=lambda: str(get_config_dir() / "SOUL.md"))
memory_path: str = field(
default_factory=lambda: str(get_config_dir() / "MEMORY.md")
)
user_path: str = field(default_factory=lambda: str(get_config_dir() / "USER.md"))
nudge_interval: int = 10
persona_name: str = "" # named persona dir under ~/.openjarvis/personas/<name>/
persona_name: str = "" # named persona dir under <config-dir>/personas/<name>/
@dataclass(slots=True)
@@ -1467,13 +1496,15 @@ class SkillsConfig:
"""Configuration for agent-authored procedural skills."""
enabled: bool = True
skills_dir: str = "~/.openjarvis/skills/"
skills_dir: str = field(default_factory=lambda: str(get_config_dir() / "skills"))
active: str = "*"
auto_discover: bool = True
auto_sync: bool = False
nudge_interval: int = 15
index_repo: str = "https://github.com/openjarvis/skill-index.git"
index_dir: str = "~/.openjarvis/skill-index/"
index_dir: str = field(
default_factory=lambda: str(get_config_dir() / "skill-index")
)
max_depth: int = 5
sandbox_dangerous: bool = True
sources: List[SkillSourceConfig] = field(default_factory=list)
@@ -1779,7 +1810,7 @@ def load_config(path: Optional[Path] = None) -> JarvisConfig:
elif os.environ.get("OPENJARVIS_CONFIG"):
config_path = Path(os.environ["OPENJARVIS_CONFIG"]).expanduser().resolve()
else:
config_path = DEFAULT_CONFIG_PATH
config_path = get_config_path()
if config_path.exists():
with open(config_path, "rb") as fh:
data = tomllib.load(fh)
@@ -2117,9 +2148,14 @@ __all__ = [
"BrowserConfig",
"CapabilitiesConfig",
"ChannelConfig",
"ConfigurationError",
"DEFAULT_CONFIG_DIR",
"DEFAULT_CONFIG_PATH",
"DiscordChannelConfig",
"get_cache_dir",
"get_config_dir",
"get_config_path",
"get_data_dir",
"EmailChannelConfig",
"EngineConfig",
"FeishuChannelConfig",
+10 -3
View File
@@ -10,13 +10,20 @@ import os
import threading
from pathlib import Path
from openjarvis.core.paths import get_config_dir
try:
import tomllib
except ModuleNotFoundError:
import tomli as tomllib # type: ignore[no-redef]
_LOCK = threading.Lock()
_DEFAULT_PATH = Path.home() / ".openjarvis" / "credentials.toml"
def _default_path() -> Path:
"""Resolve the credentials file under the OpenJarvis root (env-aware)."""
return get_config_dir() / "credentials.toml"
TOOL_CREDENTIALS: dict[str, list[str]] = {
"web_search": ["TAVILY_API_KEY"],
@@ -53,7 +60,7 @@ TOOL_CREDENTIALS: dict[str, list[str]] = {
def load_credentials(path: Path | None = None) -> dict[str, dict[str, str]]:
"""Load credentials from TOML file."""
p = Path(path) if path else _DEFAULT_PATH
p = Path(path) if path else _default_path()
if not p.exists():
return {}
with open(p, "rb") as f:
@@ -75,7 +82,7 @@ def save_credential(
if not stripped:
raise ValueError("Credential value must not be empty")
p = Path(path) if path else _DEFAULT_PATH
p = Path(path) if path else _default_path()
with _LOCK:
creds = load_credentials(path=p)
if tool_name not in creds:
+122
View File
@@ -0,0 +1,122 @@
"""Central, env-aware resolution of OpenJarvis' home directory.
OpenJarvis keeps all of its runtime state (config, databases, caches, logs,
credentials, skills, recipes, ) under a single root so it never clutters the
user's home directory beyond one directory. That root is resolved here, with
the following precedence (highest first):
1. ``$OPENJARVIS_HOME`` explicit override (also honored by the shell
installer, see ``scripts/install/install.sh``).
2. ``$XDG_DATA_HOME/openjarvis`` when ``$XDG_DATA_HOME`` is set, follow the
XDG Base Directory spec by nesting a single ``openjarvis`` directory under
it. We deliberately use ONE directory rather than splitting across XDG
config/data/cache so the install tree stays self-contained and relocatable.
3. ``~/.openjarvis`` the historical default. With no env vars set, the
resolved path is exactly this, so existing installs are untouched.
``config.py`` re-exports :func:`get_config_dir` results through the legacy
``DEFAULT_CONFIG_DIR``/``DEFAULT_CONFIG_PATH`` names (computed dynamically) so
the ~45 modules that import those names keep working while honoring the
override. Modules that previously hardcoded ``Path.home() / ".openjarvis"``
should call :func:`get_config_dir` (or :func:`get_data_dir` /
:func:`get_cache_dir`) instead.
Defense in depth: the resolved root must never live inside the OpenJarvis
source tree (a misconfigured ``$OPENJARVIS_HOME`` pointing at the repo would
otherwise scatter runtime artifacts into the working tree). This mirrors the
guard in ``learning/spec_search/storage/paths.py`` and fails loudly per
REVIEW.md's no-silent-failure discipline.
"""
from __future__ import annotations
import os
from pathlib import Path
_DEFAULT_DIR_NAME = ".openjarvis"
_XDG_SUBDIR_NAME = "openjarvis"
class ConfigurationError(RuntimeError):
"""Raised when the resolved home directory would violate isolation guarantees."""
def _find_source_root() -> Path | None:
"""Walk upward from this module to find the OpenJarvis source root.
Returns the directory containing the OpenJarvis ``pyproject.toml`` (the one
whose ``name = "openjarvis"``), or ``None`` when running from an installed
wheel rather than a source checkout.
"""
here = Path(__file__).resolve()
for candidate in (here, *here.parents):
py = candidate / "pyproject.toml"
if py.exists():
try:
content = py.read_text(encoding="utf-8")
except OSError:
continue
if 'name = "openjarvis"' in content.lower():
return candidate
return None
def _reject_source_tree(path: Path) -> Path:
"""Raise if ``path`` resolves inside the OpenJarvis source tree."""
source_root = _find_source_root()
if source_root is not None:
try:
path.relative_to(source_root)
except ValueError:
pass # Good — not inside the source tree.
else:
raise ConfigurationError(
f"OpenJarvis home ({path}) is inside the source tree "
f"({source_root}). OpenJarvis refuses to write runtime state "
"inside its own repo. Set OPENJARVIS_HOME (or XDG_DATA_HOME) "
"to a directory outside the repo (default: ~/.openjarvis)."
)
return path
def get_config_dir() -> Path:
"""Resolve OpenJarvis' single root directory, honoring env overrides.
Precedence: ``$OPENJARVIS_HOME`` > ``$XDG_DATA_HOME/openjarvis`` >
``~/.openjarvis``. The result is always absolute and is rejected if it
falls inside the OpenJarvis source tree.
"""
env_home = os.environ.get("OPENJARVIS_HOME")
if env_home:
resolved = Path(env_home).expanduser().resolve()
return _reject_source_tree(resolved)
xdg_data = os.environ.get("XDG_DATA_HOME")
if xdg_data:
resolved = (Path(xdg_data).expanduser() / _XDG_SUBDIR_NAME).resolve()
return _reject_source_tree(resolved)
return (Path.home() / _DEFAULT_DIR_NAME).resolve()
def get_config_path() -> Path:
"""Resolve the path to ``config.toml`` under the OpenJarvis root."""
return get_config_dir() / "config.toml"
def get_data_dir() -> Path:
"""Resolve the directory for persistent data (databases, blobs, …).
Consolidated under the single root; identical to :func:`get_config_dir`.
Provided as a distinct name so call sites read intentionally.
"""
return get_config_dir()
def get_cache_dir() -> Path:
"""Resolve the directory for regenerable caches (eval datasets, etc.).
Lives at ``<root>/cache`` so caches stay inside the single OpenJarvis
directory instead of scattering across ``~/.cache``.
"""
return get_config_dir() / "cache"
+166 -4
View File
@@ -1,4 +1,7 @@
"""Cloud inference engine — OpenAI, Anthropic, Google, and MiniMax API backends."""
"""Cloud inference engine.
OpenAI, Anthropic, Google, MiniMax, and DeepSeek API backends.
"""
from __future__ import annotations
@@ -48,6 +51,8 @@ PRICING: Dict[str, tuple[float, float]] = {
"MiniMax-M2.7-highspeed": (0.60, 2.40),
"MiniMax-M2.5": (0.30, 1.20),
"MiniMax-M2.5-highspeed": (0.60, 2.40),
"deepseek-v4-flash": (0.27, 1.10),
"deepseek-v4-pro": (0.55, 2.19),
}
# Well-known model IDs per provider
@@ -83,6 +88,10 @@ _MINIMAX_MODELS = [
"MiniMax-M2.5",
"MiniMax-M2.5-highspeed",
]
_DEEPSEEK_MODELS = [
"deepseek-v4-flash",
"deepseek-v4-pro",
]
# OpenRouter models — prefixed with "openrouter/" so they can be identified
_OPENROUTER_POPULAR = [
@@ -111,6 +120,10 @@ def _is_minimax_model(model: str) -> bool:
return model.lower().startswith("minimax")
def _is_deepseek_model(model: str) -> bool:
return model.lower().startswith("deepseek")
def _is_openrouter_model(model: str) -> bool:
return model.startswith("openrouter/")
@@ -127,6 +140,35 @@ def _is_google_model(model: str) -> bool:
return "gemini" in model.lower() and not _is_openrouter_model(model)
# Positive prefix predicate for genuine OpenAI models. Kept in sync with
# ``server/cloud_router.py:_OPENAI_PREFIXES`` so local-vs-cloud classification
# agrees across the codebase. Used by ``_client_for_model``/``can_serve`` so the
# cloud engine never claims it can serve an unrecognized (e.g. local Ollama)
# model name just because an OpenAI key happens to be present (see #335).
_OPENAI_PREFIXES = ("gpt-", "chatgpt-", "o1", "o3", "o4")
def _is_openai_model(model: str) -> bool:
"""True only for genuine OpenAI models (gpt-*, chatgpt-*, o1/o3/o4 series).
Defined positively so that an unrecognized model name (a local Ollama model
like ``qwen3.5:0.8b``, or a typo) is NOT treated as an OpenAI model. This is
the routing surface ``can_serve`` relies on; ``generate``/``stream`` keep
their OpenAI fall-through so an explicitly-requested unknown cloud model
still errors loudly at call time.
Caveat: a user may repoint the OpenAI client at an OpenAI-compatible server
(vLLM/LM Studio) via ``OPENAI_BASE_URL`` and legitimately serve non-gpt
names. That path is undocumented/untested in this engine; if it is added,
this predicate (or ``_client_for_model``) should treat a configured custom
base_url as "serves anything".
"""
m = model.lower()
if m in (name.lower() for name in _OPENAI_MODELS):
return True
return m.startswith(_OPENAI_PREFIXES)
def _is_openai_reasoning_model(model: str) -> bool:
"""Check if model is an OpenAI reasoning model that restricts temperature."""
m = model.lower()
@@ -269,7 +311,7 @@ def _convert_tools_to_google(
@EngineRegistry.register("cloud")
class CloudEngine(InferenceEngine):
"""Cloud inference via OpenAI, Anthropic, Google, and MiniMax SDKs."""
"""Cloud inference via OpenAI, Anthropic, Google, MiniMax, and DeepSeek SDKs."""
engine_id = "cloud"
is_cloud = True
@@ -280,6 +322,7 @@ class CloudEngine(InferenceEngine):
self._google_client: Any = None
self._openrouter_client: Any = None
self._minimax_client: Any = None
self._deepseek_client: Any = None
self._codex_client: Any = None
# Gemini thought_signatures: tool_call_id -> signature bytes
self._thought_sigs: Dict[str, bytes] = {}
@@ -332,6 +375,17 @@ class CloudEngine(InferenceEngine):
)
except ImportError:
pass
deepseek_key = os.environ.get("DEEPSEEK_API_KEY")
if deepseek_key:
try:
import openai
self._deepseek_client = openai.OpenAI(
base_url="https://api.deepseek.com/v1",
api_key=deepseek_key,
)
except ImportError:
pass
# Codex — uses the OpenAI Responses API.
# Supports both standard API keys (api.openai.com) and ChatGPT
# OAuth tokens (chatgpt.com) via OPENAI_CODEX_BASE_URL override.
@@ -985,6 +1039,56 @@ class CloudEngine(InferenceEngine):
]
return result
def _generate_deepseek(
self,
messages: Sequence[Message],
*,
model: str,
temperature: float,
max_tokens: int,
**kwargs: Any,
) -> Dict[str, Any]:
if self._deepseek_client is None:
raise EngineConnectionError(
"DeepSeek client not available — set DEEPSEEK_API_KEY"
)
kwargs.pop("response_format", None)
create_kwargs: Dict[str, Any] = {
"model": model,
"messages": messages_to_dicts(messages),
"max_tokens": max_tokens,
"temperature": temperature,
}
t0 = time.monotonic()
resp = self._deepseek_client.chat.completions.create(**create_kwargs)
elapsed = time.monotonic() - t0
choice = resp.choices[0]
usage = resp.usage
prompt_tokens = usage.prompt_tokens if usage else 0
completion_tokens = usage.completion_tokens if usage else 0
result: Dict[str, Any] = {
"content": choice.message.content or "",
"usage": {
"prompt_tokens": prompt_tokens,
"completion_tokens": completion_tokens,
"total_tokens": (usage.total_tokens if usage else 0),
},
"model": resp.model,
"finish_reason": choice.finish_reason or "stop",
"cost_usd": estimate_cost(model, prompt_tokens, completion_tokens),
"ttft": elapsed,
}
if hasattr(choice.message, "tool_calls") and choice.message.tool_calls:
result["tool_calls"] = [
{
"id": tc.id,
"name": tc.function.name,
"arguments": tc.function.arguments,
}
for tc in choice.message.tool_calls
]
return result
def generate(
self,
messages: Sequence[Message],
@@ -1006,6 +1110,8 @@ class CloudEngine(InferenceEngine):
return self._generate_openrouter(messages, **kw)
if _is_minimax_model(model):
return self._generate_minimax(messages, **kw)
if _is_deepseek_model(model):
return self._generate_deepseek(messages, **kw)
if _is_anthropic_model(model):
return self._generate_anthropic(messages, **kw)
if _is_google_model(model):
@@ -1036,6 +1142,9 @@ class CloudEngine(InferenceEngine):
elif _is_minimax_model(model):
async for token in self._stream_minimax(messages, **kw):
yield token
elif _is_deepseek_model(model):
async for token in self._stream_deepseek(messages, **kw):
yield token
elif _is_anthropic_model(model):
async for token in self._stream_anthropic(messages, **kw):
yield token
@@ -1254,6 +1363,30 @@ class CloudEngine(InferenceEngine):
if delta and delta.content:
yield delta.content
async def _stream_deepseek(
self,
messages: Sequence[Message],
*,
model: str,
temperature: float,
max_tokens: int,
**kwargs: Any,
) -> AsyncIterator[str]:
if self._deepseek_client is None:
raise EngineConnectionError("DeepSeek client not available")
create_kwargs: Dict[str, Any] = {
"model": model,
"messages": messages_to_dicts(messages),
"max_tokens": max_tokens,
"temperature": temperature,
"stream": True,
}
resp = self._deepseek_client.chat.completions.create(**create_kwargs)
for chunk in resp:
delta = chunk.choices[0].delta if chunk.choices else None
if delta and delta.content:
yield delta.content
# -- stream_full: rich streaming with tool_calls support ----------------
async def _stream_full_openai(
@@ -1307,6 +1440,18 @@ class CloudEngine(InferenceEngine):
"stream": True,
**kwargs,
}
elif _is_deepseek_model(model):
client = self._deepseek_client
if client is None:
raise EngineConnectionError("DeepSeek client not available")
create_kwargs = {
"model": model,
"messages": messages_to_dicts(messages),
"max_tokens": max_tokens,
"temperature": temperature,
"stream": True,
**kwargs,
}
else:
client = self._openai_client
if client is None:
@@ -1473,24 +1618,40 @@ class CloudEngine(InferenceEngine):
models.extend(_OPENROUTER_POPULAR)
if self._minimax_client is not None:
models.extend(_MINIMAX_MODELS)
if self._deepseek_client is not None:
models.extend(_DEEPSEEK_MODELS)
if self._codex_client is not None:
models.extend(_CODEX_MODELS)
return models
def _client_for_model(self, model: str) -> Any:
"""Return the provider client ``generate``/``stream`` will dispatch to
for *model* (mirrors the routing in those methods)."""
for *model*, or ``None`` for a model this engine cannot route.
Mirrors the routing in ``generate``/``stream``, but is intentionally
*stricter* on the OpenAI fall-through: only genuine OpenAI models map to
the OpenAI client. Unrecognized names (e.g. a local Ollama model like
``qwen3.5:0.8b``) return ``None`` so ``can_serve`` declines them and the
cloud engine is not mis-selected as a fallback when the local engine is
transiently down and any (even dummy) ``OPENAI_API_KEY`` is set (#335).
``generate``/``stream`` keep their OpenAI fall-through, so an
explicitly-requested unknown cloud model still fails loudly at call time.
"""
if _is_codex_model(model):
return self._codex_client
if _is_openrouter_model(model):
return self._openrouter_client
if _is_minimax_model(model):
return self._minimax_client
if _is_deepseek_model(model):
return self._deepseek_client
if _is_anthropic_model(model):
return self._anthropic_client
if _is_google_model(model):
return self._google_client
return self._openai_client
if _is_openai_model(model):
return self._openai_client
return None
def can_serve(self, model: str) -> bool:
"""Return ``True`` only if the provider client for *model* exists.
@@ -1512,6 +1673,7 @@ class CloudEngine(InferenceEngine):
or self._google_client is not None
or self._openrouter_client is not None
or self._minimax_client is not None
or self._deepseek_client is not None
or self._codex_client is not None
)
+67 -9
View File
@@ -22,6 +22,49 @@ from openjarvis.engine._stubs import StreamChunk
logger = logging.getLogger(__name__)
# Qwen3 treats ``/think`` and ``/no_think`` as soft-switch control tokens that
# toggle reasoning mode. Small models (e.g. qwen3:14b) fed a multi-line prompt
# sometimes emit one of these as the sole tool argument, e.g.
# ``{"command": "/no_think"}`` instead of the real command. Ollama parses that
# into a fully-formed tool_call via the model's chat template, so we have to
# drop it on our side before the agent executes garbage.
_QWEN_CONTROL_TOKENS = frozenset({"/think", "/no_think"})
def _is_control_token_only_args(raw_args: Any) -> bool:
"""Return True if tool-call arguments contain nothing but a Qwen3 token.
``raw_args`` may be a dict (Ollama's native shape) or a JSON / bare string.
A call is considered degenerate only when it carries at least one control
token and no other usable content, so legitimate calls such as
``{"command": "date"}`` or ``{"command": "echo /no_think"}`` are kept.
"""
parsed: Any = raw_args
if isinstance(raw_args, str):
try:
parsed = json.loads(raw_args)
except (json.JSONDecodeError, TypeError):
parsed = raw_args
if isinstance(parsed, str):
return parsed.strip().lower() in _QWEN_CONTROL_TOKENS
if not isinstance(parsed, dict) or not parsed:
return False
saw_token = False
for value in parsed.values():
if not isinstance(value, str):
return False # a non-string value is real content
stripped = value.strip()
if not stripped:
continue
if stripped.lower() in _QWEN_CONTROL_TOKENS:
saw_token = True
else:
return False # real string content
return saw_token
def _default_num_ctx() -> int:
"""Default context window (tokens). Override with ``JARVIS_NUM_CTX``.
@@ -168,14 +211,19 @@ class OllamaEngine(InferenceEngine):
if raw_tool_calls:
tool_calls = []
for i, tc in enumerate(raw_tool_calls):
raw_args = tc.get("function", {}).get(
"arguments",
"{}",
)
fn = tc.get("function", {})
raw_args = fn.get("arguments", "{}")
if _is_control_token_only_args(raw_args):
logger.warning(
"Dropping Qwen3 control-token tool call %s(%r)",
fn.get("name", ""),
raw_args,
)
continue
tool_calls.append(
{
"id": tc.get("id", f"call_{i}"),
"name": tc.get("function", {}).get("name", ""),
"name": fn.get("name", ""),
"arguments": (
json.dumps(raw_args)
if isinstance(raw_args, dict)
@@ -183,7 +231,8 @@ class OllamaEngine(InferenceEngine):
),
}
)
result["tool_calls"] = tool_calls
if tool_calls:
result["tool_calls"] = tool_calls
return result
async def stream(
@@ -340,14 +389,22 @@ class OllamaEngine(InferenceEngine):
# OpenAI-delta fragment shape that agent_manager_routes
# expects in _merge_tool_call_fragments.
fragments: List[Dict[str, Any]] = []
for i, tc in enumerate(raw_tool_calls):
for tc in raw_tool_calls:
fn = tc.get("function", {}) or {}
raw_args = fn.get("arguments", "{}")
if _is_control_token_only_args(raw_args):
logger.warning(
"Dropping Qwen3 control-token tool call %s(%r)",
fn.get("name", ""),
raw_args,
)
continue
args_str = (
json.dumps(raw_args)
if isinstance(raw_args, dict)
else str(raw_args)
)
i = len(fragments)
fragments.append(
{
"index": i,
@@ -359,8 +416,9 @@ class OllamaEngine(InferenceEngine):
},
}
)
yield StreamChunk(tool_calls=fragments)
finish_reason = "tool_calls"
if fragments:
yield StreamChunk(tool_calls=fragments)
finish_reason = "tool_calls"
if chunk.get("done", False):
reported_prompt = chunk.get("prompt_eval_count", 0)
+2 -1
View File
@@ -11,11 +11,12 @@ import shutil
from pathlib import Path
from typing import Iterable, List, MutableMapping, Optional, Sequence
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.splits import apply_split
from openjarvis.evals.core.types import EvalRecord
_DEFAULT_CACHE_DIR = Path.home() / ".cache" / "gaia_benchmark"
_DEFAULT_CACHE_DIR = get_cache_dir() / "gaia_benchmark"
_DEFAULT_INPUT_PROMPT = """Please answer the question below. You should:
@@ -17,6 +17,7 @@ import subprocess
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.splits import apply_split
from openjarvis.evals.core.types import EvalRecord
@@ -24,7 +25,7 @@ from openjarvis.evals.core.types import EvalRecord
LOGGER = logging.getLogger(__name__)
LIVERESEARCH_REPO = "https://github.com/Ayanami0730/deep_research_bench.git"
CACHE_DIR = Path.home() / ".cache" / "liveresearch_bench"
CACHE_DIR = get_cache_dir() / "liveresearch_bench"
def _load_jsonl(path: Path) -> List[Dict[str, Any]]:
+2 -3
View File
@@ -13,6 +13,7 @@ import re
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.types import EvalRecord
@@ -64,9 +65,7 @@ class LogHubDataset(DatasetProvider):
f"Choose from: {list(_DATASETS.keys())}"
)
self._subset = subset
self._cache_dir = (
Path(cache_dir) if cache_dir else Path.home() / ".cache" / "loghub"
)
self._cache_dir = Path(cache_dir) if cache_dir else get_cache_dir() / "loghub"
self._records: List[EvalRecord] = []
def load(
+2 -1
View File
@@ -15,6 +15,7 @@ import random
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.types import EvalRecord
@@ -41,7 +42,7 @@ class PaperArenaDataset(DatasetProvider):
cache_dir: Optional[str] = None,
) -> None:
self._cache_dir = (
Path(cache_dir) if cache_dir else Path.home() / ".cache" / "paperarena"
Path(cache_dir) if cache_dir else get_cache_dir() / "paperarena"
)
self._records: List[EvalRecord] = []
+2 -1
View File
@@ -18,6 +18,7 @@ from typing import Any, Dict, Iterable, List, Optional
import yaml
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.splits import apply_split
from openjarvis.evals.core.types import EvalRecord
@@ -25,7 +26,7 @@ from openjarvis.evals.core.types import EvalRecord
LOGGER = logging.getLogger(__name__)
PINCHBENCH_REPO = "https://github.com/pinchbench/skill.git"
CACHE_DIR = Path.home() / ".cache" / "pinchbench"
CACHE_DIR = get_cache_dir() / "pinchbench"
def _parse_task_markdown(content: str, filename: str = "") -> Dict[str, Any]:
+2 -2
View File
@@ -12,9 +12,9 @@ import logging
import os
import subprocess
import sys
from pathlib import Path
from typing import Iterable, List, Optional
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.splits import apply_split
from openjarvis.evals.core.types import EvalRecord
@@ -22,7 +22,7 @@ from openjarvis.evals.core.types import EvalRecord
LOGGER = logging.getLogger(__name__)
TAU2_REPO = "https://github.com/sierra-research/tau2-bench.git"
CACHE_DIR = Path.home() / ".cache" / "tau2-bench"
CACHE_DIR = get_cache_dir() / "tau2-bench"
DOMAINS = ("airline", "retail", "telecom")
@@ -21,6 +21,7 @@ import subprocess
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.dataset import DatasetProvider
from openjarvis.evals.core.types import EvalRecord
@@ -56,7 +57,7 @@ class WebChoreArenaDataset(DatasetProvider):
) -> None:
self._subset = subset # "all", "small", or a site name
self._cache_dir = (
Path(cache_dir) if cache_dir else Path.home() / ".cache" / "webchorearena"
Path(cache_dir) if cache_dir else get_cache_dir() / "webchorearena"
)
self._headless = headless
self._records: List[EvalRecord] = []
@@ -49,6 +49,7 @@ import tempfile
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
from openjarvis.core.paths import get_cache_dir
from openjarvis.evals.core.scorer import Scorer
from openjarvis.evals.core.types import EvalRecord
@@ -93,9 +94,7 @@ def _run_subprocess_hard_timeout(
)
try:
stdout, stderr = proc.communicate(timeout=timeout_s)
return subprocess.CompletedProcess(
cmd, proc.returncode, stdout, stderr
)
return subprocess.CompletedProcess(cmd, proc.returncode, stdout, stderr)
except subprocess.TimeoutExpired:
# Kill the whole group, not just the direct child — Modal harness
# subprocesses fork workers that would otherwise keep pipes open.
@@ -115,9 +114,7 @@ def _run_subprocess_hard_timeout(
stdout, stderr = proc.communicate(timeout=10)
except subprocess.TimeoutExpired:
stdout, stderr = "", ""
raise subprocess.TimeoutExpired(
cmd, timeout_s, output=stdout, stderr=stderr
)
raise subprocess.TimeoutExpired(cmd, timeout_s, output=stdout, stderr=stderr)
# ---------- Patch tracking ----------
@@ -208,11 +205,11 @@ def _patch_modal_sandbox_source() -> None:
# Upstream changed the line — bail rather than apply blindly.
return
replacement = (
' # ' + _CGROUP_SOURCE_SENTINEL + '\n'
' try:\n'
" # " + _CGROUP_SOURCE_SENTINEL + "\n"
" try:\n"
' self.write_file("/sys/fs/cgroup/cpu/cpu.shares", "2048")\n'
' except FileNotFoundError:\n'
' pass # cgroup v2 Modal sandbox — path missing is fine\n'
" except FileNotFoundError:\n"
" pass # cgroup v2 Modal sandbox — path missing is fine\n"
)
new_src = src.replace(needle + "\n", replacement, 1)
try:
@@ -324,22 +321,23 @@ def extract_patch(text: str) -> Optional[str]:
# ---------- Harness invocation ----------
def _harness_cache_dir() -> Path:
"""Where the swebench subprocess writes its report JSON + logs/ tree.
Defaults to ``$OPENJARVIS_HOME/.swebench-cache`` if set, otherwise to a
process-shared tempdir. Pin both so we don't pollute the project root.
Consolidated under the env-aware OpenJarvis cache root
(``<openjarvis-home>/cache/swebench``) so it never pollutes the project
root or scatters across ``$HOME``. Honors ``OPENJARVIS_HOME`` /
``XDG_DATA_HOME`` via :func:`openjarvis.core.paths.get_cache_dir`.
"""
home = os.environ.get("OPENJARVIS_HOME")
if home:
cache = Path(home) / ".swebench-cache"
else:
cache = Path(tempfile.gettempdir()) / "openjarvis-swebench-cache"
cache = get_cache_dir() / "swebench"
cache.mkdir(parents=True, exist_ok=True)
return cache
def _find_report(cache: Path, instance_id: str, run_id: str) -> Optional[Dict[str, Any]]:
def _find_report(
cache: Path, instance_id: str, run_id: str
) -> Optional[Dict[str, Any]]:
"""Find the harness's report JSON for one instance.
swebench writes ``<model_name_or_path>.<run_id>.json`` inside the
@@ -427,26 +425,40 @@ def _run_harness(
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
preds_path = tmp_path / "predictions.jsonl"
preds_path.write_text(json.dumps({
"instance_id": instance_id,
"model_name_or_path": "openjarvis-harness",
"model_patch": patch,
}) + "\n")
preds_path.write_text(
json.dumps(
{
"instance_id": instance_id,
"model_name_or_path": "openjarvis-harness",
"model_patch": patch,
}
)
+ "\n"
)
cmd = [
sys.executable, "-m", "swebench.harness.run_evaluation",
"--predictions_path", str(preds_path),
"--max_workers", "1",
"--run_id", run_id,
"--dataset_name", "SWE-bench/SWE-bench_Verified",
"--instance_ids", instance_id,
sys.executable,
"-m",
"swebench.harness.run_evaluation",
"--predictions_path",
str(preds_path),
"--max_workers",
"1",
"--run_id",
run_id,
"--dataset_name",
"SWE-bench/SWE-bench_Verified",
"--instance_ids",
instance_id,
]
if backend == "modal":
cmd += ["--modal", "true"]
try:
proc = _run_subprocess_hard_timeout(
cmd, timeout_s=timeout_s, cwd=str(cache),
cmd,
timeout_s=timeout_s,
cwd=str(cache),
)
except subprocess.TimeoutExpired as exc:
# The harness subprocess (and its Modal grandchildren) exceeded
@@ -497,6 +509,7 @@ def _run_harness(
# ---------- Scorer ----------
class SWEBenchHarnessScorer(Scorer):
"""SWE-bench Verified scorer that runs the official harness.
@@ -516,7 +529,7 @@ class SWEBenchHarnessScorer(Scorer):
timeout_s: int = 1800,
cell_name: Optional[str] = None,
judge_backend: object = None, # noqa: ARG002 — CLI factory compat
judge_model: str = "", # noqa: ARG002 — CLI factory compat
judge_model: str = "", # noqa: ARG002 — CLI factory compat
) -> None:
self._timeout_s = int(timeout_s)
# ``cell_name`` namespaces the ``run_id`` so concurrent cells scoring
@@ -538,16 +551,15 @@ class SWEBenchHarnessScorer(Scorer):
if patch is None:
return False, {"reason": "no_patch_extracted"}
instance_id = (
record.metadata.get("instance_id")
or record.record_id
or ""
)
instance_id = record.metadata.get("instance_id") or record.record_id or ""
if not instance_id:
return False, {"reason": "missing_instance_id"}
result = _run_harness(
instance_id, patch, self._timeout_s, cell_name=self._cell_name,
instance_id,
patch,
self._timeout_s,
cell_name=self._cell_name,
)
details = dict(result.get("details", {}))
details["patch"] = patch
+5 -5
View File
@@ -16,6 +16,8 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
from openjarvis.core.paths import get_config_dir
LOGGER = logging.getLogger(__name__)
CONDITIONS = (
@@ -52,14 +54,12 @@ class SkillBenchmarkConfig:
seeds: List[int] = field(default_factory=lambda: [42, 43, 44])
max_samples: Optional[int] = None
output_dir: Path = field(default_factory=lambda: Path("docs/superpowers/results/"))
skills_dir: Path = field(
default_factory=lambda: Path("~/.openjarvis/skills/").expanduser()
)
skills_dir: Path = field(default_factory=lambda: get_config_dir() / "skills")
overlay_dir_dspy: Path = field(
default_factory=lambda: Path("~/.openjarvis/learning/skills-dspy/").expanduser()
default_factory=lambda: get_config_dir() / "learning" / "skills-dspy"
)
overlay_dir_gepa: Path = field(
default_factory=lambda: Path("~/.openjarvis/learning/skills-gepa/").expanduser()
default_factory=lambda: get_config_dir() / "learning" / "skills-gepa"
)
@@ -30,6 +30,7 @@ from pathlib import Path
from typing import Any, Dict, List
from openjarvis.core.config import ACEOptimizerConfig
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import LearningRegistry
from openjarvis.learning._stubs import AgentLearningPolicy
@@ -48,7 +49,7 @@ except ImportError:
def _default_save_dir(task_name: str) -> Path:
return Path.home() / ".openjarvis" / "learning" / "ace" / task_name
return get_config_dir() / "learning" / "ace" / task_name
class _TraceDataProcessor:
@@ -87,9 +88,7 @@ class _TraceDataProcessor:
if not predictions:
return 0.0
n_correct = sum(
1
for p, g in zip(predictions, ground_truths)
if cls.answer_is_correct(p, g)
1 for p, g in zip(predictions, ground_truths) if cls.answer_is_correct(p, g)
)
return n_correct / len(predictions)
@@ -149,8 +148,7 @@ class ACEAgentOptimizer:
return {
"status": "skipped",
"reason": (
f"only {len(traces)} traces, "
f"min_traces={self.config.min_traces}"
f"only {len(traces)} traces, min_traces={self.config.min_traces}"
),
}
@@ -158,8 +156,7 @@ class ACEAgentOptimizer:
return {
"status": "error",
"reason": (
"ace not installed (pip install "
"'openjarvis[learning-ace]')"
"ace not installed (pip install 'openjarvis[learning-ace]')"
),
}
@@ -19,6 +19,7 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
from openjarvis.core.paths import get_config_dir
from openjarvis.core.types import Trace, TraceStep
from openjarvis.skills.manager import SkillManager
from openjarvis.skills.overlay import SkillOverlay, write_overlay
@@ -99,7 +100,7 @@ class SkillOptimizer:
pass
if overlay_dir is None:
overlay_dir = Path(
"~/.openjarvis/learning/skills/"
str(get_config_dir() / "learning" / "skills")
).expanduser()
overlay_dir = Path(overlay_dir).expanduser()
@@ -11,14 +11,21 @@ writing artifacts into the working tree.
from __future__ import annotations
import os
from pathlib import Path
from openjarvis.core.paths import ConfigurationError, get_config_dir
from openjarvis.security.file_utils import secure_mkdir
class ConfigurationError(RuntimeError):
"""Raised when path configuration would violate isolation guarantees."""
# ``ConfigurationError`` is re-exported from ``openjarvis.core.paths`` (it used
# to be defined here). Spec search now resolves the home dir through the unified
# core resolver, which raises the same exception type on a source-tree path, so
# we alias rather than redefine to keep ``except ConfigurationError`` callers and
# existing tests working.
__all__ = [
"ConfigurationError",
"ensure_spec_search_dirs",
"resolve_spec_search_root",
]
def _find_source_root() -> Path | None:
@@ -42,11 +49,12 @@ def _find_source_root() -> Path | None:
def _resolve_openjarvis_home() -> Path:
"""Resolve the OPENJARVIS_HOME directory (env var or default)."""
env = os.environ.get("OPENJARVIS_HOME")
if env:
return Path(env).expanduser().resolve()
return (Path.home() / ".openjarvis").resolve()
"""Resolve the OpenJarvis home directory via the unified core resolver.
Delegates to ``get_config_dir`` so spec-search honors the same env-aware
resolution (OPENJARVIS_HOME and XDG) as the rest of the framework.
"""
return get_config_dir()
def resolve_spec_search_root() -> Path:
+3 -3
View File
@@ -9,7 +9,7 @@ section 7.3 for the rev-bump workflow.
from __future__ import annotations
from pathlib import Path
from openjarvis.core.paths import get_config_dir
PEARL_REPO = "https://github.com/pearl-research-labs/pearl.git"
# TODO at implementation time: replace with the specific commit/tag verified
@@ -47,9 +47,9 @@ DEFAULT_PEARLD_RPC_URL = "http://localhost:44107"
MIN_FREE_DISK_GB = 200
# Runtime sidecar location (single-session assumption — see spec §8.8).
RUNTIME_DIR = Path.home() / ".openjarvis" / "runtime"
RUNTIME_DIR = get_config_dir() / "runtime"
SIDECAR_PATH = RUNTIME_DIR / "mining.json"
SIDECAR_LOCK_PATH = RUNTIME_DIR / "mining.lock"
# Pearl source cache for build-from-pin path (see spec §7.2).
PEARL_CACHE_DIR = Path.home() / ".openjarvis" / "cache" / "pearl"
PEARL_CACHE_DIR = get_config_dir() / "cache" / "pearl"
+2 -1
View File
@@ -17,6 +17,7 @@ import urllib.request
from pathlib import Path
from openjarvis.core.config import HardwareInfo
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import MinerRegistry
from . import _install
@@ -47,7 +48,7 @@ def _sidecar_path() -> Path:
def _log_dir() -> Path:
"""Return the logs directory. Override in tests."""
return Path.home() / ".openjarvis" / "logs" / "mining"
return get_config_dir() / "logs" / "mining"
def _parse_gateway_metrics(text: str, *, provider_id: str) -> MiningStats:
+2 -1
View File
@@ -5,6 +5,7 @@ from pathlib import Path
from typing import List, Literal, Optional, Tuple
from openjarvis.core.config import MemoryFilesConfig, SystemPromptConfig
from openjarvis.core.paths import get_config_dir
PromptCacheSegment = Literal["frozen_prefix", "dynamic_suffix"]
@@ -279,7 +280,7 @@ class SystemPromptBuilder:
f"Invalid persona name {name!r}: must be a simple "
"identifier (no path separators or '..')."
)
base = Path.home() / ".openjarvis" / "personas" / name
base = get_config_dir() / "personas" / name
return MemoryFilesConfig(
soul_path=str(base / "SOUL.md"),
memory_path=str(base / "MEMORY.md"),
+3 -2
View File
@@ -19,13 +19,14 @@ try:
except ModuleNotFoundError:
import tomli as tomllib # type: ignore[no-redef]
from openjarvis.core.paths import get_config_dir
# Built-in recipes directory (package data)
_PROJECT_RECIPES_DIR = Path(__file__).resolve().parent / "data"
_PROJECT_OPERATORS_DIR = _PROJECT_RECIPES_DIR / "operators"
# User-level directories
_USER_RECIPES_DIR = Path.home() / ".openjarvis" / "recipes"
_USER_OPERATORS_DIR = Path.home() / ".openjarvis" / "operators"
_USER_RECIPES_DIR = get_config_dir() / "recipes"
_USER_OPERATORS_DIR = get_config_dir() / "operators"
@dataclass(slots=True)
@@ -2095,10 +2095,13 @@ def create_agent_manager_router(
raise HTTPException(status_code=404, detail="Agent not found")
try:
from openjarvis.core.config import load_config
from openjarvis.core.paths import get_config_dir
from openjarvis.traces.store import TraceStore
config = load_config()
store = TraceStore(config.traces.db_path or "~/.openjarvis/traces.db")
store = TraceStore(
config.traces.db_path or str(get_config_dir() / "traces.db")
)
traces = store.list_traces(agent=agent_id, limit=limit)
return {
"traces": [
@@ -2120,10 +2123,13 @@ def create_agent_manager_router(
def get_trace(agent_id: str, trace_id: str):
try:
from openjarvis.core.config import load_config
from openjarvis.core.paths import get_config_dir
from openjarvis.traces.store import TraceStore
config = load_config()
store = TraceStore(config.traces.db_path or "~/.openjarvis/traces.db")
store = TraceStore(
config.traces.db_path or str(get_config_dir() / "traces.db")
)
trace = store.get(trace_id)
if trace is None:
raise HTTPException(status_code=404, detail="Trace not found")
+24 -2
View File
@@ -893,7 +893,15 @@ async def transcribe_speech(request: Request):
filename = getattr(audio_file, "filename", "audio.wav")
ext = filename.rsplit(".", 1)[-1] if "." in filename else "wav"
result = backend.transcribe(audio_bytes, format=ext, language=language or None)
try:
result = backend.transcribe(audio_bytes, format=ext, language=language or None)
except Exception as exc:
logger.exception("Speech transcription failed")
raise HTTPException(
status_code=500,
detail=f"Speech transcription failed: {exc}",
) from exc
return {
"text": result.text,
"language": result.language,
@@ -908,9 +916,23 @@ async def speech_health(request: Request):
backend = getattr(request.app.state, "speech_backend", None)
if backend is None:
return {"available": False, "reason": "No speech backend configured"}
try:
available = backend.health()
reason = None
except Exception as exc:
logger.exception("Speech health check failed")
available = False
reason = str(exc)
if not available and reason is None:
last_error = getattr(backend, "last_error", None)
if callable(last_error):
reason = last_error()
return {
"available": backend.health(),
"available": available,
"backend": backend.backend_id,
**({"reason": reason} if reason else {}),
}
+2 -2
View File
@@ -10,18 +10,18 @@ from __future__ import annotations
import json
import os
from collections.abc import AsyncIterator
from pathlib import Path
from typing import Any, Sequence
import httpx
from openjarvis.core.paths import get_config_dir
from openjarvis.core.types import Message
# ---------------------------------------------------------------------------
# Key / provider detection
# ---------------------------------------------------------------------------
_CLOUD_ENV_FILE = Path.home() / ".openjarvis" / "cloud-keys.env"
_CLOUD_ENV_FILE = get_config_dir() / "cloud-keys.env"
_OPENAI_PREFIXES = ("gpt-", "o1-", "o3-", "o4-", "chatgpt-")
_ANTHROPIC_PREFIXES = ("claude-",)
+95 -3
View File
@@ -3,7 +3,23 @@
from __future__ import annotations
import logging
from typing import Any, Dict, Optional
from typing import TYPE_CHECKING, Any, Dict, Optional
# ``Request`` must be importable at *module* scope so that FastAPI can resolve
# the stringized ``request: Request`` annotations on the OAuth endpoints below.
# Because this module uses ``from __future__ import annotations``, every
# annotation is a string that FastAPI evaluates against the module globals; a
# ``Request`` imported only inside ``create_connectors_router()`` is invisible
# there, which makes FastAPI mistake ``request`` for a required *query* param
# (HTTP 422 on /oauth/start) or inject ``None`` (AttributeError on
# /oauth/callback). Keep this import at top level. See issue #512.
if TYPE_CHECKING:
from starlette.requests import Request
else:
try:
from starlette.requests import Request
except ImportError: # starlette ships with fastapi; absent only without it
Request = Any # type: ignore[assignment,misc]
logger = logging.getLogger(__name__)
@@ -78,7 +94,7 @@ def create_connectors_router():
this package.
"""
try:
from fastapi import APIRouter, HTTPException, Request
from fastapi import APIRouter, HTTPException
except ImportError as exc:
raise ImportError(
"fastapi and pydantic are required for the connectors router"
@@ -125,6 +141,69 @@ def create_connectors_router():
"chunks": chunks,
}
def _maybe_oauth_client_pair(
connector_id: str, req: ConnectRequest
) -> Optional[Dict[str, Any]]:
"""Handle a pasted ``client_id:client_secret`` for an OAuth connector.
Returns an ``oauth_required`` directive (and persists the client
credentials to every credential file for the provider) when *req*
carries a Google ``client_id:client_secret`` pair, so the caller can
return early instead of triggering the silent background OAuth flow.
Returns ``None`` when there is no such pair (the caller then falls
through to the normal ``handle_callback`` / token path).
Raises ``HTTPException(400)`` when the pair is present but malformed or
the connector has no OAuth provider per the silent-failure discipline
in REVIEW.md, a bad credential surfaces an actionable error rather than
a perpetual ``pending`` state.
"""
from openjarvis.connectors.oauth import (
get_provider_for_connector,
save_client_credentials,
)
raw = (req.code or req.token or "").strip()
# Only the client-registration pair routes through the server flow.
# A raw access token (no ".apps.googleusercontent.com") is handled by
# the connector's handle_callback unchanged.
if ".apps.googleusercontent.com" not in raw or ":" not in raw:
return None
provider = get_provider_for_connector(connector_id)
if provider is None:
raise HTTPException(
status_code=400,
detail=f"No OAuth provider configured for '{connector_id}'",
)
client_id, client_secret = raw.split(":", 1)
client_id = client_id.strip()
client_secret = client_secret.strip()
if not client_id or not client_secret:
raise HTTPException(
status_code=400,
detail=(
"Malformed credentials — expected 'CLIENT_ID:CLIENT_SECRET'. "
f"Create an OAuth client at: {provider.setup_url}"
),
)
save_client_credentials(provider, client_id, client_secret)
# Cached instances may have resolved a stale credentials path before
# these client creds existed; drop them so /oauth/callback rebuilds
# them against the freshly written files.
for cid in provider.connector_ids:
_instances.pop(cid, None)
return {
"connector_id": connector_id,
"connected": False,
"status": "oauth_required",
"oauth_start": f"/v1/connectors/{connector_id}/oauth/start",
"sync_status": None,
}
# ------------------------------------------------------------------
# Background-sync state tracking. Defined here (before the endpoints)
# so that POST /connect can fire-and-forget into the same machinery
@@ -317,6 +396,19 @@ def create_connectors_router():
instance._connected = Path(req.path).is_dir()
elif auth_type == "oauth":
# A pasted ``client_id:client_secret`` pair is NOT a completed
# OAuth credential — it is the app registration. Persist it and
# hand the UI a directive to run the in-process browser consent
# flow (/oauth/start → /oauth/callback), which is the only path
# that actually exchanges a code for an access_token. Previously
# this routed into the connector's handle_callback, which spawned
# a daemon thread that popped a browser + ran its own
# localhost:8789 callback server; that thread fails silently in
# the bundled desktop context, so the connector never became
# connected and never appeared in Data Sources (issue #512).
directive = _maybe_oauth_client_pair(connector_id, req)
if directive is not None:
return directive
if req.code:
instance.handle_callback(req.code)
elif req.token:
@@ -433,9 +525,9 @@ def create_connectors_router():
@router.get("/{connector_id}/oauth/callback")
async def oauth_callback(
connector_id: str,
request: Request,
code: str = "",
error: str = "",
request: Request = None,
):
"""Handle OAuth callback from the provider."""
from fastapi.responses import HTMLResponse
+57 -3
View File
@@ -9,6 +9,7 @@ from typing import Any
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from openjarvis.core.paths import get_config_dir
from openjarvis.core.types import Message, Role
from openjarvis.server.models import (
ChatCompletionChunk,
@@ -43,6 +44,51 @@ def _to_messages(chat_messages) -> list[Message]:
return messages
def _ensure_identity_prompt(messages: list[Message], app_config) -> list[Message]:
"""Prepend OpenJarvis's identity system prompt when the client omits one.
The desktop UI's chat backend posts only user/assistant turns to
``/v1/chat/completions`` (see ``frontend/.../Chat/InputArea.tsx``), so
nothing grounds the model's identity. Without a system prompt the model
answers from its training identity (e.g. "I'm Claude", "I am Qwen"),
which is what #540 reported. The CLI paths inject this via
``SystemPromptBuilder`` / ``BaseAgent``; the engine-direct server paths
did not. This mirrors the agent fallback in ``agents/_stubs.py``.
If any message already carries a system role, the caller has supplied
their own grounding and we leave the list untouched (no double-prompting).
Resolution of the identity text: ``app_config.agent.default_system_prompt``
when a config is wired onto ``app.state``; otherwise fall back to
``load_config()``. Config resolution is wrapped so a broken/missing
config degrades to "no injection" rather than crashing the endpoint, but
the failure is logged (per REVIEW.md never silently swallow).
"""
if any(m.role == Role.SYSTEM for m in messages):
return messages
prompt = ""
try:
if app_config is not None:
prompt = app_config.agent.default_system_prompt or ""
else:
from openjarvis.core.config import load_config
prompt = load_config().agent.default_system_prompt or ""
except Exception:
logging.getLogger("openjarvis.server").debug(
"Identity system prompt resolution failed; "
"serving request without identity grounding",
exc_info=True,
)
return messages
if not prompt:
return messages
return [Message(role=Role.SYSTEM, content=prompt), *messages]
@router.post("/v1/chat/completions")
async def chat_completions(request_body: ChatCompletionRequest, request: Request):
"""Handle chat completion requests (streaming and non-streaming)."""
@@ -149,7 +195,7 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
# from the engine for true real-time output.
if request_body.tools:
return await _handle_stream_tools(
engine, model, request_body, complexity_info
engine, model, request_body, complexity_info, app_config=config
)
return await _handle_stream(
engine,
@@ -157,6 +203,7 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
request_body,
complexity_info,
trace_store=getattr(request.app.state, "trace_store", None),
app_config=config,
)
# Non-streaming: use agent if available, otherwise direct engine call.
@@ -192,6 +239,7 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
request_body,
bus=bus,
complexity_info=complexity_info,
app_config=config,
)
@@ -201,9 +249,11 @@ def _handle_direct(
req: ChatCompletionRequest,
bus=None,
complexity_info=None,
app_config=None,
) -> ChatCompletionResponse:
"""Direct engine call without agent."""
messages = _to_messages(req.messages)
messages = _ensure_identity_prompt(messages, app_config)
kwargs: dict[str, Any] = {}
if req.tools:
kwargs["tools"] = req.tools
@@ -380,6 +430,8 @@ async def _handle_stream_tools(
model: str,
req: ChatCompletionRequest,
complexity_info=None,
*,
app_config=None,
):
"""Stream a raw OpenAI-compat function-calling response via SSE.
@@ -397,6 +449,7 @@ async def _handle_stream_tools(
from openjarvis.server.cloud_router import is_cloud_model
messages = _to_messages(req.messages)
messages = _ensure_identity_prompt(messages, app_config)
chunk_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
use_cloud = is_cloud_model(model)
@@ -491,6 +544,7 @@ async def _handle_stream(
complexity_info=None,
*,
trace_store=None,
app_config=None,
):
"""Stream response using SSE format.
@@ -509,6 +563,7 @@ async def _handle_stream(
)
messages = _to_messages(req.messages)
messages = _ensure_identity_prompt(messages, app_config)
chunk_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
# Last user message — recorded as the trace query.
@@ -767,10 +822,9 @@ async def reload_cloud_engine(request: Request):
key so that cloud models become available without a full app restart.
"""
import os
from pathlib import Path
# Re-read ~/.openjarvis/cloud-keys.env and update the running process env.
keys_path = Path.home() / ".openjarvis" / "cloud-keys.env"
keys_path = get_config_dir() / "cloud-keys.env"
if keys_path.exists():
for raw_line in keys_path.read_text().splitlines():
line = raw_line.strip()
+3 -1
View File
@@ -8,6 +8,8 @@ import sqlite3
from pathlib import Path
from typing import Any, Dict, List, Optional
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
_MAX_HISTORY_TURNS = 20
@@ -22,7 +24,7 @@ class SessionStore:
def __init__(self, db_path: str = "") -> None:
if not db_path:
db_path = str(Path.home() / ".openjarvis" / "sessions.db")
db_path = str(get_config_dir() / "sessions.db")
from openjarvis.security.file_utils import secure_create
secure_create(Path(db_path))
+2 -1
View File
@@ -23,6 +23,7 @@ from typing import List
import yaml
from openjarvis.core.paths import get_config_dir
from openjarvis.skills.parser import SkillParser
from openjarvis.skills.sources.base import ResolvedSkill
from openjarvis.skills.tool_translator import ToolTranslator
@@ -56,7 +57,7 @@ class SkillImporter:
self._parser = parser
self._translator = tool_translator
if target_root is None:
target_root = Path("~/.openjarvis/skills/").expanduser()
target_root = get_config_dir() / "skills"
self._target_root = Path(target_root)
def import_skill(
+4 -3
View File
@@ -7,6 +7,7 @@ from pathlib import Path
from typing import Any, Dict, List, Optional
from openjarvis.core.events import EventBus
from openjarvis.core.paths import get_config_dir
from openjarvis.skills.dependency import validate_dependencies
from openjarvis.skills.executor import SkillExecutor, SkillResult
from openjarvis.skills.loader import discover_skills
@@ -54,7 +55,7 @@ class SkillManager:
except Exception:
pass
if overlay_dir is None:
overlay_dir = Path("~/.openjarvis/learning/skills/").expanduser()
overlay_dir = get_config_dir() / "learning" / "skills"
self._overlay_dir = Path(overlay_dir).expanduser()
# ------------------------------------------------------------------
@@ -262,7 +263,7 @@ class SkillManager:
discovered = discovery.analyze_traces(traces)
if output_dir is None:
output_dir = Path("~/.openjarvis/skills/discovered/").expanduser()
output_dir = get_config_dir() / "skills" / "discovered"
output_dir = Path(output_dir).expanduser()
output_dir.mkdir(parents=True, exist_ok=True)
@@ -375,7 +376,7 @@ class SkillManager:
manifest's ``name`` field equals ``name``.
"""
if roots is None:
roots = [Path("~/.openjarvis/skills/").expanduser(), Path("./skills")]
roots = [get_config_dir() / "skills", Path("./skills")]
matches: List[Path] = []
for root in roots:
+2 -1
View File
@@ -14,6 +14,7 @@ from typing import List
import yaml
from openjarvis.core.paths import get_config_dir
from openjarvis.skills.sources.base import ResolvedSkill, SourceResolver
LOGGER = logging.getLogger(__name__)
@@ -28,7 +29,7 @@ class HermesResolver(SourceResolver):
def __init__(self, cache_root: Path | None = None) -> None:
if cache_root is None:
cache_root = Path("~/.openjarvis/skill-cache/hermes/").expanduser()
cache_root = get_config_dir() / "skill-cache" / "hermes"
self._cache_root = Path(cache_root)
def cache_dir(self) -> Path:
+2 -1
View File
@@ -15,6 +15,7 @@ from typing import List
import yaml
from openjarvis.core.paths import get_config_dir
from openjarvis.skills.sources.base import ResolvedSkill, SourceResolver
LOGGER = logging.getLogger(__name__)
@@ -29,7 +30,7 @@ class OpenClawResolver(SourceResolver):
def __init__(self, cache_root: Path | None = None) -> None:
if cache_root is None:
cache_root = Path("~/.openjarvis/skill-cache/openclaw/").expanduser()
cache_root = get_config_dir() / "skill-cache" / "openclaw"
self._cache_root = Path(cache_root)
def cache_dir(self) -> Path:
+77 -16
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import logging
import tempfile
from typing import List, Optional
@@ -13,6 +14,13 @@ try:
except ImportError:
WhisperModel = None # type: ignore[assignment, misc]
try:
import ctranslate2
except ImportError:
ctranslate2 = None # type: ignore[assignment]
logger = logging.getLogger(__name__)
@SpeechRegistry.register("faster-whisper")
class FasterWhisperBackend(SpeechBackend):
@@ -30,20 +38,60 @@ class FasterWhisperBackend(SpeechBackend):
self._device = device
self._compute_type = compute_type
self._model: Optional[WhisperModel] = None
self._last_error: Optional[str] = None
def _resolve_compute_type(self) -> str:
"""Pick a CTranslate2 compute type supported by the configured device."""
if ctranslate2 is None:
return self._compute_type
try:
supported = set(ctranslate2.get_supported_compute_types(self._device))
except Exception as exc:
logger.debug(
"Could not inspect CTranslate2 compute types for %s: %s",
self._device,
exc,
)
return self._compute_type
if self._compute_type in supported:
return self._compute_type
preferences = (
("int8", "float32", "int8_float32", "int16")
if self._compute_type == "float16"
else ("float32", "int8", "int8_float32", "int16")
)
fallback = next((value for value in preferences if value in supported), None)
if fallback is None:
return self._compute_type
logger.warning(
"CTranslate2 compute_type=%r is not supported on device=%r; "
"using %r instead",
self._compute_type,
self._device,
fallback,
)
return fallback
def _ensure_model(self) -> WhisperModel:
"""Lazy-load the Whisper model on first use."""
if self._model is None:
if WhisperModel is None:
raise ImportError(
self._last_error = (
"faster-whisper is not installed. "
"Install with: uv sync --extra speech"
"Install with: uv sync --extra desktop"
)
raise ImportError(self._last_error)
compute_type = self._resolve_compute_type()
self._model = WhisperModel(
self._model_size,
device=self._device,
compute_type=self._compute_type,
compute_type=compute_type,
)
self._last_error = None
return self._model
def transcribe(
@@ -54,20 +102,24 @@ class FasterWhisperBackend(SpeechBackend):
language: Optional[str] = None,
) -> TranscriptionResult:
"""Transcribe audio bytes using Faster-Whisper."""
model = self._ensure_model()
try:
model = self._ensure_model()
# Write audio to a temp file (faster-whisper needs a file path)
suffix = f".{format}" if not format.startswith(".") else format
with tempfile.NamedTemporaryFile(suffix=suffix, delete=True) as tmp:
tmp.write(audio)
tmp.flush()
# Write audio to a temp file (faster-whisper needs a file path)
suffix = f".{format}" if not format.startswith(".") else format
with tempfile.NamedTemporaryFile(suffix=suffix, delete=True) as tmp:
tmp.write(audio)
tmp.flush()
kwargs = {}
if language:
kwargs["language"] = language
kwargs = {}
if language:
kwargs["language"] = language
segments_iter, info = model.transcribe(tmp.name, **kwargs)
segments_list = list(segments_iter)
segments_iter, info = model.transcribe(tmp.name, **kwargs)
segments_list = list(segments_iter)
except Exception as exc:
self._last_error = str(exc)
raise
# Build result
text = "".join(seg.text for seg in segments_list).strip()
@@ -81,6 +133,7 @@ class FasterWhisperBackend(SpeechBackend):
for seg in segments_list
]
self._last_error = None
return TranscriptionResult(
text=text,
language=getattr(info, "language", None),
@@ -91,9 +144,17 @@ class FasterWhisperBackend(SpeechBackend):
def health(self) -> bool:
"""Check if model is loaded or loadable."""
if self._model is not None:
try:
self._ensure_model()
return True
return WhisperModel is not None
except Exception as exc:
self._last_error = str(exc)
logger.debug("Faster-Whisper health check failed: %s", exc)
return False
def last_error(self) -> Optional[str]:
"""Return the last model load or transcription error, if any."""
return self._last_error
def supported_formats(self) -> List[str]:
"""Supported audio formats (same as ffmpeg/Whisper)."""
+2 -3
View File
@@ -7,6 +7,7 @@ from typing import Any, List, Optional
from openjarvis.core.config import JarvisConfig, load_config
from openjarvis.core.events import EventBus, get_event_bus
from openjarvis.core.paths import get_config_dir
from openjarvis.engine._stubs import InferenceEngine
from openjarvis.system.core import JarvisSystem
from openjarvis.tools._stubs import BaseTool, ToolExecutor
@@ -232,12 +233,10 @@ class SystemBuilder:
agent_manager = None
if config.agent_manager.enabled:
try:
from pathlib import Path
from openjarvis.agents.manager import AgentManager
am_db = config.agent_manager.db_path or str(
Path("~/.openjarvis/agents.db").expanduser()
get_config_dir() / "agents.db"
)
agent_manager = AgentManager(db_path=am_db)
except Exception as exc:
+3 -1
View File
@@ -11,6 +11,8 @@ try:
except ModuleNotFoundError:
import tomli as tomllib # type: ignore[no-redef]
from openjarvis.core.paths import get_config_dir
@dataclass(slots=True)
class AgentTemplate:
@@ -72,7 +74,7 @@ def _builtin_templates_dir() -> Path:
def _user_templates_dir() -> Path:
"""Return the path to user-defined templates (~/.openjarvis/templates/agents/)."""
return Path.home() / ".openjarvis" / "templates" / "agents"
return get_config_dir() / "templates" / "agents"
def discover_templates(
+3 -1
View File
@@ -19,6 +19,8 @@ from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
from openjarvis.core.paths import get_config_dir
# ---------------------------------------------------------------------------
# Decision constants
# ---------------------------------------------------------------------------
@@ -147,7 +149,7 @@ class ApprovalStore:
def __init__(self, db_path: str = "") -> None:
if not db_path:
db_path = str(Path.home() / ".openjarvis" / "approvals.db")
db_path = str(get_config_dir() / "approvals.db")
self._db_path = db_path
Path(db_path).parent.mkdir(parents=True, exist_ok=True)
self._conn = sqlite3.connect(db_path, check_same_thread=False)
+3 -3
View File
@@ -16,10 +16,10 @@ The TOML file format (written by the applier) is::
from __future__ import annotations
import logging
import os
from pathlib import Path
from typing import Dict, Optional
from openjarvis.core.paths import get_config_dir
logger = logging.getLogger(__name__)
_cache: Optional[Dict[str, str]] = None
@@ -27,7 +27,7 @@ _cache: Optional[Dict[str, str]] = None
def _load_overrides() -> Dict[str, str]:
"""Parse descriptions.toml and return {tool_name: description}."""
home = Path(os.environ.get("OPENJARVIS_HOME", "~/.openjarvis")).expanduser()
home = get_config_dir()
desc_path = home / "tools" / "descriptions.toml"
if not desc_path.exists():
return {}
+4 -1
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
from pathlib import Path
from typing import Any
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import ToolRegistry
from openjarvis.core.types import ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
@@ -14,7 +15,9 @@ from openjarvis.tools._stubs import BaseTool, ToolSpec
class MemoryManageTool(BaseTool):
"""Manage persistent agent memory (MEMORY.md)."""
def __init__(self, memory_path: Path | str = "~/.openjarvis/MEMORY.md") -> None:
def __init__(self, memory_path: Path | str | None = None) -> None:
if memory_path is None:
memory_path = get_config_dir() / "MEMORY.md"
self._memory_path = Path(memory_path).expanduser()
@property
+4 -1
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
from pathlib import Path
from typing import Any, List
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import ToolRegistry
from openjarvis.core.types import ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
@@ -14,7 +15,9 @@ from openjarvis.tools._stubs import BaseTool, ToolSpec
class SkillManageTool(BaseTool):
"""Manage agent-authored procedural skills."""
def __init__(self, skills_dir: Path | str = "~/.openjarvis/skills/") -> None:
def __init__(self, skills_dir: Path | str | None = None) -> None:
if skills_dir is None:
skills_dir = get_config_dir() / "skills"
self._skills_dir = Path(skills_dir).expanduser()
@property
+4 -1
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
from pathlib import Path
from typing import Any
from openjarvis.core.paths import get_config_dir
from openjarvis.core.registry import ToolRegistry
from openjarvis.core.types import ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
@@ -14,7 +15,9 @@ from openjarvis.tools._stubs import BaseTool, ToolSpec
class UserProfileManageTool(BaseTool):
"""Manage persistent user profile (USER.md)."""
def __init__(self, user_path: Path | str = "~/.openjarvis/USER.md") -> None:
def __init__(self, user_path: Path | str | None = None) -> None:
if user_path is None:
user_path = get_config_dir() / "USER.md"
self._user_path = Path(user_path).expanduser()
@property
+9 -2
View File
@@ -165,7 +165,10 @@ class WebSearchTool(BaseTool):
client = TavilyClient(api_key=self._api_key)
response = client.search(
query, max_results=max_results, search_depth="advanced"
query,
max_results=max_results,
search_depth="advanced",
include_usage=True,
)
results = response.get("results", [])
formatted_parts = []
@@ -182,7 +185,11 @@ class WebSearchTool(BaseTool):
tool_name="web_search",
content=formatted or "No results found.",
success=True,
metadata={"num_results": len(results), "engine": "tavily"},
metadata={
"num_results": len(results),
"engine": "tavily",
"credits": (response.get("usage") or {}).get("credits"),
},
)
except Exception as exc:
logger.debug(
+53
View File
@@ -10,10 +10,12 @@ from click.testing import CliRunner
from openjarvis.cli import cli
from openjarvis.cli.doctor_cmd import (
CheckResult,
_check_config_exists,
_check_default_model,
_check_nodejs,
_check_python_version,
_check_speech_backend,
)
@@ -39,6 +41,10 @@ class TestDoctorRuns:
),
patch("openjarvis.cli.doctor_cmd._check_engines", return_value=[]),
patch("openjarvis.cli.doctor_cmd._check_models", return_value=[]),
patch(
"openjarvis.cli.doctor_cmd._check_speech_backend",
return_value=CheckResult("Speech backend", "ok", "mock ready"),
),
):
result = CliRunner().invoke(cli, ["doctor"])
assert result.exit_code == 0
@@ -59,6 +65,10 @@ class TestDoctorJsonOutput:
),
patch("openjarvis.cli.doctor_cmd._check_engines", return_value=[]),
patch("openjarvis.cli.doctor_cmd._check_models", return_value=[]),
patch(
"openjarvis.cli.doctor_cmd._check_speech_backend",
return_value=CheckResult("Speech backend", "ok", "mock ready"),
),
):
result = CliRunner().invoke(cli, ["doctor", "--json"])
assert result.exit_code == 0
@@ -142,6 +152,49 @@ class TestCheckDefaultModel:
assert "auto" in result.message.lower()
class TestCheckSpeechBackend:
def test_check_speech_backend_ready(self) -> None:
backend = MagicMock()
backend.backend_id = "faster-whisper"
backend.health.return_value = True
with patch(
"openjarvis.speech._discovery.get_speech_backend",
return_value=backend,
):
result = _check_speech_backend()
assert result.status == "ok"
assert "faster-whisper" in result.message
def test_check_speech_backend_reports_load_error(self) -> None:
backend = MagicMock()
backend.backend_id = "faster-whisper"
backend.health.return_value = False
backend.last_error.return_value = "missing cublas64_12.dll"
with patch(
"openjarvis.speech._discovery.get_speech_backend",
return_value=backend,
):
result = _check_speech_backend()
assert result.status == "warn"
assert "faster-whisper unavailable" in result.message
assert result.details == "missing cublas64_12.dll"
def test_check_speech_backend_missing_uses_desktop_hint(self) -> None:
with patch(
"openjarvis.speech._discovery.get_speech_backend",
return_value=None,
):
result = _check_speech_backend()
assert result.status == "warn"
assert result.details is not None
assert "uv sync --extra desktop" in result.details
class TestCheckNodejs:
def test_check_nodejs_found(self) -> None:
"""Node.js check reports version when node is available."""
@@ -0,0 +1,273 @@
"""Regression tests for the connectors-router OAuth flow (issue #512).
These tests reproduce the three coupled defects that prevented Google Drive
(and its Google siblings) from ever completing OAuth and appearing in Data
Sources, and assert the fixed behaviour:
(A/B) ``POST /connect`` with a pasted ``client_id:client_secret`` pair must
persist the client credentials and return an ``oauth_required`` directive
pointing at ``/oauth/start`` NOT silently spawn a background browser
thread and report a perpetual ``pending`` state.
(C-1) ``GET /oauth/start`` must return a redirect to the provider's consent
page (regression: HTTP 422 because ``request: Request`` was mis-bound as
a query param under ``from __future__ import annotations`` + a local
``Request`` import).
(C-2) ``GET /oauth/callback`` must read ``request.base_url`` and exchange the
code for tokens without crashing (regression: ``request`` defaulted to
``None`` ``AttributeError``), persisting the access token to every
Google credential file and flipping ``is_connected()`` to True.
All tests are hermetic: the connectors directory, the shared Google
credentials path, and every Google connector's default credentials path are
redirected to ``tmp_path`` so the suite neither depends on nor pollutes
``~/.openjarvis/connectors`` (a real source of spurious failures see the
verifier note on ``resolve_google_credentials`` silently substituting the
shared file when the caller-supplied path does not yet exist on disk).
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any, Iterator
from unittest.mock import patch
import pytest
fastapi = pytest.importorskip("fastapi", reason="requires the 'server' extra")
from fastapi import FastAPI # noqa: E402
from fastapi.testclient import TestClient # noqa: E402
_CLIENT_PAIR = "myid-123.apps.googleusercontent.com:GOCSPX-secret"
_CLIENT_ID = "myid-123.apps.googleusercontent.com"
_ALL_GOOGLE_FILES = (
"google.json",
"gdrive.json",
"gcalendar.json",
"gcontacts.json",
"gmail.json",
"google_tasks.json",
)
@pytest.fixture()
def hermetic_connectors(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
"""Redirect all Google credential paths into *tmp_path*.
Ensures connector instances created by the router's ``_get_or_create``
resolve to the same directory the OAuth callback writes to, and that the
test leaves ``~/.openjarvis`` untouched.
Why this is more than a one-line monkeypatch: the autouse registry-clear
fixture causes ``_ensure_connectors_registered()`` to ``importlib.reload``
each connector module on the first router call, which re-executes the
module body. To survive that reload we patch ``DEFAULT_CONFIG_DIR`` at its
*source* (``openjarvis.core.config``) every connector re-derives
``_DEFAULT_CREDENTIALS_PATH`` from it on reload, so the tmp dir sticks.
We also pre-register + pre-reload the connectors inside the fixture so the
reload happens while the patch is live, then reset module state on
teardown so a later test that imports these modules fresh is unaffected.
"""
import importlib
import sys
import openjarvis.connectors.oauth as oauth_mod
import openjarvis.core.config as config_mod
import openjarvis.server.connectors_router as router_mod
from openjarvis.core.registry import ConnectorRegistry
conn_dir = tmp_path / "connectors"
conn_dir.mkdir(parents=True, exist_ok=True)
monkeypatch.setattr(config_mod, "DEFAULT_CONFIG_DIR", tmp_path)
monkeypatch.setattr(oauth_mod, "_CONNECTORS_DIR", conn_dir)
monkeypatch.setattr(
oauth_mod, "_SHARED_GOOGLE_CREDENTIALS_PATH", str(conn_dir / "google.json")
)
# Force the connector modules to re-derive their default paths from the
# patched DEFAULT_CONFIG_DIR now, before any request, and register them so
# the router's lazy reload-on-empty-registry path is a no-op.
google_mods = [
"openjarvis.connectors.gdrive",
"openjarvis.connectors.gcalendar",
"openjarvis.connectors.gcontacts",
"openjarvis.connectors.gmail",
"openjarvis.connectors.google_tasks",
]
for name in google_mods:
if name in sys.modules:
importlib.reload(sys.modules[name])
router_mod._instances.clear()
yield conn_dir
router_mod._instances.clear()
ConnectorRegistry.clear()
# Restore the connector modules to their real (unpatched) default paths so
# subsequent tests in the same process see ~/.openjarvis again.
for name in google_mods:
if name in sys.modules:
importlib.reload(sys.modules[name])
@pytest.fixture()
def client(hermetic_connectors: Path) -> Iterator[TestClient]:
from openjarvis.server.connectors_router import create_connectors_router
app = FastAPI()
app.include_router(create_connectors_router())
with TestClient(app) as c:
yield c
# ---------------------------------------------------------------------------
# Defect A/B — POST /connect must not silently spawn a background OAuth thread
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"connector_id", ["gdrive", "gcalendar", "gcontacts", "gmail", "google_tasks"]
)
def test_connect_client_pair_returns_oauth_required_no_browser(
client: TestClient, hermetic_connectors: Path, connector_id: str
) -> None:
"""Pasting client_id:secret persists creds + asks the UI to run the flow.
Covers every Google connector that shares the OAuth provider, proving the
sibling connectors are fixed too (not just gdrive).
"""
with patch("openjarvis.core.open_browser") as mock_browser:
resp = client.post(
f"/v1/connectors/{connector_id}/connect", json={"code": _CLIENT_PAIR}
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["status"] == "oauth_required"
assert body["oauth_start"] == f"/v1/connectors/{connector_id}/oauth/start"
assert body["connected"] is False
# No fire-and-forget browser thread (the root cause of "nothing happens").
mock_browser.assert_not_called()
# Client credentials persisted to EVERY Google credential file so a single
# consent covers all Google connectors.
for filename in _ALL_GOOGLE_FILES:
path = hermetic_connectors / filename
assert path.exists(), f"{filename} not written"
assert json.loads(path.read_text())["client_id"] == _CLIENT_ID
def test_connect_malformed_client_pair_raises_400(
client: TestClient,
) -> None:
"""A blank secret surfaces an actionable 400 — not a silent pending state."""
resp = client.post(
"/v1/connectors/gdrive/connect",
json={"code": "myid-123.apps.googleusercontent.com:"},
)
assert resp.status_code == 400
assert "CLIENT_ID:CLIENT_SECRET" in resp.json()["detail"]
def test_connect_raw_token_still_handled(
client: TestClient, hermetic_connectors: Path
) -> None:
"""A raw token (not a client pair) still flows through handle_callback."""
resp = client.post(
"/v1/connectors/gdrive/connect", json={"token": "ya29.raw-access-token"}
)
assert resp.status_code == 200, resp.text
saved = json.loads((hermetic_connectors / "gdrive.json").read_text())
assert saved.get("token") == "ya29.raw-access-token"
# ---------------------------------------------------------------------------
# Defect C-1 — GET /oauth/start must redirect (was HTTP 422)
# ---------------------------------------------------------------------------
def test_oauth_start_redirects_to_consent(
client: TestClient,
) -> None:
# First save client creds via the connect call.
client.post("/v1/connectors/gdrive/connect", json={"code": _CLIENT_PAIR})
resp = client.get("/v1/connectors/gdrive/oauth/start", follow_redirects=False)
# FastAPI's RedirectResponse defaults to 307; any 3xx is a pass (was 422).
assert resp.status_code in (302, 307), resp.text
location = resp.headers["location"]
assert location.startswith("https://accounts.google.com/o/oauth2/v2/auth")
assert _CLIENT_ID in location
# redirect_uri must point back at OUR in-process callback.
assert "oauth%2Fcallback" in location or "oauth/callback" in location
def test_oauth_start_without_creds_returns_400(client: TestClient) -> None:
resp = client.get("/v1/connectors/gdrive/oauth/start", follow_redirects=False)
assert resp.status_code == 400
assert "client credentials" in resp.json()["detail"].lower()
# ---------------------------------------------------------------------------
# Defect C-2 — GET /oauth/callback must exchange + persist (was 500 on None)
# ---------------------------------------------------------------------------
def test_oauth_callback_exchanges_and_connects(
client: TestClient, hermetic_connectors: Path
) -> None:
import openjarvis.connectors.oauth as oauth_mod
client.post("/v1/connectors/gdrive/connect", json={"code": _CLIENT_PAIR})
fake_tokens = {
"access_token": "ya29.REAL",
"refresh_token": "1//REAL",
"token_type": "Bearer",
"expires_in": 3600,
}
with patch.object(oauth_mod, "_exchange_token", return_value=fake_tokens) as ex:
resp = client.get("/v1/connectors/gdrive/oauth/callback?code=authcode123")
assert resp.status_code == 200, resp.text
assert "Connected!" in resp.text
ex.assert_called_once()
# Access token written to ALL Google credential files.
for filename in _ALL_GOOGLE_FILES:
saved = json.loads((hermetic_connectors / filename).read_text())
assert saved["access_token"] == "ya29.REAL"
assert saved["refresh_token"] == "1//REAL"
# The connector now reports connected, and GET /connectors agrees.
from openjarvis.connectors.gdrive import GDriveConnector
assert GDriveConnector().is_connected() is True
listing = client.get("/v1/connectors").json()["connectors"]
gdrive = next(c for c in listing if c["connector_id"] == "gdrive")
assert gdrive["connected"] is True
def test_oauth_callback_error_param_renders_failure(client: TestClient) -> None:
resp = client.get("/v1/connectors/gdrive/oauth/callback?error=access_denied")
assert resp.status_code == 400
assert "access_denied" in resp.text
def test_oauth_callback_exchange_failure_renders_error(
client: TestClient,
) -> None:
import openjarvis.connectors.oauth as oauth_mod
client.post("/v1/connectors/gdrive/connect", json={"code": _CLIENT_PAIR})
def _boom(*_a: Any, **_k: Any) -> dict[str, Any]:
raise RuntimeError("token endpoint 400")
with patch.object(oauth_mod, "_exchange_token", side_effect=_boom):
resp = client.get("/v1/connectors/gdrive/oauth/callback?code=bad")
assert resp.status_code == 500
assert "Token Exchange Failed" in resp.text
+51 -37
View File
@@ -30,19 +30,35 @@ def test_exchange_google_token_calls_endpoint() -> None:
mock_post.assert_called_once()
def test_gdrive_handle_callback_triggers_oauth(tmp_path: Path) -> None:
def test_gdrive_handle_callback_persists_creds_no_background_flow(
tmp_path: Path,
) -> None:
"""A pasted client pair persists creds ONLY — no silent browser thread.
Regression for issue #512: the previous implementation spawned a daemon
thread that popped a browser and ran its own localhost:8789 callback
server. That thread failed silently in the bundled desktop context, so the
connector never gained an access token. ``handle_callback`` must now only
save the client_id/secret; the in-process server flow owns the consent
round-trip. We assert ``open_browser`` is never invoked.
"""
from openjarvis.connectors.gdrive import GDriveConnector
from openjarvis.connectors.oauth import load_tokens
creds = str(tmp_path / "gdrive.json")
conn = GDriveConnector(credentials_path=creds)
with patch("openjarvis.connectors.gdrive.run_oauth_flow") as mock_flow:
mock_flow.return_value = {"access_token": "ya29.test"}
with patch("openjarvis.core.open_browser") as mock_browser:
conn.handle_callback("test-id.apps.googleusercontent.com:test-secret")
mock_flow.assert_called_once()
call_kwargs = mock_flow.call_args
assert "test-id.apps.googleusercontent.com" in str(call_kwargs)
mock_browser.assert_not_called()
tokens = load_tokens(creds)
assert tokens is not None
assert tokens["client_id"] == "test-id.apps.googleusercontent.com"
assert tokens["client_secret"] == "test-secret"
# No access token yet — that arrives via /oauth/callback.
assert not tokens.get("access_token")
assert conn.is_connected() is False
def test_gdrive_is_connected_requires_access_token(tmp_path: Path) -> None:
@@ -61,48 +77,46 @@ def test_gdrive_is_connected_requires_access_token(tmp_path: Path) -> None:
assert conn.is_connected() is True
def test_gcalendar_handle_callback_triggers_oauth(tmp_path: Path) -> None:
def test_gcalendar_handle_callback_persists_creds_no_background_flow(
tmp_path: Path,
) -> None:
"""Sibling connector shares the fix: creds saved, no browser thread (#512)."""
from openjarvis.connectors.gcalendar import GCalendarConnector
from openjarvis.connectors.oauth import load_tokens
creds = str(tmp_path / "gcalendar.json")
conn = GCalendarConnector(credentials_path=creds)
with patch("openjarvis.connectors.gcalendar.run_oauth_flow") as mock_flow:
mock_flow.return_value = {"access_token": "ya29.test"}
with patch("openjarvis.core.open_browser") as mock_browser:
conn.handle_callback("test-id.apps.googleusercontent.com:test-secret")
mock_flow.assert_called_once()
def test_gcontacts_handle_callback_triggers_oauth(tmp_path: Path) -> None:
from openjarvis.connectors.gcontacts import GContactsConnector
creds = str(tmp_path / "gcontacts.json")
conn = GContactsConnector(credentials_path=creds)
with patch("openjarvis.connectors.gcontacts.run_oauth_flow") as mock_flow:
mock_flow.return_value = {"access_token": "ya29.test"}
conn.handle_callback("test-id.apps.googleusercontent.com:test-secret")
mock_flow.assert_called_once()
def test_gdrive_handle_callback_fallback_on_failure(tmp_path: Path) -> None:
from openjarvis.connectors.gdrive import GDriveConnector
from openjarvis.connectors.oauth import load_tokens
creds = str(tmp_path / "gdrive.json")
conn = GDriveConnector(credentials_path=creds)
with patch("openjarvis.connectors.gdrive.run_oauth_flow") as mock_flow:
mock_flow.side_effect = RuntimeError("OAuth failed")
conn.handle_callback("test-id.apps.googleusercontent.com:test-secret")
# Should have saved client_id and client_secret as fallback
mock_browser.assert_not_called()
tokens = load_tokens(creds)
assert tokens is not None
assert tokens["client_id"] == "test-id.apps.googleusercontent.com"
assert tokens["client_secret"] == "test-secret"
assert conn.is_connected() is False
def test_gcontacts_handle_callback_persists_creds_no_background_flow(
tmp_path: Path,
) -> None:
"""Sibling connector shares the fix: creds saved, no browser thread (#512)."""
from openjarvis.connectors.gcontacts import GContactsConnector
from openjarvis.connectors.oauth import load_tokens
creds = str(tmp_path / "gcontacts.json")
conn = GContactsConnector(credentials_path=creds)
with patch("openjarvis.core.open_browser") as mock_browser:
conn.handle_callback("test-id.apps.googleusercontent.com:test-secret")
mock_browser.assert_not_called()
tokens = load_tokens(creds)
assert tokens is not None
assert tokens["client_id"] == "test-id.apps.googleusercontent.com"
assert tokens["client_secret"] == "test-secret"
assert conn.is_connected() is False
def test_gdrive_handle_callback_raw_token(tmp_path: Path) -> None:
+9
View File
@@ -237,6 +237,14 @@ class TestAgentConfigNew:
or isinstance(getattr(ac.__class__, "temperature", None), property) is False
)
def test_default_system_prompt_anchors_identity(self) -> None:
"""#540: the hardened wording must name OpenJarvis and explicitly
deny the model's training identity so distilled models stop
claiming to be Claude/ChatGPT/etc."""
prompt = AgentConfig().default_system_prompt
assert "OpenJarvis" in prompt
assert "not Claude" in prompt
class TestNestedEngineConfig:
def test_nested_access(self) -> None:
@@ -561,6 +569,7 @@ class TestWhatsAppBaileysChannelConfig:
def test_mining_config_absent_means_none(tmp_path):
from openjarvis.core.config import load_config
cfg_path = tmp_path / "config.toml"
cfg_path.write_text("") # empty config
cfg = load_config(cfg_path)
+176
View File
@@ -0,0 +1,176 @@
"""Tests for the env-aware OpenJarvis home-directory resolver (issue #462).
Covers the single-root consolidation: ``$OPENJARVIS_HOME`` >
``$XDG_DATA_HOME/openjarvis`` > ``~/.openjarvis``, backward compatibility
(no env => exactly ``~/.openjarvis``), and the source-tree rejection guard.
"""
from __future__ import annotations
from pathlib import Path
import pytest
from openjarvis.core import paths
def _clear_env(monkeypatch: pytest.MonkeyPatch) -> None:
"""Remove every env var that influences home resolution."""
for var in (
"OPENJARVIS_HOME",
"XDG_DATA_HOME",
"XDG_CONFIG_HOME",
"XDG_CACHE_HOME",
):
monkeypatch.delenv(var, raising=False)
class TestGetConfigDir:
"""Precedence and backward compatibility of get_config_dir()."""
def test_default_when_unset_is_legacy_dir(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
# Backward-compat: with nothing set, the resolved dir is exactly the
# historical ~/.openjarvis so existing installs are untouched.
_clear_env(monkeypatch)
assert paths.get_config_dir() == (Path.home() / ".openjarvis").resolve()
def test_respects_openjarvis_home(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
custom = tmp_path / "oj"
monkeypatch.setenv("OPENJARVIS_HOME", str(custom))
assert paths.get_config_dir() == custom.resolve()
def test_respects_xdg_data_home(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path))
# Single nested 'openjarvis' dir under XDG_DATA_HOME.
assert paths.get_config_dir() == (tmp_path / "openjarvis").resolve()
def test_openjarvis_home_wins_over_xdg(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
oj = tmp_path / "oj_wins"
monkeypatch.setenv("OPENJARVIS_HOME", str(oj))
monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "xdg_loses"))
assert paths.get_config_dir() == oj.resolve()
def test_expands_user_in_openjarvis_home(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("OPENJARVIS_HOME", "~/relocated-oj")
assert paths.get_config_dir() == (Path.home() / "relocated-oj").resolve()
def test_returns_absolute_path(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "rel"))
assert paths.get_config_dir().is_absolute()
class TestDerivedDirs:
"""config_path / data_dir / cache_dir all hang off the single root."""
def test_config_path(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "oj"))
assert paths.get_config_path() == (tmp_path / "oj" / "config.toml").resolve()
def test_data_dir_equals_config_dir(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "oj"))
assert paths.get_data_dir() == paths.get_config_dir()
def test_cache_dir_is_nested_cache(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "oj"))
assert paths.get_cache_dir() == (tmp_path / "oj" / "cache").resolve()
def test_cache_dir_under_xdg(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path))
assert paths.get_cache_dir() == (tmp_path / "openjarvis" / "cache").resolve()
class TestSourceTreeRejection:
"""A home pointing inside the repo must fail loudly (REVIEW.md)."""
def test_rejects_path_inside_source_tree(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
_clear_env(monkeypatch)
source_root = paths._find_source_root()
assert source_root is not None # We must be running inside the repo.
monkeypatch.setenv("OPENJARVIS_HOME", str(source_root / "junk_dir"))
with pytest.raises(paths.ConfigurationError, match="inside the source tree"):
paths.get_config_dir()
class TestLegacyConstantsHonorEnv:
"""The legacy DEFAULT_CONFIG_* names route through the env-aware resolver.
This is the exact split-brain bug from #462: the constant used to ignore
OPENJARVIS_HOME entirely. The constant is resolved once at import (the
install-script model, where the env is set before the process starts), and
every instance-level default goes through ``get_config_dir()`` so it honors
the override. ``DEFAULT_CONFIG_DIR`` stays a real attribute so existing
tests can ``monkeypatch.setattr`` it.
"""
def test_constant_matches_resolver_at_import(self) -> None:
from openjarvis.core import config
# The constant is the import-time resolution of the same function.
assert config.DEFAULT_CONFIG_DIR == paths.get_config_dir()
assert config.DEFAULT_CONFIG_PATH == paths.get_config_path()
def test_constant_is_a_real_settable_attribute(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# Install/CLI tests monkeypatch this attribute directly; it must be a
# real module attribute (not __getattr__-only) for setattr/undo to work.
from openjarvis.core import config
monkeypatch.setattr(config, "DEFAULT_CONFIG_DIR", tmp_path / "patched")
assert config.DEFAULT_CONFIG_DIR == tmp_path / "patched"
def test_dataclass_defaults_reflect_env(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# Config dataclass field defaults must resolve under the override at
# instantiation time, not freeze ~/.openjarvis at import.
_clear_env(monkeypatch)
from openjarvis.core.config import SessionConfig, StorageConfig
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "oj"))
root = (tmp_path / "oj").resolve()
assert StorageConfig().db_path == str(root / "memory.db")
assert SessionConfig().db_path == str(root / "sessions.db")
def test_downstream_consumer_honors_env(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# End-to-end: a non-config subsystem (credentials) resolves under the
# custom root, proving the override is no longer split-brain.
_clear_env(monkeypatch)
from openjarvis.core import credentials
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "oj"))
assert (
credentials._default_path()
== (tmp_path / "oj" / "credentials.toml").resolve()
)
+4 -1
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
from openjarvis.core.config import LearningConfig, SkillsLearningConfig
from openjarvis.core.paths import get_config_dir
class TestSkillsLearningConfig:
@@ -12,7 +13,9 @@ class TestSkillsLearningConfig:
assert cfg.optimizer == "dspy"
assert cfg.min_traces_per_skill == 20
assert cfg.optimization_interval_seconds == 86400
assert cfg.overlay_dir == "~/.openjarvis/learning/skills/"
# overlay_dir now resolves under the env-aware OpenJarvis root (#462),
# defaulting to <home>/learning/skills instead of the old literal.
assert cfg.overlay_dir == str(get_config_dir() / "learning" / "skills")
def test_can_be_constructed_with_all_fields(self):
cfg = SkillsLearningConfig(
+152
View File
@@ -9,9 +9,13 @@ import pytest
from openjarvis.core.registry import EngineRegistry
from openjarvis.core.types import Message, Role
from openjarvis.engine._base import EngineConnectionError
from openjarvis.engine.cloud import (
CloudEngine,
_is_codex_model,
_is_deepseek_model,
_is_openai_model,
_is_openrouter_model,
estimate_cost,
)
@@ -457,6 +461,7 @@ class TestCloudEngineCanServe:
"_google_client",
"_openrouter_client",
"_minimax_client",
"_deepseek_client",
"_codex_client",
):
setattr(eng, name, clients.get(name))
@@ -469,7 +474,154 @@ class TestCloudEngineCanServe:
assert eng.can_serve("gemini-2.5-pro") is False
assert eng.can_serve("openrouter/openai/gpt-4o") is False
def test_openai_key_does_not_claim_local_models(self) -> None:
"""#335: with only the OpenAI client set (e.g. a present-but-dummy
OPENAI_API_KEY), the cloud engine must NOT claim it can serve a local
Ollama model name otherwise it gets mis-selected as a fallback when
the local engine is transiently down and dies with "OpenAI client not
available". Only genuine OpenAI models route to the OpenAI client.
"""
eng = self._engine(_openai_client=object())
# Local Ollama / unrecognized names are NOT served by the cloud engine.
assert eng.can_serve("qwen3.5:0.8b") is False
assert eng.can_serve("llama3.2") is False
assert eng.can_serve("mistral") is False
assert eng.can_serve("phi3:mini") is False
assert eng.can_serve("some-unknown-model") is False
# Genuine OpenAI families still served.
assert eng.can_serve("gpt-4o") is True
assert eng.can_serve("gpt-5.4") is True
assert eng.can_serve("o3-mini") is True
def test_unknown_model_not_served_even_with_all_clients(self) -> None:
"""#335: an unrecognized model is declined regardless of how many
provider clients are configured it never falls through to OpenAI."""
eng = self._engine(
_openai_client=object(),
_anthropic_client=object(),
_google_client=object(),
_minimax_client=object(),
_deepseek_client=object(),
)
assert eng.can_serve("qwen3.5:0.8b") is False
assert eng.can_serve("totally-made-up") is False
def test_anthropic_only_serves_anthropic_models(self) -> None:
eng = self._engine(_anthropic_client=object())
assert eng.can_serve("claude-sonnet-4") is True
assert eng.can_serve("gpt-4o") is False
def test_deepseek_only_serves_deepseek_models(self) -> None:
"""The DeepSeek client serves deepseek-* models (and only those)."""
eng = self._engine(_deepseek_client=object())
assert eng.can_serve("deepseek-v4-flash") is True
assert eng.can_serve("deepseek-v4-pro") is True
assert eng.can_serve("DeepSeek-V4-Pro") is True # case-insensitive
assert eng.can_serve("gpt-4o") is False
# OpenRouter-prefixed deepseek is NOT the direct DeepSeek provider.
assert eng.can_serve("openrouter/deepseek/deepseek-r1") is False
class TestCloudEngineDeepSeek:
"""PR #504: DeepSeek as a first-class cloud provider (OpenAI-compatible)."""
def test_is_deepseek_model_predicate(self) -> None:
assert _is_deepseek_model("deepseek-v4-flash") is True
assert _is_deepseek_model("deepseek-v4-pro") is True
assert _is_deepseek_model("DeepSeek-V4-Pro") is True # case-insensitive
assert _is_deepseek_model("gpt-4o") is False
# No predicate collision: openrouter/deepseek/* belongs to OpenRouter.
assert _is_deepseek_model("openrouter/deepseek/deepseek-r1") is False
assert _is_openrouter_model("openrouter/deepseek/deepseek-r1") is True
# And a deepseek name is not mistaken for an OpenAI model.
assert _is_openai_model("deepseek-v4-pro") is False
def test_pricing_entries_present(self) -> None:
assert estimate_cost("deepseek-v4-flash", 1_000_000, 1_000_000) == (
pytest.approx(1.37) # 0.27 + 1.10
)
assert estimate_cost("deepseek-v4-pro", 1_000_000, 1_000_000) == (
pytest.approx(2.74) # 0.55 + 2.19
)
def test_init_wires_deepseek_client(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""DEEPSEEK_API_KEY builds an openai client pointed at api.deepseek.com."""
for var in ("OPENAI_API_KEY", "ANTHROPIC_API_KEY"):
monkeypatch.delenv(var, raising=False)
monkeypatch.setenv("DEEPSEEK_API_KEY", "sk-deepseek-test")
fake_openai = mock.MagicMock()
with mock.patch.dict("sys.modules", {"openai": fake_openai}):
EngineRegistry.register_value("cloud", CloudEngine)
engine = CloudEngine()
fake_openai.OpenAI.assert_any_call(
base_url="https://api.deepseek.com/v1",
api_key="sk-deepseek-test",
)
assert engine._deepseek_client is not None
def test_health_and_list_models_gated_on_deepseek_key(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
for var in ("OPENAI_API_KEY", "ANTHROPIC_API_KEY"):
monkeypatch.delenv(var, raising=False)
monkeypatch.setenv("DEEPSEEK_API_KEY", "sk-deepseek-test")
fake_openai = mock.MagicMock()
with mock.patch.dict("sys.modules", {"openai": fake_openai}):
EngineRegistry.register_value("cloud", CloudEngine)
engine = CloudEngine()
assert engine.health() is True
models = engine.list_models()
assert "deepseek-v4-flash" in models
assert "deepseek-v4-pro" in models
# can_serve must agree with list_models (regression for the missing
# _client_for_model deepseek branch flagged by the #504 verifier).
assert engine.can_serve("deepseek-v4-pro") is True
assert engine.can_serve("deepseek-v4-flash") is True
def test_generate_routes_to_deepseek_client(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
for var in ("OPENAI_API_KEY", "ANTHROPIC_API_KEY", "DEEPSEEK_API_KEY"):
monkeypatch.delenv(var, raising=False)
fake_usage = SimpleNamespace(
prompt_tokens=7, completion_tokens=3, total_tokens=10
)
fake_choice = SimpleNamespace(
message=SimpleNamespace(content="ds-hello"),
finish_reason="stop",
)
fake_resp = SimpleNamespace(
choices=[fake_choice], usage=fake_usage, model="deepseek-v4-pro"
)
fake_client = mock.MagicMock()
fake_client.chat.completions.create.return_value = fake_resp
EngineRegistry.register_value("cloud", CloudEngine)
engine = CloudEngine()
engine._deepseek_client = fake_client
result = engine.generate(
[Message(role=Role.USER, content="Hi")], model="deepseek-v4-pro"
)
assert result["content"] == "ds-hello"
assert result["usage"]["prompt_tokens"] == 7
# Routed to the DeepSeek client, not OpenAI.
fake_client.chat.completions.create.assert_called_once()
def test_generate_without_client_raises(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
for var in ("OPENAI_API_KEY", "ANTHROPIC_API_KEY", "DEEPSEEK_API_KEY"):
monkeypatch.delenv(var, raising=False)
EngineRegistry.register_value("cloud", CloudEngine)
engine = CloudEngine()
assert engine._deepseek_client is None
with pytest.raises(EngineConnectionError):
engine.generate(
[Message(role=Role.USER, content="Hi")], model="deepseek-v4-pro"
)

Some files were not shown because too many files have changed in this diff Show More