Compare commits

...
Author SHA1 Message Date
c25c649048 fix: strip openrouter/ prefix before forwarding to OpenRouter API (#672)
* fix: strip openrouter/ prefix before forwarding to OpenRouter API

cloud_router.get_provider() correctly detects LiteLLM-style
"openrouter/anthropic/claude-haiku-4.5" strings as the openrouter
provider, but was forwarding them verbatim, so the redundant prefix
reached OpenRouter's API and the request failed.

* fix: preserve native OpenRouter model IDs

---------

Co-authored-by: Ari <ari.silva@paipe.co>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 18:01:53 -07:00
Haotian Zheng f2df968aa5 fix(memory): replace chunks when re-indexing sources (#675) 2026-08-14 17:41:40 -07:00
Matéo H. PetelandElliot Slusky 8145597052 feat(scan): add application data-boundary diagnostic mode (#621)
* feat(security): add data boundary audit scanner

* feat(cli): add jarvis scan data-boundaries command

* test: cover data boundary audit and scan CLI

* docs(user-guide): document data boundary scan

* chore(scripts): add data boundary scan patch helper

* chore: remove patch script and harden PR verification

Drop the one-off apply patch helper, dedupe security user-guide copy, and add a CLI integration test ensuring scan --data-boundaries skips update checks.

* test/docs: verify update-check skip and trim security guide

Add integration test for scan --data-boundaries update-check suppression and replace duplicated three-layer section with a short link.

* test(scan): tighten data-boundary update-check regression

* test(scan): fix data-boundary test spacing

* test(scan): harden data-boundary tests against config defaults

* fix(scan): preserve symlink paths in data-boundary store checks

* fix(scan): address maintainer review on data-boundary audit

* test(scan): assert required tool names independently of audit constants

* fix(scan): classify outbound and remaining local-access tools

* fix(scan): sync data-boundary audit with current-main surfaces

Classify scan_chunks and browser_axtree, report knowledge.db and credentials.toml without reading contents, cover TOOL_CREDENTIALS env keys, and separate external egress from cloud API-key claims after the upstream merge.

* fix(scan): harden data-boundary audit follow-up

---------

Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 17:39:36 -07:00
aec96f9d5d fix(cli): pass prior conversation turns to agent.run() in chat REPL (#744)
* fix(cli): pass prior conversation turns to agent.run() in chat REPL

Why: jarvis chat built an AgentContext seeded only with an optional
memory-injected fact, never with the turn-by-turn conversation history
already tracked in `history`. Every agent-backed chat turn after the
first ran with no memory of what was said before it.

- src/openjarvis/cli/chat_cmd.py: always build AgentContext for
  agent-backed turns, seeded from prior non-system history messages,
  still layering the memory-fact message on top when present
- tests/cli/test_chat_cmd.py: regression test asserting the second
  turn's AgentContext carries the first turn's user/assistant messages

* fix(cli): keep memory context before chat history

---------

Co-authored-by: Ari <ari.silva@paipe.co>
Co-authored-by: Elliot Slusky <elliot@slusky.com>
2026-08-14 17:25:23 -07:00
Elliot Slusky c2e1c375aa Merge pull request #679 from gilbert-barajas/fix/serve-persona-prompt-builder
fix: served/SDK agents silently lose their persona (SOUL.md never reaches the model over HTTP)
2026-08-14 17:16:19 -07:00
Elliot Slusky ef28e5f84b fix: complete persona wiring across agent entry points 2026-08-14 17:11:25 -07:00
Ari f2483e7bf3 fix(frontend): move MessageBubble hooks above the early return
The three useMemo calls ran after the isUser early return, so they
were skipped entirely for user messages, violating React's rules of
hooks. Caught by the react-hooks/rules-of-hooks lint rule added last
commit. Pure reordering, no logic change; tsc and the full vitest
suite (38/38) still pass.
2026-08-14 17:05:00 -07:00
Elliot Slusky 156d41d2f9 Merge remote-tracking branch 'origin/main' into elliot/pr679-fix 2026-08-14 17:02:52 -07:00
Elliot Slusky 4b7bb936ff fix(desktop): complete remote server support
Allow plaintext user-configured API hosts in the macOS webview and authenticate remote agent event WebSockets with the configured API key. Add regression coverage for both paths.
2026-08-14 16:29:59 -07:00
Haotian Zheng 3c68a17ac5 fix(desktop): allow remote API connections 2026-08-14 16:29:59 -07:00
Elliot Slusky 0d32784ed6 fix(agents): harden structured tool input normalization 2026-08-14 16:29:01 -07:00
Haotian Zheng 20a7424883 fix(agents): normalize structured string tool inputs 2026-08-14 16:29:01 -07:00
github-actions[bot] a97c64c67b chore: update clone traffic data [skip ci] 2026-08-14 07:19:51 +00:00
Jon Saad-Falcon 64333651d1 Delete tools/pearl-reference-oracle directory 2026-08-13 19:27:37 -07:00
Elliot Slusky 9bee016c82 fix(server): preserve grounded agent stream content (#736)
* fix(server): preserve grounded agent stream content

* test(server): cover active grounded stream path

* fix(server): retain agent stream bridge
2026-08-13 18:15:36 -07:00
Gilbert Barajas 6e40d87eb5 fix: wire persona prompt_builder into the serve + SDK agent paths
cli/serve.py and sdk.py constructed their agents without passing
prompt_builder, so an agent reached over HTTP (jarvis serve) or via the
SDK silently lost its SOUL.md / MEMORY.md / USER.md persona while the
same agent via `jarvis ask` / `jarvis chat` kept it. cli/ask.py,
cli/chat_cmd.py, and the managed-agent executor already wired the
builder; these two entry points never did.

Found deploying a personal assistant: the server answered as a generic
assistant that explicitly denied being the persona, with SOUL.md
sitting correctly on disk the whole time. No error, no warning.

Mirrors the existing inspect.signature-guarded wiring from ask.py, so
agents whose __init__ doesn't accept the kwarg (e.g. OrchestratorAgent)
opt out automatically and keep their own system-prompt machinery.

Adds a serve-path regression test (tests/cli/test_serve_persona.py)
that fails without the fix: agent._prompt_builder is None on the
unpatched serve path, so the persona files never reach the model.
2026-07-27 00:50:56 -05:00
51 changed files with 5267 additions and 493 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"label": "Git Clones",
"message": "189,482",
"message": "190,252",
"color": "green",
"namedLogo": "git"
}
+4 -3
View File
@@ -1,6 +1,6 @@
{
"total_clones": 189482,
"last_updated": "2026-08-13T07:22:13Z",
"total_clones": 190252,
"last_updated": "2026-08-14T07:19:51Z",
"daily": {
"2026-03-27": 2189,
"2026-03-28": 1874,
@@ -140,6 +140,7 @@
"2026-08-09": 1076,
"2026-08-10": 1060,
"2026-08-11": 2182,
"2026-08-12": 641
"2026-08-12": 641,
"2026-08-13": 770
}
}
+174
View File
@@ -0,0 +1,174 @@
# Data-boundary scan
`jarvis scan --data-boundaries` reports application-level data boundaries in the
current OpenJarvis configuration. It complements the existing host/environment
scan, which checks OS posture such as disk encryption, cloud-sync agents, remote
access tools, and exposed engine ports.
The data-boundary scan is a configuration diagnostic. It is not a vulnerability
scanner, a legal privacy assessment, a network monitor, or an OAuth-scope audit.
## Run the scan
```bash
jarvis scan --data-boundaries
jarvis scan --data-boundaries --json
jarvis scan --data-boundaries --json --show-paths
jarvis scan --data-boundaries --strict
```
`--strict` exits with status code `1` when the report contains either a `fail`
or a `warn` finding. Use it when CI or pre-demo checks need to enforce a
conservative local-only posture.
Without `--strict`, the command always exits `0` even when fail or warn findings
are present. This is useful for exploratory review.
On a fresh `jarvis init` configuration, common warn findings include
`server.host = "0.0.0.0"` and `telemetry.enabled = true`. Running
`jarvis scan --data-boundaries --strict` after init therefore exits `1` until
those defaults are tightened.
Absolute paths and connector file basenames are redacted by default so JSON
reports can be pasted into issues without revealing local usernames, mount
points, or account labels. Use `--show-paths` only for local debugging.
## What it checks
The scan inspects configuration values, environment-variable presence, and the
existence of known local runtime files. It does not read private content from
memory databases, trace databases, connector credentials, prompt files, logs, or
OAuth token files.
The current checks cover:
- cloud-capable model provider, engine, and default model settings
- local memory context injection combined with cloud-capable inference
- traces, telemetry, learning, training, and spec-search settings
- automatic memory service (`tools.storage.enabled` / `[memory].enabled`)
- deep research engine and model settings
- security bypass flags when cloud inference is configured
- unset `security.profile` (informational)
- web search, browser, local file, shell, code, knowledge chunk scanning, and MCP tool surfaces
- local knowledge.db composition with cloud-capable Deep Research targets
- server binding and unauthenticated A2A exposure
- channel enablement, channel credential fields, and channel credential env vars
- skills, skill auto-sync, digest sources, and cloud speech/TTS backends such as Cartesia
- local stores such as `knowledge.db`, `credentials.toml`, `memory.db`, `traces.db`,
`telemetry.db`, `scheduler.db`, embeddings, skill index, `.vault_key`, and memory files
- connector credential files under `connectors/*.json`, without reading them
- API-key and other runtime credential environment variables (presence only)
- a scope note for frontend credential storage when cloud/API-key surfaces exist
Configured database paths (for example `traces.db_path` or `memory.db_path`)
are resolved from config when set, not only the default locations under the
OpenJarvis home directory.
Static Deep Research targeting uses configuration only (no request overrides):
`deep_research.engine` or `engine.default`, and `deep_research.model` or
`server.model` or `intelligence.default_model`.
Model identifiers that contain vendor names (for example `deepseek-r1` or
`openai/gpt-oss`) are not treated as cloud-bound when their effective engine is
explicitly local, such as Ollama.
## Status levels
| Status | Meaning |
| --- | --- |
| `fail` | A configuration composition is likely incompatible with strict local-only use. |
| `warn` | A configured surface may send data outside the local runtime or persist sensitive data. |
| `info` | A relevant setting or local store exists, with no immediate fail or warn condition. |
The command reports potential data paths. It does not prove that a path has been
used during a specific run.
JSON output includes `"schema_version": 1` for stable downstream parsing.
## Strict local-only checklist
For a conservative local-only setup, review these settings:
```toml
[analytics]
enabled = false
[traces]
enabled = false
[telemetry]
enabled = false
[agent]
context_from_memory = false
[intelligence]
provider = ""
preferred_engine = ""
default_model = "" # local model name only
[engine]
default = "ollama" # or another local engine
[tools]
enabled = ""
[tools.storage]
enabled = false
[tools.mcp]
enabled = false
servers = ""
[channel]
enabled = false
[learning]
enabled = false
auto_update = false
training_enabled = false
[learning.spec_search]
enabled = false
[server]
host = "127.0.0.1"
[security]
profile = "personal"
[a2a]
enabled = false
```
Also unset cloud and channel credentials from the process environment when they
are not needed.
## Scope and non-goals
The scan intentionally avoids reading private data. In particular, it does not:
- read connector JSON contents or OAuth scopes
- inspect browser `localStorage` or Tauri secure storage
- inspect frontend credential storage directly
- inspect installed skill source code
- intercept runtime network traffic
- classify provider retention or training policies
- prove that a configured path was used at runtime
Frontend credential storage is tracked separately from this CLI diagnostic. If a
cloud/API-key surface is present, the scan emits an informational scope note so
users know that browser/Tauri credential storage must be reviewed separately.
## Configuration resolution
The scan follows the same explicit configuration override used by the runtime:
if `OPENJARVIS_CONFIG` is set, that file is audited. Otherwise the scan uses
the default OpenJarvis config path under the resolved OpenJarvis home. If the
home directory cannot be resolved, the command reports a `config-root-error`
finding instead of crashing.
## See also
- [Security](security.md) — three-layer security model (host scan, config scan, BoundaryGuard)
- [Configuration](../getting-started/configuration.md) — full config reference
+21
View File
@@ -2,6 +2,20 @@
OpenJarvis includes a security layer that scans prompts and model outputs for secrets, personally identifiable information (PII), and sensitive file paths. The system is designed to be composable: scanners run as a pipeline, and the `GuardrailsEngine` wrapper drops in front of any inference backend without changing how the rest of your code works.
## Three layers of security review
OpenJarvis separates host posture, application data boundaries, and runtime prompt guardrails:
| Layer | Command / component | What it checks |
| --- | --- | --- |
| Host scan | `jarvis scan` | Disk encryption, cloud-sync agents, exposed engine ports, remote-access tools |
| Data-boundary scan | `jarvis scan --data-boundaries` | Configured inference, memory, traces, channels, tools, and local stores |
| Runtime guardrails | `GuardrailsEngine` / BoundaryGuard | Secrets, PII, and file-policy violations in live prompts and outputs |
Use the host scan before storing sensitive data on the machine. Use the data-boundary scan to verify whether your `config.toml` is local-only, cloud-capable, or mixed. Use BoundaryGuard during inference when you need live redaction or blocking.
See [Data Boundary Scan](data-boundary-scan.md) for the application config diagnostic and [BoundaryGuard](#guardrailsengine) below for runtime scanning.
---
## Overview
@@ -446,8 +460,15 @@ guarded = GuardrailsEngine(
---
## Data boundary scan
See [Data Boundary Scan](data-boundary-scan.md) for the application config diagnostic (`jarvis scan --data-boundaries`).
---
## See Also
- [Data Boundary Scan](data-boundary-scan.md) — application config and local-store diagnostic (`jarvis scan --data-boundaries`)
- [Architecture: Security](../architecture/security.md) — pipeline design, event flow, and file policy integration
- [API Reference: Security](../api-reference/openjarvis/security/index.md) — full class and function signatures
- [Tools](tools.md) — how `FileReadTool` uses file policy
+14
View File
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSAppTransportSecurity</key>
<dict>
<!-- The desktop API URL is user-configurable, so it cannot be represented
by Tauri's single, build-time exceptionDomain setting. Keep this
exception scoped to WKWebView; native URLSession traffic retains ATS. -->
<key>NSAllowsArbitraryLoadsInWebContent</key>
<true/>
</dict>
</dict>
</plist>
+1 -2
View File
@@ -24,7 +24,7 @@
}
],
"security": {
"csp": "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; connect-src 'self' http://localhost:* http://127.0.0.1:* ws://localhost:* ws://127.0.0.1:*; img-src 'self' data: blob:"
"csp": "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; connect-src 'self' http: https: ws: wss:; img-src 'self' data: blob:"
}
},
"bundle": {
@@ -45,7 +45,6 @@
"macOS": {
"entitlements": "Entitlements.plist",
"minimumSystemVersion": "10.15",
"exceptionDomain": "",
"frameworks": [],
"providerShortName": null,
"signingIdentity": "-"
+18 -18
View File
@@ -103,6 +103,24 @@ function CopyMessageButton({ content }: { content: string }) {
export function MessageBubble({ message, isLive = false }: Props) {
const isUser = message.role === 'user';
const cleanContent = useMemo(() => stripThinkTags(message.content), [message.content]);
// Build a ref→source lookup once per render. Memoized so the rehype plugin
// identity stays stable until the source list actually changes.
const sourcesMap = useMemo(() => {
const m = new Map<number, NonNullable<ChatMessage['researchSources']>[number]>();
for (const s of message.researchSources ?? []) {
if (typeof s.ref === 'number') m.set(s.ref, s);
}
return m;
}, [message.researchSources]);
const rehypePlugins = useMemo(() => {
const base: any[] = [[rehypeHighlight, { detect: true }], rehypeKatex];
if (sourcesMap.size > 0) base.push([rehypeCitations, { sources: sourcesMap }]);
return base;
}, [sourcesMap]);
if (isUser) {
return (
<div className="flex justify-end mb-4">
@@ -122,24 +140,6 @@ export function MessageBubble({ message, isLive = false }: Props) {
);
}
const cleanContent = useMemo(() => stripThinkTags(message.content), [message.content]);
// Build a ref→source lookup once per render. Memoized so the rehype plugin
// identity stays stable until the source list actually changes.
const sourcesMap = useMemo(() => {
const m = new Map<number, NonNullable<ChatMessage['researchSources']>[number]>();
for (const s of message.researchSources ?? []) {
if (typeof s.ref === 'number') m.set(s.ref, s);
}
return m;
}, [message.researchSources]);
const rehypePlugins = useMemo(() => {
const base: any[] = [[rehypeHighlight, { detect: true }], rehypeKatex];
if (sourcesMap.size > 0) base.push([rehypeCitations, { sources: sourcesMap }]);
return base;
}, [sourcesMap]);
return (
<div className="group mb-6">
{/* Deep Research timeline (steps + status) */}
+66
View File
@@ -0,0 +1,66 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { buildWsUrl } from './useAgentEvents';
const SETTINGS_KEY = 'openjarvis-settings';
class MemoryStorage {
private store = new Map<string, string>();
getItem(key: string): string | null {
return this.store.get(key) ?? null;
}
setItem(key: string, value: string): void {
this.store.set(key, String(value));
}
}
beforeEach(() => {
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
describe('buildWsUrl', () => {
it('authenticates agent events with the configured API key', () => {
localStorage.setItem(
SETTINGS_KEY,
JSON.stringify({
apiUrl: 'https://jarvis.example.com:8443',
apiKey: 'secret+/=',
}),
);
const url = new URL(buildWsUrl('agent/one'));
expect(url.origin).toBe('wss://jarvis.example.com:8443');
expect(url.pathname).toBe('/v1/agents/events');
expect(url.searchParams.get('agent_id')).toBe('agent/one');
expect(url.searchParams.get('token')).toBe('secret+/=');
});
it('normalizes a versioned API base without duplicating /v1', () => {
localStorage.setItem(
SETTINGS_KEY,
JSON.stringify({ apiUrl: 'http://192.0.2.10:8000/v1/' }),
);
expect(buildWsUrl()).toBe('ws://192.0.2.10:8000/v1/agents/events');
});
it('omits the token for a keyless server', () => {
localStorage.setItem(
SETTINGS_KEY,
JSON.stringify({ apiUrl: 'http://localhost:8000' }),
);
const url = new URL(buildWsUrl('agent-one'));
expect(url.searchParams.has('token')).toBe(false);
});
});
+10 -13
View File
@@ -1,5 +1,5 @@
import { useEffect, useRef } from 'react';
import { getBase } from './api';
import { getApiKey, getBase } from './api';
export interface AgentEvent {
type: string;
@@ -7,19 +7,16 @@ export interface AgentEvent {
data: Record<string, unknown>;
}
function buildWsUrl(agentId?: string): string {
export function buildWsUrl(agentId?: string): string {
const base = getBase();
let origin: string;
if (base) {
origin = base.replace(/^http/, 'ws');
} else {
const loc = window.location;
origin = `${loc.protocol === 'https:' ? 'wss:' : 'ws:'}//${loc.host}`;
}
const path = '/v1/agents/events';
return agentId
? `${origin}${path}?agent_id=${encodeURIComponent(agentId)}`
: `${origin}${path}`;
const url = new URL('/v1/agents/events', base || window.location.origin);
url.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:';
if (agentId) url.searchParams.set('agent_id', agentId);
const apiKey = getApiKey();
if (apiKey) url.searchParams.set('token', apiKey);
return url.toString();
}
/**
+1
View File
@@ -198,6 +198,7 @@ nav:
- Benchmarks: user-guide/benchmarks.md
- System Access: user-guide/system-access.md
- Security: user-guide/security.md
- Data Boundary Scan: user-guide/data-boundary-scan.md
- LLM-guided spec search: user-guide/llm-guided-spec-search.md
- Leaderboard: leaderboard.md
- Roadmap: development/roadmap.md
@@ -33,6 +33,33 @@ impl PySQLiteMemory {
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))
}
fn replace_source(
&self,
source: &str,
documents: Vec<(String, Option<String>)>,
) -> PyResult<Vec<String>> {
let parsed_documents = documents
.into_iter()
.map(|(content, metadata)| {
let metadata = metadata
.map(|value| serde_json::from_str(&value))
.transpose()
.map_err(|e| {
PyErr::new::<pyo3::exceptions::PyValueError, _>(e.to_string())
})?;
Ok((content, metadata))
})
.collect::<PyResult<Vec<_>>>()?;
let document_refs = parsed_documents
.iter()
.map(|(content, metadata)| (content.as_str(), metadata.as_ref()))
.collect::<Vec<_>>();
self.inner
.replace_source(source, &document_refs)
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))
}
#[pyo3(signature = (query, top_k=5))]
fn retrieve(&self, query: &str, top_k: usize) -> PyResult<String> {
let results = self
@@ -94,6 +94,57 @@ impl SQLiteMemory {
pub fn in_memory() -> Result<Self, OpenJarvisError> {
Self::new(Path::new(":memory:"))
}
/// Atomically replace every document for *source* with *documents*.
pub fn replace_source(
&self,
source: &str,
documents: &[(&str, Option<&Value>)],
) -> Result<Vec<String>, OpenJarvisError> {
let mut conn = self.conn.lock();
let tx = conn.transaction().map_err(|e| {
OpenJarvisError::Io(std::io::Error::other(e.to_string()))
})?;
tx.execute(
"DELETE FROM documents_fts
WHERE rowid IN (SELECT rowid FROM documents WHERE source = ?1)",
rusqlite::params![source],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
tx.execute(
"DELETE FROM documents WHERE source = ?1",
rusqlite::params![source],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
let mut doc_ids = Vec::with_capacity(documents.len());
for (content, metadata) in documents {
let doc_id = Uuid::new_v4().to_string();
let meta_str = metadata
.map(|m| serde_json::to_string(m).unwrap_or_default())
.unwrap_or_else(|| "{}".to_string());
tx.execute(
"INSERT INTO documents (id, content, source, metadata)
VALUES (?1, ?2, ?3, ?4)",
rusqlite::params![doc_id, content, source, meta_str],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
let rowid = tx.last_insert_rowid();
tx.execute(
"INSERT INTO documents_fts (rowid, content, source) VALUES (?1, ?2, ?3)",
rusqlite::params![rowid, content, source],
)
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
doc_ids.push(doc_id);
}
tx.commit()
.map_err(|e| OpenJarvisError::Io(std::io::Error::other(e.to_string())))?;
Ok(doc_ids)
}
}
impl MemoryBackend for SQLiteMemory {
@@ -306,6 +357,40 @@ mod tests {
assert_eq!(mem.count().unwrap(), 0);
}
#[test]
fn test_sqlite_replace_source_is_idempotent() {
let mem = SQLiteMemory::in_memory().unwrap();
mem.replace_source("notes.txt", &[("old project notes", None)])
.unwrap();
assert_eq!(mem.count().unwrap(), 1);
mem.replace_source("notes.txt", &[("updated project notes", None)])
.unwrap();
assert_eq!(mem.count().unwrap(), 1);
assert!(mem.retrieve("old", 5).unwrap().is_empty());
let updated = mem.retrieve("updated", 5).unwrap();
assert_eq!(updated.len(), 1);
assert_eq!(updated[0].source, "notes.txt");
}
#[test]
fn test_sqlite_replace_source_preserves_other_sources() {
let mem = SQLiteMemory::in_memory().unwrap();
mem.store("keep this manual", "manual.txt", None).unwrap();
mem.replace_source("notes.txt", &[("old project notes", None)])
.unwrap();
mem.replace_source("notes.txt", &[("updated project notes", None)])
.unwrap();
assert_eq!(mem.count().unwrap(), 2);
let manual = mem.retrieve("manual", 5).unwrap();
assert_eq!(manual.len(), 1);
assert_eq!(manual[0].source, "manual.txt");
}
#[test]
fn test_sqlite_case_insensitive_search() {
let mem = SQLiteMemory::in_memory().unwrap();
+21 -1
View File
@@ -4,8 +4,10 @@ from __future__ import annotations
from importlib.metadata import PackageNotFoundError
from importlib.metadata import version as _pkg_version
from typing import TYPE_CHECKING, Any
from openjarvis.sdk import Jarvis, JarvisSystem, MemoryHandle, SystemBuilder
if TYPE_CHECKING:
from openjarvis.sdk import Jarvis, JarvisSystem, MemoryHandle, SystemBuilder
try:
__version__ = _pkg_version("openjarvis")
@@ -13,3 +15,21 @@ except PackageNotFoundError: # pragma: no cover — uninstalled source tree
__version__ = "0.0.0+unknown"
__all__ = ["Jarvis", "JarvisSystem", "MemoryHandle", "SystemBuilder", "__version__"]
_SDK_EXPORTS = {"Jarvis", "JarvisSystem", "MemoryHandle", "SystemBuilder"}
def __getattr__(name: str) -> Any:
"""Load SDK exports lazily so lightweight CLI diagnostics can start safely."""
if name not in _SDK_EXPORTS:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
from openjarvis import sdk
value = getattr(sdk, name)
globals()[name] = value
return value
def __dir__() -> list[str]:
return sorted(set(globals()) | _SDK_EXPORTS)
+2 -1
View File
@@ -127,6 +127,7 @@ class MonitorOperativeAgent(ToolUsingAgent):
memory_backend: Optional[Any] = None,
interactive: bool = False,
confirm_callback=None,
prompt_builder: Optional[Any] = None,
**kwargs: Any,
) -> None:
super().__init__(
@@ -139,7 +140,7 @@ class MonitorOperativeAgent(ToolUsingAgent):
max_tokens=max_tokens,
interactive=interactive,
confirm_callback=confirm_callback,
prompt_builder=kwargs.get("prompt_builder"),
prompt_builder=prompt_builder,
)
# Validate strategies
if memory_extraction not in VALID_MEMORY_EXTRACTION:
+2 -1
View File
@@ -58,6 +58,7 @@ class OperativeAgent(ToolUsingAgent):
memory_backend: Optional[Any] = None,
interactive: bool = False,
confirm_callback=None,
prompt_builder: Optional[Any] = None,
**kwargs: Any,
) -> None:
super().__init__(
@@ -70,7 +71,7 @@ class OperativeAgent(ToolUsingAgent):
max_tokens=max_tokens,
interactive=interactive,
confirm_callback=confirm_callback,
prompt_builder=kwargs.get("prompt_builder"),
prompt_builder=prompt_builder,
)
self._system_prompt = system_prompt or ""
self._operator_id = operator_id
+81 -2
View File
@@ -13,6 +13,7 @@ Supports two modes:
from __future__ import annotations
import concurrent.futures
import json
import re
from typing import Any, List, Optional
@@ -142,12 +143,21 @@ class OrchestratorAgent(ToolUsingAgent):
tool_call = ToolCall(
id=f"orch_{turns}",
name=parsed["tool"],
arguments=parsed["input"] or "{}",
arguments=self._normalize_structured_tool_input(
parsed["tool"],
parsed["input"],
),
)
tool_result = self._executor.execute(tool_call)
all_tool_results.append(tool_result)
observation = f"Observation: {tool_result.content}"
if tool_result.success:
observation = f"Observation: {tool_result.content}"
else:
observation = (
f"Observation: Tool '{tool_result.tool_name}' failed: "
f"{tool_result.content}"
)
messages.append(Message(role=Role.USER, content=observation))
continue
@@ -162,6 +172,75 @@ class OrchestratorAgent(ToolUsingAgent):
# Max turns exceeded
return self._max_turns_result(all_tool_results, turns)
def _normalize_structured_tool_input(
self,
tool_name: str,
raw_input: str,
) -> str:
"""Map unambiguous structured text input to a string parameter."""
if not raw_input:
return "{}"
try:
parsed_input = json.loads(raw_input)
except json.JSONDecodeError:
invalid_json = True
string_value = raw_input
else:
invalid_json = False
if isinstance(parsed_input, dict):
return raw_input
# INPUT is a text protocol. A non-object JSON value such as 42,
# true, null, or [1, 2] may still be the intended text for a tool's
# string parameter. Quoted JSON strings are decoded to remove only
# their surrounding quotes; other values retain their source text.
string_value = parsed_input if isinstance(parsed_input, str) else raw_input
tool_spec = None
for candidate in reversed(self._tools):
candidate_spec = candidate.spec
if candidate_spec.name == tool_name:
tool_spec = candidate_spec
break
if tool_spec is None:
return raw_input
parameters = tool_spec.parameters
parameter_container_type = parameters.get("type")
if parameter_container_type not in (None, "object"):
return raw_input
properties = parameters.get("properties", {})
required = parameters.get("required", [])
if not isinstance(properties, dict) or not isinstance(required, list):
return raw_input
if len(required) == 1 and required[0] in properties:
parameter_name = required[0]
elif not required and len(properties) == 1:
parameter_name = next(iter(properties))
else:
return raw_input
parameter_schema = properties[parameter_name]
if not isinstance(parameter_schema, dict):
return raw_input
parameter_type = parameter_schema.get("type")
accepts_string = parameter_type == "string" or (
isinstance(parameter_type, list) and "string" in parameter_type
)
if not accepts_string:
return raw_input
allow_object_text = (
tool_spec.metadata.get("structured_allow_object_text") is True
)
starts_like_object = raw_input.lstrip("\ufeff \t\r\n").startswith("{")
if invalid_json and starts_like_object and not allow_object_text:
return raw_input
return json.dumps({parameter_name: string_value})
@staticmethod
def _parse_structured_response(text: str) -> dict:
"""Parse THOUGHT/TOOL/INPUT/FINAL_ANSWER from model output."""
+129 -99
View File
@@ -2,45 +2,36 @@
from __future__ import annotations
import sys
import click
import openjarvis
from openjarvis.cli._bootstrap import bootstrap_cmd
from openjarvis.cli.add_cmd import add
from openjarvis.cli.agent_cmd import agent
from openjarvis.cli.ask import ask
from openjarvis.cli.bench_cmd import bench
from openjarvis.cli.channel_cmd import channel
from openjarvis.cli.channels_cmd import channels
from openjarvis.cli.chat_cmd import chat
from openjarvis.cli.compose_cmd import compose
from openjarvis.cli.config_cmd import config
from openjarvis.cli.connect_cmd import connect
from openjarvis.cli.daemon_cmd import restart, start, status, stop
from openjarvis.cli.digest_cmd import digest
from openjarvis.cli.doctor_cmd import doctor
from openjarvis.cli.eval_cmd import eval_group
from openjarvis.cli.feedback_cmd import feedback_group
from openjarvis.cli.gateway_cmd import gateway
from openjarvis.cli.host_cmd import host
from openjarvis.cli.init_cmd import init
from openjarvis.cli.memory_cmd import memory
from openjarvis.cli.mine_cmd import mine
from openjarvis.cli.model import model
from openjarvis.cli.operators_cmd import operators
from openjarvis.cli.optimize_cmd import optimize_group
from openjarvis.cli.pearl_cmd import pearl
from openjarvis.cli.quickstart_cmd import quickstart
from openjarvis.cli.registry_cmd import registry
from openjarvis.cli.scan_cmd import scan
from openjarvis.cli.scheduler_cmd import scheduler
from openjarvis.cli.self_update_cmd import self_update
from openjarvis.cli.serve import serve
from openjarvis.cli.skill_cmd import skill
from openjarvis.cli.telemetry_cmd import telemetry
from openjarvis.cli.tool_cmd import tool
from openjarvis.cli.vault_cmd import vault
from openjarvis.cli.workflow_cmd import workflow
def _invoked_command(argv: list[str]) -> str:
"""Return the first positional CLI token after global flags."""
for arg in argv:
if arg.startswith("-"):
continue
return arg
return ""
# A data-boundary scan must be able to diagnose an invalid OPENJARVIS_HOME.
# Importing the rest of the CLI eagerly would import core.config and resolve that
# path before the scan can turn the failure into a finding.
_DATA_BOUNDARY_BOOTSTRAP = (
_invoked_command(sys.argv[1:]) == "scan" and "--data-boundaries" in sys.argv[1:]
)
def _should_skip_update_check(ctx: click.Context, argv: list[str]) -> bool:
"""Return true for commands whose diagnostics should remain local-only."""
if "--research" in argv:
return True
return ctx.invoked_subcommand == "scan" and "--data-boundaries" in argv
@click.group(
@@ -63,11 +54,13 @@ def cli(ctx: click.Context, verbose: bool, quiet: bool) -> None:
# Check for updates on interactive commands. The banner is noise in
# demo recordings of ``jarvis ask --research``, so skip it whenever
# the research flag is in argv (cheap argv sniff — Click hasn't
# parsed the subcommand's args yet at this point).
# parsed the subcommand's args yet at this point). Also skip
# ``jarvis scan --data-boundaries`` because it is intended to be a
# local application-data diagnostic with no outbound calls.
import sys
research_mode_active = "--research" in sys.argv
if not quiet and ctx.invoked_subcommand and not research_mode_active:
skip_update_check = _should_skip_update_check(ctx, sys.argv[1:])
if not quiet and ctx.invoked_subcommand and not skip_update_check:
import threading
from openjarvis.cli._version_check import check_for_updates
@@ -91,74 +84,111 @@ def cli(ctx: click.Context, verbose: bool, quiet: bool) -> None:
check_and_route(ctx)
cli.add_command(init, "init")
cli.add_command(ask, "ask")
cli.add_command(chat, "chat")
cli.add_command(serve, "serve")
cli.add_command(model, "model")
cli.add_command(memory, "memory")
cli.add_command(mine, "mine")
cli.add_command(pearl, "pearl")
cli.add_command(telemetry, "telemetry")
cli.add_command(bench, "bench")
cli.add_command(channel, "channel")
cli.add_command(channels, "channels")
cli.add_command(scheduler, "scheduler")
cli.add_command(doctor, "doctor")
cli.add_command(agent, "agents")
cli.add_command(workflow, "workflow")
cli.add_command(skill, "skill")
cli.add_command(start, "start")
cli.add_command(stop, "stop")
cli.add_command(restart, "restart")
cli.add_command(status, "status")
cli.add_command(vault, "vault")
cli.add_command(add, "add")
cli.add_command(operators, "operators")
cli.add_command(eval_group, "eval")
cli.add_command(host, "host")
cli.add_command(quickstart, "quickstart")
cli.add_command(optimize_group, "optimize")
cli.add_command(feedback_group, "feedback")
cli.add_command(compose, "compose")
cli.add_command(gateway, "gateway")
cli.add_command(tool, "tool")
cli.add_command(registry, "registry")
cli.add_command(config, "config")
cli.add_command(scan, "scan")
cli.add_command(connect, "connect")
cli.add_command(digest, "digest")
# deep-research setup pulls the ingestion pipeline (embeddings/numpy). Guard it
# so a broken or slow numpy on Windows — which can raise at IMPORT time, not
# just ImportError (#404) — can never take down the whole CLI, including
# `jarvis serve`. Invoking `jarvis deep-research-setup` without the deps still
# errors clearly on demand.
try:
from openjarvis.cli.deep_research_setup_cmd import deep_research_setup
if not _DATA_BOUNDARY_BOOTSTRAP:
from openjarvis.cli._bootstrap import bootstrap_cmd
from openjarvis.cli.add_cmd import add
from openjarvis.cli.agent_cmd import agent
from openjarvis.cli.ask import ask
from openjarvis.cli.bench_cmd import bench
from openjarvis.cli.channel_cmd import channel
from openjarvis.cli.channels_cmd import channels
from openjarvis.cli.chat_cmd import chat
from openjarvis.cli.compose_cmd import compose
from openjarvis.cli.config_cmd import config
from openjarvis.cli.connect_cmd import connect
from openjarvis.cli.daemon_cmd import restart, start, status, stop
from openjarvis.cli.digest_cmd import digest
from openjarvis.cli.doctor_cmd import doctor
from openjarvis.cli.eval_cmd import eval_group
from openjarvis.cli.feedback_cmd import feedback_group
from openjarvis.cli.gateway_cmd import gateway
from openjarvis.cli.host_cmd import host
from openjarvis.cli.init_cmd import init
from openjarvis.cli.memory_cmd import memory
from openjarvis.cli.mine_cmd import mine
from openjarvis.cli.model import model
from openjarvis.cli.operators_cmd import operators
from openjarvis.cli.optimize_cmd import optimize_group
from openjarvis.cli.pearl_cmd import pearl
from openjarvis.cli.quickstart_cmd import quickstart
from openjarvis.cli.registry_cmd import registry
from openjarvis.cli.scheduler_cmd import scheduler
from openjarvis.cli.self_update_cmd import self_update
from openjarvis.cli.serve import serve
from openjarvis.cli.skill_cmd import skill
from openjarvis.cli.telemetry_cmd import telemetry
from openjarvis.cli.tool_cmd import tool
from openjarvis.cli.vault_cmd import vault
from openjarvis.cli.workflow_cmd import workflow
cli.add_command(deep_research_setup, "deep-research-setup")
cli.add_command(deep_research_setup, "research")
except Exception as _dr_exc:
import logging as _logging
cli.add_command(init, "init")
cli.add_command(ask, "ask")
cli.add_command(chat, "chat")
cli.add_command(serve, "serve")
cli.add_command(model, "model")
cli.add_command(memory, "memory")
cli.add_command(mine, "mine")
cli.add_command(pearl, "pearl")
cli.add_command(telemetry, "telemetry")
cli.add_command(bench, "bench")
cli.add_command(channel, "channel")
cli.add_command(channels, "channels")
cli.add_command(scheduler, "scheduler")
cli.add_command(doctor, "doctor")
cli.add_command(agent, "agents")
cli.add_command(workflow, "workflow")
cli.add_command(skill, "skill")
cli.add_command(start, "start")
cli.add_command(stop, "stop")
cli.add_command(restart, "restart")
cli.add_command(status, "status")
cli.add_command(vault, "vault")
cli.add_command(add, "add")
cli.add_command(operators, "operators")
cli.add_command(eval_group, "eval")
cli.add_command(host, "host")
cli.add_command(quickstart, "quickstart")
cli.add_command(optimize_group, "optimize")
cli.add_command(feedback_group, "feedback")
cli.add_command(compose, "compose")
cli.add_command(gateway, "gateway")
cli.add_command(tool, "tool")
cli.add_command(registry, "registry")
cli.add_command(config, "config")
cli.add_command(connect, "connect")
cli.add_command(digest, "digest")
_logging.getLogger(__name__).debug("deep-research command unavailable: %s", _dr_exc)
cli.add_command(self_update, "self-update")
cli.add_command(bootstrap_cmd, "_bootstrap")
# Deep Research setup pulls the ingestion pipeline (embeddings/numpy). Guard
# it so an import-time dependency failure cannot take down the whole CLI.
try:
from openjarvis.cli.deep_research_setup_cmd import deep_research_setup
# Gateway CLI commands (lazy import to avoid pulling starlette)
try:
from openjarvis.cli.auth_cmd import auth
cli.add_command(deep_research_setup, "deep-research-setup")
cli.add_command(deep_research_setup, "research")
except Exception as _dr_exc:
import logging as _logging
cli.add_command(auth, "auth")
except ImportError:
pass
_logging.getLogger(__name__).debug(
"deep-research command unavailable: %s", _dr_exc
)
cli.add_command(self_update, "self-update")
cli.add_command(bootstrap_cmd, "_bootstrap")
try:
from openjarvis.cli.tunnel_cmd import tunnel
# Gateway CLI commands (lazy import to avoid pulling starlette)
try:
from openjarvis.cli.auth_cmd import auth
cli.add_command(tunnel, "tunnel")
except ImportError:
pass
cli.add_command(auth, "auth")
except ImportError:
pass
try:
from openjarvis.cli.tunnel_cmd import tunnel
cli.add_command(tunnel, "tunnel")
except ImportError:
pass
def main() -> None:
+2 -3
View File
@@ -398,9 +398,8 @@ def _run_agent(
# Wire the SystemPromptBuilder so SOUL.md / MEMORY.md / USER.md persona
# files actually reach the model. Only passed to agents whose __init__
# accepts a `prompt_builder` kwarg (BaseAgent does; agents that override
# __init__ without forwarding it, e.g. OrchestratorAgent, opt out
# automatically and keep their existing system-prompt machinery).
# explicitly accepts a `prompt_builder` kwarg. Agents with specialized
# prompt machinery opt in by naming and forwarding the parameter.
import inspect as _inspect
if "prompt_builder" in _inspect.signature(agent_cls.__init__).parameters:
+6 -4
View File
@@ -311,12 +311,14 @@ def chat(
# Generate response even when optional memory context is unavailable.
try:
if agent is not None:
agent_context = None
if agent_context_message is not None:
from openjarvis.agents._stubs import AgentContext
from openjarvis.agents._stubs import AgentContext
agent_context = AgentContext()
agent_context = AgentContext()
if agent_context_message is not None:
agent_context.conversation.add(agent_context_message)
for msg in history[:-1]:
if msg.role != Role.SYSTEM:
agent_context.conversation.add(msg)
response = agent.run(user_input, context=agent_context)
content = (
response.content if hasattr(response, "content") else str(response)
+33 -9
View File
@@ -89,15 +89,39 @@ def index(
mem = _get_backend(backend)
try:
for chunk in track(chunks, description="Storing chunks...", console=console):
mem.store(
chunk.content,
source=chunk.source,
metadata={
"offset": chunk.offset,
"index": chunk.index,
},
)
replace_source = getattr(mem, "replace_source", None)
if callable(replace_source):
documents_by_source = {}
for chunk in chunks:
documents_by_source.setdefault(chunk.source, []).append(
(
chunk.content,
{
"offset": chunk.offset,
"index": chunk.index,
},
)
)
for source, documents in track(
documents_by_source.items(),
description="Replacing sources...",
console=console,
):
replace_source(source, documents)
else:
for chunk in track(
chunks,
description="Storing chunks...",
console=console,
):
mem.store(
chunk.content,
source=chunk.source,
metadata={
"offset": chunk.offset,
"index": chunk.index,
},
)
finally:
if hasattr(mem, "close"):
mem.close()
+151 -5
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import json
import os
import subprocess
import sys
from dataclasses import dataclass
@@ -11,7 +12,11 @@ from typing import Callable, List
import click
from openjarvis.core.paths import get_config_dir
from openjarvis.core.paths import get_config_dir, get_config_path
from openjarvis.security.data_boundary_audit import (
DataBoundaryReport,
build_data_boundary_report,
)
# Engine ports that should only be listening on localhost.
_ENGINE_PORTS = {11434, 8080, 8000, 30000, 1234, 52415, 18181}
@@ -441,10 +446,45 @@ _RICH_ICONS = {
"ok": "[green]\u2713[/green]",
"warn": "[yellow]![/yellow]",
"fail": "[red]\u2717[/red]",
"info": "[blue]i[/blue]",
"skip": "[dim]-[/dim]",
}
def _resolve_data_boundary_config_path() -> Path:
env_config = os.environ.get("OPENJARVIS_CONFIG")
if env_config:
return Path(env_config).expanduser().resolve()
return get_config_path()
def _load_data_boundary_config():
"""Load config without treating missing config as active."""
root = None
root_error = ""
try:
root = get_config_dir()
config_path = _resolve_data_boundary_config_path()
except Exception as exc:
config_path = None
root_error = f"{type(exc).__name__}: {exc}"
if root_error:
return None, root, False, "", root_error
try:
from openjarvis.core.config import JarvisConfig, load_config
except Exception as exc:
return None, root, False, f"{type(exc).__name__}: {exc}", ""
if config_path is None or not config_path.exists():
return JarvisConfig(), root, False, "", root_error
try:
return load_config(config_path), root, True, "", root_error
except Exception as exc:
return JarvisConfig(), root, False, f"{type(exc).__name__}: {exc}", root_error
def _render_results(results: List[ScanResult]) -> None:
"""Render scan results as a Rich table."""
from rich.console import Console
@@ -494,17 +534,123 @@ def _render_results(results: List[ScanResult]) -> None:
console.print()
def _render_data_boundary_report(
report: DataBoundaryReport,
*,
show_paths: bool,
) -> None:
"""Render application data-boundary findings as a Rich table."""
from rich.console import Console
from rich.table import Table
console = Console()
console.print()
console.print("[bold]OpenJarvis Data-Boundary Scan[/bold]")
console.print(f"Verdict: [bold]{report.verdict}[/bold]")
console.print()
table = Table(show_header=True, header_style="bold", show_lines=True)
table.add_column("", width=3, justify="center")
table.add_column("Finding")
table.add_column("Recommendation")
for finding in report.findings:
icon = _RICH_ICONS.get(finding.status, "?")
style = {"fail": "red", "warn": "yellow", "info": "blue"}.get(
finding.status,
"white",
)
details = [f"[{style}]{finding.title}[/{style}]"]
details.append(f"[dim]{finding.potential_data_path}[/dim]")
if finding.location:
location = finding.absolute_location if show_paths else finding.location
details.append(f"[dim]Location: {location}[/dim]")
table.add_row(icon, "\n".join(details), finding.recommendation)
console.print(table)
summary = report.summary()
console.print()
console.print(
f" [red]{summary['fail']} fail[/red], "
f"[yellow]{summary['warn']} warning(s)[/yellow], "
f"[blue]{summary['info']} info[/blue]"
)
if not show_paths:
console.print(
" [dim]Absolute paths and connector basenames are redacted by default. "
"Use --show-paths for local debugging.[/dim]"
)
console.print()
def _emit_data_boundary_json(
report: DataBoundaryReport,
*,
show_paths: bool,
) -> None:
click.echo(json.dumps(report.to_dict(show_paths=show_paths), indent=2))
@click.command()
@click.option("--quick", is_flag=True, default=False, help="Run only critical checks.")
@click.option("--json", "as_json", is_flag=True, default=False, help="Output as JSON.")
def scan(quick: bool, as_json: bool) -> None:
@click.option(
"--data-boundaries",
is_flag=True,
default=False,
help="Run application data-boundary checks instead of host checks.",
)
@click.option(
"--strict",
is_flag=True,
default=False,
help="Exit non-zero if data-boundary fail or warn findings are present.",
)
@click.option(
"--show-paths",
is_flag=True,
default=False,
help="Show absolute paths in data-boundary output.",
)
def scan(
quick: bool,
as_json: bool,
data_boundaries: bool,
strict: bool,
show_paths: bool,
) -> None:
"""Audit your environment for privacy and security risks."""
if data_boundaries:
if quick:
raise click.UsageError("--quick cannot be combined with --data-boundaries.")
config, root, config_loaded, config_error, root_error = (
_load_data_boundary_config()
)
report = build_data_boundary_report(
config,
root,
config_loaded=config_loaded,
config_error=config_error,
root_error=root_error,
)
if as_json:
_emit_data_boundary_json(report, show_paths=show_paths)
else:
_render_data_boundary_report(report, show_paths=show_paths)
summary = report.summary()
if strict and (summary["fail"] or summary["warn"]):
raise click.exceptions.Exit(1)
return
if strict or show_paths:
raise click.UsageError(
"--strict and --show-paths are only supported with --data-boundaries."
)
scanner = PrivacyScanner()
results: List[ScanResult] = scanner.run_quick() if quick else scanner.run_all()
if as_json:
import json as json_mod
output = [
{
"name": r.name,
@@ -514,7 +660,7 @@ def scan(quick: bool, as_json: bool) -> None:
}
for r in results
]
click.echo(json_mod.dumps(output, indent=2))
click.echo(json.dumps(output, indent=2))
return
if not results:
+21
View File
@@ -367,6 +367,27 @@ def serve(
if getattr(agent_cls, "accepts_tools", False):
agent_kwargs["max_turns"] = config.agent.max_turns
# Wire the SystemPromptBuilder so SOUL.md / MEMORY.md / USER.md
# reach the model on the SERVE path too. ``ask.py`` has done
# this since the persona system landed; ``serve.py`` never did,
# so an agent served over HTTP silently answered as a generic
# assistant while the same agent via the CLI kept its persona.
# Guarded so agents with specialized prompt machinery must opt
# in by explicitly naming and forwarding the kwarg.
import inspect as _inspect
if (
"prompt_builder"
in _inspect.signature(agent_cls.__init__).parameters
):
from openjarvis.prompt.builder import SystemPromptBuilder
agent_kwargs["prompt_builder"] = SystemPromptBuilder(
agent_template=config.agent.default_system_prompt or "",
memory_files_config=config.memory_files,
system_prompt_config=config.system_prompt,
)
agent = agent_cls(engine, model_name, **agent_kwargs)
# Pin MCP transports to the agent's lifetime so HTTP
# connections don't close mid-request (#461).
+14
View File
@@ -516,6 +516,20 @@ class Jarvis:
existing = agent_kwargs.get("tools", [])
agent_kwargs["tools"] = digest_tools + list(existing)
# Wire the SystemPromptBuilder so SOUL.md / MEMORY.md / USER.md reach
# the model — mirrors ``cli/ask.py`` and ``cli/serve.py``. Guarded so
# agents whose ``__init__`` doesn't accept the kwarg opt out.
import inspect as _inspect
if "prompt_builder" in _inspect.signature(agent_cls.__init__).parameters:
from openjarvis.prompt.builder import SystemPromptBuilder
agent_kwargs["prompt_builder"] = SystemPromptBuilder(
agent_template=self._config.agent.default_system_prompt or "",
memory_files_config=self._config.memory_files,
system_prompt_config=self._config.system_prompt,
)
agent_obj = agent_cls(self._engine, model_name, **agent_kwargs)
ctx = AgentContext()
+50 -18
View File
@@ -4,27 +4,10 @@ from __future__ import annotations
import logging
from dataclasses import dataclass
from importlib import import_module
from typing import Any, Optional
from openjarvis.core.events import EventBus
from openjarvis.security._stubs import BaseScanner
from openjarvis.security.audit import AuditLogger
from openjarvis.security.file_policy import (
DEFAULT_SENSITIVE_PATTERNS,
filter_sensitive_paths,
is_sensitive_file,
)
from openjarvis.security.guardrails import GuardrailsEngine, SecurityBlockError
from openjarvis.security.scanner import PIIScanner, SecretScanner
from openjarvis.security.ssrf import check_ssrf, is_private_ip
from openjarvis.security.types import (
RedactionMode,
ScanFinding,
ScanResult,
SecurityEvent,
SecurityEventType,
ThreatLevel,
)
logger = logging.getLogger(__name__)
@@ -50,6 +33,12 @@ def setup_security(
if not config.security.enabled:
return SecurityContext(engine=engine)
from openjarvis.security._stubs import BaseScanner
from openjarvis.security.audit import AuditLogger
from openjarvis.security.guardrails import GuardrailsEngine
from openjarvis.security.scanner import PIIScanner, SecretScanner
from openjarvis.security.types import RedactionMode
# Scanners + engine wrapping
try:
scanners: list[BaseScanner] = []
@@ -121,3 +110,46 @@ __all__ = [
"is_sensitive_file",
"setup_security",
]
_LAZY_EXPORTS = {
"AuditLogger": ("openjarvis.security.audit", "AuditLogger"),
"BaseScanner": ("openjarvis.security._stubs", "BaseScanner"),
"DEFAULT_SENSITIVE_PATTERNS": (
"openjarvis.security.file_policy",
"DEFAULT_SENSITIVE_PATTERNS",
),
"GuardrailsEngine": ("openjarvis.security.guardrails", "GuardrailsEngine"),
"PIIScanner": ("openjarvis.security.scanner", "PIIScanner"),
"RedactionMode": ("openjarvis.security.types", "RedactionMode"),
"ScanFinding": ("openjarvis.security.types", "ScanFinding"),
"ScanResult": ("openjarvis.security.types", "ScanResult"),
"SecretScanner": ("openjarvis.security.scanner", "SecretScanner"),
"SecurityBlockError": (
"openjarvis.security.guardrails",
"SecurityBlockError",
),
"SecurityEvent": ("openjarvis.security.types", "SecurityEvent"),
"SecurityEventType": ("openjarvis.security.types", "SecurityEventType"),
"ThreatLevel": ("openjarvis.security.types", "ThreatLevel"),
"check_ssrf": ("openjarvis.security.ssrf", "check_ssrf"),
"filter_sensitive_paths": (
"openjarvis.security.file_policy",
"filter_sensitive_paths",
),
"is_private_ip": ("openjarvis.security.ssrf", "is_private_ip"),
"is_sensitive_file": ("openjarvis.security.file_policy", "is_sensitive_file"),
}
def __getattr__(name: str) -> Any:
target = _LAZY_EXPORTS.get(name)
if target is None:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
module_name, attribute = target
value = getattr(import_module(module_name), attribute)
globals()[name] = value
return value
def __dir__() -> list[str]:
return sorted(set(globals()) | set(_LAZY_EXPORTS))
File diff suppressed because it is too large Load Diff
+12 -1
View File
@@ -84,6 +84,17 @@ def is_cloud_model(model: str) -> bool:
return get_provider(model) is not None
def _openrouter_model_id(model: str) -> str:
"""Return the provider-facing ID for an OpenRouter model."""
prefix = "openrouter/"
candidate = model.removeprefix(prefix)
# OpenRouter owns IDs such as "openrouter/auto" itself. Only remove the
# LiteLLM routing prefix when the remainder is still a provider/model ID.
if model.startswith(prefix) and "/" in candidate:
return candidate
return model
# ---------------------------------------------------------------------------
# Message conversion
# ---------------------------------------------------------------------------
@@ -371,7 +382,7 @@ async def stream_cloud(
"OPENROUTER_API_KEY not set — add it in the Cloud Models tab"
)
async for token in _stream_openai(
model,
_openrouter_model_id(model),
messages,
temperature,
max_tokens,
+8 -55
View File
@@ -246,62 +246,15 @@ class AgentStreamBridge:
{"results": tool_results_data},
)
# Stream content using real LLM token streaming via
# engine.stream_full() when the engine is available.
# ``agent.run()`` already produced the authoritative, grounded
# response. Do not call the engine again here: a second inference
# would not have the agent's system prompt, tool transcript, or
# other internal context and could therefore contradict the
# result reported by the agent events. Replay the final content
# in chunks so the OpenAI-compatible streaming response stays
# consistent with the completed agent run.
content = agent_result.content or ""
engine = getattr(self._agent, "_engine", None)
used_real_streaming = False
if engine is not None and hasattr(engine, "stream_full") and content:
# Re-stream using the engine for real token delivery.
# Build the same messages the agent used for its final turn.
try:
from openjarvis.core.types import Message as MsgType
from openjarvis.core.types import Role as RoleType
replay_messages = []
for m in self._request.messages:
role = (
RoleType(m.role)
if m.role in {r.value for r in RoleType}
else RoleType.USER
)
replay_messages.append(
MsgType(
role=role,
content=m.content or "",
name=m.name,
tool_call_id=m.tool_call_id,
)
)
async for sc in engine.stream_full(
replay_messages,
model=self._model,
):
if sc.content:
chunk = ChatCompletionChunk(
id=self._chunk_id,
model=self._model,
choices=[
StreamChoice(
delta=DeltaMessage(content=sc.content),
)
],
)
yield f"data: {chunk.model_dump_json()}\n\n"
used_real_streaming = True
except Exception as stream_exc:
import logging as _logging
_logger = _logging.getLogger("openjarvis.server")
_logger.warning(
"Real streaming failed, falling back to word replay: %s",
stream_exc,
)
# Fallback: word-by-word replay if real streaming was not used
if not used_real_streaming and content:
if content:
words = content.split(" ")
for i, word in enumerate(words):
token = word if i == 0 else " " + word
+18
View File
@@ -136,6 +136,15 @@ class ToolExecutor:
content=f"Invalid arguments JSON: {exc}",
success=False,
)
if not isinstance(params, dict):
return ToolResult(
tool_name=tool_call.name,
content=(
"Invalid arguments: expected a JSON object, "
f"got {type(params).__name__}."
),
success=False,
)
# Boundary guard: scan external tool arguments
if self._boundary_guard is not None and not getattr(tool, "is_local", True):
@@ -143,6 +152,15 @@ class ToolExecutor:
tool_call = self._boundary_guard.check_outbound(tool_call)
# Re-parse arguments after potential redaction
params = json.loads(tool_call.arguments) if tool_call.arguments else {}
if not isinstance(params, dict):
return ToolResult(
tool_name=tool_call.name,
content=(
"Invalid arguments: expected a JSON object, "
f"got {type(params).__name__}."
),
success=False,
)
except Exception as exc:
return ToolResult(
tool_name=tool_call.name,
+1
View File
@@ -56,6 +56,7 @@ class CodeInterpreterTool(BaseTool):
"required": ["code"],
},
category="code",
metadata={"structured_allow_object_text": True},
)
def execute(self, **params: Any) -> ToolResult:
@@ -55,6 +55,7 @@ class DockerCodeInterpreterTool(BaseTool):
},
category="code",
timeout_seconds=60.0,
metadata={"structured_allow_object_text": True},
)
def execute(self, **params: Any) -> ToolResult:
+1
View File
@@ -191,6 +191,7 @@ class ReplTool(BaseTool):
"required": ["code"],
},
category="code",
metadata={"structured_allow_object_text": True},
)
def execute(self, **params: Any) -> ToolResult:
+23
View File
@@ -95,6 +95,29 @@ class SQLiteMemory(MemoryBackend):
)
return doc_id
def replace_source(
self,
source: str,
documents: List[tuple[str, Optional[Dict[str, Any]]]],
) -> List[str]:
"""Atomically replace all documents associated with *source*."""
payload = [
(content, json.dumps(metadata) if metadata else None)
for content, metadata in documents
]
doc_ids = self._rust_impl.replace_source(source, payload)
bus = get_event_bus()
for doc_id in doc_ids:
bus.publish(
EventType.MEMORY_STORE,
{
"backend": self.backend_id,
"doc_id": doc_id,
"source": source,
},
)
return doc_ids
def retrieve(
self,
query: str,
+4 -4
View File
@@ -396,11 +396,10 @@ class TestPersonaFilesReachModel:
assert "MEMORY_SENTINEL" in joined
assert "USER_SENTINEL" in joined
def test_orchestrator_keeps_its_own_system_prompt(
def test_orchestrator_accepts_persona_prompt_builder(
self, runner, monkeypatch, tmp_path
):
"""OrchestratorAgent's __init__ doesn't accept ``prompt_builder``;
the wiring must skip it silently rather than crash."""
"""Orchestrator explicitly accepts and applies persona wiring."""
from openjarvis.core.config import JarvisConfig
soul = tmp_path / "SOUL.md"
@@ -425,5 +424,6 @@ class TestPersonaFilesReachModel:
):
result = runner.invoke(cli, ["ask", "--agent", "orchestrator", "Hello"])
# Pass condition: doesn't crash with TypeError on prompt_builder kwarg.
assert result.exit_code == 0, result.output
messages = engine.generate.call_args.args[0]
assert "ORCH_PERSONA_SENTINEL" in messages[0].content
+101 -1
View File
@@ -17,7 +17,7 @@ from openjarvis.cli.chat_cmd import _read_input, chat
from openjarvis.core.config import JarvisConfig
from openjarvis.core.events import Event, EventBus, EventType
from openjarvis.core.registry import AgentRegistry, ToolRegistry
from openjarvis.core.types import ToolCall, ToolResult
from openjarvis.core.types import Role, ToolCall, ToolResult
from openjarvis.memory.store import LocalFactStore
from openjarvis.tools._stubs import BaseTool, ToolSpec
@@ -195,6 +195,106 @@ class TestChatAgents:
assert "simple ok" in result.output
assert "failed" not in result.output.lower()
def test_agent_receives_prior_turn_history(self) -> None:
"""Multi-turn chat must pass prior turns to agent.run() via AgentContext."""
captured_contexts: list[AgentContext | None] = []
class _CapturingAgent(BaseAgent):
agent_id = "capturing_chat_agent"
def run(self, input, context: AgentContext | None = None, **kwargs):
captured_contexts.append(context)
return AgentResult(content=f"reply-{len(captured_contexts)}", turns=1)
engine = MagicMock()
engine.engine_id = "mock"
config = JarvisConfig()
config.intelligence.default_model = "test-model"
AgentRegistry.register_value("capturing_chat_agent", _CapturingAgent)
with (
patch("openjarvis.cli.chat_cmd.load_config", return_value=config),
patch("openjarvis.engine.get_engine", return_value=("mock", engine)),
patch("openjarvis.intelligence.register_builtin_models"),
):
result = CliRunner().invoke(
chat,
["--agent", "capturing_chat_agent", "--model", "test-model"],
input="first turn\nsecond turn\n/quit\n",
)
assert result.exit_code == 0
assert len(captured_contexts) == 2
first_turn_context, second_turn_context = captured_contexts
assert first_turn_context is not None
assert first_turn_context.conversation.messages == []
assert second_turn_context is not None
prior_texts = [m.content for m in second_turn_context.conversation.messages]
assert "first turn" in prior_texts
assert "reply-1" in prior_texts
def test_agent_memory_context_precedes_prior_turn_history(self, tmp_path) -> None:
"""Memory system context must remain ahead of prior conversation turns."""
captured_contexts: list[AgentContext | None] = []
class _CapturingAgent(BaseAgent):
agent_id = "capturing_memory_chat_agent"
def run(self, input, context: AgentContext | None = None, **kwargs):
captured_contexts.append(context)
return AgentResult(content=f"reply-{len(captured_contexts)}", turns=1)
facts_path = tmp_path / "facts.jsonl"
LocalFactStore(facts_path).add("The user likes jazz", source="auto")
engine = MagicMock()
engine.engine_id = "mock"
config = JarvisConfig()
config.intelligence.default_model = "test-model"
config.memory.enabled = True
config.memory.facts_path = str(facts_path)
config.agent.context_from_memory = True
AgentRegistry.register_value(
"capturing_memory_chat_agent",
_CapturingAgent,
)
with (
patch("openjarvis.cli.chat_cmd.load_config", return_value=config),
patch("openjarvis.engine.get_engine", return_value=("mock", engine)),
patch("openjarvis.intelligence.register_builtin_models"),
patch("openjarvis.memory.build_memory_service", return_value=None),
patch("openjarvis.cli.ask._get_memory_backend", return_value=None),
):
result = CliRunner().invoke(
chat,
["--agent", "capturing_memory_chat_agent", "--model", "test-model"],
input="first turn\nsecond turn\n/quit\n",
)
assert result.exit_code == 0
assert len(captured_contexts) == 2
second_turn_context = captured_contexts[1]
assert second_turn_context is not None
messages = second_turn_context.conversation.messages
assert [message.role for message in messages] == [
Role.SYSTEM,
Role.USER,
Role.ASSISTANT,
]
assert "user likes jazz" in messages[0].content
assert [message.content for message in messages[1:]] == [
"first turn",
"reply-1",
]
def test_memory_service_started_fed_and_stopped(self) -> None:
"""The REPL starts memory, publishes each turn, and stops it."""
+29
View File
@@ -40,6 +40,35 @@ def test_memory_index_file(tmp_path: Path, monkeypatch):
assert "Indexed" in result.output or "chunk" in result.output
def test_memory_index_replaces_existing_source(tmp_path: Path, monkeypatch):
"""Re-indexing a file replaces its previous chunks."""
_register_sqlite()
db_path = str(tmp_path / "mem.db")
doc = tmp_path / "doc.txt"
doc.write_text(" ".join(["legacy"] * 100), encoding="utf-8")
mod = importlib.import_module("openjarvis.cli.memory_cmd")
monkeypatch.setattr(
mod,
"_get_backend",
lambda b=None: SQLiteMemory(db_path=db_path),
)
first = CliRunner().invoke(cli, ["memory", "index", str(doc)])
assert first.exit_code == 0
doc.write_text(" ".join(["updated"] * 100), encoding="utf-8")
second = CliRunner().invoke(cli, ["memory", "index", str(doc)])
assert second.exit_code == 0
backend = SQLiteMemory(db_path=db_path)
assert backend.count() == 1
assert backend.retrieve("legacy") == []
updated = backend.retrieve("updated")
assert len(updated) == 1
assert updated[0].source == str(doc)
def test_memory_index_nonexistent(tmp_path: Path):
"""Indexing a nonexistent path should fail."""
_register_sqlite()
+396
View File
@@ -0,0 +1,396 @@
from __future__ import annotations
import json
import os
import subprocess
import sys
from pathlib import Path
from click.testing import CliRunner
from openjarvis.cli.scan_cmd import PrivacyScanner, ScanResult, scan
from openjarvis.core.config import JarvisConfig
def _low_noise_config():
"""Baseline config with no warn/fail findings under an empty scan root.
JarvisConfig defaults include absolute store paths under the real
OPENJARVIS_HOME; clear those so tests only see artifacts under tmp_path.
"""
config = JarvisConfig()
config.analytics.enabled = False
config.traces.enabled = False
config.telemetry.enabled = False
config.agent.context_from_memory = False
config.agent.tools = ""
config.skills.enabled = False
config.digest.enabled = False
config.channel.enabled = False
config.learning.enabled = False
config.learning.training_enabled = False
config.learning.auto_update = False
config.learning.spec_search.enabled = False
config.tools.enabled = ""
config.tools.mcp.enabled = False
config.tools.storage.enabled = False
config.optimize.optimizer_provider = ""
config.optimize.judge_model = ""
config.server.host = "127.0.0.1"
config.security.profile = "personal"
# Avoid scanning the developer's real ~/.openjarvis store files.
config.traces.db_path = ""
config.telemetry.db_path = ""
config.security.audit_log_path = ""
config.security.vault_key_path = ""
config.tools.storage.db_path = ""
config.tools.storage.facts_path = ""
config.sessions.db_path = ""
config.agent_manager.db_path = ""
config.optimize.db_path = ""
config.scheduler.db_path = ""
config.skills.index_dir = ""
config.memory_files.soul_path = ""
config.memory_files.memory_path = ""
config.memory_files.user_path = ""
return config
def _patch_config(monkeypatch, tmp_path, config, config_loaded=True, error=""):
monkeypatch.setattr(
"openjarvis.cli.scan_cmd._load_data_boundary_config",
lambda: (config, tmp_path, config_loaded, error, ""),
)
monkeypatch.setattr("openjarvis.cli.scan_cmd.get_config_dir", lambda: tmp_path)
def test_scan_data_boundaries_json_redacts_paths(monkeypatch, tmp_path):
config = _low_noise_config()
(tmp_path / "traces.db").write_text("", encoding="utf-8")
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--data-boundaries", "--json"])
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["schema_version"] == 1
assert payload["root"] != str(tmp_path.resolve())
assert str(tmp_path.resolve()) not in result.output
assert "findings" in payload
def test_scan_data_boundaries_show_paths_json(monkeypatch, tmp_path):
config = _low_noise_config()
trace_db = tmp_path / "traces.db"
trace_db.write_text("", encoding="utf-8")
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(
scan,
["--data-boundaries", "--json", "--show-paths"],
)
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["root"] == str(tmp_path.resolve())
assert "traces.db" in result.output
def test_scan_data_boundaries_handles_config_load_error(monkeypatch, tmp_path):
config = _low_noise_config()
_patch_config(
monkeypatch,
tmp_path,
config,
config_loaded=False,
error="TOMLDecodeError: invalid config",
)
result = CliRunner().invoke(scan, ["--data-boundaries", "--json"])
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["summary"]["fail"] == 1
assert payload["findings"][0]["id"] == "config-load-error"
def test_scan_data_boundaries_strict_exits_nonzero_on_fail(
monkeypatch,
tmp_path,
):
config = _low_noise_config()
config.intelligence.provider = "openai"
config.agent.context_from_memory = True
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--data-boundaries", "--strict"])
assert result.exit_code == 1
assert "local memory may be sent to cloud inference" in result.output
def test_scan_data_boundaries_fail_exits_zero_without_strict(
monkeypatch,
tmp_path,
):
config = _low_noise_config()
config.intelligence.provider = "openai"
config.agent.context_from_memory = True
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--data-boundaries"])
assert result.exit_code == 0
assert "local memory may be sent to cloud inference" in result.output
def test_scan_data_boundaries_strict_exits_on_warning(
monkeypatch,
tmp_path,
):
config = _low_noise_config()
config.tools.enabled = "web_search"
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--data-boundaries", "--strict"])
assert result.exit_code == 1
assert "Web search tool is configured" in result.output
def test_scan_data_boundaries_strict_passes_with_info_only(
monkeypatch,
tmp_path,
):
config = _low_noise_config()
config.agent.context_from_memory = True
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--data-boundaries", "--strict"])
assert result.exit_code == 0
assert "OpenJarvis Data-Boundary Scan" in result.output
def test_scan_data_boundaries_init_defaults_strict_exits_on_warn(
monkeypatch,
tmp_path,
):
config = _low_noise_config()
config.server.host = "0.0.0.0"
config.telemetry.enabled = True
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--data-boundaries", "--strict"])
assert result.exit_code == 1
assert "bind all" in result.output
def test_scan_data_boundaries_rejects_quick(monkeypatch, tmp_path):
config = _low_noise_config()
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(scan, ["--quick", "--data-boundaries"])
assert result.exit_code != 0
assert "cannot be combined" in result.output
def test_scan_rejects_strict_without_data_boundaries():
result = CliRunner().invoke(scan, ["--strict"])
assert result.exit_code != 0
assert "only supported with --data-boundaries" in result.output
def test_existing_scan_json_still_works(monkeypatch):
monkeypatch.setattr(
PrivacyScanner,
"run_all",
lambda self: [
ScanResult(
name="Network Exposure",
status="ok",
message="No exposed ports.",
platform="all",
)
],
)
result = CliRunner().invoke(scan, ["--json"])
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload[0]["name"] == "Network Exposure"
assert payload[0]["status"] == "ok"
def test_existing_scan_quick_json_still_works(monkeypatch):
monkeypatch.setattr(
PrivacyScanner,
"run_quick",
lambda self: [
ScanResult(
name="Cloud Sync Agents",
status="ok",
message="No cloud-sync agents detected.",
platform="all",
)
],
)
result = CliRunner().invoke(scan, ["--quick", "--json"])
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload[0]["name"] == "Cloud Sync Agents"
def test_top_level_cli_registers_data_boundary_scan(monkeypatch, tmp_path):
from openjarvis.cli import cli
config = _low_noise_config()
_patch_config(monkeypatch, tmp_path, config)
result = CliRunner().invoke(cli, ["scan", "--data-boundaries", "--json"])
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["schema_version"] == 1
assert "summary" in payload
def test_top_level_scan_data_boundaries_does_not_check_for_updates(
monkeypatch,
tmp_path,
):
import sys
from openjarvis.cli import cli
from openjarvis.core.config import JarvisConfig
called = {"value": False}
def fake_check_for_updates(_subcommand):
called["value"] = True
monkeypatch.setattr(
"openjarvis.cli._version_check.check_for_updates",
fake_check_for_updates,
)
monkeypatch.setattr(
"openjarvis.cli.scan_cmd._load_data_boundary_config",
lambda: (JarvisConfig(), tmp_path, False, "", ""),
)
monkeypatch.setattr(
sys,
"argv",
["jarvis", "scan", "--data-boundaries", "--json"],
)
result = CliRunner().invoke(cli, ["scan", "--data-boundaries", "--json"])
assert result.exit_code == 0
assert called["value"] is False
def test_update_check_skip_helper_is_precise():
from click import Command, Context
from openjarvis.cli import _should_skip_update_check
ctx = Context(Command("jarvis"))
ctx.invoked_subcommand = "scan"
assert _should_skip_update_check(ctx, ["scan", "--data-boundaries"])
ctx.invoked_subcommand = "ask"
assert not _should_skip_update_check(ctx, ["ask", "scan", "--data-boundaries"])
def test_existing_scan_quick_text_still_works(monkeypatch):
monkeypatch.setattr(
PrivacyScanner,
"run_quick",
lambda self: [
ScanResult(
name="Cloud Sync Agents",
status="ok",
message="No cloud-sync agents detected.",
platform="all",
)
],
)
result = CliRunner().invoke(scan, ["--quick"])
assert result.exit_code == 0
assert "OpenJarvis Security Scan" in result.output
def test_data_boundary_loader_honors_openjarvis_config(
monkeypatch,
tmp_path,
):
from openjarvis.cli import scan_cmd
from openjarvis.core.config import load_config
config_path = tmp_path / "custom.toml"
config_path.write_text(
"[telemetry]\nenabled = false\n",
encoding="utf-8",
)
monkeypatch.setenv("OPENJARVIS_CONFIG", str(config_path))
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "home"))
load_config.cache_clear()
_config, _root, loaded, error, root_error = scan_cmd._load_data_boundary_config()
assert loaded is True
assert error == ""
assert root_error == ""
def test_data_boundary_loader_reports_root_error(monkeypatch):
from openjarvis.cli import scan_cmd
monkeypatch.setattr(
"openjarvis.cli.scan_cmd.get_config_dir",
lambda: (_ for _ in ()).throw(RuntimeError("bad home")),
)
_config, root, loaded, error, root_error = scan_cmd._load_data_boundary_config()
assert root is None
assert loaded is False
assert error == ""
assert "bad home" in root_error
def test_data_boundary_cli_reports_real_root_error_without_import_crash():
repo_root = Path(__file__).resolve().parents[2]
invalid_home = repo_root / ".invalid-openjarvis-home"
env = os.environ.copy()
env["OPENJARVIS_HOME"] = str(invalid_home)
env["PYTHONPATH"] = str(repo_root / "src")
result = subprocess.run(
[
sys.executable,
"-c",
"from openjarvis.cli import main; main()",
"scan",
"--data-boundaries",
"--json",
],
capture_output=True,
text=True,
env=env,
check=False,
)
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["findings"][0]["id"] == "config-root-error"
assert str(invalid_home) not in result.stdout
assert str(repo_root) not in result.stdout
+129
View File
@@ -0,0 +1,129 @@
"""Regression: ``jarvis serve`` must wire the ``SystemPromptBuilder`` into the
agent it constructs, so SOUL.md / MEMORY.md / USER.md reach the model over HTTP.
``cli/ask.py`` (and ``cli/chat_cmd.py`` and the managed-agent executor) have
wired the builder since the persona system landed. The serve path never did, so
an agent served over HTTP silently answered as a generic assistant explicitly
denying the persona while the same agent via the CLI kept it. Found deploying
a personal assistant: SOUL.md was correct on disk the whole time; no error, no
warning.
This test boots ``serve`` just far enough to capture the agent handed to
``create_app`` and asserts the builder (and thus the persona content) is
present. It fails on the unpatched serve path: ``agent._prompt_builder`` is
``None``, so the persona files never reach the model.
"""
from __future__ import annotations
import importlib
from unittest.mock import MagicMock, patch
import pytest
from click.testing import CliRunner
from openjarvis.cli import cli
pytest.importorskip("fastapi")
pytest.importorskip("uvicorn")
# ``openjarvis.cli.serve`` as a package attribute resolves to the click
# *command* (re-exported); grab the real module to monkeypatch its globals.
serve_mod = importlib.import_module("openjarvis.cli.serve")
def _fake_engine() -> MagicMock:
engine = MagicMock()
engine.list_models.return_value = ["test-model"]
engine.health.return_value = True
engine.name = "mock"
engine.engine_id = "mock"
return engine
@pytest.mark.parametrize(
"agent_name",
["simple", "orchestrator", "monitor_operative", "operative"],
)
def test_serve_wires_persona_builder_into_served_agent(
tmp_path, monkeypatch, agent_name
):
"""The agent built on the serve path must carry a SystemPromptBuilder whose
assembled prompt includes SOUL.md content (regression for the HTTP persona
loss)."""
from openjarvis.agents.monitor_operative import MonitorOperativeAgent
from openjarvis.agents.operative import OperativeAgent
from openjarvis.agents.orchestrator import OrchestratorAgent
from openjarvis.agents.simple import SimpleAgent
from openjarvis.core.config import JarvisConfig
from openjarvis.core.registry import AgentRegistry
# Persona file with a unique sentinel we can grep for in the built prompt.
soul = tmp_path / "SOUL.md"
soul.write_text("SERVE_PERSONA_SENTINEL", encoding="utf-8")
# conftest clears registries per-test; re-register the agent we exercise.
agent_classes = {
"simple": SimpleAgent,
"orchestrator": OrchestratorAgent,
"monitor_operative": MonitorOperativeAgent,
"operative": OperativeAgent,
}
if not AgentRegistry.contains(agent_name):
AgentRegistry.register_value(agent_name, agent_classes[agent_name])
config = JarvisConfig()
config.server.host = "127.0.0.1"
config.server.port = 8123
config.intelligence.default_model = "test-model"
config.memory_files.soul_path = str(soul)
# Keep the heavy optional subsystems off so we reach create_app cleanly.
config.telemetry.enabled = False
config.agent_manager.enabled = False
config.sessions.enabled = False
config.channel.enabled = False
config.skills.enabled = False
config.agent.context_from_memory = False
engine = _fake_engine()
monkeypatch.setattr(serve_mod, "load_config", lambda *a, **k: config)
monkeypatch.setattr(serve_mod, "get_engine", lambda *a, **k: ("mock", engine))
monkeypatch.setattr(serve_mod, "discover_engines", lambda *a, **k: {})
monkeypatch.setattr(serve_mod, "discover_models", lambda *a, **k: {})
# setup_security returns its own context; pass the engine straight through.
sec = MagicMock()
sec.engine = engine
sec.capability_policy = None
sec.audit_logger = None
monkeypatch.setattr("openjarvis.security.setup_security", lambda *a, **k: sec)
captured: dict = {}
def _capture_create_app(*args, **kwargs):
captured["agent"] = kwargs.get("agent")
return MagicMock(name="app")
with (
patch("openjarvis.server.app.create_app", side_effect=_capture_create_app),
patch("uvicorn.run", lambda *a, **k: None),
):
result = CliRunner().invoke(
cli, ["serve", "--agent", agent_name], catch_exceptions=False
)
assert result.exit_code == 0, result.output
agent = captured.get("agent")
assert agent is not None, (
"serve did not construct an agent or never reached create_app; "
f"output:\n{result.output}"
)
# The regression: without the fix ``agent._prompt_builder`` is None and the
# persona files never reach the model over HTTP.
assert agent._prompt_builder is not None, (
f"serve constructed {agent_name} without a prompt_builder — SOUL.md / "
"MEMORY.md / USER.md would be silently dropped on the HTTP path."
)
assert "SERVE_PERSONA_SENTINEL" in agent._prompt_builder.build(), (
"prompt_builder is wired on serve, but its built prompt omits SOUL.md"
)
@@ -0,0 +1,34 @@
"""Regression guards for the desktop app's outbound network policy."""
from __future__ import annotations
import json
import plistlib
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent.parent
TAURI_CONFIG = ROOT / "frontend" / "src-tauri" / "tauri.conf.json"
MACOS_INFO_PLIST = ROOT / "frontend" / "src-tauri" / "Info.plist"
def _csp_sources(directive: str) -> set[str]:
config = json.loads(TAURI_CONFIG.read_text(encoding="utf-8"))
csp = config["app"]["security"]["csp"]
directives = {
parts[0]: set(parts[1:]) for item in csp.split(";") if (parts := item.split())
}
return directives[directive]
def test_desktop_csp_allows_remote_api_servers() -> None:
"""The user-configured API URL may point beyond localhost (#649)."""
connect_sources = _csp_sources("connect-src")
assert {"http:", "https:", "ws:", "wss:"} <= connect_sources
def test_macos_webview_allows_user_configured_http_servers() -> None:
"""CSP alone cannot override App Transport Security for public hosts."""
info = plistlib.loads(MACOS_INFO_PLIST.read_bytes())
assert info["NSAppTransportSecurity"]["NSAllowsArbitraryLoadsInWebContent"] is True
+24
View File
@@ -95,6 +95,30 @@ class TestJarvisAsk:
assert result == "Agent response"
j.close()
def test_ask_with_agent_wires_persona(self, tmp_path):
from openjarvis.agents.simple import SimpleAgent
from openjarvis.core.registry import AgentRegistry
soul = tmp_path / "SOUL.md"
soul.write_text("SDK_PERSONA_SENTINEL", encoding="utf-8")
cfg = JarvisConfig()
cfg.memory_files.soul_path = str(soul)
cfg.memory_files.memory_path = ""
cfg.memory_files.user_path = ""
cfg.agent.context_from_memory = False
if not AgentRegistry.contains("simple"):
AgentRegistry.register_value("simple", SimpleAgent)
engine = _make_engine()
with patch("openjarvis.sdk.get_engine", return_value=("mock", engine)):
j = Jarvis(config=cfg, model="test-model")
j.ask("Hello", agent="simple")
messages = engine.generate.call_args.args[0]
assert "SDK_PERSONA_SENTINEL" in messages[0].content
j.close()
def test_ask_no_engine_raises(self):
with patch("openjarvis.sdk.get_engine", return_value=None):
j = Jarvis(config=JarvisConfig())
File diff suppressed because it is too large Load Diff
+49
View File
@@ -0,0 +1,49 @@
"""Regression tests for OpenRouter model ID normalization."""
from __future__ import annotations
import pytest
from openjarvis.core.types import Message
from openjarvis.server import cloud_router
def test_get_provider_detects_bare_openrouter_id():
assert cloud_router.get_provider("anthropic/claude-haiku-4.5") == "openrouter"
def test_get_provider_detects_litellm_prefixed_openrouter_id():
model = "openrouter/anthropic/claude-haiku-4.5"
assert cloud_router.get_provider(model) == "openrouter"
@pytest.mark.parametrize(
"requested_model,expected_forwarded_model",
[
("anthropic/claude-haiku-4.5", "anthropic/claude-haiku-4.5"),
("openrouter/anthropic/claude-haiku-4.5", "anthropic/claude-haiku-4.5"),
("openrouter/auto", "openrouter/auto"),
],
)
@pytest.mark.asyncio
async def test_stream_cloud_normalizes_openrouter_model_before_forwarding(
monkeypatch, requested_model, expected_forwarded_model
):
monkeypatch.setenv("OPENROUTER_API_KEY", "test-key")
captured: dict[str, str] = {}
async def fake_stream_openai(model, messages, temperature, max_tokens, **kwargs):
captured["model"] = model
yield "ok"
monkeypatch.setattr(cloud_router, "_stream_openai", fake_stream_openai)
tokens = [
token
async for token in cloud_router.stream_cloud(
requested_model, [Message(role="user", content="hi")]
)
]
assert tokens == ["ok"]
assert captured["model"] == expected_forwarded_model
+41
View File
@@ -847,6 +847,47 @@ class TestIdentityPromptInjection:
assert len(system_msgs) == 1
assert system_msgs[0].content == "Be terse."
def test_stream_uses_grounded_agent_result_without_replay(self):
"""Regression for #734: web streaming emits the agent's final answer."""
from openjarvis.core.events import EventBus
captured: list = []
engine = _make_capturing_engine(captured)
agent = _make_agent(content="My name is Jarvis Prime.")
agent._tools = [object()]
agent._engine = engine
client = TestClient(
create_app(
engine,
"test-model",
agent=agent,
bus=EventBus(),
config=_identity_config(),
)
)
resp = client.post(
"/v1/chat/completions",
json={
"model": "test-model",
"messages": [{"role": "user", "content": "who are you?"}],
"stream": True,
},
)
assert resp.status_code == 200
streamed_content = ""
for line in resp.text.splitlines():
if not line.startswith("data: {"):
continue
payload = json.loads(line.removeprefix("data: "))
choices = payload.get("choices", [])
if choices and choices[0]["delta"].get("content"):
streamed_content += choices[0]["delta"]["content"]
assert streamed_content == "My name is Jarvis Prime."
assert captured == []
agent.run.assert_called_once()
def test_direct_injects_identity_when_absent(self):
captured: list = []
engine = _make_capturing_engine(captured)
+64 -2
View File
@@ -1,6 +1,68 @@
import json
"""Regression tests for streaming completed agent responses."""
from openjarvis.server.stream_bridge import AgentStreamBridge
from __future__ import annotations
import asyncio
import json
from unittest.mock import MagicMock
import pytest
pytest.importorskip("fastapi")
from openjarvis.agents._stubs import AgentResult # noqa: E402
from openjarvis.core.events import EventBus # noqa: E402
from openjarvis.core.types import ToolResult # noqa: E402
from openjarvis.server.models import ChatCompletionRequest # noqa: E402
from openjarvis.server.stream_bridge import AgentStreamBridge # noqa: E402
def _streamed_content(events: list[str]) -> str:
"""Join assistant content from OpenAI-compatible data chunks."""
content = []
for event in events:
if not event.startswith("data: {"):
continue
payload = json.loads(event.removeprefix("data: ").strip())
choices = payload.get("choices")
if choices and choices[0]["delta"].get("content"):
content.append(choices[0]["delta"]["content"])
return "".join(content)
def test_stream_replays_grounded_agent_result_without_second_inference():
grounded_content = "My name is Jarvis. The tool reports 72 degrees."
agent = MagicMock()
agent._model = "configured-model"
agent.run.return_value = AgentResult(
content=grounded_content,
tool_results=[
ToolResult(tool_name="weather", content="72 degrees", success=True)
],
metadata={"prompt_tokens": 10, "completion_tokens": 12, "total_tokens": 22},
)
async def ungrounded_replay(*args, **kwargs):
raise AssertionError("stream_full must not run after agent.run")
yield # pragma: no cover
agent._engine.stream_full = ungrounded_replay
request = ChatCompletionRequest(
model="requested-model",
messages=[{"role": "user", "content": "Who are you, and what's outside?"}],
stream=True,
)
bridge = AgentStreamBridge(agent, EventBus(), request.model, request)
async def collect_events() -> list[str]:
return [event async for event in bridge.stream()]
events = asyncio.run(collect_events())
assert _streamed_content(events) == grounded_content
assert any(event.startswith("event: tool_results\n") for event in events)
agent.run.assert_called_once()
assert agent._model == "configured-model"
def test_tool_call_start_serializes_arguments_for_sse_without_mutating_event():
@@ -2,6 +2,8 @@
from __future__ import annotations
import pytest
from openjarvis.agents.orchestrator import OrchestratorAgent
from openjarvis.core.types import ToolResult
from openjarvis.engine._stubs import InferenceEngine
@@ -18,8 +20,10 @@ class _MockEngine(InferenceEngine):
def __init__(self, responses: list[str]) -> None:
self._responses = list(responses)
self._call_idx = 0
self.calls = []
def generate(self, messages, **kwargs) -> dict:
self.calls.append(list(messages))
if self._call_idx < len(self._responses):
content = self._responses[self._call_idx]
self._call_idx += 1
@@ -58,6 +62,109 @@ class _MockTool(BaseTool):
return ToolResult(tool_name="calculator", content=str(expr), success=True)
class _FileReadLikeTool(BaseTool):
"""Small test double with one required and one optional parameter."""
tool_id = "file_read"
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="file_read",
description="Read a file",
parameters={
"type": "object",
"properties": {
"path": {"type": "string"},
"max_lines": {"type": "integer"},
},
"required": ["path"],
},
)
def execute(self, **params) -> ToolResult:
return ToolResult(
tool_name="file_read",
content=params.get("path", ""),
success=True,
)
class _AmbiguousTool(BaseTool):
"""Test double whose bare input cannot map to one parameter safely."""
tool_id = "copy"
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="copy",
description="Copy a value",
parameters={
"type": "object",
"properties": {
"source": {"type": "string"},
"destination": {"type": "string"},
},
"required": ["source", "destination"],
},
)
def execute(self, **params) -> ToolResult:
return ToolResult(tool_name="copy", content="copied", success=True)
class _CodeLikeTool(BaseTool):
"""Test double that explicitly accepts object-prefixed source text."""
tool_id = "code"
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="code",
description="Execute source code",
parameters={
"type": "object",
"properties": {"code": {"type": "string"}},
"required": ["code"],
},
metadata={"structured_allow_object_text": True},
)
def execute(self, **params) -> ToolResult:
return ToolResult(
tool_name="code",
content=params.get("code", ""),
success=True,
)
class _UnionStringTool(BaseTool):
"""Test double with a JSON Schema union that accepts strings."""
tool_id = "union_file_read"
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="union_file_read",
description="Read a nullable path",
parameters={
"type": "object",
"properties": {"path": {"type": ["string", "null"]}},
"required": ["path"],
},
)
def execute(self, **params) -> ToolResult:
return ToolResult(
tool_name="union_file_read",
content=params.get("path", ""),
success=True,
)
# -- Tests -------------------------------------------------------------------
@@ -80,6 +187,207 @@ class TestStructuredMode:
assert result.content == "4"
assert result.turns == 2
assert len(result.tool_results) == 1
assert result.tool_results[0].success is True
assert result.tool_results[0].content == "2+2"
@pytest.mark.parametrize(
("raw_input", "expected"),
[
("notes/today.md", "notes/today.md"),
('"notes/today.md"', "notes/today.md"),
('{"path": "notes/today.md"}', "notes/today.md"),
("42", "42"),
("true", "true"),
("false", "false"),
("null", "null"),
("1e3", "1e3"),
('["notes/today.md"]', '["notes/today.md"]'),
('["notes/today.md",]', '["notes/today.md",]'),
("[draft] notes.md", "[draft] notes.md"),
(
"[x * 2 for x in range(3)]",
"[x * 2 for x in range(3)]",
),
('"notes/today.md', '"notes/today.md'),
('""', ""),
('"{draft} notes.md"', "{draft} notes.md"),
(r'"C:\\Users\\me\\notes.txt"', r"C:\Users\me\notes.txt"),
],
)
def test_single_string_parameter_accepts_text_input(self, raw_input, expected):
"""Structured text maps to the unambiguous string parameter."""
engine = _MockEngine(
[
f"TOOL: file_read\nINPUT: {raw_input}",
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_FileReadLikeTool()],
mode="structured",
)
result = agent.run("Read my notes")
assert result.tool_results[0].success is True
assert result.tool_results[0].content == expected
@pytest.mark.parametrize(
"raw_input",
[
'{"path": "notes/today.md",}',
'{path: "notes/today.md"}',
"{'path': 'notes/today.md'}",
'{"unknown": 1, "path": "notes/today.md",}',
r'{"pa\u0074h": "notes/today.md",}',
'\ufeff{"path": "notes/today.md",}',
],
)
def test_malformed_json_like_input_remains_an_argument_error(self, raw_input):
"""Malformed JSON-looking text is not reclassified as a tool value."""
engine = _MockEngine(
[
f"TOOL: file_read\nINPUT: {raw_input}",
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_FileReadLikeTool()],
mode="structured",
)
result = agent.run("Read my notes")
assert result.tool_results[0].success is False
assert "Invalid arguments JSON" in result.tool_results[0].content
assert "Tool 'file_read' failed: Invalid arguments JSON" in (
engine.calls[1][-1].content
)
@pytest.mark.parametrize(
"raw_input",
[
"{'code': value}",
'{"code": object()}',
"{'nested': {'value': 1}}",
],
)
def test_opted_in_tool_accepts_object_prefixed_text(self, raw_input):
"""Explicit raw-text metadata disambiguates dict-shaped source code."""
engine = _MockEngine(
[
f"TOOL: code\nINPUT: {raw_input}",
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_CodeLikeTool()],
mode="structured",
)
result = agent.run("Execute code")
assert result.tool_results[0].success is True
assert result.tool_results[0].content == raw_input
def test_valid_object_remains_arguments_for_opted_in_tool(self):
"""Raw-text metadata does not override valid JSON argument objects."""
engine = _MockEngine(
[
'TOOL: code\nINPUT: {"code": "print(1)"}',
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_CodeLikeTool()],
mode="structured",
)
result = agent.run("Execute code")
assert result.tool_results[0].success is True
assert result.tool_results[0].content == "print(1)"
def test_union_string_schema_accepts_json_scalar_text(self):
"""String unions normalize text that also parses as a JSON scalar."""
engine = _MockEngine(
[
"TOOL: union_file_read\nINPUT: null",
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_UnionStringTool()],
mode="structured",
)
result = agent.run("Read the path named null")
assert result.tool_results[0].success is True
assert result.tool_results[0].content == "null"
def test_ambiguous_json_scalar_gets_stable_object_error(self):
"""Ambiguous valid JSON is rejected before tool dispatch."""
engine = _MockEngine(
[
"TOOL: copy\nINPUT: 42",
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_AmbiguousTool()],
mode="structured",
)
result = agent.run("Copy my notes")
assert result.tool_results[0].success is False
assert result.tool_results[0].content == (
"Invalid arguments: expected a JSON object, got int."
)
assert "Tool 'copy' failed: Invalid arguments" in (engine.calls[1][-1].content)
@pytest.mark.parametrize(
"raw_input",
[
"notes/today.md",
'{"source": "notes/today.md",}',
],
)
def test_multiple_required_parameters_do_not_guess_string_mapping(
self,
raw_input,
):
"""Ambiguous bare input remains invalid instead of choosing a field."""
engine = _MockEngine(
[
f"TOOL: copy\nINPUT: {raw_input}",
"FINAL_ANSWER: done",
]
)
agent = OrchestratorAgent(
engine=engine,
model="test",
tools=[_AmbiguousTool()],
mode="structured",
)
result = agent.run("Copy my notes")
assert result.tool_results[0].success is False
assert "Invalid arguments JSON" in result.tool_results[0].content
def test_direct_final_answer(self):
"""Test that FINAL_ANSWER on first turn works."""
+4
View File
@@ -127,6 +127,10 @@ class TestCodeInterpreterTool:
tool = CodeInterpreterTool()
assert tool.tool_id == "code_interpreter"
def test_structured_object_text_opt_in(self):
tool = CodeInterpreterTool()
assert tool.spec.metadata["structured_allow_object_text"] is True
def test_registry_registration(self):
ToolRegistry.register_value("code_interpreter", CodeInterpreterTool)
assert ToolRegistry.contains("code_interpreter")
@@ -25,6 +25,7 @@ class TestDockerCodeInterpreterTool:
assert spec.name == "code_interpreter_docker"
assert "code" in spec.parameters["properties"]
assert spec.category == "code"
assert spec.metadata["structured_allow_object_text"] is True
def test_empty_code(self):
from openjarvis.tools.code_interpreter_docker import (
+4
View File
@@ -17,6 +17,10 @@ class TestReplSpec:
tool = ReplTool()
assert tool.spec.category == "code"
def test_structured_object_text_opt_in(self):
tool = ReplTool()
assert tool.spec.metadata["structured_allow_object_text"] is True
def test_spec_parameters(self):
tool = ReplTool()
params = tool.spec.parameters
+47
View File
@@ -2,6 +2,10 @@
from __future__ import annotations
import json
import pytest
from openjarvis.core.events import EventBus, EventType
from openjarvis.core.types import ToolCall, ToolResult
from openjarvis.tools._stubs import BaseTool, ToolExecutor, ToolSpec
@@ -50,6 +54,17 @@ class _ErrorTool(BaseTool):
raise RuntimeError("boom")
class _ScalarBoundaryGuard:
"""Test guard that rewrites outbound arguments to a JSON scalar."""
def check_outbound(self, tool_call: ToolCall) -> ToolCall:
return ToolCall(
id=tool_call.id,
name=tool_call.name,
arguments=json.dumps("redacted"),
)
# ---------------------------------------------------------------------------
# ToolSpec tests
# ---------------------------------------------------------------------------
@@ -134,6 +149,38 @@ class TestToolExecutor:
assert result.success is False
assert "Invalid arguments JSON" in result.content
@pytest.mark.parametrize(
("arguments", "decoded_type"),
[
("42", "int"),
("true", "bool"),
("null", "NoneType"),
("[]", "list"),
('"text"', "str"),
],
)
def test_execute_rejects_non_object_json(self, arguments, decoded_type):
executor = ToolExecutor([_EchoTool()])
call = ToolCall(id="1", name="echo", arguments=arguments)
result = executor.execute(call)
assert result.success is False
assert result.content == (
f"Invalid arguments: expected a JSON object, got {decoded_type}."
)
def test_execute_revalidates_boundary_guard_arguments(self):
tool = _EchoTool()
tool.is_local = False
executor = ToolExecutor([tool], boundary_guard=_ScalarBoundaryGuard())
call = ToolCall(id="1", name="echo", arguments='{"text":"safe"}')
result = executor.execute(call)
assert result.success is False
assert result.content == ("Invalid arguments: expected a JSON object, got str.")
def test_execute_empty_arguments(self):
executor = ToolExecutor([_EchoTool()])
call = ToolCall(id="1", name="echo", arguments="")
+2
View File
@@ -64,6 +64,8 @@ EXPECTED_TOOLS = {
"image_generate",
# audio_tool.py
"audio_transcribe",
# text_to_speech.py
"text_to_speech",
# knowledge_tools.py
"kg_add_entity",
"kg_add_relation",
-111
View File
@@ -1,111 +0,0 @@
# Pearl reference oracle (OpenJarvis Phase 0 deliverable)
Phase 0-B of [Spec B](../../docs/design/2026-05-05-apple-silicon-pearl-mining-design.md)
called for "build a Python reference oracle for NoisyGEMM, validate against the
Pearl CUDA reference."
**Phase 0 found the oracle already exists upstream**, in two complementary forms:
| Layer | Upstream location | What it covers |
|---|---|---|
| Pure-Rust mining algorithm exposed to Python | `pearl/py-pearl-mining` | The complete `mine()` + `verify_plain_proof()` cycle. CPU-only. Hardware-portable. |
| PyTorch reference of production NoisyGEMM | `pearl/miner/miner-base/src/miner_base/noisy_gemm.py` | The same NoisyGEMM that vllm-miner accelerates with H100 CUDA. Bit-exact denoising verified by upstream test (`tests/test_noisy_gemm.py:92`). |
So this directory contains:
1. `smoke_test.py` — a runnable script that **actually mines a block on this machine** using the upstream Rust path, demonstrating the v1 architecture works on Apple Silicon (or any platform where `py-pearl-mining` builds).
2. This README documenting where the reference math lives.
## What this is *not*
This is **not a reimplementation** of NoisyGEMM. The original Spec B planned for that;
Phase 0 made it unnecessary. If you're tempted to write `noisy_gemm.py` here, stop —
read `pearl/miner/miner-base/src/miner_base/noisy_gemm.py` instead.
## Setup
You need:
- macOS arm64 (M1/M2/M3/M4) **or** Linux x86_64 / aarch64
- Python 3.12 (`uv venv --python 3.12 .venv` is the easiest)
- Rust 1.78+ (any recent toolchain — verified with 1.94 on macOS arm64)
- The Pearl source tree somewhere on disk
Build the wheel and install it (one-time, ~60 s on a fast Mac, ~5 min on first build):
```bash
# from the Pearl repo root
cd py-pearl-mining
uv pip install maturin
maturin build --release --interpreter "$(which python)"
# install the resulting wheel
uv pip install target/wheels/py_pearl_mining-*.whl
```
Or if Pearl publishes to PyPI in the future:
```bash
uv pip install py-pearl-mining
```
## Run the smoke test
```bash
python smoke_test.py
```
Actual output on Apple Silicon M2 Max (numbers will vary by hardware and run):
```
host: macOS-26.4.1-arm64-arm-64bit (arm64)
python: 3.12.1
[ok] pearl_mining loaded from <site-packages>/pearl_mining/__init__.py
[ok] PUBLICDATA_SIZE=164 MERKLE_LEAF_SIZE=1024
[ok] mine(m=256, n=128, k=1024, rank=32) returned a proof in 0.119 s
proof.m=256 proof.n=128 proof.k=1024 noise_rank=32
a.row_indices=[177, 185, 241, 249] bt.row_indices=[80, 81, 88, 89, 112, 113, 120, 121]
[ok] verify_plain_proof: ok=True ('Mining solution verified successfully', 0.2 ms)
[ok] all checks passed — Pearl mining works on this host
```
The `a.row_indices` and `bt.row_indices` values above are not constants — they're
`(offset + ROWS_PATTERN)` and `(offset + COLS_PATTERN)` for whichever offset the
miner happened to find a jackpot at. The smoke test verifies the *deltas* match
the configured `PeriodicPattern`, not the absolute values.
If it succeeds, this host can mine Pearl using the OpenJarvis `cpu-pearl` provider
(see Spec B §13). If it fails, the `[fail]` line tells you which step broke.
## What this proves (and what it doesn't)
**Proves:**
- The Pearl mining algorithm executes correctly on this host's CPU.
- Generated proofs verify under `verify_plain_proof`. (This is the same check
validators run on the inputs to the ZK proof.)
- The whole stack — `pearl-blake3`, `zk-pow`, `py-pearl-mining` — builds and
loads as a native CPython extension.
**Does NOT prove:**
- Network-difficulty hashrate. The smoke test uses
`nbits=0x1D2FFFFF` (test difficulty), much easier than mainnet. Real mining
expected hashrate on Apple Silicon CPU is several orders of magnitude lower
per share — see Spec B §1.5.6.
- ZK proof generation throughput. The smoke test calls `verify_plain_proof`,
not `generate_proof`. Plonky2 STARK proving takes seconds-to-minutes of CPU
per block (Spec B Open Q10).
- That this host can keep up with the network's block production rate.
## When to update this
- When Pearl bumps `py-pearl-mining` API: re-run the smoke test against the
new ref pinned in `OpenJarvis/src/openjarvis/mining/_constants.py`.
- When Pearl publishes a Mac wheel to PyPI: simplify the install instructions
above, drop the local `maturin build` step.
- When Spec B v2 adds the PyTorch-MPS reference path: extend `smoke_test.py`
with an MPS path comparison. The `miner-base` reference is already in
PyTorch, so the v2 smoke test would be a different test invoking
`miner_base.NoisyGemm` and comparing CPU vs MPS outputs for parity.
-139
View File
@@ -1,139 +0,0 @@
"""Pearl mining smoke test — runs an end-to-end mine + verify cycle.
Verifies that this host can run Pearl's pure-Rust mining algorithm via the
`pearl_mining` Python package. Used as Phase 0-B of the OpenJarvis Apple Silicon
mining spec ([Spec B]).
Exit codes:
0 all checks passed
1 pearl_mining import failed
2 mine() failed
3 verify_plain_proof rejected the proof
4 timing or sanity check failed
[Spec B]: ../../docs/design/2026-05-05-apple-silicon-pearl-mining-design.md
"""
from __future__ import annotations
import platform
import sys
import time
# Test fixture values — match upstream Pearl's tests/test_python_api.py so we are
# testing the same code path that Pearl's own CI exercises. Do not change
# without re-syncing with upstream.
DEFAULT_NBITS = 0x1D2FFFFF
DEFAULT_M = 256
DEFAULT_N = 128
DEFAULT_K = 1024
DEFAULT_RANK = 32
ROWS_PATTERN = [0, 8, 64, 72]
COLS_PATTERN = [0, 1, 8, 9, 32, 33, 40, 41]
def _ok(msg: str) -> None:
print(f"[ok] {msg}")
def _fail(msg: str, code: int) -> None:
print(f"[fail] {msg}")
sys.exit(code)
def main() -> None:
print(f"host: {platform.platform()} ({platform.machine()})")
print(f"python: {sys.version.split()[0]}")
try:
import pearl_mining
except ImportError as e:
_fail(f"could not import pearl_mining — install with `uv pip install py-pearl-mining` or build from source: {e}", 1)
_ok(f"pearl_mining loaded from {pearl_mining.__file__}")
_ok(
f"PUBLICDATA_SIZE={pearl_mining.PUBLICDATA_SIZE} "
f"MERKLE_LEAF_SIZE={pearl_mining.MERKLE_LEAF_SIZE}"
)
block_header = pearl_mining.IncompleteBlockHeader(
version=0,
prev_block=b"\x00" * 32,
merkle_root=b"0123456789abcdef" * 2,
timestamp=0x66666666,
nbits=DEFAULT_NBITS,
)
mining_config = pearl_mining.MiningConfiguration(
common_dim=DEFAULT_K,
rank=DEFAULT_RANK,
mma_type=pearl_mining.MMAType.Int7xInt7ToInt32,
rows_pattern=pearl_mining.PeriodicPattern.from_list(ROWS_PATTERN),
cols_pattern=pearl_mining.PeriodicPattern.from_list(COLS_PATTERN),
reserved=pearl_mining.MiningConfiguration.RESERVED,
)
t0 = time.perf_counter()
try:
plain_proof = pearl_mining.mine(
DEFAULT_M,
DEFAULT_N,
DEFAULT_K,
block_header,
mining_config,
signal_range=None,
wrong_jackpot_hash=False,
)
except Exception as e:
_fail(f"mine() raised: {e!r}", 2)
t_mine = time.perf_counter() - t0
_ok(
f"mine(m={DEFAULT_M}, n={DEFAULT_N}, k={DEFAULT_K}, rank={DEFAULT_RANK}) "
f"returned a proof in {t_mine:.3f} s"
)
print(
f" proof.m={plain_proof.m} proof.n={plain_proof.n} proof.k={plain_proof.k} "
f"noise_rank={plain_proof.noise_rank}"
)
print(
f" a.row_indices={plain_proof.a.row_indices} "
f"bt.row_indices={plain_proof.bt.row_indices}"
)
t0 = time.perf_counter()
ok, msg = pearl_mining.verify_plain_proof(block_header, plain_proof)
t_verify_ms = (time.perf_counter() - t0) * 1000
if not ok:
_fail(f"verify_plain_proof rejected our proof: {msg}", 3)
_ok(f"verify_plain_proof: ok=True ({msg!r}, {t_verify_ms:.1f} ms)")
if plain_proof.m != DEFAULT_M or plain_proof.n != DEFAULT_N or plain_proof.k != DEFAULT_K:
_fail("plain_proof dimensions do not match request", 4)
if plain_proof.noise_rank != DEFAULT_RANK:
_fail("plain_proof noise_rank does not match request", 4)
# Row indices are (offset + base_index) for some valid offset within the
# matrix dimension — see threads_partition() in zk-pow/src/ffi/mine.rs.
# We can't assert an absolute value (different offsets are valid every run),
# but we can assert the deltas match the pattern shape.
a_idxs = list(plain_proof.a.row_indices)
bt_idxs = list(plain_proof.bt.row_indices)
a_deltas = [v - a_idxs[0] for v in a_idxs]
bt_deltas = [v - bt_idxs[0] for v in bt_idxs]
if a_deltas != ROWS_PATTERN:
_fail(f"a.row_indices deltas ({a_deltas}) != ROWS_PATTERN ({ROWS_PATTERN})", 4)
if bt_deltas != COLS_PATTERN:
_fail(f"bt.row_indices deltas ({bt_deltas}) != COLS_PATTERN ({COLS_PATTERN})", 4)
print()
print("[ok] all checks passed — Pearl mining works on this host")
print()
print("Note: this used test difficulty (nbits=0x1D2FFFFF), not mainnet.")
print("Real-network shares per second will be many orders of magnitude lower.")
print("See docs/design/2026-05-05-apple-silicon-pearl-mining-design.md §1.5.6")
if __name__ == "__main__":
main()