Compare commits

...
Author SHA1 Message Date
Elliot SluskyandClaude Opus 4.8 c686517cc7 Fix upcoming Google Calendar event retrieval (#617)
Closes #388. Parse Google Calendar all-day events from start.date instead of stamping them with the current time; treat generic next/upcoming calendar-event queries as gcalendar timeline requests that return nearest-future events first (UTC-normalized, instant-aware comparison that handles tz offsets and all-day events); and update the research planner guidance to route such queries with sources=[gcalendar] + a today-onward time_range. Real in-memory KnowledgeStore integration tests cover ordering, tz normalization, all-day inclusion, and source narrowing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 14:18:09 -07:00
Elliot SluskyandClaude Opus 4.8 904133cb25 fix(research): respect configured engine for Deep Research (#616)
Fixes #575. Web Deep Research was hardcoded to OllamaEngine + DEFAULT_PLANNER_MODEL, ignoring the user's configured/active engine and model. Resolve the planner from [deep_research] override -> live app chat engine + selected model -> config defaults -> legacy Ollama, pass the chat picker's model from the frontend into /api/research, record the actual planner engine in telemetry, and refuse to silently fall back to a different engine (raise an actionable error instead). Adds config support and focused tests for resolution and the route. Related: #576 (duplicate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 13:46:18 -07:00
Elliot SluskyandClaude Opus 4.8 299dee1f40 fix(desktop): verify Rust extension before server startup (#615)
Fixes #505. Make the desktop backend resilient to a missing/unbuilt openjarvis_rust extension: declare openjarvis-rust as a uv-managed desktop path dependency so 'uv sync --extra desktop' owns the PyO3 build (instead of pruning an undeclared package); add ~/.cargo/bin to the subprocess PATH and fail early with Rust / Windows Build Tools guidance when the toolchain is missing; verify 'import openjarvis_rust' before starting jarvis serve; and add a TCP bind preflight for port 8000 to catch non-HTTP listeners the /health probe can't classify. Includes the uv.lock entry for the new path dependency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 13:24:24 -07:00
github-actions[bot] 44ff286005 chore: update clone traffic data [skip ci] 2026-06-30 07:21:15 +00:00
Gilbert Barajas be51eb8684 docs(user-guide): document SOUL/MEMORY/USER.md persona files (#604) (#610)
* docs(user-guide): document SOUL/MEMORY/USER.md persona files (#604)

The persistent-memory showcase links to the User Guide: Agents page for how
SOUL.md / MEMORY.md / USER.md are loaded at conversation start, but that page
never covered them (site search for the filenames returns nothing).

Add a "Persistent Persona" section to user-guide/agents.md: the three files and
what each holds, where they live (config dir + [memory_files]), how they load
(after the agent template, cached per conversation, per-section truncation),
named personas (--persona / personas/<name>/), and editing by hand or via the
memory_manage / user_profile_manage tools. Cross-links the distinct retrieval
memory backend to resolve the reporter's confusion.

Closes #604

* docs(user-guide): clarify memory_manage/user_profile_manage target the default MEMORY.md/USER.md
2026-06-29 17:38:22 -07:00
Elliot Slusky 420908401c fix(engine): count tool call payloads in token estimates (#614)
Closes #608. Make Message.content officially Optional (str | None) with a Message.text accessor that treats None as empty, and count tool-call IDs/names/arguments, tool-result IDs, and reasoning/thinking metadata in estimate_prompt_tokens — all are replayed into later prompt turns, so they belong in the estimate. Extends estimator and message-type regression tests.
2026-06-29 17:34:10 -07:00
Elliot Slusky b70be55681 fix(openhands): handle none content in token estimates (#612)
Closes #607. Assistant tool-call turns can carry content=None, which crashed token estimation (len(m.content)) and think-tag stripping. Normalize with 'content or ""', route native OpenHands truncation through the shared estimate_prompt_tokens, and add tests for the estimator, the truncation helper, and an end-to-end tool-call run with None content.
2026-06-29 16:51:52 -07:00
23 changed files with 1435 additions and 74 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"total_clones": 137874,
"last_updated": "2026-06-29T07:46:59Z",
"last_updated": "2026-06-30T07:21:14Z",
"daily": {
"2026-03-27": 2189,
"2026-03-28": 1874,
+65
View File
@@ -19,6 +19,71 @@ Agents are the agentic logic layer of OpenJarvis. They determine how a query is
---
## Persistent Persona: SOUL.md, MEMORY.md, USER.md
Every agent's system prompt is assembled at conversation start by the `SystemPromptBuilder`, which injects up to three optional Markdown files -- the **persistent persona**. They are plain text you own and edit, loaded at the start of each conversation. There is no vector database or embedding cache behind them.
| File | What it holds | Example line |
|------|---------------|--------------|
| `SOUL.md` | How the agent should behave -- tone, length, what to push back on | `Be concise. Challenge weak assumptions.` |
| `MEMORY.md` | Facts about you, your projects, your preferences | `I deploy to Postgres, never MySQL.` |
| `USER.md` | Who you are -- role, team, context | `Backend engineer at Acme, on the payments team.` |
This persona is distinct from the retrieval [memory backend](memory.md): the persona is always-on Markdown context loaded into the prompt, while the memory backend is searchable long-term storage the agent queries on demand.
### Where they live
By default the files are read from the config directory:
```
~/.openjarvis/SOUL.md
~/.openjarvis/MEMORY.md
~/.openjarvis/USER.md
```
(The config directory honors `$OPENJARVIS_HOME` / `$XDG_DATA_HOME` when set.) The paths are configurable under `[memory_files]`:
```toml
[memory_files]
soul_path = "~/.openjarvis/SOUL.md"
memory_path = "~/.openjarvis/MEMORY.md"
user_path = "~/.openjarvis/USER.md"
persona_name = "" # optional named persona -- see below
```
### How they're loaded
At the start of each conversation, `SystemPromptBuilder` reads each file as UTF-8 and adds its contents as a section of the system prompt, after the agent template and before the skill catalog:
- **All three are optional.** A missing or empty file is skipped, so any subset works and an install with no persona files behaves exactly as before.
- **Edits apply to the next conversation.** The files are read once when a conversation's prompt is built, so there is no restart or re-indexing -- edit or delete a line and it takes effect the next time you start a conversation.
- **Each section is length-capped.** Files are truncated to a per-section character budget so a large `MEMORY.md` cannot crowd out the rest of the prompt.
### Named personas
A single install can answer as different personas without changing global config. A named persona lives in its own directory:
```
~/.openjarvis/personas/<name>/SOUL.md
~/.openjarvis/personas/<name>/MEMORY.md
~/.openjarvis/personas/<name>/USER.md
```
Select one per invocation, or opt out entirely:
```bash
jarvis ask --persona work "summarize my open PRs"
jarvis ask --persona none "what is 2 + 2?" # inject no persona
```
Set `persona_name` under `[memory_files]` to make a named persona the default. `persona_name = "none"` (equivalently `--persona none`) disables persona injection for that run.
### Editing them
`SOUL.md`, `MEMORY.md`, and `USER.md` are plain Markdown -- open them in any editor. `MEMORY.md` and `USER.md` can also be updated by the agent itself through the `memory_manage` and `user_profile_manage` tools when those are enabled, so the agent can record a new fact mid-conversation. These tools always target the default `MEMORY.md` and `USER.md` (under `~/.openjarvis/`), never a named persona's copies -- edit those by hand.
---
## BaseAgent ABC
All agents extend the abstract `BaseAgent` class.
+198 -13
View File
@@ -731,13 +731,20 @@ fn format_uv_sync_failure(
let code = exit_code
.map(|c| c.to_string())
.unwrap_or_else(|| "unknown".to_string());
let tail = uv_sync_stderr_tail(stderr, 800);
let rust_hint = if looks_like_rust_extension_build_error(stderr) {
format!("\n\n{}", rust_toolchain_install_hint())
} else {
String::new()
};
format!(
"`uv sync` failed in {} (exit {}). Last output:\n\n{}\n\n\
Try opening a terminal in that directory and running \
`uv sync --extra desktop` manually for the full output.",
`uv sync --extra desktop` manually for the full output.{}",
root.display(),
code,
uv_sync_stderr_tail(stderr, 800),
tail,
rust_hint,
)
}
@@ -786,6 +793,121 @@ fn format_uv_sync_spawn_error(root: &std::path::Path, uv_bin: &str, err: &str) -
)
}
fn rust_toolchain_install_hint() -> &'static str {
"The desktop app needs the Rust toolchain to build `openjarvis_rust`. \
Install Rust from https://rustup.rs. On Windows, also install Visual Studio \
Build Tools with the C++ workload, then relaunch."
}
fn looks_like_rust_extension_build_error(stderr: &str) -> bool {
let lower = stderr.to_ascii_lowercase();
[
"openjarvis-rust",
"openjarvis_rust",
"maturin",
"cargo",
"rustc",
"link.exe",
"visual studio",
]
.iter()
.any(|marker| lower.contains(marker))
}
fn format_missing_rust_toolchain() -> String {
format!(
"Could not find Rust's `cargo` command. {}\n\n\
If Rust is already installed, close and relaunch the desktop app so \
PATH includes `~/.cargo/bin`.",
rust_toolchain_install_hint(),
)
}
fn format_extension_import_failure(root: &std::path::Path, stderr: &str) -> String {
let tail = uv_sync_stderr_tail(stderr, 4000);
format!(
"`openjarvis_rust` is still not importable after building. Last output:\n\n{}\n\n\
Run these manually for the full build log:\n\n\
cd {}\n\
uv sync --extra desktop\n\
uv run python -c \"import openjarvis_rust\"",
if tail.is_empty() {
"(no stderr output)"
} else {
&tail
},
root.display(),
)
}
fn add_cargo_bin_to_path(cmd: &mut tokio::process::Command) {
let mut paths: Vec<std::path::PathBuf> = std::env::var_os("PATH")
.map(|path| std::env::split_paths(&path).collect())
.unwrap_or_default();
paths.insert(
0,
std::path::PathBuf::from(home_dir())
.join(".cargo")
.join("bin"),
);
if let Ok(joined) = std::env::join_paths(paths) {
cmd.env("PATH", joined);
}
}
async fn verify_openjarvis_rust_extension(
root: &std::path::Path,
uv_bin: &str,
) -> Result<(), String> {
let mut cmd = tokio::process::Command::new(uv_bin);
cmd.args(["run", "python", "-c", "import openjarvis_rust"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::piped())
.current_dir(root);
prepare_subprocess_for_appimage(&mut cmd);
add_cargo_bin_to_path(&mut cmd);
match cmd.output().await {
Ok(out) if out.status.success() => Ok(()),
Ok(out) => {
let stderr = String::from_utf8_lossy(&out.stderr);
Err(format_extension_import_failure(root, &stderr))
}
Err(e) => Err(format!(
"Could not verify `openjarvis_rust`: {}. Verify uv is installed at `{}`.",
e, uv_bin
)),
}
}
fn port_owner_hint() -> String {
if cfg!(target_os = "windows") {
format!("netstat -ano | findstr :{}", JARVIS_PORT)
} else {
format!("lsof -i :{}", JARVIS_PORT)
}
}
fn format_port_unavailable(port: u16, reason: &str) -> String {
format!(
"Port {} is not available: {}. Stop the process using that port or \
change the OpenJarvis port, then relaunch.\n\nTo identify it:\n {}",
port,
reason,
port_owner_hint(),
)
}
fn check_jarvis_port_available() -> Result<(), String> {
match std::net::TcpListener::bind(("127.0.0.1", JARVIS_PORT)) {
Ok(listener) => {
drop(listener);
Ok(())
}
Err(err) => Err(format_port_unavailable(JARVIS_PORT, &err.to_string())),
}
}
// ---------------------------------------------------------------------------
// Backend boot sequence (runs in background after app launch)
// ---------------------------------------------------------------------------
@@ -1171,11 +1293,6 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
// Something else (a different web server, a stale process,
// a 4xx-returning instance) is on our port. Don't kill it —
// give the user actionable info instead.
let lsof_hint = if cfg!(target_os = "windows") {
format!("netstat -ano | findstr :{}", JARVIS_PORT)
} else {
format!("lsof -i :{}", JARVIS_PORT)
};
let mut s = status.lock().await;
s.error = Some(format!(
"Port {} is already in use by another service (it answered \
@@ -1183,7 +1300,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
OpenJarvis port, then relaunch.\n\nTo identify it:\n {}",
JARVIS_PORT,
resp.status(),
lsof_hint,
port_owner_hint(),
));
return;
}
@@ -1193,8 +1310,21 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
}
}
if let Err(err) = check_jarvis_port_available() {
let mut s = status.lock().await;
s.error = Some(err);
return;
}
let root = project_root.as_ref().unwrap();
let cargo_bin = resolve_bin("cargo");
if !std::path::Path::new(&cargo_bin).exists() && cargo_bin == "cargo" {
let mut s = status.lock().await;
s.error = Some(format_missing_rust_toolchain());
return;
}
// Install dependencies automatically (handles fresh clones).
//
// Previously we ran `uv sync` with both stdout AND stderr piped to
@@ -1226,6 +1356,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
.current_dir(root);
// Avoid LD_LIBRARY_PATH leak when running inside an AppImage (#455).
prepare_subprocess_for_appimage(&mut sync_cmd);
add_cargo_bin_to_path(&mut sync_cmd);
let sync_output = sync_cmd.output().await;
match sync_output {
Ok(out) if !out.status.success() => {
@@ -1242,6 +1373,16 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
Ok(_) => {} // success — fall through
}
{
let mut s = status.lock().await;
s.detail = "Verifying Rust extension (openjarvis_rust)...".into();
}
if let Err(err) = verify_openjarvis_rust_extension(root, &uv_bin).await {
let mut s = status.lock().await;
s.error = Some(err);
return;
}
{
let mut s = status.lock().await;
s.detail = format!("Starting API server from {}...", root.display());
@@ -2720,11 +2861,12 @@ pub fn run() {
#[cfg(test)]
mod tests {
use super::{
boot_plan, default_local_model, format_uv_sync_failure, format_uv_sync_spawn_error,
matching_installed_model, model_names_match, normalize_host, parse_inference_config,
parse_ollama_model_names, preferred_installed_model, should_persist_resolved_model,
startup_installed_model,
upsert_engine_host, uv_sync_stderr_tail, InferenceConfig, SourceKind,
boot_plan, default_local_model, format_extension_import_failure,
format_missing_rust_toolchain, format_port_unavailable, format_uv_sync_failure,
format_uv_sync_spawn_error, matching_installed_model, model_names_match, normalize_host,
parse_inference_config, parse_ollama_model_names, preferred_installed_model,
should_persist_resolved_model, startup_installed_model, upsert_engine_host,
uv_sync_stderr_tail, InferenceConfig, SourceKind,
};
use std::path::Path;
@@ -2791,6 +2933,49 @@ mod tests {
assert!(msg.contains("No such file or directory"));
}
#[test]
fn missing_rust_toolchain_message_names_cargo_and_installer() {
let msg = format_missing_rust_toolchain();
assert!(msg.contains("cargo"));
assert!(msg.contains("https://rustup.rs"));
assert!(msg.contains("openjarvis_rust"));
assert!(msg.contains("Visual Studio Build Tools"));
}
#[test]
fn uv_sync_rust_failure_mentions_toolchain() {
let msg = format_uv_sync_failure(
Path::new("C:\\Users\\me\\OpenJarvis"),
Some(1),
"maturin failed: linker `link.exe` not found while building openjarvis-rust",
);
assert!(msg.contains("exit 1"));
assert!(msg.contains("link.exe"));
assert!(msg.contains("https://rustup.rs"));
assert!(msg.contains("Visual Studio Build Tools"));
}
#[test]
fn extension_import_failure_names_verification_command() {
let msg = format_extension_import_failure(
Path::new("C:\\Users\\me\\OpenJarvis"),
"ModuleNotFoundError: No module named 'openjarvis_rust'",
);
assert!(msg.contains("openjarvis_rust"));
assert!(msg.contains("uv sync --extra desktop"));
assert!(msg.contains("uv run python -c \"import openjarvis_rust\""));
assert!(msg.contains("ModuleNotFoundError"));
}
#[test]
fn port_unavailable_message_names_port_and_owner_hint() {
let msg = format_port_unavailable(8000, "address already in use");
assert!(msg.contains("Port 8000 is not available"));
assert!(msg.contains("address already in use"));
assert!(msg.contains("To identify it"));
assert!(msg.contains("8000"));
}
#[test]
fn default_local_model_picks_second_largest_that_fits() {
// QWEN35_MODELS min_ram ladder: 4,6,8,12,24,32,96 GB
+5 -1
View File
@@ -243,7 +243,11 @@ export function InputArea() {
try {
if (deepResearch) {
for await (const ev of streamResearch(content, controller.signal)) {
for await (const ev of streamResearch(
content,
selectedModel,
controller.signal,
)) {
if (ev.type === 'search_call') {
const trace: ResearchSearchTrace = {
id: generateId(),
+2 -2
View File
@@ -60,6 +60,7 @@ export async function* streamChat(
export async function* streamResearch(
query: string,
model?: string,
signal?: AbortSignal,
): AsyncGenerator<ResearchEvent> {
// /api/research is mounted at the server root — strip any trailing /v1
@@ -68,7 +69,7 @@ export async function* streamResearch(
const response = await fetch(`${base}/api/research`, {
method: 'POST',
headers: authHeaders({ 'Content-Type': 'application/json' }),
body: JSON.stringify({ query }),
body: JSON.stringify({ query, ...(model ? { model } : {}) }),
signal,
});
@@ -106,4 +107,3 @@ export async function* streamResearch(
reader.releaseLock();
}
}
+4
View File
@@ -91,6 +91,7 @@ desktop = [
"pydantic>=2.0",
"python-multipart>=0.0.9",
"faster-whisper>=1.0",
"openjarvis-rust",
]
openhands = ["openhands-sdk>=1.0; python_version >= '3.12'"]
gpu-metrics = ["pynvml>=12.0"]
@@ -186,6 +187,9 @@ git_describe_command = [
# Such builds can inject the real version via SETUPTOOLS_SCM_PRETEND_VERSION.
fallback_version = "0.0.0+unknown"
[tool.uv.sources]
openjarvis-rust = { path = "rust/crates/openjarvis-python" }
[tool.hatch.build.targets.wheel]
packages = ["src/openjarvis"]
+5 -5
View File
@@ -19,6 +19,7 @@ from openjarvis.agents.prompt_loader import (
from openjarvis.core.events import EventBus
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import Message, Role, ToolCall, ToolResult
from openjarvis.engine._base import estimate_prompt_tokens
from openjarvis.engine._stubs import InferenceEngine
from openjarvis.tools._stubs import BaseTool, build_tool_descriptions
@@ -116,8 +117,7 @@ class NativeOpenHandsAgent(ToolUsingAgent):
max_prompt_tokens: int = 3000,
) -> list[Message]:
"""Truncate messages if estimated token count exceeds limit."""
total_chars = sum(len(m.content) for m in messages)
estimated_tokens = total_chars // 4
estimated_tokens = estimate_prompt_tokens(messages)
if estimated_tokens <= max_prompt_tokens:
return messages
# Find the last user message and truncate its content
@@ -125,7 +125,7 @@ class NativeOpenHandsAgent(ToolUsingAgent):
if messages[i].role == Role.USER:
excess_tokens = estimated_tokens - max_prompt_tokens
excess_chars = excess_tokens * 4
original = messages[i].content
original = messages[i].content or ""
if len(original) > excess_chars + 200:
truncated = original[: len(original) - excess_chars]
messages[i] = Message(
@@ -258,7 +258,7 @@ class NativeOpenHandsAgent(ToolUsingAgent):
# still emitted before re-raising.
self._emit_turn_end(turns=1, error=True)
raise
content = self._strip_think_tags(result.get("content", ""))
content = self._strip_think_tags(result.get("content") or "")
usage = result.get("usage", {})
self._emit_turn_end(turns=1)
return AgentResult(
@@ -315,7 +315,7 @@ class NativeOpenHandsAgent(ToolUsingAgent):
for k in total_usage:
total_usage[k] += usage.get(k, 0)
content = result.get("content", "")
content = result.get("content") or ""
# Strip think tags so they don't interfere with parsing
content = self._strip_think_tags(content)
last_content = content
+7 -5
View File
@@ -2,7 +2,8 @@
A small, self-contained planner-executor loop:
* the planner is a local Ollama chat model (default ``gemma4:31b``),
* the planner is supplied by the caller (the web endpoint resolves it from
config, falling back to ``gemma4:31b`` on Ollama for legacy installs),
* the only tool it can call is :meth:`HybridSearch.search`,
* it gets up to ``max_iterations`` tool calls,
* tool results are trimmed before re-entering the context window, and
@@ -142,10 +143,11 @@ Strategy:
3. The `time_range` argument is a JSON object: `{{"start": "<ISO 8601>", "end": "<ISO 8601>"}}`. Either bound may be omitted, but pass at least one whenever the user gave you a temporal cue.
4. When the user names a specific data source — "my Granola notes", "in Slack", "from my email" — you MUST pass `sources=[...]` with the matching connector ID. Only use IDs that appear in the connected-sources list above; do NOT invent or assume sources that are not connected. Common synonyms: "meeting notes"/"meetings"/"transcripts" → granola; "email"/"inbox" → gmail; "DMs"/"channels" → slack. Without this filter the search returns mail/messages ABOUT a tool instead of records FROM that tool.
4a. Never apologize about sources that aren't in the connected-sources list — if the user asks about "Notion" but Notion isn't connected, just say "Notion isn't connected, but here's what I found in {available_sources}" and answer from what is available.
5. If the first structured search returns nothing useful, broaden with a semantic query and drop filters one at a time.
6. You have a clarify tool. Only use it AFTER at least one search attempt. Use it when: you found multiple ambiguous matches (e.g. 3 different people named John), search returned zero results and the query might need reframing, or the scope is too broad to synthesize meaningfully. Never use clarify before searching — always try first.
7. After receiving a clarify response, use the information to construct a precise search with the correct person, time_range, sources, and query parameters. Never send an empty query or a query with no parameters — extract every concrete signal from the user's reply (names, dates, topics, sources) and put it on the call.
8. Tool calls — search AND clarify — share a budget of 5 total. Spend wisely.
5. When the user asks for "next", "upcoming", "future", or "soon" calendar events/meetings/appointments, use `sources=["gcalendar"]` if gcalendar is connected, set `time_range={{"start": "{today}"}}`, and use `query=""` unless the user gave a specific topic such as "dentist" or "music lesson". This returns the nearest upcoming calendar items across calendars instead of keyword-matching only birthdays or event titles.
6. If the first structured search returns nothing useful, broaden with a semantic query and drop filters one at a time.
7. You have a clarify tool. Only use it AFTER at least one search attempt. Use it when: you found multiple ambiguous matches (e.g. 3 different people named John), search returned zero results and the query might need reframing, or the scope is too broad to synthesize meaningfully. Never use clarify before searching — always try first.
8. After receiving a clarify response, use the information to construct a precise search with the correct person, time_range, sources, and query parameters. Only use an empty query when structured filters carry the request; never send a search with no concrete parameters. Extract every concrete signal from the user's reply (names, dates, topics, sources) and put it on the call.
9. Tool calls — search AND clarify — share a budget of 5 total. Spend wisely.
Synthesis rules:
- Cite sources as individual numbers in square brackets. Always separate — write [4] [7] [20], never [4, 7, 20]. Never format citations as markdown links. Just the number in brackets: [1]. The `ref` field on each hit is the citation number.
+3 -2
View File
@@ -213,12 +213,13 @@ def _parse_event_timestamp(event: Dict[str, Any]) -> datetime:
"""
start = event.get("start", {})
date_time_str: str = start.get("dateTime", "")
if not date_time_str:
date_str: str = start.get("date", "")
if not date_time_str and not date_str:
return datetime.now()
try:
# RFC3339 — Python 3.11+ fromisoformat handles the trailing 'Z'.
# For older versions we replace 'Z' with '+00:00'.
normalized = date_time_str.replace("Z", "+00:00")
normalized = (date_time_str or date_str).replace("Z", "+00:00")
return datetime.fromisoformat(normalized)
except (ValueError, TypeError):
return datetime.now()
+317 -21
View File
@@ -20,8 +20,9 @@ from __future__ import annotations
import json
import logging
import re
from dataclasses import dataclass, field
from datetime import datetime
from datetime import date, datetime, timedelta, timezone
from typing import Any, Dict, List, Optional, Sequence, Tuple
# numpy imported lazily inside _vector_recall (see embeddings.py) so importing
@@ -32,6 +33,61 @@ from openjarvis.connectors.store import KnowledgeStore
logger = logging.getLogger(__name__)
_UPCOMING_TERMS = {
"next",
"upcoming",
"future",
"forthcoming",
"coming",
"soon",
}
_CALENDAR_TERMS = {
"calendar",
"calendars",
"event",
"events",
}
_CALENDAR_REQUEST_TERMS = _CALENDAR_TERMS | {
"appointment",
"appointments",
"meeting",
"meetings",
"schedule",
}
_GCALENDAR_GENERIC_TERMS = _UPCOMING_TERMS | _CALENDAR_TERMS | {
"appointment",
"appointments",
"meeting",
"meetings",
"schedule",
}
_QUERY_STOPWORDS = {
"a",
"all",
"am",
"are",
"do",
"for",
"have",
"i",
"in",
"is",
"list",
"me",
"my",
"on",
"s",
"show",
"tell",
"the",
"there",
"to",
"what",
"whats",
"when",
}
# ---------------------------------------------------------------------------
# Result types
# ---------------------------------------------------------------------------
@@ -120,6 +176,101 @@ def _snippet(content: str, max_chars: int = 500) -> str:
return flat[:max_chars].rstrip() + ""
def _query_tokens(query: str) -> set[str]:
return set(re.findall(r"[a-z0-9_]+", query.lower()))
def _sources_include_gcalendar(sources: Optional[Sequence[str]]) -> bool:
return any(str(source).lower() == "gcalendar" for source in sources or [])
def _has_upcoming_calendar_intent(
query: str,
sources: Optional[Sequence[str]],
) -> bool:
tokens = _query_tokens(query)
if not tokens or not (tokens & _UPCOMING_TERMS):
return False
if _sources_include_gcalendar(sources):
return True
if sources:
return False
return bool(tokens & _CALENDAR_REQUEST_TERMS)
def _is_generic_calendar_timeline_query(query: str) -> bool:
tokens = _query_tokens(query)
if not tokens:
return True
topic_tokens = tokens - _GCALENDAR_GENERIC_TERMS - _QUERY_STOPWORDS
return not topic_tokens
def _start_is_nowish_or_future(start: Optional[datetime]) -> bool:
if start is None:
return False
now = datetime.now(tz=start.tzinfo) if start.tzinfo else datetime.now()
return start >= now - timedelta(days=1)
def _start_of_day(ts: datetime) -> datetime:
return ts.replace(hour=0, minute=0, second=0, microsecond=0)
def _as_utc(ts: Optional[datetime]) -> Optional[datetime]:
if ts is None:
return None
if ts.tzinfo is None:
return ts.replace(tzinfo=timezone.utc)
return ts.astimezone(timezone.utc)
def _parse_timestamp_for_timeline(
raw: Any,
) -> Tuple[Optional[datetime], Optional[date]]:
if raw is None:
return None, None
text = str(raw).strip()
if not text:
return None, None
try:
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
except ValueError:
return None, None
is_naive_midnight = (
parsed.tzinfo is None
and parsed.hour == 0
and parsed.minute == 0
and parsed.second == 0
and parsed.microsecond == 0
)
return _as_utc(parsed), parsed.date() if is_naive_midnight else None
def _timestamp_in_range(
timestamp: Optional[datetime],
time_range: Optional[Tuple[Optional[datetime], Optional[datetime]]],
*,
all_day_date: Optional[date] = None,
) -> bool:
if timestamp is None or time_range is None:
return timestamp is not None
start, end = time_range
if all_day_date is not None:
if start is not None and all_day_date < start.date():
return False
if end is not None and all_day_date > end.date():
return False
return True
start_utc = _as_utc(start)
end_utc = _as_utc(end)
if start_utc is not None and timestamp < start_utc:
return False
if end_utc is not None and timestamp > end_utc:
return False
return True
# ---------------------------------------------------------------------------
# HybridSearch
# ---------------------------------------------------------------------------
@@ -377,6 +528,128 @@ class HybridSearch:
for r in rows
]
def _normalise_calendar_timeline_scope(
self,
query: str,
time_range: Optional[Tuple[Optional[datetime], Optional[datetime]]],
sources: Optional[Sequence[str]],
) -> Tuple[
Optional[Tuple[Optional[datetime], Optional[datetime]]],
Optional[Sequence[str]],
bool,
bool,
]:
"""Fill in structured filters for generic upcoming-calendar requests.
Queries like "what are my next calendar events?" often have no useful
lexical terms in the stored event text, so BM25/vector ranking can miss
nearby events. Treat that shape as a source-filtered timeline request.
"""
scoped_sources = list(sources) if sources else None
has_upcoming_intent = _has_upcoming_calendar_intent(query, scoped_sources)
if has_upcoming_intent and (
scoped_sources is None or _sources_include_gcalendar(scoped_sources)
):
scoped_sources = ["gcalendar"]
if not _sources_include_gcalendar(scoped_sources):
return time_range, scoped_sources, False, False
if has_upcoming_intent:
if time_range is None:
time_range = (_start_of_day(datetime.now(timezone.utc)), None)
else:
start, end = time_range
if start is None:
time_range = (_start_of_day(datetime.now(timezone.utc)), end)
else:
time_range = (_start_of_day(start), end)
chronological = has_upcoming_intent or (
time_range is not None
and time_range[1] is None
and _start_is_nowish_or_future(time_range[0])
)
metadata_only = chronological and _is_generic_calendar_timeline_query(query)
return time_range, scoped_sources, chronological, metadata_only
def _calendar_timeline_ids(
self,
*,
person: Optional[str],
time_range: Optional[Tuple[Optional[datetime], Optional[datetime]]],
sources: Optional[Sequence[str]],
limit: int,
) -> List[str]:
"""Return gcalendar rows sorted by normalized event start time."""
filter_sql, filter_params = self._build_filters(
person=person,
time_range=None,
sources=sources,
)
rows = self._store._conn.execute(
f"""
SELECT id, timestamp, created_at
FROM knowledge_chunks
WHERE {filter_sql}
""",
filter_params,
).fetchall()
candidates: List[Tuple[str, datetime, float]] = []
for row in rows:
timestamp, all_day_date = _parse_timestamp_for_timeline(row["timestamp"])
if not _timestamp_in_range(
timestamp,
time_range,
all_day_date=all_day_date,
):
continue
candidates.append(
(
row["id"],
timestamp or datetime.max.replace(tzinfo=timezone.utc),
float(row["created_at"] or 0.0),
)
)
candidates.sort(key=lambda item: (item[1], item[2]))
return [chunk_id for chunk_id, *_ in candidates[:limit]]
def _filter_calendar_timeline_fused(
self,
fused: List[Tuple[str, float, float, float]],
time_range: Optional[Tuple[Optional[datetime], Optional[datetime]]],
) -> List[Tuple[str, float, float, float]]:
"""Apply normalized timestamp filtering to ranked calendar candidates."""
if not fused:
return fused
ids = [chunk_id for chunk_id, *_ in fused]
placeholders = ",".join("?" for _ in ids)
rows = self._store._conn.execute(
f"""
SELECT id, timestamp
FROM knowledge_chunks
WHERE id IN ({placeholders})
""",
ids,
).fetchall()
timestamps = {
row["id"]: _parse_timestamp_for_timeline(row["timestamp"])
for row in rows
}
def _keeps_item(item: Tuple[str, float, float, float]) -> bool:
timestamp, all_day_date = timestamps.get(item[0], (None, None))
return _timestamp_in_range(
timestamp,
time_range,
all_day_date=all_day_date,
)
return [item for item in fused if _keeps_item(item)]
# ------------------------------------------------------------------
# Public entry point
# ------------------------------------------------------------------
@@ -396,42 +669,65 @@ class HybridSearch:
when callers want a pure metadata filter (e.g. "all mail from X in
May") — in that case only the vector leg runs (and only if an
embedder is configured); if neither leg yields anything the
structured filter is applied directly and the most recent rows are
returned.
structured filter is applied directly. Upcoming calendar timelines are
returned nearest-first; other fallbacks return the most recent rows.
"""
time_range, sources, chronological_order, metadata_only = (
self._normalise_calendar_timeline_scope(query, time_range, sources)
)
rank_query = "" if metadata_only else query
calendar_timeline = chronological_order and _sources_include_gcalendar(sources)
recall_time_range = None if calendar_timeline else time_range
bm25_filter_sql, bm25_filter_params = self._build_filters(
person=person, time_range=time_range, sources=sources, alias="kc"
person=person, time_range=recall_time_range, sources=sources, alias="kc"
)
unaliased_filter_sql, unaliased_filter_params = self._build_filters(
person=person, time_range=time_range, sources=sources
person=person, time_range=recall_time_range, sources=sources
)
bm25 = (
self._bm25_recall(query, bm25_filter_sql, bm25_filter_params)
if query.strip()
self._bm25_recall(rank_query, bm25_filter_sql, bm25_filter_params)
if rank_query.strip()
else []
)
vector = (
self._vector_recall(query, unaliased_filter_sql, unaliased_filter_params)
if query.strip()
self._vector_recall(
rank_query,
unaliased_filter_sql,
unaliased_filter_params,
)
if rank_query.strip()
else []
)
fused = self._fuse(bm25, vector)
if calendar_timeline:
fused = self._filter_calendar_timeline_fused(fused, time_range)
# Metadata-only fallback: empty query, or both legs produced nothing
# despite a non-empty query. Return the most recent rows matching the
# filter so the agent still gets a useful corpus snapshot.
# despite a non-empty query. Calendar timeline requests use start-time
# ascending; other searches use recency so the agent still gets a
# useful corpus snapshot.
if not fused:
sql = f"""
SELECT id FROM knowledge_chunks
WHERE {unaliased_filter_sql}
ORDER BY timestamp DESC, created_at DESC
LIMIT ?
"""
rows = self._store._conn.execute(
sql, [*unaliased_filter_params, limit]
).fetchall()
fused = [(row["id"], 0.0, 0.0, 0.0) for row in rows]
if calendar_timeline:
chunk_ids = self._calendar_timeline_ids(
person=person,
time_range=time_range,
sources=sources,
limit=limit,
)
fused = [(chunk_id, 0.0, 0.0, 0.0) for chunk_id in chunk_ids]
else:
sql = f"""
SELECT id FROM knowledge_chunks
WHERE {unaliased_filter_sql}
ORDER BY timestamp DESC, created_at DESC
LIMIT ?
"""
rows = self._store._conn.execute(
sql, [*unaliased_filter_params, limit]
).fetchall()
fused = [(row["id"], 0.0, 0.0, 0.0) for row in rows]
# Materialise the top-N rows in one IN-clause round trip.
top = fused[:limit]
+15
View File
@@ -593,6 +593,14 @@ class IntelligenceConfig:
stop_sequences: str = "" # Comma-separated stop strings
@dataclass(slots=True)
class DeepResearchConfig:
"""Planner settings for the web Deep Research endpoint."""
engine: str = "" # Empty means use the active chat engine.
model: str = "" # Empty means use the active chat model.
@dataclass(slots=True)
class RoutingLearningConfig:
"""Routing sub-policy config within Learning."""
@@ -1578,6 +1586,7 @@ class JarvisConfig:
hardware: HardwareInfo = field(default_factory=HardwareInfo)
engine: EngineConfig = field(default_factory=EngineConfig)
intelligence: IntelligenceConfig = field(default_factory=IntelligenceConfig)
deep_research: DeepResearchConfig = field(default_factory=DeepResearchConfig)
learning: LearningConfig = field(default_factory=LearningConfig)
tools: ToolsConfig = field(default_factory=ToolsConfig)
agent: AgentConfig = field(default_factory=AgentConfig)
@@ -1839,6 +1848,7 @@ def load_config(path: Optional[Path] = None) -> JarvisConfig:
top_sections = (
"engine",
"intelligence",
"deep_research",
"learning",
"agent",
"server",
@@ -2007,6 +2017,10 @@ max_tokens = 1024
# repetition_penalty = 1.0
# stop_sequences = ""
# [deep_research]
# engine = "" # empty = use [engine].default
# model = "" # empty = use [intelligence].default_model
[agent]
default_agent = "simple"
max_turns = 10
@@ -2177,6 +2191,7 @@ __all__ = [
"DEFAULT_CONFIG_DIR",
"DEFAULT_CONFIG_PATH",
"DiscordChannelConfig",
"DeepResearchConfig",
"get_cache_dir",
"get_config_dir",
"get_config_path",
+6 -1
View File
@@ -63,7 +63,7 @@ class Message:
"""A single chat message (OpenAI-compatible structure)."""
role: Role
content: str = ""
content: str | None = ""
name: Optional[str] = None
tool_calls: Optional[List[ToolCall]] = None
tool_call_id: Optional[str] = None
@@ -73,6 +73,11 @@ class Message:
# empty for text-only messages (the common case).
images: Optional[List[str]] = None
@property
def text(self) -> str:
"""Return message content as text, treating ``None`` as empty."""
return self.content or ""
@dataclass(slots=True)
class Conversation:
+20 -2
View File
@@ -13,6 +13,22 @@ class EngineConnectionError(Exception):
"""Raised when an engine is unreachable."""
_REASONING_METADATA_KEYS = ("reasoning_content", "thinking")
def _message_estimated_chars(message: Message) -> int:
parts = [message.text]
for key in _REASONING_METADATA_KEYS:
value = message.metadata.get(key)
if isinstance(value, str):
parts.append(value)
for tc in message.tool_calls or []:
parts.extend((tc.id, tc.name, tc.arguments))
if message.tool_call_id:
parts.append(message.tool_call_id)
return sum(len(part) for part in parts)
def messages_to_dicts(messages: Sequence[Message]) -> List[Dict[str, Any]]:
"""Convert ``Message`` objects to OpenAI-format dicts."""
out: List[Dict[str, Any]] = []
@@ -53,9 +69,11 @@ def estimate_prompt_tokens(messages: Sequence[Message]) -> int:
provider would charge.
Uses ~4 characters per token (standard BPE average for English) plus
a small per-message overhead for role markers and separators.
a small per-message overhead for role markers and separators. Counts
content, reasoning metadata, tool-call payloads, and tool result IDs
because all are replayed into later prompt turns when present.
"""
total_chars = sum(len(m.content) for m in messages)
total_chars = sum(_message_estimated_chars(m) for m in messages)
# ~4 tokens overhead per message for role markers / separators
overhead = len(messages) * 4
return max(1, total_chars // 4 + overhead)
+127 -20
View File
@@ -27,7 +27,7 @@ import threading
import time
from typing import Any, AsyncGenerator, Callable, Dict, List, Optional
from fastapi import APIRouter
from fastapi import APIRouter, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
@@ -38,9 +38,10 @@ from openjarvis.agents.research_loop import (
from openjarvis.connectors.embeddings import OllamaEmbedder
from openjarvis.connectors.hybrid_search import HybridSearch
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.config import DEFAULT_CONFIG_DIR, JarvisConfig, load_config
from openjarvis.core.types import TelemetryRecord
from openjarvis.engine.ollama import OllamaEngine
from openjarvis.engine._base import InferenceEngine
from openjarvis.engine._discovery import get_engine
from openjarvis.telemetry.store import TelemetryStore
logger = logging.getLogger(__name__)
@@ -48,13 +49,99 @@ logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["research"])
_WEB_CLARIFY_RESPONSE = "no clarification available in web session"
_LEGACY_PLANNER_ENGINE = "ollama"
# Sentinel placed on the queue when the agent thread terminates.
_DONE = object()
def _first_nonempty(*values: str) -> str:
for value in values:
stripped = value.strip()
if stripped:
return stripped
return ""
def _resolve_planner_config(
config: JarvisConfig,
*,
active_engine_key: str = "",
active_model: str = "",
request_model: str = "",
) -> tuple[str, str]:
"""Resolve the planner engine/model for web Deep Research.
Resolution order:
1. explicit ``[deep_research]`` overrides,
2. the active chat engine/request model,
3. server/config defaults,
4. legacy Ollama/gemma4 fallback for unconfigured installs.
"""
engine_key = _first_nonempty(
config.deep_research.engine,
active_engine_key,
config.engine.default,
_LEGACY_PLANNER_ENGINE,
)
model = _first_nonempty(
config.deep_research.model,
request_model,
active_model,
config.server.model,
config.intelligence.default_model,
DEFAULT_PLANNER_MODEL,
)
return engine_key, model
def _build_planner_engine(
config: JarvisConfig,
*,
active_engine: InferenceEngine | None = None,
active_engine_key: str = "",
active_model: str = "",
request_model: str = "",
) -> tuple[str, InferenceEngine, str]:
"""Instantiate the exact configured planner engine.
``get_engine`` intentionally falls back to any healthy engine for general
chat routing. Deep Research must not do that here: if the configured chat
engine is LM Studio but unavailable, silently falling back to Ollama would
recreate the issue this endpoint is fixing.
"""
engine_key, model = _resolve_planner_config(
config,
active_engine_key=active_engine_key,
active_model=active_model,
request_model=request_model,
)
if active_engine is not None and not config.deep_research.engine.strip():
if model and not active_engine.can_serve(model):
raise RuntimeError(
"Deep Research planner engine "
f"{engine_key!r} cannot serve model {model!r}. "
"Choose a compatible model or set [deep_research] engine/model "
"in config.toml."
)
return engine_key, active_engine, model
resolved = get_engine(config, engine_key=engine_key, model=model)
if resolved is None or resolved[0] != engine_key:
raise RuntimeError(
"Deep Research planner engine "
f"{engine_key!r} is unavailable or cannot serve model {model!r}. "
"Start the configured engine, load the configured model, or set "
"[deep_research] engine/model in config.toml."
)
resolved_key, engine = resolved
return resolved_key, engine, model
def _record_research_telemetry(
*,
engine_key: str,
model: str,
usage: Dict[str, int],
latency_seconds: float,
@@ -86,7 +173,7 @@ def _record_research_telemetry(
rec = TelemetryRecord(
timestamp=time.time(),
model_id=model,
engine="ollama",
engine=engine_key,
agent="research",
prompt_tokens=int(usage.get("prompt_tokens", 0)),
prompt_tokens_evaluated=int(usage.get("prompt_tokens", 0)),
@@ -244,12 +331,11 @@ class _LiveGPUSampler:
class ResearchRequest(BaseModel):
query: str = Field(..., description="Natural-language question to research.")
# Deep Research has its own model requirements (function-calling support,
# sufficient reasoning capability) that the chat-model selector should not
# override. We accept the field for forward-compat with older clients but
# ignore it — the planner always runs on DEFAULT_PLANNER_MODEL.
# Preferred planner model from the active chat selector. Server-side
# [deep_research] config can still override it when a dedicated planner is
# desired.
model: Optional[str] = Field(
default=None, description="Ignored; retained for client compatibility."
default=None, description="Preferred planner model for this request."
)
@@ -290,7 +376,14 @@ def _chunk_synthesis(text: str, window_chars: int = 40) -> list[str]:
# ---------------------------------------------------------------------------
async def _stream_research(query: str, model: str) -> AsyncGenerator[str, None]:
async def _stream_research(
query: str,
*,
active_engine: InferenceEngine | None = None,
active_engine_key: str = "",
active_model: str = "",
request_model: str = "",
) -> AsyncGenerator[str, None]:
"""Drive ResearchAgent on a worker thread; yield SSE frames as they land.
Three error envelopes setup, worker, consumer all funnel into the
@@ -298,7 +391,7 @@ async def _stream_research(query: str, model: str) -> AsyncGenerator[str, None]:
``{"type": "done", "usage": {...}}``. The client can rely on always
seeing a ``done`` frame, even when the agent never started.
"""
# Phase 1: setup. Failures here (Ollama daemon down, DB locked, etc.)
# Phase 1: setup. Failures here (planner engine down, DB locked, etc.)
# yield error + done and return — nothing has been emitted yet so the
# client gets a clean two-frame stream instead of a dangling connection.
try:
@@ -309,6 +402,15 @@ async def _stream_research(query: str, model: str) -> AsyncGenerator[str, None]:
# Called from the agent's worker thread; bounce onto the event loop.
loop.call_soon_threadsafe(queue.put_nowait, event)
config = load_config()
engine_key, engine, model = _build_planner_engine(
config,
active_engine=active_engine,
active_engine_key=active_engine_key,
active_model=active_model,
request_model=request_model,
)
# Each request gets its own thin set of connectors. Constructing them
# is cheap (SQLite open + HTTP keepalive) and avoids state leaks
# between concurrent requests.
@@ -320,7 +422,6 @@ async def _stream_research(query: str, model: str) -> AsyncGenerator[str, None]:
)
embedder = None
engine = OllamaEngine()
agent = ResearchAgent(
engine=engine,
search=HybridSearch(store, embedder),
@@ -367,6 +468,7 @@ async def _stream_research(query: str, model: str) -> AsyncGenerator[str, None]:
# rolls research into the same Power/Energy numbers as chat —
# this is what the launch-video System panel reads.
_record_research_telemetry(
engine_key=engine_key,
model=model,
usage=usage_dict,
latency_seconds=time.time() - t0,
@@ -472,7 +574,7 @@ async def _stream_research(query: str, model: str) -> AsyncGenerator[str, None]:
@router.post("/research")
async def research(req: ResearchRequest) -> StreamingResponse:
async def research(req: ResearchRequest, request: Request) -> StreamingResponse:
"""Run a research query and stream the agent's trace + synthesis via SSE.
Response is ``text/event-stream`` with one JSON event per frame. See the
@@ -480,14 +582,19 @@ async def research(req: ResearchRequest) -> StreamingResponse:
terminates the stream so clients can detect end-of-response without
parsing the underlying ``[DONE]`` sentinel used by OpenAI-style routes.
"""
if req.model and req.model != DEFAULT_PLANNER_MODEL:
logger.info(
"research: ignoring client model=%r; using DEFAULT_PLANNER_MODEL=%r",
req.model,
DEFAULT_PLANNER_MODEL,
)
active_engine = getattr(request.app.state, "engine", None)
active_model = str(getattr(request.app.state, "model", "") or "")
active_engine_key = str(getattr(request.app.state, "engine_name", "") or "")
if active_engine is not None and not active_engine_key:
active_engine_key = str(getattr(active_engine, "engine_id", "") or "")
return StreamingResponse(
_stream_research(req.query, DEFAULT_PLANNER_MODEL),
_stream_research(
req.query,
active_engine=active_engine,
active_engine_key=active_engine_key,
active_model=active_model,
request_model=req.model or "",
),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
+46 -1
View File
@@ -8,7 +8,7 @@ from openjarvis.agents._stubs import AgentContext
from openjarvis.agents.native_openhands import NativeOpenHandsAgent
from openjarvis.core.events import EventBus, EventType
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import Conversation, Message, Role, ToolResult
from openjarvis.core.types import Conversation, Message, Role, ToolCall, ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
# ---------------------------------------------------------------------------
@@ -118,6 +118,51 @@ class TestNativeOpenHandsRegistration:
class TestNativeOpenHandsAgent:
def test_truncate_handles_none_content_tool_call_turn(self):
"""Tool-call assistant turns may carry content=None."""
engine = MagicMock()
engine.engine_id = "mock"
agent = NativeOpenHandsAgent(engine, "test-model")
messages = [
Message(role=Role.USER, content="hi"),
Message(
role=Role.ASSISTANT,
content=None, # type: ignore[arg-type]
tool_calls=[ToolCall(id="call_1", name="calculator", arguments="{}")],
),
]
assert agent._truncate_if_needed(messages) == messages
def test_native_tool_call_with_none_content_does_not_crash(self):
"""Native tool-call responses may omit assistant text content."""
engine = MagicMock()
engine.engine_id = "mock"
engine.generate.side_effect = [
_engine_response(
None,
tool_calls=[
{
"id": "call_1",
"name": "calculator",
"arguments": '{"expression": "2+2"}',
}
],
),
_engine_response("The result is 4."),
]
agent = NativeOpenHandsAgent(
engine,
"test-model",
tools=[_CalculatorStub()],
)
result = agent.run("What is 2+2?")
assert result.content == "The result is 4."
assert result.turns == 2
assert [tr.content for tr in result.tool_results] == ["4"]
def test_simple_response(self):
"""No code -> direct answer."""
engine = MagicMock()
+7
View File
@@ -395,6 +395,13 @@ def test_system_prompt_mandates_sources_extraction() -> None:
assert "{available_sources}" in SYSTEM_PROMPT
def test_system_prompt_routes_upcoming_calendar_as_structured_search() -> None:
"""Upcoming calendar requests need source/time filters, not just keywords."""
assert 'sources=["gcalendar"]' in SYSTEM_PROMPT
assert 'time_range={{"start": "{today}"}}' in SYSTEM_PROMPT
assert 'query=""' in SYSTEM_PROMPT
# ---------------------------------------------------------------------------
# Dynamic available_sources — only list what the user actually has connected
# ---------------------------------------------------------------------------
+10
View File
@@ -6,6 +6,7 @@ All Calendar API calls are mocked; no network access is required.
from __future__ import annotations
import json
from datetime import datetime
from pathlib import Path
from typing import List
from unittest.mock import patch
@@ -134,6 +135,15 @@ def test_sync_yields_events(
mock_events.assert_called_once()
def test_parse_event_timestamp_handles_all_day_events() -> None:
"""All-day events use their calendar date, not the current wall clock."""
from openjarvis.connectors.gcalendar import _parse_event_timestamp # noqa: PLC0415
timestamp = _parse_event_timestamp({"start": {"date": "2024-05-26"}})
assert timestamp == datetime(2024, 5, 26)
# ---------------------------------------------------------------------------
# Test 4 — disconnect removes the credentials file
# ---------------------------------------------------------------------------
+181
View File
@@ -0,0 +1,181 @@
"""Tests for source-aware HybridSearch behavior."""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from openjarvis.connectors.hybrid_search import HybridSearch
from openjarvis.connectors.store import KnowledgeStore
def _store_doc(
store: KnowledgeStore,
*,
title: str,
source: str,
timestamp: datetime | str,
) -> None:
timestamp_text = (
timestamp.isoformat() if isinstance(timestamp, datetime) else timestamp
)
store.store(
content=f"Title: {title}\nWhen: {timestamp_text}",
source=source,
doc_type="event" if source == "gcalendar" else "email",
doc_id=f"{source}:{title.lower().replace(' ', '-')}",
title=title,
timestamp=timestamp,
)
def test_next_calendar_events_returns_nearest_gcalendar_rows() -> None:
"""Generic upcoming-calendar queries should be chronological timelines."""
store = KnowledgeStore(db_path=":memory:")
_store_doc(
store,
title="Calendar Digest Email",
source="gmail",
timestamp=datetime(2999, 1, 1, 9, tzinfo=timezone.utc),
)
_store_doc(
store,
title="Birthday Reminder",
source="gcalendar",
timestamp=datetime(2999, 12, 1, 9, tzinfo=timezone.utc),
)
_store_doc(
store,
title="Music Lesson",
source="gcalendar",
timestamp=datetime(2999, 5, 26, 18, tzinfo=timezone.utc),
)
_store_doc(
store,
title="Team Sync",
source="gcalendar",
timestamp=datetime(2999, 5, 27, 10, tzinfo=timezone.utc),
)
search = HybridSearch(store)
hits = search.search("what are my next calendar events?", limit=2)
contraction_hits = search.search("what's next on my calendar?", limit=2)
meetings_hits = search.search("what are my next meetings?", limit=2)
mixed_source_hits = search.search(
"what are my next calendar events?",
sources=["gmail", "gcalendar"],
limit=2,
)
assert [hit.title for hit in hits] == ["Music Lesson", "Team Sync"]
assert all(hit.source == "gcalendar" for hit in hits)
assert [hit.title for hit in contraction_hits] == ["Music Lesson", "Team Sync"]
assert all(hit.source == "gcalendar" for hit in contraction_hits)
assert [hit.title for hit in meetings_hits] == ["Music Lesson", "Team Sync"]
assert all(hit.source == "gcalendar" for hit in meetings_hits)
assert [hit.title for hit in mixed_source_hits] == ["Music Lesson", "Team Sync"]
assert all(hit.source == "gcalendar" for hit in mixed_source_hits)
def test_empty_upcoming_calendar_filter_uses_ascending_start_time() -> None:
"""Planner-emitted structured calendar searches return nearest first."""
store = KnowledgeStore(db_path=":memory:")
_store_doc(
store,
title="Later Event",
source="gcalendar",
timestamp=datetime(2999, 8, 1, 9, tzinfo=timezone.utc),
)
_store_doc(
store,
title="Sooner Event",
source="gcalendar",
timestamp=datetime(2999, 7, 1, 9, tzinfo=timezone.utc),
)
hits = HybridSearch(store).search(
"",
sources=["gcalendar"],
time_range=(datetime(2999, 1, 1, tzinfo=timezone.utc), None),
limit=2,
)
assert [hit.title for hit in hits] == ["Sooner Event", "Later Event"]
def test_upcoming_calendar_timeline_normalizes_timestamp_offsets() -> None:
"""Timeline filtering and ordering should compare instants, not ISO text."""
store = KnowledgeStore(db_path=":memory:")
_store_doc(
store,
title="Offset Earlier",
source="gcalendar",
timestamp="2999-07-01T00:30:00+02:00",
)
_store_doc(
store,
title="UTC Later",
source="gcalendar",
timestamp="2999-06-30T23:15:00+00:00",
)
search = HybridSearch(store)
hits = search.search(
"",
sources=["gcalendar"],
time_range=(datetime(2999, 6, 30, 22, tzinfo=timezone.utc), None),
limit=2,
)
later_hits = search.search(
"",
sources=["gcalendar"],
time_range=(datetime(2999, 6, 30, 23, tzinfo=timezone.utc), None),
limit=2,
)
assert [hit.title for hit in hits] == ["Offset Earlier", "UTC Later"]
assert [hit.title for hit in later_hits] == ["UTC Later"]
def test_upcoming_calendar_includes_today_all_day_events() -> None:
"""Upcoming calendar intent starts at the day boundary for all-day events."""
store = KnowledgeStore(db_path=":memory:")
_store_doc(
store,
title="All Day Today",
source="gcalendar",
timestamp="2999-07-01T00:00:00",
)
_store_doc(
store,
title="Morning Tomorrow",
source="gcalendar",
timestamp="2999-07-02T09:00:00+00:00",
)
hits = HybridSearch(store).search(
"next calendar events",
sources=["gcalendar"],
time_range=(datetime(2999, 7, 1, 12, tzinfo=timezone.utc), None),
limit=2,
)
local_tz_hits = HybridSearch(store).search(
"",
sources=["gcalendar"],
time_range=(
datetime(
2999,
7,
1,
12,
tzinfo=timezone(timedelta(hours=-7)),
),
None,
),
limit=2,
)
assert [hit.title for hit in hits] == ["All Day Today", "Morning Tomorrow"]
assert [hit.title for hit in local_tz_hits] == [
"All Day Today",
"Morning Tomorrow",
]
+58
View File
@@ -0,0 +1,58 @@
"""Tests for Deep Research planner configuration."""
from __future__ import annotations
from pathlib import Path
import pytest
from openjarvis.core.config import (
DeepResearchConfig,
HardwareInfo,
JarvisConfig,
generate_default_toml,
load_config,
validate_config_key,
)
def test_deep_research_config_defaults_to_chat_selection() -> None:
cfg = JarvisConfig()
assert isinstance(cfg.deep_research, DeepResearchConfig)
assert cfg.deep_research.engine == ""
assert cfg.deep_research.model == ""
def test_loads_deep_research_overrides(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.setenv("OPENJARVIS_HOME", str(tmp_path / "home"))
config_file = tmp_path / "config.toml"
config_file.write_text(
"\n".join(
[
"[deep_research]",
'engine = "lmstudio"',
'model = "qwen/qwen3-14b"',
]
)
)
cfg = load_config(config_file)
assert cfg.deep_research.engine == "lmstudio"
assert cfg.deep_research.model == "qwen/qwen3-14b"
def test_deep_research_keys_are_settable() -> None:
assert validate_config_key("deep_research.engine") is str
assert validate_config_key("deep_research.model") is str
def test_default_toml_documents_deep_research_override() -> None:
toml = generate_default_toml(HardwareInfo())
assert "# [deep_research]" in toml
assert '# engine = ""' in toml
assert '# model = ""' in toml
+6
View File
@@ -35,9 +35,15 @@ class TestMessage:
msg = Message(role=Role.USER, content="hello")
assert msg.role == Role.USER
assert msg.content == "hello"
assert msg.text == "hello"
assert msg.tool_calls is None
assert msg.metadata == {}
def test_none_content_text_helper(self) -> None:
msg = Message(role=Role.ASSISTANT, content=None)
assert msg.content is None
assert msg.text == ""
def test_tool_calls(self) -> None:
tc = ToolCall(id="1", name="calc", arguments='{"x": 1}')
msg = Message(role=Role.ASSISTANT, content="", tool_calls=[tc])
+60
View File
@@ -0,0 +1,60 @@
from __future__ import annotations
from openjarvis.core.types import Message, Role, ToolCall
from openjarvis.engine._base import estimate_prompt_tokens
def test_estimate_prompt_tokens_handles_none_content_tool_call_turn() -> None:
messages = [
Message(role=Role.USER, content="hi"),
Message(
role=Role.ASSISTANT,
content=None,
tool_calls=[ToolCall(id="call_1", name="lookup", arguments="{}")],
),
]
assert estimate_prompt_tokens(messages) == 12
def test_estimate_prompt_tokens_counts_tool_call_arguments() -> None:
base = [
Message(role=Role.USER, content="hi"),
Message(role=Role.ASSISTANT, content=None),
]
with_tool_call = [
Message(role=Role.USER, content="hi"),
Message(
role=Role.ASSISTANT,
content=None,
tool_calls=[ToolCall(id="", name="", arguments="abcdefgh")],
),
]
assert estimate_prompt_tokens(with_tool_call) - estimate_prompt_tokens(base) == 2
def test_estimate_prompt_tokens_counts_reasoning_metadata() -> None:
base = [
Message(role=Role.USER, content="hi"),
Message(role=Role.ASSISTANT, content=None),
]
with_reasoning = [
Message(role=Role.USER, content="hi"),
Message(
role=Role.ASSISTANT,
content=None,
metadata={"reasoning_content": "abcdefgh"},
),
]
assert estimate_prompt_tokens(with_reasoning) - estimate_prompt_tokens(base) == 2
def test_estimate_prompt_tokens_counts_tool_result_ids() -> None:
messages = [
Message(role=Role.USER, content="hi"),
Message(role=Role.TOOL, content="ok", tool_call_id="abcdefgh"),
]
assert estimate_prompt_tokens(messages) == 11
+285
View File
@@ -0,0 +1,285 @@
"""Tests for web Deep Research planner engine selection."""
from __future__ import annotations
import asyncio
from types import SimpleNamespace
import pytest
from openjarvis.agents.research_loop import DEFAULT_PLANNER_MODEL
from openjarvis.core.config import JarvisConfig
from openjarvis.server import research_router
class _DummyEngine:
def __init__(self, servable: bool = True) -> None:
self.servable = servable
def can_serve(self, model: str) -> bool:
return self.servable
def test_resolve_planner_config_uses_chat_defaults() -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "local-model"
assert research_router._resolve_planner_config(cfg) == (
"lmstudio",
"local-model",
)
def test_resolve_planner_config_prefers_active_chat_runtime() -> None:
cfg = JarvisConfig()
cfg.engine.default = "ollama"
cfg.intelligence.default_model = ""
assert research_router._resolve_planner_config(
cfg,
active_engine_key="lmstudio",
active_model="server-model",
request_model="selected-model",
) == (
"lmstudio",
"selected-model",
)
def test_resolve_planner_config_uses_server_model_before_legacy_default() -> None:
cfg = JarvisConfig()
cfg.engine.default = "ollama"
cfg.intelligence.default_model = ""
cfg.server.model = "serve-model"
assert research_router._resolve_planner_config(cfg) == (
"ollama",
"serve-model",
)
def test_resolve_planner_config_allows_deep_research_override() -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "chat-model"
cfg.deep_research.engine = "vllm"
cfg.deep_research.model = "planner-model"
assert research_router._resolve_planner_config(cfg) == (
"vllm",
"planner-model",
)
def test_resolve_planner_config_allows_partial_model_override() -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "chat-model"
cfg.deep_research.model = "planner-model"
assert research_router._resolve_planner_config(cfg) == (
"lmstudio",
"planner-model",
)
def test_resolve_planner_config_allows_partial_engine_override() -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "chat-model"
cfg.deep_research.engine = "vllm"
assert research_router._resolve_planner_config(cfg) == (
"vllm",
"chat-model",
)
def test_resolve_planner_config_keeps_legacy_fallback_when_unconfigured() -> None:
cfg = JarvisConfig()
cfg.engine.default = ""
cfg.intelligence.default_model = ""
assert research_router._resolve_planner_config(cfg) == (
"ollama",
DEFAULT_PLANNER_MODEL,
)
def test_build_planner_engine_uses_configured_engine(
monkeypatch: pytest.MonkeyPatch,
) -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "local-model"
engine = _DummyEngine()
calls: list[tuple[str | None, str | None]] = []
def fake_get_engine(
config: JarvisConfig,
engine_key: str | None = None,
model: str | None = None,
) -> tuple[str, _DummyEngine]:
calls.append((engine_key, model))
return "lmstudio", engine
monkeypatch.setattr(research_router, "get_engine", fake_get_engine)
engine_key, resolved_engine, model = research_router._build_planner_engine(cfg)
assert calls == [("lmstudio", "local-model")]
assert engine_key == "lmstudio"
assert resolved_engine is engine
assert model == "local-model"
def test_build_planner_engine_uses_active_engine_without_config_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
cfg = JarvisConfig()
cfg.engine.default = "ollama"
cfg.intelligence.default_model = ""
active_engine = _DummyEngine()
def fail_get_engine(*args: object, **kwargs: object) -> None:
raise AssertionError("should use the live app engine")
monkeypatch.setattr(research_router, "get_engine", fail_get_engine)
engine_key, resolved_engine, model = research_router._build_planner_engine(
cfg,
active_engine=active_engine,
active_engine_key="lmstudio",
active_model="server-model",
request_model="selected-model",
)
assert engine_key == "lmstudio"
assert resolved_engine is active_engine
assert model == "selected-model"
def test_build_planner_engine_rejects_active_engine_that_cannot_serve_model() -> None:
cfg = JarvisConfig()
with pytest.raises(RuntimeError, match="selected-model"):
research_router._build_planner_engine(
cfg,
active_engine=_DummyEngine(servable=False),
active_engine_key="cloud",
request_model="selected-model",
)
def test_build_planner_engine_honors_explicit_deep_research_engine(
monkeypatch: pytest.MonkeyPatch,
) -> None:
cfg = JarvisConfig()
cfg.deep_research.engine = "vllm"
cfg.deep_research.model = "planner-model"
active_engine = _DummyEngine()
planner_engine = _DummyEngine()
def fake_get_engine(
config: JarvisConfig,
engine_key: str | None = None,
model: str | None = None,
) -> tuple[str, _DummyEngine]:
assert engine_key == "vllm"
assert model == "planner-model"
return "vllm", planner_engine
monkeypatch.setattr(research_router, "get_engine", fake_get_engine)
engine_key, resolved_engine, model = research_router._build_planner_engine(
cfg,
active_engine=active_engine,
active_engine_key="lmstudio",
active_model="chat-model",
request_model="selected-model",
)
assert engine_key == "vllm"
assert resolved_engine is planner_engine
assert model == "planner-model"
def test_research_route_passes_live_engine_and_selected_model(
monkeypatch: pytest.MonkeyPatch,
) -> None:
captured: dict[str, object] = {}
active_engine = _DummyEngine()
def fake_stream(query: str, **kwargs: object):
captured["query"] = query
captured.update(kwargs)
async def gen():
yield "data: {\"type\":\"done\",\"usage\":{}}\n\n"
return gen()
request = SimpleNamespace(
app=SimpleNamespace(
state=SimpleNamespace(
engine=active_engine,
engine_name="lmstudio",
model="server-model",
)
)
)
monkeypatch.setattr(research_router, "_stream_research", fake_stream)
response = asyncio.run(
research_router.research(
research_router.ResearchRequest(
query="find notes",
model="selected-model",
),
request, # type: ignore[arg-type]
)
)
assert response.media_type == "text/event-stream"
assert captured == {
"query": "find notes",
"active_engine": active_engine,
"active_engine_key": "lmstudio",
"active_model": "server-model",
"request_model": "selected-model",
}
def test_build_planner_engine_rejects_fallback_engine(
monkeypatch: pytest.MonkeyPatch,
) -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "local-model"
def fake_get_engine(
config: JarvisConfig,
engine_key: str | None = None,
model: str | None = None,
) -> tuple[str, _DummyEngine]:
return "ollama", _DummyEngine()
monkeypatch.setattr(research_router, "get_engine", fake_get_engine)
with pytest.raises(RuntimeError, match="lmstudio"):
research_router._build_planner_engine(cfg)
def test_build_planner_engine_rejects_unavailable_engine(
monkeypatch: pytest.MonkeyPatch,
) -> None:
cfg = JarvisConfig()
cfg.engine.default = "lmstudio"
cfg.intelligence.default_model = "local-model"
monkeypatch.setattr(research_router, "get_engine", lambda *args, **kwargs: None)
with pytest.raises(RuntimeError, match="local-model"):
research_router._build_planner_engine(cfg)
Generated
+7
View File
@@ -4206,6 +4206,7 @@ dashboard = [
desktop = [
{ name = "fastapi" },
{ name = "faster-whisper" },
{ name = "openjarvis-rust" },
{ name = "pydantic" },
{ name = "python-multipart" },
{ name = "uvicorn" },
@@ -4390,6 +4391,7 @@ requires-dist = [
{ name = "openai", marker = "extra == 'inference-cloud'", specifier = ">=1.30" },
{ name = "openai", marker = "extra == 'media'", specifier = ">=1.30" },
{ name = "openhands-sdk", marker = "python_full_version >= '3.12' and extra == 'openhands'", specifier = ">=1.0" },
{ name = "openjarvis-rust", marker = "extra == 'desktop'", directory = "rust/crates/openjarvis-python" },
{ name = "pdfplumber", marker = "extra == 'memory-pdf'", specifier = ">=0.10" },
{ name = "pdfplumber", marker = "extra == 'pdf'", specifier = ">=0.10" },
{ name = "playwright", marker = "extra == 'browser'", specifier = ">=1.40" },
@@ -4445,6 +4447,11 @@ provides-extras = ["browser", "channel-discord", "channel-gmail", "channel-line"
[package.metadata.requires-dev]
dev = [{ name = "maturin", specifier = ">=1.12.6" }]
[[package]]
name = "openjarvis-rust"
version = "0.1.0"
source = { directory = "rust/crates/openjarvis-python" }
[[package]]
name = "opentelemetry-api"
version = "1.39.1"