Compare commits

...
Author SHA1 Message Date
Jon Saad-FalconandClaude Opus 4.8 e7c46c1985 fix(frontend): make Supabase anon key optional to unblock PyPI publishing (#589)
PyPI publishing had been broken since v1.0.3.dev851: #587 made VITE_SUPABASE_ANON_KEY a hard build-time requirement, but no such secret exists, so the frontend build aborted every publish run before the PyPI upload. Decouple package buildability from the leaderboard credential: a missing anon key now disables the savings leaderboard at runtime instead of failing the build, and auto-enables when the secret is provided. Verified: npm run build with the key unset succeeds; tsc + vitest pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:15:06 -07:00
github-actions[bot] 00d1e39b6d chore: update clone traffic data [skip ci] 2026-06-24 07:14:44 +00:00
Elliot SluskyandClaude Opus 4.8 843375d6ef Fix Supabase frontend build env for release builds (#588)
Follow-up to #587. Pass VITE_SUPABASE_ANON_KEY into the frontend builds of both release paths: the PyPI publish workflow (wheel-bundled frontend) and the desktop tauri-action build (npm run build:tauri -> vite build). Kept strict: a missing/empty secret fails the release by design rather than shipping a placeholder key. Requires the VITE_SUPABASE_ANON_KEY repo secret to be set for releases to succeed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:32:13 -07:00
Elliot SluskyandClaude Opus 4.8 8d33cb58fa Fix secure cloud key storage and Supabase key config (#587)
Route desktop cloud-key saves/status through the OS credential store (keyring with per-platform native backends: apple-native / windows-native / sync-secret-service), migrate the legacy plaintext ~/.openjarvis/cloud-keys.env into it, remove browser localStorage persistence of provider keys, and push key updates to the running server via /v1/cloud/reload (legacy env-file fallback retained). Remove hardcoded Supabase anon JWTs from frontend/docs source and make VITE_SUPABASE_ANON_KEY a required build var. Adds libdbus-1-dev to the Linux desktop build and a CI build var. Closes #220.

NOTE (post-merge follow-ups, not covered by CI): add the VITE_SUPABASE_ANON_KEY repo secret with the rotated key (release/docs builds otherwise use a placeholder), rotate the previously-committed Supabase anon key, and run a desktop save->restart->read smoke test to confirm keychain persistence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:11:32 -07:00
github-actions[bot] e4c4bcbae3 chore: update clone traffic data [skip ci] 2026-06-23 07:18:13 +00:00
Jon Saad-Falcon 993c24c8b9 test(server): make TestTraceRecording hermetic (env-independent) (#583)
TestTraceRecording relied on the ambient ~/.openjarvis/config.toml leaving traces.enabled at its default, so it failed on any machine with traces disabled locally (passing in CI only because the runner has no config file). Pass an explicit traces-enabled config with a tmp db_path so the tests are environment-independent and parallel-safe under pytest -n auto. Relates to #582.
2026-06-22 19:12:10 -07:00
Elliot Slusky 5bc8d3a2f6 Harden Docker and systemd deployment configs (#581)
Pin all base images and ollama to fixed versions + @sha256 digests (no floating :latest), run Docker images as an unprivileged openjarvis user (uid 10001), replace the curl|bash NodeSource install with a digest-pinned multi-stage copy, install from the committed uv.lock via uv export --frozen --no-dev (hash-verified, --no-deps), and add systemd sandboxing (NoNewPrivileges, ProtectSystem=strict, PrivateTmp, kernel/SUID protections). Closes #228, #563, #564, #565, #566, #567.
2026-06-22 19:12:07 -07:00
Elliot Slusky 433d10db5e feat(memory): native persistent memory service integrated into core (#579)
Adds the openjarvis.memory package (LocalFactStore, FactExtractor, background MemoryService), starts/stops it in the jarvis serve and jarvis chat lifecycle, feeds completed non-streaming exchanges to it, adds [memory] config support, and adds jarvis memory list/clear CLI commands. Extraction runs on a background thread and degrades to a no-op on any failure (BrokenPipe, timeouts, unparseable output) so it can never block a reply or crash the host. Disabled by default. Closes #393, #571, #572, #573.
2026-06-22 13:59:02 -07:00
Elliot Slusky 9b7b3681f6 ci: parallelize the test suite and cut install/coverage overhead (#580)
Run pytest with -n auto (pytest-xdist) and COVERAGE_CORE=sysmon, enable the uv cache, and switch test output to -q. Cuts the test job from ~40min to ~4min without changing what's tested or the 60% coverage gate.
2026-06-22 13:58:46 -07:00
44 changed files with 3205 additions and 1202 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"label": "Git Clones",
"message": "128,077",
"message": "130,895",
"color": "green",
"namedLogo": "git"
}
+5 -3
View File
@@ -1,6 +1,6 @@
{
"total_clones": 128077,
"last_updated": "2026-06-22T08:05:29Z",
"total_clones": 130895,
"last_updated": "2026-06-24T07:14:43Z",
"daily": {
"2026-03-27": 2189,
"2026-03-28": 1874,
@@ -88,6 +88,8 @@
"2026-06-18": 1408,
"2026-06-19": 1350,
"2026-06-20": 1437,
"2026-06-21": 1426
"2026-06-21": 1426,
"2026-06-22": 1350,
"2026-06-23": 1468
}
}
+12 -1
View File
@@ -23,6 +23,8 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v8.0.0
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev --extra framework-comparison --extra server
@@ -55,6 +57,8 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v8.0.0
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev --extra framework-comparison --extra server
@@ -63,8 +67,13 @@ jobs:
run: uv run maturin develop --manifest-path rust/crates/openjarvis-python/Cargo.toml
- name: Run tests
# COVERAGE_CORE=sysmon uses CPython 3.12's sys.monitoring backend,
# which is dramatically cheaper than the default C trace function.
# -n auto fans the suite out across all runner cores via pytest-xdist.
env:
COVERAGE_CORE: sysmon
run: |
uv run pytest tests/ -v --tb=short -m "not live and not cloud and not hub" \
uv run pytest tests/ -n auto -q --tb=short -m "not live and not cloud and not hub" \
--cov=openjarvis \
--cov-report=term-missing \
--cov-report=xml \
@@ -107,6 +116,8 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v8.0.0
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev --extra server
+8 -2
View File
@@ -40,7 +40,8 @@ jobs:
libappindicator3-dev \
librsvg2-dev \
patchelf \
libxdo-dev
libxdo-dev \
libdbus-1-dev
- name: Setup Node.js
uses: actions/setup-node@v6
@@ -130,7 +131,8 @@ jobs:
libappindicator3-dev \
librsvg2-dev \
patchelf \
libxdo-dev
libxdo-dev \
libdbus-1-dev
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
@@ -252,6 +254,10 @@ jobs:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
TAURI_CONFIG: '{"version":"${{ steps.release-info.outputs.tauri_version }}","bundle":{"externalBin":["binaries/ollama"]}}'
# tauri-action runs beforeBuildCommand (npm run build:tauri -> vite
# build), which requires this at build time (#587). Strict for
# releases: a missing/empty secret fails the build by design.
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
with:
projectPath: frontend
tauriScript: npx tauri
+5
View File
@@ -35,3 +35,8 @@ jobs:
- run: npm ci
- run: npx tsc --noEmit
- run: npm run build
env:
# Optional: when the secret is unset the build still succeeds and the
# leaderboard is disabled (see src/lib/supabase.ts). No placeholder,
# so a keyless CI build doesn't bake in a bogus anon key.
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
+2
View File
@@ -55,6 +55,8 @@ jobs:
cache-dependency-path: frontend/package-lock.json
- name: Build frontend and bundle into package
env:
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
run: |
set -euo pipefail
cd frontend
+31 -6
View File
@@ -1,5 +1,9 @@
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers — reproducible builds and
# safe rollbacks (#563).
# Stage 1: Build frontend SPA
FROM node:22-slim AS frontend
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS frontend
WORKDIR /frontend
COPY frontend/package.json frontend/package-lock.json* ./
@@ -8,10 +12,22 @@ COPY frontend/ .
RUN npm run build
# Stage 2: Build Python package
FROM python:3.12-slim-bookworm AS builder
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf AS builder
WORKDIR /app
COPY pyproject.toml README.md ./
# Install dependencies from the committed lockfile (#567). `uv export --frozen`
# reads uv.lock as-is (no re-resolution) and emits a fully pinned, hash-verified
# requirements set; `--no-deps` then installs exactly that set. This is a
# separate layer from the source copy so dependency installs stay cached when
# only application code changes.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt
# Copy the source and the non-src force-include paths (see pyproject
# [tool.hatch.build.targets.wheel.force-include]) before building the project.
COPY src/ src/
COPY scripts/install scripts/install
COPY deploy/windows deploy/windows
@@ -19,16 +35,25 @@ COPY deploy/windows deploy/windows
# Copy built frontend into the server static directory
COPY --from=frontend /src/openjarvis/server/static src/openjarvis/server/static/
RUN pip install --no-cache-dir uv && \
uv pip install --system ".[server]"
# Install the project itself without re-resolving dependencies.
RUN uv pip install --system --no-deps .
# Stage 3: Runtime
FROM python:3.12-slim-bookworm
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf
COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
WORKDIR /app
# Run as an unprivileged user — the server needs no root privileges, so dropping
# them limits the blast radius of a compromise (#565). The app writes only to
# $HOME (config/cache/state), which is owned by this user.
RUN groupadd --system --gid 10001 openjarvis && \
useradd --system --uid 10001 --gid openjarvis \
--create-home --home-dir /home/openjarvis openjarvis
ENV HOME=/home/openjarvis
USER openjarvis
EXPOSE 8000
ENTRYPOINT ["jarvis"]
+24 -6
View File
@@ -1,5 +1,9 @@
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers — reproducible builds and
# safe rollbacks (#563).
# Stage 1: Build frontend SPA
FROM node:22-slim AS frontend
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS frontend
WORKDIR /frontend
COPY frontend/package.json frontend/package-lock.json* ./
@@ -8,25 +12,31 @@ COPY frontend/ .
RUN npm run build
# Stage 2: Build Python package (NVIDIA CUDA 12.4)
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04 AS builder
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04@sha256:af8bd179ed3bf69d4b63b19a763662a6141f0f62ef099283f68d0b14b4bab0e3 AS builder
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip python3-venv && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY pyproject.toml README.md ./
# Install dependencies from the committed lockfile (#567). See deploy/docker/Dockerfile
# for the rationale behind the frozen export + --no-deps install.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt
COPY src/ src/
COPY scripts/install scripts/install
COPY deploy/windows deploy/windows
COPY --from=frontend /src/openjarvis/server/static src/openjarvis/server/static/
RUN pip install --no-cache-dir uv && \
uv pip install --system ".[server]"
RUN uv pip install --system --no-deps .
# Stage 3: Runtime
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04
FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04@sha256:af8bd179ed3bf69d4b63b19a763662a6141f0f62ef099283f68d0b14b4bab0e3
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip && \
@@ -36,6 +46,14 @@ COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
WORKDIR /app
# Run as an unprivileged user (#565). NVIDIA device nodes (/dev/nvidia*) are
# world-accessible, so GPU workloads do not require root.
RUN groupadd --system --gid 10001 openjarvis && \
useradd --system --uid 10001 --gid openjarvis \
--create-home --home-dir /home/openjarvis openjarvis
ENV HOME=/home/openjarvis
USER openjarvis
EXPOSE 8000
ENTRYPOINT ["jarvis"]
+28 -6
View File
@@ -1,5 +1,9 @@
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers — reproducible builds and
# safe rollbacks (#563).
# Stage 1: Build frontend SPA
FROM node:22-slim AS frontend
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS frontend
WORKDIR /frontend
COPY frontend/package.json frontend/package-lock.json* ./
@@ -8,25 +12,31 @@ COPY frontend/ .
RUN npm run build
# Stage 2: Build Python package (AMD ROCm 7.2)
FROM rocm/dev-ubuntu-22.04:7.2 AS builder
FROM rocm/dev-ubuntu-22.04:7.2@sha256:05af5f04a06b04676d4c7438997d0deadaeb7478961ad621376e199bf3aeb644 AS builder
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip python3-venv && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY pyproject.toml README.md ./
# Install dependencies from the committed lockfile (#567). See deploy/docker/Dockerfile
# for the rationale behind the frozen export + --no-deps install.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt
COPY src/ src/
COPY scripts/install scripts/install
COPY deploy/windows deploy/windows
COPY --from=frontend /src/openjarvis/server/static src/openjarvis/server/static/
RUN pip install --no-cache-dir uv && \
uv pip install --system ".[server]"
RUN uv pip install --system --no-deps .
# Stage 3: Runtime
FROM rocm/dev-ubuntu-22.04:7.2
FROM rocm/dev-ubuntu-22.04:7.2@sha256:05af5f04a06b04676d4c7438997d0deadaeb7478961ad621376e199bf3aeb644
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 python3-pip && \
@@ -36,6 +46,18 @@ COPY --from=builder /usr/local /usr/local
COPY --from=builder /app /app
WORKDIR /app
# Run as an unprivileged user (#565). ROCm GPU access is gated by the `video` and
# `render` groups (see group_add in docker-compose.gpu.rocm.yml), so the user is
# added to both; root is not required.
RUN groupadd --system --gid 10001 openjarvis && \
useradd --system --uid 10001 --gid openjarvis \
--create-home --home-dir /home/openjarvis openjarvis && \
(getent group video >/dev/null || groupadd --system video) && \
(getent group render >/dev/null || groupadd --system render) && \
usermod -aG video,render openjarvis
ENV HOME=/home/openjarvis
USER openjarvis
EXPOSE 8000
ENTRYPOINT ["jarvis"]
+34 -7
View File
@@ -1,15 +1,42 @@
FROM python:3.12-slim
# Base images are pinned to an immutable digest (in addition to a human-readable
# tag) so every build resolves the exact same layers (#563).
# Install Node.js 22
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates && \
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
apt-get install -y nodejs && \
# Node.js is sourced from the official, digest-pinned image rather than piping a
# remote setup script into bash (`curl ... | bash -`), which performed no
# checksum or signature verification of the downloaded installer (#566). The
# image digest is the integrity check, and the copy is architecture-agnostic.
FROM node:22.23.0-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS node
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf
# libstdc++6 + ca-certificates are the only runtime requirements of the Node
# binary copied below (the python slim image already provides libc/libgcc).
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates libstdc++6 && \
rm -rf /var/lib/apt/lists/*
# Transplant the Node.js runtime from the official image. Both images are Debian
# bookworm, so the glibc/libstdc++ ABI matches.
COPY --from=node /usr/local/bin/node /usr/local/bin/node
COPY --from=node /usr/local/lib/node_modules /usr/local/lib/node_modules
RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm && \
ln -sf /usr/local/lib/node_modules/npm/bin/npx-cli.js /usr/local/bin/npx
WORKDIR /app
# Install dependencies from the committed lockfile (#567): `uv export --frozen`
# reads uv.lock as-is and emits a pinned, hash-verified set installed with
# --no-deps (no re-resolution). Copied first so this layer caches independently
# of application source.
COPY pyproject.toml uv.lock README.md ./
RUN pip install --no-cache-dir uv && \
uv export --frozen --no-dev --extra server --no-emit-project > requirements.txt && \
uv pip install --system --no-deps -r requirements.txt
COPY . .
RUN pip install --no-cache-dir ".[server]"
# Install the project itself without re-resolving dependencies.
RUN uv pip install --system --no-deps .
LABEL openjarvis-sandbox=true
+3 -1
View File
@@ -18,7 +18,9 @@ services:
capabilities: [gpu]
ollama:
image: ollama/ollama:latest
# Pinned to a fixed version + digest for reproducible deployments (#563);
# must match the tag in docker-compose.yml.
image: ollama/ollama:0.30.10@sha256:bfc9c6d53cc6989aa5131a6fde6b162b2802d4d337657f3253b5f69579bddeee
environment:
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
+3 -1
View File
@@ -18,7 +18,9 @@ services:
restart: unless-stopped
ollama:
image: ollama/ollama:latest
# Pinned to a fixed version + digest for reproducible deployments and
# predictable rollbacks (#563). Bump deliberately, not implicitly via :latest.
image: ollama/ollama:0.30.10@sha256:bfc9c6d53cc6989aa5131a6fde6b162b2802d4d337657f3253b5f69579bddeee
ports:
- "11434:11434"
volumes:
+20
View File
@@ -14,7 +14,27 @@ Environment=HOME=/opt/openjarvis
# OPENJARVIS_API_KEY=<key> (generate one: `jarvis auth generate-key`)
# It is not prefixed with "-", so the unit fails to start if the file is
# missing — preventing an accidentally unauthenticated public server.
# Keep secrets here (mode 0600, owned by root) rather than inline Environment=
# lines, which leak into `systemctl show` and the journal.
EnvironmentFile=/etc/openjarvis/env
# --- Sandboxing / hardening (#564) ---
# Conservative set: tightens the unit without blocking the server's normal I/O
# or local GPU inference. ProtectSystem=strict makes the whole filesystem
# read-only except ReadWritePaths, so $HOME (config/cache/state under
# /opt/openjarvis) stays writable.
NoNewPrivileges=true
ProtectSystem=strict
ReadWritePaths=/opt/openjarvis
ProtectHome=true
PrivateTmp=true
ProtectControlGroups=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
RestrictRealtime=true
RestrictSUIDSGID=true
LockPersonality=true
[Install]
WantedBy=multi-user.target
+3 -3
View File
@@ -1,9 +1,9 @@
(function () {
"use strict";
var SUPABASE_URL = "https://mtbtgpwzrbostweaanpr.supabase.co";
var SUPABASE_ANON_KEY =
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Im10YnRncHd6cmJvc3R3ZWFhbnByIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NzMxODk0OTQsImV4cCI6MjA4ODc2NTQ5NH0._xMlqCfljtXpwPj54H-ghxfLFO-jiq4W2WhpU8vVL1c";
var SUPABASE_URL =
window.OPENJARVIS_SUPABASE_URL || "https://mtbtgpwzrbostweaanpr.supabase.co";
var SUPABASE_ANON_KEY = window.OPENJARVIS_SUPABASE_ANON_KEY || "";
var PAGE_SIZE = 50;
var allRows = [];
+1108 -1007
View File
File diff suppressed because it is too large Load Diff
+13
View File
@@ -24,9 +24,22 @@ serde_json = "1"
reqwest = { version = "0.12", features = ["json", "multipart"] }
tokio = { version = "1", features = ["full"] }
# Cloud API keys are stored in the OS credential store via `keyring`. keyring v3
# enables NO backend by default — without an explicit per-platform feature it
# silently falls back to a non-persistent in-memory mock, so keys would not
# survive an app restart. Each desktop target opts into its native store.
[target.'cfg(target_os = "macos")'.dependencies]
objc = "0.2"
dispatch = "0.2"
keyring = { version = "3", features = ["apple-native"] }
[target.'cfg(target_os = "windows")'.dependencies]
keyring = { version = "3", features = ["windows-native"] }
[target.'cfg(target_os = "linux")'.dependencies]
# Blocking Secret Service backend (no internal async runtime, so it is safe to
# call from the tokio-driven Tauri commands). Needs libdbus-1-dev at build time.
keyring = { version = "3", features = ["sync-secret-service", "crypto-rust"] }
[features]
default = ["custom-protocol"]
+162 -44
View File
@@ -1204,7 +1204,7 @@ async fn boot_backend(backend: SharedBackend, status: SharedStatus) {
// additions aren't accidentally stripped.
prepare_subprocess_for_appimage(&mut cmd);
// Inject cloud API keys from ~/.openjarvis/cloud-keys.env
// Inject cloud API keys from secure desktop storage.
for (key, value) in read_cloud_keys() {
cmd.env(&key, &value);
}
@@ -1720,17 +1720,111 @@ async fn submit_savings(
// Cloud API key management
// ---------------------------------------------------------------------------
/// Path to the cloud keys file (~/.openjarvis/cloud-keys.env).
fn cloud_keys_path() -> std::path::PathBuf {
const SECURE_KEY_SERVICE: &str = "OpenJarvis Cloud Keys";
const MANAGED_CLOUD_KEY_NAMES: &[&str] = &[
"OPENAI_API_KEY",
"ANTHROPIC_API_KEY",
"GEMINI_API_KEY",
"GOOGLE_API_KEY",
"OPENROUTER_API_KEY",
"MINIMAX_API_KEY",
"TAVILY_API_KEY",
];
/// Legacy path used by older desktop builds. New saves never write here.
fn legacy_cloud_keys_path() -> std::path::PathBuf {
let home = home_dir();
std::path::PathBuf::from(home)
.join(".openjarvis")
.join("cloud-keys.env")
}
/// Read cloud keys from disk and return as key=value pairs.
fn read_cloud_keys() -> Vec<(String, String)> {
let path = cloud_keys_path();
fn validate_cloud_key_name(key_name: &str) -> Result<(), String> {
let valid = !key_name.is_empty()
&& key_name.len() <= 128
&& key_name.ends_with("_API_KEY")
&& key_name
.chars()
.all(|ch| ch.is_ascii_uppercase() || ch.is_ascii_digit() || ch == '_');
if valid {
Ok(())
} else {
Err(format!("Invalid API key name: {}", key_name))
}
}
fn engine_api_key_name(engine: &str) -> String {
let normalized: String = engine
.chars()
.map(|ch| {
if ch.is_ascii_alphanumeric() {
ch.to_ascii_uppercase()
} else {
'_'
}
})
.collect();
let trimmed = normalized.trim_matches('_');
let engine_name = if trimmed.is_empty() {
CUSTOM_FALLBACK_ENGINE.to_ascii_uppercase()
} else {
trimmed.to_string()
};
format!("{}_API_KEY", engine_name)
}
fn managed_cloud_key_names() -> Vec<String> {
let mut names: Vec<String> = MANAGED_CLOUD_KEY_NAMES
.iter()
.map(|name| (*name).to_string())
.collect();
let cfg = read_inference_config();
if matches!(&cfg.kind, SourceKind::Custom) {
let engine = cfg.engine.unwrap_or_else(|| CUSTOM_FALLBACK_ENGINE.to_string());
let key_name = engine_api_key_name(&engine);
if validate_cloud_key_name(&key_name).is_ok() {
names.push(key_name);
}
}
names.sort();
names.dedup();
names
}
fn secure_store_get(key_name: &str) -> Result<Option<String>, String> {
validate_cloud_key_name(key_name)?;
let entry = keyring::Entry::new(SECURE_KEY_SERVICE, key_name)
.map_err(|err| format!("Failed to open secure key storage for {}: {}", key_name, err))?;
match entry.get_password() {
Ok(value) => Ok(Some(value)),
Err(keyring::Error::NoEntry) => Ok(None),
Err(err) => Err(format!("Failed to read {} from secure key storage: {}", key_name, err)),
}
}
fn secure_store_set(key_name: &str, key_value: &str) -> Result<(), String> {
validate_cloud_key_name(key_name)?;
let entry = keyring::Entry::new(SECURE_KEY_SERVICE, key_name)
.map_err(|err| format!("Failed to open secure key storage for {}: {}", key_name, err))?;
if key_value.is_empty() {
return match entry.delete_credential() {
Ok(()) => Ok(()),
Err(keyring::Error::NoEntry) => Ok(()),
Err(err) => Err(format!(
"Failed to remove {} from secure key storage: {}",
key_name, err
)),
};
}
entry
.set_password(key_value)
.map_err(|err| format!("Failed to save {} in secure key storage: {}", key_name, err))
}
fn read_legacy_cloud_keys() -> Vec<(String, String)> {
let path = legacy_cloud_keys_path();
let mut keys = Vec::new();
if let Ok(contents) = std::fs::read_to_string(&path) {
for line in contents.lines() {
@@ -1746,47 +1840,68 @@ fn read_cloud_keys() -> Vec<(String, String)> {
keys
}
/// Save a single cloud API key to the keys file.
#[tauri::command]
async fn save_cloud_key(key_name: String, key_value: String) -> Result<(), String> {
let path = cloud_keys_path();
// Ensure directory exists
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
fn migrate_legacy_cloud_keys() {
let path = legacy_cloud_keys_path();
if !path.exists() {
return;
}
// Read existing keys, update/add the one being saved
let mut keys: Vec<(String, String)> = read_cloud_keys()
let legacy_keys = read_legacy_cloud_keys();
if legacy_keys.is_empty() {
let _ = std::fs::remove_file(&path);
return;
}
let mut migrated_all = true;
for (key, value) in legacy_keys {
if value.is_empty() {
continue;
}
if secure_store_set(&key, &value).is_err() {
migrated_all = false;
}
}
if migrated_all {
let _ = std::fs::remove_file(path);
}
}
/// Read cloud keys from secure desktop storage and return key=value pairs.
fn read_cloud_keys() -> Vec<(String, String)> {
migrate_legacy_cloud_keys();
managed_cloud_key_names()
.into_iter()
.filter(|(k, _)| k != &key_name)
.collect();
if !key_value.is_empty() {
keys.push((key_name, key_value));
}
.filter_map(|key| match secure_store_get(&key) {
Ok(Some(value)) if !value.is_empty() => Some((key, value)),
_ => None,
})
.collect()
}
// Write back
let content: String = keys
.iter()
.map(|(k, v)| format!("{}={}", k, v))
.collect::<Vec<_>>()
.join("\n");
std::fs::write(&path, content + "\n").map_err(|e| format!("Failed to save key: {}", e))?;
// Set permissions to owner-only (chmod 600)
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600));
}
// Tell the running server to hot-reload its cloud engine so the user
// doesn't need to restart the app after entering an API key.
async fn reload_cloud_keys(keys: Vec<(String, String)>) {
let reload_url = format!("http://127.0.0.1:{}/v1/cloud/reload", JARVIS_PORT);
let key_map: serde_json::Map<String, serde_json::Value> = keys
.into_iter()
.map(|(key, value)| (key, serde_json::Value::String(value)))
.collect();
let _ = reqwest::Client::new()
.post(&reload_url)
.json(&serde_json::json!({ "keys": key_map }))
.timeout(std::time::Duration::from_secs(10))
.send()
.await;
}
/// Save a single cloud API key to secure desktop storage.
#[tauri::command]
async fn save_cloud_key(key_name: String, key_value: String) -> Result<(), String> {
let key_value = key_value.trim().to_string();
secure_store_set(&key_name, &key_value)?;
// Tell the running server to hot-reload its cloud engine so the user
// doesn't need to restart the app after entering an API key.
reload_cloud_keys(vec![(key_name, key_value)]).await;
Ok(())
}
@@ -1794,10 +1909,13 @@ async fn save_cloud_key(key_name: String, key_value: String) -> Result<(), Strin
/// Get which cloud providers have keys configured (without exposing values).
#[tauri::command]
async fn get_cloud_key_status() -> Result<serde_json::Value, String> {
let keys = read_cloud_keys();
let status: Vec<serde_json::Value> = keys
.iter()
.map(|(k, v)| serde_json::json!({ "key": k, "set": !v.is_empty() }))
migrate_legacy_cloud_keys();
let status: Vec<serde_json::Value> = managed_cloud_key_names()
.into_iter()
.map(|key| {
let set = matches!(secure_store_get(&key), Ok(Some(value)) if !value.is_empty());
serde_json::json!({ "key": key, "set": set })
})
.collect();
Ok(serde_json::json!(status))
}
@@ -1809,8 +1927,8 @@ async fn get_inference_source() -> Result<InferenceConfig, String> {
}
/// Persist the chosen inference source. `host` is normalized to a bare base
/// URL. For custom endpoints, an optional API key is stored in cloud-keys.env
/// under `<ENGINE>_API_KEY`. Applies on next app launch.
/// URL. For custom endpoints, an optional API key is stored in secure desktop
/// storage under `<ENGINE>_API_KEY`. Applies on next app launch.
#[tauri::command]
async fn set_inference_source(
kind: String,
@@ -1842,7 +1960,7 @@ async fn set_inference_source(
.engine
.clone()
.unwrap_or_else(|| CUSTOM_FALLBACK_ENGINE.to_string());
let key_name = format!("{}_API_KEY", engine.to_ascii_uppercase());
let key_name = engine_api_key_name(&engine);
// Save the key before persisting the config: if the key can't be
// written, surface it and DON'T record a custom source whose
// credential is missing (which would fail confusingly at runtime).
+74 -48
View File
@@ -1,7 +1,15 @@
import { useState, useRef, useEffect } from 'react';
import { useState, useRef, useEffect, useCallback } from 'react';
import { Search, Cpu, X, Download, Loader2, Trash2, Check, Cloud, Key, Eye, EyeOff } from 'lucide-react';
import { useAppStore } from '../lib/store';
import { pullModel, deleteModel, fetchModels, preloadModel, isTauri } from '../lib/api';
import {
pullModel,
deleteModel,
fetchModels,
preloadModel,
isTauri,
getCloudKeyStatus,
saveCloudKey,
} from '../lib/api';
/** Popular models that users can download from the catalogue. */
const CATALOGUE_MODELS = [
@@ -23,7 +31,6 @@ const CATALOGUE_MODELS = [
interface CloudProvider {
name: string;
envKey: string;
storageKey: string;
models: Array<{ id: string; desc: string }>;
}
@@ -31,7 +38,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'OpenAI',
envKey: 'OPENAI_API_KEY',
storageKey: 'openjarvis-openai-key',
models: [
{ id: 'gpt-4o', desc: 'GPT-4o — fast, multimodal' },
{ id: 'gpt-4o-mini', desc: 'GPT-4o Mini — cheap, fast' },
@@ -41,7 +47,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'Anthropic',
envKey: 'ANTHROPIC_API_KEY',
storageKey: 'openjarvis-anthropic-key',
models: [
{ id: 'claude-sonnet-4-6', desc: 'Claude Sonnet 4.6 — balanced' },
{ id: 'claude-opus-4-6', desc: 'Claude Opus 4.6 — most capable' },
@@ -51,7 +56,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'Google',
envKey: 'GEMINI_API_KEY',
storageKey: 'openjarvis-gemini-key',
models: [
{ id: 'gemini-2.5-pro', desc: 'Gemini 2.5 Pro — flagship' },
{ id: 'gemini-2.5-flash', desc: 'Gemini 2.5 Flash — fast' },
@@ -61,7 +65,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
{
name: 'OpenRouter',
envKey: 'OPENROUTER_API_KEY',
storageKey: 'openjarvis-openrouter-key',
models: [
{ id: 'openrouter/auto', desc: 'Auto — best model for the task' },
{ id: 'openrouter/anthropic/claude-sonnet-4', desc: 'Claude Sonnet 4 via OpenRouter' },
@@ -70,16 +73,6 @@ const CLOUD_PROVIDERS: CloudProvider[] = [
},
];
function getStoredKey(storageKey: string): string {
try { return localStorage.getItem(storageKey) || ''; } catch { return ''; }
}
function setStoredKey(storageKey: string, value: string): void {
try {
if (value) localStorage.setItem(storageKey, value);
else localStorage.removeItem(storageKey);
} catch {}
}
type Tab = 'installed' | 'catalogue' | 'cloud';
export function CommandPalette() {
@@ -92,11 +85,10 @@ export function CommandPalette() {
const [deleting, setDeleting] = useState<string | null>(null);
const [customModel, setCustomModel] = useState('');
const [showKeys, setShowKeys] = useState<Record<string, boolean>>({});
const [apiKeys, setApiKeys] = useState<Record<string, string>>(() => {
const keys: Record<string, string> = {};
for (const p of CLOUD_PROVIDERS) keys[p.storageKey] = getStoredKey(p.storageKey);
return keys;
});
const [apiKeys, setApiKeys] = useState<Record<string, string>>({});
const [cloudKeyStatus, setCloudKeyStatus] = useState<Record<string, boolean>>({});
const [cloudKeyError, setCloudKeyError] = useState<string | null>(null);
const [savingKey, setSavingKey] = useState<string | null>(null);
const inputRef = useRef<HTMLInputElement>(null);
const models = useAppStore((s) => s.models);
@@ -106,6 +98,20 @@ export function CommandPalette() {
const setCommandPaletteOpen = useAppStore((s) => s.setCommandPaletteOpen);
const installedIds = new Set(models.map((m) => m.id));
const desktopKeyStorage = isTauri();
const refreshCloudKeyStatus = useCallback(async () => {
if (!desktopKeyStorage) {
setCloudKeyStatus({});
return;
}
try {
setCloudKeyStatus(await getCloudKeyStatus());
setCloudKeyError(null);
} catch (e: any) {
setCloudKeyError(e?.message || 'Failed to read cloud key status');
}
}, [desktopKeyStorage]);
const filtered = tab === 'installed'
? (query
@@ -122,6 +128,10 @@ export function CommandPalette() {
inputRef.current?.focus();
}, []);
useEffect(() => {
void refreshCloudKeyStatus();
}, [refreshCloudKeyStatus]);
useEffect(() => {
setSelectedIdx(0);
}, [query, tab]);
@@ -210,24 +220,29 @@ export function CommandPalette() {
};
const handleSaveKey = async (provider: CloudProvider, value: string) => {
setStoredKey(provider.storageKey, value);
setApiKeys((prev) => ({ ...prev, [provider.storageKey]: value }));
const keyValue = value.trim();
setSavingKey(provider.envKey);
setCloudKeyError(null);
// Also save to Tauri backend so the server process picks up the key
if (isTauri()) {
try {
const { invoke } = await import('@tauri-apps/api/core');
await invoke('save_cloud_key', { keyName: provider.envKey, keyValue: value });
} catch {}
try {
await saveCloudKey(provider.envKey, keyValue);
setApiKeys((prev) => ({ ...prev, [provider.envKey]: '' }));
await refreshCloudKeyStatus();
useAppStore.getState().addLogEntry({
timestamp: Date.now(), level: 'info', category: 'model',
message: `${provider.name} API key ${keyValue ? 'saved' : 'removed'}. Refreshing model list...`,
});
await refreshModels();
} catch (e: any) {
setCloudKeyError(e?.message || `Failed to save ${provider.name} API key`);
} finally {
setSavingKey(null);
}
};
useAppStore.getState().addLogEntry({
timestamp: Date.now(), level: 'info', category: 'model',
message: `${provider.name} API key ${value ? 'saved' : 'removed'}. Refreshing model list…`,
});
// Refresh the model list so cloud models appear immediately.
await refreshModels();
const handleKeyBlur = (provider: CloudProvider) => {
const draft = apiKeys[provider.envKey] || '';
if (draft.trim()) void handleSaveKey(provider, draft);
};
const handleKeyDown = (e: React.KeyboardEvent) => {
@@ -323,6 +338,11 @@ export function CommandPalette() {
<Check size={12} /> Downloaded {pullSuccess} successfully
</div>
)}
{tab === 'cloud' && cloudKeyError && (
<div className="px-4 py-2 text-xs" style={{ color: 'var(--color-error)', background: 'rgba(220,38,38,0.05)' }}>
{cloudKeyError}
</div>
)}
{/* Results */}
<div className="max-h-[400px] overflow-y-auto py-2">
@@ -431,13 +451,17 @@ export function CommandPalette() {
/* ── Cloud Models tab ── */
<div className="px-4 py-2">
<div className="text-[11px] mb-3" style={{ color: 'var(--color-text-tertiary)' }}>
Add your API keys to use cloud models. Keys are stored locally on your device only.
{desktopKeyStorage
? 'Add your API keys to use cloud models. Keys are stored in secure desktop storage.'
: 'Configure cloud provider keys in the server environment to use cloud models.'}
</div>
{CLOUD_PROVIDERS.map((provider) => {
const key = apiKeys[provider.storageKey] || '';
const hasKey = !!key;
const isVisible = showKeys[provider.storageKey];
const key = apiKeys[provider.envKey] || '';
const hasSavedKey = !!cloudKeyStatus[provider.envKey];
const hasKey = hasSavedKey || !!key.trim();
const isVisible = showKeys[provider.envKey];
const isSaving = savingKey === provider.envKey;
return (
<div key={provider.name} className="mb-4">
@@ -458,26 +482,28 @@ export function CommandPalette() {
<input
type={isVisible ? 'text' : 'password'}
value={key}
onChange={(e) => setApiKeys((prev) => ({ ...prev, [provider.storageKey]: e.target.value }))}
onBlur={() => handleSaveKey(provider, apiKeys[provider.storageKey] || '')}
placeholder={`${provider.envKey}`}
onChange={(e) => setApiKeys((prev) => ({ ...prev, [provider.envKey]: e.target.value }))}
onBlur={() => handleKeyBlur(provider)}
placeholder={hasSavedKey ? 'Saved in secure storage' : provider.envKey}
disabled={!desktopKeyStorage || isSaving}
className="flex-1 text-xs px-2 py-1.5 bg-transparent outline-none font-mono"
style={{ color: 'var(--color-text)' }}
/>
<button
onClick={() => setShowKeys((prev) => ({ ...prev, [provider.storageKey]: !prev[provider.storageKey] }))}
onClick={() => setShowKeys((prev) => ({ ...prev, [provider.envKey]: !prev[provider.envKey] }))}
className="px-2 cursor-pointer" style={{ color: 'var(--color-text-tertiary)' }}
>
{isVisible ? <EyeOff size={12} /> : <Eye size={12} />}
</button>
</div>
{hasKey && (
{hasSavedKey && (
<button
onClick={() => handleSaveKey(provider, '')}
disabled={isSaving}
className="px-2 py-1 rounded-lg text-[10px] cursor-pointer"
style={{ color: 'var(--color-error)', border: '1px solid var(--color-error)' }}
style={{ color: 'var(--color-error)', border: '1px solid var(--color-error)', opacity: isSaving ? 0.5 : 1 }}
>
Remove
{isSaving ? 'Saving' : 'Remove'}
</button>
)}
</div>
@@ -1,6 +1,7 @@
import { useState, useEffect, useCallback } from 'react';
import type React from 'react';
import { invoke } from '@tauri-apps/api/core';
import { LEADERBOARD_ENABLED, SUPABASE_ANON_KEY, SUPABASE_URL } from '../../lib/supabase';
// ---------------------------------------------------------------------------
// Types
@@ -279,9 +280,6 @@ function getOrCreateAnonId(): string {
return id;
}
const SUPABASE_URL = 'https://mtbtgpwzrbostweaanpr.supabase.co';
const SUPABASE_KEY = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Im10YnRncHd6cmJvc3R3ZWFhbnByIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NzMxODk0OTQsImV4cCI6MjA4ODc2NTQ5NH0._xMlqCfljtXpwPj54H-ghxfLFO-jiq4W2WhpU8vVL1c';
const REFRESH_INTERVAL_MS = 5000;
export function SavingsDashboard({ apiUrl }: { apiUrl: string }) {
@@ -318,15 +316,16 @@ export function SavingsDashboard({ apiUrl }: { apiUrl: string }) {
return () => clearInterval(timer);
}, [fetchData]);
// Share savings to Supabase when opted in and data changes
// Share savings to Supabase when opted in and data changes. Skipped entirely
// when no anon key was built in (leaderboard disabled).
useEffect(() => {
if (!optInEnabled || !displayName || !data) return;
if (!LEADERBOARD_ENABLED || !optInEnabled || !displayName || !data) return;
const dollarSavings = data.per_provider.reduce((s, p) => s + p.total_cost, 0);
const energySaved = data.per_provider.reduce((s, p) => s + (p.energy_wh || 0), 0);
const flopsSaved = data.per_provider.reduce((s, p) => s + (p.flops || 0), 0);
invoke('submit_savings', {
supabaseUrl: SUPABASE_URL,
supabaseKey: SUPABASE_KEY,
supabaseKey: SUPABASE_ANON_KEY,
payload: {
anon_id: anonId,
display_name: displayName,
+4 -1
View File
@@ -1,4 +1,4 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
// Regression for #266: the frontend must send the local API key as a Bearer
// token on /v1 + /api requests, or `jarvis serve` with a key configured 401s
@@ -26,11 +26,14 @@ class MemoryStorage {
}
beforeEach(() => {
vi.resetModules();
vi.stubEnv('VITE_SUPABASE_ANON_KEY', 'test-anon-key');
(globalThis as unknown as { localStorage: MemoryStorage }).localStorage =
new MemoryStorage();
});
afterEach(() => {
vi.unstubAllEnvs();
(globalThis as unknown as { localStorage?: MemoryStorage }).localStorage =
undefined;
});
+27 -4
View File
@@ -1,12 +1,10 @@
import type { ModelInfo, SavingsData, ServerInfo } from '../types';
import { SUPABASE_ANON_KEY, SUPABASE_URL } from './supabase';
// ---------------------------------------------------------------------------
// Supabase config — safe to embed (RLS protects writes)
// Supabase config
// ---------------------------------------------------------------------------
const SUPABASE_URL = import.meta.env.VITE_SUPABASE_URL || 'https://mtbtgpwzrbostweaanpr.supabase.co';
const SUPABASE_ANON_KEY = import.meta.env.VITE_SUPABASE_ANON_KEY || 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Im10YnRncHd6cmJvc3R3ZWFhbnByIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NzMxODk0OTQsImV4cCI6MjA4ODc2NTQ5NH0._xMlqCfljtXpwPj54H-ghxfLFO-jiq4W2WhpU8vVL1c';
declare global {
interface Window {
__TAURI_INTERNALS__?: unknown;
@@ -15,6 +13,31 @@ declare global {
export const isTauri = () => typeof window !== 'undefined' && !!window.__TAURI_INTERNALS__;
export type CloudKeyStatus = Record<string, boolean>;
export async function getCloudKeyStatus(): Promise<CloudKeyStatus> {
if (!isTauri()) return {};
try {
const { invoke } = await import('@tauri-apps/api/core');
const rows = await invoke<Array<{ key: string; set: boolean }>>('get_cloud_key_status');
return Object.fromEntries(rows.map((row) => [row.key, row.set]));
} catch (e: any) {
throw new Error(e?.message ?? e ?? 'Failed to read cloud key status');
}
}
export async function saveCloudKey(keyName: string, keyValue: string): Promise<void> {
if (!isTauri()) {
throw new Error('Cloud API keys can be saved in the desktop app only.');
}
try {
const { invoke } = await import('@tauri-apps/api/core');
await invoke('save_cloud_key', { keyName, keyValue });
} catch (e: any) {
throw new Error(e?.message ?? e ?? 'Failed to save cloud key');
}
}
// Cached API base URL fetched from the Tauri backend at startup.
// This avoids hardcoding the port — the Rust backend is the single
// source of truth for JARVIS_PORT.
+11
View File
@@ -0,0 +1,11 @@
export const SUPABASE_URL =
import.meta.env.VITE_SUPABASE_URL || 'https://mtbtgpwzrbostweaanpr.supabase.co';
// The Supabase anon key is optional at build time. When it is unset the public
// savings leaderboard is disabled rather than failing the build — this keeps
// the `openjarvis` package and desktop app buildable without coupling
// publishability to a leaderboard credential. Set VITE_SUPABASE_ANON_KEY at
// build time (from a repo secret) to enable the leaderboard.
export const SUPABASE_ANON_KEY = import.meta.env.VITE_SUPABASE_ANON_KEY ?? '';
export const LEADERBOARD_ENABLED = SUPABASE_ANON_KEY.length > 0;
+115 -24
View File
@@ -19,9 +19,21 @@ import {
RefreshCw,
} from 'lucide-react';
import { useAppStore, type ThemeMode } from '../lib/store';
import { checkHealth, fetchSpeechHealth, getMemoryStats, getInferenceSource, setInferenceSource, type InferenceSource } from '../lib/api';
import {
checkHealth,
fetchSpeechHealth,
getMemoryStats,
getInferenceSource,
setInferenceSource,
getCloudKeyStatus,
saveCloudKey,
isTauri,
type InferenceSource,
} from '../lib/api';
import { isAutoUpdateDisabled, setAutoUpdateDisabled } from '../components/Desktop/UpdateChecker';
const CLOUD_KEY_STATUS_CHANGED = 'openjarvis-cloud-key-status-changed';
function OllamaModelList() {
const [models, setModels] = useState<Array<{ name: string; size: number }>>([]);
useEffect(() => {
@@ -44,32 +56,111 @@ function OllamaModelList() {
);
}
function ApiKeyInput({ storageKey, placeholder }: { storageKey: string; placeholder: string }) {
const [value, setValue] = useState(() => {
try { return localStorage.getItem(storageKey) || ''; } catch { return ''; }
});
function ApiKeyInput({ keyName, placeholder }: { keyName: string; placeholder: string }) {
const [value, setValue] = useState('');
const [saved, setSaved] = useState(false);
const save = (v: string) => {
setValue(v);
try { if (v) localStorage.setItem(storageKey, v); else localStorage.removeItem(storageKey); } catch {}
setSaved(true);
setTimeout(() => setSaved(false), 2000);
const [hasKey, setHasKey] = useState(false);
const [error, setError] = useState('');
const desktopKeyStorage = isTauri();
const refresh = useCallback(async () => {
if (!desktopKeyStorage) {
setHasKey(false);
return;
}
try {
const status = await getCloudKeyStatus();
setHasKey(!!status[keyName]);
} catch {
setHasKey(false);
}
}, [desktopKeyStorage, keyName]);
useEffect(() => {
void refresh();
window.addEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
return () => window.removeEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
}, [refresh]);
const save = async (v: string) => {
const next = v.trim();
if (!next) return;
setError('');
try {
await saveCloudKey(keyName, next);
setValue('');
setHasKey(true);
setSaved(true);
window.dispatchEvent(new Event(CLOUD_KEY_STATUS_CHANGED));
setTimeout(() => setSaved(false), 2000);
} catch (e: any) {
setError(e?.message || 'Failed to save API key');
}
};
const remove = async () => {
setError('');
try {
await saveCloudKey(keyName, '');
setValue('');
setHasKey(false);
setSaved(true);
window.dispatchEvent(new Event(CLOUD_KEY_STATUS_CHANGED));
setTimeout(() => setSaved(false), 2000);
} catch (e: any) {
setError(e?.message || 'Failed to remove API key');
}
};
return (
<div className="flex items-center gap-2">
<input type="password" value={value} onChange={e => save(e.target.value)} placeholder={placeholder}
<input
type="password"
value={value}
onChange={e => setValue(e.target.value)}
onBlur={() => { if (value.trim()) void save(value); }}
placeholder={hasKey ? 'Saved in secure storage' : placeholder}
disabled={!desktopKeyStorage}
className="w-48 px-2 py-1 rounded text-xs"
style={{ background: 'var(--color-bg)', border: '1px solid var(--color-border)', color: 'var(--color-text)' }} />
{hasKey && (
<button
onClick={() => void remove()}
className="px-2 py-1 rounded text-[10px] cursor-pointer"
style={{ color: 'var(--color-error)', border: '1px solid var(--color-error)' }}
>
Remove
</button>
)}
{saved && <span className="text-[10px]" style={{ color: 'var(--color-success)' }}>Saved</span>}
{error && <span className="text-[10px]" style={{ color: 'var(--color-error)' }}>{error}</span>}
</div>
);
}
function CloudProviderStatus({ label, storageKey }: { label: string; storageKey: string }) {
function CloudProviderStatus({ label, keyName }: { label: string; keyName: string }) {
const [hasKey, setHasKey] = useState(false);
const desktopKeyStorage = isTauri();
const refresh = useCallback(async () => {
if (!desktopKeyStorage) {
setHasKey(false);
return;
}
try {
const status = await getCloudKeyStatus();
setHasKey(!!status[keyName]);
} catch {
setHasKey(false);
}
}, [desktopKeyStorage, keyName]);
useEffect(() => {
try { setHasKey(!!localStorage.getItem(storageKey)); } catch { setHasKey(false); }
}, [storageKey]);
void refresh();
window.addEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
return () => window.removeEventListener(CLOUD_KEY_STATUS_CHANGED, refresh);
}, [refresh]);
return (
<span className="flex items-center gap-1 text-xs" style={{ color: 'var(--color-text-secondary)' }}>
<span style={{
@@ -424,10 +515,10 @@ export function SettingsPage() {
</div>
<SettingRow label="Cloud providers" description="Green dot means API key is configured">
<div className="flex flex-wrap gap-3">
<CloudProviderStatus label="OpenAI" storageKey="openjarvis-openai-key" />
<CloudProviderStatus label="Anthropic" storageKey="openjarvis-anthropic-key" />
<CloudProviderStatus label="Google" storageKey="openjarvis-gemini-key" />
<CloudProviderStatus label="OpenRouter" storageKey="openjarvis-openrouter-key" />
<CloudProviderStatus label="OpenAI" keyName="OPENAI_API_KEY" />
<CloudProviderStatus label="Anthropic" keyName="ANTHROPIC_API_KEY" />
<CloudProviderStatus label="Google" keyName="GEMINI_API_KEY" />
<CloudProviderStatus label="OpenRouter" keyName="OPENROUTER_API_KEY" />
</div>
</SettingRow>
</Section>
@@ -435,23 +526,23 @@ export function SettingsPage() {
{/* API Keys */}
<Section title="API Keys">
<SettingRow label="OpenAI" description="GPT-4, GPT-3.5, etc.">
<ApiKeyInput storageKey="openjarvis-openai-key" placeholder="sk-..." />
<ApiKeyInput keyName="OPENAI_API_KEY" placeholder="sk-..." />
</SettingRow>
<SettingRow label="Anthropic" description="Claude models">
<ApiKeyInput storageKey="openjarvis-anthropic-key" placeholder="sk-ant-..." />
<ApiKeyInput keyName="ANTHROPIC_API_KEY" placeholder="sk-ant-..." />
</SettingRow>
<SettingRow label="Google" description="Gemini models">
<ApiKeyInput storageKey="openjarvis-gemini-key" placeholder="AI..." />
<ApiKeyInput keyName="GEMINI_API_KEY" placeholder="AI..." />
</SettingRow>
<SettingRow label="OpenRouter" description="Multi-provider routing">
<ApiKeyInput storageKey="openjarvis-openrouter-key" placeholder="sk-or-..." />
<ApiKeyInput keyName="OPENROUTER_API_KEY" placeholder="sk-or-..." />
</SettingRow>
</Section>
{/* Tools */}
<Section title="Tools">
<SettingRow label="Web Search" description="SerpAPI or Tavily key for web search tool">
<ApiKeyInput storageKey="openjarvis-search-key" placeholder="API key..." />
<SettingRow label="Web Search" description="Tavily key for web search tool">
<ApiKeyInput keyName="TAVILY_API_KEY" placeholder="tvly-..." />
</SettingRow>
</Section>
+3 -1
View File
@@ -1,7 +1,9 @@
/// <reference types="vite/client" />
interface ImportMetaEnv {
readonly VITE_API_URL: string;
readonly VITE_API_URL?: string;
readonly VITE_SUPABASE_URL?: string;
readonly VITE_SUPABASE_ANON_KEY?: string;
}
interface ImportMeta {
+3
View File
@@ -4,6 +4,9 @@ import react from '@vitejs/plugin-react';
import tailwindcss from '@tailwindcss/vite';
import { VitePWA } from 'vite-plugin-pwa';
// VITE_SUPABASE_ANON_KEY is intentionally NOT required here: a missing key
// disables the savings leaderboard at runtime (see src/lib/supabase.ts) rather
// than failing the build, so the package/app stays publishable without it.
export default defineConfig({
resolve: {
alias: {
+1
View File
@@ -48,6 +48,7 @@ dev = [
"pytest>=8",
"pytest-asyncio>=0.24",
"pytest-cov>=5",
"pytest-xdist>=3",
"respx>=0.22",
"ruff>=0.4",
"pre-commit>=3.0",
+20
View File
@@ -179,6 +179,19 @@ def chat(
_notifications = NotificationDispatcher(get_status())
# Automatic long-term memory — extracts durable facts in the background.
memory_service = None
try:
from openjarvis.memory import build_memory_service
memory_service = build_memory_service(config, engine, model)
if memory_service is not None:
memory_service.start()
console.print("[dim] Memory: active[/dim]")
except Exception as exc:
console.print(f"[yellow]Memory service unavailable: {exc}[/yellow]")
memory_service = None
# Conversation state
if not system_prompt:
from openjarvis.prompt.builder import SystemPromptBuilder
@@ -266,10 +279,17 @@ def chat(
console.print()
console.print(Markdown(content))
console.print()
# Hand the exchange to the memory service (non-blocking).
if memory_service is not None:
memory_service.submit(user_input, content)
except KeyboardInterrupt:
console.print("\n[dim]Generation interrupted.[/dim]")
except Exception as exc:
console.print(f"\n[red]Error: {exc}[/red]\n")
if memory_service is not None:
memory_service.stop()
__all__ = ["chat"]
+60
View File
@@ -167,6 +167,66 @@ def search(
console.print(table)
def _get_fact_store():
"""Instantiate the automatic-memory fact store from config."""
from openjarvis.memory.store import create_fact_store
config = load_config()
mem = config.memory
return create_fact_store(
getattr(mem, "backend", "local"),
path=getattr(mem, "facts_path", "~/.openjarvis/memory_facts.jsonl"),
max_facts=getattr(mem, "max_facts", 1000),
)
@memory.command(name="list")
def list_facts() -> None:
"""List durable facts captured by the automatic memory service."""
console = Console()
store = _get_fact_store()
facts = store.list()
if not facts:
console.print("[yellow]No memory facts stored yet.[/yellow]")
return
table = Table(title=f"Memory Facts ({len(facts)})")
table.add_column("#", style="dim", width=4)
table.add_column("Fact")
table.add_column("Source", style="cyan")
for i, fact in enumerate(facts, 1):
table.add_row(str(i), fact.text, fact.source or "-")
console.print(table)
@memory.command()
@click.option(
"--yes",
"-y",
is_flag=True,
default=False,
help="Skip the confirmation prompt.",
)
def clear(yes: bool) -> None:
"""Remove all durable facts captured by the automatic memory service."""
console = Console()
store = _get_fact_store()
count = store.count()
if count == 0:
console.print("[yellow]No memory facts to clear.[/yellow]")
return
if not yes:
if not click.confirm(f"Remove all {count} stored memory fact(s)?"):
console.print("[dim]Aborted.[/dim]")
return
removed = store.clear()
console.print(f"[green]Cleared {removed} memory fact(s).[/green]")
@memory.command()
@click.option(
"--backend",
+14
View File
@@ -493,6 +493,19 @@ def serve(
except Exception as exc:
logger.debug("Memory backend init failed: %s", exc)
# Automatic long-term memory service (background fact extraction).
memory_service = None
try:
from openjarvis.memory import build_memory_service
memory_service = build_memory_service(config, engine, model_name)
if memory_service is not None:
memory_service.start()
console.print(" Memory svc: [cyan]active[/cyan]")
except Exception as exc:
logger.debug("Memory service init failed: %s", exc)
memory_service = None
# Set up agent manager
agent_manager = None
if config.agent_manager.enabled:
@@ -667,6 +680,7 @@ def serve(
channel_bridge=channel_bridge,
config=config,
memory_backend=memory_backend,
memory_service=memory_service,
speech_backend=speech_backend,
agent_manager=agent_manager,
agent_scheduler=agent_scheduler,
+24 -1
View File
@@ -910,7 +910,13 @@ class LearningConfig:
@dataclass(slots=True)
class StorageConfig:
"""Storage (memory) backend settings."""
"""Storage (memory) backend settings.
Covers both the retrieval/document store (``default_backend``, ``db_path``,
chunking, context injection) and the automatic long-term memory service
(``enabled``, ``backend``, ``extraction_model``, ``max_facts``,
``facts_path``) configured under ``[memory]`` in ``config.toml``.
"""
default_backend: str = "sqlite"
db_path: str = field(default_factory=lambda: str(get_config_dir() / "memory.db"))
@@ -920,6 +926,14 @@ class StorageConfig:
chunk_size: int = 512
chunk_overlap: int = 64
# Automatic memory service — extracts durable facts from conversations in
# the background and persists them across sessions (see openjarvis.memory).
enabled: bool = False # start the memory service with serve/chat
backend: str = "local" # fact-store backend ("local" = on-disk JSONL)
extraction_model: str = "" # model for fact extraction ("" = active model)
max_facts: int = 1000 # cap on stored facts (oldest evicted past the cap)
facts_path: str = str(DEFAULT_CONFIG_DIR / "memory_facts.jsonl")
# Backward-compatibility alias
MemoryConfig = StorageConfig
@@ -2003,6 +2017,15 @@ context_from_memory = true
[tools.storage]
default_backend = "sqlite"
# Automatic long-term memory: extracts durable facts from conversations in the
# background and persists them. Starts/stops with `jarvis serve` and
# `jarvis chat`; manage stored facts with `jarvis memory list` / `clear`.
[memory]
enabled = false # set true to enable the memory service
backend = "local" # fact-store backend (local = on-disk JSONL)
extraction_model = "" # model for fact extraction ("" = active model)
max_facts = 1000 # cap on stored facts
[tools.mcp]
enabled = true
+28
View File
@@ -0,0 +1,28 @@
"""Native persistent long-term memory for OpenJarvis.
This package provides the automatic memory service that extracts durable facts
from conversations in the background and persists them across sessions. It is
started and stopped as part of the ``jarvis serve`` / ``jarvis chat`` lifecycle
and configured via the ``[memory]`` section of ``config.toml``.
"""
from __future__ import annotations
from openjarvis.memory.extractor import FactExtractor
from openjarvis.memory.service import MemoryService, build_memory_service
from openjarvis.memory.store import (
Fact,
FactStore,
LocalFactStore,
create_fact_store,
)
__all__ = [
"Fact",
"FactStore",
"FactExtractor",
"LocalFactStore",
"MemoryService",
"build_memory_service",
"create_fact_store",
]
+150
View File
@@ -0,0 +1,150 @@
"""LLM-backed extraction of durable facts from a conversation turn.
The extractor takes a single (user, assistant) exchange and asks a small
local model to distill any long-term, user-specific facts worth remembering.
It is deliberately defensive: extraction runs on a background thread far from
the request path, so *any* failure — a dropped Ollama connection, a timeout, a
``BrokenPipeError`` when the client went away, or simply unparseable output —
must degrade to "no facts" rather than propagate.
"""
from __future__ import annotations
import json
import logging
import re
from typing import Any, List, Optional
from openjarvis.core.types import Message, Role
logger = logging.getLogger(__name__)
_DEFAULT_SYSTEM_PROMPT = (
"You extract durable, long-term facts about the user from a single "
"conversation exchange. A good fact is stable over time and useful in "
"future conversations: preferences, identity, goals, ongoing projects, "
"constraints, or relationships. Ignore one-off task details, small talk, "
"and anything the assistant said about itself.\n\n"
"Respond with ONLY a JSON array of short fact strings (each under 200 "
"characters). If there is nothing worth remembering, respond with []."
)
class FactExtractor:
"""Extract memory-worthy facts from a conversation turn via an engine."""
def __init__(
self,
engine: Any,
model: str,
*,
temperature: float = 0.0,
max_tokens: int = 512,
max_facts_per_turn: int = 10,
max_fact_chars: int = 200,
system_prompt: Optional[str] = None,
) -> None:
self._engine = engine
self._model = model
self._temperature = temperature
self._max_tokens = max_tokens
self._max_facts_per_turn = max_facts_per_turn
self._max_fact_chars = max_fact_chars
self._system_prompt = system_prompt or _DEFAULT_SYSTEM_PROMPT
def extract(self, user_text: str, assistant_text: str = "") -> List[str]:
"""Return durable facts from the exchange. Never raises."""
user_text = (user_text or "").strip()
if not user_text:
return []
exchange = f"User: {user_text}"
if assistant_text and assistant_text.strip():
exchange += f"\nAssistant: {assistant_text.strip()}"
messages = [
Message(role=Role.SYSTEM, content=self._system_prompt),
Message(role=Role.USER, content=exchange),
]
try:
result = self._engine.generate(
messages,
model=self._model,
temperature=self._temperature,
max_tokens=self._max_tokens,
)
except BrokenPipeError:
# The classic failure mode: the model call's transport died.
# Extraction is best-effort, so swallow it.
logger.debug("Memory extraction aborted: broken pipe", exc_info=True)
return []
except Exception: # noqa: BLE001 — extraction must never crash the worker
logger.debug("Memory extraction failed", exc_info=True)
return []
if isinstance(result, dict):
content = result.get("content", "") or ""
else:
content = str(result)
return self._parse(content)
# -- parsing ------------------------------------------------------------
def _parse(self, content: str) -> List[str]:
"""Parse model output into a clean, deduped, capped list of facts."""
if not content or not content.strip():
return []
raw = self._coerce_to_list(content)
facts: List[str] = []
seen: set[str] = set()
for item in raw:
fact = self._clean_fact(item)
if not fact:
continue
key = fact.lower()
if key in seen:
continue
seen.add(key)
facts.append(fact)
if len(facts) >= self._max_facts_per_turn:
break
return facts
def _coerce_to_list(self, content: str) -> List[str]:
"""Best-effort conversion of model output to a list of strings."""
# 1. Try to locate and parse a JSON array anywhere in the output
# (models often wrap it in prose or code fences).
match = re.search(r"\[.*\]", content, re.DOTALL)
if match:
try:
parsed = json.loads(match.group(0))
if isinstance(parsed, list):
return [str(x) for x in parsed]
except (json.JSONDecodeError, ValueError):
pass
# 2. Fall back to line-based parsing (markdown bullets / numbered).
items: List[str] = []
for line in content.splitlines():
line = line.strip()
if not line:
continue
line = re.sub(r"^\s*(?:[-*•]|\d+[.)])\s*", "", line)
items.append(line)
return items
def _clean_fact(self, item: str) -> str:
fact = str(item).strip().strip("\"'").strip()
# Drop obvious non-facts the model sometimes emits.
if not fact or fact.lower() in ("[]", "none", "n/a", "null"):
return ""
if len(fact) > self._max_fact_chars:
fact = fact[: self._max_fact_chars].rstrip()
return fact
__all__ = ["FactExtractor"]
+180
View File
@@ -0,0 +1,180 @@
"""Persistent memory service: async fact extraction integrated into core.
``MemoryService`` runs fact extraction on a dedicated background thread so it
never blocks ``jarvis serve`` request handling or the ``jarvis chat`` REPL.
Callers hand off an exchange via :meth:`submit`, which enqueues the work and
returns immediately — the slow model call and disk write happen out of band.
The worker swallows every per-job error (including ``BrokenPipeError`` when a
client disconnects mid-extraction), so a flaky extraction model can never take
down the host process.
The service is started and stopped as part of the OpenJarvis lifecycle (see
``cli/serve.py`` and ``cli/chat_cmd.py``) and is configured through the
``[memory]`` section of ``config.toml``.
"""
from __future__ import annotations
import logging
import queue
import threading
from typing import Any, List, Optional
from openjarvis.memory.extractor import FactExtractor
from openjarvis.memory.store import Fact, FactStore, create_fact_store
logger = logging.getLogger(__name__)
# Sentinel pushed onto the queue to wake the worker for shutdown.
_STOP = object()
class MemoryService:
"""Background long-term-memory extraction and persistence service."""
def __init__(
self,
store: FactStore,
extractor: FactExtractor,
*,
max_queue: int = 256,
) -> None:
self._store = store
self._extractor = extractor
self._queue: "queue.Queue[Any]" = queue.Queue(maxsize=max(1, max_queue))
self._thread: Optional[threading.Thread] = None
self._running = threading.Event()
# -- lifecycle ----------------------------------------------------------
def start(self) -> None:
"""Start the background worker thread (idempotent)."""
if self._running.is_set():
return
self._running.set()
self._thread = threading.Thread(
target=self._loop,
name="memory-service",
daemon=True,
)
self._thread.start()
logger.debug("Memory service started")
def stop(self, timeout: float = 2.0) -> None:
"""Signal the worker to drain and stop, then join it (idempotent)."""
if not self._running.is_set():
return
self._running.clear()
try:
self._queue.put_nowait(_STOP)
except queue.Full:
pass # worker will notice the cleared flag on its next loop
thread = self._thread
if thread is not None:
thread.join(timeout=timeout)
self._thread = None
logger.debug("Memory service stopped")
@property
def is_running(self) -> bool:
return self._running.is_set()
# -- submission ---------------------------------------------------------
def submit(self, user_text: str, assistant_text: str = "") -> bool:
"""Queue an exchange for extraction. Non-blocking; never raises.
Returns True if the job was enqueued, False if the service is not
running or the queue is full (in which case the exchange is dropped
rather than blocking the caller — extraction is best-effort).
"""
if not self._running.is_set():
return False
if not user_text or not user_text.strip():
return False
try:
self._queue.put_nowait((user_text, assistant_text))
return True
except queue.Full:
logger.debug("Memory service queue full; dropping exchange")
return False
# -- worker -------------------------------------------------------------
def _loop(self) -> None:
while True:
try:
job = self._queue.get(timeout=0.5)
except queue.Empty:
if not self._running.is_set():
break
continue
if job is _STOP:
self._queue.task_done()
break
try:
self._process(job)
except Exception: # noqa: BLE001 — a bad job must not kill the worker
logger.debug("Memory extraction job failed", exc_info=True)
finally:
self._queue.task_done()
if not self._running.is_set() and self._queue.empty():
break
def _process(self, job: Any) -> None:
user_text, assistant_text = job
facts = self._extractor.extract(user_text, assistant_text)
if facts:
stored = self._store.add_many(facts, source="auto")
if stored:
logger.debug("Memory service stored %d new fact(s)", stored)
# -- store passthroughs -------------------------------------------------
def list_facts(self) -> List[Fact]:
return self._store.list()
def clear_facts(self) -> int:
return self._store.clear()
def fact_count(self) -> int:
return self._store.count()
def build_memory_service(
config: Any,
engine: Any,
default_model: str = "",
) -> Optional[MemoryService]:
"""Build a :class:`MemoryService` from config, or ``None`` if disabled.
Reads the ``[memory]`` section (``config.memory`` / ``config.tools.storage``)
for ``enabled``, ``backend``, ``extraction_model``, ``max_facts`` and
``facts_path``. Returns ``None`` when memory is disabled or no engine /
extraction model is available, so callers can simply do::
svc = build_memory_service(config, engine, model)
if svc is not None:
svc.start()
"""
mem = getattr(config, "memory", None)
if mem is None or not getattr(mem, "enabled", False):
return None
if engine is None:
return None
model = getattr(mem, "extraction_model", "") or default_model
if not model:
logger.debug("Memory service disabled: no extraction model available")
return None
store = create_fact_store(
getattr(mem, "backend", "local"),
path=getattr(mem, "facts_path", "~/.openjarvis/memory_facts.jsonl"),
max_facts=getattr(mem, "max_facts", 1000),
)
extractor = FactExtractor(engine, model)
return MemoryService(store, extractor)
__all__ = ["MemoryService", "build_memory_service"]
+179
View File
@@ -0,0 +1,179 @@
"""Persistent stores for automatically extracted long-term memory facts.
A *fact* is a short, durable statement worth remembering about the user
(e.g. ``"Prefers concise answers"``). Facts are produced by the memory
service's background extractor and persisted here so they survive across
sessions. The store is intentionally small and self-contained: it dedupes,
caps the total number of facts, and is safe to call from multiple threads.
"""
from __future__ import annotations
import json
import os
import threading
import time
from abc import ABC, abstractmethod
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Iterable, List
@dataclass(slots=True)
class Fact:
"""A single durable memory entry."""
text: str
source: str = ""
created_at: float = 0.0
class FactStore(ABC):
"""Abstract persistent store for extracted memory facts."""
@abstractmethod
def add(self, text: str, source: str = "") -> bool:
"""Store *text* as a fact. Returns True if a new fact was stored."""
def add_many(self, texts: Iterable[str], source: str = "") -> int:
"""Store several facts, returning the count of newly stored ones."""
added = 0
for text in texts:
if self.add(text, source=source):
added += 1
return added
@abstractmethod
def list(self) -> List[Fact]:
"""Return all stored facts, oldest first."""
@abstractmethod
def clear(self) -> int:
"""Remove all stored facts, returning the number removed."""
@abstractmethod
def count(self) -> int:
"""Return the number of stored facts."""
class LocalFactStore(FactStore):
"""Append-only JSONL fact store on the local filesystem.
Facts are kept human-readable (one JSON object per line) so they can be
inspected or edited by hand. Writes are atomic (temp file + rename) and
guarded by a lock, so concurrent ``add`` calls from the extraction worker
and ``list``/``clear`` from the CLI never corrupt the file.
"""
def __init__(
self,
path: str | Path = "~/.openjarvis/memory_facts.jsonl",
*,
max_facts: int = 1000,
) -> None:
self._path = Path(path).expanduser()
self._max_facts = max(0, int(max_facts))
self._lock = threading.Lock()
self._facts: List[Fact] = self._load()
# -- persistence --------------------------------------------------------
def _load(self) -> List[Fact]:
if not self._path.exists():
return []
facts: List[Fact] = []
try:
text = self._path.read_text(encoding="utf-8")
except OSError:
return []
for line in text.splitlines():
line = line.strip()
if not line:
continue
try:
obj = json.loads(line)
except (json.JSONDecodeError, ValueError):
continue # skip malformed lines rather than crashing
fact_text = str(obj.get("text", "")).strip()
if not fact_text:
continue
facts.append(
Fact(
text=fact_text,
source=str(obj.get("source", "")),
created_at=float(obj.get("created_at", 0.0) or 0.0),
)
)
return facts
def _flush(self) -> None:
"""Atomically rewrite the JSONL file from the in-memory list."""
self._path.parent.mkdir(parents=True, exist_ok=True)
tmp = self._path.with_suffix(self._path.suffix + ".tmp")
payload = "".join(
json.dumps(asdict(f), ensure_ascii=False) + "\n" for f in self._facts
)
tmp.write_text(payload, encoding="utf-8")
os.replace(tmp, self._path)
# -- FactStore API ------------------------------------------------------
def add(self, text: str, source: str = "") -> bool:
text = (text or "").strip()
if not text:
return False
with self._lock:
lowered = text.lower()
if any(f.text.lower() == lowered for f in self._facts):
return False # dedupe
self._facts.append(Fact(text=text, source=source, created_at=time.time()))
# Enforce the cap by evicting the oldest entries.
if self._max_facts and len(self._facts) > self._max_facts:
self._facts = self._facts[-self._max_facts :]
self._flush()
return True
def list(self) -> List[Fact]:
with self._lock:
return list(self._facts)
def clear(self) -> int:
with self._lock:
removed = len(self._facts)
self._facts = []
if self._path.exists():
try:
self._path.unlink()
except OSError:
self._flush()
return removed
def count(self) -> int:
with self._lock:
return len(self._facts)
@property
def path(self) -> Path:
"""Filesystem location of the JSONL store."""
return self._path
def create_fact_store(
backend: str = "local",
*,
path: str | Path = "~/.openjarvis/memory_facts.jsonl",
max_facts: int = 1000,
) -> FactStore:
"""Construct a fact store for the configured *backend*.
Only the ``"local"`` (on-disk JSONL) backend is supported today; the
factory exists so additional backends can be added without changing the
service or CLI wiring.
"""
key = (backend or "local").strip().lower()
if key == "local":
return LocalFactStore(path, max_facts=max_facts)
raise ValueError(f"Unknown memory backend '{backend}'. Supported backends: local")
__all__ = ["Fact", "FactStore", "LocalFactStore", "create_fact_store"]
+14
View File
@@ -151,6 +151,7 @@ def create_app(
channel_bridge=None,
config=None,
memory_backend=None,
memory_service=None,
speech_backend=None,
agent_manager=None,
agent_scheduler=None,
@@ -221,6 +222,7 @@ def create_app(
app.state.channel_bridge = channel_bridge
app.state.config = config
app.state.memory_backend = memory_backend
app.state.memory_service = memory_service
app.state.speech_backend = speech_backend
app.state.agent_manager = agent_manager
app.state.agent_scheduler = agent_scheduler
@@ -292,6 +294,18 @@ def create_app(
except Exception as _exc:
logger.debug("Analytics init skipped: %s", _exc)
# Stop the background memory service cleanly when the server shuts down.
if memory_service is not None:
@app.on_event("shutdown")
async def _shutdown_memory_service() -> None:
svc = getattr(app.state, "memory_service", None)
if svc is not None:
try:
svc.stop()
except Exception:
pass
app.include_router(router)
app.include_router(dashboard_router)
app.include_router(comparison_router)
+4 -4
View File
@@ -1,8 +1,8 @@
"""Direct cloud API router — bypasses the engine system entirely.
Reads API keys from ~/.openjarvis/cloud-keys.env at request time so
it works even when the server was started without cloud keys in its
environment. Uses httpx directly so no cloud SDK packages are required.
Reads API keys from the process environment, with a legacy
~/.openjarvis/cloud-keys.env fallback for non-desktop/manual setups. Uses
httpx directly so no cloud SDK packages are required.
"""
from __future__ import annotations
@@ -38,7 +38,7 @@ _LOCAL_HF_ORGS = (
def _load_keys() -> dict[str, str]:
"""Read cloud-keys.env from disk every call so live updates are picked up."""
"""Read available cloud keys every call so live updates are picked up."""
keys: dict[str, str] = {}
# File first, then fall back to process environment
if _CLOUD_ENV_FILE.exists():
+62 -17
View File
@@ -223,7 +223,7 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
# the agent to execute them), add an explicit opt-in header rather
# than removing this guard — silent re-routing is what produced #414.
if agent is not None and not request_body.tools:
return _handle_agent(
response = _handle_agent(
agent,
model,
request_body,
@@ -231,16 +231,41 @@ async def chat_completions(request_body: ChatCompletionRequest, request: Request
trace_store=getattr(request.app.state, "trace_store", None),
bus=getattr(request.app.state, "bus", None),
)
else:
bus = getattr(request.app.state, "bus", None)
response = _handle_direct(
engine,
model,
request_body,
bus=bus,
complexity_info=complexity_info,
app_config=config,
)
bus = getattr(request.app.state, "bus", None)
return _handle_direct(
engine,
model,
request_body,
bus=bus,
complexity_info=complexity_info,
app_config=config,
# Hand the completed exchange to the background memory service.
_remember_exchange(
getattr(request.app.state, "memory_service", None),
query_text_for_complexity,
response,
)
return response
def _remember_exchange(memory_service, user_text: str, response) -> None:
"""Submit a completed exchange to the memory service (non-blocking)."""
if memory_service is None or not user_text:
return
try:
content = ""
choices = getattr(response, "choices", None)
if choices:
content = getattr(choices[0].message, "content", "") or ""
memory_service.submit(user_text, content)
except Exception: # noqa: BLE001 — memory is best-effort, never fail a reply
logging.getLogger("openjarvis.server").debug(
"Memory submit failed",
exc_info=True,
)
def _handle_direct(
@@ -823,14 +848,34 @@ async def reload_cloud_engine(request: Request):
"""
import os
# Re-read ~/.openjarvis/cloud-keys.env and update the running process env.
keys_path = get_config_dir() / "cloud-keys.env"
if keys_path.exists():
for raw_line in keys_path.read_text().splitlines():
line = raw_line.strip()
if line and not line.startswith("#") and "=" in line:
k, v = line.split("=", 1)
os.environ[k.strip()] = v.strip()
submitted_keys: dict[str, str] | None = None
try:
body = await request.json()
raw_keys = body.get("keys") if isinstance(body, dict) else None
if isinstance(raw_keys, dict):
submitted_keys = {
str(k): str(v)
for k, v in raw_keys.items()
if str(k).endswith("_API_KEY")
}
except Exception:
submitted_keys = None
if submitted_keys is not None:
for key, value in submitted_keys.items():
if value:
os.environ[key] = value
else:
os.environ.pop(key, None)
else:
# Compatibility fallback for non-desktop/manual configurations.
keys_path = get_config_dir() / "cloud-keys.env"
if keys_path.exists():
for raw_line in keys_path.read_text().splitlines():
line = raw_line.strip()
if line and not line.startswith("#") and "=" in line:
k, v = line.split("=", 1)
os.environ[k.strip()] = v.strip()
# Try to build a fresh CloudEngine.
try:
+48
View File
@@ -120,6 +120,54 @@ class TestChatAgents:
assert "simple ok" in result.output
assert "failed" not in result.output.lower()
def test_memory_service_started_fed_and_stopped(self) -> None:
"""The REPL starts the memory service, submits each turn, and stops it."""
class _SpyMemoryService:
def __init__(self) -> None:
self.started = False
self.stopped = False
self.submissions: list[tuple[str, str]] = []
def start(self) -> None:
self.started = True
def submit(self, user_text: str, assistant_text: str = "") -> bool:
self.submissions.append((user_text, assistant_text))
return True
def stop(self, timeout: float = 2.0) -> None:
self.stopped = True
spy = _SpyMemoryService()
engine = MagicMock()
engine.engine_id = "mock"
engine.generate.return_value = {"content": "engine fallback"}
config = JarvisConfig()
config.intelligence.default_model = "test-model"
AgentRegistry.register_value("simple_chat_agent", _SimpleChatAgent)
with (
patch("openjarvis.cli.chat_cmd.load_config", return_value=config),
patch("openjarvis.engine.get_engine", return_value=("mock", engine)),
patch("openjarvis.intelligence.register_builtin_models"),
patch(
"openjarvis.memory.build_memory_service",
return_value=spy,
),
):
result = CliRunner().invoke(
chat,
["--agent", "simple_chat_agent", "--model", "test-model"],
input="hello\n/quit\n",
)
assert result.exit_code == 0
assert spy.started is True
assert spy.stopped is True
assert spy.submissions == [("hello", "simple ok")]
def test_tool_agent_uses_legacy_agent_tools_and_prompts_confirmation(self) -> None:
engine = MagicMock()
engine.engine_id = "mock"
+65
View File
@@ -9,6 +9,7 @@ from click.testing import CliRunner
from openjarvis.cli import cli
from openjarvis.core.registry import MemoryRegistry
from openjarvis.memory.store import LocalFactStore
from openjarvis.tools.storage.sqlite import SQLiteMemory
@@ -108,3 +109,67 @@ def test_memory_stats_shows_count(tmp_path: Path, monkeypatch):
assert result.exit_code == 0
assert "2" in result.output
backend.close()
def _patch_fact_store(monkeypatch, tmp_path: Path) -> LocalFactStore:
"""Point ``jarvis memory list/clear`` at a temp fact store."""
mod = importlib.import_module("openjarvis.cli.memory_cmd")
store = LocalFactStore(tmp_path / "facts.jsonl")
monkeypatch.setattr(mod, "_get_fact_store", lambda: store)
return store
def test_memory_list_empty(tmp_path: Path, monkeypatch):
_patch_fact_store(monkeypatch, tmp_path)
result = CliRunner().invoke(cli, ["memory", "list"])
assert result.exit_code == 0
assert "No memory facts" in result.output
def test_memory_list_shows_facts(tmp_path: Path, monkeypatch):
store = _patch_fact_store(monkeypatch, tmp_path)
store.add("User prefers dark mode")
store.add("User lives in Berlin")
result = CliRunner().invoke(cli, ["memory", "list"])
assert result.exit_code == 0
assert "dark mode" in result.output
assert "Berlin" in result.output
def test_memory_clear_with_confirmation(tmp_path: Path, monkeypatch):
store = _patch_fact_store(monkeypatch, tmp_path)
store.add("fact one")
store.add("fact two")
result = CliRunner().invoke(cli, ["memory", "clear"], input="y\n")
assert result.exit_code == 0
assert "Cleared 2" in result.output
assert store.count() == 0
def test_memory_clear_aborted(tmp_path: Path, monkeypatch):
store = _patch_fact_store(monkeypatch, tmp_path)
store.add("keep me")
result = CliRunner().invoke(cli, ["memory", "clear"], input="n\n")
assert result.exit_code == 0
assert "Aborted" in result.output
assert store.count() == 1
def test_memory_clear_yes_flag(tmp_path: Path, monkeypatch):
store = _patch_fact_store(monkeypatch, tmp_path)
store.add("fact")
result = CliRunner().invoke(cli, ["memory", "clear", "--yes"])
assert result.exit_code == 0
assert "Cleared 1" in result.output
assert store.count() == 0
def test_memory_clear_empty(tmp_path: Path, monkeypatch):
_patch_fact_store(monkeypatch, tmp_path)
result = CliRunner().invoke(cli, ["memory", "clear"])
assert result.exit_code == 0
assert "No memory facts to clear" in result.output
+180 -4
View File
@@ -1,7 +1,14 @@
"""Tests for Docker and deployment files."""
"""Tests for Docker and deployment files.
These are static file-content checks (no Docker daemon required) so they run in
the default CI lane. They guard the deployment hardening from #228 and its
sub-issues (#563 image pinning, #564 systemd hardening, #565 non-root, #566
secure Node install, #567 frozen lockfile installs).
"""
from __future__ import annotations
import re
from pathlib import Path
try:
@@ -11,6 +18,35 @@ except ModuleNotFoundError: # pragma: no cover - Python < 3.11
ROOT = Path(__file__).resolve().parent.parent.parent
DOCKER_DIR = ROOT / "deploy" / "docker"
SYSTEMD_DIR = ROOT / "deploy" / "systemd"
def _dockerfiles() -> list[Path]:
return sorted(DOCKER_DIR.glob("Dockerfile*"))
def _compose_files() -> list[Path]:
return sorted(DOCKER_DIR.glob("docker-compose*.yml"))
def _from_lines(content: str) -> list[str]:
return [
ln.strip()
for ln in content.splitlines()
if ln.strip().upper().startswith("FROM ")
]
def _image_lines(content: str) -> list[str]:
"""`image: ...` lines from a compose file, ignoring comments."""
out = []
for ln in content.splitlines():
stripped = ln.strip()
if stripped.startswith("#"):
continue
if stripped.startswith("image:"):
out.append(stripped)
return out
class TestDockerFiles:
@@ -37,10 +73,12 @@ class TestDockerFiles:
]
non_src_includes = [s for s in force_include if not s.startswith("src/")]
install_marker = 'uv pip install --system ".[server]"'
# The project itself is built by the final `--no-deps .` install (#567);
# the force-includes must be present before that step.
install_marker = "uv pip install --system --no-deps ."
wheel_dockerfiles = [
p
for p in sorted(DOCKER_DIR.glob("Dockerfile*"))
for p in _dockerfiles()
if install_marker in p.read_text() and "COPY src/ src/" in p.read_text()
]
# Sanity: we actually found the wheel-building Dockerfiles to guard.
@@ -86,4 +124,142 @@ class TestDockerFiles:
assert "ollama:" in content
def test_systemd_service_exists(self):
assert (ROOT / "deploy" / "systemd" / "openjarvis.service").is_file()
assert (SYSTEMD_DIR / "openjarvis.service").is_file()
class TestImagePinning:
"""#563 — base images and ollama pinned to fixed versions + digests."""
def test_no_floating_latest_tag_in_image_directives(self):
# `:latest` only acceptable inside comments (rationale text).
for path in _dockerfiles() + _compose_files():
for ln in path.read_text().splitlines():
code = ln.split("#", 1)[0]
assert ":latest" not in code, f"{path.name}: floating :latest in {ln!r}"
def test_every_from_pins_a_digest(self):
for path in _dockerfiles():
froms = _from_lines(path.read_text())
assert froms, f"{path.name}: no FROM lines found"
for ln in froms:
assert "@sha256:" in ln, (
f"{path.name}: FROM is not digest-pinned: {ln!r}"
)
def test_ollama_image_pinned_with_version_and_digest(self):
for path in _compose_files():
for ln in _image_lines(path.read_text()):
if "ollama/ollama" in ln:
assert "@sha256:" in ln, f"{path.name}: ollama not digest-pinned"
# A concrete version tag (digits) must accompany the digest.
assert re.search(r"ollama/ollama:\d", ln), (
f"{path.name}: ollama missing a version tag"
)
class TestNonRootUser:
"""#565 — GPU images (and the base image) drop root."""
NON_ROOT = ["Dockerfile", "Dockerfile.gpu", "Dockerfile.gpu.rocm"]
def test_creates_and_switches_to_non_root_user(self):
for name in self.NON_ROOT:
content = (DOCKER_DIR / name).read_text()
assert "useradd" in content, f"{name}: no useradd"
# A USER directive switching away from root must exist and not be root.
user_lines = [
ln.strip()
for ln in content.splitlines()
if ln.strip().upper().startswith("USER ")
]
assert user_lines, f"{name}: no USER directive"
assert all("root" not in ln for ln in user_lines), (
f"{name}: USER directive still root"
)
def test_user_directive_is_last_stage(self):
# USER must appear after the final FROM so the runtime stage is non-root.
for name in self.NON_ROOT:
content = (DOCKER_DIR / name).read_text()
last_from = content.rfind("\nFROM ")
user_idx = content.rfind("\nUSER ")
assert user_idx > last_from, f"{name}: USER not in final runtime stage"
class TestSandboxNodeSecurity:
"""#566 — Node installed without an unverified curl|bash pipe."""
def test_no_curl_pipe_bash(self):
content = (DOCKER_DIR / "Dockerfile.sandbox").read_text()
for ln in content.splitlines():
code = ln.split("#", 1)[0] # ignore the explanatory comment
assert "| bash" not in code and "|bash" not in code, (
f"unverified pipe-to-bash install remains: {ln!r}"
)
assert "nodesource.com" not in code, "still using NodeSource setup script"
def test_node_sourced_from_pinned_official_image(self):
content = (DOCKER_DIR / "Dockerfile.sandbox").read_text()
assert "COPY --from=node" in content, "Node not copied from pinned image stage"
froms = _from_lines(content)
assert any("node:" in ln and "@sha256:" in ln for ln in froms), (
"no digest-pinned node base stage"
)
class TestFrozenInstall:
"""#567 — Docker builds install from the committed, frozen uv.lock."""
BUILD_DOCKERFILES = [
"Dockerfile",
"Dockerfile.gpu",
"Dockerfile.gpu.rocm",
"Dockerfile.sandbox",
]
def test_copies_lockfile(self):
for name in self.BUILD_DOCKERFILES:
content = (DOCKER_DIR / name).read_text()
assert "uv.lock" in content, f"{name}: uv.lock not referenced"
def test_uses_frozen_export(self):
for name in self.BUILD_DOCKERFILES:
content = (DOCKER_DIR / name).read_text()
assert "uv export --frozen" in content, f"{name}: no frozen export"
assert "--no-deps" in content, (
f"{name}: deps re-resolved (missing --no-deps)"
)
def test_no_unpinned_pyproject_resolution(self):
# The old `uv pip install --system ".[server]"` re-resolved deps on every
# build; it must not come back.
for name in self.BUILD_DOCKERFILES:
content = (DOCKER_DIR / name).read_text()
assert '".[server]"' not in content, (
f"{name}: still re-resolves from pyproject"
)
class TestSystemdHardening:
"""#564 — systemd unit ships secrets via EnvironmentFile and is sandboxed."""
def _service(self) -> str:
return (SYSTEMD_DIR / "openjarvis.service").read_text()
def test_environment_file_for_secrets(self):
assert "EnvironmentFile=" in self._service()
def test_core_hardening_directives_present(self):
content = self._service()
for directive in (
"NoNewPrivileges=true",
"ProtectSystem=strict",
"PrivateTmp=true",
):
assert directive in content, f"missing hardening directive: {directive}"
def test_writable_state_path_declared(self):
# ProtectSystem=strict makes the FS read-only; the working/home dir must
# be re-granted write access or the server cannot persist config/state.
content = self._service()
assert "ReadWritePaths=" in content
+105
View File
@@ -0,0 +1,105 @@
"""Tests for the LLM-backed fact extractor (openjarvis.memory.extractor)."""
from __future__ import annotations
from openjarvis.memory.extractor import FactExtractor
class FakeEngine:
"""Engine stub returning a canned completion (or raising)."""
def __init__(self, content="", *, raises=None):
self._content = content
self._raises = raises
self.calls = []
def generate(self, messages, *, model, temperature=0.7, max_tokens=1024, **kwargs):
self.calls.append((messages, model, temperature, max_tokens))
if self._raises is not None:
raise self._raises
return {"content": self._content}
def test_parses_json_array():
engine = FakeEngine('["User likes coffee", "User lives in Berlin"]')
extractor = FactExtractor(engine, "qwen3:14b")
facts = extractor.extract("I like coffee and live in Berlin", "Noted.")
assert facts == ["User likes coffee", "User lives in Berlin"]
def test_parses_json_array_wrapped_in_prose():
engine = FakeEngine('Sure! Here are the facts:\n["Fact A", "Fact B"]\nDone.')
extractor = FactExtractor(engine, "m")
assert extractor.extract("hi", "hello") == ["Fact A", "Fact B"]
def test_empty_array_returns_no_facts():
engine = FakeEngine("[]")
extractor = FactExtractor(engine, "m")
assert extractor.extract("just chatting", "ok") == []
def test_line_fallback_for_bullets():
engine = FakeEngine("- User is a teacher\n- User has two kids\n")
extractor = FactExtractor(engine, "m")
assert extractor.extract("about me", "noted") == [
"User is a teacher",
"User has two kids",
]
def test_dedupe_within_turn():
engine = FakeEngine('["likes tea", "Likes Tea", "likes tea"]')
extractor = FactExtractor(engine, "m")
assert extractor.extract("x", "y") == ["likes tea"]
def test_cap_facts_per_turn():
items = [f'"fact {i}"' for i in range(20)]
engine = FakeEngine("[" + ", ".join(items) + "]")
extractor = FactExtractor(engine, "m", max_facts_per_turn=3)
assert len(extractor.extract("x", "y")) == 3
def test_truncates_long_facts():
long_fact = "z" * 500
engine = FakeEngine(f'["{long_fact}"]')
extractor = FactExtractor(engine, "m", max_fact_chars=50)
facts = extractor.extract("x", "y")
assert len(facts) == 1
assert len(facts[0]) == 50
def test_empty_user_text_skips_engine():
engine = FakeEngine('["should not be called"]')
extractor = FactExtractor(engine, "m")
assert extractor.extract(" ", "y") == []
assert engine.calls == []
def test_broken_pipe_returns_empty():
engine = FakeEngine(raises=BrokenPipeError("client gone"))
extractor = FactExtractor(engine, "m")
# Must not raise — extraction is best-effort.
assert extractor.extract("hi", "hello") == []
def test_generic_exception_returns_empty():
engine = FakeEngine(raises=RuntimeError("ollama exploded"))
extractor = FactExtractor(engine, "m")
assert extractor.extract("hi", "hello") == []
def test_handles_non_dict_result():
class StrEngine:
def generate(self, *a, **k):
return '["plain string result"]'
extractor = FactExtractor(StrEngine(), "m")
assert extractor.extract("x", "y") == ["plain string result"]
def test_filters_non_fact_tokens():
engine = FakeEngine('["none", "N/A", "Real fact"]')
extractor = FactExtractor(engine, "m")
assert extractor.extract("x", "y") == ["Real fact"]
+112
View File
@@ -0,0 +1,112 @@
"""Tests for the persistent fact store (openjarvis.memory.store)."""
from __future__ import annotations
import json
import pytest
from openjarvis.memory.store import LocalFactStore, create_fact_store
def test_add_and_list(tmp_path):
store = LocalFactStore(tmp_path / "facts.jsonl")
assert store.add("User prefers concise answers") is True
assert store.add("User lives in Berlin") is True
facts = store.list()
assert [f.text for f in facts] == [
"User prefers concise answers",
"User lives in Berlin",
]
assert store.count() == 2
def test_add_dedupes_case_insensitive(tmp_path):
store = LocalFactStore(tmp_path / "facts.jsonl")
assert store.add("Likes coffee") is True
assert store.add("likes coffee") is False # duplicate
assert store.count() == 1
def test_add_skips_empty(tmp_path):
store = LocalFactStore(tmp_path / "facts.jsonl")
assert store.add("") is False
assert store.add(" ") is False
assert store.count() == 0
def test_add_many(tmp_path):
store = LocalFactStore(tmp_path / "facts.jsonl")
added = store.add_many(["a", "b", "a", "c"]) # one dupe
assert added == 3
assert store.count() == 3
def test_max_facts_evicts_oldest(tmp_path):
store = LocalFactStore(tmp_path / "facts.jsonl", max_facts=2)
store.add("first")
store.add("second")
store.add("third")
facts = [f.text for f in store.list()]
assert facts == ["second", "third"] # oldest dropped
def test_persistence_across_instances(tmp_path):
path = tmp_path / "facts.jsonl"
store = LocalFactStore(path)
store.add("durable fact")
reloaded = LocalFactStore(path)
assert [f.text for f in reloaded.list()] == ["durable fact"]
def test_clear(tmp_path):
path = tmp_path / "facts.jsonl"
store = LocalFactStore(path)
store.add("one")
store.add("two")
removed = store.clear()
assert removed == 2
assert store.count() == 0
# A fresh instance also sees an empty store.
assert LocalFactStore(path).count() == 0
def test_load_skips_malformed_lines(tmp_path):
path = tmp_path / "facts.jsonl"
path.write_text(
'{"text": "good fact"}\n'
"this is not json\n"
'{"text": ""}\n' # empty text ignored
'{"text": "another good"}\n',
encoding="utf-8",
)
store = LocalFactStore(path)
assert [f.text for f in store.list()] == ["good fact", "another good"]
def test_jsonl_round_trip_is_valid_json(tmp_path):
path = tmp_path / "facts.jsonl"
store = LocalFactStore(path)
store.add("fact one", source="auto")
lines = [
line for line in path.read_text(encoding="utf-8").splitlines() if line.strip()
]
assert len(lines) == 1
obj = json.loads(lines[0])
assert obj["text"] == "fact one"
assert obj["source"] == "auto"
assert "created_at" in obj
def test_create_fact_store_local(tmp_path):
store = create_fact_store("local", path=tmp_path / "f.jsonl", max_facts=5)
assert isinstance(store, LocalFactStore)
def test_create_fact_store_unknown_backend(tmp_path):
with pytest.raises(ValueError):
create_fact_store("cloud", path=tmp_path / "f.jsonl")
+171
View File
@@ -0,0 +1,171 @@
"""Tests for the background memory service (openjarvis.memory.service)."""
from __future__ import annotations
import threading
import time
from types import SimpleNamespace
from openjarvis.core.config import StorageConfig
from openjarvis.memory.service import MemoryService, build_memory_service
from openjarvis.memory.store import LocalFactStore
def _wait_until(predicate, timeout=2.0, interval=0.01):
deadline = time.time() + timeout
while time.time() < deadline:
if predicate():
return True
time.sleep(interval)
return predicate()
class FakeExtractor:
"""Extractor stub with controllable output, blocking and failures."""
def __init__(self, facts=None, *, raises=None, gate=None):
self._facts = facts or []
self._raises = raises
self._gate = gate # optional threading.Event to block on
self.calls = []
def extract(self, user_text, assistant_text=""):
self.calls.append((user_text, assistant_text))
if self._gate is not None:
self._gate.wait(timeout=2.0)
if self._raises is not None:
raise self._raises
return list(self._facts)
def _service(tmp_path, extractor, **kwargs):
store = LocalFactStore(tmp_path / "facts.jsonl")
return MemoryService(store, extractor, **kwargs)
def test_start_stop_lifecycle(tmp_path):
svc = _service(tmp_path, FakeExtractor())
assert svc.is_running is False
svc.start()
assert svc.is_running is True
svc.start() # idempotent
assert svc.is_running is True
svc.stop()
assert svc.is_running is False
svc.stop() # idempotent
def test_submit_extracts_and_stores(tmp_path):
extractor = FakeExtractor(["User likes hiking"])
svc = _service(tmp_path, extractor)
svc.start()
try:
assert svc.submit("I love hiking", "Nice!") is True
assert _wait_until(lambda: svc.fact_count() == 1)
assert [f.text for f in svc.list_facts()] == ["User likes hiking"]
finally:
svc.stop()
def test_submit_when_not_running_is_dropped(tmp_path):
extractor = FakeExtractor(["x"])
svc = _service(tmp_path, extractor)
assert svc.submit("hi", "there") is False
assert extractor.calls == []
def test_submit_empty_user_text_dropped(tmp_path):
extractor = FakeExtractor(["x"])
svc = _service(tmp_path, extractor)
svc.start()
try:
assert svc.submit(" ", "y") is False
finally:
svc.stop()
def test_worker_survives_extractor_broken_pipe(tmp_path):
"""A BrokenPipeError in one job must not kill the worker."""
extractor = FakeExtractor(raises=BrokenPipeError("client gone"))
svc = _service(tmp_path, extractor)
svc.start()
try:
svc.submit("first", "a")
assert _wait_until(lambda: len(extractor.calls) == 1)
# Service is still alive and accepting work.
assert svc.is_running is True
assert svc.submit("second", "b") is True
assert _wait_until(lambda: len(extractor.calls) == 2)
finally:
svc.stop()
def test_worker_survives_generic_exception(tmp_path):
extractor = FakeExtractor(raises=RuntimeError("boom"))
svc = _service(tmp_path, extractor)
svc.start()
try:
svc.submit("x", "y")
assert _wait_until(lambda: len(extractor.calls) == 1)
assert svc.is_running is True
finally:
svc.stop()
def test_submit_returns_false_when_queue_full(tmp_path):
"""Backpressure: a full queue drops work instead of blocking the caller."""
gate = threading.Event()
extractor = FakeExtractor(["fact"], gate=gate)
svc = _service(tmp_path, extractor, max_queue=1)
svc.start()
try:
# First submit is pulled by the worker and blocks on the gate.
assert svc.submit("job1", "a") is True
assert _wait_until(lambda: len(extractor.calls) == 1)
# Fill the (size-1) queue, then the next submit must be dropped.
assert svc.submit("job2", "b") is True
dropped = svc.submit("job3", "c")
assert dropped is False
finally:
gate.set()
svc.stop()
def test_build_memory_service_disabled_returns_none(tmp_path):
cfg = SimpleNamespace(memory=StorageConfig(enabled=False))
assert build_memory_service(cfg, object(), "model") is None
def test_build_memory_service_no_engine_returns_none(tmp_path):
cfg = SimpleNamespace(memory=StorageConfig(enabled=True))
assert build_memory_service(cfg, None, "model") is None
def test_build_memory_service_no_model_returns_none(tmp_path):
cfg = SimpleNamespace(memory=StorageConfig(enabled=True, extraction_model=""))
assert build_memory_service(cfg, object(), "") is None
def test_build_memory_service_enabled(tmp_path):
cfg = SimpleNamespace(
memory=StorageConfig(
enabled=True,
extraction_model="qwen3:14b",
facts_path=str(tmp_path / "facts.jsonl"),
max_facts=10,
)
)
svc = build_memory_service(cfg, object(), "fallback-model")
assert isinstance(svc, MemoryService)
def test_build_memory_service_falls_back_to_default_model(tmp_path):
cfg = SimpleNamespace(
memory=StorageConfig(
enabled=True,
extraction_model="",
facts_path=str(tmp_path / "facts.jsonl"),
)
)
svc = build_memory_service(cfg, object(), "active-model")
assert isinstance(svc, MemoryService)
+84 -4
View File
@@ -74,6 +74,68 @@ def client_with_agent():
# ---------------------------------------------------------------------------
class _SpyMemoryService:
"""Minimal stand-in capturing memory submissions."""
def __init__(self) -> None:
self.submissions: list[tuple[str, str]] = []
def submit(self, user_text: str, assistant_text: str = "") -> bool:
self.submissions.append((user_text, assistant_text))
return True
def stop(self, timeout: float = 2.0) -> None:
pass
class TestMemoryServiceWiring:
def test_non_streaming_completion_feeds_memory(self):
engine = _make_engine(content="remembered reply")
spy = _SpyMemoryService()
app = create_app(engine, "test-model", memory_service=spy)
client = TestClient(app)
resp = client.post(
"/v1/chat/completions",
json={
"model": "test-model",
"messages": [{"role": "user", "content": "I like jazz"}],
},
)
assert resp.status_code == 200
assert spy.submissions == [("I like jazz", "remembered reply")]
def test_agent_completion_feeds_memory(self):
engine = _make_engine()
agent = _make_agent(content="agent reply")
spy = _SpyMemoryService()
app = create_app(engine, "test-model", agent=agent, memory_service=spy)
client = TestClient(app)
resp = client.post(
"/v1/chat/completions",
json={
"model": "test-model",
"messages": [{"role": "user", "content": "remember this"}],
},
)
assert resp.status_code == 200
assert spy.submissions == [("remember this", "agent reply")]
def test_no_memory_service_is_noop(self):
engine = _make_engine()
app = create_app(engine, "test-model") # memory_service defaults to None
client = TestClient(app)
resp = client.post(
"/v1/chat/completions",
json={
"model": "test-model",
"messages": [{"role": "user", "content": "Hello"}],
},
)
assert resp.status_code == 200
class TestChatCompletions:
def test_basic_completion(self, client):
resp = client.post(
@@ -728,8 +790,25 @@ class TestCreateApp:
# ---------------------------------------------------------------------------
def _traces_enabled_config(tmp_path):
"""A config with traces explicitly enabled, isolated to *tmp_path*.
``create_app`` only builds a trace store when ``config.traces.enabled`` is
true (server/app.py). Relying on the ambient ``load_config()`` made these
tests fail on any machine whose ``~/.openjarvis/config.toml`` disables
traces; pinning an explicit config + tmp db keeps them hermetic and
parallel-safe under ``pytest -n auto``.
"""
from openjarvis.core.config import JarvisConfig
cfg = JarvisConfig()
cfg.traces.enabled = True
cfg.traces.db_path = str(tmp_path / "traces.db")
return cfg
class TestTraceRecording:
def test_agent_completion_creates_trace(self):
def test_agent_completion_creates_trace(self, tmp_path):
"""A non-streaming agent completion records exactly one trace.
The collector is the single writer: it saves directly and also
@@ -748,9 +827,10 @@ class TestTraceRecording:
"test-model",
agent=agent,
bus=EventBus(record_history=False),
config=_traces_enabled_config(tmp_path),
)
store = app.state.trace_store
assert store is not None, "traces enabled by default → store should exist"
assert store is not None, "traces explicitly enabled → store should exist"
assert store.count() == 0
client = TestClient(app)
@@ -769,10 +849,10 @@ class TestTraceRecording:
assert trace.query == "What is 2+2?"
assert trace.result == "traced reply"
def test_streaming_completion_creates_trace(self):
def test_streaming_completion_creates_trace(self, tmp_path):
"""A streamed completion (no agent) records the assembled response."""
engine = _make_engine()
app = create_app(engine, "test-model")
app = create_app(engine, "test-model", config=_traces_enabled_config(tmp_path))
store = app.state.trace_store
assert store is not None
assert store.count() == 0