Compare commits

...
Author SHA1 Message Date
dependabot[bot] 8494a55876 build(deps): bump actions/checkout from 6 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-25 08:13:27 +00:00
jaberjaber23 acf2587e46 bump v0.6.9 2026-05-12 21:42:08 +03:00
jaberjaber23 4583157b49 audit fixes 2026-05-12 21:13:41 +03:00
jaberjaber23 7185ea8808 cargo fmt 2026-05-12 20:57:57 +03:00
jaberjaber23 4aa1508f54 clippy lint 2026-05-12 20:43:56 +03:00
jaberjaber23 5447bf7f1d toctou staging 2026-05-12 17:11:36 +03:00
jaberjaber23 b77ebfb897 thread routing 2026-05-12 17:09:09 +03:00
jaberjaber23 2323cd5e67 audit hands 2026-05-12 17:06:43 +03:00
jaberjaber23 df29e5e8b9 audit endpoint 2026-05-12 17:05:42 +03:00
jaberjaber23 8b411c21c4 codex hardening 2026-05-12 16:38:50 +03:00
jaberjaber23 b00af5eddd add requesty 2026-05-12 16:35:22 +03:00
jaberjaber23 36177425c4 skill tools 2026-05-12 16:34:49 +03:00
jaberjaber23 6bed6c04ff workspace split 2026-05-12 16:33:49 +03:00
jaberjaber23 4c496be02f integration fixes 2026-05-12 16:13:08 +03:00
jaberjaber23 7f7b071528 provider refs 2026-05-12 15:53:15 +03:00
jaberjaber23 2d1fb8171c matrix refresh 2026-05-12 15:50:44 +03:00
Jaber Jaber e683acc565 Merge pull request #1045 from dongtran16092006/fix-mcp-system-prompt
fix: system prompt and identity handling, and config form hydration
2026-05-12 15:49:20 +03:00
jaberjaber23 5396889ff1 bedrock redacted 2026-05-12 15:48:54 +03:00
jaberjaber23 c9e8d31571 ws auth align 2026-05-12 15:47:38 +03:00
Jaber Jaber d8ad91572e Merge pull request #1054 from Hypn0sis/feat/discord-smart-thread
feat(discord): smart auto-thread mode (true/false/smart)
2026-05-12 15:47:05 +03:00
jaberjaber23 c27bfebd17 bump v0.6.7 2026-05-12 15:44:29 +03:00
jaberjaber23 f05ba5e42f tts image urls 2026-05-12 15:38:31 +03:00
jaberjaber23 d9e72abb4b uninstall agent 2026-05-12 15:37:44 +03:00
jaberjaber23 505a8e8080 hand stop 2026-05-12 15:37:02 +03:00
jaberjaber23 5cc865e6e6 shell env 2026-05-12 15:34:39 +03:00
jaberjaber23 6a1ce40d86 require signed 2026-05-12 15:34:01 +03:00
jaberjaber23 fbb7936234 docker docs 2026-05-12 15:30:40 +03:00
jaberjaber23 569e76c79a ws reconnect 2026-05-12 15:29:56 +03:00
jaberjaber23 88ad029999 bump v0.6.6 2026-05-12 15:27:09 +03:00
jaberjaber23 838836b29c integration fixes 2026-05-12 15:25:34 +03:00
jaberjaber23 8564872181 create directory 2026-05-12 15:12:16 +03:00
jaberjaber23 efbefa1682 chat agents 2026-05-12 15:10:59 +03:00
Jaber Jaber bdcd440cd6 Merge pull request #1143 from benhoverter/discord-file-sharing
fix(channels/discord): surface image attachments to text-only providers
2026-05-12 15:08:30 +03:00
Jaber Jaber 25516c7f87 Merge pull request #1168 from nimitbhardwaj/fix/latex-rendering
fix: render LaTeX math in chat messages
2026-05-12 15:08:15 +03:00
Jaber Jaber ae2706bdab Merge pull request #1176 from nimitbhardwaj/fix/new-line-chat
fix(chat): support Shift+Enter for multi-line input and proper newline display
2026-05-12 15:07:58 +03:00
Jaber Jaber 32299bb506 Merge pull request #1147 from benhoverter/harden-channel-id-binding
feat(channels): harden channel_id binding — adapter allowlist, strict validation, single source of truth for routing
2026-05-12 15:07:50 +03:00
Nimit Bhardwaj 5e228336e4 fix(chat): support Shift+Enter for multi-line input and proper newline display 2026-05-08 23:12:29 +05:30
Nimit Bhardwaj 5c1b1508a2 Fix Latex Rendering in Openfang Web 2026-05-07 00:24:36 +05:30
Ben Hoverter 701fcd8e2e channels/bridge: disable transparent decompression on image download
Discord's CDN edges occasionally advertise `content-encoding: gzip` (or
deflate/brotli) on PNG/JPEG passthroughs while the body is raw,
uncompressed image bytes. With the default `reqwest::Client::new()` and
the workspace's gzip/deflate/brotli features all enabled, reqwest's
transparent-decompression layer chokes on the PNG/JPEG header and
returns "error decoding response body" only on `bytes().await` (not on
`send()`), causing `download_image_to_blocks` to silently fall back to a
text-only block — the user's image never reaches the model.

Build the client explicitly with no_gzip/no_deflate/no_brotli so the
request advertises identity encoding and the body is read raw. Also set
a User-Agent (some CDN edges 403 clients without one) and a 30s timeout
aligned with the upstream 5 MB cap.

Repro: send an image attachment via Discord; the daemon logs
`Failed to read image bytes: error decoding response body` and the turn
appends as text-only with `appended_has_image=false`. After this fix the
PNG bytes are read and emitted as an Image content block as intended.
2026-05-04 12:04:31 -07:00
Ben Hoverter 118eacea64 channels: handle Discord image attachments coherently across providers
Discord MESSAGE_CREATE payloads with attachments were previously parsed
in a way that either dropped the attachment (when text was present, only
the text was kept) or dropped the whole message (when text was empty,
the early `content.is_empty()` return killed bare-image posts). The
result on text-only providers like claude-code: silent drops, then
hallucinated acknowledgements of content the model never saw.

This rewires the inbound path end-to-end:

* types: add ChannelContent::Multipart(Vec<ChannelContent>) so a single
  inbound message can carry a caption + one or more attachments as
  sibling blocks. Doc forbids nesting; consumers debug_assert.

* discord: classify attachments by MIME (with extension fallback for
  bot-relayed payloads that omit content_type) and a 5 MB vision-size
  cap matching Anthropic's image block limit. Vision-eligible images
  become ChannelContent::Image; everything else becomes File. Emit
  Multipart whenever text and attachments coexist, or when there are
  multiple attachments.

* bridge: flat-map Multipart in both dispatch paths — into Vec<ContentBlock>
  for multimodal-capable providers, and into a newline-joined text
  descriptor for text-flatten providers.

* telegram: add the Multipart arm to send_to_user for exhaustive-match
  parity; flattens defensively.

* claude_code driver: render Image blocks as
  "[attachment: <mime> image, ~N KB — not viewable on this provider]"
  instead of dropping them. The model still cannot see the image, but
  it can acknowledge it coherently rather than confabulating.

Adds 9 discord parser tests covering all (text, attachment-count) shapes
plus MIME edge cases, and 2 claude_code driver tests covering captioned
and bare-image rendering.
2026-05-02 15:16:28 -07:00
Ben Hoverter aaad1fdf32 discord: log raw MESSAGE_CREATE/UPDATE payloads at debug
Adds a single tracing::debug! at the top of parse_discord_message that
dumps the full payload JSON. Silent at default `info` level; enable with
`RUST_LOG=openfang_channels::discord=debug` to capture real attachment
JSON when developing the file-passing parse code.

Logs before any filters (bot, allowed_users, allowed_guilds, empty
content) so attachment-only messages are visible too.
2026-05-02 15:16:27 -07:00
Ben Hoverter dd8c53026e channels/bridge: support file:// URLs in download_image_to_blocks
Pick 3a-bis of the Discord file-passing plan: teach the multimodal
image fetcher to handle file:// URLs by reading from local disk
instead of going through reqwest. PR-A (Discord inbound) will
materialize attachments to a shared inbox dir and emit
ChannelContent::Image { url: "file://..." }, so this branch is what
unblocks vision on inbox-materialized images after the Discord CDN
URL has expired.

Implementation:
- Branch on url.strip_prefix("file://"); local read uses tokio::fs::read.
- HTTP path unchanged. Both paths converge on (Vec<u8>, Option<String>)
  before the existing 5MB cap, magic-byte sniffing, and base64 path.
- No content-type header on file:// — magic-byte detection and URL
  extension fallback do all the media-type work, which is fine since
  detect_image_magic and media_type_from_url already exist.
- No new deps. Vec<u8> instead of bytes::Bytes to avoid pulling in
  the bytes crate as a direct dep.
- No URL percent-decoding: the inbox writer (PR-A) controls filenames
  and avoids characters that would need encoding.

Refs: projects/openfang-fork/discord-file-passing-plan.md (step 2)
2026-05-02 15:16:27 -07:00
Ben Hoverter 218f2dba1f channels: add mime and size to ChannelContent::File
Pick 3a of the Discord file-passing plan: extend the URL-flavored File
variant with optional mime and size metadata so adapters can pass
attachment context through to bridges. FileData (bytes-flavored) is
unchanged; size is implicit in data.len() and mime_type already exists.

Match-arm sites in bridge.rs, telegram.rs, whatsapp.rs use `..` to stay
forward-compatible. Construction sites in telegram.rs and kernel.rs
pass `mime: None, size: None` for now; Discord inbound (PR-A) will
populate them.

Refs: projects/openfang-fork/discord-file-passing-plan.md
2026-05-02 15:16:27 -07:00
Ben HoverterandClaude Opus 4.7 9130811433 docs(types): drop internal spec reference from KernelConfig comment
Replaces "Spec §5.5 scoped strict-field validation to bindings" with
self-contained wording. The §5.5 reference points to an internal-fork
spec document that means nothing to upstream readers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 17:49:28 -07:00
Ben Hoverter faf2cf9211 feat(channels): harden channel_id binding (extends d336314)
Layers richer config validation, an explicit adapter allowlist, and a
stricter bridge routing path on top of upstream `d336314` ("binding
rule"), which shipped the same `channel_id` field as our PR #1127 in a
parallel implementation. Replaces upstream's `sender_user_id`/
`platform_id` heuristic with a single source of truth shared between
config validation and routing.

## What changes vs upstream `d336314`

**Data model** (`openfang-types/src/config.rs`)
- `#[serde(deny_unknown_fields)]` on `AgentBinding` so a typo at the
  binding level (e.g. `match_rules` plural) fails loudly instead of
  silently leaving the rule defaulted to "match everything". Upstream
  has it on `BindingMatchRule` only.
- New `pub const CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL` (19 adapters:
  discord, slack, telegram, matrix, mattermost, teams, webex,
  rocketchat, nextcloud, pumble, revolt, guilded, feishu, lark,
  keybase, google_chat, line, twist, flock, twitch). Single source of
  truth shared with the bridge — no drift between routing and
  validation paths possible. Hybrid adapters (IRC, Zulip) are
  excluded; see source comment.
- Startup validation: warn when a binding sets `channel_id` for a
  non-supporting adapter, or when `channel_id` is set without
  `channel`. Documents the metadata escape hatch in the warning.
- Top-level `KernelConfig` keeps no `deny_unknown_fields` — comment
  explains the §5.5 scoping decision so a future reader doesn't
  "tighten" it without realizing it would break forward-compat keys.

**Bridge** (`openfang-channels/src/bridge.rs`)
- Replaces upstream's `sender_channel_id()` heuristic ("if metadata has
  `sender_user_id` and it differs from `platform_id`, assume
  `platform_id` IS the channel") with `binding_context_for(message)`,
  which delegates to `ChannelMessage::channel_id()`. The heuristic
  worked for Discord/Slack but would fail silently on Matrix, Teams,
  Mattermost, Telegram, etc. — adapters whose `platform_id` IS the
  channel ID but whose metadata does not happen to set
  `sender_user_id` differently.
- Routes both dispatch paths (text + blocks) through
  `resolve_with_context` so `guild_id` and `channel_id` bindings can
  match. (Upstream's `resolve_with_channel_id` only handled
  channel_id.)

**Channels types** (`openfang-channels/src/types.rs`)
- New `ChannelMessage::channel_id()` accessor: reads `platform_id` for
  allowlisted adapters, falls back to `metadata["channel_id"]` for
  opt-in adapters, else `None`. Case-folds `Custom(...)` variants so
  a stray `Custom("Twitch")` cannot silently slip past the allowlist
  (the validation path lowercases user input — accessor must match).

**Tests** (+8 in `bridge.rs`, +5 in `config.rs`, +1 in `types.rs`)
- Bridge: Discord/Telegram/Matrix/custom-supported/user-id-only-adapter
  /metadata-fallback/guild-id-from-metadata/Email-returns-None
  coverage of `binding_context_for` and `channel_id()`.
- Config: typo rejection on both `BindingMatchRule` and `AgentBinding`;
  channel_id-without-channel warning; unsupported-adapter warning;
  no-warning for discord/slack/telegram.
- Types: `channel_id()` case-insensitivity for Custom variants
  including the Lark/Feishu Intl spelling.

**Docs** (`docs/channel-adapters.md`)
- Routing section rewritten: bindings are step 1 in the resolution
  order. New "Bindings" subsection documents the rule shape, the
  `peer_id` vs `channel_id` distinction (the easy confusion), full
  specificity table, the adapter allowlist with the metadata escape
  hatch, and the strict-field parsing rule.

## Why this layering instead of replacing d336314

Upstream's commit and our PR #1127 are functionally equivalent on
Discord and Slack. Shipping a richer extension on top is less churn
than ripping out the upstream commit and substituting ours, and keeps
the API surface upstream just added (`resolve_with_channel_id`)
intact for any third-party consumers.

`cargo check --workspace` and `cargo test -p openfang-types -p
openfang-channels --lib` (850 tests) pass.
2026-04-30 17:10:16 -07:00
dongtran16092006 93b57bdd52 fix: system prompt and identity handling, and config form hydration #1045 2026-04-20 18:10:16 +07:00
Matteo De Agazio 0227ff1790 fix(discord): cap dedup set on thread delete, add #[cfg(test)] to test mod
- Replace retain(|_| true) no-op with a size-capped clear: when
  threaded_message_ids exceeds MAX_DEDUP_MSG_IDS (2000) it is cleared.
  MESSAGE_UPDATE embed events arrive within seconds so old entries are
  always safe to discard; prevents unbounded growth on busy servers.
- Add #[cfg(test)] to mod tests so empty_threads() helper is only
  compiled in test mode — removes the need for #[allow(dead_code)].
2026-04-20 09:31:09 +02:00
Matteo De Agazio 525d7d844a feat(discord): smart auto-thread mode (true/false/smart) 2026-04-20 09:31:09 +02:00
62 changed files with 6726 additions and 598 deletions
+6 -6
View File
@@ -20,7 +20,7 @@ jobs:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
@@ -45,7 +45,7 @@ jobs:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
@@ -67,7 +67,7 @@ jobs:
name: Clippy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
@@ -87,7 +87,7 @@ jobs:
name: Format
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
@@ -99,7 +99,7 @@ jobs:
name: Security Audit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Install cargo-audit
@@ -111,7 +111,7 @@ jobs:
name: Secrets Scan
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Install trufflehog
+3 -3
View File
@@ -49,7 +49,7 @@ jobs:
runs-on: ${{ matrix.platform.os }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install system deps (Linux)
if: runner.os == 'Linux'
@@ -165,7 +165,7 @@ jobs:
archive: zip
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
@@ -215,7 +215,7 @@ jobs:
name: Docker Image
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Log in to GHCR
uses: docker/login-action@v4
with:
Generated
+95 -90
View File
@@ -139,7 +139,7 @@ version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -150,7 +150,7 @@ checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -893,7 +893,7 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -1029,27 +1029,27 @@ dependencies = [
[[package]]
name = "cranelift-assembler-x64"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "046d4b584c3bb9b5eb500c8f29549bec36be11000f1ba2a927cef3d1a9875691"
checksum = "adc822414b18d1f5b1b33ce1441534e311e62fef86ebb5b9d382af857d0272c9"
dependencies = [
"cranelift-assembler-x64-meta",
]
[[package]]
name = "cranelift-assembler-x64-meta"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9b194a7870becb1490366fc0ae392ccd188065ff35f8391e77ac659db6fb977"
checksum = "8c646808b06f4532478d8d6057d74f15c3322f10d995d9486e7dcea405bf521a"
dependencies = [
"cranelift-srcgen",
]
[[package]]
name = "cranelift-bforest"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb6a4ab44c6b371e661846b97dab687387a60ac4e2f864e2d4257284aad9e889"
checksum = "7b5996f01a686b2349cdb379083ec5ad3e8cb8767fb2d495d3a4f2ee4163a18d"
dependencies = [
"cranelift-entity",
"wasmtime-internal-core",
@@ -1057,9 +1057,9 @@ dependencies = [
[[package]]
name = "cranelift-bitset"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8b7a44150c2f471a94023482bda1902710746e4bed9f9973d60c5a94319b06d"
checksum = "523fea83273f6a985520f57788809a4de2165794d9ab00fb1254fceb4f5aa00c"
dependencies = [
"serde",
"serde_derive",
@@ -1068,9 +1068,9 @@ dependencies = [
[[package]]
name = "cranelift-codegen"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01b06598133b1dd76758b8b95f8d6747c124124aade50cea96a3d88b962da9fa"
checksum = "d73d1e372730b5f64ed1a2bd9f01fe4686c8ec14a28034e3084e530c8d951878"
dependencies = [
"bumpalo",
"cranelift-assembler-x64",
@@ -1096,9 +1096,9 @@ dependencies = [
[[package]]
name = "cranelift-codegen-meta"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6190e2e7bcf0a678da2f715363d34ed530fedf7a2f0ab75edaefef72a70465ff"
checksum = "b0319c18165e93dc1ebf78946a8da0b1c341c95b4a39729a69574671639bdb5f"
dependencies = [
"cranelift-assembler-x64-meta",
"cranelift-codegen-shared",
@@ -1109,24 +1109,24 @@ dependencies = [
[[package]]
name = "cranelift-codegen-shared"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f583cf203d1aa8b79560e3b01f929bdacf9070b015eec4ea9c46e22a3f83e4a0"
checksum = "9195cd8aeecb55e401aa96b2eaa55921636e8246c127ed7908f7ef7e0d40f270"
[[package]]
name = "cranelift-control"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "803159df35cc398ae54473c150b16d6c77e92ab2948be638488de126a3328fbc"
checksum = "8976c2154b74136322befc74222ab5c7249edd7e2604f8cbef2b94975541ffb9"
dependencies = [
"arbitrary",
]
[[package]]
name = "cranelift-entity"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3109e417257082d88087f5bcce677525bdaa8322b88dd7f175ed1a1fd41d546c"
checksum = "6038b3147c7982f4951150d5f96c7c06c1e7214b99d4b4a98607aadf8ded89d1"
dependencies = [
"cranelift-bitset",
"serde",
@@ -1136,9 +1136,9 @@ dependencies = [
[[package]]
name = "cranelift-frontend"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "14db6b0e0e4994c581092df78d837be2072578f7cb2528f96a6cf895e56dee63"
checksum = "4cbd294abe236e23cc3d907b0936226b6a8342db7636daa9c7c72be1e323420e"
dependencies = [
"cranelift-codegen",
"log",
@@ -1148,15 +1148,15 @@ dependencies = [
[[package]]
name = "cranelift-isle"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec66ea5025c7317383699778282ac98741d68444f956e3b1d7b62f12b7216e67"
checksum = "b5a90b6ed3aba84189352a87badeb93b2126d3724225a42dc67fdce53d1b139c"
[[package]]
name = "cranelift-native"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "373ade56438e6232619d85678477d0a88a31b3581936e0503e61e96b546b0800"
checksum = "c3ec0cc1a54e22925eacf4fc3dc815f907734d3b377899d19d52bec04863e853"
dependencies = [
"cranelift-codegen",
"libc",
@@ -1165,9 +1165,9 @@ dependencies = [
[[package]]
name = "cranelift-srcgen"
version = "0.130.1"
version = "0.130.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef53619d3cd5c78fd998c6d9420547af26b72e6456f94c2a8a2334cb76b42baa"
checksum = "948865622f87f30907bb46fbb081b235ae63c1896a99a83c26a003305c1fa82d"
[[package]]
name = "crc32fast"
@@ -1555,7 +1555,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users 0.5.2",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -1805,7 +1805,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -3739,7 +3739,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -3959,7 +3959,7 @@ dependencies = [
[[package]]
name = "openfang-api"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"argon2",
"async-trait",
@@ -4001,7 +4001,7 @@ dependencies = [
[[package]]
name = "openfang-channels"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"aes",
"async-trait",
@@ -4040,7 +4040,7 @@ dependencies = [
[[package]]
name = "openfang-cli"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"clap",
"clap_complete",
@@ -4068,7 +4068,7 @@ dependencies = [
[[package]]
name = "openfang-desktop"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"axum",
"open",
@@ -4094,7 +4094,7 @@ dependencies = [
[[package]]
name = "openfang-extensions"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"aes-gcm",
"argon2",
@@ -4122,14 +4122,16 @@ dependencies = [
[[package]]
name = "openfang-hands"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"chrono",
"dashmap",
"dirs 6.0.0",
"hex",
"openfang-types",
"serde",
"serde_json",
"sha2",
"tempfile",
"thiserror 2.0.18",
"tokio-test",
@@ -4140,7 +4142,7 @@ dependencies = [
[[package]]
name = "openfang-kernel"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"async-trait",
"chrono",
@@ -4165,6 +4167,7 @@ dependencies = [
"rustls",
"serde",
"serde_json",
"sha2",
"subtle",
"tempfile",
"thiserror 2.0.18",
@@ -4179,7 +4182,7 @@ dependencies = [
[[package]]
name = "openfang-memory"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"async-trait",
"chrono",
@@ -4199,7 +4202,7 @@ dependencies = [
[[package]]
name = "openfang-migrate"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"chrono",
"dirs 6.0.0",
@@ -4218,7 +4221,7 @@ dependencies = [
[[package]]
name = "openfang-runtime"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"anyhow",
"async-trait",
@@ -4254,11 +4257,13 @@ dependencies = [
[[package]]
name = "openfang-skills"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"chrono",
"ed25519-dalek",
"hex",
"openfang-types",
"rand 0.8.5",
"reqwest 0.12.28",
"serde",
"serde_json",
@@ -4277,7 +4282,7 @@ dependencies = [
[[package]]
name = "openfang-types"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"async-trait",
"bitflags 2.11.0",
@@ -4297,7 +4302,7 @@ dependencies = [
[[package]]
name = "openfang-wire"
version = "0.6.5"
version = "0.6.9"
dependencies = [
"async-trait",
"chrono",
@@ -4394,7 +4399,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d8fae84b431384b68627d0f9b3b1245fcf9f46f6c0e3dc902e9dce64edd1967"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.45.0",
]
[[package]]
@@ -5019,7 +5024,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27c6023962132f4b30eb4c172c91ce92d933da334c59c23cddee82358ddafb0b"
dependencies = [
"anyhow",
"itertools 0.14.0",
"itertools 0.13.0",
"proc-macro2",
"quote",
"syn 2.0.117",
@@ -5027,9 +5032,9 @@ dependencies = [
[[package]]
name = "pulley-interpreter"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "010dec3755eb61b2f1051ecb3611b718460b7a74c131e474de2af20a845938af"
checksum = "7ec12fe19a9588315a49fe5704502a9c02d6a198303314b0c7c86123b06d29e5"
dependencies = [
"cranelift-bitset",
"log",
@@ -5039,9 +5044,9 @@ dependencies = [
[[package]]
name = "pulley-macros"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad360c32e85ca4b083ac0e2b6856e8f11c3d5060dafa7d5dc57b370857fa3018"
checksum = "36f7d5ef31ebf1b46cd7e722ffef934e670d7e462f49aa01cde07b9b76dca580"
dependencies = [
"proc-macro2",
"quote",
@@ -5704,7 +5709,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys 0.12.1",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -5763,7 +5768,7 @@ dependencies = [
"security-framework",
"security-framework-sys",
"webpki-root-certs",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -5774,9 +5779,9 @@ checksum = "f87165f0995f63a9fbeea62b64d10b4d9d8e78ec6d7d51fb2125fda7bb36788f"
[[package]]
name = "rustls-webpki"
version = "0.103.10"
version = "0.103.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e"
dependencies = [
"aws-lc-rs",
"ring",
@@ -6309,7 +6314,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -7045,7 +7050,7 @@ dependencies = [
"getrandom 0.4.2",
"once_cell",
"rustix 1.1.4",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -7617,7 +7622,7 @@ checksum = "f2f6fb2847f6742cd76af783a2a2c49e9375d0a111c7bef6f71cd9e738c72d6e"
dependencies = [
"memoffset",
"tempfile",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -8061,9 +8066,9 @@ dependencies = [
[[package]]
name = "wasmtime"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce205cd643d661b5ba5ba4717e13730262e8cdbc8f2eacbc7b906d45c1a74026"
checksum = "efb1ed5899dde98357cfdcf647a4614498798719793898245b4b34e663addabf"
dependencies = [
"addr2line",
"async-trait",
@@ -8114,9 +8119,9 @@ dependencies = [
[[package]]
name = "wasmtime-environ"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b8b78abf3677d4a0a5db82e5015b4d085ff3a1b8b472cbb8c70d4b769f019ce"
checksum = "4172382dcc785c31d0e862c6780a18f5dd437914d22c4691351f965ef751c821"
dependencies = [
"anyhow",
"cpp_demangle",
@@ -8145,9 +8150,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-cache"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e4fd4103ba413c0da2e636f73490c6c8e446d708cbde7573703941bc3d6a448"
checksum = "4ed398988226d7aa0505ac6bb576e09532ad722d702ec4e66365d78ed695c95f"
dependencies = [
"base64 0.22.1",
"directories-next",
@@ -8165,9 +8170,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-component-macro"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d3d6914f34be2f9d78d8ee9f422e834dfc204e71ccce697205fae95fed87892"
checksum = "ae5ec9fff073ff13b81732d56a9515d761c245750bcda09093827f84130ebc25"
dependencies = [
"anyhow",
"proc-macro2",
@@ -8180,15 +8185,15 @@ dependencies = [
[[package]]
name = "wasmtime-internal-component-util"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3751b0616b914fdd87fe1bf804694a078f321b000338e6476bc48a4d6e454f21"
checksum = "935d9ab293ba27d1ec9aa7bc1b3a43993dbe961af2a8f23f90a11e1331b4c13f"
[[package]]
name = "wasmtime-internal-core"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22632b187e1b0716f1b9ac57ad29013bed33175fcb19e10bb6896126f82fac67"
checksum = "9a3820b174f477d2a7083209d1ad5353fcdb11eaea434b2137b8681029460dd3"
dependencies = [
"anyhow",
"hashbrown 0.16.1",
@@ -8198,9 +8203,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-cranelift"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b3ca07b3e0bb3429674b173b5800577719d600774dd81bff58f775c0aaa64ee"
checksum = "d1679d205caf9766c6aa309d45bb3e7c634d7725e3164404df33824b9f7c4fb7"
dependencies = [
"cfg-if",
"cranelift-codegen",
@@ -8225,9 +8230,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-fiber"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20c8b2c9704eb1f33ead025ec16038277ccb63d0a14c31e99d5b765d7c36da55"
checksum = "f1e505254058be5b0df458d670ee42d9eafe2349d04c1296e9dc01071dc20a85"
dependencies = [
"cc",
"cfg-if",
@@ -8240,9 +8245,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-jit-debug"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d950310d07391d34369f62c48336ebb14eacbd4d6f772bb5f349c24e838e0664"
checksum = "1c2e05b345f1773e59c20e6ad7298fd6857cdea245023d88bb659c96d8f0ea72"
dependencies = [
"cc",
"object",
@@ -8252,9 +8257,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-jit-icache-coherence"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3606662c156962d096be3127b8b8ae8ee2f8be3f896dad29259ff01ddb64abfd"
checksum = "b86701b234a4643e3f111869aa792b3a05a06e02d486ee9cb6c04dae16b52dab"
dependencies = [
"cfg-if",
"libc",
@@ -8264,9 +8269,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-unwinder"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75eef0747e52dc545b075f64fd0e0cc237ae738e641266b1970e07e2d744bc32"
checksum = "f63558d801beb83dde9b336eb4ae049019aee26627926edb32cd119d7e4c83cd"
dependencies = [
"cfg-if",
"cranelift-codegen",
@@ -8277,9 +8282,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-versioned-export-macros"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d8b0a5dab02a8fb527f547855ecc0e05f9fdc3d5bd57b8b080349408f9a6cece"
checksum = "737c4d956fc3a848541a064afb683dd2771132a6b125be5baaf95c4379aa47df"
dependencies = [
"proc-macro2",
"quote",
@@ -8288,9 +8293,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-winch"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8007342bd12ff400293a817973f7ecd6f1d9a8549a53369a9c1af357166f1f1e"
checksum = "f599b79545e3bba0b7913406055ebede5bb0dabee9ba2015ef25a9f4c9f47807"
dependencies = [
"cranelift-codegen",
"gimli",
@@ -8305,9 +8310,9 @@ dependencies = [
[[package]]
name = "wasmtime-internal-wit-bindgen"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7900c3e3c1d6e475bc225d73b02d6d5484815f260022e6964dca9558e50dd01a"
checksum = "2192a77a00b9a67800c2b4e1c70fb6abca79d6b529e53a2ef9dcdcc36090330d"
dependencies = [
"anyhow",
"bitflags 2.11.0",
@@ -8490,7 +8495,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -8501,9 +8506,9 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "winch-codegen"
version = "43.0.1"
version = "43.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb9f45f7172a2628c8317766e427babc0a400f9d10b1c0f0b0617c5ed5b79de6"
checksum = "52dbb0cf07b0dfe7b7a1ca8efb8f94ba98bd0fb144c411ea1665c78f0449e958"
dependencies = [
"cranelift-assembler-x64",
"cranelift-codegen",
@@ -9231,7 +9236,7 @@ checksum = "b9cc00251562a284751c9973bace760d86c0276c471b4be569fe6b068ee97a56"
[[package]]
name = "xtask"
version = "0.6.5"
version = "0.6.9"
[[package]]
name = "yoke"
+1 -1
View File
@@ -18,7 +18,7 @@ members = [
]
[workspace.package]
version = "0.6.5"
version = "0.6.9"
edition = "2021"
license = "Apache-2.0 OR MIT"
repository = "https://github.com/RightNow-AI/openfang"
+2 -2
View File
@@ -19,8 +19,8 @@
<p align="center">
<img src="https://img.shields.io/badge/language-Rust-orange?style=flat-square" alt="Rust" />
<img src="https://img.shields.io/badge/license-MIT-blue?style=flat-square" alt="MIT" />
<img src="https://img.shields.io/badge/version-0.6.5-green?style=flat-square" alt="v0.6.5" />
<img src="https://img.shields.io/badge/tests-2,585%2B%20passing-brightgreen?style=flat-square" alt="Tests" />
<img src="https://img.shields.io/badge/version-0.6.9-green?style=flat-square" alt="v0.6.9" />
<img src="https://img.shields.io/badge/tests-2,696%2B%20passing-brightgreen?style=flat-square" alt="Tests" />
<img src="https://img.shields.io/badge/clippy-0%20warnings-brightgreen?style=flat-square" alt="Clippy" />
<a href="https://www.buymeacoffee.com/openfang" target="_blank"><img src="https://img.shields.io/badge/Buy%20Me%20a%20Coffee-FFDD00?style=flat-square&logo=buy-me-a-coffee&logoColor=black" alt="Buy Me A Coffee" /></a>
</p>
+11 -2
View File
@@ -1166,11 +1166,12 @@ pub async fn start_channel_bridge_with_config(
if let Some(ref tg_config) = config.telegram {
if let Some(token) = read_token(&tg_config.bot_token_env, "Telegram") {
let poll_interval = Duration::from_secs(tg_config.poll_interval_secs);
let adapter = Arc::new(TelegramAdapter::new(
let adapter = Arc::new(TelegramAdapter::with_thread_routes(
token,
tg_config.allowed_users.clone(),
poll_interval,
tg_config.api_url.clone(),
tg_config.thread_routes.clone(),
));
adapters.push((adapter, tg_config.default_agent.clone()));
}
@@ -1185,6 +1186,7 @@ pub async fn start_channel_bridge_with_config(
dc_config.allowed_users.clone(),
dc_config.ignore_bots,
dc_config.intents,
dc_config.auto_thread.clone(),
));
adapters.push((adapter, dc_config.default_agent.clone()));
}
@@ -1249,10 +1251,17 @@ pub async fn start_channel_bridge_with_config(
// Matrix
if let Some(ref mx_config) = config.matrix {
if let Some(token) = read_token(&mx_config.access_token_env, "Matrix") {
let adapter = Arc::new(MatrixAdapter::new(
// MSC2918 refresh-token support: optional env var, when present the
// adapter auto-recovers from M_UNKNOWN_TOKEN 401s.
let refresh = mx_config
.refresh_token_env
.as_deref()
.and_then(|env| read_token(env, "Matrix refresh"));
let adapter = Arc::new(MatrixAdapter::with_refresh_token(
mx_config.homeserver_url.clone(),
mx_config.user_id.clone(),
token,
refresh,
mx_config.allowed_rooms.clone(),
mx_config.auto_accept_invites,
));
+414 -24
View File
@@ -692,6 +692,98 @@ pub async fn kill_agent(
}
}
/// DELETE /api/agents/{id}/uninstall — Permanently uninstall an agent.
///
/// Issue #1163: in addition to killing the agent (registry + memory + cron),
/// this also removes the on-disk `~/.openfang/agents/<name>/` directory so
/// the agent does not auto-respawn on the next daemon start.
pub async fn uninstall_agent(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
) -> impl IntoResponse {
let agent_id: AgentId = match id.parse() {
Ok(id) => id,
Err(_) => {
return (
StatusCode::BAD_REQUEST,
Json(serde_json::json!({"error": "Invalid agent ID"})),
);
}
};
// Capture the agent name BEFORE killing — registry entry is gone after.
let agent_name = match state.kernel.registry.get(agent_id) {
Some(entry) => entry.name.clone(),
None => {
return (
StatusCode::NOT_FOUND,
Json(serde_json::json!({"error": "Agent not found"})),
);
}
};
// Step 1: kill the agent (registry, memory, cron, triggers, caps).
if let Err(e) = state.kernel.kill_agent(agent_id) {
tracing::warn!("kill_agent failed during uninstall for {id}: {e}");
return (
StatusCode::NOT_FOUND,
Json(serde_json::json!({"error": "Agent not found or already terminated"})),
);
}
// Step 2: remove ~/.openfang/agents/<name>/ so the agent does NOT
// auto-respawn from disk on the next daemon start.
let agents_dir = state.kernel.config.home_dir.join("agents");
let agent_dir = agents_dir.join(&agent_name);
let dir_removed = if agent_dir.is_dir() {
// Safety: only allow removal if the parent is exactly the agents root.
let parent_ok = agent_dir
.parent()
.map(|p| p == agents_dir.as_path())
.unwrap_or(false);
if !parent_ok {
tracing::warn!(
agent = %agent_name,
path = %agent_dir.display(),
"Refusing to remove agent dir outside agents root"
);
false
} else {
match std::fs::remove_dir_all(&agent_dir) {
Ok(()) => {
tracing::info!(
agent = %agent_name,
path = %agent_dir.display(),
"Removed agent directory on uninstall (#1163)"
);
true
}
Err(e) => {
tracing::warn!(
agent = %agent_name,
path = %agent_dir.display(),
"Failed to remove agent directory: {e}"
);
false
}
}
}
} else {
false
};
(
StatusCode::OK,
Json(serde_json::json!({
"status": "uninstalled",
"agent_id": id,
"name": agent_name,
"dir_removed": dir_removed,
})),
)
}
/// POST /api/agents/{id}/restart — Restart a crashed/stuck agent.
///
/// Cancels any active task, resets agent state to Running, and updates last_active.
@@ -1417,11 +1509,15 @@ pub async fn get_agent(
"network": entry.manifest.capabilities.network,
},
"description": entry.manifest.description,
"system_prompt": entry.manifest.model.system_prompt,
"tags": entry.manifest.tags,
"identity": {
"emoji": entry.identity.emoji,
"avatar_url": entry.identity.avatar_url,
"color": entry.identity.color,
"archetype": entry.identity.archetype,
"vibe": entry.identity.vibe,
"greeting_style": entry.identity.greeting_style,
},
"skills": entry.manifest.skills,
"skills_mode": if entry.manifest.skills.is_empty() { "all" } else { "allowlist" },
@@ -3607,7 +3703,14 @@ pub async fn install_skill(
let config = openfang_skills::marketplace::MarketplaceConfig::default();
let client = openfang_skills::marketplace::MarketplaceClient::new(config);
match client.install(&req.name, &skills_dir).await {
let opts = openfang_skills::installer::InstallOptions {
require_signed: req.require_signed,
allowed_signer_keys: req.allowed_signer_keys.clone(),
};
match client
.install_with_options(&req.name, &skills_dir, &opts)
.await
{
Ok(version) => {
// Hot-reload so agents see the new skill immediately
state.kernel.reload_skills();
@@ -3664,6 +3767,112 @@ pub async fn reload_skills(State(state): State<Arc<AppState>>) -> impl IntoRespo
Json(serde_json::json!({"status": "reloaded"}))
}
/// POST /api/audit/append — Append an entry to the Merkle hash chain audit
/// trail on behalf of an external (instance-side) wrapper (issue #1174).
///
/// RBAC: gated by the same bearer-token middleware as POST /api/skills/install
/// (see `middleware::auth_middleware`). When `api_key` is configured every
/// caller must present `Authorization: Bearer <key>` — wrappers running in the
/// same trust boundary as the daemon are expected to share that key.
pub async fn audit_append(
State(state): State<Arc<AppState>>,
Json(req): Json<AuditAppendRequest>,
) -> impl IntoResponse {
use openfang_runtime::audit::AuditAction;
// SECURITY: bound input sizes so a wrapper cannot wedge the chain with
// unbounded strings. The audit table stores TEXT columns and the chain
// hash is computed over the same bytes — keep it sane.
const MAX_FIELD: usize = 16 * 1024;
if req.event_type.len() > MAX_FIELD
|| req.agent_id.len() > MAX_FIELD
|| req.detail.len() > MAX_FIELD
|| req
.outcome
.as_ref()
.map(|s| s.len() > MAX_FIELD)
.unwrap_or(false)
|| req
.signing_context
.as_ref()
.map(|s| s.len() > MAX_FIELD)
.unwrap_or(false)
{
return (
StatusCode::PAYLOAD_TOO_LARGE,
Json(serde_json::json!({"error": "field exceeds 16KB limit"})),
);
}
// Map operator-supplied event_type → AuditAction (case-insensitive).
let action = match req.event_type.trim().to_ascii_lowercase().as_str() {
"toolinvoke" | "tool_invoke" | "tool" => AuditAction::ToolInvoke,
"capabilitycheck" | "capability_check" | "capability" => AuditAction::CapabilityCheck,
"agentspawn" | "agent_spawn" | "spawn" => AuditAction::AgentSpawn,
"agentkill" | "agent_kill" | "kill" => AuditAction::AgentKill,
"agentmessage" | "agent_message" | "message" => AuditAction::AgentMessage,
"memoryaccess" | "memory_access" | "memory" => AuditAction::MemoryAccess,
"fileaccess" | "file_access" | "file" => AuditAction::FileAccess,
"networkaccess" | "network_access" | "network" => AuditAction::NetworkAccess,
"shellexec" | "shell_exec" | "shell" => AuditAction::ShellExec,
"authattempt" | "auth_attempt" | "auth" => AuditAction::AuthAttempt,
"wireconnect" | "wire_connect" | "wire" => AuditAction::WireConnect,
"configchange" | "config_change" | "config" => AuditAction::ConfigChange,
other => {
tracing::warn!(
"audit_append: unknown event_type {other:?}, falling back to ToolInvoke"
);
AuditAction::ToolInvoke
}
};
// Compose a detail string that preserves the operator's free-form detail
// plus optional signing context and structured payload, so wrappers can
// attach context without changing the on-chain schema.
let mut detail = req.detail.clone();
if let Some(ctx) = req.signing_context.as_ref().filter(|s| !s.is_empty()) {
if !detail.is_empty() {
detail.push_str(" | ");
}
detail.push_str("signer=");
detail.push_str(ctx);
}
if let Some(payload) = req.payload.as_ref() {
let serialised = serde_json::to_string(payload)
.unwrap_or_else(|_| String::from("<unserialisable payload>"));
// Cap payload contribution so a huge JSON blob cannot blow the entry.
let truncated: String = serialised.chars().take(8 * 1024).collect();
if !detail.is_empty() {
detail.push_str(" | ");
}
detail.push_str("payload=");
detail.push_str(&truncated);
}
let agent_id = if req.agent_id.trim().is_empty() {
"external-wrapper".to_string()
} else {
req.agent_id.clone()
};
let outcome = req.outcome.clone().unwrap_or_else(|| "ok".to_string());
let hash = state
.kernel
.audit_log
.record(agent_id, action, detail, outcome);
let seq = state.kernel.audit_log.len().saturating_sub(1) as u64;
(
StatusCode::OK,
Json(serde_json::json!({
"status": "appended",
"seq": seq,
"hash": hash,
"tip": state.kernel.audit_log.tip_hash(),
})),
)
}
/// GET /api/marketplace/search — Search the FangHub marketplace.
pub async fn marketplace_search(
Query(params): Query<HashMap<String, String>>,
@@ -7077,6 +7286,10 @@ pub async fn compact_session(
}
/// POST /api/agents/{id}/stop — Cancel an agent's current LLM run.
///
/// If the agent is owned by an active hand instance, the hand instance is
/// also deactivated. Otherwise the hand stays registered as `Active` and the
/// user cannot re-activate it via the wizard (issue #1164).
pub async fn stop_agent(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@@ -7090,6 +7303,33 @@ pub async fn stop_agent(
)
}
};
// If this agent is the agent of an active hand instance, deactivate the
// hand entirely — which also kills the agent and cancels the run. This
// matches what users expect when they click Stop on a hand-owned agent.
if let Some(instance) = state.kernel.hand_registry.find_by_agent(agent_id) {
match state.kernel.deactivate_hand(instance.instance_id) {
Ok(()) => {
return (
StatusCode::OK,
Json(serde_json::json!({
"status": "ok",
"message": "Hand deactivated",
"hand_deactivated": true,
"hand_id": instance.hand_id,
"instance_id": instance.instance_id,
})),
);
}
Err(e) => {
return (
StatusCode::INTERNAL_SERVER_ERROR,
Json(serde_json::json!({"error": format!("{e}")})),
);
}
}
}
match state.kernel.stop_agent_run(agent_id) {
Ok(true) => (
StatusCode::OK,
@@ -9826,7 +10066,9 @@ pub async fn clone_agent(
Err(e) => {
return (
StatusCode::INTERNAL_SERVER_ERROR,
Json(serde_json::json!({"error": format!("Failed to serialize manifest: {e}")})),
Json(
serde_json::json!({"error": format!("Failed to serialize manifest: {e}")}),
),
);
}
};
@@ -9846,8 +10088,10 @@ pub async fn clone_agent(
// new_name always wins over any name field in overrides.
cloned_manifest.name = new_name.clone();
// Let the kernel assign a fresh workspace path — never share with the template.
// Let the kernel assign fresh workspace and state directory paths.
// Never share private state with the template (see issue #868, #1097).
cloned_manifest.workspace = None;
cloned_manifest.state_dir = None;
// Spawn the cloned agent.
let new_id = match state.kernel.spawn_agent(cloned_manifest) {
@@ -9860,25 +10104,42 @@ pub async fn clone_agent(
}
};
// Copy non-memory workspace files from source to destination.
// MEMORY.md and HEARTBEAT.md are intentionally skipped — the cloned agent
// must start with independent memory (issue #868).
// Copy non-memory identity files from source state_dir to destination
// state_dir. MEMORY.md and HEARTBEAT.md are intentionally skipped — the
// cloned agent must start with independent memory (issue #868). Identity
// files live in state_dir per #1097; fall back to legacy workspace for
// older agents.
let new_entry = state.kernel.registry.get(new_id);
let src_state = source
.manifest
.state_dir
.as_ref()
.or(source.manifest.workspace.as_ref());
let dst_state = new_entry.as_ref().and_then(|e| {
e.manifest
.state_dir
.as_ref()
.or(e.manifest.workspace.as_ref())
});
if let (Some(src_ws), Some(dst_ws)) = (src_state, dst_state) {
if let (Ok(src_can), Ok(dst_can)) = (src_ws.canonicalize(), dst_ws.canonicalize()) {
for &fname in KNOWN_IDENTITY_FILES {
if MEMORY_FILES.contains(&fname) {
continue;
}
let src_file = src_can.join(fname);
let dst_file = dst_can.join(fname);
if src_file.exists() {
let _ = std::fs::copy(&src_file, &dst_file);
}
}
}
}
// Copy the user-facing `skills/` subdirectory so curated skills travel
// with the template. These live in the workspace, not the state dir.
if let (Some(ref src_ws), Some(ref new_entry)) = (&source.manifest.workspace, &new_entry) {
if let Some(ref dst_ws) = new_entry.manifest.workspace {
if let (Ok(src_can), Ok(dst_can)) = (src_ws.canonicalize(), dst_ws.canonicalize()) {
for &fname in KNOWN_IDENTITY_FILES {
if MEMORY_FILES.contains(&fname) {
continue;
}
let src_file = src_can.join(fname);
let dst_file = dst_can.join(fname);
if src_file.exists() {
let _ = std::fs::copy(&src_file, &dst_file);
}
}
// Copy the `skills/` subdirectory if present so curated skills
// travel with the template.
let src_skills = src_can.join("skills");
let dst_skills = dst_can.join("skills");
if src_skills.is_dir() {
@@ -9981,8 +10242,16 @@ pub async fn list_agent_files(
}
};
let workspace = match entry.manifest.workspace {
Some(ref ws) => ws.clone(),
// Identity files live in the agent's private state directory (see #1097).
// Fall back to the legacy workspace location for agents created before the
// split so existing on-disk files remain reachable.
let workspace = match entry
.manifest
.state_dir
.as_ref()
.or(entry.manifest.workspace.as_ref())
{
Some(ws) => ws.clone(),
None => {
return (
StatusCode::NOT_FOUND,
@@ -10043,8 +10312,15 @@ pub async fn get_agent_file(
}
};
let workspace = match entry.manifest.workspace {
Some(ref ws) => ws.clone(),
// Identity files live in the agent's private state directory (see #1097).
// Fall back to legacy workspace for agents created before the split.
let workspace = match entry
.manifest
.state_dir
.as_ref()
.or(entry.manifest.workspace.as_ref())
{
Some(ws) => ws.clone(),
None => {
return (
StatusCode::NOT_FOUND,
@@ -10150,8 +10426,15 @@ pub async fn set_agent_file(
}
};
let workspace = match entry.manifest.workspace {
Some(ref ws) => ws.clone(),
// Identity files live in the agent's private state directory (see #1097).
// Fall back to legacy workspace for agents created before the split.
let workspace = match entry
.manifest
.state_dir
.as_ref()
.or(entry.manifest.workspace.as_ref())
{
Some(ws) => ws.clone(),
None => {
return (
StatusCode::NOT_FOUND,
@@ -12583,3 +12866,110 @@ mod skill_config_tests {
assert_eq!(back, doc);
}
}
#[cfg(test)]
mod uninstall_agent_tests {
//! Issue #1163 — directory-removal portion of the uninstall flow.
//!
//! These tests exercise the same logic the route handler runs after
//! `kernel.kill_agent()`: locate `<home>/agents/<name>/`, verify it is
//! directly under the agents root, and remove it. Live end-to-end
//! coverage (real HTTP + kernel) belongs in `tests/api_integration_test.rs`.
use std::path::Path;
/// Mirror of the dir-removal logic in `uninstall_agent`. Kept in sync
/// with the route handler so the rules can be unit-tested without a
/// running kernel. Returns whether the directory was removed.
fn remove_agent_dir(home_dir: &Path, agent_name: &str) -> bool {
let agents_dir = home_dir.join("agents");
let agent_dir = agents_dir.join(agent_name);
if !agent_dir.is_dir() {
return false;
}
let parent_ok = agent_dir
.parent()
.map(|p| p == agents_dir.as_path())
.unwrap_or(false);
if !parent_ok {
return false;
}
std::fs::remove_dir_all(&agent_dir).is_ok()
}
#[test]
fn removes_agent_directory_under_agents_root() {
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path().to_path_buf();
let agents = home.join("agents");
std::fs::create_dir_all(agents.join("trash-agent")).unwrap();
std::fs::write(
agents.join("trash-agent").join("agent.toml"),
"name = \"trash-agent\"\n",
)
.unwrap();
assert!(agents.join("trash-agent").is_dir());
let removed = remove_agent_dir(&home, "trash-agent");
assert!(removed, "agent directory must be removed");
assert!(!agents.join("trash-agent").exists());
}
#[test]
fn returns_false_when_no_directory_exists() {
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path().to_path_buf();
std::fs::create_dir_all(home.join("agents")).unwrap();
let removed = remove_agent_dir(&home, "ghost-agent");
assert!(!removed, "no dir => false, but uninstall still succeeds");
}
#[test]
fn does_not_touch_siblings() {
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path().to_path_buf();
let agents = home.join("agents");
std::fs::create_dir_all(agents.join("trash-agent")).unwrap();
std::fs::create_dir_all(agents.join("keep-me")).unwrap();
std::fs::write(
agents.join("trash-agent").join("agent.toml"),
"name = \"trash-agent\"\n",
)
.unwrap();
std::fs::write(
agents.join("keep-me").join("agent.toml"),
"name = \"keep-me\"\n",
)
.unwrap();
assert!(remove_agent_dir(&home, "trash-agent"));
assert!(!agents.join("trash-agent").exists());
assert!(
agents.join("keep-me").is_dir(),
"sibling agent dirs must not be touched by uninstall"
);
}
#[test]
fn rejects_path_traversal_attempt() {
// A name like "../escape" would join to a path whose parent is the
// agents root only if the file system resolves it that way — but
// `parent()` on a non-canonicalized Path returns the textual parent,
// which for `<home>/agents/../escape` is `<home>/agents/..`, not
// `<home>/agents`. The check rejects it.
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path().to_path_buf();
std::fs::create_dir_all(home.join("agents")).unwrap();
// Create a sibling dir outside agents/ that an attacker might want
// to delete.
std::fs::create_dir_all(home.join("escape")).unwrap();
std::fs::write(home.join("escape").join("secret.toml"), "x = 1\n").unwrap();
let removed = remove_agent_dir(&home, "../escape");
assert!(!removed, "must reject path-traversal names");
assert!(
home.join("escape").is_dir(),
"sibling dir outside agents/ must NOT be deleted"
);
}
}
+9
View File
@@ -186,6 +186,10 @@ pub async fn build_router(
.delete(routes::kill_agent)
.patch(routes::patch_agent),
)
.route(
"/api/agents/{id}/uninstall",
axum::routing::delete(routes::uninstall_agent),
)
.route(
"/api/agents/{id}/mode",
axum::routing::put(routes::set_agent_mode),
@@ -390,6 +394,11 @@ pub async fn build_router(
"/api/skills/reload",
axum::routing::post(routes::reload_skills),
)
// Audit trail (issue #1174 — instance-side wrapper integration)
.route(
"/api/audit/append",
axum::routing::post(routes::audit_append),
)
.route(
"/api/skills/{id}/config",
axum::routing::get(routes::get_skill_config).put(routes::put_skill_config),
+102
View File
@@ -65,6 +65,15 @@ pub struct MessageResponse {
#[derive(Debug, Deserialize)]
pub struct SkillInstallRequest {
pub name: String,
/// When true, reject the install unless the bundle ships a valid
/// Ed25519 SignedManifest envelope bound to the on-disk manifest.
/// Maps to `InstallOptions::require_signed` (issue #1170).
#[serde(default)]
pub require_signed: bool,
/// Optional hex-encoded allow-list of acceptable signer public keys.
/// Empty = TOFU (any valid signature accepted).
#[serde(default)]
pub allowed_signer_keys: Vec<String>,
}
/// Request to uninstall a skill.
@@ -115,3 +124,96 @@ pub struct CommandsQuery {
#[serde(default)]
pub surface: Option<String>,
}
/// Request body for `POST /api/audit/append` (issue #1174).
///
/// Lets external (instance-side) wrappers append entries to the Merkle hash
/// chain audit log. The handler maps `event_type` to an `AuditAction` and
/// records the entry through `kernel.audit_log`.
#[derive(Debug, Deserialize)]
pub struct AuditAppendRequest {
/// Operator-supplied event category. Case-insensitive, matched against the
/// `AuditAction` enum variants (e.g. `tool_invoke`, `ConfigChange`,
/// `agent_message`). Unknown values fall back to `ToolInvoke`.
pub event_type: String,
/// Agent or wrapper identifier responsible for the event. When empty,
/// recorded as `"external-wrapper"`.
#[serde(default)]
pub agent_id: String,
/// Free-form detail string (e.g. tool name, URL, file path).
#[serde(default)]
pub detail: String,
/// Optional arbitrary payload. When present it is serialised to JSON and
/// appended onto the entry's detail so the wrapper retains structured
/// context without changing the on-chain schema.
#[serde(default)]
pub payload: Option<serde_json::Value>,
/// Optional outcome string (`"ok"`, `"denied"`, or an error). Defaults to
/// `"ok"` when omitted.
#[serde(default)]
pub outcome: Option<String>,
/// Optional operator-supplied signing context (e.g. wrapper identity, key
/// fingerprint). Mixed into the detail when present so the chain captures
/// who attested to the event.
#[serde(default)]
pub signing_context: Option<String>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn skill_install_request_defaults_back_compat() {
// Existing callers send `{"name": "..."}` only. New optional fields
// must default cleanly (issue #1170).
let req: SkillInstallRequest = serde_json::from_str(r#"{"name":"github-helper"}"#).unwrap();
assert_eq!(req.name, "github-helper");
assert!(!req.require_signed);
assert!(req.allowed_signer_keys.is_empty());
}
#[test]
fn skill_install_request_parses_require_signed() {
let req: SkillInstallRequest = serde_json::from_str(
r#"{"name":"x","require_signed":true,"allowed_signer_keys":["abc123"]}"#,
)
.unwrap();
assert!(req.require_signed);
assert_eq!(req.allowed_signer_keys, vec!["abc123".to_string()]);
}
#[test]
fn audit_append_request_required_only() {
// Only `event_type` is required; everything else must default.
let req: AuditAppendRequest =
serde_json::from_str(r#"{"event_type":"ToolInvoke"}"#).unwrap();
assert_eq!(req.event_type, "ToolInvoke");
assert!(req.agent_id.is_empty());
assert!(req.detail.is_empty());
assert!(req.payload.is_none());
assert!(req.outcome.is_none());
assert!(req.signing_context.is_none());
}
#[test]
fn audit_append_request_full_payload() {
let body = r#"{
"event_type": "config_change",
"agent_id": "wrapper-1",
"detail": "rotated key",
"payload": {"key_id": "k-42", "ts": 1700000000},
"outcome": "ok",
"signing_context": "ed25519:deadbeef"
}"#;
let req: AuditAppendRequest = serde_json::from_str(body).unwrap();
assert_eq!(req.event_type, "config_change");
assert_eq!(req.agent_id, "wrapper-1");
assert_eq!(req.detail, "rotated key");
assert_eq!(req.outcome.as_deref(), Some("ok"));
assert_eq!(req.signing_context.as_deref(), Some("ed25519:deadbeef"));
let payload = req.payload.expect("payload present");
assert_eq!(payload["key_id"], "k-42");
assert_eq!(payload["ts"], 1_700_000_000);
}
}
+5 -4
View File
@@ -90,11 +90,11 @@ pub async fn webchat_page() -> impl IntoResponse {
let html = WEBCHAT_HTML.replace(NONCE_PLACEHOLDER, &nonce);
let csp = format!(
"default-src 'self'; \
script-src 'self' 'nonce-{nonce}' 'unsafe-eval'; \
style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://fonts.gstatic.com; \
script-src 'self' 'nonce-{nonce}' 'unsafe-eval' https://cdn.jsdelivr.net; \
style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://fonts.gstatic.com https://cdn.jsdelivr.net; \
img-src 'self' data: blob:; \
connect-src 'self' ws://localhost:* ws://127.0.0.1:* wss://localhost:* wss://127.0.0.1:*; \
font-src 'self' https://fonts.gstatic.com; \
connect-src 'self' ws://localhost:* ws://127.0.0.1:* wss://localhost:* wss://127.0.0.1:* https://cdn.jsdelivr.net; \
font-src 'self' https://fonts.gstatic.com https://cdn.jsdelivr.net; \
media-src 'self' blob:; \
frame-src 'self' blob:; \
object-src 'none'; \
@@ -120,6 +120,7 @@ pub async fn webchat_page() -> impl IntoResponse {
/// All vendor libraries (Alpine.js, marked.js, highlight.js) are bundled
/// locally — no CDN dependency. Alpine.js is included LAST because it
/// immediately processes x-data directives and fires alpine:init on load.
/// KaTeX is loaded dynamically from jsdelivr CDN when needed for LaTeX rendering.
const WEBCHAT_HTML: &str = concat!(
include_str!("../static/index_head.html"),
"<style>\n",
+127 -20
View File
@@ -223,18 +223,31 @@ pub(crate) struct WsAuthCtx<'a> {
pub(crate) fn check_ws_auth(ctx: &WsAuthCtx<'_>) -> Result<(), axum::http::StatusCode> {
use axum::http::StatusCode;
// No api_key configured: only allow loopback or explicit opt-in.
// No api_key configured: behavior depends on whether dashboard auth is on.
//
// Issue #1189: previously this path allowed any loopback request through
// when api_key was empty, EVEN IF dashboard auth was enabled. That diverged
// from the HTTP middleware (which only opens the loopback no-auth path
// when api_key is empty AND auth.enabled is false). A local attacker with
// loopback access could chat with agents over WS even when the operator
// had configured dashboard credentials. Now mirror HTTP exactly.
if ctx.api_key.is_empty() {
// A session cookie can still rescue non-loopback requests when
// dashboard auth is enabled.
if ctx.auth_enabled && !ctx.session_secret.is_empty() {
if let Some(token) = crate::session_auth::extract_session_cookie(ctx.headers) {
if crate::session_auth::verify_session_token(&token, ctx.session_secret).is_some()
{
return Ok(());
// When dashboard auth is configured, require a valid session cookie
// regardless of bind address. Loopback no longer bypasses login.
if ctx.auth_enabled {
if !ctx.session_secret.is_empty() {
if let Some(token) = crate::session_auth::extract_session_cookie(ctx.headers) {
if crate::session_auth::verify_session_token(&token, ctx.session_secret)
.is_some()
{
return Ok(());
}
}
}
return Err(StatusCode::UNAUTHORIZED);
}
// No api_key AND dashboard auth disabled: keep the dev convenience
// path (loopback or explicit OPENFANG_ALLOW_NO_AUTH=1).
if ctx.is_loopback || ctx.allow_no_auth {
return Ok(());
}
@@ -1025,15 +1038,34 @@ async fn handle_command(
serde_json::json!({"type": "error", "content": format!("Compaction failed: {e}")})
}
},
"stop" => match state.kernel.stop_agent_run(agent_id) {
Ok(true) => {
serde_json::json!({"type": "command_result", "command": cmd, "message": "Run cancelled."})
"stop" => {
// If this agent is owned by an active hand instance, deactivate the
// hand entirely so the user can re-activate it (issue #1164).
if let Some(instance) = state.kernel.hand_registry.find_by_agent(agent_id) {
match state.kernel.deactivate_hand(instance.instance_id) {
Ok(()) => serde_json::json!({
"type": "command_result",
"command": cmd,
"message": format!("Hand '{}' deactivated.", instance.hand_id),
}),
Err(e) => {
serde_json::json!({"type": "error", "content": format!("Stop failed: {e}")})
}
}
} else {
match state.kernel.stop_agent_run(agent_id) {
Ok(true) => {
serde_json::json!({"type": "command_result", "command": cmd, "message": "Run cancelled."})
}
Ok(false) => {
serde_json::json!({"type": "command_result", "command": cmd, "message": "No active run to cancel."})
}
Err(e) => {
serde_json::json!({"type": "error", "content": format!("Stop failed: {e}")})
}
}
}
Ok(false) => {
serde_json::json!({"type": "command_result", "command": cmd, "message": "No active run to cancel."})
}
Err(e) => serde_json::json!({"type": "error", "content": format!("Stop failed: {e}")}),
},
}
"model" => {
if args.is_empty() {
if let Some(entry) = state.kernel.registry.get(agent_id) {
@@ -1784,10 +1816,7 @@ mod tests {
// Cookie signed with the wrong secret must fail.
let bad = crate::session_auth::create_session_token("alice", "other-secret", 1);
let mut headers = axum::http::HeaderMap::new();
headers.insert(
"cookie",
format!("openfang_session={bad}").parse().unwrap(),
);
headers.insert("cookie", format!("openfang_session={bad}").parse().unwrap());
let uri = empty_uri();
let ctx = WsAuthCtx {
api_key: "secret",
@@ -1918,4 +1947,82 @@ mod tests {
};
assert!(check_ws_auth(&ctx).is_ok());
}
// -----------------------------------------------------------------------
// Issue #1189: WS auth must mirror HTTP middleware. When dashboard auth
// is enabled, loopback + empty api_key + no cookie must NOT bypass.
// -----------------------------------------------------------------------
#[test]
fn ws_auth_dashboard_on_loopback_empty_key_no_cookie_rejected() {
// Issue #1189 regression guard: previously this returned Ok(()) because
// the empty-api_key branch allowed any loopback request through, even
// when dashboard credentials were configured. HTTP middleware rejects
// this path; WS must too.
let secret = "password-hash-style-secret";
let headers = axum::http::HeaderMap::new();
let uri = empty_uri();
let ctx = WsAuthCtx {
api_key: "",
auth_enabled: true,
session_secret: secret,
is_loopback: true,
allow_no_auth: false,
headers: &headers,
uri: &uri,
};
assert_eq!(
check_ws_auth(&ctx).unwrap_err(),
axum::http::StatusCode::UNAUTHORIZED,
"loopback must not bypass dashboard auth when api_key is empty"
);
}
#[test]
fn ws_auth_dashboard_on_loopback_valid_cookie_accepted() {
// With dashboard auth on, a valid session cookie is the supported
// credential and must upgrade successfully from loopback too.
let secret = "password-hash-style-secret";
let token = crate::session_auth::create_session_token("admin", secret, 1);
let mut headers = axum::http::HeaderMap::new();
headers.insert(
"cookie",
format!("openfang_session={token}").parse().unwrap(),
);
let uri = empty_uri();
let ctx = WsAuthCtx {
api_key: "",
auth_enabled: true,
session_secret: secret,
is_loopback: true,
allow_no_auth: false,
headers: &headers,
uri: &uri,
};
assert!(
check_ws_auth(&ctx).is_ok(),
"valid session cookie should authorize loopback WS upgrade"
);
}
#[test]
fn ws_auth_dashboard_off_loopback_empty_key_accepted() {
// Preserve the development convenience path: when dashboard auth is
// NOT configured AND api_key is empty, loopback still upgrades.
let headers = axum::http::HeaderMap::new();
let uri = empty_uri();
let ctx = WsAuthCtx {
api_key: "",
auth_enabled: false,
session_secret: "",
is_loopback: true,
allow_no_auth: false,
headers: &headers,
uri: &uri,
};
assert!(
check_ws_auth(&ctx).is_ok(),
"loopback dev path must work when dashboard auth is disabled"
);
}
}
+3 -1
View File
@@ -580,6 +580,7 @@
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><g x-show="!$store.app.focusMode"><path d="M8 3H5a2 2 0 0 0-2 2v3"/><path d="M21 8V5a2 2 0 0 0-2-2h-3"/><path d="M3 16v3a2 2 0 0 0 2 2h3"/><path d="M16 21h3a2 2 0 0 0 2-2v-3"/></g><g x-show="$store.app.focusMode"><path d="M8 3v3a2 2 0 0 1-2 2H3"/><path d="M21 8h-3a2 2 0 0 1-2-2V3"/><path d="M3 16h3a2 2 0 0 1 2 2v3"/><path d="M16 21v-3a2 2 0 0 1 2-2h3"/></g></svg>
</button>
<button class="btn btn-danger btn-sm" @click="killAgent()">Stop</button>
<button class="btn btn-danger btn-sm" @click="uninstallAgent()" title="Stop and remove agent files from workspace">Uninstall</button>
</div>
</div>
@@ -739,7 +740,7 @@
<span class="text-xs" style="color:var(--danger)" x-text="formatRecordingTime()"></span>
</div>
<textarea id="msg-input" rows="1" :placeholder="recording ? 'Recording... release to send' : 'Message OpenFang... (/ for commands)'"
@keydown.enter.prevent="if(!$event.isComposing && $event.keyCode !== 229 && !$event.shiftKey){if(showModelPicker && filteredModelPicker.length){pickModel(filteredModelPicker[modelPickerIdx].id)}else if(showSlashMenu && filteredSlashCommands.length){executeSlashCommand(filteredSlashCommands[slashIdx].cmd)}else{sendMessage()}}"
@keydown.enter="if(!$event.isComposing && $event.keyCode !== 229 && !$event.shiftKey){$event.preventDefault();if(showModelPicker && filteredModelPicker.length){pickModel(filteredModelPicker[modelPickerIdx].id)}else if(showSlashMenu && filteredSlashCommands.length){executeSlashCommand(filteredSlashCommands[slashIdx].cmd)}else{sendMessage()}}"
@keydown.escape="showSlashMenu = false; showModelPicker = false"
@keydown.arrow-up.prevent="if(showModelPicker){modelPickerIdx = Math.max(0, modelPickerIdx - 1)}else if(showSlashMenu){slashIdx = Math.max(0, slashIdx - 1)}"
@keydown.arrow-down.prevent="if(showModelPicker){modelPickerIdx = Math.min(filteredModelPicker.length - 1, modelPickerIdx + 1)}else if(showSlashMenu){slashIdx = Math.min(filteredSlashCommands.length - 1, slashIdx + 1)}"
@@ -988,6 +989,7 @@
<button class="btn btn-ghost" @click="cloneAgent(detailAgent)">Clone</button>
<button class="btn btn-ghost" @click="clearHistory(detailAgent)">Clear History</button>
<button class="btn btn-danger" @click="killAgent(detailAgent)">Stop</button>
<button class="btn btn-danger" @click="uninstallAgent(detailAgent)" title="Stop and remove agent files from workspace">Uninstall</button>
</div>
</div>
+1 -1
View File
@@ -18,7 +18,7 @@ if (typeof marked !== 'undefined') {
function escapeHtml(text) {
var div = document.createElement('div');
div.textContent = text || '';
return div.innerHTML;
return div.innerHTML.replace(/\n/g, '<br>');
}
function renderMarkdown(text) {
+46 -14
View File
@@ -337,29 +337,38 @@ function agentsPage() {
OpenFangAPI.wsDisconnect();
},
buildConfigForm(agent) {
var identity = (agent && agent.identity) || {};
return {
name: (agent && agent.name) || '',
system_prompt: (agent && agent.system_prompt) || '',
emoji: identity.emoji || '',
color: identity.color || '#FF5C00',
archetype: identity.archetype || '',
vibe: identity.vibe || ''
};
},
async showDetail(agent) {
this.detailAgent = agent;
this.detailAgent._fallbacks = [];
this.detailTab = 'info';
this.agentFiles = [];
this.editingFile = null;
this.fileContent = '';
this.editingFallback = false;
this.newFallbackValue = '';
this.configForm = {
name: agent.name || '',
system_prompt: agent.system_prompt || '',
emoji: (agent.identity && agent.identity.emoji) || '',
color: (agent.identity && agent.identity.color) || '#FF5C00',
archetype: (agent.identity && agent.identity.archetype) || '',
vibe: (agent.identity && agent.identity.vibe) || ''
};
this.showDetailModal = true;
// Fetch full agent detail to get fallback_models
// Load the full detail payload before opening the modal so editable
// fields such as system_prompt and identity metadata are hydrated.
var detail = agent;
try {
var full = await OpenFangAPI.get('/api/agents/' + agent.id);
this.detailAgent._fallbacks = full.fallback_models || [];
} catch(e) { /* ignore */ }
detail = Object.assign({}, agent, full, {
identity: Object.assign({}, (agent && agent.identity) || {}, (full && full.identity) || {})
});
} catch(e) { /* fall back to list payload */ }
this.detailAgent = detail;
this.detailAgent._fallbacks = detail.fallback_models || [];
this.configForm = this.buildConfigForm(detail);
this.showDetailModal = true;
},
killAgent(agent) {
@@ -376,6 +385,29 @@ function agentsPage() {
});
},
// Issue #1163: uninstall an agent (kill + remove ~/.openfang/agents/<name>/).
uninstallAgent(agent) {
var self = this;
OpenFangToast.confirm(
'Uninstall Agent',
'Uninstall agent "' + agent.name + '"? This stops the agent AND deletes its files from your workspace. This cannot be undone.',
async function() {
try {
var res = await OpenFangAPI.del('/api/agents/' + agent.id + '/uninstall');
var msg = 'Agent "' + agent.name + '" uninstalled';
if (res && res.dir_removed === false) {
msg += ' (no on-disk files found)';
}
OpenFangToast.success(msg);
self.showDetailModal = false;
await Alpine.store('app').refreshAgents();
} catch(e) {
OpenFangToast.error('Failed to uninstall agent: ' + e.message);
}
}
);
},
killAllAgents() {
var list = this.filteredAgents;
if (!list.length) return;
@@ -143,6 +143,29 @@ function chatPage() {
// Fetch dynamic commands from server
this.fetchCommands();
// Observe DOM for new messages and render LaTeX
this._latexObserver = new MutationObserver(function(mutations) {
mutations.forEach(function(mutation) {
mutation.addedNodes.forEach(function(node) {
if (node.nodeType === Node.ELEMENT_NODE) {
var bubbles = node.querySelector ? node.querySelectorAll('.message-bubble') : [];
if (node.classList && node.classList.contains('message-bubble')) {
bubbles = [node];
}
bubbles.forEach(function(bubble) {
if (bubble.textContent && hasLatexDelimiters(bubble.textContent)) {
renderLatex(bubble);
}
});
}
});
});
});
this._latexObserver.observe(document.getElementById('messages') || document.body, {
childList: true,
subtree: true
});
// Ctrl+/ keyboard shortcut
document.addEventListener('keydown', function(e) {
if ((e.ctrlKey || e.metaKey) && e.key === '/') {
@@ -175,6 +198,10 @@ function chatPage() {
if (store.pendingAgent) {
self.selectAgent(store.pendingAgent);
store.pendingAgent = null;
} else {
// Restore previously active agent after page refresh (#1179).
// The agent list may not be loaded yet, so resolve once it appears.
self._restoreActiveAgent();
}
// Watch for future pending agent selections (e.g., user clicks agent while on chat)
@@ -185,6 +212,13 @@ function chatPage() {
}
});
// Re-attempt restore once the agent list arrives from the server
this.$watch('$store.app.agents', function(agents) {
if (!self.currentAgent && agents && agents.length) {
self._restoreActiveAgent();
}
});
// Watch for slash commands + model autocomplete
this.$watch('inputText', function(val) {
var modelMatch = val.match(/^\/model\s+(.*)$/i);
@@ -528,6 +562,7 @@ function chatPage() {
self._wsAgent = null;
self.currentAgent = null;
self.messages = [];
try { localStorage.removeItem('of-active-agent'); } catch(e) { /* ignore */ }
window.dispatchEvent(new Event('close-chat'));
break;
case '/budget':
@@ -563,9 +598,27 @@ function chatPage() {
}
},
// Restore the previously-active agent (set in selectAgent) after a page
// refresh, so the WebSocket re-attaches to the same session and any
// in-flight tool output streams back into the chat (#1179).
_restoreActiveAgent: function() {
var storedId = null;
try { storedId = localStorage.getItem('of-active-agent'); } catch(e) { /* ignore */ }
if (!storedId) return;
var agents = (Alpine.store('app') && Alpine.store('app').agents) || [];
var match = null;
for (var i = 0; i < agents.length; i++) {
if (agents[i] && agents[i].id === storedId) { match = agents[i]; break; }
}
if (match) {
this.selectAgent(match);
}
},
selectAgent(agent) {
this.currentAgent = agent;
this.messages = [];
try { localStorage.setItem('of-active-agent', agent.id); } catch(e) { /* ignore */ }
this.connectWs(agent.id);
var t = typeof window.t === 'function' ? window.t : function(s) { return s; };
// Show welcome tips on first use
@@ -1150,6 +1203,7 @@ function chatPage() {
self._wsAgent = null;
self.currentAgent = null;
self.messages = [];
try { localStorage.removeItem('of-active-agent'); } catch(e) { /* ignore */ }
OpenFangToast.success(t('chat.agent_stopped') + ' "' + name + '"');
Alpine.store('app').refreshAgents();
} catch(e) {
@@ -1158,6 +1212,37 @@ function chatPage() {
});
},
// Permanently uninstall the agent: kill + remove ~/.openfang/agents/<name>/
// Issue #1163.
uninstallAgent: function() {
if (!this.currentAgent) return;
var self = this;
var name = this.currentAgent.name;
var agentId = this.currentAgent.id;
OpenFangToast.confirm(
'Uninstall Agent',
'Uninstall agent "' + name + '"? This stops the agent AND deletes its files from your workspace. This cannot be undone.',
async function() {
try {
var res = await OpenFangAPI.del('/api/agents/' + agentId + '/uninstall');
OpenFangAPI.wsDisconnect();
self._wsAgent = null;
self.currentAgent = null;
self.messages = [];
try { localStorage.removeItem('of-active-agent'); } catch(e) { /* ignore */ }
var msg = 'Agent "' + name + '" uninstalled';
if (res && res.dir_removed === false) {
msg += ' (no on-disk files found)';
}
OpenFangToast.success(msg);
Alpine.store('app').refreshAgents();
} catch(e) {
OpenFangToast.error('Failed to uninstall agent: ' + e.message);
}
}
);
},
_latexTimer: null,
scrollToBottom() {
var self = this;
@@ -1660,10 +1660,7 @@ async fn test_clone_agent_happy_path() {
assert_eq!(manifest["description"], "Cloned for user 1");
assert_eq!(
manifest["tags"].as_array().unwrap(),
&vec![
serde_json::json!("clone"),
serde_json::json!("user-1"),
]
&vec![serde_json::json!("clone"), serde_json::json!("user-1"),]
);
// Inherited from template — the system_prompt should match.
assert_eq!(
@@ -1678,10 +1675,7 @@ async fn test_clone_agent_happy_path() {
.await
.unwrap();
let agents: Vec<serde_json::Value> = resp.json().await.unwrap();
let names: Vec<&str> = agents
.iter()
.map(|a| a["name"].as_str().unwrap())
.collect();
let names: Vec<&str> = agents.iter().map(|a| a["name"].as_str().unwrap()).collect();
assert!(names.contains(&"test-agent"));
assert!(names.contains(&"cloned-user-1"));
}
@@ -1731,10 +1725,7 @@ async fn test_clone_agent_name_collision() {
"duplicate name must return 409 Conflict"
);
let body: serde_json::Value = resp.json().await.unwrap();
assert!(body["error"]
.as_str()
.unwrap()
.contains("already exists"));
assert!(body["error"].as_str().unwrap().contains("already exists"));
// Cloning into the template's own name must also be rejected.
let resp = client
+412 -69
View File
@@ -4,7 +4,7 @@
//! `BridgeManager` which owns running adapters and dispatches messages.
use crate::formatter;
use crate::router::AgentRouter;
use crate::router::{AgentRouter, BindingContext};
use crate::types::{
default_phase_emoji, AgentPhase, ChannelAdapter, ChannelContent, ChannelMessage, ChannelUser,
LifecycleReaction,
@@ -679,31 +679,33 @@ fn sender_user_id(message: &ChannelMessage) -> &str {
.unwrap_or(&message.sender.platform_id)
}
/// Extract the channel/conversation ID from a message, for bindings whose
/// `match_rule.channel_id` is set.
/// Build a `BindingContext` for routing the given inbound message.
///
/// On Discord and Slack, `sender.platform_id` already holds the channel/
/// conversation ID (per `discord.rs` and `slack.rs`, where the user ID lives
/// in metadata under `sender_user_id`). On other adapters where the platform
/// ID is the user, callers can opt-in by stashing the channel ID under the
/// `sender_channel_id` metadata key.
fn sender_channel_id(message: &ChannelMessage) -> Option<&str> {
if let Some(v) = message
.metadata
.get("sender_channel_id")
.and_then(|v| v.as_str())
{
return Some(v);
}
// On Discord/Slack, the metadata `sender_user_id` is set and differs from
// `sender.platform_id` — in that case, platform_id IS the channel ID.
let user_in_meta = message
.metadata
.get("sender_user_id")
.and_then(|v| v.as_str());
match user_in_meta {
Some(uid) if uid != message.sender.platform_id => Some(&message.sender.platform_id),
_ => None,
/// Populates `channel_id` so per-channel bindings (e.g. `channel_id = "<discord_channel>"`)
/// can route to dedicated agents. The channel ID source is delegated to
/// [`ChannelMessage::channel_id`] — the single source of truth shared with
/// config validation (see `CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL` in
/// `openfang-types::config`). `peer_id` uses the resolved user ID, not
/// `sender.platform_id`, so user-scoped bindings still match correctly on
/// Discord/Slack/etc. where `platform_id` holds the channel.
///
/// This replaces the earlier heuristic `sender_channel_id()` (which inferred
/// "platform_id is the channel" from "metadata has `sender_user_id`"). The
/// allowlist is explicit, the metadata-fallback path is documented, and
/// adapters can be added or removed in one place (`openfang-types::config`)
/// without touching this file.
fn binding_context_for(message: &ChannelMessage) -> BindingContext {
BindingContext {
channel: channel_type_str(&message.channel).to_string(),
account_id: None,
peer_id: sender_user_id(message).to_string(),
channel_id: message.channel_id(),
guild_id: message
.metadata
.get("guild_id")
.and_then(|v| v.as_str())
.map(String::from),
roles: Vec::new(),
}
}
@@ -771,12 +773,20 @@ async fn dispatch_message(
.as_ref()
.map(|o| o.lifecycle_reactions)
.unwrap_or(true);
let thread_id = if threading_enabled {
message.thread_id.as_deref()
// --- Auto-thread: decide intent now, but create AFTER all policy guards ---
let auto_thread_name = if !threading_enabled && message.thread_id.is_none() {
adapter.should_auto_thread(message).await
} else {
None
};
// thread_id is resolved later, after all guards pass.
// Always propagate an existing thread_id (message arrived inside a thread),
// regardless of threading_enabled — that flag controls explicit threading config,
// not auto-detected thread context.
let mut effective_thread_id: Option<String> = message.thread_id.clone();
// --- DM/Group policy check ---
if let Some(ref ov) = overrides {
if message.is_group {
@@ -837,12 +847,42 @@ async fn dispatch_message(
if let Err(msg) =
rate_limiter.check(ct_str, sender_user_id(message), ov.rate_limit_per_user)
{
send_response(adapter, &message.sender, msg, thread_id, output_format).await;
// Rate-limit rejection: don't create a thread, use existing thread if any
send_response(
adapter,
&message.sender,
msg,
message.thread_id.as_deref(),
output_format,
)
.await;
return;
}
}
}
// --- Create auto-thread NOW (after all policy guards have passed) ---
if let Some(ref thread_name) = auto_thread_name {
match adapter
.create_thread(&message.sender, &message.platform_message_id, thread_name)
.await
{
Ok(new_thread_id) => {
info!(
"Created auto-thread {} for message {}",
thread_name, message.platform_message_id
);
effective_thread_id = Some(new_thread_id);
}
Err(e) => {
warn!("Failed to create auto-thread: {}", e);
}
}
}
// Resolve final thread_id reference used by all downstream send_response calls
let thread_id = effective_thread_id.as_deref();
// Handle commands first (early return)
if let ChannelContent::Command { ref name, ref args } = message.content {
let result = handle_command(
@@ -858,6 +898,93 @@ async fn dispatch_message(
return;
}
// Multipart: flatten children into LLM content blocks. If any image
// succeeds, dispatch as multimodal; otherwise fall through to the text
// path (Multipart arm in the match below builds the combined descriptor).
if let ChannelContent::Multipart(parts) = &message.content {
let mut blocks: Vec<ContentBlock> = Vec::new();
for part in parts {
debug_assert!(
!matches!(part, ChannelContent::Multipart(_)),
"nested Multipart in ChannelContent — adapters should produce flat lists"
);
match part {
ChannelContent::Text(t) => blocks.push(ContentBlock::Text {
text: t.clone(),
provider_metadata: None,
}),
ChannelContent::Image { url, caption } => {
let mut img = download_image_to_blocks(url, caption.as_deref()).await;
blocks.append(&mut img);
}
ChannelContent::File { url, filename, .. } => {
blocks.push(ContentBlock::Text {
text: format!("[User sent a file ({filename}): {url}]"),
provider_metadata: None,
});
}
ChannelContent::Voice {
url,
duration_seconds,
} => {
blocks.push(ContentBlock::Text {
text: format!("[User sent a voice message ({duration_seconds}s): {url}]"),
provider_metadata: None,
});
}
ChannelContent::Location { lat, lon } => {
blocks.push(ContentBlock::Text {
text: format!("[User shared location: {lat}, {lon}]"),
provider_metadata: None,
});
}
ChannelContent::FileData { filename, .. } => {
blocks.push(ContentBlock::Text {
text: format!("[User sent a local file: {filename}]"),
provider_metadata: None,
});
}
// Commands aren't expected inside Multipart, but render as
// text rather than drop the message if one slips through.
ChannelContent::Command { name, args } => {
blocks.push(ContentBlock::Text {
text: format!("/{name} {}", args.join(" ")),
provider_metadata: None,
});
}
// Defensive: debug_assert above catches this in dev; ignore
// gracefully in release.
ChannelContent::Multipart(_) => {}
}
}
if blocks
.iter()
.any(|b| matches!(b, ContentBlock::Image { .. }))
{
let prefix_style = overrides
.as_ref()
.map(|o| o.prefix_agent_name)
.unwrap_or(PrefixStyle::Off);
dispatch_with_blocks(
blocks,
message,
handle,
router,
adapter,
adapter_arc,
ct_str,
thread_id,
output_format,
lifecycle_reactions,
prefix_style,
)
.await;
return;
}
// No image blocks — fall through to text path below.
}
// For images: download, base64 encode, and send as multimodal content blocks
if let ChannelContent::Image {
ref url,
@@ -909,6 +1036,7 @@ async fn dispatch_message(
ChannelContent::File {
ref url,
ref filename,
..
} => {
format!("[User sent a file ({filename}): {url}]")
}
@@ -924,6 +1052,37 @@ async fn dispatch_message(
ChannelContent::FileData { ref filename, .. } => {
format!("[User sent a local file: {filename}]")
}
ChannelContent::Multipart(parts) => parts
.iter()
.map(|p| match p {
ChannelContent::Text(t) => t.clone(),
ChannelContent::Image { url, caption } => match caption {
Some(c) => format!("[User sent a photo: {url}]\nCaption: {c}"),
None => format!("[User sent a photo: {url}]"),
},
ChannelContent::File { url, filename, .. } => {
format!("[User sent a file ({filename}): {url}]")
}
ChannelContent::Voice {
url,
duration_seconds,
} => format!("[User sent a voice message ({duration_seconds}s): {url}]"),
ChannelContent::Location { lat, lon } => {
format!("[User shared location: {lat}, {lon}]")
}
ChannelContent::FileData { filename, .. } => {
format!("[User sent a local file: {filename}]")
}
ChannelContent::Command { name, args } => {
format!("/{name} {}", args.join(" "))
}
// Nesting is rejected by adapters; emit empty so the join
// doesn't insert spurious separators.
ChannelContent::Multipart(_) => String::new(),
})
.filter(|s| !s.is_empty())
.collect::<Vec<_>>()
.join("\n"),
};
// Check if it's a slash command embedded in text (e.g. "/agents")
@@ -1029,14 +1188,36 @@ async fn dispatch_message(
// Route to agent (standard path).
// Use sender_user_id() so user-keyed bindings (peer_id) match for adapters like
// Discord/Slack where sender.platform_id is the channel ID, not the user ID.
// Pass the channel/conversation ID separately so bindings with `channel_id`
// can match (e.g. "messages in Discord channel X → agent Y").
let agent_id = router.resolve_with_channel_id(
&message.channel,
sender_user_id(message),
message.sender.openfang_user.as_deref(),
sender_channel_id(message),
);
// Use resolve_with_context so channel_id-scoped (and guild_id-scoped)
// bindings can route per channel — see binding_context_for() for the
// single-source-of-truth allowlist.
//
// Issue #780: when the adapter stamped a per-thread target agent in
// metadata (e.g. Telegram forum-topic routing via `thread_routes`), prefer
// it over the standard router so operators can scope topics to specific
// agents from config.toml.
let target_agent_name = message
.metadata
.get("target_agent_name")
.and_then(|v| v.as_str());
let routed_by_name = if let Some(name) = target_agent_name {
match handle.find_agent_by_name(name).await {
Ok(Some(id)) => Some(id),
_ => None,
}
} else {
None
};
let binding_ctx = binding_context_for(message);
let agent_id = routed_by_name.or_else(|| {
router.resolve_with_context(
&message.channel,
sender_user_id(message),
message.sender.openfang_user.as_deref(),
&binding_ctx,
)
});
let agent_id = match agent_id {
Some(id) => id,
@@ -1381,6 +1562,10 @@ fn media_type_from_url(url: &str) -> String {
/// Download an image from a URL and build content blocks for multimodal LLM input.
///
/// Accepts both `http(s)://` URLs (fetched via reqwest) and `file://` URLs
/// (read from local disk — used by the channel inbox materialization path so
/// agents see a stable local path even after a Discord CDN URL has expired).
///
/// Returns a `Vec<ContentBlock>` containing an image block (base64-encoded) and
/// optionally a text block for the caption. If the download fails, returns a
/// text-only block describing the failure.
@@ -1390,38 +1575,79 @@ async fn download_image_to_blocks(url: &str, caption: Option<&str>) -> Vec<Conte
// 5 MB limit to prevent memory abuse from oversized images
const MAX_IMAGE_BYTES: usize = 5 * 1024 * 1024;
let client = reqwest::Client::new();
let resp = match client.get(url).send().await {
Ok(r) => r,
Err(e) => {
warn!("Failed to download image from channel: {e}");
return vec![ContentBlock::Text {
text: format!("[Image download failed: {e}]"),
provider_metadata: None,
}];
}
};
// Branch on URL scheme: file:// reads from local disk, everything else
// goes through HTTP. We unify both paths into (bytes, header_type) before
// the size/magic-byte logic below.
let (bytes, header_type): (Vec<u8>, Option<String>) =
if let Some(path) = url.strip_prefix("file://") {
// file:// — local read. No content-type header to honor; magic-byte
// sniffing and URL extension fallback do all the work. We don't
// percent-decode: the inbox writer controls filenames and avoids
// characters that would need encoding.
match tokio::fs::read(path).await {
Ok(b) => (b, None),
Err(e) => {
warn!("Failed to read image from local path {path}: {e}");
return vec![ContentBlock::Text {
text: format!("[Image read failed: {e}]"),
provider_metadata: None,
}];
}
}
} else {
// Build the client with transparent decompression DISABLED. Discord's
// CDN edges occasionally advertise `content-encoding: gzip` (or br)
// on PNG/JPEG passthroughs while the body is the raw, uncompressed
// image bytes. With the default reqwest client (gzip/deflate/brotli
// features enabled at the workspace level), this causes the
// decompression layer to choke on the image header and reqwest
// returns "error decoding response body" only on `bytes().await`,
// not on `send()`. Forcing identity encoding sidesteps the whole
// class of CDN content-encoding-flapping bugs. We also set a UA
// (some CDNs 403 clients without one) and a 30s timeout aligned
// with the upstream 5 MB cap.
let client = reqwest::Client::builder()
.no_gzip()
.no_deflate()
.no_brotli()
.user_agent("openfang/0.1 (+https://openfang.ai)")
.timeout(std::time::Duration::from_secs(30))
.build()
.unwrap_or_else(|_| reqwest::Client::new());
let resp = match client.get(url).send().await {
Ok(r) => r,
Err(e) => {
warn!("Failed to download image from channel: {e}");
return vec![ContentBlock::Text {
text: format!("[Image download failed: {e}]"),
provider_metadata: None,
}];
}
};
// Detect media type from Content-Type header — but only trust it if it's
// actually an image/* type. Many APIs (Telegram, S3 pre-signed URLs) return
// `application/octet-stream` for all files, which breaks vision.
let header_type = resp
.headers()
.get("content-type")
.and_then(|v| v.to_str().ok())
.map(|ct| ct.split(';').next().unwrap_or(ct).trim().to_string())
.filter(|ct| ct.starts_with("image/"));
// Detect media type from Content-Type header — but only trust it if
// it's actually an image/* type. Many APIs (Telegram, S3 pre-signed
// URLs) return `application/octet-stream` for all files, which
// breaks vision.
let header_type = resp
.headers()
.get("content-type")
.and_then(|v| v.to_str().ok())
.map(|ct| ct.split(';').next().unwrap_or(ct).trim().to_string())
.filter(|ct| ct.starts_with("image/"));
let bytes = match resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn!("Failed to read image bytes: {e}");
return vec![ContentBlock::Text {
text: format!("[Image read failed: {e}]"),
provider_metadata: None,
}];
}
};
let bytes = match resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn!("Failed to read image bytes: {e}");
return vec![ContentBlock::Text {
text: format!("[Image read failed: {e}]"),
provider_metadata: None,
}];
}
};
(bytes.to_vec(), header_type)
};
// Three-tier media type detection:
// 1. Trusted Content-Type header (only if image/*)
@@ -1485,12 +1711,13 @@ async fn dispatch_with_blocks(
) {
// Route to agent (same logic as text path).
// Use sender_user_id() so user-keyed bindings match for Discord/Slack;
// pass channel_id so per-room bindings match too.
let agent_id = router.resolve_with_channel_id(
// resolve_with_context lets channel_id-scoped bindings match per room.
let binding_ctx = binding_context_for(message);
let agent_id = router.resolve_with_context(
&message.channel,
sender_user_id(message),
message.sender.openfang_user.as_deref(),
sender_channel_id(message),
&binding_ctx,
);
let agent_id = match agent_id {
@@ -2190,6 +2417,122 @@ mod tests {
assert_eq!(GroupPolicy::default(), GroupPolicy::MentionOnly);
}
// -- binding_context_for / ChannelMessage::channel_id() coverage --
//
// These tests pin the routing-time behavior so future adapter additions to
// CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL cannot silently regress the bridge.
fn make_msg_for_ctx(
channel: ChannelType,
platform_id: &str,
metadata: Vec<(&str, serde_json::Value)>,
) -> ChannelMessage {
let mut md = std::collections::HashMap::new();
for (k, v) in metadata {
md.insert(k.to_string(), v);
}
ChannelMessage {
channel,
platform_message_id: "msg-1".to_string(),
sender: crate::types::ChannelUser {
platform_id: platform_id.to_string(),
display_name: "Tester".to_string(),
openfang_user: None,
},
content: ChannelContent::Text("hi".to_string()),
target_agent: None,
timestamp: chrono::Utc::now(),
is_group: true,
thread_id: None,
metadata: md,
}
}
#[test]
fn test_binding_context_for_discord_uses_platform_id_as_channel() {
let msg = make_msg_for_ctx(ChannelType::Discord, "1234567890", vec![]);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.channel, "discord");
assert_eq!(ctx.channel_id.as_deref(), Some("1234567890"));
}
#[test]
fn test_binding_context_for_telegram_uses_platform_id_as_channel() {
// Regression guard: Telegram is on the channel-ID allowlist.
let msg = make_msg_for_ctx(ChannelType::Telegram, "-100123", vec![]);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.channel, "telegram");
assert_eq!(ctx.channel_id.as_deref(), Some("-100123"));
}
#[test]
fn test_binding_context_for_matrix_uses_room_id_from_platform_id() {
let msg = make_msg_for_ctx(ChannelType::Matrix, "!room:server.tld", vec![]);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.channel_id.as_deref(), Some("!room:server.tld"));
}
#[test]
fn test_binding_context_for_custom_supported_adapter() {
// Custom("twitch") is on the allowlist.
let msg = make_msg_for_ctx(
ChannelType::Custom("twitch".to_string()),
"channel-foo",
vec![],
);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.channel, "twitch");
assert_eq!(ctx.channel_id.as_deref(), Some("channel-foo"));
}
#[test]
fn test_binding_context_for_user_id_adapter_returns_none() {
// Reddit's platform_id is the post author, not a subreddit/conversation.
// The bridge must not surface that as `channel_id` (would silently match
// user-scoped bindings against a user ID).
let msg = make_msg_for_ctx(
ChannelType::Custom("reddit".to_string()),
"u/some-user",
vec![],
);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.channel, "reddit");
assert!(ctx.channel_id.is_none());
// peer_id still falls through to platform_id (sender_user_id default).
assert_eq!(ctx.peer_id, "u/some-user");
}
#[test]
fn test_binding_context_for_metadata_fallback() {
// For non-allowlisted adapters, metadata["channel_id"] is the
// documented escape hatch — verify the bridge honors it.
let msg = make_msg_for_ctx(
ChannelType::Custom("reddit".to_string()),
"u/some-user",
vec![("channel_id", serde_json::json!("r/rust"))],
);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.channel_id.as_deref(), Some("r/rust"));
}
#[test]
fn test_binding_context_for_metadata_guild_id() {
let msg = make_msg_for_ctx(
ChannelType::Discord,
"1234567890",
vec![("guild_id", serde_json::json!("99999"))],
);
let ctx = binding_context_for(&msg);
assert_eq!(ctx.guild_id.as_deref(), Some("99999"));
}
#[test]
fn test_channel_message_channel_id_email_returns_none() {
// Email's platform_id is the sender address — not a channel.
let msg = make_msg_for_ctx(ChannelType::Email, "alice@example.com", vec![]);
assert!(msg.channel_id().is_none());
}
#[test]
fn test_channel_type_str() {
assert_eq!(channel_type_str(&ChannelType::Telegram), "telegram");
+626 -26
View File
@@ -8,7 +8,7 @@ use crate::types::{
};
use async_trait::async_trait;
use futures::{SinkExt, Stream, StreamExt};
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::pin::Pin;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
@@ -22,6 +22,10 @@ const DISCORD_API_BASE: &str = "https://discord.com/api/v10";
const MAX_BACKOFF: Duration = Duration::from_secs(60);
const INITIAL_BACKOFF: Duration = Duration::from_secs(1);
const DISCORD_MSG_LIMIT: usize = 2000;
/// Maximum number of seen message IDs kept in the dedup set.
/// MESSAGE_UPDATE (embed resolution) events arrive within seconds of the
/// original CREATE; entries older than this cap are safe to discard.
const MAX_DEDUP_MSG_IDS: usize = 2_000;
/// Discord Gateway opcodes.
mod opcode {
@@ -56,6 +60,8 @@ pub struct DiscordAdapter {
allowed_users: Vec<String>,
ignore_bots: bool,
intents: u64,
/// Auto-thread behavior: "true", "false", or "smart"
auto_thread: String,
shutdown_tx: Arc<watch::Sender<bool>>,
shutdown_rx: watch::Receiver<bool>,
/// Bot's own user ID (populated after READY event).
@@ -64,6 +70,13 @@ pub struct DiscordAdapter {
session_id: Arc<RwLock<Option<String>>>,
/// Resume gateway URL.
resume_gateway_url: Arc<RwLock<Option<String>>>,
/// Thread channel IDs created by this bot (thread_id → parent_channel_id).
/// Used to detect when incoming messages are inside a bot-created thread.
created_thread_ids: Arc<RwLock<HashMap<String, String>>>,
/// Message IDs seen via MESSAGE_CREATE (used to drop duplicate MESSAGE_UPDATE events).
/// Populated immediately when MESSAGE_CREATE is forwarded — before bridge processing —
/// to eliminate the race window where MESSAGE_UPDATE arrives before thread creation completes.
threaded_message_ids: Arc<RwLock<HashSet<String>>>,
}
impl DiscordAdapter {
@@ -73,6 +86,7 @@ impl DiscordAdapter {
allowed_users: Vec<String>,
ignore_bots: bool,
intents: u64,
auto_thread: String,
) -> Self {
let (shutdown_tx, shutdown_rx) = watch::channel(false);
Self {
@@ -82,11 +96,14 @@ impl DiscordAdapter {
allowed_users,
ignore_bots,
intents,
auto_thread,
shutdown_tx: Arc::new(shutdown_tx),
shutdown_rx,
bot_user_id: Arc::new(RwLock::new(None)),
session_id: Arc::new(RwLock::new(None)),
resume_gateway_url: Arc::new(RwLock::new(None)),
created_thread_ids: Arc::new(RwLock::new(HashMap::new())),
threaded_message_ids: Arc::new(RwLock::new(HashSet::new())),
}
}
@@ -147,6 +164,79 @@ impl DiscordAdapter {
.await?;
Ok(())
}
/// Create a thread from a message in a Discord channel.
async fn api_create_thread(
&self,
channel_id: &str,
message_id: &str,
name: &str,
) -> Result<String, Box<dyn std::error::Error>> {
let url = format!(
"{DISCORD_API_BASE}/channels/{channel_id}/messages/{message_id}/threads",
channel_id = channel_id,
message_id = message_id
);
let body = serde_json::json!({
"name": name,
"auto_archive_duration": 1440 // 24 hours
});
let resp = self
.client
.post(&url)
.header("Authorization", format!("Bot {}", self.token.as_str()))
.json(&body)
.send()
.await?;
if !resp.status().is_success() {
let body_text = resp.text().await.unwrap_or_default();
return Err(format!("Discord createThread failed: {}", body_text).into());
}
let response: serde_json::Value = resp.json().await?;
let thread_id = response["id"].as_str().unwrap_or("").to_string();
// Track thread_id → parent channel_id so we can recognise messages
// that arrive inside this thread.
if !thread_id.is_empty() {
self.created_thread_ids
.write()
.await
.insert(thread_id.clone(), channel_id.to_string());
}
Ok(thread_id)
}
/// Send a message to an existing thread.
/// Discord threads are channels — post directly to channels/{thread_id}/messages.
async fn api_send_thread_message(
&self,
_channel_id: &str,
thread_id: &str,
text: &str,
) -> Result<(), Box<dyn std::error::Error>> {
let url = format!("{DISCORD_API_BASE}/channels/{thread_id}/messages");
let chunks = split_message(text, DISCORD_MSG_LIMIT);
for chunk in chunks {
let body = serde_json::json!({ "content": chunk });
let resp = self
.client
.post(&url)
.header("Authorization", format!("Bot {}", self.token.as_str()))
.json(&body)
.send()
.await?;
if !resp.status().is_success() {
let body_text = resp.text().await.unwrap_or_default();
warn!("Discord sendThreadMessage failed: {body_text}");
}
}
Ok(())
}
}
#[async_trait]
@@ -159,6 +249,33 @@ impl ChannelAdapter for DiscordAdapter {
ChannelType::Discord
}
async fn should_auto_thread(&self, message: &ChannelMessage) -> Option<String> {
// Only auto-thread in group channels (servers), not DMs
if !message.is_group {
return None;
}
// Check auto_thread mode
match self.auto_thread.as_str() {
"true" => Some(thread_name_from_message(message)),
"false" => None,
"smart" => {
// Only create thread if bot was @mentioned
let was_mentioned = message
.metadata
.get("was_mentioned")
.and_then(|v| v.as_bool())
.unwrap_or(false);
if was_mentioned {
Some(thread_name_from_message(message))
} else {
None
}
}
_ => None,
}
}
async fn start(
&self,
) -> Result<Pin<Box<dyn Stream<Item = ChannelMessage> + Send>>, Box<dyn std::error::Error>>
@@ -176,6 +293,8 @@ impl ChannelAdapter for DiscordAdapter {
let bot_user_id = self.bot_user_id.clone();
let session_id_store = self.session_id.clone();
let resume_url_store = self.resume_gateway_url.clone();
let created_thread_ids = self.created_thread_ids.clone();
let threaded_message_ids = self.threaded_message_ids.clone();
let mut shutdown = self.shutdown_rx.clone();
tokio::spawn(async move {
@@ -414,19 +533,66 @@ impl ChannelAdapter for DiscordAdapter {
&allowed_guilds,
&allowed_users,
ignore_bots,
&created_thread_ids,
)
.await
{
// MESSAGE_UPDATE must be suppressed if we already
// forwarded a MESSAGE_CREATE for this message ID.
// The check uses `seen_message_ids` (tracked below)
// which is populated the moment MESSAGE_CREATE is
// forwarded — before the bridge even processes it.
// This closes the race window where MESSAGE_UPDATE
// arrives before adapter.create_thread() completes.
if event_name == "MESSAGE_UPDATE"
&& threaded_message_ids
.read()
.await
.contains(&msg.platform_message_id)
{
debug!(
"Discord MESSAGE_UPDATE skipped (already seen {})",
msg.platform_message_id
);
continue;
}
debug!(
"Discord {event_name} from {}: {:?}",
msg.sender.display_name, msg.content
);
// Mark this message as seen immediately so any
// concurrent or subsequent MESSAGE_UPDATE is dropped.
if event_name == "MESSAGE_CREATE" {
threaded_message_ids
.write()
.await
.insert(msg.platform_message_id.clone());
}
if tx.send(msg).await.is_err() {
return;
}
}
}
"THREAD_DELETE" | "CHANNEL_DELETE" => {
// Clean up tracking when a thread is deleted so the
// next message in the parent channel is treated fresh.
if let Some(tid) = d["id"].as_str() {
created_thread_ids.write().await.remove(tid);
// Prune the dedup set to prevent unbounded growth.
// Entries older than MAX_DEDUP_MSG_IDS are safe to
// discard — embed UPDATE events arrive within seconds.
let mut ids = threaded_message_ids.write().await;
if ids.len() > MAX_DEDUP_MSG_IDS {
ids.clear();
}
debug!("Discord thread/channel deleted: {tid}");
}
}
"RESUMED" => {
info!("Discord session resumed successfully");
}
@@ -532,12 +698,123 @@ impl ChannelAdapter for DiscordAdapter {
self.api_send_typing(&user.platform_id).await
}
async fn send_in_thread(
&self,
user: &ChannelUser,
content: ChannelContent,
thread_id: &str,
) -> Result<(), Box<dyn std::error::Error>> {
let channel_id = &user.platform_id;
match content {
ChannelContent::Text(text) => {
self.api_send_thread_message(channel_id, thread_id, &text)
.await?;
}
_ => {
self.api_send_thread_message(channel_id, thread_id, "(Unsupported content type)")
.await?;
}
}
Ok(())
}
async fn create_thread(
&self,
user: &ChannelUser,
message_id: &str,
thread_name: &str,
) -> Result<String, Box<dyn std::error::Error>> {
let channel_id = &user.platform_id;
let thread_id = self
.api_create_thread(channel_id, message_id, thread_name)
.await?;
// Also ensure the message_id is marked as seen (belt-and-suspenders:
// the gateway loop already inserts on MESSAGE_CREATE, but keep this
// in case create_thread is ever called from another path).
self.threaded_message_ids
.write()
.await
.insert(message_id.to_string());
Ok(thread_id)
}
async fn stop(&self) -> Result<(), Box<dyn std::error::Error>> {
let _ = self.shutdown_tx.send(true);
Ok(())
}
}
/// Maximum byte size for an attachment to be classified as a vision-eligible
/// image. Anthropic's image content blocks are capped at 5 MB; oversize images
/// fall through to `File` so the bridge passes the URL as text instead of
/// attempting an inline image block.
const VISION_IMAGE_MAX_BYTES: u64 = 5 * 1024 * 1024;
/// Best-effort MIME inference from a filename extension. Used as a fallback
/// when Discord's `content_type` field is missing or empty (we've observed
/// this on some bot-relayed attachments).
fn mime_from_extension(filename: &str) -> Option<&'static str> {
let ext = filename.rsplit('.').next()?.to_ascii_lowercase();
match ext.as_str() {
"jpg" | "jpeg" => Some("image/jpeg"),
"png" => Some("image/png"),
"gif" => Some("image/gif"),
"webp" => Some("image/webp"),
"heic" => Some("image/heic"),
"heif" => Some("image/heif"),
"pdf" => Some("application/pdf"),
"txt" => Some("text/plain"),
"md" => Some("text/markdown"),
"json" => Some("application/json"),
"mp4" => Some("video/mp4"),
"mov" => Some("video/quicktime"),
"mp3" => Some("audio/mpeg"),
"wav" => Some("audio/wav"),
"ogg" => Some("audio/ogg"),
_ => None,
}
}
/// Classify a single Discord attachment JSON object into a `ChannelContent`
/// block. Vision-eligible image MIME types (jpeg/png/gif/webp) under
/// `VISION_IMAGE_MAX_BYTES` become `Image`; everything else becomes `File`
/// (URL-pass-through; the bridge will surface it as a text descriptor in v1).
///
/// MIME resolution chain: `attachments[].content_type` (if non-empty) →
/// extension lookup → `application/octet-stream`.
fn classify_discord_attachment(att: &serde_json::Value) -> ChannelContent {
let url = att["url"].as_str().unwrap_or("").to_string();
let filename = att["filename"].as_str().unwrap_or("file").to_string();
let size = att["size"].as_u64();
let resolved_mime: String = att["content_type"]
.as_str()
.filter(|s| !s.is_empty())
.map(str::to_string)
.or_else(|| mime_from_extension(&filename).map(str::to_string))
.unwrap_or_else(|| "application/octet-stream".to_string());
let is_vision_mime = matches!(
resolved_mime.as_str(),
"image/jpeg" | "image/png" | "image/gif" | "image/webp"
);
// If size is unknown, optimistically allow the image — the bridge will
// surface a 4xx if Anthropic rejects it, which is better than silently
// demoting to a text URL.
let within_vision_limit = size.map(|s| s <= VISION_IMAGE_MAX_BYTES).unwrap_or(true);
if is_vision_mime && within_vision_limit {
ChannelContent::Image { url, caption: None }
} else {
ChannelContent::File {
url,
filename,
mime: Some(resolved_mime),
size,
}
}
}
/// Parse a Discord MESSAGE_CREATE or MESSAGE_UPDATE payload into a `ChannelMessage`.
async fn parse_discord_message(
d: &serde_json::Value,
@@ -545,7 +822,13 @@ async fn parse_discord_message(
allowed_guilds: &[String],
allowed_users: &[String],
ignore_bots: bool,
created_thread_ids: &Arc<RwLock<HashMap<String, String>>>,
) -> Option<ChannelMessage> {
// Diagnostic: dump the raw Discord payload so we can ground attachment
// parsing in real JSON. Gated by RUST_LOG; silent at default `info` level.
// Enable with: RUST_LOG=openfang_channels::discord=debug
debug!(target: "openfang_channels::discord", payload = %d, "discord raw message payload");
let author = d.get("author")?;
let author_id = author["id"].as_str()?;
@@ -577,12 +860,22 @@ async fn parse_discord_message(
}
let content_text = d["content"].as_str().unwrap_or("");
if content_text.is_empty() {
return None;
}
let channel_id = d["channel_id"].as_str()?;
let message_id = d["id"].as_str().unwrap_or("0");
// Detect if this message is inside a bot-created thread.
// In Discord, a thread is its own channel — channel_id will be the thread's ID.
// If so, use the parent channel as platform_id and set thread_id so that:
// (a) auto-thread logic is skipped (message.thread_id.is_some())
// (b) responses are sent back into the same thread
let (effective_channel_id, parsed_thread_id) = {
let threads = created_thread_ids.read().await;
if let Some(parent_channel_id) = threads.get(channel_id) {
(parent_channel_id.clone(), Some(channel_id.to_string()))
} else {
(channel_id.to_string(), None)
}
};
let username = author["username"].as_str().unwrap_or("Unknown");
let discriminator = author["discriminator"].as_str().unwrap_or("0000");
let display_name = if discriminator == "0" {
@@ -597,7 +890,8 @@ async fn parse_discord_message(
.map(|dt| dt.with_timezone(&chrono::Utc))
.unwrap_or_else(chrono::Utc::now);
// Parse commands (messages starting with /)
// Parse commands (messages starting with /). Commands do not carry
// attachments in v1; attachment processing only runs in the non-command path.
let content = if content_text.starts_with('/') {
let parts: Vec<&str> = content_text.splitn(2, ' ').collect();
let cmd_name = &parts[0][1..];
@@ -611,7 +905,50 @@ async fn parse_discord_message(
args,
}
} else {
ChannelContent::Text(content_text.to_string())
let attachment_blocks: Vec<ChannelContent> = d["attachments"]
.as_array()
.map(|arr| arr.iter().map(classify_discord_attachment).collect())
.unwrap_or_default();
match (content_text.is_empty(), attachment_blocks.len()) {
// No text, no attachments → nothing to ingest.
(true, 0) => return None,
// Text only.
(false, 0) => ChannelContent::Text(content_text.to_string()),
// Single attachment, no caption.
(true, 1) => attachment_blocks.into_iter().next().unwrap(),
// Single attachment + caption: emit Multipart with the caption as
// a sibling Text block. This keeps the caption visible to providers
// that flatten content to text only (e.g. claude-code/*, which
// currently drops Image blocks) — the user gets a coherent
// text-only response instead of a hallucination. Vision-capable
// providers see the same blocks and dispatch multimodally.
(false, 1) => {
let block = attachment_blocks.into_iter().next().unwrap();
let normalized = match block {
// Drop any caption that classify_discord_attachment may have
// attached; the sibling Text block is now the caption.
ChannelContent::Image { url, caption: _ } => {
ChannelContent::Image { url, caption: None }
}
other => other,
};
ChannelContent::Multipart(vec![
ChannelContent::Text(content_text.to_string()),
normalized,
])
}
// Multiple attachments, no caption.
(true, _) => ChannelContent::Multipart(attachment_blocks),
// Multiple attachments + caption: text first, then attachments
// (matches Discord's visual ordering: text above attachments).
(false, _) => {
let mut blocks = Vec::with_capacity(attachment_blocks.len() + 1);
blocks.push(ChannelContent::Text(content_text.to_string()));
blocks.extend(attachment_blocks);
ChannelContent::Multipart(blocks)
}
}
};
// Determine if this is a group message (guild_id present = server channel)
@@ -644,7 +981,7 @@ async fn parse_discord_message(
channel: ChannelType::Discord,
platform_message_id: message_id.to_string(),
sender: ChannelUser {
platform_id: channel_id.to_string(),
platform_id: effective_channel_id,
display_name,
openfang_user: None,
},
@@ -652,15 +989,50 @@ async fn parse_discord_message(
target_agent: None,
timestamp,
is_group,
thread_id: None,
thread_id: parsed_thread_id,
metadata,
})
}
/// Build a Discord thread name from the message content.
/// Strips @mention prefixes (`<@...>`), trims whitespace, and truncates to
/// Discord's 100-character thread name limit. Falls back to the sender's
/// display name if the message has no usable text (e.g. image-only).
fn thread_name_from_message(message: &ChannelMessage) -> String {
let raw = match &message.content {
ChannelContent::Text(t) => t.clone(),
ChannelContent::Image { caption, .. } => caption.clone().unwrap_or_default(),
_ => String::new(),
};
// Strip leading Discord mention tokens (<@id> / <@!id>)
let stripped = regex_lite::Regex::new(r"^(<@!?\d+>\s*)+")
.map(|re| re.replace(&raw, "").into_owned())
.unwrap_or(raw);
let trimmed = stripped.trim().to_string();
if trimmed.is_empty() {
return message.sender.display_name.clone();
}
// Truncate to Discord's 100-char limit
if trimmed.chars().count() <= 100 {
trimmed
} else {
trimmed.chars().take(97).collect::<String>() + ""
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Convenience helper: empty thread-tracking map for tests that don't exercise threading.
fn empty_threads() -> Arc<RwLock<HashMap<String, String>>> {
Arc::new(RwLock::new(HashMap::new()))
}
#[tokio::test]
async fn test_parse_discord_message_basic() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
@@ -677,7 +1049,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true)
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert_eq!(msg.channel, ChannelType::Discord);
@@ -701,7 +1073,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true).await;
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads()).await;
assert!(msg.is_none());
}
@@ -721,7 +1093,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true).await;
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads()).await;
assert!(msg.is_none());
}
@@ -742,7 +1114,7 @@ mod tests {
});
// With ignore_bots=false, other bots' messages should be allowed
let msg = parse_discord_message(&d, &bot_id, &[], &[], false).await;
let msg = parse_discord_message(&d, &bot_id, &[], &[], false, &empty_threads()).await;
assert!(msg.is_some());
let msg = msg.unwrap();
assert_eq!(msg.sender.display_name, "somebot");
@@ -766,7 +1138,7 @@ mod tests {
});
// Even with ignore_bots=false, the bot's own messages must still be filtered
let msg = parse_discord_message(&d, &bot_id, &[], &[], false).await;
let msg = parse_discord_message(&d, &bot_id, &[], &[], false, &empty_threads()).await;
assert!(msg.is_none());
}
@@ -787,12 +1159,20 @@ mod tests {
});
// Not in allowed guilds
let msg =
parse_discord_message(&d, &bot_id, &["111".into(), "222".into()], &[], true).await;
let msg = parse_discord_message(
&d,
&bot_id,
&["111".into(), "222".into()],
&[],
true,
&empty_threads(),
)
.await;
assert!(msg.is_none());
// In allowed guilds
let msg = parse_discord_message(&d, &bot_id, &["999".into()], &[], true).await;
let msg =
parse_discord_message(&d, &bot_id, &["999".into()], &[], true, &empty_threads()).await;
assert!(msg.is_some());
}
@@ -811,7 +1191,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true)
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match &msg.content {
@@ -838,7 +1218,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true).await;
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads()).await;
assert!(msg.is_none());
}
@@ -857,7 +1237,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true)
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert_eq!(msg.sender.display_name, "alice#1234");
@@ -881,7 +1261,7 @@ mod tests {
});
// MESSAGE_UPDATE uses the same parse function as MESSAGE_CREATE
let msg = parse_discord_message(&d, &bot_id, &[], &[], true)
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert_eq!(msg.channel, ChannelType::Discord);
@@ -912,16 +1292,25 @@ mod tests {
&[],
&["user111".into(), "user222".into()],
true,
&empty_threads(),
)
.await;
assert!(msg.is_none());
// In allowed users
let msg = parse_discord_message(&d, &bot_id, &[], &["user999".into()], true).await;
let msg = parse_discord_message(
&d,
&bot_id,
&[],
&["user999".into()],
true,
&empty_threads(),
)
.await;
assert!(msg.is_some());
// Empty allowed_users = allow all
let msg = parse_discord_message(&d, &bot_id, &[], &[], true).await;
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads()).await;
assert!(msg.is_some());
}
@@ -944,7 +1333,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true)
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert!(msg.is_group);
@@ -967,7 +1356,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg2 = parse_discord_message(&d2, &bot_id, &[], &[], true)
let msg2 = parse_discord_message(&d2, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert!(msg2.is_group);
@@ -989,7 +1378,7 @@ mod tests {
"timestamp": "2024-01-01T00:00:00+00:00"
});
let msg = parse_discord_message(&d, &bot_id, &[], &[], true)
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert!(!msg.is_group);
@@ -1028,8 +1417,219 @@ mod tests {
vec![],
true,
37376,
"true".to_string(),
);
assert_eq!(adapter.name(), "discord");
assert_eq!(adapter.channel_type(), ChannelType::Discord);
}
// -- Multipart / attachment parsing tests (commit 4) ----------------------
fn att(filename: &str, content_type: Option<&str>, size: u64) -> serde_json::Value {
let mut obj = serde_json::json!({
"url": format!("https://cdn.discordapp.com/attachments/1/2/{filename}"),
"filename": filename,
"size": size,
});
if let Some(ct) = content_type {
obj["content_type"] = serde_json::Value::String(ct.to_string());
}
obj
}
fn payload_with(content: &str, attachments: Vec<serde_json::Value>) -> serde_json::Value {
serde_json::json!({
"id": "msg1",
"channel_id": "ch1",
"content": content,
"author": {
"id": "user456",
"username": "alice",
"discriminator": "0",
"bot": false
},
"timestamp": "2024-01-01T00:00:00+00:00",
"attachments": attachments,
})
}
#[tokio::test]
async fn test_parse_image_only_no_caption() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with("", vec![att("photo.png", Some("image/png"), 100_000)]);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::Image { caption, url } => {
assert!(caption.is_none());
assert!(url.contains("photo.png"));
}
other => panic!("expected Image, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_image_with_caption() {
// Single image + caption is emitted as Multipart([Text, Image]) so the
// caption survives providers that flatten content blocks to text only
// (e.g. claude-code/*). The Image carries no caption of its own; the
// sibling Text block IS the caption.
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with(
"look at this",
vec![att("photo.jpg", Some("image/jpeg"), 50_000)],
);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::Multipart(parts) => {
assert_eq!(parts.len(), 2);
assert!(matches!(&parts[0], ChannelContent::Text(t) if t == "look at this"));
match &parts[1] {
ChannelContent::Image { caption, url } => {
assert!(
caption.is_none(),
"image caption should be None; the sibling Text block is the caption"
);
assert!(url.contains("photo.jpg"));
}
other => panic!("expected Image as second part, got {other:?}"),
}
}
other => panic!("expected Multipart, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_multi_image_no_caption() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with(
"",
vec![
att("a.png", Some("image/png"), 10_000),
att("b.png", Some("image/png"), 20_000),
],
);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::Multipart(parts) => {
assert_eq!(parts.len(), 2);
assert!(parts
.iter()
.all(|p| matches!(p, ChannelContent::Image { .. })));
}
other => panic!("expected Multipart, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_multi_image_with_caption() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with(
"two pics",
vec![
att("a.png", Some("image/png"), 10_000),
att("b.png", Some("image/png"), 20_000),
],
);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::Multipart(parts) => {
assert_eq!(parts.len(), 3);
// Text first, then images.
assert!(matches!(&parts[0], ChannelContent::Text(t) if t == "two pics"));
assert!(matches!(&parts[1], ChannelContent::Image { .. }));
assert!(matches!(&parts[2], ChannelContent::Image { .. }));
}
other => panic!("expected Multipart, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_heic_falls_to_file() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with("", vec![att("photo.heic", Some("image/heic"), 100_000)]);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::File { mime, filename, .. } => {
assert_eq!(filename, "photo.heic");
assert_eq!(mime.as_deref(), Some("image/heic"));
}
other => panic!("expected File, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_oversize_image_falls_to_file() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
// 6 MB exceeds VISION_IMAGE_MAX_BYTES (5 MB).
let d = payload_with(
"",
vec![att("huge.png", Some("image/png"), 6 * 1024 * 1024)],
);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::File {
filename,
mime,
size,
..
} => {
assert_eq!(filename, "huge.png");
assert_eq!(mime.as_deref(), Some("image/png"));
assert_eq!(size, Some(6 * 1024 * 1024));
}
other => panic!("expected File, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_file_with_caption_yields_multipart() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with(
"see attached",
vec![att("doc.pdf", Some("application/pdf"), 200_000)],
);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
match msg.content {
ChannelContent::Multipart(parts) => {
assert_eq!(parts.len(), 2);
assert!(matches!(&parts[0], ChannelContent::Text(t) if t == "see attached"));
assert!(matches!(&parts[1], ChannelContent::File { .. }));
}
other => panic!("expected Multipart, got {other:?}"),
}
}
#[tokio::test]
async fn test_parse_extension_fallback_when_content_type_missing() {
// Discord occasionally omits content_type on bot-relayed attachments;
// we should fall back to the filename extension.
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with("", vec![att("pic.png", None, 50_000)]);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads())
.await
.unwrap();
assert!(matches!(msg.content, ChannelContent::Image { .. }));
}
#[tokio::test]
async fn test_parse_empty_message_with_no_attachments_returns_none() {
let bot_id = Arc::new(RwLock::new(Some("bot123".to_string())));
let d = payload_with("", vec![]);
let msg = parse_discord_message(&d, &bot_id, &[], &[], true, &empty_threads()).await;
assert!(msg.is_none());
}
}
+304 -36
View File
@@ -18,14 +18,20 @@ use zeroize::Zeroizing;
const SYNC_TIMEOUT_MS: u64 = 30000;
const MAX_MESSAGE_LEN: usize = 4096;
/// Shared access + refresh token pair. Tokens are zeroized on drop and rotated
/// in place when MSC2918 refresh succeeds.
type TokenPair = Arc<RwLock<(Zeroizing<String>, Option<Zeroizing<String>>)>>;
/// Matrix channel adapter using the Client-Server API.
pub struct MatrixAdapter {
/// Matrix homeserver URL (e.g., `"https://matrix.org"`).
homeserver_url: String,
/// Bot's user ID (e.g., "@openfang:matrix.org").
user_id: String,
/// SECURITY: Access token is zeroized on drop.
access_token: Zeroizing<String>,
/// SECURITY: Access + refresh tokens are zeroized on drop. Stored behind
/// an RwLock so the sync loop and send paths see rotated tokens after a
/// MSC2918 /refresh call (matrix.org/MAS rotates both tokens every refresh).
tokens: TokenPair,
/// HTTP client.
client: reqwest::Client,
/// Allowed room IDs (empty = all joined rooms).
@@ -40,19 +46,47 @@ pub struct MatrixAdapter {
}
impl MatrixAdapter {
/// Create a new Matrix adapter.
/// Create a new Matrix adapter without a refresh token.
pub fn new(
homeserver_url: String,
user_id: String,
access_token: String,
allowed_rooms: Vec<String>,
auto_accept_invites: bool,
) -> Self {
Self::with_refresh_token(
homeserver_url,
user_id,
access_token,
None,
allowed_rooms,
auto_accept_invites,
)
}
/// Create a new Matrix adapter with an optional refresh token (MSC2918).
///
/// When `refresh_token` is `Some`, the adapter will automatically call
/// `POST /_matrix/client/v3/refresh` on `401 M_UNKNOWN_TOKEN` responses
/// and retry the failed request once. Both tokens rotate on each refresh
/// under Matrix Authentication Service (MAS).
pub fn with_refresh_token(
homeserver_url: String,
user_id: String,
access_token: String,
refresh_token: Option<String>,
allowed_rooms: Vec<String>,
auto_accept_invites: bool,
) -> Self {
let (shutdown_tx, shutdown_rx) = watch::channel(false);
let tokens: TokenPair = Arc::new(RwLock::new((
Zeroizing::new(access_token),
refresh_token.map(Zeroizing::new),
)));
Self {
homeserver_url,
user_id,
access_token: Zeroizing::new(access_token),
tokens,
client: reqwest::Client::new(),
allowed_rooms,
shutdown_tx: Arc::new(shutdown_tx),
@@ -62,6 +96,11 @@ impl MatrixAdapter {
}
}
/// Read the current access token (cloned).
async fn current_access_token(&self) -> String {
self.tokens.read().await.0.as_str().to_string()
}
/// Send a text message to a Matrix room.
async fn api_send_message(
&self,
@@ -81,18 +120,46 @@ impl MatrixAdapter {
"body": chunk,
});
let resp = self
.client
.put(&url)
.bearer_auth(&*self.access_token)
.json(&body)
.send()
.await?;
let mut attempt = 0;
loop {
attempt += 1;
let token = self.current_access_token().await;
let resp = self
.client
.put(&url)
.bearer_auth(&token)
.json(&body)
.send()
.await?;
if resp.status().is_success() {
break;
}
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(format!("Matrix API error {status}: {body}").into());
let body_text = resp.text().await.unwrap_or_default();
// Try a single refresh+retry on M_UNKNOWN_TOKEN (MSC2918).
if attempt == 1
&& status == reqwest::StatusCode::UNAUTHORIZED
&& is_unknown_token_body(&body_text)
{
match try_refresh_tokens(&self.client, &self.homeserver_url, &self.tokens).await
{
Ok(()) => {
info!("Matrix: access token refreshed via MSC2918, retrying send");
continue;
}
Err(e) => {
return Err(format!(
"Matrix API error {status}: {body_text} (refresh failed: {e})"
)
.into());
}
}
}
return Err(format!("Matrix API error {status}: {body_text}").into());
}
}
@@ -103,21 +170,32 @@ impl MatrixAdapter {
async fn validate(&self) -> Result<String, Box<dyn std::error::Error>> {
let url = format!("{}/_matrix/client/v3/account/whoami", self.homeserver_url);
let resp = self
.client
.get(&url)
.bearer_auth(&*self.access_token)
.send()
.await?;
let mut attempt = 0;
loop {
attempt += 1;
let token = self.current_access_token().await;
let resp = self.client.get(&url).bearer_auth(&token).send().await?;
if !resp.status().is_success() {
if resp.status().is_success() {
let body: serde_json::Value = resp.json().await?;
let user_id = body["user_id"].as_str().unwrap_or("unknown").to_string();
return Ok(user_id);
}
let status = resp.status();
let body_text = resp.text().await.unwrap_or_default();
if attempt == 1
&& status == reqwest::StatusCode::UNAUTHORIZED
&& is_unknown_token_body(&body_text)
&& try_refresh_tokens(&self.client, &self.homeserver_url, &self.tokens)
.await
.is_ok()
{
info!("Matrix: access token refreshed via MSC2918, retrying /whoami");
continue;
}
return Err("Matrix authentication failed".into());
}
let body: serde_json::Value = resp.json().await?;
let user_id = body["user_id"].as_str().unwrap_or("unknown").to_string();
Ok(user_id)
}
#[cfg(test)]
@@ -126,6 +204,87 @@ impl MatrixAdapter {
}
}
/// Detect `M_UNKNOWN_TOKEN` errors in a Matrix response body.
///
/// Matrix returns 401 for multiple reasons; we only want to refresh on
/// `M_UNKNOWN_TOKEN` (the access token expired or was revoked). See
/// <https://spec.matrix.org/latest/client-server-api/#soft-logout>.
fn is_unknown_token_body(body: &str) -> bool {
serde_json::from_str::<serde_json::Value>(body)
.ok()
.and_then(|v| v.get("errcode").and_then(|c| c.as_str()).map(String::from))
.map(|c| c == "M_UNKNOWN_TOKEN")
.unwrap_or(false)
}
/// Whether a Matrix 401 body indicates a hard logout (operator must re-login).
///
/// `soft_logout: true` (or absent — default per spec) means the device is still
/// known to the server and a refresh-token grant is valid. `soft_logout: false`
/// means the device was invalidated and the operator must perform a new
/// `m.login.password` flow.
fn is_hard_logout(body: &str) -> bool {
serde_json::from_str::<serde_json::Value>(body)
.ok()
.and_then(|v| v.get("soft_logout").and_then(|s| s.as_bool()))
.map(|soft| !soft)
.unwrap_or(false)
}
/// Call `POST /_matrix/client/v3/refresh` (MSC2918) and rotate the stored tokens.
///
/// On success, replaces the access token and (if the server returned one) the
/// refresh token. MAS (matrix.org since 2025-04-07) rotates the refresh token
/// on every call, so callers must use the new value next time.
async fn try_refresh_tokens(
client: &reqwest::Client,
homeserver: &str,
tokens: &TokenPair,
) -> Result<(), String> {
let refresh_token = {
let guard = tokens.read().await;
match guard.1.as_ref() {
Some(rt) => rt.as_str().to_string(),
None => return Err("no refresh token configured".to_string()),
}
};
let url = format!("{homeserver}/_matrix/client/v3/refresh");
let resp = client
.post(&url)
.json(&serde_json::json!({ "refresh_token": refresh_token }))
.send()
.await
.map_err(|e| format!("refresh request failed: {e}"))?;
let status = resp.status();
if !status.is_success() {
let body = resp.text().await.unwrap_or_default();
return Err(format!("refresh returned {status}: {body}"));
}
let body: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("refresh response parse error: {e}"))?;
let new_access = body
.get("access_token")
.and_then(|v| v.as_str())
.ok_or_else(|| "refresh response missing access_token".to_string())?;
let new_refresh = body
.get("refresh_token")
.and_then(|v| v.as_str())
.map(String::from);
let mut guard = tokens.write().await;
guard.0 = Zeroizing::new(new_access.to_string());
if let Some(rt) = new_refresh {
guard.1 = Some(Zeroizing::new(rt));
}
Ok(())
}
/// Accept a room invite by calling POST /_matrix/client/v3/rooms/{room_id}/join.
async fn accept_invite(
client: &reqwest::Client,
@@ -218,7 +377,7 @@ impl ChannelAdapter for MatrixAdapter {
let (tx, rx) = mpsc::channel::<ChannelMessage>(256);
let homeserver = self.homeserver_url.clone();
let access_token = self.access_token.clone();
let tokens = Arc::clone(&self.tokens);
// Use the validated user ID from /whoami instead of the config value.
// Matrix server delegation or casing differences can cause self.user_id
// to not match the sender field in timeline events, making the bot
@@ -232,9 +391,10 @@ impl ChannelAdapter for MatrixAdapter {
// FIX #4: Do an initial sync to get the since token, skipping old messages.
if since_token.read().await.is_none() {
if let Some(token) = initial_sync(&client, &homeserver, access_token.as_str()).await {
let token = self.current_access_token().await;
if let Some(next) = initial_sync(&client, &homeserver, &token).await {
info!("Matrix: initial sync complete, skipping old messages");
*since_token.write().await = Some(token);
*since_token.write().await = Some(next);
}
}
@@ -257,12 +417,13 @@ impl ChannelAdapter for MatrixAdapter {
url.push_str(&format!("&since={token}"));
}
let current_token = tokens.read().await.0.as_str().to_string();
let resp = tokio::select! {
_ = shutdown_rx.changed() => {
info!("Matrix adapter shutting down");
break;
}
result = client.get(&url).bearer_auth(access_token.as_str()).send() => {
result = client.get(&url).bearer_auth(&current_token).send() => {
match result {
Ok(r) => r,
Err(e) => {
@@ -276,7 +437,38 @@ impl ChannelAdapter for MatrixAdapter {
};
if !resp.status().is_success() {
warn!("Matrix sync returned {}", resp.status());
let status = resp.status();
// MSC2918: on 401 M_UNKNOWN_TOKEN with a refresh token configured,
// try refreshing once and loop again immediately. Hard logout
// (soft_logout:false) is unrecoverable here — the operator must
// perform a fresh m.login.password.
if status == reqwest::StatusCode::UNAUTHORIZED {
let body_text = resp.text().await.unwrap_or_default();
if is_unknown_token_body(&body_text) {
if is_hard_logout(&body_text) {
warn!(
"Matrix: hard logout (soft_logout=false), operator must re-login"
);
} else {
match try_refresh_tokens(&client, &homeserver, &tokens).await {
Ok(()) => {
info!(
"Matrix: access token refreshed via MSC2918, resuming /sync"
);
backoff = Duration::from_secs(1);
continue;
}
Err(e) => {
warn!("Matrix: token refresh failed: {e}");
}
}
}
} else {
warn!("Matrix sync returned {status}: {body_text}");
}
} else {
warn!("Matrix sync returned {status}");
}
tokio::time::sleep(backoff).await;
backoff = (backoff * 2).min(Duration::from_secs(60));
continue;
@@ -309,8 +501,8 @@ impl ChannelAdapter for MatrixAdapter {
);
continue;
}
accept_invite(&client, &homeserver, access_token.as_str(), room_id)
.await;
let tok = tokens.read().await.0.as_str().to_string();
accept_invite(&client, &homeserver, &tok, room_id).await;
}
}
}
@@ -380,10 +572,11 @@ impl ChannelAdapter for MatrixAdapter {
}
// FIX #3: Determine if room is a DM (2 members) or group.
let tok_for_count = tokens.read().await.0.as_str().to_string();
let is_group = get_room_member_count(
&client,
&homeserver,
access_token.as_str(),
&tok_for_count,
room_id,
)
.await
@@ -409,10 +602,11 @@ impl ChannelAdapter for MatrixAdapter {
}
// FIX #3: Determine if room is a DM (2 members) or group.
let tok_for_count = tokens.read().await.0.as_str().to_string();
let is_group = get_room_member_count(
&client,
&homeserver,
access_token.as_str(),
&tok_for_count,
room_id,
)
.await
@@ -485,10 +679,11 @@ impl ChannelAdapter for MatrixAdapter {
"timeout": 5000,
});
let token = self.current_access_token().await;
let _ = self
.client
.put(&url)
.bearer_auth(&*self.access_token)
.bearer_auth(&token)
.json(&body)
.send()
.await;
@@ -518,6 +713,79 @@ mod tests {
assert_eq!(adapter.name(), "matrix");
}
#[test]
fn test_is_unknown_token_body() {
// Real matrix.org body for M_UNKNOWN_TOKEN under MAS.
let body =
r#"{"errcode":"M_UNKNOWN_TOKEN","error":"Token is not active","soft_logout":true}"#;
assert!(is_unknown_token_body(body));
assert!(!is_hard_logout(body));
let hard = r#"{"errcode":"M_UNKNOWN_TOKEN","error":"Invalidated","soft_logout":false}"#;
assert!(is_unknown_token_body(hard));
assert!(is_hard_logout(hard));
let other = r#"{"errcode":"M_FORBIDDEN","error":"You are not allowed"}"#;
assert!(!is_unknown_token_body(other));
assert!(!is_hard_logout(other));
// Empty / non-JSON must not trigger refresh.
assert!(!is_unknown_token_body(""));
assert!(!is_unknown_token_body("not json"));
}
#[tokio::test]
async fn test_refresh_tokens_rotates_pair() {
// Spin up a tiny axum server that mimics MSC2918 /refresh: rotates both
// access and refresh tokens and returns the new pair.
use axum::{routing::post, Json, Router};
async fn refresh_handler(Json(body): Json<serde_json::Value>) -> Json<serde_json::Value> {
let incoming = body
.get("refresh_token")
.and_then(|v| v.as_str())
.unwrap_or("");
assert_eq!(incoming, "old_refresh");
Json(serde_json::json!({
"access_token": "new_access",
"refresh_token": "new_refresh",
"expires_in_ms": 3_600_000u64,
}))
}
let app = Router::new().route("/_matrix/client/v3/refresh", post(refresh_handler));
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
let server = tokio::spawn(async move {
axum::serve(listener, app).await.unwrap();
});
let homeserver = format!("http://{addr}");
let tokens: TokenPair = Arc::new(RwLock::new((
Zeroizing::new("old_access".to_string()),
Some(Zeroizing::new("old_refresh".to_string())),
)));
let client = reqwest::Client::new();
try_refresh_tokens(&client, &homeserver, &tokens)
.await
.expect("refresh succeeds");
let guard = tokens.read().await;
assert_eq!(guard.0.as_str(), "new_access");
assert_eq!(guard.1.as_ref().map(|s| s.as_str()), Some("new_refresh"));
drop(guard);
// Refresh with no refresh token configured must fail cleanly.
let no_refresh: TokenPair = Arc::new(RwLock::new((Zeroizing::new("a".to_string()), None)));
let err = try_refresh_tokens(&client, &homeserver, &no_refresh)
.await
.unwrap_err();
assert!(err.contains("no refresh token"));
server.abort();
}
#[test]
fn test_matrix_allowed_rooms() {
let adapter = MatrixAdapter::new(
+2 -1
View File
@@ -722,7 +722,8 @@ mod tests {
assert_eq!(resolved, None);
// Missing channel_id on the wire — no match (the binding is restrictive).
let resolved = router.resolve_with_channel_id(&ChannelType::Discord, "any-user", None, None);
let resolved =
router.resolve_with_channel_id(&ChannelType::Discord, "any-user", None, None);
assert_eq!(resolved, None);
}
+1 -3
View File
@@ -329,9 +329,7 @@ impl ChannelAdapter for SlackAdapter {
// connection during the rotation overlap. Ack on
// both, but only forward to the agent once.
if is_duplicate_envelope(&seen_envelopes, envelope_id) {
debug!(
"Slack: skipping duplicate envelope_id {envelope_id}"
);
debug!("Slack: skipping duplicate envelope_id {envelope_id}");
continue;
}
+483 -73
View File
@@ -46,6 +46,11 @@ pub struct TelegramAdapter {
poll_interval: Duration,
/// Base URL for Telegram Bot API (supports proxies/mirrors).
api_base_url: String,
/// Maps Telegram forum topic `message_thread_id` to an agent name. When a
/// message arrives inside a configured topic, the inbound parser stamps
/// `target_agent` so the bridge dispatches to that agent instead of the
/// default. Empty map = default routing for every thread. Issue #780.
thread_routes: Arc<HashMap<i64, String>>,
/// Bot username (without @), populated from `getMe` during `start()`.
/// Used for @mention detection in group messages.
bot_username: Arc<tokio::sync::RwLock<Option<String>>>,
@@ -79,6 +84,19 @@ impl TelegramAdapter {
allowed_users: Vec<String>,
poll_interval: Duration,
api_url: Option<String>,
) -> Self {
Self::with_thread_routes(token, allowed_users, poll_interval, api_url, HashMap::new())
}
/// Same as [`new`] but accepts a `thread_routes` map for forum-topic
/// routing (issue #780). Keys are Telegram `message_thread_id` values,
/// values are agent names to dispatch matching messages to.
pub fn with_thread_routes(
token: String,
allowed_users: Vec<String>,
poll_interval: Duration,
api_url: Option<String>,
thread_routes: HashMap<i64, String>,
) -> Self {
let (shutdown_tx, shutdown_rx) = watch::channel(false);
let api_base_url = api_url
@@ -91,6 +109,7 @@ impl TelegramAdapter {
allowed_users,
poll_interval,
api_base_url,
thread_routes: Arc::new(thread_routes),
bot_username: Arc::new(tokio::sync::RwLock::new(None)),
rejected_reactions: Arc::new(Mutex::new(HashSet::new())),
shutdown_tx: Arc::new(shutdown_tx),
@@ -498,7 +517,7 @@ impl TelegramAdapter {
self.api_send_photo(chat_id, &url, caption.as_deref(), thread_id)
.await?;
}
ChannelContent::File { url, filename } => {
ChannelContent::File { url, filename, .. } => {
self.api_send_document(chat_id, &url, &filename, thread_id)
.await?;
}
@@ -521,6 +540,17 @@ impl TelegramAdapter {
self.api_send_message(chat_id, text.trim(), thread_id)
.await?;
}
ChannelContent::Multipart(parts) => {
// Send each child as its own Telegram message. Nested
// Multipart is rejected by adapters; flatten defensively.
for part in parts {
if let ChannelContent::Multipart(_) = part {
debug_assert!(false, "nested Multipart in send_to_user");
continue;
}
Box::pin(self.send_content(user, part, thread_id)).await?;
}
}
}
Ok(())
}
@@ -586,6 +616,7 @@ impl ChannelAdapter for TelegramAdapter {
let poll_interval = self.poll_interval;
let api_base_url = self.api_base_url.clone();
let bot_username = self.bot_username.clone();
let thread_routes = self.thread_routes.clone();
let mut shutdown = self.shutdown_rx.clone();
tokio::spawn(async move {
@@ -706,6 +737,7 @@ impl ChannelAdapter for TelegramAdapter {
&client,
&api_base_url,
bot_uname.as_deref(),
&thread_routes,
)
.await
{
@@ -815,6 +847,7 @@ async fn parse_telegram_update(
client: &reqwest::Client,
api_base_url: &str,
bot_username: Option<&str>,
thread_routes: &HashMap<i64, String>,
) -> Option<ChannelMessage> {
let update_id = update["update_id"].as_i64().unwrap_or(0);
let message = match update
@@ -934,7 +967,12 @@ async fn parse_telegram_update(
.unwrap_or("document")
.to_string();
match telegram_get_file_url(token, client, file_id, api_base_url).await {
Some(url) => ChannelContent::File { url, filename },
Some(url) => ChannelContent::File {
url,
filename,
mime: None,
size: None,
},
None => ChannelContent::Text(format!("[Document received: {filename}]")),
}
} else if message.get("voice").is_some() {
@@ -992,9 +1030,15 @@ async fn parse_telegram_update(
// Extract forum topic thread_id (Telegram sends this as `message_thread_id`
// for messages inside forum topics / reply threads).
let thread_id = message["message_thread_id"]
.as_i64()
.map(|tid| tid.to_string());
let thread_id_raw = message["message_thread_id"].as_i64();
let thread_id = thread_id_raw.map(|tid| tid.to_string());
// Forum topic routing (issue #780). If the message lives inside a
// configured topic, stash the target agent name in metadata under
// `target_agent_name` so the bridge dispatcher prefers it over the
// channel default. Topics with no entry leave metadata untouched and
// fall through to the adapter's default_agent.
let target_agent_name = thread_id_raw.and_then(|tid| thread_routes.get(&tid).cloned());
// Detect @mention of the bot in entities / caption_entities for MentionOnly group policy.
let mut metadata = HashMap::new();
@@ -1027,6 +1071,16 @@ async fn parse_telegram_update(
}
}
// Stash the forum-topic route target so the bridge can dispatch to a
// specific agent for this topic. The bridge resolves the name via
// `find_agent_by_name` before the standard router fallback.
if let Some(name) = target_agent_name {
metadata.insert(
"target_agent_name".to_string(),
serde_json::Value::String(name),
);
}
Some(ChannelMessage {
channel: ChannelType::Telegram,
platform_message_id: message_id.to_string(),
@@ -1180,9 +1234,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert_eq!(msg.channel, ChannelType::Telegram);
assert_eq!(msg.sender.display_name, "Alice Smith");
assert_eq!(msg.sender.platform_id, "111222333");
@@ -1213,9 +1275,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
// The chat_id (used for replies) stays on sender.platform_id.
assert_eq!(msg.sender.platform_id, "-1009876543210");
@@ -1254,9 +1324,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
let tg_id = msg
.metadata
@@ -1291,9 +1369,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Command { name, args } => {
assert_eq!(name, "agent");
@@ -1325,8 +1411,16 @@ mod tests {
let client = test_client();
// Empty allowed_users = allow all
let msg =
parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None).await;
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await;
assert!(msg.is_some());
// Non-matching allowed_users = filter out
@@ -1338,6 +1432,7 @@ mod tests {
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await;
assert!(msg.is_none());
@@ -1351,6 +1446,7 @@ mod tests {
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await;
assert!(msg.is_some());
@@ -1378,9 +1474,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert_eq!(msg.channel, ChannelType::Telegram);
assert_eq!(msg.sender.display_name, "Alice Smith");
assert!(matches!(msg.content, ChannelContent::Text(ref t) if t == "Edited message!"));
@@ -1416,9 +1520,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Command { name, args } => {
assert_eq!(name, "agents");
@@ -1442,9 +1554,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert!(matches!(msg.content, ChannelContent::Location { .. }));
}
@@ -1468,9 +1588,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
// With a fake token, getFile will fail, so we get a text fallback
match &msg.content {
ChannelContent::Text(t) => {
@@ -1505,9 +1633,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert!(t.contains("Document received"));
@@ -1538,9 +1674,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert!(t.contains("Voice message"));
@@ -1571,9 +1715,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert_eq!(msg.thread_id, Some("42".to_string()));
assert!(msg.is_group);
}
@@ -1593,9 +1745,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert_eq!(msg.thread_id, None);
assert!(!msg.is_group);
}
@@ -1617,12 +1777,193 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert_eq!(msg.thread_id, Some("99".to_string()));
}
// ---- Issue #780: forum topic -> agent routing ----
fn forum_topic_update(thread_id: i64, update_id: i64) -> serde_json::Value {
serde_json::json!({
"update_id": update_id,
"message": {
"message_id": 1000 + update_id,
"message_thread_id": thread_id,
"from": { "id": 555, "first_name": "Operator" },
"chat": { "id": -1009998887776_i64, "type": "supergroup" },
"date": 1700000000,
"text": "scoped message"
}
})
}
#[tokio::test]
async fn test_thread_routes_dispatch_matching_topic_to_named_agent() {
// A message inside a configured forum topic must route to that agent
// via `target_agent`, overriding the channel default.
let update = forum_topic_update(42, 1);
let mut routes = HashMap::new();
routes.insert(42_i64, "support-agent".to_string());
routes.insert(99_i64, "ops-agent".to_string());
let client = test_client();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&routes,
)
.await
.unwrap();
assert_eq!(msg.thread_id, Some("42".to_string()));
assert_eq!(
msg.metadata
.get("target_agent_name")
.and_then(|v| v.as_str()),
Some("support-agent")
);
}
#[tokio::test]
async fn test_thread_routes_unconfigured_topic_falls_through() {
// A thread id NOT in the route map must leave target_agent = None so
// the bridge applies the channel default_agent.
let update = forum_topic_update(7, 2);
let mut routes = HashMap::new();
routes.insert(42_i64, "support-agent".to_string());
let client = test_client();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&routes,
)
.await
.unwrap();
assert_eq!(msg.thread_id, Some("7".to_string()));
assert!(!msg.metadata.contains_key("target_agent_name"));
}
#[tokio::test]
async fn test_thread_routes_ignored_for_non_topic_message() {
// Messages with no message_thread_id (regular group / DM) must never
// be matched against thread_routes, even if the map is non-empty.
let update = serde_json::json!({
"update_id": 3,
"message": {
"message_id": 1003,
"from": { "id": 555, "first_name": "Operator" },
"chat": { "id": -1009998887776_i64, "type": "supergroup" },
"date": 1700000000,
"text": "general group message"
}
});
let mut routes = HashMap::new();
routes.insert(42_i64, "support-agent".to_string());
let client = test_client();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&routes,
)
.await
.unwrap();
assert_eq!(msg.thread_id, None);
assert!(!msg.metadata.contains_key("target_agent_name"));
}
#[tokio::test]
async fn test_thread_routes_multiple_topics_route_independently() {
// Two different topics in the same chat must dispatch to two different
// agents.
let mut routes = HashMap::new();
routes.insert(10_i64, "alpha".to_string());
routes.insert(20_i64, "beta".to_string());
let client = test_client();
let update_alpha = forum_topic_update(10, 4);
let msg_alpha = parse_telegram_update(
&update_alpha,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&routes,
)
.await
.unwrap();
assert_eq!(
msg_alpha
.metadata
.get("target_agent_name")
.and_then(|v| v.as_str()),
Some("alpha")
);
let update_beta = forum_topic_update(20, 5);
let msg_beta = parse_telegram_update(
&update_beta,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&routes,
)
.await
.unwrap();
assert_eq!(
msg_beta
.metadata
.get("target_agent_name")
.and_then(|v| v.as_str()),
Some("beta")
);
}
#[tokio::test]
async fn test_adapter_with_thread_routes_constructor_stores_map() {
// Sanity check that the new constructor wires the route map onto the
// adapter so the spawned polling task sees it.
let mut routes = HashMap::new();
routes.insert(1_i64, "first".to_string());
let adapter = TelegramAdapter::with_thread_routes(
"test:token".to_string(),
vec![],
Duration::from_millis(10),
Some("https://example.test".to_string()),
routes.clone(),
);
assert_eq!(adapter.thread_routes.as_ref(), &routes);
}
#[tokio::test]
async fn test_parse_sender_chat_fallback() {
// Messages sent on behalf of a channel have `sender_chat` instead of `from`.
@@ -1642,9 +1983,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
assert_eq!(msg.sender.display_name, "My Channel");
assert_eq!(msg.sender.platform_id, "-1001234567890");
assert!(
@@ -1666,8 +2015,16 @@ mod tests {
});
let client = test_client();
let msg =
parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None).await;
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await;
assert!(msg.is_none());
}
@@ -1698,6 +2055,7 @@ mod tests {
&client,
DEFAULT_API_URL,
Some("testbot"),
&HashMap::new(),
)
.await
.unwrap();
@@ -1730,6 +2088,7 @@ mod tests {
&client,
DEFAULT_API_URL,
Some("testbot"),
&HashMap::new(),
)
.await
.unwrap();
@@ -1764,6 +2123,7 @@ mod tests {
&client,
DEFAULT_API_URL,
Some("testbot"),
&HashMap::new(),
)
.await
.unwrap();
@@ -1801,6 +2161,7 @@ mod tests {
&client,
DEFAULT_API_URL,
Some("testbot"),
&HashMap::new(),
)
.await
.unwrap();
@@ -1838,6 +2199,7 @@ mod tests {
&client,
DEFAULT_API_URL,
Some("testbot"),
&HashMap::new(),
)
.await
.unwrap();
@@ -1874,6 +2236,7 @@ mod tests {
&client,
DEFAULT_API_URL,
Some("testbot"),
&HashMap::new(),
)
.await
.unwrap();
@@ -1927,9 +2290,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert!(t.starts_with("[Replying to Bob: We should use Rust]\n\n"));
@@ -1969,9 +2340,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert!(t.starts_with("[Replying to Carol: Sunset view]\n\n"));
@@ -2010,9 +2389,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert_eq!(t, "What was that?");
@@ -2048,9 +2435,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert!(t.starts_with("[Replying to Unknown: Anonymous message]\n\n"));
@@ -2075,9 +2470,17 @@ mod tests {
});
let client = test_client();
let msg = parse_telegram_update(&update, &[], "fake:token", &client, DEFAULT_API_URL, None)
.await
.unwrap();
let msg = parse_telegram_update(
&update,
&[],
"fake:token",
&client,
DEFAULT_API_URL,
None,
&HashMap::new(),
)
.await
.unwrap();
match &msg.content {
ChannelContent::Text(t) => {
assert_eq!(t, "Just a normal message");
@@ -2138,10 +2541,7 @@ mod tests {
body,
)
} else {
(
StatusCode::OK,
r#"{"ok":true,"result":true}"#.to_string(),
)
(StatusCode::OK, r#"{"ok":true,"result":true}"#.to_string())
}
}
}));
@@ -2218,7 +2618,10 @@ mod tests {
// Two-chunk message; first POST fails. Nothing delivered → Err.
let big = "a".repeat(5000); // > 4096 → split into two chunks
let stub = StubServer::new(vec![
(500, r#"{"ok":false,"error_code":500,"description":"server"}"#),
(
500,
r#"{"ok":false,"error_code":500,"description":"server"}"#,
),
(200, r#"{"ok":true,"result":{}}"#),
]);
let base = spawn_stub_server(stub.clone()).await;
@@ -2246,7 +2649,10 @@ mod tests {
let big = "a".repeat(5000);
let stub = StubServer::new(vec![
(200, r#"{"ok":true,"result":{}}"#),
(400, r#"{"ok":false,"error_code":400,"description":"some err"}"#),
(
400,
r#"{"ok":false,"error_code":400,"description":"some err"}"#,
),
]);
let base = spawn_stub_server(stub.clone()).await;
let adapter = test_adapter(base);
@@ -2257,7 +2663,11 @@ mod tests {
result.is_ok(),
"partial delivery must return Ok (best-effort), got {result:?}"
);
assert_eq!(stub.hit_count(), 2, "both chunks should have been attempted");
assert_eq!(
stub.hit_count(),
2,
"both chunks should have been attempted"
);
}
// -----------------------------------------------------------------------
+116
View File
@@ -50,6 +50,16 @@ pub enum ChannelContent {
File {
url: String,
filename: String,
/// Best-effort MIME type from the source platform (e.g. Discord's
/// `attachments[].content_type`). `None` if the platform did not
/// provide one; downstream consumers may sniff bytes or fall back
/// to extension-based detection.
#[serde(default, skip_serializing_if = "Option::is_none")]
mime: Option<String>,
/// Size in bytes, when known. Useful for capacity gating before
/// the bridge attempts to materialize or transmit the file.
#[serde(default, skip_serializing_if = "Option::is_none")]
size: Option<u64>,
},
/// Local file data (bytes read from disk). Used by the proactive `channel_send`
/// tool when `file_path` is provided instead of `file_url`.
@@ -70,6 +80,12 @@ pub enum ChannelContent {
name: String,
args: Vec<String>,
},
/// A composite message carrying multiple content blocks (e.g. a Discord
/// message with several attachments, or an image with a separate file
/// sibling). Blocks are flat-mapped by the bridge into multiple LLM
/// content blocks. Implementations should not produce nested `Multipart`
/// values; consumers may `debug_assert!` against nesting.
Multipart(Vec<ChannelContent>),
}
/// A unified message from any channel.
@@ -97,6 +113,60 @@ pub struct ChannelMessage {
pub metadata: HashMap<String, serde_json::Value>,
}
// Re-export the adapter allowlist from openfang-types so config validation
// and routing share a single source of truth (no drift between the two).
pub use openfang_types::config::CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL;
impl ChannelMessage {
/// Return the platform-native channel/conversation ID for this message,
/// suitable for matching against an `AgentBinding`'s `channel_id` field.
///
/// Resolution order:
/// 1. For adapters in [`CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL`],
/// `sender.platform_id` already *is* the channel ID (these adapters
/// overload the field because it doubles as the send target).
/// 2. Otherwise, fall back to `metadata["channel_id"]` if present (any
/// adapter can opt in by populating that key).
/// 3. Otherwise, `None`.
///
/// This is the central routing-time accessor — config validation and the
/// router both consult it (directly or via the same allowlist) so the two
/// cannot drift.
pub fn channel_id(&self) -> Option<String> {
// For builtin variants the string is already lowercase by construction.
// For `Custom(s)`, adapters _should_ register lowercase names but we
// case-fold here so a stray `Custom("Twitch")` cannot silently slip
// past the allowlist (and out of step with the validation path, which
// already lowercases user input). Allocates only on the Custom arm.
let channel_str: std::borrow::Cow<'_, str> = match &self.channel {
ChannelType::Telegram => "telegram".into(),
ChannelType::Discord => "discord".into(),
ChannelType::Slack => "slack".into(),
ChannelType::WhatsApp => "whatsapp".into(),
ChannelType::Signal => "signal".into(),
ChannelType::Matrix => "matrix".into(),
ChannelType::Email => "email".into(),
ChannelType::Teams => "teams".into(),
ChannelType::Mattermost => "mattermost".into(),
ChannelType::WebChat => "webchat".into(),
ChannelType::CLI => "cli".into(),
ChannelType::Mqtt => "mqtt".into(),
ChannelType::Custom(s) => s.to_lowercase().into(),
};
if CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL
.iter()
.any(|c| *c == channel_str.as_ref())
{
Some(self.sender.platform_id.clone())
} else {
self.metadata
.get("channel_id")
.and_then(|v| v.as_str())
.map(String::from)
}
}
}
/// Agent lifecycle phase for UX indicators.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "snake_case")]
@@ -271,6 +341,24 @@ pub trait ChannelAdapter: Send + Sync {
self.send(user, content).await
}
/// Determine whether to auto-create a thread for an incoming message.
/// Returns Some(thread_name) to create a thread, or None to reply directly.
/// Default implementation returns None (no auto-threading).
async fn should_auto_thread(&self, _message: &ChannelMessage) -> Option<String> {
None
}
/// Create a new thread (typically triggered after should_auto_thread returns Some).
/// Returns the new thread ID on success.
async fn create_thread(
&self,
_user: &ChannelUser,
_message_id: &str,
_thread_name: &str,
) -> Result<String, Box<dyn std::error::Error>> {
Err("Thread creation not supported for this adapter".into())
}
/// Whether this adapter should suppress sending internal agent errors back to the user.
///
/// Returns `true` for public broadcast channels (e.g. Mastodon) where posting
@@ -365,6 +453,34 @@ mod tests {
assert_eq!(back, ChannelType::Email);
}
#[test]
fn test_channel_id_custom_arm_is_case_insensitive() {
// A stray capitalized Custom variant must still resolve through the
// allowlist. The validation path lowercases user input; the routing
// path needs the same case-fold to stay in sync.
let make = |name: &str| ChannelMessage {
channel: ChannelType::Custom(name.to_string()),
platform_message_id: "m".to_string(),
sender: ChannelUser {
platform_id: "C123".to_string(),
display_name: "x".to_string(),
openfang_user: None,
},
content: ChannelContent::Text("hi".to_string()),
target_agent: None,
timestamp: Utc::now(),
is_group: false,
thread_id: None,
metadata: HashMap::new(),
};
assert_eq!(make("twitch").channel_id().as_deref(), Some("C123"));
assert_eq!(make("Twitch").channel_id().as_deref(), Some("C123"));
assert_eq!(make("TWITCH").channel_id().as_deref(), Some("C123"));
// Lark spelling (Feishu Intl) must also match.
assert_eq!(make("lark").channel_id().as_deref(), Some("C123"));
assert_eq!(make("Lark").channel_id().as_deref(), Some("C123"));
}
#[test]
fn test_channel_content_variants() {
let text = ChannelContent::Text("hello".to_string());
+1 -1
View File
@@ -271,7 +271,7 @@ impl ChannelAdapter for WhatsAppAdapter {
return Err(format!("WhatsApp API error {status}: {body}").into());
}
}
ChannelContent::File { url, filename } => {
ChannelContent::File { url, filename, .. } => {
let body = serde_json::json!({
"messaging_product": "whatsapp",
"to": user.platform_id,
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "OpenFang",
"version": "0.6.5",
"version": "0.6.9",
"identifier": "ai.openfang.desktop",
"build": {},
"app": {
+2
View File
@@ -16,6 +16,8 @@ uuid = { workspace = true }
chrono = { workspace = true }
dashmap = { workspace = true }
dirs = { workspace = true }
sha2 = { workspace = true }
hex = { workspace = true }
[dev-dependencies]
tokio-test = { workspace = true }
+167 -5
View File
@@ -8,10 +8,27 @@ use crate::{
use dashmap::DashMap;
use openfang_types::agent::AgentId;
use serde::Serialize;
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::sync::{Arc, RwLock};
use tracing::{info, warn};
use uuid::Uuid;
/// Callback signature invoked on every successful hand load / reload.
///
/// Arguments: `(hand_id, sha256_hex_of_hand_toml)`. The kernel wires this
/// into the Merkle audit chain so reload events leave a tamper-evident
/// record (issue #1172). The callback must be cheap and non-blocking; it
/// runs inline on the loader thread.
pub type HandAuditCallback = Arc<dyn Fn(&str, &str) + Send + Sync>;
/// Compute the SHA-256 hex digest of raw HAND.toml content.
fn hand_toml_sha256(toml_content: &str) -> String {
let mut hasher = Sha256::new();
hasher.update(toml_content.as_bytes());
hex::encode(hasher.finalize())
}
// ─── Settings availability types ────────────────────────────────────────────
/// Availability status of a single setting option.
@@ -41,6 +58,10 @@ pub struct HandRegistry {
definitions: DashMap<String, HandDefinition>,
/// Active hand instances, keyed by instance UUID.
instances: DashMap<Uuid, HandInstance>,
/// Optional callback invoked on every successful HAND.toml load with
/// the computed SHA-256 of the file content. Wired by the kernel into
/// the Merkle audit chain (issue #1172).
audit_callback: RwLock<Option<HandAuditCallback>>,
}
impl HandRegistry {
@@ -49,9 +70,37 @@ impl HandRegistry {
Self {
definitions: DashMap::new(),
instances: DashMap::new(),
audit_callback: RwLock::new(None),
}
}
/// Install a callback invoked on every successful HAND.toml load /
/// reload with the file's SHA-256. The kernel wires this to
/// `AuditLog::record(AuditAction::ConfigChange, ...)` so reload events
/// leave a tamper-evident audit record (issue #1172).
pub fn set_audit_callback(&self, callback: HandAuditCallback) {
let mut guard = self
.audit_callback
.write()
.unwrap_or_else(|e| e.into_inner());
*guard = Some(callback);
}
/// Compute SHA-256 of the given HAND.toml content and invoke the audit
/// callback if one is registered. Returns the hex digest for callers
/// that want to log or compare it.
fn emit_hand_loaded_audit(&self, hand_id: &str, toml_content: &str) -> String {
let hash = hand_toml_sha256(toml_content);
let guard = self
.audit_callback
.read()
.unwrap_or_else(|e| e.into_inner());
if let Some(cb) = guard.as_ref() {
cb(hand_id, &hash);
}
hash
}
/// Persist active hand state to disk so it survives restarts.
pub fn persist_state(&self, path: &std::path::Path) -> HandResult<()> {
let entries: Vec<serde_json::Value> = self
@@ -112,7 +161,8 @@ impl HandRegistry {
for (id, toml_content, skill_content) in bundled {
match bundled::parse_bundled(id, toml_content, skill_content) {
Ok(def) => {
info!(hand = %def.id, name = %def.name, "Loaded bundled hand");
let hash = self.emit_hand_loaded_audit(&def.id, toml_content);
info!(hand = %def.id, name = %def.name, sha256 = %hash, "Loaded bundled hand");
self.definitions.insert(def.id.clone(), def);
count += 1;
}
@@ -173,7 +223,8 @@ impl HandRegistry {
match bundled::parse_bundled("custom", &contents, &skill_content) {
Ok(def) => {
let hand_id = def.id.clone();
info!(hand = %hand_id, path = %path.display(), "Loaded workspace hand");
let hash = self.emit_hand_loaded_audit(&hand_id, &contents);
info!(hand = %hand_id, path = %path.display(), sha256 = %hash, "Loaded workspace hand");
self.definitions.insert(hand_id, def);
count += 1;
}
@@ -204,7 +255,8 @@ impl HandRegistry {
)));
}
info!(hand = %def.id, name = %def.name, path = %path.display(), "Installed hand from path");
let hash = self.emit_hand_loaded_audit(&def.id, &toml_content);
info!(hand = %def.id, name = %def.name, path = %path.display(), sha256 = %hash, "Installed hand from path");
self.definitions.insert(def.id.clone(), def.clone());
// Persist the hand to the user's data dir so it survives daemon
@@ -251,7 +303,8 @@ impl HandRegistry {
)));
}
info!(hand = %def.id, name = %def.name, "Installed hand from content");
let hash = self.emit_hand_loaded_audit(&def.id, toml_content);
info!(hand = %def.id, name = %def.name, sha256 = %hash, "Installed hand from content");
self.definitions.insert(def.id.clone(), def.clone());
Ok(def)
}
@@ -269,7 +322,8 @@ impl HandRegistry {
let def = bundled::parse_bundled("custom", toml_content, skill_content)?;
let existed = self.definitions.contains_key(&def.id);
let verb = if existed { "Updated" } else { "Installed" };
info!(hand = %def.id, name = %def.name, "{verb} hand from content");
let hash = self.emit_hand_loaded_audit(&def.id, toml_content);
info!(hand = %def.id, name = %def.name, sha256 = %hash, "{verb} hand from content");
self.definitions.insert(def.id.clone(), def.clone());
Ok(def)
}
@@ -1145,6 +1199,114 @@ metrics = []
assert!(matches!(err, HandError::AlreadyActive(_)));
}
/// Issue #1172: HAND.toml SHA-256 must be emitted to the audit
/// callback on every successful load / reload.
#[test]
fn audit_callback_records_hand_toml_hash_on_load() {
use std::sync::Mutex;
let captured: Arc<Mutex<Vec<(String, String)>>> = Arc::new(Mutex::new(Vec::new()));
let sink = Arc::clone(&captured);
let reg = HandRegistry::new();
reg.set_audit_callback(Arc::new(move |hand_id: &str, hash: &str| {
sink.lock()
.unwrap()
.push((hand_id.to_string(), hash.to_string()));
}));
let toml_str = r#"
id = "audit-hand"
name = "Audit Hand"
description = "Used to verify audit-trail wiring"
category = "other"
tools = []
[agent]
name = "audit-agent"
description = "audit"
system_prompt = "audit."
"#;
// Precompute the expected hash so the test fails loudly if the
// registry ever changes how it digests the TOML content.
let expected_hash = {
let mut h = Sha256::new();
h.update(toml_str.as_bytes());
hex::encode(h.finalize())
};
let def = reg.install_from_content(toml_str, "").unwrap();
assert_eq!(def.id, "audit-hand");
let events = captured.lock().unwrap().clone();
assert_eq!(
events.len(),
1,
"exactly one audit event should be emitted per load"
);
assert_eq!(events[0].0, "audit-hand", "hand id propagated to callback");
assert_eq!(
events[0].1, expected_hash,
"callback received SHA-256 of the HAND.toml content"
);
assert_eq!(events[0].1.len(), 64, "SHA-256 hex is 64 chars");
}
/// Issue #1172: reloading the same HAND.toml via upsert must emit a
/// fresh audit event so the chain records when the swap took effect.
/// A content change must surface a different hash.
#[test]
fn audit_callback_fires_on_reload_with_new_hash() {
use std::sync::Mutex;
let captured: Arc<Mutex<Vec<(String, String)>>> = Arc::new(Mutex::new(Vec::new()));
let sink = Arc::clone(&captured);
let reg = HandRegistry::new();
reg.set_audit_callback(Arc::new(move |hand_id: &str, hash: &str| {
sink.lock()
.unwrap()
.push((hand_id.to_string(), hash.to_string()));
}));
let v1 = r#"
id = "reload-hand"
name = "Reload Hand v1"
description = "v1"
category = "other"
tools = []
[agent]
name = "reload-agent"
description = "reload"
system_prompt = "v1."
"#;
let v2 = r#"
id = "reload-hand"
name = "Reload Hand v2"
description = "v2"
category = "other"
tools = []
[agent]
name = "reload-agent"
description = "reload"
system_prompt = "v2 — schedule changed."
"#;
reg.upsert_from_content(v1, "").unwrap();
reg.upsert_from_content(v2, "").unwrap();
let events = captured.lock().unwrap().clone();
assert_eq!(events.len(), 2, "one event per upsert (load + reload)");
assert_eq!(events[0].0, "reload-hand");
assert_eq!(events[1].0, "reload-hand");
assert_ne!(
events[0].1, events[1].1,
"different HAND.toml content must yield different SHA-256"
);
}
/// Integration test for issue #809: `hand config` round-trip.
///
/// Simulates what `openfang hand config <id> --set KEY=VAL` does against
+1
View File
@@ -32,6 +32,7 @@ futures = { workspace = true }
subtle = { workspace = true }
rand = { workspace = true }
hex = { workspace = true }
sha2 = { workspace = true }
reqwest = { workspace = true }
rustls = { workspace = true }
cron = "0.16"
+1
View File
@@ -339,6 +339,7 @@ mod tests {
autonomous: None,
pinned_model: None,
workspace: None,
state_dir: None,
generate_identity_files: true,
exec_policy: None,
tool_allowlist: vec![],
File diff suppressed because it is too large Load Diff
+6
View File
@@ -234,6 +234,12 @@ pub struct BudgetStatus {
/// Order matters: more specific patterns must come before generic ones
/// (e.g. "gpt-4o-mini" before "gpt-4o", "gpt-4.1-mini" before "gpt-4.1").
fn estimate_cost_rates(model: &str) -> (f64, f64) {
// ── Requesty (issue #995) ──────────────────────────────────
// Router-style gateway. IDs are `requesty/<upstream>/<model>` and
// resolve via substring match on the upstream model name below
// (e.g. "sonnet", "gpt-4o", "gemini", "deepseek", "llama").
// No early-return here — fall through to upstream patterns.
// ── Anthropic ──────────────────────────────────────────────
if model.contains("haiku") {
return (0.25, 1.25);
+18
View File
@@ -134,6 +134,23 @@ impl AgentRegistry {
Ok(())
}
/// Update an agent's private state directory path. The state directory
/// holds identity files, sessions, and per-agent memory and is always
/// kept separate from the user-facing workspace. See issue #1097.
pub fn update_state_dir(
&self,
id: AgentId,
state_dir: Option<std::path::PathBuf>,
) -> OpenFangResult<()> {
let mut entry = self
.agents
.get_mut(&id)
.ok_or_else(|| OpenFangError::AgentNotFound(id.to_string()))?;
entry.manifest.state_dir = state_dir;
entry.last_active = chrono::Utc::now();
Ok(())
}
/// Update an agent's visual identity (emoji, avatar, color).
pub fn update_identity(
&self,
@@ -391,6 +408,7 @@ mod tests {
autonomous: None,
pinned_model: None,
workspace: None,
state_dir: None,
generate_identity_files: true,
exec_policy: None,
tool_allowlist: vec![],
+1
View File
@@ -176,6 +176,7 @@ impl SetupWizard {
autonomous: None,
pinned_model: None,
workspace: None,
state_dir: None,
generate_identity_files: true,
profile: None,
fallback_models: vec![],
+1 -1
View File
@@ -496,7 +496,7 @@ impl SessionStore {
.conn
.lock()
.map_err(|e| OpenFangError::Internal(e.to_string()))?;
let messages_blob = rmp_serde::to_vec(&canonical.messages)
let messages_blob = rmp_serde::to_vec_named(&canonical.messages)
.map_err(|e| OpenFangError::Serialization(e.to_string()))?;
conn.execute(
"INSERT INTO canonical_sessions (agent_id, messages, compaction_cursor, compacted_summary, updated_at)
+55 -13
View File
@@ -69,8 +69,7 @@ fn env_timeout_secs(var: &str) -> Option<u64> {
fn tool_timeout_for(tool_name: &str) -> Option<Duration> {
let secs = match tool_name {
"agent_send" | "agent_spawn" => {
env_timeout_secs("OPENFANG_AGENT_TOOL_TIMEOUT_SECS")
.unwrap_or(AGENT_TOOL_TIMEOUT_SECS)
env_timeout_secs("OPENFANG_AGENT_TOOL_TIMEOUT_SECS").unwrap_or(AGENT_TOOL_TIMEOUT_SECS)
}
_ => env_timeout_secs("OPENFANG_TOOL_TIMEOUT_SECS").unwrap_or(TOOL_TIMEOUT_SECS),
};
@@ -158,22 +157,30 @@ fn build_assistant_message_preserving_thinking(
response_blocks: &[ContentBlock],
final_text: &str,
) -> Message {
let has_thinking = response_blocks
.iter()
.any(|b| matches!(b, ContentBlock::Thinking { .. }));
if !has_thinking {
// Key on either Thinking or RedactedThinking — Anthropic/Bedrock both
// reject extended-thinking history that drops the redacted variant, so a
// turn that contains only RedactedThinking must still be preserved.
let has_reasoning = response_blocks.iter().any(|b| {
matches!(
b,
ContentBlock::Thinking { .. } | ContentBlock::RedactedThinking { .. }
)
});
if !has_reasoning {
return Message::assistant(final_text.to_string());
}
// Preserve order: Thinking blocks first (in original order), then a
// single Text block carrying `final_text`. Tool blocks aren't expected
// here (StopReason::EndTurn path), but copy them through if present so
// we don't drop information.
// Preserve order: Thinking / RedactedThinking blocks first (in original
// order), then a single Text block carrying `final_text`. Tool blocks
// aren't expected here (StopReason::EndTurn path), but copy them through
// if present so we don't drop information.
let mut blocks: Vec<ContentBlock> = Vec::with_capacity(response_blocks.len() + 1);
let mut emitted_text = false;
for b in response_blocks {
match b {
ContentBlock::Thinking { .. } => blocks.push(b.clone()),
ContentBlock::Thinking { .. } | ContentBlock::RedactedThinking { .. } => {
blocks.push(b.clone())
}
ContentBlock::Text { .. } if !emitted_text => {
blocks.push(ContentBlock::Text {
text: final_text.to_string(),
@@ -3263,7 +3270,9 @@ mod tests {
assert_eq!(blocks.len(), 2, "must preserve thinking + text");
match &blocks[0] {
ContentBlock::Thinking {
thinking, signature, ..
thinking,
signature,
..
} => {
assert_eq!(thinking, "Let me reason carefully...");
assert_eq!(signature.as_deref(), Some("sig_anthropic_xyz"));
@@ -3359,7 +3368,10 @@ mod tests {
let has_redacted = blocks
.iter()
.any(|b| matches!(b, ContentBlock::RedactedThinking { .. }));
assert!(has_thinking, "Thinking block must be preserved on MaxTokens");
assert!(
has_thinking,
"Thinking block must be preserved on MaxTokens"
);
assert!(
has_redacted,
"RedactedThinking block must be preserved on MaxTokens"
@@ -3380,6 +3392,36 @@ mod tests {
assert_eq!(saved_text, Some(final_text));
}
/// Issue #1187 — a turn that contains only `RedactedThinking` (no
/// `Thinking` block) must still trigger the block-preserving path. The
/// previous gate keyed solely on `Thinking`, so redacted-only turns were
/// downgraded to plain text and the encrypted blob was lost on the next
/// request, which Anthropic/Bedrock reject.
#[test]
fn test_build_assistant_message_preserves_redacted_only() {
let response_blocks = vec![
ContentBlock::RedactedThinking {
data: "encrypted_only".to_string(),
},
ContentBlock::Text {
text: "Answer".to_string(),
provider_metadata: None,
},
];
let msg = build_assistant_message_preserving_thinking(&response_blocks, "Answer");
let blocks = match &msg.content {
MessageContent::Blocks(b) => b,
other => panic!("expected Blocks content for redacted-only turn, got {other:?}"),
};
let has_redacted = blocks.iter().any(
|b| matches!(b, ContentBlock::RedactedThinking { data } if data == "encrypted_only"),
);
assert!(
has_redacted,
"RedactedThinking-only turn must be preserved as Blocks"
);
}
#[test]
fn test_retry_constants() {
assert_eq!(MAX_RETRIES, 3);
@@ -189,7 +189,9 @@ enum ContentBlockAccum {
/// Redacted (encrypted) thinking block streamed from Anthropic.
/// The opaque `data` blob arrives on `content_block_start` and must be
/// persisted so the next turn can echo it back verbatim.
RedactedThinking { data: String },
RedactedThinking {
data: String,
},
}
#[async_trait]
@@ -454,10 +456,8 @@ impl LlmDriver for AnthropicDriver {
"thinking" => {
// Some API versions ship the signature on
// content_block_start instead of as a delta.
let initial_sig = block["signature"]
.as_str()
.unwrap_or("")
.to_string();
let initial_sig =
block["signature"].as_str().unwrap_or("").to_string();
blocks.push(ContentBlockAccum::Thinking {
thinking: String::new(),
signature: initial_sig,
@@ -470,10 +470,7 @@ impl LlmDriver for AnthropicDriver {
// verbatim so we can echo it back on the
// next request — API rejects history
// that strips redacted_thinking blocks.
let data = block["data"]
.as_str()
.unwrap_or("")
.to_string();
let data = block["data"].as_str().unwrap_or("").to_string();
blocks.push(ContentBlockAccum::RedactedThinking { data });
}
_ => {}
@@ -751,9 +748,7 @@ fn convert_message(msg: &Message) -> ApiMessage {
if data.is_empty() {
None
} else {
Some(ApiContentBlock::RedactedThinking {
data: data.clone(),
})
Some(ApiContentBlock::RedactedThinking { data: data.clone() })
}
}
ContentBlock::Unknown => None,
+77 -5
View File
@@ -92,6 +92,20 @@ enum BedrockContentBlock {
#[serde(rename = "toolResult")]
tool_result: BedrockToolResult,
},
// Bedrock Converse representation of Anthropic's `redacted_thinking`.
// The encrypted blob is echoed back verbatim under
// reasoningContent.redactedContent so Claude extended-thinking history
// is not rejected on resubmission.
ReasoningContent {
#[serde(rename = "reasoningContent")]
reasoning_content: BedrockReasoningContent,
},
}
#[derive(Debug, Serialize)]
struct BedrockReasoningContent {
#[serde(rename = "redactedContent")]
redacted_content: String,
}
#[derive(Debug, Serialize)]
@@ -299,11 +313,22 @@ fn convert_content_block(block: &ContentBlock) -> Option<BedrockContentBlock> {
},
},
}),
// Image, Thinking, RedactedThinking, and Unknown are not supported — silently drop
ContentBlock::Image { .. }
| ContentBlock::Thinking { .. }
| ContentBlock::RedactedThinking { .. }
| ContentBlock::Unknown => None,
// Echo redacted_thinking verbatim. Bedrock Converse rejects history
// that drops these blocks on Claude extended-thinking models, mirroring
// the anthropic.rs path. Drop empty blobs (e.g. interrupted stream).
ContentBlock::RedactedThinking { data } => {
if data.is_empty() {
None
} else {
Some(BedrockContentBlock::ReasoningContent {
reasoning_content: BedrockReasoningContent {
redacted_content: data.clone(),
},
})
}
}
// Image, Thinking, and Unknown are not supported — silently drop
ContentBlock::Image { .. } | ContentBlock::Thinking { .. } | ContentBlock::Unknown => None,
}
}
@@ -1126,6 +1151,53 @@ mod tests {
assert!(text_at_3 >= 1);
}
/// Issue #1187 — Bedrock Converse history must preserve
/// `redacted_thinking` blocks on Claude extended-thinking models.
/// A message containing only RedactedThinking must round-trip through
/// `convert_content_block` without being silently dropped, and the wire
/// format must use `reasoningContent.redactedContent`.
#[test]
fn test_bedrock_redacted_thinking_round_trip() {
let msg = Message::assistant_with_blocks(vec![ContentBlock::RedactedThinking {
data: "encrypted-blob-abc123".to_string(),
}]);
let bedrock_blocks = convert_message_content(&msg.content);
assert_eq!(
bedrock_blocks.len(),
1,
"RedactedThinking must survive convert_content_block"
);
match &bedrock_blocks[0] {
BedrockContentBlock::ReasoningContent { reasoning_content } => {
assert_eq!(reasoning_content.redacted_content, "encrypted-blob-abc123");
}
other => panic!("expected ReasoningContent block, got {other:?}"),
}
// Wire format check: serialized JSON must carry
// reasoningContent.redactedContent so Bedrock accepts the history.
let json = serde_json::to_value(&bedrock_blocks[0]).unwrap();
assert_eq!(
json["reasoningContent"]["redactedContent"],
"encrypted-blob-abc123"
);
}
/// Empty RedactedThinking blobs (interrupted stream) must be dropped on
/// outbound, matching the anthropic.rs behavior.
#[test]
fn test_bedrock_redacted_thinking_empty_dropped() {
let msg = Message::assistant_with_blocks(vec![ContentBlock::RedactedThinking {
data: String::new(),
}]);
let bedrock_blocks = convert_message_content(&msg.content);
assert!(
bedrock_blocks.is_empty(),
"empty redacted_thinking must be dropped"
);
}
#[test]
fn test_validate_tool_pairing_noop_on_correct() {
// already correct 2-for-2 → no change
@@ -11,7 +11,7 @@
use crate::llm_driver::{CompletionRequest, CompletionResponse, LlmDriver, LlmError, StreamEvent};
use async_trait::async_trait;
use dashmap::DashMap;
use openfang_types::message::{ContentBlock, Role, StopReason, TokenUsage};
use openfang_types::message::{ContentBlock, MessageContent, Role, StopReason, TokenUsage};
use serde::Deserialize;
use std::sync::Arc;
use tokio::io::{AsyncBufReadExt, AsyncReadExt};
@@ -130,6 +130,14 @@ impl ClaudeCodeDriver {
}
/// Build a text prompt from the completion request messages.
///
/// The Claude Code CLI is text-only (`-p <prompt>`), so non-text content
/// blocks (images, etc.) cannot be sent natively. Rather than dropping
/// them silently — which causes the model to hallucinate about content
/// it can't see — we render each non-text block as a synthetic
/// `[attachment: ...]` marker. The model still can't *view* the
/// attachment, but it knows the attachment exists and can acknowledge
/// it coherently instead of confabulating.
fn build_prompt(request: &CompletionRequest) -> String {
let mut parts = Vec::new();
@@ -139,15 +147,53 @@ impl ClaudeCodeDriver {
Role::Assistant => "Assistant",
Role::System => "System",
};
let text = msg.content.text_content();
if !text.is_empty() {
parts.push(format!("[{role_label}]\n{text}"));
let rendered = Self::render_content(&msg.content);
if !rendered.is_empty() {
parts.push(format!("[{role_label}]\n{rendered}"));
}
}
parts.join("\n\n")
}
/// Render message content for the text-only CLI prompt.
///
/// Text blocks pass through verbatim. Image blocks are rendered as
/// `[attachment: <media_type> image, ~N KB — not viewable on this
/// provider]` so the model receives a positive signal that an
/// attachment arrived. ToolUse/ToolResult/Thinking are omitted —
/// the CLI manages its own tool loop.
fn render_content(content: &MessageContent) -> String {
match content {
MessageContent::Text(s) => s.clone(),
MessageContent::Blocks(blocks) => blocks
.iter()
.filter_map(|b| match b {
ContentBlock::Text { text, .. } => {
if text.is_empty() {
None
} else {
Some(text.clone())
}
}
ContentBlock::Image { media_type, data } => {
// base64 → ~3/4 the length in decoded bytes.
let approx_kb = (data.len().saturating_mul(3) / 4) / 1024;
Some(format!(
"[attachment: {media_type} image, ~{approx_kb} KB — not viewable on this provider]"
))
}
ContentBlock::ToolUse { .. }
| ContentBlock::ToolResult { .. }
| ContentBlock::Thinking { .. }
| ContentBlock::RedactedThinking { .. }
| ContentBlock::Unknown => None,
})
.collect::<Vec<_>>()
.join("\n"),
}
}
/// Map a model ID like "claude-code/opus" to CLI --model flag value.
fn model_flag(model: &str) -> Option<String> {
let stripped = model.strip_prefix("claude-code/").unwrap_or(model);
@@ -727,6 +773,79 @@ mod tests {
assert!(prompt.contains("Hello"));
}
#[test]
fn test_build_prompt_renders_image_attachment_marker() {
use openfang_types::message::{ContentBlock, Message, MessageContent};
// ~12 KB of base64 — decoded ~9 KB.
let fake_b64 = "A".repeat(12 * 1024);
let request = CompletionRequest {
model: "claude-code/sonnet".to_string(),
messages: vec![Message {
role: Role::User,
content: MessageContent::Blocks(vec![
ContentBlock::Text {
text: "what's in this?".to_string(),
provider_metadata: None,
},
ContentBlock::Image {
media_type: "image/png".to_string(),
data: fake_b64,
},
]),
..Default::default()
}],
tools: vec![],
max_tokens: 1024,
temperature: 0.7,
system: None,
thinking: None,
};
let prompt = ClaudeCodeDriver::build_prompt(&request);
assert!(prompt.contains("what's in this?"), "text preserved");
assert!(
prompt.contains("[attachment: image/png image"),
"image rendered as synthetic attachment marker, got: {prompt}"
);
assert!(
prompt.contains("not viewable on this provider"),
"marker explains the limitation, got: {prompt}"
);
}
#[test]
fn test_build_prompt_image_only_still_emits_marker() {
use openfang_types::message::{ContentBlock, Message, MessageContent};
let request = CompletionRequest {
model: "claude-code/sonnet".to_string(),
messages: vec![Message {
role: Role::User,
content: MessageContent::Blocks(vec![ContentBlock::Image {
media_type: "image/jpeg".to_string(),
data: "Zm9v".to_string(),
}]),
..Default::default()
}],
tools: vec![],
max_tokens: 1024,
temperature: 0.7,
system: None,
thinking: None,
};
let prompt = ClaudeCodeDriver::build_prompt(&request);
assert!(
prompt.contains("[User]"),
"user role label emitted even with image-only content, got: {prompt}"
);
assert!(
prompt.contains("[attachment: image/jpeg image"),
"bare image renders marker, got: {prompt}"
);
}
#[test]
fn test_model_flag_mapping() {
assert_eq!(
+193 -4
View File
@@ -21,9 +21,9 @@ use openfang_types::model_catalog::{
HUGGINGFACE_BASE_URL, KIMI_CODING_BASE_URL, LEMONADE_BASE_URL, LMSTUDIO_BASE_URL,
MINIMAX_BASE_URL, MISTRAL_BASE_URL, MOONSHOT_BASE_URL, NOVITA_BASE_URL, NVIDIA_NIM_BASE_URL,
OLLAMA_BASE_URL, OPENAI_BASE_URL, OPENROUTER_BASE_URL, PERPLEXITY_BASE_URL, QIANFAN_BASE_URL,
QWEN_BASE_URL, REPLICATE_BASE_URL, SAMBANOVA_BASE_URL, TOGETHER_BASE_URL, VENICE_BASE_URL,
VLLM_BASE_URL, VOLCENGINE_BASE_URL, VOLCENGINE_CODING_BASE_URL, XAI_BASE_URL, ZAI_BASE_URL,
ZAI_CODING_BASE_URL, ZHIPU_BASE_URL, ZHIPU_CODING_BASE_URL,
QWEN_BASE_URL, REPLICATE_BASE_URL, REQUESTY_BASE_URL, SAMBANOVA_BASE_URL, TOGETHER_BASE_URL,
VENICE_BASE_URL, VLLM_BASE_URL, VOLCENGINE_BASE_URL, VOLCENGINE_CODING_BASE_URL, XAI_BASE_URL,
ZAI_BASE_URL, ZAI_CODING_BASE_URL, ZHIPU_BASE_URL, ZHIPU_CODING_BASE_URL,
};
use std::sync::Arc;
@@ -35,6 +35,64 @@ struct ProviderDefaults {
key_required: bool,
}
/// Resolve an OpenAI-compatible base URL for a local/self-hosted provider from
/// well-known environment variables. Returns `None` if no override is set.
///
/// This lets users point Ollama / LM Studio / vLLM / Lemonade at a remote host
/// (VPS, LXC, another box on the LAN) without editing `~/.openfang/config.toml`.
///
/// Recognised variables:
/// - `ollama` → `OLLAMA_BASE_URL`, then `OLLAMA_HOST` (Ollama CLI convention)
/// - `lmstudio` → `LMSTUDIO_BASE_URL`, then `LMSTUDIO_HOST`
/// - `vllm` → `VLLM_BASE_URL`, then `VLLM_HOST`
/// - `lemonade` → `LEMONADE_BASE_URL`, then `LEMONADE_HOST`
///
/// `*_HOST` values may omit the scheme and the `/v1` suffix
/// (e.g. `OLLAMA_HOST=192.168.1.50:11434`); both are normalised.
pub fn local_provider_url_from_env(provider: &str) -> Option<String> {
fn read(var: &str) -> Option<String> {
std::env::var(var)
.ok()
.map(|v| v.trim().to_string())
.filter(|v| !v.is_empty())
}
/// Normalise a host-style value into a full OpenAI-compatible base URL.
/// - Adds `http://` if no scheme is present.
/// - Appends `/v1` if not already present in the path.
fn normalize(raw: &str) -> String {
let mut url = if raw.contains("://") {
raw.trim_end_matches('/').to_string()
} else {
format!("http://{}", raw.trim_end_matches('/'))
};
// Add /v1 suffix if missing (OpenAI-compatible endpoints expect it).
// Be lenient: accept either `/v1` or `/v1/` already in place, and also
// `/openai/v1` style proxies.
let lower = url.to_lowercase();
if !lower.ends_with("/v1") && !lower.contains("/v1/") {
url.push_str("/v1");
}
url
}
let (primary, host_fallback) = match provider {
"ollama" => ("OLLAMA_BASE_URL", "OLLAMA_HOST"),
"lmstudio" => ("LMSTUDIO_BASE_URL", "LMSTUDIO_HOST"),
"vllm" => ("VLLM_BASE_URL", "VLLM_HOST"),
"lemonade" => ("LEMONADE_BASE_URL", "LEMONADE_HOST"),
_ => return None,
};
if let Some(v) = read(primary) {
return Some(normalize(&v));
}
if let Some(v) = read(host_fallback) {
return Some(normalize(&v));
}
None
}
/// Get defaults for known providers.
fn provider_defaults(provider: &str) -> Option<ProviderDefaults> {
match provider {
@@ -48,6 +106,11 @@ fn provider_defaults(provider: &str) -> Option<ProviderDefaults> {
api_key_env: "OPENROUTER_API_KEY",
key_required: true,
}),
"requesty" => Some(ProviderDefaults {
base_url: REQUESTY_BASE_URL,
api_key_env: "REQUESTY_API_KEY",
key_required: true,
}),
"deepseek" => Some(ProviderDefaults {
base_url: DEEPSEEK_BASE_URL,
api_key_env: "DEEPSEEK_API_KEY",
@@ -473,9 +536,14 @@ pub fn create_driver(config: &DriverConfig) -> Result<Arc<dyn LlmDriver>, LlmErr
)));
}
// Precedence for the base URL:
// 1. Explicit `DriverConfig.base_url` (from config.toml or `[provider_urls]`)
// 2. Well-known env vars for local providers (`OLLAMA_HOST`, etc.) — issue #1154
// 3. Hard-coded provider default (localhost for ollama/lmstudio/vllm/lemonade)
let base_url = config
.base_url
.clone()
.or_else(|| local_provider_url_from_env(provider))
.unwrap_or_else(|| defaults.base_url.to_string());
return Ok(Arc::new(openai::OpenAIDriver::new(api_key, base_url)));
@@ -629,13 +697,24 @@ pub fn known_providers() -> &'static [&'static str] {
]
}
/// Cross-module env-var serialisation lock for tests that mutate process env.
///
/// Several tests in this crate (drivers, model_catalog) set/unset the same
/// `OLLAMA_*` / `LMSTUDIO_*` env vars and would race under cargo's parallel
/// test runner. Anything that mutates those vars must hold this lock.
#[cfg(test)]
pub(crate) fn env_lock_for_tests() -> &'static std::sync::Mutex<()> {
use std::ops::Deref;
tests::ENV_LOCK.deref()
}
#[cfg(test)]
mod tests {
use super::*;
use std::ffi::OsString;
use std::sync::{LazyLock, Mutex};
static ENV_LOCK: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
pub(super) static ENV_LOCK: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
struct EnvVarGuard {
key: &'static str,
@@ -1118,4 +1197,114 @@ mod tests {
"unparseable env override should fall through to config field"
);
}
// ── Issue #1154: env-var URL overrides for local providers ──
#[test]
fn test_local_url_env_ollama_host_normalised() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::remove("OLLAMA_BASE_URL");
let _g2 = EnvVarGuard::set("OLLAMA_HOST", "192.168.1.50:11434");
let url = local_provider_url_from_env("ollama").expect("env should resolve");
assert_eq!(url, "http://192.168.1.50:11434/v1");
}
#[test]
fn test_local_url_env_ollama_base_url_wins() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::set("OLLAMA_BASE_URL", "https://llm.example.com/v1");
let _g2 = EnvVarGuard::set("OLLAMA_HOST", "should-be-ignored:11434");
let url = local_provider_url_from_env("ollama").expect("env should resolve");
assert_eq!(url, "https://llm.example.com/v1");
}
#[test]
fn test_local_url_env_lmstudio() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::remove("LMSTUDIO_BASE_URL");
let _g2 = EnvVarGuard::set("LMSTUDIO_HOST", "http://10.0.0.5:1234");
let url = local_provider_url_from_env("lmstudio").expect("env should resolve");
assert_eq!(url, "http://10.0.0.5:1234/v1");
}
#[test]
fn test_local_url_env_vllm() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::remove("VLLM_BASE_URL");
let _g2 = EnvVarGuard::set("VLLM_HOST", "vps.internal:8000");
let url = local_provider_url_from_env("vllm").expect("env should resolve");
assert_eq!(url, "http://vps.internal:8000/v1");
}
#[test]
fn test_local_url_env_unset_returns_none() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::remove("OLLAMA_BASE_URL");
let _g2 = EnvVarGuard::remove("OLLAMA_HOST");
assert!(local_provider_url_from_env("ollama").is_none());
}
#[test]
fn test_local_url_env_only_for_local_providers() {
// Cloud providers should never resolve via these helpers — they have
// their own *_API_KEY conventions and a fixed cloud base URL.
assert!(local_provider_url_from_env("openai").is_none());
assert!(local_provider_url_from_env("anthropic").is_none());
assert!(local_provider_url_from_env("groq").is_none());
}
#[test]
fn test_local_url_env_preserves_existing_v1_suffix() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::set("OLLAMA_BASE_URL", "http://1.2.3.4:11434/v1");
let _g2 = EnvVarGuard::remove("OLLAMA_HOST");
let url = local_provider_url_from_env("ollama").expect("env should resolve");
assert_eq!(url, "http://1.2.3.4:11434/v1");
}
#[test]
fn test_create_driver_ollama_uses_env_host() {
// End-to-end: when no explicit base_url and no OLLAMA_API_KEY, the
// driver should be constructed pointed at the env-supplied host.
// (We can't introspect the OpenAIDriver's base_url directly, but
// construction succeeds — separate unit covers URL resolution.)
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::remove("OLLAMA_BASE_URL");
let _g2 = EnvVarGuard::set("OLLAMA_HOST", "10.20.30.40:11434");
let _g3 = EnvVarGuard::remove("OLLAMA_API_KEY");
let config = DriverConfig {
provider: "ollama".to_string(),
api_key: None,
base_url: None,
skip_permissions: true,
subprocess_timeout_secs: None,
};
let driver = create_driver(&config);
assert!(
driver.is_ok(),
"ollama with OLLAMA_HOST set and no API key should construct: {:?}",
driver.err()
);
}
#[test]
fn test_create_driver_lmstudio_no_key_no_env_still_works() {
// Pre-#1154 regression guard: lmstudio with no env vars and no API key
// should still construct (falls back to localhost default).
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let _g1 = EnvVarGuard::remove("LMSTUDIO_BASE_URL");
let _g2 = EnvVarGuard::remove("LMSTUDIO_HOST");
let _g3 = EnvVarGuard::remove("LMSTUDIO_API_KEY");
let config = DriverConfig {
provider: "lmstudio".to_string(),
api_key: None,
base_url: None,
skip_permissions: true,
subprocess_timeout_secs: None,
};
let driver = create_driver(&config);
assert!(driver.is_ok(), "lmstudio default should construct");
}
}
+205 -43
View File
@@ -190,9 +190,14 @@ struct OaiMessage {
tool_calls: Option<Vec<OaiToolCall>>,
#[serde(skip_serializing_if = "Option::is_none")]
tool_call_id: Option<String>,
/// Moonshot Kimi: sent as empty string on assistant messages with tool_calls when using Kimi (thinking is disabled for multi-turn compatibility).
/// Legacy reasoning field. Pre-vLLM 0.19, DeepSeek, Moonshot/Kimi (empty string when thinking is disabled for tool_calls multi-turn).
#[serde(skip_serializing_if = "Option::is_none")]
reasoning_content: Option<String>,
/// New reasoning field per OpenAI GPT-OSS Responses-API convention.
/// vLLM 0.19+ (PR #33402) renamed `reasoning_content` to `reasoning`.
/// Issue #1157: emit both for backward compat across servers.
#[serde(skip_serializing_if = "Option::is_none")]
reasoning: Option<String>,
}
/// Content can be a plain string or an array of content parts (for images).
@@ -263,8 +268,23 @@ struct OaiResponseMessage {
content: Option<String>,
tool_calls: Option<Vec<OaiToolCall>>,
/// Reasoning/thinking content returned by some models (DeepSeek-R1, Qwen3, etc.)
/// via LM Studio, Ollama, and other local inference servers.
/// via LM Studio, Ollama, and pre-0.19 vLLM.
reasoning_content: Option<String>,
/// New reasoning field per OpenAI GPT-OSS Responses-API convention.
/// vLLM 0.19+ (PR #33402) emits this name instead of `reasoning_content`.
/// Issue #1157.
reasoning: Option<String>,
}
impl OaiResponseMessage {
/// Return whichever reasoning field the server populated.
/// vLLM ≥ 0.19 → `reasoning`. Older servers / DeepSeek / Qwen → `reasoning_content`.
fn reasoning_text(&self) -> Option<&str> {
self.reasoning
.as_deref()
.filter(|s| !s.is_empty())
.or_else(|| self.reasoning_content.as_deref().filter(|s| !s.is_empty()))
}
}
#[derive(Debug, Deserialize)]
@@ -292,16 +312,16 @@ fn strip_trailing_empty_assistant(messages: &mut Vec<OaiMessage>) {
/// Assemble an outbound assistant `OaiMessage` from `ContentBlock`s, replaying
/// any `Thinking` blocks in the format the upstream model originally emitted.
///
/// This is the fix for issue #1098 thinking-model state preservation.
/// This is the fix for issue #1098 — thinking-model state preservation.
/// Without this, `<think>...</think>` and `reasoning_content` are stripped on
/// the next turn so the model loses its prior reasoning trace and re-derives
/// the answer (degrading quality). We honour `provider_metadata.format`:
///
/// - `"reasoning_content"` emitted on the OpenAI `reasoning_content` field
/// - `"reasoning_content"` → emitted on the OpenAI `reasoning_content` field
/// (DeepSeek-R1, Qwen3, MiniMax M2 via LM Studio/Ollama)
/// - `"inline_think"` wrapped in `<think>...</think>` and prepended to
/// - `"inline_think"` → wrapped in `<think>...</think>` and prepended to
/// the visible content (MiniMax M2.5, Llama-3.3-think variants)
/// - missing/other fall back to the legacy Moonshot/Kimi behaviour
/// - missing/other → fall back to the legacy Moonshot/Kimi behaviour
/// (only emit `reasoning_content` when `needs_reasoning_content()` is true)
fn assemble_assistant_message(
blocks: &[ContentBlock],
@@ -360,7 +380,7 @@ fn assemble_assistant_message(
}
}
_ => {
// Unknown format preserve as inline_think since it's
// Unknown format — preserve as inline_think since it's
// safe (visible to the model as ordinary text). The
// legacy Moonshot path overrides this below.
let entry = format!("<think>{thinking}</think>");
@@ -387,14 +407,22 @@ fn assemble_assistant_message(
let has_tool_calls = !tool_calls.is_empty();
let needs_reasoning = driver.needs_reasoning_content(model);
// Final reasoning_content field: the per-block format hint wins; otherwise
// Final reasoning fields: the per-block format hint wins; otherwise
// fall back to legacy Moonshot/Kimi behaviour (empty string when needed).
let reasoning_content = if reasoning_field.is_some() {
reasoning_field
//
// Issue #1157: vLLM ≥ 0.19 renamed `reasoning_content` to `reasoning`.
// Emit BOTH fields so the persisted thinking trace reaches the model
// regardless of which server version we're talking to. Old servers
// ignore `reasoning`; new vLLM ignores `reasoning_content` (and would
// otherwise silently strip our thinking, see PR vllm#33402).
let (reasoning_content, reasoning) = if let Some(text) = reasoning_field {
(Some(text.clone()), Some(text))
} else if needs_reasoning {
Some(String::new())
// Moonshot/Kimi legacy contract: empty `reasoning_content` to disable
// thinking on tool-call multi-turn. The `reasoning` field stays unset.
(Some(String::new()), None)
} else {
None
(None, None)
};
OaiMessage {
@@ -415,6 +443,7 @@ fn assemble_assistant_message(
},
tool_call_id: None,
reasoning_content,
reasoning,
}
}
@@ -431,6 +460,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
@@ -444,6 +474,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
(Role::User, MessageContent::Text(text)) => {
@@ -453,6 +484,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
(Role::Assistant, MessageContent::Text(text)) => {
@@ -462,6 +494,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
(Role::User, MessageContent::Blocks(blocks)) => {
@@ -486,6 +519,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: Some(tool_use_id.clone()),
reasoning_content: None,
reasoning: None,
});
}
ContentBlock::Text { text, .. } => {
@@ -509,6 +543,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
}
@@ -676,7 +711,7 @@ impl LlmDriver for OpenAIDriver {
continue;
}
// Model doesn't support function calling retry without tools
// Model doesn't support function calling — retry without tools
// (e.g. GLM-5 on DashScope returns 500 "internal error" when tools are sent)
let body_lower = body.to_lowercase();
if !oai_request.tools.is_empty()
@@ -720,19 +755,19 @@ impl LlmDriver for OpenAIDriver {
let mut content = Vec::new();
let mut tool_calls = Vec::new();
// Capture reasoning_content from models that use a separate field
// (DeepSeek-R1, Qwen3, etc. via LM Studio/Ollama)
if let Some(ref reasoning) = choice.message.reasoning_content {
// Capture reasoning text from models that use a separate field.
// Issue #1098 (legacy `reasoning_content`) + #1157 (vLLM ≥ 0.19
// renamed it to `reasoning`). Accept either.
if let Some(reasoning) = choice.message.reasoning_text() {
if !reasoning.is_empty() {
debug!(
len = reasoning.len(),
"Captured reasoning_content from response"
);
debug!(len = reasoning.len(), "Captured reasoning from response");
// Mark the format so the outbound path knows to re-emit
// this as a `reasoning_content` field rather than as
// inline `<think>` tags. Issue #1098.
// this on the reasoning field rather than as inline
// `<think>` tags. The outbound assembler writes BOTH
// `reasoning` and `reasoning_content` for cross-server
// compat.
content.push(ContentBlock::Thinking {
thinking: reasoning.clone(),
thinking: reasoning.to_string(),
signature: None,
provider_metadata: Some(serde_json::json!({
"format": "reasoning_content"
@@ -741,14 +776,17 @@ impl LlmDriver for OpenAIDriver {
}
}
let already_has_reasoning = choice.message.reasoning_text().is_some();
if let Some(text) = choice.message.content {
if !text.is_empty() {
// Extract <think>...</think> blocks that some local models
// embed directly in the content field.
let (cleaned, thinking) = extract_think_tags(&text);
if let Some(think_text) = thinking {
// Only add if we didn't already get reasoning_content
if choice.message.reasoning_content.is_none() {
// Only add if we didn't already get a reasoning field
// (either legacy `reasoning_content` or new vLLM 0.19+
// `reasoning`). Issue #1157.
if !already_has_reasoning {
// Mark the format so we re-emit as inline `<think>`
// tags on the next turn (MiniMax/M2.5 style).
content.push(ContentBlock::Thinking {
@@ -843,7 +881,7 @@ impl LlmDriver for OpenAIDriver {
// this as a "silent failure" and loop unnecessarily.
if !content.is_empty() && usage.input_tokens == 0 && usage.output_tokens == 0 {
debug!(
"Response has content but no usage stats setting synthetic output_tokens=1"
"Response has content but no usage stats — setting synthetic output_tokens=1"
);
usage.output_tokens = 1;
}
@@ -877,6 +915,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
@@ -889,6 +928,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
(Role::User, MessageContent::Text(text)) => {
@@ -898,6 +938,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
(Role::Assistant, MessageContent::Text(text)) => {
@@ -907,6 +948,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: None,
reasoning_content: None,
reasoning: None,
});
}
(Role::User, MessageContent::Blocks(blocks)) => {
@@ -927,6 +969,7 @@ impl LlmDriver for OpenAIDriver {
tool_calls: None,
tool_call_id: Some(tool_use_id.clone()),
reasoning_content: None,
reasoning: None,
});
}
}
@@ -1091,7 +1134,7 @@ impl LlmDriver for OpenAIDriver {
continue;
}
// Provider doesn't support stream_options retry without it
// Provider doesn't support stream_options — retry without it
if status == 400
&& oai_request.stream_options.is_some()
&& attempt < max_retries
@@ -1104,7 +1147,7 @@ impl LlmDriver for OpenAIDriver {
continue;
}
// Model doesn't support function calling retry without tools
// Model doesn't support function calling — retry without tools
let body_lower = body.to_lowercase();
if !oai_request.tools.is_empty()
&& attempt < max_retries
@@ -1193,7 +1236,7 @@ impl LlmDriver for OpenAIDriver {
for choice in choices {
let delta = &choice["delta"];
// Text content delta route through think filter to
// Text content delta — route through think filter to
// strip <think>...</think> tags before they reach the client.
if let Some(text) = delta["content"].as_str() {
if !text.is_empty() {
@@ -1309,7 +1352,7 @@ impl LlmDriver for OpenAIDriver {
sse_lines = sse_line_count,
finish = ?finish_reason,
buffer_remaining = buffer.len(),
"SSE stream returned empty: 0 content, 0 tokens likely a silently failed request"
"SSE stream returned empty: 0 content, 0 tokens — likely a silently failed request"
);
} else {
debug!(
@@ -1449,7 +1492,9 @@ impl LlmDriver for OpenAIDriver {
// non-zero output_tokens so the agent loop doesn't misclassify
// this as a "silent failure" and loop unnecessarily.
if !content.is_empty() && usage.input_tokens == 0 && usage.output_tokens == 0 {
debug!("Stream has content but no usage stats — setting synthetic output_tokens=1");
debug!(
"Stream has content but no usage stats — setting synthetic output_tokens=1"
);
usage.output_tokens = 1;
}
@@ -1502,7 +1547,7 @@ fn extract_think_tags(text: &str) -> (String, Option<String>) {
break;
}
} else {
// Unclosed <think> tag treat everything after as thinking
// Unclosed <think> tag — treat everything after as thinking
let thought = cleaned[start + "<think>".len()..].trim().to_string();
if !thought.is_empty() {
thinking_parts.push(thought);
@@ -1609,7 +1654,7 @@ fn parse_groq_failed_tool_call(body: &str) -> Option<CompletionResponse> {
let args = &call_content[brace_pos..];
(name, args)
} else {
// No args just a tool name
// No args — just a tool name
(call_content.trim(), "{}")
};
@@ -1625,7 +1670,7 @@ fn parse_groq_failed_tool_call(body: &str) -> Option<CompletionResponse> {
}
if tool_calls.is_empty() {
// No tool calls found the model generated plain text but Groq rejected it.
// No tool calls found — the model generated plain text but Groq rejected it.
// Return it as a normal text response instead of failing.
if !failed.trim().is_empty() {
warn!("Recovering plain text from Groq failed_generation (no tool calls)");
@@ -1870,9 +1915,124 @@ mod tests {
let msg: OaiResponseMessage = serde_json::from_str(json).unwrap();
assert!(msg.content.is_none());
assert!(msg.reasoning_content.is_none());
assert!(msg.reasoning.is_none());
}
// ── Azure OpenAI tests ──────────────────────────────────────────
// ── Issue #1157: vLLM ≥ 0.19 reasoning field rename ─────────────────
/// vLLM 0.19+ (PR #33402) returns `reasoning` instead of
/// `reasoning_content`. We must accept the new name on ingress.
#[test]
fn test_oai_response_message_with_vllm_reasoning_field() {
let json =
r#"{"content": "Answer.", "reasoning": "I weighed A vs B.", "tool_calls": null}"#;
let msg: OaiResponseMessage = serde_json::from_str(json).unwrap();
assert_eq!(msg.content.as_deref(), Some("Answer."));
assert!(msg.reasoning_content.is_none());
assert_eq!(msg.reasoning.as_deref(), Some("I weighed A vs B."));
// reasoning_text() must surface the new field transparently.
assert_eq!(msg.reasoning_text(), Some("I weighed A vs B."));
}
/// If a server sends both fields (during the transition), prefer the
/// new `reasoning` name since that's what vLLM 0.19+ writes natively.
#[test]
fn test_reasoning_text_prefers_new_field() {
let json = r#"{"content": null, "reasoning": "new", "reasoning_content": "old"}"#;
let msg: OaiResponseMessage = serde_json::from_str(json).unwrap();
assert_eq!(msg.reasoning_text(), Some("new"));
}
/// If only the legacy field is set (older vLLM, DeepSeek, Ollama),
/// `reasoning_text()` must still return it.
#[test]
fn test_reasoning_text_falls_back_to_legacy_field() {
let json = r#"{"content": null, "reasoning_content": "legacy thinking"}"#;
let msg: OaiResponseMessage = serde_json::from_str(json).unwrap();
assert_eq!(msg.reasoning_text(), Some("legacy thinking"));
}
/// Outbound assembler must emit BOTH `reasoning` and `reasoning_content`
/// when a `Thinking` block carries the `reasoning_content` format hint,
/// so the persisted thinking reaches the model regardless of whether
/// the upstream server is pre- or post-vLLM 0.19.
#[test]
fn test_assemble_emits_both_reasoning_fields_for_vllm_compat() {
let driver = OpenAIDriver::new("test".to_string(), "http://localhost:8000/v1".to_string());
let blocks = vec![
ContentBlock::Thinking {
thinking: "MARKER-vllm-019".to_string(),
signature: None,
provider_metadata: Some(serde_json::json!({"format": "reasoning_content"})),
},
ContentBlock::Text {
text: "final".to_string(),
provider_metadata: None,
},
];
let msg = assemble_assistant_message(&blocks, "minimax-m2", &driver);
assert_eq!(
msg.reasoning_content.as_deref(),
Some("MARKER-vllm-019"),
"legacy reasoning_content field required for pre-0.19 vLLM and DeepSeek"
);
assert_eq!(
msg.reasoning.as_deref(),
Some("MARKER-vllm-019"),
"new reasoning field required for vLLM ≥ 0.19 (PR #33402)"
);
// Serialize and confirm the wire shape has both keys at top level.
let json = serde_json::to_value(&msg).unwrap();
assert_eq!(json["reasoning_content"], "MARKER-vllm-019");
assert_eq!(json["reasoning"], "MARKER-vllm-019");
}
/// Non-reasoning models (gpt-4o, claude, …) must NOT carry either
/// reasoning field on the wire. Regression guard for the dual-emit
/// change in #1157.
#[test]
fn test_assemble_no_reasoning_fields_for_plain_model() {
let driver = OpenAIDriver::new("test".to_string(), "https://api.openai.com/v1".to_string());
let blocks = vec![ContentBlock::Text {
text: "hi".to_string(),
provider_metadata: None,
}];
let msg = assemble_assistant_message(&blocks, "gpt-4o", &driver);
assert!(msg.reasoning_content.is_none());
assert!(msg.reasoning.is_none());
let json = serde_json::to_value(&msg).unwrap();
assert!(json.get("reasoning").is_none());
assert!(json.get("reasoning_content").is_none());
}
/// Moonshot/Kimi legacy contract: emit empty `reasoning_content` to
/// disable thinking on tool-call multi-turn. Issue #1157 must not
/// regress this — `reasoning` stays absent because Moonshot doesn't
/// understand the new name.
#[test]
fn test_assemble_moonshot_keeps_legacy_field_only() {
let driver =
OpenAIDriver::new("test".to_string(), "https://api.moonshot.cn/v1".to_string());
let blocks = vec![ContentBlock::ToolUse {
id: "call_1".to_string(),
name: "search".to_string(),
input: serde_json::json!({"q": "x"}),
provider_metadata: None,
}];
let msg = assemble_assistant_message(&blocks, "kimi-k2", &driver);
assert_eq!(
msg.reasoning_content.as_deref(),
Some(""),
"Moonshot Kimi requires empty reasoning_content on tool_calls turns"
);
assert!(
msg.reasoning.is_none(),
"Moonshot does not understand the new vLLM `reasoning` field"
);
}
// ── Azure OpenAI tests ──────────────────────────────────────────
#[test]
fn test_azure_driver_creation() {
@@ -1948,7 +2108,7 @@ mod tests {
assert_eq!(url, "https://api.moonshot.ai/v1/chat/completions");
}
// ── issue #1098: thinking-block round-trip ────────────────────────
// ── issue #1098: thinking-block round-trip ────────────────────────
/// Inline `<think>` blocks captured on ingress must be re-emitted in
/// historical assistant turns so MiniMax-style models retain reasoning
@@ -2008,7 +2168,10 @@ mod tests {
Some(OaiMessageContent::Text(t)) => t,
_ => panic!("expected text content"),
};
assert_eq!(content, "answer", "visible content must not include <think>");
assert_eq!(
content, "answer",
"visible content must not include <think>"
);
assert_eq!(
msg.reasoning_content.as_deref(),
Some("internal chain-of-thought"),
@@ -2017,11 +2180,10 @@ mod tests {
}
/// Without thinking blocks, the outbound message should be a plain
/// assistant message preserve the legacy shape.
/// assistant message — preserve the legacy shape.
#[test]
fn test_assemble_assistant_no_thinking_is_plain() {
let driver =
OpenAIDriver::new("test".to_string(), "https://api.openai.com/v1".to_string());
let driver = OpenAIDriver::new("test".to_string(), "https://api.openai.com/v1".to_string());
let blocks = vec![ContentBlock::Text {
text: "Hi.".to_string(),
provider_metadata: None,
@@ -2042,14 +2204,14 @@ mod tests {
// Step 1: parse server response shape.
let json = serde_json::json!({
"content": "Final answer.",
"reasoning_content": "I considered options A, B, and C",
"reasoning_content": "I considered options A, B, and C…",
"tool_calls": null
});
let server_msg: OaiResponseMessage = serde_json::from_value(json).unwrap();
assert_eq!(server_msg.content.as_deref(), Some("Final answer."));
assert_eq!(
server_msg.reasoning_content.as_deref(),
Some("I considered options A, B, and C")
Some("I considered options A, B, and C…")
);
// Step 2: simulate the driver building blocks (mirrors the live
@@ -2076,7 +2238,7 @@ mod tests {
// The reasoning_content field must round-trip verbatim.
assert_eq!(
outbound.reasoning_content.as_deref(),
Some("I considered options A, B, and C"),
Some("I considered options A, B, and C…"),
"issue #1098 regression: reasoning was stripped on resubmission"
);
}
@@ -258,7 +258,6 @@ fn host_net_fetch(state: &GuestState, params: &serde_json::Value) -> serde_json:
})
}
// ---------------------------------------------------------------------------
// Shell (capability-checked)
// ---------------------------------------------------------------------------
@@ -561,7 +560,10 @@ mod tests {
assert!(web_fetch::check_ssrf("http://127.0.0.1:8080/secret", &no_allow).is_err());
assert!(web_fetch::check_ssrf("http://localhost:3000/api", &no_allow).is_err());
assert!(web_fetch::check_ssrf("http://169.254.169.254/metadata", &no_allow).is_err());
assert!(web_fetch::check_ssrf("http://metadata.google.internal/v1/instance", &no_allow).is_err());
assert!(
web_fetch::check_ssrf("http://metadata.google.internal/v1/instance", &no_allow)
.is_err()
);
// These were previously missing from host_functions — now covered:
assert!(web_fetch::check_ssrf("http://[::1]:8080/secret", &no_allow).is_err());
assert!(web_fetch::check_ssrf("http://100.100.100.200/metadata", &no_allow).is_err());
+52 -2
View File
@@ -7,7 +7,15 @@ use tracing::warn;
/// Generate images via OpenAI's image generation API.
///
/// Requires OPENAI_API_KEY to be set.
pub async fn generate_image(request: &ImageGenRequest) -> Result<ImageGenResult, String> {
///
/// `base_url_override` (sourced from `MediaConfig.image_gen_base_url`) lets
/// callers redirect the request to a local OpenAI-compatible image service
/// (e.g. Lemonade/Flux, LM Studio). When `None`, the hardcoded
/// `https://api.openai.com/v1/images/generations` endpoint is used. Closes #1051.
pub async fn generate_image(
request: &ImageGenRequest,
base_url_override: Option<&str>,
) -> Result<ImageGenResult, String> {
// Validate request
request.validate()?;
@@ -30,9 +38,19 @@ pub async fn generate_image(request: &ImageGenRequest) -> Result<ImageGenResult,
body["quality"] = serde_json::json!(request.quality);
}
// `image_gen_base_url` (config.media.image_gen_base_url) overrides the
// hardcoded provider URL when set, allowing the same OpenAI-compat JSON
// wire format to be sent to a local image generation service
// (Lemonade/Flux, LM Studio, etc.) instead of the cloud provider. The
// Authorization header is still built from `OPENAI_API_KEY`; local
// services typically accept any non-empty bearer token. Closes #1051.
let url = base_url_override
.map(|base| format!("{}/v1/images/generations", base.trim_end_matches('/')))
.unwrap_or_else(|| "https://api.openai.com/v1/images/generations".to_string());
let client = reqwest::Client::new();
let response = client
.post("https://api.openai.com/v1/images/generations")
.post(&url)
.header("Authorization", format!("Bearer {}", api_key))
.header("Content-Type", "application/json")
.json(&body)
@@ -201,6 +219,38 @@ mod tests {
}
}
/// Closes #1051: when `image_gen_base_url` is set, the URL building
/// logic must use the override (with `/v1/images/generations` appended)
/// and strip any trailing slash from the user-supplied base. When unset,
/// the hardcoded provider URL is used.
#[test]
fn test_image_gen_base_url_override_logic() {
// Helper mirroring the URL construction in `generate_image`.
fn build(base: Option<&str>) -> String {
base.map(|b| format!("{}/v1/images/generations", b.trim_end_matches('/')))
.unwrap_or_else(|| "https://api.openai.com/v1/images/generations".to_string())
}
// Default: hardcoded URL preserved (backward compatibility).
assert_eq!(build(None), "https://api.openai.com/v1/images/generations");
// Override applied.
assert_eq!(
build(Some("http://127.0.0.1:7000")),
"http://127.0.0.1:7000/v1/images/generations"
);
// Trailing slash on the user-supplied base is stripped.
assert_eq!(
build(Some("http://127.0.0.1:7000/")),
"http://127.0.0.1:7000/v1/images/generations"
);
assert_eq!(
build(Some("https://images.example.com/")),
"https://images.example.com/v1/images/generations"
);
}
#[test]
fn test_save_images_creates_dir() {
let dir = tempfile::tempdir().unwrap();
@@ -24,6 +24,13 @@ impl MediaEngine {
}
}
/// Read-only access to the media configuration. Used by callers that
/// need the URL overrides (e.g. image_gen_base_url for #1051) without
/// taking ownership of the engine.
pub fn config(&self) -> &MediaConfig {
&self.config
}
/// Describe an image using a vision-capable LLM.
/// Auto-cascade: Anthropic -> OpenAI -> Gemini (based on API key availability).
pub async fn describe_image(
+174 -13
View File
@@ -10,8 +10,8 @@ use openfang_types::model_catalog::{
HUGGINGFACE_BASE_URL, KIMI_CODING_BASE_URL, LEMONADE_BASE_URL, LMSTUDIO_BASE_URL,
MINIMAX_BASE_URL, MISTRAL_BASE_URL, MOONSHOT_BASE_URL, NVIDIA_NIM_BASE_URL, OLLAMA_BASE_URL,
OPENAI_BASE_URL, OPENROUTER_BASE_URL, PERPLEXITY_BASE_URL, QIANFAN_BASE_URL, QWEN_BASE_URL,
REPLICATE_BASE_URL, SAMBANOVA_BASE_URL, TOGETHER_BASE_URL, VENICE_BASE_URL, VLLM_BASE_URL,
VOLCENGINE_BASE_URL, VOLCENGINE_CODING_BASE_URL, XAI_BASE_URL, ZAI_BASE_URL,
REPLICATE_BASE_URL, REQUESTY_BASE_URL, SAMBANOVA_BASE_URL, TOGETHER_BASE_URL, VENICE_BASE_URL,
VLLM_BASE_URL, VOLCENGINE_BASE_URL, VOLCENGINE_CODING_BASE_URL, XAI_BASE_URL, ZAI_BASE_URL,
ZAI_CODING_BASE_URL, ZHIPU_BASE_URL, ZHIPU_CODING_BASE_URL,
};
use std::collections::HashMap;
@@ -339,6 +339,27 @@ impl ModelCatalog {
}
}
/// Apply environment-variable URL overrides for local providers.
///
/// Honours the same env vars the drivers respect (see
/// `drivers::local_provider_url_from_env`): `OLLAMA_HOST` / `OLLAMA_BASE_URL`,
/// `LMSTUDIO_HOST` / `LMSTUDIO_BASE_URL`, `VLLM_HOST` / `VLLM_BASE_URL`,
/// `LEMONADE_HOST` / `LEMONADE_BASE_URL`. This keeps the dashboard's
/// "Providers" view in sync with what the driver actually connects to,
/// without requiring users to edit `config.toml` for remote local-LLM hosts
/// (VPS, LXC, LAN). See issue #1154.
pub fn apply_local_env_overrides(&mut self) {
for provider in ["ollama", "lmstudio", "vllm", "lemonade"] {
if let Some(url) = crate::drivers::local_provider_url_from_env(provider) {
if let Some(p) = self.providers.iter_mut().find(|p| p.id == provider) {
p.base_url = url;
// A custom host indicates intentional setup, surface it as configured.
p.auth_status = AuthStatus::Configured;
}
}
}
}
/// Apply a batch of provider URL overrides from config.
///
/// Each entry maps a provider ID to a custom base URL.
@@ -604,6 +625,15 @@ fn builtin_providers() -> Vec<ProviderInfo> {
auth_status: AuthStatus::Missing,
model_count: 0,
},
ProviderInfo {
id: "requesty".into(),
display_name: "Requesty".into(),
api_key_env: "REQUESTY_API_KEY".into(),
base_url: REQUESTY_BASE_URL.into(),
key_required: true,
auth_status: AuthStatus::Missing,
model_count: 0,
},
ProviderInfo {
id: "mistral".into(),
display_name: "Mistral AI".into(),
@@ -1022,10 +1052,7 @@ fn builtin_aliases() -> HashMap<String, String> {
),
("free", "openrouter/meta-llama/llama-3.3-70b-instruct:free"),
("free-reasoning", "openrouter/deepseek/deepseek-r1:free"),
(
"openrouter/free-coder",
"openrouter/qwen/qwen3-coder:free",
),
("openrouter/free-coder", "openrouter/qwen/qwen3-coder:free"),
(
"openrouter/free-large",
"openrouter/openai/gpt-oss-120b:free",
@@ -2053,6 +2080,80 @@ fn builtin_models() -> Vec<ModelCatalogEntry> {
aliases: vec!["hunter-alpha".into()],
},
// ══════════════════════════════════════════════════════════════
// Requesty (5) — router-style OpenAI-compatible gateway (issue #995)
// Hundreds of upstream models accessible via https://router.requesty.ai/v1
// ══════════════════════════════════════════════════════════════
ModelCatalogEntry {
id: "requesty/anthropic/claude-sonnet-4".into(),
display_name: "Claude Sonnet 4 (Requesty)".into(),
provider: "requesty".into(),
tier: ModelTier::Smart,
context_window: 200_000,
max_output_tokens: 64_000,
input_cost_per_m: 3.0,
output_cost_per_m: 15.0,
supports_tools: true,
supports_vision: true,
supports_streaming: true,
aliases: vec![],
},
ModelCatalogEntry {
id: "requesty/openai/gpt-4o".into(),
display_name: "GPT-4o (Requesty)".into(),
provider: "requesty".into(),
tier: ModelTier::Smart,
context_window: 128_000,
max_output_tokens: 16_384,
input_cost_per_m: 2.5,
output_cost_per_m: 10.0,
supports_tools: true,
supports_vision: true,
supports_streaming: true,
aliases: vec![],
},
ModelCatalogEntry {
id: "requesty/google/gemini-2.5-flash".into(),
display_name: "Gemini 2.5 Flash (Requesty)".into(),
provider: "requesty".into(),
tier: ModelTier::Smart,
context_window: 1_048_576,
max_output_tokens: 65_536,
input_cost_per_m: 0.15,
output_cost_per_m: 0.60,
supports_tools: true,
supports_vision: true,
supports_streaming: true,
aliases: vec![],
},
ModelCatalogEntry {
id: "requesty/deepseek/deepseek-chat".into(),
display_name: "DeepSeek V3 (Requesty)".into(),
provider: "requesty".into(),
tier: ModelTier::Smart,
context_window: 128_000,
max_output_tokens: 32_768,
input_cost_per_m: 0.14,
output_cost_per_m: 0.28,
supports_tools: true,
supports_vision: false,
supports_streaming: true,
aliases: vec![],
},
ModelCatalogEntry {
id: "requesty/meta-llama/llama-3.3-70b-instruct".into(),
display_name: "Llama 3.3 70B (Requesty)".into(),
provider: "requesty".into(),
tier: ModelTier::Balanced,
context_window: 128_000,
max_output_tokens: 32_768,
input_cost_per_m: 0.39,
output_cost_per_m: 0.39,
supports_tools: true,
supports_vision: false,
supports_streaming: true,
aliases: vec![],
},
// ══════════════════════════════════════════════════════════════
// Mistral (6)
// ══════════════════════════════════════════════════════════════
ModelCatalogEntry {
@@ -3982,7 +4083,7 @@ mod tests {
#[test]
fn test_catalog_has_providers() {
let catalog = ModelCatalog::new();
assert_eq!(catalog.list_providers().len(), 41);
assert_eq!(catalog.list_providers().len(), 42);
}
#[test]
@@ -4611,9 +4712,9 @@ mod tests {
#[test]
fn test_openrouter_free_alias_supports_tools() {
let catalog = ModelCatalog::new();
let entry = catalog.find_model("openrouter/free").expect(
"openrouter/free alias must resolve to a known model",
);
let entry = catalog
.find_model("openrouter/free")
.expect("openrouter/free alias must resolve to a known model");
assert_eq!(entry.provider, "openrouter");
assert!(
entry.supports_tools,
@@ -4626,9 +4727,7 @@ mod tests {
#[test]
fn test_openrouter_free_short_alias_supports_tools() {
let catalog = ModelCatalog::new();
let entry = catalog
.find_model("free")
.expect("free alias must resolve");
let entry = catalog.find_model("free").expect("free alias must resolve");
assert_eq!(entry.provider, "openrouter");
assert!(
entry.supports_tools,
@@ -4702,4 +4801,66 @@ mod tests {
"qwen-2.5-7b-instruct:free has no tool-supporting free endpoint"
);
}
// ── Requesty provider (issue #995) ────────────────────────────────────
/// Requesty must be registered as a provider with the correct base URL
/// and env var, and at least one of its catalog models must resolve.
#[test]
fn test_requesty_provider_and_models_present() {
let catalog = ModelCatalog::new();
let provider = catalog
.list_providers()
.iter()
.find(|p| p.id == "requesty")
.expect("requesty provider must be registered");
assert_eq!(provider.display_name, "Requesty");
assert_eq!(provider.api_key_env, "REQUESTY_API_KEY");
assert_eq!(provider.base_url, "https://router.requesty.ai/v1");
assert!(provider.key_required);
assert!(
provider.model_count >= 1,
"requesty must have at least one model in catalog"
);
let entry = catalog
.find_model("requesty/anthropic/claude-sonnet-4")
.expect("requesty/anthropic/claude-sonnet-4 must resolve");
assert_eq!(entry.provider, "requesty");
assert!(entry.supports_tools);
}
// ── Issue #1154: env-var overrides for local provider URLs ──
/// Local guard so this catalog test doesn't clash with the driver tests
/// that touch the same env vars. We acquire the cross-module lock from
/// the drivers module to serialise.
#[test]
fn test_apply_local_env_overrides_ollama() {
// Serialise with driver-side env tests that touch OLLAMA_*.
let _lock = crate::drivers::env_lock_for_tests()
.lock()
.unwrap_or_else(|e| e.into_inner());
let prev_base = std::env::var_os("OLLAMA_BASE_URL");
let prev_host = std::env::var_os("OLLAMA_HOST");
std::env::remove_var("OLLAMA_BASE_URL");
std::env::set_var("OLLAMA_HOST", "172.16.0.10:11434");
let mut catalog = ModelCatalog::new();
catalog.apply_local_env_overrides();
let ollama = catalog.get_provider("ollama").unwrap();
assert_eq!(ollama.base_url, "http://172.16.0.10:11434/v1");
assert_eq!(ollama.auth_status, AuthStatus::Configured);
// Restore env
if let Some(v) = prev_base {
std::env::set_var("OLLAMA_BASE_URL", v);
}
if let Some(v) = prev_host {
std::env::set_var("OLLAMA_HOST", v);
} else {
std::env::remove_var("OLLAMA_HOST");
}
}
}
@@ -35,6 +35,12 @@ pub const SAFE_ENV_VARS_WINDOWS: &[&str] = &[
/// - On Windows, the Windows-specific safe variables (`SAFE_ENV_VARS_WINDOWS`)
/// - Any additional variables the caller explicitly allows via `allowed_env_vars`
///
/// `allowed_env_vars` accepts either explicit variable names or the special
/// wildcard entry `"*"`, which forwards every variable present in the parent
/// process. Use the wildcard only when the operator has explicitly opted in
/// (e.g. `exec_policy.shell_env_passthrough = ["*"]`) — it will leak any
/// secret the parent holds into the child.
///
/// Variables that are not set in the current process environment are silently
/// skipped (rather than being set to empty strings).
pub fn sandbox_command(cmd: &mut tokio::process::Command, allowed_env_vars: &[String]) {
@@ -55,6 +61,14 @@ pub fn sandbox_command(cmd: &mut tokio::process::Command, allowed_env_vars: &[St
}
}
// Wildcard: forward every var from the parent process.
if allowed_env_vars.iter().any(|v| v == "*") {
for (key, val) in std::env::vars() {
cmd.env(key, val);
}
return;
}
// Re-add caller-specified allowed vars.
for var in allowed_env_vars {
if let Ok(val) = std::env::var(var) {
@@ -63,6 +77,22 @@ pub fn sandbox_command(cmd: &mut tokio::process::Command, allowed_env_vars: &[St
}
}
/// Merge two env-passthrough lists (hand-granted + exec-policy-granted),
/// deduplicating entries. If either contains `"*"`, the result is just `["*"]`
/// (wildcard subsumes anything else).
pub fn merge_env_passthrough(a: &[String], b: &[String]) -> Vec<String> {
if a.iter().any(|v| v == "*") || b.iter().any(|v| v == "*") {
return vec!["*".to_string()];
}
let mut out: Vec<String> = Vec::with_capacity(a.len() + b.len());
for v in a.iter().chain(b.iter()) {
if !out.iter().any(|existing| existing == v) {
out.push(v.clone());
}
}
out
}
/// Validates that an executable path does not contain directory traversal
/// components (`..`).
///
@@ -711,6 +741,40 @@ pub async fn wait_or_kill_with_idle(
mod tests {
use super::*;
// ── Env passthrough merge (issue #1169) ────────────────────────────
#[test]
fn test_merge_env_passthrough_empty() {
let merged = merge_env_passthrough(&[], &[]);
assert!(merged.is_empty());
}
#[test]
fn test_merge_env_passthrough_dedup() {
let a = vec!["TZ".to_string(), "HOME".to_string()];
let b = vec!["TZ".to_string(), "PATH".to_string()];
let merged = merge_env_passthrough(&a, &b);
assert_eq!(merged, vec!["TZ", "HOME", "PATH"]);
}
#[test]
fn test_merge_env_passthrough_wildcard_a() {
let merged = merge_env_passthrough(&["*".to_string()], &["TZ".to_string()]);
assert_eq!(merged, vec!["*"]);
}
#[test]
fn test_merge_env_passthrough_wildcard_b() {
let merged = merge_env_passthrough(&["TZ".to_string()], &["*".to_string()]);
assert_eq!(merged, vec!["*"]);
}
#[test]
fn test_exec_policy_default_has_empty_passthrough() {
let policy = openfang_types::config::ExecPolicy::default();
assert!(policy.shell_env_passthrough.is_empty());
}
#[test]
fn test_validate_path() {
// Clean paths should be accepted.
+462 -3
View File
@@ -205,6 +205,7 @@ pub async fn execute_tool(
"file_read" => tool_file_read(input, workspace_root).await,
"file_write" => tool_file_write(input, workspace_root).await,
"file_list" => tool_file_list(input, workspace_root).await,
"create_directory" => tool_create_directory(input, workspace_root).await,
"apply_patch" => tool_apply_patch(input, workspace_root).await,
// Web tools (upgraded: multi-provider search, SSRF-protected fetch)
@@ -331,7 +332,7 @@ pub async fn execute_tool(
"media_transcribe" => tool_media_transcribe(input, media_engine).await,
// Image generation tool
"image_generate" => tool_image_generate(input, workspace_root).await,
"image_generate" => tool_image_generate(input, workspace_root, media_engine).await,
// TTS/STT tools
"text_to_speech" => tool_text_to_speech(input, tts_engine, workspace_root).await,
@@ -477,6 +478,11 @@ pub async fn execute_tool(
}
},
// Skill introspection tools (issue #1038)
"skill_list" => tool_skill_list(skill_registry),
"skill_describe" => tool_skill_describe(input, skill_registry),
"skill_execute" => tool_skill_execute(input, skill_registry).await,
// Canvas / A2UI tool
"canvas_present" => tool_canvas_present(input, workspace_root).await,
@@ -595,6 +601,17 @@ pub fn builtin_tool_definitions() -> Vec<ToolDefinition> {
"required": ["path"]
}),
},
ToolDefinition {
name: "create_directory".to_string(),
description: "Create a directory (and any missing parent directories) at the given path. Paths are relative to the agent workspace. Idempotent: succeeds if the directory already exists.".to_string(),
input_schema: serde_json::json!({
"type": "object",
"properties": {
"path": { "type": "string", "description": "The directory path to create" }
},
"required": ["path"]
}),
},
ToolDefinition {
name: "apply_patch".to_string(),
description: "Apply a multi-hunk diff patch to add, update, move, or delete files. Use this for targeted edits instead of full file overwrites.".to_string(),
@@ -1296,6 +1313,42 @@ pub fn builtin_tool_definitions() -> Vec<ToolDefinition> {
"required": ["html"]
}),
},
// --- Skill introspection tools (issue #1038) ---
// These let the agent discover and read installed skills without
// touching the filesystem. Global skills live at ~/.openfang/skills/
// which is outside the workspace sandbox — file_read cannot reach them.
ToolDefinition {
name: "skill_list".to_string(),
description: "List all installed skills available to this agent. Returns name, version, description, runtime type, and provided tool names. Use this instead of file_list on the skills directory.".to_string(),
input_schema: serde_json::json!({
"type": "object",
"properties": {}
}),
},
ToolDefinition {
name: "skill_describe".to_string(),
description: "Read the full description (SKILL.md body / prompt context) of an installed skill by name. Use this instead of file_read on a skill's SKILL.md file — global skills live outside the workspace sandbox and cannot be read with file_read.".to_string(),
input_schema: serde_json::json!({
"type": "object",
"properties": {
"name": { "type": "string", "description": "The skill name (as returned by skill_list)" }
},
"required": ["name"]
}),
},
ToolDefinition {
name: "skill_execute".to_string(),
description: "Execute a tool provided by an installed skill. For code-runtime skills (Python/Node/Shell) this invokes the underlying script. For prompt-only skills this returns the skill's instruction body so the agent can follow it using built-in tools.".to_string(),
input_schema: serde_json::json!({
"type": "object",
"properties": {
"skill": { "type": "string", "description": "The skill name (as returned by skill_list)" },
"tool": { "type": "string", "description": "Optional name of a tool the skill provides. Omit to invoke the skill's default behavior (returns SKILL.md body for prompt-only skills)." },
"input": { "type": "object", "description": "Optional JSON input for the skill tool" }
},
"required": ["skill"]
}),
},
]
}
@@ -1358,6 +1411,82 @@ async fn tool_file_write(
))
}
/// Resolve a directory path for creation. Unlike `resolve_file_path`, this walks
/// up the path to find the nearest existing ancestor, canonicalizes that, and
/// re-appends the missing segments. This lets `create_directory` accept nested
/// paths like `a/b/c/d` even when none of `a`, `b`, `c` exist yet.
fn resolve_directory_path_for_create(
raw_path: &str,
workspace_root: Option<&Path>,
) -> Result<PathBuf, String> {
// Reject `..` components regardless of workspace.
let _ = validate_path(raw_path)?;
let Some(root) = workspace_root else {
return Ok(PathBuf::from(raw_path));
};
let path = Path::new(raw_path);
let candidate = if path.is_absolute() {
path.to_path_buf()
} else {
root.join(path)
};
let canon_root = root
.canonicalize()
.map_err(|e| format!("Failed to resolve workspace root: {e}"))?;
// Walk up to find the nearest existing ancestor, canonicalize it, then
// re-append the missing tail.
let mut existing: PathBuf = candidate.clone();
let mut tail: Vec<std::ffi::OsString> = Vec::new();
while !existing.exists() {
let parent = match existing.parent() {
Some(p) => p.to_path_buf(),
None => return Err("Invalid path: no existing ancestor".to_string()),
};
let name = match existing.file_name() {
Some(n) => n.to_os_string(),
None => return Err("Invalid path: no filename component".to_string()),
};
tail.push(name);
existing = parent;
}
let canon_existing = existing
.canonicalize()
.map_err(|e| format!("Failed to resolve ancestor directory: {e}"))?;
let mut resolved = canon_existing;
for segment in tail.into_iter().rev() {
resolved.push(segment);
}
if !resolved.starts_with(&canon_root) {
return Err(format!(
"Access denied: path '{raw_path}' resolves outside workspace"
));
}
Ok(resolved)
}
async fn tool_create_directory(
input: &serde_json::Value,
workspace_root: Option<&Path>,
) -> Result<String, String> {
let raw_path = input["path"].as_str().ok_or("Missing 'path' parameter")?;
if raw_path.is_empty() {
return Err("'path' parameter is empty".to_string());
}
let resolved = resolve_directory_path_for_create(raw_path, workspace_root)?;
tokio::fs::create_dir_all(&resolved)
.await
.map_err(|e| format!("Failed to create directory: {e}"))?;
Ok(format!("Created directory {}", resolved.display()))
}
async fn tool_file_list(
input: &serde_json::Value,
workspace_root: Option<&Path>,
@@ -1579,7 +1708,18 @@ async fn tool_shell_exec(
// SECURITY: Isolate environment to prevent credential leakage.
// Hand settings may grant access to specific provider API keys.
crate::subprocess_sandbox::sandbox_command(&mut cmd, allowed_env);
//
// Operators can also forward additional vars via
// `exec_policy.shell_env_passthrough` (issue #1169). This is the path
// Docker users hit: their container env (TZ, GOG_*, etc.) is present
// in PID 1 but `env_clear()` strips it. Listing names (or `"*"`) here
// re-adds them to the child.
let policy_env_passthrough: &[String] = exec_policy
.map(|p| p.shell_env_passthrough.as_slice())
.unwrap_or(&[]);
let merged_env =
crate::subprocess_sandbox::merge_env_passthrough(allowed_env, policy_env_passthrough);
crate::subprocess_sandbox::sandbox_command(&mut cmd, &merged_env);
// Ensure UTF-8 output on Windows
#[cfg(windows)]
@@ -2947,6 +3087,7 @@ async fn tool_media_transcribe(
async fn tool_image_generate(
input: &serde_json::Value,
workspace_root: Option<&Path>,
media_engine: Option<&crate::media_understanding::MediaEngine>,
) -> Result<String, String> {
let prompt = input["prompt"]
.as_str()
@@ -2976,7 +3117,10 @@ async fn tool_image_generate(
count,
};
let result = crate::image_gen::generate_image(&request).await?;
// Closes #1051: route to a local OpenAI-compatible image generation
// service when `media.image_gen_base_url` is set.
let base_url_override = media_engine.and_then(|e| e.config().image_gen_base_url.as_deref());
let result = crate::image_gen::generate_image(&request, base_url_override).await?;
// Save images to workspace if available
let saved_paths = if let Some(workspace) = workspace_root {
@@ -3433,6 +3577,165 @@ async fn tool_canvas_present(
serde_json::to_string_pretty(&response).map_err(|e| format!("Serialize error: {e}"))
}
// ---------------------------------------------------------------------------
// Skill introspection tools (issue #1038)
//
// Global skills live at ~/.openfang/skills/ which is outside the agent
// workspace sandbox. Without these tools the LLM falls back to file_read /
// shell_exec to inspect SKILL.md files — which fail with path-resolution
// errors because file_read is workspace-scoped. These tools surface the
// already-loaded skill registry directly to the agent.
// ---------------------------------------------------------------------------
/// List all skills available to this agent, with their provided tool names.
fn tool_skill_list(skill_registry: Option<&SkillRegistry>) -> Result<String, String> {
let registry = match skill_registry {
Some(r) => r,
None => return Ok("No skill registry available.".to_string()),
};
let skills = registry.list();
if skills.is_empty() {
return Ok("No skills installed. Install skills via the dashboard or `openfang skill install <name>`.".to_string());
}
let entries: Vec<serde_json::Value> = skills
.iter()
.map(|s| {
let tool_names: Vec<String> = s
.manifest
.tools
.provided
.iter()
.map(|t| t.name.clone())
.collect();
serde_json::json!({
"name": s.manifest.skill.name,
"version": s.manifest.skill.version,
"description": s.manifest.skill.description,
"runtime": format!("{:?}", s.manifest.runtime.runtime_type),
"enabled": s.enabled,
"tools": tool_names,
"has_prompt_context": s.manifest.prompt_context.as_ref().is_some_and(|c| !c.is_empty()),
})
})
.collect();
serde_json::to_string_pretty(&serde_json::json!({
"count": entries.len(),
"skills": entries,
}))
.map_err(|e| format!("Serialize error: {e}"))
}
/// Return the full description (SKILL.md body) of a named skill.
fn tool_skill_describe(
input: &serde_json::Value,
skill_registry: Option<&SkillRegistry>,
) -> Result<String, String> {
let name = input["name"]
.as_str()
.ok_or("Missing 'name' parameter")?
.trim();
let registry = skill_registry.ok_or("No skill registry available")?;
let skill = registry.get(name).ok_or_else(|| {
format!("Skill '{name}' not found. Use skill_list to see installed skills.")
})?;
let body = skill
.manifest
.prompt_context
.clone()
.unwrap_or_else(|| "(No prompt context body — this skill provides executable tools only. Use skill_execute or call its tools directly.)".to_string());
let tool_names: Vec<String> = skill
.manifest
.tools
.provided
.iter()
.map(|t| t.name.clone())
.collect();
let response = serde_json::json!({
"name": skill.manifest.skill.name,
"version": skill.manifest.skill.version,
"description": skill.manifest.skill.description,
"runtime": format!("{:?}", skill.manifest.runtime.runtime_type),
"tools": tool_names,
"body": body,
});
serde_json::to_string_pretty(&response).map_err(|e| format!("Serialize error: {e}"))
}
/// Execute a skill's tool, or for prompt-only skills return the description body.
async fn tool_skill_execute(
input: &serde_json::Value,
skill_registry: Option<&SkillRegistry>,
) -> Result<String, String> {
let skill_name = input["skill"]
.as_str()
.ok_or("Missing 'skill' parameter")?
.trim();
let registry = skill_registry.ok_or("No skill registry available")?;
let skill = registry.get(skill_name).ok_or_else(|| {
format!("Skill '{skill_name}' not found. Use skill_list to see installed skills.")
})?;
// If no tool name was given, default behavior depends on runtime.
// For prompt-only skills, return the SKILL.md body (most useful response
// for issue #1038's daily-journal style skills).
let tool_name = input["tool"].as_str().map(|s| s.trim());
let tool_input = input.get("input").cloned().unwrap_or(serde_json::json!({}));
let resolved_tool = match tool_name {
Some(t) if !t.is_empty() => t.to_string(),
_ => {
// No tool specified — return SKILL.md body so the agent can act on it.
if let Some(ref body) = skill.manifest.prompt_context {
if !body.is_empty() {
let response = serde_json::json!({
"skill": skill.manifest.skill.name,
"mode": "prompt_context",
"body": body,
"note": "This is a prompt-only skill. Follow the instructions in 'body' using your built-in tools.",
});
return serde_json::to_string_pretty(&response)
.map_err(|e| format!("Serialize error: {e}"));
}
}
// Fall through: pick the first provided tool if any
skill
.manifest
.tools
.provided
.first()
.map(|t| t.name.clone())
.ok_or_else(|| {
format!("Skill '{skill_name}' provides no tools and has no prompt body.")
})?
}
};
match openfang_skills::loader::execute_skill_tool(
&skill.manifest,
&skill.path,
&resolved_tool,
&tool_input,
)
.await
{
Ok(result) => {
let content = serde_json::to_string_pretty(&serde_json::json!({
"skill": skill.manifest.skill.name,
"tool": resolved_tool,
"output": result.output,
"is_error": result.is_error,
}))
.unwrap_or_else(|_| result.output.to_string());
if result.is_error {
Err(content)
} else {
Ok(content)
}
}
Err(e) => Err(format!("Skill execution failed: {e}")),
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -3448,6 +3751,9 @@ mod tests {
let names: Vec<&str> = tools.iter().map(|t| t.name.as_str()).collect();
// Original 12
assert!(names.contains(&"file_read"));
assert!(names.contains(&"file_write"));
assert!(names.contains(&"file_list"));
assert!(names.contains(&"create_directory"));
assert!(names.contains(&"shell_exec"));
assert!(names.contains(&"agent_send"));
assert!(names.contains(&"agent_spawn"));
@@ -3503,6 +3809,67 @@ mod tests {
assert!(names.contains(&"docker_exec"));
// Canvas tool
assert!(names.contains(&"canvas_present"));
// 3 skill introspection tools (issue #1038)
assert!(names.contains(&"skill_list"));
assert!(names.contains(&"skill_describe"));
assert!(names.contains(&"skill_execute"));
}
/// Issue #1038: skill_list, skill_describe, skill_execute work without
/// touching the filesystem so global skills (outside the workspace
/// sandbox) are reachable by the agent.
#[tokio::test]
async fn test_skill_tools_no_filesystem_access() {
use openfang_skills::registry::SkillRegistry;
use tempfile::TempDir;
// Build a skills directory containing one prompt-only SKILL.md skill
// (mirroring the user's daily-journal scenario from #1038).
let global_dir = TempDir::new().unwrap();
let skill_dir = global_dir.path().join("daily-journal");
std::fs::create_dir_all(&skill_dir).unwrap();
std::fs::write(
skill_dir.join("SKILL.md"),
"---\nname: daily-journal\ndescription: Keep a daily journal\n---\n\
# Daily Journal\n\nWrite one paragraph per day about what you learned.",
)
.unwrap();
let mut registry = SkillRegistry::new(global_dir.path().to_path_buf());
registry.load_all().unwrap();
assert_eq!(registry.count(), 1);
// skill_list returns the global skill without any filesystem call
let list_out = tool_skill_list(Some(&registry)).unwrap();
assert!(list_out.contains("daily-journal"));
assert!(list_out.contains("Keep a daily journal"));
// skill_describe returns the SKILL.md body — no file_read needed
let desc_out = tool_skill_describe(
&serde_json::json!({ "name": "daily-journal" }),
Some(&registry),
)
.unwrap();
assert!(desc_out.contains("Daily Journal"));
assert!(desc_out.contains("Write one paragraph"));
// skill_execute on a prompt-only skill returns the body in 'prompt_context' mode
let exec_out = tool_skill_execute(
&serde_json::json!({ "skill": "daily-journal" }),
Some(&registry),
)
.await
.unwrap();
assert!(exec_out.contains("prompt_context"));
assert!(exec_out.contains("Daily Journal"));
// skill_describe on a missing skill returns a helpful error
let missing = tool_skill_describe(
&serde_json::json!({ "name": "no-such-skill" }),
Some(&registry),
);
assert!(missing.is_err());
assert!(missing.unwrap_err().contains("not found"));
}
#[test]
@@ -3619,6 +3986,98 @@ mod tests {
assert!(result.content.contains("traversal"));
}
#[tokio::test]
async fn test_create_directory_path_traversal_blocked() {
let result = execute_tool(
"test-id",
"create_directory",
&serde_json::json!({"path": "../../etc/evil"}),
None,
None,
None,
None,
None,
None,
None,
None,
None,
None, // media_engine
None, // exec_policy
None, // tts_engine
None, // docker_config
None, // process_manager
)
.await;
assert!(result.is_error);
assert!(result.content.contains("traversal"));
}
#[tokio::test]
async fn test_create_directory_creates_nested() {
let tmp = tempfile::TempDir::new().unwrap();
let root = tmp.path();
let result = tool_create_directory(&serde_json::json!({"path": "a/b/c"}), Some(root)).await;
assert!(result.is_ok(), "Expected Ok, got: {:?}", result);
let expected = root.join("a").join("b").join("c");
assert!(
expected.is_dir(),
"Expected directory to exist: {}",
expected.display()
);
}
#[tokio::test]
async fn test_create_directory_idempotent() {
let tmp = tempfile::TempDir::new().unwrap();
let root = tmp.path();
// First create
let r1 = tool_create_directory(&serde_json::json!({"path": "data/logs"}), Some(root)).await;
assert!(r1.is_ok());
// Second create on existing dir should also succeed
let r2 = tool_create_directory(&serde_json::json!({"path": "data/logs"}), Some(root)).await;
assert!(r2.is_ok(), "Expected idempotent success, got: {:?}", r2);
}
#[tokio::test]
async fn test_create_directory_missing_path_param() {
let result = tool_create_directory(&serde_json::json!({}), None).await;
assert!(result.is_err());
let msg = result.unwrap_err();
assert!(msg.contains("Missing 'path'"), "got: {msg}");
}
#[tokio::test]
async fn test_create_directory_dispatch_via_execute_tool() {
let tmp = tempfile::TempDir::new().unwrap();
let root = tmp.path().to_path_buf();
let result = execute_tool(
"test-id",
"create_directory",
&serde_json::json!({"path": "nested/folder"}),
None, // kernel
None, // allowed_tools
None, // caller_agent_id
None, // skill_registry
None, // mcp_connections
None, // web_ctx
None, // browser_ctx
None, // allowed_env_vars
Some(root.as_path()), // workspace_root
None, // media_engine
None, // exec_policy
None, // tts_engine
None, // docker_config
None, // process_manager
)
.await;
assert!(
!result.is_error,
"Expected success, got: {}",
result.content
);
assert!(root.join("nested").join("folder").is_dir());
}
#[tokio::test]
async fn test_file_list_path_traversal_blocked() {
let result = execute_tool(
+136 -3
View File
@@ -19,11 +19,38 @@ pub struct TtsResult {
/// Text-to-speech engine.
pub struct TtsEngine {
config: TtsConfig,
/// Optional override for OpenAI TTS base URL. When set, the engine POSTs
/// to `<openai_base_url>/v1/audio/speech` instead of the hardcoded
/// `https://api.openai.com/v1/audio/speech`. Sourced from
/// `MediaConfig.tts_openai_base_url`. Closes #1051.
openai_base_url: Option<String>,
/// Optional override for ElevenLabs TTS base URL. When set, the engine
/// POSTs to `<elevenlabs_base_url>/v1/text-to-speech/{voice_id}` instead
/// of the hardcoded `https://api.elevenlabs.io/...`. Sourced from
/// `MediaConfig.tts_elevenlabs_base_url`. Closes #1051.
elevenlabs_base_url: Option<String>,
}
impl TtsEngine {
pub fn new(config: TtsConfig) -> Self {
Self { config }
Self {
config,
openai_base_url: None,
elevenlabs_base_url: None,
}
}
/// Attach optional base-URL overrides from `MediaConfig`. Use this to
/// route TTS calls at a local OpenAI-compatible service (e.g.
/// Lemonade/Kokoro, LM Studio) or an ElevenLabs proxy. Closes #1051.
pub fn with_base_urls(
mut self,
openai_base_url: Option<String>,
elevenlabs_base_url: Option<String>,
) -> Self {
self.openai_base_url = openai_base_url;
self.elevenlabs_base_url = elevenlabs_base_url;
self
}
/// Detect which TTS provider is available based on environment variables.
@@ -100,9 +127,21 @@ impl TtsEngine {
"speed": self.config.openai.speed,
});
// `tts_openai_base_url` (config.media.tts_openai_base_url) overrides
// the hardcoded provider URL when set, allowing the same OpenAI-compat
// JSON wire format to be sent to a local TTS service (Lemonade/Kokoro,
// LM Studio, etc.) instead of the cloud provider. The Authorization
// header is still built from `OPENAI_API_KEY`; local services typically
// accept any non-empty bearer token. Closes #1051.
let url = self
.openai_base_url
.as_deref()
.map(|base| format!("{}/v1/audio/speech", base.trim_end_matches('/')))
.unwrap_or_else(|| "https://api.openai.com/v1/audio/speech".to_string());
let client = reqwest::Client::new();
let response = client
.post("https://api.openai.com/v1/audio/speech")
.post(&url)
.header("Authorization", format!("Bearer {}", api_key))
.header("Content-Type", "application/json")
.json(&body)
@@ -161,7 +200,17 @@ impl TtsEngine {
std::env::var("ELEVENLABS_API_KEY").map_err(|_| "ELEVENLABS_API_KEY not set")?;
let voice_id = voice_override.unwrap_or(&self.config.elevenlabs.voice_id);
let url = format!("https://api.elevenlabs.io/v1/text-to-speech/{}", voice_id);
// `tts_elevenlabs_base_url` (config.media.tts_elevenlabs_base_url)
// overrides the hardcoded provider URL when set, allowing the same
// ElevenLabs JSON wire format to be routed through a proxy or
// self-hosted ElevenLabs-compatible gateway. The `xi-api-key` header
// still comes from `ELEVENLABS_API_KEY`. Closes #1051.
let base = self
.elevenlabs_base_url
.as_deref()
.map(|b| b.trim_end_matches('/').to_string())
.unwrap_or_else(|| "https://api.elevenlabs.io".to_string());
let url = format!("{}/v1/text-to-speech/{}", base, voice_id);
let body = serde_json::json!({
"text": text,
@@ -306,4 +355,88 @@ mod tests {
fn test_max_audio_constant() {
assert_eq!(MAX_AUDIO_RESPONSE_BYTES, 10 * 1024 * 1024);
}
#[test]
fn test_with_base_urls_sets_overrides() {
let engine = TtsEngine::new(default_config()).with_base_urls(
Some("http://127.0.0.1:8000".to_string()),
Some("http://127.0.0.1:9000".to_string()),
);
assert_eq!(
engine.openai_base_url.as_deref(),
Some("http://127.0.0.1:8000")
);
assert_eq!(
engine.elevenlabs_base_url.as_deref(),
Some("http://127.0.0.1:9000")
);
}
/// Closes #1051: when the OpenAI TTS base URL is overridden, the URL
/// building logic must append `/v1/audio/speech` and strip any trailing
/// slash. When unset, the hardcoded provider URL is used.
#[test]
fn test_tts_openai_base_url_override_logic() {
// Helper mirroring the URL construction in `synthesize_openai`.
fn build(base: Option<&str>) -> String {
base.map(|b| format!("{}/v1/audio/speech", b.trim_end_matches('/')))
.unwrap_or_else(|| "https://api.openai.com/v1/audio/speech".to_string())
}
// Default: hardcoded URL preserved (backward compatibility).
assert_eq!(build(None), "https://api.openai.com/v1/audio/speech");
// Override applied.
assert_eq!(
build(Some("http://127.0.0.1:8000")),
"http://127.0.0.1:8000/v1/audio/speech"
);
// Trailing slash on the user-supplied base is stripped.
assert_eq!(
build(Some("http://127.0.0.1:8000/")),
"http://127.0.0.1:8000/v1/audio/speech"
);
assert_eq!(
build(Some("https://tts.example.com/")),
"https://tts.example.com/v1/audio/speech"
);
}
/// Closes #1051: when the ElevenLabs TTS base URL is overridden, the URL
/// building logic must append `/v1/text-to-speech/{voice_id}` and strip
/// any trailing slash. When unset, the hardcoded provider URL is used.
#[test]
fn test_tts_elevenlabs_base_url_override_logic() {
fn build(base: Option<&str>, voice_id: &str) -> String {
let b = base
.map(|b| b.trim_end_matches('/').to_string())
.unwrap_or_else(|| "https://api.elevenlabs.io".to_string());
format!("{}/v1/text-to-speech/{}", b, voice_id)
}
let voice = "21m00Tcm4TlvDq8ikWAM";
// Default: hardcoded URL preserved.
assert_eq!(
build(None, voice),
format!("https://api.elevenlabs.io/v1/text-to-speech/{voice}")
);
// Override applied.
assert_eq!(
build(Some("http://127.0.0.1:9000"), voice),
format!("http://127.0.0.1:9000/v1/text-to-speech/{voice}")
);
// Trailing slash stripped.
assert_eq!(
build(Some("http://127.0.0.1:9000/"), voice),
format!("http://127.0.0.1:9000/v1/text-to-speech/{voice}")
);
assert_eq!(
build(Some("https://eleven.example.com/"), voice),
format!("https://eleven.example.com/v1/text-to-speech/{voice}")
);
}
}
+2
View File
@@ -25,3 +25,5 @@ zip = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
tokio-test = { workspace = true }
ed25519-dalek = { workspace = true }
rand = { workspace = true }
+75 -4
View File
@@ -11,6 +11,7 @@
//! - Download: `GET /api/v1/download?slug=...`
//! - File: `GET /api/v1/skills/{slug}/file?path=SKILL.md`
use crate::installer::{enforce_require_signed, InstallOptions};
use crate::openclaw_compat;
use crate::verify::{SkillVerifier, SkillWarning, WarningSeverity};
use crate::SkillError;
@@ -491,6 +492,25 @@ impl ClawHubClient {
/// Install a skill from ClawHub into the target directory.
///
/// Convenience wrapper around [`Self::install_with_options`] using the
/// default (permissive) options. Existing callers behave exactly as
/// before.
pub async fn install(
&self,
slug: &str,
target_dir: &Path,
) -> Result<ClawHubInstallResult, SkillError> {
self.install_with_options(slug, target_dir, &InstallOptions::default())
.await
}
/// Install a skill from ClawHub with explicit enforcement options.
///
/// When `opts.require_signed` is true, the skill must ship with a valid
/// Ed25519 `SignedManifest` envelope (see [`crate::installer`] for the
/// well-known filenames). Skills failing the gate are removed from disk
/// and a `SkillError::SecurityBlocked` is returned.
///
/// Security pipeline:
/// 1. Download skill zip and compute SHA256
/// 2. Detect format (SKILL.md vs package.json)
@@ -499,10 +519,12 @@ impl ClawHubClient {
/// 5. If prompt-only: run prompt injection scan
/// 6. Check binary dependencies
/// 7. Write skill.toml with `verified: false`
pub async fn install(
/// 8. Enforce `require_signed` if requested.
pub async fn install_with_options(
&self,
slug: &str,
target_dir: &Path,
opts: &InstallOptions,
) -> Result<ClawHubInstallResult, SkillError> {
// Use /api/v1/download?slug=... endpoint
let url = format!("{}/download?slug={}", self.base_url, urlencoded(slug));
@@ -525,9 +547,24 @@ impl ClawHubClient {
};
info!(slug, sha256 = %sha256, "Downloaded skill");
// Create skill directory
let skill_dir = target_dir.join(slug);
std::fs::create_dir_all(&skill_dir)?;
// Codex Finding 1 (#1170 followup): extract into a private staging
// directory and atomically rename into the final location only after
// enforce_require_signed passes. Without this, a local writer with
// access to target_dir/<slug>/ can swap files between extraction
// and the binding check (the TOCTOU window).
let final_dir = target_dir.join(slug);
let staging_dir = target_dir.join(format!(
".staging-{}-{}-{}",
slug,
std::process::id(),
chrono::Utc::now().timestamp_nanos_opt().unwrap_or(0)
));
// Pre-clean any leftover staging dir from a prior crashed install.
let _ = std::fs::remove_dir_all(&staging_dir);
std::fs::create_dir_all(&staging_dir)?;
// Use the staging path for all extraction/conversion work.
// After enforcement passes the function renames it to final_dir.
let skill_dir = staging_dir.clone();
// Detect content type and extract accordingly
let content_str = String::from_utf8_lossy(&bytes);
@@ -637,6 +674,39 @@ impl ClawHubClient {
// Step 7: Write skill.toml
openclaw_compat::write_openfang_manifest(&skill_dir, &manifest)?;
// Step 8: Enforce --require-signed gate, if requested.
if let Err(e) = enforce_require_signed(&skill_dir, opts) {
let _ = std::fs::remove_dir_all(&skill_dir);
return Err(e);
}
// Step 9: Atomic rename staging -> final. Closes the TOCTOU window
// between extraction and enforcement (Codex Finding 1). If the final
// location already exists from a prior install, replace it
// atomically (where the filesystem allows) or fall back to
// remove-then-rename.
if final_dir.exists() {
if let Err(e) = std::fs::remove_dir_all(&final_dir) {
let _ = std::fs::remove_dir_all(&skill_dir);
return Err(SkillError::SecurityBlocked(format!(
"install: failed to clear existing {} before atomic rename: {e}",
final_dir.display()
)));
}
}
if let Err(e) = std::fs::rename(&skill_dir, &final_dir) {
let _ = std::fs::remove_dir_all(&skill_dir);
return Err(SkillError::SecurityBlocked(format!(
"install: atomic rename {} -> {} failed: {e}",
skill_dir.display(),
final_dir.display()
)));
}
// From this point on the skill lives at final_dir, not staging.
// (Variable kept for clarity even though unread; future tracing
// additions reference it.)
let _skill_dir = final_dir;
let result = ClawHubInstallResult {
skill_name: manifest.skill.name.clone(),
version: manifest.skill.version.clone(),
@@ -650,6 +720,7 @@ impl ClawHubClient {
slug,
skill_name = %result.skill_name,
warnings = result.warnings.len(),
require_signed = opts.require_signed,
"Installed skill from ClawHub"
);
+540
View File
@@ -0,0 +1,540 @@
//! Skill install enforcement options.
//!
//! Wraps the per-source install clients (FangHub `marketplace`, ClawHub) with
//! optional supply-chain gates. The flagship gate is `require_signed`: when
//! true, an Ed25519 `SignedManifest` envelope must sit alongside the skill
//! payload and verify cleanly before the install is considered complete.
//!
//! The signature envelope is a JSON serialisation of
//! [`openfang_types::manifest_signing::SignedManifest`]. The installer looks
//! for it at one of these well-known names inside the freshly written skill
//! directory:
//!
//! - `signature.json`
//! - `skill.toml.sig.json`
//! - `SKILL.md.sig.json`
//!
//! On a `require_signed` failure the skill directory is removed and a
//! `SkillError::SecurityBlocked` is returned, matching the existing
//! prompt-injection-blocked path in `clawhub.rs`.
use crate::SkillError;
use openfang_types::manifest_signing::SignedManifest;
use std::path::Path;
/// Options controlling enforcement during skill install.
///
/// Defaults are permissive — `require_signed` is `false` so existing
/// callers (`Installer::install`, `Installer::install` on the marketplace)
/// behave exactly as before.
#[derive(Debug, Clone, Default)]
pub struct InstallOptions {
/// When true, reject any skill that does not ship with a valid Ed25519
/// `SignedManifest` envelope. The `--require-signed` CLI flag maps here.
pub require_signed: bool,
/// Optional allow-list of acceptable signer public keys (hex-encoded,
/// 32 bytes / 64 hex chars). When non-empty, the envelope's
/// `signer_public_key` must match one of these entries in addition to
/// passing cryptographic verification. Empty = any valid signature
/// accepted (TOFU mode).
pub allowed_signer_keys: Vec<String>,
}
impl InstallOptions {
/// Convenience: `require_signed = true`, no key pinning.
pub fn require_signed() -> Self {
Self {
require_signed: true,
allowed_signer_keys: Vec::new(),
}
}
/// Convenience: `require_signed = true` with a pinned signer key.
pub fn require_signed_by(pubkey_hex: impl Into<String>) -> Self {
Self {
require_signed: true,
allowed_signer_keys: vec![pubkey_hex.into()],
}
}
}
/// Well-known filenames the installer searches for a detached signature
/// envelope, in priority order.
const SIGNATURE_CANDIDATES: &[&str] =
&["signature.json", "skill.toml.sig.json", "SKILL.md.sig.json"];
/// Normalize a manifest text for content-binding comparison.
///
/// Strips a UTF-8 BOM if present and converts CRLF to LF. Without this,
/// a Windows checkout with git autocrlf would write `\r\n` line endings
/// to disk while the signed envelope captured `\n`, and the literal
/// byte-equality check would reject an otherwise-valid signed manifest.
/// Applied symmetrically to both sides of the binding compare.
fn normalize_manifest_text(text: &str) -> String {
let trimmed = text.strip_prefix('\u{feff}').unwrap_or(text);
trimmed.replace("\r\n", "\n")
}
/// Verify that a path resolves to a regular file *inside* `dir`, not a
/// symlink and not an escape via `..` or canonicalization. Returns
/// `Ok(true)` if the entry is safe to open, `Ok(false)` if it doesn't
/// exist, and `Err` if the entry exists but fails the safety check.
///
/// Without this, an attacker shipping a crafted archive could place
/// `signature.json` or `skill.toml` as a symlink pointing outside the
/// skill directory and redirect manifest/envelope reads — bypassing the
/// intent of the binding enforcement.
fn safe_regular_file_in(dir: &Path, name: &str) -> Result<bool, SkillError> {
let path = dir.join(name);
let md = match std::fs::symlink_metadata(&path) {
Ok(md) => md,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(false),
Err(e) => {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: failed to stat {} for safety check: {e}",
path.display()
)))
}
};
if md.file_type().is_symlink() {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: refusing to follow symlink at {} \
(skill bundles must ship regular files only)",
path.display()
)));
}
if !md.is_file() {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: {} is not a regular file",
path.display()
)));
}
let canon_dir = std::fs::canonicalize(dir).map_err(|e| {
SkillError::SecurityBlocked(format!(
"require_signed: cannot canonicalize {}: {e}",
dir.display()
))
})?;
let canon_path = std::fs::canonicalize(&path).map_err(|e| {
SkillError::SecurityBlocked(format!(
"require_signed: cannot canonicalize {}: {e}",
path.display()
))
})?;
if !canon_path.starts_with(&canon_dir) {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: {} escapes skill directory {}",
canon_path.display(),
canon_dir.display()
)));
}
Ok(true)
}
/// Locate a `SignedManifest` envelope inside `skill_dir`, if any.
///
/// Returns the parsed envelope on the first candidate that exists and parses
/// successfully. Files that exist but fail to parse return an error — a
/// malformed envelope is a stronger signal than an absent one.
pub fn load_signature(skill_dir: &Path) -> Result<Option<SignedManifest>, SkillError> {
for name in SIGNATURE_CANDIDATES {
if !safe_regular_file_in(skill_dir, name)? {
continue;
}
let path = skill_dir.join(name);
let raw = std::fs::read_to_string(&path)?;
let envelope: SignedManifest = serde_json::from_str(&raw).map_err(|e| {
SkillError::InvalidManifest(format!(
"Signature envelope at {} is not valid JSON: {e}",
path.display()
))
})?;
return Ok(Some(envelope));
}
Ok(None)
}
/// Enforce `require_signed` against a freshly installed skill directory.
///
/// Returns `Ok(())` when:
/// - `opts.require_signed` is false (no enforcement); or
/// - a `SignedManifest` envelope is found, `verify()` passes, and (when
/// `allowed_signer_keys` is non-empty) the signer key is allow-listed.
///
/// Returns `SkillError::SecurityBlocked` when enforcement is on and the
/// skill fails any of those checks. On failure the caller is expected to
/// remove `skill_dir` to keep the skills directory clean.
pub fn enforce_require_signed(skill_dir: &Path, opts: &InstallOptions) -> Result<(), SkillError> {
if !opts.require_signed {
return Ok(());
}
let envelope = match load_signature(skill_dir)? {
Some(e) => e,
None => {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: no signature envelope found in {} \
(looked for signature.json / skill.toml.sig.json / SKILL.md.sig.json)",
skill_dir.display()
)))
}
};
if let Err(e) = envelope.verify() {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: signature verification failed: {e}"
)));
}
// Bind the signature to the installed bytes.
//
// envelope.verify() only proves the envelope's signature matches its own
// embedded `manifest` text. Without comparing that text to the actual
// skill.toml / SKILL.md / package.json on disk, an attacker could ship
// a benign signed envelope alongside malicious skill files and pass the
// check.
//
// Defense layers (Codex Findings 1-4 from audit of 4c496be):
// - Symlink + canonicalization checks: redirect-via-symlink closed
// - CRLF/BOM normalization: Windows false-reject closed
// - package.json in candidate list: OpenClaw gap closed
// - clawhub::install_with_options now uses staging-then-atomic-rename
// so this function runs inside a private dir not yet visible at the
// final install path. That closes the TOCTOU window where a local
// writer could swap files between extract and enforce.
//
// Read every candidate manifest file in the installed dir and require
// that at least one byte-matches envelope.manifest after BOM strip +
// CRLF→LF normalization (applied symmetrically to both sides).
// `package.json` is included because openclaw_compat treats it as a
// valid SKILL manifest source.
const MANIFEST_CANDIDATES: &[&str] = &["skill.toml", "SKILL.md", "skill.md", "package.json"];
let normalized_envelope = normalize_manifest_text(&envelope.manifest);
let mut bound = false;
for name in MANIFEST_CANDIDATES {
if !safe_regular_file_in(skill_dir, name)? {
continue;
}
let path = skill_dir.join(name);
match std::fs::read_to_string(&path) {
Ok(actual) => {
if normalize_manifest_text(&actual) == normalized_envelope {
bound = true;
break;
}
}
Err(e) => {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: failed to read {} for binding check: {e}",
path.display()
)));
}
}
}
if !bound {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: signed envelope content does not match any \
installed manifest file in {} (signature was valid but the \
skill payload on disk differs from what was signed)",
skill_dir.display()
)));
}
if !opts.allowed_signer_keys.is_empty() {
let actual = hex::encode(&envelope.signer_public_key);
let actual_lower = actual.to_lowercase();
let matched = opts
.allowed_signer_keys
.iter()
.any(|k| k.trim().to_lowercase() == actual_lower);
if !matched {
return Err(SkillError::SecurityBlocked(format!(
"require_signed: signer key {actual} not in allow-list \
(signer_id = {:?})",
envelope.signer_id
)));
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use tempfile::TempDir;
fn write_skill_toml(dir: &Path) -> String {
let toml = r#"
[skill]
name = "signed-skill"
version = "0.1.0"
description = "A signed skill"
[runtime]
type = "python"
entry = "main.py"
"#;
std::fs::write(dir.join("skill.toml"), toml).unwrap();
toml.to_string()
}
fn write_signature(dir: &Path, envelope: &SignedManifest, name: &str) {
let json = serde_json::to_string_pretty(envelope).unwrap();
std::fs::write(dir.join(name), json).unwrap();
}
#[test]
fn require_signed_off_passes_unsigned() {
let dir = TempDir::new().unwrap();
write_skill_toml(dir.path());
let opts = InstallOptions::default();
assert!(enforce_require_signed(dir.path(), &opts).is_ok());
}
#[test]
fn require_signed_on_rejects_missing_signature() {
let dir = TempDir::new().unwrap();
write_skill_toml(dir.path());
let opts = InstallOptions::require_signed();
let err = enforce_require_signed(dir.path(), &opts).unwrap_err();
match err {
SkillError::SecurityBlocked(msg) => {
assert!(msg.contains("no signature envelope"), "got: {msg}");
}
other => panic!("expected SecurityBlocked, got {other:?}"),
}
}
#[test]
fn require_signed_on_accepts_valid_signature() {
let dir = TempDir::new().unwrap();
let toml = write_skill_toml(dir.path());
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(toml, &signing_key, "test-signer");
write_signature(dir.path(), &envelope, "signature.json");
let opts = InstallOptions::require_signed();
assert!(enforce_require_signed(dir.path(), &opts).is_ok());
}
#[test]
fn require_signed_on_rejects_tampered_envelope() {
let dir = TempDir::new().unwrap();
let toml = write_skill_toml(dir.path());
let signing_key = SigningKey::generate(&mut OsRng);
let mut envelope = SignedManifest::sign(toml, &signing_key, "test-signer");
// Tamper with the manifest body — content_hash will no longer match.
envelope.manifest.push_str("\n# evil append\n");
write_signature(dir.path(), &envelope, "signature.json");
let opts = InstallOptions::require_signed();
let err = enforce_require_signed(dir.path(), &opts).unwrap_err();
match err {
SkillError::SecurityBlocked(msg) => {
assert!(
msg.contains("signature verification failed")
|| msg.contains("content hash mismatch"),
"got: {msg}"
);
}
other => panic!("expected SecurityBlocked, got {other:?}"),
}
}
#[test]
fn require_signed_rejects_malformed_envelope() {
let dir = TempDir::new().unwrap();
write_skill_toml(dir.path());
std::fs::write(dir.path().join("signature.json"), "{not valid json").unwrap();
let opts = InstallOptions::require_signed();
let err = enforce_require_signed(dir.path(), &opts).unwrap_err();
match err {
SkillError::InvalidManifest(msg) => {
assert!(msg.contains("Signature envelope"), "got: {msg}");
}
other => panic!("expected InvalidManifest, got {other:?}"),
}
}
#[test]
fn require_signed_with_allowed_keys_accepts_listed_key() {
let dir = TempDir::new().unwrap();
let toml = write_skill_toml(dir.path());
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(toml, &signing_key, "test-signer");
let pk_hex = hex::encode(&envelope.signer_public_key);
write_signature(dir.path(), &envelope, "signature.json");
let opts = InstallOptions::require_signed_by(pk_hex);
assert!(enforce_require_signed(dir.path(), &opts).is_ok());
}
#[test]
fn require_signed_with_allowed_keys_rejects_unlisted_key() {
let dir = TempDir::new().unwrap();
let toml = write_skill_toml(dir.path());
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(toml, &signing_key, "evil-signer");
write_signature(dir.path(), &envelope, "signature.json");
// Allow only a different key.
let other_key = SigningKey::generate(&mut OsRng);
let other_hex = hex::encode(other_key.verifying_key().to_bytes());
let opts = InstallOptions::require_signed_by(other_hex);
let err = enforce_require_signed(dir.path(), &opts).unwrap_err();
match err {
SkillError::SecurityBlocked(msg) => {
assert!(msg.contains("not in allow-list"), "got: {msg}");
}
other => panic!("expected SecurityBlocked, got {other:?}"),
}
}
/// Critical: a valid signature for a different manifest must NOT pass
/// when the on-disk skill.toml differs. Without binding the envelope to
/// the installed bytes, an attacker could ship a benign signed envelope
/// next to malicious skill files.
#[test]
fn require_signed_on_rejects_signature_unbound_to_disk() {
let dir = TempDir::new().unwrap();
// Sign a BENIGN manifest body but never write that text to disk.
let benign_toml = r#"name = "benign"
version = "0.1.0"
description = "Looks fine."
[runtime]
type = "python"
entry = "main.py"
"#;
let signing_key = SigningKey::generate(&mut OsRng);
let envelope =
SignedManifest::sign(benign_toml.to_string(), &signing_key, "trusted-signer");
write_signature(dir.path(), &envelope, "signature.json");
// Write a DIFFERENT (malicious) skill.toml on disk.
let evil_toml = r#"name = "evil"
version = "0.1.0"
description = "Backdoor."
[runtime]
type = "python"
entry = "rm-rf.py"
"#;
std::fs::write(dir.path().join("skill.toml"), evil_toml).unwrap();
let opts = InstallOptions::require_signed();
let err = enforce_require_signed(dir.path(), &opts).unwrap_err();
match err {
SkillError::SecurityBlocked(msg) => {
assert!(
msg.contains("does not match any") || msg.contains("payload on disk differs"),
"got: {msg}"
);
}
other => panic!("expected SecurityBlocked, got {other:?}"),
}
}
#[test]
fn load_signature_returns_none_when_absent() {
let dir = TempDir::new().unwrap();
write_skill_toml(dir.path());
assert!(load_signature(dir.path()).unwrap().is_none());
}
#[test]
fn load_signature_finds_alternate_filename() {
let dir = TempDir::new().unwrap();
let toml = write_skill_toml(dir.path());
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(toml, &signing_key, "alt-name-signer");
write_signature(dir.path(), &envelope, "skill.toml.sig.json");
let loaded = load_signature(dir.path()).unwrap().unwrap();
assert_eq!(loaded.signer_id, "alt-name-signer");
}
// Codex audit Finding 2: CRLF/BOM normalization. A Windows checkout
// may write \r\n line endings to disk; the signed envelope captured
// \n. Literal byte equality would false-reject. Both sides are
// normalize_manifest_text-ed before compare.
#[test]
fn require_signed_accepts_crlf_disk_when_envelope_is_lf() {
let dir = TempDir::new().unwrap();
let lf_toml = "[skill]\nname = \"x\"\nversion = \"0.1\"\n[runtime]\ntype = \"python\"\nentry = \"main.py\"\n";
let crlf_toml = lf_toml.replace('\n', "\r\n");
std::fs::write(dir.path().join("skill.toml"), &crlf_toml).unwrap();
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(lf_toml, &signing_key, "lf-signer");
write_signature(dir.path(), &envelope, "signature.json");
let opts = InstallOptions::require_signed();
assert!(enforce_require_signed(dir.path(), &opts).is_ok());
}
#[test]
fn require_signed_accepts_utf8_bom_disk_when_envelope_is_clean() {
let dir = TempDir::new().unwrap();
let clean = "[skill]\nname = \"x\"\nversion = \"0.1\"\n[runtime]\ntype = \"python\"\nentry = \"main.py\"\n";
let with_bom = format!("\u{feff}{clean}");
std::fs::write(dir.path().join("skill.toml"), &with_bom).unwrap();
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(clean, &signing_key, "bom-signer");
write_signature(dir.path(), &envelope, "signature.json");
let opts = InstallOptions::require_signed();
assert!(enforce_require_signed(dir.path(), &opts).is_ok());
}
// Codex audit Finding 3: package.json must be a valid binding candidate
// because openclaw_compat treats it as a SKILL manifest source.
#[test]
fn require_signed_binds_to_package_json() {
let dir = TempDir::new().unwrap();
let pkg = r#"{"name":"x","version":"0.1.0","openfang":{"skill":"x"}}"#;
std::fs::write(dir.path().join("package.json"), pkg).unwrap();
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(pkg, &signing_key, "pkg-signer");
write_signature(dir.path(), &envelope, "signature.json");
let opts = InstallOptions::require_signed();
assert!(enforce_require_signed(dir.path(), &opts).is_ok());
}
// Codex audit Finding 4: symlinks for signature.json or manifest must
// be rejected even when they point to a valid file, to prevent crafted
// bundles from redirecting reads outside the skill directory.
#[cfg(unix)]
#[test]
fn require_signed_rejects_symlink_signature() {
use std::os::unix::fs::symlink;
let dir = TempDir::new().unwrap();
let outside = TempDir::new().unwrap();
let toml = write_skill_toml(dir.path());
let signing_key = SigningKey::generate(&mut OsRng);
let envelope = SignedManifest::sign(toml, &signing_key, "symlink-signer");
let outside_sig = outside.path().join("real-sig.json");
std::fs::write(
&outside_sig,
serde_json::to_string_pretty(&envelope).unwrap(),
)
.unwrap();
symlink(&outside_sig, dir.path().join("signature.json")).unwrap();
let opts = InstallOptions::require_signed();
let err = enforce_require_signed(dir.path(), &opts).unwrap_err();
match err {
SkillError::SecurityBlocked(msg) => {
assert!(msg.contains("symlink"), "got: {msg}");
}
other => panic!("expected SecurityBlocked, got {other:?}"),
}
}
}
+1
View File
@@ -10,6 +10,7 @@
pub mod bundled;
pub mod clawhub;
pub mod config_injection;
pub mod installer;
pub mod loader;
pub mod marketplace;
pub mod openclaw_compat;
+32 -2
View File
@@ -3,6 +3,7 @@
//! For Phase 1, uses GitHub releases as the registry backend.
//! Each skill is a GitHub repo with releases containing the skill bundle.
use crate::installer::{enforce_require_signed, InstallOptions};
use crate::SkillError;
use std::path::Path;
use tracing::info;
@@ -90,8 +91,28 @@ impl MarketplaceClient {
/// Install a skill from a GitHub repo by name.
///
/// Downloads the latest release tarball and extracts it to the target directory.
/// Convenience wrapper around [`Self::install_with_options`] using the
/// default (permissive) options. Existing callers behave exactly as
/// before.
pub async fn install(&self, skill_name: &str, target_dir: &Path) -> Result<String, SkillError> {
self.install_with_options(skill_name, target_dir, &InstallOptions::default())
.await
}
/// Install a skill from a GitHub repo with explicit enforcement options.
///
/// When `opts.require_signed` is true, the installed bundle must contain
/// a valid Ed25519 `SignedManifest` envelope. Skills failing the gate
/// are removed from disk and a `SkillError::SecurityBlocked` is
/// returned.
///
/// Downloads the latest release tarball and extracts it to the target directory.
pub async fn install_with_options(
&self,
skill_name: &str,
target_dir: &Path,
opts: &InstallOptions,
) -> Result<String, SkillError> {
let repo = format!("{}/{}", self.config.github_org, skill_name);
let url = format!(
"{}/repos/{}/releases/latest",
@@ -163,7 +184,16 @@ impl MarketplaceClient {
serde_json::to_string_pretty(&meta).unwrap_or_default(),
)?;
info!("Installed skill: {skill_name} {version}");
// Enforce --require-signed gate, if requested.
if let Err(e) = enforce_require_signed(&skill_dir, opts) {
let _ = std::fs::remove_dir_all(&skill_dir);
return Err(e);
}
info!(
"Installed skill: {skill_name} {version} (require_signed={})",
opts.require_signed
);
Ok(version)
}
}
+17 -3
View File
@@ -474,11 +474,23 @@ pub struct AgentManifest {
/// Pinned model override (used in Stable mode).
#[serde(default)]
pub pinned_model: Option<String>,
/// Agent workspace directory. Auto-created on spawn.
/// Default: `{workspaces_dir}/{agent_name}-{agent_id_prefix}/`
/// Agent workspace directory. User-facing working area for output, data,
/// skills, context.md, and uploads. When the user sets this in agent.toml
/// (e.g. `workspace = "/home/me/Documents"`) the runtime treats it as the
/// agent's working directory and will NOT scaffold private state files
/// there. Multiple agents can share the same workspace to collaborate on
/// files. See issue #1097.
/// Default: `{workspaces_dir}/{agent_name}/` (same as state_dir).
#[serde(default)]
pub workspace: Option<PathBuf>,
/// Whether to generate workspace identity files (SOUL.md, USER.md, etc.) on creation.
/// Agent private state directory. Stores identity files (SOUL.md, etc.),
/// AGENT.json, sessions/, and the daily memory log. Always lives under
/// `~/.openfang/workspaces/{name}/` regardless of where the user-facing
/// workspace points. Auto-derived on spawn. See issue #1097.
#[serde(default)]
pub state_dir: Option<PathBuf>,
/// Whether to generate identity files (SOUL.md, USER.md, etc.) in
/// `state_dir` on creation.
#[serde(default = "default_true")]
pub generate_identity_files: bool,
/// Per-agent exec policy override. If None, uses global exec_policy.
@@ -550,6 +562,7 @@ impl Default for AgentManifest {
autonomous: None,
pinned_model: None,
workspace: None,
state_dir: None,
generate_identity_files: true,
exec_policy: None,
tool_allowlist: Vec::new(),
@@ -809,6 +822,7 @@ mod tests {
autonomous: None,
pinned_model: None,
workspace: None,
state_dir: None,
generate_identity_files: true,
exec_policy: None,
tool_allowlist: Vec::new(),
+305 -1
View File
@@ -733,7 +733,12 @@ impl Default for VaultConfig {
}
/// Agent binding — routes specific channel/account/peer patterns to agents.
///
/// `deny_unknown_fields` so typos at the binding level (e.g. `match_rule` →
/// `match_rules`) fail loudly at config-load instead of silently leaving the
/// rule defaulted to "match everything".
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct AgentBinding {
/// Target agent name or ID.
pub agent: String,
@@ -741,6 +746,50 @@ pub struct AgentBinding {
pub match_rule: BindingMatchRule,
}
/// Lowercased channel-name strings whose adapters place a channel/conversation/
/// room/space/chat ID directly in `ChannelMessage::sender.platform_id` (these
/// adapters overload that field because it doubles as the send target).
///
/// Single source of truth shared between:
/// - Config validation (warn the user when their `channel_id` binding targets
/// an adapter that doesn't populate `ctx.channel_id`).
/// - `ChannelMessage::channel_id()` in `openfang-channels::types` (routing-time
/// accessor that reads from this list to decide where to source the ID).
///
/// Adapters not listed fall back to `metadata["channel_id"]` if present, then
/// `None`. Hybrid adapters whose `platform_id` flips between channel and user
/// based on `is_group` (IRC, Zulip) are intentionally excluded — a single
/// channel-scoped binding would silently match DMs.
///
/// Compared against the lowercased `channel` string from `BindingMatchRule`
/// or from `channel_type_str()` at routing time.
pub const CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL: &[&str] = &[
"discord",
"slack",
"telegram",
"matrix",
"mattermost",
"teams",
"webex",
"rocketchat",
"nextcloud",
"pumble",
"revolt",
"guilded",
"feishu",
// Feishu Intl region emits `Custom("lark")` from `feishu.rs` (region.as_str()
// returns "lark" when configured for international). Listed alongside
// "feishu" so both regional spellings resolve identically at routing and
// validation time.
"lark",
"keybase",
"google_chat",
"line",
"twist",
"flock",
"twitch",
];
/// Match rule for agent bindings. All specified (non-None) fields must match.
///
/// `#[serde(deny_unknown_fields)]` is intentional: a typo like `channnel_id` or
@@ -755,7 +804,18 @@ pub struct BindingMatchRule {
/// Specific account/bot ID within the channel.
#[serde(default)]
pub account_id: Option<String>,
/// Peer/user ID for DM routing.
/// Peer/user ID. Matches `BindingContext::peer_id`, which the bridge
/// populates from `ChannelMessage::sender_user_id()` — i.e. the platform's
/// *user* identity (Discord user ID, Slack user ID, etc.).
///
/// Note: the legacy `AgentRouter::resolve()` entry point (kept for tests
/// and any caller without bridge context) builds a synthetic context
/// where `peer_id` is filled from the raw `platform_user_id` argument. On
/// adapters that overload `platform_id` as the channel ID (Discord, Slack,
/// …), passing those callers a channel-scoped value will match here. New
/// code should route through `resolve_with_context` and a bridge-built
/// `BindingContext`. For platform-native channel/conversation matching,
/// use `channel_id` instead.
#[serde(default)]
pub peer_id: Option<String>,
/// Guild/server ID (Discord/Slack).
@@ -766,6 +826,8 @@ pub struct BindingMatchRule {
/// ID (`C…`/`D…`/`G…`); on Telegram it is the chat ID; on IRC it is the
/// channel name. Bridges populate this from the message's channel/conversation
/// identifier so bindings can route by room independent of which user posted.
/// Pair with `channel` to disambiguate across platforms (channel IDs are
/// not portable).
#[serde(default)]
pub channel_id: Option<String>,
/// Role-based routing (user must have at least one).
@@ -907,6 +969,25 @@ pub struct ExecPolicy {
/// produce no stdout/stderr output for this duration. Default: 30.
#[serde(default = "default_no_output_timeout")]
pub no_output_timeout_secs: u64,
/// Environment variables to forward from the OpenFang process into
/// `shell_exec` subprocesses.
///
/// By default, subprocesses run with `env_clear()` and only receive a
/// minimal safe set (PATH, HOME, TMPDIR, LANG, TERM, etc. — see
/// `subprocess_sandbox::SAFE_ENV_VARS`). Anything else — including
/// user-defined variables present in the container/host environment —
/// is stripped. This list lets operators explicitly re-add specific
/// variables to the subprocess environment.
///
/// Each entry is an env var name. A single entry of `"*"` forwards
/// every variable present in the parent process. Use with care — `*`
/// will leak API keys and other secrets into child processes.
///
/// Aliases `env_passthrough` and `env_allowlist` are accepted for
/// backwards compatibility with users who configured these names
/// before the field existed (issue #1169).
#[serde(default, alias = "env_passthrough", alias = "env_allowlist")]
pub shell_env_passthrough: Vec<String>,
}
fn default_no_output_timeout() -> u64 {
@@ -928,6 +1009,7 @@ impl Default for ExecPolicy {
timeout_secs: 30,
max_output_bytes: 100 * 1024,
no_output_timeout_secs: default_no_output_timeout(),
shell_env_passthrough: Vec::new(),
}
}
}
@@ -1050,6 +1132,14 @@ impl Default for ThinkingConfig {
}
/// Top-level kernel configuration.
///
/// `deny_unknown_fields` is intentionally *not* applied here. Strict-field
/// validation is scoped to bindings (`AgentBinding` + `BindingMatchRule`),
/// where silent no-op routing is the failure mode worth catching loudly.
/// Adding it to the top-level kernel struct would also reject forward-compat
/// keys, downstream-fork-only keys, and "I'm trying out a future field early"
/// workflows — a wider behavior change than this PR's mandate. If we want it
/// later, it ships as its own decision.
#[derive(Clone, Serialize, Deserialize)]
#[serde(default)]
pub struct KernelConfig {
@@ -1391,6 +1481,10 @@ fn default_true() -> bool {
true
}
fn default_auto_thread() -> String {
"false".to_string()
}
fn default_thread_ttl() -> u64 {
24
}
@@ -1839,6 +1933,13 @@ pub struct TelegramConfig {
/// Allows channel_send(channel="telegram", message="...") without a recipient.
#[serde(default)]
pub default_chat_id: Option<String>,
/// Forum topic routing: maps `message_thread_id` to an agent name.
/// When a message arrives inside a Telegram forum topic whose thread id is
/// listed here, the bridge dispatches it to the named agent instead of the
/// default. Threads not listed fall back to `default_agent`.
/// Issue #780.
#[serde(default)]
pub thread_routes: HashMap<i64, String>,
/// Per-channel behavior overrides.
#[serde(default)]
pub overrides: ChannelOverrides,
@@ -1853,6 +1954,7 @@ impl Default for TelegramConfig {
poll_interval_secs: 1,
api_url: None,
default_chat_id: None,
thread_routes: HashMap::new(),
overrides: ChannelOverrides::default(),
}
}
@@ -1886,6 +1988,10 @@ pub struct DiscordConfig {
/// In these channels, the bot responds to all group messages without needing to be mentioned.
#[serde(default, deserialize_with = "deserialize_string_or_int_vec")]
pub free_response_channels: Vec<String>,
/// Auto-thread behavior: "true" (always create thread), "false" (never), "smart" (only when @mentioned).
/// Default: "false"
#[serde(default = "default_auto_thread")]
pub auto_thread: String,
/// Per-channel behavior overrides.
#[serde(default)]
pub overrides: ChannelOverrides,
@@ -1902,6 +2008,7 @@ impl Default for DiscordConfig {
ignore_bots: true,
default_channel_id: None,
free_response_channels: vec![],
auto_thread: "false".to_string(),
overrides: ChannelOverrides::default(),
}
}
@@ -2029,6 +2136,13 @@ pub struct MatrixConfig {
pub user_id: String,
/// Env var name holding the access token.
pub access_token_env: String,
/// Env var name holding the MSC2918 refresh token (optional).
///
/// When set, the adapter auto-recovers from `M_UNKNOWN_TOKEN` 401 responses
/// by calling `POST /_matrix/client/v3/refresh`. Required for matrix.org
/// since the 2025-04-07 migration to MAS (Matrix Authentication Service).
#[serde(default)]
pub refresh_token_env: Option<String>,
/// Room IDs to listen in (empty = all joined rooms).
#[serde(default, deserialize_with = "deserialize_string_or_int_vec")]
pub allowed_rooms: Vec<String>,
@@ -2048,6 +2162,7 @@ impl Default for MatrixConfig {
homeserver_url: "https://matrix.org".to_string(),
user_id: String::new(),
access_token_env: "MATRIX_ACCESS_TOKEN".to_string(),
refresh_token_env: None,
allowed_rooms: vec![],
default_agent: None,
auto_accept_invites: false,
@@ -3723,6 +3838,42 @@ impl KernelConfig {
SearchProvider::DuckDuckGo | SearchProvider::Auto => {}
}
// --- Binding validation (channel_id) ---
// Use the shared allowlist (CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL) so this
// validation cannot drift from the routing-time accessor in
// ChannelMessage::channel_id(). Adapters not on the list may still
// populate ctx.channel_id via metadata["channel_id"], but we cannot
// detect that statically — so we warn conservatively and document the
// metadata escape hatch in docs/channel-adapters.md.
for (idx, binding) in self.bindings.iter().enumerate() {
let rule = &binding.match_rule;
if let Some(ref cid) = rule.channel_id {
match rule.channel.as_deref() {
None => {
warnings.push(format!(
"Binding #{} (agent='{}') sets channel_id='{}' without channel; \
channel IDs are not portable across platforms. Pair with channel = \"discord\" (or similar).",
idx, binding.agent, cid
));
}
Some(ch) => {
let ch_lower = ch.to_lowercase();
if !CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL
.iter()
.any(|p| *p == ch_lower)
{
warnings.push(format!(
"Binding #{} (agent='{}') sets channel_id='{}' for channel='{}', \
but the {} adapter does not populate ctx.channel_id from sender.platform_id; \
this binding will only match if the adapter writes channel_id into message metadata.",
idx, binding.agent, cid, ch, ch
));
}
}
}
}
}
// --- Production bounds validation ---
// Clamp dangerous zero/extreme values to safe defaults instead of crashing.
warnings
@@ -3782,6 +3933,106 @@ mod tests {
assert!(toml_str.contains("log_level"));
}
#[test]
fn test_binding_match_rule_deny_unknown_fields_rejects_typo() {
// Typo (`channnel_id` with three n's) should fail to parse rather than
// silently producing a no-op rule that matches every message.
let toml_input = r#"
channel = "discord"
channnel_id = "12345"
"#;
let result: Result<BindingMatchRule, _> = toml::from_str(toml_input);
assert!(
result.is_err(),
"expected deny_unknown_fields to reject typo'd field, got: {:?}",
result
);
}
#[test]
fn test_agent_binding_deny_unknown_fields() {
// A typo at the AgentBinding level (`match_rules` plural instead of
// `match_rule`) must fail config load — silent default would make the
// binding a wildcard.
let toml_str = r#"
agent = "x"
[match_rules]
channel = "discord"
"#;
let result: Result<AgentBinding, _> = toml::from_str(toml_str);
assert!(
result.is_err(),
"expected deny_unknown_fields rejection, got Ok"
);
}
#[test]
fn test_validate_warns_channel_id_without_channel() {
let mut config = KernelConfig::default();
config.bindings.push(AgentBinding {
agent: "ghost".to_string(),
match_rule: BindingMatchRule {
channel_id: Some("99999".to_string()),
..Default::default()
},
});
let warnings = config.validate();
assert!(
warnings
.iter()
.any(|w| w.contains("channel_id") && w.contains("without channel")),
"expected channel_id-without-channel warning, got: {:?}",
warnings
);
}
#[test]
fn test_validate_warns_channel_id_for_unsupported_adapter() {
// Reddit's `platform_id` is the post author, not a subreddit; no
// per-conversation channel_id, so the binding can never match.
let mut config = KernelConfig::default();
config.bindings.push(AgentBinding {
agent: "ghost".to_string(),
match_rule: BindingMatchRule {
channel: Some("reddit".to_string()),
channel_id: Some("r/something".to_string()),
..Default::default()
},
});
let warnings = config.validate();
assert!(
warnings
.iter()
.any(|w| w.contains("reddit") && w.contains("does not populate")),
"expected unsupported-adapter warning, got: {:?}",
warnings
);
}
#[test]
fn test_validate_no_warning_for_supported_adapters() {
// Discord, Slack, Telegram all overload platform_id as the channel ID —
// these must not warn.
for ch in ["discord", "slack", "telegram"] {
let mut config = KernelConfig::default();
config.bindings.push(AgentBinding {
agent: "ok".to_string(),
match_rule: BindingMatchRule {
channel: Some(ch.to_string()),
channel_id: Some("X".to_string()),
..Default::default()
},
});
let warnings = config.validate();
assert!(
!warnings.iter().any(|w| w.contains("does not populate")),
"did not expect channel_id-coverage warning for {}, got: {:?}",
ch,
warnings
);
}
}
#[test]
fn test_discord_config_defaults() {
let dc = DiscordConfig::default();
@@ -3920,6 +4171,9 @@ mod tests {
let mx = MatrixConfig::default();
assert_eq!(mx.homeserver_url, "https://matrix.org");
assert_eq!(mx.access_token_env, "MATRIX_ACCESS_TOKEN");
// MSC2918 refresh token env defaults to None; operators opt in via
// `refresh_token_env = "MATRIX_REFRESH_TOKEN"` in config.toml.
assert!(mx.refresh_token_env.is_none());
assert!(mx.allowed_rooms.is_empty());
}
@@ -4394,4 +4648,54 @@ mod tests {
let config: KernelConfig = toml::from_str(toml_str).unwrap();
assert_eq!(config.heartbeat.default_timeout_secs, 300);
}
// ── Issue #1169: shell_env_passthrough on ExecPolicy ──────────────
#[test]
fn test_exec_policy_passthrough_default_empty() {
let policy = ExecPolicy::default();
assert!(policy.shell_env_passthrough.is_empty());
}
#[test]
fn test_exec_policy_passthrough_deserializes() {
let toml_str = r#"
mode = "full"
shell_env_passthrough = ["TZ", "GOG_ACCOUNT"]
"#;
let policy: ExecPolicy = toml::from_str(toml_str).unwrap();
assert_eq!(policy.shell_env_passthrough, vec!["TZ", "GOG_ACCOUNT"]);
}
#[test]
fn test_exec_policy_passthrough_alias_env_passthrough() {
// Backwards-compat alias from the issue body (#1169).
let toml_str = r#"
mode = "full"
env_passthrough = ["TZ", "GOG_ACCOUNT"]
"#;
let policy: ExecPolicy = toml::from_str(toml_str).unwrap();
assert_eq!(policy.shell_env_passthrough, vec!["TZ", "GOG_ACCOUNT"]);
}
#[test]
fn test_exec_policy_passthrough_alias_env_allowlist() {
// Backwards-compat alias from the issue body (#1169).
let toml_str = r#"
mode = "full"
env_allowlist = ["TZ", "HOME"]
"#;
let policy: ExecPolicy = toml::from_str(toml_str).unwrap();
assert_eq!(policy.shell_env_passthrough, vec!["TZ", "HOME"]);
}
#[test]
fn test_exec_policy_passthrough_wildcard() {
let toml_str = r#"
mode = "full"
shell_env_passthrough = ["*"]
"#;
let policy: ExecPolicy = toml::from_str(toml_str).unwrap();
assert_eq!(policy.shell_env_passthrough, vec!["*"]);
}
}
+105
View File
@@ -96,6 +96,44 @@ pub struct MediaConfig {
/// # works for most local OpenAI-compat servers).
/// ```
pub audio_base_url: Option<String>,
/// Optional override for the OpenAI TTS endpoint base URL.
///
/// When set, replaces `https://api.openai.com` with
/// `<tts_openai_base_url>/v1/audio/speech`. Use this to point at a
/// local OpenAI-compatible TTS service (Lemonade/Kokoro, LM Studio,
/// etc.) while keeping the same JSON wire format. The Authorization
/// header is still built from `OPENAI_API_KEY` (local services
/// usually accept any non-empty bearer token).
///
/// Closes <https://github.com/RightNow-AI/openfang/issues/1051>.
#[serde(default)]
pub tts_openai_base_url: Option<String>,
/// Optional override for the ElevenLabs TTS endpoint base URL.
///
/// When set, replaces `https://api.elevenlabs.io` with
/// `<tts_elevenlabs_base_url>/v1/text-to-speech/{voice_id}`. Use this
/// to route through a proxy or self-hosted ElevenLabs-compatible
/// gateway. The `xi-api-key` header still comes from
/// `ELEVENLABS_API_KEY`.
///
/// Closes <https://github.com/RightNow-AI/openfang/issues/1051>.
#[serde(default)]
pub tts_elevenlabs_base_url: Option<String>,
/// Optional override for the OpenAI image generation endpoint base URL.
///
/// When set, replaces `https://api.openai.com` with
/// `<image_gen_base_url>/v1/images/generations`. Use this to point at
/// a local OpenAI-compatible image generation service
/// (Lemonade/Flux, LM Studio, etc.) while keeping the same JSON wire
/// format. The Authorization header is still built from
/// `OPENAI_API_KEY`.
///
/// Closes <https://github.com/RightNow-AI/openfang/issues/1051>.
#[serde(default)]
pub image_gen_base_url: Option<String>,
}
impl Default for MediaConfig {
@@ -108,6 +146,9 @@ impl Default for MediaConfig {
image_provider: None,
audio_provider: None,
audio_base_url: None,
tts_openai_base_url: None,
tts_elevenlabs_base_url: None,
image_gen_base_url: None,
}
}
}
@@ -382,6 +423,70 @@ mod tests {
assert_eq!(config.max_concurrency, 2);
assert!(config.image_provider.is_none());
assert!(config.audio_base_url.is_none());
assert!(config.tts_openai_base_url.is_none());
assert!(config.tts_elevenlabs_base_url.is_none());
assert!(config.image_gen_base_url.is_none());
}
#[test]
fn test_media_config_tts_openai_base_url_serde_roundtrip() {
let config = MediaConfig {
tts_openai_base_url: Some("http://127.0.0.1:8000".to_string()),
..MediaConfig::default()
};
let json = serde_json::to_string(&config).unwrap();
let parsed: MediaConfig = serde_json::from_str(&json).unwrap();
assert_eq!(
parsed.tts_openai_base_url.as_deref(),
Some("http://127.0.0.1:8000")
);
}
#[test]
fn test_media_config_tts_elevenlabs_base_url_serde_roundtrip() {
let config = MediaConfig {
tts_elevenlabs_base_url: Some("http://127.0.0.1:9000".to_string()),
..MediaConfig::default()
};
let json = serde_json::to_string(&config).unwrap();
let parsed: MediaConfig = serde_json::from_str(&json).unwrap();
assert_eq!(
parsed.tts_elevenlabs_base_url.as_deref(),
Some("http://127.0.0.1:9000")
);
}
#[test]
fn test_media_config_image_gen_base_url_serde_roundtrip() {
let config = MediaConfig {
image_gen_base_url: Some("http://127.0.0.1:7000".to_string()),
..MediaConfig::default()
};
let json = serde_json::to_string(&config).unwrap();
let parsed: MediaConfig = serde_json::from_str(&json).unwrap();
assert_eq!(
parsed.image_gen_base_url.as_deref(),
Some("http://127.0.0.1:7000")
);
}
#[test]
fn test_media_config_backward_compat_no_tts_or_image_overrides() {
// Old TOML/JSON without the three new URL override fields must still
// parse with None, thanks to #[serde(default)] on the struct.
let legacy_json = r#"{
"image_description": true,
"audio_transcription": true,
"video_description": false,
"max_concurrency": 2,
"image_provider": null,
"audio_provider": "openai",
"audio_base_url": null
}"#;
let parsed: MediaConfig = serde_json::from_str(legacy_json).unwrap();
assert!(parsed.tts_openai_base_url.is_none());
assert!(parsed.tts_elevenlabs_base_url.is_none());
assert!(parsed.image_gen_base_url.is_none());
}
#[test]
+7 -4
View File
@@ -434,7 +434,9 @@ mod tests {
let restored: ContentBlock = serde_json::from_str(&serialized).unwrap();
match restored {
ContentBlock::Thinking {
thinking, signature, ..
thinking,
signature,
..
} => {
assert_eq!(thinking, "Let me reason about this carefully...");
assert_eq!(
@@ -517,7 +519,9 @@ mod tests {
assert_eq!(blocks.len(), 2);
match &blocks[0] {
ContentBlock::Thinking {
thinking, signature, ..
thinking,
signature,
..
} => {
assert_eq!(thinking, "Internal reasoning");
assert_eq!(signature.as_deref(), Some("sig_xyz"));
@@ -547,8 +551,7 @@ mod tests {
fn test_provider_msg_id_is_recorded_but_not_primary() {
// The LLM-supplied identifier is preserved for debugging only — the
// server-generated `msg_id` is unchanged and remains the primary key.
let msg =
Message::assistant("hi").with_provider_msg_id("msg_abc123_anthropic_collidable");
let msg = Message::assistant("hi").with_provider_msg_id("msg_abc123_anthropic_collidable");
assert_eq!(
msg.provider_msg_id.as_deref(),
Some("msg_abc123_anthropic_collidable")
@@ -14,6 +14,7 @@ pub const GEMINI_BASE_URL: &str = "https://generativelanguage.googleapis.com";
pub const DEEPSEEK_BASE_URL: &str = "https://api.deepseek.com/v1";
pub const GROQ_BASE_URL: &str = "https://api.groq.com/openai/v1";
pub const OPENROUTER_BASE_URL: &str = "https://openrouter.ai/api/v1";
pub const REQUESTY_BASE_URL: &str = "https://router.requesty.ai/v1";
pub const MISTRAL_BASE_URL: &str = "https://api.mistral.ai/v1";
pub const TOGETHER_BASE_URL: &str = "https://api.together.xyz/v1";
pub const FIREWORKS_BASE_URL: &str = "https://api.fireworks.ai/inference/v1";
+13
View File
@@ -15,6 +15,9 @@ pub fn map_tool_name(openclaw_name: &str) -> Option<&'static str> {
"Edit" | "edit" => Some("file_write"),
"Glob" | "glob" | "list_files" => Some("file_list"),
"Grep" | "grep" => Some("file_list"),
"Mkdir" | "mkdir" | "make_directory" | "makeDirectory" | "make_dir" | "makedir"
| "create_dir" | "createDir" | "createDirectory" | "new_directory" | "new_folder"
| "create_folder" | "createFolder" => Some("create_directory"),
"Bash" | "bash" | "exec" | "execute_command" => Some("shell_exec"),
"WebSearch" | "web_search" => Some("web_search"),
"WebFetch" | "fetch_url" | "web_fetch" => Some("web_fetch"),
@@ -56,6 +59,7 @@ pub fn is_known_openfang_tool(name: &str) -> bool {
"file_read"
| "file_write"
| "file_list"
| "create_directory"
| "shell_exec"
| "web_search"
| "web_fetch"
@@ -147,6 +151,14 @@ mod tests {
assert_eq!(map_tool_name("execute"), Some("shell_exec"));
assert_eq!(map_tool_name("shell"), Some("shell_exec"));
// create_directory aliases
assert_eq!(map_tool_name("Mkdir"), Some("create_directory"));
assert_eq!(map_tool_name("mkdir"), Some("create_directory"));
assert_eq!(map_tool_name("make_directory"), Some("create_directory"));
assert_eq!(map_tool_name("create_dir"), Some("create_directory"));
assert_eq!(map_tool_name("createDirectory"), Some("create_directory"));
assert_eq!(map_tool_name("create_folder"), Some("create_directory"));
// Unknown
assert_eq!(map_tool_name("unknown_tool"), None);
assert_eq!(map_tool_name(""), None);
@@ -180,6 +192,7 @@ mod tests {
"file_read",
"file_write",
"file_list",
"create_directory",
"shell_exec",
"web_search",
"web_fetch",
+5
View File
@@ -11,6 +11,11 @@ services:
- "4200:4200"
volumes:
- openfang-data:/data
# Uncomment to reach host services (Ollama, whisper.cpp, local Postgres)
# from inside the container. Required on Linux and colima. See
# docs/troubleshooting.md#connecting-to-host-services-from-docker.
# extra_hosts:
# - "host.docker.internal:host-gateway"
environment:
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY:-}
- OPENAI_API_KEY=${OPENAI_API_KEY:-}
+57 -4
View File
@@ -622,10 +622,63 @@ Features:
The `AgentRouter` determines which agent receives an incoming message. The routing logic is:
1. **Per-channel default**: Each channel config has a `default_agent` field. Messages from that channel go to that agent.
2. **User-agent binding**: If a user has previously been associated with a specific agent (via commands or configuration), messages from that user route to that agent.
3. **Command prefix**: Users can switch agents by sending a command like `/agent coder` in the chat. Subsequent messages will be routed to the "coder" agent.
4. **Fallback**: If no routing applies, messages go to the first available agent.
1. **Bindings** (most specific first). Declarative `[[bindings]]` rules in `config.toml` map message attributes (channel, channel_id, peer_id, guild_id, account_id, roles) to agents. The router scores each rule by specificity and picks the highest-scoring match.
2. **Per-channel default**: Each channel config has a `default_agent` field. Messages from that channel go to that agent.
3. **User-agent binding**: If a user has previously been associated with a specific agent (via commands or configuration), messages from that user route to that agent.
4. **Command prefix**: Users can switch agents by sending a command like `/agent coder` in the chat. Subsequent messages will be routed to the "coder" agent.
5. **Fallback**: If no routing applies, messages go to the first available agent.
### Bindings
A binding has an `agent` (the target) and a `match_rule` (the criteria). All non-empty fields in the rule must match.
```toml
# Route a specific Discord channel to a dedicated agent.
[[bindings]]
agent = "researcher-medical"
match_rule = { channel = "discord", channel_id = "1234567890" }
[[bindings]]
agent = "researcher-business"
match_rule = { channel = "discord", channel_id = "9876543210" }
# Catch-all for the same user on any other channel.
[[bindings]]
agent = "assistant"
match_rule = { channel = "discord", peer_id = "user_discord_id" }
```
**`peer_id` vs `channel_id`** — these are easy to confuse and the difference matters:
- `peer_id` matches the **user** (Discord user ID, Slack user ID, etc.).
- `channel_id` matches the **channel/conversation** (Discord text channel, Slack conversation, Telegram chat).
Use `peer_id` for "messages from this person." Use `channel_id` for "messages in this room."
**Specificity scores** (higher wins):
| Field | Score |
| ------------ | ----- |
| `peer_id` | 8 |
| `channel_id` | 8 |
| `guild_id` | 4 |
| `roles` | 2 |
| `account_id` | 2 |
| `channel` | 1 |
A binding's score is the sum of its set fields. `peer_id` and `channel_id` are equally specific, so a rule with both (16) beats either alone (8). Ties are broken by declaration order in the config.
**Adapter coverage for `channel_id`** — the following adapters populate `ctx.channel_id` directly from `sender.platform_id` (their "user" field is overloaded as a channel/conversation/room/space ID because that field doubles as the send target):
`discord`, `slack`, `telegram`, `matrix`, `mattermost`, `teams`, `webex`, `rocketchat`, `nextcloud`, `pumble`, `revolt`, `guilded`, `feishu`, `lark`, `keybase`, `google_chat`, `line`, `twist`, `flock`, `twitch`.
(Feishu Intl region emits `Custom("lark")` rather than `Custom("feishu")`; both spellings are recognized.)
Adapters not on this list (Reddit, Bluesky, Mastodon, Signal, Email, ntfy, Discourse, etc.) carry a *user* ID in `platform_id` and have no per-conversation concept, or use a hybrid scheme (IRC, Zulip flip between channel and user based on `is_group`). Bindings targeting `channel_id` on those platforms will only match if the adapter writes a `channel_id` key into message metadata.
The kernel emits a startup warning when a binding sets `channel_id` for a non-supporting adapter, so misconfigurations surface early instead of silently routing nowhere. The single source of truth for this list is `CHANNELS_WITH_PLATFORM_ID_AS_CHANNEL` in `openfang-types::config`, consumed by both routing (`ChannelMessage::channel_id()`) and config validation.
**Strict parsing** — `AgentBinding` and `BindingMatchRule` use `#[serde(deny_unknown_fields)]`. Typos at the binding level (e.g. `match_rules` for `match_rule`, `channnel_id` for `channel_id`) fail config load with a clear error rather than parsing into a no-op rule that silently matches every message. Existing configs that work today are unaffected; only configs with stray/misspelled fields inside a `[[bindings]]` block need a fix. The top-level `KernelConfig` deliberately stays permissive so unrecognized top-level keys (forward-compat, downstream forks) don't break startup.
---
+17
View File
@@ -82,6 +82,23 @@ cd openfang
docker compose up -d
```
**Reaching host services from the container.** If you run a local LLM
(Ollama, whisper.cpp, vLLM) on the host and want the agent to call it, add
the host-gateway bridge. Required on Linux and colima:
```bash
docker run -d \
--add-host=host.docker.internal:host-gateway \
-e OLLAMA_HOST=http://host.docker.internal:11434 \
-p 4200:4200 \
ghcr.io/rightnow-ai/openfang:latest
```
For Compose, add `extra_hosts: ["host.docker.internal:host-gateway"]` to the
service. See [Troubleshooting → Connecting to host services from Docker](troubleshooting.md#connecting-to-host-services-from-docker)
and the [curl-equipped overlay image](troubleshooting.md#curl-equipped-reference-image)
if you need in-container `curl` for healthchecks.
### Verify Installation
```bash
+76
View File
@@ -110,6 +110,75 @@ rm ~/.config/fish/conf.d/openfang.fish
- Port already in use: change the port mapping `-p 3001:4200`
- Permission denied on volume mount: check directory permissions
### Connecting to host services from Docker
If you run OpenFang inside Docker and need to reach a service running on the
host (Ollama on `127.0.0.1:11434`, whisper.cpp on `127.0.0.1:8090`, a local
Postgres, etc.), `localhost` inside the container points at the container
itself, not the host. You must opt in to the host bridge.
On Docker Desktop (macOS/Windows) `host.docker.internal` resolves
automatically. On Linux and on colima (macOS) it does not, and you must add
the flag explicitly:
```bash
docker run --rm \
--add-host=host.docker.internal:host-gateway \
-e OLLAMA_HOST=http://host.docker.internal:11434 \
-p 4200:4200 \
ghcr.io/rightnow-ai/openfang:latest
```
Verify the bridge works:
```bash
docker exec <container> getent hosts host.docker.internal
# 192.168.x.x host.docker.internal
```
For Docker Compose use `extra_hosts:`:
```yaml
services:
openfang:
image: ghcr.io/rightnow-ai/openfang:latest
ports:
- "4200:4200"
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- OLLAMA_HOST=http://host.docker.internal:11434
```
Without this flag on Linux/colima, calls to host services fail silently with
connection refused or DNS lookup errors.
### Curl-equipped reference image
The default `ghcr.io/rightnow-ai/openfang` image does not ship `curl`, so
`docker exec openfang curl ...` returns `exec: curl: not found`. If you need
in-container probes for healthchecks or egress verification, build a thin
overlay image:
```dockerfile
# Dockerfile.curl
FROM ghcr.io/rightnow-ai/openfang:latest
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& rm -rf /var/lib/apt/lists/*
```
Build and run:
```bash
docker build -f Dockerfile.curl -t openfang-curl:latest .
docker run --rm openfang-curl:latest curl -s https://example.com
```
Use this variant when you need `HEALTHCHECK` directives or in-container
diagnostics. The base image stays slim by default.
---
## Configuration Issues
@@ -596,6 +665,13 @@ docker run -d --name openfang \
ghcr.io/rightnow-ai/openfang:latest
```
To reach a host LLM (Ollama, vLLM, whisper.cpp) from inside the container,
add `--add-host=host.docker.internal:host-gateway`. See
[Connecting to host services from Docker](#connecting-to-host-services-from-docker).
The default image does not ship `curl`; build the
[curl-equipped overlay](#curl-equipped-reference-image) if you need
in-container healthchecks.
### How do I protect the dashboard with a password?
OpenFang has built-in dashboard authentication. Enable it in `~/.openfang/config.toml`: