mirror of
https://github.com/ValueCell-ai/ClawX.git
synced 2026-08-14 00:48:10 +00:00
fix(chat): display OpenClaw transcript attachments (#1218)
This commit is contained in:
paisley
+1
-1
@@ -231,7 +231,7 @@ Chat は Electron Main が所有する ACP stdio bridge を使用します。Ren
|
||||
|
||||
ACP の assistant ターンにはターン全体の所要時間が表示されます。ライブ計時はクライアントが観測した prompt ライフサイクルに従い、アプリ内を移動しても継続します。履歴の所要時間は Electron Main が範囲を限定した OpenClaw transcript のタイムスタンプから算出し、ACP リプレイですでに復元されたターンだけに付与します。
|
||||
|
||||
ACP Chat は標準 ACP resource を添付ファイルとして表示します。ユーザーが選択した画像は、ホバー時のオーバーレイにファイル名を表示するサムネイルとして描画され、その他の利用可能な添付カードはファイル名に続いて、淡色で省略可能なソースパスを表示します。現在の OpenClaw ACP adapter が assistant のメディアを省略した場合も、明示的な assistant の `MEDIA:` ディレクティブを、元のディレクティブを表示せずに添付カードとして復元できます。現在の workspace 外を含む既存のローカルファイル参照は、プレビューまたはオープンのたびに Electron Main で正確な session と generation に対して再検証されます。AI が生成したプレビュー可能なローカル添付ファイル（20 MB 以下の `.docx` と `.pptx` を含む）は、読み取り専用のアプリ内プレビューを主要操作として維持し、対応アプリで開く操作と Finder、エクスプローラー、またはシステムのファイルマネージャーで表示する操作を副次メニューから利用できます。ローカル HTML 添付ファイルでは、そのメニューの先頭項目が右側のプレビューでファイルを開きます。ここでも Office プレビューには同じ制限があります。`.doc` と `.ppt` はシステムアプリで開く形式のままで、DOCX のページ区切りは Microsoft Word と異なる場合があり、PPTX のアニメーション、画面切り替え、メディア再生はサポートされません。対応アプリの検出は macOS と Windows のみで利用でき、Linux または検出失敗時には通知せず、ファイルの場所を表示する操作だけに切り替わります。それ以外のローカルファイル（20 MB を超える Office ファイルを含む）はユーザーのクリック後にシステムアプリで開かれます。ユーザーが選択したフォルダー添付も送信後に利用可能なまま保持され、クリックするとシステムのファイルマネージャーで開きます。ClawX はフォルダー内を読み取りまたはプレビューしません。リモートの HTTP/HTTPS 添付ファイルはクリック後に外部で開かれます。通常の文章内にある単独またはインラインのパスは添付ファイルとして扱われません。
|
||||
ACP Chat は標準 ACP resource を添付ファイルとして表示します。ユーザーが選択した画像は、ホバー時のオーバーレイにファイル名を表示するサムネイルとして描画され、その他の利用可能な添付カードはファイル名に続いて、淡色で省略可能なソースパスを表示します。現在の OpenClaw ACP adapter が assistant のメディアを省略した場合も、OpenClaw が永続化した正規メディア情報と明示的な assistant の `MEDIA:` ディレクティブを、transcript 専用メタデータを表示せずに添付カードとして復元できます。現在の workspace 外を含む既存のローカルファイル参照は、プレビューまたはオープンのたびに Electron Main で正確な session と generation に対して再検証されます。AI が生成したプレビュー可能なローカル添付ファイル（20 MB 以下の `.docx` と `.pptx` を含む）は、読み取り専用のアプリ内プレビューを主要操作として維持し、対応アプリで開く操作と Finder、エクスプローラー、またはシステムのファイルマネージャーで表示する操作を副次メニューから利用できます。ローカル HTML 添付ファイルでは、そのメニューの先頭項目が右側のプレビューでファイルを開きます。ここでも Office プレビューには同じ制限があります。`.doc` と `.ppt` はシステムアプリで開く形式のままで、DOCX のページ区切りは Microsoft Word と異なる場合があり、PPTX のアニメーション、画面切り替え、メディア再生はサポートされません。対応アプリの検出は macOS と Windows のみで利用でき、Linux または検出失敗時には通知せず、ファイルの場所を表示する操作だけに切り替わります。それ以外のローカルファイル（20 MB を超える Office ファイルを含む）はユーザーのクリック後にシステムアプリで開かれます。ユーザーが選択したフォルダー添付も送信後に利用可能なまま保持され、クリックするとシステムのファイルマネージャーで開きます。ClawX はフォルダー内を読み取りまたはプレビューしません。リモートの HTTP/HTTPS 添付ファイルはクリック後に外部で開かれます。正規メディア情報がない通常の文章内の単独またはインラインパスは添付ファイルとして扱われません。
|
||||
|
||||
ACP Chat は、runtime が画像生成メディアを信頼できる構造化メディアとして配信した場合に、生成画像のプレビューも表示できます。信頼できる OpenClaw internal-UI 配信と画像生成タスクに関連付けられた最終返信では、テキストのみの失敗説明を含む元のユーザー向け完了テキストを保持し、汎用の画像キャプションへ置き換えません。OpenClaw の履歴リプレイ中は、同じセッションで画像生成タスク開始が記録されている場合に限り、assistant の画像 `MEDIA:` マーカーがインライン画像表示へ昇格されます。ClawX は Renderer から任意にファイルシステムへアクセスするのではなく、Electron Main のホストメディア処理を通じてプレビューを読み込みます。標準 ACP の画像と resource コンテンツは引き続き推奨パスであり、そのまま描画されます。
|
||||
|
||||
|
||||
@@ -234,7 +234,7 @@ An unfinished ACP response keeps streaming when you open another conversation or
|
||||
|
||||
ACP assistant turns show whole-turn duration. Live timing follows the client-observed prompt lifecycle and survives in-app navigation; historical timing is derived in Electron Main from bounded OpenClaw transcript timestamps and only annotates a turn already restored by ACP replay.
|
||||
|
||||
ACP Chat renders standard ACP resources as attachments. User-selected images appear as thumbnails with a filename hover overlay, while other available attachment cards show the filename and a muted, truncating source path. When the current OpenClaw ACP adapter omits assistant media, explicit assistant `MEDIA:` directives can also be recovered as attachment cards without displaying the raw directive. Existing local file references, including paths outside the active workspace, are revalidated in Electron Main for the exact session and generation before every preview or open. Previewable local attachments produced by the AI, including `.docx` and `.pptx` files within the 20 MB inline-preview limit, keep their primary read-only in-app preview action and provide a secondary menu for opening with compatible applications or revealing the file in Finder, File Explorer, or the system file manager. For local HTML attachments, that menu starts with an action that opens the file in the right-side Preview tab. The same Office limitations apply here: `.doc` and `.ppt` remain system-open formats, DOCX pagination may differ from Microsoft Word, and PPTX animations, transitions, and media playback are unsupported. Compatible-application discovery is available only on macOS and Windows and silently degrades to reveal-only behavior on Linux or when discovery fails. Other local files, including Office files larger than 20 MB, open in the system application after a user click. User-selected folder attachments also remain available after send and open in the system file manager; ClawX does not read or preview their contents. Remote HTTP and HTTPS attachments open externally after a user click. Bare or inline prose paths are not treated as attachments.
|
||||
ACP Chat renders standard ACP resources as attachments. User-selected images appear as thumbnails with a filename hover overlay, while other available attachment cards show the filename and a muted, truncating source path. When the current OpenClaw ACP adapter omits assistant media, canonical persisted OpenClaw media facts and explicit assistant `MEDIA:` directives can also be recovered as attachment cards without displaying transcript-only metadata. Existing local file references, including paths outside the active workspace, are revalidated in Electron Main for the exact session and generation before every preview or open. Previewable local attachments produced by the AI, including `.docx` and `.pptx` files within the 20 MB inline-preview limit, keep their primary read-only in-app preview action and provide a secondary menu for opening with compatible applications or revealing the file in Finder, File Explorer, or the system file manager. For local HTML attachments, that menu starts with an action that opens the file in the right-side Preview tab. The same Office limitations apply here: `.doc` and `.ppt` remain system-open formats, DOCX pagination may differ from Microsoft Word, and PPTX animations, transitions, and media playback are unsupported. Compatible-application discovery is available only on macOS and Windows and silently degrades to reveal-only behavior on Linux or when discovery fails. Other local files, including Office files larger than 20 MB, open in the system application after a user click. User-selected folder attachments also remain available after send and open in the system file manager; ClawX does not read or preview their contents. Remote HTTP and HTTPS attachments open externally after a user click. Bare or inline prose paths without canonical media facts are not treated as attachments.
|
||||
|
||||
ACP Chat can also display generated image previews when image-generation media is delivered by the runtime as trusted structured media. Trusted OpenClaw internal-UI deliveries and task-correlated final replies preserve the original user-facing completion text, including text-only failure explanations, rather than replacing it with a generic image caption. During historical OpenClaw replay, assistant image `MEDIA:` markers are promoted to the inline image experience only when they follow a recorded image-generation task start for that session. ClawX loads previews through host media handling in Electron Main, not arbitrary Renderer filesystem access. Standard ACP image and resource content remains the preferred path and renders directly.
|
||||
|
||||
|
||||
+1
-1
@@ -235,7 +235,7 @@ Chat 使用由 Electron Main 持有的 ACP stdio bridge。Renderer 接收类型
|
||||
|
||||
ACP assistant 回合会显示整轮耗时。Live 计时跟随客户端观测到的 prompt 生命周期，并在应用内导航后保持连续；历史耗时由 Electron Main 根据有界的 OpenClaw transcript 时间戳计算，而且只能标注 ACP 回放已经恢复出的回合。
|
||||
|
||||
ACP Chat 会将标准 ACP resource 渲染为附件。用户选择的图片会显示为缩略图，并在悬停蒙层中显示文件名；其它可用的附件卡片会显示文件名，以及灰色、可截断的来源路径。当前 OpenClaw ACP adapter 遗漏 assistant 媒体时，显式的 assistant `MEDIA:` 指令也可恢复为附件卡片，且不会显示原始指令。现有本地文件引用（包括当前 workspace 外的路径）在每次预览或打开前，都会由 Electron Main 按精确的 session 和 generation 重新验证。AI 生成且可预览的本地附件（包括不超过 20 MB 的 `.docx` 和 `.pptx` 文件）会保留主要的只读应用内预览操作，并提供次级菜单，可通过兼容应用打开，或在 Finder、文件资源管理器或系统文件管理器中显示。对于本地 HTML 附件，该菜单第一项会在右侧预览中打开文件。Office 预览在此处也有相同限制：`.doc` 和 `.ppt` 仍通过系统应用打开，DOCX 的分页效果可能与 Microsoft Word 不同，PPTX 的动画、切换效果和媒体播放不受支持。兼容应用发现仅在 macOS 和 Windows 上可用；在 Linux 上或发现失败时，会静默降级为仅显示文件位置。其它本地文件（包括超过 20 MB 的 Office 文件）会在用户点击后通过系统应用打开。用户选择的文件夹附件在发送后也会保持可用，点击后交给系统文件管理器打开；ClawX 不会读取或预览其中内容。远程 HTTP 和 HTTPS 附件会在用户点击后从外部打开。普通文本中的裸路径或行内路径不会被当作附件。
|
||||
ACP Chat 会将标准 ACP resource 渲染为附件。用户选择的图片会显示为缩略图，并在悬停蒙层中显示文件名；其它可用的附件卡片会显示文件名，以及灰色、可截断的来源路径。当前 OpenClaw ACP adapter 遗漏 assistant 媒体时，OpenClaw 持久化的规范媒体事实和显式 assistant `MEDIA:` 指令也可恢复为附件卡片，且不会显示仅用于 transcript 的元数据。现有本地文件引用（包括当前 workspace 外的路径）在每次预览或打开前，都会由 Electron Main 按精确的 session 和 generation 重新验证。AI 生成且可预览的本地附件（包括不超过 20 MB 的 `.docx` 和 `.pptx` 文件）会保留主要的只读应用内预览操作，并提供次级菜单，可通过兼容应用打开，或在 Finder、文件资源管理器或系统文件管理器中显示。对于本地 HTML 附件，该菜单第一项会在右侧预览中打开文件。Office 预览在此处也有相同限制：`.doc` 和 `.ppt` 仍通过系统应用打开，DOCX 的分页效果可能与 Microsoft Word 不同，PPTX 的动画、切换效果和媒体播放不受支持。兼容应用发现仅在 macOS 和 Windows 上可用；在 Linux 上或发现失败时，会静默降级为仅显示文件位置。其它本地文件（包括超过 20 MB 的 Office 文件）会在用户点击后通过系统应用打开。用户选择的文件夹附件在发送后也会保持可用，点击后交给系统文件管理器打开；ClawX 不会读取或预览其中内容。远程 HTTP 和 HTTPS 附件会在用户点击后从外部打开。没有规范媒体事实佐证的普通文本裸路径或行内路径不会被当作附件。
|
||||
|
||||
ACP Chat 也可在 runtime 以可信结构化媒体投递图像生成结果时显示生成图片预览。对于可信的 OpenClaw internal-UI 投递和与生图任务关联的最终回复，ClawX 会保留原始的用户可见完成文案，包括只有文本的失败说明，而不会统一替换成通用图片文案。历史 OpenClaw 回放中，assistant 的图片 `MEDIA:` 标记只有在同一会话已记录图像生成任务启动后才会进入内联图片体验。ClawX 通过 Electron Main 的主机媒体处理加载预览，而不是让 Renderer 任意访问文件系统。标准 ACP 图片和 resource 内容仍是首选路径，并会直接渲染。
|
||||
|
||||
|
||||
@@ -17,13 +17,13 @@ Standard ACP image, `resource_link`, and URI-backed `resource` content blocks ar
|
||||
This section is the durable rationale referenced by the transcript supplement entry point. The two content exceptions are:
|
||||
|
||||
1. Image-generation completion with proven `image_generate` context. Trusted structured runtime evidence or approved transcript evidence may restore the completion caption, failure explanation, and media as the existing inline-image experience.
|
||||
2. General attachment recovery from an explicit line-leading assistant `MEDIA:` directive outside fenced code blocks. This exception does not require image-generation context, but it recovers only the attachment reference, never the surrounding assistant message.
|
||||
2. General attachment recovery from a canonical persisted assistant `__openclaw.media` fact or an explicit line-leading assistant `MEDIA:` directive outside fenced code blocks. This exception does not require image-generation context, but it recovers only attachment references and declared media metadata, never the surrounding assistant message.
|
||||
|
||||
Both content exceptions use one bounded transcript fetch coordinator, keep projected state in memory, require exact active session and generation identity, and reject stale or ambiguous evidence. Existing-session load reads at most 1000 recent transcript messages. An ordinary successful live prompt performs one immediate read and one retry 1500 milliseconds later. Only an `image_generate` task recorded for that same live prompt extends the coordinator through bounded backoff while waiting for its completion artifact; accepted completion, invalidation, or retry-window exhaustion stops it. These exceptions must be removed when the distributed OpenClaw ACP adapter emits the equivalent standard content.
|
||||
|
||||
The same historical coordinator may request metadata-only whole-turn timing from Main. This is necessary because ACP `session/load` supplies replay content and status but not the original timestamps needed to calculate duration. Main derives candidates from bounded transcript JSONL envelopes, and Renderer aligns them with the same normalized user text and duplicate occurrence-from-tail rule. Timing can annotate only an ACP-created turn and never recovers transcript content.
|
||||
|
||||
Transcript supplementation must not recover or reconstruct ordinary assistant messages, thoughts, tools, plans, permissions, file activity, or a parallel Chat history. Bare paths, inline prose paths, unknown URI schemes, incidental tool paths, and directives inside fenced code blocks are not general attachments.
|
||||
Transcript supplementation must not recover or reconstruct ordinary assistant messages, thoughts, tools, plans, permissions, file activity, or a parallel Chat history. Bare paths and inline prose paths without canonical media facts, unknown URI schemes, incidental tool paths, and directives inside fenced code blocks are not general attachments.
|
||||
|
||||
### Image-Generation Completion
|
||||
|
||||
@@ -40,11 +40,13 @@ Accepted live evidence includes structured media fields such as `mediaUrl`, `med
|
||||
|
||||
When trusted source-reply text exists, it is preserved whether or not media is present. If no source-reply text exists, successful media uses the localized generic caption; partial or failed thumbnail hydration uses the existing localized fallback. Raw `MEDIA:` paths are never displayed.
|
||||
|
||||
### Explicit MEDIA Attachments
|
||||
### Canonical And Explicit MEDIA Attachments
|
||||
|
||||
The general attachment extractor considers normalized assistant roles only. After optional leading whitespace, a whole line must start with the case-insensitive `MEDIA:` token and contain exactly one reference. Single- or double-quoted references may contain spaces and must close with the same quote; unquoted references cannot contain whitespace. One accepted line produces one candidate, and multiple lines retain transcript order. The current source-reference bound is `4096` characters.
|
||||
The general attachment extractor considers normalized assistant roles only. Its preferred transcript evidence is OpenClaw's canonical persisted `__openclaw.media` array. Each fact may contribute one ordered `path` or `url` plus bounded filename, content type, and size metadata; the containing assistant message id is retained for Main-side outgoing-record validation. Canonical facts do not authorize access: every reference and metadata value remains untrusted and passes through the existing Main attachment boundary.
|
||||
|
||||
Accepted reference forms are absolute POSIX paths, Windows drive paths, `file://` URIs, `~/` paths, paths relative to the registered execution cwd, and HTTP or HTTPS URLs. Relative paths are accepted only when execution cwd is available. Unknown URI schemes, malformed URLs or quotes, empty references, Markdown/list wrappers, inline prose, ordinary bare paths, and wrapped references are rejected. Markdown backtick and tilde fences follow the delimiter character and opening length; all content remains ignored until a valid close with the same delimiter and at least that length. The parser does not render the raw directive or surrounding transcript prose.
|
||||
For the legacy directive form, after optional leading whitespace, a whole line must start with the case-insensitive `MEDIA:` token and contain exactly one reference. Single- or double-quoted references may contain spaces and must close with the same quote; unquoted references cannot contain whitespace. One accepted line produces one candidate, and multiple lines retain transcript order. When a canonical fact and directive identify the same URI in one assistant message, the canonical fact wins. The current source-reference bound is `4096` characters.
|
||||
|
||||
Accepted reference forms are absolute POSIX paths, Windows drive paths, `file://` URIs, `~/` paths, paths relative to the registered execution cwd, and HTTP or HTTPS URLs. Relative paths are accepted only when execution cwd is available. Canonical structured values may contain spaces. Unknown URI schemes, malformed URLs or quotes, empty references, Markdown/list wrappers, inline prose without canonical evidence, ordinary bare paths without canonical evidence, and wrapped directives are rejected. Markdown backtick and tilde fences follow the delimiter character and opening length; all content remains ignored until a valid close with the same delimiter and at least that length. The parser does not recover or render surrounding transcript prose.
|
||||
|
||||
Transcript and ACP messages are partitioned by real user boundaries; leading orphan assistant content is ineligible. OpenClaw ACP does not project assistant `MEDIA:` attachments, so ClawX must read this bounded transcript supplement. To align it without parsing user-authored marker text, each ACP user segment retains only the ordered, binary-free text blocks produced by OpenClaw's prompt flattening: text and embedded text remain text, `resource_link` becomes OpenClaw's escaped `[Resource link]` form, and image/audio/blob data is omitted. User matching then removes only the known OpenClaw working-directory envelope and normalizes line endings and surrounding whitespace; it does not use broad fuzzy matching or globally strip resource markers. Because transcript history is a bounded suffix and cross-source message ids are not durable, alignment proceeds newest-to-oldest with the tuple of normalized flattened user text and duplicate occurrence from the tail. Attachment-only empty text remains eligible under the same real-user boundary and occurrence rules. A live supplement additionally requires the optimistic ACP user identity and restricts extraction to that current turn. Missing, duplicate, or ambiguous anchors are skipped instead of assigned by ordinal offset or nearest-turn guesswork.
|
||||
|
||||
@@ -64,13 +66,13 @@ After successful `loadSession` for an existing session, the store may call:
|
||||
hostApi.sessions.history({ sessionKey, limit: 1000 });
|
||||
```
|
||||
|
||||
A pure image-generation extractor scans messages in transcript order. It first records an `image_generate` start from a tool result, then accepts a later internal-UI `message` tool source reply or assistant completion associated with that task. OpenClaw's runtime-generated inter-session completion trigger remains part of the originating user turn rather than starting a new end-user turn. Assistant media captions have their `MEDIA:` directives removed before display, and a task-correlated text-only assistant reply may restore a failure explanation. A message-tool reply or image completion without preceding task context is rejected. Separately, the general attachment extractor may accept explicit assistant `MEDIA:` directives without image-generation context under the restrictions above. Read failure, no accepted evidence, duplicate evidence, or a stale generation leaves the ACP timeline unchanged.
|
||||
A pure image-generation extractor scans messages in transcript order. It first records an `image_generate` start from a tool result, then accepts a later internal-UI `message` tool source reply or assistant completion associated with that task. OpenClaw's runtime-generated inter-session completion trigger remains part of the originating user turn rather than starting a new end-user turn. Assistant media captions have their `MEDIA:` directives removed before display, and a task-correlated text-only assistant reply may restore a failure explanation. A message-tool reply or image completion without preceding task context is rejected. Separately, the general attachment extractor may accept canonical persisted assistant media facts or explicit assistant `MEDIA:` directives without image-generation context under the restrictions above. Read failure, no accepted evidence, duplicate evidence, or a stale generation leaves the ACP timeline unchanged.
|
||||
|
||||
These are the only transcript-derived Chat content supplements. Metadata-only whole-turn timing is also permitted, but it must not become a general recovery mechanism for missing turns, tool cards, file activity, plans, permissions, thoughts, or ordinary messages.
|
||||
|
||||
## Rejected Compatibility Alternatives
|
||||
|
||||
Main does not manufacture ACP `agent_message_chunk` resource events from transcript evidence because that would misrepresent compatibility data as native protocol replay. The ACP page does not reuse legacy Chat path extraction or rendering because that would restore competing history authorities. Standard-ACP-only behavior is insufficient while the distributed adapter omits assistant media, but the exception remains removable when upstream emits standard resources. Bare-path or broad prose extraction is rejected because false positives would widen the local-file trust surface.
|
||||
Main does not manufacture ACP `agent_message_chunk` resource events from transcript evidence because that would misrepresent compatibility data as native protocol replay. The ACP page does not reuse legacy Chat path extraction or rendering because that would restore competing history authorities. Standard-ACP-only behavior is insufficient while the distributed adapter omits assistant media, but the exception remains removable when upstream emits standard resources. Canonical persisted media facts are explicit structured evidence; bare-path or broad prose extraction without those facts remains rejected because false positives would widen the local-file trust surface.
|
||||
|
||||
## Trace Channel
|
||||
|
||||
|
||||
@@ -9,6 +9,6 @@ appliesTo:
|
||||
|
||||
Standard ACP content is authoritative and preferred. A compatibility supplement is allowed only when it is explicitly marked by source, retained in memory, backed by approved structured runtime evidence or explicit assistant transcript evidence, and accompanied by reason-coded diagnostics. Compatibility data must never be represented as a native ACP event.
|
||||
|
||||
Approved transcript evidence has two bounded forms: asynchronous image-generation completion with proven image-generation context, including explicit internal-UI `message` tool source replies; and general attachment recovery from whole-line, line-leading assistant OpenClaw `MEDIA:` directives outside fenced code blocks. The general form accepts only the documented local path, `file:`, execution-cwd-relative, HTTP, and HTTPS forms; quoted references may contain spaces, while unquoted references may not. It does not require image-generation context and projects only one ordered attachment reference per directive, never surrounding transcript prose. A trusted image-generation source reply may provide user-facing completion or failure text. Reject malformed or wrapped directives, bare or inline prose paths, unknown URI schemes, incidental tool paths, and unrelated assistant prose.
|
||||
Approved transcript evidence has three bounded forms: asynchronous image-generation completion with proven image-generation context, including explicit internal-UI `message` tool source replies; canonical persisted assistant `__openclaw.media` facts; and general attachment recovery from whole-line, line-leading assistant OpenClaw `MEDIA:` directives outside fenced code blocks. Canonical facts and directives accept only the documented local path, `file:`, execution-cwd-relative, HTTP, and HTTPS forms. Quoted directive references may contain spaces, while unquoted directives may not; canonical structured values may contain spaces. General recovery projects only ordered attachment references and declared media metadata, never surrounding transcript prose. A trusted image-generation source reply may provide user-facing completion or failure text. Reject malformed or wrapped directives, bare or inline prose paths without canonical media facts, unknown URI schemes, incidental tool paths, and unrelated assistant prose.
|
||||
|
||||
Compatibility logic must not reconstruct ordinary assistant messages, thoughts, tools, plans, permissions, file activity, or a parallel Chat history. User-side OpenClaw prompt projection may be reconstructed only from structured ACP content already present in the same timeline; generated-looking user prose is not evidence and must not be stripped or parsed. Unmatched or ambiguous evidence is skipped rather than attached by guesswork. Deduplication is turn-scoped and uses only a Main-authorized opaque identity; native ACP resource content wins over equivalent compatibility evidence, generated-image evidence remains inline, and an unavailable result does not block a later available upgrade.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
id: acp-media-attachments
|
||||
title: Render ACP resources and bounded OpenClaw MEDIA attachments
|
||||
title: Render ACP resources and bounded OpenClaw media attachments
|
||||
scenario: gateway-backend-communication
|
||||
taskType: runtime-bridge
|
||||
intent: Render standard ACP resources and recover only explicit OpenClaw MEDIA attachments omitted by the distributed ACP adapter through a bounded transcript compatibility projection.
|
||||
intent: Render standard ACP resources and recover canonical persisted OpenClaw media facts or explicit MEDIA attachments omitted by the distributed ACP adapter through a bounded transcript compatibility projection.
|
||||
touchedAreas:
|
||||
- package.json
|
||||
- harness/specs/tasks/acp-media-attachments.md
|
||||
@@ -23,6 +23,7 @@ touchedAreas:
|
||||
- harness/reference/acp-generated-media-and-diagnostics.md
|
||||
- harness/reference/openclaw-file-activity.md
|
||||
- shared/acp-chat/types.ts
|
||||
- shared/chat/types.ts
|
||||
- shared/host-api/contract.ts
|
||||
- shared/file-preview/limits.ts
|
||||
- electron/services/acp-session-access-registry.ts
|
||||
@@ -115,6 +116,7 @@ touchedAreas:
|
||||
- README.ja-JP.md
|
||||
expectedUserBehavior:
|
||||
- Standard ACP resource_link and URI-backed resource content renders as paperclip attachment cards.
|
||||
- Canonical assistant `__openclaw.media` facts render as attachment cards even when visible prose only mentions the output path.
|
||||
- Explicit assistant OpenClaw MEDIA directives omitted by ACP are recovered for live completions and historical session loads without displaying the raw directive.
|
||||
- MEDIA recovery remains aligned when the triggering ACP user turn contains structured resources, images, or no text.
|
||||
- Attachment rows render after assistant prose and preserve declaration order.
|
||||
@@ -157,6 +159,7 @@ requiredTests:
|
||||
- pnpm run harness:ci
|
||||
acceptance:
|
||||
- A standard ACP resource_link or URI-backed resource renders an actionable paperclip attachment card.
|
||||
- A canonical persisted assistant `__openclaw.media` fact renders an actionable attachment card and carries its declared filename, content type, size, and transcript message identity into Main validation.
|
||||
- User image attachments render as thumbnails with the filename revealed by a hover overlay.
|
||||
- User non-image attachments render the filename followed by a muted, truncating source path and no MIME label.
|
||||
- The reported OpenClaw MEDIA directive for budget_sample.xlsx renders an attachment in ACP Chat even though OpenClaw ACP emits no resource block.
|
||||
@@ -182,13 +185,13 @@ docs:
|
||||
|
||||
## Scope
|
||||
|
||||
Standard ACP resource content is the preferred attachment source. The OpenClaw transcript path is a bounded compatibility exception for explicit assistant `MEDIA:` directives that the distributed ACP adapter omits; it is not a second Chat history source.
|
||||
Standard ACP resource content is the preferred attachment source. The OpenClaw transcript path is a bounded compatibility exception for canonical persisted assistant `__openclaw.media` facts and explicit assistant `MEDIA:` directives that the distributed ACP adapter omits; it is not a second Chat history source.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- Modifying OpenClaw or its distributed package.
|
||||
- Reconstructing ordinary messages, tools, plans, permissions, thoughts, or file activity from transcripts.
|
||||
- Extracting bare paths or inline paths from ordinary assistant prose.
|
||||
- Treating bare paths or inline paths from ordinary assistant prose as attachment evidence without a canonical persisted media fact.
|
||||
- Persisting a synthetic ACP attachment ledger or compatibility cache.
|
||||
|
||||
## Acceptance Traceability
|
||||
@@ -196,7 +199,7 @@ Standard ACP resource content is the preferred attachment source. The OpenClaw t
|
||||
| Acceptance behavior | Test or durable rule |
|
||||
| --- | --- |
|
||||
| Standard ACP resources render actionable cards | `tests/unit/acp-reducer.test.ts`, `tests/unit/acp-chat-components.test.tsx`, `tests/e2e/chat-acp-attachments.spec.ts` |
|
||||
| Explicit OpenClaw `MEDIA:` recovery and hidden raw directives | `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts` |
|
||||
| Canonical persisted OpenClaw media facts, explicit `MEDIA:` recovery, and hidden raw directives | `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts` |
|
||||
| Explicit parser grammar rejects fenced, wrapped, inline, malformed, unknown-scheme, and overlong values | `tests/unit/acp-media-attachments.test.ts`, `acp-compatibility-content-safety` |
|
||||
| Transcript suffix alignment uses normalized user text and occurrence from the tail without guessing | `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-chat-store.test.ts`, `acp-chat-state-and-history` |
|
||||
| Attached and attachment-only user turns use binary-free structured prompt projection | `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-reducer.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts`, `acp-chat-state-and-history` |
|
||||
|
||||
@@ -35,6 +35,19 @@ export interface RawMessage {
|
||||
stop_reason?: string;
|
||||
errorMessage?: string;
|
||||
error_message?: string;
|
||||
/** Canonical OpenClaw-owned transcript metadata. */
|
||||
__openclaw?: {
|
||||
media?: Array<{
|
||||
path?: string;
|
||||
url?: string;
|
||||
contentType?: string;
|
||||
kind?: string;
|
||||
fileName?: string;
|
||||
sizeBytes?: number;
|
||||
messageId?: string;
|
||||
workspaceDir?: string;
|
||||
}>;
|
||||
};
|
||||
/** Local-only: file metadata for user-uploaded attachments (not sent to/from Gateway) */
|
||||
_attachedFiles?: AttachedFileMeta[];
|
||||
}
|
||||
|
||||
@@ -12,6 +12,9 @@ export type OpenClawMediaCandidate = {
|
||||
evidenceId: string;
|
||||
transcriptMessageId?: string;
|
||||
uri: string;
|
||||
name?: string;
|
||||
mimeType?: string;
|
||||
size?: number;
|
||||
order: number;
|
||||
};
|
||||
|
||||
@@ -59,6 +62,20 @@ function textFromContent(content: unknown): string {
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
function recordValue(value: unknown): Record<string, unknown> | null {
|
||||
return value && typeof value === 'object' && !Array.isArray(value)
|
||||
? value as Record<string, unknown>
|
||||
: null;
|
||||
}
|
||||
|
||||
function optionalString(value: unknown): string | undefined {
|
||||
return typeof value === 'string' && value.trim() ? value.trim() : undefined;
|
||||
}
|
||||
|
||||
function optionalSize(value: unknown): number | undefined {
|
||||
return typeof value === 'number' && Number.isFinite(value) && value >= 0 ? value : undefined;
|
||||
}
|
||||
|
||||
function normalizeUserText(text: string): string {
|
||||
return stripAcpWorkingDirectoryPrefix(text)
|
||||
.replace(/\r\n/g, '\n')
|
||||
@@ -108,6 +125,56 @@ function parseDirectiveReference(line: string, executionCwd: string): string | n
|
||||
return executionCwd.trim() ? reference : null;
|
||||
}
|
||||
|
||||
function parseStructuredReference(value: unknown, executionCwd: string): string | null {
|
||||
const reference = optionalString(value);
|
||||
if (!reference || reference.length > MAX_MEDIA_REFERENCE_LENGTH) return null;
|
||||
if (/^https?:\/\//i.test(reference)) {
|
||||
try {
|
||||
const url = new URL(reference);
|
||||
return url.protocol === 'http:' || url.protocol === 'https:' ? reference : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
if (/^file:\/\//i.test(reference)) return reference;
|
||||
if (URI_SCHEME_RE.test(reference) && !WINDOWS_ABSOLUTE_RE.test(reference)) return null;
|
||||
if (reference.startsWith('/') || reference.startsWith('~/') || WINDOWS_ABSOLUTE_RE.test(reference)) {
|
||||
return reference;
|
||||
}
|
||||
return executionCwd.trim() ? reference : null;
|
||||
}
|
||||
|
||||
type PersistedMediaFact = {
|
||||
uri: string;
|
||||
name?: string;
|
||||
mimeType?: string;
|
||||
size?: number;
|
||||
index: number;
|
||||
};
|
||||
|
||||
function persistedMediaFacts(message: RawMessage, executionCwd: string): PersistedMediaFact[] {
|
||||
const metadata = recordValue((message as RawMessage & { __openclaw?: unknown }).__openclaw);
|
||||
const media = metadata?.media;
|
||||
if (!Array.isArray(media)) return [];
|
||||
|
||||
return media.flatMap((value, index) => {
|
||||
const fact = recordValue(value);
|
||||
if (!fact) return [];
|
||||
const uri = parseStructuredReference(fact.path ?? fact.url, executionCwd);
|
||||
if (!uri) return [];
|
||||
const name = optionalString(fact.fileName);
|
||||
const mimeType = optionalString(fact.contentType);
|
||||
const size = optionalSize(fact.sizeBytes);
|
||||
return [{
|
||||
uri,
|
||||
index,
|
||||
...(name ? { name } : {}),
|
||||
...(mimeType ? { mimeType } : {}),
|
||||
...(size !== undefined ? { size } : {}),
|
||||
}];
|
||||
});
|
||||
}
|
||||
|
||||
function mediaReferences(text: string): Array<{ uri: string; line: number }> {
|
||||
const references: Array<{ uri: string; line: number }> = [];
|
||||
let fence: { marker: string; length: number } | null = null;
|
||||
@@ -166,15 +233,30 @@ export function extractOpenClawMediaTurns(
|
||||
if (role !== 'assistant' || !current) continue;
|
||||
|
||||
const text = textFromContent(message.content);
|
||||
const messageIdentity = message.id
|
||||
? `id:${message.id}`
|
||||
: message.timestamp != null
|
||||
? `timestamp:${message.timestamp}`
|
||||
: `content:${stableHash(text)}`;
|
||||
const structuredFacts = persistedMediaFacts(message, input.executionCwd);
|
||||
const structuredUris = new Set(structuredFacts.map((fact) => fact.uri));
|
||||
for (const fact of structuredFacts) {
|
||||
if (input.suppressedUris.has(fact.uri)) continue;
|
||||
const order = current.candidates.length;
|
||||
current.candidates.push({
|
||||
evidenceSeed: `${messageIdentity}:structured:${fact.index}:${fact.uri}`,
|
||||
...(message.id ? { transcriptMessageId: message.id } : {}),
|
||||
uri: fact.uri,
|
||||
order,
|
||||
...(fact.name ? { name: fact.name } : {}),
|
||||
...(fact.mimeType ? { mimeType: fact.mimeType } : {}),
|
||||
...(fact.size !== undefined ? { size: fact.size } : {}),
|
||||
});
|
||||
}
|
||||
for (const reference of mediaReferences(text)) {
|
||||
const uri = parseDirectiveReference(reference.uri, input.executionCwd);
|
||||
if (!uri || input.suppressedUris.has(uri)) continue;
|
||||
if (!uri || structuredUris.has(uri) || input.suppressedUris.has(uri)) continue;
|
||||
const order = current.candidates.length;
|
||||
const messageIdentity = message.id
|
||||
? `id:${message.id}`
|
||||
: message.timestamp != null
|
||||
? `timestamp:${message.timestamp}`
|
||||
: `content:${stableHash(text)}`;
|
||||
current.candidates.push({
|
||||
evidenceSeed: `${messageIdentity}:${reference.line}:${uri}`,
|
||||
...(message.id ? { transcriptMessageId: message.id } : {}),
|
||||
|
||||
@@ -607,6 +607,7 @@ async function resolveOpenClawMediaCandidate(
|
||||
if (!isCurrent()) return;
|
||||
|
||||
let result: ResolveAttachmentResult;
|
||||
const attachmentName = safeAttachmentName(candidate.name ?? candidate.uri);
|
||||
try {
|
||||
result = await hostApi.files.resolveAttachment({
|
||||
ref: {
|
||||
@@ -615,10 +616,12 @@ async function resolveOpenClawMediaCandidate(
|
||||
uri: candidate.uri,
|
||||
...(candidate.transcriptMessageId ? { transcriptMessageId: candidate.transcriptMessageId } : {}),
|
||||
},
|
||||
name: safeAttachmentName(candidate.uri),
|
||||
name: attachmentName,
|
||||
...(candidate.mimeType ? { mimeType: candidate.mimeType } : {}),
|
||||
...(candidate.size !== undefined ? { size: candidate.size } : {}),
|
||||
});
|
||||
} catch {
|
||||
result = { ok: false, displayName: safeAttachmentName(candidate.uri), error: 'operationFailed' };
|
||||
result = { ok: false, displayName: attachmentName, error: 'operationFailed' };
|
||||
}
|
||||
|
||||
const evidenceHash = hashOpenClawMediaDiagnostic(candidate.evidenceId);
|
||||
@@ -646,7 +649,9 @@ async function resolveOpenClawMediaCandidate(
|
||||
segmentIndex: 0,
|
||||
blockIndex: candidate.order,
|
||||
uri: candidate.uri,
|
||||
name: safeAttachmentName(candidate.uri),
|
||||
name: attachmentName,
|
||||
...(candidate.mimeType ? { mimeType: candidate.mimeType } : {}),
|
||||
...(candidate.size !== undefined ? { size: candidate.size } : {}),
|
||||
...(candidate.transcriptMessageId ? { transcriptMessageId: candidate.transcriptMessageId } : {}),
|
||||
source: 'openclaw-media',
|
||||
evidenceId: candidate.evidenceId,
|
||||
|
||||
@@ -616,6 +616,69 @@ test.describe('ACP media attachments', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('renders canonical OpenClaw transcript media when assistant prose only names the path', async ({ launchElectronApp }) => {
|
||||
const app = await launchElectronApp({ skipSetup: true });
|
||||
const prompt = 'Create the Markdown market report';
|
||||
const reply = 'Markdown 文件在这里：';
|
||||
|
||||
try {
|
||||
const fixture = await installAttachmentHostFixture(app, {
|
||||
sessions: [
|
||||
{ key: MAIN_SESSION_KEY, title: 'Main session' },
|
||||
{ key: OTHER_SESSION_KEY, title: 'Other session' },
|
||||
],
|
||||
});
|
||||
const reportPath = await fixture.createWorkspaceFile('market-report.md', '# Market report\n');
|
||||
await fixture.setSessionReplay(MAIN_SESSION_KEY, [
|
||||
userUpdate('structured-media-user', prompt),
|
||||
{
|
||||
sessionUpdate: 'agent_message',
|
||||
messageId: 'structured-media-reply',
|
||||
content: [{ type: 'text', text: `${reply}\n${reportPath}` }],
|
||||
},
|
||||
]);
|
||||
const transcript = [
|
||||
{ role: 'user', id: 'structured-transcript-user', content: prompt },
|
||||
{
|
||||
role: 'assistant',
|
||||
id: 'structured-transcript-assistant',
|
||||
content: `${reply}\n${reportPath}`,
|
||||
__openclaw: {
|
||||
media: [{
|
||||
path: reportPath,
|
||||
fileName: 'market-report.md',
|
||||
contentType: 'text/markdown',
|
||||
sizeBytes: 16,
|
||||
}],
|
||||
},
|
||||
},
|
||||
];
|
||||
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
|
||||
|
||||
const page = await openChat(app);
|
||||
await expect(page.getByText(reply)).toBeVisible({ timeout: 30_000 });
|
||||
await fixture.waitForHistoryRequestCount(MAIN_SESSION_KEY, 1);
|
||||
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [transcript]);
|
||||
await page.getByTestId(`sidebar-session-${OTHER_SESSION_KEY}`).click();
|
||||
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible();
|
||||
await page.getByTestId(`sidebar-session-${MAIN_SESSION_KEY}`).click();
|
||||
|
||||
const attachment = page.getByRole('button', { name: 'Preview market-report.md', exact: true });
|
||||
await expect(attachment).toBeEnabled();
|
||||
await expect.poll(async () => (await fixture.getHostInvocations()).some((call) => (
|
||||
call.module === 'files'
|
||||
&& call.action === 'resolveAttachment'
|
||||
&& call.payload?.name === 'market-report.md'
|
||||
&& call.payload?.mimeType === 'text/markdown'
|
||||
&& (call.payload?.ref as Record<string, unknown> | undefined)?.uri === reportPath
|
||||
&& (call.payload?.ref as Record<string, unknown> | undefined)?.transcriptMessageId
|
||||
=== 'structured-transcript-assistant'
|
||||
))).toBe(true);
|
||||
} finally {
|
||||
await closeElectronApp(app);
|
||||
}
|
||||
});
|
||||
|
||||
test('previews the reported live spreadsheet flow and restores one historical card', async ({ launchElectronApp }) => {
|
||||
const app = await launchElectronApp({ skipSetup: true });
|
||||
|
||||
|
||||
@@ -3148,6 +3148,67 @@ describe('ACP Chat store', () => {
|
||||
expect(JSON.stringify(transcriptTraces)).not.toContain('MEDIA:');
|
||||
});
|
||||
|
||||
it('projects canonical persisted OpenClaw media metadata through Main attachment resolution', async () => {
|
||||
const history = createDeferred<{ success: true; messages: Array<Record<string, unknown>> }>();
|
||||
hostApiMock.sessionsHistory.mockReturnValueOnce(history.promise);
|
||||
const { ensureAcpChatSubscriptions, useAcpChatSessionStore } = await importStore();
|
||||
ensureAcpChatSubscriptions();
|
||||
|
||||
await useAcpChatSessionStore.getState().loadSession({
|
||||
sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo',
|
||||
});
|
||||
hostEventsMock.updateListener?.({
|
||||
sessionKey: 'agent:pi:s1',
|
||||
generation: 1,
|
||||
historical: true,
|
||||
notification: {
|
||||
sessionId: 'agent:pi:s1',
|
||||
update: {
|
||||
sessionUpdate: 'user_message',
|
||||
messageId: 'structured-user',
|
||||
content: [{ type: 'text', text: 'Create report' }],
|
||||
},
|
||||
},
|
||||
});
|
||||
history.resolve({
|
||||
success: true,
|
||||
messages: [
|
||||
{ role: 'user', content: 'Create report' },
|
||||
{
|
||||
role: 'assistant',
|
||||
id: 'structured-assistant',
|
||||
content: 'Report ready at /repo/generated-report.md',
|
||||
__openclaw: {
|
||||
media: [{
|
||||
path: '/repo/generated-report.md',
|
||||
fileName: 'Generated report.md',
|
||||
contentType: 'text/markdown',
|
||||
sizeBytes: 512,
|
||||
}],
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await vi.waitFor(() => {
|
||||
expect(hostApiMock.resolveAttachment).toHaveBeenCalledWith({
|
||||
ref: {
|
||||
sessionKey: 'agent:pi:s1',
|
||||
generation: 1,
|
||||
uri: '/repo/generated-report.md',
|
||||
transcriptMessageId: 'structured-assistant',
|
||||
},
|
||||
name: 'Generated report.md',
|
||||
mimeType: 'text/markdown',
|
||||
size: 512,
|
||||
});
|
||||
});
|
||||
const attachments = Object.values(useAcpChatSessionStore.getState().timeline.itemsById)
|
||||
.flatMap((item) => item.kind === 'message-segment' ? item.parts : [])
|
||||
.filter((part) => part.kind === 'attachment' && part.source === 'openclaw-media');
|
||||
expect(attachments).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('recovers historical MEDIA when the ACP user turn contains a resource attachment', async () => {
|
||||
const history = createDeferred<{ success: true; messages: Array<Record<string, unknown>> }>();
|
||||
hostApiMock.sessionsHistory.mockReturnValueOnce(history.promise);
|
||||
|
||||
@@ -121,6 +121,74 @@ describe('OpenClaw MEDIA transcript extraction', () => {
|
||||
expect(turn?.candidates.every((candidate) => candidate.transcriptMessageId === 'assistant-files')).toBe(true);
|
||||
});
|
||||
|
||||
it('reads canonical persisted OpenClaw media facts without parsing ordinary prose paths', () => {
|
||||
const [turn] = extract(transcript(
|
||||
{ role: 'user', content: 'Create the report' },
|
||||
{
|
||||
role: 'assistant',
|
||||
id: 'assistant-structured-files',
|
||||
content: 'Markdown 文件在这里：\nC:\\Users\\Administrator\\.openclaw\\workspace\\report.md',
|
||||
__openclaw: {
|
||||
media: [
|
||||
{
|
||||
path: 'C:\\Users\\Administrator\\.openclaw\\workspace\\report.md',
|
||||
fileName: 'market-report.md',
|
||||
contentType: 'text/markdown',
|
||||
sizeBytes: 2048,
|
||||
},
|
||||
{
|
||||
url: 'https://example.test/report.pdf',
|
||||
fileName: 'report.pdf',
|
||||
contentType: 'application/pdf',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
));
|
||||
|
||||
expect(turn?.candidates).toMatchObject([
|
||||
{
|
||||
uri: 'C:\\Users\\Administrator\\.openclaw\\workspace\\report.md',
|
||||
name: 'market-report.md',
|
||||
mimeType: 'text/markdown',
|
||||
size: 2048,
|
||||
order: 0,
|
||||
transcriptMessageId: 'assistant-structured-files',
|
||||
},
|
||||
{
|
||||
uri: 'https://example.test/report.pdf',
|
||||
name: 'report.pdf',
|
||||
mimeType: 'application/pdf',
|
||||
order: 1,
|
||||
transcriptMessageId: 'assistant-structured-files',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('prefers canonical media facts over duplicate MEDIA directives', () => {
|
||||
const [turn] = extract(transcript(
|
||||
{ role: 'user', content: 'Create the report' },
|
||||
{
|
||||
role: 'assistant',
|
||||
id: 'assistant-deduped-file',
|
||||
content: 'MEDIA:/workspace/project/report.md',
|
||||
__openclaw: {
|
||||
media: [{
|
||||
path: '/workspace/project/report.md',
|
||||
fileName: 'Structured report.md',
|
||||
contentType: 'text/markdown',
|
||||
}],
|
||||
},
|
||||
},
|
||||
));
|
||||
|
||||
expect(turn?.candidates).toMatchObject([{
|
||||
uri: '/workspace/project/report.md',
|
||||
name: 'Structured report.md',
|
||||
mimeType: 'text/markdown',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('rejects fenced, wrapped, inline, unknown-scheme, malformed, and overlong references', () => {
|
||||
const tooLong = `/tmp/${'x'.repeat(4092)}`;
|
||||
const [turn] = extract(transcript(
|
||||
|
||||
Reference in New Issue
Block a user