admin/ClawX
1
0
Fork 0
mirror of https://github.com/ValueCell-ai/ClawX.git synced 2026-08-14 08:53:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: resolve a batch of problems after migrating to ACP (#1165)

Browse Source
This commit is contained in:
ZHUO Xu
2026-07-15 14:36:17 +08:00
committed by GitHub
parent c7d4345b6e
commit 702f03e055
103 changed files with 11248 additions and 735 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.ja-JP.md
+3 -1
View File
@@ -219,7 +219,9 @@ ClawXは、**デュアルプロセス + Host API 統一アクセス**構成を
Chat は Electron Main が所有する ACP stdio bridge を使用します。Renderer は型付き host event を受け取り、メモリ上の ACP timeline を描画します。Gateway は providers、models、skills、workspace、settings、diagnostics、media configuration などの非 Chat 機能を引き続き担当します。
ACP Chat は、runtime が画像生成メディアを信頼できる構造化メディアとして配信した場合に、生成画像のプレビューを表示できます。OpenClaw の履歴リプレイ中は、同じセッションで画像生成タスク開始が記録されている場合に限り、assistant の `MEDIA:/path/to/file.png` マーカーもプレビューへ昇格されます。`MEDIA: /path/to/file.png` のようなその他の単なるローカルパスのテキストはプレビューとして扱われません。ClawX は Renderer から任意にファイルシステムへアクセスするのではなく、Electron Main のホストメディア処理を通じてプレビューを読み込みます。標準 ACP 画像コンテンツは引き続き推奨パスであり、そのまま描画されます。
ACP Chat は標準 ACP resource を添付ファイルとして表示します。ユーザーが選択した画像は、ホバー時のオーバーレイにファイル名を表示するサムネイルとして描画され、その他のユーザー添付ファイルはファイル名に続いて、淡色で省略可能なソースパスを表示します。現在の OpenClaw ACP adapter が assistant のメディアを省略した場合も、明示的な assistant の `MEDIA:` ディレクティブを、元のディレクティブを表示せずに添付カードとして復元できます。対応する認可済みローカルファイルはアプリ内でプレビューされ、それ以外の認可済みローカルファイルはユーザーのクリック後にシステムアプリで開かれます。リモートの HTTP/HTTPS 添付ファイルはクリック後に外部で開かれます。通常の文章内にある単独またはインラインのパスは添付ファイルとして扱われません。
ACP Chat は、runtime が画像生成メディアを信頼できる構造化メディアとして配信した場合に、生成画像のプレビューも表示できます。信頼できる OpenClaw internal-UI 配信と画像生成タスクに関連付けられた最終返信では、テキストのみの失敗説明を含む元のユーザー向け完了テキストを保持し、汎用の画像キャプションへ置き換えません。OpenClaw の履歴リプレイ中は、同じセッションで画像生成タスク開始が記録されている場合に限り、assistant の画像 `MEDIA:` マーカーがインライン画像表示へ昇格されます。ClawX は Renderer から任意にファイルシステムへアクセスするのではなく、Electron Main のホストメディア処理を通じてプレビューを読み込みます。標準 ACP の画像と resource コンテンツは引き続き推奨パスであり、そのまま描画されます。
### ACP ファイルアクティビティのセマンティクス
README.md
+3 -1
View File
@@ -222,7 +222,9 @@ ClawX employs a **dual-process architecture** with a unified host API layer. The
Chat uses an ACP stdio bridge owned by Electron Main. Renderer receives typed host events and renders an in-memory ACP timeline. Gateway remains responsible for non-Chat capabilities such as providers, models, skills, workspace, settings, diagnostics, and media configuration.
ACP Chat can display generated image previews when image-generation media is delivered by the runtime as trusted structured media. During historical OpenClaw replay, assistant `MEDIA:/path/to/file.png` markers are also promoted only when they follow a recorded image-generation task start for that session. Other plain local-path text such as `MEDIA: /path/to/file.png` is not interpreted as a preview; ClawX loads previews through host media handling in Electron Main, not arbitrary renderer filesystem access. Standard ACP image content remains the preferred path and renders directly.
ACP Chat renders standard ACP resources as attachments. User-selected images appear as thumbnails with a filename hover overlay, while other user attachments show the filename and a muted, truncating source path. When the current OpenClaw ACP adapter omits assistant media, explicit assistant `MEDIA:` directives can also be recovered as attachment cards without displaying the raw directive. Supported authorized local files preview in-app; other authorized local files open in the system application after a user click; remote HTTP and HTTPS attachments open externally after a user click. Bare or inline prose paths are not treated as attachments.
ACP Chat can also display generated image previews when image-generation media is delivered by the runtime as trusted structured media. Trusted OpenClaw internal-UI deliveries and task-correlated final replies preserve the original user-facing completion text, including text-only failure explanations, rather than replacing it with a generic image caption. During historical OpenClaw replay, assistant image `MEDIA:` markers are promoted to the inline image experience only when they follow a recorded image-generation task start for that session. ClawX loads previews through host media handling in Electron Main, not arbitrary Renderer filesystem access. Standard ACP image and resource content remains the preferred path and renders directly.
### ACP File Activity Semantics
README.zh-CN.md
+3 -1
View File
@@ -223,7 +223,9 @@ ClawX 采用 **双进程 + Host API 统一接入架构**。渲染进程只调用
Chat 使用由 Electron Main 持有的 ACP stdio bridge。Renderer 接收类型化 host events，并渲染内存中的 ACP timeline。Gateway 仍负责 providers、models、skills、workspace、settings、diagnostics 和 media configuration 等非 Chat 能力。
ACP Chat 可在 runtime 以可信结构化媒体投递图像生成结果时显示生成图片预览。历史 OpenClaw 回放中，assistant 的 `MEDIA:/path/to/file.png` 标记只有在同一会话已记录图像生成任务启动后才会被提升为预览。其它纯文本本地路径（如 `MEDIA: /path/to/file.png`）不会被当作图片预览；ClawX 通过 Electron Main 的主机媒体处理加载预览，而不是让 Renderer 任意访问文件系统。标准 ACP 图片内容仍是首选路径，并会直接渲染。
ACP Chat 会将标准 ACP resource 渲染为附件。用户选择的图片会显示为缩略图，并在悬停蒙层中显示文件名；其它用户附件会显示文件名，以及灰色、可截断的来源路径。当前 OpenClaw ACP adapter 遗漏 assistant 媒体时，显式的 assistant `MEDIA:` 指令也可恢复为附件卡片，且不会显示原始指令。受支持且已授权的本地文件会在应用内预览；其它已授权本地文件会在用户点击后通过系统应用打开；远程 HTTP 和 HTTPS 附件会在用户点击后从外部打开。普通文本中的裸路径或行内路径不会被当作附件。
ACP Chat 也可在 runtime 以可信结构化媒体投递图像生成结果时显示生成图片预览。对于可信的 OpenClaw internal-UI 投递和与生图任务关联的最终回复，ClawX 会保留原始的用户可见完成文案，包括只有文本的失败说明，而不会统一替换成通用图片文案。历史 OpenClaw 回放中，assistant 的图片 `MEDIA:` 标记只有在同一会话已记录图像生成任务启动后才会进入内联图片体验。ClawX 通过 Electron Main 的主机媒体处理加载预览，而不是让 Renderer 任意访问文件系统。标准 ACP 图片和 resource 内容仍是首选路径，并会直接渲染。
### ACP 文件活动语义
electron/main/ipc-handlers.ts
+11 -3
View File
@@ -57,6 +57,8 @@ import { createSettingsApi } from '../services/settings-api';
import { createChannelsApi } from '../services/channels-api';
import { createAgentsApi } from '../services/agents-api';
import { createChatApi } from '../services/chat-api';
import { AcpSessionAccessRegistry } from '../services/acp-session-access-registry';
import { createAttachmentAccess, StagedAttachmentRegistry } from '../services/attachment-access';
import { createCronApi } from '../services/cron-api';
import { createFilesApi } from '../services/files-api';
import { createMediaApi } from '../services/media-api';
@@ -133,6 +135,12 @@ function registerTypedHostHandlers(
mainWindow: BrowserWindow,
hostApiRegistry: HostApiRegistry,
): void {
const acpSessionAccessRegistry = new AcpSessionAccessRegistry();
const stagedAttachments = new StagedAttachmentRegistry();
const attachmentAccess = createAttachmentAccess({
sessionAccessRegistry: acpSessionAccessRegistry,
stagedAttachments,
});
hostApiRegistry.registerCoreServices({
app: createAppApi(),
openclaw: createOpenClawApi(),
@@ -147,10 +155,10 @@ function registerTypedHostHandlers(
channels: createChannelsApi({ gatewayManager, mainWindow }),
agents: createAgentsApi({ gatewayManager }),
providers: createProvidersApi({ gatewayManager, mainWindow }),
files: createFilesApi(),
media: createMediaApi(),
files: createFilesApi({ attachmentAccess, stagedAttachments }),
media: createMediaApi({ attachmentAccess }),
sessions: createSessionsApi(),
chat: createChatApi({ gatewayManager, mainWindow }),
chat: createChatApi({ gatewayManager, mainWindow, acpSessionAccessRegistry }),
cron: createCronApi({ gatewayManager }),
skills: createSkillsApi({ clawHubService, gatewayManager }),
usage: createUsageApi(),
electron/services/acp-chat-service.ts
+148 -17
View File
@@ -29,6 +29,8 @@ import {
} from '../utils/control-ui-device-pairing';
import { logger } from '../utils/logger';
import { recordAcpTrace } from './acp-trace';
import { AcpSessionAccessRegistry, type AcpSessionAccessContext } from './acp-session-access-registry';
import { expandPath } from '../utils/paths';
type AcpConnection = Pick<ClientSideConnection, 'initialize' | 'newSession' | 'loadSession' | 'prompt' | 'cancel'>;
type MainWindowLike = {
@@ -38,14 +40,26 @@ type PermissionWaiter = {
sessionKey: string;
resolve: (response: RequestPermissionResponse) => void;
};
type AcpSessionLoadBatch = {
sessionKey: string;
generation: number;
sessionUpdates: Array<{
acpSessionId: string;
envelope: AcpSessionUpdateEnvelope;
}>;
};
type AcpChildProcess = ChildProcess & {
stdin: NonNullable<ChildProcess['stdin']>;
stdout: NonNullable<ChildProcess['stdout']>;
stderr: NonNullable<ChildProcess['stderr']>;
};
function ok(generation?: number): AcpChatOperationResult {
return { success: true, ...(generation != null ? { generation } : {}) };
function ok(generation?: number, sessionUpdates?: AcpSessionUpdateEnvelope[]): AcpChatOperationResult {
return {
success: true,
...(generation != null ? { generation } : {}),
...(sessionUpdates?.length ? { sessionUpdates } : {}),
};
}
function fail(error: unknown): AcpChatOperationResult {
@@ -117,12 +131,16 @@ export class AcpChatService {
private loadedAcpSessionId: string | null = null;
private historicalSessionKey: string | null = null;
private historicalGeneration: number | null = null;
private permissionsEnabled = false;
private loadQueue: Promise<void> | null = null;
private activeLoadBatch: AcpSessionLoadBatch | null = null;
private permissionSeq = 0;
private readonly permissionWaiters = new Map<string, PermissionWaiter>();
readonly client: Client;
constructor(
private readonly mainWindow: MainWindowLike,
private readonly accessRegistry: AcpSessionAccessRegistry,
injectedConnection?: AcpConnection,
private readonly gateway?: GatewayPairingRpcClient,
) {
@@ -155,8 +173,32 @@ export class AcpChatService {
}
}
async loadSession(payload: AcpChatLoadPayload): Promise<AcpChatOperationResult> {
if (!isValidSessionKey(payload.sessionKey) || !payload.cwd) return fail('Invalid ACP session load payload');
loadSession(payload: AcpChatLoadPayload): Promise<AcpChatOperationResult> {
const previousLoad = this.loadQueue;
let releaseLoad!: () => void;
const currentLoad = new Promise<void>((resolve) => {
releaseLoad = resolve;
});
this.loadQueue = currentLoad;
const run = async () => {
if (previousLoad) await previousLoad;
try {
return await this.performLoadSession(payload);
} finally {
releaseLoad();
if (this.loadQueue === currentLoad) this.loadQueue = null;
}
};
return run();
}
private async performLoadSession(payload: AcpChatLoadPayload): Promise<AcpChatOperationResult> {
if (!isValidSessionKey(payload.sessionKey) || !payload.workspaceRoot || !payload.cwd) {
return fail('Invalid ACP session load payload');
}
const previousPermissionsEnabled = this.permissionsEnabled;
this.permissionsEnabled = false;
this.trace('session/load:start', {
sessionKey: payload.sessionKey,
details: { createIfMissing: !!payload.createIfMissing, cwdPresent: Boolean(payload.cwd) },
@@ -171,6 +213,8 @@ export class AcpChatService {
let previousGeneration = this.generation;
let nextGeneration = previousGeneration + 1;
let stateAdvanced = false;
let loadBatch: AcpSessionLoadBatch | null = null;
let previousAccessGrant: AcpSessionAccessContext | null = null;
try {
const connection = await this.ensureConnection();
@@ -182,6 +226,13 @@ export class AcpChatService {
previousHistoricalGeneration = this.historicalGeneration;
previousGeneration = this.generation;
nextGeneration = previousGeneration + 1;
previousAccessGrant = this.accessRegistry.snapshot();
const preparedAccessGrant = await this.accessRegistry.prepareGrant({
sessionKey: payload.sessionKey,
generation: nextGeneration,
workspaceRoot: payload.workspaceRoot,
executionCwd: payload.cwd,
});
this.generation = nextGeneration;
this.activeSessionKey = payload.sessionKey;
@@ -190,15 +241,21 @@ export class AcpChatService {
this.loadedAcpSessionId = null;
this.historicalSessionKey = payload.createIfMissing ? null : payload.sessionKey;
this.historicalGeneration = payload.createIfMissing ? null : nextGeneration;
loadBatch = {
sessionKey: payload.sessionKey,
generation: nextGeneration,
sessionUpdates: [],
};
this.activeLoadBatch = loadBatch;
stateAdvanced = true;
if (previousSessionKey && previousSessionKey !== payload.sessionKey) {
if (previousSessionKey) {
this.resolvePermissionWaitersForSession(previousSessionKey, cancelledPermissionResponse());
}
let acpSessionId = payload.sessionKey;
if (payload.createIfMissing) {
const created = await connection.newSession({
cwd: payload.cwd,
cwd: preparedAccessGrant.executionCwd,
mcpServers: [],
_meta: { sessionKey: payload.sessionKey, prefixCwd: true },
});
@@ -206,24 +263,34 @@ export class AcpChatService {
} else {
await connection.loadSession({
sessionId: payload.sessionKey,
cwd: payload.cwd,
cwd: preparedAccessGrant.executionCwd,
mcpServers: [],
});
}
if (this.activeSessionKey === payload.sessionKey && this.generation === nextGeneration) {
this.activeAcpSessionId = acpSessionId;
this.loadedSessionKey = payload.sessionKey;
this.loadedAcpSessionId = acpSessionId;
}
this.activeAcpSessionId = acpSessionId;
this.loadedSessionKey = payload.sessionKey;
this.loadedAcpSessionId = acpSessionId;
this.accessRegistry.commitGrant(preparedAccessGrant);
this.trace('session/load:success', {
sessionKey: payload.sessionKey,
generation: nextGeneration,
details: { createIfMissing: !!payload.createIfMissing, acpSessionId },
});
return ok(nextGeneration);
if (this.activeLoadBatch === loadBatch) this.activeLoadBatch = null;
return ok(
nextGeneration,
loadBatch.sessionUpdates
.filter((entry) => entry.acpSessionId === acpSessionId)
.map((entry) => entry.envelope),
);
} catch (error) {
if (this.activeLoadBatch === loadBatch) this.activeLoadBatch = null;
this.resolvePermissionWaitersForSession(payload.sessionKey, cancelledPermissionResponse());
if (stateAdvanced && this.activeSessionKey === payload.sessionKey && this.generation === nextGeneration) {
if (
stateAdvanced
&& this.activeSessionKey === payload.sessionKey
&& this.generation === nextGeneration
) {
this.generation = previousGeneration;
this.activeSessionKey = previousSessionKey;
this.activeAcpSessionId = previousAcpSessionId;
@@ -231,6 +298,8 @@ export class AcpChatService {
this.loadedAcpSessionId = previousLoadedAcpSessionId;
this.historicalSessionKey = previousHistoricalSessionKey;
this.historicalGeneration = previousHistoricalGeneration;
this.permissionsEnabled = previousPermissionsEnabled;
this.accessRegistry.restore(previousAccessGrant);
}
logger.error(`[acp-chat] loadSession failed: ${String(error)}`);
this.trace('session/load:failed', {
@@ -247,6 +316,16 @@ export class AcpChatService {
if (!this.activeSessionKey) return fail('No active ACP session');
if (payload.sessionKey !== this.activeSessionKey) return fail('ACP prompt session is not active');
if (this.loadedSessionKey !== payload.sessionKey || !this.loadedAcpSessionId) return fail('ACP session is not loaded');
const accessGrant = this.accessRegistry.get(payload.sessionKey, this.generation);
if (!accessGrant) return fail('ACP session access grant is not active');
const promptCwd = payload.cwd === accessGrant.executionCwd
? payload.cwd
: await import('node:fs/promises')
.then((fsP) => fsP.realpath(expandPath(payload.cwd)))
.catch(() => null);
if (promptCwd !== accessGrant.executionCwd) {
return fail('ACP prompt cwd does not match the registered execution cwd');
}
try {
this.trace('session/prompt:start', {
@@ -259,6 +338,7 @@ export class AcpChatService {
this.historicalSessionKey = null;
this.historicalGeneration = null;
}
this.permissionsEnabled = true;
await connection.prompt({
sessionId: this.loadedAcpSessionId,
prompt,
@@ -271,6 +351,7 @@ export class AcpChatService {
});
return ok(this.generation);
} catch (error) {
this.permissionsEnabled = false;
logger.error(`[acp-chat] prompt failed: ${String(error)}`);
this.trace('session/prompt:failed', {
sessionKey: payload.sessionKey,
@@ -288,6 +369,7 @@ export class AcpChatService {
this.trace('session/cancel:start', { sessionKey: payload.sessionKey });
const connection = await this.ensureConnection();
await connection.cancel({ sessionId: this.loadedAcpSessionId });
this.permissionsEnabled = false;
this.resolvePermissionWaitersForSession(payload.sessionKey, cancelledPermissionResponse());
this.trace('session/cancel:success', { sessionKey: payload.sessionKey });
return ok(this.generation);
@@ -444,6 +526,7 @@ export class AcpChatService {
this.loadedAcpSessionId = null;
this.historicalSessionKey = null;
this.historicalGeneration = null;
this.permissionsEnabled = false;
}
private emitSessionUpdate(notification: SessionNotification): void {
@@ -480,6 +563,16 @@ export class AcpChatService {
: {}),
notification: { ...notification, sessionId: sessionKey },
};
const loadBatch = this.activeLoadBatch;
if (loadBatch?.sessionKey === sessionKey && loadBatch.generation === this.generation) {
loadBatch.sessionUpdates.push({ acpSessionId, envelope });
this.trace('session-update:buffered', {
direction: 'downstream',
sessionKey,
details: { acpSessionId, updateType, historical: !!envelope.historical },
});
return;
}
this.mainWindow.webContents.send(HOST_EVENT_CHANNELS.chat.acpSessionUpdate, envelope);
this.trace('session-update:forwarded', {
direction: 'downstream',
@@ -491,6 +584,22 @@ export class AcpChatService {
private requestPermission(request: RequestPermissionRequest): Promise<RequestPermissionResponse> {
const acpSessionId = request.sessionId;
const sessionKey = this.activeSessionKey;
if (!this.permissionsEnabled) {
this.trace('permission:ignored', {
direction: 'upstream',
sessionKey: sessionKey ?? null,
details: { reason: 'no-active-prompt', acpSessionId },
});
return Promise.resolve(cancelledPermissionResponse());
}
if (this.activeLoadBatch) {
this.trace('permission:ignored', {
direction: 'upstream',
sessionKey: sessionKey ?? null,
details: { reason: 'session-loading', acpSessionId },
});
return Promise.resolve(cancelledPermissionResponse());
}
if (!sessionKey || (this.activeAcpSessionId && acpSessionId !== this.activeAcpSessionId)) {
this.trace('permission:ignored', {
direction: 'upstream',
@@ -550,9 +659,30 @@ export class AcpChatService {
const mimeType = item.mimeType || 'application/octet-stream';
if (mimeType.startsWith('image/')) {
const data = await fsP.readFile(item.filePath, 'base64');
blocks.push({ type: 'image', data, mimeType, uri: item.filePath });
blocks.push({
type: 'image',
data,
mimeType,
uri: item.filePath,
_meta: {
clawx: {
stagingId: item.stagingId,
...(item.fileName ? { fileName: item.fileName } : {}),
},
},
});
} else {
blocks.push({ type: 'resource_link', uri: item.filePath, name: item.fileName ?? item.filePath });
blocks.push({
type: 'resource_link',
uri: item.filePath,
name: item.fileName ?? item.filePath,
mimeType: item.mimeType,
_meta: {
clawx: {
stagingId: item.stagingId,
},
},
});
}
}
}
@@ -564,7 +694,8 @@ export class AcpChatService {
export function createAcpChatService(
mainWindow: MainWindowLike,
accessRegistry: AcpSessionAccessRegistry,
gateway?: GatewayPairingRpcClient,
): AcpChatService {
return new AcpChatService(mainWindow, undefined, gateway);
return new AcpChatService(mainWindow, accessRegistry, undefined, gateway);
}
electron/services/acp-session-access-registry.ts
+53
View File
@@ -0,0 +1,53 @@
import { realpath, stat } from 'node:fs/promises';
import { isAbsolute, relative, sep } from 'node:path';
import { expandPath } from '../utils/paths';
export type AcpSessionAccessContext = {
sessionKey: string;
generation: number;
workspaceRoot: string;
executionCwd: string;
};
async function canonicalDirectory(input: string, label: string): Promise<string> {
const canonicalPath = await realpath(expandPath(input));
const directoryStat = await stat(canonicalPath);
if (!directoryStat.isDirectory()) throw new Error(`${label} must be a directory`);
return canonicalPath;
}
function isInside(child: string, parent: string): boolean {
const relativePath = relative(parent, child);
return relativePath === ''
|| (!isAbsolute(relativePath) && relativePath !== '..' && !relativePath.startsWith(`..${sep}`));
}
export class AcpSessionAccessRegistry {
private activeGrant: AcpSessionAccessContext | null = null;
async prepareGrant(input: AcpSessionAccessContext): Promise<AcpSessionAccessContext> {
const workspaceRoot = await canonicalDirectory(input.workspaceRoot, 'ACP workspace root');
const executionCwd = await canonicalDirectory(input.executionCwd, 'ACP execution cwd');
if (!isInside(executionCwd, workspaceRoot)) {
throw new Error('ACP execution cwd must be inside the workspace root');
}
return { ...input, workspaceRoot, executionCwd };
}
snapshot(): AcpSessionAccessContext | null {
return this.activeGrant ? { ...this.activeGrant } : null;
}
commitGrant(context: AcpSessionAccessContext): void {
this.activeGrant = { ...context };
}
restore(snapshot: AcpSessionAccessContext | null): void {
this.activeGrant = snapshot ? { ...snapshot } : null;
}
get(sessionKey: string, generation: number): AcpSessionAccessContext | null {
if (this.activeGrant?.sessionKey !== sessionKey || this.activeGrant.generation !== generation) return null;
return { ...this.activeGrant };
}
}
electron/services/acp-trace.ts
+48 -2
View File
@@ -1,4 +1,9 @@
import type { AcpTraceEntry, AcpTraceRecordPayload, AcpTraceSnapshot } from '@shared/host-api/contract';
import type {
AcpTraceEntry,
AcpTraceRecordPayload,
AcpTraceSnapshot,
AttachmentAccessError,
} from '@shared/host-api/contract';
import { isRecord } from './payload-utils';
type AcpTraceRecordInput = Omit<AcpTraceEntry, 'seq' | 'timestamp'>;
@@ -6,6 +11,22 @@ type AcpTraceRecordInput = Omit<AcpTraceEntry, 'seq' | 'timestamp'>;
const MAX_ACP_TRACE_ENTRIES = 500;
const MAX_STRING_LENGTH = 300;
const SENSITIVE_KEY_RE = /(authorization|api[_-]?key|token|secret|password|bearer)/i;
const OPENCLAW_MEDIA_DETAIL_KEYS = new Set([
'source',
'reason',
'candidateCount',
'matchedCount',
'rejectedCount',
'attachmentCount',
'imageCount',
'missingCount',
'previewCount',
'evidenceHash',
'identityHash',
'operationId',
'latestGeneration',
'error',
]);
let sequence = 0;
let entries: AcpTraceEntry[] = [];
@@ -68,13 +89,16 @@ export function normalizeRendererAcpTracePayload(payload: unknown): AcpTraceReco
const generation = typeof payload.generation === 'number' && Number.isFinite(payload.generation)
? payload.generation
: undefined;
const details = event.startsWith('openclaw-media:') && isRecord(payload.details)
? Object.fromEntries(Object.entries(payload.details).filter(([key]) => OPENCLAW_MEDIA_DETAIL_KEYS.has(key)))
: payload.details;
return {
source: 'renderer',
event,
direction,
...(sessionKey ? { sessionKey } : {}),
...(generation != null ? { generation } : {}),
...(payload.details !== undefined ? { details: payload.details } : {}),
...(details !== undefined ? { details } : {}),
};
}
@@ -85,6 +109,28 @@ export function recordRendererAcpTrace(payload: AcpTraceRecordPayload): { succes
return { success: true };
}
export function recordAttachmentOpenTrace(input: {
ok: boolean;
reason: AttachmentAccessError | 'success';
sourceKind: 'local' | 'remote' | 'invalid';
sessionKey: string;
generation: number;
identity: string;
}): AcpTraceEntry {
return recordAcpTrace({
source: 'main',
event: `attachment/open:${input.ok ? 'success' : 'failure'}`,
direction: 'open',
sessionKey: input.sessionKey,
generation: input.generation,
details: {
reason: input.reason,
sourceKind: input.sourceKind,
identity: input.identity.slice(0, 64),
},
});
}
export function clearAcpTraceForTests(): void {
sequence = 0;
entries = [];
electron/services/attachment-access.ts
+903
View File
@@ -0,0 +1,903 @@
import { shell as electronShell } from 'electron';
import { createHash } from 'node:crypto';
import { constants } from 'node:fs';
import type { Stats } from 'node:fs';
import type { FileHandle } from 'node:fs/promises';
import {
basename,
dirname,
extname,
isAbsolute,
join,
posix,
relative,
resolve,
sep,
win32,
} from 'node:path';
import { fileURLToPath } from 'node:url';
import type {
AttachmentAccessError,
AttachmentFileRef,
AttachmentSourceRef,
OpenAttachmentResult,
ReadAttachmentBinaryPayload,
ReadAttachmentBinaryResult,
ReadAttachmentTextResult,
ResolveAttachmentPayload,
ResolveAttachmentResult,
} from '@shared/host-api/contract';
import {
FILE_PREVIEW_MAX_BINARY_BYTES,
FILE_PREVIEW_MAX_TEXT_BYTES,
} from '@shared/file-preview/limits';
import type { AcpSessionAccessRegistry } from './acp-session-access-registry';
import { recordAttachmentOpenTrace } from './acp-trace';
import {
expandPath,
resolveOpenClawConfigDir,
resolveOpenClawStateDir,
} from '../utils/paths';
const MAX_REFERENCE_LENGTH = 4096;
const MAX_DISPLAY_NAME_LENGTH = 160;
const MAX_OUTGOING_RECORD_BYTES = 64 * 1024;
const SAFE_ATTACHMENT_ID = /^[A-Za-z0-9._-]+$/;
const EXT_MIME_MAP: Record<string, string> = {
'.bmp': 'image/bmp',
'.csv': 'text/csv',
'.gif': 'image/gif',
'.htm': 'text/html',
'.html': 'text/html',
'.jpeg': 'image/jpeg',
'.jpg': 'image/jpeg',
'.json': 'application/json',
'.md': 'text/markdown',
'.pdf': 'application/pdf',
'.png': 'image/png',
'.svg': 'image/svg+xml',
'.txt': 'text/plain',
'.webp': 'image/webp',
'.xls': 'application/vnd.ms-excel',
'.xlsx': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
'.zip': 'application/zip',
};
type AttachmentFs = {
lstat: (path: string) => Promise<Stats>;
open: (path: string, flags: number) => Promise<FileHandle>;
realpath: (path: string) => Promise<string>;
stat: (path: string) => Promise<Stats>;
};
type AttachmentShell = {
openPath: (path: string) => Promise<string>;
openExternal: (url: string) => Promise<void>;
};
export type AttachmentAccess = {
resolveAttachment: (payload: ResolveAttachmentPayload) => Promise<ResolveAttachmentResult>;
readAttachmentText: (ref: AttachmentFileRef) => Promise<ReadAttachmentTextResult>;
readAttachmentBinary: (payload: ReadAttachmentBinaryPayload) => Promise<ReadAttachmentBinaryResult>;
openAttachment: (ref: AttachmentSourceRef) => Promise<OpenAttachmentResult>;
};
type AttachmentAccessDependencies = {
sessionAccessRegistry: AcpSessionAccessRegistry;
stagedAttachments: StagedAttachmentRegistry;
stateDir?: string;
configDir?: string;
fs?: AttachmentFs;
shell?: AttachmentShell;
};
type LocalScope = 'workspace' | 'openclaw-media' | 'staging';
type ResolvedLocal = {
kind: 'local';
canonicalPath: string;
scope: LocalScope;
mimeType: string;
size: number;
authorizationRoot?: string;
};
type ResolvedRemote = {
kind: 'remote';
normalizedUrl: string;
mimeType: string;
size: number;
};
type ResolvedTarget = ResolvedLocal | ResolvedRemote;
type PinnedDirectory = {
canonicalPath: string;
dev: number;
ino: number;
};
type ManagedAuthoritySlot = {
lexicalParent: string;
parent?: PinnedDirectory;
media?: PinnedDirectory;
pinning?: Promise<void>;
mediaPinning?: Promise<void>;
};
class AttachmentFailure extends Error {
constructor(readonly code: AttachmentAccessError) {
super(code);
}
}
export class StagedAttachmentRegistry {
private readonly files = new Map<string, { canonicalPath: string; displayPath?: string }>();
register(id: string, canonicalPath: string, displayPath?: string): void {
if (id && canonicalPath) this.files.set(id, {
canonicalPath,
...(displayPath ? { displayPath } : {}),
});
}
get(id: string): string | null {
return this.files.get(id)?.canonicalPath ?? null;
}
getDisplayPath(id: string): string | null {
return this.files.get(id)?.displayPath ?? null;
}
hasPath(canonicalPath: string): boolean {
return Array.from(this.files.values()).some((file) => isSamePath(file.canonicalPath, canonicalPath));
}
}
export function resolveClawXStagingDir(stateDir = resolveOpenClawStateDir()): string {
return join(resolve(stateDir), 'media', 'outbound', 'clawx-staging');
}
function attachmentFailure(error: unknown): AttachmentAccessError {
if (error instanceof AttachmentFailure) return error.code;
const code = error && typeof error === 'object' && 'code' in error ? error.code : undefined;
if (code === 'ENOENT') return 'unavailable';
if (code === 'ELOOP') return 'outsideAllowedRoots';
return 'operationFailed';
}
function opaqueIdentity(value: string): string {
return createHash('sha256').update(value).digest('hex');
}
function safeDisplayName(value: unknown, fallback: string): string {
const raw = typeof value === 'string' && value.trim() ? value : fallback;
const filename = posix.basename(raw.replace(/\\/gu, '/'));
const withoutControls = Array.from(filename.slice(0, MAX_DISPLAY_NAME_LENGTH * 4), (character) => {
const codePoint = character.codePointAt(0) ?? 0;
const isControl = codePoint <= 0x1f || (codePoint >= 0x7f && codePoint <= 0x9f);
const isBidiFormatting = codePoint === 0x061c
|| codePoint === 0x200e
|| codePoint === 0x200f
|| (codePoint >= 0x202a && codePoint <= 0x202e)
|| (codePoint >= 0x2066 && codePoint <= 0x2069);
return isControl || isBidiFormatting ? ' ' : character;
}).join('');
const cleaned = withoutControls
.replace(/\s+/gu, ' ')
.trim()
.slice(0, MAX_DISPLAY_NAME_LENGTH);
return cleaned || 'attachment';
}
function decodedBasename(uri: string): string {
try {
if (/^https?:/i.test(uri)) {
const url = new URL(uri);
return decodeURIComponent(posix.basename(url.pathname)) || url.hostname;
}
if (/^file:/i.test(uri)) return basename(fileURLToPath(uri));
} catch {
// A safe generic label is returned below for malformed input.
}
return basename(uri.replace(/[?#].*$/u, '')) || 'attachment';
}
function mimeTypeForPath(path: string): string {
return EXT_MIME_MAP[extname(path).toLowerCase()] ?? 'application/octet-stream';
}
function hasTraversal(value: string): boolean {
return value.split(/[\\/]+/u).includes('..');
}
function validateReferenceSyntax(uri: unknown): asserts uri is string {
if (typeof uri !== 'string' || !uri.trim() || uri.length > MAX_REFERENCE_LENGTH || uri.includes('\0')) {
throw new AttachmentFailure('invalidReference');
}
if (uri.startsWith('\\\\') || uri.startsWith('//')) throw new AttachmentFailure('invalidReference');
if (hasTraversal(uri)) throw new AttachmentFailure('invalidReference');
if (uri.includes('%')) {
let decoded: string;
try {
decoded = decodeURIComponent(uri);
} catch {
throw new AttachmentFailure('invalidReference');
}
if (decoded.includes('\0') || hasTraversal(decoded)) throw new AttachmentFailure('invalidReference');
}
}
function isInside(child: string, parent: string): boolean {
const relativePath = relative(parent, child);
return relativePath === ''
|| (!isAbsolute(relativePath) && relativePath !== '..' && !relativePath.startsWith(`..${sep}`));
}
function isSamePath(left: string, right: string): boolean {
const resolvedLeft = resolve(left);
const resolvedRight = resolve(right);
return process.platform === 'win32'
? resolvedLeft.toLowerCase() === resolvedRight.toLowerCase()
: resolvedLeft === resolvedRight;
}
function localPathFromUri(uri: string, executionCwd: string): string {
if (/^file:/i.test(uri)) {
let url: URL;
try {
url = new URL(uri);
} catch {
throw new AttachmentFailure('invalidReference');
}
if (url.username || url.password || (url.hostname && url.hostname.toLowerCase() !== 'localhost')) {
throw new AttachmentFailure('invalidReference');
}
try {
return fileURLToPath(url);
} catch {
throw new AttachmentFailure('invalidReference');
}
}
if (/^[A-Za-z][A-Za-z0-9+.-]*:/u.test(uri) && !win32.isAbsolute(uri)) {
throw new AttachmentFailure('invalidReference');
}
if (uri.startsWith('~')) return resolve(expandPath(uri));
if (isAbsolute(uri) || win32.isAbsolute(uri)) return resolve(uri);
return resolve(executionCwd, uri);
}
function parseOutgoingUrl(uri: string): { attachmentId: string; sessionKey: string } | null {
let url: URL;
try {
url = uri.startsWith('/') ? new URL(uri, 'http://clawx.local') : new URL(uri);
} catch {
return null;
}
const isRelativeGatewayUrl = uri.startsWith('/');
const isLocalGatewayUrl = (url.protocol === 'http:' || url.protocol === 'https:')
&& (url.hostname === 'localhost' || url.hostname === '127.0.0.1' || url.hostname === '[::1]');
if (!isRelativeGatewayUrl && !isLocalGatewayUrl) return null;
if (url.username || url.password) throw new AttachmentFailure('unsafeUrl');
const segments = url.pathname.split('/');
if (segments.length !== 8
|| segments[1] !== 'api'
|| segments[2] !== 'chat'
|| segments[3] !== 'media'
|| segments[4] !== 'outgoing'
|| segments[7] !== 'full') {
return null;
}
let sessionKey: string;
let attachmentId: string;
try {
sessionKey = decodeURIComponent(segments[5]);
attachmentId = decodeURIComponent(segments[6]);
} catch {
throw new AttachmentFailure('invalidReference');
}
if (!sessionKey || !SAFE_ATTACHMENT_ID.test(attachmentId)) {
throw new AttachmentFailure('invalidReference');
}
return { attachmentId, sessionKey };
}
async function canonicalRoots(paths: string[], fs: AttachmentFs): Promise<string[]> {
const roots = await Promise.all(paths.map(async (path) => {
try {
const canonical = await fs.realpath(path);
return (await fs.stat(canonical)).isDirectory() ? canonical : null;
} catch {
return null;
}
}));
return Array.from(new Set(roots.filter((root): root is string => root !== null)));
}
async function canonicalManagedMediaRoots(
stateDir: string,
configDir: string,
fs: AttachmentFs,
): Promise<string[]> {
// OpenClaw 2026.6.10 exposes only resolveStateDir()/media and resolveConfigDir()/media.
// Keep this list exact until the distributed runtime adds a real media-root setting.
const managedRoots = await Promise.all([stateDir, configDir].map(async (parentPath) => {
try {
const parent = await fs.realpath(parentPath);
if (!(await fs.stat(parent)).isDirectory()) return null;
const mediaPath = join(parentPath, 'media');
if ((await fs.lstat(mediaPath)).isSymbolicLink()) return null;
const media = await fs.realpath(mediaPath);
return (await fs.stat(media)).isDirectory() && isInside(media, parent) ? media : null;
} catch {
return null;
}
}));
return Array.from(new Set([
...managedRoots.filter((root): root is string => root !== null),
]));
}
function pinnedDirectory(path: string, stat: Stats): PinnedDirectory {
return { canonicalPath: path, dev: stat.dev, ino: stat.ino };
}
async function verifyPinnedDirectory(
lexicalPath: string,
pinned: PinnedDirectory,
fs: AttachmentFs,
rejectSymlink: boolean,
): Promise<void> {
if (rejectSymlink && (await fs.lstat(lexicalPath)).isSymbolicLink()) {
throw new AttachmentFailure('outsideAllowedRoots');
}
const currentPath = await fs.realpath(lexicalPath);
const currentStat = await fs.stat(currentPath);
if (!currentStat.isDirectory()
|| !isSamePath(currentPath, pinned.canonicalPath)
|| currentStat.dev !== pinned.dev
|| currentStat.ino !== pinned.ino) {
throw new AttachmentFailure('outsideAllowedRoots');
}
}
async function ensureManagedAuthority(
slot: ManagedAuthoritySlot,
fs: AttachmentFs,
): Promise<{ parent: PinnedDirectory; media: PinnedDirectory | null } | null> {
if (!slot.parent) {
if (!slot.pinning) {
slot.pinning = (async () => {
try {
const parentPath = await fs.realpath(slot.lexicalParent);
const parentStat = await fs.stat(parentPath);
if (!parentStat.isDirectory()) return;
slot.parent = pinnedDirectory(parentPath, parentStat);
} catch {
// A configured parent that does not exist yet is retried on a later operation.
}
})().finally(() => {
slot.pinning = undefined;
});
}
await slot.pinning;
}
if (!slot.parent) return null;
await verifyPinnedDirectory(slot.lexicalParent, slot.parent, fs, false);
const mediaPath = join(slot.lexicalParent, 'media');
if (!slot.media) {
if (!slot.mediaPinning) {
slot.mediaPinning = (async () => {
if ((await fs.lstat(mediaPath)).isSymbolicLink()) {
throw new AttachmentFailure('outsideAllowedRoots');
}
const canonicalMedia = await fs.realpath(mediaPath);
const mediaStat = await fs.stat(canonicalMedia);
if (!mediaStat.isDirectory() || !isInside(canonicalMedia, slot.parent!.canonicalPath)) {
throw new AttachmentFailure('outsideAllowedRoots');
}
slot.media = pinnedDirectory(canonicalMedia, mediaStat);
})().finally(() => {
slot.mediaPinning = undefined;
});
}
try {
await slot.mediaPinning;
} catch (error) {
if (error instanceof AttachmentFailure) throw error;
return { parent: slot.parent, media: null };
}
}
const media = slot.media;
if (!media) return { parent: slot.parent, media: null };
await verifyPinnedDirectory(mediaPath, media, fs, true);
return { parent: slot.parent, media };
}
async function frozenCanonicalDirectory(path: string, fs: AttachmentFs): Promise<string> {
try {
if ((await fs.lstat(path)).isSymbolicLink()) throw new AttachmentFailure('outsideAllowedRoots');
const current = await fs.realpath(path);
if (!isSamePath(current, path) || !(await fs.stat(current)).isDirectory()) {
throw new AttachmentFailure('outsideAllowedRoots');
}
return current;
} catch (error) {
if (error instanceof AttachmentFailure) throw error;
throw new AttachmentFailure('outsideAllowedRoots');
}
}
async function canonicalCandidateForContainment(candidate: string, fs: AttachmentFs): Promise<string> {
try {
return await fs.realpath(candidate);
} catch (error) {
const code = error && typeof error === 'object' && 'code' in error ? error.code : undefined;
if (code !== 'ENOENT') throw error;
}
let parent = dirname(candidate);
while (true) {
try {
const canonicalParent = await fs.realpath(parent);
return resolve(canonicalParent, relative(parent, candidate));
} catch (error) {
const code = error && typeof error === 'object' && 'code' in error ? error.code : undefined;
if (code !== 'ENOENT') throw error;
const nextParent = dirname(parent);
if (nextParent === parent) throw error;
parent = nextParent;
}
}
}
async function readOpenedFile(handle: FileHandle, maxBytes: number): Promise<Buffer | null> {
const chunks: Buffer[] = [];
let total = 0;
while (total <= maxBytes) {
const length = Math.min(64 * 1024, maxBytes + 1 - total);
const chunk = Buffer.allocUnsafe(length);
const { bytesRead } = await handle.read(chunk, 0, length, total);
if (bytesRead === 0) break;
chunks.push(chunk.subarray(0, bytesRead));
total += bytesRead;
}
return total > maxBytes ? null : Buffer.concat(chunks, total);
}
function looksLikeBinary(buffer: Buffer): boolean {
const length = Math.min(buffer.length, 8192);
for (let index = 0; index < length; index += 1) {
if (buffer[index] === 0) return true;
}
return false;
}
async function openRevalidatedLocal(local: ResolvedLocal, fs: AttachmentFs): Promise<{
handle: FileHandle;
stat: Stats;
}> {
let handle: FileHandle | undefined;
try {
if (local.authorizationRoot) {
const currentRoot = await fs.realpath(local.authorizationRoot);
if (!isSamePath(currentRoot, local.authorizationRoot)
|| !(await fs.stat(currentRoot)).isDirectory()) {
throw new AttachmentFailure('outsideAllowedRoots');
}
}
const noFollow = process.platform === 'win32' ? 0 : constants.O_NOFOLLOW;
handle = await fs.open(local.canonicalPath, constants.O_RDONLY | noFollow);
const handleStat = await handle.stat();
const canonicalPath = await fs.realpath(local.canonicalPath);
const pathStat = await fs.stat(canonicalPath);
if (!isSamePath(canonicalPath, local.canonicalPath)
|| (local.authorizationRoot && !isInside(canonicalPath, local.authorizationRoot))
|| handleStat.dev !== pathStat.dev
|| handleStat.ino !== pathStat.ino
|| !handleStat.isFile()) {
throw new AttachmentFailure('outsideAllowedRoots');
}
return { handle, stat: handleStat };
} catch (error) {
await handle?.close().catch(() => undefined);
throw error;
}
}
function normalizeRemote(uri: string): string {
let url: URL;
try {
url = new URL(uri);
} catch {
throw new AttachmentFailure('unsafeUrl');
}
if ((url.protocol !== 'http:' && url.protocol !== 'https:') || !url.hostname || url.username || url.password) {
throw new AttachmentFailure('unsafeUrl');
}
return url.href;
}
function boundedBinaryCap(value: unknown): number {
const requested = typeof value === 'number' && Number.isFinite(value) ? value : FILE_PREVIEW_MAX_BINARY_BYTES;
return Math.max(1, Math.min(requested, FILE_PREVIEW_MAX_BINARY_BYTES));
}
export async function resolveOutgoingMediaAttachment(input: {
uri: string;
expectedSessionKey?: string;
transcriptMessageId?: string;
stateDir?: string;
configDir?: string;
managedMediaRoots?: string[];
fs?: AttachmentFs;
}): Promise<{ path: string; mimeType: string; size: number; authorizationRoot: string } | null> {
try {
validateReferenceSyntax(input.uri);
const outgoing = parseOutgoingUrl(input.uri);
if (!outgoing || (input.expectedSessionKey && outgoing.sessionKey !== input.expectedSessionKey)) return null;
const fs = input.fs ?? await import('node:fs/promises');
const stateDir = resolve(input.stateDir ?? resolveOpenClawStateDir());
const configDir = resolve(input.configDir ?? resolveOpenClawConfigDir());
const recordPath = join(stateDir, 'media', 'outgoing', 'records', `${outgoing.attachmentId}.json`);
let handle: FileHandle | undefined;
let raw: Buffer | null;
try {
const noFollow = process.platform === 'win32' ? 0 : constants.O_NOFOLLOW;
handle = await fs.open(recordPath, constants.O_RDONLY | noFollow);
if (!(await handle.stat()).isFile()) return null;
raw = await readOpenedFile(handle, MAX_OUTGOING_RECORD_BYTES);
} finally {
await handle?.close().catch(() => undefined);
}
if (!raw) return null;
const record = JSON.parse(raw.toString('utf8')) as Record<string, unknown>;
const original = record.original && typeof record.original === 'object'
? record.original as Record<string, unknown>
: null;
const recordId = typeof record.attachmentId === 'string' ? record.attachmentId : undefined;
if (recordId !== outgoing.attachmentId
|| record.sessionKey !== outgoing.sessionKey
|| (input.transcriptMessageId && typeof record.messageId === 'string'
&& record.messageId !== input.transcriptMessageId)
|| !original
|| typeof original.path !== 'string') {
return null;
}
validateReferenceSyntax(original.path);
const originalPath = localPathFromUri(original.path, stateDir);
const roots = input.managedMediaRoots ?? await canonicalManagedMediaRoots(stateDir, configDir, fs);
const canonicalPath = await fs.realpath(originalPath);
const authorizationRoot = roots.find((root) => isInside(canonicalPath, root));
if (!authorizationRoot) return null;
const fileStat = await fs.stat(canonicalPath);
if (!fileStat.isFile()) return null;
return {
path: canonicalPath,
mimeType: typeof original.contentType === 'string' && original.contentType
? original.contentType
: mimeTypeForPath(canonicalPath),
size: fileStat.size,
authorizationRoot,
};
} catch {
return null;
}
}
export function createAttachmentAccess(dependencies: AttachmentAccessDependencies): AttachmentAccess {
const stateDir = resolve(dependencies.stateDir ?? resolveOpenClawStateDir());
const configDir = resolve(dependencies.configDir ?? resolveOpenClawConfigDir());
const shell = dependencies.shell ?? electronShell;
const getFs = async (): Promise<AttachmentFs> => dependencies.fs ?? await import('node:fs/promises');
const stateAuthority: ManagedAuthoritySlot = { lexicalParent: stateDir };
const configAuthority: ManagedAuthoritySlot = { lexicalParent: configDir };
const initializeAuthorities = async () => {
const fs = await getFs();
await Promise.all([
ensureManagedAuthority(stateAuthority, fs),
ensureManagedAuthority(configAuthority, fs),
]);
};
void initializeAuthorities().catch(() => undefined);
const verifyManagedAuthorities = async (fs: AttachmentFs) => {
const [state, config] = await Promise.all([
ensureManagedAuthority(stateAuthority, fs),
ensureManagedAuthority(configAuthority, fs),
]);
return {
stateParent: state?.parent ?? null,
mediaRoots: Array.from(new Set([
...(state?.media ? [state.media.canonicalPath] : []),
...(config?.media ? [config.media.canonicalPath] : []),
])),
};
};
const resolveLocalCandidate = async (
ref: AttachmentSourceRef,
candidateInput: string,
mimeTypeHint?: string,
mediaOnly = false,
): Promise<ResolvedLocal> => {
const context = dependencies.sessionAccessRegistry.get(ref.sessionKey, ref.generation);
if (!context) throw new AttachmentFailure('staleSession');
const fs = await getFs();
const candidate = resolve(candidateInput);
if (ref.stagingId) {
const stagedPath = dependencies.stagedAttachments.get(ref.stagingId);
if (!stagedPath) throw new AttachmentFailure('invalidReference');
let canonicalCandidate: string;
try {
canonicalCandidate = await fs.realpath(candidate);
} catch (error) {
throw new AttachmentFailure(attachmentFailure(error));
}
if (!isSamePath(canonicalCandidate, stagedPath)) throw new AttachmentFailure('invalidReference');
const stagedStat = await fs.stat(canonicalCandidate);
if (!stagedStat.isFile()) throw new AttachmentFailure('notFile');
return {
kind: 'local',
canonicalPath: canonicalCandidate,
scope: 'staging',
mimeType: mimeTypeHint || mimeTypeForPath(canonicalCandidate),
size: stagedStat.size,
};
}
const { mediaRoots } = await verifyManagedAuthorities(fs);
const workspaceRoots = mediaOnly ? [] : [await frozenCanonicalDirectory(context.workspaceRoot, fs)];
const allowedRoots = [...workspaceRoots, ...mediaRoots];
let containmentCandidate: string;
try {
containmentCandidate = await canonicalCandidateForContainment(candidate, fs);
} catch (error) {
throw new AttachmentFailure(attachmentFailure(error));
}
const lexicalRoot = allowedRoots.find((root) => isInside(containmentCandidate, root));
if (!lexicalRoot) throw new AttachmentFailure('outsideAllowedRoots');
const stagingRoot = await canonicalCandidateForContainment(resolveClawXStagingDir(stateDir), fs);
if (isInside(containmentCandidate, stagingRoot)) {
throw new AttachmentFailure('outsideAllowedRoots');
}
const outgoingRecordRoots = await canonicalRoots([
join(stateDir, 'media', 'outgoing', 'records'),
], fs);
if (outgoingRecordRoots.some((root) => isInside(containmentCandidate, root))) {
throw new AttachmentFailure('outsideAllowedRoots');
}
if (dependencies.stagedAttachments.hasPath(containmentCandidate)) {
throw new AttachmentFailure('outsideAllowedRoots');
}
let canonicalCandidate: string;
try {
canonicalCandidate = await fs.realpath(candidate);
} catch (error) {
throw new AttachmentFailure(attachmentFailure(error));
}
if (!isInside(canonicalCandidate, lexicalRoot)) throw new AttachmentFailure('outsideAllowedRoots');
const targetStat = await fs.stat(canonicalCandidate);
if (!targetStat.isFile()) throw new AttachmentFailure('notFile');
const scope: LocalScope = workspaceRoots.some((root) => isInside(canonicalCandidate, root))
? 'workspace'
: 'openclaw-media';
return {
kind: 'local',
canonicalPath: canonicalCandidate,
scope,
mimeType: mimeTypeHint || mimeTypeForPath(canonicalCandidate),
size: targetStat.size,
authorizationRoot: lexicalRoot,
};
};
const resolveOutgoing = async (
ref: AttachmentSourceRef,
outgoing: { attachmentId: string; sessionKey: string },
): Promise<ResolvedLocal> => {
if (outgoing.sessionKey !== ref.sessionKey) throw new AttachmentFailure('invalidReference');
const fs = await getFs();
const { stateParent, mediaRoots } = await verifyManagedAuthorities(fs);
if (!stateParent) throw new AttachmentFailure('outsideAllowedRoots');
const resolved = await resolveOutgoingMediaAttachment({
uri: ref.uri,
expectedSessionKey: ref.sessionKey,
transcriptMessageId: ref.transcriptMessageId,
stateDir: stateParent.canonicalPath,
configDir,
managedMediaRoots: mediaRoots,
fs,
});
if (!resolved) throw new AttachmentFailure('invalidReference');
return {
kind: 'local',
canonicalPath: resolved.path,
scope: 'openclaw-media',
mimeType: resolved.mimeType,
size: resolved.size,
authorizationRoot: resolved.authorizationRoot,
};
};
const resolveTarget = async (
ref: AttachmentSourceRef,
metadata: Pick<ResolveAttachmentPayload, 'mimeType' | 'size'> = {},
): Promise<ResolvedTarget> => {
if (!ref || typeof ref.sessionKey !== 'string' || !Number.isFinite(ref.generation)) {
throw new AttachmentFailure('invalidReference');
}
validateReferenceSyntax(ref.uri);
const context = dependencies.sessionAccessRegistry.get(ref.sessionKey, ref.generation);
if (!context) throw new AttachmentFailure('staleSession');
const outgoing = parseOutgoingUrl(ref.uri);
if (outgoing) return resolveOutgoing(ref, outgoing);
if (/^https?:/i.test(ref.uri)) {
return {
kind: 'remote',
normalizedUrl: normalizeRemote(ref.uri),
mimeType: metadata.mimeType || mimeTypeForPath(new URL(ref.uri).pathname),
size: typeof metadata.size === 'number' && Number.isFinite(metadata.size) && metadata.size >= 0
? metadata.size
: 0,
};
}
const localPath = localPathFromUri(ref.uri, context.executionCwd);
return resolveLocalCandidate(ref, localPath, metadata.mimeType);
};
const resolveAttachment = async (payload: ResolveAttachmentPayload): Promise<ResolveAttachmentResult> => {
const ref = payload?.ref;
const fallbackName = decodedBasename(typeof ref?.uri === 'string' ? ref.uri : 'attachment');
const displayName = safeDisplayName(payload?.name, fallbackName);
try {
const target = await resolveTarget(ref, payload);
const localName = target.kind === 'local' ? basename(target.canonicalPath) : fallbackName;
const finalDisplayName = safeDisplayName(payload?.name, localName);
if (target.kind === 'remote') {
return {
ok: true,
identity: opaqueIdentity(target.normalizedUrl),
displayName: finalDisplayName,
mimeType: target.mimeType,
size: target.size,
target: { kind: 'remote', ref, url: target.normalizedUrl },
};
}
const displayPath = ref.stagingId
? dependencies.stagedAttachments.getDisplayPath(ref.stagingId)
: null;
return {
ok: true,
identity: opaqueIdentity(target.canonicalPath),
displayName: finalDisplayName,
...(displayPath ? { displayPath } : {}),
mimeType: target.mimeType,
size: target.size,
target: { kind: 'local', scope: target.scope, ref },
};
} catch (error) {
return { ok: false, displayName, error: attachmentFailure(error) };
}
};
const readAttachmentText = async (ref: AttachmentFileRef): Promise<ReadAttachmentTextResult> => {
let opened: { handle: FileHandle; stat: Stats } | undefined;
try {
const target = await resolveTarget(ref);
if (target.kind !== 'local') throw new AttachmentFailure('invalidReference');
opened = await openRevalidatedLocal(target, await getFs());
if (!dependencies.sessionAccessRegistry.get(ref.sessionKey, ref.generation)) {
throw new AttachmentFailure('staleSession');
}
if (opened.stat.size > FILE_PREVIEW_MAX_TEXT_BYTES) {
return { ok: false, error: 'tooLarge', size: opened.stat.size };
}
const buffer = await readOpenedFile(opened.handle, FILE_PREVIEW_MAX_TEXT_BYTES);
if (!buffer) return { ok: false, error: 'tooLarge', size: FILE_PREVIEW_MAX_TEXT_BYTES + 1 };
if (looksLikeBinary(buffer)) return { ok: false, error: 'binary', size: buffer.length };
return {
ok: true,
content: buffer.toString('utf8'),
mimeType: target.mimeType,
size: buffer.length,
readOnly: true,
};
} catch (error) {
return { ok: false, error: attachmentFailure(error) };
} finally {
await opened?.handle.close().catch(() => undefined);
}
};
const readAttachmentBinary = async (
payload: ReadAttachmentBinaryPayload,
): Promise<ReadAttachmentBinaryResult> => {
let opened: { handle: FileHandle; stat: Stats } | undefined;
try {
const target = await resolveTarget(payload?.ref);
if (target.kind !== 'local') throw new AttachmentFailure('invalidReference');
opened = await openRevalidatedLocal(target, await getFs());
if (!dependencies.sessionAccessRegistry.get(payload.ref.sessionKey, payload.ref.generation)) {
throw new AttachmentFailure('staleSession');
}
const cap = boundedBinaryCap(payload.maxBytes);
if (opened.stat.size > cap) return { ok: false, error: 'tooLarge', size: opened.stat.size };
const buffer = await readOpenedFile(opened.handle, cap);
if (!buffer) return { ok: false, error: 'tooLarge', size: cap + 1 };
return {
ok: true,
data: new Uint8Array(buffer.buffer, buffer.byteOffset, buffer.byteLength),
mimeType: target.mimeType,
size: buffer.length,
readOnly: true,
};
} catch (error) {
return { ok: false, error: attachmentFailure(error) };
} finally {
await opened?.handle.close().catch(() => undefined);
}
};
const openAttachment = async (ref: AttachmentSourceRef): Promise<OpenAttachmentResult> => {
let identity = opaqueIdentity(typeof ref?.uri === 'string' ? ref.uri : 'invalid');
let sourceKind: 'local' | 'remote' | 'invalid' = typeof ref?.uri === 'string'
? (/^https?:/i.test(ref.uri) ? 'remote' : 'local')
: 'invalid';
try {
const target = await resolveTarget(ref);
if (target.kind === 'remote') {
sourceKind = 'remote';
identity = opaqueIdentity(target.normalizedUrl);
if (!dependencies.sessionAccessRegistry.get(ref.sessionKey, ref.generation)) {
throw new AttachmentFailure('staleSession');
}
await shell.openExternal(target.normalizedUrl);
} else {
sourceKind = 'local';
identity = opaqueIdentity(target.canonicalPath);
const revalidated = await resolveTarget(ref);
if (revalidated.kind !== 'local') throw new AttachmentFailure('invalidReference');
identity = opaqueIdentity(revalidated.canonicalPath);
if (!dependencies.sessionAccessRegistry.get(ref.sessionKey, ref.generation)) {
throw new AttachmentFailure('staleSession');
}
const error = await shell.openPath(revalidated.canonicalPath);
if (error) throw new AttachmentFailure('operationFailed');
}
recordAttachmentOpenTrace({
ok: true,
reason: 'success',
sourceKind,
sessionKey: ref.sessionKey,
generation: ref.generation,
identity,
});
return { ok: true };
} catch (error) {
const reason = attachmentFailure(error);
recordAttachmentOpenTrace({
ok: false,
reason,
sourceKind,
sessionKey: typeof ref?.sessionKey === 'string' ? ref.sessionKey : '',
generation: typeof ref?.generation === 'number' ? ref.generation : -1,
identity,
});
return { ok: false, error: reason };
}
};
return {
resolveAttachment,
readAttachmentText,
readAttachmentBinary,
openAttachment,
};
}
electron/services/chat-api.ts
+4 -1
View File
@@ -3,6 +3,7 @@ import type { GatewayManager } from '../gateway/manager';
import type { CompleteHostServiceRegistry } from '../main/ipc/host-contract';
import { logger } from '../utils/logger';
import { createAcpChatService } from './acp-chat-service';
import type { AcpSessionAccessRegistry } from './acp-session-access-registry';
import { isRecord } from './payload-utils';
const VISION_MIME_TYPES = new Set([
@@ -43,11 +44,13 @@ function normalizeMedia(media: unknown): Array<{ filePath: string; mimeType: str
export function createChatApi({
gatewayManager,
mainWindow,
acpSessionAccessRegistry,
}: {
gatewayManager: GatewayManager;
mainWindow: BrowserWindow;
acpSessionAccessRegistry: AcpSessionAccessRegistry;
}): CompleteHostServiceRegistry['chat'] {
const acpChat = createAcpChatService(mainWindow, gatewayManager);
const acpChat = createAcpChatService(mainWindow, acpSessionAccessRegistry, gatewayManager);
return {
sendWithMedia: async (payload) => {
electron/services/files-api.ts
+229 -19
View File
@@ -23,8 +23,17 @@ import type {
FileReadBinaryOptions,
WorkspaceFileRef,
} from '@shared/host-api/contract';
import {
FILE_PREVIEW_MAX_BINARY_BYTES,
FILE_PREVIEW_MAX_TEXT_BYTES,
} from '@shared/file-preview/limits';
import type { CompleteHostServiceRegistry } from '../main/ipc/host-contract';
import { expandPath } from '../utils/paths';
import { expandPath, resolveOpenClawStateDir } from '../utils/paths';
import {
resolveClawXStagingDir,
type AttachmentAccess,
type StagedAttachmentRegistry,
} from './attachment-access';
import { isRecord } from './payload-utils';
const EXT_MIME_MAP: Record<string, string> = {
@@ -70,10 +79,7 @@ const EXT_MIME_MAP: Record<string, string> = {
'.pptx': 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
};
const OUTBOUND_DIR = join(homedir(), '.openclaw', 'media', 'outbound');
const DIRECTORY_MIME_TYPE = 'application/x-directory';
const FILE_PREVIEW_MAX_TEXT_BYTES = 2 * 1024 * 1024;
const FILE_PREVIEW_MAX_BINARY_BYTES = 50 * 1024 * 1024;
const FILE_PREVIEW_TREE_MAX_DEPTH = 6;
const FILE_PREVIEW_TREE_MAX_NODES = 5000;
const FILE_PREVIEW_DIR_BLACKLIST = new Set([
@@ -127,6 +133,23 @@ type WorkspaceFs = {
type FilesApiDependencies = {
workspaceFs?: WorkspaceFs;
attachmentAccess?: AttachmentAccess;
stagedAttachments?: StagedAttachmentRegistry;
stagingHooks?: {
beforeDestinationOpen?: (input: { stagingDir: string; destinationPath: string }) => Promise<void>;
};
};
type PinnedStagingDirectory = {
lexicalPath: string;
canonicalPath: string;
dev: number;
ino: number;
};
type PinnedStagingArea = {
stagingDir: string;
directories: PinnedStagingDirectory[];
};
function getMimeType(ext: string): string {
@@ -160,6 +183,24 @@ async function generateImagePreview(filePath: string, mimeType: string): Promise
}
}
function generateImageBufferPreview(buffer: Buffer, mimeType: string): string | null {
try {
const img = nativeImage.createFromBuffer(buffer);
if (img.isEmpty()) return null;
const size = img.getSize();
const maxDim = 512;
if (size.width > maxDim || size.height > maxDim) {
const resized = size.width >= size.height
? img.resize({ width: maxDim })
: img.resize({ height: maxDim });
return `data:image/png;base64,${resized.toPNG().toString('base64')}`;
}
return `data:${mimeType};base64,${buffer.toString('base64')}`;
} catch {
return null;
}
}
function requirePath(payload: unknown): string {
const path = isRecord(payload) ? payload.path : payload;
if (typeof path !== 'string' || !path.trim()) {
@@ -317,7 +358,7 @@ function getFilePreviewWriteRoots(): string[] {
} catch {
// ignore
}
roots.push(resolve(OUTBOUND_DIR));
roots.push(resolve(resolveClawXStagingDir()));
return roots;
}
@@ -376,6 +417,162 @@ function getBinaryOptions(opts: unknown): FileReadBinaryOptions {
export function createFilesApi(dependencies: FilesApiDependencies = {}): CompleteHostServiceRegistry['files'] {
const getWorkspaceFs = async (): Promise<WorkspaceFs> => dependencies.workspaceFs
?? await import('node:fs/promises');
const stagingAreaName = `clawx-${process.pid}-${crypto.randomUUID()}`;
let stagingAreaPromise: Promise<PinnedStagingArea> | null = null;
const initializeStagingArea = async (): Promise<PinnedStagingArea> => {
const fsP = await import('node:fs/promises');
const directories: PinnedStagingDirectory[] = [];
const ensureDirectory = async (lexicalPath: string, parent?: PinnedStagingDirectory) => {
let entryStat: Stats;
try {
entryStat = await fsP.lstat(lexicalPath);
} catch (error) {
const code = error && typeof error === 'object' && 'code' in error ? error.code : undefined;
if (code !== 'ENOENT') throw error;
await fsP.mkdir(lexicalPath, { mode: 0o700 });
entryStat = await fsP.lstat(lexicalPath);
}
if (entryStat.isSymbolicLink() || !entryStat.isDirectory()) {
throw new Error('Invalid ClawX staging directory');
}
const canonicalPath = await fsP.realpath(lexicalPath);
const canonicalStat = await fsP.stat(canonicalPath);
if (!canonicalStat.isDirectory()
|| (parent && !isPathInside(canonicalPath, parent.canonicalPath))) {
throw new Error('Invalid ClawX staging directory');
}
const pinned = {
lexicalPath,
canonicalPath,
dev: canonicalStat.dev,
ino: canonicalStat.ino,
};
directories.push(pinned);
return pinned;
};
const stateDir = await ensureDirectory(resolveOpenClawStateDir());
const mediaDir = await ensureDirectory(join(stateDir.canonicalPath, 'media'), stateDir);
const outboundDir = await ensureDirectory(join(mediaDir.canonicalPath, 'outbound'), mediaDir);
const stagingRoot = await ensureDirectory(join(outboundDir.canonicalPath, 'clawx-staging'), outboundDir);
const stagingArea = await ensureDirectory(join(stagingRoot.canonicalPath, stagingAreaName), stagingRoot);
return { stagingDir: stagingArea.canonicalPath, directories };
};
const getStagingArea = () => {
stagingAreaPromise ??= initializeStagingArea();
return stagingAreaPromise;
};
const verifyStagingArea = async (area: PinnedStagingArea) => {
const fsP = await import('node:fs/promises');
for (const directory of area.directories) {
const entryStat = await fsP.lstat(directory.lexicalPath);
if (entryStat.isSymbolicLink()) throw new Error('Invalid ClawX staging directory');
const currentPath = await fsP.realpath(directory.lexicalPath);
const currentStat = await fsP.stat(currentPath);
if (!currentStat.isDirectory()
|| !isSamePath(currentPath, directory.canonicalPath)
|| currentStat.dev !== directory.dev
|| currentStat.ino !== directory.ino) {
throw new Error('Invalid ClawX staging directory');
}
}
};
const cleanupOwnedDestination = async (destinationPath: string, identity?: { dev: number; ino: number }) => {
if (!identity) return;
const fsP = await import('node:fs/promises');
try {
const current = await fsP.stat(destinationPath);
if (current.dev === identity.dev && current.ino === identity.ino) {
await fsP.unlink(destinationPath);
}
} catch {
// The destination was already removed or redirected again.
}
};
const createStagedFile = async (
fileName: string,
write: (handle: FileHandle) => Promise<void>,
): Promise<{ path: string; stat: Stats }> => {
const fsP = await import('node:fs/promises');
const area = await getStagingArea();
await verifyStagingArea(area);
const destinationPath = join(area.stagingDir, fileName);
await dependencies.stagingHooks?.beforeDestinationOpen?.({
stagingDir: area.stagingDir,
destinationPath,
});
let handle: FileHandle | undefined;
let identity: { dev: number; ino: number } | undefined;
try {
const noFollow = process.platform === 'win32' ? 0 : constants.O_NOFOLLOW;
handle = await fsP.open(
destinationPath,
constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | noFollow,
0o600,
);
const openedStat = await handle.stat();
identity = { dev: openedStat.dev, ino: openedStat.ino };
// Node has no openat. Validate the unpredictable empty destination before writing bytes.
await verifyStagingArea(area);
const canonicalDestination = await fsP.realpath(destinationPath);
const pathStat = await fsP.stat(canonicalDestination);
if (!isPathInside(canonicalDestination, area.stagingDir)
|| pathStat.dev !== openedStat.dev
|| pathStat.ino !== openedStat.ino) {
throw new Error('Invalid ClawX staging destination');
}
await write(handle);
const finalStat = await handle.stat();
await verifyStagingArea(area);
const finalPath = await fsP.realpath(destinationPath);
const finalPathStat = await fsP.stat(finalPath);
if (!isSamePath(finalPath, canonicalDestination)
|| finalPathStat.dev !== finalStat.dev
|| finalPathStat.ino !== finalStat.ino) {
throw new Error('Invalid ClawX staging destination');
}
await handle.close();
handle = undefined;
await verifyStagingArea(area);
const registrationPath = await fsP.realpath(destinationPath);
const registrationStat = await fsP.stat(registrationPath);
if (!isSamePath(registrationPath, finalPath)
|| registrationStat.dev !== finalStat.dev
|| registrationStat.ino !== finalStat.ino) {
throw new Error('Invalid ClawX staging destination');
}
return { path: registrationPath, stat: finalStat };
} catch (error) {
await handle?.close().catch(() => undefined);
await cleanupOwnedDestination(destinationPath, identity);
throw error;
}
};
const copyIntoHandle = async (sourcePath: string, destination: FileHandle) => {
const fsP = await import('node:fs/promises');
const source = await fsP.open(sourcePath, constants.O_RDONLY);
try {
const buffer = Buffer.allocUnsafe(64 * 1024);
let position = 0;
while (true) {
const { bytesRead } = await source.read(buffer, 0, buffer.length, position);
if (bytesRead === 0) break;
await destination.write(buffer, 0, bytesRead, position);
position += bytesRead;
}
} finally {
await source.close();
}
};
return {
stagePaths: async (payload) => {
const body = isRecord(payload) ? payload as StagePathsPayload : {};
@@ -383,8 +580,6 @@ export function createFilesApi(dependencies: FilesApiDependencies = {}): Complet
? body.filePaths.filter((value): value is string => typeof value === 'string')
: [];
const fsP = await import('node:fs/promises');
await fsP.mkdir(OUTBOUND_DIR, { recursive: true });
const results = [];
for (const filePath of filePaths) {
const id = crypto.randomUUID();
@@ -403,14 +598,13 @@ export function createFilesApi(dependencies: FilesApiDependencies = {}): Complet
}
const ext = extname(filePath);
const stagedPath = join(OUTBOUND_DIR, `${id}${ext}`);
await fsP.copyFile(filePath, stagedPath);
const s = await fsP.stat(stagedPath);
const mimeType = getMimeType(ext);
const preview = mimeType.startsWith('image/')
? await generateImagePreview(stagedPath, mimeType)
? await generateImagePreview(filePath, mimeType)
: null;
results.push({ id, fileName, mimeType, fileSize: s.size, stagedPath, preview });
const staged = await createStagedFile(`${id}${ext}`, (handle) => copyIntoHandle(filePath, handle));
dependencies.stagedAttachments?.register(id, staged.path, filePath);
results.push({ id, fileName, mimeType, fileSize: staged.stat.size, stagedPath: staged.path, preview });
}
return results;
},
@@ -419,26 +613,25 @@ export function createFilesApi(dependencies: FilesApiDependencies = {}): Complet
if (typeof body.base64 !== 'string' || typeof body.fileName !== 'string') {
throw new Error('Invalid staged buffer payload');
}
const fsP = await import('node:fs/promises');
await fsP.mkdir(OUTBOUND_DIR, { recursive: true });
const id = crypto.randomUUID();
const payloadMimeType = typeof body.mimeType === 'string' ? body.mimeType : '';
const ext = extname(body.fileName) || mimeToExt(payloadMimeType);
const stagedPath = join(OUTBOUND_DIR, `${id}${ext}`);
const buffer = Buffer.from(body.base64, 'base64');
await fsP.writeFile(stagedPath, buffer);
const mimeType = payloadMimeType || getMimeType(ext);
const preview = mimeType.startsWith('image/')
? await generateImagePreview(stagedPath, mimeType)
? generateImageBufferPreview(buffer, mimeType)
: null;
const staged = await createStagedFile(`${id}${ext}`, async (handle) => {
await handle.writeFile(buffer);
});
dependencies.stagedAttachments?.register(id, staged.path);
return {
id,
fileName: body.fileName,
mimeType,
fileSize: buffer.length,
stagedPath,
stagedPath: staged.path,
preview,
};
},
@@ -535,6 +728,23 @@ export function createFilesApi(dependencies: FilesApiDependencies = {}): Complet
await opened?.handle.close().catch(() => undefined);
}
},
resolveAttachment: async (payload) => dependencies.attachmentAccess?.resolveAttachment(payload) ?? {
ok: false,
displayName: 'attachment',
error: 'operationFailed',
},
readAttachmentText: async (ref) => dependencies.attachmentAccess?.readAttachmentText(ref) ?? {
ok: false,
error: 'operationFailed',
},
readAttachmentBinary: async (payload) => dependencies.attachmentAccess?.readAttachmentBinary(payload) ?? {
ok: false,
error: 'operationFailed',
},
openAttachment: async (ref) => dependencies.attachmentAccess?.openAttachment(ref) ?? {
ok: false,
error: 'operationFailed',
},
readText: async (payload) => {
try {
const { realPath: real, readOnly } = await resolveSandboxedPath(requirePath(payload), 'read');
electron/services/media-api.ts
+64 -23
View File
@@ -2,6 +2,9 @@ import { dialog, nativeImage } from 'electron';
import { homedir } from 'node:os';
import { join } from 'node:path';
import type { CompleteHostServiceRegistry } from '../main/ipc/host-contract';
import type { AttachmentFileRef } from '@shared/host-api/contract';
import { resolveOutgoingMediaAttachment, type AttachmentAccess } from './attachment-access';
import { resolveOpenClawStateDir } from '../utils/paths';
import {
CLAWX_OPENAI_IMAGE_DEFAULT_MODEL,
CLAWX_OPENAI_IMAGE_PROVIDER_KEY,
@@ -19,9 +22,17 @@ import { isRecord } from './payload-utils';
type ThumbnailEntry = {
filePath?: unknown;
gatewayUrl?: unknown;
attachmentFileRef?: unknown;
key?: unknown;
mimeType?: unknown;
};
type MediaApiDependencies = {
attachmentAccess?: Pick<AttachmentAccess, 'resolveAttachment' | 'readAttachmentBinary'>;
};
const OPAQUE_ATTACHMENT_KEY = /^[a-f0-9]{64}$/;
type SaveImagePayload = {
base64?: unknown;
mimeType?: unknown;
@@ -62,28 +73,22 @@ async function generateImagePreview(filePath: string, mimeType: string): Promise
}
}
async function resolveOutgoingMediaUrl(
gatewayUrl: string,
): Promise<{ path: string; mimeType: string } | null> {
function generateImagePreviewFromBuffer(buffer: Buffer, mimeType: string): string | null {
try {
const match = gatewayUrl.match(/\/api\/chat\/media\/outgoing\/[^/]+\/([^/]+)\//);
if (!match) return null;
const attachmentId = decodeURIComponent(match[1]);
if (!/^[A-Za-z0-9._-]+$/.test(attachmentId)) return null;
const recordPath = join(homedir(), '.openclaw', 'media', 'outgoing', 'records', `${attachmentId}.json`);
const fsP = await import('node:fs/promises');
const raw = await fsP.readFile(recordPath, 'utf8');
const record = JSON.parse(raw) as {
original?: { path?: string; contentType?: string };
};
const original = record?.original;
if (!original?.path) return null;
return {
path: original.path,
mimeType: typeof original.contentType === 'string' && original.contentType
? original.contentType
: 'application/octet-stream',
};
if (mimeType === 'image/svg+xml') {
return `data:${mimeType};base64,${buffer.toString('base64')}`;
}
const img = nativeImage.createFromBuffer(buffer);
if (img.isEmpty()) return null;
const size = img.getSize();
const maxDim = 512;
if (size.width > maxDim || size.height > maxDim) {
const resized = size.width >= size.height
? img.resize({ width: maxDim })
: img.resize({ height: maxDim });
return `data:image/png;base64,${resized.toPNG().toString('base64')}`;
}
return `data:${mimeType};base64,${buffer.toString('base64')}`;
} catch {
return null;
}
@@ -94,7 +99,7 @@ function normalizeThumbnailEntries(payload: unknown): ThumbnailEntry[] {
return Array.isArray(value) ? value as ThumbnailEntry[] : [];
}
export function createMediaApi(): CompleteHostServiceRegistry['media'] {
export function createMediaApi(dependencies: MediaApiDependencies = {}): CompleteHostServiceRegistry['media'] {
return {
thumbnails: async (payload) => {
const entries = normalizeThumbnailEntries(payload);
@@ -102,6 +107,39 @@ export function createMediaApi(): CompleteHostServiceRegistry['media'] {
const results: Record<string, { preview: string | null; fileSize: number }> = {};
for (const entry of entries) {
const mimeType = typeof entry.mimeType === 'string' ? entry.mimeType : 'application/octet-stream';
if (entry.attachmentFileRef && typeof entry.attachmentFileRef === 'object') {
const key = typeof entry.key === 'string' && entry.key ? entry.key : null;
if (!key || !OPAQUE_ATTACHMENT_KEY.test(key) || !dependencies.attachmentAccess) continue;
const ref = entry.attachmentFileRef as AttachmentFileRef;
const resolution = await dependencies.attachmentAccess.resolveAttachment({ ref });
if (!resolution.ok
|| resolution.identity !== key
|| resolution.target.kind !== 'local') {
continue;
}
const readResult = await dependencies.attachmentAccess.readAttachmentBinary({
ref,
});
if (!readResult.ok) {
results[key] = { preview: null, fileSize: 0 };
continue;
}
const effectiveMimeType = mimeType === 'application/octet-stream'
? readResult.mimeType
: mimeType;
const buffer = Buffer.from(
readResult.data.buffer,
readResult.data.byteOffset,
readResult.data.byteLength,
);
results[key] = {
preview: effectiveMimeType.startsWith('image/')
? generateImagePreviewFromBuffer(buffer, effectiveMimeType)
: null,
fileSize: readResult.size,
};
continue;
}
if (typeof entry.filePath === 'string' && entry.filePath) {
try {
const stat = await fsP.stat(entry.filePath);
@@ -116,7 +154,10 @@ export function createMediaApi(): CompleteHostServiceRegistry['media'] {
}
if (typeof entry.gatewayUrl === 'string' && entry.gatewayUrl) {
const resolved = await resolveOutgoingMediaUrl(entry.gatewayUrl);
const resolved = await resolveOutgoingMediaAttachment({
uri: entry.gatewayUrl,
stateDir: resolveOpenClawStateDir(),
});
if (!resolved) {
results[entry.gatewayUrl] = { preview: null, fileSize: 0 };
continue;
electron/services/sessions-api.ts
+8 -8
View File
@@ -5,7 +5,7 @@ import type { CompleteHostServiceRegistry } from '../main/ipc/host-contract';
import { stripAcpWorkingDirectoryPrefix } from '@shared/chat/session-title';
import { isOpenClawHeartbeatPollText } from '@shared/chat/openclaw-internal';
import type { RawMessage } from '@shared/chat/types';
import { getOpenClawConfigDir } from '../utils/paths';
import { resolveOpenClawStateDir } from '../utils/paths';
import { logger } from '../utils/logger';
import {
removeSessionEntry,
@@ -151,7 +151,7 @@ async function readOpenClawAcpSessionCwds(sessionKeys: string[]): Promise<Map<st
const workspaceByKey = new Map<string, string>();
if (normalizedKeys.length === 0) return workspaceByKey;
const databasePath = join(getOpenClawConfigDir(), 'state', 'openclaw.sqlite');
const databasePath = join(resolveOpenClawStateDir(), 'state', 'openclaw.sqlite');
try {
await access(databasePath);
const sqliteSpecifier = 'node:sqlite';
@@ -305,7 +305,7 @@ function getLimit(payload: unknown, fallback = 200): number {
async function readSessionsJson(agentId: string): Promise<Record<string, unknown>> {
const fsP = await import('node:fs/promises');
const sessionsJsonPath = join(getOpenClawConfigDir(), 'agents', agentId, 'sessions', 'sessions.json');
const sessionsJsonPath = join(resolveOpenClawStateDir(), 'agents', agentId, 'sessions', 'sessions.json');
const raw = await fsP.readFile(sessionsJsonPath, 'utf8');
return JSON.parse(raw) as Record<string, unknown>;
}
@@ -367,7 +367,7 @@ async function loadSessionSummary(sessionKey: string, workspacePath: string | nu
}
try {
const sessionsDir = join(getOpenClawConfigDir(), 'agents', parsed.agentId, 'sessions');
const sessionsDir = join(resolveOpenClawStateDir(), 'agents', parsed.agentId, 'sessions');
const sessionsJson = await readSessionsJson(parsed.agentId);
const transcriptPath = resolveSessionTranscriptPathByKey(sessionKey, sessionsDir, sessionsJson);
if (!transcriptPath) {
@@ -386,7 +386,7 @@ async function loadSessionTranscriptByKey(sessionKey: string, limit: number): Pr
if (!parsed) return null;
try {
const sessionsDir = join(getOpenClawConfigDir(), 'agents', parsed.agentId, 'sessions');
const sessionsDir = join(resolveOpenClawStateDir(), 'agents', parsed.agentId, 'sessions');
const sessionsJson = await readSessionsJson(parsed.agentId);
const transcriptPath = resolveSessionTranscriptPathByKey(sessionKey, sessionsDir, sessionsJson);
if (!transcriptPath) return null;
@@ -410,7 +410,7 @@ async function deleteSession(sessionKey: string): Promise<{ success: boolean; er
return { success: false, error: `Invalid agentId: ${agentId}` };
}
const sessionsDir = join(getOpenClawConfigDir(), 'agents', agentId, 'sessions');
const sessionsDir = join(resolveOpenClawStateDir(), 'agents', agentId, 'sessions');
const sessionsJsonPath = join(sessionsDir, 'sessions.json');
logger.info(`[session:delete] key=${sessionKey} agentId=${agentId}`);
logger.info(`[session:delete] sessionsJson=${sessionsJsonPath}`);
@@ -478,7 +478,7 @@ async function renameSession(sessionKey: string, label: string): Promise<{ succe
return { success: false, error: `Invalid agentId in sessionKey: ${agentId}` };
}
const sessionsJsonPath = join(getOpenClawConfigDir(), 'agents', agentId, 'sessions', 'sessions.json');
const sessionsJsonPath = join(resolveOpenClawStateDir(), 'agents', agentId, 'sessions', 'sessions.json');
const fsP = await import('node:fs/promises');
const raw = await fsP.readFile(sessionsJsonPath, 'utf8');
const json = JSON.parse(raw) as Record<string, unknown>;
@@ -553,7 +553,7 @@ export function createSessionsApi(): CompleteHostServiceRegistry['sessions'] {
}
try {
const transcriptPath = join(getOpenClawConfigDir(), 'agents', agentId, 'sessions', `${sessionId}.jsonl`);
const transcriptPath = join(resolveOpenClawStateDir(), 'agents', agentId, 'sessions', `${sessionId}.jsonl`);
return { success: true, messages: readRecentTranscriptMessages(transcriptPath, limit) };
} catch (error) {
if (typeof error === 'object' && error !== null && 'code' in error && error.code === 'ENOENT') {
electron/utils/paths.ts
+15 -1
View File
@@ -3,7 +3,7 @@
* Cross-platform path resolution helpers
*/
import { createRequire } from 'node:module';
import { join } from 'path';
import { dirname, join, resolve } from 'path';
import { homedir } from 'os';
import { existsSync, mkdirSync, readFileSync, realpathSync } from 'fs';
@@ -54,6 +54,20 @@ export function getOpenClawConfigDir(): string {
return join(homedir(), '.openclaw');
}
export function resolveOpenClawStateDir(env: NodeJS.ProcessEnv = process.env): string {
const configured = env.OPENCLAW_STATE_DIR?.trim();
return resolve(expandPath(configured || join(homedir(), '.openclaw')));
}
export function resolveOpenClawConfigPath(env: NodeJS.ProcessEnv = process.env): string {
const configured = env.OPENCLAW_CONFIG_PATH?.trim();
return resolve(expandPath(configured || join(resolveOpenClawStateDir(env), 'openclaw.json')));
}
export function resolveOpenClawConfigDir(env: NodeJS.ProcessEnv = process.env): string {
return dirname(resolveOpenClawConfigPath(env));
}
/**
* Get OpenClaw skills directory
*/
harness/reference/acp-attachment-access-control.md
+74
View File
@@ -0,0 +1,74 @@
# ACP Attachment Access Control
Status: current security and ownership reference, reviewed 2026-07-15.
Related scenario: `acp-chat-experience`
Related rules: `attachment-access-safety`, `session-workspace-authority`, `tool-derived-file-safety`, `renderer-main-boundary`
Related task: `acp-media-attachments`
## Trust Boundaries And Ownership
Renderer owns attachment parsing, timeline projection, presentation, and click routing. Electron Main owns the ACP workspace grant, filesystem and URI authorization, scoped reads, and operating-system open actions. Renderer-provided URIs, metadata, staging ids, transcript message ids, and attachment references are untrusted inputs.
An attachment source reference conceptually identifies the active ACP session key, generation, original URI, and optional Main-issued staging or transcript evidence. A resolved local or remote reference repeats that routing identity. These references and Renderer-visible attachment ids are not bearer capabilities. Exact current fields, result unions, error values, and operation signatures in `shared/host-api/contract.ts` and `src/lib/acp/timeline-types.ts` are authoritative.
Renderer reaches attachment operations only through the typed host API. Main exposes resolution, bounded text and binary reads, and click-initiated open. A successful resolution supplies display metadata, a non-sensitive opaque identity, and an attachment-scoped target, but it does not authorize a later read or open.
## Grant Lifecycle
ACP session load or creation is the only workspace grant boundary. Main canonicalizes the selected workspace root and execution cwd, verifies that both are directories and cwd is contained by the workspace, and commits the context only after the ACP operation succeeds. A failed load restores the prior ACP state and prior grant. Switching the active session or advancing generation replaces the active grant.
Every attachment operation looks up the one active Main-owned context by exact session key and generation. Attachment, preview, and open payloads cannot provide or replace a workspace root. Generation is a revocation and race token used together with session identity; it is not a globally monotonic credential.
## Allowed Ownership Scopes
Current local access is limited to these ownership scopes:
- `workspace`: a regular file canonically contained by the active ACP workspace root. Relative references resolve from the registered execution cwd.
- `openclaw-media`: a regular file under the exact `<openclaw-state>/media` or `<openclaw-config>/media` subtree resolved from the same runtime configuration used by OpenClaw. The current distributed runtime has no separate external media-root setting, so no parent-directory or generic external-root expansion is allowed.
- `staging`: the exact canonical file bound to a Main-owned id created by ClawX staging. Merely placing or naming a path under the staging parent does not confer staging ownership.
- `remote`: a normalized HTTP or HTTPS URL without embedded credentials. Remote references remain session/generation scoped and are revalidated immediately before external open.
State and config parents, transcripts, credentials, stores, workspaces, sandboxes, canvas directories, staging files without their Main-owned id, and outgoing record files themselves are not attachment roots.
Gateway outgoing media is a record-bound form of `openclaw-media`, not a general local URL allowlist. Main validates the outgoing attachment id, requires the URL session key and managed record `sessionKey` to equal the active ACP session key, requires the record attachment id to match, and resolves the record's original file through an allowed managed-media root. If both transcript evidence and the record carry a message id, they must agree. The literal `global` session key follows exact equality and is never a wildcard.
## Path And URI Hardening
Main applies syntax checks before ownership checks and authorization again before each side effect. Current defenses include:
- Reject empty, NUL-containing, traversal-bearing, unknown-scheme, UNC, network-share, and overlong source references. The current source-reference bound is `4096` characters.
- Decode percent-encoded input once through platform URL handling, then reject encoded traversal or NUL content as well.
- Accept `file:` URLs only with an empty authority or local `localhost` authority; reject remote authorities and credentials.
- Accept only HTTP and HTTPS remote URLs, require a host, reject credentials, and use the platform URL normalization for identity and open.
- Resolve home-relative, absolute, Windows-drive, and execution-cwd-relative local references without treating a Renderer-provided path as an authorization root.
- Require an existing regular file and enforce lexical, realpath, symlink, and allowed-root containment.
- Freeze canonical workspace and managed-media authority so later parent replacement or symlink substitution cannot silently widen access.
Main re-resolves the original reference for every operation. Scoped reads open the file without following a final symlink where the platform supports it, compare the opened handle with the current canonical path and file identity, recheck containment and active generation, and read through that verified handle. Local system open re-resolves immediately before `shell.openPath`; remote open revalidates the normalized URL and active generation before `shell.openExternal`. These checks limit time-of-check/time-of-use substitution rather than relying on a prior resolve result.
## Opaque Identity And Safe Labels
After authorization, Main returns an opaque hash derived from the canonical local target or normalized remote URL. Renderer uses that value for turn-scoped deduplication and diagnostics, but it must not expose a sensitive host path or be treated as access authority.
Display labels come from approved metadata or a decoded basename. Main reduces them to a basename, removes control and bidirectional-formatting characters, collapses whitespace to one line, applies the current length bound, and falls back to a generic attachment label. Assistant and unavailable attachment cards remain basename-only. Main-owned staging metadata may separately provide the user-selected source path for the established user-attachment presentation; ACP or transcript metadata cannot do so.
## Preview And Open Routing
The shared Renderer classifier in `src/lib/file-preview-capabilities.ts` decides whether an authorized local attachment fits an existing inline viewer and its size cap. Supported text/code, HTML, CSV, image, PDF, and spreadsheet targets use the right-side Preview panel. Unsupported, known binary, audio/video, archive, office-document, or over-limit local targets use the system application only after a user click. HTTP and HTTPS targets open externally only after a user click.
Every attachment preview carries an attachment-scoped file reference. Preview components and rich viewers must use the attachment text or binary read operations and must not fall back to a naked path or general workspace read. Attachment previews also omit trusted workspace-browser reveal or folder actions.
## Failure Isolation
An invalid, stale, missing, unsafe, non-file, or outside-root reference becomes an unavailable attachment result. It cannot be previewed or opened, but it does not suppress assistant prose or independently valid attachments. Read failures remain inside the Preview panel; local or remote open failures use the localized non-blocking Chat error path. Transcript fetch or compatibility resolution failure must not turn a successful ACP prompt into a prompt error.
Diagnostics may record bounded reason codes, source kind, session/generation routing data, and hashed identities. They must not contain transcript bodies, file content, credentials, or full sensitive paths.
## Validation Anchors
Authorization and race coverage lives primarily in `tests/unit/acp-session-access-registry.test.ts`, `tests/unit/attachment-access.test.ts`, `tests/unit/files-api-workspace.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/unit/file-preview-body.test.tsx`, `tests/unit/rich-file-viewers.test.tsx`, `tests/unit/artifact-panel.test.tsx`, and `tests/e2e/chat-acp-attachments.spec.ts`.
Parser, protocol mapping, ordering, deduplication, and component behavior are covered by `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-reducer.test.ts`, `tests/unit/acp-timeline-groups.test.ts`, and `tests/unit/acp-chat-components.test.tsx`.
harness/reference/acp-chat.md
+20 -6
View File
@@ -1,12 +1,12 @@
# ACP Chat Architecture And Timeline
Status: current architecture reference, reviewed 2026-07-13.
Status: current architecture reference, reviewed 2026-07-15.
Related scenario: `acp-chat-experience`
Related rules: `acp-chat-state-and-history`, `renderer-main-boundary`
Related rules: `acp-chat-state-and-history`, `attachment-access-safety`, `renderer-main-boundary`
Related tasks: `acp-native-chat`, `filter-openclaw-heartbeat-session`
Related tasks: `acp-native-chat`, `acp-media-attachments`, `filter-openclaw-heartbeat-session`
## Ownership
@@ -23,7 +23,7 @@ Gateway remains responsible for non-Chat capabilities. Restricted Gateway host-e
## Identity And Race Protection
Renderer-visible session identity is the OpenClaw Gateway session key. Main may hold a different ACP session id returned by `newSession`; it rewrites downstream routing to the selected session key. A routing envelope carries the session key and the generation token for the active load. Renderer ignores updates, permission requests, and asynchronous hydration results whose session or generation no longer matches. Generation is an in-memory race token rather than a durable sequence; Main may restore the previous value when a load fails, so code must compare it together with active-session and current-operation state rather than assume global monotonicity.
Renderer-visible session identity is the OpenClaw Gateway session key. Main may hold a different ACP session id returned by `newSession`; it rewrites downstream routing to the selected session key. Loads on the shared ACP connection are serialized. A routing envelope carries the session key and the Main-owned generation token for the active load. Renderer uses a separate local request sequence to reject stale load completions; preparing a local-only session must not advance the ACP generation. Renderer ignores updates, permission requests, and asynchronous hydration results whose session or generation no longer matches. Generation is an in-memory race token rather than a durable sequence; Main may restore the previous value when a load fails, so code must compare it together with active-session and current-operation state rather than assume global monotonicity.
`messageId` and `toolCallId` are opaque identities within one loaded timeline. They are not durable UI identities across loads. Timeline sequence values and DOM anchors are also local to the active snapshot.
@@ -31,7 +31,9 @@ Renderer-visible session identity is the OpenClaw Gateway session key. Main may
ACP `session/load` replay is the primary source of Chat history. ClawX does not persist an ACP ledger, reduced timeline, replay cache, or reconstructed tool history. Full structured replay can restore tools and file activity; transcript-only fallback must not invent them.
There is one approved supplement: after an existing session loads, ClawX may query Main-owned transcript history to recover asynchronous image-generation completions omitted by ACP replay. This requires a preceding `image_generate` start in the same transcript and uses the same safe media hydration path as live compatibility projection. The exception does not apply to normal assistant text, tool cards, plans, permissions, or file activity.
OpenClaw emits replay through ordinary `session/update` notifications and completes the replay before `session/load` returns. Main collects those raw notifications for the active load generation and returns them with the load result instead of forwarding them incrementally. Renderer temporarily groups generation-matching host events that arrive during the IPC result handoff, then runs the normal reducer over the combined batch and publishes the resulting timeline in one state update. This is an in-flight transaction buffer only, not a history cache; after load, live updates continue through the normal host-event route. Permission requests are accepted only after the current loaded session starts a prompt, preventing load-time or handoff requests from creating invisible waiters.
There are exactly two approved transcript supplements. ClawX may recover asynchronous image-generation completions with proven `image_generate` context, and it may recover explicit line-leading assistant `MEDIA:` attachment directives omitted by OpenClaw ACP. Both are bounded, marked, memory-only projections. They do not authorize reconstruction of ordinary assistant text, thoughts, tool cards, plans, permissions, or file activity. See `harness/reference/acp-generated-media-and-diagnostics.md#bounded-transcript-exceptions` for the compatibility grammar, alignment, rationale, and removal condition.
## Timeline Model
@@ -70,6 +72,18 @@ The protocol timeline remains flat. `src/lib/acp/timeline-groups.ts` derives dis
An assistant turn has one identity column and one copy action. Copy includes textual assistant segments and excludes tool output. Tool cards render inline in original order, preserve preformatted whitespace, auto-collapse one second after live completion, respect manual override, and start collapsed when historical and completed.
## Attachments
Standard ACP `resource_link` and URI-backed `resource` content is the preferred attachment path. OpenClaw ACP currently projects assistant text and thought content but can omit assistant media while removing `MEDIA:` directives from the visible live reply. Until upstream emits standard resource content, the bounded explicit-`MEDIA:` supplement may add a marked compatibility attachment to the matching turn without manufacturing an ACP event.
Standard `resource_link` mapping preserves its URI, name, title fallback, MIME type, and size when supplied. A URI-backed embedded `resource` uses the same model and metadata precedence; embedded content without a usable URI becomes unavailable rather than entering an unrelated unsupported-content path. Exact TypeScript models remain authoritative.
Renderer keeps attachment references and compatibility projections in the active in-memory timeline. Main owns the ACP workspace grant and resolves, reads, and opens every attachment against the exact session and generation. A prior resolution is not reusable authorization, and Renderer cannot supply a replacement workspace root. Native ACP evidence wins when it resolves to the same identity as compatibility evidence. The complete authorization and URI boundary is documented in `harness/reference/acp-attachment-access-control.md`.
Assistant grouping lifts attachments from message, thought, and tool-output segments into one ordered turn list after all prose and process items and before file activity. This prevents an early resource block from appearing above later assistant prose. User grouping similarly renders all prose before ordered attachments. User-selected images render as Main-generated thumbnails whose hover overlay identifies the file. Other user-selected files show the filename followed by the muted, truncating source path retained by Main's staging registry; ACP-provided paths are not trusted, and assistant and unavailable attachments remain basename-only.
Available attachment rows are semantic buttons with keyboard activation, an accessible action and safe filename, standard focus visibility, and the established hover state. Pending and unavailable rows remain announced but disabled. Supported authorized local files use the Preview panel, other authorized local files use the system application only after a click, and HTTP or HTTPS attachments open externally only after a click. One malformed or unavailable attachment cannot suppress prose or sibling attachments. Image-generation completion remains an inline-image experience. It shares transcript coordination and opaque resolved identities with attachment recovery but is not converted into an attachment card.
## Chat Behaviors
- The primary Chat view does not render the legacy Execution Graph.
@@ -80,6 +94,6 @@ An assistant turn has one identity column and one copy action. Copy includes tex
## Validation Anchors
Key tests live in `tests/unit/acp-*.test.*`, `tests/unit/acp-timeline-groups.test.ts`, `tests/unit/chat-question-directory.test.tsx`, and `tests/e2e/chat-acp-inline-timeline.spec.ts`.
Key tests live in `tests/unit/acp-*.test.*`, `tests/unit/acp-timeline-groups.test.ts`, `tests/unit/attachment-access.test.ts`, `tests/unit/chat-question-directory.test.tsx`, `tests/e2e/chat-acp-inline-timeline.spec.ts`, and `tests/e2e/chat-acp-attachments.spec.ts`.
This reference consolidates the former ACP native Chat, Chat polish, turn grouping, and question-directory design documents. Later implementation decisions supersede the original no-optimistic-message rule, the assumption that ACP id always equals Gateway session key, and segment-level assistant copy controls.
harness/reference/acp-generated-media-and-diagnostics.md
+44 -11
View File
@@ -1,31 +1,60 @@
# ACP Generated Media And Diagnostics
Status: current compatibility reference, reviewed 2026-07-13.
Status: current compatibility reference, reviewed 2026-07-15.
Related scenario: `acp-chat-experience`
Related rules: `acp-chat-state-and-history`, `acp-compatibility-content-safety`, `diagnostics-trace-safety`
Related rules: `acp-chat-state-and-history`, `acp-compatibility-content-safety`, `attachment-access-safety`, `diagnostics-trace-safety`
Related tasks: `acp-image-generation-compatibility`, `acp-historical-transcript-supplement`, `acp-debug-trace-channel`
Related tasks: `acp-image-generation-compatibility`, `acp-historical-transcript-supplement`, `acp-media-attachments`, `acp-debug-trace-channel`
## Preferred And Compatibility Paths
Standard ACP image content blocks are preferred and render directly. OpenClaw can instead return an immediate `image_generate` background-task start and publish the final media later through ACP tool metadata, Gateway-visible delivery evidence, or transcript history. ClawX handles that gap with an in-memory compatibility projector; it does not revive the legacy Chat renderer or claim that a synthetic item came from ACP.
Standard ACP image, `resource_link`, and URI-backed `resource` content blocks are preferred and render directly. OpenClaw ACP currently projects assistant text and thought content but can omit assistant media, while Gateway processing removes `MEDIA:` directives from the visible live reply. ClawX handles those gaps through two bounded in-memory compatibility exceptions. Neither revives the legacy Chat renderer nor represents synthetic data as a native ACP event.
## Bounded Transcript Exceptions
This section is the durable rationale referenced by the transcript supplement entry point. The two exceptions are:
1. Image-generation completion with proven `image_generate` context. Trusted structured runtime evidence or approved transcript evidence may restore the completion caption, failure explanation, and media as the existing inline-image experience.
2. General attachment recovery from an explicit line-leading assistant `MEDIA:` directive outside fenced code blocks. This exception does not require image-generation context, but it recovers only the attachment reference, never the surrounding assistant message.
Both exceptions use one bounded transcript fetch coordinator, keep projected state in memory, require exact active session and generation identity, and reject stale or ambiguous evidence. Existing-session load reads at most 1000 recent transcript messages. An ordinary successful live prompt performs one immediate read and one retry 1500 milliseconds later. Only an `image_generate` task recorded for that same live prompt extends the coordinator through bounded backoff while waiting for its completion artifact; accepted completion, invalidation, or retry-window exhaustion stops it. These exceptions must be removed when the distributed OpenClaw ACP adapter emits the equivalent standard content.
Transcript supplementation must not recover or reconstruct ordinary assistant messages, thoughts, tools, plans, permissions, file activity, or a parallel Chat history. Bare paths, inline prose paths, unknown URI schemes, incidental tool paths, and directives inside fenced code blocks are not general attachments.
### Image-Generation Completion
The projector:
1. Detects a recent image-generation task start from ACP tool evidence.
2. Accepts completion candidates only from bounded, trusted fields or approved historical context.
2. Accepts completion text and media only from bounded, trusted fields or approved historical context.
3. Requires the active session and generation to remain unchanged.
4. Resolves local paths or Gateway media through `hostApi.media.thumbnails` in Main.
5. Inserts a marked synthetic assistant segment after the associated tool when possible.
6. Deduplicates repeated evidence and keeps all state in memory.
Accepted live evidence includes structured media fields such as `mediaUrl`, `mediaUrls`, nested `sourceReply` media, assistant media attachments, and OpenClaw ACP tool output explicitly associated with the internal UI sink. Arbitrary prose, unrelated tools, and unscoped local paths are rejected.
Accepted live evidence includes structured media fields such as `mediaUrl`, `mediaUrls`, nested `sourceReply` media, assistant media attachments, OpenClaw ACP tool output explicitly associated with the internal UI sink, and final Gateway assistant replies whose `image_generate:<task-id>:ok|error` run matches the recorded task. For a correlated `message` tool delivery, `sourceReply.text` is the authoritative visible caption or failure explanation. A task-correlated final assistant reply may also provide the authoritative caption or text-only failure explanation. Arbitrary prose, unrelated runs or tools, failed delivery attempts, and unscoped local paths are rejected.
If only some thumbnails resolve, successful previews remain visible with localized partial-failure text. If trusted completion evidence exists but every preview fails, a localized text fallback is allowed. Raw `MEDIA:` paths are never displayed.
When trusted source-reply text exists, it is preserved whether or not media is present. If no source-reply text exists, successful media uses the localized generic caption; partial or failed thumbnail hydration uses the existing localized fallback. Raw `MEDIA:` paths are never displayed.
## Historical Supplement
### Explicit MEDIA Attachments
The general attachment extractor considers normalized assistant roles only. After optional leading whitespace, a whole line must start with the case-insensitive `MEDIA:` token and contain exactly one reference. Single- or double-quoted references may contain spaces and must close with the same quote; unquoted references cannot contain whitespace. One accepted line produces one candidate, and multiple lines retain transcript order. The current source-reference bound is `4096` characters.
Accepted reference forms are absolute POSIX paths, Windows drive paths, `file://` URIs, `~/` paths, paths relative to the registered execution cwd, and HTTP or HTTPS URLs. Relative paths are accepted only when execution cwd is available. Unknown URI schemes, malformed URLs or quotes, empty references, Markdown/list wrappers, inline prose, ordinary bare paths, and wrapped references are rejected. Markdown backtick and tilde fences follow the delimiter character and opening length; all content remains ignored until a valid close with the same delimiter and at least that length. The parser does not render the raw directive or surrounding transcript prose.
Transcript and ACP messages are partitioned by real user boundaries; leading orphan assistant content is ineligible. User text matching removes only the known OpenClaw working-directory envelope, normalizes line endings and surrounding whitespace, and does not use broad fuzzy matching. Because transcript history is a bounded suffix and cross-source message ids are not durable, alignment proceeds newest-to-oldest with the tuple of normalized user text and duplicate occurrence from the tail. A live supplement additionally requires the optimistic ACP user identity and restricts extraction to that current turn. Missing, duplicate, or ambiguous anchors are skipped instead of assigned by ordinal offset or nearest-turn guesswork.
Every asynchronous result is valid only for the same active session key, ACP generation, supplement operation, current attempt, and, for live recovery, user-turn identity. Session changes, new loads or prompts, cancellation/invalidation, and the delayed retry superseding the immediate attempt prevent stale mutation.
Candidates already proven to be image-generation completions remain inline images and are suppressed from the paperclip-card path. General attachments are deduplicated only within a conversation turn by the opaque identity returned after Main authorization. Deduplication spans immediate and delayed reads, repeated history loads, native ACP resources, general compatibility evidence, and generated-image evidence. Native ACP attachment evidence wins regardless of arrival order; an equivalent inline generated image suppresses the paperclip card. An unavailable result does not reserve a resolved identity, so a stable candidate from the delayed read can replace the same synthetic projection and upgrade it to available.
Every standard or compatibility attachment reference is resolved through Main's session-scoped attachment boundary. Main derives the workspace grant from the successful ACP load, checks the exact session and generation, limits local access to the active workspace or verified managed media and staging records, and re-authorizes each resolve, preview read, or open. HTTP and HTTPS references are revalidated before external open. See `harness/reference/acp-attachment-access-control.md`.
Image generation and general attachments share transcript fetch coordination and opaque resolved media identities only. Generated images remain inline; general attachments render as paperclip rows after assistant prose.
## Historical Evidence
After successful `loadSession` for an existing session, the store may call:
@@ -33,9 +62,13 @@ After successful `loadSession` for an existing session, the store may call:
hostApi.sessions.history({ sessionKey, limit: 1000 });
```
A pure extractor scans messages in transcript order. It first records an `image_generate` start from a tool result, then accepts later assistant `MEDIA:` values that look like images. A `MEDIA:` value without preceding task context is rejected. Read failure, no candidate, duplicate evidence, or a stale generation leaves ACP history unchanged.
A pure image-generation extractor scans messages in transcript order. It first records an `image_generate` start from a tool result, then accepts a later internal-UI `message` tool source reply or assistant completion associated with that task. OpenClaw's runtime-generated inter-session completion trigger remains part of the originating user turn rather than starting a new end-user turn. Assistant media captions have their `MEDIA:` directives removed before display, and a task-correlated text-only assistant reply may restore a failure explanation. A message-tool reply or image completion without preceding task context is rejected. Separately, the general attachment extractor may accept explicit assistant `MEDIA:` directives without image-generation context under the restrictions above. Read failure, no accepted evidence, duplicate evidence, or a stale generation leaves the ACP timeline unchanged.
This is the sole transcript-derived Chat supplement. It must not become a general recovery mechanism for missing tool cards, file activity, plans, or messages.
These are the only transcript-derived Chat supplements. They must not become a general recovery mechanism for missing tool cards, file activity, plans, permissions, thoughts, or ordinary messages.
## Rejected Compatibility Alternatives
Main does not manufacture ACP `agent_message_chunk` resource events from transcript evidence because that would misrepresent compatibility data as native protocol replay. The ACP page does not reuse legacy Chat path extraction or rendering because that would restore competing history authorities. Standard-ACP-only behavior is insufficient while the distributed adapter omits assistant media, but the exception remains removable when upstream emits standard resources. Bare-path or broad prose extraction is rejected because false positives would widen the local-file trust surface.
## Trace Channel
@@ -47,6 +80,6 @@ Before storage, Main validates and sanitizes all entries. The sanitizer removes
## Validation Anchors
Key tests are `tests/unit/acp-image-generation-compat.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/unit/acp-trace.test.ts`, `tests/unit/acp-chat-service.test.ts`, and the generated-media cases in `tests/e2e/chat-acp-inline-timeline.spec.ts` and `tests/e2e/chat-run-state-events.spec.ts`.
Key tests are `tests/unit/acp-image-generation-compat.test.ts`, `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/unit/attachment-access.test.ts`, `tests/unit/acp-trace.test.ts`, `tests/unit/acp-chat-service.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts`, and the generated-media cases in `tests/e2e/chat-acp-inline-timeline.spec.ts` and `tests/e2e/chat-run-state-events.spec.ts`.
This reference consolidates the former image-generation completion, debug trace, and historical transcript supplement designs.
harness/reference/openclaw-file-activity.md
+9 -3
View File
@@ -1,12 +1,12 @@
# OpenClaw File Activity
Status: current compatibility and safety reference, reviewed 2026-07-13.
Status: current compatibility and safety reference, reviewed 2026-07-15.
Related scenario: `acp-file-activity`
Related rules: `tool-derived-file-safety`, `session-workspace-authority`
Related rules: `tool-derived-file-safety`, `session-workspace-authority`, `attachment-access-safety`
Related task: `restore-acp-file-activity`
Related tasks: `restore-acp-file-activity`, `acp-media-attachments`
## Semantics And Ownership
@@ -73,6 +73,12 @@ Main independently canonicalizes each read/stat request, checks real paths and n
Tool-derived targets are read-only in-app previews. They never expose system open or reveal because path validation cannot be atomic with OS shell dispatch. Existing trusted workspace-browser targets may retain their established operations.
## Separation From Attachments
File activity and user-facing attachments are separate projections and security boundaries. Incidental paths in tool input or output remain tool-derived evidence: they cannot become attachment cards and retain the preview-only restrictions above. Attachment evidence must instead come from standard ACP resource content, a Main-owned user staging record, or the bounded explicit assistant `MEDIA:` exception documented in `harness/reference/acp-generated-media-and-diagnostics.md#bounded-transcript-exceptions`.
Main establishes an attachment workspace grant only when the ACP session load or creation succeeds. Each attachment resolve, preview read, and system or external open then revalidates the exact session, generation, reference, canonical target, and allowed ownership scope. This attachment-scoped operation can safely support click-initiated system open without weakening the separate rule that incidental tool-derived targets never expose system open or reveal. The complete boundary is documented in `harness/reference/acp-attachment-access-control.md`.
## User Experience And Replay
Each assistant turn shows one file button and one summary per eligible path. Created/modified buttons open current-file Preview; deleted buttons open Changes. Changes is session-scoped, grouped by file, and shows at most one diff editor per turn and path. Empty sessions explicitly state that the session has no file changes.
harness/specs/rules/acp-chat-state-and-history.md
+4 -2
View File
@@ -8,6 +8,8 @@ appliesTo:
- gateway-backend-communication
---
Main owns ACP process, SDK, and routing lifecycle; Renderer owns semantic reduction into an in-memory timeline. Live updates and replay use the same reducer path, stale session generations are ignored, and ClawX does not persist a second ACP ledger or reduced Chat history.
Main owns ACP process, SDK, routing lifecycle, and serialization of operations on the shared ACP connection; Renderer owns semantic reduction into an in-memory timeline. Notifications emitted during `session/load` are returned as one generation-scoped raw batch and reduced in one Renderer state commit. Renderer may temporarily buffer matching host events during the IPC result handoff, while ordinary live prompt updates continue through host events. Permission requests are interactive only for an active prompt. Stale session generations are ignored, and ClawX does not persist a second ACP ledger or reduced Chat history.
ACP replay is the primary history authority. The only approved transcript supplement is best-effort recovery of asynchronous image-generation completions when the same transcript first proves an `image_generate` start. Do not generalize that exception to messages, tool cards, plans, permissions, or file activity.
ACP replay is the primary history authority. The only approved transcript supplements are best-effort recovery of asynchronous image-generation completions with proven `image_generate` context and recovery of explicit line-leading assistant OpenClaw `MEDIA:` attachment directives omitted by ACP. The general attachment exception does not require image-generation context, but it recovers only attachment references. Both exceptions remain marked and in memory; do not generalize them to bare paths, surrounding transcript prose, ordinary messages, tool cards, plans, permissions, thoughts, file activity, or any parallel history.
The shared historical transcript read is limited to the newest `1000` messages. A successful live prompt reads immediately and retries exactly once after `1500 ms`. General attachment alignment treats that history as a suffix and matches normalized user text by duplicate occurrence from the tail; live alignment also requires the current optimistic user identity. Every asynchronous result must retain the same active session, generation, supplement operation and attempt, and live turn where applicable. Unmatched, ambiguous, superseded, or stale work cannot mutate the timeline.
harness/specs/rules/acp-compatibility-content-safety.md
+14
View File
@@ -0,0 +1,14 @@
---
id: acp-compatibility-content-safety
title: ACP Compatibility Content Safety
type: ai-coding-rule
appliesTo:
- acp-chat-experience
- gateway-backend-communication
---
Standard ACP content is authoritative and preferred. A compatibility supplement is allowed only when it is explicitly marked by source, retained in memory, backed by approved structured runtime evidence or explicit assistant transcript evidence, and accompanied by reason-coded diagnostics. Compatibility data must never be represented as a native ACP event.
Approved transcript evidence has two bounded forms: asynchronous image-generation completion with proven image-generation context, including explicit internal-UI `message` tool source replies; and general attachment recovery from whole-line, line-leading assistant OpenClaw `MEDIA:` directives outside fenced code blocks. The general form accepts only the documented local path, `file:`, execution-cwd-relative, HTTP, and HTTPS forms; quoted references may contain spaces, while unquoted references may not. It does not require image-generation context and projects only one ordered attachment reference per directive, never surrounding transcript prose. A trusted image-generation source reply may provide user-facing completion or failure text. Reject malformed or wrapped directives, bare or inline prose paths, unknown URI schemes, incidental tool paths, and unrelated assistant prose.
Compatibility logic must not reconstruct ordinary assistant messages, thoughts, tools, plans, permissions, file activity, or a parallel Chat history. Unmatched or ambiguous evidence is skipped rather than attached by guesswork. Deduplication is turn-scoped and uses only a Main-authorized opaque identity; native ACP resource content wins over equivalent compatibility evidence, generated-image evidence remains inline, and an unavailable result does not block a later available upgrade.
harness/specs/rules/attachment-access-safety.md
+14
View File
@@ -0,0 +1,14 @@
---
id: attachment-access-safety
title: Attachment Access Safety
type: ai-coding-rule
appliesTo:
- acp-chat-experience
- gateway-backend-communication
---
Treat every Renderer attachment URI, metadata field, staging id, transcript id, and source reference as untrusted. ACP load or creation is the only workspace grant boundary; Main commits the canonical workspace and execution cwd only after success, then authorizes every resolve, scoped read, and local or remote open by the exact active session key and generation. Attachment refs, attachment ids, opaque identities, and a prior successful resolve are not bearer capabilities, and later requests cannot provide or replace a workspace root.
Allow local targets only as canonically contained regular files in the active workspace, the exact runtime state/config `media` subtrees, or a Main-owned staging record bound by id. Outgoing media additionally requires exact attachment, URL-session, record-session, optional message-id, and managed original-file binding. Reject traversal, NUL, unknown or unsafe schemes, remote file authorities, credentials, symlink escapes, parent-root expansion, raw staging paths, and unauthorized outgoing records. Sanitize labels, expose only opaque identities, re-resolve before every operation, and perform final file-handle and generation checks for scoped reads.
Attachment previews must use attachment-scoped read operations and cannot fall back to naked-path or general workspace APIs. System or external open is click-initiated and Main-owned. One unavailable or malformed attachment remains isolated from assistant prose and other attachments. See `harness/reference/acp-attachment-access-control.md`; exact TypeScript contracts and current constants remain code-authoritative.
harness/specs/rules/diagnostics-trace-safety.md
+12
View File
@@ -0,0 +1,12 @@
---
id: diagnostics-trace-safety
title: Diagnostics Trace Safety
type: ai-coding-rule
appliesTo:
- acp-chat-experience
- gateway-backend-communication
---
ACP diagnostics are bounded, memory-only, and reason-coded. Trace records may contain bounded counts, source categories, session and generation routing data, operation outcomes, and hashed evidence or attachment identities.
Trace producers and storage sanitization must exclude transcript bodies, user or assistant message content, credentials, tokens, secrets, file contents, raw attachment references, and full or otherwise sensitive filesystem paths. Diagnostic failures remain best effort and must not expose sensitive payloads or change successful Chat behavior.
harness/specs/rules/session-workspace-authority.md
+3 -1
View File
@@ -10,4 +10,6 @@ appliesTo:
OpenClaw ACP cwd is authoritative for a bound Chat session. Global workspace selection applies only to new or unbound sessions, and consumers use one effective workspace for ACP load/prompt, composer state, sidebar grouping, workspace browsing, and file activity. Missing paths surface unavailable state instead of silently changing roots.
Keep `_meta.prefixCwd: true`. Remove the leading working-directory envelope only from automatic titles; never normalize explicit user labels or user-visible transcript content.
The ACP load or new-session operation is the only workspace capability-grant boundary. Main canonicalizes the workspace root and execution cwd, registers them only after a successful load, restores the prior grant after failure, and authorizes later attachment operations by exact session key and generation. Attachment resolve, read, preview, and open requests cannot provide or replace a workspace root and must be re-authorized in Main on every operation. Session or generation replacement revokes the prior grant; attachment refs and prior resolution are not authority.
Keep `_meta.prefixCwd: true`. Remove the leading working-directory envelope only from automatic titles and narrowly defined turn matching; never alter explicit user labels, user-authored content, or user-visible transcript content.
harness/specs/rules/tool-derived-file-safety.md
+4
View File
@@ -4,8 +4,12 @@ title: Tool-Derived File Safety
type: ai-coding-rule
appliesTo:
- acp-file-activity
- acp-chat-experience
- gateway-backend-communication
---
Treat file-tool paths as untrusted. Renderer must enforce lexical workspace containment before projection, and Main must independently enforce canonical and symlink-safe containment for every scoped read/stat operation. Tool-derived targets are read-only in-app previews and expose no system open or reveal action.
File activity remains a record of completed canonical OpenClaw `write`, `edit`, and `apply_patch` inputs. It must not claim to be a verified disk or Git diff, scan the workspace, infer shell effects, or persist a separate ledger.
Incidental paths found in tool input or output remain tool-derived preview evidence and must not become user-facing attachments. Only explicit attachment evidence from standard ACP resource content, Main-owned user staging records, or an approved whole-line assistant OpenClaw `MEDIA:` compatibility directive may enter the attachment pipeline. Main must validate every resolve, attachment-scoped preview read, and click-initiated system or external open through the exact ACP session and generation grant; prior resolution, opaque identity, and Renderer-supplied roots are not authorization. See `harness/reference/acp-attachment-access-control.md` for the separate attachment boundary.
harness/specs/rules/ui-i18n-design-tokens.md
+15
View File
@@ -0,0 +1,15 @@
---
id: ui-i18n-design-tokens
title: UI Internationalization And Design Tokens
type: ai-coding-rule
appliesTo:
- acp-chat-experience
- acp-file-activity
- gateway-backend-communication
---
Route every new user-visible string through `react-i18next` with matching English, Chinese, Japanese, and Russian locale coverage. Do not hardcode display text in pages or components.
Use the semantic tokens and substitutions documented in `src/styles/globals.css`: raised cards and panels use `bg-surface-modal`, recessed inputs and code surfaces use `bg-surface-input`, selected state uses `bg-black/5 dark:bg-white/10`, hover state uses `hover:bg-black/5 dark:hover:bg-white/5`, status colors pair a light `-700` shade with dark `-400`, and page H1/H2 headings use `font-serif font-normal tracking-tight`. Do not add arbitrary colors or redundant dark surface companions when a named token exists.
Interactive rows use semantic controls, keyboard activation, accessible names, visible focus styling, and disabled semantics where applicable. User-visible labels must not expose sensitive absolute host paths. The only attachment-path exception is Main-owned source metadata for a file the user explicitly selected, shown back on that user's own message; assistant, tool-derived, ACP-provided, and unavailable attachment labels remain basename-only.
harness/specs/scenarios/acp-chat-experience.md
+16 -3
View File
@@ -4,14 +4,24 @@ title: ACP Chat Experience
type: user-visible-flow
ownedPaths:
- shared/acp-chat/**
- shared/host-api/contract.ts
- shared/file-preview/**
- electron/services/acp-chat-service.ts
- electron/services/acp-session-access-registry.ts
- electron/services/acp-trace.ts
- electron/services/attachment-access.ts
- electron/services/files-api.ts
- src/lib/acp/**
- src/lib/file-preview-client.ts
- src/lib/file-preview-capabilities.ts
- src/lib/generated-files.ts
- src/components/file-preview/**
- src/stores/acp-chat-session.ts
- src/pages/Chat/**
- tests/unit/acp-*.test.ts
- tests/unit/acp-*.test.tsx
- tests/e2e/chat-acp-inline-timeline.spec.ts
- tests/e2e/chat-acp-attachments.spec.ts
- tests/e2e/chat-run-state-events.spec.ts
requiredProfiles:
- fast
@@ -24,14 +34,17 @@ requiredRules:
- renderer-main-boundary
- acp-chat-state-and-history
- acp-compatibility-content-safety
- attachment-access-safety
- diagnostics-trace-safety
- session-workspace-authority
- tool-derived-file-safety
- ui-i18n-design-tokens
- comms-regression
- docs-sync
---
ACP Chat covers session load, prompt, cancel, permission, replay, timeline reduction, assistant-turn presentation, generated-media compatibility, and Chat-specific diagnostics.
ACP Chat covers session load, prompt, cancel, permission, replay, timeline reduction, assistant-turn presentation, standard ACP attachments, bounded generated-media and OpenClaw MEDIA compatibility, and Chat-specific diagnostics.
Main owns ACP transport and routing. Renderer owns the in-memory timeline and display grouping. ACP replay is authoritative except for the approved image-generation transcript supplement. Standard ACP content remains preferred over compatibility projections.
Main owns ACP transport, routing, transcript retrieval, workspace grants, and session/generation-scoped attachment authorization. Renderer owns the in-memory timeline, bounded compatibility alignment, attachment presentation, and display grouping, including user-image thumbnails and user-selected source-path labels. ACP replay is authoritative except for the approved image-generation completion and explicit line-leading assistant OpenClaw `MEDIA:` attachment supplements. Standard ACP content remains preferred over compatibility projections, and incidental tool paths never enter the attachment pipeline.
The durable architecture, exceptions, and validation anchors are documented in `harness/reference/acp-chat.md` and `harness/reference/acp-generated-media-and-diagnostics.md`.
The durable architecture, exceptions, access boundary, file-activity separation, and validation anchors are documented in `harness/reference/acp-chat.md`, `harness/reference/acp-generated-media-and-diagnostics.md`, `harness/reference/acp-attachment-access-control.md`, and `harness/reference/openclaw-file-activity.md`.
harness/specs/tasks/acp-image-generation-compatibility.md
+7 -3
View File
@@ -3,13 +3,14 @@ id: acp-image-generation-compatibility
title: Project OpenClaw image-generation completions into ACP Chat
scenario: gateway-backend-communication
taskType: runtime-bridge
intent: Restore generated image display in ClawX ACP Chat without modifying OpenClaw by projecting trusted Gateway media delivery evidence into the in-memory ACP timeline.
intent: Restore OpenClaw image-generation completion replies in ClawX ACP Chat without modifying OpenClaw by projecting trusted ACP and Gateway delivery evidence into the in-memory ACP timeline.
touchedAreas:
- harness/specs/tasks/acp-image-generation-compatibility.md
- harness/reference/acp-generated-media-and-diagnostics.md
- harness/specs/scenarios/acp-chat-experience.md
- harness/specs/rules/acp-compatibility-content-safety.md
- src/lib/acp/image-generation-compat.ts
- src/lib/acp/openclaw-media-compat.ts
- src/lib/acp/reducer.ts
- src/lib/acp/timeline-types.ts
- src/stores/acp-chat-session.ts
@@ -26,12 +27,14 @@ touchedAreas:
- README.ja-JP.md
expectedUserBehavior:
- ACP Chat first shows the image_generate background task start tool result.
- When OpenClaw later exposes structured generated-image media through Gateway host events, ClawX appends a new assistant reply containing the hydrated image preview.
- When OpenClaw later exposes a trusted internal-UI source reply through ACP or Gateway host events, ClawX preserves its exact user-facing text instead of replacing it with a generic caption.
- Successful replies include the hydrated image preview, while text-only generation failures remain visible as assistant replies.
- Arbitrary local paths and generic MEDIA: prose without approved image-generation context are not rendered as images.
- Renderer continues to use host-api/host-events and does not call Gateway HTTP directly.
requiredProfiles:
- fast
- comms
- e2e
requiredRules:
- renderer-main-boundary
- backend-communication-boundary
@@ -50,7 +53,8 @@ requiredTests:
- pnpm run comms:compare
acceptance:
- ClawX records recent image_generate background task context from ACP tool output.
- ClawX accepts only structured Gateway media delivery evidence that matches the active ACP session and recent image-generation context.
- ClawX accepts only trusted ACP or Gateway completion evidence that matches the active ACP session and recent image-generation context.
- Internal-UI sourceReply text is authoritative for both successful media replies and text-only failure replies.
- ClawX hydrates previews through hostApi.media.thumbnails before rendering images.
- Duplicate completion records do not create duplicate assistant image replies.
- Stale preview resolution does not append to a different active session or generation.
harness/specs/tasks/acp-media-attachments.md
+212
View File
@@ -0,0 +1,212 @@
---
id: acp-media-attachments
title: Render ACP resources and bounded OpenClaw MEDIA attachments
scenario: gateway-backend-communication
taskType: runtime-bridge
intent: Render standard ACP resources and recover only explicit OpenClaw MEDIA attachments omitted by the distributed ACP adapter through a bounded transcript compatibility projection.
touchedAreas:
- package.json
- docs/specs/2026-07-14-acp-media-attachments-design.md
- docs/plans/2026-07-14-acp-media-attachments.md
- docs/plans/2026-07-15-harness-spec-consolidation.md
- harness/specs/tasks/acp-media-attachments.md
- harness/specs/tasks/fix-acp-history-load-races.md
- harness/specs/scenarios/acp-chat-experience.md
- harness/specs/rules/acp-chat-state-and-history.md
- harness/specs/rules/acp-compatibility-content-safety.md
- harness/specs/rules/attachment-access-safety.md
- harness/specs/rules/diagnostics-trace-safety.md
- harness/specs/rules/session-workspace-authority.md
- harness/specs/rules/tool-derived-file-safety.md
- harness/specs/rules/ui-i18n-design-tokens.md
- harness/reference/acp-chat.md
- harness/reference/acp-attachment-access-control.md
- harness/reference/acp-generated-media-and-diagnostics.md
- harness/reference/openclaw-file-activity.md
- shared/acp-chat/types.ts
- shared/host-api/contract.ts
- shared/file-preview/limits.ts
- electron/services/acp-session-access-registry.ts
- electron/services/acp-chat-service.ts
- electron/services/chat-api.ts
- electron/services/attachment-access.ts
- electron/services/files-api.ts
- electron/services/acp-trace.ts
- electron/services/media-api.ts
- electron/services/sessions-api.ts
- electron/main/ipc-handlers.ts
- electron/utils/paths.ts
- src/lib/host-api.ts
- src/lib/file-preview-client.ts
- src/lib/file-preview-capabilities.ts
- src/lib/generated-files.ts
- src/lib/acp/attachments.ts
- src/lib/acp/timeline-types.ts
- src/lib/acp/content-blocks.ts
- src/lib/acp/reducer.ts
- src/lib/acp/image-generation-compat.ts
- src/lib/acp/openclaw-media-compat.ts
- src/lib/acp/transcript-supplement.ts
- src/lib/acp/timeline-groups.ts
- src/stores/acp-chat-session.ts
- src/pages/Chat/index.tsx
- src/pages/Chat/ChatInput.tsx
- src/pages/Chat/AcpAttachmentPart.tsx
- src/pages/Chat/AcpMessageSegment.tsx
- src/pages/Chat/AcpAssistantTurn.tsx
- src/pages/Chat/AcpTimeline.tsx
- src/pages/Chat/AcpToolCallCard.tsx
- src/components/file-preview/ArtifactPanel.tsx
- src/components/file-preview/types.ts
- src/components/file-preview/build-preview-target.ts
- src/components/file-preview/FilePreviewBody.tsx
- src/components/file-preview/ImageViewer.tsx
- src/components/file-preview/PdfViewer.tsx
- src/components/file-preview/SheetViewer.tsx
- src/components/file-preview/HtmlPreview.tsx
- src/styles/globals.css
- shared/i18n/locales/en/chat.json
- shared/i18n/locales/zh/chat.json
- shared/i18n/locales/ja/chat.json
- shared/i18n/locales/ru/chat.json
- tests/unit/harness-specs.test.ts
- tests/unit/chat-input.test.tsx
- tests/unit/acp-session-access-registry.test.ts
- tests/unit/acp-host-contract.test.ts
- tests/unit/acp-chat-service.test.ts
- tests/unit/chat-acp-page.test.tsx
- tests/unit/chat-page-execution-graph.test.tsx
- tests/unit/attachment-access.test.ts
- tests/unit/files-api-workspace.test.ts
- tests/unit/media-api.test.ts
- tests/unit/sessions-api-workspace.test.ts
- tests/unit/acp-trace.test.ts
- tests/unit/host-api-facade.test.ts
- tests/unit/host-services.test.ts
- tests/unit/acp-reducer.test.ts
- tests/unit/acp-chat-store.test.ts
- tests/unit/acp-chat-components.test.tsx
- tests/unit/artifact-panel.test.tsx
- tests/unit/file-preview-body.test.tsx
- tests/unit/rich-file-viewers.test.tsx
- tests/unit/generated-files.test.ts
- tests/unit/acp-media-attachments.test.ts
- tests/unit/acp-image-generation-compat.test.ts
- tests/unit/acp-timeline-groups.test.ts
- tests/e2e/chat-acp-attachments.spec.ts
- tests/e2e/chat-acp-inline-timeline.spec.ts
- tests/e2e/chat-assistant-markdown-plain.spec.ts
- tests/e2e/chat-code-block-wrap.spec.ts
- tests/e2e/chat-history-startup-retry.spec.ts
- tests/e2e/chat-latex-rendering.spec.ts
- tests/e2e/chat-new-session-date.spec.ts
- tests/e2e/chat-question-directory.spec.ts
- tests/e2e/chat-run-state-events.spec.ts
- tests/e2e/chat-scroll-pin-bottom.spec.ts
- tests/e2e/chat-scroll-to-latest.spec.ts
- tests/e2e/chat-table-header-light.spec.ts
- tests/e2e/chat-task-visualizer.spec.ts
- tests/e2e/chat-workspace-context.spec.ts
- tests/e2e/cron-run-live-status.spec.ts
- tests/e2e/fixtures/electron.ts
- tests/e2e/gateway-lifecycle.spec.ts
- README.md
- README.zh-CN.md
- README.ja-JP.md
expectedUserBehavior:
- Standard ACP resource_link and URI-backed resource content renders as paperclip attachment cards.
- Explicit assistant OpenClaw MEDIA directives omitted by ACP are recovered for live completions and historical session loads without displaying the raw directive.
- Attachment rows render after assistant prose and preserve declaration order.
- User image attachments render as thumbnails with a filename overlay on hover.
- User non-image attachments show their source path after the filename, omit MIME text, and truncate long paths.
- Supported authorized local files open in the right-side Preview panel after a user click.
- Unsupported authorized local files open with the operating system default application after a user click.
- HTTP and HTTPS attachments open externally after a user click.
- Arbitrary paths in ordinary prose do not become attachments.
- Unavailable or unauthorized references cannot be previewed or opened and do not suppress assistant prose or other attachments.
requiredProfiles:
- fast
- comms
- e2e
requiredRules:
- renderer-main-boundary
- backend-communication-boundary
- api-client-transport-policy
- host-api-fallback-policy
- acp-chat-state-and-history
- acp-compatibility-content-safety
- attachment-access-safety
- diagnostics-trace-safety
- session-workspace-authority
- tool-derived-file-safety
- ui-i18n-design-tokens
- comms-regression
- docs-sync
requiredTests:
- pnpm exec vitest run tests/unit/harness-specs.test.ts tests/unit/acp-session-access-registry.test.ts tests/unit/attachment-access.test.ts tests/unit/acp-host-contract.test.ts tests/unit/acp-chat-service.test.ts tests/unit/acp-reducer.test.ts tests/unit/acp-chat-store.test.ts tests/unit/acp-chat-components.test.tsx tests/unit/acp-media-attachments.test.ts tests/unit/acp-image-generation-compat.test.ts tests/unit/acp-timeline-groups.test.ts tests/unit/acp-trace.test.ts tests/unit/file-preview-body.test.tsx tests/unit/rich-file-viewers.test.tsx tests/unit/generated-files.test.ts tests/unit/media-api.test.ts tests/unit/files-api-workspace.test.ts tests/unit/sessions-api-workspace.test.ts tests/unit/host-api-facade.test.ts tests/unit/host-services.test.ts tests/unit/chat-acp-page.test.tsx
- pnpm run typecheck
- pnpm run lint:check
- pnpm run build:vite
- pnpm exec playwright test tests/e2e/chat-acp-attachments.spec.ts
- pnpm exec playwright test tests/e2e/chat-run-state-events.spec.ts
- pnpm run comms:replay
- pnpm run comms:compare
- pnpm harness validate --spec harness/specs/tasks/acp-media-attachments.md
- pnpm harness run --spec harness/specs/tasks/acp-media-attachments.md
- pnpm run harness:ci
acceptance:
- A standard ACP resource_link or URI-backed resource renders an actionable paperclip attachment card.
- User image attachments render as thumbnails with the filename revealed by a hover overlay.
- User non-image attachments render the filename followed by a muted, truncating source path and no MIME label.
- The reported OpenClaw MEDIA directive for budget_sample.xlsx renders an attachment in ACP Chat even though OpenClaw ACP emits no resource block.
- The raw MEDIA directive is not displayed.
- The attachment renders after assistant prose and preserves declaration order.
- Supported files open in the right-side Preview panel.
- Unsupported authorized local files open with the system default application after a click.
- HTTP and HTTPS attachments open externally after a click.
- Arbitrary prose paths do not become attachments.
- Local references outside the workspace and verified managed roots cannot be previewed or opened.
- Live and historical paths deduplicate and reject stale session or generation results.
- Native ACP resources take precedence over transcript compatibility evidence.
- Attachment access remains bound to Main-owned session, generation, ownership, and outgoing-record authority on every operation.
- Attachment rows use semantic controls with safe accessible labels, keyboard activation, and disabled unavailable states.
- The implementation contains the required compatibility rationale comment and links it to durable architecture documentation.
- No OpenClaw source or distributed package is modified.
- No legacy Chat renderer, direct Renderer IPC, or direct Gateway HTTP request is introduced.
- Unit, Electron E2E, typecheck, harness, and required communication regression checks pass.
docs:
required: true
---
## Scope
Standard ACP resource content is the preferred attachment source. The OpenClaw transcript path is a bounded compatibility exception for explicit assistant `MEDIA:` directives that the distributed ACP adapter omits; it is not a second Chat history source.
## Out Of Scope
- Modifying OpenClaw or its distributed package.
- Reconstructing ordinary messages, tools, plans, permissions, thoughts, or file activity from transcripts.
- Extracting bare paths or inline paths from ordinary assistant prose.
- Persisting a synthetic ACP attachment ledger or compatibility cache.
## Acceptance Traceability
| Acceptance behavior | Test or durable rule |
| --- | --- |
| Standard ACP resources render actionable cards | `tests/unit/acp-reducer.test.ts`, `tests/unit/acp-chat-components.test.tsx`, `tests/e2e/chat-acp-attachments.spec.ts` |
| Explicit OpenClaw `MEDIA:` recovery and hidden raw directives | `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-chat-store.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts` |
| Explicit parser grammar rejects fenced, wrapped, inline, malformed, unknown-scheme, and overlong values | `tests/unit/acp-media-attachments.test.ts`, `acp-compatibility-content-safety` |
| Transcript suffix alignment uses normalized user text and occurrence from the tail without guessing | `tests/unit/acp-media-attachments.test.ts`, `tests/unit/acp-chat-store.test.ts`, `acp-chat-state-and-history` |
| Body-first ordering and declaration order | `tests/unit/acp-timeline-groups.test.ts`, `tests/unit/acp-chat-components.test.tsx`, `tests/e2e/chat-acp-attachments.spec.ts` |
| User thumbnail, filename overlay, and Main-owned source-path presentation | `tests/unit/acp-chat-components.test.tsx`, `tests/unit/acp-reducer.test.ts`, `tests/unit/attachment-access.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts`, `ui-i18n-design-tokens` |
| Preview, local system open, and remote external open routing | `tests/unit/file-preview-body.test.tsx`, `tests/unit/rich-file-viewers.test.tsx`, `tests/unit/attachment-access.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts` |
| Main grant lifecycle and exact session/generation revocation | `tests/unit/acp-session-access-registry.test.ts`, `tests/unit/acp-chat-service.test.ts`, `tests/unit/attachment-access.test.ts`, `attachment-access-safety` |
| URI hardening, allowed roots, staging ownership, and outgoing-record binding | `tests/unit/attachment-access.test.ts`, `attachment-access-safety` |
| Attachment previews use scoped reads without naked-path or workspace fallback | `tests/unit/file-preview-body.test.tsx`, `tests/unit/rich-file-viewers.test.tsx`, `tests/unit/artifact-panel.test.tsx`, `attachment-access-safety` |
| Semantic controls, safe labels, keyboard activation, and disabled unavailable state | `tests/unit/acp-chat-components.test.tsx`, `tests/unit/attachment-access.test.ts`, `tests/e2e/chat-acp-attachments.spec.ts`, `ui-i18n-design-tokens` |
| Bare prose paths are rejected | `tests/unit/acp-media-attachments.test.ts`, `acp-compatibility-content-safety` |
| Outside-root and stale session/generation access is rejected | `tests/unit/attachment-access.test.ts`, `tests/unit/acp-session-access-registry.test.ts`, `attachment-access-safety`, `session-workspace-authority`, `tool-derived-file-safety` |
| Turn-scoped live/history dedupe, unavailable upgrade, and native ACP precedence | `tests/unit/acp-chat-store.test.ts`, `tests/unit/acp-media-attachments.test.ts`, `acp-chat-state-and-history`, `acp-compatibility-content-safety` |
| Compatibility rationale remains marked and bounded | `harness/reference/acp-generated-media-and-diagnostics.md#bounded-transcript-exceptions`, `acp-compatibility-content-safety` |
| No OpenClaw, legacy Renderer, direct IPC, or direct Gateway regression | `renderer-main-boundary`, `backend-communication-boundary`, `acp-chat-state-and-history`, harness validation |
| Complete regression validation passes | `requiredTests` above and `comms-regression` |
harness/specs/tasks/fix-acp-history-load-races.md
+59
View File
@@ -0,0 +1,59 @@
---
id: fix-acp-history-load-races
title: Make ACP history loading atomic and race-safe
scenario: gateway-backend-communication
taskType: runtime-bridge
intent: Prevent long ACP histories from visibly replaying and losing their prefix when users switch rapidly between sessions.
touchedAreas:
- harness/specs/tasks/fix-acp-history-load-races.md
- package.json
- harness/reference/acp-chat.md
- harness/specs/scenarios/acp-chat-experience.md
- harness/specs/rules/acp-chat-state-and-history.md
- shared/acp-chat/types.ts
- electron/services/acp-chat-service.ts
- src/stores/acp-chat-session.ts
- src/pages/Chat/index.tsx
- src/pages/Chat/AcpToolCallCard.tsx
- tests/unit/acp-chat-service.test.ts
- tests/unit/acp-chat-store.test.ts
- tests/unit/acp-chat-components.test.tsx
- tests/unit/chat-acp-page.test.tsx
- tests/e2e/chat-acp-inline-timeline.spec.ts
expectedUserBehavior:
- Opening a long historical conversation shows a loading state followed by the complete timeline, without exposing chunk-by-chunk replay.
- Switching to a new conversation and quickly returning preserves the complete historical timeline.
- Overlapping requests for the same session cannot mix replay generations or duplicate history.
requiredProfiles:
- fast
- comms
requiredRules:
- renderer-main-boundary
- backend-communication-boundary
- host-api-fallback-policy
- host-events-fallback-policy
- acp-chat-state-and-history
- comms-regression
- docs-sync
requiredTests:
- pnpm exec vitest run tests/unit/acp-chat-service.test.ts tests/unit/acp-chat-store.test.ts
- pnpm exec playwright test tests/e2e/chat-acp-inline-timeline.spec.ts
- pnpm run typecheck
- pnpm run comms:replay
- pnpm run comms:compare
acceptance:
- Main serializes ACP session load operations on the shared ACP connection.
- Main returns notifications emitted during session load as one generation-scoped raw batch instead of forwarding them incrementally.
- Renderer reduces a completed load batch and exposes the resulting timeline in one state commit.
- Renderer merges generation-matching updates from the IPC result handoff window without exposing partial state.
- Permission requests outside an active prompt are cancelled and cannot leave an invisible waiter during load handoff.
- Renderer operation identity does not advance the Main-owned ACP generation or reject valid replay events after a local-only new conversation.
- Stale load completions and stale session updates remain ignored.
- No persistent ACP history cache, second replay ledger, or OpenClaw source change is introduced.
docs:
required: true
---
## Scope
This task changes only the transaction boundary around ACP `session/load`. Live `session/prompt` notifications continue to use the existing host event path.
shared/acp-chat/types.ts
+4
View File
@@ -12,12 +12,14 @@ export type AcpSessionKeyPayload = {
};
export type AcpChatLoadPayload = AcpSessionKeyPayload & {
workspaceRoot: string;
cwd: string;
createIfMissing?: boolean;
};
export type AcpPromptMediaItem = {
filePath: string;
stagingId: string;
fileName?: string;
mimeType?: string;
};
@@ -40,6 +42,8 @@ export type AcpChatOperationResult = {
success: boolean;
error?: string;
generation?: number;
/** Raw notifications collected while session/load is in progress. */
sessionUpdates?: AcpSessionUpdateEnvelope[];
};
export type AcpSessionUpdateEnvelope = {
shared/file-preview/limits.ts
+2
View File
@@ -0,0 +1,2 @@
export const FILE_PREVIEW_MAX_TEXT_BYTES = 2 * 1024 * 1024;
export const FILE_PREVIEW_MAX_BINARY_BYTES = 50 * 1024 * 1024;
shared/host-api/contract.ts
+57
View File
@@ -437,6 +437,57 @@ export type WorkspaceContextInput = {
executionCwd: string;
};
export type FileReadBinaryOptions = { maxBytes?: number };
export type AttachmentSourceRef = {
sessionKey: string;
generation: number;
uri: string;
stagingId?: string;
transcriptMessageId?: string;
};
export type AttachmentFileRef = AttachmentSourceRef;
export type AttachmentRemoteRef = AttachmentSourceRef;
export type AttachmentAccessError =
| 'invalidReference'
| 'staleSession'
| 'outsideAllowedRoots'
| 'unavailable'
| 'notFile'
| 'unsafeUrl'
| 'operationFailed';
export type AttachmentReadError = AttachmentAccessError | 'tooLarge' | 'binary';
export type ResolveAttachmentPayload = {
ref: AttachmentSourceRef;
name?: string;
mimeType?: string;
size?: number;
};
export type ResolveAttachmentResult =
| {
ok: true;
identity: string;
displayName: string;
displayPath?: string;
mimeType: string;
size: number;
target:
| {
kind: 'local';
scope: 'workspace' | 'openclaw-media' | 'staging';
ref: AttachmentFileRef;
}
| { kind: 'remote'; ref: AttachmentRemoteRef; url: string };
}
| { ok: false; displayName: string; error: AttachmentAccessError };
export type ReadAttachmentTextResult =
| { ok: true; content: string; mimeType: string; size: number; readOnly: true }
| { ok: false; error: AttachmentReadError; size?: number };
export type ReadAttachmentBinaryPayload = { ref: AttachmentFileRef; maxBytes?: number };
export type ReadAttachmentBinaryResult =
| { ok: true; data: Uint8Array; mimeType: string; size: number; readOnly: true }
| { ok: false; error: AttachmentReadError; size?: number };
export type OpenAttachmentResult =
| { ok: true }
| { ok: false; error: AttachmentAccessError };
export type FilePreviewTreeOptions = {
maxDepth?: number;
maxNodes?: number;
@@ -514,6 +565,8 @@ export type FileListTreeResult = {
export type MediaThumbnailEntry = {
filePath?: string;
gatewayUrl?: string;
attachmentFileRef?: AttachmentFileRef;
key?: string;
mimeType?: string;
};
export type MediaThumbnailsPayload = { paths: MediaThumbnailEntry[] };
@@ -850,6 +903,10 @@ export type HostApiContract = {
readWorkspaceText: (ref: WorkspaceFileRef) => Promise<ReadTextFileResult>;
readWorkspaceBinary: (input: WorkspaceFileRef & { maxBytes?: number }) => Promise<ReadBinaryFileResult>;
statWorkspaceFile: (ref: WorkspaceFileRef) => Promise<StatFileResult>;
resolveAttachment: (payload: ResolveAttachmentPayload) => ResolveAttachmentResult;
readAttachmentText: (ref: AttachmentFileRef) => ReadAttachmentTextResult;
readAttachmentBinary: (payload: ReadAttachmentBinaryPayload) => ReadAttachmentBinaryResult;
openAttachment: (ref: AttachmentSourceRef) => OpenAttachmentResult;
};
media: {
thumbnails: (payload: MediaThumbnailsPayload) => MediaThumbnailResult;
shared/i18n/locales/en/chat.json
+10 -2
View File
@@ -60,7 +60,15 @@
"saveImage": "Save image as",
"imageSaved": "Image saved",
"copy": "Copy response",
"copied": "Copied"
"copied": "Copied",
"attachment": {
"loading": "Loading attachment",
"unavailable": "Attachment unavailable",
"open": "Open {{name}}",
"preview": "Preview {{name}}",
"mimeSize": "{{mimeType}}, {{size}}",
"openFailed": "Could not open attachment"
}
},
"artifactPanel": {
"tabs": {
@@ -222,7 +230,7 @@
"gatewayDisconnectedPlaceholder": "Gateway not connected...",
"send": "Send",
"stop": "Stop",
"working": "AI is working",
"thinking": "Thinking…",
"gatewayConnected": "connected",
"gatewayStarting": "starting",
"gatewayStatus": "gateway {{state}} | port: {{port}} | pid: {{pid}}",
shared/i18n/locales/ja/chat.json
+10 -2
View File
@@ -60,7 +60,15 @@
"saveImage": "画像を別名で保存",
"imageSaved": "画像を保存しました",
"copy": "返信をコピー",
"copied": "コピーしました"
"copied": "コピーしました",
"attachment": {
"loading": "添付ファイルを読み込み中",
"unavailable": "添付ファイルを利用できません",
"open": "{{name}} を開く",
"preview": "{{name}} をプレビュー",
"mimeSize": "{{mimeType}}, {{size}}",
"openFailed": "添付ファイルを開けませんでした"
}
},
"artifactPanel": {
"tabs": {
@@ -222,7 +230,7 @@
"gatewayDisconnectedPlaceholder": "ゲートウェイ未接続...",
"send": "送信",
"stop": "停止",
"working": "AI が処理中です",
"thinking": "考え中…",
"gatewayConnected": "接続済み",
"gatewayStarting": "起動中",
"gatewayStatus": "ゲートウェイ {{state}} | ポート: {{port}} | PID: {{pid}}",
shared/i18n/locales/ru/chat.json
+10 -2
View File
@@ -60,7 +60,15 @@
"saveImage": "Сохранить изображение как",
"imageSaved": "Изображение сохранено",
"copy": "Копировать ответ",
"copied": "Скопировано"
"copied": "Скопировано",
"attachment": {
"loading": "Загрузка вложения",
"unavailable": "Вложение недоступно",
"open": "Открыть {{name}}",
"preview": "Просмотреть {{name}}",
"mimeSize": "{{mimeType}}, {{size}}",
"openFailed": "Не удалось открыть вложение"
}
},
"artifactPanel": {
"tabs": {
@@ -222,7 +230,7 @@
"gatewayDisconnectedPlaceholder": "Шлюз не подключён...",
"send": "Отправить",
"stop": "Остановить",
"working": "AI работает",
"thinking": "Думаю…",
"gatewayConnected": "подключён",
"gatewayStarting": "запуск",
"gatewayStatus": "шлюз {{state}} | порт: {{port}} | PID: {{pid}}",
shared/i18n/locales/zh/chat.json
+10 -2
View File
@@ -60,7 +60,15 @@
"saveImage": "另存为图片",
"imageSaved": "图片已保存",
"copy": "复制回复",
"copied": "已复制"
"copied": "已复制",
"attachment": {
"loading": "正在加载附件",
"unavailable": "附件不可用",
"open": "打开 {{name}}",
"preview": "预览 {{name}}",
"mimeSize": "{{mimeType}}, {{size}}",
"openFailed": "无法打开附件"
}
},
"artifactPanel": {
"tabs": {
@@ -222,7 +230,7 @@
"gatewayDisconnectedPlaceholder": "网关未连接...",
"send": "发送",
"stop": "停止",
"working": "AI 正在工作",
"thinking": "思考中…",
"gatewayConnected": "已连接",
"gatewayStarting": "启动中",
"gatewayStatus": "网关{{state}} ｜ 端口：{{port}} ｜ 进程 ID：{{pid}}",
src/components/file-preview/ArtifactPanel.tsx
+2 -1
View File
@@ -52,13 +52,14 @@ export function ArtifactPanel({ fileGroups, uniqueFileCount, agent, workspacePat
const focusedChange = useArtifactPanel((s) => s.focusedChange);
const close = useArtifactPanel((s) => s.close);
const richFocusedFile = !!focusedFile
&& !focusedFile.attachmentFileRef
&& !focusedFile.workspaceFileRef
&& supportsRichDocumentPreview(focusedFile.ext);
const requestedTab = !WORKSPACE_BROWSER_ENABLED && tab === 'browser' ? 'changes' : tab;
const visibleTab = requestedTab;
const handleRevealFocusedFile = () => {
if (!focusedFile) return;
if (!focusedFile || focusedFile.attachmentFileRef) return;
hostApi.shell.showItemInFolder(focusedFile.filePath).catch(() => {
toast.error(t('filePreview.errors.openInFinderFailed', 'Could not reveal in file manager'));
});
src/components/file-preview/FilePreviewBody.tsx
+41 -31
View File
@@ -20,6 +20,7 @@
* here so callers only pass a `FilePreviewTarget` and a `readOnly` flag.
*/
import { lazy, Suspense, useCallback, useEffect, useMemo, useState } from 'react';
import { FILE_PREVIEW_MAX_BINARY_BYTES } from '@shared/file-preview/limits';
import { FolderOpen, Save, ShieldAlert, Undo2 } from 'lucide-react';
import { toast } from 'sonner';
import { useTranslation } from 'react-i18next';
@@ -29,6 +30,7 @@ import { LoadingSpinner } from '@/components/common/LoadingSpinner';
import { cn } from '@/lib/utils';
import {
readTextFile,
readAttachmentText,
readWorkspaceText,
statFile,
statWorkspaceFile,
@@ -37,12 +39,9 @@ import {
import { getFilePreviewTargetIdentity, type FilePreviewTarget } from './types';
import {
isHtmlPreviewExt,
isPdfPreviewExt,
isSheetPreviewExt,
supportsInlineDiff,
supportsInlineDocumentPreview,
supportsRichDocumentPreview,
} from '@/lib/generated-files';
import { filePreviewKind, richFilePreviewKind } from '@/lib/file-preview-capabilities';
import { FilePreviewIcon } from './file-card-utils';
import { formatFileSize } from './format';
import {
@@ -59,12 +58,6 @@ const MonacoDiffViewerLazy = lazy(() => import('./MonacoDiffViewer'));
const PdfViewerLazy = lazy(() => import('./PdfViewer'));
const SheetViewerLazy = lazy(() => import('./SheetViewer'));
/**
* Files past this ceiling get the direct-open fallback instead of the
* inline PDF / spreadsheet viewer. Mirrors the main-process binary cap.
*/
const RICH_PREVIEW_MAX_BYTES = 50 * 1024 * 1024;
/**
* Tab set for the body.
*
@@ -106,20 +99,18 @@ function tabsForFile(file: FilePreviewTarget, mode: FilePreviewBodyMode): Tab[]
// formats where inline diff is actually supported.
if (mode === 'diff') return supportsInlineDiff(file) ? ['diff'] : [];
const previewKind = filePreviewKind(file);
if (!previewKind) return [];
const richPreview = richFilePreviewKind(file);
const tabs: Tab[] = [];
if (file.contentType === 'document') {
if (!supportsInlineDocumentPreview(file.ext)) {
return [];
}
if (richPreview) {
tabs.push('preview');
} else if (file.contentType === 'document') {
// Markdown / HTML / rich documents: rendered preview first.
tabs.push('preview');
if (isHtmlPreviewExt(file.ext)) {
tabs.push('source');
}
} else if (file.contentType === 'snapshot') {
tabs.push('preview');
} else if (file.contentType === 'video' || file.contentType === 'audio') {
tabs.push('preview');
} else if (file.contentType === 'code') {
tabs.push('source');
} else {
@@ -228,14 +219,15 @@ export function FilePreviewBody({
// Preview / diff modes are read-only by definition — those views are
// for inspecting content, not editing it.
const enforcedReadOnly = readOnly || !!file.workspaceFileRef || mode === 'preview' || mode === 'diff';
const enforcedReadOnly = readOnly || !!file.attachmentFileRef || !!file.workspaceFileRef || mode === 'preview' || mode === 'diff';
const tabs = useMemo(() => tabsForFile(file, mode), [file, mode]);
const unsupportedPreviewFormat = file.contentType === 'document' && !supportsInlineDocumentPreview(file.ext);
const unsupportedPreviewFormat = mode !== 'diff' && filePreviewKind(file) == null;
const unsupportedDiffFormat = mode === 'diff' && !supportsInlineDiff(file);
const richPreview = richFilePreviewKind(file);
// Binary document previews (PDF, spreadsheet) own their own loading
// pipeline — we must not pipe them through `readTextFile` (which would
// reject them as binary) and the diff tab is intentionally hidden.
const isRichDocumentPreview = file.contentType === 'document' && supportsRichDocumentPreview(file.ext);
const isRichDocumentPreview = richPreview === 'pdf' || richPreview === 'sheet';
useEffect(() => {
let cancelled = false;
@@ -255,6 +247,11 @@ export function FilePreviewBody({
if (unsupportedPreviewFormat) {
setState({ identity: loadIdentity, status: 'ready', content: '', readOnly: enforcedReadOnly });
setDraft(null);
if (file.attachmentFileRef) {
return () => {
cancelled = true;
};
}
void (file.workspaceFileRef
? statWorkspaceFile(file.workspaceFileRef)
: statFile(file.filePath))
@@ -275,7 +272,7 @@ export function FilePreviewBody({
// IPC channel; the body just needs to hand off control. For files
// beyond the inline-preview ceiling we keep the existing
// "direct open" fallback so users still have a way out.
if (typeof file.size === 'number' && file.size > RICH_PREVIEW_MAX_BYTES) {
if (typeof file.size === 'number' && file.size > FILE_PREVIEW_MAX_BINARY_BYTES) {
setSize(file.size);
setState({ identity: loadIdentity, status: 'tooLarge', size: file.size });
setDraft(null);
@@ -285,12 +282,18 @@ export function FilePreviewBody({
}
setState({ identity: loadIdentity, status: 'loading' });
setDraft(null);
if (file.attachmentFileRef) {
setState({ identity: loadIdentity, status: 'ready', content: '', readOnly: true });
return () => {
cancelled = true;
};
}
void (file.workspaceFileRef
? statWorkspaceFile(file.workspaceFileRef)
: statFile(file.filePath))
.then((res) => {
if (cancelled) return;
if (res.ok && typeof res.size === 'number' && res.size > RICH_PREVIEW_MAX_BYTES) {
if (res.ok && typeof res.size === 'number' && res.size > FILE_PREVIEW_MAX_BINARY_BYTES) {
setSize(res.size);
setState({ identity: loadIdentity, status: 'tooLarge', size: res.size });
return;
@@ -307,7 +310,7 @@ export function FilePreviewBody({
};
}
if (file.contentType === 'snapshot' || file.contentType === 'video' || file.contentType === 'audio') {
if (richPreview === 'image' || file.contentType === 'video' || file.contentType === 'audio') {
setState({ identity: loadIdentity, status: 'ready', content: '', readOnly: enforcedReadOnly });
setDraft(null);
return () => {
@@ -316,7 +319,9 @@ export function FilePreviewBody({
}
setState({ identity: loadIdentity, status: 'loading' });
(file.workspaceFileRef
(file.attachmentFileRef
? readAttachmentText(file.attachmentFileRef)
: file.workspaceFileRef
? readWorkspaceText(file.workspaceFileRef)
: readTextFile(file.filePath))
.then((res) => {
@@ -358,10 +363,10 @@ export function FilePreviewBody({
return () => {
cancelled = true;
};
}, [file, loadIdentity, enforcedReadOnly, mode, tabs, unsupportedPreviewFormat, isRichDocumentPreview]);
}, [file, loadIdentity, enforcedReadOnly, mode, tabs, unsupportedPreviewFormat, isRichDocumentPreview, richPreview]);
const effectiveReadOnly = state.status === 'ready' ? state.readOnly : true;
const allowSystemActions = !file.workspaceFileRef;
const allowSystemActions = !file.attachmentFileRef && !file.workspaceFileRef;
const dirty =
state.status === 'ready' && !state.readOnly && draft != null && draft !== state.content;
@@ -564,10 +569,11 @@ export function FilePreviewBody({
>
{tabs.includes('source') && (
<TabsContent value="source" className="m-0 h-full">
{file.contentType === 'snapshot' ? (
{richPreview === 'image' ? (
<ImageViewer
filePath={file.filePath}
fileName={file.fileName}
attachmentFileRef={file.attachmentFileRef}
workspaceFileRef={file.workspaceFileRef}
/>
) : (
@@ -590,13 +596,14 @@ export function FilePreviewBody({
)}
{tabs.includes('preview') && (
<TabsContent value="preview" className="m-0 h-full overflow-auto">
{file.contentType === 'snapshot' ? (
{richPreview === 'image' ? (
<ImageViewer
filePath={file.filePath}
fileName={file.fileName}
attachmentFileRef={file.attachmentFileRef}
workspaceFileRef={file.workspaceFileRef}
/>
) : isPdfPreviewExt(file.ext) ? (
) : richPreview === 'pdf' ? (
<Suspense
fallback={
<div className="flex h-full items-center justify-center">
@@ -607,10 +614,11 @@ export function FilePreviewBody({
<PdfViewerLazy
filePath={file.filePath}
fileName={file.fileName}
attachmentFileRef={file.attachmentFileRef}
workspaceFileRef={file.workspaceFileRef}
/>
</Suspense>
) : isSheetPreviewExt(file.ext) ? (
) : richPreview === 'sheet' ? (
<Suspense
fallback={
<div className="flex h-full items-center justify-center">
@@ -621,6 +629,7 @@ export function FilePreviewBody({
<SheetViewerLazy
filePath={file.filePath}
fileName={file.fileName}
attachmentFileRef={file.attachmentFileRef}
workspaceFileRef={file.workspaceFileRef}
/>
</Suspense>
@@ -630,6 +639,7 @@ export function FilePreviewBody({
source={draft ?? state.content}
filePath={file.filePath}
fileName={file.fileName}
attachmentFileRef={file.attachmentFileRef}
workspaceFileRef={file.workspaceFileRef}
/>
) : (
src/components/file-preview/HtmlPreview.tsx
+5 -3
View File
@@ -10,12 +10,13 @@
import { useMemo } from 'react';
import { useTranslation } from 'react-i18next';
import { cn } from '@/lib/utils';
import type { WorkspaceFileRef } from '@/lib/file-preview-client';
import type { AttachmentFileRef, WorkspaceFileRef } from '@/lib/file-preview-client';
export interface HtmlPreviewProps {
source: string;
filePath: string;
fileName?: string;
attachmentFileRef?: AttachmentFileRef;
workspaceFileRef?: WorkspaceFileRef;
className?: string;
}
@@ -57,13 +58,14 @@ export default function HtmlPreview({
source,
filePath,
fileName,
attachmentFileRef,
workspaceFileRef,
className,
}: HtmlPreviewProps) {
const { t } = useTranslation('chat');
const srcDoc = useMemo(
() => workspaceFileRef ? source : injectBaseHref(source, filePath),
[source, filePath, workspaceFileRef],
() => attachmentFileRef || workspaceFileRef ? source : injectBaseHref(source, filePath),
[attachmentFileRef, source, filePath, workspaceFileRef],
);
return (
src/components/file-preview/ImageViewer.tsx
+17 -9
View File
@@ -12,17 +12,19 @@ import { Button } from '@/components/ui/button';
import { LoadingSpinner } from '@/components/common/LoadingSpinner';
import {
readBinaryFile,
readAttachmentBinary,
readWorkspaceBinary,
type AttachmentFileRef,
type WorkspaceFileRef,
} from '@/lib/file-preview-client';
import { cn } from '@/lib/utils';
import { getFilePreviewTargetIdentity } from './types';
const IMAGE_MAX_BYTES = 50 * 1024 * 1024;
import { FILE_PREVIEW_MAX_BINARY_BYTES } from '@shared/file-preview/limits';
export interface ImageViewerProps {
filePath: string;
fileName: string;
attachmentFileRef?: AttachmentFileRef;
workspaceFileRef?: WorkspaceFileRef;
className?: string;
}
@@ -33,10 +35,10 @@ type LoadState =
| { identity: string; status: 'error'; message: string }
| { identity: string; status: 'ready'; url: string };
export default function ImageViewer({ filePath, fileName, workspaceFileRef, className }: ImageViewerProps) {
export default function ImageViewer({ filePath, fileName, attachmentFileRef, workspaceFileRef, className }: ImageViewerProps) {
const { t } = useTranslation('chat');
const [zoomed, setZoomed] = useState(false);
const loadIdentity = getFilePreviewTargetIdentity({ filePath, workspaceFileRef });
const loadIdentity = getFilePreviewTargetIdentity({ filePath, attachmentFileRef, workspaceFileRef });
const [state, setState] = useState<LoadState>({ identity: loadIdentity, status: 'loading' });
const currentState: LoadState = state.identity === loadIdentity
? state
@@ -48,11 +50,13 @@ export default function ImageViewer({ filePath, fileName, workspaceFileRef, clas
void (async () => {
try {
const res = workspaceFileRef
? await readWorkspaceBinary({ ...workspaceFileRef, maxBytes: IMAGE_MAX_BYTES })
: await readBinaryFile(filePath, { maxBytes: IMAGE_MAX_BYTES });
const res = attachmentFileRef
? await readAttachmentBinary(attachmentFileRef, FILE_PREVIEW_MAX_BINARY_BYTES)
: workspaceFileRef
? await readWorkspaceBinary({ ...workspaceFileRef, maxBytes: FILE_PREVIEW_MAX_BINARY_BYTES })
: await readBinaryFile(filePath, { maxBytes: FILE_PREVIEW_MAX_BINARY_BYTES });
if (cancelled) return;
if (!res.ok || !res.data) {
if (!res.ok) {
if (res.error === 'tooLarge') {
setState({ identity: loadIdentity, status: 'tooLarge', size: res.size });
return;
@@ -60,6 +64,10 @@ export default function ImageViewer({ filePath, fileName, workspaceFileRef, clas
setState({ identity: loadIdentity, status: 'error', message: String(res.error ?? 'unknown') });
return;
}
if (!res.data) {
setState({ identity: loadIdentity, status: 'error', message: 'unknown' });
return;
}
const cloned = new Uint8Array(res.data.byteLength);
cloned.set(res.data);
objectUrl = URL.createObjectURL(new Blob([cloned], { type: res.mimeType || 'image/png' }));
@@ -84,7 +92,7 @@ export default function ImageViewer({ filePath, fileName, workspaceFileRef, clas
URL.revokeObjectURL(objectUrl);
}
};
}, [filePath, loadIdentity, workspaceFileRef]);
}, [attachmentFileRef, filePath, loadIdentity, workspaceFileRef]);
if (currentState.status === 'loading') {
return (
src/components/file-preview/PdfViewer.tsx
+17 -7
View File
@@ -14,13 +14,15 @@ import { useTranslation } from 'react-i18next';
import { LoadingSpinner } from '@/components/common/LoadingSpinner';
import {
readBinaryFile,
readAttachmentBinary,
readWorkspaceBinary,
type AttachmentFileRef,
type WorkspaceFileRef,
} from '@/lib/file-preview-client';
import { cn } from '@/lib/utils';
import { getFilePreviewTargetIdentity } from './types';
import { FILE_PREVIEW_MAX_BINARY_BYTES } from '@shared/file-preview/limits';
const PDF_MAX_BYTES = 50 * 1024 * 1024;
const PDF_VIEWER_PARAMS = 'toolbar=0&navpanes=0&scrollbar=1&view=FitH&zoom=page-width';
const PDF_NATIVE_VIEWER_REVEAL_DELAY_MS = 900;
@@ -28,6 +30,7 @@ export interface PdfViewerProps {
filePath: string;
/** Optional file name shown in screen-reader labels and titles. */
fileName?: string;
attachmentFileRef?: AttachmentFileRef;
workspaceFileRef?: WorkspaceFileRef;
surface?: 'default' | 'workspace';
className?: string;
@@ -55,12 +58,13 @@ function withViewerParams(url: string): string {
export default function PdfViewer({
filePath,
fileName,
attachmentFileRef,
workspaceFileRef,
surface = 'default',
className,
}: PdfViewerProps) {
const { t } = useTranslation('chat');
const loadIdentity = getFilePreviewTargetIdentity({ filePath, workspaceFileRef });
const loadIdentity = getFilePreviewTargetIdentity({ filePath, attachmentFileRef, workspaceFileRef });
const [state, setState] = useState<LoadState>({ identity: loadIdentity, status: 'loading' });
const [iframeState, setIframeState] = useState<IframeState>({
url: null,
@@ -79,11 +83,13 @@ export default function PdfViewer({
void (async () => {
try {
const res = workspaceFileRef
? await readWorkspaceBinary({ ...workspaceFileRef, maxBytes: PDF_MAX_BYTES })
: await readBinaryFile(filePath, { maxBytes: PDF_MAX_BYTES });
const res = attachmentFileRef
? await readAttachmentBinary(attachmentFileRef, FILE_PREVIEW_MAX_BINARY_BYTES)
: workspaceFileRef
? await readWorkspaceBinary({ ...workspaceFileRef, maxBytes: FILE_PREVIEW_MAX_BINARY_BYTES })
: await readBinaryFile(filePath, { maxBytes: FILE_PREVIEW_MAX_BINARY_BYTES });
if (cancelled) return;
if (!res.ok || !res.data) {
if (!res.ok) {
if (res.error === 'tooLarge') {
setState({ identity: loadIdentity, status: 'tooLarge', size: res.size });
return;
@@ -91,6 +97,10 @@ export default function PdfViewer({
setState({ identity: loadIdentity, status: 'error', message: String(res.error ?? 'unknown') });
return;
}
if (!res.data) {
setState({ identity: loadIdentity, status: 'error', message: 'unknown' });
return;
}
const cloned = new Uint8Array(res.data.byteLength);
cloned.set(res.data);
objectUrl = URL.createObjectURL(new Blob([cloned], { type: 'application/pdf' }));
@@ -115,7 +125,7 @@ export default function PdfViewer({
URL.revokeObjectURL(objectUrl);
}
};
}, [filePath, loadIdentity, workspaceFileRef]);
}, [attachmentFileRef, filePath, loadIdentity, workspaceFileRef]);
useEffect(() => {
if (!iframeState.loaded || !iframeState.url) return;
src/components/file-preview/SheetViewer.tsx
+17 -8
View File
@@ -20,18 +20,21 @@ import { Button } from '@/components/ui/button';
import { LoadingSpinner } from '@/components/common/LoadingSpinner';
import {
readBinaryFile,
readAttachmentBinary,
readWorkspaceBinary,
type AttachmentFileRef,
type WorkspaceFileRef,
} from '@/lib/file-preview-client';
import { cn } from '@/lib/utils';
import { getFilePreviewTargetIdentity } from './types';
import { FILE_PREVIEW_MAX_BINARY_BYTES } from '@shared/file-preview/limits';
const SHEET_MAX_BYTES = 50 * 1024 * 1024;
const ROWS_PER_PAGE = 200;
export interface SheetViewerProps {
filePath: string;
fileName?: string;
attachmentFileRef?: AttachmentFileRef;
workspaceFileRef?: WorkspaceFileRef;
className?: string;
}
@@ -72,9 +75,9 @@ function formatCellValue(value: unknown): string {
return String(value);
}
export default function SheetViewer({ filePath, fileName, workspaceFileRef, className }: SheetViewerProps) {
export default function SheetViewer({ filePath, fileName, attachmentFileRef, workspaceFileRef, className }: SheetViewerProps) {
const { t } = useTranslation('chat');
const loadIdentity = getFilePreviewTargetIdentity({ filePath, workspaceFileRef });
const loadIdentity = getFilePreviewTargetIdentity({ filePath, attachmentFileRef, workspaceFileRef });
const [storedState, setState] = useState<LoadState>({ identity: loadIdentity, status: 'loading' });
const state: LoadState = storedState.identity === loadIdentity
? storedState
@@ -91,11 +94,13 @@ export default function SheetViewer({ filePath, fileName, workspaceFileRef, clas
void (async () => {
try {
const res = workspaceFileRef
? await readWorkspaceBinary({ ...workspaceFileRef, maxBytes: SHEET_MAX_BYTES })
: await readBinaryFile(filePath, { maxBytes: SHEET_MAX_BYTES });
const res = attachmentFileRef
? await readAttachmentBinary(attachmentFileRef, FILE_PREVIEW_MAX_BINARY_BYTES)
: workspaceFileRef
? await readWorkspaceBinary({ ...workspaceFileRef, maxBytes: FILE_PREVIEW_MAX_BINARY_BYTES })
: await readBinaryFile(filePath, { maxBytes: FILE_PREVIEW_MAX_BINARY_BYTES });
if (cancelled) return;
if (!res.ok || !res.data) {
if (!res.ok) {
if (res.error === 'tooLarge') {
setState({ identity: loadIdentity, status: 'tooLarge', size: res.size });
return;
@@ -103,6 +108,10 @@ export default function SheetViewer({ filePath, fileName, workspaceFileRef, clas
setState({ identity: loadIdentity, status: 'error', message: String(res.error ?? 'unknown') });
return;
}
if (!res.data) {
setState({ identity: loadIdentity, status: 'error', message: 'unknown' });
return;
}
const xlsx = await import('xlsx');
const wb = xlsx.read(res.data, { type: 'array', cellDates: true });
const sheets: SheetSnapshot[] = wb.SheetNames.map((name) => {
@@ -147,7 +156,7 @@ export default function SheetViewer({ filePath, fileName, workspaceFileRef, clas
return () => {
cancelled = true;
};
}, [filePath, loadIdentity, workspaceFileRef]);
}, [attachmentFileRef, filePath, loadIdentity, workspaceFileRef]);
const sheets = state.status === 'ready' ? state.sheets : null;
const activeSheet = sheets?.[sheetIndex] ?? null;
src/components/file-preview/build-preview-target.ts
+25
View File
@@ -5,6 +5,8 @@
* graph (and so React Fast Refresh stays happy).
*/
import { classifyFileExt, extnameOf, getMimeTypeForExt } from '@/lib/generated-files';
import type { AttachmentRenderPart } from '@/lib/acp/timeline-types';
import { richFilePreviewKind } from '@/lib/file-preview-capabilities';
import type { WorkspaceFileRef } from '@/lib/file-preview-client';
import type { FilePreviewTarget } from './types';
@@ -42,3 +44,26 @@ export function buildWorkspacePreviewTarget(
contentType: classifyFileExt(ext),
};
}
export function buildAttachmentPreviewTarget(attachment: AttachmentRenderPart): FilePreviewTarget {
if (attachment.access.status !== 'available' || attachment.access.target.kind !== 'local') {
throw new Error('Attachment is not available for preview');
}
const fileName = attachment.reference.name;
const ext = extnameOf(fileName);
const mimeType = attachment.access.mimeType || getMimeTypeForExt(ext);
const richPreview = richFilePreviewKind({ ext, mimeType });
return {
attachmentFileRef: attachment.access.target.ref,
filePath: fileName,
fileName,
ext,
mimeType,
contentType: richPreview === 'image'
? 'snapshot'
: richPreview === 'pdf' || richPreview === 'sheet'
? 'document'
: classifyFileExt(ext),
size: attachment.access.size,
};
}
src/components/file-preview/types.ts
+12 -3
View File
@@ -6,9 +6,10 @@
* Monaco component graph.
*/
import type { FileContentType, FileEditOp, GeneratedFileBaseline } from '@/lib/generated-files';
import type { WorkspaceFileRef } from '@/lib/file-preview-client';
import type { AttachmentFileRef, WorkspaceFileRef } from '@/lib/file-preview-client';
export interface FilePreviewTarget {
attachmentFileRef?: AttachmentFileRef;
workspaceFileRef?: WorkspaceFileRef;
filePath: string;
fileName: string;
@@ -47,9 +48,17 @@ export interface FilePreviewTarget {
}
export function getFilePreviewTargetIdentity(
target: Pick<FilePreviewTarget, 'filePath' | 'workspaceFileRef'>,
target: Pick<FilePreviewTarget, 'attachmentFileRef' | 'filePath' | 'workspaceFileRef'>,
): string {
return target.workspaceFileRef
return target.attachmentFileRef
? `attachment:${JSON.stringify([
target.attachmentFileRef.sessionKey,
target.attachmentFileRef.generation,
target.attachmentFileRef.uri,
target.attachmentFileRef.stagingId,
target.attachmentFileRef.transcriptMessageId,
])}`
: target.workspaceFileRef
? `workspace:${JSON.stringify([
target.workspaceFileRef.workspaceRoot,
target.workspaceFileRef.relativePath,
src/lib/acp/attachments.ts
+238
View File
@@ -0,0 +1,238 @@
import type { ResolveAttachmentResult } from '@shared/host-api/contract';
import type {
AcpTimelineSnapshot,
AttachmentRenderPart,
AttachmentUnavailableReason,
RenderPart,
TimelineItem,
} from './timeline-types';
export type CreatePendingAttachmentInput = {
messageId: string;
segmentIndex: number;
blockIndex: number;
uri: string;
name: string;
displayPath?: string;
mimeType?: string;
size?: number;
stagingId?: string;
transcriptMessageId?: string;
source?: AttachmentRenderPart['source'];
evidenceId?: string;
unavailableReason?: AttachmentUnavailableReason;
};
export type PendingAttachmentLocation = {
itemId: string;
partIndex: number;
fingerprint: string;
attachment: AttachmentRenderPart & { access: { status: 'pending' } };
};
export type ApplyAttachmentResolutionInput = {
attachmentId: string;
expectedFingerprint: string;
result: ResolveAttachmentResult;
};
function referenceFromInput(input: CreatePendingAttachmentInput): AttachmentRenderPart['reference'] {
return {
uri: input.uri,
name: input.name,
...(input.displayPath ? { displayPath: input.displayPath } : {}),
...(input.mimeType ? { mimeType: input.mimeType } : {}),
...(typeof input.size === 'number' ? { size: input.size } : {}),
...(input.stagingId ? { stagingId: input.stagingId } : {}),
...(input.transcriptMessageId ? { transcriptMessageId: input.transcriptMessageId } : {}),
};
}
export function createPendingAttachment(input: CreatePendingAttachmentInput): AttachmentRenderPart {
return {
kind: 'attachment',
attachmentId: `attachment:${input.messageId}:${input.segmentIndex}:${input.blockIndex}`,
reference: referenceFromInput(input),
source: input.source ?? 'acp-resource',
...(input.evidenceId ? { evidenceId: input.evidenceId } : {}),
access: input.unavailableReason
? { status: 'unavailable', reason: input.unavailableReason }
: { status: 'pending' },
};
}
export function attachmentRequestFingerprint(attachment: AttachmentRenderPart): string {
const { reference } = attachment;
return JSON.stringify([
attachment.source,
reference.uri,
reference.stagingId ?? null,
reference.transcriptMessageId ?? null,
reference.name,
reference.mimeType ?? null,
reference.size ?? null,
attachment.evidenceId ?? null,
]);
}
function itemParts(item: TimelineItem): RenderPart[] | null {
if (item.kind === 'message-segment' || item.kind === 'thought') return item.parts;
if (item.kind === 'tool-call') return item.outputParts;
return null;
}
export function collectPendingAttachments(snapshot: AcpTimelineSnapshot): PendingAttachmentLocation[] {
const pending: PendingAttachmentLocation[] = [];
for (const itemId of snapshot.itemOrder) {
const item = snapshot.itemsById[itemId];
if (!item) continue;
const parts = itemParts(item);
if (!parts) continue;
parts.forEach((part, partIndex) => {
if (part.kind === 'attachment' && part.access.status === 'pending') {
pending.push({
itemId,
partIndex,
fingerprint: attachmentRequestFingerprint(part),
attachment: part as PendingAttachmentLocation['attachment'],
});
}
});
}
return pending;
}
function attachmentPriority(part: AttachmentRenderPart): number {
return part.source === 'acp-resource' ? 2 : 1;
}
export function dedupeTurnAttachments(parts: readonly RenderPart[]): AttachmentRenderPart[] {
const imageIdentities = new Set(
parts.flatMap((part) => part.kind === 'image' && part.mediaIdentity ? [part.mediaIdentity] : []),
);
const selected = new Set<AttachmentRenderPart>();
const selectedByIdentity = new Map<string, AttachmentRenderPart>();
for (const part of parts) {
if (part.kind !== 'attachment') continue;
if (part.access.status !== 'available') {
selected.add(part);
continue;
}
if (imageIdentities.has(part.access.identity)) continue;
const previous = selectedByIdentity.get(part.access.identity);
if (!previous) {
selectedByIdentity.set(part.access.identity, part);
selected.add(part);
continue;
}
if (attachmentPriority(part) > attachmentPriority(previous)) {
selected.delete(previous);
selectedByIdentity.set(part.access.identity, part);
selected.add(part);
}
}
return parts.filter((part): part is AttachmentRenderPart => part.kind === 'attachment' && selected.has(part));
}
function resolvedAttachment(
attachment: AttachmentRenderPart,
result: ResolveAttachmentResult,
): AttachmentRenderPart {
if (!result.ok) {
return {
...attachment,
reference: { ...attachment.reference, name: result.displayName },
access: { status: 'unavailable', reason: result.error },
};
}
return {
...attachment,
reference: {
...attachment.reference,
name: result.displayName,
...(result.displayPath ? { displayPath: result.displayPath } : {}),
mimeType: result.mimeType,
size: result.size,
},
access: {
status: 'available',
identity: result.identity,
target: result.target,
mimeType: result.mimeType,
size: result.size,
},
};
}
function mapItemParts(item: TimelineItem, map: (parts: RenderPart[]) => RenderPart[]): TimelineItem {
if (item.kind === 'message-segment' || item.kind === 'thought') return { ...item, parts: map(item.parts) };
if (item.kind === 'tool-call') return { ...item, outputParts: map(item.outputParts) };
return item;
}
export function dedupeTimelineAttachments(snapshot: AcpTimelineSnapshot): AcpTimelineSnapshot {
const keep = new Set<AttachmentRenderPart>();
let turnParts: RenderPart[] = [];
let turnKind: 'user' | 'assistant' | null = null;
const commitTurn = () => {
for (const attachment of dedupeTurnAttachments(turnParts)) keep.add(attachment);
turnParts = [];
turnKind = null;
};
for (const itemId of snapshot.itemOrder) {
const item = snapshot.itemsById[itemId];
if (!item) continue;
const itemKind = item.kind === 'message-segment' && item.role === 'user' ? 'user' : 'assistant';
if (turnKind && turnKind !== itemKind) commitTurn();
turnKind = itemKind;
const parts = itemParts(item);
if (parts) turnParts.push(...parts);
}
commitTurn();
let changed = false;
const itemsById = { ...snapshot.itemsById };
for (const itemId of snapshot.itemOrder) {
const item = itemsById[itemId];
if (!item) continue;
const nextItem = mapItemParts(item, (parts) => parts.filter((part) => {
if (part.kind !== 'attachment') return true;
const shouldKeep = keep.has(part);
if (!shouldKeep) changed = true;
return shouldKeep;
}));
if (nextItem !== item) itemsById[itemId] = nextItem;
}
return changed ? { ...snapshot, itemsById } : snapshot;
}
export function applyAttachmentResolution(
snapshot: AcpTimelineSnapshot,
input: ApplyAttachmentResolutionInput,
): AcpTimelineSnapshot {
let changed = false;
const itemsById = { ...snapshot.itemsById };
for (const itemId of snapshot.itemOrder) {
const item = itemsById[itemId];
if (!item) continue;
const nextItem = mapItemParts(item, (parts) => parts.map((part) => {
if (
part.kind !== 'attachment'
|| part.attachmentId !== input.attachmentId
|| attachmentRequestFingerprint(part) !== input.expectedFingerprint
) return part;
changed = true;
return resolvedAttachment(part, input.result);
}));
if (nextItem !== item) itemsById[itemId] = nextItem;
}
if (!changed) return snapshot;
return dedupeTimelineAttachments({ ...snapshot, itemsById });
}
src/lib/acp/content-blocks.ts
+115 -26
View File
@@ -1,10 +1,33 @@
import type { ContentBlock, ToolCallContent } from '@agentclientprotocol/sdk';
import { createPendingAttachment } from './attachments';
import type { RenderPart } from './timeline-types';
export type ContentBlockRenderContext = {
role: 'user' | 'assistant';
messageId: string;
segmentIndex: number;
blockIndex: number;
};
function optionalString(value: unknown): string | undefined {
return typeof value === 'string' ? value : undefined;
}
function nonEmptyString(value: unknown): string | undefined {
const string = optionalString(value);
return string?.trim() ? string : undefined;
}
function optionalNumber(value: unknown): number | undefined {
return typeof value === 'number' && Number.isFinite(value) ? value : undefined;
}
function recordValue(value: unknown): Record<string, unknown> | undefined {
return value && typeof value === 'object' && !Array.isArray(value)
? value as Record<string, unknown>
: undefined;
}
function unsupportedContent(message: string): RenderPart {
return { kind: 'error', message };
}
@@ -19,52 +42,113 @@ function imageDataSource(mimeType: string | undefined, data: string | undefined)
return `data:${mimeType};base64,${data}`;
}
export function contentBlockToRenderPart(block: ContentBlock): RenderPart {
function uriBasename(uri: string): string {
const withoutQuery = uri.split(/[?#]/, 1)[0] ?? uri;
return withoutQuery.split(/[\\/]/).filter(Boolean).at(-1) ?? uri;
}
function clawxUserMetadata(block: ContentBlock, role: ContentBlockRenderContext['role']): {
stagingId?: string;
fileName?: string;
} {
if (role !== 'user') return {};
const meta = recordValue(block._meta);
const clawx = recordValue(meta?.clawx);
const stagingId = nonEmptyString(clawx?.stagingId);
const fileName = nonEmptyString(clawx?.fileName);
return {
...(stagingId ? { stagingId } : {}),
...(fileName ? { fileName } : {}),
};
}
function attachmentPart(input: {
context: ContentBlockRenderContext;
uri: string;
name?: string;
displayPath?: string;
title?: string;
mimeType?: string;
size?: number;
stagingId?: string;
unavailable?: boolean;
}): RenderPart {
return createPendingAttachment({
...input.context,
uri: input.uri,
name: input.name ?? input.title ?? (input.uri ? uriBasename(input.uri) : ''),
...(input.displayPath ? { displayPath: input.displayPath } : {}),
...(input.mimeType ? { mimeType: input.mimeType } : {}),
...(typeof input.size === 'number' ? { size: input.size } : {}),
...(input.stagingId ? { stagingId: input.stagingId } : {}),
...(input.unavailable ? { unavailableReason: 'invalidReference' as const } : {}),
});
}
export function contentBlockToRenderPart(block: ContentBlock, context: ContentBlockRenderContext): RenderPart {
switch (block.type) {
case 'text':
return { kind: 'markdown', text: block.text };
case 'image': {
const uri = optionalString(block.uri);
const clawx = clawxUserMetadata(block, context.role);
if (context.role === 'user' && uri) {
return attachmentPart({
context,
uri,
name: clawx.fileName,
mimeType: block.mimeType,
stagingId: clawx.stagingId,
});
}
const source = isSafeImageUri(uri)
? uri
: imageDataSource(block.mimeType, optionalString(block.data)) ?? uri ?? '';
return { kind: 'image', source, mimeType: block.mimeType };
}
case 'resource_link':
return {
kind: 'file',
path: block.uri,
name: block.name,
case 'resource_link': {
const clawx = clawxUserMetadata(block, context.role);
return attachmentPart({
context,
uri: block.uri,
name: nonEmptyString(block.name),
title: nonEmptyString(block.title),
mimeType: block.mimeType ?? undefined,
};
size: block.size ?? undefined,
stagingId: clawx.stagingId,
});
}
case 'resource': {
if (!block.resource || typeof block.resource !== 'object') {
return unsupportedContent('Unsupported ACP resource content');
}
const resource = block.resource as Record<string, unknown>;
const uri = optionalString(resource.uri);
if (uri) {
return {
kind: 'file',
path: uri,
mimeType: optionalString(resource.mimeType),
};
}
return unsupportedContent('Unsupported ACP resource content');
const resource = recordValue(block.resource);
const uri = nonEmptyString(resource?.uri) ?? '';
return attachmentPart({
context,
uri,
name: nonEmptyString(resource?.name),
title: nonEmptyString(resource?.title),
mimeType: nonEmptyString(resource?.mimeType),
size: optionalNumber(resource?.size),
unavailable: !uri,
});
}
default:
return unsupportedContent(`Unsupported ACP content block: ${block.type}`);
}
}
export function contentBlocksToRenderParts(blocks: ContentBlock[] | undefined | null): RenderPart[] {
return (blocks ?? []).map(contentBlockToRenderPart);
export function contentBlocksToRenderParts(
blocks: ContentBlock[] | undefined | null,
context: Omit<ContentBlockRenderContext, 'blockIndex'>,
): RenderPart[] {
return (blocks ?? []).map((block, blockIndex) => contentBlockToRenderPart(block, { ...context, blockIndex }));
}
export function toolContentToRenderPart(entry: ToolCallContent): RenderPart {
export function toolContentToRenderPart(entry: ToolCallContent, context?: ContentBlockRenderContext): RenderPart {
switch (entry.type) {
case 'content':
return contentBlockToRenderPart(entry.content);
return contentBlockToRenderPart(entry.content, context ?? {
role: 'assistant', messageId: 'tool-content', segmentIndex: 0, blockIndex: 0,
});
case 'diff':
return { kind: 'markdown', text: `Diff: ${entry.path}\n\n${entry.newText}` };
case 'terminal':
@@ -74,6 +158,11 @@ export function toolContentToRenderPart(entry: ToolCallContent): RenderPart {
}
}
export function toolContentToRenderParts(content: ToolCallContent[] | undefined | null): RenderPart[] {
return (content ?? []).map(toolContentToRenderPart);
export function toolContentToRenderParts(
content: ToolCallContent[] | undefined | null,
context?: Omit<ContentBlockRenderContext, 'blockIndex'>,
): RenderPart[] {
return (content ?? []).map((entry, blockIndex) => toolContentToRenderPart(entry, context
? { ...context, blockIndex }
: undefined));
}
src/lib/acp/image-generation-compat.ts
+170 -29
View File
@@ -28,6 +28,7 @@ export type ImageGenerationCompletionEvidence = {
toolCallId?: string;
evidenceId: string;
caption: string;
authoritativeCaption?: true;
candidates: ImageGenerationMediaCandidate[];
};
@@ -145,7 +146,40 @@ function hasStructuredMediaFields(value: unknown): boolean {
function hasInternalUiDeliveryEvidence(value: unknown): boolean {
const record = asRecord(value);
if (!record) return false;
return stringValue(record.sourceReplySink)?.toLowerCase() === 'internal-ui';
if (stringValue(record.sourceReplySink)?.toLowerCase() !== 'internal-ui') return false;
const status = stringValue(record.status)?.toLowerCase();
const deliveryStatus = stringValue(record.deliveryStatus)?.toLowerCase();
return (status === undefined || status === 'ok')
&& (deliveryStatus === undefined || deliveryStatus === 'sent');
}
function hasRejectedInternalUiDelivery(value: unknown): boolean {
const record = asRecord(value);
if (!record) return false;
const status = stringValue(record.status)?.toLowerCase();
const deliveryStatus = stringValue(record.deliveryStatus)?.toLowerCase();
return (status !== undefined && ['error', 'failed', 'rejected', 'cancelled'].includes(status))
|| (deliveryStatus !== undefined && deliveryStatus !== 'sent');
}
function sourceReplyText(value: unknown): string | undefined {
if (!hasInternalUiDeliveryEvidence(value)) return undefined;
const text = asRecord(asRecord(value)?.sourceReply)?.text;
return typeof text === 'string' && text.trim() ? text : undefined;
}
function firstSourceReplyText(...values: unknown[]): string | undefined {
for (const value of values) {
const text = sourceReplyText(value);
if (text) return text;
}
return undefined;
}
function imageGenerationTaskId(value: unknown): string | undefined {
const text = stringValue(value);
if (!text) return undefined;
return text.match(/(?:^|:)image_generate:([0-9a-f-]{36})(?::|$)/i)?.[1];
}
function dedupeCandidates(candidates: ImageGenerationMediaCandidate[]): ImageGenerationMediaCandidate[] {
@@ -166,7 +200,7 @@ function textFromToolContent(content: unknown): string {
function textFromContentBlock(block: unknown): string {
const record = asRecord(block);
return record?.type === 'text' ? stringValue(record.text) ?? '' : '';
return record?.type === 'text' && typeof record.text === 'string' ? record.text : '';
}
function textFromMessageContent(content: unknown): string {
@@ -184,6 +218,32 @@ function collectMediaTagCandidates(text: string): ImageGenerationMediaCandidate[
return candidates;
}
function visibleAssistantText(text: string): string | undefined {
const withoutMedia = text.replace(MEDIA_TAG_RE, '');
const visible = withoutMedia
.replace(/^(?:[ \t]*\r?\n)+/, '')
.replace(/(?:\r?\n[ \t]*)+$/, '');
return visible.trim() ? visible : undefined;
}
function transcriptCompletionTrigger(message: RawMessage): { taskId: string; failed: boolean } | undefined {
if (transcriptRole(message) !== 'user') return undefined;
const record = message as RawMessage & { provenance?: unknown };
const provenance = asRecord(record.provenance);
const text = textFromMessageContent(message.content);
const failed = /^status:\s*(?:failed|error)\b/im.test(text);
if (
stringValue(provenance?.kind)?.toLowerCase() === 'inter_session'
&& stringValue(provenance?.sourceTool)?.toLowerCase() === 'image_generate'
) {
const taskId = imageGenerationTaskId(provenance?.sourceSessionKey);
if (taskId) return { taskId, failed };
}
const header = text.match(/^\[Inter-session message\][^\n]*sourceSession=(image_generate:[^\s]+)[^\n]*sourceTool=image_generate\b/i);
const taskId = imageGenerationTaskId(header?.[1]);
return taskId ? { taskId, failed } : undefined;
}
export function extractImageGenerationStartFromAcpEnvelope(
event: AcpSessionUpdateEnvelope,
): ImageGenerationTaskStart | null {
@@ -231,11 +291,12 @@ export function extractImageGenerationCompletionFromAcpEnvelope(
const rawOutput = asRecord(update.rawOutput);
const rawOutputDetails = asRecord(rawOutput?.details);
if (!hasInternalUiDeliveryEvidence(rawOutput) && !hasInternalUiDeliveryEvidence(rawOutputDetails)) return null;
const caption = firstSourceReplyText(rawOutput, rawOutputDetails);
const candidates = dedupeCandidates([
...collectStructuredMediaCandidates(rawOutput),
...collectStructuredMediaCandidates(rawOutputDetails),
]);
if (candidates.length === 0) return null;
if (!caption && candidates.length === 0) return null;
const toolCallId = stringValue(update.toolCallId) ?? 'unknown-tool';
return {
@@ -243,8 +304,9 @@ export function extractImageGenerationCompletionFromAcpEnvelope(
source: 'acp-session-update',
...(event.historical ? { historical: true } : {}),
...(toolCallId !== 'unknown-tool' ? { toolCallId } : {}),
evidenceId: `acp:${event.sessionKey}:${toolCallId}:${candidates.map((entry) => entry.key).join('|')}`,
caption: GENERATED_IMAGE_CAPTION,
evidenceId: `acp:${event.sessionKey}:${toolCallId}:${candidates.map((entry) => entry.key).join('|') || 'text-only'}`,
caption: caption ?? GENERATED_IMAGE_CAPTION,
...(caption ? { authoritativeCaption: true } : {}),
candidates,
};
}
@@ -281,35 +343,74 @@ export function extractImageGenerationTranscriptSupplement(
): ImageGenerationTranscriptSupplement {
const starts: ImageGenerationTaskStart[] = [];
const completions: ImageGenerationCompletionEvidence[] = [];
const startedTaskIds = new Set<string>();
const seenTaskIds = new Set<string>();
const activeTaskIds = new Set<string>();
const seenCompletionIds = new Set<string>();
let latestStart: ImageGenerationTaskStart | null = null;
let completionTaskId: string | undefined;
let completionFailed = false;
for (const message of messages) {
if (transcriptRole(message) === 'user') {
const trigger = transcriptCompletionTrigger(message);
if (trigger && activeTaskIds.has(trigger.taskId)) {
completionTaskId = trigger.taskId;
completionFailed = trigger.failed;
continue;
}
activeTaskIds.clear();
latestStart = null;
completionTaskId = undefined;
completionFailed = false;
continue;
}
const start = transcriptImageGenerationStart(message, sessionKey);
if (start && !startedTaskIds.has(start.taskId)) {
if (start && !seenTaskIds.has(start.taskId)) {
starts.push(start);
startedTaskIds.add(start.taskId);
seenTaskIds.add(start.taskId);
}
if (start) {
activeTaskIds.add(start.taskId);
latestStart = start;
}
if (start) latestStart = start;
if (transcriptRole(message) !== 'assistant' || startedTaskIds.size === 0) continue;
const candidates = collectMediaTagCandidates(textFromMessageContent(message.content));
if (candidates.length === 0) continue;
if (activeTaskIds.size === 0) continue;
const role = transcriptRole(message);
const details = asRecord(message.details);
const assistantText = role === 'assistant' ? textFromMessageContent(message.content) : '';
const candidates = role === 'assistant'
? collectMediaTagCandidates(assistantText)
: role === 'toolresult' && message.toolName === MESSAGE_TOOL && hasInternalUiDeliveryEvidence(details)
? collectStructuredMediaCandidates(details)
: [];
const caption = role === 'toolresult' && message.toolName === MESSAGE_TOOL
? firstSourceReplyText(details)
: role === 'assistant' && (completionTaskId !== undefined || candidates.length > 0)
? visibleAssistantText(assistantText)
: undefined;
if (!caption && candidates.length === 0) continue;
if (role === 'assistant' && completionTaskId && !completionFailed && candidates.length === 0) continue;
const messageId = message.id ?? String(message.timestamp ?? completions.length);
const evidenceId = `transcript:${sessionKey}:${messageId}:${candidates.map((entry) => entry.key).join('|')}`;
const evidenceId = `transcript:${sessionKey}:${messageId}:${candidates.map((entry) => entry.key).join('|') || 'text-only'}`;
if (seenCompletionIds.has(evidenceId)) continue;
seenCompletionIds.add(evidenceId);
const taskId = completionTaskId ?? latestStart?.taskId;
completions.push({
sessionKey,
source: 'transcript-history',
historical: true,
...(latestStart?.taskId ? { taskId: latestStart.taskId } : {}),
...(taskId ? { taskId } : {}),
...(latestStart?.toolCallId ? { toolCallId: latestStart.toolCallId } : {}),
evidenceId,
caption: GENERATED_IMAGE_CAPTION,
caption: caption ?? GENERATED_IMAGE_CAPTION,
...(caption ? { authoritativeCaption: true } : {}),
candidates,
});
if (role === 'assistant' && taskId) {
activeTaskIds.delete(taskId);
completionTaskId = undefined;
completionFailed = false;
}
}
return { starts, completions };
@@ -327,29 +428,47 @@ export function extractImageGenerationCompletionFromGatewayChatMessage(
const details = asRecord(message.details);
const role = stringValue(message.role)?.toLowerCase();
const toolName = stringValue(message.toolName);
const runId = stringValue(envelope.runId) ?? stringValue(root.runId) ?? 'unknown-run';
const sessionKey = stringValue(envelope.sessionKey) ?? stringValue(root.sessionKey);
const taskId = imageGenerationTaskId(runId) ?? imageGenerationTaskId(sessionKey);
const finalAssistantText = role === 'assistant' && taskId && stringValue(envelope.state)?.toLowerCase() === 'final'
? textFromMessageContent(message.content)
: '';
const finalAssistantCaption = visibleAssistantText(finalAssistantText);
const finalAssistantCandidates = collectMediaTagCandidates(finalAssistantText);
const assistantAttachedCandidates = role === 'assistant'
? collectStructuredMediaCandidates({ _attachedFiles: message._attachedFiles })
: [];
const trustedMessageToolResult = (role === 'toolresult' || role === 'tool_result') && toolName === MESSAGE_TOOL;
const trustedAssistantMedia = assistantAttachedCandidates.length > 0;
const trustedEnvelopeMedia = !nestedMessage && Boolean(stringValue(envelope.state)) && hasStructuredMediaFields(envelope);
if (!trustedMessageToolResult && !trustedAssistantMedia && !trustedEnvelopeMedia) return null;
const trustedFinalCompletion = Boolean(taskId && (finalAssistantCaption || finalAssistantCandidates.length > 0));
if (!trustedMessageToolResult && !trustedAssistantMedia && !trustedEnvelopeMedia && !trustedFinalCompletion) return null;
if (trustedMessageToolResult && (
message.isError === true
|| hasRejectedInternalUiDelivery(message)
|| hasRejectedInternalUiDelivery(details)
)) return null;
const sourceReplyCaption = trustedMessageToolResult
? firstSourceReplyText(message, details)
: finalAssistantCaption;
const candidates = dedupeCandidates([
...(trustedEnvelopeMedia ? collectStructuredMediaCandidates(envelope) : []),
...(trustedMessageToolResult ? collectStructuredMediaCandidates(message) : []),
...(trustedAssistantMedia ? assistantAttachedCandidates : []),
...finalAssistantCandidates,
...(trustedMessageToolResult ? collectStructuredMediaCandidates(details) : []),
]);
if (candidates.length === 0) return null;
if (!sourceReplyCaption && candidates.length === 0) return null;
const runId = stringValue(envelope.runId) ?? stringValue(root.runId) ?? 'unknown-run';
const sessionKey = stringValue(envelope.sessionKey) ?? stringValue(root.sessionKey);
return {
...(sessionKey ? { sessionKey } : {}),
source: 'gateway-chat-message',
evidenceId: `gateway:${runId}:${candidates.map((entry) => entry.key).join('|')}`,
caption: GENERATED_IMAGE_CAPTION,
...(taskId ? { taskId } : {}),
evidenceId: `gateway:${runId}:${candidates.map((entry) => entry.key).join('|') || 'text-only'}`,
caption: sourceReplyCaption ?? GENERATED_IMAGE_CAPTION,
...(sourceReplyCaption ? { authoritativeCaption: true } : {}),
candidates,
};
}
@@ -369,6 +488,11 @@ export function extractImageGenerationCompletionFromRuntimeEvent(
const resultDetails = asRecord(result?.details);
const meta = asRecord(record.meta);
const metaDetails = asRecord(meta?.details);
if (
record.isError === true
|| [record, result, resultDetails, meta, metaDetails].some(hasRejectedInternalUiDelivery)
) return null;
const caption = firstSourceReplyText(record, result, resultDetails, meta, metaDetails);
const candidates = dedupeCandidates([
...collectStructuredMediaCandidates(record),
...collectStructuredMediaCandidates(result),
@@ -376,23 +500,38 @@ export function extractImageGenerationCompletionFromRuntimeEvent(
...collectStructuredMediaCandidates(meta),
...collectStructuredMediaCandidates(metaDetails),
]);
if (name !== MESSAGE_TOOL || candidates.length === 0) return null;
if (name !== MESSAGE_TOOL || (!caption && candidates.length === 0)) return null;
const taskId = imageGenerationTaskId(sessionKey) ?? imageGenerationTaskId(record.runId);
return {
...(sessionKey ? { sessionKey } : {}),
source: 'runtime-event',
evidenceId: `runtime:tool.completed:${stringValue(record.runId) ?? 'unknown-run'}:${candidates.map((entry) => entry.key).join('|')}`,
caption: GENERATED_IMAGE_CAPTION,
...(taskId ? { taskId } : {}),
...(stringValue(record.toolCallId) ? { toolCallId: stringValue(record.toolCallId) } : {}),
evidenceId: `runtime:tool.completed:${stringValue(record.runId) ?? 'unknown-run'}:${candidates.map((entry) => entry.key).join('|') || 'text-only'}`,
caption: caption ?? GENERATED_IMAGE_CAPTION,
...(caption ? { authoritativeCaption: true } : {}),
candidates,
};
}
const candidates = collectStructuredMediaCandidates(record);
if (type !== 'assistant.delta' || candidates.length === 0) return null;
const taskId = imageGenerationTaskId(record.runId);
const phase = stringValue(record.phase)?.toLowerCase();
const finalText = taskId && phase === 'final_answer'
? stringValue(record.text) ?? stringValue(record.delta) ?? ''
: '';
const caption = visibleAssistantText(finalText);
const candidates = dedupeCandidates([
...collectStructuredMediaCandidates(record),
...collectMediaTagCandidates(finalText),
]);
if (type !== 'assistant.delta' || (!caption && candidates.length === 0)) return null;
return {
...(sessionKey ? { sessionKey } : {}),
source: 'runtime-event',
evidenceId: `runtime:assistant.delta:${stringValue(record.runId) ?? 'unknown-run'}:${candidates.map((entry) => entry.key).join('|')}`,
caption: GENERATED_IMAGE_CAPTION,
...(taskId ? { taskId } : {}),
evidenceId: `runtime:assistant.delta:${stringValue(record.runId) ?? 'unknown-run'}:${candidates.map((entry) => entry.key).join('|') || 'text-only'}`,
caption: caption ?? GENERATED_IMAGE_CAPTION,
...(caption ? { authoritativeCaption: true } : {}),
candidates,
};
}
@@ -400,6 +539,8 @@ export function extractImageGenerationCompletionFromRuntimeEvent(
export function imageGenerationEvidenceKey(evidence: ImageGenerationCompletionEvidence): string {
const candidateKeys = Array.from(new Set(evidence.candidates.map((entry) => entry.key)))
.sort();
const candidateSetKey = JSON.stringify(candidateKeys);
const candidateSetKey = candidateKeys.length > 0
? JSON.stringify(candidateKeys)
: JSON.stringify(['text-only', evidence.taskId ?? evidence.toolCallId ?? evidence.evidenceId]);
return `${evidence.sessionKey ?? 'unknown'}:image-generation:${candidateSetKey}`;
}
src/lib/acp/openclaw-media-compat.ts
+296
View File
@@ -0,0 +1,296 @@
import { stripAcpWorkingDirectoryPrefix } from '@shared/chat/session-title';
import type { RawMessage } from '@shared/chat/types';
import type { AcpTimelineSnapshot, MessageSegmentItem } from './timeline-types';
const MAX_MEDIA_REFERENCE_LENGTH = 4096;
const URI_SCHEME_RE = /^[A-Za-z][A-Za-z0-9+.-]*:/;
const WINDOWS_ABSOLUTE_RE = /^[A-Za-z]:[\\/]/;
const FENCE_OPEN_RE = /^ {0,3}(`{3,}|~{3,})(.*)$/;
const FENCE_CLOSE_RE = /^ {0,3}(`{3,}|~{3,})[ \t]*$/;
export type OpenClawMediaCandidate = {
evidenceId: string;
transcriptMessageId?: string;
uri: string;
order: number;
};
export type OpenClawMediaTurnSupplement = {
acpTurnId: string;
candidates: OpenClawMediaCandidate[];
};
export type TranscriptMediaTurn = {
normalizedUserText: string;
userOccurrenceFromTail: number;
candidates: OpenClawMediaCandidate[];
};
type MutableTranscriptTurn = Omit<TranscriptMediaTurn, 'userOccurrenceFromTail'>;
type PendingMediaCandidate = Omit<OpenClawMediaCandidate, 'evidenceId'> & { evidenceSeed: string };
type PendingTranscriptTurn = Omit<MutableTranscriptTurn, 'candidates'> & { candidates: PendingMediaCandidate[] };
type AcpUserTurn = {
turnId: string;
messageIds: Set<string>;
normalizedUserText: string;
userOccurrenceFromTail: number;
};
function stableHash(value: string): string {
let hash = 0x811c9dc5;
for (let index = 0; index < value.length; index += 1) {
hash ^= value.charCodeAt(index);
hash = Math.imul(hash, 0x01000193);
}
return (hash >>> 0).toString(16).padStart(8, '0');
}
function textFromContent(content: unknown): string {
if (typeof content === 'string') return content;
if (!Array.isArray(content)) return '';
return content
.map((entry) => {
if (!entry || typeof entry !== 'object') return '';
const block = entry as Record<string, unknown>;
return block.type === 'text' && typeof block.text === 'string' ? block.text : '';
})
.filter(Boolean)
.join('\n');
}
function normalizeUserText(text: string): string {
return stripAcpWorkingDirectoryPrefix(text)
.replace(/\r\n/g, '\n')
.trim();
}
function isInternalInterSessionUser(message: RawMessage): boolean {
if (typeof message.role !== 'string' || message.role.toLowerCase() !== 'user') return false;
const provenance = message as RawMessage & { provenance?: unknown };
if (provenance.provenance && typeof provenance.provenance === 'object') {
const kind = (provenance.provenance as Record<string, unknown>).kind;
if (typeof kind === 'string' && kind.toLowerCase() === 'inter_session') return true;
}
return /^\[Inter-session message\]\s/.test(textFromContent(message.content));
}
function parseDirectiveReference(line: string, executionCwd: string): string | null {
const match = line.match(/^\s*MEDIA:\s*(.*?)\s*$/i);
if (!match) return null;
const raw = match[1] ?? '';
if (!raw) return null;
let reference = raw;
const quote = raw[0];
if (quote === '"' || quote === "'") {
if (raw.length < 2 || raw[raw.length - 1] !== quote) return null;
reference = raw.slice(1, -1);
} else if (/\s/.test(raw)) {
return null;
}
if (!reference || reference.length > MAX_MEDIA_REFERENCE_LENGTH) return null;
if (/^[`*_[(<{]/.test(reference) || /[\]>)}`*]$/.test(reference)) return null;
if (/^https?:\/\//i.test(reference)) {
try {
const url = new URL(reference);
return url.protocol === 'http:' || url.protocol === 'https:' ? reference : null;
} catch {
return null;
}
}
if (/^file:\/\//i.test(reference)) return reference;
if (URI_SCHEME_RE.test(reference) && !WINDOWS_ABSOLUTE_RE.test(reference)) return null;
if (reference.startsWith('/') || reference.startsWith('~/') || WINDOWS_ABSOLUTE_RE.test(reference)) {
return reference;
}
return executionCwd.trim() ? reference : null;
}
function mediaReferences(text: string): Array<{ uri: string; line: number }> {
const references: Array<{ uri: string; line: number }> = [];
let fence: { marker: string; length: number } | null = null;
for (const [lineIndex, line] of text.split(/\r?\n/).entries()) {
if (fence) {
const closeMatch = line.match(FENCE_CLOSE_RE);
const delimiter = closeMatch?.[1] ?? '';
const marker = delimiter[0];
if (marker === fence.marker && delimiter.length >= fence.length) fence = null;
continue;
}
const openMatch = line.match(FENCE_OPEN_RE);
const delimiter = openMatch?.[1] ?? '';
const marker = delimiter[0];
const info = openMatch?.[2] ?? '';
if (marker && !(marker === '`' && info.includes('`'))) {
fence = { marker, length: delimiter.length };
continue;
}
references.push({ uri: line, line: lineIndex });
}
return references;
}
function assignOccurrencesFromTail<T extends { normalizedUserText: string }>(turns: T[]): Array<T & { userOccurrenceFromTail: number }> {
const occurrences = new Map<string, number>();
const result = new Array<T & { userOccurrenceFromTail: number }>(turns.length);
for (let index = turns.length - 1; index >= 0; index -= 1) {
const turn = turns[index]!;
const occurrence = (occurrences.get(turn.normalizedUserText) ?? 0) + 1;
occurrences.set(turn.normalizedUserText, occurrence);
result[index] = { ...turn, userOccurrenceFromTail: occurrence };
}
return result;
}
export function extractOpenClawMediaTurns(
messages: RawMessage[],
input: { executionCwd: string; suppressedUris: ReadonlySet<string> },
): TranscriptMediaTurn[] {
const turns: PendingTranscriptTurn[] = [];
let current: PendingTranscriptTurn | null = null;
for (const message of messages) {
const role = typeof message.role === 'string' ? message.role.toLowerCase() : '';
if (role === 'user') {
if (isInternalInterSessionUser(message)) continue;
current = {
normalizedUserText: normalizeUserText(textFromContent(message.content)),
candidates: [],
};
turns.push(current);
continue;
}
if (role !== 'assistant' || !current) continue;
const text = textFromContent(message.content);
for (const reference of mediaReferences(text)) {
const uri = parseDirectiveReference(reference.uri, input.executionCwd);
if (!uri || input.suppressedUris.has(uri)) continue;
const order = current.candidates.length;
const messageIdentity = message.id
? `id:${message.id}`
: message.timestamp != null
? `timestamp:${message.timestamp}`
: `content:${stableHash(text)}`;
current.candidates.push({
evidenceSeed: `${messageIdentity}:${reference.line}:${uri}`,
...(message.id ? { transcriptMessageId: message.id } : {}),
uri,
order,
});
}
}
return assignOccurrencesFromTail(turns).map((turn) => ({
normalizedUserText: turn.normalizedUserText,
userOccurrenceFromTail: turn.userOccurrenceFromTail,
candidates: turn.candidates.map(({ evidenceSeed, ...candidate }) => ({
...candidate,
evidenceId: `openclaw-media:${stableHash(JSON.stringify([
turn.normalizedUserText,
turn.userOccurrenceFromTail,
evidenceSeed,
]))}`,
})),
}));
}
function markdownText(item: MessageSegmentItem): string {
return item.parts
.flatMap((part) => part.kind === 'markdown' ? [part.text] : [])
.join('');
}
function acpUserTurns(snapshot: AcpTimelineSnapshot): AcpUserTurn[] {
const turns: Array<Omit<AcpUserTurn, 'userOccurrenceFromTail'>> = [];
let current: Omit<AcpUserTurn, 'userOccurrenceFromTail'> | null = null;
for (const itemId of snapshot.itemOrder) {
const item = snapshot.itemsById[itemId];
if (item?.kind === 'message-segment' && item.role === 'user') {
if (!current) {
current = { turnId: item.messageId, messageIds: new Set(), normalizedUserText: '' };
turns.push(current);
}
current.messageIds.add(item.messageId);
current.normalizedUserText += markdownText(item);
continue;
}
current = null;
}
return assignOccurrencesFromTail(turns.map((turn) => ({
...turn,
normalizedUserText: normalizeUserText(turn.normalizedUserText),
})));
}
function turnMatchKey(turn: { normalizedUserText: string; userOccurrenceFromTail: number }): string {
return JSON.stringify([turn.normalizedUserText, turn.userOccurrenceFromTail]);
}
export function selectOpenClawTranscriptTurn(
messages: RawMessage[],
snapshot: AcpTimelineSnapshot,
liveUserMessageId: string,
): RawMessage[] {
const acpMatches = acpUserTurns(snapshot).filter((turn) => turn.messageIds.has(liveUserMessageId));
if (acpMatches.length !== 1) return [];
const targetKey = turnMatchKey(acpMatches[0]!);
const rawTurns: Array<{ normalizedUserText: string; messages: RawMessage[] }> = [];
let current: { normalizedUserText: string; messages: RawMessage[] } | null = null;
for (const message of messages) {
const role = typeof message.role === 'string' ? message.role.toLowerCase() : '';
if (role === 'user' && !isInternalInterSessionUser(message)) {
current = {
normalizedUserText: normalizeUserText(textFromContent(message.content)),
messages: [message],
};
rawTurns.push(current);
} else if (current) {
current.messages.push(message);
}
}
const matches = assignOccurrencesFromTail(rawTurns)
.filter((turn) => turnMatchKey(turn) === targetKey);
return matches.length === 1 ? matches[0]!.messages : [];
}
export function alignOpenClawMediaTurns(
snapshot: AcpTimelineSnapshot,
transcriptTurns: TranscriptMediaTurn[],
input: { liveUserMessageId?: string },
): OpenClawMediaTurnSupplement[] {
const acpTurns = acpUserTurns(snapshot);
const eligibleAcpTurns = input.liveUserMessageId
? acpTurns.filter((turn) => turn.messageIds.has(input.liveUserMessageId!))
: acpTurns;
if (input.liveUserMessageId && eligibleAcpTurns.length !== 1) return [];
const acpByKey = new Map<string, AcpUserTurn>();
const ambiguousKeys = new Set<string>();
for (const turn of eligibleAcpTurns) {
if (!turn.normalizedUserText) continue;
const key = turnMatchKey(turn);
if (acpByKey.has(key)) ambiguousKeys.add(key);
else acpByKey.set(key, turn);
}
const supplements: OpenClawMediaTurnSupplement[] = [];
for (const transcriptTurn of transcriptTurns) {
if (transcriptTurn.candidates.length === 0 || !transcriptTurn.normalizedUserText) continue;
const key = turnMatchKey(transcriptTurn);
if (ambiguousKeys.has(key)) continue;
const acpTurn = acpByKey.get(key);
if (!acpTurn) continue;
supplements.push({ acpTurnId: acpTurn.turnId, candidates: transcriptTurn.candidates });
}
return supplements;
}
export function hashOpenClawMediaDiagnostic(value: string): string {
return stableHash(value);
}
src/lib/acp/reducer.ts
+144 -12
View File
@@ -9,7 +9,8 @@ import type {
ToolKind,
} from '@agentclientprotocol/sdk';
import { contentBlockToRenderPart, contentBlocksToRenderParts, toolContentToRenderPart, toolContentToRenderParts } from './content-blocks';
import type { AcpTimelineSnapshot, MessageSegmentItem, RenderPart, TimelineItem, ToolCallItem } from './timeline-types';
import { dedupeTimelineAttachments } from './attachments';
import type { AcpTimelineSnapshot, AttachmentRenderPart, MessageSegmentItem, RenderPart, TimelineItem, ToolCallItem } from './timeline-types';
type UpdateRecord = Record<string, unknown> & {
sessionUpdate?: unknown;
@@ -109,7 +110,9 @@ function nextMessageSegment(
const segmentIndex = state.segmentCounts[messageId] ?? 0;
const id = `${messageId}:${segmentIndex}`;
const item: MessageSegmentItem = { kind: 'message-segment', id, role, messageId, segmentIndex, parts: [] };
const item: MessageSegmentItem = {
kind: 'message-segment', id, role, messageId, segmentIndex, parts: [], blockCount: 0,
};
return {
state: {
@@ -134,7 +137,37 @@ function appendRenderPart(parts: RenderPart[], nextPart: RenderPart): RenderPart
return [...parts, nextPart];
}
function preserveAvailableAttachment(
previous: Extract<RenderPart, { kind: 'attachment' }>,
next: Extract<RenderPart, { kind: 'attachment' }>,
): RenderPart {
const sameReference = previous.reference.uri === next.reference.uri
&& previous.reference.stagingId === next.reference.stagingId;
return sameReference && previous.access.status === 'available'
? {
...next,
reference: {
...next.reference,
...(previous.reference.displayPath ? { displayPath: previous.reference.displayPath } : {}),
},
access: previous.access,
}
: next;
}
function appendMessageRenderPart(role: Role, parts: RenderPart[], nextPart: RenderPart): RenderPart[] {
if (nextPart.kind === 'attachment') {
const existingIndex = parts.findIndex((part) => (
part.kind === 'attachment' && part.attachmentId === nextPart.attachmentId
));
if (existingIndex >= 0) {
return parts.map((part, index) => (
index === existingIndex && part.kind === 'attachment'
? preserveAvailableAttachment(part, nextPart)
: part
));
}
}
if (role === 'user' && nextPart.kind === 'markdown') {
const markdownIndex = parts.findIndex((part) => part.kind === 'markdown');
if (markdownIndex >= 0 && markdownIndex !== parts.length - 1) {
@@ -149,20 +182,31 @@ function appendMessageRenderPart(role: Role, parts: RenderPart[], nextPart: Rend
}
function renderPartKey(part: RenderPart): string | null {
if (part.kind === 'file') return `file:${part.path ?? ''}:${part.name ?? ''}:${part.mimeType ?? ''}`;
if (part.kind === 'attachment') {
return `attachment:${part.reference.uri}:${part.reference.name}:${part.reference.mimeType ?? ''}:${part.reference.stagingId ?? ''}`;
}
if (part.kind === 'image') return `image:${part.source}:${part.mimeType ?? ''}`;
if (part.kind === 'error') return `error:${part.message}`;
return null;
}
function mergeOptimisticUserEchoParts(optimisticParts: RenderPart[], echoParts: RenderPart[]): RenderPart[] {
const echoPartKeys = new Set(echoParts.map(renderPartKey).filter((key): key is string => Boolean(key)));
const reconciledEchoParts = echoParts.map((echoPart) => {
if (echoPart.kind !== 'attachment') return echoPart;
const previous = optimisticParts.find((part) => (
part.kind === 'attachment' && renderPartKey(part) === renderPartKey(echoPart)
));
return previous?.kind === 'attachment'
? preserveAvailableAttachment(previous, echoPart)
: echoPart;
});
const echoPartKeys = new Set(reconciledEchoParts.map(renderPartKey).filter((key): key is string => Boolean(key)));
const missingOptimisticMedia = optimisticParts.filter((part) => {
if (part.kind === 'markdown') return false;
const key = renderPartKey(part);
return !key || !echoPartKeys.has(key);
});
return [...echoParts, ...missingOptimisticMedia];
return [...reconciledEchoParts, ...missingOptimisticMedia];
}
function findMessageSegmentId(state: AcpTimelineSnapshot, role: Role, messageId: string): string | undefined {
@@ -181,12 +225,19 @@ function appendMessageChunk(
if (!content) return state;
const messageId = getMessageId(state, update, role);
const result = nextMessageSegment(state, role, messageId);
const nextPart = contentBlockToRenderPart(content);
const blockIndex = result.item.blockCount ?? result.item.parts.length;
const nextPart = contentBlockToRenderPart(content, {
role,
messageId,
segmentIndex: result.item.segmentIndex,
blockIndex,
});
const parts = result.item.optimistic && role === 'user'
? mergeOptimisticUserEchoParts(result.item.parts, [nextPart])
: appendMessageRenderPart(role, result.item.parts, nextPart);
const nextItem: MessageSegmentItem = {
...result.item,
blockCount: blockIndex + 1,
optimistic: false,
parts,
};
@@ -207,9 +258,15 @@ function replaceMessage(
const existingId = findMessageSegmentId(state, role, messageId);
const existing = existingId ? state.itemsById[existingId] : undefined;
if (existing?.kind === 'message-segment') {
const parts = contentBlocksToRenderParts(contentArray(content));
const blocks = contentArray(content);
const parts = contentBlocksToRenderParts(blocks, {
role,
messageId,
segmentIndex: existing.segmentIndex,
});
const item: MessageSegmentItem = {
...existing,
blockCount: blocks.length,
optimistic: false,
parts: existing.optimistic ? mergeOptimisticUserEchoParts(existing.parts, parts) : parts,
};
@@ -221,10 +278,16 @@ function replaceMessage(
}
const result = nextMessageSegment(state, role, messageId);
const blocks = contentArray(content);
const item: MessageSegmentItem = {
...result.item,
blockCount: blocks.length,
optimistic: false,
parts: contentBlocksToRenderParts(contentArray(content)),
parts: contentBlocksToRenderParts(blocks, {
role,
messageId,
segmentIndex: result.item.segmentIndex,
}),
};
return {
@@ -249,6 +312,7 @@ export function appendSyntheticAssistantMessage(
role: 'assistant',
messageId: input.messageId,
segmentIndex: 0,
blockCount: 0,
parts: input.parts,
compat: { source: 'image-generation', evidenceId: input.evidenceId },
};
@@ -265,12 +329,66 @@ export function appendSyntheticAssistantMessage(
];
})();
return {
return dedupeTimelineAttachments({
...closed,
itemOrder: nextOrder,
itemsById: { ...closed.itemsById, [id]: item },
segmentCounts: { ...closed.segmentCounts, [input.messageId]: 1 },
});
}
export function upsertSyntheticTurnAttachments(
snapshot: AcpTimelineSnapshot,
input: {
turnId: string;
evidenceId: string;
attachments: AttachmentRenderPart[];
source: 'openclaw-media';
},
): AcpTimelineSnapshot {
const messageId = `compat:openclaw-media:${input.evidenceId}`;
const id = `${messageId}:0`;
const existingId = snapshot.itemOrder.find((itemId) => {
const item = snapshot.itemsById[itemId];
return item?.kind === 'message-segment'
&& item.compat?.source === input.source
&& item.compat.evidenceId === input.evidenceId;
});
const anchorIndex = snapshot.itemOrder.findIndex((itemId) => {
const item = snapshot.itemsById[itemId];
return item?.kind === 'message-segment' && item.role === 'user' && item.messageId === input.turnId;
});
if (anchorIndex < 0) return snapshot;
const item: MessageSegmentItem = {
kind: 'message-segment',
id,
role: 'assistant',
messageId,
segmentIndex: 0,
blockCount: 0,
parts: input.attachments,
compat: { source: input.source, evidenceId: input.evidenceId },
};
const itemsById = { ...snapshot.itemsById };
if (existingId && existingId !== id) delete itemsById[existingId];
itemsById[id] = item;
let itemOrder = snapshot.itemOrder.filter((itemId) => itemId !== existingId && itemId !== id);
const nextUserIndex = itemOrder.findIndex((itemId, index) => {
if (index <= anchorIndex) return false;
const nextItem = itemsById[itemId];
return nextItem?.kind === 'message-segment' && nextItem.role === 'user';
});
const insertionIndex = nextUserIndex < 0 ? itemOrder.length : nextUserIndex;
itemOrder = [...itemOrder.slice(0, insertionIndex), id, ...itemOrder.slice(insertionIndex)];
return dedupeTimelineAttachments({
...snapshot,
itemOrder,
itemsById,
segmentCounts: { ...snapshot.segmentCounts, [messageId]: 1 },
});
}
function normalizeToolStatus(status: ToolCallStatus | null | undefined): ToolCallItem['status'] {
@@ -313,7 +431,11 @@ function upsertToolCall(
status: propertyExists(update, 'status') ? normalizeToolStatus(rawStatus) : prev?.status ?? 'pending',
input: hasRawInput ? update.rawInput : prev?.input,
output: hasRawOutput ? update.rawOutput : prev?.output,
outputParts: hasContent ? toolContentToRenderParts(toolContentArray(update.content)) : prev?.outputParts ?? [],
outputParts: hasContent
? toolContentToRenderParts(toolContentArray(update.content), {
role: 'assistant', messageId: `tool:${toolCallId}`, segmentIndex: 0,
})
: prev?.outputParts ?? [],
locations: hasLocations ? toolLocations(update.locations) : prev?.locations ?? [],
error: rawError ?? prev?.error,
historical: !!prev?.historical || !!options.historical,
@@ -332,7 +454,12 @@ function appendToolContentChunk(
const prev = existingToolCall(state, id);
const rawContent = objectValue(update.content);
const nextPart = rawContent
? toolContentToRenderPart(rawContent as ToolCallContent)
? toolContentToRenderPart(rawContent as ToolCallContent, {
role: 'assistant',
messageId: `tool:${toolCallId}`,
segmentIndex: 0,
blockIndex: prev?.outputParts.length ?? 0,
})
: { kind: 'error' as const, message: 'Unsupported ACP tool content chunk' };
return appendItem(closeAllMessageSegments(state), {
@@ -363,7 +490,12 @@ function appendThoughtChunk(state: AcpTimelineSnapshot, update: UpdateRecord): A
kind: 'thought',
id,
messageId,
parts: [...parts, contentBlockToRenderPart(content)],
parts: [...parts, contentBlockToRenderPart(content, {
role: 'assistant',
messageId: `thought:${messageId}`,
segmentIndex: 0,
blockIndex: parts.length,
})],
});
}
src/lib/acp/timeline-groups.ts
+70 -18
View File
@@ -1,15 +1,23 @@
import type { AcpTimelineSnapshot, MessageSegmentItem, TimelineItem } from './timeline-types';
import type {
AcpTimelineSnapshot,
AttachmentRenderPart,
MessageSegmentItem,
RenderPart,
TimelineItem,
} from './timeline-types';
export type AcpUserDisplayGroup = {
kind: 'user';
id: string;
items: MessageSegmentItem[];
attachments: AttachmentRenderPart[];
};
export type AcpAssistantTurnDisplayGroup = {
kind: 'assistant-turn';
id: string;
items: TimelineItem[];
attachments: AttachmentRenderPart[];
};
export type AcpTimelineDisplayGroup = AcpUserDisplayGroup | AcpAssistantTurnDisplayGroup;
@@ -20,30 +28,74 @@ function isUserMessageSegment(item: TimelineItem): item is MessageSegmentItem {
function appendUserItem(groups: AcpTimelineDisplayGroup[], item: MessageSegmentItem): void {
const previous = groups[groups.length - 1];
if (previous?.kind === 'user') {
previous.items.push(item);
return;
const group: AcpUserDisplayGroup = previous?.kind === 'user'
? previous
: {
kind: 'user',
id: `user-group:${item.id}`,
items: [],
attachments: [],
};
if (previous !== group) groups.push(group);
const { attachments, remaining } = extractAttachments(item.parts);
group.items.push({ ...item, parts: remaining });
group.attachments.push(...attachments);
}
function extractAttachments(parts: RenderPart[]): {
attachments: AttachmentRenderPart[];
remaining: RenderPart[];
} {
const attachments: AttachmentRenderPart[] = [];
const remaining: RenderPart[] = [];
for (const part of parts) {
if (part.kind === 'attachment') attachments.push(part);
else remaining.push(part);
}
return { attachments, remaining };
}
function assistantGroupForItem(
groups: AcpTimelineDisplayGroup[],
item: TimelineItem,
): AcpAssistantTurnDisplayGroup {
const previous = groups[groups.length - 1];
if (previous?.kind === 'assistant-turn') {
return previous;
}
groups.push({
kind: 'user',
id: `user-group:${item.id}`,
items: [item],
});
const group: AcpAssistantTurnDisplayGroup = {
kind: 'assistant-turn',
id: `assistant-turn:${item.id}`,
items: [],
attachments: [],
};
groups.push(group);
return group;
}
function appendAssistantItem(groups: AcpTimelineDisplayGroup[], item: TimelineItem): void {
const previous = groups[groups.length - 1];
if (previous?.kind === 'assistant-turn') {
previous.items.push(item);
const group = assistantGroupForItem(groups, item);
if (item.kind === 'message-segment') {
const { attachments, remaining } = extractAttachments(item.parts);
group.attachments.push(...attachments);
if (remaining.length > 0) group.items.push({ ...item, parts: remaining });
return;
}
groups.push({
kind: 'assistant-turn',
id: `assistant-turn:${item.id}`,
items: [item],
});
if (item.kind === 'thought') {
const { attachments, remaining } = extractAttachments(item.parts);
group.attachments.push(...attachments);
group.items.push({ ...item, parts: remaining });
return;
}
if (item.kind === 'tool-call') {
const { attachments, remaining } = extractAttachments(item.outputParts);
group.attachments.push(...attachments);
group.items.push({ ...item, outputParts: remaining });
return;
}
group.items.push(item);
}
export function groupAcpTimelineItems(snapshot: AcpTimelineSnapshot): AcpTimelineDisplayGroup[] {
src/lib/acp/timeline-types.ts
+46 -3
View File
@@ -1,9 +1,50 @@
import type { PlanEntry, SessionConfigOption, ToolCallLocation, ToolKind } from '@agentclientprotocol/sdk';
import type {
AttachmentAccessError,
AttachmentFileRef,
AttachmentRemoteRef,
} from '@shared/host-api/contract';
export type AttachmentUnavailableReason = AttachmentAccessError;
export type AttachmentAccessTarget =
| {
kind: 'local';
scope: 'workspace' | 'openclaw-media' | 'staging';
ref: AttachmentFileRef;
}
| { kind: 'remote'; ref: AttachmentRemoteRef; url: string };
export type AttachmentRenderPart = {
kind: 'attachment';
attachmentId: string;
reference: {
uri: string;
name: string;
displayPath?: string;
mimeType?: string;
size?: number;
stagingId?: string;
transcriptMessageId?: string;
};
source: 'acp-resource' | 'openclaw-media';
evidenceId?: string;
access:
| { status: 'pending' }
| { status: 'unavailable'; reason: AttachmentUnavailableReason }
| {
status: 'available';
identity: string;
target: AttachmentAccessTarget;
mimeType: string;
size: number;
};
};
export type RenderPart =
| { kind: 'markdown'; text: string }
| { kind: 'image'; source: string; mimeType?: string; alt?: string }
| { kind: 'file'; path?: string; name?: string; mimeType?: string }
| { kind: 'image'; source: string; mimeType?: string; alt?: string; mediaIdentity?: string }
| AttachmentRenderPart
| { kind: 'error'; message: string };
export type MessageSegmentItem = {
@@ -13,9 +54,11 @@ export type MessageSegmentItem = {
messageId: string;
segmentIndex: number;
parts: RenderPart[];
/** Number of ACP blocks consumed by this segment, independent of render-part coalescing. */
blockCount?: number;
optimistic?: boolean;
/** Renderer-only compatibility projection, not an ACP protocol event. */
compat?: { source: 'image-generation'; evidenceId: string };
compat?: { source: 'image-generation' | 'openclaw-media'; evidenceId: string };
};
export type ThoughtItem = {
src/lib/acp/transcript-supplement.ts
+153
View File
@@ -0,0 +1,153 @@
import type { RawMessage } from '@shared/chat/types';
import type { ImageGenerationCompletionEvidence, ImageGenerationTranscriptSupplement } from './image-generation-compat';
import { extractImageGenerationTranscriptSupplement } from './image-generation-compat';
import { hostApi } from '../host-api';
import {
alignOpenClawMediaTurns,
extractOpenClawMediaTurns,
selectOpenClawTranscriptTurn,
type OpenClawMediaTurnSupplement,
} from './openclaw-media-compat';
import type { AcpTimelineSnapshot } from './timeline-types';
export type CoordinatedImageGenerationCompletion = ImageGenerationCompletionEvidence & {
transcriptMessageId?: string;
};
export type CoordinatedImageGenerationSupplement = Omit<ImageGenerationTranscriptSupplement, 'completions'> & {
completions: CoordinatedImageGenerationCompletion[];
};
export type TranscriptSupplementResult = {
imageGeneration: CoordinatedImageGenerationSupplement;
media: OpenClawMediaTurnSupplement[];
transcriptMediaTurnCount: number;
};
type TranscriptSupplementInput = {
sessionKey: string;
generation: number;
executionCwd: string;
snapshot: AcpTimelineSnapshot | (() => AcpTimelineSnapshot);
liveUserMessageId?: string;
isCurrent: () => boolean;
};
function recordTrace(input: TranscriptSupplementInput, event: string, details: Record<string, unknown>): void {
void hostApi.diagnostics.recordAcpTrace({
event,
direction: 'projection',
sessionKey: input.sessionKey,
generation: input.generation,
details,
}).catch(() => undefined);
}
function transcriptMessageId(
completion: ImageGenerationCompletionEvidence,
messages: RawMessage[],
sessionKey: string,
): string | undefined {
if (completion.source !== 'transcript-history') return undefined;
const prefix = `transcript:${sessionKey}:`;
return messages
.filter((message): message is RawMessage & { id: string } => typeof message.id === 'string' && message.id.length > 0)
.sort((left, right) => right.id.length - left.id.length)
.find((message) => completion.evidenceId.startsWith(`${prefix}${message.id}:`))
?.id;
}
// OpenClaw ACP currently projects only assistant text/thought content and strips MEDIA
// directives from the visible reply. This bounded transcript read recovers only missing
// resource blocks; it is not a second Chat history source. Remove it when distributed
// OpenClaw ACP emits assistant resource_link/resource content. Architecture rationale:
// harness/reference/acp-generated-media-and-diagnostics.md
export async function fetchOpenClawTranscriptSupplement(
input: TranscriptSupplementInput,
): Promise<TranscriptSupplementResult | null> {
recordTrace(input, 'openclaw-media:history-request-started', {
source: 'openclaw-media',
reason: input.liveUserMessageId ? 'live' : 'historical',
});
let response: Awaited<ReturnType<typeof hostApi.sessions.history>>;
try {
response = await hostApi.sessions.history({ sessionKey: input.sessionKey, limit: 1000 });
} catch {
if (input.isCurrent()) {
recordTrace(input, 'openclaw-media:history-request-failed', {
source: 'openclaw-media',
reason: 'request-failed',
});
} else {
recordTrace(input, 'openclaw-media:projection-stale', {
source: 'openclaw-media',
reason: 'history-failure-stale',
});
}
return null;
}
if (!input.isCurrent()) {
recordTrace(input, 'openclaw-media:projection-stale', {
source: 'openclaw-media',
reason: 'history-response-stale',
});
return null;
}
if (!response.success || !Array.isArray(response.messages)) {
recordTrace(input, 'openclaw-media:history-request-failed', {
source: 'openclaw-media',
reason: 'invalid-response',
});
return null;
}
const messages = response.messages;
const snapshot = typeof input.snapshot === 'function' ? input.snapshot() : input.snapshot;
const imageMessages = input.liveUserMessageId
? selectOpenClawTranscriptTurn(messages, snapshot, input.liveUserMessageId)
: messages;
const extractedImages = extractImageGenerationTranscriptSupplement(imageMessages, input.sessionKey);
const imageGeneration: CoordinatedImageGenerationSupplement = {
starts: extractedImages.starts,
completions: extractedImages.completions.map((completion) => {
const messageId = transcriptMessageId(completion, imageMessages, input.sessionKey);
return { ...completion, ...(messageId ? { transcriptMessageId: messageId } : {}) };
}),
};
const suppressedUris = new Set(
imageGeneration.completions.flatMap((completion) => completion.candidates.map((candidate) => candidate.key)),
);
const transcriptMediaTurns = extractOpenClawMediaTurns(messages, {
executionCwd: input.executionCwd,
suppressedUris,
});
const media = alignOpenClawMediaTurns(snapshot, transcriptMediaTurns, {
...(input.liveUserMessageId ? { liveUserMessageId: input.liveUserMessageId } : {}),
});
const transcriptMediaTurnCount = transcriptMediaTurns.filter((turn) => turn.candidates.length > 0).length;
recordTrace(input, 'openclaw-media:history-request-succeeded', {
source: 'openclaw-media',
candidateCount: transcriptMediaTurns.reduce((count, turn) => count + turn.candidates.length, 0),
matchedCount: media.length,
rejectedCount: Math.max(0, transcriptMediaTurnCount - media.length),
});
if (transcriptMediaTurnCount > media.length) {
recordTrace(input, 'openclaw-media:turn-rejected', {
source: 'openclaw-media',
reason: 'unmatched-user-anchor',
rejectedCount: transcriptMediaTurnCount - media.length,
});
}
for (const supplement of media) {
recordTrace(input, 'openclaw-media:turn-matched', {
source: 'openclaw-media',
reason: input.liveUserMessageId ? 'live-user-identity' : 'reverse-user-occurrence',
candidateCount: supplement.candidates.length,
});
}
return { imageGeneration, media, transcriptMediaTurnCount };
}
src/lib/file-preview-capabilities.ts
+89
View File
@@ -0,0 +1,89 @@
import {
FILE_PREVIEW_MAX_BINARY_BYTES,
FILE_PREVIEW_MAX_TEXT_BYTES,
} from '@shared/file-preview/limits';
import type { AttachmentAccessTarget } from '@/lib/acp/timeline-types';
import {
classifyFileExt,
supportsInlineDocumentPreview,
supportsRichDocumentPreview,
} from '@/lib/generated-files';
export type FilePreviewKind = 'text' | 'rich';
export type RichFilePreviewKind = 'image' | 'pdf' | 'sheet';
export type AttachmentOpenMode = 'preview' | 'system';
const TEXT_APPLICATION_MIME_TYPES = new Set([
'application/json',
'application/javascript',
'application/toml',
'application/xml',
'application/yaml',
]);
const SYSTEM_OPEN_ONLY_EXTENSIONS = new Set([
'.zip', '.tar', '.gz', '.tgz', '.bz2', '.xz', '.rar', '.7z',
'.doc', '.docx', '.ppt', '.pptx',
'.aac', '.aiff', '.opus', '.wma',
'.3gp', '.flv', '.m4v', '.mpeg', '.mpg', '.ogv', '.wmv',
]);
function normalizedMimeType(mimeType: string): string {
return mimeType.toLowerCase().split(';', 1)[0]?.trim() ?? '';
}
function isSystemOpenOnlyExtension(ext: string): boolean {
const contentType = classifyFileExt(ext);
return SYSTEM_OPEN_ONLY_EXTENSIONS.has(ext)
|| contentType === 'audio'
|| contentType === 'video';
}
export function richFilePreviewKind(input: { ext: string; mimeType: string }): RichFilePreviewKind | null {
const ext = input.ext.toLowerCase();
const mimeType = normalizedMimeType(input.mimeType);
const contentType = classifyFileExt(ext);
if (isSystemOpenOnlyExtension(ext)) return null;
if (contentType === 'snapshot') return 'image';
if (ext === '.pdf') return 'pdf';
if (supportsRichDocumentPreview(ext)) return 'sheet';
if (contentType === 'code' || supportsInlineDocumentPreview(ext) || ext === '.csv') return null;
if (mimeType.startsWith('image/')) return 'image';
if (mimeType === 'application/pdf') return 'pdf';
if (mimeType.includes('spreadsheet') || mimeType === 'application/vnd.ms-excel') return 'sheet';
return null;
}
export function filePreviewKind(input: { ext: string; mimeType: string }): FilePreviewKind | null {
const ext = input.ext.toLowerCase();
const mimeType = normalizedMimeType(input.mimeType);
const contentType = classifyFileExt(ext);
if (isSystemOpenOnlyExtension(ext)) return null;
if (richFilePreviewKind(input)) return 'rich';
if (
contentType === 'code'
|| supportsInlineDocumentPreview(ext)
|| ext === '.csv'
|| mimeType.startsWith('text/')
|| TEXT_APPLICATION_MIME_TYPES.has(mimeType)
) return 'text';
return null;
}
export function isFilePreviewWithinSizeLimit(kind: FilePreviewKind, size: number): boolean {
const limit = kind === 'rich' ? FILE_PREVIEW_MAX_BINARY_BYTES : FILE_PREVIEW_MAX_TEXT_BYTES;
return Number.isFinite(size) && size >= 0 && size <= limit;
}
export function attachmentOpenMode(input: {
ext: string;
mimeType: string;
size: number;
target: AttachmentAccessTarget;
}): AttachmentOpenMode {
if (input.target.kind === 'remote') return 'system';
const kind = filePreviewKind(input);
return kind && isFilePreviewWithinSizeLimit(kind, input.size) ? 'preview' : 'system';
}
src/lib/file-preview-client.ts
+11
View File
@@ -1,12 +1,17 @@
import type {
AttachmentFileRef,
AttachmentSourceRef,
FilePreviewTreeOptions,
FileReadBinaryOptions,
ResolveAttachmentPayload,
WorkspaceContextInput,
WorkspaceFileRef,
} from '@shared/host-api/contract';
import { hostApi } from './host-api';
export type {
AttachmentFileRef,
AttachmentSourceRef,
FileListDirEntry as ListDirEntry,
FileListDirResult as ListDirResult,
FilePreviewError,
@@ -43,3 +48,9 @@ export const readWorkspaceBinary = (input: WorkspaceFileRef & { maxBytes?: numbe
hostApi.files.readWorkspaceBinary(input)
);
export const statWorkspaceFile = (ref: WorkspaceFileRef) => hostApi.files.statWorkspaceFile(ref);
export const resolveAttachment = (payload: ResolveAttachmentPayload) => hostApi.files.resolveAttachment(payload);
export const readAttachmentText = (ref: AttachmentFileRef) => hostApi.files.readAttachmentText(ref);
export const readAttachmentBinary = (ref: AttachmentFileRef, maxBytes?: number) => (
hostApi.files.readAttachmentBinary({ ref, maxBytes })
);
export const openAttachment = (ref: AttachmentSourceRef) => hostApi.files.openAttachment(ref);
src/lib/host-api.ts
+19
View File
@@ -2,6 +2,8 @@ import type {
AgentCreatePayload,
AgentUpdatePayload,
AcpTraceRecordPayload,
AttachmentFileRef,
AttachmentSourceRef,
ChannelAccountsPayload,
ChannelSaveConfigPayload,
ChannelTargetsPayload,
@@ -21,6 +23,8 @@ import type {
ProviderOAuthRequestPayload,
ProviderUpdateWithKeyPayload,
ProviderValidationPayload,
ReadAttachmentBinaryPayload,
ResolveAttachmentPayload,
SaveImagePayload,
SettingsKey,
SettingsSnapshot,
@@ -44,6 +48,11 @@ import type { CronJobCreateInput, CronJobUpdateInput } from '@shared/types/cron'
import { invokeHost } from './host-api-client';
export type {
AttachmentAccessError,
AttachmentFileRef,
AttachmentRemoteRef,
AttachmentReadError,
AttachmentSourceRef,
ChannelAccountsResult,
ChannelCredentialValidationResult,
ChannelFormValuesResult,
@@ -68,9 +77,13 @@ export type {
OpenClawCliCommandResult,
OpenClawDoctorResult,
OpenClawStatusResult,
OpenAttachmentResult,
ProviderAccountKeyInfo,
ProviderDefaultAccountResult,
ProviderValidationResult,
ReadAttachmentBinaryResult,
ReadAttachmentTextResult,
ResolveAttachmentResult,
SessionHistoryResult,
SessionLabelSummary,
SessionSummariesResult,
@@ -291,6 +304,12 @@ export const hostApi = {
invokeHost('files', 'readWorkspaceBinary', input)
),
statWorkspaceFile: (ref: WorkspaceFileRef) => invokeHost('files', 'statWorkspaceFile', ref),
resolveAttachment: (input: ResolveAttachmentPayload) => invokeHost('files', 'resolveAttachment', input),
readAttachmentText: (ref: AttachmentFileRef) => invokeHost('files', 'readAttachmentText', ref),
readAttachmentBinary: (input: ReadAttachmentBinaryPayload) => (
invokeHost('files', 'readAttachmentBinary', input)
),
openAttachment: (ref: AttachmentSourceRef) => invokeHost('files', 'openAttachment', ref),
},
media: {
thumbnails: (input: { paths: MediaThumbnailEntry[] }) => invokeHost('media', 'thumbnails', input),
src/pages/Chat/AcpAssistantTurn.tsx
+5
View File
@@ -8,6 +8,7 @@ import { AcpThoughtBlock } from './AcpThoughtBlock';
import { AcpToolCallCard } from './AcpToolCallCard';
import type { AcpTurnFileSummary } from '@/lib/acp/openclaw-file-activities';
import { AcpTurnFileActivity } from './AcpTurnFileActivity';
import { AcpAttachmentPart } from './AcpAttachmentPart';
function assistantTurnClipboardText(group: AcpAssistantTurnDisplayGroup): string {
const textSegments: string[] = [];
@@ -91,6 +92,10 @@ export function AcpAssistantTurn({
return null;
})}
{group.attachments.map((attachment) => (
<AcpAttachmentPart key={attachment.attachmentId} part={attachment} />
))}
{workspaceRoot && <AcpTurnFileActivity summaries={fileSummaries} workspaceRoot={workspaceRoot} />}
{clipboardText.trim().length > 0 && (
src/pages/Chat/AcpAttachmentPart.tsx
+167
View File
@@ -0,0 +1,167 @@
import { useEffect, useState } from 'react';
import { Paperclip } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { toast } from 'sonner';
import { buildAttachmentPreviewTarget } from '@/components/file-preview/build-preview-target';
import { formatFileSize } from '@/components/file-preview/format';
import type { AttachmentRenderPart } from '@/lib/acp/timeline-types';
import { attachmentOpenMode } from '@/lib/file-preview-capabilities';
import { basenameOf, extnameOf } from '@/lib/generated-files';
import { hostApi } from '@/lib/host-api';
import { cn } from '@/lib/utils';
import { useArtifactPanel } from '@/stores/artifact-panel';
type AttachmentTone = 'assistant' | 'user';
function AcpUserImageAttachment({
part,
name,
ariaLabel,
activate,
}: {
part: AttachmentRenderPart & { access: Extract<AttachmentRenderPart['access'], { status: 'available' }> };
name: string;
ariaLabel: string;
activate: () => Promise<void>;
}) {
const [thumbnailUrl, setThumbnailUrl] = useState<string | null>(null);
useEffect(() => {
if (part.access.target.kind !== 'local') return;
let cancelled = false;
void hostApi.media.thumbnails({
paths: [{
attachmentFileRef: part.access.target.ref,
key: part.access.identity,
mimeType: part.access.mimeType,
}],
}).then((result) => {
if (cancelled) return;
setThumbnailUrl(result[part.access.identity]?.preview ?? null);
}).catch(() => undefined);
return () => {
cancelled = true;
};
}, [part.access]);
return (
<button
type="button"
aria-label={ariaLabel}
onClick={() => void activate()}
className="group/user-image relative h-18 w-auto max-w-full overflow-hidden rounded-xl border border-black/10 bg-surface-modal text-left focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 dark:border-white/10"
>
{thumbnailUrl && (
<img
data-testid="acp-user-image-attachment"
src={thumbnailUrl}
alt={name}
className="h-full w-full object-cover"
/>
)}
<span
data-testid="acp-user-image-overlay"
className="absolute inset-0 flex items-end bg-black/0 p-2.5 transition-colors group-hover/user-image:bg-black/50 group-focus-visible/user-image:bg-black/50"
>
<span
data-testid="acp-user-image-filename"
className="w-full truncate text-xs font-medium text-white opacity-0 drop-shadow transition-opacity group-hover/user-image:opacity-100 group-focus-visible/user-image:opacity-100"
>
{name}
</span>
</span>
</button>
);
}
export function AcpAttachmentPart({ part, tone = 'assistant' }: { part: AttachmentRenderPart; tone?: AttachmentTone }) {
const { t } = useTranslation('chat');
const name = basenameOf(part.reference.name) || part.reference.name;
const pending = part.access.status === 'pending';
const unavailable = part.access.status === 'unavailable';
const disabled = pending || unavailable;
const size = part.access.status === 'available' ? part.access.size : part.reference.size;
const mimeType = part.access.status === 'available' ? part.access.mimeType : part.reference.mimeType;
const secondary = size
? t('acp.attachment.mimeSize', { mimeType, size: formatFileSize(size) })
: '';
const mode = part.access.status === 'available'
? attachmentOpenMode({ ext: extnameOf(name), mimeType: part.access.mimeType, size: part.access.size, target: part.access.target })
: null;
const actionLabel = pending
? t('acp.attachment.loading')
: unavailable
? t('acp.attachment.unavailable')
: t(mode === 'preview' ? 'acp.attachment.preview' : 'acp.attachment.open', { name });
const ariaLabel = disabled ? `${actionLabel}: ${name}` : actionLabel;
const userDisplayPath = tone === 'user' ? part.reference.displayPath : undefined;
const activate = async () => {
if (part.access.status !== 'available') return;
if (mode === 'preview') {
useArtifactPanel.getState().openPreview(buildAttachmentPreviewTarget(part));
return;
}
try {
const result = await hostApi.files.openAttachment(part.access.target.ref);
if (!result.ok) toast.error(t('acp.attachment.openFailed'));
} catch {
toast.error(t('acp.attachment.openFailed'));
}
};
if (
tone === 'user'
&& part.access.status === 'available'
&& part.access.target.kind === 'local'
&& part.access.mimeType.startsWith('image/')
) {
return (
<AcpUserImageAttachment
part={part as AttachmentRenderPart & { access: Extract<AttachmentRenderPart['access'], { status: 'available' }> }}
name={name}
ariaLabel={ariaLabel}
activate={activate}
/>
);
}
return (
<button
type="button"
disabled={disabled}
aria-label={ariaLabel}
onClick={() => void activate()}
className={cn(
'flex w-full max-w-full items-center gap-3 rounded-xl border border-black/10 bg-surface-modal px-3 py-2 text-left text-sm dark:border-white/10',
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2',
disabled
? 'cursor-not-allowed text-muted-foreground opacity-70'
: 'transition-colors hover:bg-black/5 dark:hover:bg-white/5',
)}
>
<Paperclip data-testid="acp-attachment-icon" className="h-4 w-4 shrink-0" aria-hidden="true" />
{userDisplayPath && !disabled ? (
<span className="flex min-w-0 flex-1 items-baseline gap-2">
<span className="max-w-[50%] shrink-0 truncate font-medium text-foreground">{name}</span>
<span
data-testid="acp-user-attachment-path"
className="min-w-0 flex-1 truncate text-2xs text-muted-foreground"
title={userDisplayPath}
>
{userDisplayPath}
</span>
</span>
) : (
<span className="min-w-0 flex-1">
<span className="block truncate font-medium text-foreground">{name}</span>
<span className="block truncate text-2xs text-muted-foreground">
{disabled ? actionLabel : secondary}
</span>
</span>
)}
</button>
);
}
src/pages/Chat/AcpMessageSegment.tsx
+10 -16
View File
@@ -3,11 +3,12 @@ import ReactMarkdown from 'react-markdown';
import rehypeKatex from 'rehype-katex';
import remarkGfm from 'remark-gfm';
import remarkMath from 'remark-math';
import { AlertCircle, Check, Copy, FileText, Sparkles } from 'lucide-react';
import { AlertCircle, Check, Copy, Sparkles } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import type { MessageSegmentItem, RenderPart } from '@/lib/acp/timeline-types';
import { cn } from '@/lib/utils';
import { AcpImagePart, isSafeAcpImageSource } from './AcpImagePart';
import { AcpAttachmentPart } from './AcpAttachmentPart';
type RenderTone = 'assistant' | 'user' | 'process';
@@ -92,19 +93,6 @@ function AcpMarkdownPart({ text, tone }: { text: string; tone: RenderTone }) {
);
}
function AcpFilePart({ part }: { part: Extract<RenderPart, { kind: 'file' }> }) {
const { t } = useTranslation('chat');
const label = part.name || part.path || t('acp.unsupportedContent');
return (
<div className="flex max-w-full items-center gap-2 rounded-xl border border-black/10 bg-surface-input px-3 py-2 text-sm text-muted-foreground dark:border-white/10">
<FileText className="h-4 w-4 shrink-0" aria-hidden="true" />
<span className="min-w-0 truncate">{label}</span>
{part.mimeType && <span className="shrink-0 text-2xs text-muted-foreground/70">{part.mimeType}</span>}
</div>
);
}
function AcpErrorPart({ message }: { message: string }) {
const { t } = useTranslation('chat');
@@ -176,13 +164,19 @@ export function AcpRenderPart({ part, tone = 'assistant' }: { part: RenderPart;
}
if (part.kind === 'image') return <AcpImagePart part={part} />;
if (part.kind === 'file') return <AcpFilePart part={part} />;
if (part.kind === 'attachment') return <AcpAttachmentPart part={part} />;
return <AcpErrorPart message={part.message} />;
}
export function AcpMessageSegment({ item }: { item: MessageSegmentItem }) {
const isUser = item.role === 'user';
const clipboardText = useMemo(() => clipboardTextForParts(item.parts), [item.parts]);
const orderedParts = useMemo(() => isUser
? [
...item.parts.filter((part) => part.kind !== 'attachment'),
...item.parts.filter((part) => part.kind === 'attachment'),
]
: item.parts, [isUser, item.parts]);
return (
<div
@@ -197,7 +191,7 @@ export function AcpMessageSegment({ item }: { item: MessageSegmentItem }) {
</div>
)}
<div className={cn('flex min-w-0 flex-col gap-2', isUser ? 'max-w-[80%] items-end' : 'w-full items-start')}>
{item.parts.map((part, index) => (
{orderedParts.map((part, index) => (
<AcpRenderPart key={`${part.kind}:${index}`} part={part} tone={item.role} />
))}
{!isUser && clipboardText.trim().length > 0 && <AcpAssistantHoverBar text={clipboardText} />}
src/pages/Chat/AcpTimeline.tsx
+10
View File
@@ -5,6 +5,7 @@ import { AcpAssistantTurn } from './AcpAssistantTurn';
import { AcpErrorBanner } from './AcpErrorBanner';
import { AcpMessageSegment } from './AcpMessageSegment';
import type { AcpFileActivityProjection } from '@/lib/acp/openclaw-file-activities';
import { AcpAttachmentPart } from './AcpAttachmentPart';
export function AcpTimeline({
snapshot,
@@ -41,6 +42,15 @@ export function AcpTimeline({
<AcpMessageSegment item={item} />
</div>
))}
{group.attachments.length > 0 && (
<div className="flex w-full justify-end">
<div className="flex w-full max-w-[50%] flex-col items-end gap-2">
{group.attachments.map((attachment) => (
<AcpAttachmentPart key={attachment.attachmentId} part={attachment} tone="user" />
))}
</div>
</div>
)}
</div>
);
}
src/pages/Chat/AcpToolCallCard.tsx
+6 -5
View File
@@ -49,7 +49,8 @@ function AcpToolOutputPart({ part }: { part: RenderPart }) {
export function AcpToolCallCard({ item }: { item: ToolCallItem }) {
const { t } = useTranslation('chat');
const hasDetails = Boolean(item.error) || item.outputParts.length > 0;
const shouldStartExpanded = !hasDetails || !(item.historical && item.status === 'completed');
const isFinished = item.status === 'completed' || item.status === 'failed';
const shouldStartExpanded = !hasDetails || !(item.historical && isFinished);
const [expansionState, setExpansionState] = useState<ExpansionState>(() => ({
toolCallId: item.toolCallId,
expanded: shouldStartExpanded,
@@ -62,14 +63,14 @@ export function AcpToolCallCard({ item }: { item: ToolCallItem }) {
const expanded = (() => {
if (!hasDetails) return true;
if (manualOverride) return currentExpansionState.expanded;
if (item.historical && item.status === 'completed') return false;
if (item.status !== 'completed') return true;
if (item.historical && isFinished) return false;
if (!isFinished) return true;
return currentExpansionState.expanded;
})();
useEffect(() => {
if (manualOverride) return;
if (!hasDetails || item.historical || item.status !== 'completed') return;
if (!hasDetails || item.historical || !isFinished) return;
const timer = window.setTimeout(() => {
setExpansionState((state) => {
@@ -81,7 +82,7 @@ export function AcpToolCallCard({ item }: { item: ToolCallItem }) {
});
}, TOOL_AUTO_COLLAPSE_DELAY_MS);
return () => window.clearTimeout(timer);
}, [hasDetails, item.historical, item.status, item.toolCallId, manualOverride, shouldStartExpanded]);
}, [hasDetails, item.historical, isFinished, item.toolCallId, manualOverride, shouldStartExpanded]);
const toggleLabel = expanded ? t('acp.collapseTool') : t('acp.expandTool');
src/pages/Chat/ChatInput.tsx
+8 -6
View File
@@ -859,17 +859,19 @@ export function ChatInput({
data-testid="chat-composer-working-indicator"
role="status"
aria-live="polite"
aria-label={t('composer.working')}
className="relative mb-2 flex h-1 overflow-hidden rounded-full"
aria-label={t('composer.thinking')}
className="mb-2 flex h-5 items-center gap-2 text-sm text-muted-foreground"
>
<span className="sr-only">{t('composer.working')}</span>
<span
data-testid="chat-composer-zoomies"
data-testid="chat-composer-dot-pulse"
aria-hidden="true"
className="clawx-chat-working-zoomies"
className="clawx-chat-thinking-dot-pulse"
>
<span className="clawx-chat-working-zoomies-inner" />
<span className="clawx-chat-thinking-dot-pulse-inner">
<span className="clawx-chat-thinking-dot-pulse-dot" />
</span>
</span>
<span>{t('composer.thinking')}</span>
</div>
)}
src/pages/Chat/index.tsx
+27 -19
View File
@@ -166,6 +166,7 @@ export function Chat() {
const acpCancelling = useAcpChatSessionStore((s) => s.cancelling);
const acpError = useAcpChatSessionStore((s) => s.error);
const acpActiveSessionKey = useAcpChatSessionStore((s) => s.activeSessionKey);
const acpWorkspaceRoot = useAcpChatSessionStore((s) => s.workspaceRoot);
const acpCwd = useAcpChatSessionStore((s) => s.cwd);
const prepareLocalAcpSession = useAcpChatSessionStore((s) => s.prepareLocalSession);
const loadAcpSession = useAcpChatSessionStore((s) => s.loadSession);
@@ -198,7 +199,6 @@ export function Chat() {
: null;
useEffect(() => {
setResolvedWorkspaceContext(null);
if (!workspaceContextKey || !currentSessionKey || !cwd || !projectionExecutionCwd) return;
let stale = false;
void hostApi.files.resolveWorkspaceContext({
@@ -233,15 +233,16 @@ export function Chat() {
useEffect(() => {
if (!currentSessionKey || !cwd || !currentSession?.createdLocally) return;
acpLoadInFlightKeyRef.current = null;
const hasStaleTimeline = acpTimeline.sessionId !== currentSessionKey || acpTimeline.itemOrder.length > 0;
if (acpActiveSessionKey === currentSessionKey && acpCwd === cwd && !hasStaleTimeline) return;
prepareLocalAcpSession({ sessionKey: currentSessionKey, cwd });
}, [acpActiveSessionKey, acpCwd, acpTimeline.itemOrder.length, acpTimeline.sessionId, currentSession, currentSessionKey, cwd, prepareLocalAcpSession]);
if (acpActiveSessionKey === currentSessionKey && acpWorkspaceRoot === cwd && acpCwd === cwd && !hasStaleTimeline) return;
prepareLocalAcpSession({ sessionKey: currentSessionKey, workspaceRoot: cwd, cwd });
}, [acpActiveSessionKey, acpCwd, acpTimeline.itemOrder.length, acpTimeline.sessionId, acpWorkspaceRoot, currentSession, currentSessionKey, cwd, prepareLocalAcpSession]);
useEffect(() => {
if (!currentSessionKey || !cwd) return;
if (currentSessionKey === DEFAULT_SESSION_KEY && sessions.length === 0 && acpActiveSessionKey == null && !sessionDiscoveryAttempted) return;
if (acpActiveSessionKey === currentSessionKey && acpCwd === cwd) return;
if (acpActiveSessionKey === currentSessionKey && acpWorkspaceRoot === cwd && acpCwd === cwd) return;
const acpLoadKey = `${currentSessionKey}\0${cwd}`;
if (acpLoadInFlightKeyRef.current === acpLoadKey) return;
const currentSession = sessions.find((session) => session.key === currentSessionKey);
@@ -250,6 +251,7 @@ export function Chat() {
acpLoadInFlightKeyRef.current = acpLoadKey;
void loadAcpSession({
sessionKey: currentSessionKey,
workspaceRoot: cwd,
cwd,
...(createIfMissing ? { createIfMissing: true } : {}),
}).then((loaded) => {
@@ -261,7 +263,7 @@ export function Chat() {
acpLoadInFlightKeyRef.current = null;
}
});
}, [acknowledgeAcpSessionCreated, acpActiveSessionKey, acpCwd, currentSessionKey, cwd, loadAcpSession, sessionDiscoveryAttempted, sessions]);
}, [acknowledgeAcpSessionCreated, acpActiveSessionKey, acpCwd, acpWorkspaceRoot, currentSessionKey, cwd, loadAcpSession, sessionDiscoveryAttempted, sessions]);
const platform = window.electron?.platform;
const isMac = platform === 'darwin';
@@ -288,17 +290,14 @@ export function Chat() {
});
}, [acpActiveSessionKey, acpTimeline, currentSessionKey, resolvedWorkspaceContext, workspaceContextKey]);
const questionDirectoryItems = useMemo(() => {
let ordinal = 0;
return acpTimeline.itemOrder.flatMap((itemId) => {
const item = acpTimeline.itemsById[itemId];
if (item?.kind !== 'message-segment' || item.role !== 'user') return [];
ordinal += 1;
return [{
itemId: item.id,
anchorId: getAcpUserMessageAnchorId(item.id),
title: buildQuestionDirectoryTitle(item, t('questionDirectory.fallback', { number: ordinal })),
}];
});
const userItems = acpTimeline.itemOrder
.map((itemId) => acpTimeline.itemsById[itemId])
.filter((item): item is MessageSegmentItem => item?.kind === 'message-segment' && item.role === 'user');
return userItems.map((item, index) => ({
itemId: item.id,
anchorId: getAcpUserMessageAnchorId(item.id),
title: buildQuestionDirectoryTitle(item, t('questionDirectory.fallback', { number: index + 1 })),
}));
}, [acpTimeline, t]);
const questionDirectoryVisible = questionDirectoryOpenSessionKey === currentSessionKey
&& questionDirectoryItems.length > 1;
@@ -346,7 +345,9 @@ export function Chat() {
<AcpTimeline
snapshot={acpTimeline}
fileActivity={fileActivity}
workspaceRoot={resolvedWorkspaceContext?.workspaceRoot}
workspaceRoot={resolvedWorkspaceContext?.key === workspaceContextKey
? resolvedWorkspaceContext.workspaceRoot
: undefined}
onPermissionSelect={(requestId, optionId) => {
void respondAcpPermission(requestId, optionId);
}}
@@ -389,6 +390,7 @@ export function Chat() {
?.filter((file) => file.status === 'ready')
.map((file) => ({
filePath: file.stagedPath,
stagingId: file.id,
fileName: file.fileName,
mimeType: file.mimeType,
}));
@@ -398,13 +400,19 @@ export function Chat() {
void (async () => {
const existingSession = sessions.find((session) => session.key === sessionKey);
const createIfMissing = !targetAgent && (!existingSession || !!existingSession.createdLocally);
if (createIfMissing || acpActiveSessionKey !== sessionKey || acpCwd !== promptCwd) {
if (
createIfMissing
|| acpActiveSessionKey !== sessionKey
|| acpWorkspaceRoot !== promptCwd
|| acpCwd !== promptCwd
) {
const acpLoadKey = `${sessionKey}\0${promptCwd}`;
acpLoadInFlightKeyRef.current = acpLoadKey;
const loaded = await (async () => {
try {
return await loadAcpSession({
sessionKey,
workspaceRoot: promptCwd,
cwd: promptCwd,
...(createIfMissing ? { createIfMissing: true } : {}),
});
src/stores/acp-chat-session.ts
+722 -124
View File
File diff suppressed because it is too large Load Diff
src/styles/globals.css
+38 -35
View File
@@ -268,67 +268,70 @@
}
@media (prefers-reduced-motion: reduce) {
.clawx-chat-working-zoomies-inner::after {
.clawx-chat-thinking-dot-pulse-dot,
.clawx-chat-thinking-dot-pulse-inner::before,
.clawx-chat-thinking-dot-pulse-inner::after {
animation: none !important;
transform: translateX(0) !important;
transform: scale(1) !important;
}
}
.clawx-chat-working-zoomies {
--clawx-zoomies-bg-opacity: 0.14;
--clawx-zoomies-color: hsl(var(--primary));
--clawx-zoomies-speed: 1.15s;
.clawx-chat-thinking-dot-pulse {
--uib-size: 28px;
--uib-color: hsl(var(--primary));
--uib-speed: 1.3s;
--uib-dot-size: calc(var(--uib-size) * 0.24);
align-items: center;
display: inline-flex;
flex-shrink: 0;
height: 100%;
height: var(--uib-dot-size);
justify-content: center;
width: 100%;
width: var(--uib-size);
}
.clawx-chat-working-zoomies-inner {
.clawx-chat-thinking-dot-pulse-inner {
align-items: center;
border-radius: 9999px;
display: flex;
justify-content: space-between;
height: 100%;
justify-content: center;
overflow: hidden;
position: relative;
transform: translate3d(0, 0, 0);
width: 100%;
}
.clawx-chat-working-zoomies-inner::before {
background-color: var(--clawx-zoomies-color);
.clawx-chat-thinking-dot-pulse-dot,
.clawx-chat-thinking-dot-pulse-inner::before,
.clawx-chat-thinking-dot-pulse-inner::after {
background-color: var(--uib-color);
border-radius: 50%;
content: '';
height: 100%;
left: 0;
opacity: var(--clawx-zoomies-bg-opacity);
position: absolute;
top: 0;
display: block;
height: var(--uib-dot-size);
transform: scale(0);
transition: background-color 0.3s ease;
width: 100%;
width: var(--uib-dot-size);
}
.clawx-chat-working-zoomies-inner::after {
animation: clawx-chat-working-zoomies var(--clawx-zoomies-speed) ease-in-out infinite;
background-color: var(--clawx-zoomies-color);
border-radius: 9999px;
content: '';
height: 100%;
transform: translateX(-100%);
transition: background-color 0.3s ease;
width: 100%;
.clawx-chat-thinking-dot-pulse-inner::before {
animation: clawx-chat-thinking-dot-pulse var(--uib-speed) ease-in-out calc(var(--uib-speed) * -0.375) infinite;
}
.clawx-chat-thinking-dot-pulse-dot {
animation: clawx-chat-thinking-dot-pulse var(--uib-speed) ease-in-out calc(var(--uib-speed) * -0.25) infinite both;
}
.clawx-chat-thinking-dot-pulse-inner::after {
animation: clawx-chat-thinking-dot-pulse var(--uib-speed) ease-in-out calc(var(--uib-speed) * -0.125) infinite;
}
}
@keyframes clawx-chat-working-zoomies {
0% {
transform: translateX(-100%);
@keyframes clawx-chat-thinking-dot-pulse {
0%,
100% {
transform: scale(0);
}
100% {
transform: translateX(100%);
50% {
transform: scale(1);
}
}
tests/e2e/chat-acp-attachments.spec.ts
+478
View File
@@ -0,0 +1,478 @@
import type { ElectronApplication, Page } from '@playwright/test';
import * as XLSX from 'xlsx';
import {
closeElectronApp,
expect,
getRecordedLegacyIpcInvocations,
getStableWindow,
installAttachmentHostFixture,
test,
} from './fixtures/electron';
const MAIN_SESSION_KEY = 'agent:main:main';
const OTHER_SESSION_KEY = 'agent:main:other';
const PROMPT = 'Create the budget spreadsheet';
const REPLY = 'This is the budget_sample.xlsx file in the current directory.';
type AcpSessionUpdate = Record<string, unknown> & { sessionUpdate: string };
function workbookBytes(): Uint8Array {
const workbook = XLSX.utils.book_new();
XLSX.utils.book_append_sheet(workbook, XLSX.utils.aoa_to_sheet([
['Category', 'Budget'],
['Operations', 1200],
]), 'Budget');
return XLSX.write(workbook, { bookType: 'xlsx', type: 'buffer' });
}
function reportedFlowUpdates(): AcpSessionUpdate[] {
return [{
sessionUpdate: 'agent_message',
messageId: 'budget-reply',
content: [{ type: 'text', text: REPLY }],
}];
}
function userUpdate(messageId: string, text: string): AcpSessionUpdate {
return {
sessionUpdate: 'user_message',
messageId,
content: [{ type: 'text', text }],
};
}
function resourceUpdate(input: {
messageId: string;
uri: string;
name: string;
mimeType: string;
text?: string;
}): AcpSessionUpdate {
return {
sessionUpdate: 'agent_message',
messageId: input.messageId,
content: [
...(input.text ? [{ type: 'text', text: input.text }] : []),
{
type: 'resource_link',
uri: input.uri,
name: input.name,
mimeType: input.mimeType,
},
],
};
}
async function openChat(app: ElectronApplication): Promise<Page> {
const page = await getStableWindow(app);
try {
await page.reload();
} catch (error) {
if (!String(error).includes('ERR_FILE_NOT_FOUND')) throw error;
}
await expect(page.getByTestId('main-layout')).toBeVisible();
await expect(page.getByTestId('chat-page')).toBeVisible();
return page;
}
test.describe('ACP media attachments', () => {
test('renders user image thumbnails and actionable file paths', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [{ key: MAIN_SESSION_KEY, title: 'Main session' }],
});
const imageBytes = Uint8Array.from(Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
'base64',
));
const imagePath = await fixture.createWorkspaceFile('uploads/photo.png', imageBytes);
const notesPath = await fixture.createWorkspaceFile('uploads/notes.txt', 'Preview this user attachment.');
const displayImagePath = '/Users/test/Pictures/photo.png';
const displayNotesPath = '/Users/test/Documents/a/very/long/path/notes.txt';
await fixture.registerStagedAttachment('stage-photo', imagePath, displayImagePath);
await fixture.registerStagedAttachment('stage-notes', notesPath, displayNotesPath);
await fixture.setSessionReplay(MAIN_SESSION_KEY, [{
sessionUpdate: 'user_message',
messageId: 'user-attachments',
content: [
{ type: 'text', text: 'Review these files.' },
{
type: 'image',
uri: imagePath,
data: Buffer.from(imageBytes).toString('base64'),
mimeType: 'image/png',
_meta: { clawx: { stagingId: 'stage-photo', fileName: 'photo.png' } },
},
{
type: 'resource_link',
uri: notesPath,
name: 'notes.txt',
mimeType: 'text/plain',
_meta: { clawx: { stagingId: 'stage-notes' } },
},
],
}]);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
const page = await openChat(app);
const userMessage = page.getByTestId('acp-user-message');
await expect(userMessage.getByText('Review these files.')).toBeVisible({ timeout: 30_000 });
const thumbnail = page.getByTestId('acp-user-image-attachment');
await expect(thumbnail).toBeVisible();
await expect(thumbnail).toHaveAttribute('alt', 'photo.png');
const [bubbleBox, thumbnailBox] = await Promise.all([
userMessage.locator('.bg-brand').first().boundingBox(),
thumbnail.locator('..').boundingBox(),
]);
expect(bubbleBox).not.toBeNull();
expect(thumbnailBox).not.toBeNull();
expect(Math.abs((bubbleBox!.x + bubbleBox!.width) - (thumbnailBox!.x + thumbnailBox!.width))).toBeLessThanOrEqual(1);
await expect(page.getByTestId('acp-user-image-overlay')).toContainText('photo.png');
const notes = page.getByRole('button', { name: 'Preview notes.txt' });
await expect(notes).toContainText(displayNotesPath);
await expect(notes).not.toContainText('text/plain');
await notes.click();
const panel = page.getByTestId('artifact-panel');
await expect(panel).toBeVisible();
await expect(panel.getByText('Preview this user attachment.')).toBeVisible({ timeout: 30_000 });
await expect.poll(async () => (await fixture.getHostInvocations()).some((call) => (
call.module === 'media'
&& call.action === 'thumbnails'
&& Array.isArray(call.payload?.paths)
&& (call.payload.paths as Array<Record<string, unknown>>).some((entry) => (
(entry.attachmentFileRef as Record<string, unknown> | undefined)?.uri === imagePath
))
))).toBe(true);
} finally {
await closeElectronApp(app);
}
});
test('previews the reported live spreadsheet flow and restores one historical card', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [
{ key: MAIN_SESSION_KEY, title: 'Main session' },
{ key: OTHER_SESSION_KEY, title: 'Other session' },
],
});
const spreadsheetPath = await fixture.createWorkspaceFile('budget_sample.xlsx', workbookBytes());
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
await fixture.setPromptUpdates(PROMPT, reportedFlowUpdates());
const page = await openChat(app);
await fixture.waitForHistoryRequestCount(MAIN_SESSION_KEY, 1);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[
{ role: 'user', id: 'transcript-user-budget', content: PROMPT },
{
role: 'assistant',
id: 'transcript-assistant-budget',
content: `MEDIA:${spreadsheetPath}\n${REPLY}`,
},
]]);
await page.getByTestId('chat-composer-input').fill(PROMPT);
await page.getByTestId('chat-composer-send').click();
const turn = page.getByTestId('acp-assistant-turn').last();
const prose = turn.getByText(REPLY);
const attachment = turn.getByRole('button').filter({ hasText: 'budget_sample.xlsx' });
await expect(prose).toBeVisible({ timeout: 30_000 });
await fixture.waitForHistoryRequestCount(MAIN_SESSION_KEY, 2);
await expect(attachment).toBeVisible();
await expect(page.getByText(/MEDIA:/)).toHaveCount(0);
await expect.poll(async () => prose.evaluate((node, card) => (
Boolean(node.compareDocumentPosition(card as Node) & Node.DOCUMENT_POSITION_FOLLOWING)
), await attachment.elementHandle())).toBe(true);
await attachment.click();
const panel = page.getByTestId('artifact-panel');
await expect(panel).toBeVisible();
await expect(panel.getByTestId('artifact-panel-tab-preview')).toBeVisible();
await expect(panel.getByText('Operations')).toBeVisible({ timeout: 30_000 });
await expect.poll(async () => (await fixture.getHostInvocations()).some((call) => (
call.module === 'files'
&& call.action === 'readAttachmentBinary'
&& call.payload?.ref
&& (call.payload.ref as Record<string, unknown>).uri === spreadsheetPath
))).toBe(true);
await fixture.setSessionReplay(MAIN_SESSION_KEY, [
{ sessionUpdate: 'user_message', messageId: 'history-user-budget', content: [{ type: 'text', text: PROMPT }] },
...reportedFlowUpdates(),
]);
await page.getByTestId(`sidebar-session-${OTHER_SESSION_KEY}`).click();
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible();
await page.getByTestId(`sidebar-session-${MAIN_SESSION_KEY}`).click();
await expect(page.getByRole('button').filter({ hasText: 'budget_sample.xlsx' })).toHaveCount(1, { timeout: 30_000 });
await expect(page.getByText(/MEDIA:/)).toHaveCount(0);
} finally {
await closeElectronApp(app);
}
});
test('renders native ACP resources without transcript evidence and prefers them over duplicate MEDIA evidence', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [
{ key: MAIN_SESSION_KEY, title: 'Main session' },
{ key: OTHER_SESSION_KEY, title: 'Other session' },
],
});
const spreadsheetPath = await fixture.createWorkspaceFile('native-budget.xlsx', workbookBytes());
const replay = [
userUpdate('native-user', 'Show the native budget'),
resourceUpdate({
messageId: 'native-reply',
uri: spreadsheetPath,
name: 'Native budget.xlsx',
mimeType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
text: 'The native ACP resource is ready.',
}),
];
await fixture.setSessionReplay(MAIN_SESSION_KEY, replay);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
const page = await openChat(app);
const nativeCard = page.getByRole('button').filter({ hasText: 'Native budget.xlsx' });
await expect(nativeCard).toBeEnabled({ timeout: 30_000 });
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[
{ role: 'user', id: 'native-transcript-user', content: 'Show the native budget' },
{
role: 'assistant',
id: 'native-transcript-assistant',
content: `MEDIA:${spreadsheetPath}\nThe native ACP resource is ready.`,
},
]]);
await page.getByTestId(`sidebar-session-${OTHER_SESSION_KEY}`).click();
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible();
await page.getByTestId(`sidebar-session-${MAIN_SESSION_KEY}`).click();
await expect(page.getByRole('button').filter({ hasText: 'Native budget.xlsx' })).toHaveCount(1, { timeout: 30_000 });
await expect(page.getByRole('button').filter({ hasText: 'native-budget.xlsx' })).toHaveCount(0);
await expect(page.getByText(/MEDIA:/)).toHaveCount(0);
} finally {
await closeElectronApp(app);
}
});
test('routes ZIP and HTTPS attachments through validated Main system-open operations', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
const remoteUrl = 'https://example.test/files/remote-archive.zip';
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [{ key: MAIN_SESSION_KEY, title: 'Main session' }],
});
const zipPath = await fixture.createOpenClawMediaFile('exports/budget-archive.zip', Uint8Array.from([80, 75, 3, 4]));
await fixture.setSessionReplay(MAIN_SESSION_KEY, [
userUpdate('open-user', 'Open the archives'),
{
sessionUpdate: 'agent_message',
messageId: 'open-reply',
content: [
{ type: 'text', text: 'Both archives are ready.' },
{ type: 'resource_link', uri: zipPath, name: 'budget-archive.zip', mimeType: 'application/zip' },
{ type: 'resource_link', uri: remoteUrl, name: 'remote-archive.zip', mimeType: 'application/zip' },
],
},
]);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
const page = await openChat(app);
const localCard = page.getByRole('button').filter({ hasText: 'budget-archive.zip' });
const remoteCard = page.getByRole('button').filter({ hasText: 'remote-archive.zip' });
await expect(localCard).toBeEnabled({ timeout: 30_000 });
await expect(remoteCard).toBeEnabled();
await fixture.clearInvocations();
await localCard.click();
await expect.poll(async () => (await fixture.getShellInvocations()).some((call) => (
call.action === 'openPath' && call.payload?.path === zipPath
))).toBe(true);
await remoteCard.click();
await expect.poll(async () => (await fixture.getShellInvocations()).some((call) => (
call.action === 'openExternal' && call.payload?.url === remoteUrl
))).toBe(true);
const hostCalls = await fixture.getHostInvocations();
expect(hostCalls.filter((call) => call.module === 'files' && call.action === 'openAttachment')).toHaveLength(2);
expect(await getRecordedLegacyIpcInvocations(app)).toEqual([]);
} finally {
await closeElectronApp(app);
}
});
test('renders outside-workspace paths unavailable without reading or opening them', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [{ key: MAIN_SESSION_KEY, title: 'Main session' }],
});
const outsidePath = await fixture.createOutsideFile('private.txt', 'not authorized');
await fixture.setSessionReplay(MAIN_SESSION_KEY, [
userUpdate('outside-user', 'Show the private file'),
resourceUpdate({
messageId: 'outside-reply',
uri: outsidePath,
name: 'private.txt',
mimeType: 'text/plain',
text: 'The requested path is not in the workspace.',
}),
]);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
const page = await openChat(app);
const unavailableCard = page.getByRole('button').filter({ hasText: 'private.txt' });
await expect(unavailableCard).toBeDisabled({ timeout: 30_000 });
await expect(unavailableCard).toHaveAccessibleName('Attachment unavailable: private.txt');
await expect(unavailableCard).toContainText('Attachment unavailable');
const calls = await fixture.getHostInvocations();
expect(calls.some((call) => (
call.module === 'files'
&& call.action === 'resolveAttachment'
&& call.payload?.ref
&& (call.payload.ref as Record<string, unknown>).uri === outsidePath
))).toBe(true);
expect(calls.some((call) => (
call.module === 'files'
&& (call.action === 'readAttachmentBinary' || call.action === 'openAttachment')
))).toBe(false);
expect(await fixture.getShellInvocations()).toEqual([]);
} finally {
await closeElectronApp(app);
}
});
test('drops a delayed 1500 ms transcript retry after switching sessions', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
const retryPrompt = 'Prepare the delayed attachment';
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [
{ key: MAIN_SESSION_KEY, title: 'Main session' },
{ key: OTHER_SESSION_KEY, title: 'Other session' },
],
});
const delayedPath = await fixture.createWorkspaceFile('delayed.txt', 'delayed attachment');
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
await fixture.setPromptUpdates(retryPrompt, [{
sessionUpdate: 'agent_message',
messageId: 'delayed-reply',
content: [{ type: 'text', text: 'The attachment will arrive shortly.' }],
}]);
const page = await openChat(app);
await fixture.waitForHistoryRequestCount(MAIN_SESSION_KEY, 1);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [
[],
{
deferId: 'delayed-retry',
messages: [
{ role: 'user', id: 'delayed-transcript-user', content: retryPrompt },
{
role: 'assistant',
id: 'delayed-transcript-assistant',
content: `MEDIA:${delayedPath}\nThe attachment will arrive shortly.`,
},
],
},
]);
await expect(fixture.releaseTranscriptResponse('delayed-retry')).rejects.toThrow(
'Deferred transcript response is not ready: delayed-retry',
);
await page.getByTestId('chat-composer-input').fill(retryPrompt);
await page.getByTestId('chat-composer-send').click();
const requestTimes = await fixture.waitForHistoryRequestCount(MAIN_SESSION_KEY, 3, 6_000);
expect(requestTimes[2]! - requestTimes[1]!).toBeGreaterThanOrEqual(1_400);
await fixture.waitForDeferredTranscriptReady('delayed-retry');
await page.getByTestId(`sidebar-session-${OTHER_SESSION_KEY}`).click();
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible();
await fixture.releaseTranscriptResponse('delayed-retry');
await fixture.waitForDeferredTranscriptCompleted('delayed-retry');
await expect(page.getByRole('button').filter({ hasText: 'delayed.txt' })).toHaveCount(0);
await expect(page.getByText('The attachment will arrive shortly.')).toHaveCount(0);
expect((await fixture.getHostInvocations()).some((call) => (
call.module === 'files'
&& (call.action === 'readAttachmentBinary' || call.action === 'openAttachment')
))).toBe(false);
} finally {
await closeElectronApp(app);
}
});
test('lifts an early attachment after later prose and before file activity', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
const fixture = await installAttachmentHostFixture(app, {
sessions: [{ key: MAIN_SESSION_KEY, title: 'Main session' }],
});
const earlyPath = await fixture.createWorkspaceFile('early.txt', 'early attachment');
await fixture.setSessionReplay(MAIN_SESSION_KEY, [
userUpdate('ordering-user', 'Create the ordered output'),
resourceUpdate({
messageId: 'ordering-resource',
uri: earlyPath,
name: 'early.txt',
mimeType: 'text/plain',
}),
{
sessionUpdate: 'tool_call',
toolCallId: 'ordering-write',
title: 'Write: activity.txt',
status: 'in_progress',
rawInput: { path: 'activity.txt', content: 'created' },
content: [{ type: 'content', content: { type: 'text', text: 'Writing activity.txt' } }],
locations: [],
},
{
sessionUpdate: 'tool_call_update',
toolCallId: 'ordering-write',
title: 'Write: activity.txt',
status: 'completed',
content: [{ type: 'content', content: { type: 'text', text: 'Write complete' } }],
locations: [],
},
{
sessionUpdate: 'agent_message',
messageId: 'ordering-prose',
content: [{ type: 'text', text: 'The ordered output is complete.' }],
},
]);
await fixture.setTranscriptResponses(MAIN_SESSION_KEY, [[]]);
const page = await openChat(app);
const turn = page.getByTestId('acp-assistant-turn');
const prose = turn.getByText('The ordered output is complete.');
const attachment = turn.getByRole('button').filter({ hasText: 'early.txt' });
const activity = turn.getByTestId('acp-turn-file-activity');
await expect(prose).toBeVisible({ timeout: 30_000 });
await expect(attachment).toBeEnabled();
await expect(activity).toBeVisible();
const attachmentHandle = await attachment.elementHandle();
const activityHandle = await activity.elementHandle();
if (!attachmentHandle || !activityHandle) throw new Error('Expected ordered attachment and activity elements');
expect(await prose.evaluate((node, card) => (
Boolean(node.compareDocumentPosition(card as Node) & Node.DOCUMENT_POSITION_FOLLOWING)
), attachmentHandle)).toBe(true);
expect(await attachment.evaluate((node, summary) => (
Boolean(node.compareDocumentPosition(summary as Node) & Node.DOCUMENT_POSITION_FOLLOWING)
), activityHandle)).toBe(true);
} finally {
await closeElectronApp(app);
}
});
});
tests/e2e/chat-acp-inline-timeline.spec.ts
+135 -26
View File
@@ -9,6 +9,7 @@ const REVIEWER_WORKSPACE = '/workspace/reviewer';
const IMAGE_TASK_ID = '0d2ee919-2dfd-4b72-9da3-d87e6ee56747';
const GENERATED_IMAGE_PATH = '/workspace/.openclaw/media/tool-image-generation/generated-image.png';
const GENERATED_IMAGE_PREVIEW = 'data:image/png;base64,iVBORw0KGgo=';
const GENERATED_IMAGE_IDENTITY = 'e2e-transcript-generated-image';
const DEFAULT_WORKSPACE_SEGMENT = '~%2F.openclaw%2Fworkspace';
type AcpSessionUpdate = Record<string, unknown> & { sessionUpdate: string };
@@ -28,12 +29,12 @@ function defaultWorkspaceSessionGroupTestId(): string {
function baseHostApiMocks(loadResult: Record<string, unknown> = { success: true, generation: 1 }) {
return {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: MAIN_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: '/' }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: '/', createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: '/', cwd: '/' }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: '/', cwd: '/', createIfMissing: true }])]: loadResult,
[stableStringify(['/api/agents', 'GET'])]: {
ok: true,
data: {
@@ -53,7 +54,10 @@ function baseHostApiMocks(loadResult: Record<string, unknown> = { success: true,
};
}
async function installAcpChatMocks(app: ElectronApplication) {
async function installAcpChatMocks(
app: ElectronApplication,
loadResult: Record<string, unknown> = { success: true, generation: 1 },
) {
await installIpcMocks(app, {
gatewayStatus: { state: 'running', gatewayReady: true, port: 18789, pid: 12345 },
gatewayRpc: {
@@ -64,22 +68,26 @@ async function installAcpChatMocks(app: ElectronApplication) {
},
},
},
hostApi: baseHostApiMocks(),
hostApi: baseHostApiMocks(loadResult),
});
}
async function installAcpLoadReplayMock(app: ElectronApplication, updates: AcpSessionUpdate[]) {
await app.evaluate(async ({ app: _app }, payload) => {
const { BrowserWindow, ipcMain } = process.mainModule!.require('electron') as typeof import('electron');
const { ipcMain } = process.mainModule!.require('electron') as typeof import('electron');
type IpcInvokeHandler = (event: unknown, request: { id?: string; module?: string; action?: string; args?: unknown[] }) => Promise<unknown>;
const handlers = (ipcMain as unknown as { _invokeHandlers?: Map<string, IpcInvokeHandler> })._invokeHandlers;
const originalHostInvoke = handlers?.get('host:invoke');
ipcMain.removeHandler('host:invoke');
ipcMain.handle('host:invoke', async (event: unknown, request: { id?: string; module?: string; action?: string; args?: unknown[] }) => {
if (request?.module === 'chat' && request.action === 'loadAcpSession') {
for (const update of payload.updates as AcpSessionUpdate[]) {
for (const window of BrowserWindow.getAllWindows()) {
window.webContents.send('chat:acp-session-update', {
return {
id: request.id,
ok: true,
data: {
success: true,
generation: 1,
sessionUpdates: (payload.updates as AcpSessionUpdate[]).map((update) => ({
sessionKey: payload.sessionKey,
generation: 1,
historical: true,
@@ -87,10 +95,9 @@ async function installAcpLoadReplayMock(app: ElectronApplication, updates: AcpSe
sessionId: payload.sessionKey,
update,
},
});
}
}
return { id: request.id, ok: true, data: { success: true, generation: 1 } };
})),
},
};
}
return originalHostInvoke?.(event, request) ?? { id: request?.id, ok: true, data: {} };
});
@@ -276,6 +283,73 @@ async function openChat(app: ElectronApplication) {
}
test.describe('ClawX ACP inline timeline', () => {
test('commits a long historical replay without exposing partial assistant text', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
const paragraphChunks = Array.from({ length: 12 }, (_, index) => `Paragraph ${index + 1}.\n\n`);
const sessionUpdates = [
{
sessionKey: MAIN_SESSION_KEY,
generation: 1,
historical: true,
notification: {
sessionId: MAIN_SESSION_KEY,
update: {
sessionUpdate: 'user_message_chunk',
messageId: 'long-history-user',
content: { type: 'text', text: 'Write a 12-paragraph article' },
},
},
},
...paragraphChunks.map((text) => ({
sessionKey: MAIN_SESSION_KEY,
generation: 1,
historical: true,
notification: {
sessionId: MAIN_SESSION_KEY,
update: {
sessionUpdate: 'agent_message_chunk',
messageId: 'long-history-assistant',
content: { type: 'text', text },
},
},
})),
];
try {
await installAcpChatMocks(app, { success: true, generation: 1, sessionUpdates });
const initialPage = await getStableWindow(app);
await initialPage.addInitScript(() => {
const observedLengths: number[] = [];
Object.defineProperty(window, '__acpObservedAssistantLengths', {
value: observedLengths,
configurable: true,
});
const observer = new MutationObserver(() => {
const assistant = document.querySelector('[data-testid="acp-assistant-message"]');
const length = assistant?.textContent?.length ?? 0;
if (length > 0 && observedLengths.at(-1) !== length) observedLengths.push(length);
});
const observe = () => {
observer.observe(document.documentElement, { childList: true, characterData: true, subtree: true });
};
if (document.documentElement) observe();
else window.addEventListener('DOMContentLoaded', observe, { once: true });
});
const page = await openChat(app);
const assistant = page.getByTestId('acp-assistant-message');
await expect(assistant).toContainText('Paragraph 1.', { timeout: 30_000 });
await expect(assistant).toContainText('Paragraph 12.');
const finalLength = await assistant.evaluate((element) => element.textContent?.length ?? 0);
const observedLengths = await page.evaluate(() => (
(window as unknown as { __acpObservedAssistantLengths?: number[] }).__acpObservedAssistantLengths ?? []
));
expect(observedLengths).toEqual([finalLength]);
} finally {
await closeElectronApp(app);
}
});
test('renders ACP tool updates inline without the legacy execution graph', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
@@ -585,10 +659,44 @@ test.describe('ClawX ACP inline timeline', () => {
},
],
},
[stableStringify(['media', 'thumbnails', {
paths: [{ filePath: GENERATED_IMAGE_PATH, mimeType: 'image/png' }],
[stableStringify(['files', 'resolveAttachment', {
ref: {
sessionKey: MAIN_SESSION_KEY,
generation: 1,
uri: GENERATED_IMAGE_PATH,
transcriptMessageId: 'transcript-image-complete',
},
mimeType: 'image/png',
}])]: {
[GENERATED_IMAGE_PATH]: { preview: GENERATED_IMAGE_PREVIEW, fileSize: 128 },
ok: true,
identity: GENERATED_IMAGE_IDENTITY,
displayName: 'generated-image.png',
mimeType: 'image/png',
size: 128,
target: {
kind: 'local',
scope: 'openclaw-media',
ref: {
sessionKey: MAIN_SESSION_KEY,
generation: 1,
uri: GENERATED_IMAGE_PATH,
transcriptMessageId: 'transcript-image-complete',
},
},
},
[stableStringify(['media', 'thumbnails', {
paths: [{
attachmentFileRef: {
sessionKey: MAIN_SESSION_KEY,
generation: 1,
uri: GENERATED_IMAGE_PATH,
transcriptMessageId: 'transcript-image-complete',
},
key: GENERATED_IMAGE_IDENTITY,
mimeType: 'image/png',
}],
}])]: {
[GENERATED_IMAGE_IDENTITY]: { preview: GENERATED_IMAGE_PREVIEW, fileSize: 128 },
},
[stableStringify(['media', 'saveImage', {
base64: 'iVBORw0KGgo=',
@@ -640,7 +748,6 @@ test.describe('ClawX ACP inline timeline', () => {
await expect(page.getByTestId('acp-chat-timeline')).toBeVisible({ timeout: 30_000 });
await expect(page.getByTestId('acp-tool-call-card')).toContainText('Generate image');
await expect(page.getByText('Generated image is ready.')).toBeVisible({ timeout: 30_000 });
const imagePart = page.getByTestId('acp-image-part');
const image = imagePart.locator('img');
await expect(image).toBeVisible();
@@ -730,7 +837,7 @@ test.describe('ClawX ACP inline timeline', () => {
}
});
test('shows the composer Zoomies indicator only while sending', async ({ launchElectronApp }) => {
test('shows the composer dot pulse and thinking label only while sending', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
@@ -739,17 +846,19 @@ test.describe('ClawX ACP inline timeline', () => {
const page = await openChat(app);
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible({ timeout: 30_000 });
await expect(page.getByTestId('chat-composer-working-indicator')).toHaveCount(0);
await expect(page.getByTestId('chat-composer-zoomies')).toHaveCount(0);
await expect(page.getByTestId('chat-composer-dot-pulse')).toHaveCount(0);
await page.getByTestId('chat-composer-input').fill('Hold the send state');
await page.getByTestId('chat-composer-send').click();
await expect(page.getByTestId('chat-composer-working-indicator')).toBeVisible({ timeout: 30_000 });
await expect(page.getByTestId('chat-composer-zoomies')).toBeVisible();
await expect(page.getByTestId('chat-composer-working-indicator')).toContainText('Thinking…');
await expect(page.getByTestId('chat-composer-dot-pulse')).toBeVisible();
await expect(page.getByTestId('chat-composer-zoomies')).toHaveCount(0);
await resolveDeferredAcpPrompt(app);
await expect(page.getByTestId('chat-composer-working-indicator')).toHaveCount(0, { timeout: 30_000 });
await expect(page.getByTestId('chat-composer-zoomies')).toHaveCount(0);
await expect(page.getByTestId('chat-composer-dot-pulse')).toHaveCount(0);
} finally {
await closeElectronApp(app);
}
@@ -805,11 +914,11 @@ test.describe('ClawX ACP inline timeline', () => {
},
hostApi: {
...baseHostApiMocks(),
[stableStringify(['chat', 'loadAcpSession', { sessionKey: REVIEWER_SESSION_KEY, cwd: REVIEWER_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: REVIEWER_SESSION_KEY, workspaceRoot: REVIEWER_WORKSPACE, cwd: REVIEWER_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: REVIEWER_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: REVIEWER_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-assistant-markdown-plain.spec.ts
+4 -4
View File
@@ -81,19 +81,19 @@ test.describe('ClawX assistant reply Markdown styling', () => {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-code-block-wrap.spec.ts
+4 -4
View File
@@ -84,19 +84,19 @@ test.describe('ClawX chat code block wrapping', () => {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-history-startup-retry.spec.ts
+4 -4
View File
@@ -28,19 +28,19 @@ async function installAcpStartupMocks(app: ElectronApplication) {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-latex-rendering.spec.ts
+4 -4
View File
@@ -82,19 +82,19 @@ test.describe('ClawX chat LaTeX rendering', () => {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-new-session-date.spec.ts
+2 -2
View File
@@ -73,7 +73,7 @@ test.describe('ClawX chat workspace session list', () => {
json: { success: true, agents: [{ id: 'main', name: 'Main' }] },
},
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
@@ -167,7 +167,7 @@ test.describe('ClawX chat workspace session list', () => {
json: { success: true, agents: [{ id: 'main', name: 'Main' }] },
},
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-question-directory.spec.ts
+4 -4
View File
@@ -58,19 +58,19 @@ async function installQuestionDirectoryMocks(
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-run-state-events.spec.ts
+130 -45
View File
@@ -25,9 +25,21 @@ function baseHostApiMocks(
overrides: Record<string, unknown> = {},
) {
return {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: MAIN_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: '/' }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', {
sessionKey: MAIN_SESSION_KEY,
workspaceRoot: MAIN_WORKSPACE,
cwd: MAIN_WORKSPACE,
}])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', {
sessionKey: MAIN_SESSION_KEY,
workspaceRoot: '/',
cwd: '/',
}])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', {
sessionKey: MAIN_SESSION_KEY,
workspaceRoot: DEFAULT_WORKSPACE,
cwd: DEFAULT_WORKSPACE,
}])]: loadResult,
[stableStringify(['sessions', 'summaries', { sessionKeys: [MAIN_SESSION_KEY] }])]: { summaries: [] },
[stableStringify(['/api/agents', 'GET'])]: {
ok: true,
@@ -49,6 +61,28 @@ function baseHostApiMocks(
};
}
function generatedImageHostApiMocks(generatedPath: string, identity: string): Record<string, unknown> {
const ref = { sessionKey: MAIN_SESSION_KEY, generation: 1, uri: generatedPath };
return {
[stableStringify(['files', 'resolveAttachment', {
ref,
mimeType: 'image/png',
}])]: {
ok: true,
identity,
displayName: generatedPath.split('/').at(-1) ?? 'generated.png',
mimeType: 'image/png',
size: 67,
target: { kind: 'local', scope: 'openclaw-media', ref },
},
[stableStringify(['media', 'thumbnails', {
paths: [{ attachmentFileRef: ref, key: identity, mimeType: 'image/png' }],
}])]: {
[identity]: { preview: ONE_PIXEL_PNG_DATA_URL, fileSize: 67 },
},
};
}
async function installAcpChatMocks(
app: ElectronApplication,
loadResult: Record<string, unknown> = { success: true, generation: 1 },
@@ -68,15 +102,15 @@ async function installAcpChatMocks(
});
}
async function emitChatRuntimeEvent(
async function emitGatewayChatMessage(
app: ElectronApplication,
payload: Record<string, unknown>,
) {
await app.evaluate(
async ({ app: _app }, runtimePayload) => {
async ({ app: _app }, chatPayload) => {
const { BrowserWindow } = process.mainModule!.require('electron') as typeof import('electron');
for (const window of BrowserWindow.getAllWindows()) {
window.webContents.send('chat:runtime-event', runtimePayload);
window.webContents.send('gateway:chat-message', chatPayload);
}
},
payload,
@@ -249,13 +283,11 @@ test.describe('ClawX chat run state events', () => {
const generatedPath = '/tmp/openclaw-generated-sky.png';
try {
await installAcpChatMocks(app, { success: true, generation: 1 }, {
[stableStringify(['media', 'thumbnails', {
paths: [{ filePath: generatedPath, mimeType: 'image/png' }],
}])]: {
[generatedPath]: { preview: ONE_PIXEL_PNG_DATA_URL, fileSize: 67 },
},
});
await installAcpChatMocks(
app,
{ success: true, generation: 1 },
generatedImageHostApiMocks(generatedPath, 'e2e-live-generated-image'),
);
const page = await openChat(app);
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible({ timeout: 30_000 });
@@ -278,25 +310,22 @@ test.describe('ClawX chat run state events', () => {
await expect(timeline).toBeVisible({ timeout: 30_000 });
await expect(page.getByTestId('acp-tool-call-card')).toContainText('Background task started for image generation');
await emitChatRuntimeEvent(app, {
type: 'tool.completed',
sessionKey: `image_generate:${IMAGE_GENERATION_TASK_ID}`,
runId: `image_generate:${IMAGE_GENERATION_TASK_ID}:ok`,
toolCallId: 'message-tool',
name: 'message',
result: {
details: {
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: {
mediaUrls: [generatedPath],
},
await emitGatewayChatMessage(app, {
message: {
runId: `image_generate:${IMAGE_GENERATION_TASK_ID}:ok`,
sessionKey: MAIN_SESSION_KEY,
state: 'final',
message: {
role: 'assistant',
content: [{
type: 'text',
text: `Here is the exact sky scene you requested.\n\nMEDIA:${generatedPath}`,
}],
},
},
});
await expect(page.getByText('Generated image is ready.')).toBeVisible();
await expect(page.getByText('Here is the exact sky scene you requested.')).toBeVisible();
await expect(timeline.getByTestId('acp-image-part')).toBeVisible();
const image = timeline.getByRole('img', { name: 'Image' });
await expect(image).toBeVisible();
@@ -308,18 +337,60 @@ test.describe('ClawX chat run state events', () => {
}
});
test('projects OpenClaw image-generation failures as ACP Chat replies', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
try {
await installAcpChatMocks(app, { success: true, generation: 1 });
const page = await openChat(app);
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible({ timeout: 30_000 });
await emitAcpSessionUpdates(app, [{
sessionUpdate: 'tool_call_update',
toolCallId: 'image-tool',
status: 'completed',
content: [{
type: 'content',
content: {
type: 'text',
text: `Background task started for image generation (${IMAGE_GENERATION_TASK_ID}).`,
},
}],
locations: [],
}]);
await emitGatewayChatMessage(app, {
message: {
runId: `image_generate:${IMAGE_GENERATION_TASK_ID}:error`,
sessionKey: MAIN_SESSION_KEY,
state: 'final',
message: {
role: 'assistant',
content: [{
type: 'text',
text: 'Image generation failed because no image model is available.',
}],
},
},
});
await expect(page.getByText('Image generation failed because no image model is available.')).toBeVisible();
await expect(page.getByTestId('acp-image-part')).toHaveCount(0);
} finally {
await closeElectronApp(app);
}
});
test('replays OpenClaw image-generation previews from historical ACP tool output', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
const generatedPath = '/tmp/openclaw-replayed-sky.png';
try {
await installAcpChatMocks(app, { success: true, generation: 1 }, {
[stableStringify(['media', 'thumbnails', {
paths: [{ filePath: generatedPath, mimeType: 'image/png' }],
}])]: {
[generatedPath]: { preview: ONE_PIXEL_PNG_DATA_URL, fileSize: 67 },
},
});
await installAcpChatMocks(
app,
{ success: true, generation: 1 },
generatedImageHostApiMocks(generatedPath, 'e2e-replayed-generated-image'),
);
const page = await openChat(app);
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible({ timeout: 30_000 });
@@ -353,6 +424,7 @@ test.describe('ClawX chat run state events', () => {
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: {
text: 'Generated image is ready.',
mediaUrls: [generatedPath],
},
},
@@ -377,13 +449,11 @@ test.describe('ClawX chat run state events', () => {
const generatedPath = '/Users/me/.openclaw/media/tool-image-generation/clawx-image-1.png';
try {
await installAcpChatMocks(app, { success: true, generation: 1 }, {
[stableStringify(['media', 'thumbnails', {
paths: [{ filePath: generatedPath, mimeType: 'image/png' }],
}])]: {
[generatedPath]: { preview: ONE_PIXEL_PNG_DATA_URL, fileSize: 67 },
},
});
await installAcpChatMocks(
app,
{ success: true, generation: 1 },
generatedImageHostApiMocks(generatedPath, 'e2e-transcript-generated-image'),
);
const page = await openChat(app);
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible({ timeout: 30_000 });
@@ -406,14 +476,14 @@ test.describe('ClawX chat run state events', () => {
messageId: 'replayed-image-result',
content: {
type: 'text',
text: `图片生成完成！这是为你创建的蓝天白云风景图。\n\nMEDIA:${generatedPath}`,
text: `Generated image is ready.\n\nMEDIA:${generatedPath}`,
},
},
], 1, true);
const timeline = page.getByTestId('acp-chat-timeline');
await expect(timeline).toBeVisible({ timeout: 30_000 });
await expect(page.getByText('Generated image is ready.')).toBeVisible();
await expect(page.getByText('Generated image is ready.').first()).toBeVisible();
await expect(timeline.getByTestId('acp-image-part')).toBeVisible();
const image = timeline.getByRole('img', { name: 'Image' });
await expect(image).toBeVisible();
@@ -479,9 +549,23 @@ test.describe('ClawX chat run state events', () => {
test('renders ACP SVG image content without legacy MEDIA marker leakage', async ({ launchElectronApp }) => {
const app = await launchElectronApp({ skipSetup: true });
const filePath = String.raw`C:\Users\Administrator\.openclaw\workspace\japan-kansai-4d3n-plan.svg`;
const attachmentRef = { sessionKey: MAIN_SESSION_KEY, generation: 1, uri: filePath };
try {
await installAcpChatMocks(app);
await installAcpChatMocks(app, { success: true, generation: 1 }, {
[stableStringify(['files', 'resolveAttachment', {
ref: attachmentRef,
name: 'japan-kansai-4d3n-plan.svg',
mimeType: 'image/svg+xml',
}])]: {
ok: true,
identity: 'e2e-windows-svg',
displayName: 'japan-kansai-4d3n-plan.svg',
mimeType: 'image/svg+xml',
size: 128,
target: { kind: 'local', scope: 'workspace', ref: attachmentRef },
},
});
const page = await openChat(app);
await expect(page.getByTestId('acp-chat-empty-state')).toBeVisible({ timeout: 30_000 });
@@ -506,6 +590,7 @@ test.describe('ClawX chat run state events', () => {
await expect(page.getByText('SVG file is ready:')).toBeVisible();
await expect(page.getByText('MEDIA:C:')).toHaveCount(0);
await expect(page.getByText('japan-kansai-4d3n-plan.svg')).toBeVisible();
await expect(page.getByRole('button').filter({ hasText: 'japan-kansai-4d3n-plan.svg' })).toBeEnabled();
await expect(page.getByTestId('acp-image-part').locator('img')).toBeVisible();
await expect(page.getByTestId('chat-execution-graph')).toHaveCount(0);
} finally {
tests/e2e/chat-scroll-pin-bottom.spec.ts
+4 -4
View File
@@ -94,19 +94,19 @@ test.describe('ClawX chat scroll pin-to-bottom during runs', () => {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-scroll-to-latest.spec.ts
+4 -4
View File
@@ -66,19 +66,19 @@ test.describe('ClawX chat scroll-to-latest affordance', () => {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-table-header-light.spec.ts
+4 -4
View File
@@ -99,19 +99,19 @@ test.describe('ClawX chat table header styling', () => {
},
},
hostApi: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: MAIN_WORKSPACE, cwd: MAIN_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/chat-task-visualizer.spec.ts
+5 -5
View File
@@ -18,11 +18,11 @@ function stableStringify(value: unknown): string {
function baseHostApiMocks(loadResult: Record<string, unknown> = { success: true, generation: 1 }) {
return {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, cwd: PROJECT_MANAGER_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, cwd: PROJECT_MANAGER_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, cwd: '/' }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, workspaceRoot: PROJECT_MANAGER_WORKSPACE, cwd: PROJECT_MANAGER_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, workspaceRoot: PROJECT_MANAGER_WORKSPACE, cwd: PROJECT_MANAGER_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, workspaceRoot: '/', cwd: '/' }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: loadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: PROJECT_MANAGER_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: loadResult,
[stableStringify(['/api/agents', 'GET'])]: {
ok: true,
data: {
tests/e2e/chat-workspace-context.spec.ts
+2 -2
View File
@@ -159,8 +159,8 @@ async function installWorkspaceMocks(app: ElectronApplication, options: Workspac
}),
[stableStringify(['sessions', 'summaries', { sessionKeys: [SESSION_KEY] }])]: sessionSummaries,
[stableStringify(['/api/sessions/summaries', 'POST'])]: hostJson(sessionSummaries),
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: acpLoadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, cwd: SESSION_WORKSPACE }])]: acpLoadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: acpLoadResult,
[stableStringify(['chat', 'loadAcpSession', { sessionKey: SESSION_KEY, workspaceRoot: SESSION_WORKSPACE, cwd: SESSION_WORKSPACE }])]: acpLoadResult,
},
});
tests/e2e/cron-run-live-status.spec.ts
+2 -2
View File
@@ -25,11 +25,11 @@ const cronTriggerUpdate: AcpSessionUpdate = {
function acpLoadMocks(sessionKey: string) {
return {
[stableStringify(['chat', 'loadAcpSession', { sessionKey, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE, createIfMissing: true }])]: {
success: true,
generation: 1,
},
tests/e2e/fixtures/electron.ts
+522 -2
View File
@@ -1,9 +1,12 @@
import electronBinaryPath from 'electron';
import { _electron as electron, expect, test as base, type ElectronApplication, type Page } from '@playwright/test';
import { access, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
import { build as buildWithEsbuild } from 'esbuild';
import { access, mkdir, mkdtemp, realpath, rm, writeFile } from 'node:fs/promises';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { dirname, join, relative, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
import type { RawMessage } from '../../../shared/chat/types';
type LaunchElectronOptions = {
skipSetup?: boolean;
@@ -29,6 +32,51 @@ export type RecordedLegacyIpcInvocation = {
args: unknown[];
};
export type AttachmentFixtureSession = {
key: string;
title: string;
};
export type AttachmentFixtureTranscriptResponse = RawMessage[] | {
messages: RawMessage[];
deferId: string;
};
export type AttachmentHostFixture = {
workspaceDir: string;
openClawMediaDir: string;
outsideDir: string;
createWorkspaceFile: (relativePath: string, data: string | Uint8Array) => Promise<string>;
createOpenClawMediaFile: (relativePath: string, data: string | Uint8Array) => Promise<string>;
createOutsideFile: (relativePath: string, data: string | Uint8Array) => Promise<string>;
registerStagedAttachment: (id: string, stagedPath: string, displayPath?: string) => Promise<void>;
emitAcpSessionUpdates: (input: {
sessionKey: string;
updates: Array<Record<string, unknown> & { sessionUpdate: string }>;
generation?: number;
historical?: boolean;
}) => Promise<void>;
setPromptUpdates: (
prompt: string,
updates: Array<Record<string, unknown> & { sessionUpdate: string }>,
) => Promise<void>;
setSessionReplay: (
sessionKey: string,
updates: Array<Record<string, unknown> & { sessionUpdate: string }>,
) => Promise<void>;
setTranscriptResponses: (
sessionKey: string,
responses: AttachmentFixtureTranscriptResponse[],
) => Promise<void>;
releaseTranscriptResponse: (deferId: string) => Promise<void>;
waitForDeferredTranscriptReady: (deferId: string, timeoutMs?: number) => Promise<void>;
waitForDeferredTranscriptCompleted: (deferId: string, timeoutMs?: number) => Promise<void>;
waitForHistoryRequestCount: (sessionKey: string, count: number, timeoutMs?: number) => Promise<number[]>;
getHostInvocations: () => Promise<RecordedHostInvocation[]>;
getShellInvocations: () => Promise<RecordedHostInvocation[]>;
clearInvocations: () => Promise<void>;
};
type ElectronFixtures = {
electronApp: ElectronApplication;
page: Page;
@@ -39,6 +87,40 @@ type ElectronFixtures = {
const repoRoot = resolve(process.cwd());
const electronEntry = join(repoRoot, 'dist-electron/main/index.js');
let productionAttachmentBundlePromise: Promise<string> | undefined;
function productionAttachmentBundle(): Promise<string> {
// The app's production registry is closure-owned and cannot accept a provider-free
// test grant. Bundle the real access service into Electron Main instead of
// reimplementing its authorization or shell delegation in this fixture.
productionAttachmentBundlePromise ??= buildWithEsbuild({
stdin: {
contents: [
`export { createAttachmentAccess, StagedAttachmentRegistry } from ${JSON.stringify(join(repoRoot, 'electron/services/attachment-access.ts'))};`,
`export { AcpSessionAccessRegistry } from ${JSON.stringify(join(repoRoot, 'electron/services/acp-session-access-registry.ts'))};`,
`export { createMediaApi } from ${JSON.stringify(join(repoRoot, 'electron/services/media-api.ts'))};`,
].join('\n'),
loader: 'ts',
resolveDir: repoRoot,
sourcefile: 'attachment-e2e-production-entry.ts',
},
bundle: true,
define: {
'import.meta.url': JSON.stringify(pathToFileURL(join(repoRoot, 'electron/utils/paths.ts')).href),
},
external: ['electron'],
format: 'cjs',
platform: 'node',
target: 'node22',
tsconfig: join(repoRoot, 'tsconfig.node.json'),
write: false,
}).then((result) => {
const output = result.outputFiles?.[0];
if (!output) throw new Error('Failed to bundle production attachment services for Electron E2E');
return output.text;
});
return productionAttachmentBundlePromise;
}
async function allocatePort(): Promise<number> {
return await new Promise((resolvePort, reject) => {
@@ -483,6 +565,444 @@ export async function installIpcMocks(
);
}
function stableStringify(value: unknown): string {
if (value == null || typeof value !== 'object') return JSON.stringify(value);
if (Array.isArray(value)) return `[${value.map((item) => stableStringify(item)).join(',')}]`;
const entries = Object.entries(value as Record<string, unknown>)
.sort(([left], [right]) => left.localeCompare(right))
.map(([key, entryValue]) => `${JSON.stringify(key)}:${stableStringify(entryValue)}`);
return `{${entries.join(',')}}`;
}
export async function installAttachmentHostFixture(
app: ElectronApplication,
options: { sessions: AttachmentFixtureSession[] },
): Promise<AttachmentHostFixture> {
if (options.sessions.length === 0) throw new Error('Attachment fixture requires at least one session');
const homeDir = await app.evaluate(async () => process.env.HOME || process.env.USERPROFILE || '');
if (!homeDir) throw new Error('Attachment fixture could not resolve the isolated home directory');
const fixtureRoot = join(homeDir, 'attachment-e2e');
const workspacePath = join(fixtureRoot, 'workspace');
const outsidePath = join(fixtureRoot, 'outside');
const openClawMediaPath = join(homeDir, '.openclaw', 'media');
await Promise.all([
mkdir(workspacePath, { recursive: true }),
mkdir(outsidePath, { recursive: true }),
mkdir(openClawMediaPath, { recursive: true }),
]);
const [workspaceDir, outsideDir, openClawMediaDir] = await Promise.all([
realpath(workspacePath),
realpath(outsidePath),
realpath(openClawMediaPath),
]);
const productionAttachmentBundlePath = join(fixtureRoot, 'production-attachment-access.cjs');
await writeFile(productionAttachmentBundlePath, await productionAttachmentBundle(), 'utf-8');
const now = Date.now();
const sessionRecords = options.sessions.map((session, index) => ({
key: session.key,
displayName: session.title,
derivedTitle: session.title,
workspacePath: workspaceDir,
updatedAt: new Date(now - index).toISOString(),
}));
const sessionsList = { success: true, result: { sessions: sessionRecords } };
const sessionKeys = options.sessions.map((session) => session.key);
await installIpcMocks(app, {
gatewayStatus: { state: 'running', gatewayReady: true, port: 18789, pid: 12345, connectedAt: now },
gatewayRpc: {
[stableStringify(['sessions.list', {}])]: sessionsList,
[stableStringify(['sessions.list', { includeDerivedTitles: true, includeLastMessage: true }])]: sessionsList,
[stableStringify(['chat.history', null])]: { success: true, result: { messages: [] } },
},
hostApi: {
[stableStringify(['settings', 'getAll', null])]: {
language: 'en',
setupComplete: true,
chatWorkspacePath: workspaceDir,
recentWorkspacePaths: [workspaceDir],
},
[stableStringify(['agents', 'list', null])]: {
success: true,
agents: [{
id: 'main',
name: 'main',
workspace: workspaceDir,
mainSessionKey: options.sessions[0]!.key,
}],
},
[stableStringify(['sessions', 'summaries', { sessionKeys }])]: {
success: true,
summaries: options.sessions.map((session, index) => ({
sessionKey: session.key,
firstUserText: session.title,
lastTimestamp: now - index,
workspacePath: workspaceDir,
})),
},
},
recordLegacyIpcInvocations: true,
});
await app.evaluate(async ({ app: _app }, payload) => {
const { BrowserWindow, ipcMain, shell } = process.mainModule!.require('electron') as typeof import('electron');
type AcpUpdate = Record<string, unknown> & { sessionUpdate: string };
type TranscriptResponse = { messages: RawMessage[]; deferId?: string };
type HostRequest = {
id?: string;
module?: string;
action?: string;
payload?: Record<string, unknown>;
};
type HostHandler = (event: unknown, request: HostRequest) => Promise<unknown>;
type FixtureState = {
activeSessionKey: string;
generation: number;
replays: Record<string, AcpUpdate[]>;
promptUpdates: Record<string, AcpUpdate[]>;
transcriptResponses: Record<string, TranscriptResponse[]>;
transcriptIndexes: Record<string, number>;
historyRequestTimes: Record<string, number[]>;
replayReady: Record<string, Promise<void> | undefined>;
deferredTranscriptResolvers: Record<string, (() => void) | undefined>;
deferredTranscriptReady: Record<string, boolean>;
deferredTranscriptReturned: Record<string, boolean>;
deferredTranscriptCompleted: Record<string, boolean>;
hostInvocations: RecordedHostInvocation[];
shellInvocations: RecordedHostInvocation[];
stagedAttachments?: { register: (id: string, canonicalPath: string, displayPath?: string) => void };
};
const globals = globalThis as unknown as { __e2eAttachmentFixture?: FixtureState };
const state: FixtureState = {
activeSessionKey: '',
generation: 0,
replays: {},
promptUpdates: {},
transcriptResponses: {},
transcriptIndexes: {},
historyRequestTimes: {},
replayReady: {},
deferredTranscriptResolvers: {},
deferredTranscriptReady: {},
deferredTranscriptReturned: {},
deferredTranscriptCompleted: {},
hostInvocations: [],
shellInvocations: [],
};
globals.__e2eAttachmentFixture = state;
const instrumentedShell = shell as unknown as {
openPath: (path: string) => Promise<string>;
openExternal: (url: string) => Promise<void>;
};
instrumentedShell.openPath = async (path) => {
state.shellInvocations.push({ module: 'shell', action: 'openPath', payload: { path } });
return '';
};
instrumentedShell.openExternal = async (url) => {
state.shellInvocations.push({ module: 'shell', action: 'openExternal', payload: { url } });
};
type ProductionAttachmentModule = {
AcpSessionAccessRegistry: new () => {
prepareGrant: (input: {
sessionKey: string;
generation: number;
workspaceRoot: string;
executionCwd: string;
}) => Promise<unknown>;
commitGrant: (grant: unknown) => void;
};
StagedAttachmentRegistry: new () => {
register: (id: string, canonicalPath: string, displayPath?: string) => void;
};
createMediaApi: (dependencies: { attachmentAccess: unknown }) => {
thumbnails: (input: unknown) => Promise<unknown>;
};
createAttachmentAccess: (dependencies: {
sessionAccessRegistry: unknown;
stagedAttachments: unknown;
}) => {
resolveAttachment: (input: unknown) => Promise<unknown>;
readAttachmentText: (input: unknown) => Promise<unknown>;
readAttachmentBinary: (input: unknown) => Promise<unknown>;
openAttachment: (input: unknown) => Promise<unknown>;
};
};
const production = process.mainModule!.require(payload.productionAttachmentBundlePath) as ProductionAttachmentModule;
const productionSessionAccess = new production.AcpSessionAccessRegistry();
const productionStagedAttachments = new production.StagedAttachmentRegistry();
state.stagedAttachments = productionStagedAttachments;
const productionAttachmentAccess = production.createAttachmentAccess({
sessionAccessRegistry: productionSessionAccess,
stagedAttachments: productionStagedAttachments,
});
const productionMediaApi = production.createMediaApi({ attachmentAccess: productionAttachmentAccess });
const respond = (id: unknown, data: unknown) => ({
id: typeof id === 'string' ? id : undefined,
ok: true,
data,
});
const emitUpdates = (sessionKey: string, generation: number, historical: boolean, updates: AcpUpdate[]) => {
for (const update of updates) {
for (const window of BrowserWindow.getAllWindows()) {
window.webContents.send('chat:acp-session-update', {
sessionKey,
generation,
...(historical ? { historical: true } : {}),
notification: { sessionId: sessionKey, update },
});
}
}
};
const handlers = (ipcMain as unknown as { _invokeHandlers?: Map<string, HostHandler> })._invokeHandlers;
const originalHostInvoke = handlers?.get('host:invoke');
ipcMain.removeHandler('host:invoke');
ipcMain.handle('host:invoke', async (event: unknown, request: HostRequest) => {
state.hostInvocations.push({ module: request.module, action: request.action, payload: request.payload });
if (request.module === 'chat' && request.action === 'loadAcpSession') {
const sessionKey = String(request.payload?.sessionKey ?? '');
state.generation += 1;
state.activeSessionKey = sessionKey;
const generation = state.generation;
const grant = await productionSessionAccess.prepareGrant({
sessionKey,
generation,
workspaceRoot: payload.workspaceDir,
executionCwd: payload.workspaceDir,
});
productionSessionAccess.commitGrant(grant);
const replay = state.replays[sessionKey] ?? [];
state.replayReady[sessionKey] = new Promise((resolveReplay) => {
setTimeout(() => {
emitUpdates(sessionKey, generation, true, replay);
resolveReplay();
}, 0);
});
return respond(request.id, { success: true, generation });
}
if (request.module === 'chat' && request.action === 'sendAcpPrompt') {
const sessionKey = String(request.payload?.sessionKey ?? '');
const prompt = String(request.payload?.message ?? '');
if (sessionKey === state.activeSessionKey) {
emitUpdates(sessionKey, state.generation, false, state.promptUpdates[prompt] ?? []);
}
return respond(request.id, { success: true, generation: state.generation });
}
if (request.module === 'sessions' && request.action === 'history') {
const sessionKey = String(request.payload?.sessionKey ?? '');
const times = state.historyRequestTimes[sessionKey] ?? [];
times.push(Date.now());
state.historyRequestTimes[sessionKey] = times;
await state.replayReady[sessionKey];
const responses = state.transcriptResponses[sessionKey] ?? [{ messages: [] }];
const index = state.transcriptIndexes[sessionKey] ?? 0;
state.transcriptIndexes[sessionKey] = index + 1;
const response = responses[Math.min(index, responses.length - 1)] ?? { messages: [] };
if (response.deferId) {
state.deferredTranscriptReady[response.deferId] = true;
await new Promise<void>((resolveResponse) => {
state.deferredTranscriptResolvers[response.deferId!] = resolveResponse;
});
delete state.deferredTranscriptResolvers[response.deferId];
state.deferredTranscriptReturned[response.deferId] = true;
}
return respond(request.id, { success: true, messages: response.messages });
}
if (request.module === 'files' && request.action === 'resolveWorkspaceContext') {
return respond(request.id, {
ok: true,
workspaceRoot: payload.workspaceDir,
executionCwd: payload.workspaceDir,
});
}
if (request.module === 'files' && request.action === 'resolveAttachment') {
return respond(request.id, await productionAttachmentAccess.resolveAttachment(request.payload));
}
if (request.module === 'files' && request.action === 'readAttachmentText') {
return respond(request.id, await productionAttachmentAccess.readAttachmentText(request.payload));
}
if (request.module === 'files' && request.action === 'readAttachmentBinary') {
return respond(request.id, await productionAttachmentAccess.readAttachmentBinary(request.payload));
}
if (request.module === 'files' && request.action === 'openAttachment') {
return respond(request.id, await productionAttachmentAccess.openAttachment(request.payload));
}
if (request.module === 'media' && request.action === 'thumbnails') {
return respond(request.id, await productionMediaApi.thumbnails(request.payload));
}
if (request.module === 'diagnostics' && request.action === 'recordAcpTrace') {
if (request.payload?.event === 'openclaw-media:projection-stale') {
for (const deferId of Object.keys(state.deferredTranscriptReturned)) {
state.deferredTranscriptCompleted[deferId] = true;
}
}
return respond(request.id, { success: true });
}
return originalHostInvoke?.(event, request) ?? respond(request.id, {});
});
}, { workspaceDir, productionAttachmentBundlePath });
const writeFixtureFile = async (root: string, relativePath: string, data: string | Uint8Array) => {
const filePath = resolve(root, relativePath);
const fromRoot = relative(root, filePath);
if (!fromRoot || fromRoot === '..' || fromRoot.startsWith(`..${process.platform === 'win32' ? '\\' : '/'}`)) {
throw new Error(`Attachment fixture path escapes its root: ${relativePath}`);
}
await mkdir(dirname(filePath), { recursive: true });
await writeFile(filePath, data);
return filePath;
};
const readState = async () => await app.evaluate(async () => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: {
historyRequestTimes: Record<string, number[]>;
deferredTranscriptReady: Record<string, boolean>;
deferredTranscriptCompleted: Record<string, boolean>;
hostInvocations: RecordedHostInvocation[];
shellInvocations: RecordedHostInvocation[];
};
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
return {
historyRequestTimes: state.historyRequestTimes,
deferredTranscriptReady: state.deferredTranscriptReady,
deferredTranscriptCompleted: state.deferredTranscriptCompleted,
hostInvocations: state.hostInvocations,
shellInvocations: state.shellInvocations,
};
});
return {
workspaceDir,
openClawMediaDir,
outsideDir,
createWorkspaceFile: async (path, data) => await writeFixtureFile(workspaceDir, path, data),
createOpenClawMediaFile: async (path, data) => await writeFixtureFile(openClawMediaDir, path, data),
createOutsideFile: async (path, data) => await writeFixtureFile(outsideDir, path, data),
registerStagedAttachment: async (id, stagedPath, displayPath) => {
await app.evaluate(async ({ app: _app }, input) => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: {
stagedAttachments?: { register: (id: string, canonicalPath: string, displayPath?: string) => void };
};
}).__e2eAttachmentFixture;
if (!state?.stagedAttachments) throw new Error('Attachment staging fixture is not installed');
state.stagedAttachments.register(input.id, input.stagedPath, input.displayPath);
}, { id, stagedPath, displayPath });
},
emitAcpSessionUpdates: async (input) => {
await app.evaluate(async ({ app: _app }, event) => {
const { BrowserWindow } = process.mainModule!.require('electron') as typeof import('electron');
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: { activeSessionKey: string; generation: number };
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
const generation = event.generation ?? state.generation;
for (const update of event.updates) {
for (const window of BrowserWindow.getAllWindows()) {
window.webContents.send('chat:acp-session-update', {
sessionKey: event.sessionKey,
generation,
...(event.historical ? { historical: true } : {}),
notification: { sessionId: event.sessionKey, update },
});
}
}
}, input);
},
setPromptUpdates: async (prompt, updates) => {
await app.evaluate(async ({ app: _app }, input) => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: { promptUpdates: Record<string, unknown[]> };
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
state.promptUpdates[input.prompt] = input.updates;
}, { prompt, updates });
},
setSessionReplay: async (sessionKey, updates) => {
await app.evaluate(async ({ app: _app }, input) => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: { replays: Record<string, unknown[]> };
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
state.replays[input.sessionKey] = input.updates;
}, { sessionKey, updates });
},
setTranscriptResponses: async (sessionKey, responses) => {
const normalized = responses.map((response) => Array.isArray(response)
? { messages: response }
: response);
await app.evaluate(async ({ app: _app }, input) => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: {
transcriptResponses: Record<string, unknown[]>;
transcriptIndexes: Record<string, number>;
};
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
state.transcriptResponses[input.sessionKey] = input.responses;
state.transcriptIndexes[input.sessionKey] = 0;
}, { sessionKey, responses: normalized });
},
releaseTranscriptResponse: async (deferId) => {
await app.evaluate(async ({ app: _app }, id) => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: { deferredTranscriptResolvers: Record<string, (() => void) | undefined> };
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
const release = state.deferredTranscriptResolvers[id];
if (!release) throw new Error(`Deferred transcript response is not ready: ${id}`);
delete state.deferredTranscriptResolvers[id];
release();
}, deferId);
},
waitForDeferredTranscriptReady: async (deferId, timeoutMs = 5_000) => {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if ((await readState()).deferredTranscriptReady[deferId]) return;
await new Promise((resolveWait) => setTimeout(resolveWait, 25));
}
throw new Error(`Timed out waiting for deferred transcript response to become ready: ${deferId}`);
},
waitForDeferredTranscriptCompleted: async (deferId, timeoutMs = 5_000) => {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if ((await readState()).deferredTranscriptCompleted[deferId]) return;
await new Promise((resolveWait) => setTimeout(resolveWait, 25));
}
throw new Error(`Timed out waiting for deferred transcript response to complete: ${deferId}`);
},
waitForHistoryRequestCount: async (sessionKey, count, timeoutMs = 5_000) => {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const times = (await readState()).historyRequestTimes[sessionKey] ?? [];
if (times.length >= count) return times;
await new Promise((resolveWait) => setTimeout(resolveWait, 25));
}
throw new Error(`Timed out waiting for ${count} transcript requests for ${sessionKey}`);
},
getHostInvocations: async () => (await readState()).hostInvocations,
getShellInvocations: async () => (await readState()).shellInvocations,
clearInvocations: async () => {
await app.evaluate(async () => {
const state = (globalThis as unknown as {
__e2eAttachmentFixture?: {
hostInvocations: RecordedHostInvocation[];
shellInvocations: RecordedHostInvocation[];
};
}).__e2eAttachmentFixture;
if (!state) throw new Error('Attachment fixture is not installed');
state.hostInvocations = [];
state.shellInvocations = [];
});
},
};
}
export async function getRecordedHostInvocations(app: ElectronApplication): Promise<RecordedHostInvocation[]> {
return await app.evaluate(async ({ app: _app }) => (
(globalThis as unknown as { __e2eHostInvocations?: RecordedHostInvocation[] }).__e2eHostInvocations ?? []
tests/e2e/gateway-lifecycle.spec.ts
+2 -2
View File
@@ -234,7 +234,7 @@ test.describe('ClawX gateway lifecycle resilience', () => {
json: { success: true, agents: [{ id: 'main', name: 'main', workspace: DEFAULT_WORKSPACE, mainSessionKey: MAIN_SESSION_KEY }] },
},
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
@@ -270,7 +270,7 @@ test.describe('ClawX gateway lifecycle resilience', () => {
json: { success: true, agents: [{ id: 'main', name: 'main', workspace: DEFAULT_WORKSPACE, mainSessionKey: MAIN_SESSION_KEY }] },
},
},
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, cwd: DEFAULT_WORKSPACE }])]: {
[stableStringify(['chat', 'loadAcpSession', { sessionKey: MAIN_SESSION_KEY, workspaceRoot: DEFAULT_WORKSPACE, cwd: DEFAULT_WORKSPACE }])]: {
success: true,
generation: 1,
},
tests/unit/acp-chat-components.test.tsx
+478 -4
View File
@@ -1,14 +1,28 @@
import { act, fireEvent, render, screen } from '@testing-library/react';
import { describe, expect, it, vi } from 'vitest';
import { act, fireEvent, render, screen, waitFor } from '@testing-library/react';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { AcpToolCallCard } from '@/pages/Chat/AcpToolCallCard';
import { AcpTimeline } from '@/pages/Chat/AcpTimeline';
import type { AcpTimelineSnapshot, ToolCallItem } from '@/lib/acp/timeline-types';
import type { AcpFileActivityProjection } from '@/lib/acp/openclaw-file-activities';
import { useArtifactPanel } from '@/stores/artifact-panel';
const openAttachmentMock = vi.hoisted(() => vi.fn());
const thumbnailsMock = vi.hoisted(() => vi.fn());
vi.mock('@/lib/host-api', () => ({
hostApi: {
files: {
openAttachment: openAttachmentMock,
},
media: {
thumbnails: thumbnailsMock,
},
},
}));
vi.mock('react-i18next', () => ({
useTranslation: () => ({
t: (key: string) => {
t: (key: string, options?: Record<string, unknown>) => {
const labels: Record<string, string> = {
'acp.thought': 'Thought',
'acp.tool': 'Tool',
@@ -25,13 +39,19 @@ vi.mock('react-i18next', () => ({
'acp.promptFailed': 'Prompt failed',
'acp.unsupportedContent': 'Unsupported content',
'acp.dismiss': 'Dismiss',
'acp.attachment.loading': 'Loading attachment',
'acp.attachment.unavailable': 'Attachment unavailable',
'acp.attachment.open': 'Open {{name}}',
'acp.attachment.preview': 'Preview {{name}}',
'acp.attachment.mimeSize': '{{size}}',
'acp.attachment.openFailed': 'Could not open attachment',
'fileActivity.created': 'Created',
'fileActivity.modified': 'Modified',
'fileActivity.deleted': 'Deleted',
'fileActivity.fileButton': '{{action}} {{path}}',
'fileActivity.changeRecord': 'View changes for {{path}}',
};
return labels[key] ?? key;
return (labels[key] ?? key).replace(/{{(\w+)}}/g, (_match, name: string) => String(options?.[name] ?? ''));
},
}),
}));
@@ -63,6 +83,13 @@ function toolCallItem(overrides: Partial<ToolCallItem>): ToolCallItem {
}
describe('ACP chat timeline components', () => {
beforeEach(() => {
vi.clearAllMocks();
openAttachmentMock.mockResolvedValue({ ok: true });
thumbnailsMock.mockResolvedValue({});
useArtifactPanel.setState({ open: false, tab: 'changes', focusedFile: null });
});
it('renders tool-only turn file controls once after timeline items and routes preview and changes', () => {
const state = snapshot({
itemOrder: ['tool:write-file'],
@@ -276,6 +303,45 @@ describe('ACP chat timeline components', () => {
}
});
it('auto-collapses failed tool results on the same delay as completed ones', () => {
vi.useFakeTimers();
try {
render(<AcpToolCallCard item={toolCallItem({ status: 'failed', error: 'Boom.' })} />);
const card = screen.getByTestId('acp-tool-call-card');
expect(card).toHaveAttribute('data-expanded', 'true');
act(() => {
vi.advanceTimersByTime(999);
});
expect(card).toHaveAttribute('data-expanded', 'true');
act(() => {
vi.advanceTimersByTime(1);
});
expect(card).toHaveAttribute('data-expanded', 'false');
} finally {
vi.useRealTimers();
}
});
it('mounts historical failed tool results collapsed without waiting for auto-collapse', () => {
vi.useFakeTimers();
try {
render(<AcpToolCallCard item={toolCallItem({ status: 'failed', error: 'Boom.', historical: true })} />);
const card = screen.getByTestId('acp-tool-call-card');
expect(card).toHaveAttribute('data-expanded', 'false');
act(() => {
vi.advanceTimersByTime(1_000);
});
expect(card).toHaveAttribute('data-expanded', 'false');
} finally {
vi.useRealTimers();
}
});
it('starts a fresh delayed auto-collapse when a completed tool call id changes', () => {
vi.useFakeTimers();
try {
@@ -388,6 +454,414 @@ describe('ACP chat timeline components', () => {
expect(screen.getByAltText('Chart preview')).toHaveAttribute('src', 'data:image/png;base64,abc');
});
it('renders pending and unavailable attachments as disabled paperclip rows', () => {
const state = snapshot({
itemOrder: ['msg-a:0', 'msg-a:1'],
itemsById: {
'msg-a:0': {
kind: 'message-segment',
id: 'msg-a:0',
role: 'assistant',
messageId: 'msg-a',
segmentIndex: 0,
parts: [{
kind: 'attachment',
attachmentId: 'attachment:msg-a:0:0',
reference: { uri: 'file:///repo/report.txt', name: '/repo/report.txt', mimeType: 'text/plain' },
source: 'acp-resource',
access: { status: 'pending' },
}],
},
'msg-a:1': {
kind: 'message-segment',
id: 'msg-a:1',
role: 'assistant',
messageId: 'msg-a',
segmentIndex: 1,
parts: [{
kind: 'attachment',
attachmentId: 'attachment:msg-a:1:0',
reference: { uri: 'file:///secret/missing.zip', name: 'missing.zip' },
source: 'acp-resource',
access: { status: 'unavailable', reason: 'outsideAllowedRoots' },
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
expect(screen.getByText('report.txt')).toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Loading attachment: report.txt' })).toBeDisabled();
expect(screen.getByRole('button', { name: 'Attachment unavailable: missing.zip' })).toBeDisabled();
expect(screen.getAllByTestId('acp-attachment-icon')).toHaveLength(2);
expect(screen.queryByText('file:///secret/missing.zip')).not.toBeInTheDocument();
});
it('previews a supported attachment with safe metadata and native button semantics', async () => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///secret/budget.xlsx' };
const state = snapshot({
itemOrder: ['msg-a:0'],
itemsById: {
'msg-a:0': {
kind: 'message-segment',
id: 'msg-a:0',
role: 'assistant',
messageId: 'msg-a',
segmentIndex: 0,
parts: [{
kind: 'attachment',
attachmentId: 'attachment:msg-a:0:0',
reference: { uri: ref.uri, name: 'budget.xlsx' },
source: 'acp-resource',
access: {
status: 'available',
identity: 'opaque-budget',
target: { kind: 'local', scope: 'workspace', ref },
mimeType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
size: 2048,
},
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
const button = screen.getByRole('button', { name: 'Preview budget.xlsx' });
expect(button.tagName).toBe('BUTTON');
expect(button).toHaveClass('focus-visible:ring-2');
expect(screen.getByText('2.0 KB')).toBeInTheDocument();
expect(button).not.toHaveAttribute('title', expect.stringContaining('/secret/'));
button.focus();
fireEvent.click(button);
expect(useArtifactPanel.getState().focusedFile).toMatchObject({
filePath: 'budget.xlsx',
fileName: 'budget.xlsx',
attachmentFileRef: ref,
});
expect(openAttachmentMock).not.toHaveBeenCalled();
});
it.each([
['archive.zip', 'application/zip', 1024],
['report.docx', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 1024],
['clip.mp3', 'audio/mpeg', 1024],
['movie.mp4', 'video/mp4', 1024],
['large.pdf', 'application/pdf', 50 * 1024 * 1024 + 1],
])('opens unsupported or oversized local attachment %s through files.openAttachment', async (name, mimeType, size) => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: `file:///workspace/${name}` };
const state = snapshot({
itemOrder: ['msg-a:0'],
itemsById: {
'msg-a:0': {
kind: 'message-segment', id: 'msg-a:0', role: 'assistant', messageId: 'msg-a', segmentIndex: 0,
parts: [{
kind: 'attachment', attachmentId: `attachment:${name}`, reference: { uri: ref.uri, name }, source: 'acp-resource',
access: { status: 'available', identity: `opaque-${name}`, target: { kind: 'local', scope: 'workspace', ref }, mimeType, size },
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
fireEvent.click(screen.getByRole('button', { name: `Open ${name}` }));
await waitFor(() => expect(openAttachmentMock).toHaveBeenCalledWith(ref));
expect(useArtifactPanel.getState().focusedFile).toBeNull();
});
it.each(['http://example.com/report.pdf', 'https://example.com/report.pdf'])('routes %s through files.openAttachment', async (url) => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: url };
const state = snapshot({
itemOrder: ['msg-a:0'],
itemsById: {
'msg-a:0': {
kind: 'message-segment', id: 'msg-a:0', role: 'assistant', messageId: 'msg-a', segmentIndex: 0,
parts: [{
kind: 'attachment', attachmentId: 'attachment:remote', reference: { uri: ref.uri, name: 'report.pdf' }, source: 'acp-resource',
access: { status: 'available', identity: 'opaque-remote', target: { kind: 'remote', ref, url: ref.uri }, mimeType: 'application/pdf', size: 1024 },
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
fireEvent.click(screen.getByRole('button', { name: 'Open report.pdf' }));
await waitFor(() => expect(openAttachmentMock).toHaveBeenCalledWith(ref));
});
it('lifts early assistant attachments after later process and prose items and before file activity', () => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///workspace/report.txt' };
const state = snapshot({
itemOrder: ['msg-a:0', 'tool:read-file', 'msg-a:1'],
itemsById: {
'msg-a:0': {
kind: 'message-segment', id: 'msg-a:0', role: 'assistant', messageId: 'msg-a', segmentIndex: 0,
parts: [
{ kind: 'markdown', text: 'First prose.' },
{ kind: 'attachment', attachmentId: 'attachment:early', reference: { uri: ref.uri, name: 'report.txt' }, source: 'acp-resource', access: { status: 'available', identity: 'opaque-report', target: { kind: 'local', scope: 'workspace', ref }, mimeType: 'text/plain', size: 12 } },
],
},
'tool:read-file': toolCallItem({ id: 'tool:read-file', toolCallId: 'read-file' }),
'msg-a:1': {
kind: 'message-segment', id: 'msg-a:1', role: 'assistant', messageId: 'msg-a', segmentIndex: 1,
parts: [{ kind: 'markdown', text: 'Final prose.' }],
},
},
});
const turnId = 'assistant-turn:msg-a:0';
const activity: AcpFileActivityProjection = {
activities: [],
turnSummariesByTurnId: {
[turnId]: [{ turnId, relativePath: 'changed.txt', action: 'modified', activities: [], added: 1, removed: 0 }],
},
fileGroups: [],
uniqueFileCount: 1,
};
const { container } = render(<AcpTimeline snapshot={state} fileActivity={activity} workspaceRoot="/workspace" />);
const tool = screen.getByTestId('acp-tool-call-card');
const finalProse = screen.getByText('Final prose.');
const attachment = screen.getByRole('button', { name: 'Preview report.txt' });
const fileActivity = screen.getByTestId('acp-turn-file-activity');
const ordered = [tool, finalProse, attachment, fileActivity].map((node) => Array.from(container.querySelectorAll('*')).indexOf(node));
expect(ordered).toEqual([...ordered].sort((a, b) => a - b));
expect(screen.getAllByText('report.txt')).toHaveLength(1);
});
it('renders user attachments after all prose in the user message', () => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///workspace/notes.txt', stagingId: 'stage-1' };
const state = snapshot({
itemOrder: ['msg-u:0'],
itemsById: {
'msg-u:0': {
kind: 'message-segment', id: 'msg-u:0', role: 'user', messageId: 'msg-u', segmentIndex: 0,
parts: [
{ kind: 'attachment', attachmentId: 'attachment:user', reference: { uri: ref.uri, name: 'notes.txt' }, source: 'acp-resource', access: { status: 'available', identity: 'opaque-notes', target: { kind: 'local', scope: 'staging', ref }, mimeType: 'text/plain', size: 12 } },
{ kind: 'markdown', text: 'Please review this file.' },
],
},
},
});
render(<AcpTimeline snapshot={state} />);
const prose = screen.getByText('Please review this file.');
const attachment = screen.getByRole('button', { name: 'Preview notes.txt' });
expect(prose.compareDocumentPosition(attachment) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy();
});
it('renders a user image attachment as a thumbnail with a filename hover overlay', async () => {
const ref = {
sessionKey: 'agent:main:s1', generation: 1, uri: '/tmp/clawx-staging/photo.png', stagingId: 'stage-photo',
};
thumbnailsMock.mockResolvedValueOnce({
'opaque-photo': {
preview: 'data:image/png;base64,iVBORw0KGgo=',
fileSize: 4,
},
});
const state = snapshot({
itemOrder: ['msg-u:0'],
itemsById: {
'msg-u:0': {
kind: 'message-segment', id: 'msg-u:0', role: 'user', messageId: 'msg-u', segmentIndex: 0,
parts: [{
kind: 'attachment', attachmentId: 'attachment:user-photo',
reference: {
uri: ref.uri,
name: 'photo.png',
displayPath: '/Users/test/Pictures/photo.png',
mimeType: 'image/png',
stagingId: 'stage-photo',
},
source: 'acp-resource',
access: {
status: 'available', identity: 'opaque-photo',
target: { kind: 'local', scope: 'staging', ref }, mimeType: 'image/png', size: 4,
},
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
const thumbnail = await screen.findByTestId('acp-user-image-attachment');
expect(thumbnail).toHaveAttribute('alt', 'photo.png');
expect(thumbnail).toHaveAttribute('src', 'data:image/png;base64,iVBORw0KGgo=');
expect(screen.getByTestId('acp-user-image-overlay')).toHaveTextContent('photo.png');
expect(thumbnail.parentElement?.parentElement).toHaveClass('items-end');
expect(thumbnailsMock).toHaveBeenCalledWith({
paths: [{ attachmentFileRef: ref, key: 'opaque-photo', mimeType: 'image/png' }],
});
expect(screen.queryByTestId('acp-attachment-icon')).not.toBeInTheDocument();
});
it('shows a user file path after its name without MIME or size and keeps preview routing', () => {
const ref = {
sessionKey: 'agent:main:s1', generation: 1, uri: '/tmp/clawx-staging/notes.txt', stagingId: 'stage-notes',
};
const state = snapshot({
itemOrder: ['msg-u:0'],
itemsById: {
'msg-u:0': {
kind: 'message-segment', id: 'msg-u:0', role: 'user', messageId: 'msg-u', segmentIndex: 0,
parts: [{
kind: 'attachment', attachmentId: 'attachment:user-notes',
reference: {
uri: ref.uri,
name: 'notes.txt',
displayPath: '/Users/test/Documents/a/very/long/path/notes.txt',
mimeType: 'text/plain',
stagingId: 'stage-notes',
},
source: 'acp-resource',
access: {
status: 'available', identity: 'opaque-notes',
target: { kind: 'local', scope: 'staging', ref }, mimeType: 'text/plain', size: 2048,
},
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
const button = screen.getByRole('button', { name: 'Preview notes.txt' });
const path = screen.getByTestId('acp-user-attachment-path');
expect(path).toHaveTextContent('/Users/test/Documents/a/very/long/path/notes.txt');
expect(path).toHaveClass('truncate', 'text-muted-foreground');
expect(button).not.toHaveTextContent('2.0 KB');
expect(button).not.toHaveTextContent('text/plain');
fireEvent.click(button);
expect(useArtifactPanel.getState().focusedFile).toMatchObject({
fileName: 'notes.txt',
attachmentFileRef: ref,
});
});
it('opens an unsupported user file through the scoped system-open route', async () => {
const ref = {
sessionKey: 'agent:main:s1', generation: 1, uri: '/tmp/clawx-staging/archive.zip', stagingId: 'stage-zip',
};
const state = snapshot({
itemOrder: ['msg-u:0'],
itemsById: {
'msg-u:0': {
kind: 'message-segment', id: 'msg-u:0', role: 'user', messageId: 'msg-u', segmentIndex: 0,
parts: [{
kind: 'attachment', attachmentId: 'attachment:user-zip',
reference: {
uri: ref.uri, name: 'archive.zip', displayPath: '/Users/test/Downloads/archive.zip', stagingId: 'stage-zip',
},
source: 'acp-resource',
access: {
status: 'available', identity: 'opaque-zip',
target: { kind: 'local', scope: 'staging', ref }, mimeType: 'application/zip', size: 128,
},
}],
},
},
});
render(<AcpTimeline snapshot={state} />);
fireEvent.click(screen.getByRole('button', { name: 'Open archive.zip' }));
await waitFor(() => expect(openAttachmentMock).toHaveBeenCalledWith(ref));
expect(useArtifactPanel.getState().focusedFile).toBeNull();
});
it('renders thought and collapsed-tool attachments once after all assistant items', () => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///workspace/file.txt' };
const attachment = (attachmentId: string, name: string) => ({
kind: 'attachment' as const,
attachmentId,
reference: { uri: ref.uri, name },
source: 'acp-resource' as const,
access: {
status: 'available' as const,
identity: `opaque-${attachmentId}`,
target: { kind: 'local' as const, scope: 'workspace' as const, ref },
mimeType: 'text/plain',
size: 12,
},
});
const state = snapshot({
itemOrder: ['thought:msg-a', 'tool:read', 'msg-a:1'],
itemsById: {
'thought:msg-a': {
kind: 'thought', id: 'thought:msg-a', messageId: 'msg-a',
parts: [{ kind: 'markdown', text: 'Inspecting.' }, attachment('thought-file', 'thought.txt')],
},
'tool:read': {
kind: 'tool-call', id: 'tool:read', toolCallId: 'read', title: 'Read file', status: 'completed', historical: true,
outputParts: [{ kind: 'markdown', text: 'Hidden tool output.' }, attachment('tool-file', 'tool.txt')], locations: [],
},
'msg-a:1': {
kind: 'message-segment', id: 'msg-a:1', role: 'assistant', messageId: 'msg-a', segmentIndex: 1,
parts: [{ kind: 'markdown', text: 'Final answer.' }],
},
},
});
const { container } = render(<AcpTimeline snapshot={state} />);
expect(screen.getByTestId('acp-tool-call-card')).toHaveAttribute('data-expanded', 'false');
const ordered = [
screen.getByTestId('acp-thought-block'),
screen.getByTestId('acp-tool-call-card'),
screen.getByText('Final answer.'),
screen.getByRole('button', { name: 'Preview thought.txt' }),
screen.getByRole('button', { name: 'Preview tool.txt' }),
].map((node) => Array.from(container.querySelectorAll('*')).indexOf(node));
expect(ordered).toEqual([...ordered].sort((a, b) => a - b));
expect(screen.getAllByTestId('acp-attachment-icon')).toHaveLength(2);
});
it('renders all user prose segments before the group attachment list', () => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///workspace/file.txt', stagingId: 'stage-1' };
const attachment = (attachmentId: string, name: string) => ({
kind: 'attachment' as const,
attachmentId,
reference: { uri: ref.uri, name },
source: 'acp-resource' as const,
access: {
status: 'available' as const,
identity: `opaque-${attachmentId}`,
target: { kind: 'local' as const, scope: 'staging' as const, ref },
mimeType: 'text/plain',
size: 12,
},
});
const state = snapshot({
itemOrder: ['msg-u:0', 'msg-u:1'],
itemsById: {
'msg-u:0': {
kind: 'message-segment', id: 'msg-u:0', role: 'user', messageId: 'msg-u', segmentIndex: 0,
parts: [attachment('user-a', 'a.txt'), { kind: 'markdown', text: 'First user prose.' }],
},
'msg-u:1': {
kind: 'message-segment', id: 'msg-u:1', role: 'user', messageId: 'msg-u', segmentIndex: 1,
parts: [{ kind: 'markdown', text: 'Second user prose.' }, attachment('user-b', 'b.txt')],
},
},
});
const { container } = render(<AcpTimeline snapshot={state} />);
const ordered = [
screen.getByText('First user prose.'),
screen.getByText('Second user prose.'),
screen.getByRole('button', { name: 'Preview a.txt' }),
screen.getByRole('button', { name: 'Preview b.txt' }),
].map((node) => Array.from(container.querySelectorAll('*')).indexOf(node));
expect(ordered).toEqual([...ordered].sort((a, b) => a - b));
expect(screen.getAllByTestId('acp-attachment-icon')).toHaveLength(2);
});
it('dismisses the session error banner', () => {
const onDismissError = vi.fn();
render(<AcpTimeline snapshot={snapshot({})} error="Connection lost" onDismissError={onDismissError} />);
tests/unit/acp-chat-service.test.ts
+377 -42
View File
@@ -1,6 +1,6 @@
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { rmSync, writeFileSync } from 'node:fs';
import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { EventEmitter } from 'node:events';
import { PassThrough } from 'node:stream';
import { beforeEach, describe, expect, it, vi } from 'vitest';
@@ -58,11 +58,36 @@ function createConnection() {
};
}
async function createService(connection = createConnection()) {
function createPassthroughAccessRegistry() {
let activeGrant: {
sessionKey: string;
generation: number;
workspaceRoot: string;
executionCwd: string;
} | null = null;
return {
prepareGrant: vi.fn(async (input) => ({ ...input })),
snapshot: vi.fn(() => activeGrant ? { ...activeGrant } : null),
commitGrant: vi.fn((context) => { activeGrant = { ...context }; }),
restore: vi.fn((snapshot) => { activeGrant = snapshot ? { ...snapshot } : null; }),
get: vi.fn((sessionKey, generation) => (
activeGrant?.sessionKey === sessionKey && activeGrant.generation === generation
? { ...activeGrant }
: null
)),
};
}
async function createService(connection = createConnection(), accessRegistry = createPassthroughAccessRegistry()) {
const send = vi.fn();
const { AcpChatService } = await import('../../electron/services/acp-chat-service');
const service = new AcpChatService({ webContents: { send } } as never, connection as never);
return { service, connection, send };
const service = new AcpChatService(
{ webContents: { send } } as never,
accessRegistry as never,
connection as never,
undefined,
);
return { service, connection, send, accessRegistry };
}
function createFakeChild() {
@@ -83,7 +108,12 @@ async function createSpawnedService(connection = createConnection()) {
acpSdkMock.state.connectionForSpawn = connection;
childProcessMock.state.child = child;
const { AcpChatService } = await import('../../electron/services/acp-chat-service');
const service = new AcpChatService({ webContents: { send } } as never);
const service = new AcpChatService(
{ webContents: { send } } as never,
createPassthroughAccessRegistry() as never,
undefined,
undefined,
);
return { service, connection, send, child };
}
@@ -118,7 +148,7 @@ describe('AcpChatService', () => {
it('forks the embedded OpenClaw entry for ACP instead of spawning a public CLI wrapper', async () => {
const { service } = await createSpawnedService();
await expect(service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' })).resolves.toEqual({
await expect(service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' })).resolves.toEqual({
success: true,
generation: 1,
});
@@ -144,7 +174,7 @@ describe('AcpChatService', () => {
it('filters non-JSON stdout diagnostics before the ACP SDK parser sees them', async () => {
const { service, child } = await createSpawnedService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
const output = acpSdkMock.ndJsonStream.mock.calls[0]?.[1] as ReadableStream<Uint8Array>;
const reader = output.getReader();
@@ -162,7 +192,7 @@ describe('AcpChatService', () => {
it('loads historical sessions without explicit routing metadata so replay can resolve by session key', async () => {
const { service, connection } = await createService();
await expect(service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' })).resolves.toEqual({
await expect(service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' })).resolves.toEqual({
success: true,
generation: 1,
});
@@ -182,7 +212,7 @@ describe('AcpChatService', () => {
it('creates fresh generated sessions with ACP session/new so replay ledgers are complete', async () => {
const { service, connection } = await createService();
await expect(service.loadSession({ sessionKey: 'agent:pi:session-123', cwd: '/repo', createIfMissing: true })).resolves.toEqual({
await expect(service.loadSession({ sessionKey: 'agent:pi:session-123', workspaceRoot: '/repo', cwd: '/repo', createIfMissing: true })).resolves.toEqual({
success: true,
generation: 1,
});
@@ -198,7 +228,7 @@ describe('AcpChatService', () => {
it('routes fresh-session prompts through the ACP session id returned by session/new', async () => {
const { service, connection } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:session-123', cwd: '/repo', createIfMissing: true });
await service.loadSession({ sessionKey: 'agent:pi:session-123', workspaceRoot: '/repo', cwd: '/repo', createIfMissing: true });
await expect(service.sendPrompt({
sessionKey: 'agent:pi:session-123',
cwd: '/repo',
@@ -217,7 +247,7 @@ describe('AcpChatService', () => {
it('rewrites fresh-session ACP updates to the ClawX session key for the renderer', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:session-123', cwd: '/repo', createIfMissing: true });
await service.loadSession({ sessionKey: 'agent:pi:session-123', workspaceRoot: '/repo', cwd: '/repo', createIfMissing: true });
await service.client.sessionUpdate({
sessionId: 'acp-session-1',
update: {
@@ -246,7 +276,7 @@ describe('AcpChatService', () => {
it('emits raw ACP session updates with sessionKey and generation for the active session', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'live', messageId: 'msg-live' });
send.mockClear();
await service.client.sessionUpdate({
@@ -286,7 +316,7 @@ describe('AcpChatService', () => {
clearAcpTraceForTests();
const { service } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.client.sessionUpdate({
sessionId: 'agent:pi:s1',
update: {
@@ -326,7 +356,7 @@ describe('AcpChatService', () => {
clearAcpTraceForTests();
const { service } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.client.sessionUpdate({
sessionId: 'agent:other:s2',
update: {
@@ -348,7 +378,7 @@ describe('AcpChatService', () => {
it('marks ACP session updates from historical loads until the next live prompt starts', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.client.sessionUpdate({
sessionId: 'agent:pi:s1',
update: {
@@ -394,7 +424,8 @@ describe('AcpChatService', () => {
it('emits permission requests separately and resolves them from respondPermission', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'edit the file' });
send.mockClear();
const pending = service.client.requestPermission({
@@ -429,7 +460,7 @@ describe('AcpChatService', () => {
it('returns cancelled for permission requests from non-active sessions', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
send.mockClear();
await expectCancelledSoon(service.client.requestPermission({
@@ -444,7 +475,8 @@ describe('AcpChatService', () => {
it('cancels pending permission requests when switching sessions', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'edit the file' });
send.mockClear();
const pending = service.client.requestPermission({
sessionId: 'agent:pi:s1',
@@ -452,7 +484,7 @@ describe('AcpChatService', () => {
options: [{ optionId: 'allow-once', name: 'Allow once', kind: 'allow_once' }],
} as never);
await expect(service.loadSession({ sessionKey: 'agent:pi:s2', cwd: '/repo' })).resolves.toEqual({
await expect(service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' })).resolves.toEqual({
success: true,
generation: 2,
});
@@ -460,11 +492,62 @@ describe('AcpChatService', () => {
await expectCancelledSoon(pending);
});
it('cancels pending permission requests when reloading the same session', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'edit the file' });
send.mockClear();
const pending = service.client.requestPermission({
sessionId: 'agent:pi:s1',
toolCall: { toolCallId: 'tool-1', title: 'Edit file', status: 'pending' },
options: [{ optionId: 'allow-once', name: 'Allow once', kind: 'allow_once' }],
} as never);
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await expectCancelledSoon(pending);
});
it('cancels permission requests received while session/load is in progress', async () => {
const connection = createConnection();
const load = createDeferred<unknown>();
connection.loadSession.mockReturnValueOnce(load.promise);
const { service, send } = await createService(connection);
const loadPromise = service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await vi.waitFor(() => expect(connection.loadSession).toHaveBeenCalledTimes(1));
send.mockClear();
await expectCancelledSoon(service.client.requestPermission({
sessionId: 'agent:pi:s1',
toolCall: { toolCallId: 'tool-load', title: 'Unexpected load permission', status: 'pending' },
options: [{ optionId: 'allow-once', name: 'Allow once', kind: 'allow_once' }],
} as never));
expect(send).not.toHaveBeenCalledWith(HOST_EVENT_CHANNELS.chat.acpPermissionRequest, expect.anything());
load.resolve({});
await loadPromise;
});
it('cancels permission requests after load until the current session starts a prompt', async () => {
const { service, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
send.mockClear();
await expectCancelledSoon(service.client.requestPermission({
sessionId: 'agent:pi:s1',
toolCall: { toolCallId: 'tool-handoff', title: 'Late load permission', status: 'pending' },
options: [{ optionId: 'allow-once', name: 'Allow once', kind: 'allow_once' }],
} as never));
expect(send).not.toHaveBeenCalledWith(HOST_EVENT_CHANNELS.chat.acpPermissionRequest, expect.anything());
});
it('cancels pending permission requests and drops the connection when the ACP child exits', async () => {
const firstConnection = createConnection();
const { service, child } = await createSpawnedService(firstConnection);
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'edit the file' });
const pending = service.client.requestPermission({
sessionId: 'agent:pi:s1',
toolCall: { toolCallId: 'tool-1', title: 'Edit file', status: 'pending' },
@@ -477,7 +560,7 @@ describe('AcpChatService', () => {
child.emit('exit', 1);
await expectCancelledSoon(pending);
await service.loadSession({ sessionKey: 'agent:pi:s2', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' });
expect(childProcessMock.fork).toHaveBeenCalledTimes(2);
expect(secondConnection.initialize).toHaveBeenCalledTimes(1);
});
@@ -486,7 +569,8 @@ describe('AcpChatService', () => {
const firstConnection = createConnection();
const { service, child } = await createSpawnedService(firstConnection);
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'edit the file' });
const pending = service.client.requestPermission({
sessionId: 'agent:pi:s1',
toolCall: { toolCallId: 'tool-1', title: 'Edit file', status: 'pending' },
@@ -499,7 +583,7 @@ describe('AcpChatService', () => {
child.emit('error', new Error('spawn failed'));
await expectCancelledSoon(pending);
await service.loadSession({ sessionKey: 'agent:pi:s2', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' });
expect(childProcessMock.fork).toHaveBeenCalledTimes(2);
expect(secondConnection.initialize).toHaveBeenCalledTimes(1);
});
@@ -510,8 +594,8 @@ describe('AcpChatService', () => {
connection.initialize.mockReturnValue(initialized.promise);
const { service } = await createService(connection);
const firstLoad = service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
const secondLoad = service.loadSession({ sessionKey: 'agent:pi:s2', cwd: '/repo' });
const firstLoad = service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
const secondLoad = service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' });
await Promise.resolve();
expect(connection.initialize).toHaveBeenCalledTimes(1);
@@ -520,6 +604,84 @@ describe('AcpChatService', () => {
await expect(Promise.all([firstLoad, secondLoad])).resolves.toHaveLength(2);
});
it('serializes overlapping session loads on the shared ACP connection', async () => {
const connection = createConnection();
const firstLoad = createDeferred<unknown>();
connection.loadSession
.mockReturnValueOnce(firstLoad.promise)
.mockResolvedValueOnce({});
const { service } = await createService(connection);
const first = service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await vi.waitFor(() => expect(connection.loadSession).toHaveBeenCalledTimes(1));
const second = service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' });
await Promise.resolve();
expect(connection.loadSession).toHaveBeenCalledTimes(1);
firstLoad.resolve({});
await expect(first).resolves.toMatchObject({ success: true, generation: 1 });
await expect(second).resolves.toMatchObject({ success: true, generation: 2 });
expect(connection.loadSession).toHaveBeenCalledTimes(2);
});
it('returns session/load replay as one batch without forwarding incremental events', async () => {
const connection = createConnection();
const { service, send } = await createService(connection);
connection.loadSession.mockImplementationOnce(async () => {
await service.client.sessionUpdate({
sessionId: 'agent:pi:s1',
update: {
sessionUpdate: 'agent_message_chunk',
messageId: 'history-message',
content: { type: 'text', text: 'complete history' },
},
} as never);
return {};
});
await expect(service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' })).resolves.toEqual({
success: true,
generation: 1,
sessionUpdates: [{
sessionKey: 'agent:pi:s1',
generation: 1,
historical: true,
notification: {
sessionId: 'agent:pi:s1',
update: {
sessionUpdate: 'agent_message_chunk',
messageId: 'history-message',
content: { type: 'text', text: 'complete history' },
},
},
}],
});
expect(send).not.toHaveBeenCalledWith(HOST_EVENT_CHANNELS.chat.acpSessionUpdate, expect.anything());
});
it('filters old-session updates received before session/new returns its ACP id', async () => {
const connection = createConnection();
const { service } = await createService(connection);
connection.newSession.mockImplementationOnce(async () => {
await service.client.sessionUpdate({
sessionId: 'acp-old-session',
update: {
sessionUpdate: 'agent_message_chunk',
messageId: 'stale-message',
content: { type: 'text', text: 'old session tail' },
},
} as never);
return { sessionId: 'acp-new-session' };
});
await expect(service.loadSession({
sessionKey: 'agent:pi:session-new',
workspaceRoot: '/repo',
cwd: '/repo',
createIfMissing: true,
})).resolves.toEqual({ success: true, generation: 1 });
});
it('rejects prompts before any ACP session has loaded', async () => {
const { service, connection } = await createService();
@@ -534,7 +696,7 @@ describe('AcpChatService', () => {
it('rejects prompts for inactive ACP sessions', async () => {
const { service, connection } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
connection.prompt.mockClear();
await expect(service.sendPrompt({
@@ -552,7 +714,7 @@ describe('AcpChatService', () => {
connection.loadSession.mockReturnValueOnce(load.promise);
const { service, connection: activeConnection } = await createService(connection);
const loadPromise = service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
const loadPromise = service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await vi.waitFor(() => expect(connection.loadSession).toHaveBeenCalledTimes(1));
await expect(service.sendPrompt({
@@ -569,10 +731,10 @@ describe('AcpChatService', () => {
it('rolls back active session and generation when loadSession fails', async () => {
const { service, connection, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
connection.loadSession.mockRejectedValueOnce(new Error('load failed'));
await expect(service.loadSession({ sessionKey: 'agent:pi:s2', cwd: '/repo' })).resolves.toEqual({
await expect(service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' })).resolves.toEqual({
success: false,
error: 'load failed',
});
@@ -609,24 +771,118 @@ describe('AcpChatService', () => {
});
});
it('does not let an older failed overlapping load roll back a newer loaded session', async () => {
const { service, connection, send } = await createService();
it.each([
{ createIfMissing: false, operation: 'loadSession' as const },
{ createIfMissing: true, operation: 'newSession' as const },
])('commits a canonical access grant only after $operation resolves', async ({ createIfMissing, operation }) => {
const parent = mkdtempSync(join(tmpdir(), 'clawx-acp-service-access-'));
const workspaceRoot = join(parent, 'workspace');
const executionCwd = join(workspaceRoot, 'nested');
mkdirSync(executionCwd, { recursive: true });
const connection = createConnection();
const pending = createDeferred<unknown>();
connection[operation].mockReturnValueOnce(pending.promise);
try {
const { AcpSessionAccessRegistry } = await import('../../electron/services/acp-session-access-registry');
const accessRegistry = new AcpSessionAccessRegistry();
const { service } = await createService(connection, accessRegistry);
const load = service.loadSession({
sessionKey: 'agent:pi:grant',
workspaceRoot: join(workspaceRoot, '.'),
cwd: join(executionCwd, '.'),
...(createIfMissing ? { createIfMissing: true } : {}),
});
await vi.waitFor(() => expect(connection[operation]).toHaveBeenCalledTimes(1));
expect(accessRegistry.get('agent:pi:grant', 1)).toBeNull();
pending.resolve(createIfMissing ? { sessionId: 'created-session' } : {});
await expect(load).resolves.toEqual({ success: true, generation: 1 });
expect(accessRegistry.get('agent:pi:grant', 1)).toEqual({
sessionKey: 'agent:pi:grant',
generation: 1,
workspaceRoot: realpathSync(workspaceRoot),
executionCwd: realpathSync(executionCwd),
});
} finally {
rmSync(parent, { recursive: true, force: true });
}
});
it('restores the previous access grant when a later load fails', async () => {
const parent = mkdtempSync(join(tmpdir(), 'clawx-acp-service-rollback-'));
const firstRoot = join(parent, 'first');
const secondRoot = join(parent, 'second');
mkdirSync(firstRoot);
mkdirSync(secondRoot);
try {
const { AcpSessionAccessRegistry } = await import('../../electron/services/acp-session-access-registry');
const accessRegistry = new AcpSessionAccessRegistry();
const { service, connection } = await createService(createConnection(), accessRegistry);
await service.loadSession({ sessionKey: 'agent:pi:first', workspaceRoot: firstRoot, cwd: firstRoot });
connection.loadSession.mockRejectedValueOnce(new Error('load failed'));
await expect(service.loadSession({
sessionKey: 'agent:pi:second', workspaceRoot: secondRoot, cwd: secondRoot,
})).resolves.toEqual({ success: false, error: 'load failed' });
expect(accessRegistry.get('agent:pi:first', 1)).toEqual({
sessionKey: 'agent:pi:first',
generation: 1,
workspaceRoot: realpathSync(firstRoot),
executionCwd: realpathSync(firstRoot),
});
expect(accessRegistry.get('agent:pi:second', 2)).toBeNull();
} finally {
rmSync(parent, { recursive: true, force: true });
}
});
it('rejects a prompt cwd that differs from the registered execution cwd', async () => {
const workspaceRoot = mkdtempSync(join(tmpdir(), 'clawx-acp-service-prompt-cwd-'));
try {
const { AcpSessionAccessRegistry } = await import('../../electron/services/acp-session-access-registry');
const { service, connection } = await createService(createConnection(), new AcpSessionAccessRegistry());
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot, cwd: workspaceRoot });
await expect(service.sendPrompt({
sessionKey: 'agent:pi:s1',
cwd: join(workspaceRoot, 'replacement'),
message: 'wrong cwd',
})).resolves.toEqual({
success: false,
error: 'ACP prompt cwd does not match the registered execution cwd',
});
expect(connection.prompt).not.toHaveBeenCalled();
} finally {
rmSync(workspaceRoot, { recursive: true, force: true });
}
});
it('continues with a queued session load after the older load fails', async () => {
const { service, connection, send, accessRegistry } = await createService();
const firstLoad = createDeferred<unknown>();
connection.loadSession
.mockReturnValueOnce(firstLoad.promise)
.mockResolvedValueOnce({});
const older = service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
const older = service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await vi.waitFor(() => expect(connection.loadSession).toHaveBeenCalledTimes(1));
await expect(service.loadSession({ sessionKey: 'agent:pi:s2', cwd: '/repo' })).resolves.toEqual({
success: true,
generation: 2,
});
const newer = service.loadSession({ sessionKey: 'agent:pi:s2', workspaceRoot: '/repo', cwd: '/repo' });
firstLoad.reject(new Error('older load failed'));
await expect(older).resolves.toEqual({ success: false, error: 'older load failed' });
await expect(newer).resolves.toEqual({ success: true, generation: 1 });
expect(accessRegistry.get('agent:pi:s2', 1)).toEqual({
sessionKey: 'agent:pi:s2',
generation: 1,
workspaceRoot: '/repo',
executionCwd: '/repo',
});
await service.client.sessionUpdate({
sessionId: 'agent:pi:s1',
@@ -648,7 +904,7 @@ describe('AcpChatService', () => {
expect(send).toHaveBeenCalledTimes(1);
expect(send).toHaveBeenCalledWith(HOST_EVENT_CHANNELS.chat.acpSessionUpdate, {
sessionKey: 'agent:pi:s2',
generation: 2,
generation: 1,
historical: true,
notification: {
sessionId: 'agent:pi:s2',
@@ -661,10 +917,86 @@ describe('AcpChatService', () => {
});
});
it.each(['success', 'failure'] as const)(
'serializes deferred access preparation and keeps the later grant after older %s',
async (olderOutcome) => {
type AccessContext = {
sessionKey: string;
generation: number;
workspaceRoot: string;
executionCwd: string;
};
const preparations: Array<{
input: AccessContext;
deferred: ReturnType<typeof createDeferred<AccessContext>>;
}> = [];
let activeGrant: AccessContext | null = null;
const accessRegistry = {
prepareGrant: vi.fn((input: AccessContext) => {
const deferred = createDeferred<AccessContext>();
preparations.push({ input, deferred });
return deferred.promise;
}),
snapshot: vi.fn(() => activeGrant ? { ...activeGrant } : null),
commitGrant: vi.fn((context: AccessContext) => { activeGrant = { ...context }; }),
restore: vi.fn((snapshot: AccessContext | null) => { activeGrant = snapshot ? { ...snapshot } : null; }),
get: vi.fn((sessionKey: string, generation: number) => (
activeGrant?.sessionKey === sessionKey && activeGrant.generation === generation
? { ...activeGrant }
: null
)),
};
const connection = createConnection();
const { service } = await createService(connection, accessRegistry);
const olderLoad = service.loadSession({
sessionKey: 'agent:pi:older', workspaceRoot: '/older', cwd: '/older',
});
await vi.waitFor(() => expect(preparations).toHaveLength(1));
const laterLoad = service.loadSession({
sessionKey: 'agent:pi:later', workspaceRoot: '/later', cwd: '/later',
});
await Promise.resolve();
expect(preparations).toHaveLength(1);
if (olderOutcome === 'success') {
preparations[0].deferred.resolve(preparations[0].input);
await expect(olderLoad).resolves.toEqual({ success: true, generation: 1 });
} else {
preparations[0].deferred.reject(new Error('older preparation failed'));
await expect(olderLoad).resolves.toEqual({
success: false,
error: 'older preparation failed',
});
}
await vi.waitFor(() => expect(preparations).toHaveLength(2));
const laterGeneration = olderOutcome === 'success' ? 2 : 1;
expect(preparations.map(({ input }) => input.generation)).toEqual([1, laterGeneration]);
preparations[1].deferred.resolve(preparations[1].input);
await expect(laterLoad).resolves.toEqual({ success: true, generation: laterGeneration });
expect(accessRegistry.get('agent:pi:later', laterGeneration)).toEqual({
sessionKey: 'agent:pi:later',
generation: laterGeneration,
workspaceRoot: '/later',
executionCwd: '/later',
});
expect(accessRegistry.get('agent:pi:older', 1)).toBeNull();
expect(connection.loadSession).toHaveBeenCalledWith({
sessionId: 'agent:pi:later',
cwd: '/later',
mcpServers: [],
});
expect(accessRegistry.restore).not.toHaveBeenCalled();
},
);
it('cancels the ACP session and resolves pending permission requests for that session', async () => {
const { service, connection, send } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await service.sendPrompt({ sessionKey: 'agent:pi:s1', cwd: '/repo', message: 'edit the file' });
send.mockClear();
const pending = service.client.requestPermission({
@@ -691,15 +1023,15 @@ describe('AcpChatService', () => {
try {
const { service, connection } = await createService();
await service.loadSession({ sessionKey: 'agent:pi:s1', cwd: '/repo' });
await service.loadSession({ sessionKey: 'agent:pi:s1', workspaceRoot: '/repo', cwd: '/repo' });
await expect(service.sendPrompt({
sessionKey: 'agent:pi:s1',
cwd: '/repo',
message: 'Inspect attachments',
messageId: 'msg-user-1',
media: [
{ filePath: imagePath, mimeType: 'image/png', fileName: 'image.png' },
{ filePath, mimeType: 'text/plain', fileName: 'notes.txt' },
{ filePath: imagePath, stagingId: 'staged-image', mimeType: 'image/png', fileName: 'image.png' },
{ filePath, stagingId: 'staged-notes', mimeType: 'text/plain', fileName: 'notes.txt' },
],
})).resolves.toEqual({ success: true, generation: 1 });
@@ -713,11 +1045,14 @@ describe('AcpChatService', () => {
data: Buffer.from('fake-image').toString('base64'),
mimeType: 'image/png',
uri: imagePath,
_meta: { clawx: { stagingId: 'staged-image', fileName: 'image.png' } },
},
{
type: 'resource_link',
uri: filePath,
name: 'notes.txt',
mimeType: 'text/plain',
_meta: { clawx: { stagingId: 'staged-notes' } },
},
],
_meta: { sessionKey: 'agent:pi:s1', prefixCwd: true },
tests/unit/acp-chat-store.test.ts
+2022 -61
View File
File diff suppressed because it is too large Load Diff
tests/unit/acp-image-generation-compat.test.ts
+411 -3
View File
@@ -101,7 +101,7 @@ describe('ACP image-generation compatibility extraction', () => {
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: {
text: 'Generated image is ready.',
text: 'Your sunset over the mountains is ready.',
mediaUrls: ['/tmp/generated-sky.png'],
},
},
@@ -111,13 +111,271 @@ describe('ACP image-generation compatibility extraction', () => {
expect(evidence).toMatchObject({
sessionKey: `image_generate:${TASK_ID}`,
source: 'runtime-event',
caption: 'Generated image is ready.',
caption: 'Your sunset over the mountains is ready.',
});
expect(evidence?.candidates).toEqual([
{ key: '/tmp/generated-sky.png', filePath: '/tmp/generated-sky.png', mimeType: 'image/png' },
]);
});
it('extracts authoritative source-reply text from ACP tool output with or without media', () => {
const completion = (sourceReply: { text: string; mediaUrls?: string[] }) => extractImageGenerationCompletionFromAcpEnvelope({
sessionKey: SESSION_KEY,
generation: 1,
notification: {
sessionId: SESSION_KEY,
update: {
sessionUpdate: 'tool_call_update',
toolCallId: 'message-tool',
status: 'completed',
rawOutput: {
details: {
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply,
},
},
},
},
});
expect(completion({
text: 'Here is the watercolor skyline you requested.',
mediaUrls: ['/tmp/generated-skyline.png'],
})).toMatchObject({
source: 'acp-session-update',
caption: 'Here is the watercolor skyline you requested.',
candidates: [{ key: '/tmp/generated-skyline.png' }],
});
expect(completion({
text: 'Image generation failed because no image model is available.',
})).toMatchObject({
source: 'acp-session-update',
caption: 'Image generation failed because no image model is available.',
candidates: [],
});
});
it('preserves source-reply text exactly and rejects unsuccessful internal-UI deliveries', () => {
const completion = (details: Record<string, unknown>) => extractImageGenerationCompletionFromAcpEnvelope({
sessionKey: SESSION_KEY,
generation: 1,
notification: {
sessionId: SESSION_KEY,
update: {
sessionUpdate: 'tool_call_update',
toolCallId: 'message-tool',
status: 'completed',
rawOutput: { details },
},
},
});
const exactText = ' First line\n indented line \n';
expect(completion({
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: { text: exactText },
})?.caption).toBe(exactText);
expect(completion({
status: 'error',
deliveryStatus: 'failed',
sourceReplySink: 'internal-ui',
sourceReply: { text: 'This was not delivered.', mediaUrls: ['/tmp/not-delivered.png'] },
})).toBeNull();
expect(completion({
status: 'ok',
deliveryStatus: 'cancelled',
sourceReplySink: 'internal-ui',
sourceReply: { text: 'This was cancelled.' },
})).toBeNull();
});
it('extracts text-only internal-UI failures from runtime message-tool deliveries', () => {
const evidence = extractImageGenerationCompletionFromRuntimeEvent({
type: 'tool.completed',
runId: `image_generate:${TASK_ID}:error`,
sessionKey: `image_generate:${TASK_ID}`,
toolCallId: 'message-tool',
name: 'message',
result: {
details: {
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: {
text: 'Image generation timed out before an image was produced.',
},
},
},
});
expect(evidence).toMatchObject({
sessionKey: `image_generate:${TASK_ID}`,
source: 'runtime-event',
taskId: TASK_ID,
caption: 'Image generation timed out before an image was produced.',
candidates: [],
});
});
it('accepts OpenClaw message-tool delivery details when lifecycle meta is completed', () => {
const evidence = extractImageGenerationCompletionFromRuntimeEvent({
type: 'tool.completed',
runId: `image_generate:${TASK_ID}:ok`,
sessionKey: `image_generate:${TASK_ID}`,
toolCallId: 'message-tool',
name: 'message',
meta: { status: 'completed' },
result: {
details: {
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: {
text: 'The generated puppy is ready.',
mediaUrls: ['/tmp/generated-puppy.png'],
},
},
},
});
expect(evidence).toMatchObject({
taskId: TASK_ID,
caption: 'The generated puppy is ready.',
candidates: [{ key: '/tmp/generated-puppy.png' }],
});
});
it('extracts final assistant text from a correlated image-generation completion run', () => {
expect(extractImageGenerationCompletionFromRuntimeEvent({
type: 'assistant.delta',
runId: `image_generate:${TASK_ID}:error`,
sessionKey: SESSION_KEY,
phase: 'final_answer',
text: '抱歉，这次小猫图生成失败了：当前图像生成模型通道不可用。',
})).toMatchObject({
taskId: TASK_ID,
caption: '抱歉，这次小猫图生成失败了：当前图像生成模型通道不可用。',
authoritativeCaption: true,
candidates: [],
});
});
it('extracts final Gateway assistant MEDIA replies from a correlated completion run', () => {
const evidence = extractImageGenerationCompletionFromGatewayChatMessage({
message: {
runId: `image_generate:${TASK_ID}:ok`,
sessionKey: SESSION_KEY,
state: 'final',
message: {
role: 'assistant',
content: [{
type: 'text',
text: '生成好了 🐶\n\nMEDIA:/tmp/generated-puppy.png',
}],
},
},
});
expect(evidence).toMatchObject({
taskId: TASK_ID,
caption: '生成好了 🐶',
authoritativeCaption: true,
candidates: [{ key: '/tmp/generated-puppy.png' }],
});
expect(extractImageGenerationCompletionFromGatewayChatMessage({
message: {
runId: `image_generate:${TASK_ID}:ok`,
sessionKey: SESSION_KEY,
state: 'final',
message: {
role: 'assistant',
content: [{
type: 'text',
text: ' 橘子来了~MEDIA:/tmp/generated-orange.png',
}],
},
},
})).toMatchObject({
caption: ' 橘子来了~',
candidates: [{ key: '/tmp/generated-orange.png' }],
});
});
it('rejects failed runtime deliveries and text-only assistant deltas', () => {
expect(extractImageGenerationCompletionFromRuntimeEvent({
type: 'tool.completed',
runId: `image_generate:${TASK_ID}:error`,
sessionKey: `image_generate:${TASK_ID}`,
toolCallId: 'message-tool',
name: 'message',
isError: true,
result: {
details: {
status: 'error',
deliveryStatus: 'not-sent',
sourceReplySink: 'internal-ui',
sourceReply: { text: 'Not delivered.', mediaUrls: ['/tmp/not-delivered.png'] },
},
},
})).toBeNull();
expect(extractImageGenerationCompletionFromRuntimeEvent({
type: 'assistant.delta',
runId: `image_generate:${TASK_ID}:error`,
sessionKey: SESSION_KEY,
text: 'Partial untrusted completion text',
})).toBeNull();
expect(extractImageGenerationCompletionFromRuntimeEvent({
type: 'tool.completed',
runId: `image_generate:${TASK_ID}:error`,
sessionKey: `image_generate:${TASK_ID}`,
toolCallId: 'message-tool',
name: 'message',
result: {
details: {
status: 'ok',
deliveryStatus: 'timeout',
sourceReplySink: 'internal-ui',
sourceReply: { mediaUrls: ['/tmp/timed-out.png'] },
},
},
})).toBeNull();
expect(extractImageGenerationCompletionFromGatewayChatMessage({
message: {
sessionKey: `image_generate:${TASK_ID}`,
state: 'final',
runId: `image_generate:${TASK_ID}:error`,
message: {
role: 'toolresult',
toolName: 'message',
details: {
status: 'rejected',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: { mediaUrls: ['/tmp/rejected.png'] },
},
},
},
})).toBeNull();
expect(extractImageGenerationCompletionFromGatewayChatMessage({
message: {
sessionKey: `image_generate:${TASK_ID}`,
state: 'final',
runId: `image_generate:${TASK_ID}:error`,
message: {
role: 'toolresult',
toolName: 'message',
details: {
status: 'error',
mediaUrls: ['/tmp/unconfirmed-error.png'],
},
},
},
})).toBeNull();
});
it('extracts historical ACP image-generation media from assistant MEDIA text', () => {
const evidence = extractImageGenerationCompletionFromAcpEnvelope({
sessionKey: SESSION_KEY,
@@ -184,7 +442,7 @@ describe('ACP image-generation compatibility extraction', () => {
sessionKey: SESSION_KEY,
source: 'transcript-history',
historical: true,
caption: 'Generated image is ready.',
caption: '图片生成完成！',
});
expect(supplement.completions[0]?.candidates).toEqual([
{
@@ -195,6 +453,119 @@ describe('ACP image-generation compatibility extraction', () => {
]);
});
it('extracts text-only internal-UI failures from transcript fallback after an image_generate start', () => {
const supplement = extractImageGenerationTranscriptSupplement([
{
role: 'toolresult',
toolCallId: 'tool-image',
toolName: 'image_generate',
content: `Background task started for image generation (${TASK_ID}).`,
details: { taskId: TASK_ID },
},
{
role: 'toolresult',
id: 'message-delivery-failure',
toolCallId: 'message-tool',
toolName: 'message',
content: 'Sent visible reply to the current source conversation via internal-ui.',
details: {
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: {
text: 'Image generation failed because the provider returned no images.',
},
},
},
], SESSION_KEY);
expect(supplement.completions).toEqual([
expect.objectContaining({
taskId: TASK_ID,
toolCallId: 'tool-image',
caption: 'Image generation failed because the provider returned no images.',
candidates: [],
}),
]);
});
it('does not associate transcript MEDIA evidence across user-turn boundaries', () => {
const supplement = extractImageGenerationTranscriptSupplement([
{
role: 'toolresult',
toolCallId: 'tool-image',
toolName: 'image_generate',
content: `Background task started for image generation (${TASK_ID}).`,
details: { taskId: TASK_ID },
},
{ role: 'user', content: 'A different request' },
{
role: 'assistant',
id: 'unrelated-media',
content: 'MEDIA:/tmp/unrelated.png',
},
], SESSION_KEY);
expect(supplement.starts).toHaveLength(1);
expect(supplement.completions).toEqual([]);
});
it('keeps internal image-completion triggers in the originating transcript turn', () => {
const failure = extractImageGenerationTranscriptSupplement([
{ role: 'user', content: '生成一张小猫图' },
{
role: 'toolresult',
toolCallId: 'tool-image',
toolName: 'image_generate',
content: `Background task started for image generation (${TASK_ID}).`,
details: { taskId: TASK_ID },
},
{
role: 'user',
content: `[Inter-session message] sourceSession=image_generate:${TASK_ID} sourceChannel=webchat sourceTool=image_generate isUser=false\n[Internal task completion event]\nstatus: failed`,
},
{
role: 'assistant',
id: 'kitten-failure',
content: '抱歉，这次小猫图生成失败了：当前图像生成模型通道不可用。',
},
], SESSION_KEY);
const success = extractImageGenerationTranscriptSupplement([
{ role: 'user', content: '生成一张小狗图' },
{
role: 'toolresult',
toolCallId: 'tool-image',
toolName: 'image_generate',
content: `Background task started for image generation (${TASK_ID}).`,
details: { taskId: TASK_ID },
},
{
role: 'user',
content: `[Inter-session message] sourceSession=image_generate:${TASK_ID} sourceChannel=webchat sourceTool=image_generate isUser=false\n[Internal task completion event]\nstatus: completed successfully`,
},
{
role: 'assistant',
id: 'puppy-success',
content: '生成好了 🐶\n\nMEDIA:/tmp/generated-puppy.png',
},
], SESSION_KEY);
expect(failure.completions).toEqual([
expect.objectContaining({
taskId: TASK_ID,
caption: '抱歉，这次小猫图生成失败了：当前图像生成模型通道不可用。',
candidates: [],
}),
]);
expect(success.completions).toEqual([
expect.objectContaining({
taskId: TASK_ID,
caption: '生成好了 🐶',
candidates: [expect.objectContaining({ key: '/tmp/generated-puppy.png' })],
}),
]);
});
it('does not extract transcript assistant MEDIA text without a prior image_generate start', () => {
const supplement = extractImageGenerationTranscriptSupplement([
{
@@ -347,5 +718,42 @@ describe('ACP image-generation compatibility extraction', () => {
expect(collisionEvidenceA).not.toBeNull();
expect(collisionEvidenceB).not.toBeNull();
expect(imageGenerationEvidenceKey(collisionEvidenceA!)).not.toBe(imageGenerationEvidenceKey(collisionEvidenceB!));
const gatewayFailure = extractImageGenerationCompletionFromGatewayChatMessage({
message: {
sessionKey: `image_generate:${TASK_ID}`,
state: 'final',
runId: `image_generate:${TASK_ID}:error`,
message: {
role: 'toolresult',
toolName: 'message',
details: {
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: { text: 'Image generation failed.' },
},
},
},
});
const runtimeFailure = extractImageGenerationCompletionFromRuntimeEvent({
type: 'tool.completed',
runId: `image_generate:${TASK_ID}:error`,
sessionKey: `image_generate:${TASK_ID}`,
toolCallId: 'message-tool',
name: 'message',
result: {
details: {
status: 'ok',
deliveryStatus: 'sent',
sourceReplySink: 'internal-ui',
sourceReply: { text: 'Image generation failed.' },
},
},
});
expect(gatewayFailure).not.toBeNull();
expect(runtimeFailure).not.toBeNull();
expect(imageGenerationEvidenceKey(gatewayFailure!)).toBe(imageGenerationEvidenceKey(runtimeFailure!));
});
});
tests/unit/acp-media-attachments.test.ts
+491
View File
@@ -0,0 +1,491 @@
import { describe, expect, it } from 'vitest';
import type { RawMessage } from '@shared/chat/types';
import {
alignOpenClawMediaTurns,
extractOpenClawMediaTurns,
} from '@/lib/acp/openclaw-media-compat';
import {
appendSyntheticAssistantMessage,
createEmptyAcpTimeline,
upsertSyntheticTurnAttachments,
} from '@/lib/acp/reducer';
import type { AcpTimelineSnapshot, AttachmentRenderPart } from '@/lib/acp/timeline-types';
function transcript(...messages: RawMessage[]): RawMessage[] {
return messages;
}
function extract(messages: RawMessage[], suppressedUris = new Set<string>()) {
return extractOpenClawMediaTurns(messages, {
executionCwd: '/workspace/project',
suppressedUris,
});
}
function timeline(
turns: Array<{ userId: string; userText: string; assistantId?: string; assistantText?: string }>,
): AcpTimelineSnapshot {
const snapshot = createEmptyAcpTimeline('agent:main:session-1', 4);
for (const turn of turns) {
const userItemId = `${turn.userId}:0`;
snapshot.itemOrder.push(userItemId);
snapshot.itemsById[userItemId] = {
kind: 'message-segment',
id: userItemId,
role: 'user',
messageId: turn.userId,
segmentIndex: 0,
parts: [{ kind: 'markdown', text: turn.userText }],
};
if (turn.assistantId) {
const assistantItemId = `${turn.assistantId}:0`;
snapshot.itemOrder.push(assistantItemId);
snapshot.itemsById[assistantItemId] = {
kind: 'message-segment',
id: assistantItemId,
role: 'assistant',
messageId: turn.assistantId,
segmentIndex: 0,
parts: [{ kind: 'markdown', text: turn.assistantText ?? '' }],
};
}
}
return snapshot;
}
function availableAttachment(id: string, identity: string): AttachmentRenderPart {
const uri = `/workspace/project/${id}.png`;
return {
kind: 'attachment',
attachmentId: id,
reference: { uri, name: `${id}.png` },
source: 'openclaw-media',
evidenceId: `evidence:${id}`,
access: {
status: 'available',
identity,
mimeType: 'image/png',
size: 12,
target: {
kind: 'local',
scope: 'workspace',
ref: { sessionKey: 'agent:main:session-1', generation: 4, uri },
},
},
};
}
describe('OpenClaw MEDIA transcript extraction', () => {
it('accepts the approved explicit reference forms and preserves directive order', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Create the exports' },
{
role: 'assistant',
id: 'assistant-files',
content: [
{
type: 'text',
text: [
' MEDIA:/tmp/report.pdf',
'MEDIA:C:\\Users\\alex\\report.xlsx',
'MEDIA:file:///tmp/report.csv',
'MEDIA:~/report.txt',
'MEDIA:exports/report.json',
'MEDIA:https://example.test/report.zip',
'MEDIA:http://example.test/report.txt',
'MEDIA:"/tmp/quarterly report.xlsx"',
].join('\n'),
},
],
},
));
expect(turn?.candidates.map((candidate) => candidate.uri)).toEqual([
'/tmp/report.pdf',
'C:\\Users\\alex\\report.xlsx',
'file:///tmp/report.csv',
'~/report.txt',
'exports/report.json',
'https://example.test/report.zip',
'http://example.test/report.txt',
'/tmp/quarterly report.xlsx',
]);
expect(turn?.candidates.map((candidate) => candidate.order)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]);
expect(turn?.candidates.every((candidate) => candidate.transcriptMessageId === 'assistant-files')).toBe(true);
});
it('rejects fenced, wrapped, inline, unknown-scheme, malformed, and overlong references', () => {
const tooLong = `/tmp/${'x'.repeat(4092)}`;
const [turn] = extract(transcript(
{ role: 'user', content: 'Create the export' },
{
role: 'assistant',
content: [
{
type: 'text',
text: [
'```text',
'MEDIA:/tmp/fenced.txt',
'```',
'- MEDIA:/tmp/list.txt',
'**MEDIA:/tmp/bold.txt**',
'Here is MEDIA:/tmp/inline.txt',
'ftp://example.test/bare.txt',
'MEDIA:ftp://example.test/file.txt',
'MEDIA:/tmp/path with unquoted spaces.txt',
`MEDIA:${tooLong}`,
'/tmp/bare.txt',
].join('\n'),
},
],
},
{ role: 'toolresult', content: 'MEDIA:/tmp/tool.txt' },
{ role: 'system', content: 'MEDIA:/tmp/system.txt' },
{ role: 'user', content: 'MEDIA:/tmp/user.txt' },
));
expect(turn?.candidates).toEqual([]);
});
it('requires a fence close to use the same delimiter and at least the opening length', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Create exports' },
{
role: 'assistant',
content: [
'````markdown',
'MEDIA:/tmp/hidden-backtick-a.txt',
'```',
'MEDIA:/tmp/hidden-backtick-b.txt',
'````',
'MEDIA:/tmp/visible-backtick.txt',
'~~~~text',
'MEDIA:/tmp/hidden-tilde-a.txt',
'```',
'MEDIA:/tmp/hidden-tilde-b.txt',
'~~~',
'MEDIA:/tmp/hidden-tilde-c.txt',
'~~~~~',
'MEDIA:/tmp/visible-tilde.txt',
].join('\n'),
},
));
expect(turn?.candidates.map((candidate) => candidate.uri)).toEqual([
'/tmp/visible-backtick.txt',
'/tmp/visible-tilde.txt',
]);
});
it('keeps backtick MEDIA fenced after trailing-text and over-indented close candidates', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Create exports' },
{
role: 'assistant',
content: [
'```text',
'MEDIA:/tmp/hidden-backtick-a.txt',
'```not-a-close',
'MEDIA:/tmp/hidden-backtick-b.txt',
' ```',
'MEDIA:/tmp/hidden-backtick-c.txt',
' ``` \t',
'MEDIA:/tmp/visible-backtick.txt',
].join('\n'),
},
));
expect(turn?.candidates.map((candidate) => candidate.uri)).toEqual([
'/tmp/visible-backtick.txt',
]);
});
it('keeps tilde MEDIA fenced after trailing-text and over-indented close candidates', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Create exports' },
{
role: 'assistant',
content: [
'~~~~text',
'MEDIA:/tmp/hidden-tilde-a.txt',
'~~~~not-a-close',
'MEDIA:/tmp/hidden-tilde-b.txt',
' ~~~~~',
'MEDIA:/tmp/hidden-tilde-c.txt',
' ~~~~~\t',
'MEDIA:/tmp/visible-tilde.txt',
].join('\n'),
},
));
expect(turn?.candidates.map((candidate) => candidate.uri)).toEqual([
'/tmp/visible-tilde.txt',
]);
});
it('does not open a fence indented by more than three spaces', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Create exports' },
{
role: 'assistant',
content: [
' ```text',
'MEDIA:/tmp/visible-after-indented-backtick.txt',
' ~~~~text',
'MEDIA:/tmp/visible-after-indented-tilde.txt',
].join('\n'),
},
));
expect(turn?.candidates.map((candidate) => candidate.uri)).toEqual([
'/tmp/visible-after-indented-backtick.txt',
'/tmp/visible-after-indented-tilde.txt',
]);
});
it('removes only a leading ACP working-directory envelope while matching user text', () => {
const turns = extract(transcript(
{ role: 'user', content: '[Working directory: /workspace/project]\n\nCreate report' },
{ role: 'assistant', content: 'MEDIA:report.pdf' },
{ role: 'user', content: 'Keep this text\n[Working directory: /authored/example]' },
{ role: 'assistant', content: 'MEDIA:notes.txt' },
));
expect(turns.map((turn) => turn.normalizedUserText)).toEqual([
'Create report',
'Keep this text\n[Working directory: /authored/example]',
]);
});
it('uses stable evidence ids across immediate and delayed reads', () => {
const messages = transcript(
{ role: 'user', content: 'Create report' },
{ role: 'assistant', id: 'assistant-report', content: 'MEDIA:report.pdf' },
);
expect(extract(messages)[0]?.candidates[0]?.evidenceId)
.toBe(extract(structuredClone(messages))[0]?.candidates[0]?.evidenceId);
});
it('keeps fallback evidence stable when unrelated messages shift the transcript window', () => {
const targetTurn = transcript(
{ role: 'user', content: 'Create report' },
{ role: 'assistant', content: 'MEDIA:report.pdf' },
);
const shifted = transcript(
{ role: 'user', content: 'Unrelated prompt' },
{ role: 'assistant', content: 'Unrelated answer' },
...targetTurn,
);
expect(extract(targetTurn)[0]?.candidates[0]?.evidenceId)
.toBe(extract(shifted)[1]?.candidates[0]?.evidenceId);
});
it('gives repeated identical turns distinct fallback evidence ids', () => {
const turns = extract(transcript(
{ role: 'user', content: 'Create report' },
{ role: 'assistant', content: 'MEDIA:report.pdf' },
{ role: 'user', content: 'Create report' },
{ role: 'assistant', content: 'MEDIA:report.pdf' },
));
expect(turns[0]?.candidates[0]?.evidenceId).not.toBe(turns[1]?.candidates[0]?.evidenceId);
});
it('suppresses only exact image-generation candidate keys', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Create assets' },
{
role: 'assistant',
content: 'MEDIA:/tmp/generated.png\nMEDIA:file:///tmp/generated.png\nMEDIA:/tmp/report.pdf',
},
), new Set(['/tmp/generated.png']));
expect(turn?.candidates.map((candidate) => candidate.uri)).toEqual([
'file:///tmp/generated.png',
'/tmp/report.pdf',
]);
});
it('keeps an explicit image directive without proven image-generation evidence', () => {
const [turn] = extract(transcript(
{ role: 'user', content: 'Send the screenshot' },
{ role: 'assistant', content: 'MEDIA:/tmp/screenshot.png' },
));
expect(turn?.candidates).toMatchObject([{ uri: '/tmp/screenshot.png' }]);
});
});
describe('OpenClaw MEDIA transcript turn alignment', () => {
it('aligns a bounded transcript suffix newest-to-oldest', () => {
const snapshot = timeline([
{ userId: 'user-old', userText: 'Old prompt', assistantId: 'assistant-old' },
{ userId: 'user-middle', userText: 'Middle prompt', assistantId: 'assistant-middle' },
{ userId: 'user-new', userText: 'New prompt', assistantId: 'assistant-new' },
]);
const turns = extract(transcript(
{ role: 'user', content: 'Middle prompt' },
{ role: 'assistant', content: 'MEDIA:middle.pdf' },
{ role: 'user', content: 'New prompt' },
{ role: 'assistant', content: 'MEDIA:new.pdf' },
));
expect(alignOpenClawMediaTurns(snapshot, turns, {})).toMatchObject([
{ acpTurnId: 'user-middle', candidates: [{ uri: 'middle.pdf' }] },
{ acpTurnId: 'user-new', candidates: [{ uri: 'new.pdf' }] },
]);
});
it('matches repeated prompts by occurrence from the tail', () => {
const snapshot = timeline([
{ userId: 'user-first', userText: 'Repeat', assistantId: 'assistant-first' },
{ userId: 'user-other', userText: 'Other', assistantId: 'assistant-other' },
{ userId: 'user-last', userText: 'Repeat', assistantId: 'assistant-last' },
]);
const turns = extract(transcript(
{ role: 'user', content: 'Repeat' },
{ role: 'assistant', content: 'MEDIA:first.pdf' },
{ role: 'user', content: 'Other' },
{ role: 'assistant', content: 'No attachment' },
{ role: 'user', content: 'Repeat' },
{ role: 'assistant', content: 'MEDIA:last.pdf' },
));
expect(alignOpenClawMediaTurns(snapshot, turns, {})).toMatchObject([
{ acpTurnId: 'user-first', candidates: [{ uri: 'first.pdf' }] },
{ acpTurnId: 'user-last', candidates: [{ uri: 'last.pdf' }] },
]);
});
it('rejects missing anchors and restricts live matching to the optimistic user identity', () => {
const snapshot = timeline([
{ userId: 'user-first', userText: 'First', assistantId: 'assistant-first' },
{ userId: 'user-live', userText: 'Live prompt' },
]);
const turns = extract(transcript(
{ role: 'user', content: 'Missing' },
{ role: 'assistant', content: 'MEDIA:missing.pdf' },
{ role: 'user', content: 'Live prompt' },
{ role: 'assistant', content: 'MEDIA:live.pdf' },
));
expect(alignOpenClawMediaTurns(snapshot, turns, { liveUserMessageId: 'user-first' })).toEqual([]);
expect(alignOpenClawMediaTurns(snapshot, turns, { liveUserMessageId: 'user-live' })).toMatchObject([
{ acpTurnId: 'user-live', candidates: [{ uri: 'live.pdf' }] },
]);
});
it('aligns an attachment-only assistant output to a user turn without an ACP assistant segment', () => {
const snapshot = timeline([{ userId: 'user-only', userText: 'Create report' }]);
const turns = extract(transcript(
{ role: 'user', content: 'Create report' },
{ role: 'assistant', content: 'MEDIA:report.pdf' },
));
expect(alignOpenClawMediaTurns(snapshot, turns, {})).toMatchObject([
{ acpTurnId: 'user-only', candidates: [{ uri: 'report.pdf' }] },
]);
});
});
describe('synthetic OpenClaw MEDIA projection', () => {
it('anchors marked attachment-only segments inside the matching turn', () => {
const snapshot = timeline([
{ userId: 'user-report', userText: 'Create report' },
{ userId: 'user-next', userText: 'Next', assistantId: 'assistant-next' },
]);
const projected = upsertSyntheticTurnAttachments(snapshot, {
turnId: 'user-report',
evidenceId: 'evidence-report',
attachments: [availableAttachment('report', 'report-identity')],
source: 'openclaw-media',
});
expect(projected.itemOrder).toEqual([
'user-report:0',
'compat:openclaw-media:evidence-report:0',
'user-next:0',
'assistant-next:0',
]);
expect(projected.itemsById['compat:openclaw-media:evidence-report:0']).toMatchObject({
role: 'assistant',
compat: { source: 'openclaw-media', evidenceId: 'evidence-report' },
parts: [{ attachmentId: 'report' }],
});
});
it('lets an inline image remove an earlier same-identity attachment card', () => {
const snapshot = upsertSyntheticTurnAttachments(
timeline([{ userId: 'user-image', userText: 'Create image' }]),
{
turnId: 'user-image',
evidenceId: 'attachment-evidence',
attachments: [availableAttachment('generated', 'same-media')],
source: 'openclaw-media',
},
);
const projected = appendSyntheticAssistantMessage(snapshot, {
messageId: 'compat:image-generation:image-evidence',
evidenceId: 'image-evidence',
parts: [{ kind: 'image', source: 'data:image/png;base64,abc', mediaIdentity: 'same-media' }],
});
const attachments = Object.values(projected.itemsById)
.flatMap((item) => item.kind === 'message-segment' ? item.parts : [])
.filter((part) => part.kind === 'attachment');
expect(attachments).toEqual([]);
});
it('rejects a same-identity attachment card when the inline image arrived first', () => {
const withImage = appendSyntheticAssistantMessage(
timeline([{ userId: 'user-image', userText: 'Create image' }]),
{
messageId: 'compat:image-generation:image-evidence',
evidenceId: 'image-evidence',
parts: [{ kind: 'image', source: 'data:image/png;base64,abc', mediaIdentity: 'same-media' }],
},
);
const projected = upsertSyntheticTurnAttachments(withImage, {
turnId: 'user-image',
evidenceId: 'attachment-evidence',
attachments: [availableAttachment('generated', 'same-media')],
source: 'openclaw-media',
});
const attachments = Object.values(projected.itemsById)
.flatMap((item) => item.kind === 'message-segment' ? item.parts : [])
.filter((part) => part.kind === 'attachment');
expect(attachments).toEqual([]);
});
it.each([
'/workspace/project/generated.png',
'file:///workspace/project/generated.png',
'~/project/generated.png',
'generated.png',
'/api/chat/media/outgoing/agent%3Amain%3Asession-1/attachment/full',
])('dedupes equivalent resolved reference %s by opaque identity', (uri) => {
const withImage = appendSyntheticAssistantMessage(
timeline([{ userId: 'user-image', userText: 'Create image' }]),
{
messageId: 'compat:image-generation:image-evidence',
evidenceId: 'image-evidence',
parts: [{ kind: 'image', source: 'data:image/png;base64,abc', mediaIdentity: 'canonical-image' }],
},
);
const attachment = availableAttachment('generated', 'canonical-image');
attachment.reference.uri = uri;
const projected = upsertSyntheticTurnAttachments(withImage, {
turnId: 'user-image',
evidenceId: `attachment-${attachment.attachmentId}`,
attachments: [attachment],
source: 'openclaw-media',
});
expect(Object.values(projected.itemsById)
.flatMap((item) => item.kind === 'message-segment' ? item.parts : [])
.filter((part) => part.kind === 'attachment')).toEqual([]);
});
});
tests/unit/acp-reducer.test.ts
+180 -12
View File
@@ -1,7 +1,15 @@
import { describe, expect, it } from 'vitest';
import { applyAttachmentResolution, attachmentRequestFingerprint } from '@/lib/acp/attachments';
import { contentBlockToRenderPart, toolContentToRenderPart } from '@/lib/acp/content-blocks';
import { appendSyntheticAssistantMessage, applyAcpSessionUpdate, createEmptyAcpTimeline } from '@/lib/acp/reducer';
const assistantBlockContext = {
role: 'assistant' as const,
messageId: 'msg-a',
segmentIndex: 0,
blockIndex: 0,
};
describe('ACP timeline reducer', () => {
it('segments assistant text when process blocks interleave with the same messageId', () => {
let state = createEmptyAcpTimeline('agent:pi:s1', 1);
@@ -228,7 +236,13 @@ describe('ACP timeline reducer', () => {
optimistic: true,
parts: [
{ kind: 'markdown', text: 'inspect this' },
{ kind: 'file', path: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
{
kind: 'attachment',
attachmentId: 'attachment:user-msg:0:1',
reference: { uri: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
source: 'acp-resource',
access: { status: 'pending' },
},
],
},
},
@@ -258,7 +272,13 @@ describe('ACP timeline reducer', () => {
optimistic: false,
parts: [
{ kind: 'markdown', text: 'inspect this' },
{ kind: 'file', path: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
{
kind: 'attachment',
attachmentId: 'attachment:user-msg:0:1',
reference: { uri: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
source: 'acp-resource',
access: { status: 'pending' },
},
],
});
});
@@ -278,7 +298,13 @@ describe('ACP timeline reducer', () => {
optimistic: true,
parts: [
{ kind: 'markdown', text: 'inspect this' },
{ kind: 'file', path: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
{
kind: 'attachment',
attachmentId: 'attachment:user-msg:0:1',
reference: { uri: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
source: 'acp-resource',
access: { status: 'pending' },
},
],
},
},
@@ -300,7 +326,13 @@ describe('ACP timeline reducer', () => {
optimistic: false,
parts: [
{ kind: 'markdown', text: 'inspect this' },
{ kind: 'file', path: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
{
kind: 'attachment',
attachmentId: 'attachment:user-msg:0:1',
reference: { uri: '/repo/notes.txt', name: 'notes.txt', mimeType: 'text/plain' },
source: 'acp-resource',
access: { status: 'pending' },
},
],
});
});
@@ -370,7 +402,13 @@ describe('ACP timeline reducer', () => {
expect(item.parts).toEqual([
{ kind: 'markdown', text: 'complete' },
{ kind: 'image', source: 'file:///tmp/plot.png', mimeType: 'image/png' },
{ kind: 'file', path: 'file:///tmp/result.txt', name: 'result.txt', mimeType: 'text/plain' },
{
kind: 'attachment',
attachmentId: 'attachment:msg-a:0:2',
reference: { uri: 'file:///tmp/result.txt', name: 'result.txt', mimeType: 'text/plain' },
source: 'acp-resource',
access: { status: 'pending' },
},
]);
}
});
@@ -519,11 +557,17 @@ describe('ACP timeline reducer', () => {
});
});
it('converts embedded resources with a uri into file render parts', () => {
it('converts embedded resources with a uri into attachment render parts', () => {
expect(contentBlockToRenderPart({
type: 'resource',
resource: { uri: 'file:///tmp/report.md', text: '# Report', mimeType: 'text/markdown' },
})).toEqual({ kind: 'file', path: 'file:///tmp/report.md', mimeType: 'text/markdown' });
}, assistantBlockContext)).toEqual({
kind: 'attachment',
attachmentId: 'attachment:msg-a:0:0',
reference: { uri: 'file:///tmp/report.md', name: 'report.md', mimeType: 'text/markdown' },
source: 'acp-resource',
access: { status: 'pending' },
});
});
it('prefers image data when an image uri is not render-safe', () => {
@@ -532,13 +576,137 @@ describe('ACP timeline reducer', () => {
uri: '/tmp/staged-image.png',
data: 'abc123',
mimeType: 'image/png',
})).toEqual({ kind: 'image', source: 'data:image/png;base64,abc123', mimeType: 'image/png' });
}, assistantBlockContext)).toEqual({ kind: 'image', source: 'data:image/png;base64,abc123', mimeType: 'image/png' });
});
it('returns an error render part for malformed embedded resources', () => {
expect(contentBlockToRenderPart({ type: 'resource', resource: null } as never)).toEqual({
kind: 'error',
message: 'Unsupported ACP resource content',
it('returns an unavailable attachment for embedded resources without a usable uri', () => {
expect(contentBlockToRenderPart({
type: 'resource',
resource: { text: '# Report', mimeType: 'text/markdown' },
} as never, assistantBlockContext)).toEqual({
kind: 'attachment',
attachmentId: 'attachment:msg-a:0:0',
reference: { uri: '', name: '' , mimeType: 'text/markdown' },
source: 'acp-resource',
access: { status: 'unavailable', reason: 'invalidReference' },
});
});
it('preserves resource link metadata and accepts ClawX staging ownership only from user content', () => {
const resource = {
type: 'resource_link' as const,
uri: 'file:///tmp/budget.xlsx',
name: '',
title: 'Budget workbook',
mimeType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
size: 2048,
_meta: { clawx: { stagingId: 'stage-1' } },
};
expect(contentBlockToRenderPart(resource, {
...assistantBlockContext,
role: 'user',
})).toMatchObject({
kind: 'attachment',
attachmentId: 'attachment:msg-a:0:0',
reference: {
uri: 'file:///tmp/budget.xlsx',
name: 'Budget workbook',
mimeType: resource.mimeType,
size: 2048,
stagingId: 'stage-1',
},
source: 'acp-resource',
access: { status: 'pending' },
});
expect(contentBlockToRenderPart(resource, assistantBlockContext)).not.toMatchObject({
reference: { stagingId: 'stage-1' },
});
});
it('renders a staged user image block as an attachment and ignores untrusted display paths', () => {
expect(contentBlockToRenderPart({
type: 'image',
data: 'abc123',
mimeType: 'image/png',
uri: '/tmp/clawx-staging/photo.png',
_meta: {
clawx: {
stagingId: 'stage-photo',
displayPath: '/spoofed/private/path/photo.png',
fileName: 'photo.png',
},
},
}, {
role: 'user', messageId: 'user-photo', segmentIndex: 0, blockIndex: 0,
})).toMatchObject({
kind: 'attachment',
reference: {
uri: '/tmp/clawx-staging/photo.png',
name: 'photo.png',
mimeType: 'image/png',
stagingId: 'stage-photo',
},
});
expect(contentBlockToRenderPart({
type: 'resource_link',
uri: '/tmp/clawx-staging/notes.txt',
name: 'notes.txt',
_meta: { clawx: { stagingId: 'stage-notes', displayPath: '/spoofed/private/notes.txt' } },
}, {
role: 'user', messageId: 'user-notes', segmentIndex: 0, blockIndex: 0,
})).not.toMatchObject({ reference: { displayPath: expect.anything() } });
});
it('uses segment and block positions for distinct attachment ids around a tool call', () => {
let state = createEmptyAcpTimeline('agent:pi:s1', 1);
const resource = { type: 'resource_link' as const, uri: 'file:///tmp/report.txt', name: 'report.txt' };
state = applyAcpSessionUpdate(state, {
sessionId: 'agent:pi:s1',
update: { sessionUpdate: 'agent_message_chunk', messageId: 'msg-a', content: resource },
});
state = applyAcpSessionUpdate(state, {
sessionId: 'agent:pi:s1',
update: { sessionUpdate: 'tool_call', toolCallId: 'tool-1', title: 'Inspect', status: 'completed' },
});
state = applyAcpSessionUpdate(state, {
sessionId: 'agent:pi:s1',
update: { sessionUpdate: 'agent_message_chunk', messageId: 'msg-a', content: resource },
});
expect(state.itemsById['msg-a:0']).toMatchObject({
parts: [{ attachmentId: 'attachment:msg-a:0:0' }],
});
expect(state.itemsById['msg-a:1']).toMatchObject({
parts: [{ attachmentId: 'attachment:msg-a:1:0' }],
});
const secondAttachment = state.itemsById['msg-a:1']?.kind === 'message-segment'
? state.itemsById['msg-a:1'].parts[0]
: undefined;
if (!secondAttachment || secondAttachment.kind !== 'attachment') throw new Error('missing second attachment');
state = applyAttachmentResolution(state, {
attachmentId: 'attachment:msg-a:1:0',
expectedFingerprint: attachmentRequestFingerprint(secondAttachment),
result: {
ok: true,
identity: 'second-resource',
displayName: 'report.txt',
mimeType: 'text/plain',
size: 42,
target: {
kind: 'local',
scope: 'workspace',
ref: { sessionKey: 'agent:pi:s1', generation: 1, uri: 'file:///tmp/report.txt' },
},
},
});
expect(state.itemsById['msg-a:0']).toMatchObject({
parts: [{ access: { status: 'pending' } }],
});
expect(state.itemsById['msg-a:1']).toMatchObject({
parts: [{ access: { status: 'available', identity: 'second-resource' } }],
});
});
tests/unit/acp-session-access-registry.test.ts
+119
View File
@@ -0,0 +1,119 @@
import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, describe, expect, it } from 'vitest';
import { AcpSessionAccessRegistry } from '../../electron/services/acp-session-access-registry';
const temporaryDirectories: string[] = [];
function createDirectories() {
const parent = mkdtempSync(join(tmpdir(), 'clawx-acp-access-'));
temporaryDirectories.push(parent);
const workspaceRoot = join(parent, 'workspace');
const executionCwd = join(workspaceRoot, 'packages', 'app');
const outsideCwd = join(parent, 'outside');
mkdirSync(executionCwd, { recursive: true });
mkdirSync(outsideCwd);
return { parent, workspaceRoot, executionCwd, outsideCwd };
}
afterEach(() => {
for (const directory of temporaryDirectories.splice(0)) {
rmSync(directory, { recursive: true, force: true });
}
});
describe('AcpSessionAccessRegistry', () => {
it('canonicalizes workspace directories and rejects an execution cwd outside the workspace', async () => {
const registry = new AcpSessionAccessRegistry();
const { workspaceRoot, executionCwd, outsideCwd } = createDirectories();
await expect(registry.prepareGrant({
sessionKey: 'agent:main:session-1',
generation: 1,
workspaceRoot: join(workspaceRoot, 'packages', '..'),
executionCwd: join(executionCwd, '.'),
})).resolves.toEqual({
sessionKey: 'agent:main:session-1',
generation: 1,
workspaceRoot: realpathSync(workspaceRoot),
executionCwd: realpathSync(executionCwd),
});
await expect(registry.prepareGrant({
sessionKey: 'agent:main:session-1',
generation: 1,
workspaceRoot,
executionCwd: outsideCwd,
})).rejects.toThrow('ACP execution cwd must be inside the workspace root');
const filePath = join(workspaceRoot, 'file.txt');
writeFileSync(filePath, 'not a directory');
await expect(registry.prepareGrant({
sessionKey: 'agent:main:session-1',
generation: 1,
workspaceRoot: filePath,
executionCwd,
})).rejects.toThrow('ACP workspace root must be a directory');
});
it('commits access only for the exact session and generation', async () => {
const registry = new AcpSessionAccessRegistry();
const { workspaceRoot, executionCwd } = createDirectories();
const prepared = await registry.prepareGrant({
sessionKey: 'agent:main:session-1',
generation: 3,
workspaceRoot,
executionCwd,
});
expect(registry.get('agent:main:session-1', 3)).toBeNull();
registry.commitGrant(prepared);
expect(registry.get('agent:main:session-1', 3)).toEqual(prepared);
expect(registry.get('agent:main:session-1', 2)).toBeNull();
expect(registry.get('agent:main:other', 3)).toBeNull();
});
it('invalidates the older generation when a later grant is committed', async () => {
const registry = new AcpSessionAccessRegistry();
const { workspaceRoot, executionCwd } = createDirectories();
const first = await registry.prepareGrant({
sessionKey: 'agent:main:session-1', generation: 1, workspaceRoot, executionCwd,
});
const second = await registry.prepareGrant({
sessionKey: 'agent:main:session-1', generation: 2, workspaceRoot, executionCwd,
});
registry.commitGrant(first);
registry.commitGrant(second);
expect(registry.get('agent:main:session-1', 1)).toBeNull();
expect(registry.get('agent:main:session-1', 2)).toEqual(second);
});
it('restores a previous snapshot after a failed load', async () => {
const registry = new AcpSessionAccessRegistry();
const { workspaceRoot, executionCwd } = createDirectories();
const previous = await registry.prepareGrant({
sessionKey: 'agent:main:session-1', generation: 4, workspaceRoot, executionCwd,
});
const attempted = await registry.prepareGrant({
sessionKey: 'agent:main:session-2', generation: 5, workspaceRoot, executionCwd,
});
registry.commitGrant(previous);
const snapshot = registry.snapshot();
registry.commitGrant(attempted);
registry.restore(snapshot);
expect(registry.get('agent:main:session-1', 4)).toEqual(previous);
expect(registry.get('agent:main:session-2', 5)).toBeNull();
});
it('does not accept a replacement workspace root during lookup', () => {
const registry = new AcpSessionAccessRegistry();
expect(registry.get.length).toBe(2);
});
});
tests/unit/acp-timeline-groups.test.ts
+113
View File
@@ -159,4 +159,117 @@ describe('groupAcpTimelineItems', () => {
expect(groups).toHaveLength(1);
expect(groups[0]?.items.map((item) => item.id)).toEqual(['assistant-shared:0', 'tool:shared', 'assistant-other:0']);
});
it('lifts ordered attachments from assistant messages, thoughts, and tool output', () => {
const ref = { sessionKey: 'agent:main:session-1', generation: 1, uri: 'file:///workspace/file.txt' };
const attachment = (attachmentId: string, name: string) => ({
kind: 'attachment' as const,
attachmentId,
reference: { uri: ref.uri, name },
source: 'acp-resource' as const,
access: {
status: 'available' as const,
identity: `opaque-${attachmentId}`,
target: { kind: 'local' as const, scope: 'workspace' as const, ref },
mimeType: 'text/plain',
size: 12,
},
});
const groups = groupAcpTimelineItems(timelineWithItems({
'assistant-a:0': {
kind: 'message-segment', id: 'assistant-a:0', role: 'assistant', messageId: 'assistant-a', segmentIndex: 0,
parts: [{ kind: 'markdown', text: 'Message body' }, attachment('message-file', 'message.txt')],
},
'thought:assistant-a': {
kind: 'thought', id: 'thought:assistant-a', messageId: 'assistant-a',
parts: [attachment('thought-file', 'thought.txt'), { kind: 'markdown', text: 'Thought body' }],
},
'tool:read': {
kind: 'tool-call', id: 'tool:read', toolCallId: 'read', title: 'Read file', status: 'completed', historical: true,
outputParts: [{ kind: 'markdown', text: 'Tool body' }, attachment('tool-file', 'tool.txt')], locations: [],
},
}));
expect(groups).toHaveLength(1);
const group = groups[0];
expect(group?.kind).toBe('assistant-turn');
if (!group || group.kind !== 'assistant-turn') throw new Error('expected assistant group');
expect(group.attachments.map((part) => part.attachmentId)).toEqual(['message-file', 'thought-file', 'tool-file']);
expect(group.items[0]).toMatchObject({ parts: [{ kind: 'markdown', text: 'Message body' }] });
expect(group.items[1]).toMatchObject({ parts: [{ kind: 'markdown', text: 'Thought body' }] });
expect(group.items[2]).toMatchObject({ outputParts: [{ kind: 'markdown', text: 'Tool body' }] });
});
it('lifts user attachments across segments while retaining every prose segment first', () => {
const ref = { sessionKey: 'agent:main:session-1', generation: 1, uri: 'file:///workspace/file.txt' };
const attachment = (attachmentId: string, name: string) => ({
kind: 'attachment' as const,
attachmentId,
reference: { uri: ref.uri, name },
source: 'acp-resource' as const,
access: {
status: 'available' as const,
identity: `opaque-${attachmentId}`,
target: { kind: 'local' as const, scope: 'staging' as const, ref },
mimeType: 'text/plain',
size: 12,
},
});
const groups = groupAcpTimelineItems(timelineWithItems({
'user-a:0': {
kind: 'message-segment', id: 'user-a:0', role: 'user', messageId: 'user-a', segmentIndex: 0,
parts: [attachment('user-file-a', 'a.txt'), { kind: 'markdown', text: 'First prose' }],
},
'user-a:1': {
kind: 'message-segment', id: 'user-a:1', role: 'user', messageId: 'user-a', segmentIndex: 1,
parts: [{ kind: 'markdown', text: 'Second prose' }, attachment('user-file-b', 'b.txt')],
},
}));
expect(groups).toHaveLength(1);
const group = groups[0];
expect(group?.kind).toBe('user');
if (!group || group.kind !== 'user') throw new Error('expected user group');
expect(group.items.map((item) => item.parts)).toEqual([
[{ kind: 'markdown', text: 'First prose' }],
[{ kind: 'markdown', text: 'Second prose' }],
]);
expect(group.attachments.map((part) => part.attachmentId)).toEqual(['user-file-a', 'user-file-b']);
});
it('keeps a marked compatibility attachment visible when the turn has no ordinary assistant message', () => {
const ref = { sessionKey: 'agent:main:session-1', generation: 1, uri: '/workspace/report.pdf' };
const groups = groupAcpTimelineItems(timelineWithItems({
'user-report:0': {
kind: 'message-segment', id: 'user-report:0', role: 'user', messageId: 'user-report', segmentIndex: 0,
parts: [{ kind: 'markdown', text: 'Create report' }],
},
'compat:openclaw-media:evidence:0': {
kind: 'message-segment',
id: 'compat:openclaw-media:evidence:0',
role: 'assistant',
messageId: 'compat:openclaw-media:evidence',
segmentIndex: 0,
compat: { source: 'openclaw-media', evidenceId: 'evidence' },
parts: [{
kind: 'attachment',
attachmentId: 'report',
reference: { uri: ref.uri, name: 'report.pdf' },
source: 'openclaw-media',
evidenceId: 'evidence',
access: {
status: 'available', identity: 'opaque-report', mimeType: 'application/pdf', size: 12,
target: { kind: 'local', scope: 'workspace', ref },
},
}],
},
}));
expect(groups).toHaveLength(2);
expect(groups[1]).toMatchObject({
kind: 'assistant-turn',
items: [],
attachments: [{ attachmentId: 'report', source: 'openclaw-media' }],
});
});
});
tests/unit/acp-trace.test.ts
+53
View File
@@ -4,6 +4,7 @@ import {
getAcpTraceSnapshot,
normalizeRendererAcpTracePayload,
recordAcpTrace,
recordAttachmentOpenTrace,
} from '../../electron/services/acp-trace';
describe('ACP trace diagnostics store', () => {
@@ -55,4 +56,56 @@ describe('ACP trace diagnostics store', () => {
expect(normalizeRendererAcpTracePayload({ event: '' })).toBeNull();
expect(normalizeRendererAcpTracePayload(null)).toBeNull();
});
it('allowlists OpenClaw transcript projection details without transcript bodies or raw references', () => {
const normalized = normalizeRendererAcpTracePayload({
event: 'openclaw-media:resolution-available',
sessionKey: 'agent:pi:s1',
generation: 2,
details: {
source: 'openclaw-media',
reason: 'available',
candidateCount: 1,
evidenceHash: 'evidence-hash',
identityHash: 'identity-hash',
transcriptBody: 'MEDIA:/private/secret.txt',
uri: 'file:///private/secret.txt',
path: '/private/secret.txt',
},
});
expect(normalized?.details).toEqual({
source: 'openclaw-media',
reason: 'available',
candidateCount: 1,
evidenceHash: 'evidence-hash',
identityHash: 'identity-hash',
});
expect(JSON.stringify(normalized)).not.toContain('/private/secret.txt');
});
it('records attachment opens with an allowlisted redacted detail shape', () => {
recordAttachmentOpenTrace({
ok: false,
reason: 'operationFailed',
sourceKind: 'local',
sessionKey: 'agent:pi:s1',
generation: 2,
identity: 'a'.repeat(64),
uri: 'file:///private/secret.txt',
path: '/private/secret.txt',
} as never);
const entry = getAcpTraceSnapshot().entries[0];
expect(entry).toMatchObject({
event: 'attachment/open:failure',
details: {
reason: 'operationFailed',
sourceKind: 'local',
identity: 'a'.repeat(64),
},
});
expect(Object.keys(entry.details as object).sort()).toEqual(['identity', 'reason', 'sourceKind']);
expect(JSON.stringify(entry)).not.toContain('/private/secret.txt');
});
});
tests/unit/artifact-panel.test.tsx
+25
View File
@@ -143,6 +143,31 @@ describe('ArtifactPanel', () => {
expect(shellShowItemInFolder).not.toHaveBeenCalled();
});
it('renders attachment previews without trusted rich-file folder actions', () => {
useArtifactPanel.setState({
open: true,
tab: 'preview',
focusedFile: {
filePath: 'report.pdf',
fileName: 'report.pdf',
ext: '.pdf',
mimeType: 'application/pdf',
contentType: 'document',
attachmentFileRef: {
sessionKey: 'agent:main:s1',
generation: 2,
uri: 'file:///secret/report.pdf',
},
},
});
render(<ArtifactPanel fileGroups={groups()} uniqueFileCount={2} agent={null} />);
expect(screen.getByTestId('file-preview-body')).toHaveTextContent('preview:report.pdf');
expect(screen.queryByTestId('artifact-panel-action-open-folder')).not.toBeInTheDocument();
expect(shellShowItemInFolder).not.toHaveBeenCalled();
});
it('retains the rich open-folder action for trusted files', () => {
useArtifactPanel.setState({
open: true,
tests/unit/attachment-access.test.ts
+500
View File
@@ -0,0 +1,500 @@
// @vitest-environment node
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { lstat, mkdir, mkdtemp, open, realpath, rename, rm, stat, symlink, writeFile } from 'node:fs/promises';
import { pathToFileURL } from 'node:url';
import { join, resolve } from 'node:path';
import { tmpdir } from 'node:os';
import { AcpSessionAccessRegistry } from '../../electron/services/acp-session-access-registry';
import {
StagedAttachmentRegistry,
createAttachmentAccess,
} from '../../electron/services/attachment-access';
import {
clearAcpTraceForTests,
getAcpTraceSnapshot,
} from '../../electron/services/acp-trace';
import {
resolveOpenClawConfigDir,
resolveOpenClawConfigPath,
resolveOpenClawStateDir,
} from '../../electron/utils/paths';
describe('attachment access boundary', () => {
const sessionKey = 'agent:main:session-a';
let testDir: string;
let workspaceRoot: string;
let stateDir: string;
let configDir: string;
let externalMediaRoot: string;
let outsideDir: string;
let registry: AcpSessionAccessRegistry;
let stagedAttachments: StagedAttachmentRegistry;
let openPath: ReturnType<typeof vi.fn>;
let openExternal: ReturnType<typeof vi.fn>;
beforeEach(async () => {
clearAcpTraceForTests();
testDir = await mkdtemp(join(tmpdir(), 'clawx-attachment-access-'));
workspaceRoot = join(testDir, 'workspace');
stateDir = join(testDir, 'state');
configDir = join(testDir, 'config');
externalMediaRoot = join(testDir, 'runtime-media');
outsideDir = join(testDir, 'outside');
await Promise.all([
mkdir(join(workspaceRoot, 'nested'), { recursive: true }),
mkdir(join(stateDir, 'media', 'outgoing', 'records'), { recursive: true }),
mkdir(join(stateDir, 'agents', 'main', 'sessions'), { recursive: true }),
mkdir(join(configDir, 'media'), { recursive: true }),
mkdir(externalMediaRoot, { recursive: true }),
mkdir(outsideDir, { recursive: true }),
]);
await Promise.all([
writeFile(join(workspaceRoot, 'notes.txt'), 'workspace notes'),
writeFile(join(workspaceRoot, 'binary.bin'), Buffer.from([1, 2, 3])),
writeFile(join(stateDir, 'media', 'state.png'), 'state image'),
writeFile(join(configDir, 'media', 'config.png'), 'config image'),
writeFile(join(externalMediaRoot, 'runtime.png'), 'runtime image'),
writeFile(join(stateDir, 'openclaw.json'), '{}'),
writeFile(join(configDir, 'openclaw.json'), '{}'),
writeFile(join(stateDir, 'media', 'outgoing', 'records', 'leak.json'), JSON.stringify({
original: { path: '/private/account/secret.png' },
})),
writeFile(join(stateDir, 'agents', 'main', 'sessions', 'secret.jsonl'), 'secret'),
writeFile(join(outsideDir, 'secret.txt'), 'outside secret'),
]);
registry = new AcpSessionAccessRegistry();
registry.commitGrant(await registry.prepareGrant({
sessionKey,
generation: 1,
workspaceRoot,
executionCwd: join(workspaceRoot, 'nested'),
}));
stagedAttachments = new StagedAttachmentRegistry();
openPath = vi.fn().mockResolvedValue('');
openExternal = vi.fn().mockResolvedValue(undefined);
});
afterEach(async () => {
await rm(testDir, { recursive: true, force: true });
});
function getAccess() {
return createAttachmentAccess({
sessionAccessRegistry: registry,
stagedAttachments,
stateDir,
configDir,
shell: { openPath, openExternal },
});
}
function ref(uri: string, extra: Record<string, unknown> = {}) {
return { sessionKey, generation: 1, uri, ...extra };
}
it('resolves workspace files only through the exact registered session and generation', async () => {
const access = getAccess();
const workspaceFile = join(workspaceRoot, 'notes.txt');
await expect(access.resolveAttachment({ ref: ref(workspaceFile) })).resolves.toMatchObject({
ok: true,
displayName: 'notes.txt',
target: { kind: 'local', scope: 'workspace', ref: ref(workspaceFile) },
});
await expect(access.resolveAttachment({ ref: { ...ref(workspaceFile), sessionKey: 'agent:main:other' } }))
.resolves.toMatchObject({ ok: false, error: 'staleSession' });
await expect(access.resolveAttachment({ ref: { ...ref(workspaceFile), generation: 2 } }))
.resolves.toMatchObject({ ok: false, error: 'staleSession' });
await expect(access.resolveAttachment({
ref: ref(join(outsideDir, 'secret.txt')),
workspaceRoot: outsideDir,
} as never)).resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
});
it('allows only exact declared OpenClaw media roots', async () => {
const access = getAccess();
await expect(access.resolveAttachment({ ref: ref(join(stateDir, 'media', 'state.png')) }))
.resolves.toMatchObject({ ok: true, target: { kind: 'local', scope: 'openclaw-media' } });
await expect(access.resolveAttachment({ ref: ref(join(configDir, 'media', 'config.png')) }))
.resolves.toMatchObject({ ok: true, target: { kind: 'local', scope: 'openclaw-media' } });
for (const denied of [
join(stateDir, 'openclaw.json'),
join(configDir, 'openclaw.json'),
join(stateDir, 'agents', 'main', 'sessions', 'secret.jsonl'),
join(stateDir, 'media', 'outgoing', 'records', 'leak.json'),
join(externalMediaRoot, 'runtime.png'),
]) {
await expect(access.resolveAttachment({ ref: ref(denied) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
}
});
it('freezes the registered canonical workspace root across rename and symlink replacement', async () => {
const originalWorkspace = join(testDir, 'workspace-original');
await rename(workspaceRoot, originalWorkspace);
await symlink(outsideDir, workspaceRoot);
await expect(getAccess().resolveAttachment({ ref: ref(join(workspaceRoot, 'secret.txt')) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
});
it.each(['state-external', 'state-sibling', 'config-external', 'config-sibling'])(
'rejects a symlinked managed media root targeting %s',
async (scenario) => {
const isState = scenario.startsWith('state');
const parent = isState ? stateDir : configDir;
const mediaPath = join(parent, 'media');
const sibling = join(parent, 'sensitive');
const target = scenario.endsWith('sibling') ? sibling : externalMediaRoot;
await rm(mediaPath, { recursive: true, force: true });
if (scenario.endsWith('sibling')) {
await mkdir(sibling);
await writeFile(join(sibling, 'secret.txt'), 'sibling secret');
}
await symlink(target, mediaPath);
const targetFile = join(target, scenario.endsWith('sibling') ? 'secret.txt' : 'runtime.png');
await expect(getAccess().resolveAttachment({ ref: ref(targetFile) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
},
);
it.each(['state', 'config'])('pins the %s parent across rename and symlink replacement', async (kind) => {
const parent = kind === 'state' ? stateDir : configDir;
const originalParent = `${parent}-original`;
const replacementParent = join(testDir, `${kind}-replacement`);
const replacementMedia = join(replacementParent, 'media');
const attachmentId = `${kind}-parent-swap`;
const gatewayUrl = `/api/chat/media/outgoing/${encodeURIComponent(sessionKey)}/${attachmentId}/full`;
const initialMediaPath = join(parent, 'media', `${kind}.png`);
const replacementMediaPath = join(replacementMedia, `${kind}.png`);
await writeFile(initialMediaPath, `${kind} initial`);
const recordPath = join(stateDir, 'media', 'outgoing', 'records', `${attachmentId}.json`);
await writeFile(recordPath, JSON.stringify({
attachmentId,
sessionKey,
original: { path: initialMediaPath, contentType: 'image/png' },
}));
const access = getAccess();
await expect(access.resolveAttachment({ ref: ref(initialMediaPath) }))
.resolves.toMatchObject({ ok: true });
await expect(access.resolveAttachment({ ref: ref(gatewayUrl) }))
.resolves.toMatchObject({ ok: true });
await mkdir(replacementMedia, { recursive: true });
await writeFile(replacementMediaPath, `${kind} replacement`);
if (kind === 'state') {
await mkdir(join(replacementMedia, 'outgoing', 'records'), { recursive: true });
await writeFile(
join(replacementMedia, 'outgoing', 'records', `${attachmentId}.json`),
JSON.stringify({
attachmentId,
sessionKey,
original: { path: replacementMediaPath, contentType: 'image/png' },
}),
);
}
await rename(parent, originalParent);
await symlink(replacementParent, parent);
await expect(access.resolveAttachment({ ref: ref(join(parent, 'media', `${kind}.png`)) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
await expect(access.resolveAttachment({ ref: ref(gatewayUrl) }))
.resolves.toMatchObject({ ok: false });
});
it('requires a Main-owned staging id and matching staged path', async () => {
const stagingDir = join(stateDir, 'media', 'outbound', 'clawx-staging');
const stagedPath = join(stagingDir, 'owned.txt');
const previousRunPath = join(stagingDir, 'previous-run.txt');
await mkdir(stagingDir, { recursive: true });
await writeFile(stagedPath, 'owned');
await writeFile(previousRunPath, 'unregistered');
stagedAttachments.register('stage-1', await realpath(stagedPath), '/Users/test/Documents/owned.txt');
const access = getAccess();
await expect(access.resolveAttachment({ ref: ref(stagedPath) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
await expect(access.resolveAttachment({ ref: ref(previousRunPath) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
await expect(access.resolveAttachment({ ref: ref(stagedPath, { stagingId: 'stage-1' }) }))
.resolves.toMatchObject({
ok: true,
displayPath: '/Users/test/Documents/owned.txt',
target: { kind: 'local', scope: 'staging' },
});
await expect(access.resolveAttachment({
ref: ref(join(outsideDir, 'secret.txt'), { stagingId: 'stage-1' }),
})).resolves.toMatchObject({ ok: false, error: 'invalidReference' });
});
it('binds outgoing records to attachment, URL session, record session, and message ids', async () => {
const attachmentId = 'generated-1';
const originalPath = join(stateDir, 'media', 'state.png');
const recordPath = join(stateDir, 'media', 'outgoing', 'records', `${attachmentId}.json`);
const gatewayUrl = `/api/chat/media/outgoing/${encodeURIComponent(sessionKey)}/${attachmentId}/full`;
const writeRecord = (record: Record<string, unknown>) => writeFile(recordPath, JSON.stringify(record));
const access = getAccess();
await writeRecord({
attachmentId,
sessionKey,
messageId: 'message-1',
original: { path: originalPath, contentType: 'image/png' },
});
await expect(access.resolveAttachment({
ref: ref(gatewayUrl, { transcriptMessageId: 'message-1' }),
})).resolves.toMatchObject({ ok: true, mimeType: 'image/png', target: { scope: 'openclaw-media' } });
const rejected = [
`/api/chat/media/outgoing/${encodeURIComponent('agent:main:other')}/${attachmentId}/full`,
`/api/chat/media/outgoing/${encodeURIComponent(sessionKey)}/different-id/full`,
];
for (const uri of rejected) {
await expect(access.resolveAttachment({ ref: ref(uri, { transcriptMessageId: 'message-1' }) }))
.resolves.toMatchObject({ ok: false });
}
await expect(access.resolveAttachment({
ref: ref(gatewayUrl, { transcriptMessageId: 'message-2' }),
})).resolves.toMatchObject({ ok: false, error: 'invalidReference' });
await writeRecord({
attachmentId,
sessionKey: 'agent:main:other',
original: { path: originalPath, contentType: 'image/png' },
});
await expect(access.resolveAttachment({ ref: ref(gatewayUrl) }))
.resolves.toMatchObject({ ok: false, error: 'invalidReference' });
});
it('treats global as a literal outgoing session rather than a wildcard', async () => {
const attachmentId = 'global-image';
const recordPath = join(stateDir, 'media', 'outgoing', 'records', `${attachmentId}.json`);
await writeFile(recordPath, JSON.stringify({
attachmentId,
sessionKey: 'global',
original: { path: join(stateDir, 'media', 'state.png') },
}));
const access = getAccess();
const globalUrl = `/api/chat/media/outgoing/global/${attachmentId}/full`;
await expect(access.resolveAttachment({ ref: ref(globalUrl) }))
.resolves.toMatchObject({ ok: false, error: 'invalidReference' });
registry.commitGrant(await registry.prepareGrant({
sessionKey: 'global',
generation: 2,
workspaceRoot,
executionCwd: workspaceRoot,
}));
await expect(access.resolveAttachment({
ref: { sessionKey: 'global', generation: 2, uri: globalUrl },
})).resolves.toMatchObject({ ok: true });
});
it.each([
['missing file', () => join(workspaceRoot, 'missing.txt'), 'unavailable'],
['directory', () => workspaceRoot, 'notFile'],
['traversal', () => `${workspaceRoot}/nested/../notes.txt`, 'invalidReference'],
['encoded traversal', () => `file://${workspaceRoot}/nested/%2e%2e/notes.txt`, 'invalidReference'],
['encoded NUL', () => `file://${workspaceRoot}/notes.txt%00`, 'invalidReference'],
['remote file authority', () => 'file://evil.example/etc/passwd', 'invalidReference'],
['UNC path', () => '\\\\server\\share\\file.txt', 'invalidReference'],
['URL credentials', () => 'https://user:pass@example.com/file.txt', 'unsafeUrl'],
['unknown scheme', () => 'ftp://example.com/file.txt', 'invalidReference'],
['overlong reference', () => `https://example.com/${'a'.repeat(4096)}`, 'invalidReference'],
])('fails closed for %s', async (_label, makeUri, error) => {
await expect(getAccess().resolveAttachment({ ref: ref(makeUri()) }))
.resolves.toMatchObject({ ok: false, error });
});
it('rejects symlink escapes', async () => {
await symlink(join(outsideDir, 'secret.txt'), join(workspaceRoot, 'escape.txt'));
await expect(getAccess().resolveAttachment({ ref: ref(join(workspaceRoot, 'escape.txt')) }))
.resolves.toMatchObject({ ok: false, error: 'outsideAllowedRoots' });
});
it('normalizes safe remote URLs without granting unsafe variants', async () => {
const access = getAccess();
const result = await access.resolveAttachment({
ref: ref('HTTPS://Example.COM:443/assets/report%20final.pdf?x=1'),
});
expect(result).toMatchObject({
ok: true,
displayName: 'report final.pdf',
target: {
kind: 'remote',
url: 'https://example.com/assets/report%20final.pdf?x=1',
},
});
expect(result.ok && result.identity).toMatch(/^[a-f0-9]{64}$/);
expect(JSON.stringify(result)).not.toContain(workspaceRoot);
});
it('re-resolves text, binary, and open operations after prior success', async () => {
const access = getAccess();
const textRef = ref(join(workspaceRoot, 'notes.txt'));
const binaryRef = ref(join(workspaceRoot, 'binary.bin'));
await expect(access.resolveAttachment({ ref: textRef })).resolves.toMatchObject({ ok: true });
await expect(access.resolveAttachment({ ref: binaryRef })).resolves.toMatchObject({ ok: true });
registry.commitGrant(await registry.prepareGrant({
sessionKey,
generation: 2,
workspaceRoot,
executionCwd: workspaceRoot,
}));
await expect(access.readAttachmentText(textRef)).resolves.toEqual({ ok: false, error: 'staleSession' });
await expect(access.readAttachmentBinary({ ref: binaryRef })).resolves.toEqual({ ok: false, error: 'staleSession' });
await expect(access.openAttachment(textRef)).resolves.toEqual({ ok: false, error: 'staleSession' });
expect(openPath).not.toHaveBeenCalled();
});
it('reads bounded text and binary data through attachment refs', async () => {
const access = getAccess();
await expect(access.readAttachmentText(ref(join(workspaceRoot, 'notes.txt')))).resolves.toMatchObject({
ok: true,
content: 'workspace notes',
size: 15,
});
await expect(access.readAttachmentBinary({ ref: ref(join(workspaceRoot, 'binary.bin')), maxBytes: 3 }))
.resolves.toMatchObject({ ok: true, data: new Uint8Array([1, 2, 3]), size: 3 });
await expect(access.readAttachmentBinary({ ref: ref(join(workspaceRoot, 'binary.bin')), maxBytes: 2 }))
.resolves.toMatchObject({ ok: false, error: 'tooLarge', size: 3 });
});
it('delegates validated local and remote opens to the correct shell operation', async () => {
const access = getAccess();
const localPath = join(workspaceRoot, 'notes.txt');
const remoteUrl = 'https://example.com/download/report.pdf';
await expect(access.openAttachment(ref(localPath))).resolves.toEqual({ ok: true });
await expect(access.openAttachment(ref(remoteUrl))).resolves.toEqual({ ok: true });
expect(openPath).toHaveBeenCalledWith(await realpath(localPath));
expect(openExternal).toHaveBeenCalledWith(remoteUrl);
const trace = getAcpTraceSnapshot().entries;
expect(trace).toHaveLength(2);
expect(trace.every((entry) => entry.event === 'attachment/open:success')).toBe(true);
expect(trace.every((entry) => (
Object.keys(entry.details as object).sort().join(',') === 'identity,reason,sourceKind'
))).toBe(true);
expect(JSON.stringify(trace)).not.toContain(localPath);
expect(JSON.stringify(trace)).not.toContain(remoteUrl);
});
it('rechecks generation after final local validation before shell.openPath', async () => {
const localPath = await realpath(join(workspaceRoot, 'notes.txt'));
let targetStatCount = 0;
let releaseFinalValidation!: () => void;
let signalFinalValidation!: () => void;
const finalValidationReached = new Promise<void>((resolveSignal) => {
signalFinalValidation = resolveSignal;
});
const finalValidationRelease = new Promise<void>((resolveRelease) => {
releaseFinalValidation = resolveRelease;
});
const access = createAttachmentAccess({
sessionAccessRegistry: registry,
stagedAttachments,
stateDir,
configDir,
shell: { openPath, openExternal },
fs: {
lstat,
open,
realpath,
stat: async (path) => {
const result = await stat(path);
if (path === localPath && ++targetStatCount === 2) {
signalFinalValidation();
await finalValidationRelease;
}
return result;
},
},
});
const opening = access.openAttachment(ref(localPath));
await finalValidationReached;
registry.commitGrant(await registry.prepareGrant({
sessionKey,
generation: 2,
workspaceRoot,
executionCwd: workspaceRoot,
}));
releaseFinalValidation();
await expect(opening).resolves.toEqual({ ok: false, error: 'staleSession' });
expect(openPath).not.toHaveBeenCalled();
});
it('records only bounded, redacted attachment open trace metadata', async () => {
openPath.mockResolvedValueOnce('application failed');
const localPath = join(workspaceRoot, 'notes.txt');
await expect(getAccess().openAttachment(ref(localPath))).resolves.toEqual({
ok: false,
error: 'operationFailed',
});
const entry = getAcpTraceSnapshot().entries.at(-1);
expect(entry).toMatchObject({
source: 'main',
event: 'attachment/open:failure',
sessionKey,
generation: 1,
details: {
reason: 'operationFailed',
sourceKind: 'local',
identity: expect.stringMatching(/^[a-f0-9]{64}$/),
},
});
expect(Object.keys(entry?.details as object).sort()).toEqual(['identity', 'reason', 'sourceKind']);
expect(JSON.stringify(entry)).not.toContain(localPath);
expect(JSON.stringify(entry)).not.toContain(pathToFileURL(localPath).href);
});
it('sanitizes hostile display labels to one bounded line', async () => {
const hostile = `/private/account/secrets/report\n\u202Egpj.exe\u0007 ${'x'.repeat(300)} `;
const result = await getAccess().resolveAttachment({
ref: ref(join(workspaceRoot, 'notes.txt')),
name: hostile,
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(Array.from(result.displayName).some((character) => {
const codePoint = character.codePointAt(0) ?? 0;
return codePoint <= 0x1f
|| (codePoint >= 0x7f && codePoint <= 0x9f)
|| (codePoint >= 0x202a && codePoint <= 0x202e);
})).toBe(false);
expect(result.displayName).not.toContain('\n');
expect(result.displayName).not.toContain('/private/account/secrets');
expect(result.displayName.length).toBeLessThanOrEqual(160);
});
});
describe('OpenClaw attachment path resolution', () => {
it('keeps state and config paths distinct and absolute', () => {
const stateDir = resolveOpenClawStateDir({ OPENCLAW_STATE_DIR: '~/custom-state' });
const configPath = resolveOpenClawConfigPath({
OPENCLAW_STATE_DIR: '~/custom-state',
OPENCLAW_CONFIG_PATH: './runtime/openclaw.json',
});
expect(stateDir).toBe(resolve(process.env.HOME!, 'custom-state'));
expect(configPath).toBe(resolve('runtime/openclaw.json'));
expect(resolveOpenClawConfigDir({ OPENCLAW_CONFIG_PATH: configPath })).toBe(resolve('runtime'));
expect(resolveOpenClawConfigPath({ OPENCLAW_STATE_DIR: stateDir })).toBe(join(stateDir, 'openclaw.json'));
});
});
tests/unit/chat-acp-page.test.tsx
+110 -18
View File
@@ -45,6 +45,7 @@ const { acpState, agentsState, artifactPanelState, artifactPanelProps, chatState
cancelling: false,
error: null as string | null,
activeSessionKey: 'agent:main:main' as string | null,
workspaceRoot: null as string | null,
cwd: null as string | null,
acceptedPromptSessionKeys: [] as string[],
prepareLocalSession: vi.fn(),
@@ -182,12 +183,14 @@ vi.mock('@/pages/Chat/ChatInput', () => ({
onClick={() => onSend('Ship it', [
{
status: 'ready',
id: 'staged-ready',
stagedPath: '/tmp/ready.png',
fileName: 'ready.png',
mimeType: 'image/png',
},
{
status: 'staging',
id: 'staged-pending',
stagedPath: '/tmp/staging.txt',
fileName: 'staging.txt',
mimeType: 'text/plain',
@@ -298,18 +301,21 @@ describe('ACP Chat page', () => {
acpState.cancelling = false;
acpState.error = null;
acpState.activeSessionKey = 'agent:main:main';
acpState.workspaceRoot = null;
acpState.cwd = null;
acpState.acceptedPromptSessionKeys = [];
acpState.timeline = populatedTimeline();
acpState.prepareLocalSession.mockReset();
acpState.prepareLocalSession.mockImplementation((input: { sessionKey: string; cwd: string }) => {
acpState.prepareLocalSession.mockImplementation((input: { sessionKey: string; workspaceRoot: string; cwd: string }) => {
acpState.activeSessionKey = input.sessionKey;
acpState.workspaceRoot = input.workspaceRoot;
acpState.cwd = input.cwd;
acpState.timeline = { ...emptyTimeline(), sessionId: input.sessionKey };
});
acpState.loadSession.mockReset();
acpState.loadSession.mockImplementation(async (input: { sessionKey: string; cwd: string }) => {
acpState.loadSession.mockImplementation(async (input: { sessionKey: string; workspaceRoot: string; cwd: string }) => {
acpState.activeSessionKey = input.sessionKey;
acpState.workspaceRoot = input.workspaceRoot;
acpState.cwd = input.cwd;
return true;
});
@@ -366,11 +372,14 @@ describe('ACP Chat page', () => {
await waitFor(() => {
expect(ensureAcpChatSubscriptions).toHaveBeenCalledTimes(1);
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:main:main', cwd: '/workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:main:main', workspaceRoot: '/workspace', cwd: '/workspace',
});
});
});
it('sends ready staged attachments and cancels through the ACP session store', () => {
acpState.workspaceRoot = '/workspace';
acpState.cwd = '/workspace';
render(<Chat />);
@@ -380,7 +389,9 @@ describe('ACP Chat page', () => {
sessionKey: 'agent:main:main',
cwd: '/workspace',
message: 'Ship it',
media: [{ filePath: '/tmp/ready.png', fileName: 'ready.png', mimeType: 'image/png' }],
media: [{
filePath: '/tmp/ready.png', stagingId: 'staged-ready', fileName: 'ready.png', mimeType: 'image/png',
}],
});
fireEvent.click(screen.getByTestId('mock-stop'));
@@ -398,7 +409,9 @@ describe('ACP Chat page', () => {
const { rerender } = render(<Chat />);
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:main:main', cwd: '/session-workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:main:main', workspaceRoot: '/session-workspace', cwd: '/session-workspace',
});
});
agentsState.agents = [{ id: 'main', name: 'Main', workspace: '/resolved-workspace', mainSessionKey: 'agent:main:main' }];
@@ -419,7 +432,9 @@ describe('ACP Chat page', () => {
expect(chatState.loadSessions).toHaveBeenCalledTimes(1);
});
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:main:main', cwd: '/workspace', createIfMissing: true });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:main:main', workspaceRoot: '/workspace', cwd: '/workspace', createIfMissing: true,
});
});
});
@@ -434,6 +449,7 @@ describe('ACP Chat page', () => {
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:main:session-a',
workspaceRoot: '/Users/alex/workspace/ClawX',
cwd: '/Users/alex/workspace/ClawX',
});
});
@@ -459,18 +475,62 @@ describe('ACP Chat page', () => {
const { rerender } = render(<Chat />);
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey, cwd: '~/.openclaw/workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey, workspaceRoot: '~/.openclaw/workspace', cwd: '~/.openclaw/workspace',
});
});
chatState.sessions = [{ key: sessionKey, updatedAt: 1000, workspacePath: '/Users/alex/workspace/ClawX' }];
rerender(<Chat />);
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey, cwd: '/Users/alex/workspace/ClawX' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey,
workspaceRoot: '/Users/alex/workspace/ClawX',
cwd: '/Users/alex/workspace/ClawX',
});
});
resolveInitialLoad(true);
});
it('starts a new load when returning to a session whose earlier load is still pending', async () => {
const sessionKey = 'agent:main:session-a';
let resolveInitialLoad!: (loaded: boolean) => void;
const initialLoad = new Promise<boolean>((resolve) => {
resolveInitialLoad = resolve;
});
const localSessionKey = 'agent:main:session-local';
chatState.sessions = [{ key: sessionKey, workspacePath: '/workspace' }];
chatState.currentSessionKey = sessionKey;
acpState.activeSessionKey = null;
acpState.loadSession
.mockReturnValueOnce(initialLoad)
.mockResolvedValueOnce(true);
const { rerender } = render(<Chat />);
await waitFor(() => expect(acpState.loadSession).toHaveBeenCalledTimes(1));
chatState.sessions = [
{ key: sessionKey, workspacePath: '/workspace' },
{ key: localSessionKey, workspacePath: '/workspace', createdLocally: true },
];
chatState.currentSessionKey = localSessionKey;
acpState.activeSessionKey = localSessionKey;
acpState.cwd = '/workspace';
rerender(<Chat />);
chatState.currentSessionKey = sessionKey;
rerender(<Chat />);
await waitFor(() => expect(acpState.loadSession).toHaveBeenCalledTimes(2));
expect(acpState.loadSession).toHaveBeenLastCalledWith({
sessionKey,
workspaceRoot: '/workspace',
cwd: '/workspace',
});
resolveInitialLoad(false);
});
it('does not create a local ACP session before first send', async () => {
chatState.sessions = [{ key: 'agent:main:session-local', createdLocally: true }];
chatState.currentSessionKey = 'agent:main:session-local';
@@ -494,7 +554,9 @@ describe('ACP Chat page', () => {
const { rerender } = render(<Chat />);
await waitFor(() => {
expect(acpState.prepareLocalSession).toHaveBeenCalledWith({ sessionKey, cwd: '/workspace' });
expect(acpState.prepareLocalSession).toHaveBeenCalledWith({
sessionKey, workspaceRoot: '/workspace', cwd: '/workspace',
});
});
rerender(<Chat />);
@@ -517,7 +579,9 @@ describe('ACP Chat page', () => {
fireEvent.click(screen.getByTestId('mock-send'));
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey, cwd: '/workspace', createIfMissing: true });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey, workspaceRoot: '/workspace', cwd: '/workspace', createIfMissing: true,
});
});
await waitFor(() => {
expect(chatState.acknowledgeAcpSessionCreated).toHaveBeenCalledWith(sessionKey, '/workspace');
@@ -535,7 +599,9 @@ describe('ACP Chat page', () => {
fireEvent.click(screen.getByTestId('mock-send'));
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey, cwd: '/workspace', createIfMissing: true });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey, workspaceRoot: '/workspace', cwd: '/workspace', createIfMissing: true,
});
});
await waitFor(() => {
expect(chatState.acknowledgeAcpSessionCreated).toHaveBeenCalledWith(sessionKey, '/workspace');
@@ -586,7 +652,9 @@ describe('ACP Chat page', () => {
expect(screen.getByTestId('mock-chat-input')).toHaveAttribute('data-disabled', 'false');
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:main:main', cwd: '/workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:main:main', workspaceRoot: '/workspace', cwd: '/workspace',
});
});
});
@@ -603,7 +671,9 @@ describe('ACP Chat page', () => {
await waitFor(() => {
expect(acpState.acceptedPromptSessionKeys).toContain('agent:research:desk');
});
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:research:desk', cwd: '/research-workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:research:desk', workspaceRoot: '/research-workspace', cwd: '/research-workspace',
});
expect(chatState.selectAcpSession).toHaveBeenCalledWith('agent:research:desk');
expect(acpState.sendPrompt).toHaveBeenCalledWith({
sessionKey: 'agent:research:desk',
@@ -639,7 +709,9 @@ describe('ACP Chat page', () => {
fireEvent.click(screen.getByTestId('mock-send-target'));
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey, cwd: '/research-workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey, workspaceRoot: '/research-workspace', cwd: '/research-workspace',
});
});
expect(acpState.sendPrompt).toHaveBeenCalledWith({
sessionKey,
@@ -668,7 +740,9 @@ describe('ACP Chat page', () => {
fireEvent.click(screen.getByTestId('mock-send-target'));
await waitFor(() => {
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:research:desk', cwd: '/research-workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:research:desk', workspaceRoot: '/research-workspace', cwd: '/research-workspace',
});
});
expect(chatState.selectAcpSession).toHaveBeenCalledWith('agent:research:desk');
expect(acpState.sendPrompt).not.toHaveBeenCalled();
@@ -703,7 +777,13 @@ describe('ACP Chat page', () => {
role: 'user',
messageId: 'msg-user',
segmentIndex: 0,
parts: [{ kind: 'file', path: '/workspace/user-upload.md', name: 'user-upload.md', mimeType: 'text/markdown' }],
parts: [{
kind: 'attachment',
attachmentId: 'attachment:msg-user:0:0',
reference: { uri: '/workspace/user-upload.md', name: 'user-upload.md', mimeType: 'text/markdown' },
source: 'acp-resource',
access: { status: 'pending' },
}],
},
'msg-assistant:0': {
kind: 'message-segment',
@@ -711,7 +791,13 @@ describe('ACP Chat page', () => {
role: 'assistant',
messageId: 'msg-assistant',
segmentIndex: 0,
parts: [{ kind: 'file', path: '/workspace/report.md', name: 'report.md', mimeType: 'text/markdown' }],
parts: [{
kind: 'attachment',
attachmentId: 'attachment:msg-assistant:0:0',
reference: { uri: '/workspace/report.md', name: 'report.md', mimeType: 'text/markdown' },
source: 'acp-resource',
access: { status: 'pending' },
}],
},
'tool:write-file': {
kind: 'tool-call',
@@ -720,7 +806,13 @@ describe('ACP Chat page', () => {
title: 'write: app',
status: 'completed',
input: { path: '/Users/test/.openclaw/workspace/src/app.tsx', content: 'export {}' },
outputParts: [{ kind: 'file', path: '/workspace/src/app.tsx', name: 'app.tsx' }],
outputParts: [{
kind: 'attachment',
attachmentId: 'attachment:tool:write-file:0:0',
reference: { uri: '/workspace/src/app.tsx', name: 'app.tsx' },
source: 'acp-resource',
access: { status: 'pending' },
}],
locations: [],
},
},
tests/unit/chat-input.test.tsx
+10 -3
View File
@@ -109,6 +109,8 @@ function translate(key: string, vars?: Record<string, unknown>): string {
return 'Send';
case 'composer.stop':
return 'Stop';
case 'composer.thinking':
return 'Thinking…';
case 'composer.gatewayConnected':
return 'connected';
case 'composer.gatewayStarting':
@@ -230,15 +232,20 @@ describe('ChatInput agent targeting', () => {
artifactPanelMocks.openPreview.mockReset();
});
it('renders the Zoomies working indicator while a message is sending', () => {
it('renders a dot pulse and visible thinking label while a message is sending', () => {
render(
<TooltipProvider>
<ChatInput onSend={vi.fn()} sending />
</TooltipProvider>,
);
expect(screen.getByTestId('chat-composer-working-indicator')).toBeInTheDocument();
expect(screen.getByTestId('chat-composer-zoomies')).toBeInTheDocument();
const indicator = screen.getByRole('status', { name: 'Thinking…' });
expect(indicator).toHaveAttribute('data-testid', 'chat-composer-working-indicator');
expect(indicator).toHaveTextContent('Thinking…');
expect(indicator).toHaveAttribute('aria-label', 'Thinking…');
expect(indicator).toHaveAttribute('aria-live', 'polite');
expect(screen.getByTestId('chat-composer-dot-pulse')).toBeInTheDocument();
expect(screen.queryByTestId('chat-composer-zoomies')).not.toBeInTheDocument();
});
it('renders editable workspace selector in the composer footer', () => {
tests/unit/chat-page-execution-graph.test.tsx
+11 -1
View File
@@ -10,6 +10,9 @@ const { acpState, agentsState, artifactPanelState, chatState, gatewayState, stic
cancelling: false,
error: null as string | null,
activeSessionKey: 'agent:main:main' as string | null,
workspaceRoot: null as string | null,
cwd: null as string | null,
prepareLocalSession: vi.fn(),
loadSession: vi.fn(),
sendPrompt: vi.fn(),
cancel: vi.fn(),
@@ -209,6 +212,9 @@ describe('ACP Chat page inline timeline lifecycle', () => {
acpState.cancelling = false;
acpState.error = null;
acpState.activeSessionKey = 'agent:main:main';
acpState.workspaceRoot = null;
acpState.cwd = null;
acpState.prepareLocalSession.mockReset();
acpState.loadSession.mockReset();
acpState.loadSession.mockResolvedValue(undefined);
acpState.sendPrompt.mockReset();
@@ -253,7 +259,11 @@ describe('ACP Chat page inline timeline lifecycle', () => {
await waitFor(() => {
expect(ensureAcpChatSubscriptions).toHaveBeenCalledTimes(1);
expect(acpState.loadSession).toHaveBeenCalledWith({ sessionKey: 'agent:main:main', cwd: '/workspace' });
expect(acpState.loadSession).toHaveBeenCalledWith({
sessionKey: 'agent:main:main',
workspaceRoot: '/workspace',
cwd: '/workspace',
});
});
});
tests/unit/file-preview-body.test.tsx
+53
View File
@@ -16,6 +16,7 @@ const shellOpenPathMock = vi.fn(async () => '');
const shellShowItemInFolderMock = vi.fn(async () => undefined);
const readTextFile = vi.fn();
const readWorkspaceText = vi.fn();
const readAttachmentText = vi.fn();
const statFile = vi.fn();
const statWorkspaceFile = vi.fn();
const writeTextFile = vi.fn();
@@ -23,6 +24,7 @@ const writeTextFile = vi.fn();
vi.mock('@/lib/file-preview-client', () => ({
readTextFile: (...args: unknown[]) => readTextFile(...args),
readWorkspaceText: (...args: unknown[]) => readWorkspaceText(...args),
readAttachmentText: (...args: unknown[]) => readAttachmentText(...args),
statFile: (...args: unknown[]) => statFile(...args),
statWorkspaceFile: (...args: unknown[]) => statWorkspaceFile(...args),
writeTextFile: (...args: unknown[]) => writeTextFile(...args),
@@ -88,6 +90,57 @@ describe('FilePreviewBody', () => {
expect(screen.queryByText('<!doctype html>')).not.toBeInTheDocument();
});
it('uses attachment-scoped text reads for HTML without a naked-path fallback', async () => {
const attachmentFileRef = { sessionKey: 'agent:main:s1', generation: 4, uri: 'file:///secret/site.html' };
readAttachmentText.mockResolvedValueOnce({
ok: true,
content: '<h1>Scoped HTML</h1>',
mimeType: 'text/html',
size: 20,
readOnly: true,
});
render(
<FilePreviewBody
file={makePreviewTarget({
filePath: 'site.html',
fileName: 'site.html',
ext: '.html',
mimeType: 'text/html',
contentType: 'document',
size: 20,
attachmentFileRef,
})}
mode="preview"
/>,
);
expect(await screen.findByTestId('html-preview-frame')).toHaveAttribute('srcdoc', '<h1>Scoped HTML</h1>');
expect(readAttachmentText).toHaveBeenCalledWith(attachmentFileRef);
expect(readWorkspaceText).not.toHaveBeenCalled();
expect(readTextFile).not.toHaveBeenCalled();
expect(screen.queryByRole('button', { name: 'Show in file manager' })).not.toBeInTheDocument();
});
it('uses attachment-scoped text reads for source previews without workspace fallback', async () => {
const attachmentFileRef = { sessionKey: 'agent:main:s1', generation: 4, uri: 'file:///secret/demo.ts' };
readAttachmentText.mockResolvedValueOnce({
ok: true,
content: 'export const scoped = true;',
mimeType: 'text/typescript',
size: 27,
readOnly: true,
});
render(<FilePreviewBody file={makePreviewTarget({
filePath: 'demo.ts', fileName: 'demo.ts', ext: '.ts', mimeType: 'text/typescript', contentType: 'code', size: 27, attachmentFileRef,
})} />);
await waitFor(() => expect(readAttachmentText).toHaveBeenCalledWith(attachmentFileRef));
expect(readWorkspaceText).not.toHaveBeenCalled();
expect(readTextFile).not.toHaveBeenCalled();
});
it('uses known attachment size to show direct-open fallback for large PDFs', async () => {
render(
<FilePreviewBody
tests/unit/files-api-workspace.test.ts
+52 -1
View File
@@ -1,7 +1,7 @@
// @vitest-environment node
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { mkdir, mkdtemp, open, realpath, rename, rm, stat, symlink, truncate, writeFile } from 'node:fs/promises';
import { mkdir, mkdtemp, open, readdir, realpath, rename, rm, stat, symlink, truncate, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
@@ -73,6 +73,57 @@ describe('workspace-scoped files api', () => {
});
}
it('registers every staged file id with Main-owned staging storage', async () => {
const { StagedAttachmentRegistry } = await import('../../electron/services/attachment-access');
const { createFilesApi } = await import('../../electron/services/files-api');
const stagedAttachments = new StagedAttachmentRegistry();
const api = createFilesApi({ stagedAttachments });
const result = await api.stageBuffer({
base64: Buffer.from('staged text').toString('base64'),
fileName: 'staged.txt',
mimeType: 'text/plain',
});
const [pathResult] = await api.stagePaths({ filePaths: [join(workspaceRoot, 'hello.txt')] });
expect(stagedAttachments.get(result.id)).toBe(await realpath(result.stagedPath));
expect(stagedAttachments.get(pathResult.id)).toBe(await realpath(pathResult.stagedPath));
expect(stagedAttachments.getDisplayPath(result.id)).toBeNull();
expect(stagedAttachments.getDisplayPath(pathResult.id)).toBe(join(workspaceRoot, 'hello.txt'));
expect(result.stagedPath).toContain(join('media', 'outbound', 'clawx-staging'));
expect(pathResult.stagedPath).toContain(join('media', 'outbound', 'clawx-staging'));
});
it.each(['buffer', 'path'])('rejects a %s stage when the pinned staging directory is replaced', async (kind) => {
const { StagedAttachmentRegistry } = await import('../../electron/services/attachment-access');
const { createFilesApi } = await import('../../electron/services/files-api');
const stagedAttachments = new StagedAttachmentRegistry();
const register = vi.spyOn(stagedAttachments, 'register');
const outsideDir = join(testDir, `outside-stage-${kind}`);
await mkdir(outsideDir);
const api = createFilesApi({
stagedAttachments,
stagingHooks: {
beforeDestinationOpen: async ({ stagingDir }) => {
await rename(stagingDir, `${stagingDir}-moved`);
await symlink(outsideDir, stagingDir);
},
},
} as never);
const operation = kind === 'buffer'
? api.stageBuffer({
base64: Buffer.from('must not escape').toString('base64'),
fileName: 'escape.txt',
mimeType: 'text/plain',
})
: api.stagePaths({ filePaths: [join(workspaceRoot, 'hello.txt')] });
await expect(operation).rejects.toThrow();
expect(register).not.toHaveBeenCalled();
expect(await readdir(outsideDir)).toEqual([]);
});
it('does not expose path-only scoped shell capabilities', async () => {
const api = await getApi();
tests/unit/generated-files.test.ts
+52
View File
@@ -8,6 +8,7 @@ import {
type GeneratedFileBaseline,
} from '@/lib/generated-files';
import type { RawMessage } from '@/stores/chat';
import { attachmentOpenMode, richFilePreviewKind } from '@/lib/file-preview-capabilities';
function makeWriteFile(overrides: Partial<GeneratedFile> = {}): GeneratedFile {
return {
@@ -89,6 +90,57 @@ describe('generated-files utilities', () => {
expect(stats).toBeNull();
});
it('uses shared preview limits and routes remote or unsupported attachments to system open', () => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///workspace/file.txt' };
const local = { kind: 'local' as const, scope: 'workspace' as const, ref };
const remote = { kind: 'remote' as const, ref, url: 'https://example.com/file.txt' };
expect(attachmentOpenMode({ ext: '.txt', mimeType: 'text/plain', size: 2 * 1024 * 1024, target: local })).toBe('preview');
expect(attachmentOpenMode({ ext: '.txt', mimeType: 'text/plain', size: 2 * 1024 * 1024 + 1, target: local })).toBe('system');
expect(attachmentOpenMode({ ext: '.pdf', mimeType: 'application/pdf', size: 50 * 1024 * 1024, target: local })).toBe('preview');
expect(attachmentOpenMode({ ext: '', mimeType: 'application/pdf', size: 1024, target: local })).toBe('preview');
expect(attachmentOpenMode({ ext: '.xlsx', mimeType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', size: 50 * 1024 * 1024 + 1, target: local })).toBe('system');
expect(attachmentOpenMode({ ext: '.zip', mimeType: 'application/zip', size: 100, target: local })).toBe('system');
expect(attachmentOpenMode({ ext: '.txt', mimeType: 'text/plain', size: 100, target: remote })).toBe('system');
});
it.each([
'.zip', '.tar', '.gz', '.rar', '.7z',
'.doc', '.docx', '.ppt', '.pptx',
'.mp3', '.wav', '.mp4', '.webm',
])('forces known unsupported extension %s to system open despite previewable MIME', (ext) => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: `/workspace/file${ext}` };
expect(attachmentOpenMode({
ext,
mimeType: 'text/plain',
size: 100,
target: { kind: 'local', scope: 'workspace', ref },
})).toBe('system');
});
it.each([
['.txt', 'application/octet-stream'],
['.ts', 'application/octet-stream'],
['.csv', 'application/zip'],
['.pdf', 'text/plain'],
['.xlsx', 'text/plain'],
])('preserves supported extension %s despite conflicting MIME', (ext, mimeType) => {
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: `/workspace/file${ext}` };
expect(attachmentOpenMode({
ext,
mimeType,
size: 100,
target: { kind: 'local', scope: 'workspace', ref },
})).toBe('preview');
});
it('uses supported extensions before conflicting rich MIME viewer hints', () => {
expect(richFilePreviewKind({ ext: '.pdf', mimeType: 'image/png' })).toBe('pdf');
expect(richFilePreviewKind({ ext: '.xlsx', mimeType: 'image/png' })).toBe('sheet');
expect(richFilePreviewKind({ ext: '.txt', mimeType: 'image/png' })).toBeNull();
expect(richFilePreviewKind({ ext: '', mimeType: 'image/png' })).toBe('image');
});
it('extracts write files with per-run baseline state and action', () => {
const messages: RawMessage[] = [
{ role: 'user', content: 'update file', timestamp: 1 },
tests/unit/harness-specs.test.ts
+34
View File
@@ -2,6 +2,9 @@ import { describe, expect, it } from 'vitest';
import {
isPluginLifecycleTask,
loadRuleSpecs,
loadScenarioSpecs,
loadSpec,
parseFrontmatter,
pathMatchesAny,
} from '../../harness/src/specs.mjs';
@@ -13,6 +16,37 @@ import {
} from '../../harness/src/rules.mjs';
describe('harness specs', () => {
it('defines the ACP media attachment harness contract', async () => {
const expectedRules = [
'renderer-main-boundary',
'backend-communication-boundary',
'api-client-transport-policy',
'host-api-fallback-policy',
'acp-chat-state-and-history',
'acp-compatibility-content-safety',
'attachment-access-safety',
'diagnostics-trace-safety',
'session-workspace-authority',
'tool-derived-file-safety',
'ui-i18n-design-tokens',
'comms-regression',
'docs-sync',
];
const [task, rules, scenarios] = await Promise.all([
loadSpec('harness/specs/tasks/acp-media-attachments.md'),
loadRuleSpecs(),
loadScenarioSpecs(),
]);
const ruleIds = new Set(rules.map((rule) => rule.data.id));
const acpChatScenario = scenarios.find((scenario) => scenario.data.id === 'acp-chat-experience');
expect(task.data.id).toBe('acp-media-attachments');
expect(task.data.requiredRules).toEqual(expectedRules);
expect(expectedRules.filter((ruleId) => !ruleIds.has(ruleId))).toEqual([]);
expect(task.data.requiredProfiles).toContain('e2e');
expect(acpChatScenario?.data.ownedPaths).toContain('tests/e2e/chat-acp-attachments.spec.ts');
});
it('parses Markdown frontmatter with arrays and nested docs', () => {
const spec = parseFrontmatter(`---
id: example
tests/unit/host-api-facade.test.ts
+53 -2
View File
@@ -297,6 +297,53 @@ describe('hostApi facade', () => {
});
});
it('passes attachment-scoped file actions without a workspace root', async () => {
hostInvoke.mockResolvedValue({ id: 'req', ok: true, data: { ok: true } });
const { hostApi } = await import('@/lib/host-api');
const ref = {
sessionKey: 'agent:main:session-a',
generation: 4,
uri: 'file:///workspace/report.pdf',
};
const resolvePayload = { ref, name: 'report.pdf', mimeType: 'application/pdf' };
await hostApi.files.resolveAttachment(resolvePayload);
await hostApi.files.readAttachmentText(ref);
await hostApi.files.readAttachmentBinary({ ref, maxBytes: 2048 });
await hostApi.files.openAttachment(ref);
const actions = [
['resolveAttachment', resolvePayload],
['readAttachmentText', ref],
['readAttachmentBinary', { ref, maxBytes: 2048 }],
['openAttachment', ref],
];
actions.forEach(([action, payload], index) => {
expect(hostInvoke).toHaveBeenNthCalledWith(index + 1, expect.objectContaining({
module: 'files',
action,
payload,
}));
expect(payload).not.toHaveProperty('workspaceRoot');
});
});
it('exports attachment resolution from the file preview client', async () => {
hostInvoke.mockResolvedValue({ id: 'req', ok: true, data: { ok: false, error: 'unavailable' } });
const { resolveAttachment } = await import('@/lib/file-preview-client');
const payload = {
ref: { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///workspace/a.txt' },
};
await resolveAttachment(payload);
expect(hostInvoke).toHaveBeenCalledWith(expect.objectContaining({
module: 'files',
action: 'resolveAttachment',
payload,
}));
});
it('does not expose workspace-scoped shell actions', async () => {
const { hostApi } = await import('@/lib/host-api');
@@ -323,7 +370,11 @@ describe('hostApi facade', () => {
.mockResolvedValueOnce({ id: 'req-4', ok: true, data: { success: true } });
const { hostApi } = await import('@/lib/host-api');
await hostApi.chat.loadAcpSession({ sessionKey: 'main', cwd: '/workspace/project' });
await hostApi.chat.loadAcpSession({
sessionKey: 'main',
workspaceRoot: '/workspace',
cwd: '/workspace/project',
});
await hostApi.chat.sendAcpPrompt({
sessionKey: 'main',
cwd: '/workspace/project',
@@ -339,7 +390,7 @@ describe('hostApi facade', () => {
expect(hostInvoke).toHaveBeenNthCalledWith(1, expect.objectContaining({
module: 'chat',
action: 'loadAcpSession',
payload: { sessionKey: 'main', cwd: '/workspace/project' },
payload: { sessionKey: 'main', workspaceRoot: '/workspace', cwd: '/workspace/project' },
}));
expect(hostInvoke).toHaveBeenNthCalledWith(2, expect.objectContaining({
module: 'chat',
tests/unit/host-services.test.ts
+25
View File
@@ -250,9 +250,12 @@ vi.mock('@electron/utils/whatsapp-login', () => ({
}));
vi.mock('@electron/utils/paths', () => ({
expandPath: (path: string) => path,
getOpenClawConfigDir: () => testOpenClawConfigDir,
getOpenClawDir: () => testOpenClawConfigDir,
getOpenClawResolvedDir: () => testOpenClawConfigDir,
resolveOpenClawConfigDir: () => testOpenClawConfigDir,
resolveOpenClawStateDir: () => testOpenClawConfigDir,
}));
vi.mock('@electron/utils/proxy-fetch', () => ({
@@ -1061,4 +1064,26 @@ describe('host services', () => {
],
});
});
it('exposes four attachment-scoped file operations from the files service', async () => {
const attachmentAccess = {
resolveAttachment: vi.fn().mockResolvedValue({ ok: false, error: 'unavailable', displayName: 'file' }),
readAttachmentText: vi.fn().mockResolvedValue({ ok: false, error: 'unavailable' }),
readAttachmentBinary: vi.fn().mockResolvedValue({ ok: false, error: 'unavailable' }),
openAttachment: vi.fn().mockResolvedValue({ ok: false, error: 'unavailable' }),
};
const { createFilesApi } = await import('@electron/services/files-api');
const filesApi = createFilesApi({ attachmentAccess: attachmentAccess as never });
const ref = { sessionKey: 'agent:main:s1', generation: 1, uri: 'file:///tmp/a.txt' };
await filesApi.resolveAttachment({ ref });
await filesApi.readAttachmentText(ref);
await filesApi.readAttachmentBinary({ ref, maxBytes: 5 });
await filesApi.openAttachment(ref);
expect(attachmentAccess.resolveAttachment).toHaveBeenCalledWith({ ref });
expect(attachmentAccess.readAttachmentText).toHaveBeenCalledWith(ref);
expect(attachmentAccess.readAttachmentBinary).toHaveBeenCalledWith({ ref, maxBytes: 5 });
expect(attachmentAccess.openAttachment).toHaveBeenCalledWith(ref);
});
});

Some files were not shown because too many files have changed in this diff Show More