fix(outputs): make asset hashes optional

Keep filename-backed output references previewable when hosts omit hash metadata, while preserving the explicit no-go path for asset-only references.

Update public docs and generated OpenAPI contract to describe optional hash metadata and the ComfyUI asset hashing flag.

Validation: targeted backend/unit/E2E checks passed; full Windows test gate passed.
This commit is contained in:
rookiestar28
2026-07-08 02:58:39 +08:00
parent c612a67053
commit 886e91c491
13 changed files with 131 additions and 31 deletions
+3 -3
View File
@@ -120,7 +120,7 @@ Deployment profiles and hardening references:
- Reconciled active prompt state after backend or SSE reconnects so completed prompts are not left in the active queue lane after a host recovery.
- Updated sidebar registration to prefer the current ComfyUI sidebar store API and keep the deprecated frontend facade as a compatibility fallback for older hosts.
- Aligned Model Manager and preflight diagnostics with current ComfyUI model folder names, including newer managed keys such as `gligen`, `latent_upscale_models`, `hypernetworks`, `photomaker`, `model_patches`, `geometry_estimation`, and `detection`, while retaining legacy aliases such as `clip` and `unet`.
- Made output parsing media-aware for current previewable result groups (`images`, `video`, `audio`, `3d`, and bounded `text`) while keeping image callbacks compatible and keeping asset-only identifiers as explicit fallback states instead of silently upgrading to `/api/assets`.
- Made output parsing media-aware for current previewable result groups (`images`, `video`, `audio`, `3d`, and bounded `text`) while keeping image callbacks compatible, supporting optional hash-backed refs when host metadata is present, and keeping asset-only identifiers as explicit fallback states instead of silently upgrading to `/api/assets`.
- OpenClaw prompt submissions now include stable `comfy_usage_source` attribution when missing, without overwriting caller-provided attribution or copying prompt/tenant/trace content into that field.
- Updated public release/support/troubleshooting docs to match the refreshed host facts and avoid exposing maintainer-only planning paths or machine-local links.
@@ -421,7 +421,7 @@ The OpenClaw sidebar includes these built-in tabs. Some tabs are capability-gate
| Tab | What it does | Related docs |
| --- | --- | --- |
| `Settings` | Health/config/log visibility, provider/model setup, model connectivity checks, and optional localhost key storage. | [Quick Start](#quick-start-minimal), [API Overview](#api-overview), [Troubleshooting](#troubleshooting) |
| `Jobs` | Tracks prompt IDs, consumes deterministic event/task cursor metadata for polling, and shows recent outputs across classic history refs, `asset_hash`/`hash`-backed refs, and current previewable media groups (`images`, `video`, `audio`, `3d`, bounded `text`). File-like refs stay on the `/view` contract; asset-service-only refs stay explicit instead of silently upgrading to `/api/assets`. | [API Overview](#api-overview), [Remote Control (Connector)](#remote-control-connector) |
| `Jobs` | Tracks prompt IDs, consumes deterministic event/task cursor metadata for polling, and shows recent outputs across classic history refs, optional `asset_hash`/`hash`-backed refs when host metadata is present, and current previewable media groups (`images`, `video`, `audio`, `3d`, bounded `text`). File-like refs stay on the `/view` contract; asset-service-only refs stay explicit instead of silently upgrading to `/api/assets`. | [API Overview](#api-overview), [Remote Control (Connector)](#remote-control-connector) |
| `Planner` | Uses assist endpoint to generate structured prompt plans (positive/negative/params). | [Configure an LLM key](#1-configure-an-llm-key-for-plannerrefinervision-helpers), [Nodes](#nodes) |
| `Refiner` | Refines existing prompts with optional image context and issue/goal input. | [Configure an LLM key](#1-configure-an-llm-key-for-plannerrefinervision-helpers), [Nodes](#nodes) |
| `Variants` | Local helper for generating batch variant parameter JSON (seed/range-style sweeps). | [Nodes](#nodes), [Operator UX Features](#operator-ux-features) |
@@ -525,7 +525,7 @@ Key operational notes:
- Observability remains token-gated for remote access and redacts provider reasoning-like content plus marked internal maintenance/helper content by default.
- Event/model-download polling and preflight inventory are snapshot/cursor-driven contracts; clients should consume `snapshot_ts`, `scan_state`, `stale`, and cursor metadata instead of assuming full-refresh polling.
- Model Manager and preflight consumers should use current ComfyUI folder keys for model types where possible, including `text_encoders`, `diffusion_models`, `gligen`, `latent_upscale_models`, `hypernetworks`, `photomaker`, `model_patches`, `geometry_estimation`, and `detection`; compatibility aliases such as `clip`, `unet`, `ckpt`, and plural legacy names are normalized before lookup/import.
- Output/history-facing consumers should keep using the bounded `/history` + `/view` contract; current previewable output groups include `images`, `video`, `audio`, `3d`, and bounded `text`, while refs that only upstream asset services can resolve remain explicit `asset_api_required` compatibility states.
- Output/history-facing consumers should keep using the bounded `/history` + `/view` contract; current previewable output groups include `images`, `video`, `audio`, `3d`, and bounded `text`, while optional hash-backed refs are used only when host metadata is present and refs that only upstream asset services can resolve remain explicit `asset_api_required` compatibility states.
- Queue submissions add stable ComfyUI usage-source attribution (`comfyui-openclaw`) when callers do not provide one; callers that already supply `extra_data.comfy_usage_source` keep ownership of that value.
- Connector diagnostics expose redacted token references only, and `/openclaw/connector/extraction-contract` is structural packaging metadata and static SecretRef policy rather than a live installation-health, environment, or token-status feed.
+10 -8
View File
@@ -3,20 +3,21 @@
## 2026-07-08 reference anchor update
- Current reference anchor is ComfyUI `51bf508a` (`v0.27.0-25-g51bf508a`, pyproject `0.27.0`).
- SaveImage output sockets, 3D preview refs, typed asset dimensions, grouped asset downloads, and `hash` / `asset_hash` aliases do not change the no-go decision.
- SaveImage output sockets, 3D preview refs, typed asset dimensions, grouped asset downloads, and optional `hash` / `asset_hash` aliases do not change the no-go decision.
- ComfyUI asset hashing is host-side opt-in through `--enable-asset-hashing`, so normal filename-backed output refs must not require hash metadata.
- OpenClaw continues to use `/history` + `/view`; asset-service-only refs stay explicit `asset_api_required` states.
## 2026-06-12 reconfirmation
- Current output parsing is media-aware for ComfyUI result groups `images`, `video`, `audio`, `3d`, and bounded `text`.
- File-like media refs still use `/view` when they provide `filename` or hash-backed preview metadata.
- File-like media refs still use `/view` when they provide `filename`, or optional hash-backed preview metadata when the host provides it.
- Text output previews are bounded and rendered as text, not HTML.
- Asset-service-only identifiers remain explicit fallback states and still do not trigger automatic direct `/api/assets` fetches.
## 2026-05-31 reconfirmation
- Current host reference evidence shows upstream asset responses may expose `hash` alongside `asset_hash`.
- OpenClaw accepts `hash` as an alias for hash-backed previews, but still resolves those refs through `/view?filename=blake3:...`.
- Current host reference evidence shows upstream asset responses may expose optional `hash` alongside `asset_hash`.
- OpenClaw accepts `hash` as an alias for hash-backed previews when present, but still resolves those refs through `/view?filename=blake3:...`.
- This does not change the no-go decision for automatic direct `/api/assets` runtime fetches.
## Scope
@@ -27,13 +28,14 @@
- Current history/output-facing interop already accepts:
- classic ComfyUI output refs (`filename`, `subfolder`, `type`)
- asset-hash-backed refs that still resolve through `/view?filename=blake3:...`
- optional asset-hash-backed refs that still resolve through `/view?filename=blake3:...` when host metadata is present
- media-aware output groups (`images`, `video`, `audio`, `3d`, and bounded `text`)
- Current ComfyUI `51bf508a` / `v0.27.0-25-g51bf508a` / pyproject `0.27.0` reference facts:
- `/api/assets*` routes exist, but operational use is feature-gated behind `--enable-assets`
- content hashing is opt-in through `--enable-asset-hashing`, so normal filename-backed refs may omit `asset_hash` / `hash`
- `/features` exposes the `assets` capability flag so hosts can report whether the asset system is enabled
- frontend preview still resolves `blake3:...` asset hashes through `/view`, so hash-backed outputs do not require a direct `/api/assets` fetch
- asset responses may expose `hash` alongside `asset_hash`; OpenClaw treats both as hash-backed preview aliases
- asset responses may expose optional `hash` alongside `asset_hash`; OpenClaw treats both as hash-backed preview aliases when present
- Current operator/runtime surfaces in scope:
- sidebar `Jobs`
- callback delivery payloads
@@ -49,7 +51,7 @@
## Rationale
1. Current OpenClaw output surfaces still succeed on the existing bounded `/view` contract, including asset-hash-backed refs.
1. Current OpenClaw output surfaces still succeed on the existing bounded `/view` contract, including optional asset-hash-backed refs when metadata exists.
2. Adding `/api/assets` as a normal dependency would widen runtime coupling to upstream host behavior without a demonstrated operator need in current features.
3. A silent fallback from `asset id only` to `/api/assets` would weaken boundary clarity and make host drift harder to reason about.
@@ -57,7 +59,7 @@
- Preserve current supported refs exactly:
- classic refs -> `/view?filename=...&type=...`
- asset-hash-backed refs -> `/view?filename=blake3:...`
- optional asset-hash-backed refs -> `/view?filename=blake3:...` when metadata exists
- file-like media refs -> `/view` fallback/link surfaces when preview metadata is present
- bounded text refs -> escaped text surfaces, not HTML
- For refs that expose only asset-service identifiers and are not representable through `/view`:
+2 -2
View File
@@ -14,7 +14,7 @@ This document summarizes the current OpenClaw sidebar UI structure and how to ve
- Tabs: `web/openclaw_tabs.js` manages tab registration, rendering, and remount safety.
- API: `web/openclaw_api.js` provides a normalized fetch wrapper and OpenClaw endpoints (legacy Moltbot endpoints still work).
- Host surface: `web/openclaw_host_surface.js` resolves the active frontend host surface and stamps explicit metadata so standalone frontend vs desktop-embedded behavior stays testable.
- Output refs: `web/openclaw_asset_refs.js` normalizes classic history refs, `asset_hash`/`hash` refs, and current previewable media groups (`images`, `video`, `audio`, `3d`, bounded `text`) onto one media-aware contract. File-like refs stay on the bounded `/view` preview path, text refs stay escaped and bounded, and asset-service-only refs remain explicit fallback states instead of silently auto-fetching `/api/assets`.
- Output refs: `web/openclaw_asset_refs.js` normalizes classic history refs, optional `asset_hash`/`hash` refs when host metadata is present, and current previewable media groups (`images`, `video`, `audio`, `3d`, bounded `text`) onto one media-aware contract. File-like refs stay on the bounded `/view` preview path, text refs stay escaped and bounded, and asset-service-only refs remain explicit fallback states instead of silently auto-fetching `/api/assets`.
- Styles: `web/openclaw.css` provides shared design tokens and component classes.
- Errors and compatibility helpers: `web/openclaw_utils.js` provides `showError()` / `clearError()` plus runtime legacy-class alias helpers used to keep canonical `openclaw-*` markup compatible with existing `moltbot-*` selectors.
@@ -83,7 +83,7 @@ If `assist_streaming` is unavailable or the stream transport degrades, Planner/R
3. Confirm the sidebar host-surface metadata resolves correctly for the current environment instead of defaulting silently.
4. Planner: click **Plan Generation** with minimal input and confirm either live preview/stage updates appear (when streaming is supported) or a readable fallback result/error appears.
5. Refiner: click **Refine Prompts** (with or without image) and confirm either live preview/stage updates appear (when streaming is supported) or a readable fallback result/error appears.
6. Jobs: verify output previews still resolve for classic history refs, hash-backed refs, and supported media-aware refs (`images`, `video`, `audio`, `3d`, bounded `text`); asset-service-only refs should stay explicit as a bounded fallback state, and repeated polls should not duplicate rows after reconnect/resume.
6. Jobs: verify output previews still resolve for classic history refs, optional hash-backed refs when host metadata is present, and supported media-aware refs (`images`, `video`, `audio`, `3d`, bounded `text`); asset-service-only refs should stay explicit as a bounded fallback state, and repeated polls should not duplicate rows after reconnect/resume.
7. Explorer: verify preflight inventory can show `refreshing` / `stale` / `error` state without freezing the tab while deep scan work continues, and verify inactive-branch suppressed findings render separately from actionable failures.
8. Library/Approvals: if backend endpoints are not enabled, confirm the UI shows a clear error state (no crashes).
9. If you simulate/fake a stream failure in dev tools, confirm Planner/Refiner retry through the classic non-stream path without duplicate submits or broken loading state.
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
title: "ComfyUI-OpenClaw API"
version: "1.0.12"
version: "1.0.13"
description: "Generated from docs/release/api_contract.md."
servers:
- url: "/openclaw"
+4 -3
View File
@@ -1,8 +1,8 @@
# OpenClaw API Contract (v1)
> **Status**: normative
> **Version**: 1.0.12
> **Date**: 2026-06-12
> **Version**: 1.0.13
> **Date**: 2026-07-08
This document defines the public API contract for OpenClaw. It serves as the authoritative baseline for client compatibility and breaking change policies.
@@ -80,7 +80,8 @@ History and output-ref contract:
- history/output consumers SHOULD treat the normalized output-ref contract as media-aware
- current previewable output groups are `images`, `video`, `audio`, `3d`, and bounded `text`
- file-like refs that can be represented through `/view` remain on the bounded `/history` + `/view` preview path
- refs with `asset_hash` or `hash` values that map to `blake3:...` preview through `/view?filename=blake3:...`
- `asset_hash` / `hash` metadata is optional because current ComfyUI host asset hashing is opt-in through `--enable-asset-hashing`; clients MUST NOT require hashes for normal filename-backed previews
- refs with `asset_hash` or `hash` values, when host metadata provides them, preview through `/view?filename=blake3:...`
- refs that only expose upstream asset-service identifiers remain explicit `asset_api_required` states; clients MUST NOT silently infer direct `/api/assets` fetching from that marker
- legacy callback/image-only consumers may continue using image-only extraction paths; non-image media refs should be rendered as explicit fallback/link/text surfaces unless the client implements a safe media-specific renderer
+1 -1
View File
@@ -43,7 +43,7 @@ This document tracks the current reference anchors and validated environments fo
## Residual Host-Contract Decisions
- **SaveImage output refs**: OpenClaw consumes runtime `/history` output refs and does not infer graph-rewrite behavior from output-node socket shape. `SaveImage` output sockets are allowed to exist without changing the normalized output-ref contract.
- **3D output refs**: `Load3DAdvanced` and related 3D preview refs remain media-aware output refs. File-like and hash-backed 3D refs stay on the bounded `/view` preview contract; clients without a 3D renderer should show an explicit fallback/link surface.
- **3D output refs**: `Load3DAdvanced` and related 3D preview refs remain media-aware output refs. File-like refs and optional hash-backed 3D refs stay on the bounded `/view` preview contract; clients without a 3D renderer should show an explicit fallback/link surface.
- **Asset dimensions and grouped assets**: typed width/height metadata and grouped multi-download behavior are host-frontend display/download concerns. They do not change OpenClaw fetch routing, and asset-service-only identifiers remain explicit `asset_api_required` states rather than implicit `/api/assets` fetches.
- **Sidebar registration**: prefer the current `sidebarTab.registerSidebarTab` host API and retain the deprecated `extensionManager.registerSidebarTab` fallback for older or desktop-embedded frontend hosts.
- **Node runtime policy**: the standalone ComfyUI frontend development workspace currently declares `node >=25 <26` and `pnpm >=11.3`, but OpenClaw keeps its package engine at `>=18.0.0` because this custom-node package runs its own Playwright/Vitest harness and does not build the host frontend workspace. OpenClaw acceptance remains governed by `tests/TEST_SOP.md` and `tests/E2E_TESTING_SOP.md`, which require Node.js 18+ and CI-parity validation on the project test harness.
+1 -1
View File
@@ -100,7 +100,7 @@ If a result ref only exposes an upstream asset-service identifier and cannot be
What this means:
- `asset_api_required` is a bounded compatibility state, not a generic parser failure.
- Classic history refs and hash-backed refs exposed as `asset_hash` or `hash` that still map onto `/view` should continue to preview normally.
- Classic history refs should continue to preview normally even when hash metadata is absent. Optional hash-backed refs exposed as `asset_hash` or `hash` still map onto `/view` when ComfyUI host metadata provides them.
- Current media-aware outputs can include `images`, `video`, `audio`, `3d`, and bounded `text`; images render as thumbnails, text renders as escaped bounded text, and other file-like media may appear as explicit fallback/link tiles instead of image elements.
- If an operator workflow starts depending on direct asset-service identifiers, treat that as a contract gap and review [`docs/asset_api_adoption_decision.md`](asset_api_adoption_decision.md) before widening the runtime dependency.
+4 -3
View File
@@ -131,9 +131,10 @@ def normalize_history_output_ref(
resolution = "asset_api_required" if asset_api_required else "view"
if not asset_api_required:
# IMPORTANT: keep OpenClaw on the bounded /view contract. Asset-hash refs
# are accepted because they still resolve through /view; do not escalate
# asset-api-only identifiers into implicit /api/assets runtime fetches.
# IMPORTANT: keep OpenClaw on the bounded /view contract. Optional
# asset-hash metadata still resolves through /view when hosts provide it;
# do not escalate asset-api-only identifiers into implicit /api/assets
# runtime fetches.
if asset_hash:
params = {"filename": asset_hash}
else:
+22 -7
View File
@@ -123,7 +123,7 @@ test.describe('R107 Live Backend Parity', () => {
await expect(page.locator('img[src*="test_img.png"]')).toBeVisible();
});
test('Job Monitor keeps the phase-2 asset API no-go contract explicit', async ({ page }) => {
test('Job Monitor keeps asset hashing optional and asset API no-go explicit', async ({ page }) => {
const jobId = "job-asset-phase2";
let assetApiCalls = 0;
@@ -137,6 +137,13 @@ test.describe('R107 Live Backend Parity', () => {
outputs: {
"9": {
images: [
{
filename: "filename-only.png",
type: "output",
asset: {
id: "asset-without-hash",
},
},
{
filename: "preview.png",
type: "temp",
@@ -175,12 +182,19 @@ test.describe('R107 Live Backend Parity', () => {
await page.route('**/view**', async route => {
const request = route.request();
const url = new URL(request.url());
if (
request.method() !== 'GET'
|| url.searchParams.get('filename') !== 'blake3:abc123'
|| url.searchParams.has('type')
|| url.searchParams.has('subfolder')
) {
const filename = url.searchParams.get('filename');
const isFilenameOnlyPreview = (
request.method() === 'GET'
&& filename === 'filename-only.png'
&& url.searchParams.get('type') === 'output'
);
const isHashBackedPreview = (
request.method() === 'GET'
&& filename === 'blake3:abc123'
&& !url.searchParams.has('type')
&& !url.searchParams.has('subfolder')
);
if (!isFilenameOnlyPreview && !isHashBackedPreview) {
await route.fallback();
return;
}
@@ -197,6 +211,7 @@ test.describe('R107 Live Backend Parity', () => {
await page.getByText('Add').click();
await expect(page.locator('.openclaw-kv-val.ok')).toHaveText('completed', { timeout: 10000 });
await expect(page.locator('img[src*="filename-only.png"]')).toBeVisible();
await expect(page.locator('img[src*="blake3%3Aabc123"]')).toBeVisible();
await expect(page.locator('.openclaw-job-output-fallback')).toContainText('Asset API output requires /api/assets');
expect(assetApiCalls).toBe(0);
+31
View File
@@ -104,6 +104,37 @@ class TestComfyUIHistoryParsing(unittest.TestCase):
self.assertFalse(images[0]["asset_api_required"])
self.assertEqual(images[0]["resolution"], "view")
def test_extract_images_filename_refs_do_not_require_asset_hash(self):
from services.comfyui_history import extract_images
history_item = {
"outputs": {
"2": {
"images": [
{
"filename": "filename-only.png",
"subfolder": "session-a",
"type": "output",
"asset": {
"id": "asset-without-hash",
},
}
]
}
}
}
images = extract_images(history_item)
self.assertEqual(len(images), 1)
self.assertEqual(images[0]["filename"], "filename-only.png")
self.assertEqual(images[0]["asset_hash"], "")
self.assertEqual(images[0]["asset_api_id"], "asset-without-hash")
self.assertFalse(images[0]["asset_api_required"])
self.assertEqual(images[0]["resolution"], "view")
self.assertIn("filename=filename-only.png", images[0]["view_url"])
self.assertIn("type=output", images[0]["view_url"])
self.assertIn("subfolder=session-a", images[0]["view_url"])
def test_extract_images_accepts_top_level_hash_alias(self):
from services.comfyui_history import extract_images
+18
View File
@@ -13,10 +13,28 @@ class TestR167AssetApiContract(unittest.IsolatedAsyncioTestCase):
self.assertIn("/api/assets", doc)
self.assertIn("--enable-assets", doc)
self.assertIn("--enable-asset-hashing", doc)
self.assertIn("optional", doc.lower())
self.assertIn("/features", doc)
self.assertIn("blake3", doc)
self.assertIn("/view", doc)
def test_public_output_ref_docs_state_hashes_are_optional(self):
repo_root = Path(__file__).resolve().parents[1]
public_docs = [
repo_root / "README.md",
repo_root / "docs" / "release" / "api_contract.md",
repo_root / "docs" / "troubleshooting.md",
repo_root / "docs" / "frontend_ux_walkthrough.md",
]
for doc_path in public_docs:
with self.subTest(doc=str(doc_path.relative_to(repo_root))):
doc = doc_path.read_text(encoding="utf-8").lower()
self.assertIn("optional", doc)
self.assertIn("/view", doc)
self.assertIn("hash", doc)
async def test_callback_delivery_preserves_asset_api_only_refs_without_view_fetch(
self,
):
+3 -2
View File
@@ -142,8 +142,9 @@ export function normalizeComfyOutputRef(imageRef = {}, mediaType = "images") {
const explicitAssetApiRequired = outputRef.asset_api_required === true;
const assetApiRequired = Boolean(explicitAssetApiRequired || (assetApiId && !assetHash && !namedFilename));
// IMPORTANT: asset-backed refs still resolve through /view when possible; do
// not promote asset-api-only identifiers into implicit /api/assets fetches.
// IMPORTANT: optional asset-hash metadata still resolves through /view when
// hosts provide it; do not promote asset-api-only identifiers into implicit
// /api/assets fetches.
const viewParams = assetApiRequired
? null
: (
@@ -61,6 +61,37 @@ describe("openclaw asset refs", () => {
});
});
it("keeps filename refs previewable when host omits hash metadata", () => {
expect(
normalizeComfyOutputRef({
filename: "filename-only.png",
subfolder: "session-a",
type: "output",
asset: {
id: "asset-without-hash",
},
})
).toEqual({
filename: "filename-only.png",
subfolder: "session-a",
type: "output",
media_type: "images",
asset_hash: "",
asset_api_id: "asset-without-hash",
asset_api_required: false,
resolution: "view",
unsupported_reason: "",
is_asset_backed: true,
content: "",
text_truncated: false,
viewParams: {
filename: "filename-only.png",
subfolder: "session-a",
type: "output",
},
});
});
it("accepts upload-style nested asset metadata", () => {
expect(
normalizeComfyOutputRef({