mirror of
https://github.com/rookiestar28/ComfyUI-OpenClaw.git
synced 2026-08-14 00:48:07 +00:00
docs: refresh host alignment documentation
This commit is contained in:
rookiestar28
@@ -103,13 +103,14 @@ Deployment profiles and hardening references:
|
||||
|
||||
<details>
|
||||
|
||||
<summary><strong>ComfyUI host compatibility, queue recovery, model folders, and asset-output posture refreshed</strong></summary>
|
||||
<summary><strong>ComfyUI host compatibility, media outputs, model folders, and prompt attribution refreshed</strong></summary>
|
||||
|
||||
- Refreshed the published compatibility baseline for ComfyUI `822aca19` (`v0.24.0-60-g822aca19`, pyproject `0.24.0`), standalone frontend `1.46.13`, and Desktop `0.9.4` with core `0.22.3` plus embedded frontend `1.43.18`.
|
||||
- Reconciled active prompt state after backend or SSE reconnects so completed prompts are not left in the active queue lane after a host recovery.
|
||||
- Updated sidebar registration to prefer the current ComfyUI sidebar store API and keep the deprecated frontend facade as a compatibility fallback for older hosts.
|
||||
- Aligned Model Manager and preflight diagnostics with current ComfyUI model folder names such as `text_encoders`, `diffusion_models`, `geometry_estimation`, and `detection`, while retaining legacy aliases such as `clip` and `unet`.
|
||||
- Kept output previews on the bounded `/history` + `/view` contract, including `asset_hash` and `hash` aliases for hash-backed previews; upstream asset-only identifiers remain explicit fallback states unless a future feature requires direct `/api/assets` use.
|
||||
- Aligned Model Manager and preflight diagnostics with current ComfyUI model folder names, including newer managed keys such as `gligen`, `latent_upscale_models`, `hypernetworks`, `photomaker`, `model_patches`, `geometry_estimation`, and `detection`, while retaining legacy aliases such as `clip` and `unet`.
|
||||
- Made output parsing media-aware for current previewable result groups (`images`, `video`, `audio`, `3d`, and bounded `text`) while keeping image callbacks compatible and keeping asset-only identifiers as explicit fallback states instead of silently upgrading to `/api/assets`.
|
||||
- OpenClaw prompt submissions now include stable `comfy_usage_source` attribution when missing, without overwriting caller-provided attribution or copying prompt/tenant/trace content into that field.
|
||||
- Updated public release/support/troubleshooting docs to match the refreshed host facts and avoid exposing maintainer-only planning paths or machine-local links.
|
||||
|
||||
</details>
|
||||
@@ -419,7 +420,7 @@ The OpenClaw sidebar includes these built-in tabs. Some tabs are capability-gate
|
||||
| Tab | What it does | Related docs |
|
||||
| --- | --- | --- |
|
||||
| `Settings` | Health/config/log visibility, provider/model setup, model connectivity checks, and optional localhost key storage. | [Quick Start](#quick-start-minimal), [API Overview](#api-overview), [Troubleshooting](#troubleshooting) |
|
||||
| `Jobs` | Tracks prompt IDs, consumes deterministic event/task cursor metadata for polling, and shows output previews for recent jobs across classic history refs plus `asset_hash`/`hash`-backed output refs through the same `/view` contract; refs that only expose asset-service identifiers stay explicit as an operator-visible fallback state instead of silently upgrading to `/api/assets`. | [API Overview](#api-overview), [Remote Control (Connector)](#remote-control-connector) |
|
||||
| `Jobs` | Tracks prompt IDs, consumes deterministic event/task cursor metadata for polling, and shows recent outputs across classic history refs, `asset_hash`/`hash`-backed refs, and current previewable media groups (`images`, `video`, `audio`, `3d`, bounded `text`). File-like refs stay on the `/view` contract; asset-service-only refs stay explicit instead of silently upgrading to `/api/assets`. | [API Overview](#api-overview), [Remote Control (Connector)](#remote-control-connector) |
|
||||
| `Planner` | Uses assist endpoint to generate structured prompt plans (positive/negative/params). | [Configure an LLM key](#1-configure-an-llm-key-for-plannerrefinervision-helpers), [Nodes](#nodes) |
|
||||
| `Refiner` | Refines existing prompts with optional image context and issue/goal input. | [Configure an LLM key](#1-configure-an-llm-key-for-plannerrefinervision-helpers), [Nodes](#nodes) |
|
||||
| `Variants` | Local helper for generating batch variant parameter JSON (seed/range-style sweeps). | [Nodes](#nodes), [Operator UX Features](#operator-ux-features) |
|
||||
@@ -428,7 +429,7 @@ The OpenClaw sidebar includes these built-in tabs. Some tabs are capability-gate
|
||||
| `Explorer` | Inventory/preflight diagnostics and snapshot/checkpoint troubleshooting workflows, including snapshot-first inventory refresh state (`snapshot_ts`, `scan_state`, `stale`, `last_error`) and suppressed inactive-branch findings. | [Operator UX Features](#operator-ux-features), [Troubleshooting](#troubleshooting) |
|
||||
| `Packs` | Dedicated pack lifecycle tab for import/export/delete under admin boundary. | [API Overview](#api-overview) |
|
||||
| `PNG Info` | Inspects saved generation images through drag-and-drop, file picker, or scoped paste, parses A1111 infotext plus ComfyUI `prompt` / `workflow` metadata, shows extracted prompt and generation fields when recoverable, and keeps raw metadata visible for operator inspection. | [API Overview](#api-overview), [Troubleshooting](#troubleshooting) |
|
||||
| `Model Manager` | Searches model catalog/install records, queues managed downloads, monitors task lifecycle, and imports completed tasks into the managed install root with current ComfyUI folder-key normalization, including `geometry_estimation` and `detection`, plus legacy type aliases. | [API Overview](#api-overview), [Troubleshooting](#troubleshooting) |
|
||||
| `Model Manager` | Searches model catalog/install records, queues managed downloads, monitors task lifecycle, and imports completed tasks into the managed install root with current ComfyUI folder-key normalization, including `gligen`, `latent_upscale_models`, `hypernetworks`, `photomaker`, `model_patches`, `geometry_estimation`, and `detection`, plus legacy type aliases. | [API Overview](#api-overview), [Troubleshooting](#troubleshooting) |
|
||||
| `Parameter Lab` | Runs bounded sweep/compare experiments, stores history, and replays parameters back into the graph. | [Operator UX Features](#operator-ux-features) |
|
||||
|
||||
## Operator UX Features
|
||||
@@ -522,8 +523,9 @@ Key operational notes:
|
||||
|
||||
- Observability remains token-gated for remote access and redacts provider reasoning-like content plus marked internal maintenance/helper content by default.
|
||||
- Event/model-download polling and preflight inventory are snapshot/cursor-driven contracts; clients should consume `snapshot_ts`, `scan_state`, `stale`, and cursor metadata instead of assuming full-refresh polling.
|
||||
- Model Manager and preflight consumers should use current ComfyUI folder keys for model types where possible, including `text_encoders`, `diffusion_models`, `geometry_estimation`, and `detection`; compatibility aliases such as `clip`, `unet`, `ckpt`, and plural legacy names are normalized before lookup/import.
|
||||
- Output/history-facing consumers should keep using the bounded `/history` + `/view` contract; `asset_hash` and `hash` values that map to `blake3:...` preview normally, while refs that only upstream asset services can resolve remain explicit `asset_api_required` compatibility states.
|
||||
- Model Manager and preflight consumers should use current ComfyUI folder keys for model types where possible, including `text_encoders`, `diffusion_models`, `gligen`, `latent_upscale_models`, `hypernetworks`, `photomaker`, `model_patches`, `geometry_estimation`, and `detection`; compatibility aliases such as `clip`, `unet`, `ckpt`, and plural legacy names are normalized before lookup/import.
|
||||
- Output/history-facing consumers should keep using the bounded `/history` + `/view` contract; current previewable output groups include `images`, `video`, `audio`, `3d`, and bounded `text`, while refs that only upstream asset services can resolve remain explicit `asset_api_required` compatibility states.
|
||||
- Queue submissions add stable ComfyUI usage-source attribution (`comfyui-openclaw`) when callers do not provide one; callers that already supply `extra_data.comfy_usage_source` keep ownership of that value.
|
||||
- Connector diagnostics expose redacted token references only, and `/openclaw/connector/extraction-contract` is structural packaging metadata and static SecretRef policy rather than a live installation-health, environment, or token-status feed.
|
||||
|
||||
## Advanced Security and Runtime Setup
|
||||
@@ -693,7 +695,7 @@ Quick jumps:
|
||||
|
||||
- backend not loaded / route 404 startup failures
|
||||
- Operator Doctor usage
|
||||
- Jobs preview fallback for asset-api-only output refs
|
||||
- Jobs preview and media fallback for asset-api-only output refs
|
||||
- audit chain verification after restart or rotation
|
||||
- external-tool allowlist, sandbox runtime, interpreter, timeout, or workspace diagnostics
|
||||
- webhook auth not configured
|
||||
|
||||
@@ -1,5 +1,12 @@
|
||||
# ComfyUI Asset API Adoption Decision (2026-04-16)
|
||||
|
||||
## 2026-06-12 reconfirmation
|
||||
|
||||
- Current output parsing is media-aware for ComfyUI result groups `images`, `video`, `audio`, `3d`, and bounded `text`.
|
||||
- File-like media refs still use `/view` when they provide `filename` or hash-backed preview metadata.
|
||||
- Text output previews are bounded and rendered as text, not HTML.
|
||||
- Asset-service-only identifiers remain explicit fallback states and still do not trigger automatic direct `/api/assets` fetches.
|
||||
|
||||
## 2026-05-31 reconfirmation
|
||||
|
||||
- Current host reference evidence shows upstream asset responses may expose `hash` alongside `asset_hash`.
|
||||
@@ -15,6 +22,7 @@
|
||||
- Current history/output-facing interop already accepts:
|
||||
- classic ComfyUI output refs (`filename`, `subfolder`, `type`)
|
||||
- asset-hash-backed refs that still resolve through `/view?filename=blake3:...`
|
||||
- media-aware output groups (`images`, `video`, `audio`, `3d`, and bounded `text`)
|
||||
- Current ComfyUI `822aca19` / `v0.24.0-60-g822aca19` / pyproject `0.24.0` reference facts:
|
||||
- `/api/assets*` routes exist, but operational use is feature-gated behind `--enable-assets`
|
||||
- `/features` exposes the `assets` capability flag so hosts can report whether the asset system is enabled
|
||||
@@ -31,7 +39,7 @@
|
||||
|
||||
- **No-go for first-class `/api/assets` runtime adoption in phase 2.**
|
||||
- OpenClaw keeps `/history` + `/view` as the supported runtime contract for normal output handling.
|
||||
- Asset-api-only identifiers are now treated as explicit unsupported contracts rather than implicit fetch targets.
|
||||
- Asset-api-only identifiers are treated as explicit unsupported contracts rather than implicit fetch targets.
|
||||
|
||||
## Rationale
|
||||
|
||||
@@ -44,6 +52,8 @@
|
||||
- Preserve current supported refs exactly:
|
||||
- classic refs -> `/view?filename=...&type=...`
|
||||
- asset-hash-backed refs -> `/view?filename=blake3:...`
|
||||
- file-like media refs -> `/view` fallback/link surfaces when preview metadata is present
|
||||
- bounded text refs -> escaped text surfaces, not HTML
|
||||
- For refs that expose only asset-service identifiers and are not representable through `/view`:
|
||||
- keep them in normalized output payloads
|
||||
- mark them as `asset_api_required`
|
||||
|
||||
@@ -14,7 +14,7 @@ This document summarizes the current OpenClaw sidebar UI structure and how to ve
|
||||
- Tabs: `web/openclaw_tabs.js` manages tab registration, rendering, and remount safety.
|
||||
- API: `web/openclaw_api.js` provides a normalized fetch wrapper and OpenClaw endpoints (legacy Moltbot endpoints still work).
|
||||
- Host surface: `web/openclaw_host_surface.js` resolves the active frontend host surface and stamps explicit metadata so standalone frontend vs desktop-embedded behavior stays testable.
|
||||
- Output refs: `web/openclaw_asset_refs.js` normalizes classic history refs and newer asset-backed output refs onto the same bounded `/view` preview contract, while keeping asset-service-only refs explicit as a fallback state instead of silently auto-fetching `/api/assets`.
|
||||
- Output refs: `web/openclaw_asset_refs.js` normalizes classic history refs, `asset_hash`/`hash` refs, and current previewable media groups (`images`, `video`, `audio`, `3d`, bounded `text`) onto one media-aware contract. File-like refs stay on the bounded `/view` preview path, text refs stay escaped and bounded, and asset-service-only refs remain explicit fallback states instead of silently auto-fetching `/api/assets`.
|
||||
- Styles: `web/openclaw.css` provides shared design tokens and component classes.
|
||||
- Errors and compatibility helpers: `web/openclaw_utils.js` provides `showError()` / `clearError()` plus runtime legacy-class alias helpers used to keep canonical `openclaw-*` markup compatible with existing `moltbot-*` selectors.
|
||||
|
||||
@@ -24,7 +24,7 @@ Refactor note:
|
||||
- New tab markup should use canonical `openclaw-*` classes; legacy `moltbot-*` aliases are generated centrally at runtime instead of being duplicated in each template.
|
||||
- New host sidebar registration changes should stay in `web/openclaw_sidebar_registration.js` rather than duplicating ComfyUI frontend API detection inside the extension entrypoint.
|
||||
- Host-sensitive behaviors should consume the shared host-surface helper rather than inferring desktop vs standalone frontend from ad-hoc globals.
|
||||
- Output preview flows should consume the shared asset-ref normalizer rather than assembling `/view` URLs independently in each tab or silently widening runtime behavior to direct `/api/assets` fetches.
|
||||
- Output preview flows should consume the shared asset-ref normalizer rather than assembling `/view` URLs independently in each tab, treating non-image media as broken images, or silently widening runtime behavior to direct `/api/assets` fetches.
|
||||
- Explorer/preflight consumers should treat inventory diagnostics as snapshot-first and surface `snapshot_ts`, `scan_state`, `stale`, and `last_error` instead of blocking the UI on full rescans.
|
||||
- Explorer/preflight rendering should keep actionable missing-node/model failures separate from suppressed inactive-branch findings returned by the backend.
|
||||
|
||||
@@ -83,7 +83,7 @@ If `assist_streaming` is unavailable or the stream transport degrades, Planner/R
|
||||
3. Confirm the sidebar host-surface metadata resolves correctly for the current environment instead of defaulting silently.
|
||||
4. Planner: click **Plan Generation** with minimal input and confirm either live preview/stage updates appear (when streaming is supported) or a readable fallback result/error appears.
|
||||
5. Refiner: click **Refine Prompts** (with or without image) and confirm either live preview/stage updates appear (when streaming is supported) or a readable fallback result/error appears.
|
||||
6. Jobs: verify output previews still resolve for both classic history refs and any asset-backed refs surfaced by callback/history payloads, that asset-service-only refs stay explicit as a bounded fallback state, and that repeated polls do not duplicate rows after reconnect/resume.
|
||||
6. Jobs: verify output previews still resolve for classic history refs, hash-backed refs, and supported media-aware refs (`images`, `video`, `audio`, `3d`, bounded `text`); asset-service-only refs should stay explicit as a bounded fallback state, and repeated polls should not duplicate rows after reconnect/resume.
|
||||
7. Explorer: verify preflight inventory can show `refreshing` / `stale` / `error` state without freezing the tab while deep scan work continues, and verify inactive-branch suppressed findings render separately from actionable failures.
|
||||
8. Library/Approvals: if backend endpoints are not enabled, confirm the UI shows a clear error state (no crashes).
|
||||
9. If you simulate/fake a stream failure in dev tools, confirm Planner/Refiner retry through the classic non-stream path without duplicate submits or broken loading state.
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
openapi: "3.0.3"
|
||||
info:
|
||||
title: "ComfyUI-OpenClaw API"
|
||||
version: "1.0.11"
|
||||
version: "1.0.12"
|
||||
description: "Generated from docs/release/api_contract.md."
|
||||
servers:
|
||||
- url: "/openclaw"
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# OpenClaw API Contract (v1)
|
||||
|
||||
> **Status**: normative
|
||||
> **Version**: 1.0.11
|
||||
> **Date**: 2026-06-04
|
||||
> **Version**: 1.0.12
|
||||
> **Date**: 2026-06-12
|
||||
|
||||
This document defines the public API contract for OpenClaw. It serves as the authoritative baseline for client compatibility and breaking change policies.
|
||||
|
||||
@@ -75,6 +75,15 @@ Preflight workflow diagnostics contract:
|
||||
- suppressed findings represent muted or bypassed root nodes or subgraph branches when the submitted workflow shape provides enough frontend ancestry metadata
|
||||
- clients SHOULD display suppressed findings as informational context rather than blocking workflow readiness
|
||||
|
||||
History and output-ref contract:
|
||||
|
||||
- history/output consumers SHOULD treat the normalized output-ref contract as media-aware
|
||||
- current previewable output groups are `images`, `video`, `audio`, `3d`, and bounded `text`
|
||||
- file-like refs that can be represented through `/view` remain on the bounded `/history` + `/view` preview path
|
||||
- refs with `asset_hash` or `hash` values that map to `blake3:...` preview through `/view?filename=blake3:...`
|
||||
- refs that only expose upstream asset-service identifiers remain explicit `asset_api_required` states; clients MUST NOT silently infer direct `/api/assets` fetching from that marker
|
||||
- legacy callback/image-only consumers may continue using image-only extraction paths; non-image media refs should be rendered as explicit fallback/link/text surfaces unless the client implements a safe media-specific renderer
|
||||
|
||||
### 1.2 Webhooks & Triggers
|
||||
|
||||
**Auth**: Requires configured webhook secret or Admin Token.
|
||||
@@ -86,6 +95,13 @@ Preflight workflow diagnostics contract:
|
||||
| `POST` | `/webhook/validate` | `/moltbot/webhook/validate` | Webhook Secret | Dry-run validation of webhook payload. |
|
||||
| `POST` | `/triggers/fire` | `/moltbot/triggers/fire` | Admin | Fire an ad-hoc workflow trigger from external system. |
|
||||
|
||||
ComfyUI prompt submission interoperability:
|
||||
|
||||
- OpenClaw-generated ComfyUI `/prompt` payloads include `extra_data.comfy_usage_source = "comfyui-openclaw"` when the caller has not supplied a value
|
||||
- caller-provided `extra_data.comfy_usage_source` is preserved
|
||||
- attribution is a stable product identifier and MUST NOT include prompt text, tenant ids, trace ids, URLs, tokens, or secrets
|
||||
- existing `extra_data.openclaw` and legacy `extra_data.moltbot` metadata remain caller-owned except for OpenClaw tenant metadata insertion under `extra_data.openclaw.tenant_id`
|
||||
|
||||
### 1.3 Assist, LLM & Chat
|
||||
|
||||
**Assist Base Path**: `/openclaw/assist/`
|
||||
|
||||
@@ -19,13 +19,14 @@ Newest entries appear first.
|
||||
|
||||
<details>
|
||||
|
||||
<summary><strong>ComfyUI host compatibility, queue recovery, model folders, and asset-output posture refreshed</strong></summary>
|
||||
<summary><strong>ComfyUI host compatibility, media outputs, model folders, and prompt attribution refreshed</strong></summary>
|
||||
|
||||
- Refreshed the published compatibility baseline for ComfyUI `822aca19` (`v0.24.0-60-g822aca19`, pyproject `0.24.0`), standalone frontend `1.46.13`, and Desktop `0.9.4` with core `0.22.3` plus embedded frontend `1.43.18`.
|
||||
- Reconciled active prompt state after backend or SSE reconnects so completed prompts are not left in the active queue lane after a host recovery.
|
||||
- Updated sidebar registration to prefer the current ComfyUI sidebar store API and keep the deprecated frontend facade as a compatibility fallback for older hosts.
|
||||
- Aligned Model Manager and preflight diagnostics with current ComfyUI model folder names such as `text_encoders`, `diffusion_models`, `geometry_estimation`, and `detection`, while retaining legacy aliases such as `clip` and `unet`.
|
||||
- Kept output previews on the bounded `/history` + `/view` contract, including `asset_hash` and `hash` aliases for hash-backed previews; upstream asset-only identifiers remain explicit fallback states unless a future feature requires direct `/api/assets` use.
|
||||
- Aligned Model Manager and preflight diagnostics with current ComfyUI model folder names, including newer managed keys such as `gligen`, `latent_upscale_models`, `hypernetworks`, `photomaker`, `model_patches`, `geometry_estimation`, and `detection`, while retaining legacy aliases such as `clip` and `unet`.
|
||||
- Made output parsing media-aware for current previewable result groups (`images`, `video`, `audio`, `3d`, and bounded `text`) while keeping image callbacks compatible and keeping asset-only identifiers as explicit fallback states instead of silently upgrading to `/api/assets`.
|
||||
- OpenClaw prompt submissions now include stable `comfy_usage_source` attribution when missing, without overwriting caller-provided attribution or copying prompt/tenant/trace content into that field.
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -91,7 +91,7 @@ Notes:
|
||||
- Runtime cache and sandbox scratch paths are generated state, not package resources.
|
||||
- OpenClaw does not automatically repair, migrate, or delete runtime dependency caches.
|
||||
|
||||
## Jobs preview shows an explicit asset fallback state instead of an image preview
|
||||
## Jobs preview shows an explicit media or asset fallback state
|
||||
|
||||
Current OpenClaw builds keep `/history` + `/view` as the supported runtime preview contract for job results.
|
||||
|
||||
@@ -101,6 +101,7 @@ What this means:
|
||||
|
||||
- `asset_api_required` is a bounded compatibility state, not a generic parser failure.
|
||||
- Classic history refs and hash-backed refs exposed as `asset_hash` or `hash` that still map onto `/view` should continue to preview normally.
|
||||
- Current media-aware outputs can include `images`, `video`, `audio`, `3d`, and bounded `text`; images render as thumbnails, text renders as escaped bounded text, and other file-like media may appear as explicit fallback/link tiles instead of image elements.
|
||||
- If an operator workflow starts depending on direct asset-service identifiers, treat that as a contract gap and review [`docs/asset_api_adoption_decision.md`](asset_api_adoption_decision.md) before widening the runtime dependency.
|
||||
|
||||
## Verify audit-chain continuity after restart or rotation
|
||||
|
||||
Reference in New Issue
Block a user