diff --git a/README.md b/README.md index 647543c..7353f4d 100644 --- a/README.md +++ b/README.md @@ -91,10 +91,11 @@ Deployment profiles and hardening references:
-Connector media safety and HDR output fallbacks tightened +Connector media safety, HDR fallbacks, and graph guards tightened - LINE and WhatsApp connector media URLs now force dangerous active content such as SVG/HTML/JS/CSS/XML to download with no-sniff response headers while preserving safe image delivery. - Job Monitor now treats HDR `.exr` and `.hdr` image outputs as explicit source-preview fallback links instead of normal thumbnails, matching the current host expectation without bundling a HDR viewer. +- Parameter Lab and graph-helper coverage now preserve non-numeric node IDs and promoted-widget source metadata, while structured color/box widget inputs stay out of missing-model diagnostics.
diff --git a/docs/frontend_ux_walkthrough.md b/docs/frontend_ux_walkthrough.md index 1738811..f49a37a 100644 --- a/docs/frontend_ux_walkthrough.md +++ b/docs/frontend_ux_walkthrough.md @@ -24,6 +24,7 @@ Refactor note: - New tab markup should use canonical `openclaw-*` classes; legacy `moltbot-*` aliases are generated centrally at runtime instead of being duplicated in each template. - New host sidebar registration changes should stay in `web/openclaw_sidebar_registration.js` rather than duplicating ComfyUI frontend API detection inside the extension entrypoint. - Host-sensitive behaviors should consume the shared host-surface helper rather than inferring desktop vs standalone frontend from ad-hoc globals. +- Graph/widget flows should preserve host-shaped promoted-widget source metadata and non-numeric node IDs, including Parameter Lab replay/apply paths. - Output preview flows should consume the shared asset-ref normalizer rather than assembling `/view` URLs independently in each tab, treating non-image or HDR media as broken images, or silently widening runtime behavior to direct `/api/assets` fetches. - Explorer/preflight consumers should treat inventory diagnostics as snapshot-first and surface `snapshot_ts`, `scan_state`, `stale`, and `last_error` instead of blocking the UI on full rescans. - Explorer/preflight rendering should keep actionable missing-node/model failures separate from suppressed inactive-branch findings returned by the backend. @@ -47,7 +48,7 @@ If `assist_streaming` is unavailable or the stream transport degrades, Planner/R - OpenClaw treats standalone `ComfyUI_frontend` and `desktop` as distinct frontend host surfaces. - The sidebar stamps its resolved host surface and refreshed host-reference metadata at mount time so desktop bundle drift is explicit in diagnostics and regression tests. - The standalone Remote Admin Console now stamps the same host-surface metadata on its document root, including desktop `0.9.4`, bundled core `0.22.3`, embedded frontend `1.43.18`, and lagging parity relative to standalone frontend `1.47.6`. -- Graph/widget compatibility code should route through shared host helpers to keep nested-subgraph and promoted-widget behavior aligned with current upstream host semantics. +- Graph/widget compatibility code should route through shared host helpers to keep nested-subgraph and promoted-widget behavior aligned with current upstream host semantics, including preserving source metadata and string-shaped node IDs. ## Standalone Remote Admin Console diff --git a/docs/release/compatibility_matrix.md b/docs/release/compatibility_matrix.md index 06f41bb..5593b2e 100644 --- a/docs/release/compatibility_matrix.md +++ b/docs/release/compatibility_matrix.md @@ -45,6 +45,7 @@ This document tracks the current reference anchors and validated environments fo - **SaveImage output refs**: OpenClaw consumes runtime `/history` output refs and does not infer graph-rewrite behavior from output-node socket shape. `SaveImage` output sockets are allowed to exist without changing the normalized output-ref contract. - **3D output refs**: `Load3DAdvanced` and related 3D preview refs remain media-aware output refs. File-like refs and optional hash-backed 3D refs stay on the bounded `/view` preview contract; clients without a 3D renderer should show an explicit fallback/link surface. - **HDR image output refs**: `.exr` and `.hdr` image refs stay on the bounded `/view` source-preview contract but render as explicit fallback/link surfaces unless a client implements a safe HDR-specific viewer. +- **Promoted widget source scope and structured widgets**: OpenClaw graph helpers preserve host-shaped promoted-widget source metadata and keep non-numeric node IDs stable. Backend preflight remains a conservative model-key whitelist; structured `COLORS` / `BOUNDING_BOXES` inputs and frontend source metadata are not treated as model references, and OpenClaw does not claim full host frontend active-scope parity without a richer graph-instance contract. - **Asset dimensions and grouped assets**: typed width/height metadata and grouped multi-download behavior are host-frontend display/download concerns. They do not change OpenClaw fetch routing, and asset-service-only identifiers remain explicit `asset_api_required` states rather than implicit `/api/assets` fetches. - **Sidebar registration**: prefer the current `sidebarTab.registerSidebarTab` host API and retain the deprecated `extensionManager.registerSidebarTab` fallback for older or desktop-embedded frontend hosts. - **Node runtime policy**: the standalone ComfyUI frontend development workspace currently declares `node >=25 <26` and `pnpm >=11.3`, but OpenClaw keeps its package engine at `>=18.0.0` because this custom-node package runs its own Playwright/Vitest harness and does not build the host frontend workspace. OpenClaw acceptance remains governed by `tests/TEST_SOP.md` and `tests/E2E_TESTING_SOP.md`, which require Node.js 18+ and CI-parity validation on the project test harness. diff --git a/docs/release/recent_updates.md b/docs/release/recent_updates.md index a203d03..a8ce0ed 100644 --- a/docs/release/recent_updates.md +++ b/docs/release/recent_updates.md @@ -7,10 +7,11 @@ Newest entries appear first.
-Connector media safety and HDR output fallbacks tightened +Connector media safety, HDR fallbacks, and graph guards tightened - LINE and WhatsApp connector media URLs now force dangerous active content such as SVG/HTML/JS/CSS/XML to download with no-sniff response headers while preserving safe image delivery. - Job Monitor now treats HDR `.exr` and `.hdr` image outputs as explicit source-preview fallback links instead of normal thumbnails, matching the current host expectation without bundling a HDR viewer. +- Parameter Lab and graph-helper coverage now preserve non-numeric node IDs and promoted-widget source metadata, while structured color/box widget inputs stay out of missing-model diagnostics.
diff --git a/tests/e2e/specs/parameter_lab.spec.js b/tests/e2e/specs/parameter_lab.spec.js index 70dcc8c..7429b6e 100644 --- a/tests/e2e/specs/parameter_lab.spec.js +++ b/tests/e2e/specs/parameter_lab.spec.js @@ -141,6 +141,91 @@ test.describe('Parameter Lab - Dynamic Dimensions', () => { }); }); + test('preserves non-numeric node ids through payload generation and replay apply', async ({ page }) => { + await page.evaluate(() => { + window.app.graph = { + _nodes: [ + { + id: "loader-alpha", + type: "CheckpointLoader", + title: "String Loader", + widgets: [ + { + name: "ckpt_name", + type: "combo", + value: "base.ckpt", + options: { values: ["base.ckpt", "xl.ckpt"] } + } + ] + } + ], + getNodeById(id) { + return this._nodes.find((node) => String(node.id) === String(id)); + }, + serialize() { return { "string_graph": true }; } + }; + window.confirm = () => true; + }); + + await page.evaluate(async () => { + const mod = await import('/web/openclaw_api.js'); + window.__labSweepPayload = null; + + const originalFetch = mod.openclawApi.fetch.bind(mod.openclawApi); + mod.openclawApi.fetch = async (url, options = {}) => { + const normalizedPath = String(url || '').replace(/^\/moltbot/, '/openclaw'); + if (normalizedPath.endsWith('/lab/sweep')) { + const payload = JSON.parse(options?.body || '{}'); + window.__labSweepPayload = payload; + return { + ok: true, + status: 200, + data: { + plan: { + experiment_id: 'exp_string_ids', + dimensions: payload.params, + runs: [ + { "loader-alpha.ckpt_name": "xl.ckpt" } + ] + } + } + }; + } + return originalFetch(url, options); + }; + }); + + await page.click('#lab-add-dim'); + await expect(page.locator('.dim-node-select option[value="loader-alpha"]')).toHaveText('[loader-alpha] String Loader'); + await page.selectOption('.dim-node-select', { value: 'loader-alpha' }); + await page.selectOption('.dim-widget-select', { value: 'ckpt_name' }); + await page.selectOption('.dim-candidate-select', { value: 'xl.ckpt' }); + + await page.click('#lab-generate'); + await expect + .poll(() => page.evaluate(() => (window.__labSweepPayload ? 'ready' : 'pending'))) + .toBe('ready'); + + const payload = await page.evaluate(() => window.__labSweepPayload); + expect(payload.params[0]).toEqual({ + node_id: 'loader-alpha', + widget_name: 'ckpt_name', + values: ['xl.ckpt'], + strategy: 'grid' + }); + + await page.click('.replay-run'); + await expect + .poll(() => + page.evaluate(() => + window.app.graph + .getNodeById('loader-alpha') + .widgets.find((widget) => widget.name === 'ckpt_name').value + ) + ) + .toBe('xl.ckpt'); + }); + test('supports nested subgraph nodes and promoted widget candidates', async ({ page }) => { await page.evaluate(() => { const nestedLoader = { diff --git a/tests/test_parameter_lab.py b/tests/test_parameter_lab.py index ff4b6b5..38b295b 100644 --- a/tests/test_parameter_lab.py +++ b/tests/test_parameter_lab.py @@ -56,6 +56,22 @@ class TestSweepPlanner(unittest.TestCase): self.assertEqual(plan1.runs, plan2.runs) self.assertEqual(plan1.runs[0], {"1.steps": 10}) + def test_generate_preserves_non_numeric_node_id_keys(self): + planner = SweepPlanner() + plan = planner.generate( + workflow='{"nodes":[]}', + params=[ + { + "node_id": "loader-alpha", + "widget_name": "ckpt_name", + "values": ["xl.ckpt"], + }, + ], + ) + + self.assertEqual(plan.dimensions[0].node_id, "loader-alpha") + self.assertEqual(plan.runs, [{"loader-alpha.ckpt_name": "xl.ckpt"}]) + class TestComparePlanner(unittest.TestCase): # ... existing tests ... diff --git a/tests/test_preflight.py b/tests/test_preflight.py index bea01d2..62107ee 100644 --- a/tests/test_preflight.py +++ b/tests/test_preflight.py @@ -161,6 +161,55 @@ class TestPreflightBackend(AioHTTPTestCase): "detection", ) + def test_structured_widget_metadata_does_not_create_missing_model_diagnostics( + self, + ): + services.preflight._CACHE.clear() + + with ( + patch.object( + services.preflight, "nodes", MagicMock(), create=True + ) as mock_nodes, + patch.object( + services.preflight, "folder_paths", MagicMock(), create=True + ) as mock_folder_paths, + ): + mock_nodes.NODE_CLASS_MAPPINGS = { + "ColorAndBoxNode": object, + "PromptMetadataNode": object, + } + mock_folder_paths.folder_names_and_paths = {} + mock_folder_paths.get_filename_list.return_value = [] + + workflow = { + "loader-alpha": { + "class_type": "ColorAndBoxNode", + "inputs": { + "colors": ["#ff0000", "#00ff00"], + "COLORS": ["#ff0000"], + "bounding_boxes": [{"x": 1, "y": 2, "width": 3, "height": 4}], + "BOUNDING_BOXES": { + "boxes": [{"x": 0, "y": 0, "width": 8, "height": 8}] + }, + "sourceExecutionId": "host-pack:source-loader", + }, + }, + "metadata-node": { + "class_type": "PromptMetadataNode", + "inputs": { + "sourceExecutionId": "host-pack:source-loader", + "widgetName": "palette", + "nodeId": "loader-alpha", + }, + }, + } + + report = services.preflight.run_preflight_check(workflow) + + self.assertTrue(report["ok"]) + self.assertEqual(report["summary"]["missing_models"], 0) + self.assertEqual(report["missing_models"], []) + def test_inventory_model_types_track_current_comfyui_keys_and_exclude_custom_nodes( self, ): diff --git a/web/openclaw_graph_host.js b/web/openclaw_graph_host.js index ac7f584..9418aea 100644 --- a/web/openclaw_graph_host.js +++ b/web/openclaw_graph_host.js @@ -268,6 +268,7 @@ export function getGraphWidgetCatalog(graph, nodeId) { options: widget?.options, isPromoted: isPromotedWidgetView(widget), sourceNodeId: widget?.sourceNodeId, + sourceExecutionId: widget?.sourceExecutionId, sourceWidgetName: widget?.sourceWidgetName, resolvedNodeId: resolved?.nodeEntry?.id || entry.id, resolvedWidgetName: resolved?.widget?.name || widgetName, @@ -320,6 +321,7 @@ export function findComparableWidget(graph, nodeRefOrId) { widget: resolved.widget, hostWidgetName: widgetName, nodeId: resolved.nodeEntry.id, + sourceExecutionId: widget?.sourceExecutionId, widgetName: resolved.widget.name || widgetName, }; } diff --git a/web/tabs/parameter_lab_tab.js b/web/tabs/parameter_lab_tab.js index 695f14e..3fe1797 100644 --- a/web/tabs/parameter_lab_tab.js +++ b/web/tabs/parameter_lab_tab.js @@ -7,6 +7,7 @@ import { getGraphNodeCatalog, getGraphWidgetCatalog, getGraphWidgetValueCandidates, + resolveGraphWidget, } from "../openclaw_graph_host.js"; import { openclawUI } from "../openclaw_ui.js"; @@ -747,13 +748,23 @@ export const ParameterLabTab = { applyOverrides(run) { Object.entries(run).forEach(([key, value]) => { if (key === "prompt_id" || key === "status") return; - const [nodeId, widgetName] = key.split("."); - const node = app.graph.getNodeById(parseInt(nodeId)); - if (node) { - const widget = node.widgets.find(w => w.name === widgetName); - if (widget) { - widget.value = value; - } + const separatorIndex = key.indexOf("."); + if (separatorIndex <= 0) return; + const nodeId = key.slice(0, separatorIndex); + const widgetName = key.slice(separatorIndex + 1); + const resolved = resolveGraphWidget(app.graph, nodeId, widgetName); + const fallbackNode = app.graph.getNodeById?.(this._coerceSelectedNodeId(nodeId)); + const node = resolved?.node || fallbackNode; + if (!node) { + return; + } + const widget = + resolved?.widget || + (Array.isArray(node.widgets) + ? node.widgets.find((entry) => entry.name === widgetName) + : null); + if (widget) { + widget.value = value; } }); } diff --git a/web/tests/unit/openclaw_graph_host.test.js b/web/tests/unit/openclaw_graph_host.test.js index 50c62c7..a0e1bee 100644 --- a/web/tests/unit/openclaw_graph_host.test.js +++ b/web/tests/unit/openclaw_graph_host.test.js @@ -86,6 +86,77 @@ function createGraphFixture() { }; } +function createHostShapedGraphFixture() { + const nestedLoader = { + id: "source-loader", + type: "CheckpointLoaderSimple", + title: "Nested String Loader", + widgets: [ + { + name: "ckpt_name", + type: "combo", + value: "base.ckpt", + options: { values: ["base.ckpt", "xl.ckpt"] }, + }, + ], + }; + const subgraph = { + _nodes: [nestedLoader], + getNodeById(id) { + return this._nodes.find((node) => String(node.id) === String(id)); + }, + }; + const host = { + id: "host-pack", + type: "SubgraphNode", + title: "String Workflow Pack", + widgets: [ + { + name: "ckpt_name", + type: "combo", + value: "base.ckpt", + options: {}, + sourceNodeId: "source-loader", + sourceExecutionId: "host-pack:source-loader", + sourceWidgetName: "ckpt_name", + }, + ], + subgraph, + }; + const stringNode = { + id: "loader-alpha", + type: "ColorAndBoxNode", + title: "String Node", + widgets: [ + { + name: "palette", + type: "COLORS", + value: ["#ff0000", "#00ff00"], + options: {}, + }, + { + name: "regions", + type: "BOUNDING_BOXES", + value: [{ x: 1, y: 2, width: 3, height: 4 }], + options: {}, + }, + { + name: "ckpt_name", + type: "combo", + value: "base.ckpt", + options: { values: ["base.ckpt", "xl.ckpt"] }, + }, + ], + }; + + return { + _nodes: [host, stringNode], + getNodeById(id) { + return this._nodes.find((node) => String(node.id) === String(id)); + }, + }; +} + describe("openclaw_graph_host", () => { it("builds catalog entries for nested subgraph nodes", () => { const graph = createGraphFixture(); @@ -120,4 +191,41 @@ describe("openclaw_graph_host", () => { expect(resolved?.nodeEntry.id).toBe("50:7"); expect(resolved?.widget.name).toBe("ckpt_name"); }); + + it("preserves host-shaped sourceExecutionId metadata for promoted widgets", () => { + const graph = createHostShapedGraphFixture(); + const widgetCatalog = getGraphWidgetCatalog(graph, "host-pack"); + const promotedWidget = widgetCatalog.find((entry) => entry.name === "ckpt_name"); + const resolved = resolveGraphWidget(graph, "host-pack", "ckpt_name"); + + expect(promotedWidget).toMatchObject({ + isPromoted: true, + sourceNodeId: "source-loader", + sourceExecutionId: "host-pack:source-loader", + sourceWidgetName: "ckpt_name", + resolvedNodeId: "host-pack:source-loader", + resolvedWidgetName: "ckpt_name", + }); + expect(resolved?.nodeEntry.id).toBe("host-pack:source-loader"); + expect(getGraphWidgetValueCandidates(graph, "host-pack", "ckpt_name")).toEqual([ + "base.ckpt", + "xl.ckpt", + ]); + }); + + it("keeps non-numeric node ids stable and catalogs new structured widget types", () => { + const graph = createHostShapedGraphFixture(); + const catalog = getGraphNodeCatalog(graph); + const widgetCatalog = getGraphWidgetCatalog(graph, "loader-alpha"); + const compareTarget = findComparableWidget(graph, "loader-alpha"); + + expect(catalog.map((entry) => entry.id)).toContain("loader-alpha"); + expect(widgetCatalog.map((entry) => [entry.name, entry.type])).toEqual([ + ["palette", "COLORS"], + ["regions", "BOUNDING_BOXES"], + ["ckpt_name", "combo"], + ]); + expect(resolveGraphWidget(graph, "loader-alpha", "palette")?.nodeEntry.id).toBe("loader-alpha"); + expect(compareTarget?.nodeId).toBe("loader-alpha"); + }); });