fix(ui): preserve host-shaped widget ids

This commit is contained in:
rookiestar28
2026-07-08 03:40:02 +08:00
parent 6a35056631
commit ed9c2bcda4
10 changed files with 285 additions and 10 deletions
+2 -1
View File
@@ -91,10 +91,11 @@ Deployment profiles and hardening references:
<details>
<summary><strong>Connector media safety and HDR output fallbacks tightened</strong></summary>
<summary><strong>Connector media safety, HDR fallbacks, and graph guards tightened</strong></summary>
- LINE and WhatsApp connector media URLs now force dangerous active content such as SVG/HTML/JS/CSS/XML to download with no-sniff response headers while preserving safe image delivery.
- Job Monitor now treats HDR `.exr` and `.hdr` image outputs as explicit source-preview fallback links instead of normal thumbnails, matching the current host expectation without bundling a HDR viewer.
- Parameter Lab and graph-helper coverage now preserve non-numeric node IDs and promoted-widget source metadata, while structured color/box widget inputs stay out of missing-model diagnostics.
</details>
+2 -1
View File
@@ -24,6 +24,7 @@ Refactor note:
- New tab markup should use canonical `openclaw-*` classes; legacy `moltbot-*` aliases are generated centrally at runtime instead of being duplicated in each template.
- New host sidebar registration changes should stay in `web/openclaw_sidebar_registration.js` rather than duplicating ComfyUI frontend API detection inside the extension entrypoint.
- Host-sensitive behaviors should consume the shared host-surface helper rather than inferring desktop vs standalone frontend from ad-hoc globals.
- Graph/widget flows should preserve host-shaped promoted-widget source metadata and non-numeric node IDs, including Parameter Lab replay/apply paths.
- Output preview flows should consume the shared asset-ref normalizer rather than assembling `/view` URLs independently in each tab, treating non-image or HDR media as broken images, or silently widening runtime behavior to direct `/api/assets` fetches.
- Explorer/preflight consumers should treat inventory diagnostics as snapshot-first and surface `snapshot_ts`, `scan_state`, `stale`, and `last_error` instead of blocking the UI on full rescans.
- Explorer/preflight rendering should keep actionable missing-node/model failures separate from suppressed inactive-branch findings returned by the backend.
@@ -47,7 +48,7 @@ If `assist_streaming` is unavailable or the stream transport degrades, Planner/R
- OpenClaw treats standalone `ComfyUI_frontend` and `desktop` as distinct frontend host surfaces.
- The sidebar stamps its resolved host surface and refreshed host-reference metadata at mount time so desktop bundle drift is explicit in diagnostics and regression tests.
- The standalone Remote Admin Console now stamps the same host-surface metadata on its document root, including desktop `0.9.4`, bundled core `0.22.3`, embedded frontend `1.43.18`, and lagging parity relative to standalone frontend `1.47.6`.
- Graph/widget compatibility code should route through shared host helpers to keep nested-subgraph and promoted-widget behavior aligned with current upstream host semantics.
- Graph/widget compatibility code should route through shared host helpers to keep nested-subgraph and promoted-widget behavior aligned with current upstream host semantics, including preserving source metadata and string-shaped node IDs.
## Standalone Remote Admin Console
+1
View File
@@ -45,6 +45,7 @@ This document tracks the current reference anchors and validated environments fo
- **SaveImage output refs**: OpenClaw consumes runtime `/history` output refs and does not infer graph-rewrite behavior from output-node socket shape. `SaveImage` output sockets are allowed to exist without changing the normalized output-ref contract.
- **3D output refs**: `Load3DAdvanced` and related 3D preview refs remain media-aware output refs. File-like refs and optional hash-backed 3D refs stay on the bounded `/view` preview contract; clients without a 3D renderer should show an explicit fallback/link surface.
- **HDR image output refs**: `.exr` and `.hdr` image refs stay on the bounded `/view` source-preview contract but render as explicit fallback/link surfaces unless a client implements a safe HDR-specific viewer.
- **Promoted widget source scope and structured widgets**: OpenClaw graph helpers preserve host-shaped promoted-widget source metadata and keep non-numeric node IDs stable. Backend preflight remains a conservative model-key whitelist; structured `COLORS` / `BOUNDING_BOXES` inputs and frontend source metadata are not treated as model references, and OpenClaw does not claim full host frontend active-scope parity without a richer graph-instance contract.
- **Asset dimensions and grouped assets**: typed width/height metadata and grouped multi-download behavior are host-frontend display/download concerns. They do not change OpenClaw fetch routing, and asset-service-only identifiers remain explicit `asset_api_required` states rather than implicit `/api/assets` fetches.
- **Sidebar registration**: prefer the current `sidebarTab.registerSidebarTab` host API and retain the deprecated `extensionManager.registerSidebarTab` fallback for older or desktop-embedded frontend hosts.
- **Node runtime policy**: the standalone ComfyUI frontend development workspace currently declares `node >=25 <26` and `pnpm >=11.3`, but OpenClaw keeps its package engine at `>=18.0.0` because this custom-node package runs its own Playwright/Vitest harness and does not build the host frontend workspace. OpenClaw acceptance remains governed by `tests/TEST_SOP.md` and `tests/E2E_TESTING_SOP.md`, which require Node.js 18+ and CI-parity validation on the project test harness.
+2 -1
View File
@@ -7,10 +7,11 @@ Newest entries appear first.
<details>
<summary><strong>Connector media safety and HDR output fallbacks tightened</strong></summary>
<summary><strong>Connector media safety, HDR fallbacks, and graph guards tightened</strong></summary>
- LINE and WhatsApp connector media URLs now force dangerous active content such as SVG/HTML/JS/CSS/XML to download with no-sniff response headers while preserving safe image delivery.
- Job Monitor now treats HDR `.exr` and `.hdr` image outputs as explicit source-preview fallback links instead of normal thumbnails, matching the current host expectation without bundling a HDR viewer.
- Parameter Lab and graph-helper coverage now preserve non-numeric node IDs and promoted-widget source metadata, while structured color/box widget inputs stay out of missing-model diagnostics.
</details>
+85
View File
@@ -141,6 +141,91 @@ test.describe('Parameter Lab - Dynamic Dimensions', () => {
});
});
test('preserves non-numeric node ids through payload generation and replay apply', async ({ page }) => {
await page.evaluate(() => {
window.app.graph = {
_nodes: [
{
id: "loader-alpha",
type: "CheckpointLoader",
title: "String Loader",
widgets: [
{
name: "ckpt_name",
type: "combo",
value: "base.ckpt",
options: { values: ["base.ckpt", "xl.ckpt"] }
}
]
}
],
getNodeById(id) {
return this._nodes.find((node) => String(node.id) === String(id));
},
serialize() { return { "string_graph": true }; }
};
window.confirm = () => true;
});
await page.evaluate(async () => {
const mod = await import('/web/openclaw_api.js');
window.__labSweepPayload = null;
const originalFetch = mod.openclawApi.fetch.bind(mod.openclawApi);
mod.openclawApi.fetch = async (url, options = {}) => {
const normalizedPath = String(url || '').replace(/^\/moltbot/, '/openclaw');
if (normalizedPath.endsWith('/lab/sweep')) {
const payload = JSON.parse(options?.body || '{}');
window.__labSweepPayload = payload;
return {
ok: true,
status: 200,
data: {
plan: {
experiment_id: 'exp_string_ids',
dimensions: payload.params,
runs: [
{ "loader-alpha.ckpt_name": "xl.ckpt" }
]
}
}
};
}
return originalFetch(url, options);
};
});
await page.click('#lab-add-dim');
await expect(page.locator('.dim-node-select option[value="loader-alpha"]')).toHaveText('[loader-alpha] String Loader');
await page.selectOption('.dim-node-select', { value: 'loader-alpha' });
await page.selectOption('.dim-widget-select', { value: 'ckpt_name' });
await page.selectOption('.dim-candidate-select', { value: 'xl.ckpt' });
await page.click('#lab-generate');
await expect
.poll(() => page.evaluate(() => (window.__labSweepPayload ? 'ready' : 'pending')))
.toBe('ready');
const payload = await page.evaluate(() => window.__labSweepPayload);
expect(payload.params[0]).toEqual({
node_id: 'loader-alpha',
widget_name: 'ckpt_name',
values: ['xl.ckpt'],
strategy: 'grid'
});
await page.click('.replay-run');
await expect
.poll(() =>
page.evaluate(() =>
window.app.graph
.getNodeById('loader-alpha')
.widgets.find((widget) => widget.name === 'ckpt_name').value
)
)
.toBe('xl.ckpt');
});
test('supports nested subgraph nodes and promoted widget candidates', async ({ page }) => {
await page.evaluate(() => {
const nestedLoader = {
+16
View File
@@ -56,6 +56,22 @@ class TestSweepPlanner(unittest.TestCase):
self.assertEqual(plan1.runs, plan2.runs)
self.assertEqual(plan1.runs[0], {"1.steps": 10})
def test_generate_preserves_non_numeric_node_id_keys(self):
planner = SweepPlanner()
plan = planner.generate(
workflow='{"nodes":[]}',
params=[
{
"node_id": "loader-alpha",
"widget_name": "ckpt_name",
"values": ["xl.ckpt"],
},
],
)
self.assertEqual(plan.dimensions[0].node_id, "loader-alpha")
self.assertEqual(plan.runs, [{"loader-alpha.ckpt_name": "xl.ckpt"}])
class TestComparePlanner(unittest.TestCase):
# ... existing tests ...
+49
View File
@@ -161,6 +161,55 @@ class TestPreflightBackend(AioHTTPTestCase):
"detection",
)
def test_structured_widget_metadata_does_not_create_missing_model_diagnostics(
self,
):
services.preflight._CACHE.clear()
with (
patch.object(
services.preflight, "nodes", MagicMock(), create=True
) as mock_nodes,
patch.object(
services.preflight, "folder_paths", MagicMock(), create=True
) as mock_folder_paths,
):
mock_nodes.NODE_CLASS_MAPPINGS = {
"ColorAndBoxNode": object,
"PromptMetadataNode": object,
}
mock_folder_paths.folder_names_and_paths = {}
mock_folder_paths.get_filename_list.return_value = []
workflow = {
"loader-alpha": {
"class_type": "ColorAndBoxNode",
"inputs": {
"colors": ["#ff0000", "#00ff00"],
"COLORS": ["#ff0000"],
"bounding_boxes": [{"x": 1, "y": 2, "width": 3, "height": 4}],
"BOUNDING_BOXES": {
"boxes": [{"x": 0, "y": 0, "width": 8, "height": 8}]
},
"sourceExecutionId": "host-pack:source-loader",
},
},
"metadata-node": {
"class_type": "PromptMetadataNode",
"inputs": {
"sourceExecutionId": "host-pack:source-loader",
"widgetName": "palette",
"nodeId": "loader-alpha",
},
},
}
report = services.preflight.run_preflight_check(workflow)
self.assertTrue(report["ok"])
self.assertEqual(report["summary"]["missing_models"], 0)
self.assertEqual(report["missing_models"], [])
def test_inventory_model_types_track_current_comfyui_keys_and_exclude_custom_nodes(
self,
):
+2
View File
@@ -268,6 +268,7 @@ export function getGraphWidgetCatalog(graph, nodeId) {
options: widget?.options,
isPromoted: isPromotedWidgetView(widget),
sourceNodeId: widget?.sourceNodeId,
sourceExecutionId: widget?.sourceExecutionId,
sourceWidgetName: widget?.sourceWidgetName,
resolvedNodeId: resolved?.nodeEntry?.id || entry.id,
resolvedWidgetName: resolved?.widget?.name || widgetName,
@@ -320,6 +321,7 @@ export function findComparableWidget(graph, nodeRefOrId) {
widget: resolved.widget,
hostWidgetName: widgetName,
nodeId: resolved.nodeEntry.id,
sourceExecutionId: widget?.sourceExecutionId,
widgetName: resolved.widget.name || widgetName,
};
}
+18 -7
View File
@@ -7,6 +7,7 @@ import {
getGraphNodeCatalog,
getGraphWidgetCatalog,
getGraphWidgetValueCandidates,
resolveGraphWidget,
} from "../openclaw_graph_host.js";
import { openclawUI } from "../openclaw_ui.js";
@@ -747,13 +748,23 @@ export const ParameterLabTab = {
applyOverrides(run) {
Object.entries(run).forEach(([key, value]) => {
if (key === "prompt_id" || key === "status") return;
const [nodeId, widgetName] = key.split(".");
const node = app.graph.getNodeById(parseInt(nodeId));
if (node) {
const widget = node.widgets.find(w => w.name === widgetName);
if (widget) {
widget.value = value;
}
const separatorIndex = key.indexOf(".");
if (separatorIndex <= 0) return;
const nodeId = key.slice(0, separatorIndex);
const widgetName = key.slice(separatorIndex + 1);
const resolved = resolveGraphWidget(app.graph, nodeId, widgetName);
const fallbackNode = app.graph.getNodeById?.(this._coerceSelectedNodeId(nodeId));
const node = resolved?.node || fallbackNode;
if (!node) {
return;
}
const widget =
resolved?.widget ||
(Array.isArray(node.widgets)
? node.widgets.find((entry) => entry.name === widgetName)
: null);
if (widget) {
widget.value = value;
}
});
}
+108
View File
@@ -86,6 +86,77 @@ function createGraphFixture() {
};
}
function createHostShapedGraphFixture() {
const nestedLoader = {
id: "source-loader",
type: "CheckpointLoaderSimple",
title: "Nested String Loader",
widgets: [
{
name: "ckpt_name",
type: "combo",
value: "base.ckpt",
options: { values: ["base.ckpt", "xl.ckpt"] },
},
],
};
const subgraph = {
_nodes: [nestedLoader],
getNodeById(id) {
return this._nodes.find((node) => String(node.id) === String(id));
},
};
const host = {
id: "host-pack",
type: "SubgraphNode",
title: "String Workflow Pack",
widgets: [
{
name: "ckpt_name",
type: "combo",
value: "base.ckpt",
options: {},
sourceNodeId: "source-loader",
sourceExecutionId: "host-pack:source-loader",
sourceWidgetName: "ckpt_name",
},
],
subgraph,
};
const stringNode = {
id: "loader-alpha",
type: "ColorAndBoxNode",
title: "String Node",
widgets: [
{
name: "palette",
type: "COLORS",
value: ["#ff0000", "#00ff00"],
options: {},
},
{
name: "regions",
type: "BOUNDING_BOXES",
value: [{ x: 1, y: 2, width: 3, height: 4 }],
options: {},
},
{
name: "ckpt_name",
type: "combo",
value: "base.ckpt",
options: { values: ["base.ckpt", "xl.ckpt"] },
},
],
};
return {
_nodes: [host, stringNode],
getNodeById(id) {
return this._nodes.find((node) => String(node.id) === String(id));
},
};
}
describe("openclaw_graph_host", () => {
it("builds catalog entries for nested subgraph nodes", () => {
const graph = createGraphFixture();
@@ -120,4 +191,41 @@ describe("openclaw_graph_host", () => {
expect(resolved?.nodeEntry.id).toBe("50:7");
expect(resolved?.widget.name).toBe("ckpt_name");
});
it("preserves host-shaped sourceExecutionId metadata for promoted widgets", () => {
const graph = createHostShapedGraphFixture();
const widgetCatalog = getGraphWidgetCatalog(graph, "host-pack");
const promotedWidget = widgetCatalog.find((entry) => entry.name === "ckpt_name");
const resolved = resolveGraphWidget(graph, "host-pack", "ckpt_name");
expect(promotedWidget).toMatchObject({
isPromoted: true,
sourceNodeId: "source-loader",
sourceExecutionId: "host-pack:source-loader",
sourceWidgetName: "ckpt_name",
resolvedNodeId: "host-pack:source-loader",
resolvedWidgetName: "ckpt_name",
});
expect(resolved?.nodeEntry.id).toBe("host-pack:source-loader");
expect(getGraphWidgetValueCandidates(graph, "host-pack", "ckpt_name")).toEqual([
"base.ckpt",
"xl.ckpt",
]);
});
it("keeps non-numeric node ids stable and catalogs new structured widget types", () => {
const graph = createHostShapedGraphFixture();
const catalog = getGraphNodeCatalog(graph);
const widgetCatalog = getGraphWidgetCatalog(graph, "loader-alpha");
const compareTarget = findComparableWidget(graph, "loader-alpha");
expect(catalog.map((entry) => entry.id)).toContain("loader-alpha");
expect(widgetCatalog.map((entry) => [entry.name, entry.type])).toEqual([
["palette", "COLORS"],
["regions", "BOUNDING_BOXES"],
["ckpt_name", "combo"],
]);
expect(resolveGraphWidget(graph, "loader-alpha", "palette")?.nodeEntry.id).toBe("loader-alpha");
expect(compareTarget?.nodeId).toBe("loader-alpha");
});
});