admin/airi
1
0
Fork 0
mirror of https://github.com/moeru-ai/airi.git synced 2026-08-14 00:48:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(.agents/skills): updated agent-browser-electron

Browse Source
This commit is contained in:
Neko Ayaka
2026-05-22 15:23:45 +08:00
parent 9d9cded60a
commit e53515ae8d
1 changed files with 120 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/agent-browser-electron/SKILL.md
+120 -35
View File
@@ -7,17 +7,20 @@ description: Use when Codex needs to inspect, debug, or automate an Electron app
## Overview
Inspect Electron renderer windows reliably when `agent-browser` alone is not enough to tell which CDP target maps to which `BrowserWindow`.
Inspect Electron renderer windows reliably when `agent-browser` alone is not enough to tell which CDP target maps to which visible app window.
Prefer raw CDP target discovery over guessing from `tab list`, then connect `agent-browser` directly to the exact renderer target you want.
Prefer raw CDP target discovery over guessing from `tab list`, then use `agent-browser --cdp <port> tab` stable tab IDs to interact with the renderer target you want.
## Why Raw CDP Discovery
Use raw CDP target discovery because `agent-browser` is operating as a convenience layer on top of Chrome DevTools Protocol, and that layer can hide or flatten details that matter in Electron.
Raw `/json/list` is the source of truth for target discovery because it exposes the browser's own target inventory without extra interpretation. In Electron, that matters because:
Raw `/json/list` is the source of truth for CDP target discovery because it exposes Chromium's target inventory without extra interpretation. It is not the same thing as Electron's `BrowserWindow.getAllWindows()`.
In Electron, that distinction matters because:
- multiple `BrowserWindow` instances can share the same URL
- some windows are created lazily and appear only after an app action
- a visible Electron window is inspectable only after its renderer/webContents exists and is exposed as a CDP target
- detached DevTools pages and workers add noise
- `agent-browser tab list` may show only a subset of targets or present them with reduced metadata
- session state inside `agent-browser` can keep you attached to a previous renderer unless you reset and verify
@@ -27,15 +30,43 @@ In practice, the higher-level `tab list` view is useful for quick browsing, but
- the title is empty or collapsed
- a chat or settings window exists in CDP but is not obvious in the simplified tab output
Use `curl http://127.0.0.1:<port>/json/list` first whenever correct target selection matters. Treat `agent-browser` as the interaction client after target discovery, not as the discovery source.
Use `curl http://127.0.0.1:<port>/json/list` first whenever correct target selection matters. Treat `agent-browser` as the interaction client after target discovery, not as the only discovery source.
If `/json/list` does not contain a window that the user says exists, do not assume `agent-browser` hid it. First consider that the window may not have been created yet, may not have loaded a renderer route yet, or may not currently be exposed as a CDP target. Trigger the window from the app UI or Electron main-process action, then enumerate `/json/list` again.
## Workflow
1. Ensure the Electron window exists.
1. Confirm the app exposes a CDP port.
The CDP port is project- and run-command-specific. Do not assume `9222`, `9250`, or any other value is universal. If an Electron process is running but no CDP port responds, tell the user to relaunch the app with the project's remote-debug environment variables or launch flags.
For AIRI stage-tamagotchi on POSIX shells:
```bash
APP_REMOTE_DEBUG=true APP_REMOTE_DEBUG_PORT=9250 pnpm dev:tamagotchi
```
For Windows PowerShell:
```powershell
$env:APP_REMOTE_DEBUG = "true"
$env:APP_REMOTE_DEBUG_PORT = "9250"
pnpm dev:tamagotchi
```
For Windows Git Bash:
```bash
APP_REMOTE_DEBUG=true APP_REMOTE_DEBUG_PORT=9250 pnpm dev:tamagotchi
```
Adjust the port to match what the user actually started. If the project uses a different mechanism, inspect its Electron launch code before giving command advice.
2. Ensure the Electron window exists.
If the app uses lazy window creation, `agent-browser` cannot inspect a window that has not been created yet. Open it from the app UI or trigger its Electron-side open handler first.
2. Inspect raw CDP targets instead of trusting `agent-browser --cdp <port> tab list`.
3. Inspect raw CDP targets instead of trusting `agent-browser --cdp <port> tab`.
```bash
curl -sS http://127.0.0.1:<port>/json/list
@@ -47,13 +78,13 @@ Read these fields:
- `type`
- `webSocketDebuggerUrl`
Use `/json/version` if you need the browser-level debugger URL:
Use `/json/version` if you need to confirm the port is a Chromium/Electron CDP endpoint or need the browser-level debugger URL:
```bash
curl -sS http://127.0.0.1:<port>/json/version
```
3. Match the target to the Electron window.
4. Match the target to the Electron window.
Common patterns:
- Distinct route: chat may be `http://localhost:5173/#/chat` while the main window is `http://localhost:5173/#/`.
@@ -61,32 +92,37 @@ Common patterns:
- Duplicate URLs: two windows may both report `http://localhost:5173/#/`; in that case use screenshots, snapshots, and Electron app knowledge to disambiguate.
- Hidden noise: worker targets and detached DevTools targets are not your app window.
4. Reset `agent-browser` session state before switching targets.
5. List the targets through `agent-browser` and switch with stable tab IDs.
```bash
agent-browser close --all
agent-browser --cdp <port> tab
agent-browser --cdp <port> tab t2
```
5. Connect directly to the target you want.
```bash
agent-browser connect <webSocketDebuggerUrl>
```
Do not use positional integers. If `agent-browser tab` prints `[t2]`, switch with `tab t2`.
6. Verify the target immediately.
```bash
agent-browser --cdp <port> get url
agent-browser --cdp <port> get title
agent-browser --cdp <port> snapshot -i
```
7. If stable tab switching is not enough, reset session state and reconnect.
```bash
agent-browser close --all
agent-browser connect <webSocketDebuggerUrl>
agent-browser get url
agent-browser get title
agent-browser snapshot -i
```
If needed:
```bash
agent-browser screenshot /tmp/electron-target.png --annotate
agent-browser console
agent-browser errors
agent-browser --cdp <port> screenshot /tmp/electron-target.png --annotate
agent-browser --cdp <port> console
agent-browser --cdp <port> errors
```
## Fast Triage
@@ -97,14 +133,23 @@ Use this order when the Electron app has multiple windows:
2. Find the renderer page target with the route or title you expect
3. Ignore `worker` targets unless the task is specifically about workers
4. Ignore `DevTools` page targets unless debugging DevTools itself
5. `agent-browser close --all`
6. `agent-browser connect <webSocketDebuggerUrl>`
7. `agent-browser get url`
8. `agent-browser snapshot -i`
5. `agent-browser --cdp <port> tab`
6. Switch with the stable tab ID, for example `agent-browser --cdp <port> tab t3`
7. `agent-browser --cdp <port> get url`
8. `agent-browser --cdp <port> snapshot -i`
9. If the expected route is missing, trigger the lazy window from the UI and repeat from step 1
## AIRI Example
In `apps/stage-tamagotchi`, the chat window is lazy-created and loaded with `/#/chat`. The main window loads `/#/`.
In `apps/stage-tamagotchi`, some windows are lazy-created. The main window loads `/#/`; settings loads `/#/settings`; chat loads `/#/chat`; BeatSync loads `/beat-sync.html`.
The `9250` examples below assume the app was started by the person running it with:
```bash
APP_REMOTE_DEBUG=true APP_REMOTE_DEBUG_PORT=9250 pnpm dev:tamagotchi
```
That port is not intrinsic to AIRI or Electron. It depends on the current command and environment. Most stage-tamagotchi code lives under `apps/stage-tamagotchi`; inspect that app's Electron startup and window code when the port, routes, or remote-debug behavior differ.
Relevant files:
- `apps/stage-tamagotchi/src/main/windows/chat/index.ts`
@@ -114,27 +159,67 @@ Relevant files:
That means:
- chat does not exist in CDP until something calls `chatWindow()`
- once created, raw CDP target discovery will show a page target whose URL is `http://localhost:5173/#/chat`
- the stable way to inspect chat is to connect `agent-browser` to that target's `webSocketDebuggerUrl`
- settings and chat can be opened from the main window controls
- once created, raw CDP target discovery will show page targets such as `http://localhost:5173/#/settings` and `http://localhost:5173/#/chat`
- the stable way to inspect a window is to enumerate raw CDP targets, map them to `agent-browser` stable tab IDs, switch, and verify with `get url`
Example:
```bash
curl -sS http://127.0.0.1:9250/json/list
agent-browser close --all
agent-browser connect ws://127.0.0.1:9250/devtools/page/<chat-target-id>
agent-browser get url
agent-browser snapshot -i
agent-browser --cdp 9250 tab
agent-browser --cdp 9250 tab t4
agent-browser --cdp 9250 get url
agent-browser --cdp 9250 snapshot -i
```
If settings or chat is missing, start from the main window:
```bash
agent-browser --cdp 9250 tab t2
agent-browser --cdp 9250 snapshot -i
# Click the main window control that opens the panel or launcher.
# In AIRI this is the arrow-up control in the bottom-right controls island.
# Re-snapshot after every click because refs are stale after UI changes.
agent-browser --cdp 9250 click '[i-solar\:alt-arrow-up-line-duotone]'
agent-browser --cdp 9250 snapshot -i
```
Then open the desired entry from the expanded controls and enumerate again:
```bash
agent-browser --cdp 9250 click '[i-solar\:settings-minimalistic-outline]'
agent-browser --cdp 9250 click '[i-solar\:chat-line-line-duotone]'
curl -sS http://127.0.0.1:9250/json/list
agent-browser --cdp 9250 tab
```
If the controls island is visible but the accessibility refs or CSS icon click do not expand it, inspect the Vue component as a diagnostic fallback. AIRI currently nests tooltip trigger buttons around icon buttons, so a CDP click can report success while landing on the wrapper instead of the Vue `ControlButton` listener. This fallback is for automation/debugging only; do not use it as evidence that end-user clicking works.
```bash
agent-browser --cdp 9250 eval '(() => {
const icon = document.querySelector("[i-solar\\:alt-arrow-up-line-duotone]")
let controls = icon.__vueParentComponent
for (let i = 0; i < 10; i++) controls = controls.parent
controls.devtoolsRawSetupState.expanded.value = true
controls.proxy.$nextTick()
return controls.devtoolsRawSetupState.expanded.value
})()'
```
Expected verification for chat:
- `agent-browser get url` returns `http://localhost:5173/#/chat`
- `agent-browser --cdp 9250 get url` returns `http://localhost:5173/#/chat`
- the snapshot exposes chat UI controls such as the message textbox or send button
Expected verification for settings:
- `agent-browser --cdp 9250 get url` returns `http://localhost:5173/#/settings`
- the snapshot exposes settings navigation or configuration controls
## Failure Modes
- `tab list` omits or flattens the target you need: use raw `/json/list`.
- `connect` appears to succeed but later commands still point at the old renderer: run `agent-browser close --all`, then reconnect and verify with `get url`.
- `agent-browser tab` omits or flattens the target you need: use raw `/json/list`.
- `/json/list` and browser-level target discovery both omit the window: the renderer target is not currently exposed. Trigger the window creation/loading path, then enumerate again.
- `connect <webSocketDebuggerUrl>` appears to succeed but later commands still point at another renderer: switch with `agent-browser --cdp <port> tab tN`, or run `agent-browser close --all`, reconnect, and verify with `get url`.
- multiple windows share the same URL: use the target title, annotated screenshots, and app code to correlate them.
- `eval` returns `{}` for object values: prefer `get url`, `get title`, `snapshot -i`, or primitive-only eval return values.
- no chat target appears: the window may not have been created yet.
- no chat or settings target appears: the window may not have been created yet, or may exist as an Electron object without an inspectable renderer target.