mirror of
https://github.com/gianfrancopiana/openclaw-autoresearch.git
synced 2026-08-14 00:48:06 +00:00
Port pi-autoresearch to OpenClaw
Restructure monolithic pi-autoresearch extension into modular OpenClaw plugin with proper tool cwd resolution via api.resolvePath and explicit cwd overrides for nested repo targeting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
2227029fa5
commit
9e651fcda8
@@ -0,0 +1,2 @@
|
||||
node_modules/
|
||||
.tmp-*.md
|
||||
@@ -1,6 +1,7 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Tobi Lutke, David Cortés
|
||||
Copyright (c) 2026 Gianfranco Piana
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
||||
@@ -1,134 +1,103 @@
|
||||
# pi-autoresearch — autonomous experiment loop for pi
|
||||
# openclaw-autoresearch
|
||||
|
||||
**[Install](#install)** · **[Usage](#usage)** · **[How it works](#how-it-works)**
|
||||
Autonomous experiment loop for any optimization target.
|
||||
|
||||
*Try an idea, measure it, keep what works, discard what doesn't, repeat forever.*
|
||||
Faithful OpenClaw port of [`davebcn87/pi-autoresearch`](https://github.com/davebcn87/pi-autoresearch).
|
||||
|
||||
Inspired by [karpathy/autoresearch](https://github.com/karpathy/autoresearch). Works for any optimization target: test speed, bundle size, LLM training, build times, Lighthouse scores.
|
||||
## How it works
|
||||
|
||||
---
|
||||
The agent runs a loop: edit code, run a benchmark, measure the result, keep or discard. Each iteration is logged. The loop runs autonomously until interrupted.
|
||||
|
||||

|
||||
Three tools drive the loop:
|
||||
|
||||
---
|
||||
|
||||
## What's included
|
||||
|
||||
| | |
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| **Extension** | Tools + live widget + `/autoresearch` dashboard |
|
||||
| **Skill** | Gathers what to optimize, writes session files, starts the loop |
|
||||
| `init_experiment` | Configures the session: name, primary metric, unit, direction (lower/higher). Re-calling starts a new segment. |
|
||||
| `run_experiment` | Executes a shell command, times it, captures stdout/stderr, returns pass/fail via exit code. |
|
||||
| `log_experiment` | Records the result. `keep` auto-commits to git. `discard`/`crash` log without committing. Tracks secondary metrics alongside the primary. |
|
||||
|
||||
### Extension tools
|
||||
Each tool also accepts an optional `cwd` so callers can target a nested repo explicitly instead of relying on the current session working directory.
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| `init_experiment` | One-time session config — name, metric, unit, direction |
|
||||
| `run_experiment` | Runs any command, times wall-clock duration, captures output |
|
||||
| `log_experiment` | Records result, auto-commits, updates widget and dashboard |
|
||||
|
||||
### UI
|
||||
|
||||
- **Status widget** — always visible above the editor: `🔬 autoresearch 12 runs 8 kept │ best: 42.3s`
|
||||
- **`/autoresearch`** — full results dashboard (`Ctrl+X` to toggle, `Escape` to close)
|
||||
|
||||
### Skill
|
||||
|
||||
`autoresearch-create` asks a few questions (or infers from context) about your goal, command, metric, and files in scope — then writes two files and starts the loop immediately:
|
||||
All state lives in four repo-root files:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `autoresearch.md` | Session document — objective, metrics, files in scope, what's been tried. A fresh agent can resume from this alone. |
|
||||
| `autoresearch.sh` | Benchmark script — pre-checks, runs the workload, outputs `METRIC name=number` lines. |
|
||||
|---|---|
|
||||
| `autoresearch.md` | Session doc: objective, metrics, files in scope, constraints, what's been tried. A fresh agent reads this to resume. |
|
||||
| `autoresearch.sh` | Benchmark script. Outputs `METRIC name=number` lines. |
|
||||
| `autoresearch.jsonl` | Structured log: config headers + experiment entries (metric, status, timestamp, segment, commit hash). |
|
||||
| `autoresearch.ideas.md` | Backlog of promising ideas not yet tried. Optional. |
|
||||
|
||||
---
|
||||
The design is file-first: any agent can pick up the repo-root files and continue the loop without prior context.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
pi install https://github.com/davebcn87/pi-autoresearch
|
||||
npm install
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary>Manual install</summary>
|
||||
Then load this repo path in OpenClaw plugin discovery and restart the gateway:
|
||||
|
||||
```yaml
|
||||
plugins:
|
||||
load:
|
||||
paths:
|
||||
- /absolute/path/to/openclaw-autoresearch
|
||||
entries:
|
||||
openclaw-autoresearch:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
OpenClaw discovers `openclaw.plugin.json`, loads `extensions/openclaw-autoresearch/index.ts`, and exposes `autoresearch-create`.
|
||||
|
||||
Manual install is also possible: copy the plugin root, `extensions/openclaw-autoresearch/`, and `skills/autoresearch-create/` into your managed OpenClaw locations, then restart.
|
||||
|
||||
Verify:
|
||||
|
||||
- skill: `autoresearch-create`
|
||||
- tools: `init_experiment`, `run_experiment`, `log_experiment`
|
||||
- command: `/autoresearch` (recommended)
|
||||
- direct skill fallback: `/skill autoresearch-create`
|
||||
|
||||
Prefer the explicit `/autoresearch` command surface in OpenClaw. The auto-generated native skill alias `/autoresearch_create` may not trigger reliably on some hosts, so use `/skill autoresearch-create` if you need to invoke the skill directly.
|
||||
|
||||
## Use
|
||||
|
||||
In the repo you want to optimize:
|
||||
|
||||
1. Load the plugin.
|
||||
2. Run `/autoresearch` or `/autoresearch setup <goal>`.
|
||||
3. Send a normal message with the goal, command, metric (+ direction), files in scope, and constraints.
|
||||
4. If you need the raw skill invocation, use `/skill autoresearch-create`.
|
||||
5. The agent writes `autoresearch.md` and `autoresearch.sh`, runs a baseline, then starts looping.
|
||||
6. Use `/autoresearch` or `/autoresearch status` to re-prime context on a later turn.
|
||||
|
||||
To resume an existing session, a new agent reads the repo-root files and continues from where the last one stopped.
|
||||
|
||||
### User steers
|
||||
|
||||
Messages sent while an experiment is running are queued and surfaced after the next `log_experiment`. The agent finishes the current experiment before incorporating the steer.
|
||||
|
||||
### Ideas backlog
|
||||
|
||||
When the agent discovers promising but complex ideas mid-loop, it appends them to `autoresearch.ideas.md`. On resume, the agent reads the backlog, prunes stale entries, and uses the remaining ideas as experiment paths.
|
||||
|
||||
## Upstream reference
|
||||
|
||||
This port preserves upstream semantics, names, and file contracts while adapting presentation to OpenClaw. There is no Pi-style widget, dashboard, or editor shortcut layer. Remaining differences are tracked in [`docs/non-parity.md`](docs/non-parity.md).
|
||||
|
||||
- upstream repo: `https://github.com/davebcn87/pi-autoresearch`
|
||||
- pinned upstream commit: `2227029fa5712944a36938b5fe59f709cb30ed22` (`2227029f`)
|
||||
|
||||
## Validation
|
||||
|
||||
```bash
|
||||
cp -r extensions/pi-autoresearch ~/.pi/agent/extensions/
|
||||
cp -r skills/autoresearch-create ~/.pi/agent/skills/
|
||||
npm install --include=dev
|
||||
npm run typecheck
|
||||
npm test
|
||||
npm run validate
|
||||
```
|
||||
|
||||
Then `/reload` in pi.
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### 1. Start autoresearch
|
||||
|
||||
```
|
||||
/skill:autoresearch-create
|
||||
```
|
||||
|
||||
The agent asks about your goal, command, metric, and files in scope — or infers them from context. It then creates a branch, writes `autoresearch.md` and `autoresearch.sh`, runs the baseline, and starts looping immediately.
|
||||
|
||||
### 2. The loop
|
||||
|
||||
The agent runs autonomously: edit → commit → `run_experiment` → `log_experiment` → keep or revert → repeat. It never stops unless interrupted.
|
||||
|
||||
Every result is appended to `autoresearch.jsonl` in your project — one line per run. This means:
|
||||
|
||||
- **Survives restarts** — the agent can resume a session by reading the file
|
||||
- **Survives context resets** — `autoresearch.md` captures what's been tried so a fresh agent has full context
|
||||
- **Human readable** — open it anytime to see the full history
|
||||
- **Branch-aware** — each branch has its own session
|
||||
|
||||
### 3. Monitor progress
|
||||
|
||||
- **Widget** — always visible above the editor
|
||||
- **`/autoresearch`** — full dashboard with results table and best run
|
||||
- **`Escape`** — interrupt anytime and ask for a summary
|
||||
|
||||
---
|
||||
|
||||
## Example domains
|
||||
|
||||
| Domain | Metric | Command |
|
||||
|--------|--------|---------|
|
||||
| Test speed | seconds ↓ | `pnpm test` |
|
||||
| Bundle size | KB ↓ | `pnpm build && du -sb dist` |
|
||||
| LLM training | val_bpb ↓ | `uv run train.py` |
|
||||
| Build speed | seconds ↓ | `pnpm build` |
|
||||
| Lighthouse | perf score ↑ | `lighthouse http://localhost:3000 --output=json` |
|
||||
|
||||
---
|
||||
|
||||
## How it works
|
||||
|
||||
The **extension** is domain-agnostic infrastructure. The **skill** encodes domain knowledge. This separation means one extension serves unlimited domains.
|
||||
|
||||
```
|
||||
┌──────────────────────┐ ┌──────────────────────────┐
|
||||
│ Extension (global) │ │ Skill (per-domain) │
|
||||
│ │ │ │
|
||||
│ run_experiment │◄────│ command: pnpm test │
|
||||
│ log_experiment │ │ metric: seconds (lower) │
|
||||
│ widget + dashboard │ │ scope: vitest configs │
|
||||
│ │ │ ideas: pool, parallel… │
|
||||
└──────────────────────┘ └──────────────────────────┘
|
||||
```
|
||||
|
||||
Two files keep the session alive across restarts and context resets:
|
||||
|
||||
```
|
||||
autoresearch.jsonl — append-only log of every run (metric, status, commit, description)
|
||||
autoresearch.md — living document: objective, what's been tried, dead ends, key wins
|
||||
```
|
||||
|
||||
A fresh agent with no memory can read these two files and continue exactly where the previous session left off.
|
||||
|
||||
---
|
||||
The local test shim supports typechecking and tests without a full OpenClaw host checkout. Runtime behavior depends on a real OpenClaw host.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
# Non-Parity Notes for the OpenClaw Port
|
||||
|
||||
This repo aims to be a **faithful port** of `davebcn87/pi-autoresearch`, but it will not be a literal 1:1 port of Pi's UI/runtime.
|
||||
|
||||
Pinned upstream reference:
|
||||
|
||||
- Repo: `https://github.com/davebcn87/pi-autoresearch`
|
||||
- Commit: `2227029fa5712944a36938b5fe59f709cb30ed22` (`2227029f`)
|
||||
|
||||
## Principle
|
||||
|
||||
Non-parity is acceptable only when it is forced by the host/runtime difference.
|
||||
|
||||
That means:
|
||||
- preserve semantics first
|
||||
- preserve names and file layout second
|
||||
- adapt presentation/runtime integration last
|
||||
|
||||
## Expected non-parity in v1
|
||||
|
||||
### 1. No Pi widget parity
|
||||
The Pi extension renders an always-visible status widget above the editor.
|
||||
|
||||
OpenClaw may provide a thinner status surface instead, such as:
|
||||
- tool output
|
||||
- command output
|
||||
- lightweight summaries
|
||||
|
||||
### 2. No fullscreen dashboard / TUI parity
|
||||
Pi provides an inline dashboard with keyboard interaction.
|
||||
|
||||
OpenClaw v1 should not try to fake this with a new UI system. If a status view exists, it should be thin and optional.
|
||||
|
||||
### 3. No keyboard shortcut parity
|
||||
Pi has `Ctrl+X` and `Escape` affordances tied to its editor runtime.
|
||||
|
||||
These are considered host-specific and are not part of the core port contract.
|
||||
|
||||
### 4. Lifecycle hook names will differ
|
||||
Pi uses hooks such as:
|
||||
- `session_start`
|
||||
- `session_switch`
|
||||
- `session_fork`
|
||||
- `session_tree`
|
||||
- `before_agent_start`
|
||||
- `agent_end`
|
||||
- `input`
|
||||
|
||||
OpenClaw has a different hook model. We should preserve intent, not literal event names.
|
||||
|
||||
In practice, this port now uses documented OpenClaw lifecycle hooks such as:
|
||||
- `before_prompt_build`
|
||||
- `message_received`
|
||||
- `agent_end`
|
||||
- `session_end`
|
||||
|
||||
The result should preserve queued-steer handling and ideas-backlog continuation intent without pretending that Pi's hook names or UI affordances exist in OpenClaw.
|
||||
|
||||
### 5. `/autoresearch` command is thinner than Pi
|
||||
The upstream repo includes a dedicated `/autoresearch` dashboard/entry surface.
|
||||
|
||||
OpenClaw v1 keeps the main UX command-first and implements `/autoresearch` as a mode and status helper that:
|
||||
- provides a stable explicit entrypoint when host-native skill aliases vary
|
||||
- detects canonical repo-root files
|
||||
- enables or disables autoresearch mode for later agent turns
|
||||
- offers terse status text
|
||||
- points the agent back to `autoresearch.md`
|
||||
|
||||
It is intentionally not a dashboard replacement or a fullscreen UI.
|
||||
|
||||
## Non-parity that is **not** acceptable
|
||||
|
||||
The following would be design drift, not justified non-parity:
|
||||
|
||||
- moving canonical runtime files under `.autoresearch/` in v1
|
||||
- renaming `init_experiment`, `run_experiment`, or `log_experiment`
|
||||
- making a provider/runtime the product identity
|
||||
- replacing the explicit command and direct-skill setup surface with a provider-specific worker-first UX
|
||||
- changing keep/discard/crash behavior for convenience
|
||||
- relying on hidden runtime state instead of file-first resumability
|
||||
|
||||
## Honest product statement for v1
|
||||
|
||||
The correct way to describe v1 is:
|
||||
|
||||
> A faithful OpenClaw port of `pi-autoresearch` that preserves upstream semantics, names, and file contracts, while explicitly not matching Pi's editor widget/dashboard UX.
|
||||
|
||||
## Future parity work
|
||||
|
||||
Possible later work, if OpenClaw surfaces support it cleanly:
|
||||
- richer status presentation
|
||||
- optional command polish
|
||||
- optional provider adapters behind the plugin boundary
|
||||
|
||||
These should remain secondary to semantic fidelity.
|
||||
@@ -0,0 +1,2 @@
|
||||
- try a lighter parser
|
||||
- remove redundant setup
|
||||
@@ -0,0 +1,6 @@
|
||||
{"type":"config","name":"Speed up tests","metricName":"seconds","metricUnit":"s","bestDirection":"lower"}
|
||||
{"run":1,"commit":"aaaaaaa","metric":12.5,"metrics":{"compile_ms":40},"status":"keep","description":"baseline","timestamp":1,"segment":0}
|
||||
{"run":2,"commit":"bbbbbbb","metric":11.8,"metrics":{"compile_ms":38},"status":"keep","description":"cache tweak","timestamp":2,"segment":0}
|
||||
{"type":"config","name":"Speed up tests","metricName":"seconds","metricUnit":"s","bestDirection":"lower"}
|
||||
{"run":1,"commit":"ccccccc","metric":10.9,"metrics":{"compile_ms":34,"bundle_kb":120},"status":"keep","description":"new baseline","timestamp":3,"segment":1}
|
||||
{"run":2,"commit":"ddddddd","metric":11.4,"metrics":{"compile_ms":33,"bundle_kb":121},"status":"discard","description":"bad tradeoff","timestamp":4,"segment":1}
|
||||
@@ -0,0 +1,8 @@
|
||||
# Autoresearch: Speed up tests
|
||||
|
||||
## Objective
|
||||
Make the benchmark faster.
|
||||
|
||||
## What's Been Tried
|
||||
- cache tweak
|
||||
- baseline reset after a strategy shift
|
||||
@@ -0,0 +1,30 @@
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import {
|
||||
AUTORESEARCH_PLUGIN_DESCRIPTION,
|
||||
AUTORESEARCH_PLUGIN_ID,
|
||||
AUTORESEARCH_PLUGIN_NAME,
|
||||
autoresearchPluginConfigSchema,
|
||||
} from "./src/config.js";
|
||||
import { createInitExperimentTool } from "./src/tools/init-experiment.js";
|
||||
import { createRunExperimentTool } from "./src/tools/run-experiment.js";
|
||||
import { createLogExperimentTool } from "./src/tools/log-experiment.js";
|
||||
import { createAutoresearchStatusTool } from "./src/tools/autoresearch-status.js";
|
||||
import { registerAutoresearchHooks } from "./src/hooks.js";
|
||||
import { registerAutoresearchCommand } from "./src/commands/autoresearch.js";
|
||||
|
||||
const plugin = {
|
||||
id: AUTORESEARCH_PLUGIN_ID,
|
||||
name: AUTORESEARCH_PLUGIN_NAME,
|
||||
description: AUTORESEARCH_PLUGIN_DESCRIPTION,
|
||||
configSchema: autoresearchPluginConfigSchema,
|
||||
register(api: OpenClawPluginApi) {
|
||||
registerAutoresearchHooks(api);
|
||||
registerAutoresearchCommand(api);
|
||||
api.registerTool(createInitExperimentTool(api));
|
||||
api.registerTool(createRunExperimentTool(api));
|
||||
api.registerTool(createLogExperimentTool(api));
|
||||
api.registerTool(createAutoresearchStatusTool(api));
|
||||
},
|
||||
};
|
||||
|
||||
export default plugin;
|
||||
@@ -0,0 +1,182 @@
|
||||
import * as fs from "node:fs";
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import {
|
||||
AUTORESEARCH_ROOT_FILES,
|
||||
getAutoresearchRootFilePath,
|
||||
type AutoresearchRootFileKey,
|
||||
} from "../files.js";
|
||||
import { reconstructStateFromJsonl } from "../state.js";
|
||||
import { formatAutoresearchStatusText } from "../tools/autoresearch-status.js";
|
||||
import {
|
||||
clearAutoresearchSteers,
|
||||
getAutoresearchRuntimeState,
|
||||
setAutoresearchPendingCommand,
|
||||
setAutoresearchRunInFlight,
|
||||
setAutoresearchRuntimeMode,
|
||||
} from "../runtime-state.js";
|
||||
|
||||
type CommandContext = {
|
||||
args?: string;
|
||||
channel?: string;
|
||||
senderId?: string;
|
||||
cwd?: string;
|
||||
};
|
||||
|
||||
const COMMAND_USAGE = [
|
||||
"Enable or inspect repo-root autoresearch mode.",
|
||||
"",
|
||||
"Usage:",
|
||||
"/autoresearch",
|
||||
"/autoresearch on",
|
||||
"/autoresearch off",
|
||||
"/autoresearch setup",
|
||||
"/autoresearch status",
|
||||
"/autoresearch help",
|
||||
].join("\n");
|
||||
|
||||
export function registerAutoresearchCommand(api: OpenClawPluginApi): void {
|
||||
api.registerCommand({
|
||||
name: "autoresearch",
|
||||
description: "Enable, disable, or inspect repo-root autoresearch mode.",
|
||||
acceptsArgs: true,
|
||||
handler: (ctx: CommandContext) => {
|
||||
const cwd = resolveCommandCwd(api, ctx);
|
||||
const rawArgs = (ctx.args ?? "").trim();
|
||||
const [verb, ...rest] = rawArgs.split(/\s+/).filter(Boolean);
|
||||
const action = (verb ?? "").toLowerCase();
|
||||
const remainder = rest.join(" ").trim() || null;
|
||||
|
||||
if (!rawArgs || action === "resume" || action === "on") {
|
||||
return {
|
||||
text: enableAutoresearchMode(cwd, rawArgs && action !== "resume" && action !== "on" ? rawArgs : remainder),
|
||||
};
|
||||
}
|
||||
if (action === "setup") {
|
||||
return { text: primeAutoresearchSetup(cwd, remainder) };
|
||||
}
|
||||
if (action === "off") {
|
||||
setAutoresearchRuntimeMode(cwd, "off");
|
||||
setAutoresearchPendingCommand(cwd, null);
|
||||
clearAutoresearchSteers(cwd);
|
||||
setAutoresearchRunInFlight(cwd, false);
|
||||
return {
|
||||
text: [
|
||||
"Autoresearch mode OFF.",
|
||||
`Canonical files remain at repo root: ${Object.values(AUTORESEARCH_ROOT_FILES).join(", ")}`,
|
||||
].join("\n"),
|
||||
};
|
||||
}
|
||||
if (action === "status") {
|
||||
return { text: buildAutoresearchCommandText(cwd, "status") };
|
||||
}
|
||||
if (action === "help") {
|
||||
return { text: `${COMMAND_USAGE}\n\n${buildAutoresearchCommandText(cwd, "default")}` };
|
||||
}
|
||||
|
||||
return {
|
||||
text: enableAutoresearchMode(cwd, rawArgs),
|
||||
};
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function buildAutoresearchCommandText(
|
||||
cwd: string,
|
||||
mode: "default" | "status",
|
||||
): string {
|
||||
const runtimeState = getAutoresearchRuntimeState(cwd);
|
||||
const presentFiles = getPresentCanonicalFiles(cwd);
|
||||
const hasSession = presentFiles.length > 0;
|
||||
|
||||
if (!hasSession) {
|
||||
return [
|
||||
"No repo-root autoresearch session detected.",
|
||||
"",
|
||||
`Expected canonical files: ${Object.values(AUTORESEARCH_ROOT_FILES).join(", ")}`,
|
||||
"Recommended OpenClaw entrypoint: `/autoresearch` or `/autoresearch setup <goal>`.",
|
||||
"Direct skill fallback: `/skill autoresearch-create`.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
const state = reconstructStateFromJsonl(cwd);
|
||||
const lines = [
|
||||
`Autoresearch session detected at repo root: ${presentFiles.join(", ")}`,
|
||||
`Read \`${AUTORESEARCH_ROOT_FILES.sessionDoc}\` before resuming or changing the loop.`,
|
||||
];
|
||||
|
||||
if (mode === "status") {
|
||||
lines.push("", formatAutoresearchStatusText(state, runtimeState));
|
||||
} else if (state.mode === "active" || state.hasSessionDoc) {
|
||||
lines.push(
|
||||
"Use `/autoresearch` or `/autoresearch on` to enable mode for the next agent turn, then continue the upstream loop with `init_experiment`, `run_experiment`, and `log_experiment` as needed.",
|
||||
);
|
||||
} else {
|
||||
lines.push(
|
||||
`The canonical files exist, but the session brief looks incomplete. Open \`${AUTORESEARCH_ROOT_FILES.sessionDoc}\` and finish setup, or restart via \`/skill autoresearch-create\` if needed.`,
|
||||
);
|
||||
}
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function enableAutoresearchMode(cwd: string, args: string | null): string {
|
||||
setAutoresearchRuntimeMode(cwd, "on");
|
||||
const presentFiles = getPresentCanonicalFiles(cwd);
|
||||
const hasSession = presentFiles.length > 0;
|
||||
|
||||
if (!hasSession) {
|
||||
setAutoresearchPendingCommand(cwd, {
|
||||
kind: "setup",
|
||||
args,
|
||||
});
|
||||
return [
|
||||
"Autoresearch mode ON.",
|
||||
"No repo-root session was detected, so the next agent turn will be primed for setup.",
|
||||
"Next step: send a normal message so the next agent turn can gather setup details.",
|
||||
"Direct skill fallback: `/skill autoresearch-create`.",
|
||||
args ? `Captured setup instruction: ${args}` : "Run `/autoresearch setup <goal>` to attach a setup hint.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
setAutoresearchPendingCommand(cwd, {
|
||||
kind: "resume",
|
||||
args,
|
||||
});
|
||||
return [
|
||||
"Autoresearch mode ON.",
|
||||
`Next agent turn will be primed from \`${AUTORESEARCH_ROOT_FILES.sessionDoc}\` and the canonical repo-root files.`,
|
||||
args ? `Captured resume instruction: ${args}` : "Send a normal message to continue the loop, or use `/autoresearch status` for a snapshot first.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function primeAutoresearchSetup(cwd: string, args: string | null): string {
|
||||
setAutoresearchRuntimeMode(cwd, "on");
|
||||
setAutoresearchPendingCommand(cwd, {
|
||||
kind: "setup",
|
||||
args,
|
||||
});
|
||||
|
||||
return [
|
||||
"Autoresearch setup primed.",
|
||||
"The next agent turn will be told to create the canonical repo-root files and start the loop.",
|
||||
"Continue with a normal message on the next turn, or invoke the skill directly with `/skill autoresearch-create`.",
|
||||
args ? `Captured setup instruction: ${args}` : "Add an argument to `/autoresearch setup` if you want a specific goal or constraint carried forward.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function resolveCommandCwd(api: OpenClawPluginApi, ctx: CommandContext): string {
|
||||
if (typeof ctx.cwd === "string" && ctx.cwd.trim().length > 0) {
|
||||
return ctx.cwd;
|
||||
}
|
||||
return api.resolvePath(".");
|
||||
}
|
||||
|
||||
function getPresentCanonicalFiles(cwd: string): string[] {
|
||||
const present: string[] = [];
|
||||
for (const key of Object.keys(AUTORESEARCH_ROOT_FILES) as AutoresearchRootFileKey[]) {
|
||||
if (fs.existsSync(getAutoresearchRootFilePath(cwd, key))) {
|
||||
present.push(AUTORESEARCH_ROOT_FILES[key]);
|
||||
}
|
||||
}
|
||||
return present;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
import { emptyPluginConfigSchema } from "openclaw/plugin-sdk/core";
|
||||
|
||||
export const AUTORESEARCH_PLUGIN_ID = "openclaw-autoresearch";
|
||||
export const AUTORESEARCH_PLUGIN_NAME = "Autoresearch";
|
||||
export const AUTORESEARCH_PLUGIN_DESCRIPTION = "Faithful OpenClaw port of pi-autoresearch.";
|
||||
|
||||
export const autoresearchPluginConfigSchema = emptyPluginConfigSchema();
|
||||
@@ -0,0 +1,122 @@
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
const OUTPUT_TAIL_LINES = 80;
|
||||
const DEFAULT_TIMEOUT_SECONDS = 600;
|
||||
const FORCE_KILL_GRACE_MS = 1_000;
|
||||
|
||||
export type ExperimentExecutionResult = {
|
||||
readonly command: string;
|
||||
readonly exitCode: number | null;
|
||||
readonly durationSeconds: number;
|
||||
readonly passed: boolean;
|
||||
readonly crashed: boolean;
|
||||
readonly timedOut: boolean;
|
||||
readonly tailOutput: string;
|
||||
readonly stdout: string;
|
||||
readonly stderr: string;
|
||||
};
|
||||
|
||||
export async function executeExperimentCommand(options: {
|
||||
command: string;
|
||||
cwd: string;
|
||||
timeoutSeconds?: number;
|
||||
signal?: AbortSignal;
|
||||
}): Promise<ExperimentExecutionResult> {
|
||||
const timeoutSeconds = options.timeoutSeconds ?? DEFAULT_TIMEOUT_SECONDS;
|
||||
const timeoutMs = Math.max(0, timeoutSeconds) * 1_000;
|
||||
const startedAt = Date.now();
|
||||
|
||||
return await new Promise<ExperimentExecutionResult>((resolve) => {
|
||||
const child = spawn("bash", ["-c", options.command], {
|
||||
cwd: options.cwd,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let timedOut = false;
|
||||
let forceKillTimer: NodeJS.Timeout | undefined;
|
||||
|
||||
const timeoutTimer =
|
||||
timeoutMs > 0
|
||||
? setTimeout(() => {
|
||||
timedOut = true;
|
||||
child.kill("SIGTERM");
|
||||
forceKillTimer = setTimeout(() => {
|
||||
if (!child.killed) {
|
||||
child.kill("SIGKILL");
|
||||
}
|
||||
}, FORCE_KILL_GRACE_MS);
|
||||
}, timeoutMs)
|
||||
: undefined;
|
||||
|
||||
const abortHandler = () => {
|
||||
child.kill("SIGTERM");
|
||||
forceKillTimer = setTimeout(() => {
|
||||
if (!child.killed) {
|
||||
child.kill("SIGKILL");
|
||||
}
|
||||
}, FORCE_KILL_GRACE_MS);
|
||||
};
|
||||
|
||||
if (options.signal) {
|
||||
if (options.signal.aborted) {
|
||||
abortHandler();
|
||||
} else {
|
||||
options.signal.addEventListener("abort", abortHandler, { once: true });
|
||||
}
|
||||
}
|
||||
|
||||
child.stdout.on("data", (chunk: string | Buffer) => {
|
||||
stdout += chunk.toString();
|
||||
});
|
||||
|
||||
child.stderr.on("data", (chunk: string | Buffer) => {
|
||||
stderr += chunk.toString();
|
||||
});
|
||||
|
||||
child.on("error", (error) => {
|
||||
stderr += `${stderr ? "\n" : ""}${String(error.message || error)}`;
|
||||
});
|
||||
|
||||
child.on("close", (code) => {
|
||||
if (timeoutTimer) {
|
||||
clearTimeout(timeoutTimer);
|
||||
}
|
||||
if (forceKillTimer) {
|
||||
clearTimeout(forceKillTimer);
|
||||
}
|
||||
if (options.signal) {
|
||||
options.signal.removeEventListener("abort", abortHandler);
|
||||
}
|
||||
|
||||
const durationSeconds = (Date.now() - startedAt) / 1_000;
|
||||
const passed = code === 0 && !timedOut;
|
||||
|
||||
resolve({
|
||||
command: options.command,
|
||||
exitCode: code,
|
||||
durationSeconds,
|
||||
passed,
|
||||
crashed: !passed,
|
||||
timedOut,
|
||||
tailOutput: createOutputTail(stdout, stderr),
|
||||
stdout,
|
||||
stderr,
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function createOutputTail(stdout: string, stderr: string): string {
|
||||
const combined = [stdout, stderr]
|
||||
.filter((value) => value.trim().length > 0)
|
||||
.join("\n")
|
||||
.trim();
|
||||
|
||||
if (!combined) {
|
||||
return "";
|
||||
}
|
||||
|
||||
return combined.split(/\r?\n/).slice(-OUTPUT_TAIL_LINES).join("\n");
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import * as fs from "node:fs";
|
||||
|
||||
export const AUTORESEARCH_ROOT_FILES = {
|
||||
sessionDoc: "autoresearch.md",
|
||||
runnerScript: "autoresearch.sh",
|
||||
resultsLog: "autoresearch.jsonl",
|
||||
ideasBacklog: "autoresearch.ideas.md",
|
||||
} as const;
|
||||
|
||||
export type AutoresearchRootFileKey = keyof typeof AUTORESEARCH_ROOT_FILES;
|
||||
|
||||
export function getAutoresearchRootFilePath(
|
||||
cwd: string,
|
||||
file: AutoresearchRootFileKey,
|
||||
): string {
|
||||
return `${cwd}/${AUTORESEARCH_ROOT_FILES[file]}`;
|
||||
}
|
||||
|
||||
export function readAutoresearchRootFile(
|
||||
cwd: string,
|
||||
file: AutoresearchRootFileKey,
|
||||
): string | null {
|
||||
const filePath = getAutoresearchRootFilePath(cwd, file);
|
||||
if (!fs.existsSync(filePath)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return fs.readFileSync(filePath, "utf8");
|
||||
}
|
||||
|
||||
/**
|
||||
* PR 2 skeleton only.
|
||||
* This module will own canonical root-level file IO helpers in later PRs.
|
||||
*/
|
||||
export function describeCanonicalFiles(): typeof AUTORESEARCH_ROOT_FILES {
|
||||
return AUTORESEARCH_ROOT_FILES;
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
|
||||
export type GitCommandResult = {
|
||||
readonly code: number | null;
|
||||
readonly stdout: string;
|
||||
readonly stderr: string;
|
||||
readonly combinedOutput: string;
|
||||
};
|
||||
|
||||
export type GitKeepResult = {
|
||||
readonly attempted: true;
|
||||
readonly committed: boolean;
|
||||
readonly commit: string;
|
||||
readonly summary: string;
|
||||
readonly command: GitCommandResult;
|
||||
};
|
||||
|
||||
function runGitCommand(cwd: string, args: readonly string[]): GitCommandResult {
|
||||
const result = spawnSync("git", [...args], {
|
||||
cwd,
|
||||
encoding: "utf8",
|
||||
});
|
||||
|
||||
const stdout = result.stdout ?? "";
|
||||
const stderr = result.stderr ?? "";
|
||||
|
||||
return {
|
||||
code: result.status,
|
||||
stdout,
|
||||
stderr,
|
||||
combinedOutput: `${stdout}${stderr}`.trim(),
|
||||
};
|
||||
}
|
||||
|
||||
export function commitKeptExperiment(options: {
|
||||
cwd: string;
|
||||
description: string;
|
||||
metricName: string;
|
||||
metric: number;
|
||||
metrics: Record<string, number>;
|
||||
commit: string;
|
||||
status: "keep";
|
||||
}): GitKeepResult {
|
||||
const resultData: Record<string, unknown> = {
|
||||
status: options.status,
|
||||
[options.metricName || "metric"]: options.metric,
|
||||
...options.metrics,
|
||||
};
|
||||
const commitMessage = `${options.description}\n\nResult: ${JSON.stringify(resultData)}`;
|
||||
|
||||
const repoRootResult = runGitCommand(options.cwd, ["rev-parse", "--show-toplevel"]);
|
||||
if (repoRootResult.code !== 0 || repoRootResult.stdout.trim().length === 0) {
|
||||
return {
|
||||
attempted: true,
|
||||
committed: false,
|
||||
commit: options.commit,
|
||||
summary: `Git repo check failed${formatExit(repoRootResult.code)}: ${truncateOutput(repoRootResult.combinedOutput)}`,
|
||||
command: repoRootResult,
|
||||
};
|
||||
}
|
||||
|
||||
const repoRoot = repoRootResult.stdout.trim();
|
||||
|
||||
const addResult = runGitCommand(repoRoot, ["add", "-A"]);
|
||||
if (addResult.code !== 0) {
|
||||
return {
|
||||
attempted: true,
|
||||
committed: false,
|
||||
commit: options.commit,
|
||||
summary: `Git add failed${formatExit(addResult.code)}: ${truncateOutput(addResult.combinedOutput)}`,
|
||||
command: addResult,
|
||||
};
|
||||
}
|
||||
|
||||
const diffResult = runGitCommand(repoRoot, ["diff", "--cached", "--quiet"]);
|
||||
if (diffResult.code === 0) {
|
||||
return {
|
||||
attempted: true,
|
||||
committed: false,
|
||||
commit: options.commit,
|
||||
summary: "Git: nothing to commit (working tree clean)",
|
||||
command: diffResult,
|
||||
};
|
||||
}
|
||||
if (diffResult.code !== 1) {
|
||||
return {
|
||||
attempted: true,
|
||||
committed: false,
|
||||
commit: options.commit,
|
||||
summary: `Git diff check failed${formatExit(diffResult.code)}: ${truncateOutput(diffResult.combinedOutput)}`,
|
||||
command: diffResult,
|
||||
};
|
||||
}
|
||||
|
||||
const commitResult = runGitCommand(repoRoot, ["commit", "-m", commitMessage]);
|
||||
if (commitResult.code !== 0) {
|
||||
return {
|
||||
attempted: true,
|
||||
committed: false,
|
||||
commit: options.commit,
|
||||
summary: `Git commit failed${formatExit(commitResult.code)}: ${truncateOutput(commitResult.combinedOutput)}`,
|
||||
command: commitResult,
|
||||
};
|
||||
}
|
||||
|
||||
const revParseResult = runGitCommand(repoRoot, ["rev-parse", "--short=7", "HEAD"]);
|
||||
const actualCommit =
|
||||
revParseResult.code === 0 && revParseResult.stdout.trim().length >= 7
|
||||
? revParseResult.stdout.trim().slice(0, 7)
|
||||
: options.commit;
|
||||
const firstLine = commitResult.combinedOutput.split("\n")[0]?.trim() || "commit created";
|
||||
|
||||
return {
|
||||
attempted: true,
|
||||
committed: true,
|
||||
commit: actualCommit,
|
||||
summary: `Git: committed - ${firstLine}`,
|
||||
command: commitResult,
|
||||
};
|
||||
}
|
||||
|
||||
function truncateOutput(output: string): string {
|
||||
const normalized = output.trim();
|
||||
if (!normalized) {
|
||||
return "no output";
|
||||
}
|
||||
return normalized.length > 200 ? `${normalized.slice(0, 200)}...` : normalized;
|
||||
}
|
||||
|
||||
function formatExit(code: number | null): string {
|
||||
return code === null ? "" : ` (exit ${code})`;
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import { AUTORESEARCH_ROOT_FILES } from "./files.js";
|
||||
import { reconstructStateFromJsonl } from "./state.js";
|
||||
import {
|
||||
clearAutoresearchRuntimeState,
|
||||
consumeAutoresearchContinuationReminder,
|
||||
consumeAutoresearchPendingCommand,
|
||||
getAutoresearchRuntimeState,
|
||||
queueAutoresearchSteer,
|
||||
setAutoresearchContinuationReminder,
|
||||
} from "./runtime-state.js";
|
||||
|
||||
type BeforeAgentStartEvent = {
|
||||
systemPrompt?: string;
|
||||
};
|
||||
|
||||
type HookContext = {
|
||||
cwd?: string;
|
||||
};
|
||||
|
||||
type HookCapablePluginApi = OpenClawPluginApi & {
|
||||
on?: (hookName: string, handler: (event: unknown, ctx: HookContext) => unknown) => void;
|
||||
registerHook?: (
|
||||
hookName: string,
|
||||
handler: (event: BeforeAgentStartEvent, ctx: HookContext) => BeforeAgentStartEvent | void,
|
||||
) => void;
|
||||
};
|
||||
|
||||
export function registerAutoresearchHooks(api: OpenClawPluginApi): void {
|
||||
const hookApi = api as HookCapablePluginApi;
|
||||
if (typeof hookApi.on === "function") {
|
||||
hookApi.on("before_prompt_build", (_event, ctx) => {
|
||||
const cwd = resolveHookCwd(api, ctx);
|
||||
if (cwd === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
const addition = buildBeforePromptBuildContext(cwd);
|
||||
if (addition === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
return {
|
||||
appendSystemContext: addition,
|
||||
};
|
||||
});
|
||||
|
||||
hookApi.on("message_received", (event, ctx) => {
|
||||
const cwd = resolveHookCwd(api, ctx);
|
||||
if (cwd === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
const state = getAutoresearchRuntimeState(cwd);
|
||||
if (!state.runInFlight) {
|
||||
return;
|
||||
}
|
||||
|
||||
const messageText = extractMessageText(event);
|
||||
if (messageText === null || isCommandLikeMessage(messageText)) {
|
||||
return;
|
||||
}
|
||||
|
||||
queueAutoresearchSteer(cwd, messageText);
|
||||
});
|
||||
|
||||
hookApi.on("agent_end", (_event, ctx) => {
|
||||
const cwd = resolveHookCwd(api, ctx);
|
||||
if (cwd === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
const state = reconstructStateFromJsonl(cwd);
|
||||
if (state.mode === "active" && state.ideas.hasBacklog) {
|
||||
setAutoresearchContinuationReminder(cwd, true);
|
||||
}
|
||||
});
|
||||
|
||||
hookApi.on("session_end", (_event, ctx) => {
|
||||
const cwd = resolveHookCwd(api, ctx);
|
||||
if (cwd === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
clearAutoresearchRuntimeState(cwd);
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
if (typeof hookApi.registerHook !== "function") {
|
||||
return;
|
||||
}
|
||||
|
||||
hookApi.registerHook("before_agent_start", (event, ctx) => {
|
||||
const cwd = resolveHookCwd(api, ctx);
|
||||
if (cwd === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
const addition = buildBeforePromptBuildContext(cwd);
|
||||
if (addition === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
return {
|
||||
...event,
|
||||
systemPrompt: `${event.systemPrompt ?? ""}${addition}`,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
export function buildBeforePromptBuildContext(cwd: string): string | null {
|
||||
const state = reconstructStateFromJsonl(cwd);
|
||||
const runtimeState = getAutoresearchRuntimeState(cwd);
|
||||
const modeEnabled =
|
||||
runtimeState.mode === "on" ||
|
||||
(runtimeState.mode !== "off" && (state.mode === "active" || state.hasSessionDoc));
|
||||
|
||||
if (!modeEnabled) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const canonicalFiles = [
|
||||
AUTORESEARCH_ROOT_FILES.sessionDoc,
|
||||
AUTORESEARCH_ROOT_FILES.runnerScript,
|
||||
AUTORESEARCH_ROOT_FILES.resultsLog,
|
||||
];
|
||||
const pendingCommand = consumeAutoresearchPendingCommand(cwd);
|
||||
const needsContinuationReminder = consumeAutoresearchContinuationReminder(cwd);
|
||||
|
||||
const lines = ["", "", "## Autoresearch Mode (ACTIVE)"];
|
||||
|
||||
if (pendingCommand?.kind === "setup" || !state.hasSessionDoc) {
|
||||
lines.push(
|
||||
`No ${AUTORESEARCH_ROOT_FILES.sessionDoc} was detected. Gather context and set up the experiment now with the canonical repo-root files.`,
|
||||
`Create ${AUTORESEARCH_ROOT_FILES.sessionDoc} and ${AUTORESEARCH_ROOT_FILES.runnerScript}, then initialize the loop with init_experiment, run_experiment, and log_experiment.`,
|
||||
);
|
||||
if (pendingCommand?.args) {
|
||||
lines.push(`Additional setup instruction from /autoresearch: ${pendingCommand.args}`);
|
||||
}
|
||||
} else {
|
||||
lines.push(
|
||||
`Autoresearch files live at repo root: ${canonicalFiles.join(", ")}.`,
|
||||
`Read ${AUTORESEARCH_ROOT_FILES.sessionDoc} before resuming or changing the experiment loop, and re-read it after compaction.`,
|
||||
"Resume the autonomous upstream loop: edit, run_experiment, log_experiment, keep/discard/crash, repeat.",
|
||||
"Use init_experiment, run_experiment, and log_experiment for experiment state changes. Never stop unless the user explicitly interrupts the loop.",
|
||||
);
|
||||
if (pendingCommand?.args) {
|
||||
lines.push(`Additional resume instruction from /autoresearch: ${pendingCommand.args}`);
|
||||
}
|
||||
}
|
||||
|
||||
lines.push(
|
||||
`For discard or crash results, log_experiment records the outcome but does not revert your tree for you. Run \`git checkout -- .\` yourself after logging when you want to discard tracked changes.`,
|
||||
);
|
||||
|
||||
if (state.ideas.hasBacklog) {
|
||||
lines.push(
|
||||
`${AUTORESEARCH_ROOT_FILES.ideasBacklog} exists with ${state.ideas.pendingCount} pending idea${state.ideas.pendingCount === 1 ? "" : "s"}; use it as continuation fuel for promising paths you have not exhausted.`,
|
||||
);
|
||||
}
|
||||
|
||||
if (needsContinuationReminder && state.ideas.hasBacklog) {
|
||||
lines.push(
|
||||
`The previous autoresearch run ended with pending ideas. Read ${AUTORESEARCH_ROOT_FILES.ideasBacklog}, prune stale items, and spin those ideas into the next experiments before declaring the work done.`,
|
||||
);
|
||||
}
|
||||
|
||||
if (runtimeState.queuedSteers.length > 0) {
|
||||
lines.push(
|
||||
`${runtimeState.queuedSteers.length} user steer${runtimeState.queuedSteers.length === 1 ? "" : "s"} arrived during the current experiment window. If the next followup turn repeats a steer already surfaced in log_experiment output, treat it as the same request rather than a new branch of work.`,
|
||||
);
|
||||
}
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function resolveHookCwd(api: OpenClawPluginApi, ctx: HookContext | undefined): string | null {
|
||||
if (ctx && typeof ctx.cwd === "string" && ctx.cwd.trim().length > 0) {
|
||||
return ctx.cwd;
|
||||
}
|
||||
|
||||
try {
|
||||
const resolved = api.resolvePath(".");
|
||||
return resolved.trim().length > 0 ? resolved : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function extractMessageText(event: unknown): string | null {
|
||||
if (!event || typeof event !== "object") {
|
||||
return null;
|
||||
}
|
||||
|
||||
const record = event as Record<string, unknown>;
|
||||
const direct = firstString(record.text, record.content, record.body, record.prompt);
|
||||
if (direct) {
|
||||
return direct;
|
||||
}
|
||||
|
||||
const message = record.message;
|
||||
if (message && typeof message === "object") {
|
||||
const nested = message as Record<string, unknown>;
|
||||
const messageText = firstString(nested.text, nested.content, nested.body);
|
||||
if (messageText) {
|
||||
return messageText;
|
||||
}
|
||||
}
|
||||
|
||||
const context = record.context;
|
||||
if (context && typeof context === "object") {
|
||||
const nested = context as Record<string, unknown>;
|
||||
const contextText = firstString(nested.content, nested.commandBody, nested.text);
|
||||
if (contextText) {
|
||||
return contextText;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function firstString(...values: unknown[]): string | null {
|
||||
for (const value of values) {
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
return value.trim();
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function isCommandLikeMessage(text: string): boolean {
|
||||
return /^[\/!]/.test(text.trim());
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import * as fs from "node:fs";
|
||||
import { getAutoresearchRootFilePath } from "./files.js";
|
||||
|
||||
export type AutoresearchConfigHeader = {
|
||||
type: "config";
|
||||
name: string;
|
||||
metricName: string;
|
||||
metricUnit: string;
|
||||
bestDirection: "lower" | "higher";
|
||||
};
|
||||
|
||||
export function createConfigHeader(config: {
|
||||
name: string;
|
||||
metricName: string;
|
||||
metricUnit: string;
|
||||
bestDirection: "lower" | "higher";
|
||||
}): AutoresearchConfigHeader {
|
||||
return {
|
||||
type: "config",
|
||||
name: config.name,
|
||||
metricName: config.metricName,
|
||||
metricUnit: config.metricUnit,
|
||||
bestDirection: config.bestDirection,
|
||||
};
|
||||
}
|
||||
|
||||
export function writeConfigHeader(
|
||||
cwd: string,
|
||||
header: AutoresearchConfigHeader,
|
||||
mode: "create" | "append",
|
||||
): void {
|
||||
const jsonlPath = getAutoresearchRootFilePath(cwd, "resultsLog");
|
||||
const line = `${JSON.stringify(header)}\n`;
|
||||
if (mode === "append") {
|
||||
fs.appendFileSync(jsonlPath, line);
|
||||
return;
|
||||
}
|
||||
fs.writeFileSync(jsonlPath, line);
|
||||
}
|
||||
|
||||
export type AutoresearchResultEntry = {
|
||||
readonly run: number;
|
||||
readonly commit: string;
|
||||
readonly metric: number;
|
||||
readonly metrics: Record<string, number>;
|
||||
readonly status: "keep" | "discard" | "crash";
|
||||
readonly description: string;
|
||||
readonly timestamp: number;
|
||||
readonly segment: number;
|
||||
};
|
||||
|
||||
export function appendResultEntry(cwd: string, entry: AutoresearchResultEntry): void {
|
||||
const jsonlPath = getAutoresearchRootFilePath(cwd, "resultsLog");
|
||||
fs.appendFileSync(jsonlPath, `${JSON.stringify(entry)}\n`);
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
export type AutoresearchRuntimeMode = "auto" | "on" | "off";
|
||||
|
||||
export type PendingAutoresearchCommand =
|
||||
| {
|
||||
readonly kind: "resume" | "setup";
|
||||
readonly args: string | null;
|
||||
}
|
||||
| null;
|
||||
|
||||
export type AutoresearchRuntimeSnapshot = {
|
||||
readonly mode: AutoresearchRuntimeMode;
|
||||
readonly runInFlight: boolean;
|
||||
readonly queuedSteers: readonly string[];
|
||||
readonly needsContinuationReminder: boolean;
|
||||
readonly pendingCommand: PendingAutoresearchCommand;
|
||||
};
|
||||
|
||||
type MutableAutoresearchRuntimeState = {
|
||||
mode: AutoresearchRuntimeMode;
|
||||
runInFlight: boolean;
|
||||
queuedSteers: string[];
|
||||
needsContinuationReminder: boolean;
|
||||
pendingCommand: PendingAutoresearchCommand;
|
||||
};
|
||||
|
||||
const MAX_QUEUED_STEERS = 20;
|
||||
const runtimeStates = new Map<string, MutableAutoresearchRuntimeState>();
|
||||
|
||||
function createDefaultRuntimeState(): MutableAutoresearchRuntimeState {
|
||||
return {
|
||||
mode: "auto",
|
||||
runInFlight: false,
|
||||
queuedSteers: [],
|
||||
needsContinuationReminder: false,
|
||||
pendingCommand: null,
|
||||
};
|
||||
}
|
||||
|
||||
function getMutableRuntimeState(cwd: string): MutableAutoresearchRuntimeState {
|
||||
let state = runtimeStates.get(cwd);
|
||||
if (!state) {
|
||||
state = createDefaultRuntimeState();
|
||||
runtimeStates.set(cwd, state);
|
||||
}
|
||||
return state;
|
||||
}
|
||||
|
||||
export function getAutoresearchRuntimeState(cwd: string): AutoresearchRuntimeSnapshot {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
return {
|
||||
mode: state.mode,
|
||||
runInFlight: state.runInFlight,
|
||||
queuedSteers: [...state.queuedSteers],
|
||||
needsContinuationReminder: state.needsContinuationReminder,
|
||||
pendingCommand: state.pendingCommand,
|
||||
};
|
||||
}
|
||||
|
||||
export function setAutoresearchRuntimeMode(
|
||||
cwd: string,
|
||||
mode: AutoresearchRuntimeMode,
|
||||
): AutoresearchRuntimeSnapshot {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
state.mode = mode;
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
export function setAutoresearchRunInFlight(
|
||||
cwd: string,
|
||||
runInFlight: boolean,
|
||||
): AutoresearchRuntimeSnapshot {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
state.runInFlight = runInFlight;
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
export function queueAutoresearchSteer(
|
||||
cwd: string,
|
||||
steer: string,
|
||||
): AutoresearchRuntimeSnapshot {
|
||||
const normalized = steer.trim();
|
||||
if (!normalized) {
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
state.queuedSteers.push(normalized);
|
||||
if (state.queuedSteers.length > MAX_QUEUED_STEERS) {
|
||||
state.queuedSteers = state.queuedSteers.slice(-MAX_QUEUED_STEERS);
|
||||
}
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
export function consumeAutoresearchSteers(cwd: string): readonly string[] {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
const queued = [...state.queuedSteers];
|
||||
state.queuedSteers = [];
|
||||
return queued;
|
||||
}
|
||||
|
||||
export function clearAutoresearchSteers(cwd: string): AutoresearchRuntimeSnapshot {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
state.queuedSteers = [];
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
export function setAutoresearchPendingCommand(
|
||||
cwd: string,
|
||||
pendingCommand: PendingAutoresearchCommand,
|
||||
): AutoresearchRuntimeSnapshot {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
state.pendingCommand = pendingCommand;
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
export function consumeAutoresearchPendingCommand(
|
||||
cwd: string,
|
||||
): PendingAutoresearchCommand {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
const pending = state.pendingCommand;
|
||||
state.pendingCommand = null;
|
||||
return pending;
|
||||
}
|
||||
|
||||
export function setAutoresearchContinuationReminder(
|
||||
cwd: string,
|
||||
needsReminder: boolean,
|
||||
): AutoresearchRuntimeSnapshot {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
state.needsContinuationReminder = needsReminder;
|
||||
return getAutoresearchRuntimeState(cwd);
|
||||
}
|
||||
|
||||
export function consumeAutoresearchContinuationReminder(cwd: string): boolean {
|
||||
const state = getMutableRuntimeState(cwd);
|
||||
const needsReminder = state.needsContinuationReminder;
|
||||
state.needsContinuationReminder = false;
|
||||
return needsReminder;
|
||||
}
|
||||
|
||||
export function clearAutoresearchRuntimeState(cwd: string): void {
|
||||
runtimeStates.delete(cwd);
|
||||
}
|
||||
@@ -0,0 +1,290 @@
|
||||
import { readAutoresearchRootFile } from "./files.js";
|
||||
|
||||
export type SecondaryMetricDef = {
|
||||
readonly name: string;
|
||||
readonly unit: string;
|
||||
};
|
||||
|
||||
export type AutoresearchMode = "inactive" | "active";
|
||||
|
||||
export type AutoresearchIdeasSnapshot = {
|
||||
readonly hasBacklog: boolean;
|
||||
readonly pendingCount: number;
|
||||
readonly preview: readonly string[];
|
||||
};
|
||||
|
||||
export type AutoresearchRunSnapshot = {
|
||||
readonly run: number;
|
||||
readonly commit: string;
|
||||
readonly metric: number;
|
||||
readonly metrics: Record<string, number>;
|
||||
readonly status: "keep" | "discard" | "crash";
|
||||
readonly description: string;
|
||||
readonly timestamp: number;
|
||||
readonly segment: number;
|
||||
};
|
||||
|
||||
export type AutoresearchStateSnapshot = {
|
||||
readonly name: string | null;
|
||||
readonly metricName: string;
|
||||
readonly metricUnit: string;
|
||||
readonly bestDirection: "lower" | "higher";
|
||||
readonly secondaryMetrics: readonly SecondaryMetricDef[];
|
||||
readonly currentSegment: number;
|
||||
readonly currentRunCount: number;
|
||||
readonly totalRunCount: number;
|
||||
readonly currentBaselineMetric: number | null;
|
||||
readonly currentBestMetric: number | null;
|
||||
readonly lastRun: AutoresearchRunSnapshot | null;
|
||||
readonly mode: AutoresearchMode;
|
||||
readonly hasSessionDoc: boolean;
|
||||
readonly ideas: AutoresearchIdeasSnapshot;
|
||||
};
|
||||
|
||||
type MutableStateSnapshot = {
|
||||
name: string | null;
|
||||
metricName: string;
|
||||
metricUnit: string;
|
||||
bestDirection: "lower" | "higher";
|
||||
currentSegment: number;
|
||||
currentRunCount: number;
|
||||
totalRunCount: number;
|
||||
currentBaselineMetric: number | null;
|
||||
currentBestMetric: number | null;
|
||||
lastRun: AutoresearchRunSnapshot | null;
|
||||
mode: AutoresearchMode;
|
||||
hasSessionDoc: boolean;
|
||||
ideas: AutoresearchIdeasSnapshot;
|
||||
};
|
||||
|
||||
type JsonlEntry = {
|
||||
readonly type?: string;
|
||||
readonly name?: string;
|
||||
readonly metricName?: string;
|
||||
readonly metricUnit?: string;
|
||||
readonly bestDirection?: "lower" | "higher";
|
||||
readonly run?: number;
|
||||
readonly commit?: string;
|
||||
readonly metric?: number;
|
||||
readonly metrics?: Record<string, number>;
|
||||
readonly status?: "keep" | "discard" | "crash";
|
||||
readonly description?: string;
|
||||
readonly timestamp?: number;
|
||||
readonly segment?: number;
|
||||
};
|
||||
|
||||
export function createEmptyStateSnapshot(): AutoresearchStateSnapshot {
|
||||
return {
|
||||
name: null,
|
||||
metricName: "metric",
|
||||
metricUnit: "",
|
||||
bestDirection: "lower",
|
||||
secondaryMetrics: [],
|
||||
currentSegment: 0,
|
||||
currentRunCount: 0,
|
||||
totalRunCount: 0,
|
||||
currentBaselineMetric: null,
|
||||
currentBestMetric: null,
|
||||
lastRun: null,
|
||||
mode: "inactive",
|
||||
hasSessionDoc: false,
|
||||
ideas: {
|
||||
hasBacklog: false,
|
||||
pendingCount: 0,
|
||||
preview: [],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function createMutableEmptyStateSnapshot(): MutableStateSnapshot {
|
||||
return {
|
||||
...createEmptyStateSnapshot(),
|
||||
};
|
||||
}
|
||||
|
||||
export function reconstructStateFromJsonl(cwd: string): AutoresearchStateSnapshot {
|
||||
const sessionDoc = readAutoresearchRootFile(cwd, "sessionDoc");
|
||||
const ideasBacklog = readAutoresearchRootFile(cwd, "ideasBacklog");
|
||||
const jsonl = readAutoresearchRootFile(cwd, "resultsLog");
|
||||
|
||||
const state = createMutableEmptyStateSnapshot();
|
||||
state.hasSessionDoc = sessionDoc !== null;
|
||||
state.mode = detectAutoresearchMode(sessionDoc);
|
||||
state.ideas = summarizeIdeasBacklog(ideasBacklog);
|
||||
|
||||
if (jsonl === null) {
|
||||
return {
|
||||
...state,
|
||||
secondaryMetrics: [],
|
||||
};
|
||||
}
|
||||
|
||||
const currentSecondaryMetrics = new Map<string, SecondaryMetricDef>();
|
||||
let currentRunIndex = 0;
|
||||
let hasSeenAnyRun = false;
|
||||
|
||||
const lines = jsonl
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
for (const line of lines) {
|
||||
let entry: JsonlEntry;
|
||||
try {
|
||||
entry = JSON.parse(line) as JsonlEntry;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (entry.type === "config") {
|
||||
if (entry.name) {
|
||||
state.name = entry.name;
|
||||
}
|
||||
if (entry.metricName) {
|
||||
state.metricName = entry.metricName;
|
||||
}
|
||||
if (entry.metricUnit !== undefined) {
|
||||
state.metricUnit = entry.metricUnit;
|
||||
}
|
||||
if (entry.bestDirection === "lower" || entry.bestDirection === "higher") {
|
||||
state.bestDirection = entry.bestDirection;
|
||||
}
|
||||
if (hasSeenAnyRun) {
|
||||
state.currentSegment += 1;
|
||||
}
|
||||
state.currentRunCount = 0;
|
||||
state.currentBaselineMetric = null;
|
||||
state.currentBestMetric = null;
|
||||
currentRunIndex = 0;
|
||||
currentSecondaryMetrics.clear();
|
||||
continue;
|
||||
}
|
||||
|
||||
if (typeof entry.metric !== "number") {
|
||||
continue;
|
||||
}
|
||||
|
||||
hasSeenAnyRun = true;
|
||||
currentRunIndex += 1;
|
||||
state.currentRunCount = currentRunIndex;
|
||||
state.totalRunCount += 1;
|
||||
|
||||
const run: AutoresearchRunSnapshot = {
|
||||
run: typeof entry.run === "number" ? entry.run : currentRunIndex,
|
||||
commit: entry.commit ?? "",
|
||||
metric: entry.metric,
|
||||
metrics: normalizeMetrics(entry.metrics),
|
||||
status: entry.status ?? "keep",
|
||||
description: entry.description ?? "",
|
||||
timestamp: typeof entry.timestamp === "number" ? entry.timestamp : 0,
|
||||
segment: typeof entry.segment === "number" ? entry.segment : state.currentSegment,
|
||||
};
|
||||
|
||||
if (state.currentBaselineMetric === null) {
|
||||
state.currentBaselineMetric = run.metric;
|
||||
}
|
||||
|
||||
if (run.status === "keep" && run.metric > 0) {
|
||||
if (
|
||||
state.currentBestMetric === null ||
|
||||
isBetter(run.metric, state.currentBestMetric, state.bestDirection)
|
||||
) {
|
||||
state.currentBestMetric = run.metric;
|
||||
}
|
||||
}
|
||||
|
||||
for (const metricName of Object.keys(run.metrics)) {
|
||||
if (!currentSecondaryMetrics.has(metricName)) {
|
||||
currentSecondaryMetrics.set(metricName, {
|
||||
name: metricName,
|
||||
unit: inferMetricUnit(metricName),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
state.lastRun = run;
|
||||
}
|
||||
|
||||
return {
|
||||
...state,
|
||||
secondaryMetrics: [...currentSecondaryMetrics.values()],
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeMetrics(metrics: Record<string, number> | undefined): Record<string, number> {
|
||||
if (!metrics || typeof metrics !== "object") {
|
||||
return {};
|
||||
}
|
||||
|
||||
return Object.fromEntries(
|
||||
Object.entries(metrics).filter(([, value]) => typeof value === "number"),
|
||||
);
|
||||
}
|
||||
|
||||
function detectAutoresearchMode(sessionDoc: string | null): AutoresearchMode {
|
||||
if (sessionDoc === null) {
|
||||
return "inactive";
|
||||
}
|
||||
|
||||
const normalized = sessionDoc.toLowerCase();
|
||||
if (
|
||||
normalized.includes("# autoresearch") ||
|
||||
normalized.includes("## objective") ||
|
||||
normalized.includes("## what's been tried") ||
|
||||
normalized.includes("## how to run")
|
||||
) {
|
||||
return "active";
|
||||
}
|
||||
|
||||
return sessionDoc.trim().length > 0 ? "active" : "inactive";
|
||||
}
|
||||
|
||||
function summarizeIdeasBacklog(ideasBacklog: string | null): AutoresearchIdeasSnapshot {
|
||||
if (ideasBacklog === null) {
|
||||
return {
|
||||
hasBacklog: false,
|
||||
pendingCount: 0,
|
||||
preview: [],
|
||||
};
|
||||
}
|
||||
|
||||
const ideas = ideasBacklog
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => /^([-*+]|\d+\.)\s+/.test(line))
|
||||
.map((line) => line.replace(/^([-*+]|\d+\.)\s+/, "").trim())
|
||||
.filter(Boolean);
|
||||
|
||||
return {
|
||||
hasBacklog: ideas.length > 0,
|
||||
pendingCount: ideas.length,
|
||||
preview: ideas.slice(0, 3),
|
||||
};
|
||||
}
|
||||
|
||||
function isBetter(
|
||||
current: number,
|
||||
best: number,
|
||||
direction: "lower" | "higher",
|
||||
): boolean {
|
||||
return direction === "lower" ? current < best : current > best;
|
||||
}
|
||||
|
||||
function inferMetricUnit(name: string): string {
|
||||
if (name.endsWith("_µs") || name.includes("µs")) {
|
||||
return "µs";
|
||||
}
|
||||
if (name.endsWith("_ms") || name.includes("ms")) {
|
||||
return "ms";
|
||||
}
|
||||
if (name.endsWith("_s") || name.includes("sec")) {
|
||||
return "s";
|
||||
}
|
||||
if (name.endsWith("_kb") || name.includes("kb")) {
|
||||
return "kb";
|
||||
}
|
||||
if (name.endsWith("_mb") || name.includes("mb")) {
|
||||
return "mb";
|
||||
}
|
||||
return "";
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import { Type } from "@sinclair/typebox";
|
||||
import { reconstructStateFromJsonl, type AutoresearchStateSnapshot } from "../state.js";
|
||||
import { getAutoresearchRuntimeState, type AutoresearchRuntimeSnapshot } from "../runtime-state.js";
|
||||
import { resolveToolCwd } from "./tool-cwd.js";
|
||||
|
||||
const AutoresearchStatusParams = Type.Object(
|
||||
{
|
||||
cwd: Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
"Optional working directory for repo-local autoresearch state. Use this when the tool call originates outside the target repo session cwd.",
|
||||
}),
|
||||
),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
);
|
||||
|
||||
export function createAutoresearchStatusTool(api: OpenClawPluginApi) {
|
||||
return {
|
||||
name: "autoresearch_status",
|
||||
label: "Autoresearch Status",
|
||||
description:
|
||||
"Read-only summary of autoresearch state reconstructed from root-level files.",
|
||||
parameters: AutoresearchStatusParams,
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: {
|
||||
cwd?: string;
|
||||
},
|
||||
_signal: AbortSignal,
|
||||
_onUpdate: unknown,
|
||||
) {
|
||||
const cwd = resolveToolCwd(api, params.cwd);
|
||||
const state = reconstructStateFromJsonl(cwd);
|
||||
const runtimeState = getAutoresearchRuntimeState(cwd);
|
||||
|
||||
return {
|
||||
content: [{ type: "text" as const, text: formatAutoresearchStatusText(state, runtimeState) }],
|
||||
details: {
|
||||
status: "ok",
|
||||
state,
|
||||
runtime: runtimeState,
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function formatAutoresearchStatusText(
|
||||
state: AutoresearchStateSnapshot,
|
||||
runtimeState?: AutoresearchRuntimeSnapshot,
|
||||
): string {
|
||||
const lines = [
|
||||
`Mode: ${state.mode}`,
|
||||
`Session doc: ${state.hasSessionDoc ? "present" : "missing"}`,
|
||||
`Ideas backlog: ${state.ideas.hasBacklog ? `${state.ideas.pendingCount} pending` : "empty"}`,
|
||||
`Metric: ${state.metricName} (${state.metricUnit || "unitless"}, ${state.bestDirection} is better)`,
|
||||
`Current segment: ${state.currentSegment}`,
|
||||
`Runs: ${state.currentRunCount} current / ${state.totalRunCount} total`,
|
||||
`Baseline: ${formatMetric(state.currentBaselineMetric, state.metricUnit)}`,
|
||||
`Best kept: ${formatMetric(state.currentBestMetric, state.metricUnit)}`,
|
||||
];
|
||||
|
||||
if (state.name) {
|
||||
lines.splice(1, 0, `Session: ${state.name}`);
|
||||
}
|
||||
|
||||
if (runtimeState) {
|
||||
lines.splice(
|
||||
state.name ? 2 : 1,
|
||||
0,
|
||||
`Runtime mode: ${runtimeState.mode}`,
|
||||
`Experiment window: ${runtimeState.runInFlight ? "running" : "idle"}`,
|
||||
`Queued steers: ${runtimeState.queuedSteers.length}`,
|
||||
);
|
||||
}
|
||||
|
||||
if (state.lastRun) {
|
||||
lines.push(
|
||||
`Last run: #${state.lastRun.run} ${state.lastRun.status} ${formatMetric(state.lastRun.metric, state.metricUnit)} ${state.lastRun.commit} ${state.lastRun.description}`.trim(),
|
||||
);
|
||||
}
|
||||
|
||||
if (state.ideas.preview.length > 0) {
|
||||
lines.push(`Ideas preview: ${state.ideas.preview.join(" | ")}`);
|
||||
}
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function formatMetric(value: number | null, unit: string): string {
|
||||
if (value === null) {
|
||||
return "n/a";
|
||||
}
|
||||
|
||||
const rendered = value === Math.round(value) ? `${Math.round(value)}` : value.toFixed(2);
|
||||
return `${rendered}${unit}`;
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import { InitExperimentParams } from "./schemas.js";
|
||||
import { createConfigHeader, writeConfigHeader } from "../logging.js";
|
||||
import {
|
||||
createEmptyStateSnapshot,
|
||||
reconstructStateFromJsonl,
|
||||
type AutoresearchStateSnapshot,
|
||||
} from "../state.js";
|
||||
import { resolveToolCwd } from "./tool-cwd.js";
|
||||
|
||||
export function createInitExperimentTool(api: OpenClawPluginApi) {
|
||||
return {
|
||||
name: "init_experiment",
|
||||
label: "Init Experiment",
|
||||
description:
|
||||
"Initialize the experiment session. Call once before the first run_experiment to set the name, primary metric, unit, and direction. Writes the config header to autoresearch.jsonl.",
|
||||
parameters: InitExperimentParams,
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: {
|
||||
cwd?: string;
|
||||
name: string;
|
||||
metric_name: string;
|
||||
metric_unit?: string;
|
||||
direction?: "lower" | "higher";
|
||||
},
|
||||
_signal: AbortSignal,
|
||||
_onUpdate: unknown,
|
||||
) {
|
||||
const cwd = resolveToolCwd(api, params.cwd);
|
||||
const previousState = reconstructStateFromJsonl(cwd);
|
||||
const isReinit = previousState.currentRunCount > 0;
|
||||
const nextState: AutoresearchStateSnapshot = {
|
||||
...createEmptyStateSnapshot(),
|
||||
name: params.name,
|
||||
metricName: params.metric_name,
|
||||
metricUnit: params.metric_unit ?? "",
|
||||
bestDirection: params.direction ?? "lower",
|
||||
currentSegment: isReinit ? previousState.currentSegment + 1 : previousState.currentSegment,
|
||||
};
|
||||
|
||||
try {
|
||||
writeConfigHeader(
|
||||
cwd,
|
||||
createConfigHeader({
|
||||
name: nextState.name ?? params.name,
|
||||
metricName: nextState.metricName,
|
||||
metricUnit: nextState.metricUnit,
|
||||
bestDirection: nextState.bestDirection,
|
||||
}),
|
||||
isReinit ? "append" : "create",
|
||||
);
|
||||
} catch (error) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: `Failed to write autoresearch.jsonl: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
},
|
||||
],
|
||||
details: {
|
||||
status: "error",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const reinitNote = isReinit
|
||||
? " (re-initialized - previous results archived, new baseline needed)"
|
||||
: "";
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text:
|
||||
`Experiment initialized: "${nextState.name}"${reinitNote}\n` +
|
||||
`Metric: ${nextState.metricName} (${nextState.metricUnit || "unitless"}, ${nextState.bestDirection} is better)\n` +
|
||||
"Config written to autoresearch.jsonl. Now run the baseline with run_experiment.",
|
||||
},
|
||||
],
|
||||
details: {
|
||||
status: "ok",
|
||||
state: nextState,
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,378 @@
|
||||
import * as fs from "node:fs";
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import { LogExperimentParams } from "./schemas.js";
|
||||
import { commitKeptExperiment } from "../git.js";
|
||||
import { appendResultEntry, type AutoresearchResultEntry } from "../logging.js";
|
||||
import { getAutoresearchRootFilePath } from "../files.js";
|
||||
import {
|
||||
reconstructStateFromJsonl,
|
||||
type AutoresearchStateSnapshot,
|
||||
type SecondaryMetricDef,
|
||||
} from "../state.js";
|
||||
import {
|
||||
consumeAutoresearchSteers,
|
||||
setAutoresearchRunInFlight,
|
||||
} from "../runtime-state.js";
|
||||
import { resolveToolCwd } from "./tool-cwd.js";
|
||||
|
||||
export function createLogExperimentTool(api: OpenClawPluginApi) {
|
||||
return {
|
||||
name: "log_experiment",
|
||||
label: "Log Experiment",
|
||||
description:
|
||||
"Record an experiment result. Tracks metrics and preserves keep/discard/crash semantics. Call after every run_experiment.",
|
||||
parameters: LogExperimentParams,
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: {
|
||||
cwd?: string;
|
||||
commit: string;
|
||||
metric: number;
|
||||
status: "keep" | "discard" | "crash";
|
||||
description: string;
|
||||
metrics?: Record<string, number>;
|
||||
force?: boolean;
|
||||
},
|
||||
_signal: AbortSignal,
|
||||
_onUpdate: unknown,
|
||||
) {
|
||||
const cwd = resolveToolCwd(api, params.cwd);
|
||||
const state = reconstructStateFromJsonl(cwd);
|
||||
const secondaryMetrics = params.metrics ?? {};
|
||||
|
||||
if (state.secondaryMetrics.length > 0) {
|
||||
const validationError = validateSecondaryMetrics(
|
||||
state.secondaryMetrics,
|
||||
secondaryMetrics,
|
||||
params.force ?? false,
|
||||
);
|
||||
if (validationError) {
|
||||
return {
|
||||
content: [{ type: "text" as const, text: validationError }],
|
||||
details: {
|
||||
status: "error",
|
||||
phase: "validate",
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const knownSecondaryMetrics = mergeSecondaryMetrics(state.secondaryMetrics, secondaryMetrics);
|
||||
const currentResults = readCurrentSegmentResults(cwd, state.currentSegment);
|
||||
const experiment: AutoresearchResultEntry = {
|
||||
run: state.currentRunCount + 1,
|
||||
commit: params.commit.slice(0, 7),
|
||||
metric: params.metric,
|
||||
metrics: secondaryMetrics,
|
||||
status: params.status,
|
||||
description: params.description,
|
||||
timestamp: Date.now(),
|
||||
segment: state.currentSegment,
|
||||
};
|
||||
let finalExperiment = experiment;
|
||||
|
||||
let gitSummary = "";
|
||||
let gitAction: Record<string, unknown> = {
|
||||
action: params.status === "keep" ? "commit" : "skip",
|
||||
attempted: params.status === "keep",
|
||||
};
|
||||
|
||||
if (params.status === "keep") {
|
||||
const gitResult = commitKeptExperiment({
|
||||
cwd: cwd,
|
||||
description: params.description,
|
||||
metricName: state.metricName,
|
||||
metric: params.metric,
|
||||
metrics: secondaryMetrics,
|
||||
commit: experiment.commit,
|
||||
status: "keep",
|
||||
});
|
||||
gitSummary = gitResult.summary;
|
||||
gitAction = {
|
||||
action: "commit",
|
||||
attempted: true,
|
||||
committed: gitResult.committed,
|
||||
commit: gitResult.commit,
|
||||
};
|
||||
if (gitResult.committed) {
|
||||
finalExperiment = {
|
||||
...experiment,
|
||||
commit: gitResult.commit,
|
||||
};
|
||||
}
|
||||
} else {
|
||||
gitSummary =
|
||||
`Git: skipped commit (${params.status}) - ` +
|
||||
"revert tracked changes yourself with git checkout -- . when you want to discard them.";
|
||||
gitAction = {
|
||||
action: "skip",
|
||||
attempted: false,
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
appendResultEntry(cwd, finalExperiment);
|
||||
} catch (error) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: `Failed to append autoresearch.jsonl: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
},
|
||||
],
|
||||
details: {
|
||||
status: "error",
|
||||
phase: "write",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const baselineMetric =
|
||||
currentResults.length > 0 ? currentResults[0].metric : experiment.metric;
|
||||
const baselineSecondaryMetrics = findBaselineSecondaryMetrics(
|
||||
currentResults,
|
||||
knownSecondaryMetrics,
|
||||
);
|
||||
const nextState: AutoresearchStateSnapshot = reconstructStateFromJsonl(cwd);
|
||||
const queuedSteers = consumeAutoresearchSteers(cwd);
|
||||
setAutoresearchRunInFlight(cwd, false);
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: buildResultText({
|
||||
state,
|
||||
experiment: finalExperiment,
|
||||
baselineMetric,
|
||||
baselineSecondaryMetrics,
|
||||
totalRunCount: finalExperiment.run,
|
||||
gitSummary,
|
||||
knownSecondaryMetrics,
|
||||
queuedSteers,
|
||||
}),
|
||||
},
|
||||
],
|
||||
details: {
|
||||
status: "ok",
|
||||
experiment: finalExperiment,
|
||||
state: nextState,
|
||||
git: gitAction,
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
type CurrentSegmentResult = {
|
||||
readonly metric: number;
|
||||
readonly metrics: Record<string, number>;
|
||||
};
|
||||
|
||||
function validateSecondaryMetrics(
|
||||
knownMetrics: readonly SecondaryMetricDef[],
|
||||
providedMetrics: Record<string, number>,
|
||||
force: boolean,
|
||||
): string | null {
|
||||
const knownNames = new Set(knownMetrics.map((metric) => metric.name));
|
||||
const providedNames = new Set(Object.keys(providedMetrics));
|
||||
|
||||
const missing = [...knownNames].filter((name) => !providedNames.has(name));
|
||||
if (missing.length > 0) {
|
||||
return (
|
||||
`Missing secondary metrics: ${missing.join(", ")}\n\n` +
|
||||
`You must provide all previously tracked metrics. Expected: ${[...knownNames].join(", ")}\n` +
|
||||
`Got: ${[...providedNames].join(", ") || "(none)"}\n\n` +
|
||||
`Fix: include ${missing.map((name) => `"${name}": <value>`).join(", ")} in the metrics parameter.`
|
||||
);
|
||||
}
|
||||
|
||||
const added = [...providedNames].filter((name) => !knownNames.has(name));
|
||||
if (added.length > 0 && !force) {
|
||||
return (
|
||||
`New secondary metric${added.length > 1 ? "s" : ""} not previously tracked: ${added.join(", ")}\n\n` +
|
||||
`Existing metrics: ${[...knownNames].join(", ")}\n\n` +
|
||||
"If this metric has proven very valuable to watch, call log_experiment again with force: true to add it. Otherwise, remove it from the metrics parameter."
|
||||
);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function mergeSecondaryMetrics(
|
||||
knownMetrics: readonly SecondaryMetricDef[],
|
||||
providedMetrics: Record<string, number>,
|
||||
): readonly SecondaryMetricDef[] {
|
||||
const merged = [...knownMetrics];
|
||||
for (const metricName of Object.keys(providedMetrics)) {
|
||||
if (!merged.find((metric) => metric.name === metricName)) {
|
||||
merged.push({
|
||||
name: metricName,
|
||||
unit: inferMetricUnit(metricName),
|
||||
});
|
||||
}
|
||||
}
|
||||
return merged;
|
||||
}
|
||||
|
||||
function readCurrentSegmentResults(cwd: string, segment: number): CurrentSegmentResult[] {
|
||||
const jsonlPath = getAutoresearchRootFilePath(cwd, "resultsLog");
|
||||
if (!fs.existsSync(jsonlPath)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const results: CurrentSegmentResult[] = [];
|
||||
let currentSegment = 0;
|
||||
let hasSeenResult = false;
|
||||
const lines = fs
|
||||
.readFileSync(jsonlPath, "utf8")
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
for (const line of lines) {
|
||||
let entry: Record<string, unknown>;
|
||||
try {
|
||||
entry = JSON.parse(line) as Record<string, unknown>;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (entry.type === "config") {
|
||||
if (hasSeenResult) {
|
||||
currentSegment += 1;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
hasSeenResult = true;
|
||||
if (currentSegment !== segment || typeof entry.metric !== "number") {
|
||||
continue;
|
||||
}
|
||||
|
||||
results.push({
|
||||
metric: entry.metric,
|
||||
metrics:
|
||||
entry.metrics && typeof entry.metrics === "object"
|
||||
? (entry.metrics as Record<string, number>)
|
||||
: {},
|
||||
});
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
function findBaselineSecondaryMetrics(
|
||||
currentResults: readonly CurrentSegmentResult[],
|
||||
secondaryMetrics: readonly SecondaryMetricDef[],
|
||||
): Record<string, number> {
|
||||
const baseline =
|
||||
currentResults.length > 0 ? { ...currentResults[0].metrics } : {};
|
||||
|
||||
for (const metric of secondaryMetrics) {
|
||||
if (baseline[metric.name] !== undefined) {
|
||||
continue;
|
||||
}
|
||||
for (const result of currentResults) {
|
||||
const value = result.metrics[metric.name];
|
||||
if (value !== undefined) {
|
||||
baseline[metric.name] = value;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return baseline;
|
||||
}
|
||||
|
||||
function buildResultText(options: {
|
||||
state: AutoresearchStateSnapshot;
|
||||
experiment: {
|
||||
run: number;
|
||||
commit: string;
|
||||
metric: number;
|
||||
metrics: Record<string, number>;
|
||||
status: "keep" | "discard" | "crash";
|
||||
description: string;
|
||||
};
|
||||
baselineMetric: number;
|
||||
baselineSecondaryMetrics: Record<string, number>;
|
||||
totalRunCount: number;
|
||||
gitSummary: string;
|
||||
knownSecondaryMetrics: readonly SecondaryMetricDef[];
|
||||
queuedSteers: readonly string[];
|
||||
}): string {
|
||||
let text = `Logged #${options.experiment.run}: ${options.experiment.status} - ${options.experiment.description}`;
|
||||
text += `\nBaseline ${options.state.metricName}: ${formatMetric(options.baselineMetric, options.state.metricUnit)}`;
|
||||
|
||||
if (options.experiment.run > 1 && options.experiment.status === "keep" && options.experiment.metric > 0) {
|
||||
const delta = options.experiment.metric - options.baselineMetric;
|
||||
const pct = options.baselineMetric === 0 ? null : (delta / options.baselineMetric) * 100;
|
||||
text += ` | this: ${formatMetric(options.experiment.metric, options.state.metricUnit)}`;
|
||||
if (pct !== null) {
|
||||
const sign = delta > 0 ? "+" : "";
|
||||
text += ` (${sign}${pct.toFixed(1)}%)`;
|
||||
}
|
||||
}
|
||||
|
||||
if (Object.keys(options.experiment.metrics).length > 0) {
|
||||
const parts = Object.entries(options.experiment.metrics).map(([name, value]) => {
|
||||
const metricDef = options.knownSecondaryMetrics.find((metric) => metric.name === name);
|
||||
let part = `${name}: ${formatMetric(value, metricDef?.unit ?? "")}`;
|
||||
const baselineValue = options.baselineSecondaryMetrics[name];
|
||||
if (
|
||||
baselineValue !== undefined &&
|
||||
options.experiment.run > 1 &&
|
||||
baselineValue !== 0
|
||||
) {
|
||||
const delta = value - baselineValue;
|
||||
const sign = delta > 0 ? "+" : "";
|
||||
part += ` (${sign}${((delta / baselineValue) * 100).toFixed(1)}%)`;
|
||||
}
|
||||
return part;
|
||||
});
|
||||
text += `\nSecondary: ${parts.join(" ")}`;
|
||||
}
|
||||
|
||||
text += `\n(${options.totalRunCount} experiments in current segment)`;
|
||||
text += `\n${options.gitSummary}`;
|
||||
|
||||
if (options.queuedSteers.length > 0) {
|
||||
const steerLabel = options.queuedSteers.length === 1 ? "steer" : "steers";
|
||||
text += `\n\nQueued user ${steerLabel} captured during this experiment:`;
|
||||
for (const steer of options.queuedSteers) {
|
||||
text += `\n- ${steer}`;
|
||||
}
|
||||
text +=
|
||||
"\nTreat any immediate followup turn that repeats the same steer as the normal OpenClaw queue/backlog delivery for these messages.";
|
||||
}
|
||||
|
||||
return text;
|
||||
}
|
||||
|
||||
function inferMetricUnit(name: string): string {
|
||||
if (name.endsWith("_µs") || name.includes("µs")) {
|
||||
return "µs";
|
||||
}
|
||||
if (name.endsWith("_ms") || name.includes("ms")) {
|
||||
return "ms";
|
||||
}
|
||||
if (name.endsWith("_s") || name.includes("sec")) {
|
||||
return "s";
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
function formatMetric(value: number, unit: string): string {
|
||||
const rendered =
|
||||
value === Math.round(value) ? `${Math.round(value)}` : value.toFixed(2);
|
||||
return `${addCommas(rendered)}${unit}`;
|
||||
}
|
||||
|
||||
function addCommas(value: string): string {
|
||||
const [integerPart, fractionalPart] = value.split(".");
|
||||
const normalizedInteger = integerPart.replace(/\B(?=(\d{3})+(?!\d))/g, ",");
|
||||
return fractionalPart ? `${normalizedInteger}.${fractionalPart}` : normalizedInteger;
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import { RunExperimentParams } from "./schemas.js";
|
||||
import { executeExperimentCommand } from "../execute.js";
|
||||
import { setAutoresearchRunInFlight } from "../runtime-state.js";
|
||||
import { resolveToolCwd } from "./tool-cwd.js";
|
||||
|
||||
export function createRunExperimentTool(api: OpenClawPluginApi) {
|
||||
return {
|
||||
name: "run_experiment",
|
||||
label: "Run Experiment",
|
||||
description:
|
||||
"Run a shell command as an experiment. Times wall-clock duration, captures output, detects pass/fail via exit code. Use for any autoresearch experiment.",
|
||||
parameters: RunExperimentParams,
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: {
|
||||
cwd?: string;
|
||||
command: string;
|
||||
timeout_seconds?: number;
|
||||
},
|
||||
signal: AbortSignal,
|
||||
onUpdate: ((update: unknown) => void | Promise<void>) | undefined,
|
||||
) {
|
||||
const cwd = resolveToolCwd(api, params.cwd);
|
||||
setAutoresearchRunInFlight(cwd, true);
|
||||
|
||||
if (onUpdate) {
|
||||
await onUpdate({
|
||||
content: [{ type: "text" as const, text: `Running: ${params.command}` }],
|
||||
details: { phase: "running" },
|
||||
});
|
||||
}
|
||||
|
||||
let details;
|
||||
try {
|
||||
details = await executeExperimentCommand({
|
||||
command: params.command,
|
||||
cwd,
|
||||
timeoutSeconds: params.timeout_seconds,
|
||||
signal,
|
||||
});
|
||||
} catch (error) {
|
||||
setAutoresearchRunInFlight(cwd, false);
|
||||
throw error;
|
||||
}
|
||||
|
||||
let text = "";
|
||||
if (details.timedOut) {
|
||||
text += `TIMEOUT after ${details.durationSeconds.toFixed(1)}s\n`;
|
||||
} else if (!details.passed) {
|
||||
text += `FAILED (exit code ${details.exitCode ?? "null"}) in ${details.durationSeconds.toFixed(1)}s\n`;
|
||||
} else {
|
||||
text += `PASSED in ${details.durationSeconds.toFixed(1)}s\n`;
|
||||
}
|
||||
|
||||
text += `\nLast 80 lines of output:\n${details.tailOutput || "(no output)"}`;
|
||||
|
||||
return {
|
||||
content: [{ type: "text" as const, text }],
|
||||
details,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
import { Type } from "@sinclair/typebox";
|
||||
|
||||
const CwdParam = Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
"Optional working directory for repo-local autoresearch state. Use this when the tool call originates outside the target repo session cwd.",
|
||||
}),
|
||||
);
|
||||
|
||||
export const InitExperimentParams = Type.Object({
|
||||
cwd: CwdParam,
|
||||
name: Type.String({
|
||||
description:
|
||||
'Human-readable name for this experiment session (e.g. "Optimizing liquid for fastest execution and parsing")',
|
||||
}),
|
||||
metric_name: Type.String({
|
||||
description:
|
||||
'Display name for the primary metric (e.g. "total_µs", "bundle_kb", "val_bpb").',
|
||||
}),
|
||||
metric_unit: Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
'Unit for the primary metric. Use "µs", "ms", "s", "kb", "mb", or "" for unitless. Default: "".',
|
||||
}),
|
||||
),
|
||||
direction: Type.Optional(
|
||||
Type.String({
|
||||
description: 'Whether "lower" or "higher" is better for the primary metric. Default: "lower".',
|
||||
enum: ["lower", "higher"],
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
export const RunExperimentParams = Type.Object({
|
||||
cwd: CwdParam,
|
||||
command: Type.String({
|
||||
description: "Shell command to run (e.g. 'pnpm test:vitest', 'uv run train.py')",
|
||||
}),
|
||||
timeout_seconds: Type.Optional(
|
||||
Type.Number({
|
||||
description: "Kill after this many seconds (default: 600)",
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
export const LogExperimentParams = Type.Object({
|
||||
cwd: CwdParam,
|
||||
commit: Type.String({ description: "Git commit hash (short, 7 chars)" }),
|
||||
metric: Type.Number({
|
||||
description:
|
||||
"The primary optimization metric value (e.g. seconds, val_bpb). Use 0 for crashes.",
|
||||
}),
|
||||
status: Type.String({
|
||||
description: "Result status for this experiment.",
|
||||
enum: ["keep", "discard", "crash"],
|
||||
}),
|
||||
description: Type.String({
|
||||
description: "Short description of what this experiment tried.",
|
||||
}),
|
||||
metrics: Type.Optional(
|
||||
Type.Record(Type.String(), Type.Number(), {
|
||||
description:
|
||||
'Additional metrics to track as { name: value } pairs, e.g. { "compile_µs": 4200, "render_µs": 9800 }.',
|
||||
}),
|
||||
),
|
||||
force: Type.Optional(
|
||||
Type.Boolean({
|
||||
description:
|
||||
"Set to true to allow adding a new secondary metric that was not tracked before.",
|
||||
}),
|
||||
),
|
||||
});
|
||||
@@ -0,0 +1,19 @@
|
||||
import path from "node:path";
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
|
||||
export function resolveToolCwd(api: OpenClawPluginApi, requestedCwd?: unknown): string {
|
||||
const normalizedRequestedCwd =
|
||||
typeof requestedCwd === "string" ? requestedCwd.trim() : "";
|
||||
|
||||
const cwd = normalizedRequestedCwd
|
||||
? path.isAbsolute(normalizedRequestedCwd)
|
||||
? normalizedRequestedCwd
|
||||
: api.resolvePath(normalizedRequestedCwd)
|
||||
: api.resolvePath(".");
|
||||
|
||||
if (typeof cwd !== "string" || cwd.trim().length === 0) {
|
||||
throw new Error("Could not resolve repo cwd for autoresearch tool execution.");
|
||||
}
|
||||
|
||||
return cwd;
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"id": "openclaw-autoresearch",
|
||||
"name": "Autoresearch",
|
||||
"description": "Faithful OpenClaw port of pi-autoresearch.",
|
||||
"skills": [
|
||||
"./skills"
|
||||
],
|
||||
"version": "1.0.0",
|
||||
"configSchema": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {}
|
||||
}
|
||||
}
|
||||
Generated
+1293
-3883
File diff suppressed because it is too large
Load Diff
+24
-12
@@ -1,17 +1,29 @@
|
||||
{
|
||||
"name": "pi-autoresearch",
|
||||
"name": "openclaw-autoresearch",
|
||||
"version": "1.0.0",
|
||||
"description": "Autonomous experiment loop for pi — run, measure, keep or discard. Inspired by karpathy/autoresearch.",
|
||||
"keywords": ["pi-package"],
|
||||
"license": "MIT",
|
||||
"pi": {
|
||||
"extensions": ["./extensions"],
|
||||
"skills": ["./skills"]
|
||||
"description": "Faithful OpenClaw port of pi-autoresearch.",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
||||
"test": "vitest run",
|
||||
"validate": "npm run typecheck && npm run test"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@mariozechner/pi-ai": "*",
|
||||
"@mariozechner/pi-coding-agent": "*",
|
||||
"@mariozechner/pi-tui": "*",
|
||||
"@sinclair/typebox": "*"
|
||||
"keywords": [
|
||||
"openclaw-plugin",
|
||||
"autoresearch"
|
||||
],
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"@types/node": "24.5.2",
|
||||
"typescript": "5.9.2",
|
||||
"vitest": "3.2.4"
|
||||
},
|
||||
"dependencies": {
|
||||
"@sinclair/typebox": "0.34.48"
|
||||
},
|
||||
"openclaw": {
|
||||
"extensions": [
|
||||
"./extensions/openclaw-autoresearch/index.ts"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 874 KiB |
@@ -11,7 +11,7 @@ Autonomous experiment loop: try ideas, keep what works, discard what doesn't, ne
|
||||
|
||||
- **`init_experiment`** — configure session (name, metric, unit, direction). Call again to re-initialize with a new baseline when the optimization target changes.
|
||||
- **`run_experiment`** — runs command, times it, captures output.
|
||||
- **`log_experiment`** — records result. `keep` auto-commits. `discard`/`crash` → `git checkout -- .` to revert. Always include secondary `metrics` dict. Dashboard: ctrl+x.
|
||||
- **`log_experiment`** — records result. `keep` auto-commits. `discard`/`crash` → `git checkout -- .` to revert. Always include secondary `metrics` dict.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -86,4 +86,6 @@ When there is no `autoresearch.ideas.md` file and the loop ends, the research is
|
||||
|
||||
## User Steers
|
||||
|
||||
User messages sent while an experiment is running are **automatically queued** and delivered to you after your next `log_experiment` call. Finish your current experiment first — don't stop or ask for confirmation. Incorporate the user's idea in the next experiment.
|
||||
If the host exposes the OpenClaw message hooks, user steers that arrive while an experiment is running are captured and surfaced after your next `log_experiment` call. OpenClaw may also preserve the same steer in the normal followup backlog, so if the next turn repeats a steer you already saw in `log_experiment`, treat it as the same request rather than a brand new branch of work.
|
||||
|
||||
Finish the current experiment first, then incorporate the user's idea in the next experiment. Don't stop mid-experiment or ask for confirmation unless the user explicitly interrupts the loop.
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { AUTORESEARCH_ROOT_FILES } from "../extensions/openclaw-autoresearch/src/files.js";
|
||||
import {
|
||||
buildAutoresearchCommandText,
|
||||
registerAutoresearchCommand,
|
||||
} from "../extensions/openclaw-autoresearch/src/commands/autoresearch.js";
|
||||
import { getAutoresearchRuntimeState } from "../extensions/openclaw-autoresearch/src/runtime-state.js";
|
||||
|
||||
function createTempDir(): string {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), "autoresearch-command-test-"));
|
||||
}
|
||||
|
||||
describe("buildAutoresearchCommandText", () => {
|
||||
it("guides setup when no canonical files exist", () => {
|
||||
const cwd = createTempDir();
|
||||
|
||||
expect(buildAutoresearchCommandText(cwd, "default")).toContain(
|
||||
"Recommended OpenClaw entrypoint: `/autoresearch` or `/autoresearch setup <goal>`.",
|
||||
);
|
||||
expect(buildAutoresearchCommandText(cwd, "default")).toContain(
|
||||
"Direct skill fallback: `/skill autoresearch-create`.",
|
||||
);
|
||||
});
|
||||
|
||||
it("routes active sessions back to the canonical files", () => {
|
||||
const cwd = createTempDir();
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, AUTORESEARCH_ROOT_FILES.sessionDoc),
|
||||
"# Autoresearch\n\n## Objective\n\nReduce runtime.\n",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, AUTORESEARCH_ROOT_FILES.resultsLog),
|
||||
[
|
||||
JSON.stringify({
|
||||
type: "config",
|
||||
name: "Runtime optimization",
|
||||
metricName: "total_ms",
|
||||
metricUnit: "ms",
|
||||
bestDirection: "lower",
|
||||
}),
|
||||
JSON.stringify({
|
||||
run: 1,
|
||||
commit: "abc1234",
|
||||
metric: 101,
|
||||
metrics: {},
|
||||
status: "keep",
|
||||
description: "baseline",
|
||||
timestamp: 1700000000000,
|
||||
segment: 0,
|
||||
}),
|
||||
].join("\n"),
|
||||
);
|
||||
|
||||
const text = buildAutoresearchCommandText(cwd, "status");
|
||||
|
||||
expect(text).toContain("Autoresearch session detected at repo root:");
|
||||
expect(text).toContain("Read `autoresearch.md` before resuming or changing the loop.");
|
||||
expect(text).toContain("Mode: active");
|
||||
expect(text).toContain("Runtime mode: auto");
|
||||
expect(text).toContain("Baseline: 101ms");
|
||||
});
|
||||
|
||||
it("registers a mode-aware /autoresearch command that primes resume instructions", () => {
|
||||
const cwd = createTempDir();
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, AUTORESEARCH_ROOT_FILES.sessionDoc),
|
||||
"# Autoresearch\n\n## Objective\n\nReduce runtime.\n",
|
||||
);
|
||||
|
||||
const api = {
|
||||
resolvePath: vi.fn(() => cwd),
|
||||
registerCommand: vi.fn(),
|
||||
};
|
||||
|
||||
registerAutoresearchCommand(api as never);
|
||||
|
||||
const command = api.registerCommand.mock.calls[0]?.[0];
|
||||
expect(command?.name).toBe("autoresearch");
|
||||
|
||||
const result = command.handler({ args: "resume focus parser cache", cwd });
|
||||
expect(result.text).toContain("Autoresearch mode ON.");
|
||||
expect(result.text).toContain("Captured resume instruction: focus parser cache");
|
||||
expect(getAutoresearchRuntimeState(cwd)).toMatchObject({
|
||||
mode: "on",
|
||||
pendingCommand: {
|
||||
kind: "resume",
|
||||
args: "focus parser cache",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("uses the explicit command flow for setup before falling back to the raw skill call", () => {
|
||||
const cwd = createTempDir();
|
||||
const api = {
|
||||
resolvePath: vi.fn(() => cwd),
|
||||
registerCommand: vi.fn(),
|
||||
};
|
||||
|
||||
registerAutoresearchCommand(api as never);
|
||||
|
||||
const command = api.registerCommand.mock.calls[0]?.[0];
|
||||
const result = command.handler({ args: "", cwd });
|
||||
|
||||
expect(result.text).toContain("Autoresearch mode ON.");
|
||||
expect(result.text).toContain(
|
||||
"Next step: send a normal message so the next agent turn can gather setup details.",
|
||||
);
|
||||
expect(result.text).toContain("Direct skill fallback: `/skill autoresearch-create`.");
|
||||
expect(getAutoresearchRuntimeState(cwd)).toMatchObject({
|
||||
mode: "on",
|
||||
pendingCommand: {
|
||||
kind: "setup",
|
||||
args: null,
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,3 @@
|
||||
- Retry the parser change with a safer cache key
|
||||
- Separate compile and runtime metrics
|
||||
- Investigate benchmark startup overhead
|
||||
@@ -0,0 +1,6 @@
|
||||
{"type":"config","name":"Parser optimization","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
|
||||
{"run":1,"commit":"abc1234","metric":125,"metrics":{"compile_ms":50},"status":"keep","description":"baseline","timestamp":1700000000000,"segment":0}
|
||||
{"run":2,"commit":"def5678","metric":120,"metrics":{"compile_ms":48},"status":"discard","description":"attempt 1","timestamp":1700000001000,"segment":0}
|
||||
{"type":"config","name":"Parser optimization","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
|
||||
{"run":1,"commit":"9876543","metric":130,"metrics":{"compile_ms":55,"bundle_kb":10},"status":"keep","description":"new baseline","timestamp":1700000002000,"segment":1}
|
||||
{"run":2,"commit":"7654321","metric":118,"metrics":{"compile_ms":44,"bundle_kb":9},"status":"keep","description":"keep winner","timestamp":1700000003000,"segment":1}
|
||||
@@ -0,0 +1,5 @@
|
||||
# Autoresearch
|
||||
|
||||
## Objective
|
||||
|
||||
Reduce benchmark runtime without changing behavior.
|
||||
@@ -0,0 +1,106 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { AUTORESEARCH_ROOT_FILES } from "../extensions/openclaw-autoresearch/src/files.js";
|
||||
import {
|
||||
buildBeforePromptBuildContext,
|
||||
registerAutoresearchHooks,
|
||||
} from "../extensions/openclaw-autoresearch/src/hooks.js";
|
||||
import {
|
||||
getAutoresearchRuntimeState,
|
||||
queueAutoresearchSteer,
|
||||
setAutoresearchContinuationReminder,
|
||||
setAutoresearchPendingCommand,
|
||||
setAutoresearchRunInFlight,
|
||||
setAutoresearchRuntimeMode,
|
||||
} from "../extensions/openclaw-autoresearch/src/runtime-state.js";
|
||||
|
||||
function createTempDir(prefix: string): string {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), prefix));
|
||||
}
|
||||
|
||||
function seedActiveSession(cwd: string): void {
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, AUTORESEARCH_ROOT_FILES.sessionDoc),
|
||||
"# Autoresearch\n\n## Objective\n\nReduce runtime.\n\n## What's Been Tried\n\n- baseline\n",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, AUTORESEARCH_ROOT_FILES.ideasBacklog),
|
||||
"- retry the parser cache\n- measure compile_ms separately\n",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, AUTORESEARCH_ROOT_FILES.resultsLog),
|
||||
[
|
||||
JSON.stringify({
|
||||
type: "config",
|
||||
name: "Runtime optimization",
|
||||
metricName: "total_ms",
|
||||
metricUnit: "ms",
|
||||
bestDirection: "lower",
|
||||
}),
|
||||
JSON.stringify({
|
||||
run: 1,
|
||||
commit: "abc1234",
|
||||
metric: 101,
|
||||
metrics: {},
|
||||
status: "keep",
|
||||
description: "baseline",
|
||||
timestamp: 1700000000000,
|
||||
segment: 0,
|
||||
}),
|
||||
].join("\n"),
|
||||
);
|
||||
}
|
||||
|
||||
describe("autoresearch hooks", () => {
|
||||
it("builds a stronger before_prompt_build context for active sessions", () => {
|
||||
const cwd = createTempDir("autoresearch-hooks-context-");
|
||||
seedActiveSession(cwd);
|
||||
setAutoresearchRuntimeMode(cwd, "on");
|
||||
setAutoresearchPendingCommand(cwd, {
|
||||
kind: "resume",
|
||||
args: "focus parser cache",
|
||||
});
|
||||
queueAutoresearchSteer(cwd, "try branchless parsing");
|
||||
setAutoresearchContinuationReminder(cwd, true);
|
||||
|
||||
const context = buildBeforePromptBuildContext(cwd);
|
||||
|
||||
expect(context).toContain("## Autoresearch Mode (ACTIVE)");
|
||||
expect(context).toContain("Never stop unless the user explicitly interrupts the loop.");
|
||||
expect(context).toContain("Additional resume instruction from /autoresearch: focus parser cache");
|
||||
expect(context).toContain("The previous autoresearch run ended with pending ideas.");
|
||||
expect(context).toContain("1 user steer arrived during the current experiment window");
|
||||
|
||||
const secondContext = buildBeforePromptBuildContext(cwd);
|
||||
expect(secondContext).not.toContain("The previous autoresearch run ended with pending ideas.");
|
||||
});
|
||||
|
||||
it("registers OpenClaw-compatible hooks and queues steer messages only while experiments are in flight", () => {
|
||||
const cwd = createTempDir("autoresearch-hooks-runtime-");
|
||||
const handlers = new Map<string, (event: unknown, ctx: { cwd?: string }) => unknown>();
|
||||
const api = {
|
||||
resolvePath: vi.fn(() => cwd),
|
||||
on: vi.fn((hookName: string, handler: (event: unknown, ctx: { cwd?: string }) => unknown) => {
|
||||
handlers.set(hookName, handler);
|
||||
}),
|
||||
};
|
||||
|
||||
registerAutoresearchHooks(api as never);
|
||||
|
||||
expect(handlers.has("message_received")).toBe(true);
|
||||
|
||||
setAutoresearchRunInFlight(cwd, false);
|
||||
handlers.get("message_received")?.({ text: "ignore this" }, { cwd });
|
||||
expect(getAutoresearchRuntimeState(cwd).queuedSteers).toEqual([]);
|
||||
|
||||
setAutoresearchRunInFlight(cwd, true);
|
||||
handlers.get("message_received")?.({ text: "/help" }, { cwd });
|
||||
handlers.get("message_received")?.({ text: "try branchless parsing" }, { cwd });
|
||||
|
||||
expect(getAutoresearchRuntimeState(cwd).queuedSteers).toEqual([
|
||||
"try branchless parsing",
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,528 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { createInitExperimentTool } from "../extensions/openclaw-autoresearch/src/tools/init-experiment.js";
|
||||
import { createRunExperimentTool } from "../extensions/openclaw-autoresearch/src/tools/run-experiment.js";
|
||||
import { createLogExperimentTool } from "../extensions/openclaw-autoresearch/src/tools/log-experiment.js";
|
||||
import { getAutoresearchRootFilePath } from "../extensions/openclaw-autoresearch/src/files.js";
|
||||
import * as gitModule from "../extensions/openclaw-autoresearch/src/git.js";
|
||||
import {
|
||||
getAutoresearchRuntimeState,
|
||||
queueAutoresearchSteer,
|
||||
} from "../extensions/openclaw-autoresearch/src/runtime-state.js";
|
||||
|
||||
function createTempDir(prefix: string): string {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), prefix));
|
||||
}
|
||||
|
||||
function createAbortSignal(): AbortSignal {
|
||||
return new AbortController().signal;
|
||||
}
|
||||
|
||||
function createApi(cwd: string) {
|
||||
return {
|
||||
resolvePath: vi.fn(() => cwd),
|
||||
};
|
||||
}
|
||||
|
||||
function readJsonl(cwd: string): Array<Record<string, unknown>> {
|
||||
const jsonlPath = getAutoresearchRootFilePath(cwd, "resultsLog");
|
||||
return fs
|
||||
.readFileSync(jsonlPath, "utf8")
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter(Boolean)
|
||||
.map((line) => JSON.parse(line) as Record<string, unknown>);
|
||||
}
|
||||
|
||||
async function initExperiment(
|
||||
cwd: string,
|
||||
params: {
|
||||
name: string;
|
||||
metric_name: string;
|
||||
metric_unit?: string;
|
||||
direction?: "lower" | "higher";
|
||||
},
|
||||
) {
|
||||
return await createInitExperimentTool(createApi(cwd) as never).execute(
|
||||
"tool-call",
|
||||
params,
|
||||
createAbortSignal(),
|
||||
undefined,
|
||||
);
|
||||
}
|
||||
|
||||
async function runExperiment(
|
||||
cwd: string,
|
||||
params: {
|
||||
command: string;
|
||||
timeout_seconds?: number;
|
||||
},
|
||||
onUpdate?: (update: unknown) => void | Promise<void>,
|
||||
) {
|
||||
return await createRunExperimentTool(createApi(cwd) as never).execute(
|
||||
"tool-call",
|
||||
params,
|
||||
createAbortSignal(),
|
||||
onUpdate,
|
||||
);
|
||||
}
|
||||
|
||||
async function logExperiment(
|
||||
cwd: string,
|
||||
params: {
|
||||
commit: string;
|
||||
metric: number;
|
||||
status: "keep" | "discard" | "crash";
|
||||
description: string;
|
||||
metrics?: Record<string, number>;
|
||||
force?: boolean;
|
||||
},
|
||||
) {
|
||||
return await createLogExperimentTool(createApi(cwd) as never).execute(
|
||||
"tool-call",
|
||||
params,
|
||||
createAbortSignal(),
|
||||
undefined,
|
||||
);
|
||||
}
|
||||
|
||||
async function seedExperiment(cwd: string) {
|
||||
await initExperiment(cwd, {
|
||||
name: "Parser optimization",
|
||||
metric_name: "total_ms",
|
||||
metric_unit: "ms",
|
||||
direction: "lower",
|
||||
});
|
||||
}
|
||||
|
||||
describe("experiment lifecycle tools", () => {
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("writes the first config header to the root autoresearch.jsonl file", async () => {
|
||||
const cwd = createTempDir("autoresearch-init-");
|
||||
|
||||
const result = await initExperiment(cwd, {
|
||||
name: "Parser optimization",
|
||||
metric_name: "total_ms",
|
||||
metric_unit: "ms",
|
||||
direction: "lower",
|
||||
});
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
status: "ok",
|
||||
state: {
|
||||
name: "Parser optimization",
|
||||
metricName: "total_ms",
|
||||
metricUnit: "ms",
|
||||
bestDirection: "lower",
|
||||
currentSegment: 0,
|
||||
},
|
||||
});
|
||||
expect(readJsonl(cwd)).toEqual([
|
||||
{
|
||||
type: "config",
|
||||
name: "Parser optimization",
|
||||
metricName: "total_ms",
|
||||
metricUnit: "ms",
|
||||
bestDirection: "lower",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("appends a new config header on re-init instead of overwriting prior history", async () => {
|
||||
const cwd = createTempDir("autoresearch-reinit-");
|
||||
|
||||
await initExperiment(cwd, {
|
||||
name: "Parser optimization",
|
||||
metric_name: "total_ms",
|
||||
metric_unit: "ms",
|
||||
direction: "lower",
|
||||
});
|
||||
fs.appendFileSync(
|
||||
getAutoresearchRootFilePath(cwd, "resultsLog"),
|
||||
`${JSON.stringify({
|
||||
run: 1,
|
||||
commit: "abc1234",
|
||||
metric: 120,
|
||||
metrics: {},
|
||||
status: "keep",
|
||||
description: "baseline",
|
||||
timestamp: 1700000000000,
|
||||
segment: 0,
|
||||
})}\n`,
|
||||
);
|
||||
|
||||
const result = await initExperiment(cwd, {
|
||||
name: "Parser optimization v2",
|
||||
metric_name: "total_ms",
|
||||
metric_unit: "ms",
|
||||
direction: "lower",
|
||||
});
|
||||
|
||||
expect(result.content[0]?.type).toBe("text");
|
||||
expect((result.content[0] as { text: string }).text).toContain("re-initialized");
|
||||
expect(readJsonl(cwd)).toEqual([
|
||||
{
|
||||
type: "config",
|
||||
name: "Parser optimization",
|
||||
metricName: "total_ms",
|
||||
metricUnit: "ms",
|
||||
bestDirection: "lower",
|
||||
},
|
||||
{
|
||||
run: 1,
|
||||
commit: "abc1234",
|
||||
metric: 120,
|
||||
metrics: {},
|
||||
status: "keep",
|
||||
description: "baseline",
|
||||
timestamp: 1700000000000,
|
||||
segment: 0,
|
||||
},
|
||||
{
|
||||
type: "config",
|
||||
name: "Parser optimization v2",
|
||||
metricName: "total_ms",
|
||||
metricUnit: "ms",
|
||||
bestDirection: "lower",
|
||||
},
|
||||
]);
|
||||
expect(result.details).toMatchObject({
|
||||
status: "ok",
|
||||
state: {
|
||||
currentSegment: 1,
|
||||
currentRunCount: 0,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("reports a successful run and emits a running update", async () => {
|
||||
const cwd = createTempDir("autoresearch-run-success-");
|
||||
const updates: unknown[] = [];
|
||||
|
||||
const result = await runExperiment(
|
||||
cwd,
|
||||
{
|
||||
command: "printf 'hello\\nworld\\n'",
|
||||
},
|
||||
(update) => {
|
||||
updates.push(update);
|
||||
},
|
||||
);
|
||||
|
||||
expect(updates).toEqual([
|
||||
{
|
||||
content: [{ type: "text", text: "Running: printf 'hello\\nworld\\n'" }],
|
||||
details: { phase: "running" },
|
||||
},
|
||||
]);
|
||||
expect(result.details).toMatchObject({
|
||||
command: "printf 'hello\\nworld\\n'",
|
||||
exitCode: 0,
|
||||
passed: true,
|
||||
crashed: false,
|
||||
timedOut: false,
|
||||
stdout: "hello\nworld\n",
|
||||
stderr: "",
|
||||
tailOutput: "hello\nworld",
|
||||
});
|
||||
expect((result.content[0] as { text: string }).text).toContain("PASSED");
|
||||
expect((result.content[0] as { text: string }).text).toContain(
|
||||
"Last 80 lines of output:\nhello\nworld",
|
||||
);
|
||||
});
|
||||
|
||||
it("reports failed runs with exit code and stderr in the captured tail", async () => {
|
||||
const cwd = createTempDir("autoresearch-run-failure-");
|
||||
|
||||
const result = await runExperiment(cwd, {
|
||||
command: "echo 'bad stderr' >&2; exit 7",
|
||||
});
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
exitCode: 7,
|
||||
passed: false,
|
||||
crashed: true,
|
||||
timedOut: false,
|
||||
stdout: "",
|
||||
stderr: "bad stderr\n",
|
||||
tailOutput: "bad stderr",
|
||||
});
|
||||
expect((result.content[0] as { text: string }).text).toContain(
|
||||
"FAILED (exit code 7)",
|
||||
);
|
||||
});
|
||||
|
||||
it("marks timed out runs and keeps the output-tail shape to the last 80 lines", async () => {
|
||||
const cwd = createTempDir("autoresearch-run-timeout-");
|
||||
|
||||
const timeoutResult = await runExperiment(cwd, {
|
||||
command: "echo start; sleep 0.2; echo done",
|
||||
timeout_seconds: 0.05,
|
||||
});
|
||||
|
||||
expect(timeoutResult.details).toMatchObject({
|
||||
exitCode: null,
|
||||
passed: false,
|
||||
crashed: true,
|
||||
timedOut: true,
|
||||
});
|
||||
expect((timeoutResult.content[0] as { text: string }).text).toContain("TIMEOUT");
|
||||
expect(timeoutResult.details.stdout).toContain("start\n");
|
||||
expect(timeoutResult.details.stdout).not.toContain("done\n");
|
||||
|
||||
const tailResult = await runExperiment(cwd, {
|
||||
command:
|
||||
"i=1; while [ $i -le 100 ]; do echo line-$i; i=$((i+1)); done",
|
||||
});
|
||||
|
||||
const tailLines = tailResult.details.tailOutput.split("\n");
|
||||
expect(tailLines).toHaveLength(80);
|
||||
expect(tailLines[0]).toBe("line-21");
|
||||
expect(tailLines[79]).toBe("line-100");
|
||||
expect(tailResult.details.tailOutput).not.toContain("line-20");
|
||||
});
|
||||
|
||||
it("appends a result row to autoresearch.jsonl and preserves secondary metrics in state", async () => {
|
||||
const cwd = createTempDir("autoresearch-log-append-");
|
||||
await seedExperiment(cwd);
|
||||
|
||||
const result = await logExperiment(cwd, {
|
||||
commit: "abc1234",
|
||||
metric: 120,
|
||||
status: "discard",
|
||||
description: "baseline",
|
||||
metrics: {
|
||||
compile_ms: 15,
|
||||
},
|
||||
});
|
||||
|
||||
const rows = readJsonl(cwd);
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows[1]).toMatchObject({
|
||||
run: 1,
|
||||
commit: "abc1234",
|
||||
metric: 120,
|
||||
metrics: {
|
||||
compile_ms: 15,
|
||||
},
|
||||
status: "discard",
|
||||
description: "baseline",
|
||||
segment: 0,
|
||||
});
|
||||
expect(result.details).toMatchObject({
|
||||
status: "ok",
|
||||
experiment: {
|
||||
run: 1,
|
||||
commit: "abc1234",
|
||||
},
|
||||
state: {
|
||||
currentRunCount: 1,
|
||||
totalRunCount: 1,
|
||||
secondaryMetrics: [{ name: "compile_ms", unit: "ms" }],
|
||||
},
|
||||
git: {
|
||||
action: "skip",
|
||||
attempted: false,
|
||||
},
|
||||
});
|
||||
expect((result.content[0] as { text: string }).text).toContain(
|
||||
"Git: skipped commit (discard) - revert tracked changes yourself with git checkout -- .",
|
||||
);
|
||||
});
|
||||
|
||||
it("validates missing and newly added secondary metrics unless forced", async () => {
|
||||
const cwd = createTempDir("autoresearch-log-metrics-");
|
||||
await seedExperiment(cwd);
|
||||
const baselineResult = await logExperiment(cwd, {
|
||||
commit: "abc1234",
|
||||
metric: 120,
|
||||
status: "discard",
|
||||
description: "baseline",
|
||||
metrics: {
|
||||
compile_ms: 15,
|
||||
},
|
||||
});
|
||||
expect(baselineResult.details).toMatchObject({ status: "ok" });
|
||||
|
||||
const missingMetric = await logExperiment(cwd, {
|
||||
commit: "def5678",
|
||||
metric: 110,
|
||||
status: "discard",
|
||||
description: "missing secondary metric",
|
||||
metrics: {},
|
||||
});
|
||||
|
||||
expect(missingMetric.details).toMatchObject({
|
||||
status: "error",
|
||||
phase: "validate",
|
||||
});
|
||||
expect((missingMetric.content[0] as { text: string }).text).toContain(
|
||||
"Missing secondary metrics: compile_ms",
|
||||
);
|
||||
|
||||
const newMetricWithoutForce = await logExperiment(cwd, {
|
||||
commit: "def5678",
|
||||
metric: 110,
|
||||
status: "discard",
|
||||
description: "add bundle size",
|
||||
metrics: {
|
||||
compile_ms: 12,
|
||||
bundle_kb: 7,
|
||||
},
|
||||
});
|
||||
|
||||
expect(newMetricWithoutForce.details).toMatchObject({
|
||||
status: "error",
|
||||
phase: "validate",
|
||||
});
|
||||
expect((newMetricWithoutForce.content[0] as { text: string }).text).toContain(
|
||||
"New secondary metric not previously tracked: bundle_kb",
|
||||
);
|
||||
|
||||
const forcedNewMetric = await logExperiment(cwd, {
|
||||
commit: "def5678",
|
||||
metric: 110,
|
||||
status: "discard",
|
||||
description: "add bundle size",
|
||||
metrics: {
|
||||
compile_ms: 12,
|
||||
bundle_kb: 7,
|
||||
},
|
||||
force: true,
|
||||
});
|
||||
|
||||
expect(forcedNewMetric.details).toMatchObject({
|
||||
status: "ok",
|
||||
state: {
|
||||
secondaryMetrics: [
|
||||
{ name: "compile_ms", unit: "ms" },
|
||||
{ name: "bundle_kb", unit: "kb" },
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("routes keep to commit and leaves discard/crash as manual reverts while preserving logged statuses", async () => {
|
||||
const cwd = createTempDir("autoresearch-log-status-");
|
||||
await seedExperiment(cwd);
|
||||
const commitSpy = vi.spyOn(gitModule, "commitKeptExperiment").mockReturnValue({
|
||||
attempted: true,
|
||||
committed: true,
|
||||
commit: "def5678",
|
||||
summary: "Git: committed - [main def5678] keep improved parser",
|
||||
command: {
|
||||
code: 0,
|
||||
stdout: "[main def5678] keep improved parser\n",
|
||||
stderr: "",
|
||||
combinedOutput: "[main def5678] keep improved parser",
|
||||
},
|
||||
});
|
||||
const keepResult = await logExperiment(cwd, {
|
||||
commit: "abc1234",
|
||||
metric: 95,
|
||||
status: "keep",
|
||||
description: "keep improved parser",
|
||||
});
|
||||
|
||||
expect(keepResult.details).toMatchObject({
|
||||
status: "ok",
|
||||
git: {
|
||||
action: "commit",
|
||||
attempted: true,
|
||||
committed: true,
|
||||
commit: "def5678",
|
||||
},
|
||||
experiment: {
|
||||
commit: "def5678",
|
||||
status: "keep",
|
||||
},
|
||||
});
|
||||
expect(commitSpy).toHaveBeenCalledTimes(1);
|
||||
|
||||
const discardResult = await logExperiment(cwd, {
|
||||
commit: "def5678",
|
||||
metric: 130,
|
||||
status: "discard",
|
||||
description: "discard regression",
|
||||
});
|
||||
|
||||
expect(discardResult.details).toMatchObject({
|
||||
status: "ok",
|
||||
git: {
|
||||
action: "skip",
|
||||
attempted: false,
|
||||
},
|
||||
experiment: {
|
||||
status: "discard",
|
||||
},
|
||||
});
|
||||
expect((discardResult.content[0] as { text: string }).text).toContain(
|
||||
"revert tracked changes yourself with git checkout -- .",
|
||||
);
|
||||
|
||||
const crashResult = await logExperiment(cwd, {
|
||||
commit: "def5678",
|
||||
metric: 0,
|
||||
status: "crash",
|
||||
description: "crashed benchmark",
|
||||
});
|
||||
|
||||
expect(crashResult.details).toMatchObject({
|
||||
status: "ok",
|
||||
git: {
|
||||
action: "skip",
|
||||
attempted: false,
|
||||
},
|
||||
experiment: {
|
||||
status: "crash",
|
||||
metric: 0,
|
||||
},
|
||||
});
|
||||
expect((crashResult.content[0] as { text: string }).text).toContain(
|
||||
"revert tracked changes yourself with git checkout -- .",
|
||||
);
|
||||
|
||||
const statuses = readJsonl(cwd)
|
||||
.filter((entry) => entry.type !== "config")
|
||||
.map((entry) => entry.status);
|
||||
expect(statuses).toEqual(["keep", "discard", "crash"]);
|
||||
});
|
||||
|
||||
it("keeps the experiment window open across run_experiment and surfaces queued steers in log_experiment", async () => {
|
||||
const cwd = createTempDir("autoresearch-queued-steers-");
|
||||
await seedExperiment(cwd);
|
||||
|
||||
await runExperiment(cwd, {
|
||||
command: "printf 'baseline\\n'",
|
||||
});
|
||||
|
||||
expect(getAutoresearchRuntimeState(cwd)).toMatchObject({
|
||||
runInFlight: true,
|
||||
});
|
||||
|
||||
queueAutoresearchSteer(cwd, "try a parser cache");
|
||||
queueAutoresearchSteer(cwd, "watch compile_ms too");
|
||||
|
||||
const result = await logExperiment(cwd, {
|
||||
commit: "abc1234",
|
||||
metric: 120,
|
||||
status: "discard",
|
||||
description: "baseline",
|
||||
});
|
||||
|
||||
expect((result.content[0] as { text: string }).text).toContain(
|
||||
"Queued user steers captured during this experiment:",
|
||||
);
|
||||
expect((result.content[0] as { text: string }).text).toContain("- try a parser cache");
|
||||
expect((result.content[0] as { text: string }).text).toContain("- watch compile_ms too");
|
||||
expect(getAutoresearchRuntimeState(cwd)).toMatchObject({
|
||||
runInFlight: false,
|
||||
queuedSteers: [],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import plugin from "../extensions/openclaw-autoresearch/index.js";
|
||||
|
||||
describe("plugin registration", () => {
|
||||
it("registers the command, OpenClaw hooks, and all tool surfaces", () => {
|
||||
const api = {
|
||||
resolvePath: vi.fn(() => "/tmp/repo"),
|
||||
registerTool: vi.fn(),
|
||||
registerCommand: vi.fn(),
|
||||
on: vi.fn(),
|
||||
};
|
||||
|
||||
plugin.register(api);
|
||||
|
||||
expect(api.registerCommand).toHaveBeenCalledTimes(1);
|
||||
expect(api.on).toHaveBeenCalledTimes(4);
|
||||
expect(api.on.mock.calls.map(([hookName]) => hookName)).toEqual([
|
||||
"before_prompt_build",
|
||||
"message_received",
|
||||
"agent_end",
|
||||
"session_end",
|
||||
]);
|
||||
expect(api.registerTool).toHaveBeenCalledTimes(4);
|
||||
expect(api.registerTool.mock.calls.map(([tool]) => tool.name)).toEqual([
|
||||
"init_experiment",
|
||||
"run_experiment",
|
||||
"log_experiment",
|
||||
"autoresearch_status",
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
export type CommandRegistration = {
|
||||
name: string;
|
||||
description: string;
|
||||
acceptsArgs?: boolean;
|
||||
handler: (ctx: { args?: string; cwd?: string }) => { text: string };
|
||||
};
|
||||
|
||||
export type ToolRegistration = {
|
||||
name: string;
|
||||
};
|
||||
|
||||
export type OpenClawPluginApi = {
|
||||
resolvePath(path: string): string;
|
||||
registerTool(tool: ToolRegistration): void;
|
||||
registerCommand(command: CommandRegistration): void;
|
||||
on?(hookName: string, handler: (event: unknown, ctx: { cwd?: string }) => unknown): void;
|
||||
registerHook?(
|
||||
hookName: string,
|
||||
handler: (event: { systemPrompt?: string }, ctx: { cwd?: string }) => { systemPrompt?: string } | void,
|
||||
): void;
|
||||
};
|
||||
|
||||
export function emptyPluginConfigSchema(): Record<string, never> {
|
||||
return {};
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import path from "node:path";
|
||||
import { reconstructStateFromJsonl } from "../extensions/openclaw-autoresearch/src/state.js";
|
||||
|
||||
const activeSessionFixture = path.resolve("test/fixtures/active-session");
|
||||
|
||||
describe("reconstructStateFromJsonl", () => {
|
||||
it("rebuilds the active session snapshot from canonical fixtures", () => {
|
||||
const state = reconstructStateFromJsonl(activeSessionFixture);
|
||||
|
||||
expect(state.name).toBe("Parser optimization");
|
||||
expect(state.mode).toBe("active");
|
||||
expect(state.hasSessionDoc).toBe(true);
|
||||
expect(state.currentSegment).toBe(1);
|
||||
expect(state.currentRunCount).toBe(2);
|
||||
expect(state.totalRunCount).toBe(4);
|
||||
expect(state.currentBaselineMetric).toBe(130);
|
||||
expect(state.currentBestMetric).toBe(118);
|
||||
expect(state.lastRun).toMatchObject({
|
||||
run: 2,
|
||||
commit: "7654321",
|
||||
metric: 118,
|
||||
status: "keep",
|
||||
description: "keep winner",
|
||||
segment: 1,
|
||||
});
|
||||
expect(state.secondaryMetrics).toEqual([
|
||||
{ name: "compile_ms", unit: "ms" },
|
||||
{ name: "bundle_kb", unit: "kb" },
|
||||
]);
|
||||
expect(state.ideas).toEqual({
|
||||
hasBacklog: true,
|
||||
pendingCount: 3,
|
||||
preview: [
|
||||
"Retry the parser change with a safer cache key",
|
||||
"Separate compile and runtime metrics",
|
||||
"Investigate benchmark startup overhead",
|
||||
],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,28 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { formatAutoresearchStatusText } from "../extensions/openclaw-autoresearch/src/tools/autoresearch-status.js";
|
||||
import { reconstructStateFromJsonl } from "../extensions/openclaw-autoresearch/src/state.js";
|
||||
import { getAutoresearchRuntimeState } from "../extensions/openclaw-autoresearch/src/runtime-state.js";
|
||||
import path from "node:path";
|
||||
|
||||
const activeSessionFixture = path.resolve("test/fixtures/active-session");
|
||||
|
||||
describe("formatAutoresearchStatusText", () => {
|
||||
it("renders a concise status summary from reconstructed state", () => {
|
||||
const text = formatAutoresearchStatusText(
|
||||
reconstructStateFromJsonl(activeSessionFixture),
|
||||
getAutoresearchRuntimeState(activeSessionFixture),
|
||||
);
|
||||
|
||||
expect(text).toContain("Session: Parser optimization");
|
||||
expect(text).toContain("Runtime mode: auto");
|
||||
expect(text).toContain("Experiment window: idle");
|
||||
expect(text).toContain("Queued steers: 0");
|
||||
expect(text).toContain("Metric: total_ms (ms, lower is better)");
|
||||
expect(text).toContain("Runs: 2 current / 4 total");
|
||||
expect(text).toContain("Best kept: 118ms");
|
||||
expect(text).toContain("Last run: #2 keep 118ms 7654321 keep winner");
|
||||
expect(text).toContain(
|
||||
"Ideas preview: Retry the parser change with a safer cache key | Separate compile and runtime metrics | Investigate benchmark startup overhead",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,184 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { createInitExperimentTool } from "../extensions/openclaw-autoresearch/src/tools/init-experiment.js";
|
||||
import { createAutoresearchStatusTool } from "../extensions/openclaw-autoresearch/src/tools/autoresearch-status.js";
|
||||
import { createRunExperimentTool } from "../extensions/openclaw-autoresearch/src/tools/run-experiment.js";
|
||||
import { createLogExperimentTool } from "../extensions/openclaw-autoresearch/src/tools/log-experiment.js";
|
||||
import { AUTORESEARCH_ROOT_FILES } from "../extensions/openclaw-autoresearch/src/files.js";
|
||||
|
||||
function createTempDir(): string {
|
||||
return fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "autoresearch-tool-test-")));
|
||||
}
|
||||
|
||||
function createApi(cwd: string) {
|
||||
return {
|
||||
resolvePath: vi.fn(() => cwd),
|
||||
};
|
||||
}
|
||||
|
||||
describe("autoresearch tools", () => {
|
||||
it("init_experiment resolves cwd from api.resolvePath instead of expecting a fifth execute arg", async () => {
|
||||
const cwd = createTempDir();
|
||||
const api = createApi(cwd);
|
||||
const tool = createInitExperimentTool(api as never);
|
||||
|
||||
const result = await tool.execute(
|
||||
"call-1",
|
||||
{
|
||||
name: "Repo robustness",
|
||||
metric_name: "escaped_mutations",
|
||||
metric_unit: "",
|
||||
direction: "lower",
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(api.resolvePath).toHaveBeenCalledWith(".");
|
||||
expect(result.details).toMatchObject({ status: "ok" });
|
||||
expect(fs.existsSync(path.join(cwd, AUTORESEARCH_ROOT_FILES.resultsLog))).toBe(true);
|
||||
});
|
||||
|
||||
it("autoresearch_status resolves cwd from api.resolvePath", async () => {
|
||||
const cwd = createTempDir();
|
||||
const api = createApi(cwd);
|
||||
await createInitExperimentTool(api as never).execute(
|
||||
"call-1",
|
||||
{
|
||||
name: "Repo robustness",
|
||||
metric_name: "escaped_mutations",
|
||||
metric_unit: "",
|
||||
direction: "lower",
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
const result = await createAutoresearchStatusTool(api as never).execute(
|
||||
"call-2",
|
||||
{},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(api.resolvePath).toHaveBeenCalledWith(".");
|
||||
expect(result.content[0]?.text).toContain("Session: Repo robustness");
|
||||
expect(result.content[0]?.text).toContain("Metric: escaped_mutations");
|
||||
});
|
||||
|
||||
it("run_experiment resolves cwd from api.resolvePath", async () => {
|
||||
const cwd = createTempDir();
|
||||
const api = createApi(cwd);
|
||||
const updates: unknown[] = [];
|
||||
|
||||
const result = await createRunExperimentTool(api as never).execute(
|
||||
"call-3",
|
||||
{ command: "node -e \"console.log('ok')\"", timeout_seconds: 10 },
|
||||
new AbortController().signal,
|
||||
async (update) => {
|
||||
updates.push(update);
|
||||
},
|
||||
);
|
||||
|
||||
expect(api.resolvePath).toHaveBeenCalledWith(".");
|
||||
expect(updates.length).toBe(1);
|
||||
expect(result.details).toMatchObject({ passed: true, timedOut: false });
|
||||
});
|
||||
|
||||
it("log_experiment resolves cwd from api.resolvePath", async () => {
|
||||
const cwd = createTempDir();
|
||||
const api = createApi(cwd);
|
||||
await createInitExperimentTool(api as never).execute(
|
||||
"call-1",
|
||||
{
|
||||
name: "Repo robustness",
|
||||
metric_name: "escaped_mutations",
|
||||
metric_unit: "",
|
||||
direction: "lower",
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
const result = await createLogExperimentTool(api as never).execute(
|
||||
"call-4",
|
||||
{
|
||||
commit: "abc1234",
|
||||
metric: 5,
|
||||
status: "discard",
|
||||
description: "baseline",
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(api.resolvePath).toHaveBeenCalledWith(".");
|
||||
expect(result.details).toMatchObject({ status: "ok" });
|
||||
expect(result.content[0]?.text).toContain("Logged #1: discard - baseline");
|
||||
});
|
||||
|
||||
it("supports explicit cwd overrides for nested repo tool execution", async () => {
|
||||
const workspaceCwd = createTempDir();
|
||||
const repoCwd = createTempDir();
|
||||
const api = createApi(workspaceCwd);
|
||||
|
||||
const initResult = await createInitExperimentTool(api as never).execute(
|
||||
"call-1",
|
||||
{
|
||||
cwd: repoCwd,
|
||||
name: "Nested repo robustness",
|
||||
metric_name: "escaped_mutations",
|
||||
metric_unit: "",
|
||||
direction: "lower",
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(initResult.details).toMatchObject({ status: "ok" });
|
||||
expect(api.resolvePath).not.toHaveBeenCalled();
|
||||
expect(fs.existsSync(path.join(repoCwd, AUTORESEARCH_ROOT_FILES.resultsLog))).toBe(true);
|
||||
expect(fs.existsSync(path.join(workspaceCwd, AUTORESEARCH_ROOT_FILES.resultsLog))).toBe(false);
|
||||
|
||||
const runResult = await createRunExperimentTool(api as never).execute(
|
||||
"call-2",
|
||||
{
|
||||
cwd: repoCwd,
|
||||
command: "node -e \"console.log(process.cwd())\"",
|
||||
timeout_seconds: 10,
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(runResult.details).toMatchObject({ passed: true, timedOut: false });
|
||||
expect(runResult.details.stdout.trim()).toBe(repoCwd);
|
||||
|
||||
const statusResult = await createAutoresearchStatusTool(api as never).execute(
|
||||
"call-3",
|
||||
{ cwd: repoCwd },
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(statusResult.content[0]?.text).toContain("Session: Nested repo robustness");
|
||||
|
||||
const logResult = await createLogExperimentTool(api as never).execute(
|
||||
"call-4",
|
||||
{
|
||||
cwd: repoCwd,
|
||||
commit: "abc1234",
|
||||
metric: 5,
|
||||
status: "discard",
|
||||
description: "baseline",
|
||||
},
|
||||
new AbortController().signal,
|
||||
undefined,
|
||||
);
|
||||
|
||||
expect(logResult.details).toMatchObject({ status: "ok" });
|
||||
expect(logResult.content[0]?.text).toContain("Logged #1: discard - baseline");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"skipLibCheck": true,
|
||||
"types": ["node", "vitest/globals"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"openclaw/plugin-sdk/core": ["./test/shims/openclaw-plugin-sdk-core.ts"]
|
||||
}
|
||||
},
|
||||
"include": [
|
||||
"extensions/openclaw-autoresearch/**/*.ts",
|
||||
"test/**/*.ts"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
import { defineConfig } from "vitest/config";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const rootDir = fileURLToPath(new URL(".", import.meta.url));
|
||||
const shimPath = fileURLToPath(
|
||||
new URL("./test/shims/openclaw-plugin-sdk-core.ts", import.meta.url),
|
||||
);
|
||||
|
||||
export default defineConfig({
|
||||
resolve: {
|
||||
alias: {
|
||||
"openclaw/plugin-sdk/core": shimPath,
|
||||
},
|
||||
},
|
||||
test: {
|
||||
environment: "node",
|
||||
include: ["test/**/*.test.ts"],
|
||||
root: rootDir,
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user