admin/lobster
1
0
Fork 0
mirror of https://github.com/openclaw/lobster.git synced 2026-08-14 00:48:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: admin/lobster:v2026.6.11
Branches Tags
admin/lobster:main admin/lobster:chore/setup-baseline-20260522
admin/lobster:v2026.6.11 admin/lobster:v2026.5.22 admin/lobster:v2026.4.6
...
compare: admin/lobster:main
Branches Tags
admin/lobster:main admin/lobster:chore/setup-baseline-20260522
admin/lobster:v2026.6.11 admin/lobster:v2026.5.22 admin/lobster:v2026.4.6
20 Commits
v2026.6.11 ... main
Author SHA1 Message Date
Krasimir Kralev afb198a96b test(commands): cover the where filter command (#121) 
Adds characterization tests for src/commands/stdlib/where.ts, which shipped
with no direct coverage. Exercises the real command through parsePipeline +
runPipeline: literal coercion (number/boolean/null), the = to == normalization,
dotted-path resolution with non-object safety, all six comparison operators,
the loose-equality/missing-path quirk, and both throw paths (missing expression,
operator-less expression). No production code changes.
 2026-08-13 12:29:32 -07:00
Yiğit ERDOĞANandClaude Opus 5 096f5fedcb fix: workflow steps keep waiting for the model after timeout_ms expires (#132) 
* fix(llm): honor step timeouts and cancellation during model calls

llm.invoke never read ctx.signal, and both adapter helpers called fetch
without one, so nothing could end a request once it was in flight. A
workflow step with timeout_ms kept waiting for the adapter and, when the
model eventually answered, reported the step as completed well past its
budget. Shell steps already honor the same signal, so the two step kinds
disagreed about what timeout_ms means.

Thread ctx.signal into both adapter fetch calls, check it before each
attempt so an already-cancelled run does not pay for another call, and
rethrow abort errors unwrapped: the workflow runner recognizes timeouts
and external cancellation by the error's identity, so wrapping them in
"request failed" would misreport the outcome even once the request stops.

* fix(llm): stop waiting for adapters that ignore the cancellation signal

Threading the signal into fetch only covers the HTTP adapters. An injected
ctx.llmAdapters adapter is awaited directly, so one that never observes
ctx.signal still holds a timed-out step open for as long as it likes, which
leaves SDK and tool-runtime users on the weaker timeout contract the rest of
this change removes.

Await the adapter call against the signal instead of the adapter alone. The
adapter's own work cannot be killed from here, but the step stops waiting on
it and the abort reason reaches the workflow unchanged. Cooperative adapters
are unaffected: they still receive ctx and can end their own work.

* fix(llm): do not finish a cancelled run from cached results

Run-state and file-cache hits return before any adapter call, so a run that
was already cancelled still completed successfully from cached data. That
contradicts the rest of this change: the same invocation reports cancellation
when it has to reach a model and success when it does not, purely on cache
contents.

Check the signal once at entry, before either lookup. The retry-loop check
stays, since cancellation can also arrive between validation attempts.

* docs(readme): state the cancellation contract for injected LLM adapters

Racing an adapter promise against the step signal restores workflow liveness,
but it cannot cancel work the adapter has already started: an injected adapter
that ignores `ctx.signal` keeps its model request running after Lobster stops
awaiting it, and a configured retry can then overlap it. That is a contract for
the host to meet, so the host-facing docs have to say it rather than leaving it
to be discovered from a duplicated charge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(llm): re-check cancellation after each awaited reuse boundary

The entry check only caught a run that was already cancelled when the
command started. Draining pipeline input waits on the upstream step, and
the run-state and cache lookups are file I/O, so a step timeout could
fire during any of them and the reuse branch would still return its
stored answer as a successful result.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(llm): do not report a step that finished after its deadline

The cancellation checks stopped at the adapter call, but both return
paths then await run-state and cache writes. A deadline crossed during
those writes was never observed, so the step returned success and the
workflow cleared its timer -- a model step could still complete late.

The check goes after both writes rather than between them: the answer
has already been paid for, and leaving it stored lets the retry replay
it instead of calling the model a second time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* test(llm): let the persistence tests run where the state fix has not landed

The two cancellation-during-write tests let the command create the cache
namespace directory itself, which is where #125 throws on Windows: the
recursive mkdir reports an extended-length path and the chain sync then
opens `<dir>\C:`. Both tests failed there before reaching an assertion,
so this branch added two Windows failures over main for a reason that is
not its own.

Creating the directory up front is what the seeded-cache tests in this
file already do. ensureDirectory only syncs a chain it created, so an
existing directory never reaches the broken path, and the tests still
fail against the code they cover -- now on the missing rejection rather
than on ENOENT.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwU1bvb9Pt3cR662P52CTp

* fix(llm): let a host's own abort reason still read as cancellation

The workflow runner classifies cancellation by name alone: only
`AbortError` or `ABORT_ERR` bypasses the step's retry and `on_error`
policy. Every abort path here rejected with `signal.reason` untouched,
so a host that calls `controller.abort(new Error("stop"))` handed the
runner an ordinary-looking error. A cancelled `llm.invoke` step under
`on_error: continue` was recorded as a step failure and the run carried
on -- returning `status: "ok"` for a run the host had cancelled.

The gap is narrow, which is why it survived earlier review. An HTTP
adapter is saved by fetch rejecting with its own `AbortError`, and the
runner's own step timer is not affected. It opens only on the injected
`ctx.llmAdapters` path, where this command's rejection is the only thing
the runner ever sees.

Normalize at the boundary instead of guessing upstream: a reason that
already reads as an abort passes through untouched, anything else keeps
its message and moves to `cause` under an abort-identified error. The
host's reason is never discarded, and callers that already match on
`AbortError` keep matching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WHhk7xRwXzYcSXvw2kp5F5

* fix(llm): watch an adapter that rejects after aborting the run

`abortable` guards against an adapter that ignores `ctx.signal`, but its
already-aborted branch rejected the wrapper and returned before anything
was attached to the adapter's own promise. An adapter that cancels the
run from inside its own `invoke` -- an SDK client tearing itself down on
a fatal error -- and then rejects leaves that rejection unobserved.

The step still fails correctly, with the `AbortError` the runner needs.
The damage lands afterwards: under Node's default unhandled-rejection
handling the process exits on an error nothing is waiting for, seconds
after the run was cancelled cleanly.

Attach settlement handling first in both branches, so the promise is
watched before an abort can win the race. Removing a listener that was
never added is harmless, which keeps the two paths one shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hFZxjivm9JTTvmRjQ4A8S

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 12:28:51 -07:00
Yiğit ERDOĞANClaude Opus 5Peter Steinberger
 		911f35c9b8 fix: workflows are charged for cached model answers and stop on a budget they never spent (#134) 
* fix(workflows): do not bill cached llm.invoke replays against cost_limit

A cache or run-state hit returns the stored result verbatim, including the
usage of the call that produced it. trackStepCost recorded that usage again,
so every replay was charged as if it had reached a provider. A workflow that
asks the same question in several steps therefore reports a multiple of what
it spent, and cost_limit with action: stop aborts a run that stayed well
inside its budget.

Skip items whose source is "cache" or "run_state". Those are the only two
paths that re-emit a stored result, and no tokens are consumed on either, so
the reported token totals stay truthful too.

* fix(llm): mark replayed results so live calls are never mistaken for them

The cost guard keyed on source being "cache" or "run_state", but source is not
a replay marker: for a direct adapter it defaults to the provider name
(llm_invoke.ts:549), so an embedder registering an adapter under either name
produces live results that the guard would silently drop from _meta.cost and
cost_limit. Any step emitting that source field had the same effect.

Set an explicit replayed flag at the two sites that re-emit a stored item, and
key the guard on it. The flag is added when the item is replayed, not when it
is stored, so entries already on disk are covered.

* fix(workflows): keep billing workflow output that only looks replayed

trackStepCost inspects every JSON result a workflow step produces, not just
LLM results, so keying the exemption on a bare replayed field let any command
that reports its own replay state beside a real usage object drop out of
_meta.cost and slip past cost_limit. Under-reporting spend is the mirror of
the bug this branch fixes.

Gate the exemption on the normalized item llm.invoke and llm_task.invoke
actually re-emit: the replay marker plus cached, a known item kind, and the
string cacheKey, status, createdAt, and source fields the command always
writes. A shell step or an unrelated tool that happens to carry replayed no
longer qualifies. Checking the shape alone would not be enough either, since
a live call through an adapter registered as "cache" carries the same kind.

* fix(workflows): key the replay exemption to in-process provenance

The replay marker was a set of JSON fields, and workflow cost accounting
reads the JSON of every step — including shell steps, whose stdout is
parsed straight into the same shape. A step that printed a replay-shaped
object beside a real `usage` object dropped out of `_meta.cost`, which
under-reports spend and lets a configured `cost_limit` be bypassed.

Mark replays with a symbol key instead. `JSON.parse` cannot produce one,
so only items this process built in `llm.invoke` are exempt; the public
`replayed` field stays for consumers but no longer grants the exemption.

Tests now drive the exemption through the real path (a pipeline step
running `llm.invoke` against a stub provider, for both the cache and
run-state replays) and pin the forged case: a shell step printing every
accepted replay field plus real usage is still billed and still trips
`cost_limit`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6UVfFPP39RkoYx5jZ3KKM

* fix(workflows): keep replay provenance through a rendered pipeline

`llm.invoke | json` is a supported step shape, and it defeated the
symbol-keyed replay exemption. The renderer prints its items and returns
an empty stream, so the step has no pipeline items and its JSON is parsed
back out of stdout — objects that no symbol can survive into. Cached and
run-state replays were billed again there, inflating `_meta.cost` and
stopping workflows below their real provider spend.

Keep the originals instead of trying to recover them. Each stage's
renderer now records the objects it was handed, the pipeline returns them
alongside its items, and a step whose pipeline produced no items carries
them on a non-enumerable symbol key. Cost accounting reads those in
preference to the re-parsed JSON, so a rendered replay is recognized for
what it is.

Nothing about the forged case changes: the side channel holds only items
this process built, so a step that prints the full replay shape — through
a renderer or not — is still billed and still trips `cost_limit`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YJKpXm7WiSbyjQzrFS5Ps

* fix(llm): mark the usage record so a projection keeps replay provenance

`pick model,usage` builds a new object out of named fields, so a replayed
item's own mark never reaches workflow cost accounting and the replay is
billed again — the same defect the renderer had, one command over.

Mark the usage record as well as the item. A projection carries the usage
record across by reference, and the usage record is what gets billed, so
the exemption survives without accounting having to recognize every
command that can build a new object. `where`, `head`, `sort` and `dedupe`
yield the original item and were already covered; `group_by` and `map
--wrap` nest the usage where nothing bills it.

`JSON.parse` still cannot produce the key, so a step that prints a usage
object is billed whether or not it is projected or rendered first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YJKpXm7WiSbyjQzrFS5Ps

* fix(llm): bill a replay that stands in for a retried step's live call

A workflow records a step's cost only once the step succeeds, but `llm.invoke`
persists its answer to run state and the cache before it returns. So a step that
fails *after* its LLM call and is retried never bills the live item, and the
retry replays the stored answer -- which the replay exemption then dropped,
reporting $0 for a provider call that really happened.

Provenance now carries the cache key and whether the item was replayed, and a
live call opens an unbilled charge against that key. Accounting claims the
charge, so the live item and every replay of it settle it exactly once: a replay
is exempt only while the call behind it has already been paid for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(llm): scope the unbilled-spend ledger to the run that opened it

The ledger of live calls awaiting a charge was process-global, so a call made
outside a cost-tracked run -- the SDK pipeline API, or a workflow whose step
never succeeded -- left a key behind that a later, unrelated workflow could
claim by replaying the same cache entry. That workflow called no provider, yet
its `_meta.cost` and `cost_limit` grew by someone else's spend.

The ledger is now created per workflow run and reaches commands through the
pipeline context, so only the run that paid for a call can recover its charge
from a replay. A live call with no ledger in context opens no charge at all,
which is what an SDK caller outside cost accounting should do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(llm): count outstanding charges per cache key, not just their presence

Two identical calls that race on a cold cache -- two parallel branches asking
the same question -- are two provider charges under one cache key. The ledger
held a set of keys, so both recorded the same entry and only one of the replays
that later stood in for them could settle it. A retried step therefore reported
one call's spend for two, in `_meta.cost` and against `cost_limit`.

Charges are now counted per key, so N live calls can be settled N times and
never fewer. A replay still settles at most one charge, so nothing is billed
twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(workflows): carry recorded spend across an approval or input resume

A gate pauses a run; it does not reset what the run has spent. The resume is a
separate run with its own accounting, so the provider call made before the gate
lived only in the paused run's record: a later step repeating that prompt
replayed it and correctly billed nothing, leaving the call in no total at all.
The same reset let a `cost_limit` be walked past one gate at a time, on `main`
as much as here.

The paused run now stores its cost summary in the resume state, and the run that
resumes seeds its tracker from it, so `_meta.cost` covers the whole workflow and
the budget survives the pause. Stored entries are rebuilt through the same
normalization as live usage, and a resume state written before this change
simply seeds nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(workflows): keep an unbilled call across a pipeline input resume

A pipeline can pause between paying for a call and billing it: `llm.invoke |
ask` suspends in tool mode after the model has answered, and `ask` consumes the
item that carried the usage, so the step never reports it. The charge existed
only as an outstanding entry in the paused run's ledger, which the resumed run
recreated empty -- so the replay a later step produced was exempt and the call
was billed nowhere.

Outstanding charges now travel in the same resume state as the cost summary and
reopen on resume. It is the same run continuing, so this cannot hand a charge to
an unrelated one, and a resume state written before this change reopens nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(llm): open the charge when the provider answers, not after it is stored

`record()` ran after `persistOutputs` and the cache write. Either can fail --
an unwritable `LOBSTER_CACHE_DIR`, a full disk -- after run state already holds
a replayable copy of the answer, so the step failed, the retry succeeded from
that replay, and the replay had no charge to settle: a completed provider call
was billed nowhere.

The charge exists the moment the provider answers, so it is opened there. A
store that then fails leaves an entry nothing claims, which is bounded and
per-run; the reverse dropped real spend.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* docs(readme): state that spend is counted once and survives a pause

The two accounting properties this branch establishes are invisible from the
outside until a bill disagrees with a budget: a replayed answer is not charged
again, and a workflow that pauses at a gate keeps what it has spent, so a cost
limit covers the run rather than the steps after the last pause. The second one
changes what an existing resumed workflow does, which is a thing to read before
it happens.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

* fix(workflows): carry replay provenance across a composed workflow

A `workflow:` step rebuilds its result from the child's serialized
output, so the in-process marker that says which model call an item
replays did not survive the boundary. A child ending in a rendered
pipeline such as `llm.invoke | json` therefore handed back an unmarked
replay, and the composing run billed it again and could stop on a
cost_limit it never spent.

The child now exposes the items it produced on its run result, and its
ledger is chained to the composing run's, so a live call is opened as a
charge in both: each of them counts it once at its own boundary, and a
replay neither of them paid for still settles nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): settle live results before the replays billed with them

A step's parallel branches are accounted in declaration order. A cached
replay declared before an identical --refresh branch claimed the charge
that branch had just opened and was billed for it, and the live branch
was billed whatever its own claim returned -- one provider call charged
twice, enough to stop a run at a budget it had not spent.

Replays are now held back until every live result accounted alongside
them has settled its own charge, so a replay can only draw on a call
nothing else will bill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): bill a call no item ever carried to the accounting point

An item is the usual carrier of a call's usage, but three paths lose it
before anything is billed: a renderer consumes the pipeline and a later
stage emits items of its own, an `ask` gate swallows the only item and
the resume produces no replay, and a composed run ends on a step that
carries nothing back. In each the run had already paid the provider, and
the spend was missing from both `_meta.cost` and `cost_limit`.

Each charge now carries what the call cost, and a run settles whatever
is still open when it completes. A paused run never reaches that point,
so its charges keep travelling in the resume state instead.

The same rule reaches a call made by a step that then failed: nothing
billed its items, so the charge is the only record left that the
provider answered, and the run that made the call is where it belongs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): settle a charge from the copy of an item that lost its mark

A stage that JSON round-trips the stream hands on an item that kept
`model` and `usage` and lost everything this process attached to it.
That copy is billed, as it always was, but nothing settled the charge
behind it, so the end-of-run pass billed the same provider call again.

The copy now settles it. The public cache key alone cannot -- any step
can print one -- so the ledger settles a charge only when the cost being
billed is the cost it recorded. An invented item can still add spend to
a run, which was always true, but it cannot hide any.

Comparing only the numbers matters: a live item's usage record carries
this module's provenance symbol, which a copy that went through JSON
never can, so comparing the objects whole never matches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): bill each call at its own cost, and before the next step

Two ways the ledger still reported the wrong number.

A replay was billed from the copy it carries. Identical calls that race
on a cold cache are separate charges under one key, only one of their
answers is stored, and every replay repeats that one -- so a retried
step reported both calls at the price of whichever won the cache write.
The charge records what its own call cost, so that is what is billed.

A charge no item carried waited for the end of the run, so `cost_limit`
saw no spend at the step that blew the budget and let every later step
run its side effects first. It is settled as each step finishes instead.

Settling per step means a copy of that item can surface a step later, so
the ledger now remembers what it has already accounted for: such a copy
either settles an open charge and is billed as its carrier, or stands
behind a call already billed and is not billed twice. Both answers need
the cost to be one the run recorded, so an invented item still cannot
hide spend.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): enforce the budget a failed step already spent

A step that pays a provider and then fails takes its items with it, so
`on_error: continue` advanced the run with the charge still open and the
limit unchecked. A budget already gone let the next step run its side
effects, which is exactly what `action: stop` exists to prevent.

The charge is settled and the limit checked on that path too, before the
run is allowed to move on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): match a copied item on the tokens that are billed

Comparing every number in a usage record asked a stricter question than
the accounting does. A stage that rebuilds an item can keep the token
counts and drop or recompute a `totalTokens` nobody is charged for, and
the copy then settled nothing: it was billed beside a charge that still
went on to be settled at the end, doubling one provider call.

`CostTracker` now exposes the counts it bills, and the ledger asks it
rather than keeping a second opinion, so the two cannot drift. The
provider spellings of those counts read as one charge for the same
reason: they already bill as one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): read a copied item by its cost, and open no empty charge

Two ways one provider call could still be billed twice.

A transform can emit `{ model, usage }` and drop the cache key with the
symbols. That copy took the untouched path, was billed, and left the
charge behind it open for the settlement to bill again. The key was only
ever the index; the cost is the evidence. A copy that still names a key
is read against that key alone, so one call's copy cannot settle
another's, and a copy that named none is read against every open charge.

An answer that reports no usage is billed nowhere however many times it
is replayed, but still opened a charge. The next caller with a real call
to account for was handed that empty charge, leaving its own open to be
billed on top of the item already recorded. No charge is opened for an
answer that costs nothing, and `claim` prefers one that can be billed so
a charge restored without a cost cannot stand in either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): settle a live call against the charge it opened

`claim()` took the first charge under the key, and a step retried after
an attempt that failed *after* paying has more than one. The retry's
item settled the failed attempt's charge and was billed at its own
price, leaving the retry's charge for the step settlement to bill at
that same price -- the retry counted twice and the attempt before it
not at all, at whatever the two really cost.

A caller that knows what its own call cost now says so, and the ledger
settles that call's charge. Replays still take the first billable
charge: a replay is not a witness of what it stands in for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(llm): let a keyless copy add a record of a call, never withhold one

Matching an unmarked item to a charge by cost alone was allowed to
answer "already billed", and that is not an answer cost alone can earn.
Two unrelated objects can carry the same model and the same token
counts, so an ordinary step printing `{ model, usage }` could drop
itself out of `_meta.cost` and `cost_limit` by resembling a call the run
had made earlier.

The two answers are not equally cheap now. Settling an open charge only
ever adds a record of a call, so a copy carrying the cost may do it.
Withholding one removes a record, so that needs the cache key as well: a
copy that names the call it came from *and* costs what that call cost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): read a copy only after the marked results have settled

A copy that lost its cache key is read by cost alone, and it was read
while every charge in the step was still open. Two live calls costing
the same under different keys, one branch handing on a filtered copy:
the copy settled the other branch's charge, that branch was billed
regardless, and the copy's own call was left for the step settlement.
Two provider calls, three records.

Copies now wait alongside replays and are read last, when the only
charges still open are the ones no marked result claimed. Replays go
first among the two: a replay carries the mark of the call it stands in
for, which is better evidence than a cost on its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(state): carry what a stored value stands in for back out of storage

`llm.invoke | state.set k | state.get k` rebuilds the item from its own
JSON, and the marks this process attached are deliberately not in that
JSON. A replay of a call an earlier run paid for came back looking like
an ordinary record of spend and was billed, so a run that called no
provider could still report cost and trip `cost_limit`.

`state.set` remembers what it wrote, and `state.get` carries the marks
back onto the value it rebuilds -- only when the file still matches
those bytes exactly, so a value this process was never handed cannot
pick up marks it was never given.

What is carried is always a replay, whatever the source was: a value
read back out of storage re-emits an answer that already exists, and
that is not the call happening again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(llm): open the charge for an attempt the validator sends back

The charge was opened at the two returns, and an attempt rejected by
`--output-schema` reaches neither: it goes round the loop and asks the
provider again. That call was as real as the one that eventually
satisfies the schema, and it was in no total and against no budget.

It is opened where the provider answers instead, which is also where it
was always meant to be -- ahead of the writes that store the answer, so
a retry replaying a stored copy still finds a charge to settle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): restore what a paused run billed, not only what it spent

A resumed run gets back the money the paused run spent but no record of
what it was for, so the two halves of its accounting disagree. The
completed steps arrive as JSON, which carries a cache key but none of the
marks this process attached, and a later step that re-emits a completed
`llm.invoke` output hands on a copy the fresh ledger has never seen. It is
billed on top of the restored total: one provider call, twice in
`_meta.cost` and twice against `cost_limit`.

The settled charges now travel in the same resume state as the open ones.
They are read only against the cache key they name, so a restored record
can excuse a copy of the call it names and nothing else; a live call still
settles out of the open charges, and the spend being reconstructed is
already in the `cost` the same state carries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* docs(readme): bound the replay exemption at the process edge

The exemption needs a mark this process attached, and a stage that hands
items to an external program and reads its stdout back gets whatever that
program printed. State the boundary rather than leave it to be discovered
by a `cost_limit` that trips on a run which called no provider.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

* fix(workflows): keep a discarded wait:any branch out of the run's total

`wait: "any"` returns the first branch to answer and abandons the rest,
but an abandoned branch is not a stopped one: it can still be waiting on
a provider and it pays when the answer arrives. That charge went straight
to the run ledger, so whether the workflow billed it turned on nothing
but how long the run happened to live afterwards — the same workflow over
the same three calls reported 2000 tokens or 3000, and `cost_limit`
inherited the same coin flip. `main` is deterministic here, so this was
the ledger's own regression rather than a boundary it merely exposed.

A losing branch now opens its charges in a buffer of its own and only the
winner releases them into the run: the output of a discarded branch is
thrown away and its accounting goes with it, which is what `main` already
did. Reads still pass through, because they settle charges opened before
the race and are made by the workflow after a branch returns, never from
inside one. `wait: "all"` keeps every branch, so it bills as before.

Whether losers should instead be drained before the step settles is a
separate question about wait-any latency, and is left open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AVQmKec1UHHXyUhx4BNvS

* fix(workflows): price a live call from the charge it opened

A step's item reaches cost accounting after its pipeline has run, and a
pipeline can drop the model on the way -- `| pick usage` hands on the marked
usage record without it -- or rewrite it. The step was then priced from what
the item said rather than what the provider was asked for: nothing at all in
the first case, another model's rate in the second, with `cost_limit` reading
the difference as room left.

The charge the call opened already carries the real model and usage, and the
deferred replay path is billed from it for the same reason. A charge restored
from resume state written before it carried a cost still falls back to the
item.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JYzZx7UUfvMLvdtkiMaMs1

* fix(llm): harden replay cost provenance

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
 2026-08-13 12:26:26 -07:00
Yiğit ERDOĞANandClaude Opus 5 e88e8d3970 fix: cached LLM answers are returned when temperature or max output tokens change (#130) 
* fix(llm): key the response cache on sampling parameters

--temperature and --max-output-tokens were sent to the adapter but left out
of the cache key, so changing either one replayed the previous answer
instead of calling the model. The same key drives --state-key reuse, so a
resumed workflow replayed the stale answer too.

Hash each parameter only when it is set, guarding on the same predicate
that decides whether it goes on the wire. stableStringify sorts keys, so an
invocation that sets neither serializes exactly as before and keeps its
current hash - the cache is content-addressed by filename and a blanket key
change would orphan every entry written by an earlier release.

metadata stays out: it is set unconditionally with --metadata-json and
carries correlation data rather than decode-time parameters, so hashing it
would invalidate existing caches and collapse the hit rate. --schema-version
already covers callers who want metadata to segment the cache.

* fix(llm): version the cache identity so old entries cannot replay sampling

Keying only the parameters that were supplied kept the omitted-parameter key
byte-identical to what earlier releases wrote. That preserved existing caches,
but it also left one collision: an entry those releases stored for a request
sampled at an explicit temperature sits under the key a request that omits
sampling computes, so after upgrading, an unsampled request could be served a
sampled answer.

Put both parameters in the payload unconditionally, with `null` as the identity
of an omitted one, and add a version field to the payload. Entries written under
the previous identity are unreachable rather than ambiguous: the cost is one
re-invocation per prompt after upgrade, never a wrong replay.

The key-stability test is replaced by its inverse — a cache entry written under
the pre-upgrade key is not returned to a request that omits sampling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6UVfFPP39RkoYx5jZ3KKM

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 12:23:12 -07:00
Yiğit ERDOĞANandClaude Opus 5 9769237d36 fix: Lobster cannot create its cache or state directories on Windows (#126) 
* fix(state): normalize extended-length mkdir paths

On Windows fs.mkdir(recursive) reports the first created directory as an
extended-length path (\?\C:\...) while the requested directory is a plain
drive path. path.resolve keeps the prefix, so the two never compare equal
and path.relative between them yields an absolute path. The chain walk then
takes "C:" as its next segment and syncs a directory that does not exist,
so ensureDirectory throws ENOENT every time it actually creates something.

That breaks LLM cache writes, state.set, diff snapshots, and approval index
publication on the first Windows run.

Strip the prefix before resolving so both ends of the chain share one root
form. On POSIX the prefixes never occur and the walk is unchanged.

* fix(state): keep device namespaces out of the path normalization

Stripping every \?\ prefix also rewrote namespaces that have no plain
equivalent, so an explicitly configured LOBSTER_STATE_DIR such as
\?\Volume{GUID}\lobster\state became relative and resolved against the
current drive. That regressed a form main handles today.

Map only the drive-letter and UNC namespaces, which do have a plain
equivalent, and return anything else untouched. Cover the mapping directly
so the device-namespace case is pinned.

* fix(state): recognise the UNC namespace whatever its case

Windows compares path namespace components case-insensitively, so
`\?\unc\server\share` names the same share as `\?\UNC\server\share`.
Matching only the uppercase form left the lowercase spelling extended
while the directory it walks toward is plain, so the sync walk could
never reach the requested final path on such a setup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD5sJTbh2jn1uvgNCmkLq

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 12:22:56 -07:00
Martin KesslerClaude Fable 5Peter Steinberger
 		f14e22d94a fix: workflow run crashes when a step exits before reading its stdin (#122) 
* fix: workflow run crashes when a step exits before reading its stdin

A step command that exits during its startup (for example a shell script
failing a preamble check under set -e) before draining a large piped stdin
leaves the engine's pending stdin write to fail with EPIPE. The stdin socket
had no 'error' listener, so Node raised an unhandled 'error' event and the
entire lobster process crashed -- losing the approval gate, the resume token,
and the step's real exit code and stderr.

Ignore stdin write errors in both shell-step and stdlib exec spawns: the
close handler already reports the true failure. Regression test drives a
300KB stdout through a fast-exiting step and asserts the run rejects with
'workflow command failed (1)' instead of crashing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* simplify: condense EPIPE inline comments to essential context

The 5-line comment blocks explaining EPIPE behavior were excessive for
1-line handlers. The test comment describing the 300KB mechanism was
similarly verbose. Trim each to the non-obvious why only.

* simplify: remove EPIPE inline comments

The child.stdin.on('error', () => {}) pattern is self-explanatory to
Node.js developers; the rationale is fully documented in the PR body.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
 2026-08-13 12:22:15 -07:00
Yiğit ERDOĞANandClaude Opus 5 f2438f52ad fix: schema validation failures cost one more model call than configured (#128) 
* fix(llm): stop spending an extra call on validation retries

The retry budget was compared against a 1-based attempt counter plus one,
so the initial request never counted against it. Every configured value
bought one more billed adapter call than requested, and
--max-validation-retries 0 still issued a second call after the first
response failed the output schema.

Compare the attempt counter against the retry count directly: attempt
counts calls, max-validation-retries counts retries, so calls = retries+1.
Only the failure bound moves; the success path is unchanged.

* docs(llm): say what --max-validation-retries counts

The flag's own description said only "retries when schema validation fails",
which is exactly the ambiguity this fix resolves: whether the budget includes
the first call. The compatibility question in front of a maintainer is easier to
answer when the option states that it allows N extra calls after the first, and
that its default is 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Bu6MvbjKGtfRiNesARMyZ

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 10:55:13 -07:00
xingzhouandPeter Steinberger c440ca57d1 fix(runtime): cancelled workflows no longer continue external commands (#119) 
* fix(runtime): stop process-backed work on cancellation

* fix(runtime): invalidate cancelled resume state

* Revert "fix(runtime): invalidate cancelled resume state"

This reverts commit 0f7d291f98.

* fix(runtime): narrow cancellation to safe child processes

* fix(runtime): stop after completed search cancellation

* fix(runtime): halt direct pipelines after cancellation

Preserve completed in-flight stage results while preventing later direct pipeline stages from starting after parent cancellation.

* fix(runtime): consume aborted approval resumes

* fix(runtime): cancelled workflows no longer continue external commands

* fix(workflow): propagate custom parent cancellation

* fix(runtime): preserve pre-aborted resume state

* fix(runtime): stop lazy handoff after cancellation

* fix(workflow): close remaining cancellation boundaries

* fix(runtime): preserve workflow resumes during setup cancellation

* fix(runtime): close final cancellation persistence gaps

* fix(runtime): stop lazy handoff after cancellation

* fix(runtime): interrupt blocked lazy handoff reads

* fix(runtime): terminate cancellation process trees

* fix(runtime): await process tree termination

* fix(runtime): terminate workflow process trees

* fix(runtime): bridge CLI cancellation

* fix(cli): preserve cancellation lifecycle

* fix(cli): abort stalled signal-aware commands

* fix(cli): release aborted interactive prompts

* fix(cli): preserve sequential prompt input

* fix(cli): preserve buffered prompt input

* fix(cli): handle prompt EOF after buffered input

* fix(runtime): preserve UTF-8 subprocess output

* fix(workflow): preserve retryable resume before execution

* fix(workflow): roll back cancelled resume replacement

* fix(state): roll back cancelled monitor snapshot

* fix(resume): preserve cancelled gate capabilities

* fix(resume): close cancellation rollback windows

* fix(resume): harden cancellation state cleanup

* fix(runtime): close resumed cancellation gaps

* fix(runtime): preserve cancellation cleanup

* fix(runtime): close cancellation lifecycle gaps

* fix(runtime): harden resumed cancellation boundaries

* fix(workflow): consume timed-out resume capabilities

* fix(workflow): preserve resume policy boundaries

* fix(runtime): preserve cancellation cleanup liveness

* fix(runtime): harden cancellation cleanup

* fix(runtime): stop lazy output after cancellation

* fix(runtime): settle cancellation cleanup

* fix(runtime): preserve safe input resumes

* fix(runtime): prevent consumed resume replays

* fix(runtime): serialize approval resume consumption

* fix(runtime): prevent concurrent safe gate forks

* fix(runtime): close cancellation review gaps

* fix(llm): restore cache after cancelled refresh

* fix(runtime): close remaining cancellation windows

* fix(runtime): preserve cancellation recovery invariants

* fix(runtime): prevent stale resume recovery

* fix(runtime): preserve resume claim recovery

* fix(runtime): retry pre-dispatch claims safely

* fix(state): synchronize rollback-safe reads

* fix(resume): discard cancelled pipeline successors

* fix(runtime): harden cancellation and state locking

* fix(runtime): recover durable cancellation failures

* fix(runtime): preserve legacy workflow cancellation

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
 2026-08-13 10:54:20 -07:00
Vincent Koc 0ac962e90b fix(deps): override vulnerable fast-uri (#123) 2026-07-28 15:13:11 +08:00
Peter Steinberger 386a201ced ci: add ClawSweeper dispatch workflow 2026-07-26 18:53:47 -07:00
Hannes Rudolph c2ff7d0ace chore: align pull request template (#120) 2026-07-13 12:42:54 -06:00
Vincent Koc affca75be8 chore(repo): enforce pnpm runtime policy 2026-07-11 13:15:14 +08:00
Peter Steinberger 42b09e2077 fix(ci): pin TypeScript age exemptions 2026-07-09 15:03:09 +01:00
Peter Steinberger 1dfedd5ae8 fix(ci): enable TypeScript age exemptions 2026-07-09 14:56:20 +01:00
Peter Steinberger e77e279a01 fix(ci): match TypeScript age exemptions 2026-07-09 14:51:20 +01:00
Peter Steinberger 378f30beba chore(deps): adopt TypeScript 7 and update dependencies 2026-07-09 14:25:36 +01:00
Vincent Koc d9a0fe7e02 style: format lobster sources 2026-06-22 14:26:35 +08:00
Vincent Koc 7a57f25720 fix: harden lobster release and invoke paths 2026-06-22 13:53:21 +08:00
Peter Steinberger d759d5c3ba feat: add OpenClaw agent invocation (#118) 2026-06-18 08:01:23 +02:00
Peter Steinberger 7d3b6a1512 chore: reopen changelog after release 2026-06-11 02:58:26 +01:00
129 changed files with 31243 additions and 16895 deletions
Expand all files Collapse all files
.github/pull_request_template.md
+65
View File
@@ -0,0 +1,65 @@
<!--
Optional linked context:
Add a visible `Closes #<issue-number>` or `Related: #<issue-number>` line
below this comment.
Required PR title:
type: user-facing description
Use a parenthesized scope only when it adds clarity:
fix(auth): login redirect loops when session cookie is expired
Types: feat, fix, improve, refactor, docs, chore.
For fixes, describe the user-visible symptom and trigger:
fix: task list fails to load when user has no environments
Avoid implementation details such as:
fix: add null check to task query
-->
<details>
<summary>Additional instructions</summary>
**MUST:** Keep **Allow edits from maintainers** enabled for this PR so maintainers
can help update the branch when needed.
</details>
## What Problem This Solves
<!--
Describe the concrete user, product, or operational problem.
For fixes, begin with:
"Fixes an issue where users <do X> would <experience Y> when <condition>."
or:
"Resolves a problem where..."
Name the affected UI surface or workflow. Do not describe the code-level cause here.
-->
## Why This Change Was Made
<!--
In one or two sentences, explain the complete shipped solution, key design
decisions, and relevant boundaries or non-goals. Include implementation detail
only when it helps reviewers understand user-visible behavior or risk.
Avoid file-by-file narration.
-->
## User Impact
<!--
State what users, operators, or developers can now do or expect. Lead with the
concrete benefit and use user-facing language. If there is no user-visible
impact, say so plainly.
-->
## Evidence
<!--
Show the most useful proof that this change works. Screenshots, screencasts,
terminal output, focused tests, CI results, live observations, redacted logs,
and artifact links are all useful. Include before/after evidence for visual
changes when it clarifies the result.
Reviewers will inspect the code, tests, and CI. Use this section to make the
validation easy to understand, not to restate the diff.
-->
.github/workflows/clawsweeper-dispatch.yml
+202
View File
@@ -0,0 +1,202 @@
name: ClawSweeper Dispatch
on:
issues:
types: [opened, reopened, edited, labeled, unlabeled]
issue_comment:
types: [created, edited]
pull_request_target: # zizmor: ignore[dangerous-triggers] maintainer-owned external dispatch; no checkout or untrusted PR code execution
types: [opened, reopened, synchronize, ready_for_review, edited, labeled, unlabeled]
permissions:
contents: read
concurrency:
group: clawsweeper-dispatch-${{ github.repository }}-${{ github.event.issue.number || github.event.pull_request.number || github.run_id }}
cancel-in-progress: ${{ github.event.action == 'edited' || github.event.action == 'synchronize' || github.event.action == 'ready_for_review' }}
jobs:
dispatch:
runs-on: ubuntu-latest
if: ${{ !(endsWith(github.actor, '[bot]') && (github.event.action == 'labeled' || github.event.action == 'unlabeled')) }}
env:
HAS_CLAWSWEEPER_APP_PRIVATE_KEY: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY != '' }}
CLAWSWEEPER_APP_CLIENT_ID: Iv23liOECG0slfuhz093
SUPERSEDES_IN_PROGRESS: ${{ (github.event.action == 'edited' || github.event.action == 'synchronize' || github.event.action == 'ready_for_review') && 'true' || 'false' }}
steps:
- name: Debounce bursty metadata events
if: ${{ github.event.action == 'labeled' || github.event.action == 'unlabeled' }}
run: sleep 20
- name: Create ClawSweeper dispatch token
id: token
if: ${{ env.HAS_CLAWSWEEPER_APP_PRIVATE_KEY == 'true' }}
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ env.CLAWSWEEPER_APP_CLIENT_ID }}
private-key: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY }}
owner: openclaw
repositories: clawsweeper
permission-contents: write
- name: Pre-filter ClawSweeper comment
id: comment_filter
if: ${{ github.event_name == 'issue_comment' }}
env:
COMMENT_BODY: ${{ github.event.comment.body }}
run: |
set -euo pipefail
if grep -Eiq '(^|[[:space:]])@(clawsweeper|openclaw-clawsweeper)\b(\[bot\])?|(^|[[:space:]])/(clawsweeper|review|autoclose|auto([[:space:]]+|-)?merge)\b' <<< "$COMMENT_BODY"; then
echo "is_command=true" >> "$GITHUB_OUTPUT"
else
echo "is_command=false" >> "$GITHUB_OUTPUT"
fi
- name: Create target comment token
id: target_token
if: >-
${{
github.event_name == 'issue_comment' &&
steps.comment_filter.outputs.is_command == 'true' &&
env.HAS_CLAWSWEEPER_APP_PRIVATE_KEY == 'true'
}}
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ env.CLAWSWEEPER_APP_CLIENT_ID }}
private-key: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: ${{ github.event.repository.name }}
permission-issues: write
permission-pull-requests: read
- name: Dispatch exact ClawSweeper review
if: ${{ github.event_name != 'issue_comment' }}
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
TARGET_REPO: ${{ github.repository }}
ITEM_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
ITEM_KIND: ${{ github.event_name == 'pull_request_target' && 'pull_request' || 'issue' }}
SOURCE_EVENT: ${{ github.event_name }}
SOURCE_ACTION: ${{ github.event.action }}
run: |
if [ -z "$GH_TOKEN" ]; then
echo "::notice::Skipping ClawSweeper dispatch because no dispatch credential is configured."
exit 0
fi
ingress_fingerprint="$(node <<'NODE'
const crypto = require("node:crypto");
const fs = require("node:fs");
const event = JSON.parse(fs.readFileSync(process.env.GITHUB_EVENT_PATH, "utf8"));
const pullRequest = event.pull_request && typeof event.pull_request === "object"
? event.pull_request
: {};
const headSha = String(pullRequest.head?.sha || "").trim().toLowerCase();
const updatedAt = String(pullRequest.updated_at || "").trim();
if (
process.env.ITEM_KIND !== "pull_request" ||
!/^[0-9a-f]{40}$/.test(headSha) ||
!updatedAt
) {
process.stdout.write("");
} else {
process.stdout.write(
crypto
.createHash("sha256")
.update(
JSON.stringify({
version: 1,
target_repo: String(process.env.TARGET_REPO || "").toLowerCase(),
item_number: Number(process.env.ITEM_NUMBER),
action: String(process.env.SOURCE_ACTION || ""),
head_sha: headSha,
updated_at: updatedAt,
body: typeof pullRequest.body === "string" ? pullRequest.body : "",
label: String(event.label?.name || ""),
}),
)
.digest("hex"),
);
}
NODE
)"
payload="$(jq -nc \
--arg target_repo "$TARGET_REPO" \
--argjson item_number "$ITEM_NUMBER" \
--arg item_kind "$ITEM_KIND" \
--arg source_event "$SOURCE_EVENT" \
--arg source_action "$SOURCE_ACTION" \
--arg ingress_fingerprint "$ingress_fingerprint" \
--argjson supersedes_in_progress "$SUPERSEDES_IN_PROGRESS" \
'{event_type:"clawsweeper_item",client_payload:({target_repo:$target_repo,item_number:$item_number,item_kind:$item_kind,source_event:$source_event,source_action:$source_action,supersedes_in_progress:$supersedes_in_progress} + (if $ingress_fingerprint != "" then {ingress_route:"target_dispatcher",ingress_fingerprint:$ingress_fingerprint} else {} end))}')"
gh api repos/openclaw/clawsweeper/dispatches \
--method POST \
--input - <<< "$payload"
- name: Acknowledge and dispatch ClawSweeper comment
if: >-
${{
github.event_name == 'issue_comment' &&
steps.comment_filter.outputs.is_command == 'true'
}}
env:
DISPATCH_TOKEN: ${{ steps.token.outputs.token }}
TARGET_TOKEN: ${{ steps.target_token.outputs.token }}
TARGET_REPO: ${{ github.repository }}
ITEM_NUMBER: ${{ github.event.issue.number }}
COMMENT_ID: ${{ github.event.comment.id }}
COMMENT_BODY: ${{ github.event.comment.body }}
AUTHOR_ASSOCIATION: ${{ github.event.comment.author_association }}
SOURCE_ACTION: ${{ github.event.action }}
run: |
if [ -z "$DISPATCH_TOKEN" ]; then
echo "::notice::Skipping ClawSweeper dispatch because no dispatch credential is configured."
exit 0
fi
body_file="$RUNNER_TEMP/clawsweeper-comment-body.txt"
printf '%s\n' "$COMMENT_BODY" > "$body_file"
if grep -Eiq '<!--[[:space:]]*clawsweeper-proof-nudge([[:space:]]|-->)' "$body_file"; then
echo "Ignoring ClawSweeper proof-nudge comment."
exit 0
fi
if [ -n "$TARGET_TOKEN" ]; then
GH_TOKEN="$TARGET_TOKEN" gh api -X POST \
-H "Accept: application/vnd.github+json" \
"repos/$TARGET_REPO/issues/comments/$COMMENT_ID/reactions" \
-f content="eyes" >/dev/null || true
fi
status_comment_id=""
if [ -n "$TARGET_TOKEN" ]; then
case "$AUTHOR_ASSOCIATION" in
OWNER|MEMBER|COLLABORATOR)
status_body="$(printf '%s\n' \
"<!-- clawsweeper-command-ack:$COMMENT_ID -->" \
"🦞👀" \
"ClawSweeper picked this up." \
"" \
"Command router queued. I will update this comment with the next step.")"
status_payload="$(jq -nc --arg body "$status_body" '{body:$body}')"
status_err="$(mktemp)"
if status_response="$(GH_TOKEN="$TARGET_TOKEN" gh api \
"repos/$TARGET_REPO/issues/$ITEM_NUMBER/comments" \
--method POST \
--input - <<< "$status_payload" 2>"$status_err")"; then
status_comment_id="$(jq -r '.id // empty' <<< "$status_response")"
else
cat "$status_err" >&2
echo "::warning::Could not create ClawSweeper queued status comment; dispatching command router without one."
fi
rm -f "$status_err"
;;
esac
fi
payload="$(jq -nc \
--arg target_repo "$TARGET_REPO" \
--argjson item_number "$ITEM_NUMBER" \
--argjson comment_id "$COMMENT_ID" \
--arg status_comment_id "$status_comment_id" \
--arg source_event "issue_comment" \
--arg source_action "$SOURCE_ACTION" \
'{event_type:"clawsweeper_comment",client_payload:({target_repo:$target_repo,item_number:$item_number,comment_id:$comment_id,source_event:$source_event,source_action:$source_action,max_comments:"1"} + (if $status_comment_id != "" then {status_comment_id:($status_comment_id|tonumber)} else {} end))}')"
GH_TOKEN="$DISPATCH_TOKEN" gh api repos/openclaw/clawsweeper/dispatches \
--method POST \
--input - <<< "$payload"
.github/workflows/crabbox-hydrate.yml
+6
View File
@@ -77,6 +77,12 @@ jobs:
exit 2
;;
esac
case "$job" in
''|*[!A-Za-z0-9._-]*)
echo "Invalid crabbox_job" >&2
exit 2
;;
esac
mkdir -p "$HOME/.crabbox/actions"
state="$HOME/.crabbox/actions/${CRABBOX_ID}.env"
env_file="$HOME/.crabbox/actions/${CRABBOX_ID}.env.sh"
.github/workflows/lobster-npm-release.yml
+27
View File
@@ -517,6 +517,33 @@ jobs:
exit 1
fi
if ! VERSION_A="${latest_version}" VERSION_B="${RELEASE_VERSION}" node <<'NODE'
function parseVersion(value) {
const match = /^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9]+))?$/.exec(value);
if (!match) throw new Error(`Invalid version: ${value}`);
return {
major: Number(match[1]),
minor: Number(match[2]),
patch: Number(match[3]),
prerelease: match[4] === undefined ? null : Number(match[4]),
};
}
const latest = parseVersion(process.env.VERSION_A);
const release = parseVersion(process.env.VERSION_B);
const fields = ["major", "minor", "patch"];
for (const field of fields) {
if (latest[field] > release[field]) process.exit(1);
if (latest[field] < release[field]) process.exit(0);
}
if (latest.prerelease === null && release.prerelease !== null) process.exit(1);
if (latest.prerelease !== null && release.prerelease === null) process.exit(0);
if ((latest.prerelease ?? 0) > (release.prerelease ?? 0)) process.exit(1);
NODE
then
echo "npm latest ${latest_version} is newer than release ${RELEASE_VERSION}; refusing downgrade promotion." >&2
exit 1
fi
if ! npm view "${PACKAGE_NAME}@${RELEASE_VERSION}" version >/dev/null 2>&1; then
echo "${PACKAGE_NAME}@${RELEASE_VERSION} is not published on npm." >&2
exit 1
.oxfmtrc.json
+3
View File
@@ -0,0 +1,3 @@
{
"useTabs": true
}
CHANGELOG.md
+4
View File
@@ -2,6 +2,10 @@
All notable changes to Lobster will be documented in this file.
## Unreleased
- Add first-class `openclaw.agent` workflow turns with configured agent, session, model, thinking, and timeout selection delegated to OpenClaw. Thanks to [@Stoff81](https://github.com/Stoff81) (Issue [#117](https://github.com/openclaw/lobster/issues/117)).
## 2026.6.11
- Add command-level `ctx.requestInput(...)` for CLI/tool/SDK pipeline commands, with state-backed same-command resume, bounded command-input replay, and workflow `pipeline:` propagation (Issue [#101](https://github.com/openclaw/lobster/issues/101)).
README.md
+16 -1
View File
@@ -271,13 +271,28 @@ Built-in providers today:
- `pi` via `LOBSTER_PI_LLM_ADAPTER_URL` (typically supplied by the Pi extension)
- `http` via `LOBSTER_LLM_ADAPTER_URL`
A host embedding Lobster can supply its own adapters through `ctx.llmAdapters`. Step `timeout_ms` and workflow cancellation reach an adapter as `ctx.signal`: Lobster stops waiting as soon as that signal aborts, so the step fails or retries on time either way, but it cannot cancel work an adapter has already started. An injected adapter should observe `ctx.signal` and abort its own request — otherwise a timed-out step can leave a model call running, and billed, in the background.
Workflow `_meta.cost` and `cost_limit` use a static pricing table plus optional overrides from `LOBSTER_LLM_PRICING_JSON`, for example `{"my-model":{"input":1.0,"output":2.0}}` in USD per million tokens. Unknown or missing model IDs still record token counts with zero estimated cost, but Lobster warns on stderr so stale or missing pricing does not fail silently.
A cached or replayed answer is not billed again: a model call is counted once, in the run that made it, however many later steps re-emit its answer. This holds while the answer stays inside Lobster: through pipelines, renderers, projections, run state, `workflow:` steps and a resume. It does not survive a stage that hands the items to an external process and reads them back — `exec --stdin json --json ...` — because what comes back is whatever that process printed, and Lobster cannot tell a faithful copy of a replay from a fresh claim to have made the call. Such a step is billed as a call, which is what earlier versions did everywhere. A `workflow:` step counts what its sub-workflow spent, and a replay the sub-workflow returned is not billed a second time by the run that composed it. Spend also survives a pause — a workflow that stops at an approval or `input` gate keeps what it has recorded, so `_meta.cost` covers the whole run after a resume and `cost_limit` applies to the whole run rather than to the steps after the last gate.
`llm_task.invoke` remains available as a backward-compatible alias for the OpenClaw provider.
### Calling configured OpenClaw agents
Use `openclaw.agent` when a workflow needs a configured OpenClaw agent rather than a direct model call:
```bash
openclaw.agent --agent ops --prompt 'Summarize these logs'
openclaw.agent --agent ops --session-key incident-42 --model openai/gpt-5.4 --prompt 'Continue the investigation'
```
The command delegates agent identity, model defaults and overrides, sessions, authentication, and execution to the installed `openclaw agent` CLI. It accepts `--agent`, `--session-key`, `--session-id`, `--model`, `--thinking`, `--timeout`, and `--local`, and returns OpenClaw's structured `--json` response. Pipeline input is appended to the prompt as labeled JSONL.
### `pipeline:` vs `run:` for LLM calls
- Use `pipeline:` for `llm.invoke` and `llm_task.invoke` (they are Lobster pipeline stages, not shell executables).
- Use `pipeline:` for `openclaw.agent`, `llm.invoke`, and `llm_task.invoke` (they are Lobster pipeline stages, not shell executables).
- Use `run:` only for real binaries in your shell (for example `openclaw.invoke`).
Example (`stdin` from a prior step is passed to the LLM as artifacts):
bin/clawd.invoke.js
+13 -7
View File
@@ -1,18 +1,24 @@
#!/usr/bin/env node
import { spawnSync } from 'node:child_process';
import { spawnSync } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
function shellQuote(arg) {
if (/^[A-Za-z0-9_\-./:=@]+$/.test(arg)) return arg;
return `'${String(arg).replace(/'/g, `'\\''`)}'`;
if (/^[A-Za-z0-9_\-./:=@]+$/.test(arg)) return arg;
return `'${String(arg).replace(/'/g, `'\\''`)}'`;
}
const argv = process.argv.slice(2);
const pipeline = ['clawd.invoke', ...argv.map(shellQuote)].join(' ');
const pipeline = ["clawd.invoke", ...argv.map(shellQuote)].join(" ");
const lobsterBin = join(dirname(fileURLToPath(import.meta.url)), "lobster.js");
const res = spawnSync('lobster', [pipeline], {
stdio: 'inherit',
env: process.env,
const res = spawnSync(process.execPath, [lobsterBin, pipeline], {
stdio: "inherit",
env: process.env,
});
if (res.error) {
console.error(`clawd.invoke failed to spawn lobster: ${res.error.message}`);
}
process.exit(res.status ?? 1);
bin/lobster.js
+7 -7
View File
@@ -8,17 +8,17 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
async function load() {
const distEntry = join(__dirname, "../dist/src/cli.js");
if (existsSync(distEntry)) {
return import(pathToFileURL(distEntry).href);
}
const srcEntry = join(__dirname, "../src/cli.js");
return import(pathToFileURL(srcEntry).href);
const distEntry = join(__dirname, "../dist/src/cli.js");
if (existsSync(distEntry)) {
return import(pathToFileURL(distEntry).href);
}
const srcEntry = join(__dirname, "../src/cli.js");
return import(pathToFileURL(srcEntry).href);
}
const mod = await load();
if (typeof mod.runCli !== "function") {
throw new Error("lobster CLI entrypoint missing runCli()");
throw new Error("lobster CLI entrypoint missing runCli()");
}
await mod.runCli(process.argv.slice(2));
bin/openclaw.invoke.js
+16 -10
View File
@@ -1,21 +1,27 @@
#!/usr/bin/env node
import { spawnSync } from 'node:child_process';
import { spawnSync } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
function shellQuote(arg) {
// Conservative POSIX-ish quoting for embedding argv into a single pipeline string.
// Lobster's pipeline parser preserves quoted substrings.
if (/^[A-Za-z0-9_\-./:=@]+$/.test(arg)) return arg;
// single-quote, escaping embedded single quotes: ' -> '\''
return `'${String(arg).replace(/'/g, `'\\''`)}'`;
// Conservative POSIX-ish quoting for embedding argv into a single pipeline string.
// Lobster's pipeline parser preserves quoted substrings.
if (/^[A-Za-z0-9_\-./:=@]+$/.test(arg)) return arg;
// single-quote, escaping embedded single quotes: ' -> '\''
return `'${String(arg).replace(/'/g, `'\\''`)}'`;
}
const argv = process.argv.slice(2);
const pipeline = ['openclaw.invoke', ...argv.map(shellQuote)].join(' ');
const pipeline = ["openclaw.invoke", ...argv.map(shellQuote)].join(" ");
const lobsterBin = join(dirname(fileURLToPath(import.meta.url)), "lobster.js");
const res = spawnSync('lobster', [pipeline], {
stdio: 'inherit',
env: process.env,
const res = spawnSync(process.execPath, [lobsterBin, pipeline], {
stdio: "inherit",
env: process.env,
});
if (res.error) {
console.error(`openclaw.invoke failed to spawn lobster: ${res.error.message}`);
}
process.exit(res.status ?? 1);
package.json
+74 -73
View File
@@ -1,75 +1,76 @@
{
"name": "@clawdbot/lobster",
"version": "2026.6.11",
"description": "Workflow runtime for AI agents - deterministic pipelines with approval gates",
"keywords": [
"ai-agent",
"approval",
"automation",
"lobster",
"openclaw",
"pipeline",
"workflow"
],
"homepage": "https://github.com/openclaw/lobster#readme",
"bugs": {
"url": "https://github.com/openclaw/lobster/issues"
},
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/openclaw/lobster.git"
},
"bin": {
"clawd.invoke": "bin/clawd.invoke.js",
"lobster": "bin/lobster.js",
"openclaw.invoke": "bin/openclaw.invoke.js"
},
"files": [
"bin",
"dist",
"README.md",
"LICENSE",
"VISION.md"
],
"type": "module",
"main": "./dist/src/sdk/index.js",
"exports": {
".": "./dist/src/sdk/index.js",
"./sdk": "./dist/src/sdk/index.js",
"./core": "./dist/src/core/index.js",
"./recipes/github": "./dist/src/recipes/github/index.js"
},
"scripts": {
"clean": "node -e \"require('fs').rmSync('dist', { recursive: true, force: true })\"",
"build": "pnpm clean && tsgo -p tsconfig.json",
"prepack": "pnpm build",
"typecheck": "tsgo -p tsconfig.json --noEmit",
"format": "oxfmt --write package.json tsconfig.json src test",
"format:check": "oxfmt --check package.json tsconfig.json src test",
"lint": "pnpm format:check && oxlint --tsconfig tsconfig.json src test",
"fmt": "pnpm format",
"test": "pnpm build && node --test dist/test/*.test.js",
"check:changed": "pnpm run test",
"test:changed": "pnpm run test",
"crabbox:hydrate": "crabbox actions hydrate",
"crabbox:run": "crabbox run",
"crabbox:stop": "crabbox stop",
"crabbox:warmup": "crabbox warmup"
},
"dependencies": {
"ajv": "^8.20.0",
"yaml": "^2.9.0"
},
"devDependencies": {
"@types/node": "^25.9.2",
"@typescript/native-preview": "7.0.0-dev.20260609.1",
"oxfmt": "^0.54.0",
"oxlint": "^1.69.0",
"oxlint-tsgolint": "^0.23.0",
"typescript": "^6.0.3"
},
"engines": {
"node": ">=22"
}
"name": "@clawdbot/lobster",
"version": "2026.6.11",
"description": "Workflow runtime for AI agents - deterministic pipelines with approval gates",
"keywords": [
"ai-agent",
"approval",
"automation",
"lobster",
"openclaw",
"pipeline",
"workflow"
],
"homepage": "https://github.com/openclaw/lobster#readme",
"bugs": {
"url": "https://github.com/openclaw/lobster/issues"
},
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/openclaw/lobster.git"
},
"bin": {
"clawd.invoke": "bin/clawd.invoke.js",
"lobster": "bin/lobster.js",
"openclaw.invoke": "bin/openclaw.invoke.js"
},
"files": [
"bin",
"dist",
"README.md",
"LICENSE",
"VISION.md"
],
"type": "module",
"main": "./dist/src/sdk/index.js",
"exports": {
".": "./dist/src/sdk/index.js",
"./sdk": "./dist/src/sdk/index.js",
"./core": "./dist/src/core/index.js",
"./recipes/github": "./dist/src/recipes/github/index.js"
},
"scripts": {
"clean": "node -e \"require('fs').rmSync('dist', { recursive: true, force: true })\"",
"build": "pnpm clean && tsc -p tsconfig.build.json",
"prepack": "pnpm build",
"typecheck": "tsc -p tsconfig.json --noEmit",
"format": "oxfmt --write package.json tsconfig.json tsconfig.build.json bin src test",
"format:check": "oxfmt --check package.json tsconfig.json tsconfig.build.json bin src test",
"lint": "pnpm format:check && oxlint --tsconfig tsconfig.json bin src test",
"fmt": "pnpm format",
"test": "pnpm clean && tsc -p tsconfig.json && node -e \"if (!require('fs').existsSync('dist/test')) process.exit(1)\" && node --test dist/test/*.test.js",
"check:changed": "pnpm run test",
"test:changed": "pnpm run test",
"crabbox:hydrate": "crabbox actions hydrate",
"crabbox:run": "crabbox run",
"crabbox:stop": "crabbox stop",
"crabbox:warmup": "crabbox warmup"
},
"dependencies": {
"ajv": "^8.20.0",
"yaml": "^2.9.0"
},
"devDependencies": {
"@types/node": "^26.1.1",
"@typescript/native": "npm:typescript@^7.0.2",
"oxfmt": "^0.54.0",
"oxlint": "^1.73.0",
"oxlint-tsgolint": "^0.24.0",
"typescript": "npm:@typescript/typescript6@^6.0.2"
},
"engines": {
"node": ">=22"
},
"packageManager": "pnpm@10.34.4"
}
pnpm-lock.yaml
Generated
+304 -163
View File
@@ -4,6 +4,9 @@ settings:
autoInstallPeers: true
excludeLinksFromLockfile: false
overrides:
fast-uri: 3.1.4
importers:
.:
@@ -16,23 +19,23 @@ importers:
version: 2.9.0
devDependencies:
'@types/node':
specifier: ^25.9.2
version: 25.9.2
'@typescript/native-preview':
specifier: 7.0.0-dev.20260609.1
version: 7.0.0-dev.20260609.1
specifier: ^26.1.1
version: 26.1.1
'@typescript/native':
specifier: npm:typescript@^7.0.2
version: typescript@7.0.2
oxfmt:
specifier: ^0.54.0
version: 0.54.0
oxlint:
specifier: ^1.69.0
version: 1.69.0(oxlint-tsgolint@0.23.0)
specifier: ^1.73.0
version: 1.73.0(oxlint-tsgolint@0.24.0)
oxlint-tsgolint:
specifier: ^0.23.0
version: 0.23.0
specifier: ^0.24.0
version: 0.24.0
typescript:
specifier: ^6.0.3
version: 6.0.3
specifier: npm:@typescript/typescript6@^6.0.2
version: '@typescript/typescript6@6.0.2'
packages:
@@ -158,206 +161,283 @@ packages:
cpu: [x64]
os: [win32]
'@oxlint-tsgolint/darwin-arm64@0.23.0':
resolution: {integrity: sha512-gOs9PVr2wEg4ox9z0aJo+RKhhImW86YL5N6yav8BK/rgPsIrwN/igSZ+pbRr723NFvUNKde9fgMhRA6JrXAOZw==}
'@oxlint-tsgolint/darwin-arm64@0.24.0':
resolution: {integrity: sha512-C2uMmwK5Bc4ri4ysZ6sA8Rcu+A5zBQTp6ml2u0CLLbRZp4kMFPV3yWk8B5DK9Aw7y9bbjogIm75tUwGLFzlsYQ==}
cpu: [arm64]
os: [darwin]
'@oxlint-tsgolint/darwin-x64@0.23.0':
resolution: {integrity: sha512-kjJ8B+7n4tB9VJdxS5A9GdJt6/bYpzbu4lXp2uO1S3sRmCB5gDEABlGoiePNApRWaW+xqL4b4xgiE727jSLhuA==}
'@oxlint-tsgolint/darwin-x64@0.24.0':
resolution: {integrity: sha512-Wgvt/1lRbDxmoNqWQKKcL+UIiqLmdJ+EWLpQa1qzoNVAfNB0PJpa82/8dH1twT/3rSs4zrP5TXPWl4juB71WuQ==}
cpu: [x64]
os: [darwin]
'@oxlint-tsgolint/linux-arm64@0.23.0':
resolution: {integrity: sha512-6dCZuKNu135seMXilkRk9SpCx6i1XgmiipYGalLij5WVRX6ZYS8c4xI7preN/zv9fCXhsQclTIMDu2Y/cytTjw==}
'@oxlint-tsgolint/linux-arm64@0.24.0':
resolution: {integrity: sha512-PB1rxII7KV83+ASY4sSkXtqvpij6ME66+QCRL49uksi/ofs2Rf/UVboYr095n0Rkbl2wgvlsHGl6DHC361jQUQ==}
cpu: [arm64]
os: [linux]
'@oxlint-tsgolint/linux-x64@0.23.0':
resolution: {integrity: sha512-3bdilnyA7kmSTjK27rvjIjSxL5SIg3wt7vwNiRkouWB83ytssyKnuGvxSYJxgMEmFpSutzaBzcCUM2jDtPGcgA==}
'@oxlint-tsgolint/linux-x64@0.24.0':
resolution: {integrity: sha512-xcz3CxKmjTQLREtE/UShh+ruWmm9nAb7UM9zKcD65BStiuYgOakAKkPHl4YS5DztpVcDrE0+HqbOolTlRKYWmw==}
cpu: [x64]
os: [linux]
'@oxlint-tsgolint/win32-arm64@0.23.0':
resolution: {integrity: sha512-j+OEp44SVYiQ+ZD+uttsX7u6L9SvmbbQ77SO1pSFCcJlsVMeCk8qZsjhKfGKuT/jIA+ipOJMVs/+pqUfObBWNw==}
'@oxlint-tsgolint/win32-arm64@0.24.0':
resolution: {integrity: sha512-A2i6ZGBec3i20S7RaxkgHc6r3HYtD5Mn7j/mb22NkTz14u0JuudvTu6JggAnbGMcv8+dBKQI//EasxSPJLD8pw==}
cpu: [arm64]
os: [win32]
'@oxlint-tsgolint/win32-x64@0.23.0':
resolution: {integrity: sha512-5MyjFuqf+g8OUPJBSGWHJtmoWnzFJYyOg4To9WMQshZYEWig/vtu7JtJ03VWnzHv9LJkAUeApY0gVCOywFR/iQ==}
'@oxlint-tsgolint/win32-x64@0.24.0':
resolution: {integrity: sha512-0ZbGd9qRB6zs82moekaKdEvncRANq49EAwfNX62JpTS46feXUhKAuoyVDvZMj6Rywejylrmmu79Wo6faYCo4Ew==}
cpu: [x64]
os: [win32]
'@oxlint/binding-android-arm-eabi@1.69.0':
resolution: {integrity: sha512-DKQQbD5cZ/MYfDgDI7YGyGD9FSxABlsBsYFo5p26lloob543tP9+4N3guwdXIYJN+7HSZxLe8YJuwcOWw5qnHg==}
'@oxlint/binding-android-arm-eabi@1.73.0':
resolution: {integrity: sha512-HZQRN/UMBu+Ut+/9MiAChkbP4qZqrNOWBcNI45vOT40GVhbGR0JgHB87L48D4iAqFQIdVmeQYtV9RF89AjTKkg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm]
os: [android]
'@oxlint/binding-android-arm64@1.69.0':
resolution: {integrity: sha512-lEhb+I5pr4inux+JFwfCa1HRq3Os7NirEFQ0H1I35SVEHPm6byX0Ah47xmRha3qi6LAkxUcxViL8o/9PivjzBg==}
'@oxlint/binding-android-arm64@1.73.0':
resolution: {integrity: sha512-Gp+KJRylv2aW7thRpG5p1KTxZq4ZJFbWowrKzufNq9d3ssl3r3JviYV45/+p+7CN1Nv0zDd1e8Ex0b/HUDq4TQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [android]
'@oxlint/binding-darwin-arm64@1.69.0':
resolution: {integrity: sha512-GY2YE8lOZW59BW1Ia1y+1gR0XyjrZRvVWHAr8LGeGhYHE0OQJ/7cRKXTkx1P+E9/6awEc3SX8a68SFTjh/E//A==}
'@oxlint/binding-darwin-arm64@1.73.0':
resolution: {integrity: sha512-3de96NdtXhxERMjIz7wsp2HYMY6pMQycGxFWac2mFecAx6VeARF/IqFb1QIaqiCRIdfzBwzTed+pCTCoiS+CYA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [darwin]
'@oxlint/binding-darwin-x64@1.69.0':
resolution: {integrity: sha512-ax1oZnOjHX3LB7myQyHEaQkDwfLb6str3/nSP6O7EVUviQGNkEGzGV0EqcBJWK+Ufwx0l4xPgyYayurvhAdl2Q==}
'@oxlint/binding-darwin-x64@1.73.0':
resolution: {integrity: sha512-5zx/uPW32TiaOeVY1dQ/H5iOf0K1HOdFKOJhLqGl4o63+i1fpzoqqu/mKtd7OFgFjNCdhlyTGgjVkQTZm1ELcg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [darwin]
'@oxlint/binding-freebsd-x64@1.69.0':
resolution: {integrity: sha512-kHWeHv4g2h8NY+mpCxzCtY4uerMJWTN/TSnNj1CPbakFpHEJ6cTya2wWV0pDSYWOJ2+0UiEbhn3AtXxHtsnKjg==}
'@oxlint/binding-freebsd-x64@1.73.0':
resolution: {integrity: sha512-qNe4gKHaGnLuZJ8toUg90JAa0S2vTVvDw+0bRi3q1avXZXDT4u5mMeECf3nD4HYrbdn1O7dXqWut4onY/yx/Xg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [freebsd]
'@oxlint/binding-linux-arm-gnueabihf@1.69.0':
resolution: {integrity: sha512-gq84vM1a1oEehXo27YCDzGVcxPsZDI1yswZwz2Da1/cbnWtrL16XZZnz0G/+gIU8edtHpfjxq5c+vWEHqJfWoQ==}
'@oxlint/binding-linux-arm-gnueabihf@1.73.0':
resolution: {integrity: sha512-cCehYh5hTbfShm/fxTD6wwrGUWIpvX+N5OxmAMhFhDeTGXvw+BeNj889tpxsFQ9ZLatQ6wImuY8tsKLZ+FMz7w==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm]
os: [linux]
'@oxlint/binding-linux-arm-musleabihf@1.69.0':
resolution: {integrity: sha512-kIqEa98JQ0VRyrcncxA417m2AzasqTlD+FyVT1AksjvjkqQcvm7pBWYvoW3/mpyOP2XYvi5nSCCTIe6De1yu5g==}
'@oxlint/binding-linux-arm-musleabihf@1.73.0':
resolution: {integrity: sha512-d5j5GDU/2dMgjVhw7TQT9ITrsIr1Y02KEXKyVGIXUkD+KiaxE9TP65FS2ZdgTBemQvoRL+gSBdbrIm3cQIeacg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm]
os: [linux]
'@oxlint/binding-linux-arm64-gnu@1.69.0':
resolution: {integrity: sha512-j+xYiXozxGWx2cpjCrwwGR4awTxPFsRv3JZrv23RCogEPMc4R7UqjHW47p/RG0aRlbWiROCJ8coUfCwy0dvzHA==}
'@oxlint/binding-linux-arm64-gnu@1.73.0':
resolution: {integrity: sha512-Eyf1SrP3+yR1DI3OJgOY2Pvrr9dWP9TK37xPaDYycwTtlGlI45erJAVIfH5/m/xosDt6BupJYEFi47bvbTuuyw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-arm64-musl@1.69.0':
resolution: {integrity: sha512-xEPpNppTfN1l/nM7gYSf9iocscu/as+p/7vxkLeLEKnYU+09Dm+5V6IhDYDh+Uz6FajEupWwCLt5SOG0y1PCKg==}
'@oxlint/binding-linux-arm64-musl@1.73.0':
resolution: {integrity: sha512-IlT/OJApEDKaMmCooHuncgJZbbCe7T5QIWmTZBEtYscWvzPQuuEinVcid6kwQRVQOUdb7PUCz4jQHnaYXdfJXw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [musl]
'@oxlint/binding-linux-ppc64-gnu@1.69.0':
resolution: {integrity: sha512-Ug0+eU7HJBlek+SjklYH62IlOMirEJsdxpihH0kSqX0XdrDD4NdHpQc10fK1JC35yn6KrrcN+uYzlHD38XAf8Q==}
'@oxlint/binding-linux-ppc64-gnu@1.73.0':
resolution: {integrity: sha512-L+JYcb/vdg5fmcH08V6o0YYLU28cTH1SPNulwJdvK9NK49aXSkYy6oNpKBmddArVOXYqNepriDGiZ04G54kh1Q==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-riscv64-gnu@1.69.0':
resolution: {integrity: sha512-iEyI3GIg0l/s3G4qy2TlaaWKdzj4PJJStwtlocpDTC00PY9hZueotf6OKUj9+yfQh0lrpBW/pLMgTztbAHKJEg==}
'@oxlint/binding-linux-riscv64-gnu@1.73.0':
resolution: {integrity: sha512-Qtk0g3bKV6OwWjIm7R8kQN1uOZRKQt/MODK2a8QfkwhTpXBD53ozx5XLVWLGDQAVyp2otLW4D2wB98XfAfMPGA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-riscv64-musl@1.69.0':
resolution: {integrity: sha512-NjHjpiI4WIKSMwuoJSZi5VToPeoYOS1FR52HLIDG6lidMdqquusgtODb4iLk0+lb1q3Z0nv2/aPRcC/olmpQGg==}
'@oxlint/binding-linux-riscv64-musl@1.73.0':
resolution: {integrity: sha512-wX0NQKZVxltkAOVmzFcpOaMpdaUvsq1Eqpx9tkAfl71UdkTlSo1R4AdAnGccR1Fm2+TzFgZ22CyyGuZ41RDr/A==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@oxlint/binding-linux-s390x-gnu@1.69.0':
resolution: {integrity: sha512-Ai/prDewoItkDXbp38gwGZi41DycZbUTZJ3UidwoHgQC0/DaqC2TGdtBTQLJ6hSD+SAxASzh8+/eSBPmxfOacA==}
'@oxlint/binding-linux-s390x-gnu@1.73.0':
resolution: {integrity: sha512-vPe7UGBMWyiLTtnqS4xxgMQFSFGmtQwhwCxuiw6lXygaO6bVt0D8dFVg8Xv05eaiN3ybC0HXXHUAohFMFvqoCQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-x64-gnu@1.69.0':
resolution: {integrity: sha512-Gt3KHgp46mRKz4sJeaASmKvD8ayXookRw07RMf+NowhEztGGDZ7VrXpoW96XuKJLjFukWizOFVNjmYb/u7caNQ==}
'@oxlint/binding-linux-x64-gnu@1.73.0':
resolution: {integrity: sha512-2CwIWr9cemFC/CbRBWZvuk5mffz6ObmfFkfcC/9rTQ7f+icNhYr2kOjf9Rt8lLvugvkdGDOmkoVoFFHh6ClCTw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-x64-musl@1.69.0':
resolution: {integrity: sha512-7tQhJ2+p/oHv1zcfnjYI7YVzC/7iBaVOfIvFYtxdJ5F45mWgEdrCyXZXZGfiLey5t/5JhOhsaMnnv1kAzckd7g==}
'@oxlint/binding-linux-x64-musl@1.73.0':
resolution: {integrity: sha512-nDadfJgg7NBBxG0N560wOe7LLX5QiYp6qBaI7viuk5EUORFBktU/NfV0MbTqU3gTqQDCh4VyxKdo5VADxk9w8Q==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [musl]
'@oxlint/binding-openharmony-arm64@1.69.0':
resolution: {integrity: sha512-vmWz6TKp/3hfA4lksR0zHBv/6xuX1jhym6eqOjdH2DXsDDHZWcp2f0KG0VCAnlVbIrjk29G4wAWMXb/Hn1YobA==}
'@oxlint/binding-openharmony-arm64@1.73.0':
resolution: {integrity: sha512-wGjJC+NLH9xP+IKGn9RDW94ojJR/wPbg5WCnQjj/oReaOtCQthr8ws1zICe77JFmo4ouUdeTHHZL/ESGiF6Pmw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [openharmony]
'@oxlint/binding-win32-arm64-msvc@1.69.0':
resolution: {integrity: sha512-9RExaLgmaw6IoIkU9cTpT71mLfI0xZ86iZH8x518LVsOkjquJMYqb9P7KpC8lgd1t0Dxs41p2pxynq4XR3Ttzw==}
'@oxlint/binding-win32-arm64-msvc@1.73.0':
resolution: {integrity: sha512-I7X47GPGljw225YUQ5SbC/rb1Kkdrd0yQf0x+hYxeKS6DpfjMbo9ccQPQ6LNY6BoJQ1sHhgDUGuMn5Vg5gHT6w==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [win32]
'@oxlint/binding-win32-ia32-msvc@1.69.0':
resolution: {integrity: sha512-1907kRPF8/PrcIw1E7LMs9JbVrpgnt/MvFdss3an8oDkYNAACXzTntV3t3869ZZhMZxb2AzRGbz1pA/jdFatXA==}
'@oxlint/binding-win32-ia32-msvc@1.73.0':
resolution: {integrity: sha512-5lWj+3h+74Fm1jYOO9qkJA4xkAlZA099DkXppuXsk7UpnpZLttsefrZU469vChGaG6hcSqrkKXQOvMTZtbjeNg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ia32]
os: [win32]
'@oxlint/binding-win32-x64-msvc@1.69.0':
resolution: {integrity: sha512-w8SOXv3mT9Fi6jY8OXdXCfnvX/3KNLXGNr4HEz2TA7S4Mv/PYAOmpB8y/ge40mxvBMgGNaSaaDwZpAsQn7HtWA==}
'@oxlint/binding-win32-x64-msvc@1.73.0':
resolution: {integrity: sha512-WaNRvh4f6zY9CvUQk2YoA1O90ieWrIklI84+HXFr9Isjz9CSESrdqo/RtIYt4Dll/cAchqGDMehfaZd0vqEFZw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [win32]
'@types/node@25.9.2':
resolution: {integrity: sha512-G05zqtJhcDLb8uslf5EjCxXg9G1KQxiV8OS0R26IC//Eoyitzqe8z37I7cqvnZlrlSfgocQRfSn/AHBZJJFyGw==}
'@types/node@26.1.1':
resolution: {integrity: sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw==}
'@typescript/native-preview-darwin-arm64@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-Yf/zHEadP/yUiWUdM/mZVfEVFJuGMf6nhRSFif0vp+FwtfGU4jmlpNF7BTJJdOHrrcWkwEJKzAoMCtEtyxhuyQ==}
'@typescript/typescript-aix-ppc64@7.0.2':
resolution: {integrity: sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==}
engines: {node: '>=16.20.0'}
cpu: [ppc64]
os: [aix]
'@typescript/typescript-darwin-arm64@7.0.2':
resolution: {integrity: sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==}
engines: {node: '>=16.20.0'}
cpu: [arm64]
os: [darwin]
'@typescript/native-preview-darwin-x64@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-z4dYWI57CPHs0wV/FWFth8fWmqYH7iOm7THOfZ5Fv0jo/SWK6kE1kEUIqIAExqo7ueRNqSrCw0I8U1J4TJszAw==}
'@typescript/typescript-darwin-x64@7.0.2':
resolution: {integrity: sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [darwin]
'@typescript/native-preview-linux-arm64@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-OxNVWH9IhrMAzNlDyDit1dPO64GFIDPOUKoruIkJ9A1ZEONfIHXG5f+V3si9jtuNmuomiz9FjpbzOqLsgaxt+w==}
'@typescript/typescript-freebsd-arm64@7.0.2':
resolution: {integrity: sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==}
engines: {node: '>=16.20.0'}
cpu: [arm64]
os: [freebsd]
'@typescript/typescript-freebsd-x64@7.0.2':
resolution: {integrity: sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [freebsd]
'@typescript/typescript-linux-arm64@7.0.2':
resolution: {integrity: sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==}
engines: {node: '>=16.20.0'}
cpu: [arm64]
os: [linux]
'@typescript/native-preview-linux-arm@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-mEtN8BbAgVtBu/5MVomYquXNvgok2C0KG6V0D4SV1jfBJNtlcqbp0WuIqT0bnM9DA4TgzcHvnFMpwGSK/dqI5A==}
'@typescript/typescript-linux-arm@7.0.2':
resolution: {integrity: sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==}
engines: {node: '>=16.20.0'}
cpu: [arm]
os: [linux]
'@typescript/native-preview-linux-x64@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-KO8WO1gBIC09T3255RlTY42TGu8en5mEkLPQu2wkMn+dX2T8KYL64zXrCeLeUWa0NvmVdJUeyWu3pFOn3zKemw==}
'@typescript/typescript-linux-loong64@7.0.2':
resolution: {integrity: sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==}
engines: {node: '>=16.20.0'}
cpu: [loong64]
os: [linux]
'@typescript/typescript-linux-mips64el@7.0.2':
resolution: {integrity: sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==}
engines: {node: '>=16.20.0'}
cpu: [mips64el]
os: [linux]
'@typescript/typescript-linux-ppc64@7.0.2':
resolution: {integrity: sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==}
engines: {node: '>=16.20.0'}
cpu: [ppc64]
os: [linux]
'@typescript/typescript-linux-riscv64@7.0.2':
resolution: {integrity: sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==}
engines: {node: '>=16.20.0'}
cpu: [riscv64]
os: [linux]
'@typescript/typescript-linux-s390x@7.0.2':
resolution: {integrity: sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==}
engines: {node: '>=16.20.0'}
cpu: [s390x]
os: [linux]
'@typescript/typescript-linux-x64@7.0.2':
resolution: {integrity: sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [linux]
'@typescript/native-preview-win32-arm64@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-+8q19LWjnMKK6SF3PLeMEalbfWDYWHs0AU8kSFCBCke/RLoDG4FjQzVtLgUo+KWhsmZMosiEyqEnZmSlED2tIQ==}
'@typescript/typescript-netbsd-arm64@7.0.2':
resolution: {integrity: sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==}
engines: {node: '>=16.20.0'}
cpu: [arm64]
os: [netbsd]
'@typescript/typescript-netbsd-x64@7.0.2':
resolution: {integrity: sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [netbsd]
'@typescript/typescript-openbsd-arm64@7.0.2':
resolution: {integrity: sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==}
engines: {node: '>=16.20.0'}
cpu: [arm64]
os: [openbsd]
'@typescript/typescript-openbsd-x64@7.0.2':
resolution: {integrity: sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [openbsd]
'@typescript/typescript-sunos-x64@7.0.2':
resolution: {integrity: sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [sunos]
'@typescript/typescript-win32-arm64@7.0.2':
resolution: {integrity: sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==}
engines: {node: '>=16.20.0'}
cpu: [arm64]
os: [win32]
'@typescript/native-preview-win32-x64@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-qNPcss+6yRoNFfFIKQbPwJWYxDfOZwyL8JBJh4J+yMLOad/+/AOjsO4EtZsIpv5PMCjpnD75coBoDkw+5NkItw==}
'@typescript/typescript-win32-x64@7.0.2':
resolution: {integrity: sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==}
engines: {node: '>=16.20.0'}
cpu: [x64]
os: [win32]
'@typescript/native-preview@7.0.0-dev.20260609.1':
resolution: {integrity: sha512-1HOuH/u/451O3hx4Z9fesNqarpeit6UfkgwK96sCVWi5p69F0N3v+6bI969lLIjF7K9dbYQNiWUaZ6Wik87iKg==}
engines: {node: '>=16.20.0'}
'@typescript/typescript6@6.0.2':
resolution: {integrity: sha512-mbCddXd+jm7hfx7w2YU64/Av4/NqqeG3GoRZgxPcgoTxYjhrcfJRw9ULch71SS4G+Q3bOXFhRvPqjguN0Hyp5w==}
hasBin: true
ajv@8.20.0:
@@ -366,8 +446,8 @@ packages:
fast-deep-equal@3.1.3:
resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==}
fast-uri@3.1.2:
resolution: {integrity: sha512-rVjf7ArG3LTk+FS6Yw81V1DLuZl1bRbNrev6Tmd/9RaroeeRRJhAt7jg/6YFxbvAQXUCavSoZhPPj6oOx+5KjQ==}
fast-uri@3.1.4:
resolution: {integrity: sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw==}
json-schema-traverse@1.0.0:
resolution: {integrity: sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==}
@@ -385,16 +465,16 @@ packages:
vite-plus:
optional: true
oxlint-tsgolint@0.23.0:
resolution: {integrity: sha512-3mBv3CoPbh8dFbzfDGIWa2ytZjn2v+3EX4aKRXjIhsoGFzG8GCjfRirz3rwZf1wYbZzsNLTSgpw8VjQuWdp/jA==}
oxlint-tsgolint@0.24.0:
resolution: {integrity: sha512-giCk5sEvG02d5tzPmFMX3hem8ndzEEu1xvGYS5OwNfO2WGl6ZVxt5LjE0yiMDoz94INI7XkXwgFAQiydPvVHDw==}
hasBin: true
oxlint@1.69.0:
resolution: {integrity: sha512-ypZkK/aDc5NQV8zIR6s2H2Tl3aNW8FmJ1m9+2qsaYuRenl8vgnHNCGwTHviWJdUQzglOlHFchgopdtGhSy17Rw==}
oxlint@1.73.0:
resolution: {integrity: sha512-u91G9TJzU6yqKWNZUYprQB07W7YvntZXaRxQ6CkoytepYhLWUXWsr1M8zUJ34VatNPuUAr3Z8GH+O2A331CluQ==}
engines: {node: ^20.19.0 || >=22.12.0}
hasBin: true
peerDependencies:
oxlint-tsgolint: '>=0.22.1'
oxlint-tsgolint: '>=0.24.0'
vite-plus: '*'
peerDependenciesMeta:
oxlint-tsgolint:
@@ -415,8 +495,13 @@ packages:
engines: {node: '>=14.17'}
hasBin: true
undici-types@7.24.6:
resolution: {integrity: sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg==}
typescript@7.0.2:
resolution: {integrity: sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==}
engines: {node: '>=16.20.0'}
hasBin: true
undici-types@8.3.0:
resolution: {integrity: sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==}
yaml@2.9.0:
resolution: {integrity: sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==}
@@ -482,126 +567,159 @@ snapshots:
'@oxfmt/binding-win32-x64-msvc@0.54.0':
optional: true
'@oxlint-tsgolint/darwin-arm64@0.23.0':
'@oxlint-tsgolint/darwin-arm64@0.24.0':
optional: true
'@oxlint-tsgolint/darwin-x64@0.23.0':
'@oxlint-tsgolint/darwin-x64@0.24.0':
optional: true
'@oxlint-tsgolint/linux-arm64@0.23.0':
'@oxlint-tsgolint/linux-arm64@0.24.0':
optional: true
'@oxlint-tsgolint/linux-x64@0.23.0':
'@oxlint-tsgolint/linux-x64@0.24.0':
optional: true
'@oxlint-tsgolint/win32-arm64@0.23.0':
'@oxlint-tsgolint/win32-arm64@0.24.0':
optional: true
'@oxlint-tsgolint/win32-x64@0.23.0':
'@oxlint-tsgolint/win32-x64@0.24.0':
optional: true
'@oxlint/binding-android-arm-eabi@1.69.0':
'@oxlint/binding-android-arm-eabi@1.73.0':
optional: true
'@oxlint/binding-android-arm64@1.69.0':
'@oxlint/binding-android-arm64@1.73.0':
optional: true
'@oxlint/binding-darwin-arm64@1.69.0':
'@oxlint/binding-darwin-arm64@1.73.0':
optional: true
'@oxlint/binding-darwin-x64@1.69.0':
'@oxlint/binding-darwin-x64@1.73.0':
optional: true
'@oxlint/binding-freebsd-x64@1.69.0':
'@oxlint/binding-freebsd-x64@1.73.0':
optional: true
'@oxlint/binding-linux-arm-gnueabihf@1.69.0':
'@oxlint/binding-linux-arm-gnueabihf@1.73.0':
optional: true
'@oxlint/binding-linux-arm-musleabihf@1.69.0':
'@oxlint/binding-linux-arm-musleabihf@1.73.0':
optional: true
'@oxlint/binding-linux-arm64-gnu@1.69.0':
'@oxlint/binding-linux-arm64-gnu@1.73.0':
optional: true
'@oxlint/binding-linux-arm64-musl@1.69.0':
'@oxlint/binding-linux-arm64-musl@1.73.0':
optional: true
'@oxlint/binding-linux-ppc64-gnu@1.69.0':
'@oxlint/binding-linux-ppc64-gnu@1.73.0':
optional: true
'@oxlint/binding-linux-riscv64-gnu@1.69.0':
'@oxlint/binding-linux-riscv64-gnu@1.73.0':
optional: true
'@oxlint/binding-linux-riscv64-musl@1.69.0':
'@oxlint/binding-linux-riscv64-musl@1.73.0':
optional: true
'@oxlint/binding-linux-s390x-gnu@1.69.0':
'@oxlint/binding-linux-s390x-gnu@1.73.0':
optional: true
'@oxlint/binding-linux-x64-gnu@1.69.0':
'@oxlint/binding-linux-x64-gnu@1.73.0':
optional: true
'@oxlint/binding-linux-x64-musl@1.69.0':
'@oxlint/binding-linux-x64-musl@1.73.0':
optional: true
'@oxlint/binding-openharmony-arm64@1.69.0':
'@oxlint/binding-openharmony-arm64@1.73.0':
optional: true
'@oxlint/binding-win32-arm64-msvc@1.69.0':
'@oxlint/binding-win32-arm64-msvc@1.73.0':
optional: true
'@oxlint/binding-win32-ia32-msvc@1.69.0':
'@oxlint/binding-win32-ia32-msvc@1.73.0':
optional: true
'@oxlint/binding-win32-x64-msvc@1.69.0':
'@oxlint/binding-win32-x64-msvc@1.73.0':
optional: true
'@types/node@25.9.2':
'@types/node@26.1.1':
dependencies:
undici-types: 7.24.6
undici-types: 8.3.0
'@typescript/native-preview-darwin-arm64@7.0.0-dev.20260609.1':
'@typescript/typescript-aix-ppc64@7.0.2':
optional: true
'@typescript/native-preview-darwin-x64@7.0.0-dev.20260609.1':
'@typescript/typescript-darwin-arm64@7.0.2':
optional: true
'@typescript/native-preview-linux-arm64@7.0.0-dev.20260609.1':
'@typescript/typescript-darwin-x64@7.0.2':
optional: true
'@typescript/native-preview-linux-arm@7.0.0-dev.20260609.1':
'@typescript/typescript-freebsd-arm64@7.0.2':
optional: true
'@typescript/native-preview-linux-x64@7.0.0-dev.20260609.1':
'@typescript/typescript-freebsd-x64@7.0.2':
optional: true
'@typescript/native-preview-win32-arm64@7.0.0-dev.20260609.1':
'@typescript/typescript-linux-arm64@7.0.2':
optional: true
'@typescript/native-preview-win32-x64@7.0.0-dev.20260609.1':
'@typescript/typescript-linux-arm@7.0.2':
optional: true
'@typescript/native-preview@7.0.0-dev.20260609.1':
optionalDependencies:
'@typescript/native-preview-darwin-arm64': 7.0.0-dev.20260609.1
'@typescript/native-preview-darwin-x64': 7.0.0-dev.20260609.1
'@typescript/native-preview-linux-arm': 7.0.0-dev.20260609.1
'@typescript/native-preview-linux-arm64': 7.0.0-dev.20260609.1
'@typescript/native-preview-linux-x64': 7.0.0-dev.20260609.1
'@typescript/native-preview-win32-arm64': 7.0.0-dev.20260609.1
'@typescript/native-preview-win32-x64': 7.0.0-dev.20260609.1
'@typescript/typescript-linux-loong64@7.0.2':
optional: true
'@typescript/typescript-linux-mips64el@7.0.2':
optional: true
'@typescript/typescript-linux-ppc64@7.0.2':
optional: true
'@typescript/typescript-linux-riscv64@7.0.2':
optional: true
'@typescript/typescript-linux-s390x@7.0.2':
optional: true
'@typescript/typescript-linux-x64@7.0.2':
optional: true
'@typescript/typescript-netbsd-arm64@7.0.2':
optional: true
'@typescript/typescript-netbsd-x64@7.0.2':
optional: true
'@typescript/typescript-openbsd-arm64@7.0.2':
optional: true
'@typescript/typescript-openbsd-x64@7.0.2':
optional: true
'@typescript/typescript-sunos-x64@7.0.2':
optional: true
'@typescript/typescript-win32-arm64@7.0.2':
optional: true
'@typescript/typescript-win32-x64@7.0.2':
optional: true
'@typescript/typescript6@6.0.2':
dependencies:
'@typescript/old': typescript@6.0.3
ajv@8.20.0:
dependencies:
fast-deep-equal: 3.1.3
fast-uri: 3.1.2
fast-uri: 3.1.4
json-schema-traverse: 1.0.0
require-from-string: 2.0.2
fast-deep-equal@3.1.3: {}
fast-uri@3.1.2: {}
fast-uri@3.1.4: {}
json-schema-traverse@1.0.0: {}
@@ -629,37 +747,37 @@ snapshots:
'@oxfmt/binding-win32-ia32-msvc': 0.54.0
'@oxfmt/binding-win32-x64-msvc': 0.54.0
oxlint-tsgolint@0.23.0:
oxlint-tsgolint@0.24.0:
optionalDependencies:
'@oxlint-tsgolint/darwin-arm64': 0.23.0
'@oxlint-tsgolint/darwin-x64': 0.23.0
'@oxlint-tsgolint/linux-arm64': 0.23.0
'@oxlint-tsgolint/linux-x64': 0.23.0
'@oxlint-tsgolint/win32-arm64': 0.23.0
'@oxlint-tsgolint/win32-x64': 0.23.0
'@oxlint-tsgolint/darwin-arm64': 0.24.0
'@oxlint-tsgolint/darwin-x64': 0.24.0
'@oxlint-tsgolint/linux-arm64': 0.24.0
'@oxlint-tsgolint/linux-x64': 0.24.0
'@oxlint-tsgolint/win32-arm64': 0.24.0
'@oxlint-tsgolint/win32-x64': 0.24.0
oxlint@1.69.0(oxlint-tsgolint@0.23.0):
oxlint@1.73.0(oxlint-tsgolint@0.24.0):
optionalDependencies:
'@oxlint/binding-android-arm-eabi': 1.69.0
'@oxlint/binding-android-arm64': 1.69.0
'@oxlint/binding-darwin-arm64': 1.69.0
'@oxlint/binding-darwin-x64': 1.69.0
'@oxlint/binding-freebsd-x64': 1.69.0
'@oxlint/binding-linux-arm-gnueabihf': 1.69.0
'@oxlint/binding-linux-arm-musleabihf': 1.69.0
'@oxlint/binding-linux-arm64-gnu': 1.69.0
'@oxlint/binding-linux-arm64-musl': 1.69.0
'@oxlint/binding-linux-ppc64-gnu': 1.69.0
'@oxlint/binding-linux-riscv64-gnu': 1.69.0
'@oxlint/binding-linux-riscv64-musl': 1.69.0
'@oxlint/binding-linux-s390x-gnu': 1.69.0
'@oxlint/binding-linux-x64-gnu': 1.69.0
'@oxlint/binding-linux-x64-musl': 1.69.0
'@oxlint/binding-openharmony-arm64': 1.69.0
'@oxlint/binding-win32-arm64-msvc': 1.69.0
'@oxlint/binding-win32-ia32-msvc': 1.69.0
'@oxlint/binding-win32-x64-msvc': 1.69.0
oxlint-tsgolint: 0.23.0
'@oxlint/binding-android-arm-eabi': 1.73.0
'@oxlint/binding-android-arm64': 1.73.0
'@oxlint/binding-darwin-arm64': 1.73.0
'@oxlint/binding-darwin-x64': 1.73.0
'@oxlint/binding-freebsd-x64': 1.73.0
'@oxlint/binding-linux-arm-gnueabihf': 1.73.0
'@oxlint/binding-linux-arm-musleabihf': 1.73.0
'@oxlint/binding-linux-arm64-gnu': 1.73.0
'@oxlint/binding-linux-arm64-musl': 1.73.0
'@oxlint/binding-linux-ppc64-gnu': 1.73.0
'@oxlint/binding-linux-riscv64-gnu': 1.73.0
'@oxlint/binding-linux-riscv64-musl': 1.73.0
'@oxlint/binding-linux-s390x-gnu': 1.73.0
'@oxlint/binding-linux-x64-gnu': 1.73.0
'@oxlint/binding-linux-x64-musl': 1.73.0
'@oxlint/binding-openharmony-arm64': 1.73.0
'@oxlint/binding-win32-arm64-msvc': 1.73.0
'@oxlint/binding-win32-ia32-msvc': 1.73.0
'@oxlint/binding-win32-x64-msvc': 1.73.0
oxlint-tsgolint: 0.24.0
require-from-string@2.0.2: {}
@@ -667,6 +785,29 @@ snapshots:
typescript@6.0.3: {}
undici-types@7.24.6: {}
typescript@7.0.2:
optionalDependencies:
'@typescript/typescript-aix-ppc64': 7.0.2
'@typescript/typescript-darwin-arm64': 7.0.2
'@typescript/typescript-darwin-x64': 7.0.2
'@typescript/typescript-freebsd-arm64': 7.0.2
'@typescript/typescript-freebsd-x64': 7.0.2
'@typescript/typescript-linux-arm': 7.0.2
'@typescript/typescript-linux-arm64': 7.0.2
'@typescript/typescript-linux-loong64': 7.0.2
'@typescript/typescript-linux-mips64el': 7.0.2
'@typescript/typescript-linux-ppc64': 7.0.2
'@typescript/typescript-linux-riscv64': 7.0.2
'@typescript/typescript-linux-s390x': 7.0.2
'@typescript/typescript-linux-x64': 7.0.2
'@typescript/typescript-netbsd-arm64': 7.0.2
'@typescript/typescript-netbsd-x64': 7.0.2
'@typescript/typescript-openbsd-arm64': 7.0.2
'@typescript/typescript-openbsd-x64': 7.0.2
'@typescript/typescript-sunos-x64': 7.0.2
'@typescript/typescript-win32-arm64': 7.0.2
'@typescript/typescript-win32-x64': 7.0.2
undici-types@8.3.0: {}
yaml@2.9.0: {}
pnpm-workspace.yaml
+29
View File
@@ -0,0 +1,29 @@
packages:
- '.'
overrides:
fast-uri: 3.1.4
minimumReleaseAge: 2880
minimumReleaseAgeExclude:
- '@typescript/typescript-aix-ppc64@7.0.2'
- '@typescript/typescript-darwin-arm64@7.0.2'
- '@typescript/typescript-darwin-x64@7.0.2'
- '@typescript/typescript-freebsd-arm64@7.0.2'
- '@typescript/typescript-freebsd-x64@7.0.2'
- '@typescript/typescript-linux-arm64@7.0.2'
- '@typescript/typescript-linux-arm@7.0.2'
- '@typescript/typescript-linux-loong64@7.0.2'
- '@typescript/typescript-linux-mips64el@7.0.2'
- '@typescript/typescript-linux-ppc64@7.0.2'
- '@typescript/typescript-linux-riscv64@7.0.2'
- '@typescript/typescript-linux-s390x@7.0.2'
- '@typescript/typescript-linux-x64@7.0.2'
- '@typescript/typescript-netbsd-arm64@7.0.2'
- '@typescript/typescript-netbsd-x64@7.0.2'
- '@typescript/typescript-openbsd-arm64@7.0.2'
- '@typescript/typescript-openbsd-x64@7.0.2'
- '@typescript/typescript-sunos-x64@7.0.2'
- '@typescript/typescript-win32-arm64@7.0.2'
- '@typescript/typescript-win32-x64@7.0.2'
- typescript@7.0.2
src/abortable_process.ts
+237
View File
@@ -0,0 +1,237 @@
import { spawn, type ChildProcess } from "node:child_process";
const ABORT_FORCE_KILL_AFTER_MS = 250;
const forceTerminationCallbacks = new WeakMap<AbortSignal, Set<() => void>>();
type ProcessResult = {
stdout: string;
stderr: string;
code: number | null;
};
type RunAbortableProcessOptions = {
command: string;
argv: string[];
env: NodeJS.ProcessEnv;
cwd?: string;
stdin?: string | null;
signal?: AbortSignal;
forceTerminationSignal?: AbortSignal;
killSignal?: NodeJS.Signals | (() => NodeJS.Signals | undefined);
maxOutputBytes?: number;
outputLimitMessage?: string;
notFoundMessage: string;
};
export function forceTerminateAbortableProcesses(signal: AbortSignal) {
for (const terminate of forceTerminationCallbacks.get(signal) ?? []) terminate();
}
function abortError(signal: AbortSignal) {
if (signal.reason instanceof Error) return signal.reason;
const error = new Error("This operation was aborted");
error.name = "AbortError";
return error;
}
function terminateProcessTree(child: ChildProcess, signal: NodeJS.Signals): Promise<void> {
if (!child.pid) return Promise.resolve();
if (process.platform === "win32") {
return new Promise((resolve) => {
const taskkill = spawn("taskkill", ["/pid", String(child.pid), "/T", "/F"], {
stdio: "ignore",
windowsHide: true,
});
taskkill.once("error", () => {
child.kill(signal);
resolve();
});
taskkill.once("close", resolve);
});
}
try {
process.kill(-child.pid, signal);
} catch {
child.kill(signal);
}
return Promise.resolve();
}
export function runAbortableProcess({
command,
argv,
env,
cwd,
stdin,
signal,
forceTerminationSignal,
killSignal,
maxOutputBytes,
outputLimitMessage,
notFoundMessage,
}: RunAbortableProcessOptions): Promise<ProcessResult> {
return new Promise((resolve, reject) => {
if (
maxOutputBytes !== undefined &&
(!Number.isSafeInteger(maxOutputBytes) || maxOutputBytes < 0)
) {
reject(new Error("maxOutputBytes must be a non-negative safe integer"));
return;
}
try {
signal?.throwIfAborted();
} catch (err) {
reject(err);
return;
}
const child = spawn(command, argv, {
env,
cwd,
stdio: ["pipe", "pipe", "pipe"],
// Create a dedicated POSIX process group only when this runner owns a
// cancellation signal for it. Direct APIs without one must retain the
// caller's terminal process group so Ctrl-C still reaches their child.
detached: process.platform !== "win32" && signal !== undefined,
});
let stdout = "";
let stderr = "";
let stdoutBytes = 0;
let stderrBytes = 0;
let terminationError: Error | undefined;
let forceKillTimer: ReturnType<typeof setTimeout> | undefined;
let processClosed = false;
let forceKillRequested = false;
let forceKillIssued = false;
let settled = false;
const forceTerminationRegistrations: Set<() => void>[] = [];
let forceTerminate: (() => void) | undefined;
const cleanup = () => {
signal?.removeEventListener("abort", onAbort);
if (forceKillTimer) clearTimeout(forceKillTimer);
if (forceTerminate) {
for (const listeners of forceTerminationRegistrations) listeners.delete(forceTerminate);
}
};
const failTerminationWhenTreeIsStopped = () => {
if (!terminationError || !processClosed || !forceKillIssued || settled) return;
settled = true;
cleanup();
reject(terminationError);
};
const fail = (error: Error) => {
if (settled) return;
settled = true;
cleanup();
reject(error);
};
const forceKill = () => {
if (forceKillRequested) return;
forceKillRequested = true;
void terminateProcessTree(child, "SIGKILL").finally(() => {
forceKillIssued = true;
failTerminationWhenTreeIsStopped();
});
};
const startTermination = (error: Error) => {
if (settled || terminationError) return;
terminationError = error;
const initialKillSignal =
(typeof killSignal === "function" ? killSignal() : killSignal) ?? "SIGTERM";
if (initialKillSignal === "SIGKILL") {
forceKill();
return;
}
void terminateProcessTree(child, initialKillSignal);
forceKillTimer = setTimeout(() => {
forceKillTimer = undefined;
forceKill();
}, ABORT_FORCE_KILL_AFTER_MS);
};
forceTerminate = () => {
if (settled) return;
if (!terminationError) {
terminationError = signal ? abortError(signal) : new Error("Process termination requested");
}
if (forceKillTimer) {
clearTimeout(forceKillTimer);
forceKillTimer = undefined;
}
forceKill();
};
const onAbort = () => {
if (!signal) return;
startTermination(abortError(signal));
};
const appendOutput = (stream: "stdout" | "stderr", data: string) => {
const bytes = Buffer.byteLength(data);
const total = stream === "stdout" ? stdoutBytes + bytes : stderrBytes + bytes;
if (maxOutputBytes !== undefined && total > maxOutputBytes) {
startTermination(
new Error(outputLimitMessage ?? `Process output exceeded ${maxOutputBytes} bytes`),
);
return;
}
if (stream === "stdout") {
stdoutBytes = total;
stdout += data;
} else {
stderrBytes = total;
stderr += data;
}
};
child.stdout?.setEncoding("utf8");
child.stderr?.setEncoding("utf8");
child.stdout?.on("data", (data: string) => appendOutput("stdout", data));
child.stderr?.on("data", (data: string) => appendOutput("stderr", data));
child.stdin?.on("error", () => {});
if (typeof stdin === "string") child.stdin?.write(stdin);
child.stdin?.end();
child.on("error", (error: NodeJS.ErrnoException) => {
if (terminationError) {
processClosed = true;
failTerminationWhenTreeIsStopped();
return;
}
if (error.code === "ENOENT") {
fail(new Error(notFoundMessage));
return;
}
fail(error);
});
child.on("close", (code) => {
if (settled) return;
processClosed = true;
if (terminationError) {
failTerminationWhenTreeIsStopped();
return;
}
settled = true;
cleanup();
resolve({ stdout, stderr, code });
});
for (const registrationSignal of new Set(
[signal, forceTerminationSignal].filter(
(candidate): candidate is AbortSignal => candidate !== undefined,
),
)) {
let listeners = forceTerminationCallbacks.get(registrationSignal);
if (!listeners) {
listeners = new Set();
forceTerminationCallbacks.set(registrationSignal, listeners);
}
listeners.add(forceTerminate);
forceTerminationRegistrations.push(listeners);
}
if (signal) {
signal.addEventListener("abort", onAbort, { once: true });
if (signal.aborted) onAbort();
}
});
}
src/cli.ts
+641 -768
View File
File diff suppressed because it is too large Load Diff
src/commands/commands_list.ts
+45 -45
View File
@@ -1,55 +1,55 @@
import type { CommandMeta, LobsterCommand } from "./types.js";
function parseDescriptionFromHelp(helpText: string): string {
const firstLine = helpText.split("\n").find((l) => l.trim().length > 0) ?? "";
// Expected pattern: "name — description" but fall back to the line as-is.
return firstLine.includes("—")
? firstLine.split("—").slice(1).join("—").trim()
: firstLine.trim();
const firstLine = helpText.split("\n").find((l) => l.trim().length > 0) ?? "";
// Expected pattern: "name — description" but fall back to the line as-is.
return firstLine.includes("—")
? firstLine.split("—").slice(1).join("—").trim()
: firstLine.trim();
}
export const commandsListCommand: LobsterCommand = {
name: "commands.list",
help() {
return (
`commands.list — list available Lobster pipeline commands\n\n` +
`Usage:\n` +
` commands.list\n\n` +
`Notes:\n` +
` - Intended for agents (e.g. OpenClaw) to discover available pipeline stages dynamically.\n` +
` - Output includes name/description plus optional metadata (argsSchema/examples/sideEffects) when provided by commands.\n`
);
},
meta: {
description: "List available Lobster pipeline commands",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
} satisfies CommandMeta,
async run({ input, ctx }) {
// Drain input
for await (const _ of input) {
// no-op
}
name: "commands.list",
help() {
return (
`commands.list — list available Lobster pipeline commands\n\n` +
`Usage:\n` +
` commands.list\n\n` +
`Notes:\n` +
` - Intended for agents (e.g. OpenClaw) to discover available pipeline stages dynamically.\n` +
` - Output includes name/description plus optional metadata (argsSchema/examples/sideEffects) when provided by commands.\n`
);
},
meta: {
description: "List available Lobster pipeline commands",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
} satisfies CommandMeta,
async run({ input, ctx }) {
// Drain input
for await (const _ of input) {
// no-op
}
const names = ctx.registry.list();
const output = names.map((name) => {
const cmd = ctx.registry.get(name) as LobsterCommand | undefined;
const help = typeof cmd?.help === "function" ? String(cmd.help()) : "";
const description = cmd?.meta?.description ?? parseDescriptionFromHelp(help);
const names = ctx.registry.list();
const output = names.map((name) => {
const cmd = ctx.registry.get(name) as LobsterCommand | undefined;
const help = typeof cmd?.help === "function" ? String(cmd.help()) : "";
const description = cmd?.meta?.description ?? parseDescriptionFromHelp(help);
return {
name,
description,
argsSchema: cmd?.meta?.argsSchema ?? null,
examples: cmd?.meta?.examples ?? null,
sideEffects: cmd?.meta?.sideEffects ?? null,
};
});
return {
name,
description,
argsSchema: cmd?.meta?.argsSchema ?? null,
examples: cmd?.meta?.examples ?? null,
sideEffects: cmd?.meta?.sideEffects ?? null,
};
});
return {
output: (async function* () {
for (const item of output) yield item;
})(),
};
},
return {
output: (async function* () {
for (const item of output) yield item;
})(),
};
},
};
src/commands/registry.ts
+41 -39
View File
@@ -12,6 +12,7 @@ import { groupByCommand } from "./stdlib/group_by.js";
import { approveCommand } from "./stdlib/approve.js";
import { askCommand } from "./stdlib/ask.js";
import { clawdInvokeCommand, openclawInvokeCommand } from "./stdlib/openclaw_invoke.js";
import { openclawAgentCommand } from "./stdlib/openclaw_agent.js";
import { llmInvokeCommand } from "./stdlib/llm_invoke.js";
import { llmTaskInvokeCommand } from "./stdlib/llm_task_invoke.js";
import { stateGetCommand, stateSetCommand } from "./stdlib/state.js";
@@ -24,45 +25,46 @@ import { gogGmailSendCommand } from "./stdlib/gog_gmail_send.js";
import { emailTriageCommand } from "./stdlib/email_triage.js";
export function createDefaultRegistry() {
const commands = new Map();
const commands = new Map();
for (const cmd of [
execCommand,
headCommand,
jsonCommand,
pickCommand,
tableCommand,
whereCommand,
sortCommand,
dedupeCommand,
templateCommand,
mapCommand,
groupByCommand,
approveCommand,
askCommand,
openclawInvokeCommand,
clawdInvokeCommand,
llmInvokeCommand,
llmTaskInvokeCommand,
stateGetCommand,
stateSetCommand,
diffLastCommand,
workflowsListCommand,
workflowsRunCommand,
commandsListCommand,
gogGmailSearchCommand,
gogGmailSendCommand,
emailTriageCommand,
]) {
commands.set(cmd.name, cmd);
}
for (const cmd of [
execCommand,
headCommand,
jsonCommand,
pickCommand,
tableCommand,
whereCommand,
sortCommand,
dedupeCommand,
templateCommand,
mapCommand,
groupByCommand,
approveCommand,
askCommand,
openclawInvokeCommand,
clawdInvokeCommand,
openclawAgentCommand,
llmInvokeCommand,
llmTaskInvokeCommand,
stateGetCommand,
stateSetCommand,
diffLastCommand,
workflowsListCommand,
workflowsRunCommand,
commandsListCommand,
gogGmailSearchCommand,
gogGmailSendCommand,
emailTriageCommand,
]) {
commands.set(cmd.name, cmd);
}
return {
get(name) {
return commands.get(name);
},
list() {
return [...commands.keys()].sort();
},
};
return {
get(name) {
return commands.get(name);
},
list() {
return [...commands.keys()].sort();
},
};
}
src/commands/stdlib/approve.ts
+63 -61
View File
@@ -1,83 +1,85 @@
import { readLineFromStream } from "../../read_line.js";
function isInteractive(stdin) {
return Boolean(stdin.isTTY);
return Boolean(stdin.isTTY);
}
export const approveCommand = {
name: "approve",
meta: {
description: "Require confirmation to continue",
argsSchema: {
type: "object",
properties: {
prompt: { type: "string", description: "Approval prompt text", default: "Approve?" },
emit: { type: "boolean", description: "Force emit approval request + halt" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return `approve — require confirmation to continue\n\nUsage:\n ... | approve --prompt "Send these emails?"\n ... | approve --emit --prompt "Send these emails?"\n ... | approve --emit --preview-from-stdin --limit 5 --prompt "Proceed?"\n\nModes:\n - Interactive (default): prompts on TTY and passes items through if approved.\n - Emit (--emit): returns an approval request object and stops the pipeline.\n\nNotes:\n - In tool mode (or non-interactive), this emits an approval request and halts.\n`;
},
async run({ input, args, ctx }) {
const prompt = args.prompt ?? "Approve?";
const previewFromStdin = Boolean(args.previewFromStdin ?? args["preview-from-stdin"]);
const previewLimitRaw = args.limit ?? args.previewLimit ?? args["preview-limit"];
const previewLimit = Number.isFinite(Number(previewLimitRaw)) ? Number(previewLimitRaw) : 5;
name: "approve",
meta: {
description: "Require confirmation to continue",
argsSchema: {
type: "object",
properties: {
prompt: { type: "string", description: "Approval prompt text", default: "Approve?" },
emit: { type: "boolean", description: "Force emit approval request + halt" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
resumeSafeBeforeInput: true,
},
help() {
return `approve — require confirmation to continue\n\nUsage:\n ... | approve --prompt "Send these emails?"\n ... | approve --emit --prompt "Send these emails?"\n ... | approve --emit --preview-from-stdin --limit 5 --prompt "Proceed?"\n\nModes:\n - Interactive (default): prompts on TTY and passes items through if approved.\n - Emit (--emit): returns an approval request object and stops the pipeline.\n\nNotes:\n - In tool mode (or non-interactive), this emits an approval request and halts.\n`;
},
async run({ input, args, ctx }) {
const prompt = args.prompt ?? "Approve?";
const previewFromStdin = Boolean(args.previewFromStdin ?? args["preview-from-stdin"]);
const previewLimitRaw = args.limit ?? args.previewLimit ?? args["preview-limit"];
const previewLimit = Number.isFinite(Number(previewLimitRaw)) ? Number(previewLimitRaw) : 5;
const items = [];
for await (const item of input) items.push(item);
const items = [];
for await (const item of input) items.push(item);
const emit = Boolean(args.emit) || ctx.mode === "tool" || !isInteractive(ctx.stdin);
const emit = Boolean(args.emit) || ctx.mode === "tool" || !isInteractive(ctx.stdin);
if (emit) {
const preview = previewFromStdin
? buildPreview(items.slice(0, Math.max(0, previewLimit)))
: undefined;
return {
halt: true,
output: (async function* () {
yield {
type: "approval_request",
prompt,
items,
...(preview ? { preview } : null),
};
})(),
};
}
if (emit) {
const preview = previewFromStdin
? buildPreview(items.slice(0, Math.max(0, previewLimit)))
: undefined;
return {
halt: true,
output: (async function* () {
yield {
type: "approval_request",
prompt,
items,
...(preview ? { preview } : null),
};
})(),
};
}
ctx.stdout.write(`${prompt} [y/N] `);
const answer = await readLineFromStream(ctx.stdin, {
timeoutMs: parseApprovalTimeoutMs(ctx.env),
});
ctx.stdout.write(`${prompt} [y/N] `);
const answer = await readLineFromStream(ctx.stdin, {
timeoutMs: parseApprovalTimeoutMs(ctx.env),
signal: ctx.signal,
});
if (!/^y(es)?$/i.test(String(answer).trim())) {
throw new Error("Not approved");
}
if (!/^y(es)?$/i.test(String(answer).trim())) {
throw new Error("Not approved");
}
return { output: asStream(items) };
},
return { output: asStream(items) };
},
};
function buildPreview(items) {
if (!items.length) return "";
if (items.every((item) => typeof item === "string")) {
return items.join("\n");
}
return JSON.stringify(items, null, 2);
if (!items.length) return "";
if (items.every((item) => typeof item === "string")) {
return items.join("\n");
}
return JSON.stringify(items, null, 2);
}
function parseApprovalTimeoutMs(env) {
const raw = env?.LOBSTER_APPROVAL_INPUT_TIMEOUT_MS;
const value = Number(raw);
if (!Number.isFinite(value) || value <= 0) return 0;
return Math.floor(value);
const raw = env?.LOBSTER_APPROVAL_INPUT_TIMEOUT_MS;
const value = Number(raw);
if (!Number.isFinite(value) || value <= 0) return 0;
return Math.floor(value);
}
async function* asStream(items) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/commands/stdlib/ask.ts
+154 -152
View File
@@ -1,179 +1,181 @@
import { compileCached } from "../../validation.js";
function isInteractive(stdin) {
return Boolean(stdin.isTTY);
return Boolean(stdin.isTTY);
}
function compileAskValidator(schema) {
try {
return compileCached(schema);
} catch {
throw new Error("ask response schema is invalid");
}
try {
return compileCached(schema);
} catch {
throw new Error("ask response schema is invalid");
}
}
function validateAskResponse(validator, response) {
const ok = validator(response);
if (ok) return;
const first = validator.errors?.[0];
const pathValue = first?.instancePath || "/";
const reason = first?.message ? ` ${first.message}` : "";
throw new Error(`ask response failed schema validation at ${pathValue}:${reason}`);
const ok = validator(response);
if (ok) return;
const first = validator.errors?.[0];
const pathValue = first?.instancePath || "/";
const reason = first?.message ? ` ${first.message}` : "";
throw new Error(`ask response failed schema validation at ${pathValue}:${reason}`);
}
function parseInteractiveCandidates(text) {
let parsed;
try {
parsed = JSON.parse(text);
} catch {
return [text, { decision: text }];
}
if (typeof parsed === "string") {
return [parsed, { decision: parsed }];
}
return [parsed];
let parsed;
try {
parsed = JSON.parse(text);
} catch {
return [text, { decision: text }];
}
if (typeof parsed === "string") {
return [parsed, { decision: parsed }];
}
return [parsed];
}
export const askCommand = {
name: "ask",
meta: {
description: "Pause and request structured input from the user",
argsSchema: {
type: "object",
properties: {
prompt: {
type: "string",
description: "Question or instruction to show",
default: "Input required",
},
schema: { type: "string", description: "JSON Schema string for the expected response" },
"subject-from-stdin": {
type: "boolean",
description: "Use stdin content as the subject (preview text)",
},
emit: { type: "boolean", description: "Force emit mode" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return [
"ask — pause and request structured input from the user",
"",
"Usage:",
' ... | ask --prompt "Approve, reject, or send feedback:"',
' ... | ask --prompt "Feedback?" --schema \'{"type":"object","properties":{"decision":{"type":"string"},"feedback":{"type":"string"}},"required":["decision"]}\'',
' ... | ask --subject-from-stdin --prompt "Review this draft:"',
"",
"Notes:",
" - In tool mode (or non-interactive), emits a needs_input envelope and halts.",
" - Use --schema to constrain the response shape (JSON Schema).",
" - Use --subject-from-stdin to embed the current pipeline value as preview text.",
].join("\n");
},
async run({ input, args, ctx }) {
const prompt = typeof args.prompt === "string" ? args.prompt : "Input required";
const subjectFromStdin = Boolean(args["subject-from-stdin"] ?? args.subjectFromStdin);
const schemaRaw = typeof args.schema === "string" ? args.schema : null;
name: "ask",
meta: {
description: "Pause and request structured input from the user",
argsSchema: {
type: "object",
properties: {
prompt: {
type: "string",
description: "Question or instruction to show",
default: "Input required",
},
schema: { type: "string", description: "JSON Schema string for the expected response" },
"subject-from-stdin": {
type: "boolean",
description: "Use stdin content as the subject (preview text)",
},
emit: { type: "boolean", description: "Force emit mode" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
resumeSafeBeforeInput: true,
resumeSafeAfterInput: true,
},
help() {
return [
"ask — pause and request structured input from the user",
"",
"Usage:",
' ... | ask --prompt "Approve, reject, or send feedback:"',
' ... | ask --prompt "Feedback?" --schema \'{"type":"object","properties":{"decision":{"type":"string"},"feedback":{"type":"string"}},"required":["decision"]}\'',
' ... | ask --subject-from-stdin --prompt "Review this draft:"',
"",
"Notes:",
" - In tool mode (or non-interactive), emits a needs_input envelope and halts.",
" - Use --schema to constrain the response shape (JSON Schema).",
" - Use --subject-from-stdin to embed the current pipeline value as preview text.",
].join("\n");
},
async run({ input, args, ctx }) {
const prompt = typeof args.prompt === "string" ? args.prompt : "Input required";
const subjectFromStdin = Boolean(args["subject-from-stdin"] ?? args.subjectFromStdin);
const schemaRaw = typeof args.schema === "string" ? args.schema : null;
const defaultSchema = {
type: "object",
properties: {
decision: { type: "string", enum: ["approve", "reject", "redraft"] },
feedback: { type: "string", description: "Feedback for redraft" },
},
required: ["decision"],
};
const defaultSchema = {
type: "object",
properties: {
decision: { type: "string", enum: ["approve", "reject", "redraft"] },
feedback: { type: "string", description: "Feedback for redraft" },
},
required: ["decision"],
};
let responseSchema = defaultSchema;
if (schemaRaw) {
let parsedSchema;
try {
parsedSchema = JSON.parse(schemaRaw);
} catch {
throw new Error("ask --schema must be valid JSON");
}
if (!parsedSchema || typeof parsedSchema !== "object" || Array.isArray(parsedSchema)) {
throw new Error("ask --schema must decode to a JSON schema object");
}
responseSchema = parsedSchema;
}
const responseValidator = compileAskValidator(responseSchema);
let responseSchema = defaultSchema;
if (schemaRaw) {
let parsedSchema;
try {
parsedSchema = JSON.parse(schemaRaw);
} catch {
throw new Error("ask --schema must be valid JSON");
}
if (!parsedSchema || typeof parsedSchema !== "object" || Array.isArray(parsedSchema)) {
throw new Error("ask --schema must decode to a JSON schema object");
}
responseSchema = parsedSchema;
}
const responseValidator = compileAskValidator(responseSchema);
const forceEmit = Boolean(args.emit);
const emit = forceEmit || ctx.mode === "tool" || !isInteractive(ctx.stdin);
const canRequestInput =
!forceEmit && ctx.mode === "tool" && typeof ctx.requestInput === "function";
const restoredState = canRequestInput ? ctx.requestInput.getSuspendedState?.() : undefined;
const restoredAskState =
restoredState && typeof restoredState === "object" && restoredState.type === "ask"
? restoredState
: null;
const forceEmit = Boolean(args.emit);
const emit = forceEmit || ctx.mode === "tool" || !isInteractive(ctx.stdin);
const canRequestInput =
!forceEmit && ctx.mode === "tool" && typeof ctx.requestInput === "function";
const restoredState = canRequestInput ? ctx.requestInput.getSuspendedState?.() : undefined;
const restoredAskState =
restoredState && typeof restoredState === "object" && restoredState.type === "ask"
? restoredState
: null;
const items = [];
if (!restoredAskState) {
for await (const item of input) items.push(item);
}
const items = [];
if (!restoredAskState) {
for await (const item of input) items.push(item);
}
let subject = restoredAskState?.subject;
if (subjectFromStdin && items.length > 0) {
const preview = items
.map((item) => (typeof item === "string" ? item : JSON.stringify(item)))
.join("\n")
.slice(0, 2000);
subject = { text: preview };
}
let subject = restoredAskState?.subject;
if (subjectFromStdin && items.length > 0) {
const preview = items
.map((item) => (typeof item === "string" ? item : JSON.stringify(item)))
.join("\n")
.slice(0, 2000);
subject = { text: preview };
}
if (emit) {
if (canRequestInput) {
const response = await ctx.requestInput({
prompt,
responseSchema,
...(subject ? { subject } : null),
suspendedState: { type: "ask", ...(subject ? { subject } : null) },
});
return {
output: asStream([response]),
};
}
return {
halt: true,
output: (async function* () {
yield {
type: "input_request",
prompt,
responseSchema,
...(subject ? { subject } : null),
items,
};
})(),
};
}
if (emit) {
if (canRequestInput) {
const response = await ctx.requestInput({
prompt,
responseSchema,
...(subject ? { subject } : null),
suspendedState: { type: "ask", ...(subject ? { subject } : null) },
});
return {
output: asStream([response]),
};
}
return {
halt: true,
output: (async function* () {
yield {
type: "input_request",
prompt,
responseSchema,
...(subject ? { subject } : null),
items,
};
})(),
};
}
ctx.stdout.write(`${prompt}\n> `);
const { readLineFromStream } = await import("../../read_line.js");
const raw = await readLineFromStream(ctx.stdin, { timeoutMs: 0 });
const text = String(raw ?? "").trim();
ctx.stdout.write(`${prompt}\n> `);
const { readLineFromStream } = await import("../../read_line.js");
const raw = await readLineFromStream(ctx.stdin, { timeoutMs: 0, signal: ctx.signal });
const text = String(raw ?? "").trim();
let lastError;
for (const candidate of parseInteractiveCandidates(text)) {
try {
validateAskResponse(responseValidator, candidate);
return {
output: (async function* () {
yield candidate;
})(),
};
} catch (err) {
lastError = err;
}
}
throw lastError ?? new Error("ask response failed schema validation");
},
let lastError;
for (const candidate of parseInteractiveCandidates(text)) {
try {
validateAskResponse(responseValidator, candidate);
return {
output: (async function* () {
yield candidate;
})(),
};
} catch (err) {
lastError = err;
}
}
throw lastError ?? new Error("ask response failed schema validation");
},
};
async function* asStream(items) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/commands/stdlib/dedupe.ts
+49 -49
View File
@@ -1,55 +1,55 @@
function getByPath(obj: any, path: string): any {
if (!path) return obj;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
if (!path) return obj;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
}
export const dedupeCommand = {
name: "dedupe",
meta: {
description: "Remove duplicate items, keeping first occurrence (stable)",
argsSchema: {
type: "object",
properties: {
key: {
type: "string",
description: "Dot-path key used for identity (defaults to whole item)",
},
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`dedupe — remove duplicate items (stable)\n\n` +
`Usage:\n` +
` ... | dedupe\n` +
` ... | dedupe --key id\n\n` +
`Notes:\n` +
` - Keeps the first occurrence.\n`
);
},
async run({ input, args }: any) {
const key = typeof args.key === "string" ? args.key : undefined;
const seen = new Set<string>();
name: "dedupe",
meta: {
description: "Remove duplicate items, keeping first occurrence (stable)",
argsSchema: {
type: "object",
properties: {
key: {
type: "string",
description: "Dot-path key used for identity (defaults to whole item)",
},
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`dedupe — remove duplicate items (stable)\n\n` +
`Usage:\n` +
` ... | dedupe\n` +
` ... | dedupe --key id\n\n` +
`Notes:\n` +
` - Keeps the first occurrence.\n`
);
},
async run({ input, args }: any) {
const key = typeof args.key === "string" ? args.key : undefined;
const seen = new Set<string>();
return {
output: (async function* () {
for await (const item of input) {
const id = key ? getByPath(item, key) : item;
const k = JSON.stringify(id);
if (seen.has(k)) continue;
seen.add(k);
yield item;
}
})(),
};
},
return {
output: (async function* () {
for await (const item of input) {
const id = key ? getByPath(item, key) : item;
const k = JSON.stringify(id);
if (seen.has(k)) continue;
seen.add(k);
yield item;
}
})(),
};
},
};
src/commands/stdlib/diff_last.ts
+34 -29
View File
@@ -1,36 +1,41 @@
import { diffAndStore } from "../../state/store.js";
export const diffLastCommand = {
name: "diff.last",
meta: {
description: "Compare current items to last stored snapshot",
argsSchema: {
type: "object",
properties: {
key: { type: "string", description: "State key to diff against" },
_: { type: "array", items: { type: "string" } },
},
required: ["key"],
},
sideEffects: ["writes_state"],
},
help() {
return `diff.last — compare current items to last stored snapshot\n\nUsage:\n <items> | diff.last --key <stateKey>\n\nOutput:\n { changed, key, before, after }\n`;
},
async run({ input, args, ctx }) {
const key = args.key ?? args._[0];
if (!key) throw new Error("diff.last requires --key");
name: "diff.last",
meta: {
description: "Compare current items to last stored snapshot",
argsSchema: {
type: "object",
properties: {
key: { type: "string", description: "State key to diff against" },
_: { type: "array", items: { type: "string" } },
},
required: ["key"],
},
sideEffects: ["writes_state"],
},
help() {
return `diff.last — compare current items to last stored snapshot\n\nUsage:\n <items> | diff.last --key <stateKey>\n\nOutput:\n { changed, key, before, after }\n`;
},
async run({ input, args, ctx }) {
const key = args.key ?? args._[0];
if (!key) throw new Error("diff.last requires --key");
const afterItems = [];
for await (const item of input) afterItems.push(item);
const afterItems = [];
for await (const item of input) afterItems.push(item);
const after = afterItems.length === 1 ? afterItems[0] : afterItems;
const { before, changed } = await diffAndStore({ env: ctx.env, key, value: after });
const after = afterItems.length === 1 ? afterItems[0] : afterItems;
const { before, changed } = await diffAndStore({
env: ctx.env,
key,
value: after,
signal: ctx.signal,
});
return {
output: (async function* () {
yield { kind: "diff.last", key, changed, before, after };
})(),
};
},
return {
output: (async function* () {
yield { kind: "diff.last", key, changed, before, after };
})(),
};
},
};
src/commands/stdlib/email_triage.ts
+289 -289
View File
@@ -1,352 +1,352 @@
type EmailLike = {
id?: string;
threadId?: string;
from?: string;
subject?: string;
date?: string;
snippet?: string;
labels?: string[];
id?: string;
threadId?: string;
from?: string;
subject?: string;
date?: string;
snippet?: string;
labels?: string[];
};
type NormalizedEmail = Required<
Pick<EmailLike, "id" | "threadId" | "from" | "subject" | "date" | "snippet">
Pick<EmailLike, "id" | "threadId" | "from" | "subject" | "date" | "snippet">
> & {
labels: string[];
labels: string[];
};
function normalizeEmail(raw: any): NormalizedEmail {
const id = String(raw?.id ?? raw?.messageId ?? "").trim();
const threadId = String(raw?.threadId ?? raw?.thread_id ?? id).trim();
const from = String(raw?.from ?? raw?.sender ?? "").trim();
const subject = String(raw?.subject ?? "").trim();
const date = String(raw?.date ?? raw?.internalDate ?? raw?.timestamp ?? "").trim();
const snippet = String(raw?.snippet ?? raw?.bodyPreview ?? "").trim();
const labels = Array.isArray(raw?.labels) ? raw.labels.map((x: any) => String(x)) : [];
const id = String(raw?.id ?? raw?.messageId ?? "").trim();
const threadId = String(raw?.threadId ?? raw?.thread_id ?? id).trim();
const from = String(raw?.from ?? raw?.sender ?? "").trim();
const subject = String(raw?.subject ?? "").trim();
const date = String(raw?.date ?? raw?.internalDate ?? raw?.timestamp ?? "").trim();
const snippet = String(raw?.snippet ?? raw?.bodyPreview ?? "").trim();
const labels = Array.isArray(raw?.labels) ? raw.labels.map((x: any) => String(x)) : [];
return {
id,
threadId: threadId || id,
from,
subject,
date,
snippet,
labels,
};
return {
id,
threadId: threadId || id,
from,
subject,
date,
snippet,
labels,
};
}
function isLikelyNoReply(from: string) {
const f = from.toLowerCase();
return (
f.includes("no-reply") ||
f.includes("noreply") ||
f.includes("do-not-reply") ||
f.includes("donotreply")
);
const f = from.toLowerCase();
return (
f.includes("no-reply") ||
f.includes("noreply") ||
f.includes("do-not-reply") ||
f.includes("donotreply")
);
}
function extractEmailAddress(from: string): string {
const m = String(from).match(/<([^>]+)>/);
if (m?.[1]) return m[1].trim();
// fallback: find first email-ish token
const m2 = String(from).match(/[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}/i);
return (m2?.[0] ?? "").trim();
const m = String(from).match(/<([^>]+)>/);
if (m?.[1]) return m[1].trim();
// fallback: find first email-ish token
const m2 = String(from).match(/[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}/i);
return (m2?.[0] ?? "").trim();
}
function ensureRe(subject: string) {
const s = String(subject ?? "").trim();
if (!s) return "Re:";
return /^re:\s*/i.test(s) ? s : `Re: ${s}`;
const s = String(subject ?? "").trim();
if (!s) return "Re:";
return /^re:\s*/i.test(s) ? s : `Re: ${s}`;
}
type TriageCategory = "needs_reply" | "needs_action" | "fyi";
type TriageDecision = {
id: string;
category: TriageCategory;
rationale?: string;
reply?: {
subject?: string;
body: string;
};
id: string;
category: TriageCategory;
rationale?: string;
reply?: {
subject?: string;
body: string;
};
};
type EmailTriageReport = {
summary: string;
buckets: {
needsReply: string[];
needsAction: string[];
fyi: string[];
};
emails: NormalizedEmail[];
decisions?: TriageDecision[];
drafts?: { to: string; subject: string; body: string; emailId: string }[];
mode: "deterministic" | "llm";
summary: string;
buckets: {
needsReply: string[];
needsAction: string[];
fyi: string[];
};
emails: NormalizedEmail[];
decisions?: TriageDecision[];
drafts?: { to: string; subject: string; body: string; emailId: string }[];
mode: "deterministic" | "llm";
};
function buildDeterministicReport(emails: NormalizedEmail[]): EmailTriageReport {
const buckets = {
needsReply: [] as NormalizedEmail[],
needsAction: [] as NormalizedEmail[],
fyi: [] as NormalizedEmail[],
};
const buckets = {
needsReply: [] as NormalizedEmail[],
needsAction: [] as NormalizedEmail[],
fyi: [] as NormalizedEmail[],
};
for (const e of emails) {
const subjLower = e.subject.toLowerCase();
const unread = e.labels.some((l) => l.toUpperCase() === "UNREAD");
for (const e of emails) {
const subjLower = e.subject.toLowerCase();
const unread = e.labels.some((l) => l.toUpperCase() === "UNREAD");
if (subjLower.includes("action required") || subjLower.includes("urgent")) {
buckets.needsAction.push(e);
continue;
}
if (subjLower.includes("action required") || subjLower.includes("urgent")) {
buckets.needsAction.push(e);
continue;
}
if (unread && !isLikelyNoReply(e.from)) {
buckets.needsReply.push(e);
continue;
}
if (unread && !isLikelyNoReply(e.from)) {
buckets.needsReply.push(e);
continue;
}
buckets.fyi.push(e);
}
buckets.fyi.push(e);
}
const summary = `${buckets.needsReply.length} need replies, ${buckets.needsAction.length} need action, ${buckets.fyi.length} FYI`;
const summary = `${buckets.needsReply.length} need replies, ${buckets.needsAction.length} need action, ${buckets.fyi.length} FYI`;
return {
summary,
buckets: {
needsReply: buckets.needsReply.map((x) => x.id),
needsAction: buckets.needsAction.map((x) => x.id),
fyi: buckets.fyi.map((x) => x.id),
},
emails,
mode: "deterministic",
};
return {
summary,
buckets: {
needsReply: buckets.needsReply.map((x) => x.id),
needsAction: buckets.needsAction.map((x) => x.id),
fyi: buckets.fyi.map((x) => x.id),
},
emails,
mode: "deterministic",
};
}
function triagePrompt(emails: NormalizedEmail[]) {
return (
`You are an email triage assistant.\n` +
`Given the following emails, return JSON that categorizes each email and (when category is needs_reply) drafts a short reply.\n` +
`Guidelines:\n` +
`- Keep replies concise, friendly, and professional.\n` +
`- If sender appears to be automated (no-reply), do not draft a reply; categorize as fyi unless it is clearly urgent/actionable.\n` +
`- Use one of categories: needs_reply, needs_action, fyi.\n` +
`- The reply body should be plain text, no markdown.\n\n` +
`Emails (JSON):\n` +
JSON.stringify(
emails.map((e) => ({
id: e.id,
from: e.from,
subject: e.subject,
date: e.date,
snippet: e.snippet,
labels: e.labels,
})),
null,
2,
)
);
return (
`You are an email triage assistant.\n` +
`Given the following emails, return JSON that categorizes each email and (when category is needs_reply) drafts a short reply.\n` +
`Guidelines:\n` +
`- Keep replies concise, friendly, and professional.\n` +
`- If sender appears to be automated (no-reply), do not draft a reply; categorize as fyi unless it is clearly urgent/actionable.\n` +
`- Use one of categories: needs_reply, needs_action, fyi.\n` +
`- The reply body should be plain text, no markdown.\n\n` +
`Emails (JSON):\n` +
JSON.stringify(
emails.map((e) => ({
id: e.id,
from: e.from,
subject: e.subject,
date: e.date,
snippet: e.snippet,
labels: e.labels,
})),
null,
2,
)
);
}
const TRIAGE_OUTPUT_SCHEMA = {
type: "object",
properties: {
decisions: {
type: "array",
items: {
type: "object",
properties: {
id: { type: "string" },
category: { type: "string", enum: ["needs_reply", "needs_action", "fyi"] },
rationale: { type: "string" },
reply: {
type: "object",
properties: {
subject: { type: "string" },
body: { type: "string" },
},
required: ["body"],
additionalProperties: false,
},
},
required: ["id", "category"],
additionalProperties: false,
},
},
},
required: ["decisions"],
additionalProperties: false,
type: "object",
properties: {
decisions: {
type: "array",
items: {
type: "object",
properties: {
id: { type: "string" },
category: { type: "string", enum: ["needs_reply", "needs_action", "fyi"] },
rationale: { type: "string" },
reply: {
type: "object",
properties: {
subject: { type: "string" },
body: { type: "string" },
},
required: ["body"],
additionalProperties: false,
},
},
required: ["id", "category"],
additionalProperties: false,
},
},
},
required: ["decisions"],
additionalProperties: false,
};
export const emailTriageCommand = {
name: "email.triage",
meta: {
description: "Email triage (deterministic by default, optionally LLM-assisted via llm.invoke)",
argsSchema: {
type: "object",
properties: {
limit: {
type: "number",
description: "Maximum items to consume from input stream",
default: 20,
},
llm: { type: "boolean", description: "Use llm.invoke for categorization + draft replies" },
model: {
type: "string",
description: "Model for llm.invoke (optional; adapter defaults may apply)",
},
url: {
type: "string",
description: "Reserved for compatibility (ignored in OpenClaw mode)",
},
token: { type: "string", description: "Bearer token (or OPENCLAW_TOKEN/CLAWD_TOKEN)" },
temperature: { type: "number", description: "LLM temperature" },
"max-output-tokens": { type: "number", description: "Max completion tokens" },
emit: {
type: "string",
description: "Output mode: 'report' (default) or 'drafts'",
default: "report",
},
"state-key": { type: "string", description: "Run-state key forwarded to llm.invoke" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`email.triage — categorize emails and draft replies (optional LLM)\n\n` +
`Usage (deterministic):\n` +
` gog.gmail.search --query 'newer_than:1d' --max 20 | email.triage\n\n` +
`Usage (LLM-assisted drafts):\n` +
` gog.gmail.search --query 'newer_than:1d' --max 20 | email.triage --llm --model <model>\n\n` +
`Send drafts (requires approval):\n` +
` ... | email.triage --llm --model <model> --emit drafts | approve --prompt 'Send replies?' | gog.gmail.send\n\n` +
`Notes:\n` +
` - Read-only by default: does not send anything.\n` +
` - LLM mode uses llm.invoke (and its cache/resume semantics).\n`
);
},
async run({ input, args, ctx }) {
const limit = Number(args.limit ?? 20);
const emit = String(args.emit ?? "report").trim() || "report";
name: "email.triage",
meta: {
description: "Email triage (deterministic by default, optionally LLM-assisted via llm.invoke)",
argsSchema: {
type: "object",
properties: {
limit: {
type: "number",
description: "Maximum items to consume from input stream",
default: 20,
},
llm: { type: "boolean", description: "Use llm.invoke for categorization + draft replies" },
model: {
type: "string",
description: "Model for llm.invoke (optional; adapter defaults may apply)",
},
url: {
type: "string",
description: "Reserved for compatibility (ignored in OpenClaw mode)",
},
token: { type: "string", description: "Bearer token (or OPENCLAW_TOKEN/CLAWD_TOKEN)" },
temperature: { type: "number", description: "LLM temperature" },
"max-output-tokens": { type: "number", description: "Max completion tokens" },
emit: {
type: "string",
description: "Output mode: 'report' (default) or 'drafts'",
default: "report",
},
"state-key": { type: "string", description: "Run-state key forwarded to llm.invoke" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`email.triage — categorize emails and draft replies (optional LLM)\n\n` +
`Usage (deterministic):\n` +
` gog.gmail.search --query 'newer_than:1d' --max 20 | email.triage\n\n` +
`Usage (LLM-assisted drafts):\n` +
` gog.gmail.search --query 'newer_than:1d' --max 20 | email.triage --llm --model <model>\n\n` +
`Send drafts (requires approval):\n` +
` ... | email.triage --llm --model <model> --emit drafts | approve --prompt 'Send replies?' | gog.gmail.send\n\n` +
`Notes:\n` +
` - Read-only by default: does not send anything.\n` +
` - LLM mode uses llm.invoke (and its cache/resume semantics).\n`
);
},
async run({ input, args, ctx }) {
const limit = Number(args.limit ?? 20);
const emit = String(args.emit ?? "report").trim() || "report";
const emails: NormalizedEmail[] = [];
for await (const item of input) {
emails.push(normalizeEmail(item));
if (emails.length >= limit) break;
}
const emails: NormalizedEmail[] = [];
for await (const item of input) {
emails.push(normalizeEmail(item));
if (emails.length >= limit) break;
}
const wantLlm = Boolean(args.llm ?? false);
const env = ctx?.env ?? process.env;
const hasLlmProvider = Boolean(
String(env.LOBSTER_LLM_PROVIDER ?? "").trim() ||
String(env.LOBSTER_PI_LLM_ADAPTER_URL ?? "").trim() ||
String(env.LOBSTER_LLM_ADAPTER_URL ?? "").trim() ||
String(env.OPENCLAW_URL ?? env.CLAWD_URL ?? "").trim(),
);
const wantLlm = Boolean(args.llm ?? false);
const env = ctx?.env ?? process.env;
const hasLlmProvider = Boolean(
String(env.LOBSTER_LLM_PROVIDER ?? "").trim() ||
String(env.LOBSTER_PI_LLM_ADAPTER_URL ?? "").trim() ||
String(env.LOBSTER_LLM_ADAPTER_URL ?? "").trim() ||
String(env.OPENCLAW_URL ?? env.CLAWD_URL ?? "").trim(),
);
if (!wantLlm || !hasLlmProvider) {
const report = buildDeterministicReport(emails);
if (emit === "drafts") {
return { output: streamOf([]) };
}
return { output: streamOf([report]) };
}
if (!wantLlm || !hasLlmProvider) {
const report = buildDeterministicReport(emails);
if (emit === "drafts") {
return { output: streamOf([]) };
}
return { output: streamOf([report]) };
}
const model = String(args.model ?? "").trim();
const model = String(args.model ?? "").trim();
if (!ctx?.registry) throw new Error("email.triage (LLM mode) requires ctx.registry");
const llmCmd = ctx.registry.get("llm.invoke") ?? ctx.registry.get("llm_task.invoke");
if (!llmCmd) throw new Error("email.triage requires llm.invoke to be registered");
if (!ctx?.registry) throw new Error("email.triage (LLM mode) requires ctx.registry");
const llmCmd = ctx.registry.get("llm.invoke") ?? ctx.registry.get("llm_task.invoke");
if (!llmCmd) throw new Error("email.triage requires llm.invoke to be registered");
const llmRes = await llmCmd.run({
input: streamOf(emails),
args: {
_: [],
url: args.url,
token: args.token,
...(model ? { model } : null),
prompt: triagePrompt(emails),
"output-schema": JSON.stringify(TRIAGE_OUTPUT_SCHEMA),
"schema-version": "email_triage.v1",
temperature: args.temperature,
"max-output-tokens": args["max-output-tokens"],
"state-key": args["state-key"] ?? env.LOBSTER_RUN_STATE_KEY,
},
ctx,
} as any);
const llmRes = await llmCmd.run({
input: streamOf(emails),
args: {
_: [],
url: args.url,
token: args.token,
...(model ? { model } : null),
prompt: triagePrompt(emails),
"output-schema": JSON.stringify(TRIAGE_OUTPUT_SCHEMA),
"schema-version": "email_triage.v1",
temperature: args.temperature,
"max-output-tokens": args["max-output-tokens"],
"state-key": args["state-key"] ?? env.LOBSTER_RUN_STATE_KEY,
},
ctx,
} as any);
const llmItems: any[] = [];
for await (const it of llmRes.output) llmItems.push(it);
const first = llmItems[0];
const data = first?.output?.data;
const decisionsRaw = Array.isArray(data?.decisions) ? data.decisions : [];
const decisions: TriageDecision[] = decisionsRaw
.map((d: any) => ({
id: String(d?.id ?? "").trim(),
category: String(d?.category ?? "fyi") as TriageCategory,
rationale: d?.rationale ? String(d.rationale) : undefined,
reply:
d?.reply && typeof d.reply === "object"
? { subject: d.reply.subject, body: String(d.reply.body ?? "") }
: undefined,
}))
.filter((d: any) => d.id);
const llmItems: any[] = [];
for await (const it of llmRes.output) llmItems.push(it);
const first = llmItems[0];
const data = first?.output?.data;
const decisionsRaw = Array.isArray(data?.decisions) ? data.decisions : [];
const decisions: TriageDecision[] = decisionsRaw
.map((d: any) => ({
id: String(d?.id ?? "").trim(),
category: String(d?.category ?? "fyi") as TriageCategory,
rationale: d?.rationale ? String(d.rationale) : undefined,
reply:
d?.reply && typeof d.reply === "object"
? { subject: d.reply.subject, body: String(d.reply.body ?? "") }
: undefined,
}))
.filter((d: any) => d.id);
const byId = new Map(emails.map((e) => [e.id, e] as const));
const buckets = {
needsReply: [] as string[],
needsAction: [] as string[],
fyi: [] as string[],
};
const byId = new Map(emails.map((e) => [e.id, e] as const));
const buckets = {
needsReply: [] as string[],
needsAction: [] as string[],
fyi: [] as string[],
};
const drafts: { to: string; subject: string; body: string; emailId: string }[] = [];
const drafts: { to: string; subject: string; body: string; emailId: string }[] = [];
for (const d of decisions) {
if (d.category === "needs_reply") buckets.needsReply.push(d.id);
else if (d.category === "needs_action") buckets.needsAction.push(d.id);
else buckets.fyi.push(d.id);
for (const d of decisions) {
if (d.category === "needs_reply") buckets.needsReply.push(d.id);
else if (d.category === "needs_action") buckets.needsAction.push(d.id);
else buckets.fyi.push(d.id);
if (d.category === "needs_reply" && d.reply?.body) {
const email = byId.get(d.id);
const to = email ? extractEmailAddress(email.from) : "";
if (to && !isLikelyNoReply(email?.from ?? "")) {
drafts.push({
emailId: d.id,
to,
subject: d.reply.subject ? String(d.reply.subject) : ensureRe(email?.subject ?? ""),
body: String(d.reply.body),
});
}
}
}
if (d.category === "needs_reply" && d.reply?.body) {
const email = byId.get(d.id);
const to = email ? extractEmailAddress(email.from) : "";
if (to && !isLikelyNoReply(email?.from ?? "")) {
drafts.push({
emailId: d.id,
to,
subject: d.reply.subject ? String(d.reply.subject) : ensureRe(email?.subject ?? ""),
body: String(d.reply.body),
});
}
}
}
const summary = `${buckets.needsReply.length} need replies, ${buckets.needsAction.length} need action, ${buckets.fyi.length} FYI`;
const summary = `${buckets.needsReply.length} need replies, ${buckets.needsAction.length} need action, ${buckets.fyi.length} FYI`;
if (emit === "drafts") {
return {
output: (async function* () {
for (const d of drafts) {
// gog.gmail.send expects: {to, subject, body}
yield { to: d.to, subject: d.subject, body: d.body, emailId: d.emailId };
}
})(),
};
}
if (emit === "drafts") {
return {
output: (async function* () {
for (const d of drafts) {
// gog.gmail.send expects: {to, subject, body}
yield { to: d.to, subject: d.subject, body: d.body, emailId: d.emailId };
}
})(),
};
}
const report: EmailTriageReport = {
summary,
buckets,
emails,
decisions,
drafts,
mode: "llm",
};
const report: EmailTriageReport = {
summary,
buckets,
emails,
decisions,
drafts,
mode: "llm",
};
return { output: streamOf([report]) };
},
return { output: streamOf([report]) };
},
};
async function* streamOf(items: any[]) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/commands/stdlib/exec.ts
+103 -122
View File
@@ -1,143 +1,124 @@
import { spawn } from "node:child_process";
import { runAbortableProcess } from "../../abortable_process.js";
import { resolveInlineShellCommand } from "../../shell.js";
export const execCommand = {
name: "exec",
meta: {
description: "Run an OS command",
argsSchema: {
type: "object",
properties: {
json: { type: "boolean", description: "Parse stdout as JSON (single value)." },
shell: { type: "string", description: "Run via the system shell with this command line." },
_: { type: "array", items: { type: "string" }, description: "Command + args." },
},
required: ["_"],
},
sideEffects: ["local_exec"],
},
help() {
return (
`exec — run an OS command\n\n` +
`Usage:\n` +
` exec <command...>\n` +
` exec --stdin raw|json|jsonl <command...>\n` +
` exec --json <command...>\n` +
` exec --shell "<command line>"\n\n` +
`Notes:\n` +
` - With --json, parses stdout as JSON (single value).\n` +
` - With --stdin, writes pipeline input to stdin.\n` +
` - With --shell (or a single arg containing spaces), runs via the system shell.\n`
);
},
async run({ input, args, ctx }) {
const cmd = args._;
const cwd = ctx?.cwd ?? process.cwd();
name: "exec",
meta: {
description: "Run an OS command",
argsSchema: {
type: "object",
properties: {
json: { type: "boolean", description: "Parse stdout as JSON (single value)." },
shell: { type: "string", description: "Run via the system shell with this command line." },
_: { type: "array", items: { type: "string" }, description: "Command + args." },
},
required: ["_"],
},
sideEffects: ["local_exec"],
},
help() {
return (
`exec — run an OS command\n\n` +
`Usage:\n` +
` exec <command...>\n` +
` exec --stdin raw|json|jsonl <command...>\n` +
` exec --json <command...>\n` +
` exec --shell "<command line>"\n\n` +
`Notes:\n` +
` - With --json, parses stdout as JSON (single value).\n` +
` - With --stdin, writes pipeline input to stdin.\n` +
` - With --shell (or a single arg containing spaces), runs via the system shell.\n`
);
},
async run({ input, args, ctx }) {
const cmd = args._;
const cwd = ctx?.cwd ?? process.cwd();
const shellLine = typeof args.shell === "string" ? args.shell : null;
const useShell = Boolean(args.shell) || (cmd.length === 1 && /\s/.test(cmd[0]));
const stdinMode = typeof args.stdin === "string" ? String(args.stdin).toLowerCase() : null;
const shellLine = typeof args.shell === "string" ? args.shell : null;
const useShell = Boolean(args.shell) || (cmd.length === 1 && /\s/.test(cmd[0]));
const stdinMode = typeof args.stdin === "string" ? String(args.stdin).toLowerCase() : null;
if (!cmd.length && !shellLine) throw new Error("exec requires a command");
if (!cmd.length && !shellLine) throw new Error("exec requires a command");
let stdinPayload = null;
if (stdinMode) {
const items = [];
for await (const item of input) items.push(item);
stdinPayload = encodeStdin(items, stdinMode);
} else {
// Drain input to avoid dangling streams.
for await (const _item of input) {
// no-op
}
}
let stdinPayload = null;
if (stdinMode) {
const items = [];
for await (const item of input) items.push(item);
stdinPayload = encodeStdin(items, stdinMode);
} else {
// Drain input to avoid dangling streams.
for await (const _item of input) {
// no-op
}
}
const result = useShell
? await runShellLine(shellLine ?? cmd[0] ?? "", {
env: ctx.env,
cwd,
stdin: stdinPayload,
signal: ctx.signal,
})
: await runProcess(cmd[0], cmd.slice(1), {
env: ctx.env,
cwd,
stdin: stdinPayload,
signal: ctx.signal,
});
const result = useShell
? await runShellLine(shellLine ?? cmd[0] ?? "", {
env: ctx.env,
cwd,
stdin: stdinPayload,
signal: ctx.signal,
forceTerminationSignal: ctx.forceTerminationSignal,
})
: await runProcess(cmd[0], cmd.slice(1), {
env: ctx.env,
cwd,
stdin: stdinPayload,
signal: ctx.signal,
forceTerminationSignal: ctx.forceTerminationSignal,
});
if (args.json) {
let parsed;
try {
parsed = JSON.parse(result.stdout.trim() || "null");
} catch (err) {
throw new Error(
`exec --json could not parse stdout as JSON: ${err?.message ?? String(err)}`,
);
}
if (args.json) {
let parsed;
try {
parsed = JSON.parse(result.stdout.trim() || "null");
} catch (err) {
throw new Error(
`exec --json could not parse stdout as JSON: ${err?.message ?? String(err)}`,
);
}
return {
output: asStream(Array.isArray(parsed) ? parsed : [parsed]),
};
}
return {
output: asStream(Array.isArray(parsed) ? parsed : [parsed]),
};
}
const lines = result.stdout.split(/\r?\n/).filter(Boolean);
return { output: asStream(lines) };
},
const lines = result.stdout.split(/\r?\n/).filter(Boolean);
return { output: asStream(lines) };
},
};
function runProcess(command, argv, { env, cwd, stdin, signal }) {
return new Promise<any>((resolve, reject) => {
const child = spawn(command, argv, {
env,
cwd,
signal,
stdio: ["pipe", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
if (typeof stdin === "string") {
child.stdin.setDefaultEncoding("utf8");
child.stdin.write(stdin);
}
child.stdin.end();
child.on("error", reject);
child.on("close", (code) => {
if (code === 0) return resolve({ stdout, stderr });
reject(new Error(`exec failed (${code}): ${stderr.trim() || stdout.trim() || command}`));
});
});
async function runProcess(command, argv, { env, cwd, stdin, signal, forceTerminationSignal }) {
const { stdout, stderr, code } = await runAbortableProcess({
command,
argv,
env,
cwd,
stdin,
signal,
forceTerminationSignal,
notFoundMessage: `exec command not found: ${command}`,
});
if (code === 0) return { stdout, stderr };
throw new Error(`exec failed (${code}): ${stderr.trim() || stdout.trim() || command}`);
}
function runShellLine(commandLine, { env, cwd, stdin, signal }) {
const shell = resolveInlineShellCommand({ command: commandLine, env });
return runProcess(shell.command, shell.argv, { env, cwd, stdin, signal });
function runShellLine(commandLine, { env, cwd, stdin, signal, forceTerminationSignal }) {
const shell = resolveInlineShellCommand({ command: commandLine, env });
return runProcess(shell.command, shell.argv, { env, cwd, stdin, signal, forceTerminationSignal });
}
function encodeStdin(items, mode) {
if (mode === "json") return JSON.stringify(items);
if (mode === "jsonl") {
return items.map((item) => JSON.stringify(item)).join("\n") + (items.length ? "\n" : "");
}
if (mode === "raw") {
return items.map((item) => (typeof item === "string" ? item : JSON.stringify(item))).join("\n");
}
throw new Error(`exec --stdin must be raw, json, or jsonl (got ${mode})`);
if (mode === "json") return JSON.stringify(items);
if (mode === "jsonl") {
return items.map((item) => JSON.stringify(item)).join("\n") + (items.length ? "\n" : "");
}
if (mode === "raw") {
return items.map((item) => (typeof item === "string" ? item : JSON.stringify(item))).join("\n");
}
throw new Error(`exec --stdin must be raw, json, or jsonl (got ${mode})`);
}
async function* asStream(items) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/commands/stdlib/gog_gmail_search.ts
+78 -95
View File
@@ -1,106 +1,89 @@
import { spawn } from "node:child_process";
function run(cmd: string, argv: string[], env: Record<string, string | undefined>, cwd?: string) {
return new Promise<{ stdout: string; stderr: string; code: number | null }>((resolve, reject) => {
const child = spawn(cmd, argv, {
env: { ...process.env, ...env },
cwd,
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout?.on("data", (d) => (stdout += String(d)));
child.stderr?.on("data", (d) => (stderr += String(d)));
child.on("error", (err: any) => {
if (err?.code === "ENOENT") {
reject(new Error("gog not found on PATH (install: https://github.com/steipete/gogcli)"));
return;
}
reject(err);
});
child.on("close", (code) => {
resolve({ stdout, stderr, code });
});
});
}
import { runAbortableProcess } from "../../abortable_process.js";
export const gogGmailSearchCommand = {
name: "gog.gmail.search",
meta: {
description: "Fetch Gmail threads via gog (JSON)",
argsSchema: {
type: "object",
properties: {
query: { type: "string", description: "Gmail search query", default: "newer_than:1d" },
max: { type: "number", description: "Max results", default: 20 },
limit: { type: "number", description: "Alias for max" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: ["reads_email"],
},
help() {
return (
`gog.gmail.search — fetch Gmail messages via gog (JSON)\n\n` +
`Usage:\n` +
` gog.gmail.search --query 'newer_than:1d' --max 20\n\n` +
`Notes:\n` +
` - Requires the gog CLI: https://github.com/steipete/gogcli\n` +
` - Set GOG_BIN to override the executable used (default: gog).\n` +
` - This command outputs an array of message objects (as a stream).\n`
);
},
async run({ input, args, ctx }) {
// Drain input
for await (const _item of input) {
// no-op
}
name: "gog.gmail.search",
meta: {
description: "Fetch Gmail threads via gog (JSON)",
argsSchema: {
type: "object",
properties: {
query: { type: "string", description: "Gmail search query", default: "newer_than:1d" },
max: { type: "number", description: "Max results", default: 20 },
limit: { type: "number", description: "Alias for max" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: ["reads_email"],
},
help() {
return (
`gog.gmail.search — fetch Gmail messages via gog (JSON)\n\n` +
`Usage:\n` +
` gog.gmail.search --query 'newer_than:1d' --max 20\n\n` +
`Notes:\n` +
` - Requires the gog CLI: https://github.com/steipete/gogcli\n` +
` - Set GOG_BIN to override the executable used (default: gog).\n` +
` - This command outputs an array of message objects (as a stream).\n`
);
},
async run({ input, args, ctx }) {
ctx.signal?.throwIfAborted();
// Drain input
for await (const _item of input) {
// no-op
}
ctx.signal?.throwIfAborted();
const query = String(args.query ?? "newer_than:1d");
const max = Number(args.max ?? args.limit ?? 20);
const query = String(args.query ?? "newer_than:1d");
const max = Number(args.max ?? args.limit ?? 20);
const gogBinRaw = String(ctx.env.GOG_BIN ?? "gog");
const gogBinRaw = String(ctx.env.GOG_BIN ?? "gog");
// gog CLI (v0.9.x) expects the query as a positional argument:
// gog gmail search <query> ... --json
// Earlier draft versions used --query; keep Lobster's --query flag but translate it.
const argvBase = ["gmail", "search", query, "--json", "--max", String(max)];
// gog CLI (v0.9.x) expects the query as a positional argument:
// gog gmail search <query> ... --json
// Earlier draft versions used --query; keep Lobster's --query flag but translate it.
const argvBase = ["gmail", "search", query, "--json", "--max", String(max)];
// Test-friendly: allow pointing GOG_BIN at a node script.
const isScript = /\.(mjs|cjs|js|ts)$/i.test(gogBinRaw);
const gogBin = isScript ? process.execPath : gogBinRaw;
const argv = isScript ? [gogBinRaw, ...argvBase] : argvBase;
// Test-friendly: allow pointing GOG_BIN at a node script.
const isScript = /\.(mjs|cjs|js|ts)$/i.test(gogBinRaw);
const gogBin = isScript ? process.execPath : gogBinRaw;
const argv = isScript ? [gogBinRaw, ...argvBase] : argvBase;
const res = await run(gogBin, argv, ctx.env, process.cwd());
if (res.code !== 0) {
throw new Error(`gog.gmail.search failed (${res.code ?? "?"}): ${res.stderr.slice(0, 400)}`);
}
const res = await runAbortableProcess({
command: gogBin,
argv,
env: { ...process.env, ...ctx.env },
cwd: process.cwd(),
signal: ctx.signal,
forceTerminationSignal: ctx.forceTerminationSignal,
notFoundMessage: "gog not found on PATH (install: https://github.com/steipete/gogcli)",
});
if (res.code !== 0) {
throw new Error(`gog.gmail.search failed (${res.code ?? "?"}): ${res.stderr.slice(0, 400)}`);
}
let parsed: any;
try {
parsed = JSON.parse(res.stdout);
} catch (_err) {
throw new Error("gog.gmail.search expected JSON output");
}
let parsed: any;
try {
parsed = JSON.parse(res.stdout);
} catch (_err) {
throw new Error("gog.gmail.search expected JSON output");
}
// gog gmail search --json returns either:
// - an array of message/thread objects (older versions / some commands), or
// - an object like { nextPageToken, threads: [...] } (gog v0.9.x).
const items = Array.isArray(parsed)
? // Some gog versions return: [ { nextPageToken, threads: [...] } ]
(parsed as any[]).flatMap((x) => (Array.isArray(x?.threads) ? x.threads : [x]))
: Array.isArray((parsed as any)?.threads)
? (parsed as any).threads
: [parsed];
// gog gmail search --json returns either:
// - an array of message/thread objects (older versions / some commands), or
// - an object like { nextPageToken, threads: [...] } (gog v0.9.x).
const items = Array.isArray(parsed)
? // Some gog versions return: [ { nextPageToken, threads: [...] } ]
(parsed as any[]).flatMap((x) => (Array.isArray(x?.threads) ? x.threads : [x]))
: Array.isArray((parsed as any)?.threads)
? (parsed as any).threads
: [parsed];
return {
output: (async function* () {
for (const item of items) yield item;
})(),
};
},
return {
output: (async function* () {
for (const item of items) yield item;
})(),
};
},
};
src/commands/stdlib/gog_gmail_send.ts
+104 -98
View File
@@ -1,122 +1,128 @@
import { spawn } from "node:child_process";
function run(cmd: string, argv: string[], env: Record<string, string | undefined>, cwd?: string) {
return new Promise<{ stdout: string; stderr: string; code: number | null }>((resolve, reject) => {
const child = spawn(cmd, argv, {
env: { ...process.env, ...env },
cwd,
stdio: ["ignore", "pipe", "pipe"],
});
return new Promise<{ stdout: string; stderr: string; code: number | null }>((resolve, reject) => {
const child = spawn(cmd, argv, {
env: { ...process.env, ...env },
cwd,
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout?.on("data", (d) => (stdout += String(d)));
child.stderr?.on("data", (d) => (stderr += String(d)));
let stdout = "";
let stderr = "";
child.stdout?.on("data", (d) => (stdout += String(d)));
child.stderr?.on("data", (d) => (stderr += String(d)));
child.on("error", (err: any) => {
if (err?.code === "ENOENT") {
reject(new Error("gog not found on PATH (install: https://github.com/steipete/gogcli)"));
return;
}
reject(err);
});
child.on("error", (err: any) => {
if (err?.code === "ENOENT") {
reject(new Error("gog not found on PATH (install: https://github.com/steipete/gogcli)"));
return;
}
reject(err);
});
child.on("close", (code) => {
resolve({ stdout, stderr, code });
});
});
child.on("close", (code) => {
resolve({ stdout, stderr, code });
});
});
}
type Draft = {
to: string;
subject: string;
body: string;
to: string;
subject: string;
body: string;
};
function parseDraft(item: any): Draft {
if (!item || typeof item !== "object") {
throw new Error("gog.gmail.send expects draft objects");
}
const to = String((item as any).to ?? "").trim();
const subject = String((item as any).subject ?? "").trim();
const body = String((item as any).body ?? "").trim();
if (!to) throw new Error("gog.gmail.send draft missing to");
return { to, subject, body };
if (!item || typeof item !== "object") {
throw new Error("gog.gmail.send expects draft objects");
}
const to = String((item as any).to ?? "").trim();
const subject = String((item as any).subject ?? "").trim();
const body = String((item as any).body ?? "").trim();
if (!to) throw new Error("gog.gmail.send draft missing to");
return { to, subject, body };
}
export const gogGmailSendCommand = {
name: "gog.gmail.send",
meta: {
description: "Send Gmail messages via gog",
argsSchema: {
type: "object",
properties: {
dryRun: { type: "boolean", description: "If true, do not send; echo drafts" },
"dry-run": { type: "boolean", description: "Alias for dryRun" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: ["sends_email"],
},
help() {
return (
`gog.gmail.send — send Gmail messages via gog\n\n` +
`Usage:\n` +
` ... | approve --prompt 'Send replies?' | gog.gmail.send\n\n` +
`Input:\n` +
` Stream of draft objects: { to, subject, body }\n\n` +
`Notes:\n` +
` - Requires the gog CLI: https://github.com/steipete/gogcli\n` +
` - Set GOG_BIN to override the executable used (default: gog).\n`
);
},
async run({ input, args, ctx }) {
const dryRun = Boolean(args.dryRun ?? args["dry-run"] ?? false);
const gogBinRaw = String(ctx.env.GOG_BIN ?? "gog");
const isScript = /\.(mjs|cjs|js|ts)$/i.test(gogBinRaw);
const gogBin = isScript ? process.execPath : gogBinRaw;
name: "gog.gmail.send",
meta: {
description: "Send Gmail messages via gog",
argsSchema: {
type: "object",
properties: {
dryRun: { type: "boolean", description: "If true, do not send; echo drafts" },
"dry-run": { type: "boolean", description: "Alias for dryRun" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: ["sends_email"],
},
help() {
return (
`gog.gmail.send — send Gmail messages via gog\n\n` +
`Usage:\n` +
` ... | approve --prompt 'Send replies?' | gog.gmail.send\n\n` +
`Input:\n` +
` Stream of draft objects: { to, subject, body }\n\n` +
`Notes:\n` +
` - Requires the gog CLI: https://github.com/steipete/gogcli\n` +
` - Set GOG_BIN to override the executable used (default: gog).\n`
);
},
async run({ input, args, ctx }) {
ctx.signal?.throwIfAborted();
const dryRun = Boolean(args.dryRun ?? args["dry-run"] ?? false);
const gogBinRaw = String(ctx.env.GOG_BIN ?? "gog");
const isScript = /\.(mjs|cjs|js|ts)$/i.test(gogBinRaw);
const gogBin = isScript ? process.execPath : gogBinRaw;
const results: any[] = [];
const results: any[] = [];
for await (const item of input) {
const draft = parseDraft(item);
for await (const item of input) {
if (ctx.signal?.aborted) {
if (results.length > 0) break;
ctx.signal.throwIfAborted();
}
const draft = parseDraft(item);
if (dryRun) {
results.push({ ok: true, dryRun: true, ...draft });
continue;
}
if (dryRun) {
results.push({ ok: true, dryRun: true, ...draft });
continue;
}
const argvBase = [
"gmail",
"send",
"--to",
draft.to,
...(draft.subject ? ["--subject", draft.subject] : []),
...(draft.body ? ["--body", draft.body] : []),
"--json",
];
const argvBase = [
"gmail",
"send",
"--to",
draft.to,
...(draft.subject ? ["--subject", draft.subject] : []),
...(draft.body ? ["--body", draft.body] : []),
"--json",
];
const argv = isScript ? [gogBinRaw, ...argvBase] : argvBase;
const res = await run(gogBin, argv, ctx.env, process.cwd());
if (res.code !== 0) {
throw new Error(`gog.gmail.send failed (${res.code ?? "?"}): ${res.stderr.slice(0, 400)}`);
}
const argv = isScript ? [gogBinRaw, ...argvBase] : argvBase;
const res = await run(gogBin, argv, ctx.env, process.cwd());
if (res.code !== 0) {
throw new Error(`gog.gmail.send failed (${res.code ?? "?"}): ${res.stderr.slice(0, 400)}`);
}
let parsed: any;
try {
parsed = res.stdout ? JSON.parse(res.stdout) : { ok: true };
} catch (_err) {
parsed = { ok: true, raw: res.stdout };
}
let parsed: any;
try {
parsed = res.stdout ? JSON.parse(res.stdout) : { ok: true };
} catch (_err) {
parsed = { ok: true, raw: res.stdout };
}
results.push(parsed);
}
results.push(parsed);
if (ctx.signal?.aborted) break;
}
return {
output: (async function* () {
for (const r of results) yield r;
})(),
};
},
return {
output: (async function* () {
for (const r of results) yield r;
})(),
};
},
};
src/commands/stdlib/group_by.ts
+54 -54
View File
@@ -1,62 +1,62 @@
function getByPath(obj: any, path: string): any {
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
}
export const groupByCommand = {
name: "groupBy",
meta: {
description: "Group items by a key (stable group order)",
argsSchema: {
type: "object",
properties: {
key: { type: "string", description: "Dot-path key to group by (required)" },
_: { type: "array", items: { type: "string" } },
},
required: ["key"],
},
sideEffects: [],
},
help() {
return (
`groupBy — group items by a key\n\n` +
`Usage:\n` +
` ... | groupBy --key from\n\n` +
`Output:\n` +
` Stream of { key, items, count } objects\n\n` +
`Notes:\n` +
` - Group order is stable (order of first appearance).\n`
);
},
async run({ input, args }: any) {
const keyPath = String(args.key ?? "").trim();
if (!keyPath) throw new Error("groupBy requires --key");
name: "groupBy",
meta: {
description: "Group items by a key (stable group order)",
argsSchema: {
type: "object",
properties: {
key: { type: "string", description: "Dot-path key to group by (required)" },
_: { type: "array", items: { type: "string" } },
},
required: ["key"],
},
sideEffects: [],
},
help() {
return (
`groupBy — group items by a key\n\n` +
`Usage:\n` +
` ... | groupBy --key from\n\n` +
`Output:\n` +
` Stream of { key, items, count } objects\n\n` +
`Notes:\n` +
` - Group order is stable (order of first appearance).\n`
);
},
async run({ input, args }: any) {
const keyPath = String(args.key ?? "").trim();
if (!keyPath) throw new Error("groupBy requires --key");
const groups = new Map<string, { key: any; items: any[] }>();
const order: string[] = [];
const groups = new Map<string, { key: any; items: any[] }>();
const order: string[] = [];
for await (const item of input) {
const keyVal = getByPath(item, keyPath);
const k = JSON.stringify(keyVal);
if (!groups.has(k)) {
groups.set(k, { key: keyVal, items: [] });
order.push(k);
}
groups.get(k)!.items.push(item);
}
for await (const item of input) {
const keyVal = getByPath(item, keyPath);
const k = JSON.stringify(keyVal);
if (!groups.has(k)) {
groups.set(k, { key: keyVal, items: [] });
order.push(k);
}
groups.get(k)!.items.push(item);
}
return {
output: (async function* () {
for (const k of order) {
const g = groups.get(k)!;
yield { key: g.key, items: g.items, count: g.items.length };
}
})(),
};
},
return {
output: (async function* () {
for (const k of order) {
const g = groups.get(k)!;
yield { key: g.key, items: g.items, count: g.items.length };
}
})(),
};
},
};
src/commands/stdlib/head.ts
+29 -29
View File
@@ -1,32 +1,32 @@
export const headCommand = {
name: "head",
meta: {
description: "Take first N items",
argsSchema: {
type: "object",
properties: {
n: { type: "number", description: "Number of items to take", default: 10 },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return `head — take first N items\n\nUsage:\n head --n 10\n`;
},
async run({ input, args }) {
const n = args.n === undefined ? 10 : Number(args.n);
if (!Number.isFinite(n) || n < 0) throw new Error("head --n must be a non-negative number");
name: "head",
meta: {
description: "Take first N items",
argsSchema: {
type: "object",
properties: {
n: { type: "number", description: "Number of items to take", default: 10 },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return `head — take first N items\n\nUsage:\n head --n 10\n`;
},
async run({ input, args }) {
const n = args.n === undefined ? 10 : Number(args.n);
if (!Number.isFinite(n) || n < 0) throw new Error("head --n must be a non-negative number");
return {
output: (async function* () {
let i = 0;
for await (const item of input) {
if (i++ >= n) break;
yield item;
}
})(),
};
},
return {
output: (async function* () {
let i = 0;
for await (const item of input) {
if (i++ >= n) break;
yield item;
}
})(),
};
},
};
src/commands/stdlib/json.ts
+15 -15
View File
@@ -1,19 +1,19 @@
export const jsonCommand = {
name: "json",
meta: {
description: "Render pipeline output as JSON",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
},
help() {
return `json — render pipeline output as JSON\n\nUsage:\n ... | json\n`;
},
async run({ input, ctx }) {
const items = [];
for await (const item of input) items.push(item);
ctx.render.json(items);
return { output: emptyStream(), rendered: true };
},
name: "json",
meta: {
description: "Render pipeline output as JSON",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
},
help() {
return `json — render pipeline output as JSON\n\nUsage:\n ... | json\n`;
},
async run({ input, ctx }) {
const items = [];
for await (const item of input) items.push(item);
ctx.render.json(items);
return { output: emptyStream(), rendered: true };
},
};
async function* emptyStream() {}
src/commands/stdlib/llm_invoke.ts
+1390 -776
View File
File diff suppressed because it is too large Load Diff
src/commands/stdlib/map.ts
+88 -88
View File
@@ -1,105 +1,105 @@
function getByPath(obj: any, path: string): any {
if (path === "." || path === "this") return obj;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
if (path === "." || path === "this") return obj;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
}
function renderTemplate(tpl: string, ctx: any): string {
return tpl.replace(/\{\{\s*([^}]+?)\s*\}\}/g, (_m, expr) => {
const key = String(expr ?? "").trim();
const val = getByPath(ctx, key);
if (val === undefined || val === null) return "";
if (typeof val === "string") return val;
return JSON.stringify(val);
});
return tpl.replace(/\{\{\s*([^}]+?)\s*\}\}/g, (_m, expr) => {
const key = String(expr ?? "").trim();
const val = getByPath(ctx, key);
if (val === undefined || val === null) return "";
if (typeof val === "string") return val;
return JSON.stringify(val);
});
}
function parseAssignments(tokens: any[]): Array<{ key: string; value: string }> {
const out: Array<{ key: string; value: string }> = [];
for (const tok of tokens ?? []) {
const s = String(tok);
const idx = s.indexOf("=");
if (idx === -1) continue;
const key = s.slice(0, idx).trim();
const value = s.slice(idx + 1);
if (!key) continue;
out.push({ key, value });
}
return out;
const out: Array<{ key: string; value: string }> = [];
for (const tok of tokens ?? []) {
const s = String(tok);
const idx = s.indexOf("=");
if (idx === -1) continue;
const key = s.slice(0, idx).trim();
const value = s.slice(idx + 1);
if (!key) continue;
out.push({ key, value });
}
return out;
}
export const mapCommand = {
name: "map",
meta: {
description: "Transform items (wrap/unwrap/add fields)",
argsSchema: {
type: "object",
properties: {
wrap: { type: "string", description: "Wrap each item as {wrap: item}" },
unwrap: { type: "string", description: "Unwrap a field (yield item[unwrap])" },
_: {
type: "array",
items: { type: "string" },
description: "Optional assignments like key=value (value supports {{path}})",
},
},
required: [],
},
sideEffects: [],
},
help() {
return (
`map — transform items\n\n` +
`Usage:\n` +
` ... | map --wrap item\n` +
` ... | map --unwrap item\n` +
` ... | map foo=bar id={{id}}\n\n` +
`Notes:\n` +
` - Assignments are added to an object item (preserves existing fields).\n` +
` - Assignment values support template placeholders like {{id}} and {{nested.field}}.\n`
);
},
async run({ input, args }: any) {
const wrap = typeof args.wrap === "string" ? args.wrap : undefined;
const unwrap = typeof args.unwrap === "string" ? args.unwrap : undefined;
const assignments = parseAssignments(Array.isArray(args._) ? args._ : []);
name: "map",
meta: {
description: "Transform items (wrap/unwrap/add fields)",
argsSchema: {
type: "object",
properties: {
wrap: { type: "string", description: "Wrap each item as {wrap: item}" },
unwrap: { type: "string", description: "Unwrap a field (yield item[unwrap])" },
_: {
type: "array",
items: { type: "string" },
description: "Optional assignments like key=value (value supports {{path}})",
},
},
required: [],
},
sideEffects: [],
},
help() {
return (
`map — transform items\n\n` +
`Usage:\n` +
` ... | map --wrap item\n` +
` ... | map --unwrap item\n` +
` ... | map foo=bar id={{id}}\n\n` +
`Notes:\n` +
` - Assignments are added to an object item (preserves existing fields).\n` +
` - Assignment values support template placeholders like {{id}} and {{nested.field}}.\n`
);
},
async run({ input, args }: any) {
const wrap = typeof args.wrap === "string" ? args.wrap : undefined;
const unwrap = typeof args.unwrap === "string" ? args.unwrap : undefined;
const assignments = parseAssignments(Array.isArray(args._) ? args._ : []);
if (wrap && unwrap) throw new Error("map cannot use both --wrap and --unwrap");
if (wrap && unwrap) throw new Error("map cannot use both --wrap and --unwrap");
return {
output: (async function* () {
for await (const item of input) {
let cur: any = item;
return {
output: (async function* () {
for await (const item of input) {
let cur: any = item;
if (unwrap) {
if (cur && typeof cur === "object") cur = cur[unwrap];
else cur = undefined;
yield cur;
continue;
}
if (unwrap) {
if (cur && typeof cur === "object") cur = cur[unwrap];
else cur = undefined;
yield cur;
continue;
}
if (wrap) {
cur = { [wrap]: cur };
}
if (wrap) {
cur = { [wrap]: cur };
}
if (assignments.length > 0) {
if (cur === null || typeof cur !== "object" || Array.isArray(cur)) {
// If current is not an object, turn it into one so we can attach fields.
cur = { value: cur };
}
for (const { key, value } of assignments) {
cur[key] = renderTemplate(String(value), item);
}
}
if (assignments.length > 0) {
if (cur === null || typeof cur !== "object" || Array.isArray(cur)) {
// If current is not an object, turn it into one so we can attach fields.
cur = { value: cur };
}
for (const { key, value } of assignments) {
cur[key] = renderTemplate(String(value), item);
}
}
yield cur;
}
})(),
};
},
yield cur;
}
})(),
};
},
};
src/commands/stdlib/openclaw_agent.ts
+164
View File
@@ -0,0 +1,164 @@
import { runAbortableProcess } from "../../abortable_process.js";
import type { LobsterCommand } from "../types.js";
const OPENCLAW_AGENT_MAX_OUTPUT_BYTES = 10 * 1024 * 1024;
type AgentCliRunner = (params: {
executable: string;
argv: string[];
cwd: string;
env: NodeJS.ProcessEnv;
signal?: AbortSignal;
forceTerminationSignal?: AbortSignal;
}) => Promise<unknown>;
export const openclawAgentCommand = createOpenClawAgentCommand();
export function createOpenClawAgentCommand(
runCli: AgentCliRunner = runOpenClawAgentCli,
): LobsterCommand {
return {
name: "openclaw.agent",
meta: {
description: "Run a configured OpenClaw agent turn",
argsSchema: {
type: "object",
properties: {
agent: { type: "string", description: "Configured OpenClaw agent id" },
prompt: { type: "string", description: "Message for the agent" },
message: { type: "string", description: "Alias for prompt" },
model: { type: "string", description: "OpenClaw model override for this turn" },
sessionKey: { type: "string", description: "OpenClaw session key" },
"session-key": { type: "string", description: "Alias for sessionKey" },
sessionId: { type: "string", description: "OpenClaw session id" },
"session-id": { type: "string", description: "Alias for sessionId" },
thinking: { type: "string", description: "OpenClaw thinking level" },
timeout: { type: "number", description: "Agent timeout in seconds" },
local: { type: "boolean", description: "Force OpenClaw embedded execution" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: ["calls_openclaw_agent"],
},
help() {
return (
`openclaw.agent — run a configured OpenClaw agent turn\n\n` +
`Usage:\n` +
` openclaw.agent --agent ops --prompt "Summarize logs"\n` +
` openclaw.agent --agent ops --session-key incident-42 --model openai/gpt-5.4 --prompt "Continue"\n` +
` ... | openclaw.agent --agent ops --prompt "Review this input"\n\n` +
`Notes:\n` +
` - Delegates agent, session, model, and auth behavior to the installed OpenClaw CLI.\n` +
` - Requires --agent, --session-key, or --session-id.\n` +
` - Pipeline input is appended to the message as JSONL under a labeled section.\n` +
` - Returns the structured OpenClaw --json response unchanged.\n`
);
},
async run({ input, args, ctx }) {
const prompt = extractPrompt(args);
if (!prompt) {
throw new Error("openclaw.agent requires --prompt, --message, or positional text");
}
const agent = optionalString(args.agent);
const sessionKey = optionalString(args.sessionKey ?? args["session-key"]);
const sessionId = optionalString(args.sessionId ?? args["session-id"]);
if (!agent && !sessionKey && !sessionId) {
throw new Error("openclaw.agent requires --agent, --session-key, or --session-id");
}
const inputItems: unknown[] = [];
for await (const item of input) inputItems.push(item);
const argv = ["agent", "--json", "--message", appendPipelineInput(prompt, inputItems)];
pushOption(argv, "--agent", agent);
pushOption(argv, "--model", optionalString(args.model));
pushOption(argv, "--session-key", sessionKey);
pushOption(argv, "--session-id", sessionId);
pushOption(argv, "--thinking", optionalString(args.thinking));
if (args.timeout !== undefined && args.timeout !== null) {
const timeout = Number(args.timeout);
if (!Number.isInteger(timeout) || timeout < 0) {
throw new Error("openclaw.agent --timeout must be a non-negative integer in seconds");
}
pushOption(argv, "--timeout", String(timeout));
}
if (args.local === true) argv.push("--local");
const env = (ctx?.env ?? process.env) as NodeJS.ProcessEnv;
const executable = optionalString(env.LOBSTER_OPENCLAW_BIN) ?? "openclaw";
const response = await runCli({
executable,
argv,
cwd: ctx?.cwd ?? process.cwd(),
env,
signal: ctx?.signal,
forceTerminationSignal: ctx?.forceTerminationSignal,
});
return { output: streamOf([response]) };
},
};
}
export function runOpenClawAgentCli(params: {
executable: string;
argv: string[];
cwd: string;
env: NodeJS.ProcessEnv;
signal?: AbortSignal;
forceTerminationSignal?: AbortSignal;
}): Promise<unknown> {
return runAbortableProcess({
command: params.executable,
argv: params.argv,
cwd: params.cwd,
env: params.env,
signal: params.signal,
forceTerminationSignal: params.forceTerminationSignal,
maxOutputBytes: OPENCLAW_AGENT_MAX_OUTPUT_BYTES,
outputLimitMessage: `openclaw.agent output exceeded ${OPENCLAW_AGENT_MAX_OUTPUT_BYTES} bytes`,
notFoundMessage: "openclaw.agent could not find the OpenClaw CLI",
}).then(({ code, stdout, stderr }) => {
if (code !== 0) {
const detail = String(stderr || stdout || `exited with code ${code}`).trim();
throw new Error(`openclaw.agent failed: ${detail}`);
}
try {
const parsed = JSON.parse(stdout.trim() || "null");
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
throw new Error("response must be an object");
}
return parsed;
} catch {
throw new Error("openclaw.agent expected JSON output from `openclaw agent --json`");
}
});
}
function extractPrompt(args: Record<string, unknown>): string {
const explicit = optionalString(args.prompt ?? args.message);
if (explicit) return explicit;
return Array.isArray(args._) ? args._.map(String).join(" ").trim() : "";
}
function appendPipelineInput(prompt: string, items: unknown[]): string {
if (items.length === 0) return prompt;
const jsonl = items.map((item) => JSON.stringify(item)).join("\n");
return `${prompt}\n\nPipeline input (JSONL):\n${jsonl}`;
}
function optionalString(value: unknown): string | undefined {
if (value === undefined || value === null) return undefined;
const text = String(value).trim();
return text || undefined;
}
function pushOption(argv: string[], name: string, value: string | undefined): void {
if (value !== undefined) argv.push(name, value);
}
async function* streamOf(items: unknown[]) {
for (const item of items) yield item;
}
src/commands/stdlib/openclaw_invoke.ts
+148 -128
View File
@@ -1,149 +1,169 @@
function createInvokeCommand(commandName: string) {
return {
name: commandName,
meta: {
description: "Call a local OpenClaw tool endpoint",
argsSchema: {
type: "object",
properties: {
url: {
type: "string",
description: "OpenClaw control URL (or OPENCLAW_URL / CLAWD_URL)",
},
token: { type: "string", description: "Bearer token (or OPENCLAW_TOKEN / CLAWD_TOKEN)" },
tool: { type: "string", description: "Tool name (e.g. message, cron, github, etc.)" },
action: { type: "string", description: "Tool action" },
"args-json": { type: "string", description: "JSON string of tool args" },
sessionKey: { type: "string", description: "Optional session key attribution" },
"session-key": { type: "string", description: "Alias for sessionKey" },
dryRun: { type: "boolean", description: "Dry run" },
"dry-run": { type: "boolean", description: "Alias for dryRun" },
each: { type: "boolean", description: "Map each pipeline item into tool args" },
itemKey: {
type: "string",
description: "Key to set from the pipeline item (default: item)",
},
"item-key": { type: "string", description: "Alias for itemKey" },
_: { type: "array", items: { type: "string" } },
},
required: ["tool", "action"],
},
sideEffects: ["calls_clawd_tool"],
},
help() {
return (
`${commandName} — call a local OpenClaw tool endpoint\n\n` +
`Usage:\n` +
` ${commandName} --tool message --action send --args-json '{"provider":"telegram","to":"...","message":"..."}'\n` +
` ${commandName} --tool message --action send --args-json '{...}' --dry-run\n` +
` ... | ${commandName} --tool message --action send --each --item-key message --args-json '{"provider":"telegram","to":"..."}'\n\n` +
`Config:\n` +
` - Uses OPENCLAW_URL env var by default (or pass --url).\n` +
` - Backward compatible: CLAWD_URL is also supported.\n` +
` - Optional Bearer token via OPENCLAW_TOKEN env var (or pass --token).\n` +
` - Backward compatible: CLAWD_TOKEN is also supported.\n` +
` - Optional attribution via --session-key <sessionKey>.\n\n` +
`Notes:\n` +
` - This is a thin transport bridge. Lobster should not own OAuth/secrets.\n`
);
},
async run({ input, args, ctx }) {
const each = Boolean(args.each);
const itemKey = String(args.itemKey ?? args["item-key"] ?? "item");
return {
name: commandName,
meta: {
description: "Call a local OpenClaw tool endpoint",
argsSchema: {
type: "object",
properties: {
url: {
type: "string",
description: "OpenClaw control URL (or OPENCLAW_URL / CLAWD_URL)",
},
token: { type: "string", description: "Bearer token (or OPENCLAW_TOKEN / CLAWD_TOKEN)" },
tool: { type: "string", description: "Tool name (e.g. message, cron, github, etc.)" },
action: { type: "string", description: "Tool action" },
"args-json": { type: "string", description: "JSON string of tool args" },
sessionKey: { type: "string", description: "Optional session key attribution" },
"session-key": { type: "string", description: "Alias for sessionKey" },
dryRun: { type: "boolean", description: "Dry run" },
"dry-run": { type: "boolean", description: "Alias for dryRun" },
each: { type: "boolean", description: "Map each pipeline item into tool args" },
itemKey: {
type: "string",
description: "Key to set from the pipeline item (default: item)",
},
"item-key": { type: "string", description: "Alias for itemKey" },
_: { type: "array", items: { type: "string" } },
},
required: ["tool", "action"],
},
sideEffects: ["calls_clawd_tool"],
},
help() {
return (
`${commandName} — call a local OpenClaw tool endpoint\n\n` +
`Usage:\n` +
` ${commandName} --tool message --action send --args-json '{"provider":"telegram","to":"...","message":"..."}'\n` +
` ${commandName} --tool message --action send --args-json '{...}' --dry-run\n` +
` ... | ${commandName} --tool message --action send --each --item-key message --args-json '{"provider":"telegram","to":"..."}'\n\n` +
`Config:\n` +
` - Uses OPENCLAW_URL env var by default (or pass --url).\n` +
` - Backward compatible: CLAWD_URL is also supported.\n` +
` - Optional Bearer token via OPENCLAW_TOKEN env var (or pass --token).\n` +
` - Backward compatible: CLAWD_TOKEN is also supported.\n` +
` - Optional attribution via --session-key <sessionKey>.\n\n` +
`Notes:\n` +
` - This is a thin transport bridge. Lobster should not own OAuth/secrets.\n`
);
},
async run({ input, args, ctx }) {
const each = Boolean(args.each);
const itemKey = String(args.itemKey ?? args["item-key"] ?? "item");
const url = String(args.url ?? ctx.env.OPENCLAW_URL ?? ctx.env.CLAWD_URL ?? "").trim();
if (!url) throw new Error(`${commandName} requires --url or OPENCLAW_URL`);
const url = String(args.url ?? ctx.env.OPENCLAW_URL ?? ctx.env.CLAWD_URL ?? "").trim();
if (!url) throw new Error(`${commandName} requires --url or OPENCLAW_URL`);
const tool = args.tool;
const action = args.action;
if (!tool || !action) throw new Error(`${commandName} requires --tool and --action`);
const tool = args.tool;
const action = args.action;
if (!tool || !action) throw new Error(`${commandName} requires --tool and --action`);
const token = String(
args.token ?? ctx.env.OPENCLAW_TOKEN ?? ctx.env.CLAWD_TOKEN ?? "",
).trim();
const explicitToken = args.token !== undefined && args.token !== null;
const token = String(
explicitToken ? args.token : (ctx.env.OPENCLAW_TOKEN ?? ctx.env.CLAWD_TOKEN ?? ""),
).trim();
let toolArgs: any = {};
if (args["args-json"]) {
try {
toolArgs = JSON.parse(String(args["args-json"]));
} catch (_err) {
throw new Error(`${commandName} --args-json must be valid JSON`);
}
}
let toolArgs: any = {};
if (args["args-json"]) {
try {
toolArgs = JSON.parse(String(args["args-json"]));
} catch (_err) {
throw new Error(`${commandName} --args-json must be valid JSON`);
}
}
if (each && (toolArgs === null || typeof toolArgs !== "object" || Array.isArray(toolArgs))) {
throw new Error(`${commandName} --each requires --args-json to be an object`);
}
if (each && (toolArgs === null || typeof toolArgs !== "object" || Array.isArray(toolArgs))) {
throw new Error(`${commandName} --each requires --args-json to be an object`);
}
const endpoint = new URL("/tools/invoke", url);
const sessionKey = args.sessionKey ?? args["session-key"] ?? null;
const dryRun = args.dryRun ?? args["dry-run"] ?? null;
const endpoint = new URL("/tools/invoke", url);
if (endpoint.protocol !== "http:" && endpoint.protocol !== "https:") {
throw new Error(`${commandName} requires an http(s) --url`);
}
if (token && !explicitToken && !isLocalOpenClawOrigin(endpoint)) {
throw new Error(
`${commandName} refuses to send OPENCLAW_TOKEN/CLAWD_TOKEN to non-local --url; pass --token explicitly for remote endpoints`,
);
}
const sessionKey = args.sessionKey ?? args["session-key"] ?? null;
const dryRun = args.dryRun ?? args["dry-run"] ?? null;
const invokeOnce = async (argsValue: unknown) => {
const res = await fetch(endpoint, {
method: "POST",
headers: {
"content-type": "application/json",
...(token ? { authorization: `Bearer ${token}` } : null),
},
body: JSON.stringify({
tool: String(tool),
action: String(action),
args: argsValue,
...(sessionKey ? { sessionKey: String(sessionKey) } : null),
...(dryRun !== null ? { dryRun: Boolean(dryRun) } : null),
}),
});
const invokeOnce = async (argsValue: unknown) => {
const res = await fetch(endpoint, {
method: "POST",
signal: ctx.signal,
headers: {
"content-type": "application/json",
...(token ? { authorization: `Bearer ${token}` } : null),
},
body: JSON.stringify({
tool: String(tool),
action: String(action),
args: argsValue,
...(sessionKey ? { sessionKey: String(sessionKey) } : null),
...(dryRun !== null ? { dryRun: Boolean(dryRun) } : null),
}),
});
const text = await res.text();
if (!res.ok) {
throw new Error(`${commandName} failed (${res.status}): ${text.slice(0, 400)}`);
}
const text = await res.text();
if (!res.ok) {
throw new Error(`${commandName} failed (${res.status}): ${text.slice(0, 400)}`);
}
let parsed: any;
try {
parsed = text ? JSON.parse(text) : null;
} catch (_err) {
throw new Error(`${commandName} expected JSON response`);
}
let parsed: any;
try {
parsed = text ? JSON.parse(text) : null;
} catch (_err) {
throw new Error(`${commandName} expected JSON response`);
}
// Preferred: { ok: true, result: ... }
if (parsed && typeof parsed === "object" && !Array.isArray(parsed) && "ok" in parsed) {
if (parsed.ok !== true) {
const msg = parsed?.error?.message ?? "Unknown error";
throw new Error(`${commandName} tool error: ${msg}`);
}
const result = parsed.result;
return Array.isArray(result) ? result : [result];
}
// Preferred: { ok: true, result: ... }
if (parsed && typeof parsed === "object" && !Array.isArray(parsed) && "ok" in parsed) {
if (parsed.ok !== true) {
const msg = parsed?.error?.message ?? "Unknown error";
throw new Error(`${commandName} tool error: ${msg}`);
}
const result = parsed.result;
return Array.isArray(result) ? result : [result];
}
// Compatibility: raw JSON result
return Array.isArray(parsed) ? parsed : [parsed];
};
// Compatibility: raw JSON result
return Array.isArray(parsed) ? parsed : [parsed];
};
if (!each) {
// Drain input: for now we don't stream input into clawd calls.
for await (const _item of input) {
// no-op
}
const items = await invokeOnce(toolArgs);
return { output: asStream(items) };
}
if (!each) {
// Drain input: for now we don't stream input into clawd calls.
for await (const _item of input) {
// no-op
}
const items = await invokeOnce(toolArgs);
return { output: asStream(items) };
}
const out: any[] = [];
for await (const item of input) {
const argsValue = { ...(toolArgs as any), [itemKey]: item };
const items = await invokeOnce(argsValue);
out.push(...items);
}
const out: any[] = [];
for await (const item of input) {
const argsValue = { ...(toolArgs as any), [itemKey]: item };
const items = await invokeOnce(argsValue);
out.push(...items);
}
return { output: asStream(out) };
},
};
return { output: asStream(out) };
},
};
}
async function* asStream(items: any[]) {
for (const item of items) yield item;
for (const item of items) yield item;
}
function isLocalOpenClawOrigin(url: URL): boolean {
const hostname = url.hostname.toLowerCase();
return (
hostname === "localhost" ||
hostname === "127.0.0.1" ||
hostname === "::1" ||
hostname === "[::1]"
);
}
export const openclawInvokeCommand = createInvokeCommand("openclaw.invoke");
src/commands/stdlib/pick.ts
+40 -40
View File
@@ -1,43 +1,43 @@
export const pickCommand = {
name: "pick",
meta: {
description: "Project fields from objects",
argsSchema: {
type: "object",
properties: {
_: {
type: "array",
items: { type: "string" },
description: "First positional arg is a comma-separated list of fields",
},
},
required: ["_"],
},
sideEffects: [],
},
help() {
return `pick — project fields from objects\n\nUsage:\n ... | pick id,subject,from\n`;
},
async run({ input, args }) {
const spec = args._[0];
if (!spec) throw new Error("pick requires a comma-separated field list");
const fields = spec
.split(",")
.map((s) => s.trim())
.filter(Boolean);
name: "pick",
meta: {
description: "Project fields from objects",
argsSchema: {
type: "object",
properties: {
_: {
type: "array",
items: { type: "string" },
description: "First positional arg is a comma-separated list of fields",
},
},
required: ["_"],
},
sideEffects: [],
},
help() {
return `pick — project fields from objects\n\nUsage:\n ... | pick id,subject,from\n`;
},
async run({ input, args }) {
const spec = args._[0];
if (!spec) throw new Error("pick requires a comma-separated field list");
const fields = spec
.split(",")
.map((s) => s.trim())
.filter(Boolean);
return {
output: (async function* () {
for await (const item of input) {
if (item === null || typeof item !== "object") {
yield item;
continue;
}
const out = {};
for (const f of fields) out[f] = item[f];
yield out;
}
})(),
};
},
return {
output: (async function* () {
for await (const item of input) {
if (item === null || typeof item !== "object") {
yield item;
continue;
}
const out = {};
for (const f of fields) out[f] = item[f];
yield out;
}
})(),
};
},
};
src/commands/stdlib/sort.ts
+71 -71
View File
@@ -1,84 +1,84 @@
function getByPath(obj: any, path: string): any {
if (!path) return undefined;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
if (!path) return undefined;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
}
function defaultCompare(a: any, b: any): number {
// Treat undefined/null as last
const aU = a === undefined || a === null;
const bU = b === undefined || b === null;
if (aU && bU) return 0;
if (aU) return 1;
if (bU) return -1;
// Treat undefined/null as last
const aU = a === undefined || a === null;
const bU = b === undefined || b === null;
if (aU && bU) return 0;
if (aU) return 1;
if (bU) return -1;
// number compare if both numbers
if (typeof a === "number" && typeof b === "number") return a - b;
// number compare if both numbers
if (typeof a === "number" && typeof b === "number") return a - b;
// Deterministic lexical compare independent of process locale.
const aStr = String(a);
const bStr = String(b);
if (aStr < bStr) return -1;
if (aStr > bStr) return 1;
return 0;
// Deterministic lexical compare independent of process locale.
const aStr = String(a);
const bStr = String(b);
if (aStr < bStr) return -1;
if (aStr > bStr) return 1;
return 0;
}
export const sortCommand = {
name: "sort",
meta: {
description: "Sort items (stable) by a key or by stringified value",
argsSchema: {
type: "object",
properties: {
key: { type: "string", description: "Dot-path key to sort by (e.g. updatedAt, pr.number)" },
desc: { type: "boolean", description: "Sort descending" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`sort — sort items (stable) by a key\n\n` +
`Usage:\n` +
` ... | sort\n` +
` ... | sort --key updatedAt\n` +
` ... | sort --key prNumber --desc\n\n` +
`Notes:\n` +
` - Sorting is stable (preserves order for equal keys).\n` +
` - undefined/null keys sort last.\n`
);
},
async run({ input, args }: any) {
const key = typeof args.key === "string" ? args.key : undefined;
const desc = Boolean(args.desc);
name: "sort",
meta: {
description: "Sort items (stable) by a key or by stringified value",
argsSchema: {
type: "object",
properties: {
key: { type: "string", description: "Dot-path key to sort by (e.g. updatedAt, pr.number)" },
desc: { type: "boolean", description: "Sort descending" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`sort — sort items (stable) by a key\n\n` +
`Usage:\n` +
` ... | sort\n` +
` ... | sort --key updatedAt\n` +
` ... | sort --key prNumber --desc\n\n` +
`Notes:\n` +
` - Sorting is stable (preserves order for equal keys).\n` +
` - undefined/null keys sort last.\n`
);
},
async run({ input, args }: any) {
const key = typeof args.key === "string" ? args.key : undefined;
const desc = Boolean(args.desc);
const items: any[] = [];
let idx = 0;
for await (const item of input) {
items.push({ item, idx });
idx++;
}
const items: any[] = [];
let idx = 0;
for await (const item of input) {
items.push({ item, idx });
idx++;
}
items.sort((a, b) => {
const av = key ? getByPath(a.item, key) : a.item;
const bv = key ? getByPath(b.item, key) : b.item;
const c = defaultCompare(av, bv);
if (c !== 0) return desc ? -c : c;
// stable tie-break
return a.idx - b.idx;
});
items.sort((a, b) => {
const av = key ? getByPath(a.item, key) : a.item;
const bv = key ? getByPath(b.item, key) : b.item;
const c = defaultCompare(av, bv);
if (c !== 0) return desc ? -c : c;
// stable tie-break
return a.idx - b.idx;
});
return {
output: (async function* () {
for (const x of items) yield x.item;
})(),
};
},
return {
output: (async function* () {
for (const x of items) yield x.item;
})(),
};
},
};
src/commands/stdlib/state.ts
+90 -64
View File
@@ -1,81 +1,107 @@
import { promises as fsp } from "node:fs";
import { defaultStateDir, ensureDirectory, keyToPath, writeFileAtomic } from "../../state/store.js";
import { defaultStateDir, keyToPath, withFileLock, writeStateJson } from "../../state/store.js";
import { carryLlmProvenance } from "./llm_invoke.js";
// What this process last wrote to each state file. A value read straight back is the same value
// rebuilt from its own JSON, and the marks a command attached in-process are not in that JSON:
// without this, `llm.invoke | state.set k | state.get k` turns a replay that cost nothing into an
// item indistinguishable from one that was paid for. The remembered text has to match the file
// byte for byte, so nothing written by anything else can pick up marks it was never given.
const lastWritten = new Map<string, { text: string; value: unknown }>();
const MAX_REMEMBERED_WRITES = 64;
function rememberWrite(filePath: string, text: string, value: unknown) {
lastWritten.set(filePath, { text, value });
for (const oldest of lastWritten.keys()) {
if (lastWritten.size <= MAX_REMEMBERED_WRITES) break;
lastWritten.delete(oldest);
}
}
async function readRememberedState({ env, key, signal }) {
const filePath = keyToPath(defaultStateDir(env), key);
const read = async () => {
try {
const text = await fsp.readFile(filePath, "utf8");
const value = JSON.parse(text);
const written = lastWritten.get(filePath);
if (written?.text === text) carryLlmProvenance(written.value, value);
return value;
} catch (err: any) {
if (err?.code === "ENOENT") return null;
throw err;
}
};
try {
return await withFileLock({ filePath, signal, task: read });
} catch (err: any) {
if (["EACCES", "EPERM", "EROFS"].includes(err?.code)) return read();
throw err;
}
}
export const stateGetCommand = {
name: "state.get",
meta: {
description: "Read a JSON value from Lobster state",
argsSchema: {
type: "object",
properties: {
_: { type: "array", items: { type: "string" }, description: "Key" },
},
required: ["_"],
},
sideEffects: ["reads_state"],
},
help() {
return `state.get — read a JSON value from Lobster state\n\nUsage:\n state.get <key>\n\nEnv:\n LOBSTER_STATE_DIR overrides storage directory\n`;
},
async run({ args, ctx }) {
const key = args._[0];
if (!key) throw new Error("state.get requires a key");
name: "state.get",
meta: {
description: "Read a JSON value from Lobster state",
argsSchema: {
type: "object",
properties: {
_: { type: "array", items: { type: "string" }, description: "Key" },
},
required: ["_"],
},
sideEffects: ["reads_state"],
},
help() {
return `state.get — read a JSON value from Lobster state\n\nUsage:\n state.get <key>\n\nEnv:\n LOBSTER_STATE_DIR overrides storage directory\n`;
},
async run({ args, ctx }) {
const key = args._[0];
if (!key) throw new Error("state.get requires a key");
const stateDir = defaultStateDir(ctx.env);
const filePath = keyToPath(stateDir, key);
const value = await readRememberedState({ env: ctx.env, key, signal: ctx.signal });
let value = null;
try {
const text = await fsp.readFile(filePath, "utf8");
value = JSON.parse(text);
} catch (err) {
if (err?.code === "ENOENT") {
value = null;
} else {
throw err;
}
}
return { output: asStream([value]) };
},
return { output: asStream([value]) };
},
};
export const stateSetCommand = {
name: "state.set",
meta: {
description: "Write a JSON value to Lobster state",
argsSchema: {
type: "object",
properties: {
_: { type: "array", items: { type: "string" }, description: "Key" },
},
required: ["_"],
},
sideEffects: ["writes_state"],
},
help() {
return `state.set — write a JSON value to Lobster state\n\nUsage:\n <value> | state.set <key>\n\nNotes:\n - Consumes the entire input stream; stores a single JSON value.\n`;
},
async run({ input, args, ctx }) {
const key = args._[0];
if (!key) throw new Error("state.set requires a key");
name: "state.set",
meta: {
description: "Write a JSON value to Lobster state",
argsSchema: {
type: "object",
properties: {
_: { type: "array", items: { type: "string" }, description: "Key" },
},
required: ["_"],
},
sideEffects: ["writes_state"],
},
help() {
return `state.set — write a JSON value to Lobster state\n\nUsage:\n <value> | state.set <key>\n\nNotes:\n - Consumes the entire input stream; stores a single JSON value.\n`;
},
async run({ input, args, ctx }) {
const key = args._[0];
if (!key) throw new Error("state.set requires a key");
const items = [];
for await (const item of input) items.push(item);
const items = [];
for await (const item of input) items.push(item);
const value = items.length === 1 ? items[0] : items;
const value = items.length === 1 ? items[0] : items;
const stateDir = defaultStateDir(ctx.env);
const filePath = keyToPath(stateDir, key);
const text = JSON.stringify(value, null, 2) + "\n";
await writeStateJson({ env: ctx.env, key, value, signal: ctx.signal });
const filePath = keyToPath(defaultStateDir(ctx.env), key);
rememberWrite(filePath, text, value);
await ensureDirectory(stateDir);
await writeFileAtomic(filePath, JSON.stringify(value, null, 2) + "\n");
return { output: asStream([value]) };
},
return { output: asStream([value]) };
},
};
async function* asStream(items) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/commands/stdlib/table.ts
+47 -47
View File
@@ -1,62 +1,62 @@
function stringifyCell(v) {
if (v === null || v === undefined) return "";
if (typeof v === "string") return v;
if (typeof v === "number" || typeof v === "boolean") return String(v);
return JSON.stringify(v);
if (v === null || v === undefined) return "";
if (typeof v === "string") return v;
if (typeof v === "number" || typeof v === "boolean") return String(v);
return JSON.stringify(v);
}
export const tableCommand = {
name: "table",
meta: {
description: "Render items as a simple table",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
},
help() {
return `table — render items as a simple table\n\nUsage:\n ... | table\n\nNotes:\n - If items are objects, columns are union of keys (first 20 items).\n`;
},
async run({ input, ctx }) {
const items = [];
for await (const item of input) items.push(item);
name: "table",
meta: {
description: "Render items as a simple table",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
},
help() {
return `table — render items as a simple table\n\nUsage:\n ... | table\n\nNotes:\n - If items are objects, columns are union of keys (first 20 items).\n`;
},
async run({ input, ctx }) {
const items = [];
for await (const item of input) items.push(item);
if (items.length === 0) {
ctx.stdout.write("(no results)\n");
return { output: emptyStream(), rendered: true };
}
if (items.length === 0) {
ctx.stdout.write("(no results)\n");
return { output: emptyStream(), rendered: true };
}
const sample = items.slice(0, 20);
const objectItems = sample.filter((x) => x && typeof x === "object" && !Array.isArray(x));
const sample = items.slice(0, 20);
const objectItems = sample.filter((x) => x && typeof x === "object" && !Array.isArray(x));
if (objectItems.length === sample.length) {
const cols = [];
const seen = new Set();
for (const obj of objectItems) {
for (const k of Object.keys(obj)) {
if (!seen.has(k)) {
seen.add(k);
cols.push(k);
}
}
}
if (objectItems.length === sample.length) {
const cols = [];
const seen = new Set();
for (const obj of objectItems) {
for (const k of Object.keys(obj)) {
if (!seen.has(k)) {
seen.add(k);
cols.push(k);
}
}
}
const rows = [cols, ...items.map((it) => cols.map((c) => stringifyCell(it?.[c])))].map(
(row) => row.map((cell) => cell.replace(/\n/g, " ")),
);
const rows = [cols, ...items.map((it) => cols.map((c) => stringifyCell(it?.[c])))].map(
(row) => row.map((cell) => cell.replace(/\n/g, " ")),
);
const widths = cols.map((_, i) => Math.max(...rows.map((r) => r[i].length), 3));
const widths = cols.map((_, i) => Math.max(...rows.map((r) => r[i].length), 3));
const renderRow = (row) => row.map((cell, i) => cell.padEnd(widths[i])).join(" ");
ctx.stdout.write(renderRow(rows[0]) + "\n");
ctx.stdout.write(widths.map((w) => "-".repeat(w)).join(" ") + "\n");
for (const row of rows.slice(1)) ctx.stdout.write(renderRow(row) + "\n");
const renderRow = (row) => row.map((cell, i) => cell.padEnd(widths[i])).join(" ");
ctx.stdout.write(renderRow(rows[0]) + "\n");
ctx.stdout.write(widths.map((w) => "-".repeat(w)).join(" ") + "\n");
for (const row of rows.slice(1)) ctx.stdout.write(renderRow(row) + "\n");
return { output: emptyStream(), rendered: true };
}
return { output: emptyStream(), rendered: true };
}
// Fallback: render each item on a line.
for (const item of items) ctx.stdout.write(stringifyCell(item) + "\n");
return { output: emptyStream(), rendered: true };
},
// Fallback: render each item on a line.
for (const item of items) ctx.stdout.write(stringifyCell(item) + "\n");
return { output: emptyStream(), rendered: true };
},
};
async function* emptyStream() {}
src/commands/stdlib/template.ts
+108 -108
View File
@@ -2,125 +2,125 @@ import fs from "node:fs/promises";
import { applyFilters } from "../../core/filters.js";
function getByPath(obj: any, path: string): any {
if (path === "." || path === "this") return obj;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
if (path === "." || path === "this") return obj;
const parts = path.split(".").filter(Boolean);
let cur: any = obj;
for (const p of parts) {
if (cur == null) return undefined;
cur = cur[p];
}
return cur;
}
function splitFilterChain(expr: string): string[] {
const parts: string[] = [];
let current = "";
let i = 0;
while (i < expr.length) {
const ch = expr[i];
if (ch === '"' || ch === "'") {
const quote = ch;
current += ch;
i += 1;
while (i < expr.length && expr[i] !== quote) {
if (expr[i] === "\\" && i + 1 < expr.length) {
current += expr[i] + expr[i + 1];
i += 2;
} else {
current += expr[i];
i += 1;
}
}
if (i < expr.length) {
current += expr[i];
i += 1;
}
} else if (ch === "|") {
parts.push(current.trim());
current = "";
i += 1;
} else {
current += ch;
i += 1;
}
}
parts.push(current.trim());
return parts;
const parts: string[] = [];
let current = "";
let i = 0;
while (i < expr.length) {
const ch = expr[i];
if (ch === '"' || ch === "'") {
const quote = ch;
current += ch;
i += 1;
while (i < expr.length && expr[i] !== quote) {
if (expr[i] === "\\" && i + 1 < expr.length) {
current += expr[i] + expr[i + 1];
i += 2;
} else {
current += expr[i];
i += 1;
}
}
if (i < expr.length) {
current += expr[i];
i += 1;
}
} else if (ch === "|") {
parts.push(current.trim());
current = "";
i += 1;
} else {
current += ch;
i += 1;
}
}
parts.push(current.trim());
return parts;
}
function renderTemplate(tpl: string, ctx: any): string {
return tpl.replace(/\{\{\s*([^}]+?)\s*\}\}/g, (_m, expr) => {
const rawExpr = String(expr ?? "").trim();
const parts = splitFilterChain(rawExpr);
const key = parts[0];
let val: unknown = getByPath(ctx, key);
if (parts.length > 1) {
val = applyFilters(val, parts.slice(1));
}
if (val === undefined || val === null) return "";
if (typeof val === "string") return val;
if (typeof val === "number" || typeof val === "boolean") return String(val);
return JSON.stringify(val);
});
return tpl.replace(/\{\{\s*([^}]+?)\s*\}\}/g, (_m, expr) => {
const rawExpr = String(expr ?? "").trim();
const parts = splitFilterChain(rawExpr);
const key = parts[0];
let val: unknown = getByPath(ctx, key);
if (parts.length > 1) {
val = applyFilters(val, parts.slice(1));
}
if (val === undefined || val === null) return "";
if (typeof val === "string") return val;
if (typeof val === "number" || typeof val === "boolean") return String(val);
return JSON.stringify(val);
});
}
export const templateCommand = {
name: "template",
meta: {
description: "Render a simple {{path}} template against each input item",
argsSchema: {
type: "object",
properties: {
text: {
type: "string",
description:
"Template text (supports {{path}}, {{path | filter}}, {{.}} for the whole item)",
},
file: { type: "string", description: "Template file path" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`template — render a simple template against each item\n\n` +
`Usage:\n` +
` ... | template --text 'PR {{number}}: {{title}}'\n` +
` ... | template --file ./draft.txt\n\n` +
`Template syntax:\n` +
` - {{field}} or {{nested.field}}\n` +
` - {{.}} for the whole item\n` +
` - {{field | filter}} with pipe-based filters\n` +
` - Missing values render as empty string\n\n` +
`Filters:\n` +
` upper, lower, trim, truncate N, replace "from" "to", split sep\n` +
` first, last, length, join sep\n` +
` json, string, default val, round N, date fmt\n`
);
},
async run({ input, args }: any) {
let tpl = typeof args.text === "string" ? args.text : undefined;
const file = typeof args.file === "string" ? args.file : undefined;
name: "template",
meta: {
description: "Render a simple {{path}} template against each input item",
argsSchema: {
type: "object",
properties: {
text: {
type: "string",
description:
"Template text (supports {{path}}, {{path | filter}}, {{.}} for the whole item)",
},
file: { type: "string", description: "Template file path" },
_: { type: "array", items: { type: "string" } },
},
required: [],
},
sideEffects: [],
},
help() {
return (
`template — render a simple template against each item\n\n` +
`Usage:\n` +
` ... | template --text 'PR {{number}}: {{title}}'\n` +
` ... | template --file ./draft.txt\n\n` +
`Template syntax:\n` +
` - {{field}} or {{nested.field}}\n` +
` - {{.}} for the whole item\n` +
` - {{field | filter}} with pipe-based filters\n` +
` - Missing values render as empty string\n\n` +
`Filters:\n` +
` upper, lower, trim, truncate N, replace "from" "to", split sep\n` +
` first, last, length, join sep\n` +
` json, string, default val, round N, date fmt\n`
);
},
async run({ input, args }: any) {
let tpl = typeof args.text === "string" ? args.text : undefined;
const file = typeof args.file === "string" ? args.file : undefined;
if (!tpl && file) {
tpl = await fs.readFile(file, "utf8");
}
if (!tpl && file) {
tpl = await fs.readFile(file, "utf8");
}
if (!tpl) {
const positional = Array.isArray(args._) ? args._ : [];
if (positional.length) tpl = positional.join(" ");
}
if (!tpl) {
const positional = Array.isArray(args._) ? args._ : [];
if (positional.length) tpl = positional.join(" ");
}
if (!tpl) throw new Error("template requires --text or --file (or positional text)");
if (!tpl) throw new Error("template requires --text or --file (or positional text)");
return {
output: (async function* () {
for await (const item of input) {
yield renderTemplate(String(tpl), item);
}
})(),
};
},
return {
output: (async function* () {
for await (const item of input) {
yield renderTemplate(String(tpl), item);
}
})(),
};
},
};
src/commands/stdlib/where.ts
+64 -64
View File
@@ -1,78 +1,78 @@
function parsePredicate(expr) {
const m = expr.match(/^([a-zA-Z0-9_.]+)\s*(==|=|!=|<=|>=|<|>)\s*(.+)$/);
if (!m) throw new Error(`Invalid where expression: ${expr}`);
const [, path, op, rawValue] = m;
const m = expr.match(/^([a-zA-Z0-9_.]+)\s*(==|=|!=|<=|>=|<|>)\s*(.+)$/);
if (!m) throw new Error(`Invalid where expression: ${expr}`);
const [, path, op, rawValue] = m;
let value = rawValue;
if (rawValue === "true") value = true;
else if (rawValue === "false") value = false;
else if (rawValue === "null") value = null;
else if (!Number.isNaN(Number(rawValue)) && rawValue.trim() !== "") value = Number(rawValue);
let value = rawValue;
if (rawValue === "true") value = true;
else if (rawValue === "false") value = false;
else if (rawValue === "null") value = null;
else if (!Number.isNaN(Number(rawValue)) && rawValue.trim() !== "") value = Number(rawValue);
return { path, op: op === "=" ? "==" : op, value };
return { path, op: op === "=" ? "==" : op, value };
}
function getPath(obj, path) {
const parts = path.split(".");
let cur = obj;
for (const p of parts) {
if (cur === null || typeof cur !== "object") return undefined;
cur = cur[p];
}
return cur;
const parts = path.split(".");
let cur = obj;
for (const p of parts) {
if (cur === null || typeof cur !== "object") return undefined;
cur = cur[p];
}
return cur;
}
function compare(left, op, right) {
switch (op) {
case "==":
return left == right; // intentional loose equality for convenience
case "!=":
return left != right;
case "<":
return left < right;
case "<=":
return left <= right;
case ">":
return left > right;
case ">=":
return left >= right;
default:
throw new Error(`Unsupported operator: ${op}`);
}
switch (op) {
case "==":
return left == right; // intentional loose equality for convenience
case "!=":
return left != right;
case "<":
return left < right;
case "<=":
return left <= right;
case ">":
return left > right;
case ">=":
return left >= right;
default:
throw new Error(`Unsupported operator: ${op}`);
}
}
export const whereCommand = {
name: "where",
meta: {
description: "Filter objects by a simple predicate",
argsSchema: {
type: "object",
properties: {
_: {
type: "array",
items: { type: "string" },
description: "First positional arg is an expression like field=value or minutes>=30",
},
},
required: ["_"],
},
sideEffects: [],
},
help() {
return `where — filter objects by a simple predicate\n\nUsage:\n ... | where unread=true\n ... | where minutes>=30\n ... | where sender.domain==example.com\n`;
},
async run({ input, args }) {
const expr = args._[0];
if (!expr) throw new Error("where requires an expression (e.g. field=value)");
const pred = parsePredicate(expr);
name: "where",
meta: {
description: "Filter objects by a simple predicate",
argsSchema: {
type: "object",
properties: {
_: {
type: "array",
items: { type: "string" },
description: "First positional arg is an expression like field=value or minutes>=30",
},
},
required: ["_"],
},
sideEffects: [],
},
help() {
return `where — filter objects by a simple predicate\n\nUsage:\n ... | where unread=true\n ... | where minutes>=30\n ... | where sender.domain==example.com\n`;
},
async run({ input, args }) {
const expr = args._[0];
if (!expr) throw new Error("where requires an expression (e.g. field=value)");
const pred = parsePredicate(expr);
return {
output: (async function* () {
for await (const item of input) {
const left = getPath(item, pred.path);
if (compare(left, pred.op, pred.value)) yield item;
}
})(),
};
},
return {
output: (async function* () {
for await (const item of input) {
const left = getPath(item, pred.path);
if (compare(left, pred.op, pred.value)) yield item;
}
})(),
};
},
};
src/commands/types.ts
+20 -8
View File
@@ -1,13 +1,25 @@
export type CommandMeta = {
description?: string;
argsSchema?: unknown;
examples?: Array<{ args: Record<string, unknown>; description?: string }>;
sideEffects?: string[];
description?: string;
argsSchema?: unknown;
examples?: Array<{ args: Record<string, unknown>; description?: string }>;
sideEffects?: string[];
/**
* The command may create an input/approval gate before it begins execution.
* Commands that omit this are treated conservatively when a resumed pipeline
* is cancelled: its original capability cannot be replayed after dispatch.
*/
resumeSafeBeforeInput?: boolean;
/**
* The command remains side-effect-free after returning a resumed input until
* the next pipeline stage dispatches. This is intentionally opt-in so a
* command that acts on a resumed response consumes its capability first.
*/
resumeSafeAfterInput?: boolean;
};
export type LobsterCommand = {
name: string;
help: () => string;
run: (params: any) => Promise<any>;
meta?: CommandMeta;
name: string;
help: () => string;
run: (params: any) => Promise<any>;
meta?: CommandMeta;
};
src/commands/workflows/workflows_list.ts
+17 -17
View File
@@ -1,25 +1,25 @@
import { listWorkflows } from "../../workflows/registry.js";
export const workflowsListCommand = {
name: "workflows.list",
meta: {
description: "List available Lobster workflows",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
},
help() {
return `workflows.list — list available Lobster workflows\n\nUsage:\n workflows.list\n\nNotes:\n - Intended for OpenClaw to discover workflows dynamically.\n`;
},
async run({ input }) {
// Drain input.
for await (const _item of input) {
// no-op
}
name: "workflows.list",
meta: {
description: "List available Lobster workflows",
argsSchema: { type: "object", properties: {}, required: [] },
sideEffects: [],
},
help() {
return `workflows.list — list available Lobster workflows\n\nUsage:\n workflows.list\n\nNotes:\n - Intended for OpenClaw to discover workflows dynamically.\n`;
},
async run({ input }) {
// Drain input.
for await (const _item of input) {
// no-op
}
return { output: asStream(listWorkflows()) };
},
return { output: asStream(listWorkflows()) };
},
};
async function* asStream(items) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/commands/workflows/workflows_run.ts
+59 -59
View File
@@ -1,80 +1,80 @@
import { workflowRegistry } from "../../workflows/registry.js";
import {
runGithubPrMonitorWorkflow,
runGithubPrMonitorNotifyWorkflow,
runGithubPrMonitorWorkflow,
runGithubPrMonitorNotifyWorkflow,
} from "../../workflows/github_pr_monitor.js";
const runners = {
"github.pr.monitor": runGithubPrMonitorWorkflow,
"github.pr.monitor.notify": runGithubPrMonitorNotifyWorkflow,
"github.pr.monitor": runGithubPrMonitorWorkflow,
"github.pr.monitor.notify": runGithubPrMonitorNotifyWorkflow,
};
// Recipe runners - adapt SDK recipes to workflow runner interface
const recipeRunners = {};
export const workflowsRunCommand = {
name: "workflows.run",
meta: {
description: "Run a named Lobster workflow",
argsSchema: {
type: "object",
properties: {
name: { type: "string", description: "Workflow name" },
"args-json": { type: "string", description: "JSON string of workflow args" },
_: { type: "array", items: { type: "string" } },
},
required: ["name"],
},
sideEffects: [],
},
help() {
return `workflows.run — run a named Lobster workflow\n\nUsage:\n workflows.run --name <workflow> [--args-json '{...}']\n\nExample:\n workflows.run --name github.pr.monitor.notify --args-json '{"repo":"openclaw/openclaw","pr":1152}'\n`;
},
async run({ input, args, ctx }) {
// Drain input.
for await (const _item of input) {
// no-op
}
name: "workflows.run",
meta: {
description: "Run a named Lobster workflow",
argsSchema: {
type: "object",
properties: {
name: { type: "string", description: "Workflow name" },
"args-json": { type: "string", description: "JSON string of workflow args" },
_: { type: "array", items: { type: "string" } },
},
required: ["name"],
},
sideEffects: [],
},
help() {
return `workflows.run — run a named Lobster workflow\n\nUsage:\n workflows.run --name <workflow> [--args-json '{...}']\n\nExample:\n workflows.run --name github.pr.monitor.notify --args-json '{"repo":"openclaw/openclaw","pr":1152}'\n`;
},
async run({ input, args, ctx }) {
// Drain input.
for await (const _item of input) {
// no-op
}
const name = args.name ?? args._[0];
if (!name) throw new Error("workflows.run requires --name");
const name = args.name ?? args._[0];
if (!name) throw new Error("workflows.run requires --name");
// Check for recipe-based workflow first
const recipeRunner = recipeRunners[name];
if (recipeRunner) {
let workflowArgs = {};
if (args["args-json"]) {
try {
workflowArgs = JSON.parse(String(args["args-json"]));
} catch {
throw new Error("workflows.run --args-json must be valid JSON");
}
}
const result = await recipeRunner({ args: workflowArgs, ctx });
return { output: asStream([result]) };
}
// Check for recipe-based workflow first
const recipeRunner = recipeRunners[name];
if (recipeRunner) {
let workflowArgs = {};
if (args["args-json"]) {
try {
workflowArgs = JSON.parse(String(args["args-json"]));
} catch {
throw new Error("workflows.run --args-json must be valid JSON");
}
}
const result = await recipeRunner({ args: workflowArgs, ctx });
return { output: asStream([result]) };
}
// Fall back to legacy workflow registry
const meta = workflowRegistry[name];
if (!meta) throw new Error(`Unknown workflow: ${name}`);
// Fall back to legacy workflow registry
const meta = workflowRegistry[name];
if (!meta) throw new Error(`Unknown workflow: ${name}`);
const runner = runners[name];
if (!runner) throw new Error(`Workflow runner not implemented: ${name}`);
const runner = runners[name];
if (!runner) throw new Error(`Workflow runner not implemented: ${name}`);
let workflowArgs = {};
if (args["args-json"]) {
try {
workflowArgs = JSON.parse(String(args["args-json"]));
} catch {
throw new Error("workflows.run --args-json must be valid JSON");
}
}
let workflowArgs = {};
if (args["args-json"]) {
try {
workflowArgs = JSON.parse(String(args["args-json"]));
} catch {
throw new Error("workflows.run --args-json must be valid JSON");
}
}
const result = await runner({ args: workflowArgs, ctx });
return { output: asStream([result]) };
},
const result = await runner({ args: workflowArgs, ctx });
return { output: asStream([result]) };
},
};
async function* asStream(items) {
for (const item of items) yield item;
for (const item of items) yield item;
}
src/core/cost_tracker.ts
+159 -126
View File
@@ -1,168 +1,201 @@
export type StepCost = {
stepId: string;
model: string | null;
inputTokens: number;
outputTokens: number;
costUsd: number;
stepId: string;
model: string | null;
inputTokens: number;
outputTokens: number;
costUsd: number;
};
export type CostSummary = {
totalInputTokens: number;
totalOutputTokens: number;
estimatedCostUsd: number;
byStep: StepCost[];
totalInputTokens: number;
totalOutputTokens: number;
estimatedCostUsd: number;
byStep: StepCost[];
};
export type CostLimit = {
max_usd: number;
action?: "warn" | "stop";
max_usd: number;
action?: "warn" | "stop";
};
const DEFAULT_PRICING: Record<string, { input: number; output: number }> = {
"gpt-4o": { input: 2.5, output: 10.0 },
"gpt-4o-mini": { input: 0.15, output: 0.6 },
"gpt-4-turbo": { input: 10.0, output: 30.0 },
"gpt-3.5-turbo": { input: 0.5, output: 1.5 },
"claude-opus-4-20250514": { input: 15.0, output: 75.0 },
"claude-sonnet-4-5-20250514": { input: 3.0, output: 15.0 },
"claude-haiku-3-5": { input: 0.8, output: 4.0 },
"gemini-1.5-pro": { input: 1.25, output: 5.0 },
"gemini-1.5-flash": { input: 0.075, output: 0.3 },
"gpt-4o": { input: 2.5, output: 10.0 },
"gpt-4o-mini": { input: 0.15, output: 0.6 },
"gpt-4-turbo": { input: 10.0, output: 30.0 },
"gpt-3.5-turbo": { input: 0.5, output: 1.5 },
"claude-opus-4-20250514": { input: 15.0, output: 75.0 },
"claude-sonnet-4-5-20250514": { input: 3.0, output: 15.0 },
"claude-haiku-3-5": { input: 0.8, output: 4.0 },
"gemini-1.5-pro": { input: 1.25, output: 5.0 },
"gemini-1.5-flash": { input: 0.075, output: 0.3 },
};
const INVALID_PRICING_JSON_WARNING =
"[WARN] Ignoring invalid LOBSTER_LLM_PRICING_JSON; custom LLM pricing must be a JSON object whose model entries have finite non-negative input and output rates.\n";
"[WARN] Ignoring invalid LOBSTER_LLM_PRICING_JSON; custom LLM pricing must be a JSON object whose model entries have finite non-negative input and output rates.\n";
function toTokenCount(value: unknown): number {
const parsed = Number(value ?? 0);
if (!Number.isFinite(parsed) || parsed < 0) return 0;
return Math.floor(parsed);
const parsed = Number(value ?? 0);
if (!Number.isFinite(parsed) || parsed < 0) return 0;
return Math.floor(parsed);
}
/**
* The token counts a usage record is billed for, under any of the spellings providers use.
* Everything else a record carries — a `totalTokens`, a cache breakdown — costs nothing, so
* two records agreeing on these two numbers cost the same.
*/
export function billableTokens(usage: Record<string, unknown>) {
return {
inputTokens: toTokenCount(usage.inputTokens ?? usage.input_tokens ?? usage.prompt_tokens),
outputTokens: toTokenCount(
usage.outputTokens ?? usage.output_tokens ?? usage.completion_tokens,
),
};
}
export class CostTracker {
private steps: StepCost[] = [];
private steps: StepCost[] = [];
private pricing: Record<string, { input: number; output: number }>;
private pricing: Record<string, { input: number; output: number }>;
private stderr?: NodeJS.WritableStream;
private stderr?: NodeJS.WritableStream;
private warnedUnknownModels = new Set<string>();
private warnedUnknownModels = new Set<string>();
constructor(
customPricing?: Record<string, { input: number; output: number }>,
stderr?: NodeJS.WritableStream,
) {
this.pricing = { ...DEFAULT_PRICING, ...customPricing };
this.stderr = stderr;
}
constructor(
customPricing?: Record<string, { input: number; output: number }>,
stderr?: NodeJS.WritableStream,
) {
this.pricing = { ...DEFAULT_PRICING, ...customPricing };
this.stderr = stderr;
}
recordUsage(stepId: string, model: string | null, usage: Record<string, unknown>) {
const inputTokens = toTokenCount(
usage.inputTokens ?? usage.input_tokens ?? usage.prompt_tokens,
);
const outputTokens = toTokenCount(
usage.outputTokens ?? usage.output_tokens ?? usage.completion_tokens,
);
const pricingKey = typeof model === "string" && model.trim() ? model : null;
const pricing =
pricingKey && Object.prototype.hasOwnProperty.call(this.pricing, pricingKey)
? this.pricing[pricingKey]
: undefined;
if (!pricing) {
this.warnUnknownModel(pricingKey ?? "<missing>");
}
const effectivePricing = pricing ?? { input: 0, output: 0 };
const costUsd =
(inputTokens * effectivePricing.input + outputTokens * effectivePricing.output) / 1_000_000;
this.steps.push({ stepId, model, inputTokens, outputTokens, costUsd });
}
recordUsage(stepId: string, model: string | null, usage: Record<string, unknown>) {
const { inputTokens, outputTokens } = billableTokens(usage);
const pricingKey = typeof model === "string" && model.trim() ? model : null;
const pricing =
pricingKey && Object.prototype.hasOwnProperty.call(this.pricing, pricingKey)
? this.pricing[pricingKey]
: undefined;
if (!pricing) {
this.warnUnknownModel(pricingKey ?? "<missing>");
}
const effectivePricing = pricing ?? { input: 0, output: 0 };
const costUsd =
(inputTokens * effectivePricing.input + outputTokens * effectivePricing.output) / 1_000_000;
this.steps.push({ stepId, model, inputTokens, outputTokens, costUsd });
}
getSummary(): CostSummary {
let totalInputTokens = 0;
let totalOutputTokens = 0;
let estimatedCostUsd = 0;
/**
* Seeds this tracker with spend an earlier run of the same workflow already recorded — the
* steps completed before an approval or input gate paused it. A pause is not a spend reset:
* without this, `_meta.cost` after a resume would report only the steps that ran after it,
* and a `cost_limit` could be walked past one gate at a time. Entries are rebuilt through
* the same normalization as live usage rather than trusted verbatim, so a malformed stored
* record cannot poison later totals.
*/
restore(steps: readonly StepCost[] | undefined) {
if (!Array.isArray(steps)) return;
for (const step of steps) {
if (!step || typeof step !== "object") continue;
if (typeof step.stepId !== "string" || !step.stepId) continue;
const costUsd = Number(step.costUsd ?? 0);
this.steps.push({
stepId: step.stepId,
model: typeof step.model === "string" ? step.model : null,
inputTokens: toTokenCount(step.inputTokens),
outputTokens: toTokenCount(step.outputTokens),
costUsd: Number.isFinite(costUsd) && costUsd > 0 ? costUsd : 0,
});
}
}
for (const step of this.steps) {
totalInputTokens += step.inputTokens;
totalOutputTokens += step.outputTokens;
estimatedCostUsd += step.costUsd;
}
getSummary(): CostSummary {
let totalInputTokens = 0;
let totalOutputTokens = 0;
let estimatedCostUsd = 0;
return {
totalInputTokens,
totalOutputTokens,
estimatedCostUsd: Math.round(estimatedCostUsd * 1_000_000) / 1_000_000,
byStep: [...this.steps],
};
}
for (const step of this.steps) {
totalInputTokens += step.inputTokens;
totalOutputTokens += step.outputTokens;
estimatedCostUsd += step.costUsd;
}
hasUsage() {
return this.steps.length > 0;
}
return {
totalInputTokens,
totalOutputTokens,
estimatedCostUsd: Math.round(estimatedCostUsd * 1_000_000) / 1_000_000,
byStep: [...this.steps],
};
}
checkLimit(limit: CostLimit, stderr?: NodeJS.WritableStream) {
const summary = this.getSummary();
if (summary.estimatedCostUsd <= limit.max_usd) return;
hasUsage() {
return this.steps.length > 0;
}
if (limit.action === "stop") {
throw new Error(
`Cost limit exceeded: $${summary.estimatedCostUsd.toFixed(4)} > $${limit.max_usd.toFixed(2)} limit`,
);
}
checkLimit(limit: CostLimit, stderr?: NodeJS.WritableStream) {
const summary = this.getSummary();
if (summary.estimatedCostUsd <= limit.max_usd) return;
if (stderr) {
stderr.write(
`[WARN] Cost $${summary.estimatedCostUsd.toFixed(4)} exceeds limit $${limit.max_usd.toFixed(2)}\n`,
);
}
}
if (limit.action === "stop") {
throw new Error(
`Cost limit exceeded: $${summary.estimatedCostUsd.toFixed(4)} > $${limit.max_usd.toFixed(2)} limit`,
);
}
static parsePricingFromEnv(
env: Record<string, string | undefined>,
stderr?: NodeJS.WritableStream,
): Record<string, { input: number; output: number }> | undefined {
const raw = env.LOBSTER_LLM_PRICING_JSON;
if (!raw) return undefined;
try {
const parsed = JSON.parse(raw);
if (!isPricingMap(parsed)) {
stderr?.write(INVALID_PRICING_JSON_WARNING);
return undefined;
}
return parsed;
} catch {
stderr?.write(INVALID_PRICING_JSON_WARNING);
return undefined;
}
}
if (stderr) {
stderr.write(
`[WARN] Cost $${summary.estimatedCostUsd.toFixed(4)} exceeds limit $${limit.max_usd.toFixed(2)}\n`,
);
}
}
private warnUnknownModel(model: string) {
if (this.warnedUnknownModels.has(model)) return;
this.warnedUnknownModels.add(model);
const safeModel = safeJsonString(model);
this.stderr?.write(
`[WARN] No LLM pricing configured for model ${safeModel}; recording zero cost. Set LOBSTER_LLM_PRICING_JSON to enable cost_limit enforcement for this model.\n`,
);
}
static parsePricingFromEnv(
env: Record<string, string | undefined>,
stderr?: NodeJS.WritableStream,
): Record<string, { input: number; output: number }> | undefined {
const raw = env.LOBSTER_LLM_PRICING_JSON;
if (!raw) return undefined;
try {
const parsed = JSON.parse(raw);
if (!isPricingMap(parsed)) {
stderr?.write(INVALID_PRICING_JSON_WARNING);
return undefined;
}
return parsed;
} catch {
stderr?.write(INVALID_PRICING_JSON_WARNING);
return undefined;
}
}
private warnUnknownModel(model: string) {
if (this.warnedUnknownModels.has(model)) return;
this.warnedUnknownModels.add(model);
const safeModel = safeJsonString(model);
this.stderr?.write(
`[WARN] No LLM pricing configured for model ${safeModel}; recording zero cost. Set LOBSTER_LLM_PRICING_JSON to enable cost_limit enforcement for this model.\n`,
);
}
}
function isPricingMap(value: unknown): value is Record<string, { input: number; output: number }> {
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
for (const [model, entry] of Object.entries(value as Record<string, unknown>)) {
if (!model.trim()) return false;
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return false;
const rates = entry as Record<string, unknown>;
if (!isValidRate(rates.input) || !isValidRate(rates.output)) return false;
}
return true;
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
for (const [model, entry] of Object.entries(value as Record<string, unknown>)) {
if (!model.trim()) return false;
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return false;
const rates = entry as Record<string, unknown>;
if (!isValidRate(rates.input) || !isValidRate(rates.output)) return false;
}
return true;
}
function isValidRate(value: unknown) {
return typeof value === "number" && Number.isFinite(value) && value >= 0;
return typeof value === "number" && Number.isFinite(value) && value >= 0;
}
function safeJsonString(value: string) {
return JSON.stringify(value).replace(/[\u007f-\u009f\u2028\u2029]/g, (char) => {
return `\\u${char.charCodeAt(0).toString(16).padStart(4, "0")}`;
});
return JSON.stringify(value).replace(/[\u007f-\u009f\u2028\u2029]/g, (char) => {
return `\\u${char.charCodeAt(0).toString(16).padStart(4, "0")}`;
});
}
src/core/filters.ts
+66 -66
View File
@@ -6,96 +6,96 @@ FILTERS.set("upper", (v) => String(v ?? "").toUpperCase());
FILTERS.set("lower", (v) => String(v ?? "").toLowerCase());
FILTERS.set("trim", (v) => String(v ?? "").trim());
FILTERS.set("truncate", (v, n) => {
const s = String(v ?? "");
const parsed = parseInt(n ?? "", 10);
const len = Number.isNaN(parsed) ? 80 : parsed;
return s.length > len ? `${s.slice(0, len)}...` : s;
const s = String(v ?? "");
const parsed = parseInt(n ?? "", 10);
const len = Number.isNaN(parsed) ? 80 : parsed;
return s.length > len ? `${s.slice(0, len)}...` : s;
});
FILTERS.set("replace", (v, from, to) => String(v ?? "").replaceAll(from ?? "", to ?? ""));
FILTERS.set("split", (v, sep) => String(v ?? "").split(sep ?? ","));
FILTERS.set("first", (v) => (Array.isArray(v) ? v[0] : v));
FILTERS.set("last", (v) => (Array.isArray(v) ? v[v.length - 1] : v));
FILTERS.set("length", (v) => {
if (Array.isArray(v)) return v.length;
if (typeof v === "string") return v.length;
return 0;
if (Array.isArray(v)) return v.length;
if (typeof v === "string") return v.length;
return 0;
});
FILTERS.set("join", (v, sep) => (Array.isArray(v) ? v.join(sep ?? ", ") : String(v ?? "")));
FILTERS.set("json", (v) => JSON.stringify(v, null, 2));
FILTERS.set("string", (v) => String(v ?? ""));
FILTERS.set("default", (v, def) => (v == null || v === "" ? def : v));
FILTERS.set("round", (v, n) => {
const num = Number(v);
const dec = parseInt(n ?? "", 10) || 0;
return Number.isNaN(num) ? v : Number(num.toFixed(dec));
const num = Number(v);
const dec = parseInt(n ?? "", 10) || 0;
return Number.isNaN(num) ? v : Number(num.toFixed(dec));
});
FILTERS.set("date", (v, fmt) => {
const d =
typeof v === "number" || (typeof v === "string" && /^\d+$/.test(v.trim()))
? new Date(Number(v))
: new Date(String(v));
if (Number.isNaN(d.getTime())) return String(v);
if (!fmt) return d.toISOString();
return fmt
.replace("YYYY", String(d.getUTCFullYear()))
.replace("MM", String(d.getUTCMonth() + 1).padStart(2, "0"))
.replace("DD", String(d.getUTCDate()).padStart(2, "0"))
.replace("HH", String(d.getUTCHours()).padStart(2, "0"))
.replace("mm", String(d.getUTCMinutes()).padStart(2, "0"))
.replace("ss", String(d.getUTCSeconds()).padStart(2, "0"));
const d =
typeof v === "number" || (typeof v === "string" && /^\d+$/.test(v.trim()))
? new Date(Number(v))
: new Date(String(v));
if (Number.isNaN(d.getTime())) return String(v);
if (!fmt) return d.toISOString();
return fmt
.replace("YYYY", String(d.getUTCFullYear()))
.replace("MM", String(d.getUTCMonth() + 1).padStart(2, "0"))
.replace("DD", String(d.getUTCDate()).padStart(2, "0"))
.replace("HH", String(d.getUTCHours()).padStart(2, "0"))
.replace("mm", String(d.getUTCMinutes()).padStart(2, "0"))
.replace("ss", String(d.getUTCSeconds()).padStart(2, "0"));
});
export function getFilter(name: string): FilterFn | undefined {
return FILTERS.get(name);
return FILTERS.get(name);
}
export function parseFilterExpression(expr: string): [string, ...string[]] {
const trimmed = expr.trim();
const parts: string[] = [];
let i = 0;
const trimmed = expr.trim();
const parts: string[] = [];
let i = 0;
while (i < trimmed.length) {
while (i < trimmed.length && trimmed[i] === " ") i += 1;
if (i >= trimmed.length) break;
while (i < trimmed.length) {
while (i < trimmed.length && trimmed[i] === " ") i += 1;
if (i >= trimmed.length) break;
if (trimmed[i] === '"' || trimmed[i] === "'") {
const quote = trimmed[i];
i += 1;
let arg = "";
while (i < trimmed.length && trimmed[i] !== quote) {
if (trimmed[i] === "\\" && i + 1 < trimmed.length) {
arg += trimmed[i + 1];
i += 2;
} else {
arg += trimmed[i];
i += 1;
}
}
if (i < trimmed.length) i += 1;
parts.push(arg);
} else {
let arg = "";
while (i < trimmed.length && trimmed[i] !== " ") {
arg += trimmed[i];
i += 1;
}
parts.push(arg);
}
}
if (trimmed[i] === '"' || trimmed[i] === "'") {
const quote = trimmed[i];
i += 1;
let arg = "";
while (i < trimmed.length && trimmed[i] !== quote) {
if (trimmed[i] === "\\" && i + 1 < trimmed.length) {
arg += trimmed[i + 1];
i += 2;
} else {
arg += trimmed[i];
i += 1;
}
}
if (i < trimmed.length) i += 1;
parts.push(arg);
} else {
let arg = "";
while (i < trimmed.length && trimmed[i] !== " ") {
arg += trimmed[i];
i += 1;
}
parts.push(arg);
}
}
if (parts.length === 0) {
return [trimmed];
}
return parts as [string, ...string[]];
if (parts.length === 0) {
return [trimmed];
}
return parts as [string, ...string[]];
}
export function applyFilters(value: unknown, filterChain: string[]): unknown {
let result = value;
for (const filterExpr of filterChain) {
const [name, ...args] = parseFilterExpression(filterExpr);
const fn = FILTERS.get(name);
if (!fn) throw new Error(`Unknown template filter: ${name}`);
result = fn(result, ...args);
}
return result;
let result = value;
for (const filterExpr of filterChain) {
const [name, ...args] = parseFilterExpression(filterExpr);
const fn = FILTERS.get(name);
if (!fn) throw new Error(`Unknown template filter: ${name}`);
result = fn(result, ...args);
}
return result;
}
src/core/retry.ts
+78 -78
View File
@@ -1,64 +1,64 @@
export type RetryConfig = {
max?: number;
backoff?: "fixed" | "exponential";
delay_ms?: number;
max_delay_ms?: number;
jitter?: boolean;
max?: number;
backoff?: "fixed" | "exponential";
delay_ms?: number;
max_delay_ms?: number;
jitter?: boolean;
};
const DEFAULTS = {
max: 1,
backoff: "fixed" as const,
delay_ms: 1000,
max_delay_ms: 30000,
jitter: false,
max: 1,
backoff: "fixed" as const,
delay_ms: 1000,
max_delay_ms: 30000,
jitter: false,
};
export function resolveRetryConfig(raw: RetryConfig | undefined): Required<RetryConfig> {
if (!raw) return { ...DEFAULTS };
return {
max: raw.max ?? DEFAULTS.max,
backoff: raw.backoff ?? DEFAULTS.backoff,
delay_ms: raw.delay_ms ?? DEFAULTS.delay_ms,
max_delay_ms: raw.max_delay_ms ?? DEFAULTS.max_delay_ms,
jitter: raw.jitter ?? DEFAULTS.jitter,
};
if (!raw) return { ...DEFAULTS };
return {
max: raw.max ?? DEFAULTS.max,
backoff: raw.backoff ?? DEFAULTS.backoff,
delay_ms: raw.delay_ms ?? DEFAULTS.delay_ms,
max_delay_ms: raw.max_delay_ms ?? DEFAULTS.max_delay_ms,
jitter: raw.jitter ?? DEFAULTS.jitter,
};
}
function computeDelay(config: Required<RetryConfig>, attempt: number): number {
let delay: number;
if (config.backoff === "exponential") {
delay = Math.min(config.delay_ms * Math.pow(2, attempt), config.max_delay_ms);
} else {
delay = config.delay_ms;
}
if (config.jitter) {
// +/- 10% randomization, clamped to max_delay_ms
const jitterRange = delay * 0.1;
delay += (Math.random() * 2 - 1) * jitterRange;
delay = Math.min(delay, config.max_delay_ms);
}
return Math.max(0, Math.round(delay));
let delay: number;
if (config.backoff === "exponential") {
delay = Math.min(config.delay_ms * Math.pow(2, attempt), config.max_delay_ms);
} else {
delay = config.delay_ms;
}
if (config.jitter) {
// +/- 10% randomization, clamped to max_delay_ms
const jitterRange = delay * 0.1;
delay += (Math.random() * 2 - 1) * jitterRange;
delay = Math.min(delay, config.max_delay_ms);
}
return Math.max(0, Math.round(delay));
}
function abortableSleep(ms: number, signal?: AbortSignal): Promise<void> {
if (!signal) return new Promise((r) => setTimeout(r, ms));
return new Promise((resolve, reject) => {
if (signal.aborted) {
reject(signal.reason ?? new DOMException("The operation was aborted.", "AbortError"));
return;
}
let timer: ReturnType<typeof setTimeout>;
const onAbort = () => {
clearTimeout(timer);
reject(signal.reason ?? new DOMException("The operation was aborted.", "AbortError"));
};
timer = setTimeout(() => {
signal.removeEventListener("abort", onAbort);
resolve();
}, ms);
signal.addEventListener("abort", onAbort, { once: true });
});
if (!signal) return new Promise((r) => setTimeout(r, ms));
return new Promise((resolve, reject) => {
if (signal.aborted) {
reject(signal.reason ?? new DOMException("The operation was aborted.", "AbortError"));
return;
}
let timer: ReturnType<typeof setTimeout>;
const onAbort = () => {
clearTimeout(timer);
reject(signal.reason ?? new DOMException("The operation was aborted.", "AbortError"));
};
timer = setTimeout(() => {
signal.removeEventListener("abort", onAbort);
resolve();
}, ms);
signal.addEventListener("abort", onAbort, { once: true });
});
}
/**
@@ -70,35 +70,35 @@ function abortableSleep(ms: number, signal?: AbortSignal): Promise<void> {
* the last error after all retries are exhausted.
*/
export async function withRetry<T>(
fn: () => Promise<T>,
config: Required<RetryConfig>,
options?: {
signal?: AbortSignal;
shouldRetry?: (error: any, attempt: number) => boolean;
onRetry?: (attempt: number, error: Error, delayMs: number) => void;
},
fn: () => Promise<T>,
config: Required<RetryConfig>,
options?: {
signal?: AbortSignal;
shouldRetry?: (error: any, attempt: number) => boolean;
onRetry?: (attempt: number, error: Error, delayMs: number) => void;
},
): Promise<T> {
let lastError: Error | undefined;
for (let attempt = 0; attempt < config.max; attempt++) {
try {
return await fn();
} catch (err: any) {
// Only propagate AbortError immediately for external workflow cancellation.
// Per-attempt timeout AbortErrors (options.signal not aborted) flow through
// shouldRetry so timeout_ms + retry.max combinations work as documented.
if ((err?.name === "AbortError" || err?.code === "ABORT_ERR") && options?.signal?.aborted) {
throw err;
}
lastError = err;
if (attempt + 1 < config.max) {
if (options?.shouldRetry && !options.shouldRetry(err, attempt + 1)) {
throw err;
}
const delay = computeDelay(config, attempt);
options?.onRetry?.(attempt + 1, err, delay);
await abortableSleep(delay, options?.signal);
}
}
}
throw lastError;
let lastError: Error | undefined;
for (let attempt = 0; attempt < config.max; attempt++) {
try {
return await fn();
} catch (err: any) {
// Only propagate AbortError immediately for external workflow cancellation.
// Per-attempt timeout AbortErrors (options.signal not aborted) flow through
// shouldRetry so timeout_ms + retry.max combinations work as documented.
if ((err?.name === "AbortError" || err?.code === "ABORT_ERR") && options?.signal?.aborted) {
throw err;
}
lastError = err;
if (attempt + 1 < config.max) {
if (options?.shouldRetry && !options.shouldRetry(err, attempt + 1)) {
throw err;
}
const delay = computeDelay(config, attempt);
options?.onRetry?.(attempt + 1, err, delay);
await abortableSleep(delay, options?.signal);
}
}
}
throw lastError;
}
src/core/tool_runtime.ts
+469 -329
View File
@@ -7,384 +7,524 @@ import { decodeResumeToken, kindFromStateKey } from "../resume.js";
import { runPipeline } from "../runtime.js";
import { encodeToken } from "../token.js";
import {
deleteStateJson,
deleteApprovalId,
findStateKeyByApprovalId,
cleanupApprovalIndexByStateKey,
deleteStateJsonWithBoundedResumeCleanup,
deleteUnconsumedResumeState,
deleteApprovalId,
findStateKeyByApprovalId,
cleanupApprovalIndexByStateKey,
consumeResumeState,
restoreConsumedResumeState,
stateJsonExists,
} from "../state/store.js";
import { WorkflowResumeArgumentError, runWorkflowFile } from "../workflows/file.js";
import {
finalizePipelineToolRun,
loadPipelineResumeState,
validatePipelineInputResponse,
WorkflowResumeArgumentError,
alternateWorkflowResumeStateKey,
runWorkflowFile,
} from "../workflows/file.js";
import {
finalizePipelineToolRun,
loadPipelineResumeState,
validatePipelineInputResponse,
} from "../pipeline_resume_state.js";
type ToolRunContext = {
cwd?: string;
env?: Record<string, string | undefined>;
mode?: "tool" | "human" | "sdk";
stdin?: NodeJS.ReadableStream;
stdout?: NodeJS.WritableStream;
stderr?: NodeJS.WritableStream;
signal?: AbortSignal;
registry?: any;
llmAdapters?: Record<string, any>;
cwd?: string;
env?: Record<string, string | undefined>;
mode?: "tool" | "human" | "sdk";
stdin?: NodeJS.ReadableStream;
stdout?: NodeJS.WritableStream;
stderr?: NodeJS.WritableStream;
signal?: AbortSignal;
forceTerminationSignal?: AbortSignal;
registry?: any;
llmAdapters?: Record<string, any>;
};
type ToolEnvelope = {
protocolVersion: 1;
ok: boolean;
status?: "ok" | "needs_approval" | "needs_input" | "cancelled";
output?: unknown[];
requiresApproval?: {
type?: "approval_request";
prompt: string;
items: unknown[];
preview?: string;
resumeToken?: string;
approvalId?: string;
} | null;
requiresInput?: {
type?: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
resumeToken?: string;
} | null;
error?: {
type: string;
message: string;
};
protocolVersion: 1;
ok: boolean;
status?: "ok" | "needs_approval" | "needs_input" | "cancelled";
output?: unknown[];
requiresApproval?: {
type?: "approval_request";
prompt: string;
items: unknown[];
preview?: string;
resumeToken?: string;
approvalId?: string;
} | null;
requiresInput?: {
type?: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
resumeToken?: string;
} | null;
error?: {
type: string;
message: string;
};
};
export async function runToolRequest({
pipeline,
filePath,
args,
ctx = {},
pipeline,
filePath,
args,
ctx = {},
}: {
pipeline?: string;
filePath?: string;
args?: Record<string, unknown>;
ctx?: ToolRunContext;
pipeline?: string;
filePath?: string;
args?: Record<string, unknown>;
ctx?: ToolRunContext;
}): Promise<ToolEnvelope> {
const runtime = createToolContext(ctx);
const hasPipeline = typeof pipeline === "string" && pipeline.trim().length > 0;
const hasFile = typeof filePath === "string" && filePath.trim().length > 0;
const runtime = createToolContext(ctx);
const hasPipeline = typeof pipeline === "string" && pipeline.trim().length > 0;
const hasFile = typeof filePath === "string" && filePath.trim().length > 0;
if (!hasPipeline && !hasFile) {
return errorEnvelope("parse_error", "run requires either pipeline or filePath");
}
if (hasPipeline && hasFile) {
return errorEnvelope("parse_error", "run accepts either pipeline or filePath, not both");
}
if (!hasPipeline && !hasFile) {
return errorEnvelope("parse_error", "run requires either pipeline or filePath");
}
if (hasPipeline && hasFile) {
return errorEnvelope("parse_error", "run accepts either pipeline or filePath, not both");
}
if (hasFile) {
let resolvedFilePath: string;
try {
resolvedFilePath = await resolveWorkflowFile(filePath!, runtime.cwd);
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
if (hasFile) {
let resolvedFilePath: string;
try {
resolvedFilePath = await resolveWorkflowFile(filePath!, runtime.cwd);
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
try {
const output = await runWorkflowFile({
filePath: resolvedFilePath,
args,
ctx: runtime,
});
try {
const output = await runWorkflowFile({
filePath: resolvedFilePath,
args,
ctx: runtime,
});
if (output.status === "needs_approval") {
return okEnvelope("needs_approval", [], output.requiresApproval ?? null, null);
}
if (output.status === "needs_input") {
return okEnvelope("needs_input", [], null, output.requiresInput ?? null);
}
if (output.status === "cancelled") {
return okEnvelope("cancelled", [], null, null);
}
return okEnvelope("ok", output.output, null, null);
} catch (err: any) {
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
}
if (output.status === "needs_approval") {
return okEnvelope("needs_approval", [], output.requiresApproval ?? null, null);
}
if (output.status === "needs_input") {
return okEnvelope("needs_input", [], null, output.requiresInput ?? null);
}
if (output.status === "cancelled") {
return okEnvelope("cancelled", [], null, null);
}
return okEnvelope("ok", output.output, null, null);
} catch (err: any) {
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
}
let parsed;
try {
parsed = parsePipeline(String(pipeline));
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
let parsed;
try {
parsed = parsePipeline(String(pipeline));
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
try {
const output = await runPipeline({
pipeline: parsed,
registry: runtime.registry,
input: [],
stdin: runtime.stdin,
stdout: runtime.stdout,
stderr: runtime.stderr,
env: runtime.env,
mode: "tool",
cwd: runtime.cwd,
llmAdapters: runtime.llmAdapters,
signal: runtime.signal,
});
try {
const output = await runPipeline({
pipeline: parsed,
registry: runtime.registry,
input: [],
stdin: runtime.stdin,
stdout: runtime.stdout,
stderr: runtime.stderr,
env: runtime.env,
mode: "tool",
cwd: runtime.cwd,
llmAdapters: runtime.llmAdapters,
signal: runtime.signal,
forceTerminationSignal: runtime.forceTerminationSignal,
haltAfterStageOnAbort: true,
});
const finalized = await finalizePipelineToolRun({
env: runtime.env,
pipeline: parsed,
output,
});
return okEnvelope(
finalized.status,
finalized.output,
finalized.requiresApproval,
finalized.requiresInput,
);
} catch (err: any) {
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
const finalized = await finalizePipelineToolRun({
env: runtime.env,
pipeline: parsed,
output,
signal: runtime.signal,
});
return okEnvelope(
finalized.status,
finalized.output,
finalized.requiresApproval,
finalized.requiresInput,
);
} catch (err: any) {
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
}
export async function resumeToolRequest({
token,
approvalId,
approved,
response,
cancel,
ctx = {},
token,
approvalId,
approved,
response,
cancel,
ctx = {},
}: {
token?: string;
approvalId?: string;
approved?: boolean;
response?: unknown;
cancel?: boolean;
ctx?: ToolRunContext;
token?: string;
approvalId?: string;
approved?: boolean;
response?: unknown;
cancel?: boolean;
ctx?: ToolRunContext;
}): Promise<ToolEnvelope> {
const runtime = createToolContext(ctx);
let payload: any;
let resolvedApprovalId = approvalId ?? null;
const runtime = createToolContext(ctx);
let payload: any;
let resolvedApprovalId = approvalId ?? null;
try {
// Resolve short approval ID to token if provided
let resolvedToken: string;
if (approvalId) {
const stateKey = await findStateKeyByApprovalId({ env: runtime.env, approvalId });
if (!stateKey) {
return errorEnvelope("parse_error", `Approval ID "${approvalId}" not found or expired`);
}
const kind = kindFromStateKey(stateKey);
resolvedToken = encodeToken({
protocolVersion: 1,
v: 1,
kind,
stateKey,
});
} else if (token) {
resolvedToken = token;
} else {
return errorEnvelope("parse_error", "resume requires token or approvalId");
}
payload = decodeResumeToken(resolvedToken);
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
try {
// Resolve short approval ID to token if provided
let resolvedToken: string;
if (approvalId) {
const stateKey = await findStateKeyByApprovalId({ env: runtime.env, approvalId });
if (!stateKey) {
return errorEnvelope("parse_error", `Approval ID "${approvalId}" not found or expired`);
}
const kind = kindFromStateKey(stateKey);
resolvedToken = encodeToken({
protocolVersion: 1,
v: 1,
kind,
stateKey,
});
} else if (token) {
resolvedToken = token;
} else {
return errorEnvelope("parse_error", "resume requires token or approvalId");
}
payload = decodeResumeToken(resolvedToken);
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
// Helper: clean up approval ID index after successful use
const cleanupIndex = async () => {
if (resolvedApprovalId) {
await deleteApprovalId({ env: runtime.env, approvalId: resolvedApprovalId });
} else if (payload?.stateKey) {
await cleanupApprovalIndexByStateKey({ env: runtime.env, stateKey: payload.stateKey });
}
};
// Helper: clean up approval ID index after successful use
const cleanupIndex = async (stateKey = payload?.stateKey) => {
if (resolvedApprovalId) {
await deleteApprovalId({ env: runtime.env, approvalId: resolvedApprovalId });
} else if (stateKey) {
await cleanupApprovalIndexByStateKey({ env: runtime.env, stateKey });
}
};
if (cancel === true) {
await cleanupIndex();
if (payload.kind === "workflow-file" && payload.stateKey) {
await deleteStateJson({ env: runtime.env, key: payload.stateKey });
}
if (payload.kind === "pipeline-resume" && payload.stateKey) {
await deleteStateJson({ env: runtime.env, key: payload.stateKey });
}
return okEnvelope("cancelled", [], null, null);
}
if (cancel === true) {
let stateKeys = payload.stateKey ? [payload.stateKey] : [];
if (payload.kind === "workflow-file" && payload.stateKey) {
const alternateStateKey = alternateWorkflowResumeStateKey(payload.stateKey);
if (alternateStateKey) {
// Delete a non-authoritative spelling first. If cancellation interrupts
// its lock wait, the state that makes this capability resumable remains.
const [primaryExists, alternateExists] = await Promise.all([
stateJsonExists({ env: runtime.env, key: payload.stateKey }),
stateJsonExists({ env: runtime.env, key: alternateStateKey }),
]);
if (primaryExists || !alternateExists) {
stateKeys = [alternateStateKey, payload.stateKey];
} else {
stateKeys = [payload.stateKey, alternateStateKey];
}
}
}
// Keep the capability indexed until every state deletion succeeds. A
// cancelled request must not orphan a resume state by dropping its
// approval ID while waiting on another writer's state lock.
const deletionResults = [];
for (const stateKey of new Set(stateKeys)) {
deletionResults.push(
await deleteUnconsumedResumeState({
env: runtime.env,
key: stateKey,
signal: runtime.signal,
}),
);
}
if (
stateKeys.length > 0 &&
(deletionResults.includes("claimed") ||
deletionResults.every((result) => result === "missing"))
) {
return errorEnvelope("runtime_error", "Resume state not found");
}
if (resolvedApprovalId) {
await cleanupIndex();
} else {
for (const stateKey of stateKeys) await cleanupIndex(stateKey);
}
return okEnvelope("cancelled", [], null, null);
}
if (payload.kind === "workflow-file") {
try {
const output = await runWorkflowFile({
filePath: payload.filePath,
ctx: runtime,
resume: payload,
approved,
response,
cancel,
});
if (payload.kind === "workflow-file") {
let workflowResumeStateKey = payload.stateKey;
try {
const output = await runWorkflowFile({
filePath: payload.filePath,
ctx: {
...runtime,
_onResumeStateResolved: (stateKey) => {
workflowResumeStateKey = stateKey;
},
},
resume: payload,
approved,
response,
cancel,
});
if (output.status === "needs_approval") {
// Don't clean up index — next gate will issue a new approvalId
return okEnvelope("needs_approval", [], output.requiresApproval ?? null, null);
}
if (output.status === "needs_input") {
return okEnvelope("needs_input", [], null, output.requiresInput ?? null);
}
await cleanupIndex();
if (output.status === "cancelled") {
return okEnvelope("cancelled", [], null, null);
}
return okEnvelope("ok", output.output, null, null);
} catch (err: any) {
if (err instanceof WorkflowResumeArgumentError) {
return errorEnvelope("parse_error", err.message);
}
// Don't clean up index on error — allow retry by --id
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
}
if (output.status === "needs_approval") {
return okEnvelope("needs_approval", [], output.requiresApproval ?? null, null);
}
if (output.status === "needs_input") {
return okEnvelope("needs_input", [], null, output.requiresInput ?? null);
}
await cleanupIndex(workflowResumeStateKey);
if (output.status === "cancelled") {
return okEnvelope("cancelled", [], null, null);
}
return okEnvelope("ok", output.output, null, null);
} catch (err: any) {
if (err instanceof WorkflowResumeArgumentError) {
return errorEnvelope("parse_error", err.message);
}
// Non-abort failures and cancellations before step execution remain retryable.
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
}
let resumeState;
try {
resumeState = await loadPipelineResumeState(runtime.env, payload.stateKey);
} catch (err: any) {
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
if (runtime.signal?.aborted) {
return errorEnvelope(
"runtime_error",
runtime.signal.reason instanceof Error
? runtime.signal.reason.message
: "This operation was aborted",
);
}
if (resumeState.haltType === "input_request") {
if (approved !== undefined) {
return errorEnvelope("parse_error", "pipeline input resumes require response");
}
if (response === undefined) {
return errorEnvelope("parse_error", "pipeline input resumes require response");
}
try {
validatePipelineInputResponse(resumeState.inputSchema, response);
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
} else {
if (response !== undefined) {
return errorEnvelope(
"parse_error",
"approval resumes require approved=true|false, not response",
);
}
if (approved !== true) {
await cleanupIndex();
await deleteStateJson({ env: runtime.env, key: payload.stateKey });
return okEnvelope("cancelled", [], null, null);
}
}
let resumeState;
try {
// No state has been claimed yet, so a cancelled resume can return before
// touching the lock and leave its capability safely retryable.
resumeState = await loadPipelineResumeState(runtime.env, payload.stateKey, runtime.signal);
} catch (err: any) {
// Approval rejection historically reaches the signal-aware deletion path
// below. Keep its direct cancellation propagation while other resume modes
// retain their structured tool envelope.
if (runtime.signal?.aborted && approved === false) throw err;
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
const isSameStageInput =
resumeState.haltType === "input_request" && resumeState.resumeMode === "same_stage";
const remaining = resumeState.pipeline.slice(resumeState.resumeAtIndex);
const input = isSameStageInput
? resumeState.items
: resumeState.haltType === "input_request"
? [response]
: resumeState.items;
const requestInputResume = isSameStageInput
? {
state: resumeState.commandInput!,
response,
onConsumed: async () => {
await cleanupIndex();
await deleteStateJson({ env: runtime.env, key: payload.stateKey });
},
}
: undefined;
if (resumeState.haltType === "input_request") {
if (approved !== undefined) {
return errorEnvelope("parse_error", "pipeline input resumes require response");
}
if (response === undefined) {
return errorEnvelope("parse_error", "pipeline input resumes require response");
}
try {
validatePipelineInputResponse(resumeState.inputSchema, response);
} catch (err: any) {
return errorEnvelope("parse_error", err?.message ?? String(err));
}
} else {
if (response !== undefined) {
return errorEnvelope(
"parse_error",
"approval resumes require approved=true|false, not response",
);
}
if (approved !== true) {
// Keep the approval ID usable while this may still be waiting on a
// concurrent state writer. Dropping the index first would orphan the
// capability if cancellation interrupts the deletion.
const deletion = await deleteUnconsumedResumeState({
env: runtime.env,
key: payload.stateKey,
signal: runtime.signal,
});
if (deletion !== "deleted") {
return errorEnvelope("runtime_error", "Pipeline resume state not found");
}
await cleanupIndex();
return okEnvelope("cancelled", [], null, null);
}
}
try {
const output = await runPipeline({
pipeline: remaining,
registry: runtime.registry,
stdin: runtime.stdin,
stdout: runtime.stdout,
stderr: runtime.stderr,
env: runtime.env,
mode: "tool",
cwd: runtime.cwd,
llmAdapters: runtime.llmAdapters,
signal: runtime.signal,
input,
requestInputResume,
});
const isSameStageInput =
resumeState.haltType === "input_request" && resumeState.resumeMode === "same_stage";
const remaining = resumeState.pipeline.slice(resumeState.resumeAtIndex);
const input = isSameStageInput
? resumeState.items
: resumeState.haltType === "input_request"
? [response]
: resumeState.items;
const abortedBeforeResume = runtime.signal?.aborted === true;
let pipelineResumeStateRestored = false;
let pipelineExecutionStarted = false;
let pipelineResumeStateClaimId: string | undefined;
const requestInputResume = isSameStageInput
? {
state: resumeState.commandInput!,
response,
}
: undefined;
await cleanupIndex();
const finalized = await finalizePipelineToolRun({
env: runtime.env,
pipeline: remaining,
output,
previousStateKey: payload.stateKey,
});
return okEnvelope(
finalized.status,
finalized.output,
finalized.requiresApproval,
finalized.requiresInput,
);
} catch (err: any) {
// Don't clean up index on error — allow retry by --id
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
try {
const output = await runPipeline({
pipeline: remaining,
registry: runtime.registry,
stdin: runtime.stdin,
stdout: runtime.stdout,
stderr: runtime.stderr,
env: runtime.env,
mode: "tool",
cwd: runtime.cwd,
llmAdapters: runtime.llmAdapters,
signal: runtime.signal,
forceTerminationSignal: runtime.forceTerminationSignal,
haltAfterStageOnAbort: true,
input,
requestInputResume,
onExecutionStart: async () => {
const consumption = await consumeResumeState({
env: runtime.env,
key: payload.stateKey,
expectedState: resumeState,
signal: runtime.signal,
});
if (!consumption.consumed) {
throw new Error("Pipeline resume state not found");
}
pipelineResumeStateClaimId = consumption.claimId;
if (consumption.signalAbortedAfterCommit) {
const restored = await restoreConsumedResumeState({
env: runtime.env,
key: payload.stateKey,
expectedState: resumeState,
claimId: consumption.claimId,
});
if (restored) {
pipelineResumeStateRestored = true;
pipelineResumeStateClaimId = undefined;
}
runtime.signal?.throwIfAborted();
}
runtime.signal?.throwIfAborted();
pipelineExecutionStarted = true;
},
});
const finalized = await finalizePipelineToolRun({
env: runtime.env,
pipeline: remaining,
output,
previousStateKey: payload.stateKey,
previousState: resumeState,
previousStateConsumed: pipelineExecutionStarted,
restorePreviousStateOnAbort: !pipelineExecutionStarted,
onPreviousStateRestored: () => {
pipelineResumeStateRestored = true;
},
signal: runtime.signal,
});
if (finalized.status === "ok" && pipelineExecutionStarted) await cleanupIndex();
return okEnvelope(
finalized.status,
finalized.output,
finalized.requiresApproval,
finalized.requiresInput,
);
} catch (err: any) {
const abortedResume = runtime.signal?.aborted === true;
if (
abortedResume &&
!pipelineExecutionStarted &&
!pipelineResumeStateRestored &&
pipelineResumeStateClaimId
) {
pipelineResumeStateRestored = await restoreConsumedResumeState({
env: runtime.env,
key: payload.stateKey,
expectedState: resumeState,
claimId: pipelineResumeStateClaimId,
}).catch(() => false);
}
if (pipelineExecutionStarted && !pipelineResumeStateRestored) {
if (abortedResume && !abortedBeforeResume) {
await deleteStateJsonWithBoundedResumeCleanup({
env: runtime.env,
key: payload.stateKey,
}).catch(() => {});
}
// Keep the short approval ID through the pre-dispatch claim window. Once
// the unsafe stage has actually been entered, the tombstone makes retry
// unsafe and the old index may be retired just as it was before this fix.
await cleanupIndex().catch(() => {});
}
// Non-abort failures and pre-aborted resumes remain retryable by token or approval ID.
return errorEnvelope("runtime_error", err?.message ?? String(err));
}
}
export function createToolContext(ctx: ToolRunContext = {}) {
return {
cwd: ctx.cwd ?? process.cwd(),
env: { ...process.env, ...ctx.env },
mode: "tool" as const,
stdin: ctx.stdin ?? process.stdin,
stdout: ctx.stdout ?? createCaptureStream(),
stderr: ctx.stderr ?? createCaptureStream(),
signal: ctx.signal,
registry: ctx.registry ?? createDefaultRegistry(),
llmAdapters: ctx.llmAdapters,
};
return {
cwd: ctx.cwd ?? process.cwd(),
env: { ...process.env, ...ctx.env },
mode: "tool" as const,
stdin: ctx.stdin ?? process.stdin,
stdout: ctx.stdout ?? createCaptureStream(),
stderr: ctx.stderr ?? createCaptureStream(),
signal: ctx.signal,
forceTerminationSignal: ctx.forceTerminationSignal,
registry: ctx.registry ?? createDefaultRegistry(),
llmAdapters: ctx.llmAdapters,
};
}
export function createCaptureStream() {
return new Writable({
write(_chunk, _encoding, callback) {
callback();
},
});
return new Writable({
write(_chunk, _encoding, callback) {
callback();
},
});
}
function okEnvelope(
status: "ok" | "needs_approval" | "needs_input" | "cancelled",
output: unknown[],
requiresApproval: ToolEnvelope["requiresApproval"],
requiresInput: ToolEnvelope["requiresInput"],
status: "ok" | "needs_approval" | "needs_input" | "cancelled",
output: unknown[],
requiresApproval: ToolEnvelope["requiresApproval"],
requiresInput: ToolEnvelope["requiresInput"],
) {
return {
protocolVersion: 1 as const,
ok: true,
status,
output,
requiresApproval,
requiresInput,
};
return {
protocolVersion: 1 as const,
ok: true,
status,
output,
requiresApproval,
requiresInput,
};
}
function errorEnvelope(type: string, message: string): ToolEnvelope {
return {
protocolVersion: 1,
ok: false,
error: { type, message },
};
return {
protocolVersion: 1,
ok: false,
error: { type, message },
};
}
async function resolveWorkflowFile(candidate: string, cwd: string) {
const { stat } = await import("node:fs/promises");
const resolved = path.isAbsolute(candidate) ? candidate : path.resolve(cwd, candidate);
const fileStat = await stat(resolved);
if (!fileStat.isFile()) throw new Error("Workflow path is not a file");
const ext = path.extname(resolved).toLowerCase();
if (![".lobster", ".yaml", ".yml", ".json"].includes(ext)) {
throw new Error("Workflow file must end in .lobster, .yaml, .yml, or .json");
}
return resolved;
const { stat } = await import("node:fs/promises");
const resolved = path.isAbsolute(candidate) ? candidate : path.resolve(cwd, candidate);
const fileStat = await stat(resolved);
if (!fileStat.isFile()) throw new Error("Workflow path is not a file");
const ext = path.extname(resolved).toLowerCase();
if (![".lobster", ".yaml", ".yml", ".json"].includes(ext)) {
throw new Error("Workflow file must end in .lobster, .yaml, .yml, or .json");
}
return resolved;
}
src/input_request.ts
+422 -418
View File
@@ -2,54 +2,54 @@ import { stableStringify } from "./state/store.js";
import { compileCached } from "./validation.js";
export type RequestInputParams = {
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
suspendedState?: unknown;
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
suspendedState?: unknown;
};
export type RequestInputMetadata = {
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
signature: string;
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
signature: string;
};
export type CommandInputHistoryEntry = {
requestIndex: number;
metadata: RequestInputMetadata;
suspendedState?: unknown;
response: unknown;
requestIndex: number;
metadata: RequestInputMetadata;
suspendedState?: unknown;
response: unknown;
};
export type CommandInputPendingRequest = {
requestIndex: number;
metadata: RequestInputMetadata;
suspendedState?: unknown;
requestIndex: number;
metadata: RequestInputMetadata;
suspendedState?: unknown;
};
export type CommandInputState = {
pending: CommandInputPendingRequest;
history: CommandInputHistoryEntry[];
pending: CommandInputPendingRequest;
history: CommandInputHistoryEntry[];
};
export type CommandInputResume = {
state: CommandInputState;
response: unknown;
consumed?: boolean;
onConsumed?: () => void | Promise<void>;
state: CommandInputState;
response: unknown;
consumed?: boolean;
onConsumed?: () => void | Promise<void>;
};
export type PipelineCommandInputRequest = {
type: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
items: unknown[];
commandInput: CommandInputState;
type: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
items: unknown[];
commandInput: CommandInputState;
};
const MAX_REPLAY_ITEMS = 1000;
@@ -57,471 +57,475 @@ const MAX_REPLAY_BYTES = 1024 * 1024;
const MAX_REQUEST_HISTORY = 100;
export class InputRequestSuspension extends Error {
stageIndex: number;
request: PipelineCommandInputRequest;
stageIndex: number;
request: PipelineCommandInputRequest;
constructor(stageIndex: number, request: PipelineCommandInputRequest) {
super("Input request suspended");
this.name = "InputRequestSuspension";
this.stageIndex = stageIndex;
this.request = request;
}
constructor(stageIndex: number, request: PipelineCommandInputRequest) {
super("Input request suspended");
this.name = "InputRequestSuspension";
this.stageIndex = stageIndex;
this.request = request;
}
}
export class RequestInputResumeError extends Error {
constructor(message: string) {
super(message);
this.name = "RequestInputResumeError";
}
constructor(message: string) {
super(message);
this.name = "RequestInputResumeError";
}
}
export function createInputTracker(input: AsyncIterable<unknown> | Iterable<unknown>) {
const knownItems = Array.isArray(input) ? input : null;
let iterator: AsyncIterator<unknown> | null = null;
let completed = false;
let closed = false;
let replayEnabled = true;
let replaySnapshotItems: unknown[] = [];
let replaySnapshotIndex = 0;
let replaySnapshotError: Error | null = null;
let replaySnapshotBytes = 0;
const knownItems = Array.isArray(input) ? input : null;
let iterator: AsyncIterator<unknown> | null = null;
let completed = false;
let closed = false;
let replayEnabled = true;
let replaySnapshotItems: unknown[] = [];
let replaySnapshotIndex = 0;
let replaySnapshotError: Error | null = null;
let replaySnapshotBytes = 0;
const iterable = {
async *[Symbol.asyncIterator]() {
const iter = getIterator();
let hasPrimaryError = false;
let yieldedIndex = 0;
try {
while (true) {
const next = await iter.next();
if (next.done) {
completed = true;
return;
}
if (knownItems) {
snapshotKnownItemsThrough(yieldedIndex);
yieldedIndex += 1;
}
yield next.value;
}
} catch (err) {
hasPrimaryError = true;
throw err;
} finally {
await closeTrackedIterator({ suppressErrors: hasPrimaryError });
}
},
};
const iterable = {
async *[Symbol.asyncIterator]() {
const iter = getIterator();
let hasPrimaryError = false;
let yieldedIndex = 0;
try {
while (true) {
const next = await iter.next();
if (next.done) {
completed = true;
return;
}
if (knownItems) {
snapshotKnownItemsThrough(yieldedIndex);
yieldedIndex += 1;
}
yield next.value;
}
} catch (err) {
hasPrimaryError = true;
throw err;
} finally {
await closeTrackedIterator({ suppressErrors: hasPrimaryError });
}
},
};
return {
iterable,
getReplayItems(hasSuspendedState: boolean) {
if (!replayEnabled) {
throw new Error("requestInput replay is no longer available after command output");
}
if (!knownItems) {
if (hasSuspendedState) return [];
throw new Error("requestInput requires suspendedState when command input is streaming");
}
snapshotKnownItemsThrough(knownItems.length - 1);
if (replaySnapshotError) throw replaySnapshotError;
return snapshotArray(replaySnapshotItems, "requestInput replay input");
},
disableReplay() {
replayEnabled = false;
},
async close(options: { suppressErrors?: boolean } = {}) {
await closeTrackedIterator({ suppressErrors: options.suppressErrors === true });
},
};
return {
iterable,
getReplayItems(hasSuspendedState: boolean) {
if (!replayEnabled) {
throw new Error("requestInput replay is no longer available after command output");
}
if (!knownItems) {
if (hasSuspendedState) return [];
throw new Error("requestInput requires suspendedState when command input is streaming");
}
snapshotKnownItemsThrough(knownItems.length - 1);
if (replaySnapshotError) throw replaySnapshotError;
return snapshotArray(replaySnapshotItems, "requestInput replay input");
},
disableReplay() {
replayEnabled = false;
},
async close(options: { suppressErrors?: boolean } = {}) {
await closeTrackedIterator({ suppressErrors: options.suppressErrors === true });
},
};
function getIterator() {
iterator ??= toAsyncIterator(input);
return iterator;
}
function getIterator() {
iterator ??= toAsyncIterator(input);
return iterator;
}
async function closeTrackedIterator({ suppressErrors }: { suppressErrors: boolean }) {
if (!iterator || completed || closed) return;
closed = true;
await closeIterator(iterator, { suppressErrors });
}
async function closeTrackedIterator({ suppressErrors }: { suppressErrors: boolean }) {
if (!iterator || completed || closed) return;
closed = true;
await closeIterator(iterator, { suppressErrors });
}
function snapshotKnownItemsThrough(index: number) {
if (!knownItems || replaySnapshotError) return;
while (replaySnapshotIndex <= index && replaySnapshotIndex < knownItems.length) {
try {
const snapshot = snapshotJson(knownItems[replaySnapshotIndex], "requestInput replay input");
replaySnapshotBytes += Buffer.byteLength(JSON.stringify(snapshot), "utf8");
if (
replaySnapshotItems.length + 1 > MAX_REPLAY_ITEMS ||
replaySnapshotBytes > MAX_REPLAY_BYTES
) {
throw new Error("requestInput replay limit exceeded");
}
replaySnapshotItems.push(snapshot);
replaySnapshotIndex += 1;
} catch (err) {
replaySnapshotError = err instanceof Error ? err : new Error(String(err));
return;
}
}
}
function snapshotKnownItemsThrough(index: number) {
if (!knownItems || replaySnapshotError) return;
while (replaySnapshotIndex <= index && replaySnapshotIndex < knownItems.length) {
try {
const snapshot = snapshotJson(knownItems[replaySnapshotIndex], "requestInput replay input");
replaySnapshotBytes += Buffer.byteLength(JSON.stringify(snapshot), "utf8");
if (
replaySnapshotItems.length + 1 > MAX_REPLAY_ITEMS ||
replaySnapshotBytes > MAX_REPLAY_BYTES
) {
throw new Error("requestInput replay limit exceeded");
}
replaySnapshotItems.push(snapshot);
replaySnapshotIndex += 1;
} catch (err) {
replaySnapshotError = err instanceof Error ? err : new Error(String(err));
return;
}
}
}
}
export function createStageRequestInput({
ctx,
stageIndex,
mode,
inputTracker,
isCommandActive,
getInactiveReason,
isOutputStarted,
resume,
ctx,
stageIndex,
mode,
inputTracker,
isCommandActive,
getInactiveReason,
isOutputStarted,
resume,
onResumedInput,
}: {
ctx: any;
stageIndex: number;
mode: string;
inputTracker: ReturnType<typeof createInputTracker>;
isCommandActive: () => boolean;
getInactiveReason?: () => string | undefined;
isOutputStarted: () => boolean;
resume?: CommandInputResume;
ctx: any;
stageIndex: number;
mode: string;
inputTracker: ReturnType<typeof createInputTracker>;
isCommandActive: () => boolean;
getInactiveReason?: () => string | undefined;
isOutputStarted: () => boolean;
resume?: CommandInputResume;
onResumedInput?: () => void | Promise<void>;
}) {
let requestIndex = 0;
const history: CommandInputHistoryEntry[] = [...(resume?.state.history ?? [])];
let requestIndex = 0;
const history: CommandInputHistoryEntry[] = [...(resume?.state.history ?? [])];
const requestInput = async function requestInput(params: RequestInputParams) {
if (!isCommandActive()) {
throw new Error(
getInactiveReason?.() ?? "requestInput cannot run after the command has completed",
);
}
const requestInput = async function requestInput(params: RequestInputParams) {
if (!isCommandActive()) {
throw new Error(
getInactiveReason?.() ?? "requestInput cannot run after the command has completed",
);
}
const metadata = snapshotRequestMetadata(params);
const requestedSuspendedState =
params.suspendedState === undefined
? undefined
: snapshotJson(params.suspendedState, "requestInput suspendedState");
const historical = history[requestIndex];
if (historical) {
assertMetadataMatches(historical.requestIndex, historical.metadata, requestIndex, metadata);
assertSuspendedStateMatches(historical.suspendedState, requestedSuspendedState);
const response = snapshotJson(historical.response, "requestInput response");
validateRequestInputResponse(metadata.responseSchema, response, "requestInput");
requestIndex += 1;
return response;
}
const metadata = snapshotRequestMetadata(params);
const requestedSuspendedState =
params.suspendedState === undefined
? undefined
: snapshotJson(params.suspendedState, "requestInput suspendedState");
const historical = history[requestIndex];
if (historical) {
assertMetadataMatches(historical.requestIndex, historical.metadata, requestIndex, metadata);
assertSuspendedStateMatches(historical.suspendedState, requestedSuspendedState);
const response = snapshotJson(historical.response, "requestInput response");
validateRequestInputResponse(metadata.responseSchema, response, "requestInput");
requestIndex += 1;
await onResumedInput?.();
return response;
}
if (resume && !resume.consumed) {
assertMetadataMatches(
resume.state.pending.requestIndex,
resume.state.pending.metadata,
requestIndex,
metadata,
);
assertSuspendedStateMatches(resume.state.pending.suspendedState, requestedSuspendedState);
const response = snapshotJson(resume.response, "requestInput response");
validateRequestInputResponse(metadata.responseSchema, response, "requestInput");
const historyResponse = snapshotJson(response, "requestInput response");
await resume.onConsumed?.();
resume.consumed = true;
history.push({
requestIndex,
metadata,
...(requestedSuspendedState !== undefined
? { suspendedState: requestedSuspendedState }
: null),
response: historyResponse,
});
requestIndex += 1;
return response;
}
if (resume && !resume.consumed) {
assertMetadataMatches(
resume.state.pending.requestIndex,
resume.state.pending.metadata,
requestIndex,
metadata,
);
assertSuspendedStateMatches(resume.state.pending.suspendedState, requestedSuspendedState);
const response = snapshotJson(resume.response, "requestInput response");
validateRequestInputResponse(metadata.responseSchema, response, "requestInput");
const historyResponse = snapshotJson(response, "requestInput response");
await resume.onConsumed?.();
resume.consumed = true;
history.push({
requestIndex,
metadata,
...(requestedSuspendedState !== undefined
? { suspendedState: requestedSuspendedState }
: null),
response: historyResponse,
});
requestIndex += 1;
await onResumedInput?.();
return response;
}
if (mode === "human" && isInteractive(ctx.stdin)) {
return requestInputInteractively(ctx, metadata);
}
if (mode === "human" && isInteractive(ctx.stdin)) {
return requestInputInteractively(ctx, metadata);
}
if (isOutputStarted()) {
throw new Error("requestInput cannot suspend after this command has produced output");
}
if (history.length >= MAX_REQUEST_HISTORY) {
throw new Error("requestInput replay history limit exceeded");
}
if (isOutputStarted()) {
throw new Error("requestInput cannot suspend after this command has produced output");
}
if (history.length >= MAX_REQUEST_HISTORY) {
throw new Error("requestInput replay history limit exceeded");
}
const items = inputTracker.getReplayItems(requestedSuspendedState !== undefined);
const pending: CommandInputPendingRequest = {
requestIndex,
metadata,
...(requestedSuspendedState !== undefined
? { suspendedState: requestedSuspendedState }
: null),
};
throw new InputRequestSuspension(stageIndex, {
type: "input_request",
prompt: metadata.prompt,
responseSchema: metadata.responseSchema,
...(metadata.defaults !== undefined ? { defaults: metadata.defaults } : null),
...(metadata.subject !== undefined ? { subject: metadata.subject } : null),
items,
commandInput: {
pending,
history,
},
});
};
requestInput.getSuspendedState = function getSuspendedState() {
if (requestIndex < history.length) {
return snapshotOptionalState(history[requestIndex].suspendedState);
}
if (resume && !resume.consumed && resume.state.pending.requestIndex === requestIndex) {
return snapshotOptionalState(resume.state.pending.suspendedState);
}
return undefined;
};
return requestInput;
const items = inputTracker.getReplayItems(requestedSuspendedState !== undefined);
const pending: CommandInputPendingRequest = {
requestIndex,
metadata,
...(requestedSuspendedState !== undefined
? { suspendedState: requestedSuspendedState }
: null),
};
throw new InputRequestSuspension(stageIndex, {
type: "input_request",
prompt: metadata.prompt,
responseSchema: metadata.responseSchema,
...(metadata.defaults !== undefined ? { defaults: metadata.defaults } : null),
...(metadata.subject !== undefined ? { subject: metadata.subject } : null),
items,
commandInput: {
pending,
history,
},
});
};
requestInput.getSuspendedState = function getSuspendedState() {
if (requestIndex < history.length) {
return snapshotOptionalState(history[requestIndex].suspendedState);
}
if (resume && !resume.consumed && resume.state.pending.requestIndex === requestIndex) {
return snapshotOptionalState(resume.state.pending.suspendedState);
}
return undefined;
};
return requestInput;
}
export function assertRequestInputResumeConsumed(resume?: CommandInputResume) {
if (resume && !resume.consumed) {
throw new RequestInputResumeError("resume input response was not consumed by requestInput");
}
if (resume && !resume.consumed) {
throw new RequestInputResumeError("resume input response was not consumed by requestInput");
}
}
export function snapshotRequestMetadata(params: RequestInputParams): RequestInputMetadata {
validateRequestInputParams(params);
const responseSchema = snapshotJson(params.responseSchema, "requestInput responseSchema");
const defaults =
params.defaults === undefined
? undefined
: snapshotJson(params.defaults, "requestInput defaults");
const subject =
params.subject === undefined ? undefined : snapshotJson(params.subject, "requestInput subject");
const unsigned = {
prompt: params.prompt,
responseSchema,
...(defaults !== undefined ? { defaults } : null),
...(subject !== undefined ? { subject } : null),
};
return {
...unsigned,
signature: stableStringify(unsigned),
};
validateRequestInputParams(params);
const responseSchema = snapshotJson(params.responseSchema, "requestInput responseSchema");
const defaults =
params.defaults === undefined
? undefined
: snapshotJson(params.defaults, "requestInput defaults");
const subject =
params.subject === undefined ? undefined : snapshotJson(params.subject, "requestInput subject");
const unsigned = {
prompt: params.prompt,
responseSchema,
...(defaults !== undefined ? { defaults } : null),
...(subject !== undefined ? { subject } : null),
};
return {
...unsigned,
signature: stableStringify(unsigned),
};
}
export function validateCommandInputState(value: unknown): CommandInputState {
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<CommandInputState>;
validatePending(data.pending);
if (!Array.isArray(data.history)) throw new Error("Invalid pipeline resume state");
if (data.history.length !== data.pending.requestIndex) {
throw new Error("Invalid pipeline resume state");
}
data.history.forEach(validateHistoryEntry);
return data as CommandInputState;
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<CommandInputState>;
validatePending(data.pending);
if (!Array.isArray(data.history)) throw new Error("Invalid pipeline resume state");
if (data.history.length !== data.pending.requestIndex) {
throw new Error("Invalid pipeline resume state");
}
data.history.forEach(validateHistoryEntry);
return data as CommandInputState;
}
export function validateRequestInputResponse(schema: unknown, response: unknown, label: string) {
let validator;
try {
validator = compileCached(schema as any);
} catch {
throw new Error(`${label} response schema is invalid`);
}
if (validator(response)) return;
const first = validator.errors?.[0];
const pathValue = first?.instancePath || "/";
const reason = first?.message ? ` ${first.message}` : "";
throw new Error(`${label} response failed schema validation at ${pathValue}:${reason}`);
let validator;
try {
validator = compileCached(schema as any);
} catch {
throw new Error(`${label} response schema is invalid`);
}
if (validator(response)) return;
const first = validator.errors?.[0];
const pathValue = first?.instancePath || "/";
const reason = first?.message ? ` ${first.message}` : "";
throw new Error(`${label} response failed schema validation at ${pathValue}:${reason}`);
}
function validateRequestInputParams(params: RequestInputParams) {
if (!params || typeof params !== "object") {
throw new Error("requestInput params must be an object");
}
if (typeof params.prompt !== "string" || params.prompt.length === 0) {
throw new Error("requestInput prompt is required");
}
if (params.responseSchema === undefined) {
throw new Error("requestInput responseSchema is required");
}
try {
compileCached(params.responseSchema as any);
} catch {
throw new Error("requestInput response schema is invalid");
}
if (!params || typeof params !== "object") {
throw new Error("requestInput params must be an object");
}
if (typeof params.prompt !== "string" || params.prompt.length === 0) {
throw new Error("requestInput prompt is required");
}
if (params.responseSchema === undefined) {
throw new Error("requestInput responseSchema is required");
}
try {
compileCached(params.responseSchema as any);
} catch {
throw new Error("requestInput response schema is invalid");
}
}
function validatePending(value: unknown): asserts value is CommandInputPendingRequest {
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<CommandInputPendingRequest>;
if (
typeof data.requestIndex !== "number" ||
!Number.isInteger(data.requestIndex) ||
data.requestIndex < 0
) {
throw new Error("Invalid pipeline resume state");
}
validateStoredMetadata(data.metadata);
if (data.suspendedState !== undefined) {
snapshotJson(data.suspendedState, "requestInput suspendedState");
}
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<CommandInputPendingRequest>;
if (
typeof data.requestIndex !== "number" ||
!Number.isInteger(data.requestIndex) ||
data.requestIndex < 0
) {
throw new Error("Invalid pipeline resume state");
}
validateStoredMetadata(data.metadata);
if (data.suspendedState !== undefined) {
snapshotJson(data.suspendedState, "requestInput suspendedState");
}
}
function validateHistoryEntry(value: unknown, index: number) {
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<CommandInputHistoryEntry>;
if (data.requestIndex !== index) throw new Error("Invalid pipeline resume state");
validateStoredMetadata(data.metadata);
if (data.suspendedState !== undefined) {
snapshotJson(data.suspendedState, "requestInput suspendedState");
}
if (data.response === undefined) throw new Error("Invalid pipeline resume state");
snapshotJson(data.response, "requestInput response");
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<CommandInputHistoryEntry>;
if (data.requestIndex !== index) throw new Error("Invalid pipeline resume state");
validateStoredMetadata(data.metadata);
if (data.suspendedState !== undefined) {
snapshotJson(data.suspendedState, "requestInput suspendedState");
}
if (data.response === undefined) throw new Error("Invalid pipeline resume state");
snapshotJson(data.response, "requestInput response");
}
function validateStoredMetadata(value: unknown): asserts value is RequestInputMetadata {
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<RequestInputMetadata>;
if (typeof data.prompt !== "string" || data.prompt.length === 0) {
throw new Error("Invalid pipeline resume state");
}
if (typeof data.signature !== "string" || data.signature.length === 0) {
throw new Error("Invalid pipeline resume state");
}
if (data.responseSchema === undefined) throw new Error("Invalid pipeline resume state");
const actual = snapshotRequestMetadata({
prompt: data.prompt,
responseSchema: data.responseSchema,
...(data.defaults !== undefined ? { defaults: data.defaults } : null),
...(data.subject !== undefined ? { subject: data.subject } : null),
});
if (actual.signature !== data.signature) throw new Error("Invalid pipeline resume state");
if (!value || typeof value !== "object") throw new Error("Invalid pipeline resume state");
const data = value as Partial<RequestInputMetadata>;
if (typeof data.prompt !== "string" || data.prompt.length === 0) {
throw new Error("Invalid pipeline resume state");
}
if (typeof data.signature !== "string" || data.signature.length === 0) {
throw new Error("Invalid pipeline resume state");
}
if (data.responseSchema === undefined) throw new Error("Invalid pipeline resume state");
const actual = snapshotRequestMetadata({
prompt: data.prompt,
responseSchema: data.responseSchema,
...(data.defaults !== undefined ? { defaults: data.defaults } : null),
...(data.subject !== undefined ? { subject: data.subject } : null),
});
if (actual.signature !== data.signature) throw new Error("Invalid pipeline resume state");
}
function assertMetadataMatches(
storedIndex: number,
stored: RequestInputMetadata,
actualIndex: number,
actual: RequestInputMetadata,
storedIndex: number,
stored: RequestInputMetadata,
actualIndex: number,
actual: RequestInputMetadata,
) {
if (
storedIndex !== actualIndex ||
stored.prompt !== actual.prompt ||
stored.signature !== actual.signature
) {
throw new RequestInputResumeError(
"requestInput resume request does not match suspended request",
);
}
if (
storedIndex !== actualIndex ||
stored.prompt !== actual.prompt ||
stored.signature !== actual.signature
) {
throw new RequestInputResumeError(
"requestInput resume request does not match suspended request",
);
}
}
function assertSuspendedStateMatches(stored: unknown, actual: unknown) {
if (stableStringify(stored) !== stableStringify(actual)) {
throw new RequestInputResumeError("requestInput resume state does not match suspended request");
}
if (stableStringify(stored) !== stableStringify(actual)) {
throw new RequestInputResumeError("requestInput resume state does not match suspended request");
}
}
function snapshotOptionalState(state: unknown) {
return state === undefined ? undefined : snapshotJson(state, "requestInput suspendedState");
return state === undefined ? undefined : snapshotJson(state, "requestInput suspendedState");
}
function snapshotArray(items: readonly unknown[], label: string) {
if (items.length > MAX_REPLAY_ITEMS) throw new Error("requestInput replay item limit exceeded");
let totalBytes = 0;
return items.map((item) => {
const snapshot = snapshotJson(item, label);
totalBytes += Buffer.byteLength(JSON.stringify(snapshot), "utf8");
if (totalBytes > MAX_REPLAY_BYTES) {
throw new Error("requestInput replay byte limit exceeded");
}
return snapshot;
});
if (items.length > MAX_REPLAY_ITEMS) throw new Error("requestInput replay item limit exceeded");
let totalBytes = 0;
return items.map((item) => {
const snapshot = snapshotJson(item, label);
totalBytes += Buffer.byteLength(JSON.stringify(snapshot), "utf8");
if (totalBytes > MAX_REPLAY_BYTES) {
throw new Error("requestInput replay byte limit exceeded");
}
return snapshot;
});
}
function snapshotJson(value: unknown, label: string): unknown {
assertJsonSerializable(value, label, new WeakSet());
const text = JSON.stringify(value);
if (text === undefined) throw new Error(`${label} must be JSON-serializable`);
return JSON.parse(text);
assertJsonSerializable(value, label, new WeakSet());
const text = JSON.stringify(value);
if (text === undefined) throw new Error(`${label} must be JSON-serializable`);
return JSON.parse(text);
}
function assertJsonSerializable(value: unknown, label: string, seen: WeakSet<object>) {
if (value === null) return;
const type = typeof value;
if (type === "string" || type === "boolean") return;
if (type === "number") {
if (!Number.isFinite(value)) throw new Error(`${label} must be JSON-serializable`);
return;
}
if (type === "undefined" || type === "function" || type === "symbol" || type === "bigint") {
throw new Error(`${label} must be JSON-serializable`);
}
const object = value as object;
if (seen.has(object)) throw new Error(`${label} must be JSON-serializable`);
const prototype = Object.getPrototypeOf(object);
if (prototype !== Object.prototype && prototype !== null && !Array.isArray(value)) {
throw new Error(`${label} must be JSON-serializable`);
}
seen.add(object);
if (Array.isArray(value)) {
for (let index = 0; index < value.length; index++) {
if (!(index in value)) throw new Error(`${label} must be JSON-serializable`);
assertJsonSerializable(value[index], label, seen);
}
} else {
for (const key of Object.keys(value as Record<string, unknown>)) {
assertJsonSerializable((value as Record<string, unknown>)[key], label, seen);
}
}
seen.delete(object);
if (value === null) return;
const type = typeof value;
if (type === "string" || type === "boolean") return;
if (type === "number") {
if (!Number.isFinite(value)) throw new Error(`${label} must be JSON-serializable`);
return;
}
if (type === "undefined" || type === "function" || type === "symbol" || type === "bigint") {
throw new Error(`${label} must be JSON-serializable`);
}
const object = value as object;
if (seen.has(object)) throw new Error(`${label} must be JSON-serializable`);
const prototype = Object.getPrototypeOf(object);
if (prototype !== Object.prototype && prototype !== null && !Array.isArray(value)) {
throw new Error(`${label} must be JSON-serializable`);
}
seen.add(object);
if (Array.isArray(value)) {
for (let index = 0; index < value.length; index++) {
if (!(index in value)) throw new Error(`${label} must be JSON-serializable`);
assertJsonSerializable(value[index], label, seen);
}
} else {
for (const key of Object.keys(value as Record<string, unknown>)) {
assertJsonSerializable((value as Record<string, unknown>)[key], label, seen);
}
}
seen.delete(object);
}
async function requestInputInteractively(ctx: any, metadata: RequestInputMetadata) {
ctx.stdout.write(`${metadata.prompt}\n> `);
const { readLineFromStream } = await import("./read_line.js");
const raw = await readLineFromStream(ctx.stdin, { timeoutMs: 0 });
let response;
try {
response = JSON.parse(String(raw ?? "").trim());
} catch {
throw new Error("requestInput response must be valid JSON");
}
validateRequestInputResponse(metadata.responseSchema, response, "requestInput");
return snapshotJson(response, "requestInput response");
ctx.stdout.write(`${metadata.prompt}\n> `);
const { readLineFromStream } = await import("./read_line.js");
const raw = await readLineFromStream(ctx.stdin, { timeoutMs: 0, signal: ctx.signal });
let response;
try {
response = JSON.parse(String(raw ?? "").trim());
} catch {
throw new Error("requestInput response must be valid JSON");
}
validateRequestInputResponse(metadata.responseSchema, response, "requestInput");
return snapshotJson(response, "requestInput response");
}
function isInteractive(stdin: any) {
return Boolean(stdin?.isTTY);
return Boolean(stdin?.isTTY);
}
async function closeIterator(
iterator: AsyncIterator<unknown>,
{ suppressErrors }: { suppressErrors: boolean },
iterator: AsyncIterator<unknown>,
{ suppressErrors }: { suppressErrors: boolean },
) {
if (typeof iterator.return !== "function") return;
try {
await iterator.return();
} catch (err) {
if (!suppressErrors) throw err;
// Cleanup must not mask the original command error or suspension.
}
if (typeof iterator.return !== "function") return;
try {
await iterator.return();
} catch (err) {
if (!suppressErrors) throw err;
// Cleanup must not mask the original command error or suspension.
}
}
function toAsyncIterator(input: AsyncIterable<unknown> | Iterable<unknown>) {
if (typeof (input as any)[Symbol.asyncIterator] === "function") {
return (input as AsyncIterable<unknown>)[Symbol.asyncIterator]();
}
if (typeof (input as any)[Symbol.iterator] === "function") {
const iterator = (input as Iterable<unknown>)[Symbol.iterator]();
return {
async next() {
return iterator.next();
},
async return() {
if (typeof iterator.return === "function") iterator.return();
return { done: true, value: undefined };
},
};
}
throw new Error("input is not iterable");
if (typeof (input as any)[Symbol.asyncIterator] === "function") {
return (input as AsyncIterable<unknown>)[Symbol.asyncIterator]();
}
if (typeof (input as any)[Symbol.iterator] === "function") {
const iterator = (input as Iterable<unknown>)[Symbol.iterator]();
return {
async next() {
return iterator.next();
},
async return() {
if (typeof iterator.return === "function") iterator.return();
return { done: true, value: undefined };
},
};
}
throw new Error("input is not iterable");
}
src/parser.ts
+128 -128
View File
@@ -1,164 +1,164 @@
function isWhitespace(ch) {
return ch === " " || ch === "\t" || ch === "\n" || ch === "\r";
return ch === " " || ch === "\t" || ch === "\n" || ch === "\r";
}
function splitPipes(input) {
const parts = [];
let current = "";
let quote = null;
const parts = [];
let current = "";
let quote = null;
for (let i = 0; i < input.length; i++) {
const ch = input[i];
for (let i = 0; i < input.length; i++) {
const ch = input[i];
if (quote) {
if (ch === "\\") {
const next = input[i + 1];
if (next) {
current += ch + next;
i++;
continue;
}
}
current += ch;
if (ch === quote) {
quote = null;
}
continue;
}
if (quote) {
if (ch === "\\") {
const next = input[i + 1];
if (next) {
current += ch + next;
i++;
continue;
}
}
current += ch;
if (ch === quote) {
quote = null;
}
continue;
}
if (ch === '"' || ch === "'") {
quote = ch;
current += ch;
continue;
}
if (ch === '"' || ch === "'") {
quote = ch;
current += ch;
continue;
}
if (ch === "|") {
parts.push(current.trim());
current = "";
continue;
}
if (ch === "|") {
parts.push(current.trim());
current = "";
continue;
}
current += ch;
}
current += ch;
}
if (quote) throw new Error("Unclosed quote");
if (current.trim().length > 0) parts.push(current.trim());
return parts;
if (quote) throw new Error("Unclosed quote");
if (current.trim().length > 0) parts.push(current.trim());
return parts;
}
function tokenizeCommand(input) {
const tokens = [];
let current = "";
let quote = null;
const tokens = [];
let current = "";
let quote = null;
const push = () => {
if (current.length > 0) tokens.push(current);
current = "";
};
const push = () => {
if (current.length > 0) tokens.push(current);
current = "";
};
for (let i = 0; i < input.length; i++) {
const ch = input[i];
for (let i = 0; i < input.length; i++) {
const ch = input[i];
if (quote) {
if (quote === "'") {
if (ch === "\\" && input[i + 1] === quote) {
current += quote;
i++;
continue;
}
if (ch === quote) {
quote = null;
continue;
}
current += ch;
continue;
}
if (quote) {
if (quote === "'") {
if (ch === "\\" && input[i + 1] === quote) {
current += quote;
i++;
continue;
}
if (ch === quote) {
quote = null;
continue;
}
current += ch;
continue;
}
// Double-quoted mode: preserve unknown escapes (\n, \t, etc) while
// unescaping only shell-like quote/backslash escapes.
if (ch === "\\") {
const next = input[i + 1];
if (next === '"' || next === "\\" || next === "$" || next === "`") {
current += next;
i++;
continue;
}
if (next === "\n") {
i++;
continue;
}
current += ch;
continue;
}
// Double-quoted mode: preserve unknown escapes (\n, \t, etc) while
// unescaping only shell-like quote/backslash escapes.
if (ch === "\\") {
const next = input[i + 1];
if (next === '"' || next === "\\" || next === "$" || next === "`") {
current += next;
i++;
continue;
}
if (next === "\n") {
i++;
continue;
}
current += ch;
continue;
}
if (ch === quote) {
quote = null;
continue;
}
if (ch === quote) {
quote = null;
continue;
}
current += ch;
continue;
}
current += ch;
continue;
}
if (ch === '"' || ch === "'") {
quote = ch;
continue;
}
if (ch === '"' || ch === "'") {
quote = ch;
continue;
}
if (isWhitespace(ch)) {
push();
continue;
}
if (isWhitespace(ch)) {
push();
continue;
}
current += ch;
}
current += ch;
}
if (quote) throw new Error("Unclosed quote");
push();
return tokens;
if (quote) throw new Error("Unclosed quote");
push();
return tokens;
}
function parseArgs(tokens) {
const args = { _: [] };
const args = { _: [] };
for (let i = 0; i < tokens.length; i++) {
const tok = tokens[i];
for (let i = 0; i < tokens.length; i++) {
const tok = tokens[i];
if (tok.startsWith("--")) {
const eq = tok.indexOf("=");
if (eq !== -1) {
const key = tok.slice(2, eq);
const value = tok.slice(eq + 1);
args[key] = value;
continue;
}
if (tok.startsWith("--")) {
const eq = tok.indexOf("=");
if (eq !== -1) {
const key = tok.slice(2, eq);
const value = tok.slice(eq + 1);
args[key] = value;
continue;
}
const key = tok.slice(2);
const next = tokens[i + 1];
if (!next || next.startsWith("--")) {
args[key] = true;
continue;
}
args[key] = next;
i++;
continue;
}
const key = tok.slice(2);
const next = tokens[i + 1];
if (!next || next.startsWith("--")) {
args[key] = true;
continue;
}
args[key] = next;
i++;
continue;
}
args._.push(tok);
}
args._.push(tok);
}
return args;
return args;
}
export function parsePipeline(input) {
const stages = splitPipes(input);
if (stages.length === 0) throw new Error("Empty pipeline");
const stages = splitPipes(input);
if (stages.length === 0) throw new Error("Empty pipeline");
return stages.map((stage) => {
const tokens = tokenizeCommand(stage);
if (tokens.length === 0) throw new Error("Empty command stage");
const name = tokens[0];
const args = parseArgs(tokens.slice(1));
return { name, args, raw: stage };
});
return stages.map((stage) => {
const tokens = tokenizeCommand(stage);
if (tokens.length === 0) throw new Error("Empty command stage");
const name = tokens[0];
const args = parseArgs(tokens.slice(1));
return { name, args, raw: stage };
});
}
src/pipeline_resume_state.ts
+462 -235
View File
@@ -2,278 +2,505 @@ import { randomUUID } from "node:crypto";
import { encodeToken } from "./token.js";
import {
cleanupApprovalIndexByStateKey,
createApprovalIndex,
deleteStateJson,
readStateJson,
writeStateJson,
cleanupApprovalIndexByStateKey,
consumeResumeState,
createApprovalIndex,
deleteResumeStateWithRollback,
deleteStateJson,
isConsumedResumeState,
readStateJsonWithLock,
restoreConsumedResumeState,
writeStateJson,
} from "./state/store.js";
import { compileCached } from "./validation.js";
import { validateCommandInputState, type CommandInputState } from "./input_request.js";
export type PipelineResumeState = {
pipeline: Array<{ name: string; args: Record<string, unknown>; raw: string }>;
resumeAtIndex: number;
items: unknown[];
haltType?: "approval_request" | "input_request";
resumeMode?: "next_stage" | "same_stage";
inputSchema?: unknown;
prompt?: string;
commandInput?: CommandInputState;
createdAt: string;
pipeline: Array<{ name: string; args: Record<string, unknown>; raw: string }>;
resumeAtIndex: number;
items: unknown[];
haltType?: "approval_request" | "input_request";
resumeMode?: "next_stage" | "same_stage";
inputSchema?: unknown;
prompt?: string;
commandInput?: CommandInputState;
supersededResumeStateKeys?: string[];
createdAt: string;
};
export type PipelineApprovalRequest = {
type: "approval_request";
prompt: string;
items: unknown[];
preview?: string;
type: "approval_request";
prompt: string;
items: unknown[];
preview?: string;
};
export type PipelineInputRequest = {
type: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
items?: unknown[];
commandInput?: CommandInputState;
type: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
items?: unknown[];
commandInput?: CommandInputState;
};
export type PipelineRunOutput = {
items: unknown[];
halted?: boolean;
haltedAt?: { index: number } | null;
items: unknown[];
halted?: boolean;
haltedAt?: { index: number } | null;
executionStarted?: boolean;
};
export type PipelineToolRunResolution =
| {
status: "needs_approval";
output: [];
requiresApproval: {
type: "approval_request";
prompt: string;
items: unknown[];
preview?: string;
resumeToken: string;
approvalId?: string;
};
requiresInput: null;
}
| {
status: "needs_input";
output: [];
requiresApproval: null;
requiresInput: {
type: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
resumeToken: string;
};
}
| {
status: "ok";
output: unknown[];
requiresApproval: null;
requiresInput: null;
};
| {
status: "needs_approval";
output: [];
requiresApproval: {
type: "approval_request";
prompt: string;
items: unknown[];
preview?: string;
resumeToken: string;
approvalId?: string;
};
requiresInput: null;
}
| {
status: "needs_input";
output: [];
requiresApproval: null;
requiresInput: {
type: "input_request";
prompt: string;
responseSchema: unknown;
defaults?: unknown;
subject?: unknown;
resumeToken: string;
};
}
| {
status: "ok";
output: unknown[];
requiresApproval: null;
requiresInput: null;
};
export function extractPipelineHalt(output: { halted?: boolean; items: unknown[] }) {
const halted =
output.halted && output.items.length === 1
? (output.items[0] as Record<string, unknown>)
: null;
const approval =
halted?.type === "approval_request" ? (halted as unknown as PipelineApprovalRequest) : null;
const inputRequest =
halted?.type === "input_request" ? (halted as unknown as PipelineInputRequest) : null;
return { approval, inputRequest };
const halted =
output.halted && output.items.length === 1
? (output.items[0] as Record<string, unknown>)
: null;
const approval =
halted?.type === "approval_request" ? (halted as unknown as PipelineApprovalRequest) : null;
const inputRequest =
halted?.type === "input_request" ? (halted as unknown as PipelineInputRequest) : null;
return { approval, inputRequest };
}
export async function finalizePipelineToolRun(params: {
env: Record<string, string | undefined>;
pipeline: PipelineResumeState["pipeline"];
output: PipelineRunOutput;
previousStateKey?: string;
env: Record<string, string | undefined>;
pipeline: PipelineResumeState["pipeline"];
output: PipelineRunOutput;
previousStateKey?: string;
previousState?: PipelineResumeState;
previousStateConsumed?: boolean;
restorePreviousStateOnAbort?: boolean;
onPreviousStateRestored?: () => void;
signal?: AbortSignal;
}): Promise<PipelineToolRunResolution> {
const { approval, inputRequest } = extractPipelineHalt(params.output);
if (approval) {
const nextStateKey = await savePipelineResumeState(params.env, {
pipeline: params.pipeline,
resumeAtIndex: (params.output.haltedAt?.index ?? -1) + 1,
items: approval.items,
haltType: "approval_request",
prompt: approval.prompt,
createdAt: new Date().toISOString(),
});
if (params.previousStateKey) {
await cleanupApprovalIndexByStateKey({ env: params.env, stateKey: params.previousStateKey });
await deleteStateJson({ env: params.env, key: params.previousStateKey });
}
let approvalId: string | null;
try {
approvalId = await createApprovalIndex({ env: params.env, stateKey: nextStateKey });
} catch (err) {
await deleteStateJson({ env: params.env, key: nextStateKey }).catch(() => {});
throw err;
}
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
kind: "pipeline-resume",
stateKey: nextStateKey,
});
return {
status: "needs_approval",
output: [],
requiresApproval: {
...approval,
resumeToken,
...(approvalId ? { approvalId } : null),
},
requiresInput: null,
};
}
params.signal?.throwIfAborted();
const { approval, inputRequest } = extractPipelineHalt(params.output);
if (approval) {
let nextStateKey: string | undefined;
try {
nextStateKey = await savePipelineResumeState(
params.env,
{
pipeline: params.pipeline,
resumeAtIndex: (params.output.haltedAt?.index ?? -1) + 1,
items: approval.items,
haltType: "approval_request",
prompt: approval.prompt,
supersededResumeStateKeys: collectSupersededPipelineResumeStateKeys(
params.previousStateKey,
params.previousState,
),
createdAt: new Date().toISOString(),
},
params.signal,
);
let approvalId: string | null;
approvalId = await createApprovalIndex({ env: params.env, stateKey: nextStateKey });
const replaced = await replacePipelineResumeState({
env: params.env,
previousStateKey: params.previousStateKey,
expectedPreviousState: params.previousState,
previousStateConsumed: params.previousStateConsumed,
replacementStateKey: nextStateKey,
signal: params.signal,
});
if (!replaced) throw new Error("Pipeline resume state not found");
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
kind: "pipeline-resume",
stateKey: nextStateKey,
});
await retirePreviousPipelineApprovalIndex(params.env, params.previousStateKey, nextStateKey);
return {
status: "needs_approval",
output: [],
requiresApproval: {
...approval,
resumeToken,
...(approvalId ? { approvalId } : null),
},
requiresInput: null,
};
} catch (err) {
try {
if (!params.previousStateConsumed) await restorePreviousPipelineResumeState(params);
} finally {
if (nextStateKey) await discardPipelineResumeState(params.env, nextStateKey);
}
throw err;
}
}
if (inputRequest) {
const resumeMode = inputRequest.commandInput ? "same_stage" : "next_stage";
const nextStateKey = await savePipelineResumeState(params.env, {
pipeline: params.pipeline,
resumeAtIndex:
resumeMode === "same_stage"
? (params.output.haltedAt?.index ?? -1)
: (params.output.haltedAt?.index ?? -1) + 1,
items: resumeMode === "same_stage" ? (inputRequest.items ?? []) : [],
haltType: "input_request",
resumeMode,
inputSchema: inputRequest.responseSchema,
prompt: inputRequest.prompt,
...(inputRequest.commandInput ? { commandInput: inputRequest.commandInput } : null),
createdAt: new Date().toISOString(),
});
if (params.previousStateKey) {
await cleanupApprovalIndexByStateKey({ env: params.env, stateKey: params.previousStateKey });
await deleteStateJson({ env: params.env, key: params.previousStateKey });
}
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
kind: "pipeline-resume",
stateKey: nextStateKey,
});
return {
status: "needs_input",
output: [],
requiresApproval: null,
requiresInput: {
type: "input_request",
prompt: inputRequest.prompt,
responseSchema: inputRequest.responseSchema,
...(inputRequest.defaults !== undefined ? { defaults: inputRequest.defaults } : null),
...(inputRequest.subject !== undefined ? { subject: inputRequest.subject } : null),
resumeToken,
},
};
}
if (inputRequest) {
const resumeMode = inputRequest.commandInput ? "same_stage" : "next_stage";
let nextStateKey: string | undefined;
try {
nextStateKey = await savePipelineResumeState(
params.env,
{
pipeline: params.pipeline,
resumeAtIndex:
resumeMode === "same_stage"
? (params.output.haltedAt?.index ?? -1)
: (params.output.haltedAt?.index ?? -1) + 1,
items: resumeMode === "same_stage" ? (inputRequest.items ?? []) : [],
haltType: "input_request",
resumeMode,
inputSchema: inputRequest.responseSchema,
prompt: inputRequest.prompt,
...(inputRequest.commandInput ? { commandInput: inputRequest.commandInput } : null),
supersededResumeStateKeys: collectSupersededPipelineResumeStateKeys(
params.previousStateKey,
params.previousState,
),
createdAt: new Date().toISOString(),
},
params.signal,
);
const replaced = await replacePipelineResumeState({
env: params.env,
previousStateKey: params.previousStateKey,
expectedPreviousState: params.previousState,
previousStateConsumed: params.previousStateConsumed,
replacementStateKey: nextStateKey,
signal: params.signal,
});
if (!replaced) throw new Error("Pipeline resume state not found");
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
kind: "pipeline-resume",
stateKey: nextStateKey,
});
await retirePreviousPipelineApprovalIndex(params.env, params.previousStateKey, nextStateKey);
return {
status: "needs_input",
output: [],
requiresApproval: null,
requiresInput: {
type: "input_request",
prompt: inputRequest.prompt,
responseSchema: inputRequest.responseSchema,
...(inputRequest.defaults !== undefined ? { defaults: inputRequest.defaults } : null),
...(inputRequest.subject !== undefined ? { subject: inputRequest.subject } : null),
resumeToken,
},
};
} catch (err) {
try {
if (!params.previousStateConsumed) await restorePreviousPipelineResumeState(params);
} finally {
if (nextStateKey) await discardPipelineResumeState(params.env, nextStateKey);
}
throw err;
}
}
if (params.previousStateKey) {
await cleanupApprovalIndexByStateKey({ env: params.env, stateKey: params.previousStateKey });
await deleteStateJson({ env: params.env, key: params.previousStateKey });
}
return {
status: "ok",
output: params.output.items,
requiresApproval: null,
requiresInput: null,
};
params.signal?.throwIfAborted();
if (params.previousStateKey) {
try {
if (params.previousStateConsumed) {
await deleteStateJson({
env: params.env,
key: params.previousStateKey,
signal: params.signal,
});
params.signal?.throwIfAborted();
} else {
const deleted = await deleteResumeStateWithRollback({
env: params.env,
key: params.previousStateKey,
expectedState: params.previousState,
signal: params.signal,
});
if (!deleted) throw new Error("Pipeline resume state not found");
}
await cleanupSupersededPipelineResumeStates(
params.env,
params.previousState?.supersededResumeStateKeys,
);
} catch (err) {
if (!params.previousStateConsumed) await restorePreviousPipelineResumeState(params);
throw err;
}
}
return {
status: "ok",
output: params.output.items,
requiresApproval: null,
requiresInput: null,
};
}
export async function savePipelineResumeState(
env: Record<string, string | undefined>,
state: PipelineResumeState,
env: Record<string, string | undefined>,
state: PipelineResumeState,
signal?: AbortSignal,
) {
const stateKey = `pipeline_resume_${randomUUID()}`;
await writeStateJson({ env, key: stateKey, value: state });
return stateKey;
const stateKey = `pipeline_resume_${randomUUID()}`;
try {
signal?.throwIfAborted();
await writeStateJson({ env, key: stateKey, value: state, signal });
signal?.throwIfAborted();
return stateKey;
} catch (err) {
if (signal?.aborted) await discardPipelineResumeState(env, stateKey);
throw err;
}
}
async function replacePipelineResumeState({
env,
previousStateKey,
expectedPreviousState,
previousStateConsumed,
replacementStateKey,
signal,
}: {
env: Record<string, string | undefined>;
previousStateKey?: string;
expectedPreviousState?: PipelineResumeState;
previousStateConsumed?: boolean;
replacementStateKey: string;
signal?: AbortSignal;
}) {
if (!previousStateKey || previousStateKey === replacementStateKey) {
signal?.throwIfAborted();
return true;
}
// The current resume has already crossed an unsafe boundary and owns the
// predecessor's consumed marker. It may safely publish the next gate, but
// must retain that marker rather than attempting a stale snapshot CAS.
if (previousStateConsumed) {
signal?.throwIfAborted();
return true;
}
if (!expectedPreviousState) return false;
let claimId: string | undefined;
try {
const consumption = await consumeResumeState({
env,
key: previousStateKey,
expectedState: expectedPreviousState,
signal,
});
if (!consumption.consumed) return false;
claimId = consumption.claimId;
// The predecessor remains as a durable tombstone until terminal cleanup.
// A concurrent caller can therefore never turn the same approval into a
// second successor capability.
signal?.throwIfAborted();
return true;
} catch (err) {
// A cancellation after the atomic marker publication has not exposed the
// successor token yet. Restore only the marker created by this caller so a
// competing transition can never be overwritten.
if (claimId && signal?.aborted) {
await restoreConsumedResumeState({
env,
key: previousStateKey,
expectedState: expectedPreviousState,
claimId,
}).catch(() => {});
}
throw err;
}
}
async function restorePreviousPipelineResumeState({
env,
previousStateKey,
previousState,
restorePreviousStateOnAbort,
onPreviousStateRestored,
signal,
}: {
env: Record<string, string | undefined>;
previousStateKey?: string;
previousState?: PipelineResumeState;
restorePreviousStateOnAbort?: boolean;
onPreviousStateRestored?: () => void;
signal?: AbortSignal;
}) {
if (!signal?.aborted || !restorePreviousStateOnAbort || !previousStateKey || !previousState) {
return;
}
// Safe terminal cleanup restores its own claimed marker while still holding
// the state lock. Never recreate a missing snapshot here: this caller may
// have only observed it before another resume completed.
if ((await readStateJsonWithLock({ env, key: previousStateKey, signal })) !== null) {
onPreviousStateRestored?.();
}
}
async function retirePreviousPipelineApprovalIndex(
env: Record<string, string | undefined>,
previousStateKey: string | undefined,
replacementStateKey: string,
) {
if (!previousStateKey || previousStateKey === replacementStateKey) return;
// This runs only after replacement deletion has passed its cancellation
// checkpoint. Do not add a later cancellation check: the transition is
// committed once the old approval capability is retired.
await cleanupApprovalIndexByStateKey({ env, stateKey: previousStateKey }).catch(() => {});
}
function collectSupersededPipelineResumeStateKeys(
previousStateKey: string | undefined,
previousState: PipelineResumeState | undefined,
) {
return [
...(previousState?.supersededResumeStateKeys ?? []),
...(previousStateKey ? [previousStateKey] : []),
].filter((stateKey, index, all) => stateKey && all.indexOf(stateKey) === index);
}
async function cleanupSupersededPipelineResumeStates(
env: Record<string, string | undefined>,
stateKeys: string[] | undefined,
) {
for (const stateKey of stateKeys ?? []) {
try {
// Retire only a non-executable marker after the successor itself has
// committed. A restored state must remain available for retry.
if (isConsumedResumeState(await readStateJsonWithLock({ env, key: stateKey }))) {
await deleteStateJson({ env, key: stateKey });
}
} catch {
// Leaving a tombstone is safe if best-effort cleanup cannot complete.
}
}
}
async function discardPipelineResumeState(
env: Record<string, string | undefined>,
stateKey: string,
) {
await cleanupApprovalIndexByStateKey({ env, stateKey }).catch(() => {});
await deleteStateJson({ env, key: stateKey }).catch(() => {});
}
export async function loadPipelineResumeState(
env: Record<string, string | undefined>,
stateKey: string,
env: Record<string, string | undefined>,
stateKey: string,
signal?: AbortSignal,
) {
const stored = await readStateJson({ env, key: stateKey });
if (!stored || typeof stored !== "object") {
throw new Error("Pipeline resume state not found");
}
const data = stored as Partial<PipelineResumeState>;
if (!Array.isArray(data.pipeline)) throw new Error("Invalid pipeline resume state");
validatePipelineShape(data.pipeline);
if (
typeof data.resumeAtIndex !== "number" ||
!Number.isInteger(data.resumeAtIndex) ||
data.resumeAtIndex < 0 ||
data.resumeAtIndex > data.pipeline.length
) {
throw new Error("Invalid pipeline resume state");
}
if (!Array.isArray(data.items)) throw new Error("Invalid pipeline resume state");
if (
data.haltType !== undefined &&
!["approval_request", "input_request"].includes(data.haltType)
) {
throw new Error("Invalid pipeline resume state");
}
if (data.resumeMode !== undefined && !["next_stage", "same_stage"].includes(data.resumeMode)) {
throw new Error("Invalid pipeline resume state");
}
if (data.haltType === "input_request") {
if (data.inputSchema === undefined || typeof data.prompt !== "string") {
throw new Error("Invalid pipeline resume state");
}
if (data.resumeMode === "same_stage") {
if (data.resumeAtIndex >= data.pipeline.length) {
throw new Error("Invalid pipeline resume state");
}
data.commandInput = validateCommandInputState(data.commandInput);
} else if (data.commandInput !== undefined) {
throw new Error("Invalid pipeline resume state");
}
} else if (data.resumeMode === "same_stage" || data.commandInput !== undefined) {
throw new Error("Invalid pipeline resume state");
}
return data as PipelineResumeState;
const stored = await readStateJsonWithLock({ env, key: stateKey, signal });
if (!stored || typeof stored !== "object" || isConsumedResumeState(stored)) {
throw new Error("Pipeline resume state not found");
}
const data = stored as Partial<PipelineResumeState>;
if (!Array.isArray(data.pipeline)) throw new Error("Invalid pipeline resume state");
validatePipelineShape(data.pipeline);
if (
typeof data.resumeAtIndex !== "number" ||
!Number.isInteger(data.resumeAtIndex) ||
data.resumeAtIndex < 0 ||
data.resumeAtIndex > data.pipeline.length
) {
throw new Error("Invalid pipeline resume state");
}
if (!Array.isArray(data.items)) throw new Error("Invalid pipeline resume state");
if (
data.supersededResumeStateKeys !== undefined &&
(!Array.isArray(data.supersededResumeStateKeys) ||
data.supersededResumeStateKeys.some((stateKey) => typeof stateKey !== "string"))
) {
throw new Error("Invalid pipeline resume state");
}
if (
data.haltType !== undefined &&
!["approval_request", "input_request"].includes(data.haltType)
) {
throw new Error("Invalid pipeline resume state");
}
if (data.resumeMode !== undefined && !["next_stage", "same_stage"].includes(data.resumeMode)) {
throw new Error("Invalid pipeline resume state");
}
if (data.haltType === "input_request") {
if (data.inputSchema === undefined || typeof data.prompt !== "string") {
throw new Error("Invalid pipeline resume state");
}
if (data.resumeMode === "same_stage") {
if (data.resumeAtIndex >= data.pipeline.length) {
throw new Error("Invalid pipeline resume state");
}
data.commandInput = validateCommandInputState(data.commandInput);
} else if (data.commandInput !== undefined) {
throw new Error("Invalid pipeline resume state");
}
} else if (data.resumeMode === "same_stage" || data.commandInput !== undefined) {
throw new Error("Invalid pipeline resume state");
}
return data as PipelineResumeState;
}
export function validatePipelineInputResponse(schema: unknown, response: unknown) {
if (schema === undefined) {
throw new Error("pipeline input response schema is missing");
}
let validator;
try {
validator = compileCached(schema as any);
} catch {
throw new Error("pipeline input response schema is invalid");
}
const ok = validator(response);
if (ok) return;
const first = validator.errors?.[0];
const pathValue = first?.instancePath || "/";
const reason = first?.message ? ` ${first.message}` : "";
throw new Error(`pipeline input response failed schema validation at ${pathValue}:${reason}`);
if (schema === undefined) {
throw new Error("pipeline input response schema is missing");
}
let validator;
try {
validator = compileCached(schema as any);
} catch {
throw new Error("pipeline input response schema is invalid");
}
const ok = validator(response);
if (ok) return;
const first = validator.errors?.[0];
const pathValue = first?.instancePath || "/";
const reason = first?.message ? ` ${first.message}` : "";
throw new Error(`pipeline input response failed schema validation at ${pathValue}:${reason}`);
}
function validatePipelineShape(pipeline: unknown[]) {
for (const stage of pipeline) {
if (!stage || typeof stage !== "object") throw new Error("Invalid pipeline resume state");
const data = stage as Record<string, unknown>;
if (typeof data.name !== "string" || data.name.length === 0) {
throw new Error("Invalid pipeline resume state");
}
if (!data.args || typeof data.args !== "object" || Array.isArray(data.args)) {
throw new Error("Invalid pipeline resume state");
}
if (typeof data.raw !== "string") throw new Error("Invalid pipeline resume state");
}
for (const stage of pipeline) {
if (!stage || typeof stage !== "object") throw new Error("Invalid pipeline resume state");
const data = stage as Record<string, unknown>;
if (typeof data.name !== "string" || data.name.length === 0) {
throw new Error("Invalid pipeline resume state");
}
if (!data.args || typeof data.args !== "object" || Array.isArray(data.args)) {
throw new Error("Invalid pipeline resume state");
}
if (typeof data.raw !== "string") throw new Error("Invalid pipeline resume state");
}
}
src/read_line.ts
+96 -45
View File
@@ -1,54 +1,105 @@
export function readLineFromStream(stream: NodeJS.ReadableStream, opts?: { timeoutMs?: number }) {
const timeoutMs = Number(opts?.timeoutMs ?? 0);
const unreadInput = new WeakMap<NodeJS.ReadableStream, string>();
type ObservableReadableStream = NodeJS.ReadableStream & {
readableEnded?: boolean;
destroyed?: boolean;
closed?: boolean;
};
return new Promise<string>((resolve, reject) => {
let settled = false;
let buf = "";
let timer: NodeJS.Timeout | null = null;
export function readLineFromStream(
stream: NodeJS.ReadableStream,
opts?: { timeoutMs?: number; signal?: AbortSignal },
) {
const timeoutMs = Number(opts?.timeoutMs ?? 0);
const signal = opts?.signal;
const observableStream = stream as ObservableReadableStream;
const cleanup = () => {
stream.off("data", onData);
stream.off("end", onEnd);
stream.off("close", onClose);
stream.off("error", onError);
if (timer) clearTimeout(timer);
};
return new Promise<string>((resolve, reject) => {
let settled = false;
let buf = unreadInput.get(stream) ?? "";
unreadInput.delete(stream);
let timer: NodeJS.Timeout | null = null;
const finish = (value: string) => {
if (settled) return;
settled = true;
cleanup();
resolve(value);
};
const cleanup = () => {
stream.off("data", onData);
stream.off("end", onEnd);
stream.off("close", onClose);
stream.off("error", onError);
stream.pause();
signal?.removeEventListener("abort", onAbort);
if (timer) clearTimeout(timer);
};
const fail = (err: Error) => {
if (settled) return;
settled = true;
cleanup();
reject(err);
};
const finish = (value: string) => {
if (settled) return;
settled = true;
cleanup();
resolve(value);
};
const onData = (chunk: Buffer | string) => {
buf += Buffer.isBuffer(chunk) ? chunk.toString("utf8") : String(chunk);
const idx = buf.indexOf("\n");
if (idx !== -1) {
finish(buf.slice(0, idx));
}
};
const fail = (err: Error) => {
if (settled) return;
settled = true;
unreadInput.delete(stream);
cleanup();
reject(err);
};
const onEnd = () => finish(buf);
const onClose = () => finish(buf);
const onError = (err: Error) => fail(err);
const consumeLine = () => {
const idx = buf.indexOf("\n");
if (idx === -1) return false;
unreadInput.set(stream, buf.slice(idx + 1));
finish(buf.slice(0, idx));
return true;
};
if (timeoutMs > 0) {
timer = setTimeout(() => {
fail(new Error(`Timed out waiting for input (${timeoutMs}ms)`));
}, timeoutMs);
}
const onData = (chunk: Buffer | string) => {
buf += Buffer.isBuffer(chunk) ? chunk.toString("utf8") : String(chunk);
consumeLine();
};
const drainBuffered = () => {
let chunk: Buffer | string | null;
while (!settled && (chunk = stream.read()) !== null) onData(chunk);
};
stream.on("data", onData);
stream.on("end", onEnd);
stream.on("close", onClose);
stream.on("error", onError);
});
const onEnd = () => {
drainBuffered();
if (settled) return;
unreadInput.delete(stream);
finish(buf);
};
const onClose = () => {
drainBuffered();
if (settled) return;
unreadInput.delete(stream);
finish(buf);
};
const onError = (err: Error) => fail(err);
const onAbort = () => {
const reason = signal?.reason;
fail(reason instanceof Error ? reason : new Error("Input read aborted"));
};
if (signal?.aborted) {
onAbort();
return;
}
if (timeoutMs > 0) {
timer = setTimeout(() => {
fail(new Error(`Timed out waiting for input (${timeoutMs}ms)`));
}, timeoutMs);
}
stream.on("data", onData);
stream.on("end", onEnd);
stream.on("close", onClose);
stream.on("error", onError);
signal?.addEventListener("abort", onAbort, { once: true });
drainBuffered();
if (!settled && !consumeLine()) {
if (observableStream.readableEnded || observableStream.destroyed || observableStream.closed)
onEnd();
else stream.resume();
}
});
}
src/recipes/github/pr-monitor.ts
+173 -173
View File
@@ -21,19 +21,19 @@ import { ghPrView } from "./stages/pr-view.js";
* @returns {Object|null}
*/
function pickSubset(snapshot) {
if (!snapshot || typeof snapshot !== "object") return null;
return {
number: snapshot.number,
title: snapshot.title,
url: snapshot.url,
state: snapshot.state,
isDraft: snapshot.isDraft,
mergeable: snapshot.mergeable,
reviewDecision: snapshot.reviewDecision,
updatedAt: snapshot.updatedAt,
baseRefName: snapshot.baseRefName,
headRefName: snapshot.headRefName,
};
if (!snapshot || typeof snapshot !== "object") return null;
return {
number: snapshot.number,
title: snapshot.title,
url: snapshot.url,
state: snapshot.state,
isDraft: snapshot.isDraft,
mergeable: snapshot.mergeable,
reviewDecision: snapshot.reviewDecision,
updatedAt: snapshot.updatedAt,
baseRefName: snapshot.baseRefName,
headRefName: snapshot.headRefName,
};
}
/**
@@ -43,28 +43,28 @@ function pickSubset(snapshot) {
* @returns {{ changedFields: string[], changes: Object }}
*/
function buildChangeSummary(before, after) {
const a = pickSubset(after);
const b = pickSubset(before);
const a = pickSubset(after);
const b = pickSubset(before);
if (!a) return { changedFields: [], changes: {} };
if (!b) {
return {
changedFields: Object.keys(a),
changes: Object.fromEntries(Object.keys(a).map((k) => [k, { from: null, to: a[k] }])),
};
}
if (!a) return { changedFields: [], changes: {} };
if (!b) {
return {
changedFields: Object.keys(a),
changes: Object.fromEntries(Object.keys(a).map((k) => [k, { from: null, to: a[k] }])),
};
}
const changes = {};
for (const key of Object.keys(a)) {
if (JSON.stringify(a[key]) !== JSON.stringify(b[key])) {
changes[key] = { from: b[key], to: a[key] };
}
}
const changes = {};
for (const key of Object.keys(a)) {
if (JSON.stringify(a[key]) !== JSON.stringify(b[key])) {
changes[key] = { from: b[key], to: a[key] };
}
}
return {
changedFields: Object.keys(changes),
changes,
};
return {
changedFields: Object.keys(changes),
changes,
};
}
/**
@@ -73,10 +73,10 @@ function buildChangeSummary(before, after) {
* @returns {string}
*/
function formatChangeMessage({ repo, pr, changedFields, prInfo }) {
const fields = changedFields.length ? ` (${changedFields.join(", ")})` : "";
const title = prInfo?.title ? `: ${prInfo.title}` : "";
const url = prInfo?.url ? ` ${prInfo.url}` : "";
return `PR updated: ${repo}#${pr}${title}${fields}.${url}`.replace(/\s+/g, " ").trim();
const fields = changedFields.length ? ` (${changedFields.join(", ")})` : "";
const title = prInfo?.title ? `: ${prInfo.title}` : "";
const url = prInfo?.url ? ` ${prInfo.url}` : "";
return `PR updated: ${repo}#${pr}${title}${fields}.${url}`.replace(/\s+/g, " ").trim();
}
/**
@@ -91,97 +91,97 @@ function formatChangeMessage({ repo, pr, changedFields, prInfo }) {
* @returns {Lobster}
*/
export function prMonitor(options) {
const { repo, pr, changesOnly = false, summaryOnly = false } = options;
const key = options.key ?? `github.pr:${repo}#${pr}`;
const { repo, pr, changesOnly = false, summaryOnly = false } = options;
const key = options.key ?? `github.pr:${repo}#${pr}`;
if (!repo) throw new Error("prMonitor requires repo");
if (!pr) throw new Error("prMonitor requires pr");
if (!repo) throw new Error("prMonitor requires repo");
if (!pr) throw new Error("prMonitor requires pr");
const workflow = new Lobster()
.pipe(ghPrView({ repo, pr }))
.pipe(diffLast(key))
.pipe((results) => {
const diffResult = results[0];
const current = diffResult.after;
const before = diffResult.before;
const changed = diffResult.changed;
const workflow = new Lobster()
.pipe(ghPrView({ repo, pr }))
.pipe(diffLast(key))
.pipe((results) => {
const diffResult = results[0];
const current = diffResult.after;
const before = diffResult.before;
const changed = diffResult.changed;
// If changesOnly and no change, suppress output
if (changesOnly && !changed) {
return [
{
kind: "github.pr.monitor",
repo,
pr: Number(pr),
key,
changed: false,
suppressed: true,
},
];
}
// If changesOnly and no change, suppress output
if (changesOnly && !changed) {
return [
{
kind: "github.pr.monitor",
repo,
pr: Number(pr),
key,
changed: false,
suppressed: true,
},
];
}
const summary = buildChangeSummary(before, current);
const summary = buildChangeSummary(before, current);
if (summaryOnly) {
return [
{
kind: "github.pr.monitor",
repo,
pr: Number(pr),
key,
changed,
summary,
prInfo: {
number: current.number,
title: current.title,
url: current.url,
state: current.state,
updatedAt: current.updatedAt,
},
},
];
}
if (summaryOnly) {
return [
{
kind: "github.pr.monitor",
repo,
pr: Number(pr),
key,
changed,
summary,
prInfo: {
number: current.number,
title: current.title,
url: current.url,
state: current.state,
updatedAt: current.updatedAt,
},
},
];
}
return [
{
kind: "github.pr.monitor",
repo,
pr: Number(pr),
key,
changed,
summary,
prSnapshot: current,
},
];
})
.meta({
name: "github.pr.monitor",
description: "Monitor PR state and detect changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true, description: "Repository (owner/repo)" },
pr: { type: "number", required: true, description: "PR number" },
key: { type: "string", description: "State key override" },
changesOnly: { type: "boolean", default: false, description: "Only output when changed" },
summaryOnly: { type: "boolean", default: false, description: "Return compact summary" },
},
});
return [
{
kind: "github.pr.monitor",
repo,
pr: Number(pr),
key,
changed,
summary,
prSnapshot: current,
},
];
})
.meta({
name: "github.pr.monitor",
description: "Monitor PR state and detect changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true, description: "Repository (owner/repo)" },
pr: { type: "number", required: true, description: "PR number" },
key: { type: "string", description: "State key override" },
changesOnly: { type: "boolean", default: false, description: "Only output when changed" },
summaryOnly: { type: "boolean", default: false, description: "Return compact summary" },
},
});
return workflow;
return workflow;
}
// Attach metadata
prMonitor.meta = {
name: "github.pr.monitor",
description: "Monitor PR state and detect changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true },
pr: { type: "number", required: true },
key: { type: "string" },
changesOnly: { type: "boolean", default: false },
summaryOnly: { type: "boolean", default: false },
},
name: "github.pr.monitor",
description: "Monitor PR state and detect changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true },
pr: { type: "number", required: true },
key: { type: "string" },
changesOnly: { type: "boolean", default: false },
summaryOnly: { type: "boolean", default: false },
},
};
/**
@@ -195,72 +195,72 @@ prMonitor.meta = {
* @returns {Lobster}
*/
export function prMonitorNotify(options) {
const { repo, pr } = options;
const key = options.key ?? `github.pr:${repo}#${pr}`;
const { repo, pr } = options;
const key = options.key ?? `github.pr:${repo}#${pr}`;
if (!repo) throw new Error("prMonitorNotify requires repo");
if (!pr) throw new Error("prMonitorNotify requires pr");
if (!repo) throw new Error("prMonitorNotify requires repo");
if (!pr) throw new Error("prMonitorNotify requires pr");
const workflow = new Lobster()
.pipe(ghPrView({ repo, pr }))
.pipe(diffLast(key, { changesOnly: true }))
.pipe((results) => {
const diffResult = results[0];
const workflow = new Lobster()
.pipe(ghPrView({ repo, pr }))
.pipe(diffLast(key, { changesOnly: true }))
.pipe((results) => {
const diffResult = results[0];
if (diffResult.suppressed) {
return [{ kind: "github.pr.monitor.notify", suppressed: true }];
}
if (diffResult.suppressed) {
return [{ kind: "github.pr.monitor.notify", suppressed: true }];
}
const current = diffResult.after;
const before = diffResult.before;
const summary = buildChangeSummary(before, current);
const current = diffResult.after;
const before = diffResult.before;
const summary = buildChangeSummary(before, current);
const message = formatChangeMessage({
repo,
pr: Number(pr),
changedFields: summary.changedFields,
prInfo: current,
});
const message = formatChangeMessage({
repo,
pr: Number(pr),
changedFields: summary.changedFields,
prInfo: current,
});
return [
{
kind: "github.pr.monitor.notify",
changed: true,
repo,
pr: Number(pr),
message,
prInfo: {
number: current.number,
title: current.title,
url: current.url,
state: current.state,
},
summary,
},
];
})
.meta({
name: "github.pr.monitor.notify",
description: "Emit a notification message when PR changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true },
pr: { type: "number", required: true },
key: { type: "string" },
},
});
return [
{
kind: "github.pr.monitor.notify",
changed: true,
repo,
pr: Number(pr),
message,
prInfo: {
number: current.number,
title: current.title,
url: current.url,
state: current.state,
},
summary,
},
];
})
.meta({
name: "github.pr.monitor.notify",
description: "Emit a notification message when PR changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true },
pr: { type: "number", required: true },
key: { type: "string" },
},
});
return workflow;
return workflow;
}
// Attach metadata
prMonitorNotify.meta = {
name: "github.pr.monitor.notify",
description: "Emit a notification message when PR changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true },
pr: { type: "number", required: true },
key: { type: "string" },
},
name: "github.pr.monitor.notify",
description: "Emit a notification message when PR changes",
requires: ["gh"],
args: {
repo: { type: "string", required: true },
pr: { type: "number", required: true },
key: { type: "string" },
},
};
src/recipes/github/stages/pr-view.ts
+71 -71
View File
@@ -19,42 +19,42 @@ import { spawn } from "node:child_process";
* @returns {Promise<{stdout: string, stderr: string}>}
*/
function runGh(argv, { env, cwd }) {
return new Promise<any>((resolve, reject) => {
const child = spawn("gh", argv, {
env,
cwd,
stdio: ["ignore", "pipe", "pipe"],
});
return new Promise<any>((resolve, reject) => {
const child = spawn("gh", argv, {
env,
cwd,
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
let stdout = "";
let stderr = "";
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
child.on("error", (err: any) => {
if (err?.code === "ENOENT") {
reject(new Error("gh not found on PATH (install GitHub CLI)"));
return;
}
reject(err);
});
child.on("error", (err: any) => {
if (err?.code === "ENOENT") {
reject(new Error("gh not found on PATH (install GitHub CLI)"));
return;
}
reject(err);
});
child.on("close", (code) => {
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(new Error(`gh failed (${code}): ${stderr.trim() || stdout.trim()}`));
}
});
});
child.on("close", (code) => {
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(new Error(`gh failed (${code}): ${stderr.trim() || stdout.trim()}`));
}
});
});
}
/**
@@ -67,51 +67,51 @@ function runGh(argv, { env, cwd }) {
* @returns {Object} Stage object with run method
*/
export function ghPrView(options) {
const { repo, pr } = options;
const fields = options.fields ?? [
"number",
"title",
"url",
"state",
"isDraft",
"mergeable",
"reviewDecision",
"author",
"baseRefName",
"headRefName",
"updatedAt",
];
const { repo, pr } = options;
const fields = options.fields ?? [
"number",
"title",
"url",
"state",
"isDraft",
"mergeable",
"reviewDecision",
"author",
"baseRefName",
"headRefName",
"updatedAt",
];
if (!repo) throw new Error("ghPrView requires repo");
if (!pr) throw new Error("ghPrView requires pr");
if (!repo) throw new Error("ghPrView requires repo");
if (!pr) throw new Error("ghPrView requires pr");
return {
type: "github.pr.view",
repo,
pr,
return {
type: "github.pr.view",
repo,
pr,
async run({ input, ctx }) {
// Drain input
for await (const _item of input) {
// no-op
}
async run({ input, ctx }) {
// Drain input
for await (const _item of input) {
// no-op
}
const argv = ["pr", "view", String(pr), "--repo", String(repo), "--json", fields.join(",")];
const argv = ["pr", "view", String(pr), "--repo", String(repo), "--json", fields.join(",")];
const { stdout } = (await runGh(argv, { env: ctx.env, cwd: process.cwd() })) as any;
const { stdout } = (await runGh(argv, { env: ctx.env, cwd: process.cwd() })) as any;
let parsed;
try {
parsed = JSON.parse(stdout.trim());
} catch {
throw new Error("gh returned non-JSON output");
}
let parsed;
try {
parsed = JSON.parse(stdout.trim());
} catch {
throw new Error("gh returned non-JSON output");
}
return {
output: (async function* () {
yield parsed;
})(),
};
},
};
return {
output: (async function* () {
yield parsed;
})(),
};
},
};
}
src/recipes/registry.ts
+16 -16
View File
@@ -5,29 +5,29 @@
import { prMonitor, prMonitorNotify } from "./github/pr-monitor.js";
const recipes: Record<string, any> = {
"github.pr.monitor": prMonitor,
"github.pr.monitor.notify": prMonitorNotify,
"github.pr.monitor": prMonitor,
"github.pr.monitor.notify": prMonitorNotify,
};
export function registerRecipe(fn) {
const meta = fn?.meta ?? {};
const name = meta.name;
if (!name) throw new Error("Recipe is missing meta.name");
recipes[name] = fn;
const meta = fn?.meta ?? {};
const name = meta.name;
if (!name) throw new Error("Recipe is missing meta.name");
recipes[name] = fn;
}
export function listRecipes() {
return Object.entries(recipes).map(([name, fn]) => {
const meta: any = (fn as any).meta ?? {};
return {
name,
description: meta.description ?? "",
requires: meta.requires ?? [],
args: meta.args ?? {},
};
});
return Object.entries(recipes).map(([name, fn]) => {
const meta: any = (fn as any).meta ?? {};
return {
name,
description: meta.description ?? "",
requires: meta.requires ?? [],
args: meta.args ?? {},
};
});
}
export function getRecipe(name) {
return recipes[name];
return recipes[name];
}
src/renderers/json.ts
+9 -9
View File
@@ -1,11 +1,11 @@
export function createJsonRenderer(stdout) {
return {
json(items) {
stdout.write(JSON.stringify(items, null, 2));
stdout.write("\n");
},
lines(lines) {
for (const line of lines) stdout.write(String(line) + "\n");
},
};
return {
json(items) {
stdout.write(JSON.stringify(items, null, 2));
stdout.write("\n");
},
lines(lines) {
for (const line of lines) stdout.write(String(line) + "\n");
},
};
}
src/resume.ts
+146 -146
View File
@@ -7,133 +7,133 @@ import { findStateKeyByApprovalId } from "./state/store.js";
* State keys use naming conventions: pipeline_resume_<uuid> or workflow_resume_<uuid>.
*/
export function kindFromStateKey(stateKey: string): "pipeline-resume" | "workflow-file" {
if (stateKey.startsWith("pipeline_resume_")) return "pipeline-resume";
if (stateKey.startsWith("workflow_resume_")) return "workflow-file";
// Fallback for unknown prefixes — workflow-file is the original behavior
return "workflow-file";
if (stateKey.startsWith("pipeline_resume_")) return "pipeline-resume";
if (stateKey.startsWith("workflow_resume_")) return "workflow-file";
// Fallback for unknown prefixes — workflow-file is the original behavior
return "workflow-file";
}
export type PipelineResumePayload = {
protocolVersion: 1;
v: 1;
kind: "pipeline-resume";
stateKey: string;
protocolVersion: 1;
v: 1;
kind: "pipeline-resume";
stateKey: string;
};
export function parseResumeArgs(argv) {
const args = { decision: null, token: null, approvalId: null, responseJson: null, cancel: false };
const args = { decision: null, token: null, approvalId: null, responseJson: null, cancel: false };
for (let i = 0; i < argv.length; i++) {
const tok = argv[i];
if (tok === "--token") {
args.token = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--token=")) {
args.token = tok.slice("--token=".length);
continue;
}
if (tok === "--id") {
args.approvalId = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--id=")) {
args.approvalId = tok.slice("--id=".length);
continue;
}
if (tok === "--response-json") {
args.responseJson = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--response-json=")) {
args.responseJson = tok.slice("--response-json=".length);
continue;
}
if (tok === "--cancel") {
const next = argv[i + 1];
if (typeof next === "string" && !next.startsWith("--")) {
const parsed = parseBooleanArg(next);
if (parsed === null) {
throw new Error("resume --cancel must be true or false");
}
args.cancel = parsed;
i++;
continue;
}
args.cancel = true;
continue;
}
if (tok.startsWith("--cancel=")) {
const parsed = parseBooleanArg(tok.slice("--cancel=".length));
if (parsed === null) throw new Error("resume --cancel must be true or false");
args.cancel = parsed;
continue;
}
if (tok === "--approve" || tok === "--decision") {
args.decision = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--approve=")) {
args.decision = tok.slice("--approve=".length);
continue;
}
if (tok.startsWith("--decision=")) {
args.decision = tok.slice("--decision=".length);
continue;
}
}
for (let i = 0; i < argv.length; i++) {
const tok = argv[i];
if (tok === "--token") {
args.token = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--token=")) {
args.token = tok.slice("--token=".length);
continue;
}
if (tok === "--id") {
args.approvalId = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--id=")) {
args.approvalId = tok.slice("--id=".length);
continue;
}
if (tok === "--response-json") {
args.responseJson = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--response-json=")) {
args.responseJson = tok.slice("--response-json=".length);
continue;
}
if (tok === "--cancel") {
const next = argv[i + 1];
if (typeof next === "string" && !next.startsWith("--")) {
const parsed = parseBooleanArg(next);
if (parsed === null) {
throw new Error("resume --cancel must be true or false");
}
args.cancel = parsed;
i++;
continue;
}
args.cancel = true;
continue;
}
if (tok.startsWith("--cancel=")) {
const parsed = parseBooleanArg(tok.slice("--cancel=".length));
if (parsed === null) throw new Error("resume --cancel must be true or false");
args.cancel = parsed;
continue;
}
if (tok === "--approve" || tok === "--decision") {
args.decision = argv[i + 1];
i++;
continue;
}
if (tok.startsWith("--approve=")) {
args.decision = tok.slice("--approve=".length);
continue;
}
if (tok.startsWith("--decision=")) {
args.decision = tok.slice("--decision=".length);
continue;
}
}
if (!args.token && !args.approvalId) throw new Error("resume requires --token or --id");
const intentCount =
Number(Boolean(args.decision)) + Number(args.responseJson !== null) + Number(args.cancel);
if (intentCount > 1) {
throw new Error("resume accepts only one of --approve, --response-json, or --cancel");
}
if (intentCount === 0) {
throw new Error("resume requires --approve yes|no, --response-json, or --cancel");
}
if (!args.token && !args.approvalId) throw new Error("resume requires --token or --id");
const intentCount =
Number(Boolean(args.decision)) + Number(args.responseJson !== null) + Number(args.cancel);
if (intentCount > 1) {
throw new Error("resume accepts only one of --approve, --response-json, or --cancel");
}
if (intentCount === 0) {
throw new Error("resume requires --approve yes|no, --response-json, or --cancel");
}
if (args.cancel) {
return {
token: args.token ? String(args.token) : null,
approvalId: args.approvalId ? String(args.approvalId) : null,
cancel: true,
};
}
if (args.cancel) {
return {
token: args.token ? String(args.token) : null,
approvalId: args.approvalId ? String(args.approvalId) : null,
cancel: true,
};
}
if (args.responseJson !== null) {
try {
return {
token: args.token ? String(args.token) : null,
approvalId: args.approvalId ? String(args.approvalId) : null,
response: JSON.parse(String(args.responseJson)),
};
} catch {
throw new Error("resume --response-json must be valid JSON");
}
}
if (args.responseJson !== null) {
try {
return {
token: args.token ? String(args.token) : null,
approvalId: args.approvalId ? String(args.approvalId) : null,
response: JSON.parse(String(args.responseJson)),
};
} catch {
throw new Error("resume --response-json must be valid JSON");
}
}
const decision = String(args.decision).toLowerCase();
if (!["yes", "y", "no", "n"].includes(decision))
throw new Error("resume --approve must be yes or no");
return {
token: args.token ? String(args.token) : null,
approvalId: args.approvalId ? String(args.approvalId) : null,
approved: decision === "yes" || decision === "y",
};
const decision = String(args.decision).toLowerCase();
if (!["yes", "y", "no", "n"].includes(decision))
throw new Error("resume --approve must be yes or no");
return {
token: args.token ? String(args.token) : null,
approvalId: args.approvalId ? String(args.approvalId) : null,
approved: decision === "yes" || decision === "y",
};
}
function parseBooleanArg(value: string): boolean | null {
const raw = String(value ?? "")
.trim()
.toLowerCase();
if (["1", "true", "yes", "y"].includes(raw)) return true;
if (["0", "false", "no", "n"].includes(raw)) return false;
return null;
const raw = String(value ?? "")
.trim()
.toLowerCase();
if (["1", "true", "yes", "y"].includes(raw)) return true;
if (["0", "false", "no", "n"].includes(raw)) return false;
return null;
}
/**
@@ -141,46 +141,46 @@ function parseBooleanArg(value: string): boolean | null {
* Detects the kind (workflow-file vs pipeline-resume) from the state key prefix.
*/
export async function resolveApprovalId(
approvalId: string,
env: Record<string, string | undefined>,
approvalId: string,
env: Record<string, string | undefined>,
): Promise<string> {
const stateKey = await findStateKeyByApprovalId({ env, approvalId });
if (!stateKey) {
throw new Error(`Approval ID "${approvalId}" not found or expired`);
}
const stateKey = await findStateKeyByApprovalId({ env, approvalId });
if (!stateKey) {
throw new Error(`Approval ID "${approvalId}" not found or expired`);
}
const kind = kindFromStateKey(stateKey);
const kind = kindFromStateKey(stateKey);
return encodeToken({
protocolVersion: 1,
v: 1,
kind,
stateKey,
});
return encodeToken({
protocolVersion: 1,
v: 1,
kind,
stateKey,
});
}
export function decodeResumeToken(token) {
const payload = decodeToken(token);
if (!payload || typeof payload !== "object") throw new Error("Invalid token");
if (payload.protocolVersion !== 1) throw new Error("Unsupported protocol version");
if (payload.v !== 1) throw new Error("Unsupported token version");
const workflowPayload = decodeWorkflowResumePayload(payload);
if (workflowPayload) return workflowPayload;
const pipelinePayload = decodePipelineResumePayload(payload);
if (pipelinePayload) return pipelinePayload;
throw new Error("Invalid token");
const payload = decodeToken(token);
if (!payload || typeof payload !== "object") throw new Error("Invalid token");
if (payload.protocolVersion !== 1) throw new Error("Unsupported protocol version");
if (payload.v !== 1) throw new Error("Unsupported token version");
const workflowPayload = decodeWorkflowResumePayload(payload);
if (workflowPayload) return workflowPayload;
const pipelinePayload = decodePipelineResumePayload(payload);
if (pipelinePayload) return pipelinePayload;
throw new Error("Invalid token");
}
function decodePipelineResumePayload(payload: unknown): PipelineResumePayload | null {
if (!payload || typeof payload !== "object") return null;
const data = payload as Partial<PipelineResumePayload>;
if (data.kind !== "pipeline-resume") return null;
if (data.protocolVersion !== 1 || data.v !== 1) throw new Error("Unsupported token version");
if (!data.stateKey || typeof data.stateKey !== "string") throw new Error("Invalid token");
return {
protocolVersion: 1,
v: 1,
kind: "pipeline-resume",
stateKey: data.stateKey,
};
if (!payload || typeof payload !== "object") return null;
const data = payload as Partial<PipelineResumePayload>;
if (data.kind !== "pipeline-resume") return null;
if (data.protocolVersion !== 1 || data.v !== 1) throw new Error("Unsupported token version");
if (!data.stateKey || typeof data.stateKey !== "string") throw new Error("Invalid token");
return {
protocolVersion: 1,
v: 1,
kind: "pipeline-resume",
stateKey: data.stateKey,
};
}
src/runtime.ts
+431 -253
View File
@@ -1,298 +1,476 @@
import { createJsonRenderer } from "./renderers/json.js";
import type { LlmSpendLedger } from "./commands/stdlib/llm_invoke.js";
import {
InputRequestSuspension,
RequestInputResumeError,
assertRequestInputResumeConsumed,
createInputTracker,
createStageRequestInput,
type CommandInputResume,
InputRequestSuspension,
RequestInputResumeError,
assertRequestInputResumeConsumed,
createInputTracker,
createStageRequestInput,
type CommandInputResume,
} from "./input_request.js";
export async function runPipeline({
pipeline,
registry,
stdin,
stdout,
stderr,
env,
mode = "human",
input,
cwd = undefined,
llmAdapters = undefined,
signal = undefined,
dryRun = false,
requestInputResume = undefined,
requestInputEnabled = true,
pipeline,
registry,
stdin,
stdout,
stderr,
env,
mode = "human",
input,
cwd = undefined,
llmAdapters = undefined,
llmSpendLedger = undefined,
signal = undefined,
forceTerminationSignal = undefined,
haltAfterStageOnAbort = false,
dryRun = false,
requestInputResume = undefined,
requestInputEnabled = true,
onExecutionStart = undefined,
}: {
pipeline: any[];
registry: any;
stdin: any;
stdout: any;
stderr: any;
env: any;
mode?: string;
input?: any;
cwd?: string | undefined;
llmAdapters?: Record<string, any> | undefined;
signal?: AbortSignal | undefined;
dryRun?: boolean;
requestInputResume?: CommandInputResume | undefined;
requestInputEnabled?: boolean;
pipeline: any[];
registry: any;
stdin: any;
stdout: any;
stderr: any;
env: any;
mode?: string;
input?: any;
cwd?: string | undefined;
llmAdapters?: Record<string, any> | undefined;
llmSpendLedger?: LlmSpendLedger | undefined;
signal?: AbortSignal | undefined;
forceTerminationSignal?: AbortSignal | undefined;
haltAfterStageOnAbort?: boolean;
dryRun?: boolean;
requestInputResume?: CommandInputResume | undefined;
requestInputEnabled?: boolean;
onExecutionStart?: (() => void | Promise<void>) | undefined;
}) {
if (dryRun) {
return dryRunPipeline({ pipeline, registry, stderr });
}
if (dryRun) {
return dryRunPipeline({ pipeline, registry, stderr });
}
let stream = input ?? [];
let rendered = false;
let halted = false;
let haltedAt = null;
let pipelineOutputStarted = false;
let stream = input ?? [];
let rendered = false;
const renderedItems: unknown[] = [];
let halted = false;
let haltedAt = null;
let pipelineOutputStarted = false;
let executionStarted = false;
let executionStart: Promise<void> | undefined;
const markExecutionStarted = async () => {
if (executionStart) return executionStart;
executionStart = (async () => {
await onExecutionStart?.();
executionStarted = true;
})();
return executionStart;
};
const baseCtx = {
stdin,
stdout,
stderr,
env,
registry,
mode,
cwd,
llmAdapters,
signal,
};
const baseCtx = {
stdin,
stdout,
stderr,
env,
registry,
mode,
cwd,
llmAdapters,
// The ledger of live LLM calls this run has not billed yet, so a replay of one of them
// can still be charged to the run that made it. Absent outside a cost-tracked run.
llmSpendLedger,
signal,
forceTerminationSignal,
};
for (let idx = 0; idx < pipeline.length; idx++) {
const stage = pipeline[idx];
const command = registry.get(stage.name);
if (!command) {
throw new Error(`Unknown command: ${stage.name}`);
}
for (let idx = 0; idx < pipeline.length; idx++) {
if (haltAfterStageOnAbort) signal?.throwIfAborted();
const stage = pipeline[idx];
const command = registry.get(stage.name);
if (!command) {
throw new Error(`Unknown command: ${stage.name}`);
}
if (command.meta?.resumeSafeBeforeInput !== true) {
await markExecutionStarted();
}
const inputTracker = createInputTracker(stream);
const stageResume = idx === 0 ? requestInputResume : undefined;
let commandActive = true;
let inactiveReason: string | undefined;
let commandOutputStarted = false;
let stageFinished = false;
async function finishStage({ assertResume = true, suppressCloseErrors = false } = {}) {
if (stageFinished) return;
stageFinished = true;
commandActive = false;
inputTracker.disableReplay();
await inputTracker.close({ suppressErrors: suppressCloseErrors });
if (assertResume) assertRequestInputResumeConsumed(stageResume);
}
const stageStdout = trackWritableOutput(stdout, () => {
pipelineOutputStarted = true;
});
const ctx = {
...baseCtx,
stdout: stageStdout,
render: createJsonRenderer(stageStdout),
};
const stageCtx = {
...ctx,
requestInput: requestInputEnabled
? createStageRequestInput({
ctx,
stageIndex: idx,
mode,
inputTracker,
isCommandActive: () => commandActive,
getInactiveReason: () => inactiveReason,
isOutputStarted: () => pipelineOutputStarted || commandOutputStarted,
resume: stageResume,
})
: createUnsupportedRequestInput(),
};
const inputTracker = createInputTracker(stream);
const stageResume = idx === 0 ? requestInputResume : undefined;
let commandActive = true;
let inactiveReason: string | undefined;
let commandOutputStarted = false;
let stageFinished = false;
async function finishStage({ assertResume = true, suppressCloseErrors = false } = {}) {
if (stageFinished) return;
stageFinished = true;
commandActive = false;
inputTracker.disableReplay();
await inputTracker.close({ suppressErrors: suppressCloseErrors });
if (assertResume) assertRequestInputResumeConsumed(stageResume);
}
const stageStdout = trackWritableOutput(stdout, () => {
pipelineOutputStarted = true;
});
const ctx = {
...baseCtx,
stdout: stageStdout,
render: createRecordingJsonRenderer(stageStdout, renderedItems),
};
const stageCtx = {
...ctx,
requestInput: requestInputEnabled
? createStageRequestInput({
ctx,
stageIndex: idx,
mode,
inputTracker,
isCommandActive: () => commandActive,
getInactiveReason: () => inactiveReason,
isOutputStarted: () => pipelineOutputStarted || commandOutputStarted,
resume: stageResume,
onResumedInput:
command.meta?.resumeSafeAfterInput === true ? undefined : markExecutionStarted,
})
: createUnsupportedRequestInput(),
};
let result;
try {
result = await command.run({ input: inputTracker.iterable, args: stage.args, ctx: stageCtx });
} catch (err) {
await finishStage({ assertResume: false, suppressCloseErrors: true });
if (haltForInputRequest(err)) break;
assertNoUnconsumedResumeAfterError(stageResume, err);
throw err;
}
let result;
try {
result = await command.run({ input: inputTracker.iterable, args: stage.args, ctx: stageCtx });
} catch (err) {
await finishStage({ assertResume: false, suppressCloseErrors: true });
if (haltForInputRequest(err)) break;
assertNoUnconsumedResumeAfterError(stageResume, err);
throw err;
}
if (result?.rendered) {
rendered = true;
}
if (result?.rendered) {
rendered = true;
}
const output = result?.output;
if (Array.isArray(output)) {
stream = output;
await finishStage();
} else if (output && !result?.halt && idx < pipeline.length - 1) {
commandActive = false;
inactiveReason = "requestInput cannot suspend from lazy output before downstream stages";
assertRequestInputResumeConsumed(stageResume);
stream = trackCommandOutput(
output,
() => {
commandOutputStarted = true;
},
() => assertRequestInputResumeConsumed(stageResume),
(err) => assertNoUnconsumedResumeAfterError(stageResume, err),
finishStage,
);
} else {
stream = output
? trackCommandOutput(
output,
() => {
commandOutputStarted = true;
},
() => assertRequestInputResumeConsumed(stageResume),
(err) => assertNoUnconsumedResumeAfterError(stageResume, err),
finishStage,
)
: [];
if (!output) await finishStage();
}
const terminalOutput = Boolean(result?.halt);
let stageHalted = Boolean(terminalOutput || (haltAfterStageOnAbort && signal?.aborted));
const output = result?.output;
if (Array.isArray(output)) {
stream = output;
await finishStage();
} else if (output && idx < pipeline.length - 1 && !terminalOutput) {
commandActive = false;
inactiveReason = "requestInput cannot suspend from lazy output before downstream stages";
assertRequestInputResumeConsumed(stageResume);
const trackedOutput = trackCommandOutput(
output,
() => {
commandOutputStarted = true;
},
() => assertRequestInputResumeConsumed(stageResume),
(err) => assertNoUnconsumedResumeAfterError(stageResume, err),
finishStage,
);
stream = haltAfterStageOnAbort
? throwIfAbortedAfterDrain(trackedOutput, signal)
: trackedOutput;
} else {
if (output) {
const trackedOutput = trackCommandOutput(
output,
() => {
commandOutputStarted = true;
},
() => assertRequestInputResumeConsumed(stageResume),
(err) => assertNoUnconsumedResumeAfterError(stageResume, err),
finishStage,
);
// Terminal output is drained after the stage loop as well. It needs the
// same abort-aware read as a handoff, otherwise a stalled final iterator
// can prevent the tool cancellation from settling forever.
stream = haltAfterStageOnAbort
? throwIfAbortedAfterDrain(trackedOutput, signal)
: trackedOutput;
} else {
stream = [];
await finishStage();
}
}
if (result?.halt) {
halted = true;
haltedAt = { index: idx, stage };
break;
}
}
stageHalted ||= Boolean(haltAfterStageOnAbort && signal?.aborted);
if (stageHalted) {
halted = true;
haltedAt = { index: idx, stage };
break;
}
}
const items = [];
try {
for await (const item of stream) items.push(item);
} catch (err) {
if (haltForInputRequest(err)) {
items.length = 0;
for await (const item of stream) items.push(item);
} else {
throw err;
}
}
assertRequestInputResumeConsumed(requestInputResume);
const items = [];
try {
for await (const item of stream) items.push(item);
} catch (err) {
if (haltForInputRequest(err)) {
items.length = 0;
for await (const item of stream) items.push(item);
} else {
throw err;
}
}
if (haltAfterStageOnAbort) signal?.throwIfAborted();
assertRequestInputResumeConsumed(requestInputResume);
return { items, rendered, halted, haltedAt };
return { items, rendered, renderedItems, halted, haltedAt, executionStarted };
function haltForInputRequest(err: unknown) {
if (!(err instanceof InputRequestSuspension)) return false;
const stageIndex = err.stageIndex;
halted = true;
haltedAt = {
index: stageIndex,
stage: pipeline[stageIndex],
inPlace: true,
};
stream = streamFromItems([err.request]);
return true;
}
function haltForInputRequest(err: unknown) {
if (!(err instanceof InputRequestSuspension)) return false;
const stageIndex = err.stageIndex;
halted = true;
haltedAt = {
index: stageIndex,
stage: pipeline[stageIndex],
inPlace: true,
};
stream = streamFromItems([err.request]);
return true;
}
}
function dryRunPipeline({
pipeline,
registry,
stderr,
pipeline,
registry,
stderr,
}: {
pipeline: any[];
registry: any;
stderr: any;
pipeline: any[];
registry: any;
stderr: any;
}) {
const lines: string[] = [];
lines.push(`[DRY RUN] Pipeline (${pipeline.length} stage${pipeline.length !== 1 ? "s" : ""}):`);
const lines: string[] = [];
lines.push(`[DRY RUN] Pipeline (${pipeline.length} stage${pipeline.length !== 1 ? "s" : ""}):`);
for (let idx = 0; idx < pipeline.length; idx++) {
const stage = pipeline[idx];
const command = registry.get(stage.name);
if (!command) {
throw new Error(`Unknown command: ${stage.name}`);
}
const formattedArgs = stage.args ? formatStageArgs(stage.args) : "";
const argsStr = formattedArgs ? ` args: ${formattedArgs}` : "";
lines.push(` ${idx + 1}. ${stage.name}${argsStr}`);
}
for (let idx = 0; idx < pipeline.length; idx++) {
const stage = pipeline[idx];
const command = registry.get(stage.name);
if (!command) {
throw new Error(`Unknown command: ${stage.name}`);
}
const formattedArgs = stage.args ? formatStageArgs(stage.args) : "";
const argsStr = formattedArgs ? ` args: ${formattedArgs}` : "";
lines.push(` ${idx + 1}. ${stage.name}${argsStr}`);
}
lines.push("");
stderr.write(lines.join("\n"));
// Return rendered:true so the CLI does not print an empty JSON array to stdout.
return { items: [], rendered: true, halted: false, haltedAt: null };
lines.push("");
stderr.write(lines.join("\n"));
// Return rendered:true so the CLI does not print an empty JSON array to stdout.
return {
items: [],
rendered: true,
renderedItems: [],
halted: false,
haltedAt: null,
executionStarted: false,
};
}
function formatStageArgs(args: Record<string, unknown>) {
const parts: string[] = [];
for (const [key, value] of Object.entries(args)) {
if (key === "_") {
const positional = Array.isArray(value) ? value : [value];
for (const v of positional) {
if (v !== undefined && v !== null) parts.push(String(v));
}
} else {
parts.push(`${key}=${typeof value === "string" ? value : JSON.stringify(value)}`);
}
}
return parts.join(", ");
const parts: string[] = [];
for (const [key, value] of Object.entries(args)) {
if (key === "_") {
const positional = Array.isArray(value) ? value : [value];
for (const v of positional) {
if (v !== undefined && v !== null) parts.push(String(v));
}
} else {
parts.push(`${key}=${typeof value === "string" ? value : JSON.stringify(value)}`);
}
}
return parts.join(", ");
}
/**
* Wraps the stage renderer so the pipeline keeps the objects a renderer was handed. A renderer
* writes them to stdout and returns no items, so a caller that reads the pipeline back from that
* text only ever sees what JSON can express. Provenance a consumer must not take from the text
* itself — whether an LLM result was replayed rather than paid for — lives on these originals.
* They stay in this process and are never written or serialized.
*/
function createRecordingJsonRenderer(stdout: any, collected: unknown[]) {
const renderer = createJsonRenderer(stdout);
return {
...renderer,
json(items: unknown) {
if (Array.isArray(items)) collected.push(...items);
else collected.push(items);
return renderer.json(items);
},
};
}
function streamFromItems(items: unknown[]) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
function throwIfAbortedAfterDrain(input: AsyncIterable<unknown>, signal?: AbortSignal) {
return (async function* () {
const iterator = input[Symbol.asyncIterator]();
let completed = false;
try {
while (true) {
const next = await nextWithAbort(iterator, signal);
if (next.done) {
completed = true;
break;
}
signal?.throwIfAborted();
yield next.value;
}
signal?.throwIfAborted();
} finally {
if (!completed) await closeAfterAbortedRead(input, iterator, signal);
}
})();
}
type CancellableLazyOutput = {
abort?: (reason?: unknown) => void | Promise<void>;
};
async function closeAfterAbortedRead(
input: AsyncIterable<unknown>,
iterator: AsyncIterator<unknown>,
signal?: AbortSignal,
) {
const cancellable = iterator as AsyncIterator<unknown> & CancellableLazyOutput;
const inputCancellable = input as AsyncIterable<unknown> & CancellableLazyOutput;
const abort = cancellable.abort ?? inputCancellable.abort;
try {
// A source that owns a pending timer, socket, or process can expose this
// small cancellation hook. It must release the pending next() operation.
if (signal?.aborted && abort) await abort(signal.reason);
if (typeof iterator.return !== "function") return;
const close = iterator.return();
if (signal?.aborted && !abort) {
// Legacy iterators cannot interrupt an in-flight next(). Keep the
// existing prompt cancellation behavior, while resource-owning sources
// opt into the abort hook above so their cleanup is awaited.
void Promise.resolve(close).catch(() => {});
return;
}
await close;
} catch (err) {
if (!signal?.aborted) throw err;
}
}
async function nextWithAbort(iterator: AsyncIterator<unknown>, signal?: AbortSignal) {
if (!signal) return iterator.next();
signal.throwIfAborted();
let onAbort!: () => void;
const aborted = new Promise<never>((_resolve, reject) => {
onAbort = () => {
try {
signal.throwIfAborted();
} catch (err) {
reject(err);
}
};
signal.addEventListener("abort", onAbort, { once: true });
});
if (signal.aborted) onAbort();
try {
return await Promise.race([iterator.next(), aborted]);
} finally {
signal.removeEventListener("abort", onAbort);
}
}
function trackCommandOutput(
output: AsyncIterable<unknown> | Iterable<unknown>,
markOutput: () => void,
assertResumeConsumed: () => void,
assertNoUnconsumedResumeAfterError: (err: unknown) => void,
finishStage: (options?: {
assertResume?: boolean;
suppressCloseErrors?: boolean;
}) => Promise<void>,
output: AsyncIterable<unknown> | Iterable<unknown>,
markOutput: () => void,
assertResumeConsumed: () => void,
assertNoUnconsumedResumeAfterError: (err: unknown) => void,
finishStage: (options?: {
assertResume?: boolean;
suppressCloseErrors?: boolean;
}) => Promise<void>,
) {
return (async function* () {
let completed = false;
try {
for await (const item of output) {
assertResumeConsumed();
markOutput();
yield item;
}
completed = true;
} catch (err) {
await finishStage({ assertResume: false, suppressCloseErrors: true });
assertNoUnconsumedResumeAfterError(err);
throw err;
} finally {
await finishStage({ assertResume: completed });
}
})();
const source = output as AsyncIterable<unknown> & CancellableLazyOutput & AsyncIterator<unknown>;
let sourceIterator: AsyncIterator<unknown> | undefined;
const tracked = (async function* () {
let completed = false;
try {
if (typeof source[Symbol.asyncIterator] === "function") {
sourceIterator = source[Symbol.asyncIterator]();
const iteratorInput: AsyncIterable<unknown> = {
[Symbol.asyncIterator]: () => sourceIterator!,
};
for await (const item of iteratorInput) {
assertResumeConsumed();
markOutput();
yield item;
}
} else {
for (const item of output as Iterable<unknown>) {
assertResumeConsumed();
markOutput();
yield item;
}
}
completed = true;
} catch (err) {
await finishStage({ assertResume: false, suppressCloseErrors: true });
assertNoUnconsumedResumeAfterError(err);
throw err;
} finally {
await finishStage({ assertResume: completed });
}
})();
// Iterator-owned abort hooks are only discoverable after iterator acquisition.
// A getter lets the abort-aware wrapper find that hook once a read is pending,
// without performing acquisition outside the generator's guarded cleanup path.
Object.defineProperty(tracked, "abort", {
configurable: true,
get() {
const cancellationOwner =
sourceIterator && typeof (sourceIterator as CancellableLazyOutput).abort === "function"
? (sourceIterator as CancellableLazyOutput)
: source;
const abort = cancellationOwner.abort;
return typeof abort === "function"
? (reason?: unknown) => abort.call(cancellationOwner, reason)
: undefined;
},
});
return tracked;
}
function assertNoUnconsumedResumeAfterError(resume: CommandInputResume | undefined, err: unknown) {
if (err instanceof RequestInputResumeError) return;
assertRequestInputResumeConsumed(resume);
if (err instanceof RequestInputResumeError) return;
assertRequestInputResumeConsumed(resume);
}
function trackWritableOutput(stdout: any, markOutput: () => void) {
return new Proxy(stdout, {
get(target, prop, receiver) {
const value = Reflect.get(target, prop, receiver);
if (prop === "write" || prop === "end") {
return (...args: unknown[]) => {
markOutput();
return value.apply(target, args);
};
}
return typeof value === "function" ? value.bind(target) : value;
},
});
return new Proxy(stdout, {
get(target, prop, receiver) {
const value = Reflect.get(target, prop, receiver);
if (prop === "write" || prop === "end") {
return (...args: unknown[]) => {
markOutput();
return value.apply(target, args);
};
}
return typeof value === "function" ? value.bind(target) : value;
},
});
}
function createUnsupportedRequestInput() {
const requestInput = async function requestInput() {
throw new Error("requestInput is not supported in this pipeline context");
};
requestInput.getSuspendedState = function getSuspendedState() {
return undefined;
};
return requestInput;
const requestInput = async function requestInput() {
throw new Error("requestInput is not supported in this pipeline context");
};
requestInput.getSuspendedState = function getSuspendedState() {
return undefined;
};
return requestInput;
}
src/sdk/Lobster.ts
+395 -391
View File
@@ -3,27 +3,27 @@ import { runPipelineInternal } from "./runtime.js";
import { encodeToken, decodeToken } from "./token.js";
import { compileCached } from "../validation.js";
import { validateCommandInputState, type CommandInputState } from "../input_request.js";
import { deleteStateJson, readStateJson, writeStateJson } from "../state/store.js";
import { deleteStateJson, readStateJsonWithLock, writeStateJson } from "../state/store.js";
type SdkResumePayload = {
protocolVersion: 1;
v: 1;
stageIndex?: number;
resumeAtIndex: number;
items?: unknown[];
prompt?: string;
inputSchema?: unknown;
inputSubject?: unknown;
resumeMode?: "next_stage" | "same_stage";
stateKey?: string;
protocolVersion: 1;
v: 1;
stageIndex?: number;
resumeAtIndex: number;
items?: unknown[];
prompt?: string;
inputSchema?: unknown;
inputSubject?: unknown;
resumeMode?: "next_stage" | "same_stage";
stateKey?: string;
};
type SdkCommandInputResumeState = {
resumeAtIndex: number;
items: unknown[];
inputSchema: unknown;
inputSubject?: unknown;
commandInput: CommandInputState;
resumeAtIndex: number;
items: unknown[];
inputSchema: unknown;
inputSubject?: unknown;
commandInput: CommandInputState;
};
/**
@@ -50,417 +50,421 @@ type SdkCommandInputResumeState = {
*/
export class Lobster {
#stages = [];
#options: any = {} as any;
#meta = null;
#stages = [];
#options: any = {} as any;
#meta = null;
constructor(options: any = {}) {
this.#options = {
env: options.env ?? process.env,
stateDir: options.stateDir,
};
}
constructor(options: any = {}) {
this.#options = {
env: options.env ?? process.env,
stateDir: options.stateDir,
};
}
pipe(stage) {
if (typeof stage !== "function" && typeof stage?.run !== "function") {
throw new Error("Stage must be a function or have a run() method");
}
this.#stages.push(stage);
return this;
}
pipe(stage) {
if (typeof stage !== "function" && typeof stage?.run !== "function") {
throw new Error("Stage must be a function or have a run() method");
}
this.#stages.push(stage);
return this;
}
meta(meta) {
this.#meta = meta;
return this;
}
meta(meta) {
this.#meta = meta;
return this;
}
getMeta() {
return this.#meta;
}
getMeta() {
return this.#meta;
}
async run(initialInput = []) {
const ctx = {
env: this.#options.env,
stateDir: this.#options.stateDir,
mode: "sdk",
};
async run(initialInput = []) {
const ctx = {
env: this.#options.env,
stateDir: this.#options.stateDir,
mode: "sdk",
};
try {
const result = await runPipelineInternal({
stages: this.#stages,
ctx,
input: initialInput,
});
try {
const result = await runPipelineInternal({
stages: this.#stages,
ctx,
input: initialInput,
});
if (
result.halted &&
result.items.length === 1 &&
result.items[0]?.type === "approval_request"
) {
const approval = result.items[0];
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: result.haltedAt?.index ?? -1,
resumeAtIndex: (result.haltedAt?.index ?? -1) + 1,
items: approval.items,
prompt: approval.prompt,
});
if (
result.halted &&
result.items.length === 1 &&
result.items[0]?.type === "approval_request"
) {
const approval = result.items[0];
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: result.haltedAt?.index ?? -1,
resumeAtIndex: (result.haltedAt?.index ?? -1) + 1,
items: approval.items,
prompt: approval.prompt,
});
return {
ok: true,
status: "needs_approval",
output: [],
requiresApproval: {
prompt: approval.prompt,
items: approval.items,
resumeToken,
},
requiresInput: null,
};
}
return {
ok: true,
status: "needs_approval",
output: [],
requiresApproval: {
prompt: approval.prompt,
items: approval.items,
resumeToken,
},
requiresInput: null,
};
}
if (result.halted && result.items.length === 1 && result.items[0]?.type === "input_request") {
const input = result.items[0];
const resumeMode = input.commandInput ? "same_stage" : "next_stage";
const resumeAtIndex =
resumeMode === "same_stage"
? (result.haltedAt?.index ?? -1)
: (result.haltedAt?.index ?? -1) + 1;
const stateKey =
resumeMode === "same_stage"
? await saveSdkCommandInputResumeState(this.#options, {
resumeAtIndex,
items: input.items ?? [],
inputSchema: input.responseSchema,
...(input.subject !== undefined ? { inputSubject: input.subject } : null),
commandInput: input.commandInput,
})
: undefined;
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: result.haltedAt?.index ?? -1,
resumeAtIndex,
resumeMode,
...(resumeMode === "same_stage"
? { stateKey }
: { items: [], inputSchema: input.responseSchema }),
inputSubject: input.subject,
});
if (result.halted && result.items.length === 1 && result.items[0]?.type === "input_request") {
const input = result.items[0];
const resumeMode = input.commandInput ? "same_stage" : "next_stage";
const resumeAtIndex =
resumeMode === "same_stage"
? (result.haltedAt?.index ?? -1)
: (result.haltedAt?.index ?? -1) + 1;
const stateKey =
resumeMode === "same_stage"
? await saveSdkCommandInputResumeState(this.#options, {
resumeAtIndex,
items: input.items ?? [],
inputSchema: input.responseSchema,
...(input.subject !== undefined ? { inputSubject: input.subject } : null),
commandInput: input.commandInput,
})
: undefined;
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: result.haltedAt?.index ?? -1,
resumeAtIndex,
resumeMode,
...(resumeMode === "same_stage"
? { stateKey }
: { items: [], inputSchema: input.responseSchema }),
inputSubject: input.subject,
});
return {
ok: true,
status: "needs_input",
output: [],
requiresApproval: null,
requiresInput: {
prompt: input.prompt,
responseSchema: input.responseSchema,
defaults: input.defaults,
subject: input.subject,
resumeToken,
},
};
}
return {
ok: true,
status: "needs_input",
output: [],
requiresApproval: null,
requiresInput: {
prompt: input.prompt,
responseSchema: input.responseSchema,
defaults: input.defaults,
subject: input.subject,
resumeToken,
},
};
}
return {
ok: true,
status: "ok",
output: result.items,
requiresApproval: null,
requiresInput: null,
};
} catch (err) {
return {
ok: false,
status: "error",
output: [],
requiresApproval: null,
requiresInput: null,
error: {
type: "runtime_error",
message: err?.message ?? String(err),
},
};
}
}
return {
ok: true,
status: "ok",
output: result.items,
requiresApproval: null,
requiresInput: null,
};
} catch (err) {
return {
ok: false,
status: "error",
output: [],
requiresApproval: null,
requiresInput: null,
error: {
type: "runtime_error",
message: err?.message ?? String(err),
},
};
}
}
async resume(
token: string,
options: { approved?: boolean; response?: unknown; cancel?: boolean } = {},
) {
const { approved, response, cancel } = options;
const intentCount =
Number(typeof approved === "boolean") +
Number(response !== undefined) +
Number(cancel === true);
if (intentCount > 1) {
throw new Error("resume accepts only one of approved, response, or cancel");
}
if (intentCount === 0) {
throw new Error("resume requires approved, response, or cancel");
}
async resume(
token: string,
options: { approved?: boolean; response?: unknown; cancel?: boolean } = {},
) {
const { approved, response, cancel } = options;
const intentCount =
Number(typeof approved === "boolean") +
Number(response !== undefined) +
Number(cancel === true);
if (intentCount > 1) {
throw new Error("resume accepts only one of approved, response, or cancel");
}
if (intentCount === 0) {
throw new Error("resume requires approved, response, or cancel");
}
const payload = decodeSdkResumePayload(token);
const payload = decodeSdkResumePayload(token);
let sdkCommandInputState: SdkCommandInputResumeState | undefined;
if (payload.resumeMode === "same_stage") {
sdkCommandInputState = await loadSdkCommandInputResumeState(this.#options, payload.stateKey!);
}
let sdkCommandInputState: SdkCommandInputResumeState | undefined;
if (payload.resumeMode === "same_stage") {
sdkCommandInputState = await loadSdkCommandInputResumeState(this.#options, payload.stateKey!);
}
if (cancel === true) {
if (payload.resumeMode === "same_stage") {
await deleteStateJson({ env: sdkStateEnv(this.#options), key: payload.stateKey! });
}
return {
ok: true,
status: "cancelled",
output: [],
requiresApproval: null,
requiresInput: null,
};
}
if (cancel === true) {
if (payload.resumeMode === "same_stage") {
await deleteStateJson({ env: sdkStateEnv(this.#options), key: payload.stateKey! });
}
return {
ok: true,
status: "cancelled",
output: [],
requiresApproval: null,
requiresInput: null,
};
}
const expectsInput = payload.inputSchema !== undefined || payload.resumeMode === "same_stage";
if (expectsInput) {
if (approved !== undefined) {
throw new Error("resume token expects an input response, not approved");
}
if (response === undefined) {
throw new Error("resume token expects response");
}
} else {
if (response !== undefined) {
throw new Error("resume token expects approved=true|false, not response");
}
if (typeof approved !== "boolean") {
throw new Error("resume token expects approved=true|false");
}
if (approved === false) {
return {
ok: true,
status: "cancelled",
output: [],
requiresApproval: null,
requiresInput: null,
};
}
}
const expectsInput = payload.inputSchema !== undefined || payload.resumeMode === "same_stage";
if (expectsInput) {
if (approved !== undefined) {
throw new Error("resume token expects an input response, not approved");
}
if (response === undefined) {
throw new Error("resume token expects response");
}
} else {
if (response !== undefined) {
throw new Error("resume token expects approved=true|false, not response");
}
if (typeof approved !== "boolean") {
throw new Error("resume token expects approved=true|false");
}
if (approved === false) {
return {
ok: true,
status: "cancelled",
output: [],
requiresApproval: null,
requiresInput: null,
};
}
}
const resumeIndex = sdkCommandInputState?.resumeAtIndex ?? payload.resumeAtIndex ?? 0;
let resumeItems = sdkCommandInputState?.items ?? payload.items ?? [];
let requestInputResume;
if (response !== undefined) {
const schema = sdkCommandInputState?.inputSchema ?? payload.inputSchema;
if (schema === undefined) {
throw new Error("resume token does not support input responses");
}
let validator;
try {
validator = compileCached(schema as any);
} catch {
throw new Error("resume token input schema is invalid");
}
const ok = validator(response);
if (!ok) {
const first = validator.errors?.[0];
throw new Error(
`response does not match schema at ${first?.instancePath || "/"}: ${first?.message || "invalid"}`,
);
}
if (payload.resumeMode === "same_stage") {
resumeItems = sdkCommandInputState!.items;
requestInputResume = {
state: sdkCommandInputState!.commandInput,
response,
onConsumed: async () => {
await deleteStateJson({ env: sdkStateEnv(this.#options), key: payload.stateKey! });
},
};
} else {
resumeItems = [response];
}
}
const resumeIndex = sdkCommandInputState?.resumeAtIndex ?? payload.resumeAtIndex ?? 0;
let resumeItems = sdkCommandInputState?.items ?? payload.items ?? [];
let requestInputResume;
if (response !== undefined) {
const schema = sdkCommandInputState?.inputSchema ?? payload.inputSchema;
if (schema === undefined) {
throw new Error("resume token does not support input responses");
}
let validator;
try {
validator = compileCached(schema as any);
} catch {
throw new Error("resume token input schema is invalid");
}
const ok = validator(response);
if (!ok) {
const first = validator.errors?.[0];
throw new Error(
`response does not match schema at ${first?.instancePath || "/"}: ${first?.message || "invalid"}`,
);
}
if (payload.resumeMode === "same_stage") {
resumeItems = sdkCommandInputState!.items;
requestInputResume = {
state: sdkCommandInputState!.commandInput,
response,
onConsumed: async () => {
await deleteStateJson({ env: sdkStateEnv(this.#options), key: payload.stateKey! });
},
};
} else {
resumeItems = [response];
}
}
const remainingStages = this.#stages.slice(resumeIndex);
const ctx = {
env: this.#options.env,
stateDir: this.#options.stateDir,
mode: "sdk",
};
const remainingStages = this.#stages.slice(resumeIndex);
const ctx = {
env: this.#options.env,
stateDir: this.#options.stateDir,
mode: "sdk",
};
try {
const result = await runPipelineInternal({
stages: remainingStages,
ctx,
input: resumeItems,
requestInputResume,
});
try {
const result = await runPipelineInternal({
stages: remainingStages,
ctx,
input: resumeItems,
requestInputResume,
});
if (
result.halted &&
result.items.length === 1 &&
result.items[0]?.type === "approval_request"
) {
const approval = result.items[0];
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: resumeIndex + (result.haltedAt?.index ?? 0),
resumeAtIndex: resumeIndex + (result.haltedAt?.index ?? 0) + 1,
items: approval.items,
prompt: approval.prompt,
});
if (
result.halted &&
result.items.length === 1 &&
result.items[0]?.type === "approval_request"
) {
const approval = result.items[0];
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: resumeIndex + (result.haltedAt?.index ?? 0),
resumeAtIndex: resumeIndex + (result.haltedAt?.index ?? 0) + 1,
items: approval.items,
prompt: approval.prompt,
});
return {
ok: true,
status: "needs_approval",
output: [],
requiresApproval: {
prompt: approval.prompt,
items: approval.items,
resumeToken,
},
requiresInput: null,
};
}
return {
ok: true,
status: "needs_approval",
output: [],
requiresApproval: {
prompt: approval.prompt,
items: approval.items,
resumeToken,
},
requiresInput: null,
};
}
if (result.halted && result.items.length === 1 && result.items[0]?.type === "input_request") {
const input = result.items[0];
const inputStageIndex = resumeIndex + (result.haltedAt?.index ?? 0);
const resumeMode = input.commandInput ? "same_stage" : "next_stage";
const stateKey =
resumeMode === "same_stage"
? await saveSdkCommandInputResumeState(this.#options, {
resumeAtIndex: inputStageIndex,
items: input.items ?? [],
inputSchema: input.responseSchema,
...(input.subject !== undefined ? { inputSubject: input.subject } : null),
commandInput: input.commandInput,
})
: undefined;
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: inputStageIndex,
resumeAtIndex: resumeMode === "same_stage" ? inputStageIndex : inputStageIndex + 1,
resumeMode,
...(resumeMode === "same_stage"
? { stateKey }
: { items: [], inputSchema: input.responseSchema }),
inputSubject: input.subject,
});
if (result.halted && result.items.length === 1 && result.items[0]?.type === "input_request") {
const input = result.items[0];
const inputStageIndex = resumeIndex + (result.haltedAt?.index ?? 0);
const resumeMode = input.commandInput ? "same_stage" : "next_stage";
const stateKey =
resumeMode === "same_stage"
? await saveSdkCommandInputResumeState(this.#options, {
resumeAtIndex: inputStageIndex,
items: input.items ?? [],
inputSchema: input.responseSchema,
...(input.subject !== undefined ? { inputSubject: input.subject } : null),
commandInput: input.commandInput,
})
: undefined;
const resumeToken = encodeToken({
protocolVersion: 1,
v: 1,
stageIndex: inputStageIndex,
resumeAtIndex: resumeMode === "same_stage" ? inputStageIndex : inputStageIndex + 1,
resumeMode,
...(resumeMode === "same_stage"
? { stateKey }
: { items: [], inputSchema: input.responseSchema }),
inputSubject: input.subject,
});
return {
ok: true,
status: "needs_input",
output: [],
requiresApproval: null,
requiresInput: {
prompt: input.prompt,
responseSchema: input.responseSchema,
defaults: input.defaults,
subject: input.subject,
resumeToken,
},
};
}
return {
ok: true,
status: "needs_input",
output: [],
requiresApproval: null,
requiresInput: {
prompt: input.prompt,
responseSchema: input.responseSchema,
defaults: input.defaults,
subject: input.subject,
resumeToken,
},
};
}
return {
ok: true,
status: "ok",
output: result.items,
requiresApproval: null,
requiresInput: null,
};
} catch (err) {
return {
ok: false,
status: "error",
output: [],
requiresApproval: null,
requiresInput: null,
error: {
type: "runtime_error",
message: err?.message ?? String(err),
},
};
}
}
return {
ok: true,
status: "ok",
output: result.items,
requiresApproval: null,
requiresInput: null,
};
} catch (err) {
return {
ok: false,
status: "error",
output: [],
requiresApproval: null,
requiresInput: null,
error: {
type: "runtime_error",
message: err?.message ?? String(err),
},
};
}
}
clone() {
const cloned = new Lobster(this.#options);
cloned.#stages = [...this.#stages];
cloned.#meta = this.#meta ? { ...this.#meta } : null;
return cloned;
}
clone() {
const cloned = new Lobster(this.#options);
cloned.#stages = [...this.#stages];
cloned.#meta = this.#meta ? { ...this.#meta } : null;
return cloned;
}
}
function decodeSdkResumePayload(token: string): SdkResumePayload {
const payload = decodeToken(token);
if (!payload || typeof payload !== "object") {
throw new Error("Invalid token");
}
const data = payload as Record<string, unknown>;
if (data.protocolVersion !== 1 || data.v !== 1) {
throw new Error("Invalid token");
}
if (
typeof data.resumeAtIndex !== "number" ||
!Number.isInteger(data.resumeAtIndex) ||
data.resumeAtIndex < 0
) {
throw new Error("Invalid token");
}
if (data.items !== undefined && !Array.isArray(data.items)) {
throw new Error("Invalid token");
}
if (
data.resumeMode !== undefined &&
(typeof data.resumeMode !== "string" || !["next_stage", "same_stage"].includes(data.resumeMode))
) {
throw new Error("Invalid token");
}
if (data.resumeMode === "same_stage") {
if (typeof data.stateKey !== "string" || data.stateKey.length === 0) {
throw new Error("Invalid token");
}
} else if (data.stateKey !== undefined) {
throw new Error("Invalid token");
}
if (data.commandInput !== undefined) throw new Error("Invalid token");
return data as unknown as SdkResumePayload;
const payload = decodeToken(token);
if (!payload || typeof payload !== "object") {
throw new Error("Invalid token");
}
const data = payload as Record<string, unknown>;
if (data.protocolVersion !== 1 || data.v !== 1) {
throw new Error("Invalid token");
}
if (
typeof data.resumeAtIndex !== "number" ||
!Number.isInteger(data.resumeAtIndex) ||
data.resumeAtIndex < 0
) {
throw new Error("Invalid token");
}
if (data.items !== undefined && !Array.isArray(data.items)) {
throw new Error("Invalid token");
}
if (
data.resumeMode !== undefined &&
(typeof data.resumeMode !== "string" || !["next_stage", "same_stage"].includes(data.resumeMode))
) {
throw new Error("Invalid token");
}
if (data.resumeMode === "same_stage") {
if (typeof data.stateKey !== "string" || data.stateKey.length === 0) {
throw new Error("Invalid token");
}
} else if (data.stateKey !== undefined) {
throw new Error("Invalid token");
}
if (data.commandInput !== undefined) throw new Error("Invalid token");
return data as unknown as SdkResumePayload;
}
function sdkStateEnv(options: any) {
return options.stateDir
? { ...(options.env ?? process.env), LOBSTER_STATE_DIR: options.stateDir }
: (options.env ?? process.env);
return options.stateDir
? { ...(options.env ?? process.env), LOBSTER_STATE_DIR: options.stateDir }
: (options.env ?? process.env);
}
async function saveSdkCommandInputResumeState(options: any, state: SdkCommandInputResumeState) {
const stateKey = `sdk_resume_${randomUUID()}`;
await writeStateJson({ env: sdkStateEnv(options), key: stateKey, value: state });
return stateKey;
const stateKey = `sdk_resume_${randomUUID()}`;
await writeStateJson({ env: sdkStateEnv(options), key: stateKey, value: state });
return stateKey;
}
async function loadSdkCommandInputResumeState(
options: any,
stateKey: string,
options: any,
stateKey: string,
): Promise<SdkCommandInputResumeState> {
const stored = await readStateJson({ env: sdkStateEnv(options), key: stateKey });
if (!stored || typeof stored !== "object") throw new Error("SDK resume state not found");
const data = stored as Partial<SdkCommandInputResumeState>;
if (
typeof data.resumeAtIndex !== "number" ||
!Number.isInteger(data.resumeAtIndex) ||
data.resumeAtIndex < 0
) {
throw new Error("Invalid SDK resume state");
}
if (!Array.isArray(data.items)) throw new Error("Invalid SDK resume state");
if (data.inputSchema === undefined) throw new Error("Invalid SDK resume state");
data.commandInput = validateCommandInputState(data.commandInput);
return data as SdkCommandInputResumeState;
const stored = await readStateJsonWithLock({
env: sdkStateEnv(options),
key: stateKey,
signal: options?.signal,
});
if (!stored || typeof stored !== "object") throw new Error("SDK resume state not found");
const data = stored as Partial<SdkCommandInputResumeState>;
if (
typeof data.resumeAtIndex !== "number" ||
!Number.isInteger(data.resumeAtIndex) ||
data.resumeAtIndex < 0
) {
throw new Error("Invalid SDK resume state");
}
if (!Array.isArray(data.items)) throw new Error("Invalid SDK resume state");
if (data.inputSchema === undefined) throw new Error("Invalid SDK resume state");
data.commandInput = validateCommandInputState(data.commandInput);
return data as SdkCommandInputResumeState;
}
src/sdk/primitives/approve.ts
+25 -25
View File
@@ -19,32 +19,32 @@
* @returns {Object} Stage object with run method
*/
export function approve(options: any = {}) {
const prompt = options.prompt ?? "Approve?";
const preview = options.preview !== false;
const prompt = options.prompt ?? "Approve?";
const preview = options.preview !== false;
return {
type: "approve",
prompt,
return {
type: "approve",
prompt,
async run({ input, ctx: _ctx }) {
// Collect all items
const items = [];
for await (const item of input) {
items.push(item);
}
async run({ input, ctx: _ctx }) {
// Collect all items
const items = [];
for await (const item of input) {
items.push(item);
}
// In SDK mode, always emit approval request and halt
return {
halt: true,
output: (async function* () {
yield {
type: "approval_request",
prompt,
items: preview ? items : [],
itemCount: items.length,
};
})(),
};
},
};
// In SDK mode, always emit approval request and halt
return {
halt: true,
output: (async function* () {
yield {
type: "approval_request",
prompt,
items: preview ? items : [],
itemCount: items.length,
};
})(),
};
},
};
}
src/sdk/primitives/diff.ts
+48 -127
View File
@@ -14,56 +14,12 @@
* });
*/
import { promises as fsp } from "node:fs";
import os from "node:os";
import path from "node:path";
import { ensureDirectory, isJsonSyntaxError, writeFileAtomic } from "../../state/store.js";
import { diffAndStore } from "../../state/store.js";
/**
* Get the state directory
* @param {Object} ctx
* @returns {string}
*/
function getStateDir(ctx) {
return (
ctx?.stateDir ||
(ctx?.env?.LOBSTER_STATE_DIR && String(ctx.env.LOBSTER_STATE_DIR).trim()) ||
path.join(os.homedir(), ".lobster", "state")
);
}
/**
* Convert a key to a safe file path
* @param {string} stateDir
* @param {string} key
* @returns {string}
*/
function keyToPath(stateDir, key) {
const safe = String(key)
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, "_")
.replace(/_+/g, "_")
.replace(/^_+|_+$/g, "");
if (!safe) throw new Error("state key is empty/invalid");
return path.join(stateDir, `${safe}.json`);
}
/**
* Stable JSON stringify for comparison
* @param {any} value
* @returns {string}
*/
function stableStringify(value) {
return JSON.stringify(value, (_k, v) => {
if (v && typeof v === "object" && !Array.isArray(v)) {
return Object.fromEntries(
Object.keys(v)
.sort()
.map((k) => [k, v[k]]),
);
}
return v;
});
function stateEnv(ctx) {
return ctx?.stateDir
? { ...(ctx?.env ?? process.env), LOBSTER_STATE_DIR: ctx.stateDir }
: (ctx?.env ?? process.env);
}
/**
@@ -78,69 +34,55 @@ function stableStringify(value) {
* @returns {Object} Stage object with run method
*/
export function diffLast(key, options: any = {}) {
if (!key) throw new Error("diffLast requires a key");
if (!key) throw new Error("diffLast requires a key");
const changesOnly = options.changesOnly === true;
const changesOnly = options.changesOnly === true;
return {
type: "diff.last",
key,
return {
type: "diff.last",
key,
async run({ input, ctx }) {
// Collect all input items
const items = [];
for await (const item of input) {
items.push(item);
}
async run({ input, ctx }) {
// Collect all input items
const items = [];
for await (const item of input) {
items.push(item);
}
const value = items.length === 1 ? items[0] : items;
const value = items.length === 1 ? items[0] : items;
const stateDir = getStateDir(ctx);
const filePath = keyToPath(stateDir, key);
const { before, after, changed } = await diffAndStore({
env: stateEnv(ctx),
key,
value,
signal: ctx?.signal,
});
// Read previous value
let before = null;
try {
const text = await fsp.readFile(filePath, "utf8");
before = JSON.parse(text);
} catch (err) {
if (err?.code !== "ENOENT" && !isJsonSyntaxError(err)) {
throw err;
}
}
// Build result
const result = {
kind: "diff.last",
key,
changed,
before,
after,
};
// Compare
const changed = stableStringify(before) !== stableStringify(value);
// If changesOnly and no change, output suppressed marker
if (changesOnly && !changed) {
return {
output: (async function* () {
yield { kind: "diff.last", key, changed: false, suppressed: true };
})(),
};
}
// Store new value
await ensureDirectory(stateDir);
await writeFileAtomic(filePath, JSON.stringify(value, null, 2) + "\n");
// Build result
const result = {
kind: "diff.last",
key,
changed,
before,
after: value,
};
// If changesOnly and no change, output suppressed marker
if (changesOnly && !changed) {
return {
output: (async function* () {
yield { kind: "diff.last", key, changed: false, suppressed: true };
})(),
};
}
return {
output: (async function* () {
yield result;
})(),
};
},
};
return {
output: (async function* () {
yield result;
})(),
};
},
};
}
/**
@@ -150,27 +92,6 @@ export function diffLast(key, options: any = {}) {
* @param {Object} [ctx]
* @returns {Promise<{before: any, after: any, changed: boolean}>}
*/
export async function diffAndStoreValue(key, value, ctx = {}) {
const stateDir = getStateDir(ctx);
const filePath = keyToPath(stateDir, key);
// Read previous value
let before = null;
try {
const text = await fsp.readFile(filePath, "utf8");
before = JSON.parse(text);
} catch (err) {
if (err?.code !== "ENOENT" && !isJsonSyntaxError(err)) {
throw err;
}
}
// Compare
const changed = stableStringify(before) !== stableStringify(value);
// Store new value
await ensureDirectory(stateDir);
await writeFileAtomic(filePath, JSON.stringify(value, null, 2) + "\n");
return { before, after: value, changed };
export async function diffAndStoreValue(key, value, ctx: any = {}) {
return diffAndStore({ env: stateEnv(ctx), key, value, signal: ctx?.signal });
}
src/sdk/primitives/exec.ts
+111 -111
View File
@@ -20,39 +20,39 @@ import { resolveInlineShellCommand } from "../../shell.js";
* @returns {Promise<{stdout: string, stderr: string}>}
*/
function runProcess(command, argv, { env, cwd }) {
return new Promise<any>((resolve, reject) => {
const child = spawn(command, argv, {
env,
cwd,
stdio: ["ignore", "pipe", "pipe"],
shell: false,
});
return new Promise<any>((resolve, reject) => {
const child = spawn(command, argv, {
env,
cwd,
stdio: ["ignore", "pipe", "pipe"],
shell: false,
});
let stdout = "";
let stderr = "";
let stdout = "";
let stderr = "";
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
child.on("error", (err) => {
reject(new Error(`Failed to execute ${command}: ${err.message}`));
});
child.on("error", (err) => {
reject(new Error(`Failed to execute ${command}: ${err.message}`));
});
child.on("close", (code) => {
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(new Error(`${command} exited with code ${code}: ${stderr.trim() || stdout.trim()}`));
}
});
});
child.on("close", (code) => {
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(new Error(`${command} exited with code ${code}: ${stderr.trim() || stdout.trim()}`));
}
});
});
}
/**
@@ -62,49 +62,49 @@ function runProcess(command, argv, { env, cwd }) {
* @returns {{command: string, args: string[]}}
*/
function parseCommand(cmdString) {
const tokens = [];
let current = "";
let quote = null;
const tokens = [];
let current = "";
let quote = null;
for (let i = 0; i < cmdString.length; i++) {
const ch = cmdString[i];
for (let i = 0; i < cmdString.length; i++) {
const ch = cmdString[i];
if (quote) {
if (ch === "\\" && cmdString[i + 1]) {
current += cmdString[i + 1];
i++;
continue;
}
if (ch === quote) {
quote = null;
continue;
}
current += ch;
continue;
}
if (quote) {
if (ch === "\\" && cmdString[i + 1]) {
current += cmdString[i + 1];
i++;
continue;
}
if (ch === quote) {
quote = null;
continue;
}
current += ch;
continue;
}
if (ch === '"' || ch === "'") {
quote = ch;
continue;
}
if (ch === '"' || ch === "'") {
quote = ch;
continue;
}
if (ch === " " || ch === "\t") {
if (current.length > 0) {
tokens.push(current);
current = "";
}
continue;
}
if (ch === " " || ch === "\t") {
if (current.length > 0) {
tokens.push(current);
current = "";
}
continue;
}
current += ch;
}
current += ch;
}
if (current.length > 0) {
tokens.push(current);
}
if (current.length > 0) {
tokens.push(current);
}
const [command, ...args] = tokens;
return { command, args };
const [command, ...args] = tokens;
return { command, args };
}
/**
@@ -118,60 +118,60 @@ function parseCommand(cmdString) {
* @returns {Object} Stage object with run method
*/
export function exec(cmdString, options: any = {}) {
const parseJson = options.json !== false;
const useShell = options.shell === true;
const cwd = options.cwd ?? process.cwd();
const parseJson = options.json !== false;
const useShell = options.shell === true;
const cwd = options.cwd ?? process.cwd();
return {
type: "exec",
command: cmdString,
return {
type: "exec",
command: cmdString,
async run({ input, ctx }) {
// Drain input (exec doesn't use input stream)
for await (const _item of input) {
// no-op
}
async run({ input, ctx }) {
// Drain input (exec doesn't use input stream)
for await (const _item of input) {
// no-op
}
const env = ctx.env ?? process.env;
const env = ctx.env ?? process.env;
let stdout;
let stdout;
if (useShell) {
// Shell execution
const shell = resolveInlineShellCommand({ command: cmdString, env });
const result = await runProcess(shell.command, shell.argv, { env, cwd });
stdout = result.stdout;
} else {
// Direct execution
const { command, args } = parseCommand(cmdString);
const result = await runProcess(command, args, { env, cwd });
stdout = result.stdout;
}
if (useShell) {
// Shell execution
const shell = resolveInlineShellCommand({ command: cmdString, env });
const result = await runProcess(shell.command, shell.argv, { env, cwd });
stdout = result.stdout;
} else {
// Direct execution
const { command, args } = parseCommand(cmdString);
const result = await runProcess(command, args, { env, cwd });
stdout = result.stdout;
}
// Parse output
let output;
if (parseJson) {
try {
output = JSON.parse(stdout.trim() || "[]");
} catch {
throw new Error(`exec output is not valid JSON: ${stdout.slice(0, 100)}`);
}
} else {
output = stdout;
}
// Parse output
let output;
if (parseJson) {
try {
output = JSON.parse(stdout.trim() || "[]");
} catch {
throw new Error(`exec output is not valid JSON: ${stdout.slice(0, 100)}`);
}
} else {
output = stdout;
}
// Normalize to array
const items = Array.isArray(output) ? output : [output];
// Normalize to array
const items = Array.isArray(output) ? output : [output];
return {
output: (async function* () {
for (const item of items) {
yield item;
}
})(),
};
},
};
return {
output: (async function* () {
for (const item of items) {
yield item;
}
})(),
};
},
};
}
/**
@@ -183,5 +183,5 @@ export function exec(cmdString, options: any = {}) {
* @returns {Object}
*/
export function shell(cmdString, options = {}) {
return exec(cmdString, { ...options, shell: true });
return exec(cmdString, { ...options, shell: true });
}
src/sdk/primitives/state.ts
+48 -142
View File
@@ -15,77 +15,12 @@
* .pipe(stateSet('my-key'));
*/
import { randomBytes } from "node:crypto";
import { promises as fsp } from "node:fs";
import os from "node:os";
import path from "node:path";
import { readStateJsonWithLock, writeStateJson } from "../../state/store.js";
/**
* Write a file atomically (stage to a sibling temp file, fsync, then rename).
* `rename(2)` is atomic on a single filesystem, so a concurrent reader or a
* crash never observes a truncated/partial file. Plain `fsp.writeFile`
* truncates the target up front, leaving a corruption window on SIGKILL/OOM/
* power loss. New state files are private by default; existing file modes are
* preserved across replacement. Kept local to keep the SDK self-contained.
* @param {string} filePath
* @param {string} data
*/
async function writeFileAtomic(filePath, data) {
const dir = path.dirname(filePath);
const tmpPath = path.join(
dir,
`.${path.basename(filePath)}.${randomBytes(6).toString("hex")}.tmp`,
);
let mode = 0o600;
let handle;
let cleanup = true;
try {
try {
mode = (await fsp.stat(filePath)).mode & 0o777;
} catch (err) {
if (err?.code !== "ENOENT") throw err;
}
handle = await fsp.open(tmpPath, "wx", mode);
await handle.writeFile(data, "utf8");
await handle.sync();
await handle.close();
handle = undefined;
await fsp.chmod(tmpPath, mode);
await fsp.rename(tmpPath, filePath);
cleanup = false;
} finally {
if (handle) await handle.close().catch(() => {});
if (cleanup) await fsp.rm(tmpPath, { force: true }).catch(() => {});
}
}
/**
* Get the state directory
* @param {Object} ctx
* @returns {string}
*/
function getStateDir(ctx) {
return (
ctx?.stateDir ||
(ctx?.env?.LOBSTER_STATE_DIR && String(ctx.env.LOBSTER_STATE_DIR).trim()) ||
path.join(os.homedir(), ".lobster", "state")
);
}
/**
* Convert a key to a safe file path
* @param {string} stateDir
* @param {string} key
* @returns {string}
*/
function keyToPath(stateDir, key) {
const safe = String(key)
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, "_")
.replace(/_+/g, "_")
.replace(/^_+|_+$/g, "");
if (!safe) throw new Error("state key is empty/invalid");
return path.join(stateDir, `${safe}.json`);
function stateEnv(ctx) {
return ctx?.stateDir
? { ...(ctx?.env ?? process.env), LOBSTER_STATE_DIR: ctx.stateDir }
: (ctx?.env ?? process.env);
}
/**
@@ -95,39 +30,27 @@ function keyToPath(stateDir, key) {
* @returns {Object} Stage object with run method
*/
export function stateGet(key) {
if (!key) throw new Error("stateGet requires a key");
if (!key) throw new Error("stateGet requires a key");
return {
type: "state.get",
key,
return {
type: "state.get",
key,
async run({ input, ctx }) {
// Drain input
for await (const _item of input) {
// no-op
}
async run({ input, ctx }) {
// Drain input
for await (const _item of input) {
// no-op
}
const stateDir = getStateDir(ctx);
const filePath = keyToPath(stateDir, key);
const value = await readStateJsonWithLock({ env: stateEnv(ctx), key, signal: ctx?.signal });
let value = null;
try {
const text = await fsp.readFile(filePath, "utf8");
value = JSON.parse(text);
} catch (err) {
if (err?.code !== "ENOENT") {
throw err;
}
// File doesn't exist, return null
}
return {
output: (async function* () {
yield value;
})(),
};
},
};
return {
output: (async function* () {
yield value;
})(),
};
},
};
}
/**
@@ -137,35 +60,31 @@ export function stateGet(key) {
* @returns {Object} Stage object with run method
*/
export function stateSet(key) {
if (!key) throw new Error("stateSet requires a key");
if (!key) throw new Error("stateSet requires a key");
return {
type: "state.set",
key,
return {
type: "state.set",
key,
async run({ input, ctx }) {
// Collect all input items
const items = [];
for await (const item of input) {
items.push(item);
}
async run({ input, ctx }) {
// Collect all input items
const items = [];
for await (const item of input) {
items.push(item);
}
const value = items.length === 1 ? items[0] : items;
const value = items.length === 1 ? items[0] : items;
const stateDir = getStateDir(ctx);
const filePath = keyToPath(stateDir, key);
await writeStateJson({ env: stateEnv(ctx), key, value, signal: ctx?.signal });
await fsp.mkdir(stateDir, { recursive: true });
await writeFileAtomic(filePath, JSON.stringify(value, null, 2) + "\n");
// Pass through the value
return {
output: (async function* () {
yield value;
})(),
};
},
};
// Pass through the value
return {
output: (async function* () {
yield value;
})(),
};
},
};
}
/**
@@ -179,8 +98,8 @@ export function stateSet(key) {
* .pipe(state.set('my-key'));
*/
export const state = {
get: stateGet,
set: stateSet,
get: stateGet,
set: stateSet,
};
/**
@@ -189,17 +108,8 @@ export const state = {
* @param {Object} [ctx]
* @returns {Promise<any>}
*/
export async function readState(key, ctx = {}) {
const stateDir = getStateDir(ctx);
const filePath = keyToPath(stateDir, key);
try {
const text = await fsp.readFile(filePath, "utf8");
return JSON.parse(text);
} catch (err) {
if (err?.code === "ENOENT") return null;
throw err;
}
export async function readState(key, ctx: any = {}) {
return readStateJsonWithLock({ env: stateEnv(ctx), key, signal: ctx?.signal });
}
/**
@@ -209,10 +119,6 @@ export async function readState(key, ctx = {}) {
* @param {Object} [ctx]
* @returns {Promise<void>}
*/
export async function writeState(key, value, ctx = {}) {
const stateDir = getStateDir(ctx);
const filePath = keyToPath(stateDir, key);
await fsp.mkdir(stateDir, { recursive: true });
await writeFileAtomic(filePath, JSON.stringify(value, null, 2) + "\n");
export async function writeState(key, value, ctx: any = {}) {
await writeStateJson({ env: stateEnv(ctx), key, value, signal: ctx?.signal });
}
src/sdk/runtime.ts
+103 -103
View File
@@ -27,34 +27,34 @@ import { runPipeline as runCorePipeline } from "../runtime.js";
* @returns {Promise<any[]>}
*/
async function collectItems(iterable) {
const items = [];
for await (const item of iterable) {
items.push(item);
}
return items;
const items = [];
for await (const item of iterable) {
items.push(item);
}
return items;
}
function normalizeSdkOutput(output) {
if (output === null || output === undefined) return [];
if (Array.isArray(output)) return output;
if (
typeof output?.[Symbol.asyncIterator] === "function" ||
typeof output?.[Symbol.iterator] === "function"
) {
return output;
}
return [output];
if (output === null || output === undefined) return [];
if (Array.isArray(output)) return output;
if (
typeof output?.[Symbol.asyncIterator] === "function" ||
typeof output?.[Symbol.iterator] === "function"
) {
return output;
}
return [output];
}
function createNullWritable() {
return {
write() {
return true;
},
end() {
return undefined;
},
};
return {
write() {
return true;
},
end() {
return undefined;
},
};
}
/**
@@ -67,98 +67,98 @@ function createNullWritable() {
* @returns {Promise<PipelineResult>}
*/
export async function runPipelineInternal({
stages,
ctx,
input = [],
requestInputResume = undefined,
stages,
ctx,
input = [],
requestInputResume = undefined,
}) {
const runtimeCtx = ctx ?? {};
const pipeline = stages.map((_stage, index) => ({
name: `sdk.stage.${index}`,
args: {},
raw: `sdk.stage.${index}`,
}));
const commands = new Map(
stages.map((stage, index) => [
`sdk.stage.${index}`,
{
async run({ input, ctx }) {
const stageCtx = { ...runtimeCtx, ...ctx };
if (typeof stage === "function") {
const isGenerator =
stage.constructor?.name === "AsyncGeneratorFunction" ||
stage.constructor?.name === "GeneratorFunction";
const runtimeCtx = ctx ?? {};
const pipeline = stages.map((_stage, index) => ({
name: `sdk.stage.${index}`,
args: {},
raw: `sdk.stage.${index}`,
}));
const commands = new Map(
stages.map((stage, index) => [
`sdk.stage.${index}`,
{
async run({ input, ctx }) {
const stageCtx = { ...runtimeCtx, ...ctx };
if (typeof stage === "function") {
const isGenerator =
stage.constructor?.name === "AsyncGeneratorFunction" ||
stage.constructor?.name === "GeneratorFunction";
if (isGenerator) {
return { output: normalizeSdkOutput(stage(input, stageCtx)) };
}
if (isGenerator) {
return { output: normalizeSdkOutput(stage(input, stageCtx)) };
}
const items = await collectItems(input);
return { output: normalizeSdkOutput(await stage(items, stageCtx)) };
}
const items = await collectItems(input);
return { output: normalizeSdkOutput(await stage(items, stageCtx)) };
}
if (typeof stage?.run === "function") {
const result = await stage.run({ input, ctx: stageCtx });
return result && "output" in result
? { ...result, output: normalizeSdkOutput(result.output) }
: result;
}
if (typeof stage?.run === "function") {
const result = await stage.run({ input, ctx: stageCtx });
return result && "output" in result
? { ...result, output: normalizeSdkOutput(result.output) }
: result;
}
throw new Error(
`Invalid stage at index ${index}: must be a function or have run() method`,
);
},
},
]),
);
const stdout = runtimeCtx.stdout ?? createNullWritable();
const stderr = runtimeCtx.stderr ?? createNullWritable();
throw new Error(
`Invalid stage at index ${index}: must be a function or have run() method`,
);
},
},
]),
);
const stdout = runtimeCtx.stdout ?? createNullWritable();
const stderr = runtimeCtx.stderr ?? createNullWritable();
return runCorePipeline({
pipeline,
registry: {
get(name) {
return commands.get(name);
},
},
stdin: runtimeCtx.stdin ?? { isTTY: false },
stdout,
stderr,
env: runtimeCtx.env ?? process.env,
mode: runtimeCtx.mode ?? "sdk",
cwd: runtimeCtx.cwd,
llmAdapters: runtimeCtx.llmAdapters,
signal: runtimeCtx.signal,
input: normalizeSdkOutput(input),
requestInputResume,
});
return runCorePipeline({
pipeline,
registry: {
get(name) {
return commands.get(name);
},
},
stdin: runtimeCtx.stdin ?? { isTTY: false },
stdout,
stderr,
env: runtimeCtx.env ?? process.env,
mode: runtimeCtx.mode ?? "sdk",
cwd: runtimeCtx.cwd,
llmAdapters: runtimeCtx.llmAdapters,
signal: runtimeCtx.signal,
input: normalizeSdkOutput(input),
requestInputResume,
});
}
/**
* Re-export for compatibility with CLI runtime
*/
export async function runPipeline({
pipeline,
registry,
stdin,
stdout,
stderr,
env,
mode = "human",
input,
requestInputResume = undefined,
requestInputEnabled = true,
pipeline,
registry,
stdin,
stdout,
stderr,
env,
mode = "human",
input,
requestInputResume = undefined,
requestInputEnabled = true,
}) {
return runCorePipeline({
pipeline,
registry,
stdin,
stdout,
stderr,
env,
mode,
input,
requestInputResume,
requestInputEnabled,
});
return runCorePipeline({
pipeline,
registry,
stdin,
stdout,
stderr,
env,
mode,
input,
requestInputResume,
requestInputEnabled,
});
}
src/shell.ts
+47 -47
View File
@@ -1,60 +1,60 @@
export function resolveInlineShellCommand({
command,
env,
platform = process.platform,
command,
env,
platform = process.platform,
}: {
command: string;
env: Record<string, string | undefined>;
platform?: string;
command: string;
env: Record<string, string | undefined>;
platform?: string;
}) {
const shellOverride = String(env?.LOBSTER_SHELL ?? "").trim();
const isWindows = platform === "win32";
const shellOverride = String(env?.LOBSTER_SHELL ?? "").trim();
const isWindows = platform === "win32";
if (shellOverride) {
return {
command: shellOverride,
argv: buildShellArgs({ shellCommand: shellOverride, command, isWindows }),
};
}
if (shellOverride) {
return {
command: shellOverride,
argv: buildShellArgs({ shellCommand: shellOverride, command, isWindows }),
};
}
if (isWindows) {
const comspec = String(env?.ComSpec ?? env?.COMSPEC ?? "cmd.exe").trim() || "cmd.exe";
return {
command: comspec,
argv: ["/d", "/s", "/c", command],
};
}
if (isWindows) {
const comspec = String(env?.ComSpec ?? env?.COMSPEC ?? "cmd.exe").trim() || "cmd.exe";
return {
command: comspec,
argv: ["/d", "/s", "/c", command],
};
}
// Keep default behavior deterministic and POSIX-compatible across environments.
const shell = "/bin/sh";
return {
command: shell,
argv: ["-lc", command],
};
// Keep default behavior deterministic and POSIX-compatible across environments.
const shell = "/bin/sh";
return {
command: shell,
argv: ["-lc", command],
};
}
function buildShellArgs({
shellCommand,
command,
isWindows,
shellCommand,
command,
isWindows,
}: {
shellCommand: string;
command: string;
isWindows: boolean;
shellCommand: string;
command: string;
isWindows: boolean;
}) {
const lowered = shellCommand.toLowerCase();
const looksLikeCmd = lowered.endsWith("cmd") || lowered.endsWith("cmd.exe");
const looksLikePowerShell =
lowered.endsWith("powershell") ||
lowered.endsWith("powershell.exe") ||
lowered.endsWith("pwsh") ||
lowered.endsWith("pwsh.exe");
const lowered = shellCommand.toLowerCase();
const looksLikeCmd = lowered.endsWith("cmd") || lowered.endsWith("cmd.exe");
const looksLikePowerShell =
lowered.endsWith("powershell") ||
lowered.endsWith("powershell.exe") ||
lowered.endsWith("pwsh") ||
lowered.endsWith("pwsh.exe");
if (looksLikePowerShell) {
return ["-NoProfile", "-Command", command];
}
if (looksLikeCmd || isWindows) {
return ["/d", "/s", "/c", command];
}
return ["-lc", command];
if (looksLikePowerShell) {
return ["-NoProfile", "-Command", command];
}
if (looksLikeCmd || isWindows) {
return ["/d", "/s", "/c", command];
}
return ["-lc", command];
}
src/state/store.ts
+977 -266
View File
File diff suppressed because it is too large Load Diff
src/token.ts
+8 -8
View File
@@ -1,15 +1,15 @@
import { Buffer } from "node:buffer";
export function encodeToken(obj) {
const json = JSON.stringify(obj);
return Buffer.from(json, "utf8").toString("base64url");
const json = JSON.stringify(obj);
return Buffer.from(json, "utf8").toString("base64url");
}
export function decodeToken(token) {
try {
const json = Buffer.from(String(token), "base64url").toString("utf8");
return JSON.parse(json);
} catch (_err) {
throw new Error("Invalid token");
}
try {
const json = Buffer.from(String(token), "base64url").toString("utf8");
return JSON.parse(json);
} catch (_err) {
throw new Error("Invalid token");
}
}
src/validation.ts
+14 -14
View File
@@ -3,10 +3,10 @@ import { Ajv, type AnySchema, type ValidateFunction } from "ajv";
import { stableStringify } from "./state/store.js";
export const sharedAjv = new Ajv({
allErrors: false,
strict: false,
// User-provided schemas may repeat `$id` across runs/resumes.
addUsedSchema: false,
allErrors: false,
strict: false,
// User-provided schemas may repeat `$id` across runs/resumes.
addUsedSchema: false,
});
/**
@@ -25,16 +25,16 @@ export const sharedAjv = new Ajv({
* interchangeable across different Ajv configurations.
*/
export function createCompileCached(ajv: Ajv): (schema: AnySchema) => ValidateFunction {
const cache = new Map<string, ValidateFunction>();
return function compileCached(schema: AnySchema): ValidateFunction {
const key = stableStringify(schema);
let validator = cache.get(key);
if (!validator) {
validator = ajv.compile(schema);
cache.set(key, validator);
}
return validator;
};
const cache = new Map<string, ValidateFunction>();
return function compileCached(schema: AnySchema): ValidateFunction {
const key = stableStringify(schema);
let validator = cache.get(key);
if (!validator) {
validator = ajv.compile(schema);
cache.set(key, validator);
}
return validator;
};
}
/** Memoized compile bound to the module-level `sharedAjv` instance. */
src/workflows/file.ts
+3375 -2666
View File
File diff suppressed because it is too large Load Diff
src/workflows/github_pr_monitor.ts
+148 -154
View File
@@ -1,188 +1,182 @@
import { spawn } from "node:child_process";
import { runAbortableProcess } from "../abortable_process.js";
function runProcess(command, argv, { env, cwd }) {
return new Promise((resolve, reject) => {
const child = spawn(command, argv, { env, cwd, stdio: ["ignore", "pipe", "pipe"] });
let stdout = "";
let stderr = "";
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
child.on("error", (err: any) => {
if (err?.code === "ENOENT") {
reject(new Error("gh not found on PATH (install GitHub CLI)"));
return;
}
reject(err);
});
child.on("close", (code) => {
if (code === 0) return resolve({ stdout, stderr });
reject(new Error(`gh failed (${code}): ${stderr.trim() || stdout.trim()}`));
});
});
async function runProcess(command, argv, { env, cwd, signal, forceTerminationSignal }) {
const { stdout, stderr, code } = await runAbortableProcess({
command,
argv,
env,
cwd,
signal,
forceTerminationSignal,
notFoundMessage: "gh not found on PATH (install GitHub CLI)",
});
if (code === 0) return { stdout, stderr };
throw new Error(`gh failed (${code}): ${stderr.trim() || stdout.trim()}`);
}
import { diffAndStore } from "../state/store.js";
function pickSubset(snapshot) {
if (!snapshot || typeof snapshot !== "object") return null;
return {
number: snapshot.number,
title: snapshot.title,
url: snapshot.url,
state: snapshot.state,
isDraft: snapshot.isDraft,
mergeable: snapshot.mergeable,
reviewDecision: snapshot.reviewDecision,
updatedAt: snapshot.updatedAt,
baseRefName: snapshot.baseRefName,
headRefName: snapshot.headRefName,
};
if (!snapshot || typeof snapshot !== "object") return null;
return {
number: snapshot.number,
title: snapshot.title,
url: snapshot.url,
state: snapshot.state,
isDraft: snapshot.isDraft,
mergeable: snapshot.mergeable,
reviewDecision: snapshot.reviewDecision,
updatedAt: snapshot.updatedAt,
baseRefName: snapshot.baseRefName,
headRefName: snapshot.headRefName,
};
}
export function buildPrChangeSummary(before, after) {
const a = pickSubset(after);
const b = pickSubset(before);
const a = pickSubset(after);
const b = pickSubset(before);
if (!a) return { changedFields: [], changes: {} };
if (!b) {
return {
changedFields: Object.keys(a),
changes: Object.fromEntries(Object.keys(a).map((k) => [k, { from: null, to: a[k] }])),
};
}
if (!a) return { changedFields: [], changes: {} };
if (!b) {
return {
changedFields: Object.keys(a),
changes: Object.fromEntries(Object.keys(a).map((k) => [k, { from: null, to: a[k] }])),
};
}
const changes = {};
for (const key of Object.keys(a)) {
if (JSON.stringify(a[key]) !== JSON.stringify(b[key])) {
changes[key] = { from: b[key], to: a[key] };
}
}
const changes = {};
for (const key of Object.keys(a)) {
if (JSON.stringify(a[key]) !== JSON.stringify(b[key])) {
changes[key] = { from: b[key], to: a[key] };
}
}
return {
changedFields: Object.keys(changes),
changes,
};
return {
changedFields: Object.keys(changes),
changes,
};
}
function formatPrChangeMessage({ repo, pr, changedFields, prInfo }) {
const fields = changedFields.length ? ` (${changedFields.join(", ")})` : "";
const title = prInfo?.title ? `: ${prInfo.title}` : "";
const url = prInfo?.url ? ` ${prInfo.url}` : "";
return `PR updated: ${repo}#${pr}${title}${fields}.${url}`.replace(/\s+/g, " ").trim();
const fields = changedFields.length ? ` (${changedFields.join(", ")})` : "";
const title = prInfo?.title ? `: ${prInfo.title}` : "";
const url = prInfo?.url ? ` ${prInfo.url}` : "";
return `PR updated: ${repo}#${pr}${title}${fields}.${url}`.replace(/\s+/g, " ").trim();
}
export async function runGithubPrMonitorWorkflow({ args, ctx }) {
const repo = args.repo;
const pr = args.pr;
if (!repo || !pr) throw new Error("github.pr.monitor requires args.repo and args.pr");
ctx.signal?.throwIfAborted();
const repo = args.repo;
const pr = args.pr;
if (!repo || !pr) throw new Error("github.pr.monitor requires args.repo and args.pr");
const key = args.key ?? `github.pr:${repo}#${pr}`;
const changesOnly = Boolean(args.changesOnly);
const summaryOnly = Boolean(args.summaryOnly);
const key = args.key ?? `github.pr:${repo}#${pr}`;
const changesOnly = Boolean(args.changesOnly);
const summaryOnly = Boolean(args.summaryOnly);
const argv = [
"pr",
"view",
String(pr),
"--repo",
String(repo),
"--json",
"number,title,url,state,isDraft,mergeable,reviewDecision,author,baseRefName,headRefName,updatedAt",
];
const argv = [
"pr",
"view",
String(pr),
"--repo",
String(repo),
"--json",
"number,title,url,state,isDraft,mergeable,reviewDecision,author,baseRefName,headRefName,updatedAt",
];
const { stdout } = (await runProcess("gh", argv, { env: ctx.env, cwd: process.cwd() })) as any;
const { stdout } = (await runProcess("gh", argv, {
env: ctx.env,
cwd: process.cwd(),
signal: ctx.signal,
forceTerminationSignal: ctx.forceTerminationSignal,
})) as any;
ctx.signal?.throwIfAborted();
let current;
try {
current = JSON.parse(stdout.trim());
} catch {
throw new Error("gh returned non-JSON output");
}
let current;
try {
current = JSON.parse(stdout.trim());
} catch {
throw new Error("gh returned non-JSON output");
}
const { changed, before } = await diffAndStore({ env: ctx.env, key, value: current });
const { changed, before } = await diffAndStore({
env: ctx.env,
key,
value: current,
signal: ctx.signal,
});
if (changesOnly && !changed) {
return {
kind: "github.pr.monitor",
repo,
prNumber: Number(pr),
key,
changed: false,
suppressed: true,
};
}
if (changesOnly && !changed) {
return {
kind: "github.pr.monitor",
repo,
prNumber: Number(pr),
key,
changed: false,
suppressed: true,
};
}
const summary = buildPrChangeSummary(before, current);
const summary = buildPrChangeSummary(before, current);
if (summaryOnly) {
return {
kind: "github.pr.monitor",
repo,
prNumber: Number(pr),
key,
changed,
summary,
pr: {
number: current.number,
title: current.title,
url: current.url,
state: current.state,
updatedAt: current.updatedAt,
},
};
}
if (summaryOnly) {
return {
kind: "github.pr.monitor",
repo,
prNumber: Number(pr),
key,
changed,
summary,
pr: {
number: current.number,
title: current.title,
url: current.url,
state: current.state,
updatedAt: current.updatedAt,
},
};
}
return {
kind: "github.pr.monitor",
repo,
prNumber: Number(pr),
key,
changed,
summary,
prSnapshot: current,
};
return {
kind: "github.pr.monitor",
repo,
prNumber: Number(pr),
key,
changed,
summary,
prSnapshot: current,
};
}
export async function runGithubPrMonitorNotifyWorkflow({ args, ctx }) {
const base = await runGithubPrMonitorWorkflow({
args: {
...args,
changesOnly: true,
summaryOnly: true,
},
ctx,
});
const base = await runGithubPrMonitorWorkflow({
args: {
...args,
changesOnly: true,
summaryOnly: true,
},
ctx,
});
if (base.suppressed) {
return { kind: "github.pr.monitor.notify", suppressed: true };
}
if (base.suppressed) {
return { kind: "github.pr.monitor.notify", suppressed: true };
}
const changedFields = base.summary?.changedFields ?? [];
const prInfo = base.pr ?? {};
const changedFields = base.summary?.changedFields ?? [];
const prInfo = base.pr ?? {};
return {
kind: "github.pr.monitor.notify",
changed: Boolean(base.changed),
repo: args.repo,
prNumber: Number(args.pr),
message: formatPrChangeMessage({
repo: args.repo,
pr: Number(args.pr),
changedFields,
prInfo,
}),
pr: prInfo,
summary: base.summary,
};
return {
kind: "github.pr.monitor.notify",
changed: Boolean(base.changed),
repo: args.repo,
prNumber: Number(args.pr),
message: formatPrChangeMessage({
repo: args.repo,
pr: Number(args.pr),
changedFields,
prInfo,
}),
pr: prInfo,
summary: base.summary,
};
}
src/workflows/graph.ts
+184 -184
View File
@@ -3,251 +3,251 @@ import type { WorkflowFile, WorkflowStep } from "./file.js";
export type WorkflowGraphFormat = "mermaid" | "dot" | "ascii";
type GraphNode = {
id: string;
type: string;
label: string;
shape: "box" | "diamond";
id: string;
type: string;
label: string;
shape: "box" | "diamond";
};
type GraphEdge = {
from: string;
to: string;
label?: string;
from: string;
to: string;
label?: string;
};
type RenderGraphParams = {
workflow: WorkflowFile;
format: WorkflowGraphFormat;
args?: Record<string, unknown>;
workflow: WorkflowFile;
format: WorkflowGraphFormat;
args?: Record<string, unknown>;
};
function resolveArgsTemplate(input: string, args: Record<string, unknown>) {
return input.replace(/\$\{([A-Za-z0-9_-]+)\}/g, (match, key) => {
if (key in args) return String(args[key]);
return match;
});
return input.replace(/\$\{([A-Za-z0-9_-]+)\}/g, (match, key) => {
if (key in args) return String(args[key]);
return match;
});
}
function isApprovalStep(step: WorkflowStep) {
if (step.approval === true) return true;
if (typeof step.approval === "string" && step.approval.trim().length > 0) return true;
if (step.approval && typeof step.approval === "object" && !Array.isArray(step.approval))
return true;
return false;
if (step.approval === true) return true;
if (typeof step.approval === "string" && step.approval.trim().length > 0) return true;
if (step.approval && typeof step.approval === "object" && !Array.isArray(step.approval))
return true;
return false;
}
function isInputStep(step: WorkflowStep) {
return Boolean(step.input && typeof step.input === "object" && !Array.isArray(step.input));
return Boolean(step.input && typeof step.input === "object" && !Array.isArray(step.input));
}
function stepType(step: WorkflowStep) {
if (step.parallel) return "parallel";
if (typeof step.for_each === "string") return "for_each";
if (typeof step.workflow === "string" && step.workflow.trim()) return "workflow";
if (typeof step.pipeline === "string" && step.pipeline.trim()) return "pipeline";
if (typeof step.run === "string" || typeof step.command === "string") return "run";
if (isApprovalStep(step)) return "approval";
if (isInputStep(step)) return "input";
return "step";
if (step.parallel) return "parallel";
if (typeof step.for_each === "string") return "for_each";
if (typeof step.workflow === "string" && step.workflow.trim()) return "workflow";
if (typeof step.pipeline === "string" && step.pipeline.trim()) return "pipeline";
if (typeof step.run === "string" || typeof step.command === "string") return "run";
if (isApprovalStep(step)) return "approval";
if (isInputStep(step)) return "input";
return "step";
}
function stepDetails(step: WorkflowStep, args: Record<string, unknown>) {
if (step.parallel) {
return `parallel (${step.parallel.wait ?? "all"})`;
}
if (typeof step.for_each === "string") {
return `for_each: ${resolveArgsTemplate(step.for_each, args)}`;
}
if (typeof step.workflow === "string" && step.workflow.trim()) {
return `workflow: ${resolveArgsTemplate(step.workflow, args)}`;
}
if (typeof step.pipeline === "string" && step.pipeline.trim()) {
return `pipeline: ${resolveArgsTemplate(step.pipeline, args)}`;
}
const shell = typeof step.run === "string" ? step.run : step.command;
if (typeof shell === "string" && shell.trim()) {
return `run: ${resolveArgsTemplate(shell, args)}`;
}
if (isApprovalStep(step)) return "approval gate";
if (isInputStep(step)) return "input request";
return "";
if (step.parallel) {
return `parallel (${step.parallel.wait ?? "all"})`;
}
if (typeof step.for_each === "string") {
return `for_each: ${resolveArgsTemplate(step.for_each, args)}`;
}
if (typeof step.workflow === "string" && step.workflow.trim()) {
return `workflow: ${resolveArgsTemplate(step.workflow, args)}`;
}
if (typeof step.pipeline === "string" && step.pipeline.trim()) {
return `pipeline: ${resolveArgsTemplate(step.pipeline, args)}`;
}
const shell = typeof step.run === "string" ? step.run : step.command;
if (typeof shell === "string" && shell.trim()) {
return `run: ${resolveArgsTemplate(shell, args)}`;
}
if (isApprovalStep(step)) return "approval gate";
if (isInputStep(step)) return "input request";
return "";
}
function extractStepRefsFromString(value: string): string[] {
const refs = new Set<string>();
const rx = /\$([A-Za-z0-9_-]+)\.[A-Za-z0-9_]+(?:\.[A-Za-z0-9_]+)*/g;
for (const m of value.matchAll(rx)) {
if (m[1]) refs.add(m[1]);
}
return [...refs];
const refs = new Set<string>();
const rx = /\$([A-Za-z0-9_-]+)\.[A-Za-z0-9_]+(?:\.[A-Za-z0-9_]+)*/g;
for (const m of value.matchAll(rx)) {
if (m[1]) refs.add(m[1]);
}
return [...refs];
}
function extractStepRefs(value: unknown): string[] {
if (typeof value === "string") return extractStepRefsFromString(value);
if (Array.isArray(value)) {
const refs = new Set<string>();
for (const item of value) {
for (const ref of extractStepRefs(item)) refs.add(ref);
}
return [...refs];
}
if (value && typeof value === "object") {
const refs = new Set<string>();
for (const v of Object.values(value as Record<string, unknown>)) {
for (const ref of extractStepRefs(v)) refs.add(ref);
}
return [...refs];
}
return [];
if (typeof value === "string") return extractStepRefsFromString(value);
if (Array.isArray(value)) {
const refs = new Set<string>();
for (const item of value) {
for (const ref of extractStepRefs(item)) refs.add(ref);
}
return [...refs];
}
if (value && typeof value === "object") {
const refs = new Set<string>();
for (const v of Object.values(value as Record<string, unknown>)) {
for (const ref of extractStepRefs(v)) refs.add(ref);
}
return [...refs];
}
return [];
}
function truncate(value: string, max = 80) {
if (value.length <= max) return value;
return `${value.slice(0, max - 1)}…`;
if (value.length <= max) return value;
return `${value.slice(0, max - 1)}…`;
}
function collectGraph(workflow: WorkflowFile, args: Record<string, unknown>) {
const nodes: GraphNode[] = [];
const edges: GraphEdge[] = [];
const knownStepIds = new Set(workflow.steps.map((s) => s.id));
let prevStepId: string | null = null;
const nodes: GraphNode[] = [];
const edges: GraphEdge[] = [];
const knownStepIds = new Set(workflow.steps.map((s) => s.id));
let prevStepId: string | null = null;
const seenEdgeKeys = new Set<string>();
const addEdge = (edge: GraphEdge) => {
const key = `${edge.from}|${edge.to}|${edge.label ?? ""}`;
if (seenEdgeKeys.has(key)) return;
seenEdgeKeys.add(key);
edges.push(edge);
};
const seenEdgeKeys = new Set<string>();
const addEdge = (edge: GraphEdge) => {
const key = `${edge.from}|${edge.to}|${edge.label ?? ""}`;
if (seenEdgeKeys.has(key)) return;
seenEdgeKeys.add(key);
edges.push(edge);
};
for (const step of workflow.steps) {
const type = stepType(step);
const details = stepDetails(step, args);
const label = details ? `${step.id}\\n${truncate(details)}` : step.id;
nodes.push({
id: step.id,
type,
label,
shape: isApprovalStep(step) ? "diamond" : "box",
});
for (const step of workflow.steps) {
const type = stepType(step);
const details = stepDetails(step, args);
const label = details ? `${step.id}\\n${truncate(details)}` : step.id;
nodes.push({
id: step.id,
type,
label,
shape: isApprovalStep(step) ? "diamond" : "box",
});
if (prevStepId) {
addEdge({ from: prevStepId, to: step.id, label: "next" });
}
prevStepId = step.id;
if (prevStepId) {
addEdge({ from: prevStepId, to: step.id, label: "next" });
}
prevStepId = step.id;
for (const ref of extractStepRefs(step.stdin)) {
if (knownStepIds.has(ref)) addEdge({ from: ref, to: step.id, label: "stdin" });
}
for (const ref of extractStepRefs(step.stdin)) {
if (knownStepIds.has(ref)) addEdge({ from: ref, to: step.id, label: "stdin" });
}
if (typeof step.for_each === "string") {
for (const ref of extractStepRefs(step.for_each)) {
if (knownStepIds.has(ref)) addEdge({ from: ref, to: step.id, label: "for_each" });
}
}
if (typeof step.for_each === "string") {
for (const ref of extractStepRefs(step.for_each)) {
if (knownStepIds.has(ref)) addEdge({ from: ref, to: step.id, label: "for_each" });
}
}
const condition = step.when ?? step.condition;
if (typeof condition === "string" && condition.trim()) {
const labelValue = truncate(`when: ${condition.trim()}`, 70);
for (const ref of extractStepRefs(condition)) {
if (knownStepIds.has(ref)) addEdge({ from: ref, to: step.id, label: labelValue });
}
}
}
const condition = step.when ?? step.condition;
if (typeof condition === "string" && condition.trim()) {
const labelValue = truncate(`when: ${condition.trim()}`, 70);
for (const ref of extractStepRefs(condition)) {
if (knownStepIds.has(ref)) addEdge({ from: ref, to: step.id, label: labelValue });
}
}
}
return { nodes, edges };
return { nodes, edges };
}
function sanitizeMermaidId(id: string) {
return id.replace(/[^A-Za-z0-9_]/g, "_");
return id.replace(/[^A-Za-z0-9_]/g, "_");
}
function escapeMermaidLabel(value: string) {
return value.replace(/"/g, '\\"');
return value.replace(/"/g, '\\"');
}
function renderMermaid(nodes: GraphNode[], edges: GraphEdge[]) {
const idMap = new Map<string, string>();
const used = new Set<string>();
for (const node of nodes) {
let key = sanitizeMermaidId(node.id) || "step";
if (/^\d/.test(key)) key = `s_${key}`;
let i = 2;
while (used.has(key)) {
key = `${sanitizeMermaidId(node.id)}_${i}`;
i += 1;
}
used.add(key);
idMap.set(node.id, key);
}
const idMap = new Map<string, string>();
const used = new Set<string>();
for (const node of nodes) {
let key = sanitizeMermaidId(node.id) || "step";
if (/^\d/.test(key)) key = `s_${key}`;
let i = 2;
while (used.has(key)) {
key = `${sanitizeMermaidId(node.id)}_${i}`;
i += 1;
}
used.add(key);
idMap.set(node.id, key);
}
const lines = ["flowchart TD"];
for (const node of nodes) {
const key = idMap.get(node.id)!;
const label = escapeMermaidLabel(node.label);
if (node.shape === "diamond") {
lines.push(` ${key}{"${label}"}`);
} else {
lines.push(` ${key}["${label}"]`);
}
}
if (nodes.length) lines.push("");
const lines = ["flowchart TD"];
for (const node of nodes) {
const key = idMap.get(node.id)!;
const label = escapeMermaidLabel(node.label);
if (node.shape === "diamond") {
lines.push(` ${key}{"${label}"}`);
} else {
lines.push(` ${key}["${label}"]`);
}
}
if (nodes.length) lines.push("");
for (const edge of edges) {
const from = idMap.get(edge.from);
const to = idMap.get(edge.to);
if (!from || !to) continue;
if (edge.label) {
lines.push(` ${from} -->|${escapeMermaidLabel(edge.label)}| ${to}`);
} else {
lines.push(` ${from} --> ${to}`);
}
}
return lines.join("\n");
for (const edge of edges) {
const from = idMap.get(edge.from);
const to = idMap.get(edge.to);
if (!from || !to) continue;
if (edge.label) {
lines.push(` ${from} -->|${escapeMermaidLabel(edge.label)}| ${to}`);
} else {
lines.push(` ${from} --> ${to}`);
}
}
return lines.join("\n");
}
function escapeDot(value: string) {
return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
}
function renderDot(nodes: GraphNode[], edges: GraphEdge[]) {
const lines = ["digraph workflow {", " rankdir=TB;"];
for (const node of nodes) {
const shape = node.shape === "diamond" ? "diamond" : "box";
lines.push(` "${escapeDot(node.id)}" [shape=${shape},label="${escapeDot(node.label)}"];`);
}
if (nodes.length) lines.push("");
for (const edge of edges) {
if (edge.label) {
lines.push(
` "${escapeDot(edge.from)}" -> "${escapeDot(edge.to)}" [label="${escapeDot(edge.label)}"];`,
);
} else {
lines.push(` "${escapeDot(edge.from)}" -> "${escapeDot(edge.to)}";`);
}
}
lines.push("}");
return lines.join("\n");
const lines = ["digraph workflow {", " rankdir=TB;"];
for (const node of nodes) {
const shape = node.shape === "diamond" ? "diamond" : "box";
lines.push(` "${escapeDot(node.id)}" [shape=${shape},label="${escapeDot(node.label)}"];`);
}
if (nodes.length) lines.push("");
for (const edge of edges) {
if (edge.label) {
lines.push(
` "${escapeDot(edge.from)}" -> "${escapeDot(edge.to)}" [label="${escapeDot(edge.label)}"];`,
);
} else {
lines.push(` "${escapeDot(edge.from)}" -> "${escapeDot(edge.to)}";`);
}
}
lines.push("}");
return lines.join("\n");
}
function renderAscii(nodes: GraphNode[], edges: GraphEdge[]) {
const lines = ["Workflow Graph", "", "Nodes:"];
for (const node of nodes) {
lines.push(
`- ${node.id} [${node.type}] ${node.label.includes("\\n") ? `(${node.label.split("\\n")[1]})` : ""}`.trim(),
);
}
lines.push("", "Edges:");
for (const edge of edges) {
lines.push(`- ${edge.from} -> ${edge.to}${edge.label ? ` (${edge.label})` : ""}`);
}
if (edges.length === 0) lines.push("- (none)");
return lines.join("\n");
const lines = ["Workflow Graph", "", "Nodes:"];
for (const node of nodes) {
lines.push(
`- ${node.id} [${node.type}] ${node.label.includes("\\n") ? `(${node.label.split("\\n")[1]})` : ""}`.trim(),
);
}
lines.push("", "Edges:");
for (const edge of edges) {
lines.push(`- ${edge.from} -> ${edge.to}${edge.label ? ` (${edge.label})` : ""}`);
}
if (edges.length === 0) lines.push("- (none)");
return lines.join("\n");
}
export function renderWorkflowGraph({ workflow, format, args = {} }: RenderGraphParams) {
const { nodes, edges } = collectGraph(workflow, args);
if (format === "dot") return renderDot(nodes, edges);
if (format === "ascii") return renderAscii(nodes, edges);
return renderMermaid(nodes, edges);
const { nodes, edges } = collectGraph(workflow, args);
if (format === "dot") return renderDot(nodes, edges);
if (format === "ascii") return renderAscii(nodes, edges);
return renderMermaid(nodes, edges);
}
src/workflows/registry.ts
+52 -52
View File
@@ -1,57 +1,57 @@
export const workflowRegistry = {
"github.pr.monitor": {
name: "github.pr.monitor",
description: "Fetch PR state via gh, diff against last run, emit only on change.",
argsSchema: {
type: "object",
properties: {
repo: { type: "string", description: "owner/repo (e.g. openclaw/openclaw)" },
pr: { type: "number", description: "Pull request number" },
key: { type: "string", description: "Optional state key override." },
changesOnly: { type: "boolean", description: "If true, suppress output when unchanged." },
summaryOnly: {
type: "boolean",
description: "If true, return only a compact change summary (smaller output).",
},
},
required: ["repo", "pr"],
},
examples: [
{
args: { repo: "openclaw/openclaw", pr: 1152 },
description: "Monitor a PR and report when it changes.",
},
],
sideEffects: [],
},
"github.pr.monitor.notify": {
name: "github.pr.monitor.notify",
description: "Monitor a PR and emit a single human-friendly message when it changes.",
argsSchema: {
type: "object",
properties: {
repo: { type: "string", description: "owner/repo (e.g. openclaw/openclaw)" },
pr: { type: "number", description: "Pull request number" },
key: { type: "string", description: "Optional state key override." },
},
required: ["repo", "pr"],
},
examples: [
{
args: { repo: "openclaw/openclaw", pr: 1152 },
description: 'Emit "PR updated" message only when changed.',
},
],
sideEffects: [],
},
"github.pr.monitor": {
name: "github.pr.monitor",
description: "Fetch PR state via gh, diff against last run, emit only on change.",
argsSchema: {
type: "object",
properties: {
repo: { type: "string", description: "owner/repo (e.g. openclaw/openclaw)" },
pr: { type: "number", description: "Pull request number" },
key: { type: "string", description: "Optional state key override." },
changesOnly: { type: "boolean", description: "If true, suppress output when unchanged." },
summaryOnly: {
type: "boolean",
description: "If true, return only a compact change summary (smaller output).",
},
},
required: ["repo", "pr"],
},
examples: [
{
args: { repo: "openclaw/openclaw", pr: 1152 },
description: "Monitor a PR and report when it changes.",
},
],
sideEffects: [],
},
"github.pr.monitor.notify": {
name: "github.pr.monitor.notify",
description: "Monitor a PR and emit a single human-friendly message when it changes.",
argsSchema: {
type: "object",
properties: {
repo: { type: "string", description: "owner/repo (e.g. openclaw/openclaw)" },
pr: { type: "number", description: "Pull request number" },
key: { type: "string", description: "Optional state key override." },
},
required: ["repo", "pr"],
},
examples: [
{
args: { repo: "openclaw/openclaw", pr: 1152 },
description: 'Emit "PR updated" message only when changed.',
},
],
sideEffects: [],
},
};
export function listWorkflows() {
return Object.values(workflowRegistry).map((w) => ({
name: w.name,
description: w.description,
argsSchema: w.argsSchema,
examples: w.examples,
sideEffects: w.sideEffects,
}));
return Object.values(workflowRegistry).map((w) => ({
name: w.name,
description: w.description,
argsSchema: w.argsSchema,
examples: w.examples,
sideEffects: w.sideEffects,
}));
}
test/approval_id.test.ts
+165 -165
View File
@@ -8,224 +8,224 @@ import { spawnSync } from "node:child_process";
import { findStateKeyByApprovalId, writeApprovalIndex } from "../src/state/store.js";
function runCli(args: string[], env: Record<string, string | undefined>) {
const bin = path.join(process.cwd(), "bin", "lobster.js");
return spawnSync("node", [bin, ...args], {
encoding: "utf8",
env: { ...process.env, ...env },
});
const bin = path.join(process.cwd(), "bin", "lobster.js");
return spawnSync("node", [bin, ...args], {
encoding: "utf8",
env: { ...process.env, ...env },
});
}
test("approval gate returns approvalId alongside resumeToken", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-"));
const stateDir = path.join(tmpDir, "state");
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{a:1}]))'\" | approve --prompt 'ok?' | pick a";
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{a:1}]))'\" | approve --prompt 'ok?' | pick a";
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
assert.equal(first.status, 0);
const json = JSON.parse(first.stdout);
assert.equal(json.status, "needs_approval");
assert.ok(json.requiresApproval?.resumeToken, "should have resumeToken");
assert.ok(json.requiresApproval?.approvalId, "should have approvalId");
assert.equal(json.requiresApproval.approvalId.length, 8, "approvalId should be 8 hex chars");
assert.match(json.requiresApproval.approvalId, /^[a-f0-9]{8}$/, "approvalId should be hex");
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
assert.equal(first.status, 0);
const json = JSON.parse(first.stdout);
assert.equal(json.status, "needs_approval");
assert.ok(json.requiresApproval?.resumeToken, "should have resumeToken");
assert.ok(json.requiresApproval?.approvalId, "should have approvalId");
assert.equal(json.requiresApproval.approvalId.length, 8, "approvalId should be 8 hex chars");
assert.match(json.requiresApproval.approvalId, /^[a-f0-9]{8}$/, "approvalId should be hex");
// Verify index file was written
const files = await fsp.readdir(stateDir);
const indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 1, "should have one approval index file");
// Verify index file was written
const files = await fsp.readdir(stateDir);
const indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 1, "should have one approval index file");
});
test("resume with --id works as alternative to --token", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-resume-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-resume-"));
const stateDir = path.join(tmpDir, "state");
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{b:2}]))'\" | approve --prompt 'ok?' | pick b";
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{b:2}]))'\" | approve --prompt 'ok?' | pick b";
// Step 1: Run pipeline, get approval ID
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
assert.equal(first.status, 0);
const firstJson = JSON.parse(first.stdout);
assert.equal(firstJson.status, "needs_approval");
const approvalId = firstJson.requiresApproval.approvalId;
assert.ok(approvalId);
// Step 1: Run pipeline, get approval ID
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
assert.equal(first.status, 0);
const firstJson = JSON.parse(first.stdout);
assert.equal(firstJson.status, "needs_approval");
const approvalId = firstJson.requiresApproval.approvalId;
assert.ok(approvalId);
// Step 2: Resume using --id instead of --token
const resumed = runCli(["resume", "--id", approvalId, "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
assert.equal(resumed.status, 0, `stderr: ${resumed.stderr}`);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
assert.deepEqual(resumedJson.output, [{ b: 2 }]);
// Step 2: Resume using --id instead of --token
const resumed = runCli(["resume", "--id", approvalId, "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
assert.equal(resumed.status, 0, `stderr: ${resumed.stderr}`);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
assert.deepEqual(resumedJson.output, [{ b: 2 }]);
// Step 3: Verify cleanup — approval index should be deleted
const files = await fsp.readdir(stateDir);
const indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 0, "approval index should be cleaned up after resume");
// Step 3: Verify cleanup — approval index should be deleted
const files = await fsp.readdir(stateDir);
const indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 0, "approval index should be cleaned up after resume");
});
test("resume with --id cancellation works", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-cancel-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-cancel-"));
const stateDir = path.join(tmpDir, "state");
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{c:3}]))'\" | approve --prompt 'ok?' | pick c";
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{c:3}]))'\" | approve --prompt 'ok?' | pick c";
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
const approvalId = firstJson.requiresApproval.approvalId;
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
const approvalId = firstJson.requiresApproval.approvalId;
const cancelled = runCli(["resume", "--id", approvalId, "--approve", "no"], {
LOBSTER_STATE_DIR: stateDir,
});
assert.equal(cancelled.status, 0);
const cancelledJson = JSON.parse(cancelled.stdout);
assert.equal(cancelledJson.status, "cancelled");
const cancelled = runCli(["resume", "--id", approvalId, "--approve", "no"], {
LOBSTER_STATE_DIR: stateDir,
});
assert.equal(cancelled.status, 0);
const cancelledJson = JSON.parse(cancelled.stdout);
assert.equal(cancelledJson.status, "cancelled");
});
test("resume with invalid --id returns clear error", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-invalid-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-invalid-"));
const stateDir = path.join(tmpDir, "state");
const result = runCli(["resume", "--id", "deadbeef", "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
// Should fail with a clear error message
const json = JSON.parse(result.stdout);
assert.equal(json.ok, false);
assert.ok(
json.error?.message?.includes("not found"),
`Error should mention not found: ${json.error?.message}`,
);
const result = runCli(["resume", "--id", "deadbeef", "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
// Should fail with a clear error message
const json = JSON.parse(result.stdout);
assert.equal(json.ok, false);
assert.ok(
json.error?.message?.includes("not found"),
`Error should mention not found: ${json.error?.message}`,
);
});
test("--token resume cleans up orphaned approval index", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-orphan-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-orphan-"));
const stateDir = path.join(tmpDir, "state");
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{e:5}]))'\" | approve --prompt 'ok?' | pick e";
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{e:5}]))'\" | approve --prompt 'ok?' | pick e";
// Step 1: Run pipeline, get both approvalId and resumeToken
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
assert.ok(firstJson.requiresApproval?.approvalId);
assert.ok(firstJson.requiresApproval?.resumeToken);
// Step 1: Run pipeline, get both approvalId and resumeToken
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
assert.ok(firstJson.requiresApproval?.approvalId);
assert.ok(firstJson.requiresApproval?.resumeToken);
// Verify index file exists
let files = await fsp.readdir(stateDir);
let indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 1, "approval index should exist before resume");
// Verify index file exists
let files = await fsp.readdir(stateDir);
let indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 1, "approval index should exist before resume");
// Step 2: Resume using --token (NOT --id)
const resumed = runCli(
["resume", "--token", firstJson.requiresApproval.resumeToken, "--approve", "yes"],
{ LOBSTER_STATE_DIR: stateDir },
);
assert.equal(resumed.status, 0);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
// Step 2: Resume using --token (NOT --id)
const resumed = runCli(
["resume", "--token", firstJson.requiresApproval.resumeToken, "--approve", "yes"],
{ LOBSTER_STATE_DIR: stateDir },
);
assert.equal(resumed.status, 0);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
// Step 3: Verify approval index was cleaned up despite using --token
files = await fsp.readdir(stateDir);
indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 0, "approval index should be cleaned up even when using --token");
// Step 3: Verify approval index was cleaned up despite using --token
files = await fsp.readdir(stateDir);
indexFiles = files.filter((name) => name.startsWith("approval_"));
assert.equal(indexFiles.length, 0, "approval index should be cleaned up even when using --token");
});
test("double-resume with same --id returns clear error", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-double-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-double-"));
const stateDir = path.join(tmpDir, "state");
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{f:6}]))'\" | approve --prompt 'ok?' | pick f";
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{f:6}]))'\" | approve --prompt 'ok?' | pick f";
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
const approvalId = firstJson.requiresApproval.approvalId;
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
const approvalId = firstJson.requiresApproval.approvalId;
// First resume — should succeed
const resumed = runCli(["resume", "--id", approvalId, "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
assert.equal(resumed.status, 0);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
// First resume — should succeed
const resumed = runCli(["resume", "--id", approvalId, "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
assert.equal(resumed.status, 0);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
// Second resume with same ID — should fail cleanly, not crash
const second = runCli(["resume", "--id", approvalId, "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
const secondJson = JSON.parse(second.stdout);
assert.equal(secondJson.ok, false);
assert.ok(
secondJson.error?.message?.includes("not found"),
`Should report not found: ${secondJson.error?.message}`,
);
// Second resume with same ID — should fail cleanly, not crash
const second = runCli(["resume", "--id", approvalId, "--approve", "yes"], {
LOBSTER_STATE_DIR: stateDir,
});
const secondJson = JSON.parse(second.stdout);
assert.equal(secondJson.ok, false);
assert.ok(
secondJson.error?.message?.includes("not found"),
`Should report not found: ${secondJson.error?.message}`,
);
});
test("backward compat: --token still works when approvalId is present", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-compat-"));
const stateDir = path.join(tmpDir, "state");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-compat-"));
const stateDir = path.join(tmpDir, "state");
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{d:4}]))'\" | approve --prompt 'ok?' | pick d";
const pipeline =
"exec --json --shell \"node -e 'process.stdout.write(JSON.stringify([{d:4}]))'\" | approve --prompt 'ok?' | pick d";
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
assert.ok(firstJson.requiresApproval?.approvalId, "approvalId present");
assert.ok(firstJson.requiresApproval?.resumeToken, "resumeToken present");
const first = runCli(["run", "--mode", "tool", pipeline], { LOBSTER_STATE_DIR: stateDir });
const firstJson = JSON.parse(first.stdout);
assert.ok(firstJson.requiresApproval?.approvalId, "approvalId present");
assert.ok(firstJson.requiresApproval?.resumeToken, "resumeToken present");
// Resume using the old --token approach — should still work
const resumed = runCli(
["resume", "--token", firstJson.requiresApproval.resumeToken, "--approve", "yes"],
{ LOBSTER_STATE_DIR: stateDir },
);
assert.equal(resumed.status, 0);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
assert.deepEqual(resumedJson.output, [{ d: 4 }]);
// Resume using the old --token approach — should still work
const resumed = runCli(
["resume", "--token", firstJson.requiresApproval.resumeToken, "--approve", "yes"],
{ LOBSTER_STATE_DIR: stateDir },
);
assert.equal(resumed.status, 0);
const resumedJson = JSON.parse(resumed.stdout);
assert.equal(resumedJson.status, "ok");
assert.deepEqual(resumedJson.output, [{ d: 4 }]);
});
test("approval index writes never overwrite an existing approval ID mapping", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-collision-"));
const stateDir = path.join(tmpDir, "state");
const env = { LOBSTER_STATE_DIR: stateDir };
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-collision-"));
const stateDir = path.join(tmpDir, "state");
const env = { LOBSTER_STATE_DIR: stateDir };
await writeApprovalIndex({
env,
stateKey: "workflow_resume_original",
approvalId: "deadbeef",
});
await writeApprovalIndex({
env,
stateKey: "workflow_resume_original",
approvalId: "deadbeef",
});
await assert.rejects(
() =>
writeApprovalIndex({
env,
stateKey: "workflow_resume_replacement",
approvalId: "deadbeef",
}),
(err: NodeJS.ErrnoException) => err?.code === "EEXIST",
);
await assert.rejects(
() =>
writeApprovalIndex({
env,
stateKey: "workflow_resume_replacement",
approvalId: "deadbeef",
}),
(err: NodeJS.ErrnoException) => err?.code === "EEXIST",
);
const resolved = await findStateKeyByApprovalId({ env, approvalId: "deadbeef" });
assert.equal(resolved, "workflow_resume_original");
const resolved = await findStateKeyByApprovalId({ env, approvalId: "deadbeef" });
assert.equal(resolved, "workflow_resume_original");
});
test("corrupt approval index is treated as expired instead of crashing (#113)", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-corrupt-"));
const stateDir = path.join(tmpDir, "state");
const env = { LOBSTER_STATE_DIR: stateDir };
await fsp.mkdir(stateDir, { recursive: true });
await fsp.writeFile(path.join(stateDir, "approval_deadbeef.json"), '{"stateKey"', "utf8");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-aid-corrupt-"));
const stateDir = path.join(tmpDir, "state");
const env = { LOBSTER_STATE_DIR: stateDir };
await fsp.mkdir(stateDir, { recursive: true });
await fsp.writeFile(path.join(stateDir, "approval_deadbeef.json"), '{"stateKey"', "utf8");
const resolved = await findStateKeyByApprovalId({ env, approvalId: "deadbeef" });
assert.equal(resolved, null);
const resolved = await findStateKeyByApprovalId({ env, approvalId: "deadbeef" });
assert.equal(resolved, null);
const resumed = runCli(["resume", "--id", "deadbeef", "--approve", "yes"], env);
const json = JSON.parse(resumed.stdout);
assert.equal(json.ok, false);
assert.match(json.error?.message ?? "", /not found or expired/);
const resumed = runCli(["resume", "--id", "deadbeef", "--approve", "yes"], env);
const json = JSON.parse(resumed.stdout);
assert.equal(json.ok, false);
assert.match(json.error?.message ?? "", /not found or expired/);
});
test/approve_preview.test.ts
+28 -28
View File
@@ -3,37 +3,37 @@ import assert from "node:assert/strict";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
test("approve preview includes stdin sample when requested", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("approve");
const registry = createDefaultRegistry();
const cmd = registry.get("approve");
const result = await cmd.run({
input: streamOf([{ a: 1 }, { a: 2 }]),
args: {
_: [],
emit: true,
prompt: "ok?",
"preview-from-stdin": true,
limit: 1,
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const result = await cmd.run({
input: streamOf([{ a: 1 }, { a: 2 }]),
args: {
_: [],
emit: true,
prompt: "ok?",
"preview-from-stdin": true,
limit: 1,
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const items = [];
for await (const item of result.output) items.push(item);
assert.equal(items[0].type, "approval_request");
assert.ok(String(items[0].preview).includes('"a": 1'));
const items = [];
for await (const item of result.output) items.push(item);
assert.equal(items[0].type, "approval_request");
assert.ok(String(items[0].preview).includes('"a": 1'));
});
test/bin_smoke.test.ts
+14
View File
@@ -0,0 +1,14 @@
import test from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import path from "node:path";
test("packaged lobster bin starts and prints help", () => {
const bin = path.join(process.cwd(), "bin", "lobster.js");
const res = spawnSync(process.execPath, [bin, "--help"], {
encoding: "utf8",
});
assert.equal(res.status, 0, res.stderr);
assert.match(res.stdout, /Usage:/);
});
test/cancellation.test.ts
+5684
View File
File diff suppressed because it is too large Load Diff
test/clawd_invoke.test.ts
+135 -108
View File
@@ -4,128 +4,155 @@ import http from "node:http";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
test("openclaw.invoke posts to /tools/invoke and returns JSON", async () => {
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let body = "";
req.setEncoding("utf8");
req.on("data", (d) => (body += d));
req.on("end", () => {
const parsed = JSON.parse(body);
assert.equal(parsed.tool, "demo");
assert.equal(parsed.action, "ping");
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, result: [{ ok: true, echo: parsed.args }] }));
});
});
let body = "";
req.setEncoding("utf8");
req.on("data", (d) => (body += d));
req.on("end", () => {
const parsed = JSON.parse(body);
assert.equal(parsed.tool, "demo");
assert.equal(parsed.action, "ping");
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, result: [{ ok: true, echo: parsed.args }] }));
});
});
await new Promise<void>((resolve) => server.listen(0, () => resolve()));
const addr = server.address();
const port = typeof addr === "string" || addr == null ? 0 : addr.port;
await new Promise<void>((resolve) => server.listen(0, () => resolve()));
const addr = server.address();
const port = typeof addr === "string" || addr == null ? 0 : addr.port;
try {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
try {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
const result = await cmd.run({
input: streamOf([]),
args: {
_: [],
url: `http://127.0.0.1:${port}`,
tool: "demo",
action: "ping",
"args-json": '{"hello":"world"}',
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const result = await cmd.run({
input: streamOf([]),
args: {
_: [],
url: `http://127.0.0.1:${port}`,
tool: "demo",
action: "ping",
"args-json": '{"hello":"world"}',
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const items = [];
for await (const it of result.output) items.push(it);
assert.deepEqual(items, [{ ok: true, echo: { hello: "world" } }]);
} finally {
server.close();
}
const items = [];
for await (const it of result.output) items.push(it);
assert.deepEqual(items, [{ ok: true, echo: { hello: "world" } }]);
} finally {
server.close();
}
});
test("openclaw.invoke --each maps input items into tool args", async () => {
const seen: Array<{ call: number; args: unknown }> = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
const seen: Array<{ call: number; args: unknown }> = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let body = "";
req.setEncoding("utf8");
req.on("data", (d) => (body += d));
req.on("end", () => {
const parsed = JSON.parse(body);
seen.push({ call: seen.length + 1, args: parsed.args });
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, result: [{ ok: true, call: seen.length }] }));
});
});
let body = "";
req.setEncoding("utf8");
req.on("data", (d) => (body += d));
req.on("end", () => {
const parsed = JSON.parse(body);
seen.push({ call: seen.length + 1, args: parsed.args });
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, result: [{ ok: true, call: seen.length }] }));
});
});
await new Promise<void>((resolve) => server.listen(0, () => resolve()));
const addr = server.address();
const port = typeof addr === "string" || addr == null ? 0 : addr.port;
await new Promise<void>((resolve) => server.listen(0, () => resolve()));
const addr = server.address();
const port = typeof addr === "string" || addr == null ? 0 : addr.port;
try {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
try {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
const result = await cmd.run({
input: streamOf(["a", "b"]),
args: {
_: [],
url: `http://127.0.0.1:${port}`,
tool: "demo",
action: "ping",
each: true,
"item-key": "message",
"args-json": '{"channel":"test"}',
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const result = await cmd.run({
input: streamOf(["a", "b"]),
args: {
_: [],
url: `http://127.0.0.1:${port}`,
tool: "demo",
action: "ping",
each: true,
"item-key": "message",
"args-json": '{"channel":"test"}',
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const items = [];
for await (const it of result.output) items.push(it);
assert.deepEqual(items, [
{ ok: true, call: 1 },
{ ok: true, call: 2 },
]);
assert.deepEqual(seen, [
{ call: 1, args: { channel: "test", message: "a" } },
{ call: 2, args: { channel: "test", message: "b" } },
]);
} finally {
server.close();
}
const items = [];
for await (const it of result.output) items.push(it);
assert.deepEqual(items, [
{ ok: true, call: 1 },
{ ok: true, call: 2 },
]);
assert.deepEqual(seen, [
{ call: 1, args: { channel: "test", message: "a" } },
{ call: 2, args: { channel: "test", message: "b" } },
]);
} finally {
server.close();
}
});
test("openclaw.invoke refuses env tokens for non-local URLs", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
await assert.rejects(
cmd.run({
input: streamOf([]),
args: {
_: [],
url: "https://example.com",
tool: "demo",
action: "ping",
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, OPENCLAW_TOKEN: "secret" },
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
}),
/refuses to send OPENCLAW_TOKEN\/CLAWD_TOKEN to non-local --url/,
);
});
test/clawd_invoke_legacy.test.ts
+51 -51
View File
@@ -4,63 +4,63 @@ import http from "node:http";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
test("openclaw.invoke accepts legacy raw JSON response", async () => {
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let body = "";
req.setEncoding("utf8");
req.on("data", (d) => (body += d));
req.on("end", () => {
const parsed = JSON.parse(body);
assert.equal(parsed.tool, "demo");
assert.equal(parsed.action, "ping");
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify([{ ok: true, legacy: true, echo: parsed.args }]));
});
});
let body = "";
req.setEncoding("utf8");
req.on("data", (d) => (body += d));
req.on("end", () => {
const parsed = JSON.parse(body);
assert.equal(parsed.tool, "demo");
assert.equal(parsed.action, "ping");
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify([{ ok: true, legacy: true, echo: parsed.args }]));
});
});
await new Promise<void>((resolve) => server.listen(0, () => resolve()));
const addr = server.address();
const port = typeof addr === "string" || addr == null ? 0 : addr.port;
await new Promise<void>((resolve) => server.listen(0, () => resolve()));
const addr = server.address();
const port = typeof addr === "string" || addr == null ? 0 : addr.port;
try {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
try {
const registry = createDefaultRegistry();
const cmd = registry.get("openclaw.invoke");
const result = await cmd.run({
input: streamOf([]),
args: {
_: [],
url: `http://127.0.0.1:${port}`,
tool: "demo",
action: "ping",
"args-json": '{"hello":"world"}',
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const result = await cmd.run({
input: streamOf([]),
args: {
_: [],
url: `http://127.0.0.1:${port}`,
tool: "demo",
action: "ping",
"args-json": '{"hello":"world"}',
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const items = [];
for await (const it of result.output) items.push(it);
assert.deepEqual(items, [{ ok: true, legacy: true, echo: { hello: "world" } }]);
} finally {
server.close();
}
const items = [];
for await (const it of result.output) items.push(it);
assert.deepEqual(items, [{ ok: true, legacy: true, echo: { hello: "world" } }]);
} finally {
server.close();
}
});
test/cli_run_file_args_json.test.ts
+20 -20
View File
@@ -6,32 +6,32 @@ import os from "node:os";
import { spawnSync } from "node:child_process";
function runLobster(args: string[], opts?: { env?: Record<string, string | undefined> }) {
const res = spawnSync(process.execPath, [path.join("bin", "lobster.js"), ...args], {
cwd: path.resolve("."),
env: { ...process.env, ...(opts?.env ?? undefined) },
encoding: "utf8",
});
return res;
const res = spawnSync(process.execPath, [path.join("bin", "lobster.js"), ...args], {
cwd: path.resolve("."),
env: { ...process.env, ...(opts?.env ?? undefined) },
encoding: "utf8",
});
return res;
}
test("cli: run --file passes --args-json into workflow args", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-cli-"));
const filePath = path.join(tmpDir, "workflow.lobster");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-cli-"));
const filePath = path.join(tmpDir, "workflow.lobster");
// Print both template-substituted arg and env-injected arg (LOBSTER_ARG_TASK)
// so we catch regressions in either path.
const workflow = `name: test\nargs:\n task:\n default: ""\nsteps:\n - id: s\n command: >\n node -e "process.stdout.write(JSON.stringify({task: '\${task}', env: process.env.LOBSTER_ARG_TASK}))"\n`;
// Print both template-substituted arg and env-injected arg (LOBSTER_ARG_TASK)
// so we catch regressions in either path.
const workflow = `name: test\nargs:\n task:\n default: ""\nsteps:\n - id: s\n command: >\n node -e "process.stdout.write(JSON.stringify({task: '\${task}', env: process.env.LOBSTER_ARG_TASK}))"\n`;
await fsp.writeFile(filePath, workflow, "utf8");
await fsp.writeFile(filePath, workflow, "utf8");
const res = runLobster(["run", "--file", filePath, "--args-json", '{"task":"test"}']);
const res = runLobster(["run", "--file", filePath, "--args-json", '{"task":"test"}']);
assert.equal(
res.status,
0,
`expected exit 0, got ${res.status}\nstdout=${res.stdout}\nstderr=${res.stderr}`,
);
assert.equal(
res.status,
0,
`expected exit 0, got ${res.status}\nstdout=${res.stdout}\nstderr=${res.stderr}`,
);
const parsed = JSON.parse(String(res.stdout).trim());
assert.deepEqual(parsed, [{ task: "test", env: "test" }]);
const parsed = JSON.parse(String(res.stdout).trim());
assert.deepEqual(parsed, [{ task: "test", env: "test" }]);
});
test/commands_list.test.ts
+34 -34
View File
@@ -3,46 +3,46 @@ import assert from "node:assert/strict";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
test("commands.list returns command inventory including stdlib + workflows", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("commands.list");
assert.ok(cmd, "commands.list should be registered");
const registry = createDefaultRegistry();
const cmd = registry.get("commands.list");
assert.ok(cmd, "commands.list should be registered");
const res = await cmd.run({
input: streamOf([]),
args: { _: [] },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const res = await cmd.run({
input: streamOf([]),
args: { _: [] },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const items = [];
for await (const it of res.output) items.push(it);
const items = [];
for await (const it of res.output) items.push(it);
const names = items.map((x) => x.name).sort();
const names = items.map((x) => x.name).sort();
// A couple representative commands we always expect.
assert.ok(names.includes("exec"));
assert.ok(names.includes("json"));
assert.ok(names.includes("llm.invoke"));
assert.ok(names.includes("workflows.list"));
assert.ok(names.includes("commands.list"));
// A couple representative commands we always expect.
assert.ok(names.includes("exec"));
assert.ok(names.includes("json"));
assert.ok(names.includes("llm.invoke"));
assert.ok(names.includes("workflows.list"));
assert.ok(names.includes("commands.list"));
const self = items.find((x) => x.name === "commands.list");
assert.ok(self);
assert.equal(typeof self.description, "string");
assert.ok(self.description.length > 0);
// Schema should be present for commands that declare it.
assert.ok(self.argsSchema);
const self = items.find((x) => x.name === "commands.list");
assert.ok(self);
assert.equal(typeof self.description, "string");
assert.ok(self.description.length > 0);
// Schema should be present for commands that declare it.
assert.ok(self.argsSchema);
});
test/condition_comparison.test.ts
+98 -98
View File
@@ -7,133 +7,133 @@ import path from "node:path";
import { runWorkflowFile } from "../src/workflows/file.js";
async function runWorkflow(workflow: unknown) {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-cond-"));
const stateDir = path.join(tmpDir, "state");
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(filePath, JSON.stringify(workflow, null, 2), "utf8");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-cond-"));
const stateDir = path.join(tmpDir, "state");
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(filePath, JSON.stringify(workflow, null, 2), "utf8");
return runWorkflowFile({
filePath,
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, LOBSTER_STATE_DIR: stateDir },
mode: "tool",
},
});
return runWorkflowFile({
filePath,
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, LOBSTER_STATE_DIR: stateDir },
mode: "tool",
},
});
}
test("condition > works with numbers", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({count:5}))"' },
{ id: "check", command: 'echo "big"', when: "$data.json.count > 3" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["big\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({count:5}))"' },
{ id: "check", command: 'echo "big"', when: "$data.json.count > 3" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["big\n"]);
});
test("condition > skips when false", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({count:1}))"' },
{ id: "check", command: 'echo "big"', when: "$data.json.count > 3" },
{ id: "fallback", command: 'echo "small"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["small\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({count:1}))"' },
{ id: "check", command: 'echo "big"', when: "$data.json.count > 3" },
{ id: "fallback", command: 'echo "small"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["small\n"]);
});
test("condition < works", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:2}))"' },
{ id: "check", command: 'echo "low"', when: "$data.json.val < 10" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["low\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:2}))"' },
{ id: "check", command: 'echo "low"', when: "$data.json.val < 10" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["low\n"]);
});
test("condition >= works at boundary", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:5}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val >= 5" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["yes\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:5}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val >= 5" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["yes\n"]);
});
test("condition <= works at boundary", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:5}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val <= 5" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["yes\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:5}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val <= 5" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["yes\n"]);
});
test("comparison operators combine with boolean operators", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({a:5,b:20}))"' },
{ id: "check", command: 'echo "in range"', when: "$data.json.a >= 1 && $data.json.b < 100" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["in range\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({a:5,b:20}))"' },
{ id: "check", command: 'echo "in range"', when: "$data.json.a >= 1 && $data.json.b < 100" },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["in range\n"]);
});
test("comparison with non-numeric string returns false", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:\\"hello\\"}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val > 3" },
{ id: "fallback", command: 'echo "no"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["no\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:\\"hello\\"}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val > 3" },
{ id: "fallback", command: 'echo "no"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["no\n"]);
});
test("comparison rejects boolean as non-numeric", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:true}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val > 0" },
{ id: "fallback", command: 'echo "no"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["no\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:true}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val > 0" },
{ id: "fallback", command: 'echo "no"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["no\n"]);
});
test("comparison rejects null as non-numeric", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:null}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val >= 0" },
{ id: "fallback", command: 'echo "no"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["no\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({val:null}))"' },
{ id: "check", command: 'echo "yes"', when: "$data.json.val >= 0" },
{ id: "fallback", command: 'echo "no"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["no\n"]);
});
test("existing == and != still work with new operators", async () => {
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({status:\\"ok\\"}))"' },
{ id: "check", command: 'echo "good"', when: '$data.json.status == "ok"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["good\n"]);
const result = await runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({status:\\"ok\\"}))"' },
{ id: "check", command: 'echo "good"', when: '$data.json.status == "ok"' },
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, ["good\n"]);
});
test/core_tool_runtime.test.ts
+162 -162
View File
@@ -7,188 +7,188 @@ import os from "node:os";
import { resumeToolRequest, runToolRequest } from "../src/core/index.js";
function createDirectAdapter(resultText: string) {
const calls: Array<Record<string, unknown>> = [];
return {
calls,
adapter: {
source: "test",
async invoke({ payload }: { payload: Record<string, unknown> }) {
calls.push(payload);
return {
ok: true,
result: {
runId: "adapter_1",
model: "test/model",
prompt: payload.prompt,
status: "completed",
output: {
format: "json",
text: resultText,
data: JSON.parse(resultText),
},
},
};
},
},
};
const calls: Array<Record<string, unknown>> = [];
return {
calls,
adapter: {
source: "test",
async invoke({ payload }: { payload: Record<string, unknown> }) {
calls.push(payload);
return {
ok: true,
result: {
runId: "adapter_1",
model: "test/model",
prompt: payload.prompt,
status: "completed",
output: {
format: "json",
text: resultText,
data: JSON.parse(resultText),
},
},
};
},
},
};
}
test("runToolRequest executes pipeline with injected llm adapter", async () => {
const { adapter, calls } = createDirectAdapter('{"recommendation":"no jacket"}');
const envelope = await runToolRequest({
pipeline:
'exec --json=true node -e "process.stdout.write(JSON.stringify({location:\'Phoenix\',temp_f:73.8}))" | llm.invoke --provider pi --prompt "Should I wear a jacket?" --disable-cache',
ctx: {
env: {
...process.env,
LOBSTER_LLM_PROVIDER: "pi",
LOBSTER_LLM_MODEL: "test/model",
},
llmAdapters: {
pi: adapter,
},
},
});
const { adapter, calls } = createDirectAdapter('{"recommendation":"no jacket"}');
const envelope = await runToolRequest({
pipeline:
'exec --json=true node -e "process.stdout.write(JSON.stringify({location:\'Phoenix\',temp_f:73.8}))" | llm.invoke --provider pi --prompt "Should I wear a jacket?" --disable-cache',
ctx: {
env: {
...process.env,
LOBSTER_LLM_PROVIDER: "pi",
LOBSTER_LLM_MODEL: "test/model",
},
llmAdapters: {
pi: adapter,
},
},
});
assert.equal(envelope.ok, true);
assert.equal(envelope.status, "ok");
assert.equal(envelope.output?.length, 1);
assert.equal((envelope.output![0] as any).output.data.recommendation, "no jacket");
assert.equal(calls.length, 1);
assert.equal((calls[0] as any).model, "test/model");
assert.equal(envelope.ok, true);
assert.equal(envelope.status, "ok");
assert.equal(envelope.output?.length, 1);
assert.equal((envelope.output![0] as any).output.data.recommendation, "no jacket");
assert.equal(calls.length, 1);
assert.equal((calls[0] as any).model, "test/model");
});
test("resumeToolRequest completes approval-gated workflow with injected llm adapter", async () => {
const { adapter, calls } = createDirectAdapter('{"recommendation":"no","reason":"warm"}');
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-core-tool-runtime-"));
const filePath = path.join(tmpDir, "workflow.lobster");
const { adapter, calls } = createDirectAdapter('{"recommendation":"no","reason":"warm"}');
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-core-tool-runtime-"));
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(
filePath,
JSON.stringify(
{
steps: [
{
id: "fetch",
run: "node -e \"process.stdout.write(JSON.stringify({location:'Phoenix',temp_f:73.8}))\"",
},
{
id: "confirm",
approval: "Want jacket advice?",
stdin: "$fetch.json",
},
{
id: "advice",
pipeline: 'llm.invoke --provider pi --prompt "Return JSON." --disable-cache',
stdin: "$fetch.json",
when: "$confirm.approved",
},
],
},
null,
2,
),
"utf8",
);
await fsp.writeFile(
filePath,
JSON.stringify(
{
steps: [
{
id: "fetch",
run: "node -e \"process.stdout.write(JSON.stringify({location:'Phoenix',temp_f:73.8}))\"",
},
{
id: "confirm",
approval: "Want jacket advice?",
stdin: "$fetch.json",
},
{
id: "advice",
pipeline: 'llm.invoke --provider pi --prompt "Return JSON." --disable-cache',
stdin: "$fetch.json",
when: "$confirm.approved",
},
],
},
null,
2,
),
"utf8",
);
const env = {
...process.env,
LOBSTER_STATE_DIR: path.join(tmpDir, "state"),
LOBSTER_LLM_PROVIDER: "pi",
LOBSTER_LLM_MODEL: "test/model",
};
const env = {
...process.env,
LOBSTER_STATE_DIR: path.join(tmpDir, "state"),
LOBSTER_LLM_PROVIDER: "pi",
LOBSTER_LLM_MODEL: "test/model",
};
const first = await runToolRequest({
filePath,
ctx: {
cwd: tmpDir,
env,
llmAdapters: { pi: adapter },
},
});
const first = await runToolRequest({
filePath,
ctx: {
cwd: tmpDir,
env,
llmAdapters: { pi: adapter },
},
});
assert.equal(first.ok, true);
assert.equal(first.status, "needs_approval");
assert.ok(first.requiresApproval?.resumeToken);
assert.equal(first.ok, true);
assert.equal(first.status, "needs_approval");
assert.ok(first.requiresApproval?.resumeToken);
const resumed = await resumeToolRequest({
token: first.requiresApproval?.resumeToken ?? "",
approved: true,
ctx: {
cwd: tmpDir,
env,
llmAdapters: { pi: adapter },
},
});
const resumed = await resumeToolRequest({
token: first.requiresApproval?.resumeToken ?? "",
approved: true,
ctx: {
cwd: tmpDir,
env,
llmAdapters: { pi: adapter },
},
});
assert.equal(resumed.ok, true);
assert.equal(resumed.status, "ok");
assert.equal((resumed.output![0] as any).output.data.reason, "warm");
assert.equal(calls.length, 1);
assert.equal(resumed.ok, true);
assert.equal(resumed.status, "ok");
assert.equal((resumed.output![0] as any).output.data.reason, "warm");
assert.equal(calls.length, 1);
});
test("runToolRequest/resumeToolRequest handles needs_input workflow pauses", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-core-tool-input-"));
const filePath = path.join(tmpDir, "workflow.lobster");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-core-tool-input-"));
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(
filePath,
JSON.stringify(
{
steps: [
{
id: "draft",
run: "node -e \"process.stdout.write(JSON.stringify({text:'hello'}))\"",
},
{
id: "review",
input: {
prompt: "Review draft?",
responseSchema: {
type: "object",
properties: { decision: { type: "string" } },
required: ["decision"],
},
},
},
{
id: "finish",
run: 'node -e "process.stdout.write(JSON.stringify({decision:process.env.DECISION,subject:process.env.SUBJECT}))"',
env: {
DECISION: "$review.response.decision",
SUBJECT: "$review.subject.text",
},
},
],
},
null,
2,
),
"utf8",
);
await fsp.writeFile(
filePath,
JSON.stringify(
{
steps: [
{
id: "draft",
run: "node -e \"process.stdout.write(JSON.stringify({text:'hello'}))\"",
},
{
id: "review",
input: {
prompt: "Review draft?",
responseSchema: {
type: "object",
properties: { decision: { type: "string" } },
required: ["decision"],
},
},
},
{
id: "finish",
run: 'node -e "process.stdout.write(JSON.stringify({decision:process.env.DECISION,subject:process.env.SUBJECT}))"',
env: {
DECISION: "$review.response.decision",
SUBJECT: "$review.subject.text",
},
},
],
},
null,
2,
),
"utf8",
);
const env = {
...process.env,
LOBSTER_STATE_DIR: path.join(tmpDir, "state"),
};
const env = {
...process.env,
LOBSTER_STATE_DIR: path.join(tmpDir, "state"),
};
const first = await runToolRequest({
filePath,
ctx: { cwd: tmpDir, env },
});
const first = await runToolRequest({
filePath,
ctx: { cwd: tmpDir, env },
});
assert.equal(first.ok, true);
assert.equal(first.status, "needs_input");
assert.deepEqual(first.requiresInput?.subject, { text: "hello" });
assert.ok(first.requiresInput?.resumeToken);
assert.equal(first.ok, true);
assert.equal(first.status, "needs_input");
assert.deepEqual(first.requiresInput?.subject, { text: "hello" });
assert.ok(first.requiresInput?.resumeToken);
const resumed = await resumeToolRequest({
token: first.requiresInput?.resumeToken ?? "",
response: { decision: "approve" },
ctx: { cwd: tmpDir, env },
});
const resumed = await resumeToolRequest({
token: first.requiresInput?.resumeToken ?? "",
response: { decision: "approve" },
ctx: { cwd: tmpDir, env },
});
assert.equal(resumed.ok, true);
assert.equal(resumed.status, "ok");
assert.deepEqual(resumed.output, [{ decision: "approve", subject: "hello" }]);
assert.equal(resumed.ok, true);
assert.equal(resumed.status, "ok");
assert.deepEqual(resumed.output, [{ decision: "approve", subject: "hello" }]);
});
test/cost_tracker.test.ts
+2255 -205
View File
File diff suppressed because it is too large Load Diff
test/dedupe.test.ts
+33 -33
View File
@@ -6,46 +6,46 @@ import { createDefaultRegistry } from "../src/commands/registry.js";
import { parsePipeline } from "../src/parser.js";
async function run(pipelineText: string, input: any[]) {
const pipeline = parsePipeline(pipelineText);
const registry = createDefaultRegistry();
const res = await runPipeline({
pipeline,
registry,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
input: (async function* () {
for (const x of input) yield x;
})(),
});
return res.items;
const pipeline = parsePipeline(pipelineText);
const registry = createDefaultRegistry();
const res = await runPipeline({
pipeline,
registry,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
input: (async function* () {
for (const x of input) yield x;
})(),
});
return res.items;
}
test("dedupe removes duplicate primitives (stable)", async () => {
const out = await run("dedupe", [1, 2, 1, 3, 2]);
assert.deepEqual(out, [1, 2, 3]);
const out = await run("dedupe", [1, 2, 1, 3, 2]);
assert.deepEqual(out, [1, 2, 3]);
});
test("dedupe supports --key", async () => {
const input = [
{ id: "a", v: 1 },
{ id: "b", v: 2 },
{ id: "a", v: 3 },
];
const out = await run("dedupe --key id", input);
assert.deepEqual(out, [input[0], input[1]]);
const input = [
{ id: "a", v: 1 },
{ id: "b", v: 2 },
{ id: "a", v: 3 },
];
const out = await run("dedupe --key id", input);
assert.deepEqual(out, [input[0], input[1]]);
});
test("dedupe treats undefined keys as a key value", async () => {
const input = [
{ id: undefined, v: 1 },
{ id: undefined, v: 2 },
{ id: "x", v: 3 },
];
const out = await run("dedupe --key id", input);
assert.equal(out.length, 2);
assert.equal(out[0].v, 1);
assert.equal(out[1].v, 3);
const input = [
{ id: undefined, v: 1 },
{ id: undefined, v: 2 },
{ id: "x", v: 3 },
];
const out = await run("dedupe --key id", input);
assert.equal(out.length, 2);
assert.equal(out[0].v, 1);
assert.equal(out[1].v, 3);
});
test/diff_last.test.ts
+55 -55
View File
@@ -6,65 +6,65 @@ import { mkdtempSync } from "node:fs";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
test("diff.last reports changed on first run and not changed on same input", async () => {
const tmp = mkdtempSync(path.join(os.tmpdir(), "lobster-diff-"));
const env = { ...process.env, LOBSTER_STATE_DIR: tmp };
const registry = createDefaultRegistry();
const cmd = registry.get("diff.last");
const tmp = mkdtempSync(path.join(os.tmpdir(), "lobster-diff-"));
const env = { ...process.env, LOBSTER_STATE_DIR: tmp };
const registry = createDefaultRegistry();
const cmd = registry.get("diff.last");
const first = await cmd.run({
input: streamOf([{ a: 1 }]),
args: { _: [], key: "k" },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const out1 = [];
for await (const it of first.output) out1.push(it);
assert.equal(out1[0].changed, true);
const first = await cmd.run({
input: streamOf([{ a: 1 }]),
args: { _: [], key: "k" },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const out1 = [];
for await (const it of first.output) out1.push(it);
assert.equal(out1[0].changed, true);
const second = await cmd.run({
input: streamOf([{ a: 1 }]),
args: { _: [], key: "k" },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const out2 = [];
for await (const it of second.output) out2.push(it);
assert.equal(out2[0].changed, false);
const second = await cmd.run({
input: streamOf([{ a: 1 }]),
args: { _: [], key: "k" },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const out2 = [];
for await (const it of second.output) out2.push(it);
assert.equal(out2[0].changed, false);
const third = await cmd.run({
input: streamOf([{ a: 2 }]),
args: { _: [], key: "k" },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const out3 = [];
for await (const it of third.output) out3.push(it);
assert.equal(out3[0].changed, true);
const third = await cmd.run({
input: streamOf([{ a: 2 }]),
args: { _: [], key: "k" },
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env,
registry,
mode: "tool",
render: { json() {}, lines() {} },
},
});
const out3 = [];
for await (const it of third.output) out3.push(it);
assert.equal(out3[0].changed, true);
});
test/doctor.test.ts
+12 -12
View File
@@ -4,16 +4,16 @@ import { spawnSync } from "node:child_process";
import path from "node:path";
test("doctor returns tool-mode ok with version", () => {
const bin = path.join(process.cwd(), "bin", "lobster.js");
const res = spawnSync("node", [bin, "doctor"], {
encoding: "utf8",
env: { ...process.env, LOBSTER_STATE_DIR: path.join(process.cwd(), ".tmp-test-state") },
});
assert.equal(res.status, 0);
const out = JSON.parse(res.stdout);
assert.equal(out.ok, true);
assert.equal(out.protocolVersion, 1);
assert.equal(out.status, "ok");
assert.equal(out.output[0].toolMode, true);
assert.ok(typeof out.output[0].version === "string");
const bin = path.join(process.cwd(), "bin", "lobster.js");
const res = spawnSync("node", [bin, "doctor"], {
encoding: "utf8",
env: { ...process.env, LOBSTER_STATE_DIR: path.join(process.cwd(), ".tmp-test-state") },
});
assert.equal(res.status, 0);
const out = JSON.parse(res.stdout);
assert.equal(out.ok, true);
assert.equal(out.protocolVersion, 1);
assert.equal(out.status, "ok");
assert.equal(out.output[0].toolMode, true);
assert.ok(typeof out.output[0].version === "string");
});
test/dry_run.test.ts
+476 -476
View File
File diff suppressed because it is too large Load Diff
test/email_triage.test.ts
+272 -272
View File
@@ -13,314 +13,314 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
async function closeServer(server: http.Server) {
await new Promise<void>((resolve) => server.close(() => resolve()));
await new Promise<void>((resolve) => server.close(() => resolve()));
}
test("gog.gmail.search | email.triage works end-to-end (mock gog)", async () => {
const registry = createDefaultRegistry();
const registry = createDefaultRegistry();
// Tests run from dist/, but fixtures live in source tree.
const repoRoot = join(__dirname, "..", "..");
const mockGog = join(repoRoot, "test", "fixtures", "mock-gog.mjs");
// Tests run from dist/, but fixtures live in source tree.
const repoRoot = join(__dirname, "..", "..");
const mockGog = join(repoRoot, "test", "fixtures", "mock-gog.mjs");
const result = await runPipeline({
pipeline: [
{ name: "gog.gmail.search", args: { query: "newer_than:1d", max: 20 }, raw: "" },
{ name: "email.triage", args: { limit: 20 }, raw: "" },
],
registry,
input: [],
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, GOG_BIN: mockGog },
mode: "tool",
} as any);
const result = await runPipeline({
pipeline: [
{ name: "gog.gmail.search", args: { query: "newer_than:1d", max: 20 }, raw: "" },
{ name: "email.triage", args: { limit: 20 }, raw: "" },
],
registry,
input: [],
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, GOG_BIN: mockGog },
mode: "tool",
} as any);
assert.equal(result.items.length, 1);
assert.equal(result.items[0].summary, "1 need replies, 1 need action, 1 FYI");
assert.equal(result.items.length, 1);
assert.equal(result.items[0].summary, "1 need replies, 1 need action, 1 FYI");
});
test("email.triage buckets based on subject/from/labels", async () => {
const registry = createDefaultRegistry();
const registry = createDefaultRegistry();
const emails = [
{
id: "m1",
threadId: "t1",
from: "Alice <alice@example.com>",
subject: "Quick question",
date: "2026-01-22T07:00:00Z",
snippet: "Hey, can you take a look?",
labels: ["INBOX", "UNREAD"],
},
{
id: "m2",
threadId: "t2",
from: "no-reply@service.com",
subject: "Your receipt",
date: "2026-01-22T06:00:00Z",
snippet: "Thanks",
labels: ["INBOX", "UNREAD"],
},
{
id: "m3",
threadId: "t3",
from: "Bob <bob@example.com>",
subject: "Action required: NDA",
date: "2026-01-21T23:00:00Z",
snippet: "Please sign",
labels: ["INBOX"],
},
];
const emails = [
{
id: "m1",
threadId: "t1",
from: "Alice <alice@example.com>",
subject: "Quick question",
date: "2026-01-22T07:00:00Z",
snippet: "Hey, can you take a look?",
labels: ["INBOX", "UNREAD"],
},
{
id: "m2",
threadId: "t2",
from: "no-reply@service.com",
subject: "Your receipt",
date: "2026-01-22T06:00:00Z",
snippet: "Thanks",
labels: ["INBOX", "UNREAD"],
},
{
id: "m3",
threadId: "t3",
from: "Bob <bob@example.com>",
subject: "Action required: NDA",
date: "2026-01-21T23:00:00Z",
snippet: "Please sign",
labels: ["INBOX"],
},
];
const input = (async function* () {
for (const e of emails) yield e;
})();
const input = (async function* () {
for (const e of emails) yield e;
})();
const result = await runPipeline({
pipeline: [{ name: "email.triage", args: { limit: 20 }, raw: "" }],
registry,
input,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
} as any);
const result = await runPipeline({
pipeline: [{ name: "email.triage", args: { limit: 20 }, raw: "" }],
registry,
input,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
} as any);
assert.equal(result.items.length, 1);
const out = result.items[0];
assert.equal(out.summary, "1 need replies, 1 need action, 1 FYI");
assert.deepEqual(out.buckets.needsReply, ["m1"]);
assert.deepEqual(out.buckets.needsAction, ["m3"]);
assert.deepEqual(out.buckets.fyi, ["m2"]);
assert.equal(result.items.length, 1);
const out = result.items[0];
assert.equal(out.summary, "1 need replies, 1 need action, 1 FYI");
assert.deepEqual(out.buckets.needsReply, ["m1"]);
assert.deepEqual(out.buckets.needsAction, ["m3"]);
assert.deepEqual(out.buckets.fyi, ["m2"]);
});
test("email.triage --llm uses llm_task.invoke to draft replies (and can emit drafts)", async () => {
const registry = createDefaultRegistry();
const registry = createDefaultRegistry();
const emails = [
{
id: "m1",
threadId: "t1",
from: "Alice <alice@example.com>",
subject: "Quick question",
date: "2026-01-22T07:00:00Z",
snippet: "Hey, can you take a look?",
labels: ["INBOX", "UNREAD"],
},
{
id: "m2",
threadId: "t2",
from: "Bob <bob@example.com>",
subject: "Action required: NDA",
date: "2026-01-21T23:00:00Z",
snippet: "Please sign",
labels: ["INBOX"],
},
];
const emails = [
{
id: "m1",
threadId: "t1",
from: "Alice <alice@example.com>",
subject: "Quick question",
date: "2026-01-22T07:00:00Z",
snippet: "Hey, can you take a look?",
labels: ["INBOX", "UNREAD"],
},
{
id: "m2",
threadId: "t2",
from: "Bob <bob@example.com>",
subject: "Action required: NDA",
date: "2026-01-21T23:00:00Z",
snippet: "Please sign",
labels: ["INBOX"],
},
];
const cacheDir = await mkdtemp(join(tmpdir(), "lobster-cache-"));
const cacheDir = await mkdtemp(join(tmpdir(), "lobster-cache-"));
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
const parsed = JSON.parse(buf || "{}");
bodyLog.push(parsed);
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
const parsed = JSON.parse(buf || "{}");
bodyLog.push(parsed);
res.writeHead(200, { "content-type": "application/json" });
// OpenClaw tool router envelope -> llm-task tool envelope
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "triage_1",
output: {
data: {
decisions: [
{
id: "m1",
category: "needs_reply",
rationale: "Unclear question",
reply: { body: "Sure — what’s the deadline?" },
},
{ id: "m2", category: "needs_action", rationale: "NDA" },
],
},
},
},
},
}),
);
});
});
res.writeHead(200, { "content-type": "application/json" });
// OpenClaw tool router envelope -> llm-task tool envelope
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "triage_1",
output: {
data: {
decisions: [
{
id: "m1",
category: "needs_reply",
rationale: "Unclear question",
reply: { body: "Sure — what’s the deadline?" },
},
{ id: "m2", category: "needs_action", rationale: "NDA" },
],
},
},
},
},
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
// Report mode
const input1 = (async function* () {
for (const e of emails) yield e;
})();
try {
// Report mode
const input1 = (async function* () {
for (const e of emails) yield e;
})();
const res1 = await runPipeline({
pipeline: [
{ name: "email.triage", args: { llm: true, model: "claude-test", limit: 20 }, raw: "" },
],
registry,
input: input1,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: {
...process.env,
CLAWD_URL: `http://127.0.0.1:${port}`,
LOBSTER_CACHE_DIR: cacheDir,
LLM_TASK_FORCE_REFRESH: "1",
},
mode: "tool",
} as any);
const res1 = await runPipeline({
pipeline: [
{ name: "email.triage", args: { llm: true, model: "claude-test", limit: 20 }, raw: "" },
],
registry,
input: input1,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: {
...process.env,
CLAWD_URL: `http://127.0.0.1:${port}`,
LOBSTER_CACHE_DIR: cacheDir,
LLM_TASK_FORCE_REFRESH: "1",
},
mode: "tool",
} as any);
assert.equal(res1.items.length, 1);
assert.equal(res1.items[0].mode, "llm");
assert.equal(res1.items[0].buckets.needsReply.length, 1);
assert.equal(res1.items[0].drafts.length, 1);
assert.equal(res1.items[0].drafts[0].to, "alice@example.com");
assert.equal(res1.items.length, 1);
assert.equal(res1.items[0].mode, "llm");
assert.equal(res1.items[0].buckets.needsReply.length, 1);
assert.equal(res1.items[0].drafts.length, 1);
assert.equal(res1.items[0].drafts[0].to, "alice@example.com");
// Draft emit mode
const input2 = (async function* () {
for (const e of emails) yield e;
})();
// Draft emit mode
const input2 = (async function* () {
for (const e of emails) yield e;
})();
const res2 = await runPipeline({
pipeline: [
{
name: "email.triage",
args: { llm: true, model: "claude-test", limit: 20, emit: "drafts" },
raw: "",
},
],
registry,
input: input2,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: {
...process.env,
CLAWD_URL: `http://127.0.0.1:${port}`,
LOBSTER_CACHE_DIR: cacheDir,
LLM_TASK_FORCE_REFRESH: "1",
},
mode: "tool",
} as any);
const res2 = await runPipeline({
pipeline: [
{
name: "email.triage",
args: { llm: true, model: "claude-test", limit: 20, emit: "drafts" },
raw: "",
},
],
registry,
input: input2,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: {
...process.env,
CLAWD_URL: `http://127.0.0.1:${port}`,
LOBSTER_CACHE_DIR: cacheDir,
LLM_TASK_FORCE_REFRESH: "1",
},
mode: "tool",
} as any);
assert.equal(res2.items.length, 1);
assert.equal(res2.items[0].to, "alice@example.com");
assert.ok(res2.items[0].subject.toLowerCase().startsWith("re:"));
assert.equal(bodyLog.length >= 1, true);
assert.equal(bodyLog[0].args?.model ?? bodyLog[0].model, "claude-test");
assert.ok(bodyLog[0].prompt || bodyLog[0].args?.prompt);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
assert.equal(res2.items.length, 1);
assert.equal(res2.items[0].to, "alice@example.com");
assert.ok(res2.items[0].subject.toLowerCase().startsWith("re:"));
assert.equal(bodyLog.length >= 1, true);
assert.equal(bodyLog[0].args?.model ?? bodyLog[0].model, "claude-test");
assert.ok(bodyLog[0].prompt || bodyLog[0].args?.prompt);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("email.triage --llm honors OPENCLAW_URL (not just CLAWD_URL)", async () => {
const registry = createDefaultRegistry();
const cacheDir = await mkdtemp(join(tmpdir(), "lobster-cache-"));
const registry = createDefaultRegistry();
const cacheDir = await mkdtemp(join(tmpdir(), "lobster-cache-"));
const emails = [
{
id: "m1",
threadId: "t1",
from: "Alice <alice@example.com>",
subject: "Quick question",
date: "2026-01-22T07:00:00Z",
snippet: "Hey, can you take a look?",
labels: ["INBOX", "UNREAD"],
},
];
const emails = [
{
id: "m1",
threadId: "t1",
from: "Alice <alice@example.com>",
subject: "Quick question",
date: "2026-01-22T07:00:00Z",
snippet: "Hey, can you take a look?",
labels: ["INBOX", "UNREAD"],
},
];
let callCount = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let callCount = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
callCount++;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "triage_openclaw_url",
output: {
data: {
decisions: [
{
id: "m1",
category: "needs_reply",
reply: { body: "Absolutely — I can help." },
},
],
},
},
},
},
}),
);
});
callCount++;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "triage_openclaw_url",
output: {
data: {
decisions: [
{
id: "m1",
category: "needs_reply",
reply: { body: "Absolutely — I can help." },
},
],
},
},
},
},
}),
);
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
const input = (async function* () {
for (const e of emails) yield e;
})();
try {
const input = (async function* () {
for (const e of emails) yield e;
})();
const result = await runPipeline({
pipeline: [{ name: "email.triage", args: { llm: true, limit: 20 }, raw: "" }],
registry,
input,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: {
...process.env,
OPENCLAW_URL: `http://127.0.0.1:${port}`,
LOBSTER_CACHE_DIR: cacheDir,
LLM_TASK_FORCE_REFRESH: "1",
},
mode: "tool",
} as any);
const result = await runPipeline({
pipeline: [{ name: "email.triage", args: { llm: true, limit: 20 }, raw: "" }],
registry,
input,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: {
...process.env,
OPENCLAW_URL: `http://127.0.0.1:${port}`,
LOBSTER_CACHE_DIR: cacheDir,
LLM_TASK_FORCE_REFRESH: "1",
},
mode: "tool",
} as any);
assert.equal(callCount, 1);
assert.equal(result.items.length, 1);
assert.equal(result.items[0].mode, "llm");
assert.equal(result.items[0].drafts.length, 1);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
assert.equal(callCount, 1);
assert.equal(result.items.length, 1);
assert.equal(result.items[0].mode, "llm");
assert.equal(result.items[0].drafts.length, 1);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test/exec_stdin.test.ts
+33 -33
View File
@@ -3,43 +3,43 @@ import assert from "node:assert/strict";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
test("exec --stdin jsonl feeds pipeline input to subprocess", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("exec");
const registry = createDefaultRegistry();
const cmd = registry.get("exec");
const nodeScript = [
"let d='';",
"process.stdin.on('data',c=>d+=c);",
"process.stdin.on('end',()=>{",
" const lines=d.trim().split('\\n').filter(Boolean);",
" console.log(JSON.stringify(lines));",
"});",
].join("");
const nodeScript = [
"let d='';",
"process.stdin.on('data',c=>d+=c);",
"process.stdin.on('end',()=>{",
" const lines=d.trim().split('\\n').filter(Boolean);",
" console.log(JSON.stringify(lines));",
"});",
].join("");
const result = await cmd.run({
input: streamOf([{ a: 1 }, { a: 2 }]),
args: {
_: ["node", "-e", nodeScript],
stdin: "jsonl",
json: true,
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "human",
render: { json() {}, lines() {} },
},
});
const result = await cmd.run({
input: streamOf([{ a: 1 }, { a: 2 }]),
args: {
_: ["node", "-e", nodeScript],
stdin: "jsonl",
json: true,
},
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
registry,
mode: "human",
render: { json() {}, lines() {} },
},
});
const items = [];
for await (const item of result.output) items.push(item);
assert.deepEqual(items, ['{"a":1}', '{"a":2}']);
const items = [];
for await (const item of result.output) items.push(item);
assert.deepEqual(items, ['{"a":1}', '{"a":2}']);
});
test/fixtures/abortable-process-short-lived.mjs
Vendored
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env node
import { pathToFileURL } from "node:url";
import { access } from "node:fs/promises";
const [runnerPath, mockGog] = process.argv.slice(2);
const { runAbortableProcess } = await import(pathToFileURL(runnerPath).href);
async function waitFor(path) {
for (let attempt = 0; attempt < 300; attempt += 1) {
try {
await access(path);
return;
} catch {
await new Promise((resolve) => setTimeout(resolve, 10));
}
}
throw new Error(`Timed out waiting for ${path}`);
}
const controller = new AbortController();
const run = runAbortableProcess({
command: process.execPath,
argv: [mockGog, "gmail", "search"],
env: process.env,
signal: controller.signal,
notFoundMessage: "mock gog not found",
});
await waitFor(process.env.MOCK_GOG_SEARCH_STARTED_FILE);
await waitFor(process.env.MOCK_GOG_DESCENDANT_STARTED_FILE);
controller.abort(new Error("cancelled by short-lived caller"));
await run.catch(() => undefined);
test/fixtures/abortable-process-terminal-direct.mjs
Vendored
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env node
import { readFileSync, writeFileSync } from "node:fs";
import { pathToFileURL } from "node:url";
const [runnerPath] = process.argv.slice(2);
const { runAbortableProcess } = await import(pathToFileURL(runnerPath).href);
function processGroupId() {
const stat = readFileSync("/proc/self/stat", "utf8");
const fields = stat
.slice(stat.lastIndexOf(")") + 1)
.trim()
.split(/\s+/);
return Number(fields[2]);
}
writeFileSync(process.env.LOBSTER_TERMINAL_DIRECT_GROUP_FILE, String(processGroupId()), "utf8");
await runAbortableProcess({
command: process.execPath,
argv: [
"-e",
`const { writeFileSync } = require("node:fs");
writeFileSync(process.env.LOBSTER_TERMINAL_DIRECT_STARTED_FILE, String(process.pid));
setTimeout(() => writeFileSync(process.env.LOBSTER_TERMINAL_DIRECT_COMPLETED_FILE, "completed"), 700);`,
],
env: process.env,
notFoundMessage: "node missing",
});
test/fixtures/gog_gmail_search.json
Vendored
+27 -27
View File
@@ -1,29 +1,29 @@
[
{
"id": "m1",
"threadId": "t1",
"from": "Alice <alice@example.com>",
"subject": "Quick question",
"date": "2026-01-22T07:00:00Z",
"snippet": "Hey, can you take a look?",
"labels": ["INBOX", "UNREAD"]
},
{
"id": "m2",
"threadId": "t2",
"from": "no-reply@service.com",
"subject": "Your receipt",
"date": "2026-01-22T06:00:00Z",
"snippet": "Thanks for your purchase",
"labels": ["INBOX", "UNREAD"]
},
{
"id": "m3",
"threadId": "t3",
"from": "Bob <bob@example.com>",
"subject": "Action required: NDA",
"date": "2026-01-21T23:00:00Z",
"snippet": "Please sign",
"labels": ["INBOX"]
}
{
"id": "m1",
"threadId": "t1",
"from": "Alice <alice@example.com>",
"subject": "Quick question",
"date": "2026-01-22T07:00:00Z",
"snippet": "Hey, can you take a look?",
"labels": ["INBOX", "UNREAD"]
},
{
"id": "m2",
"threadId": "t2",
"from": "no-reply@service.com",
"subject": "Your receipt",
"date": "2026-01-22T06:00:00Z",
"snippet": "Thanks for your purchase",
"labels": ["INBOX", "UNREAD"]
},
{
"id": "m3",
"threadId": "t3",
"from": "Bob <bob@example.com>",
"subject": "Action required: NDA",
"date": "2026-01-21T23:00:00Z",
"snippet": "Please sign",
"labels": ["INBOX"]
}
]
test/fixtures/mock-gh-cancellation.mjs
Vendored
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env node
import { writeFileSync } from "node:fs";
function mark(path, value) {
if (path) writeFileSync(path, value, "utf8");
}
process.once("SIGTERM", () => {
mark(process.env.MOCK_GH_TERMINATED_FILE, "SIGTERM");
const terminationDelayMs = Number(process.env.MOCK_GH_TERMINATION_DELAY_MS ?? 0);
setTimeout(() => process.exit(143), terminationDelayMs);
});
mark(process.env.MOCK_GH_STARTED_FILE, String(process.pid));
setTimeout(
() => {
mark(process.env.MOCK_GH_COMPLETED_FILE, "completed");
process.stdout.write(
JSON.stringify({
number: 1,
title: "Fixture PR",
url: "https://example.invalid/pr/1",
state: "OPEN",
isDraft: false,
mergeable: "MERGEABLE",
reviewDecision: "",
updatedAt: "2026-07-11T00:00:00Z",
baseRefName: "main",
headRefName: "fixture",
}),
);
},
Number(process.env.MOCK_GH_COMPLETION_DELAY_MS ?? 1200),
);
test/fixtures/mock-gog-cancellation.mjs
Vendored
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env node
import { appendFileSync, writeFileSync } from "node:fs";
import { spawn } from "node:child_process";
const argv = process.argv.slice(2);
function mark(path, value) {
if (path) writeFileSync(path, value, "utf8");
}
function startDescendant() {
if (!process.env.MOCK_GOG_DESCENDANT_STARTED_FILE) return;
const helper = spawn(
process.execPath,
[
"-e",
`const { writeFileSync } = require("node:fs");
process.once("SIGTERM", () => {});
writeFileSync(process.env.MOCK_GOG_DESCENDANT_STARTED_FILE, String(process.pid));
setTimeout(() => writeFileSync(process.env.MOCK_GOG_DESCENDANT_COMPLETED_FILE, "completed"), 650);`,
],
{ env: process.env, stdio: "ignore" },
);
helper.unref();
}
function waitForCompletion({ startedFile, terminatedFile, completedFile, output }) {
const terminationDelayMs = Number(process.env.MOCK_GOG_TERMINATION_DELAY_MS ?? 0);
const completionDelayMs = Number(process.env.MOCK_GOG_COMPLETION_DELAY_MS ?? 1200);
process.once("SIGTERM", () => {
mark(terminatedFile, "SIGTERM");
setTimeout(() => process.exit(143), terminationDelayMs);
});
mark(startedFile, String(process.pid));
startDescendant();
setTimeout(() => {
mark(completedFile, "completed");
process.stdout.write(JSON.stringify(output));
}, completionDelayMs);
}
if (argv[0] === "gmail" && argv[1] === "search") {
waitForCompletion({
startedFile: process.env.MOCK_GOG_SEARCH_STARTED_FILE,
terminatedFile: process.env.MOCK_GOG_SEARCH_TERMINATED_FILE,
completedFile: process.env.MOCK_GOG_SEARCH_COMPLETED_FILE,
output: [{ to: "user@example.com", subject: "Reply", body: "Hello" }],
});
} else if (argv[0] === "gmail" && argv[1] === "send") {
if (process.env.MOCK_GOG_SEND_INVOCATIONS_FILE) {
appendFileSync(process.env.MOCK_GOG_SEND_INVOCATIONS_FILE, `${process.pid}\n`, "utf8");
}
waitForCompletion({
startedFile: process.env.MOCK_GOG_SEND_STARTED_FILE,
terminatedFile: process.env.MOCK_GOG_SEND_TERMINATED_FILE,
completedFile: process.env.MOCK_GOG_SEND_COMPLETED_FILE,
output: { ok: true },
});
} else {
process.stderr.write(`mock-gog-cancellation: unsupported args: ${argv.join(" ")}\n`);
process.exit(2);
}
test/fixtures/mock-gog.mjs
Vendored
+6 -6
View File
@@ -10,15 +10,15 @@ const argv = process.argv.slice(2);
// Minimal mock for `gog gmail search` and `gog gmail send`.
if (argv[0] === "gmail" && argv[1] === "search") {
const data = readFileSync(join(__dirname, "gog_gmail_search.json"), "utf8");
process.stdout.write(data);
process.exit(0);
const data = readFileSync(join(__dirname, "gog_gmail_search.json"), "utf8");
process.stdout.write(data);
process.exit(0);
}
if (argv[0] === "gmail" && argv[1] === "send") {
// Echo a json success object.
process.stdout.write(JSON.stringify({ ok: true }));
process.exit(0);
// Echo a json success object.
process.stdout.write(JSON.stringify({ ok: true }));
process.exit(0);
}
process.stderr.write("mock-gog: unsupported args: " + argv.join(" ") + "\n");
test/fixtures/mock-openclaw-agent.mjs
Vendored
+32
View File
@@ -0,0 +1,32 @@
import { spawn } from "node:child_process";
const writeResponse = () => {
process.stdout.write(
JSON.stringify({
runId: "fixture-run",
status: "ok",
result: { payloads: [{ text: "fixture reply" }] },
}),
);
};
if (process.argv.includes("--spawn-descendant")) {
const helper = spawn(
process.execPath,
[
"-e",
`const { writeFileSync } = require("node:fs");
process.once("SIGTERM", () => {});
writeFileSync(process.env.MOCK_OPENCLAW_AGENT_DESCENDANT_STARTED_FILE, String(process.pid));
setTimeout(() => writeFileSync(process.env.MOCK_OPENCLAW_AGENT_DESCENDANT_COMPLETED_FILE, "completed"), 650);`,
],
{ env: process.env, stdio: "ignore" },
);
helper.unref();
}
if (process.argv.includes("--sleep")) {
setTimeout(writeResponse, 10_000);
} else {
writeResponse();
}
test/for_each.test.ts
+234 -234
View File
@@ -9,276 +9,276 @@ import { createDefaultRegistry } from "../src/commands/registry.js";
import { loadWorkflowFile, runWorkflowFile } from "../src/workflows/file.js";
async function runWorkflow(workflow: unknown) {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const stateDir = path.join(tmpDir, "state");
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(filePath, JSON.stringify(workflow, null, 2), "utf8");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const stateDir = path.join(tmpDir, "state");
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(filePath, JSON.stringify(workflow, null, 2), "utf8");
return runWorkflowFile({
filePath,
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, LOBSTER_STATE_DIR: stateDir },
mode: "tool",
registry: createDefaultRegistry(),
},
});
return runWorkflowFile({
filePath,
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, LOBSTER_STATE_DIR: stateDir },
mode: "tool",
registry: createDefaultRegistry(),
},
});
}
test("for_each iterates items and collects per-iteration results", async () => {
const result = await runWorkflow({
steps: [
{
id: "data",
command: 'node -e "process.stdout.write(JSON.stringify([{name:\\"a\\"},{name:\\"b\\"}]))"',
},
{
id: "loop",
for_each: "$data.json",
steps: [
{
id: "transform",
command:
'node -e "process.stdout.write(JSON.stringify({upper: process.env.NAME.toUpperCase()}))"',
env: { NAME: "$item.json.name" },
},
],
},
],
});
assert.equal(result.status, "ok");
const output = result.output as any[];
assert.equal(output.length, 2);
assert.equal(output[0].index, 0);
assert.equal(output[1].index, 1);
assert.equal(output[0].transform.upper, "A");
assert.equal(output[1].transform.upper, "B");
const result = await runWorkflow({
steps: [
{
id: "data",
command: 'node -e "process.stdout.write(JSON.stringify([{name:\\"a\\"},{name:\\"b\\"}]))"',
},
{
id: "loop",
for_each: "$data.json",
steps: [
{
id: "transform",
command:
'node -e "process.stdout.write(JSON.stringify({upper: process.env.NAME.toUpperCase()}))"',
env: { NAME: "$item.json.name" },
},
],
},
],
});
assert.equal(result.status, "ok");
const output = result.output as any[];
assert.equal(output.length, 2);
assert.equal(output[0].index, 0);
assert.equal(output[1].index, 1);
assert.equal(output[0].transform.upper, "A");
assert.equal(output[1].transform.upper, "B");
});
test("for_each supports custom item_var and index_var", async () => {
const result = await runWorkflow({
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([10,20]))"' },
{
id: "loop",
for_each: "$vals.json",
item_var: "num",
index_var: "idx",
steps: [
{
id: "emit",
command:
'node -e "process.stdout.write(JSON.stringify({num:$num.json,idx:$idx.json}))"',
},
],
},
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, [
{ num: 10, idx: 0, emit: { num: 10, idx: 0 } },
{ num: 20, idx: 1, emit: { num: 20, idx: 1 } },
]);
const result = await runWorkflow({
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([10,20]))"' },
{
id: "loop",
for_each: "$vals.json",
item_var: "num",
index_var: "idx",
steps: [
{
id: "emit",
command:
'node -e "process.stdout.write(JSON.stringify({num:$num.json,idx:$idx.json}))"',
},
],
},
],
});
assert.equal(result.status, "ok");
assert.deepEqual(result.output, [
{ num: 10, idx: 0, emit: { num: 10, idx: 0 } },
{ num: 20, idx: 1, emit: { num: 20, idx: 1 } },
]);
});
test("for_each pipeline sub-steps reject command-level requestInput", async () => {
await assert.rejects(
() =>
runWorkflow({
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([1]))"' },
{
id: "loop",
for_each: "$vals.json",
steps: [
{
id: "review",
pipeline: "ask --prompt 'Review?'",
},
],
},
],
}),
/requestInput is not supported in this pipeline context/,
);
await assert.rejects(
() =>
runWorkflow({
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([1]))"' },
{
id: "loop",
for_each: "$vals.json",
steps: [
{
id: "review",
pipeline: "ask --prompt 'Review?'",
},
],
},
],
}),
/requestInput is not supported in this pipeline context/,
);
});
test("for_each throws when source is not an array", async () => {
await assert.rejects(
() =>
runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({x:1}))"' },
{ id: "loop", for_each: "$data.json", steps: [{ id: "x", command: "echo hi" }] },
],
}),
/for_each: expected array/,
);
await assert.rejects(
() =>
runWorkflow({
steps: [
{ id: "data", command: 'node -e "process.stdout.write(JSON.stringify({x:1}))"' },
{ id: "loop", for_each: "$data.json", steps: [{ id: "x", command: "echo hi" }] },
],
}),
/for_each: expected array/,
);
});
test("for_each validation rejects empty sub-step list", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [{ id: "loop", for_each: "$x.json", steps: [] }],
}),
"utf8",
);
await assert.rejects(
() => loadWorkflowFile(filePath),
/for_each requires a non-empty steps array/,
);
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [{ id: "loop", for_each: "$x.json", steps: [] }],
}),
"utf8",
);
await assert.rejects(
() => loadWorkflowFile(filePath),
/for_each requires a non-empty steps array/,
);
});
test("for_each validation rejects run/command/pipeline/workflow/parallel on loop step", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
run: "echo no",
steps: [{ id: "s", command: "echo hi" }],
},
],
}),
"utf8",
);
await assert.rejects(
() => loadWorkflowFile(filePath),
/for_each cannot also define run, command, pipeline, workflow, or parallel/,
);
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
run: "echo no",
steps: [{ id: "s", command: "echo hi" }],
},
],
}),
"utf8",
);
await assert.rejects(
() => loadWorkflowFile(filePath),
/for_each cannot also define run, command, pipeline, workflow, or parallel/,
);
});
test("for_each validation rejects approval/input in sub-steps", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
steps: [{ id: "s", command: "echo hi", approval: true }],
},
],
}),
"utf8",
);
await assert.rejects(() => loadWorkflowFile(filePath), /cannot contain approval or input/);
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
steps: [{ id: "s", command: "echo hi", approval: true }],
},
],
}),
"utf8",
);
await assert.rejects(() => loadWorkflowFile(filePath), /cannot contain approval or input/);
});
test("for_each validation rejects duplicate sub-step ids", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
steps: [
{ id: "dup", command: "echo a" },
{ id: "dup", command: "echo b" },
],
},
],
}),
"utf8",
);
await assert.rejects(() => loadWorkflowFile(filePath), /duplicate for_each sub-step id/);
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
steps: [
{ id: "dup", command: "echo a" },
{ id: "dup", command: "echo b" },
],
},
],
}),
"utf8",
);
await assert.rejects(() => loadWorkflowFile(filePath), /duplicate for_each sub-step id/);
});
test("for_each validation rejects item_var/index_var collisions", async () => {
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
item_var: "x",
index_var: "x",
steps: [{ id: "s", command: "echo hi" }],
},
],
}),
"utf8",
);
await assert.rejects(
() => loadWorkflowFile(filePath),
/item_var and index_var cannot be the same/,
);
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const filePath = path.join(tmpDir, "bad.lobster");
await fsp.writeFile(
filePath,
JSON.stringify({
steps: [
{
id: "loop",
for_each: "$x.json",
item_var: "x",
index_var: "x",
steps: [{ id: "s", command: "echo hi" }],
},
],
}),
"utf8",
);
await assert.rejects(
() => loadWorkflowFile(filePath),
/item_var and index_var cannot be the same/,
);
});
test("for_each pause_ms and batch_size are accepted and executable", async () => {
const result = await runWorkflow({
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([1,2,3]))"' },
{
id: "loop",
for_each: "$vals.json",
batch_size: 2,
pause_ms: 10,
steps: [
{ id: "emit", command: 'node -e "process.stdout.write(JSON.stringify({v:$item.json}))"' },
],
},
],
});
assert.equal(result.status, "ok");
assert.equal((result.output as any[]).length, 3);
const result = await runWorkflow({
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([1,2,3]))"' },
{
id: "loop",
for_each: "$vals.json",
batch_size: 2,
pause_ms: 10,
steps: [
{ id: "emit", command: 'node -e "process.stdout.write(JSON.stringify({v:$item.json}))"' },
],
},
],
});
assert.equal(result.status, "ok");
assert.equal((result.output as any[]).length, 3);
});
test("for_each dry-run renders loop structure", async () => {
const workflow = {
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([1,2]))"' },
{
id: "loop",
for_each: "$vals.json",
batch_size: 2,
steps: [{ id: "emit", command: "echo hi" }],
},
],
};
const workflow = {
steps: [
{ id: "vals", command: 'node -e "process.stdout.write(JSON.stringify([1,2]))"' },
{
id: "loop",
for_each: "$vals.json",
batch_size: 2,
steps: [{ id: "emit", command: "echo hi" }],
},
],
};
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const stateDir = path.join(tmpDir, "state");
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(filePath, JSON.stringify(workflow, null, 2), "utf8");
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), "lobster-foreach-"));
const stateDir = path.join(tmpDir, "state");
const filePath = path.join(tmpDir, "workflow.lobster");
await fsp.writeFile(filePath, JSON.stringify(workflow, null, 2), "utf8");
const stderr = new PassThrough();
let out = "";
stderr.on("data", (d: Buffer | string) => {
out += String(d);
});
const stderr = new PassThrough();
let out = "";
stderr.on("data", (d: Buffer | string) => {
out += String(d);
});
await runWorkflowFile({
filePath,
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr,
env: { ...process.env, LOBSTER_STATE_DIR: stateDir },
mode: "tool",
dryRun: true,
registry: createDefaultRegistry(),
},
});
await runWorkflowFile({
filePath,
ctx: {
stdin: process.stdin,
stdout: process.stdout,
stderr,
env: { ...process.env, LOBSTER_STATE_DIR: stateDir },
mode: "tool",
dryRun: true,
registry: createDefaultRegistry(),
},
});
assert.match(out, /\[for_each\]/);
assert.match(out, /sub-steps: 1/);
assert.match(out, /batch_size: 2/);
assert.match(out, /\[for_each\]/);
assert.match(out, /sub-steps: 1/);
assert.match(out, /batch_size: 2/);
});
test/github_pr_notify_format.test.ts
+11 -11
View File
@@ -3,19 +3,19 @@ import assert from "node:assert/strict";
import { buildPrChangeSummary } from "../src/workflows/github_pr_monitor.js";
function formatLikeWorkflow({ repo, pr, after, before }) {
const summary = buildPrChangeSummary(before, after);
const fields = summary.changedFields.length ? ` (${summary.changedFields.join(", ")})` : "";
const title = after?.title ? `: ${after.title}` : "";
const url = after?.url ? ` ${after.url}` : "";
return `PR updated: ${repo}#${pr}${title}${fields}.${url}`.replace(/\s+/g, " ").trim();
const summary = buildPrChangeSummary(before, after);
const fields = summary.changedFields.length ? ` (${summary.changedFields.join(", ")})` : "";
const title = after?.title ? `: ${after.title}` : "";
const url = after?.url ? ` ${after.url}` : "";
return `PR updated: ${repo}#${pr}${title}${fields}.${url}`.replace(/\s+/g, " ").trim();
}
test("notify message includes repo/pr and changed fields", () => {
const before = { number: 1, title: "A", url: "u", state: "OPEN", updatedAt: "t1" };
const after = { ...before, title: "B", updatedAt: "t2" };
const before = { number: 1, title: "A", url: "u", state: "OPEN", updatedAt: "t1" };
const after = { ...before, title: "B", updatedAt: "t2" };
const msg = formatLikeWorkflow({ repo: "o/r", pr: 1, before, after });
assert.ok(msg.includes("o/r#1"));
assert.ok(msg.includes("title"));
assert.ok(msg.includes("updatedAt"));
const msg = formatLikeWorkflow({ repo: "o/r", pr: 1, before, after });
assert.ok(msg.includes("o/r#1"));
assert.ok(msg.includes("title"));
assert.ok(msg.includes("updatedAt"));
});
test/github_pr_summary.test.ts
+32 -32
View File
@@ -5,41 +5,41 @@ import { buildPrChangeSummary } from "../src/workflows/github_pr_monitor.js";
const build = buildPrChangeSummary as any;
test("buildPrChangeSummary reports all fields on first snapshot", () => {
const after = {
number: 1,
title: "A",
url: "u",
state: "OPEN",
isDraft: false,
mergeable: "MERGEABLE",
reviewDecision: "REVIEW_REQUIRED",
updatedAt: "t1",
baseRefName: "main",
headRefName: "feat",
};
const after = {
number: 1,
title: "A",
url: "u",
state: "OPEN",
isDraft: false,
mergeable: "MERGEABLE",
reviewDecision: "REVIEW_REQUIRED",
updatedAt: "t1",
baseRefName: "main",
headRefName: "feat",
};
const res = build(null, after);
assert.ok(res.changedFields.length > 0);
assert.equal(res.changes.title.to, "A");
const res = build(null, after);
assert.ok(res.changedFields.length > 0);
assert.equal(res.changes.title.to, "A");
});
test("buildPrChangeSummary only includes changed fields", () => {
const before = {
number: 1,
title: "A",
url: "u",
state: "OPEN",
isDraft: false,
mergeable: "MERGEABLE",
reviewDecision: null,
updatedAt: "t1",
baseRefName: "main",
headRefName: "feat",
};
const after = { ...before, title: "B", updatedAt: "t2" };
const before = {
number: 1,
title: "A",
url: "u",
state: "OPEN",
isDraft: false,
mergeable: "MERGEABLE",
reviewDecision: null,
updatedAt: "t1",
baseRefName: "main",
headRefName: "feat",
};
const after = { ...before, title: "B", updatedAt: "t2" };
const res = build(before, after);
assert.deepEqual(res.changedFields.sort(), ["title", "updatedAt"].sort());
assert.equal(res.changes.title.from, "A");
assert.equal(res.changes.title.to, "B");
const res = build(before, after);
assert.deepEqual(res.changedFields.sort(), ["title", "updatedAt"].sort());
assert.equal(res.changes.title.from, "A");
assert.equal(res.changes.title.to, "B");
});
test/group_by.test.ts
+36 -36
View File
@@ -6,46 +6,46 @@ import { createDefaultRegistry } from "../src/commands/registry.js";
import { parsePipeline } from "../src/parser.js";
async function run(pipelineText: string, input: any[]) {
const pipeline = parsePipeline(pipelineText);
const registry = createDefaultRegistry();
const res = await runPipeline({
pipeline,
registry,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
input: (async function* () {
for (const x of input) yield x;
})(),
});
return res.items;
const pipeline = parsePipeline(pipelineText);
const registry = createDefaultRegistry();
const res = await runPipeline({
pipeline,
registry,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
input: (async function* () {
for (const x of input) yield x;
})(),
});
return res.items;
}
test("groupBy groups items by key and preserves group order", async () => {
const input = [
{ from: "a", id: 1 },
{ from: "b", id: 2 },
{ from: "a", id: 3 },
];
const out = await run("groupBy --key from", input);
assert.equal(out.length, 2);
assert.deepEqual(out[0].key, "a");
assert.deepEqual(
out[0].items.map((x: any) => x.id),
[1, 3],
);
assert.equal(out[0].count, 2);
assert.deepEqual(out[1].key, "b");
const input = [
{ from: "a", id: 1 },
{ from: "b", id: 2 },
{ from: "a", id: 3 },
];
const out = await run("groupBy --key from", input);
assert.equal(out.length, 2);
assert.deepEqual(out[0].key, "a");
assert.deepEqual(
out[0].items.map((x: any) => x.id),
[1, 3],
);
assert.equal(out[0].count, 2);
assert.deepEqual(out[1].key, "b");
});
test("groupBy supports nested key paths", async () => {
const input = [{ user: { id: "u1" } }, { user: { id: "u2" } }, { user: { id: "u1" } }];
const out = await run("groupBy --key user.id", input);
assert.deepEqual(
out.map((g: any) => g.key),
["u1", "u2"],
);
assert.equal(out[0].count, 2);
const input = [{ user: { id: "u1" } }, { user: { id: "u2" } }, { user: { id: "u1" } }];
const out = await run("groupBy --key user.id", input);
assert.deepEqual(
out.map((g: any) => g.key),
["u1", "u2"],
);
assert.equal(out[0].count, 2);
});
test/llm_invoke.test.ts
+1263 -152
View File
File diff suppressed because it is too large Load Diff
test/llm_task_invoke.test.ts
+528 -402
View File
@@ -8,466 +8,592 @@ import path from "node:path";
import { createDefaultRegistry } from "../src/commands/registry.js";
function streamOf(items: any[]) {
return (async function* () {
for (const item of items) yield item;
})();
return (async function* () {
for (const item of items) yield item;
})();
}
async function collect(iterable: AsyncIterable<any>) {
const items = [];
for await (const item of iterable) items.push(item);
return items;
const items = [];
for await (const item of iterable) items.push(item);
return items;
}
test("llm_task.invoke posts to /tools/invoke (clawd) and normalizes result", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd, "llm_task.invoke should be registered");
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd, "llm_task.invoke should be registered");
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("nope");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
const parsed = JSON.parse(buf || "{}");
bodyLog.push(parsed);
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "task_1",
model: parsed.args?.model,
prompt: parsed.args?.prompt,
output: {
text: "done",
data: { summary: "hello world" },
},
usage: { inputTokens: 12, outputTokens: 2, totalTokens: 14 },
},
},
}),
);
});
});
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("nope");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
const parsed = JSON.parse(buf || "{}");
bodyLog.push(parsed);
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "task_1",
model: parsed.args?.model,
prompt: parsed.args?.prompt,
output: {
text: "done",
data: { summary: "hello world" },
},
usage: { inputTokens: 12, outputTokens: 2, totalTokens: 14 },
},
},
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
const result = await cmd.run({
input: streamOf([{ kind: "text", text: "doc" }]),
args: {
_: [],
token: "test-token",
model: "claude-3-sonnet",
prompt: "Summarize",
},
ctx: baseCtx(
{ LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` },
registry,
),
} as any);
try {
const result = await cmd.run({
input: streamOf([{ kind: "text", text: "doc" }]),
args: {
_: [],
token: "test-token",
model: "claude-3-sonnet",
prompt: "Summarize",
},
ctx: baseCtx(
{ LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` },
registry,
),
} as any);
const items = await collect(result.output!);
assert.equal(items.length, 1);
const payload = items[0];
assert.equal(payload.kind, "llm_task.invoke");
assert.equal(payload.runId, "task_1");
assert.equal(payload.output.data.summary, "hello world");
assert.equal(payload.model, "claude-3-sonnet");
assert.equal(payload.source, "clawd");
assert.equal(payload.cached, false);
assert.ok(payload.cacheKey);
const items = await collect(result.output!);
assert.equal(items.length, 1);
const payload = items[0];
assert.equal(payload.kind, "llm_task.invoke");
assert.equal(payload.runId, "task_1");
assert.equal(payload.output.data.summary, "hello world");
assert.equal(payload.model, "claude-3-sonnet");
assert.equal(payload.source, "clawd");
assert.equal(payload.cached, false);
assert.ok(payload.cacheKey);
assert.equal(bodyLog.length, 1);
assert.equal(bodyLog[0].tool, "llm-task");
assert.equal(bodyLog[0].action, "invoke");
assert.equal(bodyLog[0].args.prompt, "Summarize");
assert.equal(bodyLog[0].args.model, "claude-3-sonnet");
assert.equal(bodyLog[0].args.artifacts.length, 1);
assert.equal(bodyLog[0].args.artifactHashes.length, 1);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
assert.equal(bodyLog.length, 1);
assert.equal(bodyLog[0].tool, "llm-task");
assert.equal(bodyLog[0].action, "invoke");
assert.equal(bodyLog[0].args.prompt, "Summarize");
assert.equal(bodyLog[0].args.model, "claude-3-sonnet");
assert.equal(bodyLog[0].args.artifacts.length, 1);
assert.equal(bodyLog[0].args.artifactHashes.length, 1);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke retries when schema validation fails", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
let calls = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end();
return;
}
calls += 1;
const valid = calls >= 2;
const payload = {
ok: true,
result: {
ok: true,
result: {
runId: `attempt_${calls}`,
output: valid ? { data: { decision: "send" } } : { data: { foo: "bar" } },
},
},
};
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify(payload));
});
let calls = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end();
return;
}
calls += 1;
const valid = calls >= 2;
const payload = {
ok: true,
result: {
ok: true,
result: {
runId: `attempt_${calls}`,
output: valid ? { data: { decision: "send" } } : { data: { foo: "bar" } },
},
},
};
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify(payload));
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
const result = await cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude-3-opus",
prompt: "Decide",
"output-schema": '{"type":"object","required":["decision"]}',
"max-validation-retries": 2,
},
ctx: baseCtx(
{ LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` },
registry,
),
} as any);
try {
const result = await cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude-3-opus",
prompt: "Decide",
"output-schema": '{"type":"object","required":["decision"]}',
"max-validation-retries": 2,
},
ctx: baseCtx(
{ LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` },
registry,
),
} as any);
const items = await collect(result.output!);
assert.equal(items.length, 1);
assert.equal(items[0].runId, "attempt_2");
assert.equal(items[0].output.data.decision, "send");
assert.equal(calls, 2);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
const items = await collect(result.output!);
assert.equal(items.length, 1);
assert.equal(items[0].runId, "attempt_2");
assert.equal(items[0].output.data.decision, "send");
assert.equal(calls, 2);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke makes a single model call when --max-validation-retries is 0", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
let calls = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end();
return;
}
calls += 1;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: { runId: `attempt_${calls}`, output: { data: { foo: "bar" } } },
},
}),
);
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
await assert.rejects(
cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude-3-opus",
prompt: "Decide",
"output-schema": '{"type":"object","required":["decision"]}',
"max-validation-retries": 0,
},
ctx: baseCtx(
{ LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` },
registry,
),
} as any),
/output failed schema validation/,
);
assert.equal(calls, 1);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke retries validation exactly once by default", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end();
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (chunk) => (buf += chunk));
req.on("end", () => {
bodyLog.push(JSON.parse(buf || "{}"));
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: `attempt_${bodyLog.length}`,
output: { data: { foo: "bar" } },
},
},
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
await assert.rejects(
cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude-3-opus",
prompt: "Decide",
"output-schema": '{"type":"object","required":["decision"]}',
},
ctx: baseCtx(
{
LOBSTER_CACHE_DIR: cacheDir,
CLAWD_URL: `http://localhost:${port}`,
LOBSTER_LLM_VALIDATION_RETRIES: "",
LLM_TASK_VALIDATION_RETRIES: "",
},
registry,
),
} as any),
/output failed schema validation/,
);
assert.equal(bodyLog.length, 2);
assert.equal(bodyLog[0].args.retryContext, undefined);
assert.equal(bodyLog[1].args.retryContext.attempt, 2);
assert.ok(bodyLog[1].args.retryContext.validationErrors.length >= 1);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke persists to run state so resume skips remote call", async () => {
const stateDir = await mkdtemp(path.join(tmpdir(), "lobster-state-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const stateDir = await mkdtemp(path.join(tmpdir(), "lobster-state-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
void buf;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: { ok: true, result: { runId: "state_run", output: { data: { ok: true } } } },
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
void buf;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: { ok: true, result: { runId: "state_run", output: { data: { ok: true } } } },
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const ctxEnv = { LOBSTER_STATE_DIR: stateDir, LOBSTER_CACHE_DIR: cacheDir };
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const ctxEnv = { LOBSTER_STATE_DIR: stateDir, LOBSTER_CACHE_DIR: cacheDir };
try {
const first = await cmd.run({
input: streamOf([{ foo: "bar" }]),
args: {
_: [],
model: "claude",
prompt: "Do thing",
"state-key": "run123",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const firstItems = await collect(first.output!);
assert.equal(firstItems[0].source, "clawd");
try {
const first = await cmd.run({
input: streamOf([{ foo: "bar" }]),
args: {
_: [],
model: "claude",
prompt: "Do thing",
"state-key": "run123",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const firstItems = await collect(first.output!);
assert.equal(firstItems[0].source, "clawd");
await closeServer(server);
await closeServer(server);
const second = await cmd.run({
input: streamOf([{ foo: "bar" }]),
args: {
_: [],
model: "claude",
prompt: "Do thing",
"state-key": "run123",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const secondItems = await collect(second.output!);
assert.equal(secondItems.length, 1);
assert.equal(secondItems[0].source, "run_state");
} finally {
await rm(stateDir, { recursive: true, force: true });
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
const second = await cmd.run({
input: streamOf([{ foo: "bar" }]),
args: {
_: [],
model: "claude",
prompt: "Do thing",
"state-key": "run123",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const secondItems = await collect(second.output!);
assert.equal(secondItems.length, 1);
assert.equal(secondItems[0].source, "run_state");
} finally {
await rm(stateDir, { recursive: true, force: true });
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke reuses file cache when URL unavailable", async () => {
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
void buf;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: { ok: true, result: { runId: "cache_run", output: { text: "cached" } } },
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
void buf;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: { ok: true, result: { runId: "cache_run", output: { text: "cached" } } },
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
const ctxEnv = { LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` };
const ctxEnv = { LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` };
try {
const first = await cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude",
prompt: "Cache me",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const firstItems = await collect(first.output!);
assert.equal(firstItems[0].source, "clawd");
try {
const first = await cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude",
prompt: "Cache me",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const firstItems = await collect(first.output!);
assert.equal(firstItems[0].source, "clawd");
await closeServer(server);
await closeServer(server);
const second = await cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude",
prompt: "Cache me",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const secondItems = await collect(second.output!);
assert.equal(secondItems.length, 1);
assert.equal(secondItems[0].source, "cache");
assert.equal(secondItems[0].cached, true);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
const second = await cmd.run({
input: streamOf([]),
args: {
_: [],
model: "claude",
prompt: "Cache me",
},
ctx: baseCtx({ ...ctxEnv, CLAWD_URL: `http://localhost:${port}` }, registry),
} as any);
const secondItems = await collect(second.output!);
assert.equal(secondItems.length, 1);
assert.equal(secondItems[0].source, "cache");
assert.equal(secondItems[0].cached, true);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke treats corrupt file cache as a miss and rewrites it atomically (#111)", async () => {
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-corrupt-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-corrupt-"));
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
let calls = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
calls += 1;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: { runId: `cache_repair_${calls}`, output: { text: `fresh ${calls}` } },
},
}),
);
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
const ctxEnv = { LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` };
let calls = 0;
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
calls += 1;
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: { runId: `cache_repair_${calls}`, output: { text: `fresh ${calls}` } },
},
}),
);
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
const ctxEnv = { LOBSTER_CACHE_DIR: cacheDir, CLAWD_URL: `http://localhost:${port}` };
try {
const args = { _: [], model: "claude", prompt: "Repair corrupt cache" };
const first = await cmd.run({
input: streamOf([]),
args,
ctx: baseCtx(ctxEnv, registry),
} as any);
const firstItems = await collect(first.output!);
assert.equal(firstItems[0].runId, "cache_repair_1");
assert.equal(calls, 1);
try {
const args = { _: [], model: "claude", prompt: "Repair corrupt cache" };
const first = await cmd.run({
input: streamOf([]),
args,
ctx: baseCtx(ctxEnv, registry),
} as any);
const firstItems = await collect(first.output!);
assert.equal(firstItems[0].runId, "cache_repair_1");
assert.equal(calls, 1);
const namespaceDir = path.join(cacheDir, "llm_task.invoke");
const cacheFiles = (await readdir(namespaceDir)).filter((name) => name.endsWith(".json"));
assert.equal(cacheFiles.length, 1);
const cachePath = path.join(namespaceDir, cacheFiles[0]);
assert.equal((await stat(cachePath)).mode & 0o777, 0o600);
await writeFile(cachePath, '{"items"', "utf8");
const namespaceDir = path.join(cacheDir, "llm_task.invoke");
const cacheFiles = (await readdir(namespaceDir)).filter((name) => name.endsWith(".json"));
assert.equal(cacheFiles.length, 1);
const cachePath = path.join(namespaceDir, cacheFiles[0]);
assert.equal((await stat(cachePath)).mode & 0o777, 0o600);
await writeFile(cachePath, '{"items"', "utf8");
const second = await cmd.run({
input: streamOf([]),
args,
ctx: baseCtx(ctxEnv, registry),
} as any);
const secondItems = await collect(second.output!);
assert.equal(secondItems[0].runId, "cache_repair_2");
assert.equal(secondItems[0].source, "clawd");
assert.equal(secondItems[0].cached, false);
assert.equal(calls, 2);
const second = await cmd.run({
input: streamOf([]),
args,
ctx: baseCtx(ctxEnv, registry),
} as any);
const secondItems = await collect(second.output!);
assert.equal(secondItems[0].runId, "cache_repair_2");
assert.equal(secondItems[0].source, "clawd");
assert.equal(secondItems[0].cached, false);
assert.equal(calls, 2);
const repaired = JSON.parse(await readFile(cachePath, "utf8"));
assert.equal(repaired.items[0].runId, "cache_repair_2");
const repaired = JSON.parse(await readFile(cachePath, "utf8"));
assert.equal(repaired.items[0].runId, "cache_repair_2");
await writeFile(
cachePath,
JSON.stringify({ cacheKey: repaired.cacheKey, items: null }),
"utf8",
);
const third = await cmd.run({
input: streamOf([]),
args,
ctx: baseCtx(ctxEnv, registry),
} as any);
const thirdItems = await collect(third.output!);
assert.equal(thirdItems[0].runId, "cache_repair_3");
assert.equal(calls, 3);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
await writeFile(
cachePath,
JSON.stringify({ cacheKey: repaired.cacheKey, items: null }),
"utf8",
);
const third = await cmd.run({
input: streamOf([]),
args,
ctx: baseCtx(ctxEnv, registry),
} as any);
const thirdItems = await collect(third.output!);
assert.equal(thirdItems[0].runId, "cache_repair_3");
assert.equal(calls, 3);
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
test("llm_task.invoke uses CLAWD_URL (/tools/invoke) without requiring --url/--model", async () => {
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const registry = createDefaultRegistry();
const cmd = registry.get("llm_task.invoke");
assert.ok(cmd);
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const cacheDir = await mkdtemp(path.join(tmpdir(), "lobster-cache-"));
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
const bodyLog: any[] = [];
const server = http.createServer((req, res) => {
if (req.method !== "POST" || req.url !== "/tools/invoke") {
res.writeHead(404);
res.end("not found");
return;
}
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
const parsed = JSON.parse(buf || "{}");
bodyLog.push(parsed);
let buf = "";
req.setEncoding("utf8");
req.on("data", (d) => (buf += d));
req.on("end", () => {
const parsed = JSON.parse(buf || "{}");
bodyLog.push(parsed);
// This is the OpenClaw tool router envelope.
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "task_clawd_1",
output: { data: { hello: "world" } },
},
},
}),
);
});
});
// This is the OpenClaw tool router envelope.
res.writeHead(200, { "content-type": "application/json" });
res.end(
JSON.stringify({
ok: true,
result: {
ok: true,
result: {
runId: "task_clawd_1",
output: { data: { hello: "world" } },
},
},
}),
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
await new Promise<void>((resolve) => server.listen(0, resolve));
const addr = server.address();
const port = typeof addr === "object" && addr ? addr.port : 0;
try {
const result = await cmd.run({
input: streamOf([{ kind: "text", text: "doc" }]),
args: {
_: [],
// no url, no model
prompt: "Summarize",
refresh: true,
},
ctx: baseCtx(
{ CLAWD_URL: `http://localhost:${port}`, LOBSTER_CACHE_DIR: cacheDir },
registry,
),
} as any);
try {
const result = await cmd.run({
input: streamOf([{ kind: "text", text: "doc" }]),
args: {
_: [],
// no url, no model
prompt: "Summarize",
refresh: true,
},
ctx: baseCtx(
{ CLAWD_URL: `http://localhost:${port}`, LOBSTER_CACHE_DIR: cacheDir },
registry,
),
} as any);
const items = await collect(result.output!);
assert.equal(items.length, 1);
assert.equal(items[0].source, "clawd");
assert.equal(items[0].cached, false);
assert.equal(items[0].runId, "task_clawd_1");
assert.equal(items[0].output.data.hello, "world");
const items = await collect(result.output!);
assert.equal(items.length, 1);
assert.equal(items[0].source, "clawd");
assert.equal(items[0].cached, false);
assert.equal(items[0].runId, "task_clawd_1");
assert.equal(items[0].output.data.hello, "world");
assert.equal(bodyLog.length, 1);
assert.equal(bodyLog[0].tool, "llm-task");
assert.equal(bodyLog[0].action, "invoke");
assert.equal(bodyLog[0].args.prompt, "Summarize");
assert.ok(Array.isArray(bodyLog[0].args.artifactHashes));
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
assert.equal(bodyLog.length, 1);
assert.equal(bodyLog[0].tool, "llm-task");
assert.equal(bodyLog[0].action, "invoke");
assert.equal(bodyLog[0].args.prompt, "Summarize");
assert.ok(Array.isArray(bodyLog[0].args.artifactHashes));
} finally {
await rm(cacheDir, { recursive: true, force: true });
await closeServer(server);
}
});
function baseCtx(envOverrides: Record<string, string>, registry?) {
return {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, ...envOverrides },
registry: registry ?? null,
mode: "tool",
render: { json() {}, lines() {} },
};
return {
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: { ...process.env, ...envOverrides },
registry: registry ?? null,
mode: "tool",
render: { json() {}, lines() {} },
};
}
async function closeServer(server: http.Server) {
if (!server.listening) return;
await new Promise<void>((resolve) => server.close(() => resolve()));
if (!server.listening) return;
await new Promise<void>((resolve) => server.close(() => resolve()));
}
test/map.test.ts
+24 -24
View File
@@ -6,40 +6,40 @@ import { createDefaultRegistry } from "../src/commands/registry.js";
import { parsePipeline } from "../src/parser.js";
async function run(pipelineText: string, input: any[]) {
const pipeline = parsePipeline(pipelineText);
const registry = createDefaultRegistry();
const res = await runPipeline({
pipeline,
registry,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
input: (async function* () {
for (const x of input) yield x;
})(),
});
return res.items;
const pipeline = parsePipeline(pipelineText);
const registry = createDefaultRegistry();
const res = await runPipeline({
pipeline,
registry,
stdin: process.stdin,
stdout: process.stdout,
stderr: process.stderr,
env: process.env,
mode: "tool",
input: (async function* () {
for (const x of input) yield x;
})(),
});
return res.items;
}
test("map --wrap wraps items", async () => {
const out = await run("map --wrap item", [1, 2]);
assert.deepEqual(out, [{ item: 1 }, { item: 2 }]);
const out = await run("map --wrap item", [1, 2]);
assert.deepEqual(out, [{ item: 1 }, { item: 2 }]);
});
test("map --unwrap unwraps fields", async () => {
const out = await run("map --unwrap x", [{ x: 1 }, { x: 2 }]);
assert.deepEqual(out, [1, 2]);
const out = await run("map --unwrap x", [{ x: 1 }, { x: 2 }]);
assert.deepEqual(out, [1, 2]);
});
test("map adds fields via assignments with template values", async () => {
const out = await run("map kind=pr id={{id}}", [{ id: 123, title: "t" }]);
// assignment overwrites existing id with rendered string
assert.deepEqual(out, [{ id: "123", title: "t", kind: "pr" }]);
const out = await run("map kind=pr id={{id}}", [{ id: 123, title: "t" }]);
// assignment overwrites existing id with rendered string
assert.deepEqual(out, [{ id: "123", title: "t", kind: "pr" }]);
});
test("map converts non-object items to {value: item} when adding fields", async () => {
const out = await run("map kind=num", [5]);
assert.deepEqual(out, [{ value: 5, kind: "num" }]);
const out = await run("map kind=num", [5]);
assert.deepEqual(out, [{ value: 5, kind: "num" }]);
});

Some files were not shown because too many files have changed in this diff Show More