Compare commits

...
65 Commits
Author SHA1 Message Date
Peter Steinberger ffa25f47da chore: rebrand user-facing to OpenClaw 2026-01-30 07:02:35 +01:00
Peter Steinberger f866dc05d1 style: format moderation flags 2026-01-30 05:27:19 +01:00
Peter Steinberger 69436fd79b fix: allow moltbot parsed data 2026-01-30 05:26:09 +01:00
Peter Steinberger f185ca6f55 feat: release 0.4.0 2026-01-30 05:23:12 +01:00
Peter Steinberger f3fc8d62b6 Revert "chore: rename molthub branding"
This reverts commit 8d68b55333.
2026-01-30 05:23:12 +01:00
Shakker 71f94a774f Merge pull request #70 from moltbot/remove-clawd-authenticator-tool
chore(moderation): block ClawdAuthenticatorTool (suspected malware)
2026-01-29 17:18:09 +00:00
vignesh07 07349e6107 chore(moderation): block ClawdAuthenticatorTool listing 2026-01-29 09:14:41 -08:00
Vignesh 5ef00e8c6a chore(security): harden file endpoints CSP + XFO + svg detection (#67) 2026-01-28 23:21:16 -06:00
Jamieson O'Reillyandtheonejvo c5e5e657dd fix: add CSP headers and Content-Disposition to prevent SVG XSS (#61)
Co-authored-by: theonejvo <theonejvo@users.noreply.github.com>
2026-01-28 22:21:39 -06:00
Jamie Turner 69e1e5c507 A few fixes for search. (#64) 2026-01-28 22:21:26 -06:00
Josh Palmer 481f1b9188 Merge pull request #66 from moltbot/fix/ci-lint
fix: restore lint compliance
2026-01-28 21:27:21 +01:00
Josh Palmer d4f5832554 🤖 chore: merge main into fix/ci-lint
- resolve conflicts in search/skill publish files
- apply Biome formatting updates from main

Tests: bun run lint:biome; bun run lint:oxlint
2026-01-28 21:23:50 +01:00
Josh Palmer ca3275ba92 🤖 fix: restore lint compliance
- apply Biome formatting and import ordering across linted files
- fix management useEffect dependencies flagged by Biome

Tests: bun run lint:biome; bun run lint:oxlint
2026-01-28 21:18:19 +01:00
Shadow d577add8a0 fix: unblock convex deploy typecheck 2026-01-28 13:15:18 -06:00
Shadow 90bd065f7e fix: correct public skill entries build 2026-01-28 13:12:41 -06:00
Shadow 219f05b160 fix: sanitize public skill and soul data 2026-01-28 13:03:19 -06:00
Shadow c8091ee8a8 chore: remove unauthenticated badge backfill 2026-01-28 08:33:14 -06:00
Shadow 63f367e1d6 chore: add unauthenticated badge backfill 2026-01-28 08:31:04 -06:00
Shadow e701d7b713 feat: add skill badges table 2026-01-28 01:11:46 -06:00
Shadow 79bd1f1d6c fix: query highlighted skills by batch index 2026-01-28 00:34:33 -06:00
Shadow ac51cc0236 fix: rely on highlighted badge 2026-01-27 23:27:36 -06:00
Shadow b9c23dc00e feat: add reports dashboard for moderation 2026-01-27 22:27:20 -06:00
Shadow f4e96995bc fix: allow clawdis parsed metadata 2026-01-27 21:39:23 -06:00
Shadow 33165db598 feat: unify skill routes with owner slugs 2026-01-27 21:28:46 -06:00
Shadow 4c10e4847b feat: add moderation management and backfill 2026-01-27 21:11:37 -06:00
Shadow 8d68b55333 chore: rename molthub branding 2026-01-27 18:25:02 -06:00
Jamie Turner 251de1f540 Performance optimizations for /skills page 2026-01-27 15:18:21 -06:00
Shadow 123f60fa93 Revert "Performance optimizations for /skills page" temporarily until we can deploy
This reverts commit 643faf71c8.
2026-01-27 12:20:15 -06:00
Jamie Turner 643faf71c8 Performance optimizations for /skills page 2026-01-27 12:09:57 -06:00
Aaron NgandPeter Steinberger a2c46fbb5d Search Fixes (#30)
* more search fixes

* update tests

* comments

* fix: tune search filters and limits (#30) (thanks @aaronn)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-01-25 00:08:29 +00:00
Peter Steinberger f51e0a087d test: fix lockfile mock version 2026-01-24 22:56:39 +00:00
emilianoandPeter Steinberger d9108b0948 feat: show published skills on user profile (#20)
* fix: resolve typecheck and lint errors

* fix: stabilize publish paths and token types

* feat: show published skills on user profile

* fix: document profile published skills (#20) (thanks @njoylab)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-01-24 22:55:18 +00:00
Peter Steinberger d7a017e1c3 fix: add update lookup test (#22) (thanks @daveonkels) 2026-01-24 22:30:09 +00:00
Dave OnkelsandClaude Opus 4.5 fffdf82540 fix: use path instead of url for skill metadata API call (#22)
The `cmdUpdate` function was passing a relative path to `apiRequest`
using the `url` property, but `url` expects a full URL. When `url` is
provided, it's used as-is without combining with the registry base URL.

This caused "Failed to parse URL from /api/v1/skills/<slug>" errors
when updating skills that don't have a local fingerprint match.

Changed to use `path` property which correctly combines with the
registry base URL via `new URL(args.path, registry)`.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 22:28:54 +00:00
Ahmed Fuad MireClaude Opus 4.5vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>Ahmed
a16e624766 fix: relax search token matching to require at least one match (#27)
* fix: relax search token matching to require at least one match

The search was requiring ALL query tokens to exist in the skill's
displayName, slug, or summary. This was too strict and caused valid
results to be filtered out. For example, searching "HTTP API client"
would fail to match skills about "HTTP API" that didn't mention "client".

Changed from `.every()` to `.some()` so at least one token must match,
allowing the vector similarity to determine relevance for the rest.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update matchesExactTokens to require prefix matching for query tokens

* more inclusive token check

* Update convex/lib/searchText.ts

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

---------

Co-authored-by: Ahmed <ahmed.mire@kaluza.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2026-01-24 21:23:03 +00:00
Peter Steinberger decce1d35c fix: skip missing skills in search hydration (#28) (thanks @aaronn) 2026-01-24 21:11:46 +00:00
Aaron Ng 468832af3f fix search (#28) 2026-01-24 21:11:06 +00:00
Shadow 54c793a660 fix: handle search embedding errors 2026-01-23 15:48:53 -06:00
Shadow 5d9a89a885 fix search 2026-01-23 15:24:26 -06:00
Peter Steinberger 31e9a57678 feat: add installs/trending sorts 2026-01-19 07:06:46 +00:00
Peter Steinberger 7680cc4ce8 feat: add idempotent star endpoints 2026-01-19 03:09:27 +00:00
Peter Steinberger de56255b95 fix: normalize monaco surface color 2026-01-19 02:21:51 +00:00
Peter Steinberger eaaa5e4423 Merge pull request #12 from NACC96/fix/search-mode-navigation
fix: search mode navigation and state management
2026-01-18 23:54:17 +00:00
Peter Steinberger 49e9c3c071 fix: stabilize search mode routing (#12) (thanks @NACC96) 2026-01-18 23:53:42 +00:00
NACC96 ef96fbee84 fix: auto-focus search input when search mode activates 2026-01-18 23:51:49 +00:00
NACC96 9fc032216e fix: update Header search links to use URL params instead of /search redirect 2026-01-18 23:51:49 +00:00
NACC96 236058b1b5 fix: preserve search flag in OnlyCrabsHome URL sync 2026-01-18 23:51:49 +00:00
NACC96 595bd2cf05 fix: search mode navigation and state management
- Fix "Explore search" button causing page refresh by using URL params
- Enable /search URL deep linking via beforeLoad redirect
- Fix logo click not closing search mode by properly syncing state with URL
2026-01-18 23:51:49 +00:00
Peter Steinberger 918b5528df chore: format peer check script 2026-01-18 23:42:04 +00:00
Peter Steinberger 2f6f11af3f ci: add peer dependency check 2026-01-18 23:40:55 +00:00
Peter Steinberger c90f92dbc7 fix: align auth core with convex auth 2026-01-18 23:33:51 +00:00
Peter Steinberger 8db8c89206 chore: update deps and adjust vite convex resolution 2026-01-18 23:29:59 +00:00
Peter Steinberger 00c31dcdd4 fix: derive auth from user query 2026-01-18 23:13:29 +00:00
Peter Steinberger 47646937b6 fix: dedupe convex auth modules 2026-01-18 22:33:09 +00:00
Shadow 10d250ea32 fix: keep ConvexAuthProvider during SSR 2026-01-18 14:43:24 -06:00
Peter Steinberger e7fa7afdf7 chore: bump clawdhub to 0.2.1 2026-01-18 16:28:40 +00:00
Peter Steinberger aa97727be8 fix: harden explore limit + tests/docs (#14) (thanks @jdrhyne) 2026-01-18 16:26:28 +00:00
Peter Steinberger d375496c17 Merge pull request #14 from jdrhyne/feat/explore-command
feat(cli): add explore command to browse latest updated skills
2026-01-18 16:25:48 +00:00
Peter Steinberger 9a912ee5eb chore: update dependencies 2026-01-18 14:10:09 +00:00
Peter Steinberger 11b257a062 fix: harden search and cli http 2026-01-18 14:04:35 +00:00
Peter Steinberger 02e509404a chore: update convex api types 2026-01-18 09:12:52 +00:00
Peter Steinberger 2cf6182991 fix: tighten search matching 2026-01-18 09:11:16 +00:00
Jonathan Rhyne e108789ab1 feat(cli): add explore command to browse latest updated skills
Adds a new `clawdhub explore` command that fetches the most recently
updated skills from the registry, sorted by updatedAt descending.

Usage:
  clawdhub explore           # Show latest 25 skills
  clawdhub explore --limit 10

Output includes slug, version, relative time since update, and summary.

The API endpoint already exists and returns skills sorted by updatedAt,
this just exposes it via the CLI.
2026-01-17 23:07:54 -05:00
Peter Steinberger 18bfea5035 fix: enable explore search button 2026-01-17 21:54:29 +00:00
Peter Steinberger 5f93ddb390 fix: rename ClawdBot to Clawdbot 2026-01-16 01:14:24 +00:00
166 changed files with 7488 additions and 1813 deletions
+3 -2
View File
@@ -15,10 +15,12 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.5
bun-version: 1.3.6
- name: Install
run: bun install --frozen-lockfile
- name: Peer deps
run: bun run check:peers
- name: Lint
run: bun run lint
@@ -36,4 +38,3 @@ jobs:
- name: Build
run: bun run build
+2 -1
View File
@@ -40,5 +40,6 @@
- OAuth: GitHub OAuth App credentials required for login.
## Convex Ops (Gotchas)
- New Convex functions must be pushed before `convex run`: use `bunx convex dev --once` (dev) or `bunx convex deploy --prod` (prod).
- New Convex functions must be pushed before `convex run`: use `bunx convex dev --once` (dev) or `bunx convex deploy` (prod).
- For non-interactive prod deploys, use `bunx convex deploy -y` to skip confirmation.
- If `bunx convex run --env-file .env.local ...` returns `401 MissingAccessToken` despite `bunx convex login`, workaround: omit `--env-file` and use `--deployment-name <name>` / `--prod`.
+42 -5
View File
@@ -1,5 +1,42 @@
# Changelog
## Unreleased
### Added
### Changed
### Fixed
## 0.4.0 - 2026-01-30
### Added
- Web: show published skills on user profiles (thanks @njoylab, #20).
- CLI: include OpenClaw + Moltbot fallback skill roots for sync scans.
- CLI: support OpenClaw configuration files (`OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR`).
### Changed
- Brand: rebrand to OpenClaw and publish CLI as `clawhub` (legacy `clawdhub` supported).
- Domain: default site/registry now `https://clawhub.ai`; `.well-known/clawhub.json` preferred.
- Theme: persist theme under `clawhub-theme` (legacy key still read).
### Fixed
- Registry: drop missing skills during search hydration (thanks @aaronn, #28).
- CLI: use path-based skill metadata lookup for updates (thanks @daveonkels, #22).
- Search: keep highlighted-only filtering and clamp vector candidates to Convex limits (thanks @aaronn, #30).
## 0.3.0 - 2026-01-19
### Added
- CLI: add `explore` command for latest updates, with limit clamping + tests/docs (thanks @jdrhyne, #14).
- CLI: `explore --json` output + new sorts (`installs`, `installsAllTime`, `trending`) and limit up to 200.
- API: `/api/v1/skills` supports installs + trending sorts (7-day installs).
- API: idempotent `POST/DELETE /api/v1/stars/{slug}` endpoints.
- Registry: trending leaderboard + daily stats backfill for installs-based sorts.
### Fixed
- Web: keep search mode navigation and state in sync (thanks @NACC96, #12).
## 0.2.0 - 2026-01-13
### Added
@@ -14,7 +51,7 @@
- Registry: make SoulHub auto-seed idempotent and non-user-owned.
- Registry: keep GitHub backup state + publish backups intact (thanks @joshp123, #1).
- CLI/Registry: restore fork lineage on sync + clamp bulk list queries (thanks @joshp123, #1).
- CLI: default workdir falls back to Clawdbot workspace (override with `--workdir` / `CLAWDHUB_WORKDIR`).
- CLI: default workdir falls back to Clawdbot workspace (override with `--workdir` / `CLAWHUB_WORKDIR`).
## 0.0.6 - 2026-01-07
@@ -30,7 +67,7 @@
## 0.0.5 - 2026-01-06
### Added
- Telemetry: track installs via `clawdhub sync` (logged-in only), per root, with 120-day staleness.
- Telemetry: track installs via `clawhub sync` (logged-in only), per root, with 120-day staleness.
- Skills: show current + all-time installs; sort by installs.
- Profile: private "Installed" tab with JSON export + delete telemetry controls.
- Docs: add `docs/telemetry.md` (what we track + how to opt out).
@@ -38,7 +75,7 @@
- Web: dashboard for managing your published skills (thanks @dbhurley!).
### Changed
- CLI: telemetry opt-out via `CLAWDHUB_DISABLE_TELEMETRY=1`.
- CLI: telemetry opt-out via `CLAWHUB_DISABLE_TELEMETRY=1`.
- Web: move theme picker into mobile menu.
### Fixed
@@ -86,8 +123,8 @@
### Fixed
- CLI sync: wrap note output to avoid terminal overflow; cap list lengths.
- CLI sync: label fallback scans as fallback locations.
- CLI package: bundle schema internally (no external `clawdhub-schema` publish).
- Repo: mark `clawdhub-schema` as private to prevent publishing.
- CLI package: bundle schema internally (no external `clawhub-schema` publish).
- Repo: mark `clawhub-schema` as private to prevent publishing.
## 0.0.2 - 2026-01-04
+9 -9
View File
@@ -1,17 +1,17 @@
# ClawdHub
# OpenClaw
<p align="center">
<a href="https://github.com/clawdbot/clawdhub/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/clawdbot/clawdhub/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://github.com/openclaw/clawhub/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/openclaw/clawhub/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://discord.gg/clawd"><img src="https://img.shields.io/discord/1456350064065904867?label=Discord&logo=discord&logoColor=white&color=5865F2&style=for-the-badge" alt="Discord"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
</p>
ClawdHub is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
OpenClaw is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
Its designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
onlycrabs.ai is the **SOUL.md registry**: publish and share system lore the same way you publish skills.
Live: `https://clawdhub.com`
Live: `https://clawhub.ai`
onlycrabs.ai: `https://onlycrabs.ai`
## What you can do
@@ -27,7 +27,7 @@ onlycrabs.ai: `https://onlycrabs.ai`
- Entry point is host-based: `onlycrabs.ai`.
- On the onlycrabs.ai host, the home page and nav default to souls.
- On ClawdHub, souls live under `/souls`.
- On OpenClaw, souls live under `/souls`.
- Soul bundles only accept `SOUL.md` for now (no extra files).
## How it works (high level)
@@ -35,15 +35,15 @@ onlycrabs.ai: `https://onlycrabs.ai`
- Web app: TanStack Start (React, Vite/Nitro).
- Backend: Convex (DB + file storage + HTTP actions) + Convex Auth (GitHub OAuth).
- Search: OpenAI embeddings (`text-embedding-3-small`) + Convex vector search.
- API schema + routes: `packages/schema` (`clawdhub-schema`).
- API schema + routes: `packages/schema` (`clawhub-schema`).
## Telemetry
ClawdHub tracks minimal **install telemetry** (to compute install counts) when you run `clawdhub sync` while logged in.
OpenClaw tracks minimal **install telemetry** (to compute install counts) when you run `clawhub sync` while logged in.
Disable via:
```bash
export CLAWDHUB_DISABLE_TELEMETRY=1
export CLAWHUB_DISABLE_TELEMETRY=1
```
Details: `docs/telemetry.md`.
@@ -95,7 +95,7 @@ This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints values for y
## Nix plugins (nixmode skills)
ClawdHub can store a nix-clawdbot plugin pointer in SKILL frontmatter so the registry knows which
OpenClaw can store a nix-clawdbot plugin pointer in SKILL frontmatter so the registry knows which
Nix package bundle to install. A nix plugin is different from a regular skill pack: it bundles the
skill pack, the CLI binary, and its config flags/requirements together.
+1 -1
View File
@@ -1,5 +1,5 @@
{
"$schema": "https://biomejs.dev/schemas/2.3.11/schema.json",
"$schema": "https://biomejs.dev/schemas/2.3.13/schema.json",
"files": {
"includes": [
"**",
+275 -241
View File
File diff suppressed because it is too large Load Diff
Executable
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bun
import { existsSync } from 'node:fs'
import { stat } from 'node:fs/promises'
import { fileURLToPath } from 'node:url'
const distCliUrl = new URL('./packages/clawdhub/dist/cli.js', import.meta.url)
const distCliPath = fileURLToPath(distCliUrl)
const srcRootPath = fileURLToPath(new URL('./packages/clawdhub/src/', import.meta.url))
const shouldBuild = await (async () => {
if (!existsSync(distCliPath)) return true
try {
const dist = await stat(distCliPath)
const latestSrcMtime = await getLatestMtime(srcRootPath)
return latestSrcMtime > dist.mtimeMs
} catch {
return true
}
})()
if (shouldBuild) {
const proc = Bun.spawn(['bunx', 'tsc', '-p', 'packages/clawdhub/tsconfig.json'], {
stdin: 'inherit',
stdout: 'inherit',
stderr: 'inherit',
})
const code = await proc.exited
if (code !== 0) process.exit(code)
}
await import(distCliUrl.href)
async function getLatestMtime(root: string) {
let latest = 0
const glob = new Bun.Glob('**/*.ts')
for await (const rel of glob.scan({ cwd: root, onlyFiles: true })) {
const path = `${root}${root.endsWith('/') ? '' : '/'}${rel}`
try {
const entry = await stat(path)
latest = Math.max(latest, entry.mtimeMs)
} catch {
// ignore
}
}
return latest
}
+20
View File
@@ -12,6 +12,7 @@ import type * as auth from "../auth.js";
import type * as comments from "../comments.js";
import type * as crons from "../crons.js";
import type * as devSeed from "../devSeed.js";
import type * as devSeedExtra from "../devSeedExtra.js";
import type * as downloads from "../downloads.js";
import type * as githubBackups from "../githubBackups.js";
import type * as githubBackupsNode from "../githubBackupsNode.js";
@@ -21,15 +22,22 @@ import type * as githubSoulBackupsNode from "../githubSoulBackupsNode.js";
import type * as http from "../http.js";
import type * as httpApi from "../httpApi.js";
import type * as httpApiV1 from "../httpApiV1.js";
import type * as leaderboards from "../leaderboards.js";
import type * as lib_access from "../lib/access.js";
import type * as lib_apiTokenAuth from "../lib/apiTokenAuth.js";
import type * as lib_badges from "../lib/badges.js";
import type * as lib_changelog from "../lib/changelog.js";
import type * as lib_embeddings from "../lib/embeddings.js";
import type * as lib_githubBackup from "../lib/githubBackup.js";
import type * as lib_githubImport from "../lib/githubImport.js";
import type * as lib_githubSoulBackup from "../lib/githubSoulBackup.js";
import type * as lib_leaderboards from "../lib/leaderboards.js";
import type * as lib_moderation from "../lib/moderation.js";
import type * as lib_public from "../lib/public.js";
import type * as lib_searchText from "../lib/searchText.js";
import type * as lib_skillBackfill from "../lib/skillBackfill.js";
import type * as lib_skillPublish from "../lib/skillPublish.js";
import type * as lib_skillStats from "../lib/skillStats.js";
import type * as lib_skills from "../lib/skills.js";
import type * as lib_soulChangelog from "../lib/soulChangelog.js";
import type * as lib_soulPublish from "../lib/soulPublish.js";
@@ -40,12 +48,14 @@ import type * as rateLimits from "../rateLimits.js";
import type * as search from "../search.js";
import type * as seed from "../seed.js";
import type * as seedSouls from "../seedSouls.js";
import type * as skillStatEvents from "../skillStatEvents.js";
import type * as skills from "../skills.js";
import type * as soulComments from "../soulComments.js";
import type * as soulDownloads from "../soulDownloads.js";
import type * as soulStars from "../soulStars.js";
import type * as souls from "../souls.js";
import type * as stars from "../stars.js";
import type * as statsMaintenance from "../statsMaintenance.js";
import type * as telemetry from "../telemetry.js";
import type * as tokens from "../tokens.js";
import type * as uploads from "../uploads.js";
@@ -63,6 +73,7 @@ declare const fullApi: ApiFromModules<{
comments: typeof comments;
crons: typeof crons;
devSeed: typeof devSeed;
devSeedExtra: typeof devSeedExtra;
downloads: typeof downloads;
githubBackups: typeof githubBackups;
githubBackupsNode: typeof githubBackupsNode;
@@ -72,15 +83,22 @@ declare const fullApi: ApiFromModules<{
http: typeof http;
httpApi: typeof httpApi;
httpApiV1: typeof httpApiV1;
leaderboards: typeof leaderboards;
"lib/access": typeof lib_access;
"lib/apiTokenAuth": typeof lib_apiTokenAuth;
"lib/badges": typeof lib_badges;
"lib/changelog": typeof lib_changelog;
"lib/embeddings": typeof lib_embeddings;
"lib/githubBackup": typeof lib_githubBackup;
"lib/githubImport": typeof lib_githubImport;
"lib/githubSoulBackup": typeof lib_githubSoulBackup;
"lib/leaderboards": typeof lib_leaderboards;
"lib/moderation": typeof lib_moderation;
"lib/public": typeof lib_public;
"lib/searchText": typeof lib_searchText;
"lib/skillBackfill": typeof lib_skillBackfill;
"lib/skillPublish": typeof lib_skillPublish;
"lib/skillStats": typeof lib_skillStats;
"lib/skills": typeof lib_skills;
"lib/soulChangelog": typeof lib_soulChangelog;
"lib/soulPublish": typeof lib_soulPublish;
@@ -91,12 +109,14 @@ declare const fullApi: ApiFromModules<{
search: typeof search;
seed: typeof seed;
seedSouls: typeof seedSouls;
skillStatEvents: typeof skillStatEvents;
skills: typeof skills;
soulComments: typeof soulComments;
soulDownloads: typeof soulDownloads;
soulStars: typeof soulStars;
souls: typeof souls;
stars: typeof stars;
statsMaintenance: typeof statsMaintenance;
telemetry: typeof telemetry;
tokens: typeof tokens;
uploads: typeof uploads;
+5 -4
View File
@@ -1,7 +1,8 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { assertRole, requireUser } from './lib/access'
import { assertModerator, requireUser } from './lib/access'
import { type PublicUser, toPublicUser } from './lib/public'
export const listBySkill = query({
args: { skillId: v.id('skills'), limit: v.optional(v.number()) },
@@ -13,10 +14,10 @@ export const listBySkill = query({
.order('desc')
.take(limit)
const results: Array<{ comment: Doc<'comments'>; user: Doc<'users'> | null }> = []
const results: Array<{ comment: Doc<'comments'>; user: PublicUser | null }> = []
for (const comment of comments) {
if (comment.softDeletedAt) continue
const user = await ctx.db.get(comment.userId)
const user = toPublicUser(await ctx.db.get(comment.userId))
results.push({ comment, user })
}
return results
@@ -59,7 +60,7 @@ export const remove = mutation({
const isOwner = comment.userId === user._id
if (!isOwner) {
assertRole(user, ['admin', 'moderator'])
assertModerator(user)
}
await ctx.db.patch(comment._id, {
+21
View File
@@ -10,4 +10,25 @@ crons.interval(
{ batchSize: 50, maxBatches: 5 },
)
crons.interval(
'trending-leaderboard',
{ minutes: 60 },
internal.leaderboards.rebuildTrendingLeaderboardInternal,
{ limit: 200 },
)
crons.interval(
'skill-stats-backfill',
{ minutes: 10 },
internal.statsMaintenance.runSkillStatBackfillInternal,
{ batchSize: 200, maxBatches: 5 },
)
crons.interval(
'skill-stat-events',
{ minutes: 15 },
internal.skillStatEvents.processSkillStatEventsAction,
{},
)
export default crons
+71 -41
View File
@@ -1,5 +1,6 @@
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { ActionCtx } from './_generated/server'
import { internalAction, internalMutation } from './_generated/server'
import { EMBEDDING_DIMENSIONS } from './lib/embeddings'
import { parseClawdisMetadata, parseFrontmatter } from './lib/skills'
@@ -13,6 +14,17 @@ type SeedSkillSpec = {
rawSkillMd: string
}
type SeedActionArgs = {
reset?: boolean
}
type SeedActionResult = {
ok: true
results: Array<Record<string, unknown> & { slug: string }>
}
type SeedMutationResult = Record<string, unknown>
const SEED_SKILLS: SeedSkillSpec[] = [
{
slug: 'padel',
@@ -237,53 +249,19 @@ function injectMetadata(rawSkillMd: string, metadata: Record<string, unknown>) {
)}${rawSkillMd.slice(frontmatterEnd)}`
}
export const seedNixSkills = internalAction({
args: {
reset: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const results = []
for (const spec of SEED_SKILLS) {
const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
const frontmatter = parseFrontmatter(skillMd)
const clawdis = parseClawdisMetadata(frontmatter)
const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
const result = await ctx.runMutation(internal.devSeed.seedSkillMutation, {
reset: args.reset,
storageId,
metadata: spec.metadata,
frontmatter,
clawdis,
skillMd,
slug: spec.slug,
displayName: spec.displayName,
summary: spec.summary,
version: spec.version,
})
results.push({ slug: spec.slug, ...result })
}
return { ok: true, results }
},
})
export const seedPadelSkill = internalAction({
args: {
reset: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const spec = SEED_SKILLS.find((entry) => entry.slug === 'padel')
if (!spec) throw new Error('padel seed spec missing')
async function seedNixSkillsHandler(
ctx: ActionCtx,
args: SeedActionArgs,
): Promise<SeedActionResult> {
const results: Array<Record<string, unknown> & { slug: string }> = []
for (const spec of SEED_SKILLS) {
const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
const frontmatter = parseFrontmatter(skillMd)
const clawdis = parseClawdisMetadata(frontmatter)
const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
return ctx.runMutation(internal.devSeed.seedSkillMutation, {
const result: SeedMutationResult = await ctx.runMutation(internal.devSeed.seedSkillMutation, {
reset: args.reset,
storageId,
metadata: spec.metadata,
@@ -295,7 +273,51 @@ export const seedPadelSkill = internalAction({
summary: spec.summary,
version: spec.version,
})
results.push({ slug: spec.slug, ...result })
}
return { ok: true, results }
}
export const seedNixSkills: ReturnType<typeof internalAction> = internalAction({
args: {
reset: v.optional(v.boolean()),
},
handler: seedNixSkillsHandler,
})
async function seedPadelSkillHandler(
ctx: ActionCtx,
args: SeedActionArgs,
): Promise<SeedMutationResult> {
const spec = SEED_SKILLS.find((entry) => entry.slug === 'padel')
if (!spec) throw new Error('padel seed spec missing')
const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
const frontmatter = parseFrontmatter(skillMd)
const clawdis = parseClawdisMetadata(frontmatter)
const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
return (await ctx.runMutation(internal.devSeed.seedSkillMutation, {
reset: args.reset,
storageId,
metadata: spec.metadata,
frontmatter,
clawdis,
skillMd,
slug: spec.slug,
displayName: spec.displayName,
summary: spec.summary,
version: spec.version,
})) as SeedMutationResult
}
export const seedPadelSkill: ReturnType<typeof internalAction> = internalAction({
args: {
reset: v.optional(v.boolean()),
},
handler: seedPadelSkillHandler,
})
export const seedSkillMutation = internalMutation({
@@ -364,6 +386,10 @@ export const seedSkillMutation = internalMutation({
tags: {},
softDeletedAt: undefined,
badges: { redactionApproved: undefined },
statsDownloads: 0,
statsStars: 0,
statsInstallsCurrent: 0,
statsInstallsAllTime: 0,
stats: {
downloads: 0,
installsCurrent: 0,
@@ -413,6 +439,10 @@ export const seedSkillMutation = internalMutation({
await ctx.db.patch(skillId, {
latestVersionId: versionId,
tags: { latest: versionId },
statsDownloads: 0,
statsStars: 0,
statsInstallsCurrent: 0,
statsInstallsAllTime: 0,
stats: {
downloads: 0,
installsCurrent: 0,
+541
View File
@@ -0,0 +1,541 @@
/**
* Extra seed skills for pagination testing.
*
* This file contains 50 placeholder skills to test pagination behavior.
* Run with: bunx convex run internal.devSeedExtra.seedExtraSkillsInternal
* Or with reset: bunx convex run internal.devSeedExtra.seedExtraSkillsInternal '{"reset": true}'
*/
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Id } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
import { internalAction, internalMutation } from './_generated/server'
import { parseClawdisMetadata, parseFrontmatter } from './lib/skills'
type SeedSkillSpec = {
slug: string
displayName: string
summary: string
version: string
metadata: Record<string, unknown>
rawSkillMd: string
}
function makeSkill(
slug: string,
displayName: string,
summary: string,
envVars: string[] = [],
commands: string[] = ['help', 'status', 'run'],
): SeedSkillSpec {
const cliHelp = `${slug} - ${summary}
Usage:
${slug} [command]
Commands:
${commands.map((cmd) => ` ${cmd.padEnd(12)} Run ${cmd} operation`).join('\n')}
Flags:
-h, --help Show help
--json Output as JSON
`
const rawSkillMd = `---
name: ${slug}
description: ${summary}
---
# ${displayName}
## CLI
\`\`\`bash
${commands.map((cmd) => `${slug} ${cmd}`).join('\n')}
\`\`\`
## Usage
Use this skill to ${summary.toLowerCase()}.
`
return {
slug,
displayName,
summary,
version: '0.1.0',
metadata: {
clawdbot: {
nix: {
plugin: `github:example/${slug}`,
systems: ['aarch64-darwin', 'x86_64-linux'],
},
config: {
requiredEnv: envVars,
},
cliHelp,
},
},
rawSkillMd,
}
}
// 50 placeholder skills for pagination testing
const EXTRA_SEED_SKILLS: SeedSkillSpec[] = [
// DevOps & Infrastructure (10)
makeSkill(
'kubectl-helper',
'Kubectl Helper',
'Simplified kubectl commands for common Kubernetes operations.',
['KUBECONFIG'],
['pods', 'logs', 'exec', 'describe', 'apply'],
),
makeSkill(
'terraform-runner',
'Terraform Runner',
'Execute Terraform plans and applies with safety checks.',
['TF_VAR_region', 'AWS_PROFILE'],
['plan', 'apply', 'destroy', 'output', 'state'],
),
makeSkill(
'ansible-exec',
'Ansible Exec',
'Run Ansible playbooks and ad-hoc commands.',
['ANSIBLE_INVENTORY'],
['playbook', 'adhoc', 'inventory', 'facts', 'vault'],
),
makeSkill(
'docker-compose-mgr',
'Docker Compose Manager',
'Manage Docker Compose stacks and services.',
['DOCKER_HOST'],
['up', 'down', 'logs', 'ps', 'restart'],
),
makeSkill(
'k9s-wrapper',
'K9s Wrapper',
'Interactive Kubernetes cluster management via K9s.',
['KUBECONFIG'],
['launch', 'contexts', 'namespaces', 'pods', 'logs'],
),
makeSkill(
'helm-charts',
'Helm Charts',
'Manage Helm chart deployments and releases.',
['KUBECONFIG', 'HELM_REPO'],
['install', 'upgrade', 'rollback', 'list', 'search'],
),
makeSkill(
'prometheus-alerts',
'Prometheus Alerts',
'Query Prometheus metrics and manage alerting rules.',
['PROMETHEUS_URL'],
['query', 'alerts', 'rules', 'targets', 'status'],
),
makeSkill(
'grafana-dash',
'Grafana Dashboards',
'Create and manage Grafana dashboards programmatically.',
['GRAFANA_URL', 'GRAFANA_API_KEY'],
['list', 'export', 'import', 'create', 'delete'],
),
makeSkill(
'nginx-config',
'Nginx Config',
'Generate and validate Nginx configuration files.',
['NGINX_CONF_DIR'],
['generate', 'validate', 'reload', 'test', 'sites'],
),
makeSkill(
'jenkins-jobs',
'Jenkins Jobs',
'Manage Jenkins jobs and pipelines.',
['JENKINS_URL', 'JENKINS_TOKEN'],
['list', 'build', 'status', 'logs', 'config'],
),
// Productivity (8)
makeSkill(
'todoist-sync',
'Todoist Sync',
'Sync and manage Todoist tasks from the command line.',
['TODOIST_API_TOKEN'],
['list', 'add', 'complete', 'projects', 'labels'],
),
makeSkill(
'notion-backup',
'Notion Backup',
'Export and backup Notion workspaces.',
['NOTION_TOKEN'],
['export', 'backup', 'restore', 'pages', 'databases'],
),
makeSkill(
'gcal-manager',
'Google Calendar Manager',
'Manage Google Calendar events and schedules.',
['GOOGLE_CREDENTIALS_FILE'],
['events', 'create', 'delete', 'calendars', 'reminders'],
),
makeSkill(
'time-tracker',
'Time Tracker',
'Track time spent on projects and tasks.',
['TIMETRACK_DB'],
['start', 'stop', 'status', 'report', 'projects'],
),
makeSkill(
'email-digest',
'Email Digest',
'Generate email digests and summaries.',
['IMAP_SERVER', 'IMAP_USER'],
['fetch', 'digest', 'search', 'folders', 'unread'],
),
makeSkill(
'habit-tracker',
'Habit Tracker',
'Track daily habits and streaks.',
['HABITS_DB'],
['log', 'streak', 'stats', 'habits', 'remind'],
),
makeSkill(
'bookmark-sync',
'Bookmark Sync',
'Sync bookmarks across browsers and devices.',
['BOOKMARKS_DIR'],
['sync', 'export', 'import', 'search', 'tags'],
),
makeSkill(
'notes-export',
'Notes Export',
'Export notes to various formats.',
['NOTES_DIR'],
['export', 'convert', 'search', 'list', 'tags'],
),
// Media & Entertainment (6)
makeSkill(
'spotify-ctl',
'Spotify Control',
'Control Spotify playback from the terminal.',
['SPOTIFY_CLIENT_ID', 'SPOTIFY_CLIENT_SECRET'],
['play', 'pause', 'next', 'prev', 'search'],
),
makeSkill(
'plex-manager',
'Plex Manager',
'Manage Plex media libraries and playback.',
['PLEX_URL', 'PLEX_TOKEN'],
['libraries', 'scan', 'search', 'play', 'sessions'],
),
makeSkill(
'ytdl-wrapper',
'YouTube Downloader',
'Download videos from YouTube and other platforms.',
['YTDL_OUTPUT_DIR'],
['download', 'info', 'playlist', 'audio', 'formats'],
),
makeSkill(
'podcast-dl',
'Podcast Downloader',
'Download and manage podcast episodes.',
['PODCAST_DIR'],
['subscribe', 'download', 'list', 'play', 'search'],
),
makeSkill(
'audiobook-player',
'Audiobook Player',
'Manage and play audiobook collections.',
['AUDIOBOOK_DIR'],
['play', 'pause', 'bookmark', 'list', 'progress'],
),
makeSkill(
'music-lib',
'Music Library',
'Organize and query local music libraries.',
['MUSIC_DIR'],
['scan', 'search', 'play', 'playlist', 'stats'],
),
// Smart Home (8)
makeSkill(
'hass-control',
'Home Assistant Control',
'Control Home Assistant entities and automations.',
['HASS_URL', 'HASS_TOKEN'],
['entities', 'services', 'automations', 'scenes', 'history'],
),
makeSkill(
'zigbee-mqtt',
'Zigbee2MQTT',
'Manage Zigbee devices via MQTT.',
['MQTT_BROKER', 'ZIGBEE_TOPIC'],
['devices', 'pair', 'remove', 'rename', 'groups'],
),
makeSkill(
'tasmota-ctl',
'Tasmota Control',
'Control Tasmota-flashed devices.',
['TASMOTA_HOSTS'],
['status', 'power', 'config', 'update', 'backup'],
),
makeSkill(
'esphome-mgr',
'ESPHome Manager',
'Manage ESPHome device configurations.',
['ESPHOME_DIR'],
['compile', 'upload', 'logs', 'dashboard', 'config'],
),
makeSkill(
'mqtt-broker',
'MQTT Broker',
'Interact with MQTT brokers for IoT messaging.',
['MQTT_BROKER', 'MQTT_USER'],
['pub', 'sub', 'topics', 'clients', 'stats'],
),
makeSkill(
'hue-lights',
'Philips Hue',
'Control Philips Hue lights and scenes.',
['HUE_BRIDGE_IP', 'HUE_API_KEY'],
['lights', 'scenes', 'groups', 'schedules', 'sensors'],
),
makeSkill(
'smart-thermo',
'Smart Thermostat',
'Control smart thermostats and HVAC systems.',
['THERMOSTAT_API_KEY'],
['status', 'set', 'schedule', 'history', 'zones'],
),
makeSkill(
'cam-viewer',
'Camera Viewer',
'View and manage security camera feeds.',
['CAMERA_URLS'],
['list', 'snapshot', 'stream', 'record', 'events'],
),
// Finance (5)
makeSkill(
'budget-track',
'Budget Tracker',
'Track budgets and spending across categories.',
['BUDGET_DB'],
['summary', 'add', 'categories', 'report', 'goals'],
),
makeSkill(
'crypto-watch',
'Crypto Watcher',
'Monitor cryptocurrency prices and portfolios.',
['CRYPTO_API_KEY'],
['prices', 'portfolio', 'alerts', 'history', 'convert'],
),
makeSkill(
'stock-alerts',
'Stock Alerts',
'Set up stock price alerts and notifications.',
['STOCK_API_KEY'],
['quote', 'watch', 'alerts', 'portfolio', 'news'],
),
makeSkill(
'expense-cat',
'Expense Categorizer',
'Automatically categorize expenses.',
['EXPENSE_DB'],
['import', 'categorize', 'report', 'rules', 'export'],
),
makeSkill(
'invoice-gen',
'Invoice Generator',
'Generate and manage invoices.',
['INVOICE_DIR', 'COMPANY_INFO'],
['create', 'list', 'send', 'paid', 'overdue'],
),
// Communication (5)
makeSkill(
'slack-bot',
'Slack Bot',
'Interact with Slack channels and messages.',
['SLACK_TOKEN'],
['send', 'channels', 'users', 'search', 'files'],
),
makeSkill(
'discord-mgr',
'Discord Manager',
'Manage Discord servers and messages.',
['DISCORD_TOKEN'],
['send', 'servers', 'channels', 'members', 'roles'],
),
makeSkill(
'telegram-bot',
'Telegram Bot',
'Send and receive Telegram messages.',
['TELEGRAM_BOT_TOKEN'],
['send', 'receive', 'chats', 'files', 'inline'],
),
makeSkill(
'matrix-cli',
'Matrix CLI',
'Interact with Matrix chat rooms.',
['MATRIX_HOMESERVER', 'MATRIX_TOKEN'],
['send', 'rooms', 'join', 'leave', 'sync'],
),
makeSkill(
'irc-bridge',
'IRC Bridge',
'Bridge IRC channels to other platforms.',
['IRC_SERVER', 'IRC_NICK'],
['connect', 'join', 'send', 'channels', 'users'],
),
// Data & Analytics (5)
makeSkill(
'pg-queries',
'PostgreSQL Queries',
'Execute PostgreSQL queries and manage databases.',
['DATABASE_URL'],
['query', 'tables', 'schema', 'backup', 'restore'],
),
makeSkill(
'clickhouse-ql',
'ClickHouse Queries',
'Run ClickHouse analytics queries.',
['CLICKHOUSE_URL'],
['query', 'tables', 'insert', 'system', 'optimize'],
),
makeSkill(
'redis-cli',
'Redis CLI',
'Interact with Redis cache and data structures.',
['REDIS_URL'],
['get', 'set', 'keys', 'info', 'flush'],
),
makeSkill(
'elastic-search',
'Elasticsearch',
'Search and manage Elasticsearch indices.',
['ELASTICSEARCH_URL'],
['search', 'index', 'mapping', 'cluster', 'aliases'],
),
makeSkill(
'mongo-shell',
'MongoDB Shell',
'Query and manage MongoDB collections.',
['MONGODB_URI'],
['find', 'insert', 'update', 'delete', 'aggregate'],
),
// Security (3)
makeSkill(
'vault-secrets',
'Vault Secrets',
'Manage secrets in HashiCorp Vault.',
['VAULT_ADDR', 'VAULT_TOKEN'],
['read', 'write', 'list', 'delete', 'seal'],
),
makeSkill(
'gpg-keys',
'GPG Keys',
'Manage GPG keys and encryption.',
['GNUPGHOME'],
['list', 'generate', 'export', 'import', 'encrypt'],
),
makeSkill(
'ssh-rotate',
'SSH Key Rotator',
'Rotate and manage SSH keys.',
['SSH_KEY_DIR'],
['generate', 'rotate', 'deploy', 'list', 'revoke'],
),
]
function injectMetadata(rawSkillMd: string, metadata: Record<string, unknown>) {
const frontmatterEnd = rawSkillMd.indexOf('\n---', 3)
if (frontmatterEnd === -1) return rawSkillMd
return `${rawSkillMd.slice(0, frontmatterEnd)}\nmetadata: ${JSON.stringify(
metadata,
)}${rawSkillMd.slice(frontmatterEnd)}`
}
function randomStats() {
return {
downloads: Math.floor(Math.random() * 5000),
stars: Math.floor(Math.random() * 500),
installsCurrent: Math.floor(Math.random() * 200),
installsAllTime: Math.floor(Math.random() * 1000),
}
}
export const applyRandomStats = internalMutation({
args: {
skillId: v.id('skills'),
stats: v.object({
downloads: v.number(),
stars: v.number(),
installsCurrent: v.number(),
installsAllTime: v.number(),
}),
},
handler: async (ctx, args) => {
await ctx.db.patch(args.skillId, {
statsDownloads: args.stats.downloads,
statsStars: args.stats.stars,
statsInstallsCurrent: args.stats.installsCurrent,
statsInstallsAllTime: args.stats.installsAllTime,
stats: {
downloads: args.stats.downloads,
stars: args.stats.stars,
installsCurrent: args.stats.installsCurrent,
installsAllTime: args.stats.installsAllTime,
versions: 1,
comments: 0,
},
})
},
})
export const seedExtraSkillsInternal = internalAction({
args: {
reset: v.optional(v.boolean()),
},
handler: async (ctx: ActionCtx, args) => {
const results: Array<{ slug: string; ok: boolean; skipped?: boolean }> = []
for (const spec of EXTRA_SEED_SKILLS) {
const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
const frontmatter = parseFrontmatter(skillMd)
const clawdis = parseClawdisMetadata(frontmatter)
const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
const result = (await ctx.runMutation(internal.devSeed.seedSkillMutation, {
reset: args.reset,
storageId,
metadata: spec.metadata,
frontmatter,
clawdis,
skillMd,
slug: spec.slug,
displayName: spec.displayName,
summary: spec.summary,
version: spec.version,
})) as { ok: boolean; skipped?: boolean; skillId?: string }
// Apply random stats after creation (only if not skipped)
if (result.skillId && !result.skipped) {
const stats = randomStats()
await ctx.runMutation(internal.devSeedExtra.applyRandomStats, {
skillId: result.skillId as Id<'skills'>,
stats,
})
}
results.push({ slug: spec.slug, ok: result.ok, skipped: result.skipped })
}
const created = results.filter((r) => !r.skipped).length
const skipped = results.filter((r) => r.skipped).length
return { ok: true, total: results.length, created, skipped }
},
})
+4 -3
View File
@@ -2,6 +2,7 @@ import { v } from 'convex/values'
import { zipSync } from 'fflate'
import { api } from './_generated/api'
import { httpAction, mutation } from './_generated/server'
import { insertStatEvent } from './skillStatEvents'
export const downloadZip = httpAction(async (ctx, request) => {
const url = new URL(request.url)
@@ -69,9 +70,9 @@ export const increment = mutation({
handler: async (ctx, args) => {
const skill = await ctx.db.get(args.skillId)
if (!skill) return
await ctx.db.patch(skill._id, {
stats: { ...skill.stats, downloads: skill.stats.downloads + 1 },
updatedAt: Date.now(),
await insertStatEvent(ctx, {
skillId: skill._id,
kind: 'download',
})
},
})
+15 -1
View File
@@ -1,4 +1,4 @@
import { ApiRoutes, LegacyApiRoutes } from 'clawdhub-schema'
import { ApiRoutes, LegacyApiRoutes } from 'clawhub-schema'
import { httpRouter } from 'convex/server'
import { auth } from './auth'
import { downloadZip } from './downloads'
@@ -26,6 +26,8 @@ import {
soulsDeleteRouterV1Http,
soulsGetRouterV1Http,
soulsPostRouterV1Http,
starsDeleteRouterV1Http,
starsPostRouterV1Http,
whoamiV1Http,
} from './httpApiV1'
@@ -81,6 +83,18 @@ http.route({
handler: skillsDeleteRouterV1Http,
})
http.route({
pathPrefix: `${ApiRoutes.stars}/`,
method: 'POST',
handler: starsPostRouterV1Http,
})
http.route({
pathPrefix: `${ApiRoutes.stars}/`,
method: 'DELETE',
handler: starsDeleteRouterV1Http,
})
http.route({
path: ApiRoutes.whoami,
method: 'GET',
+22 -7
View File
@@ -11,7 +11,7 @@ vi.mock('./skills', () => ({
const { requireApiTokenUser } = await import('./lib/apiTokenAuth')
const { publishVersionForUser } = await import('./skills')
const { __handlers, cliSkillDeleteHttp, cliSkillUndeleteHttp } = await import('./httpApi')
const { __handlers } = await import('./httpApi')
const { hashSkillFiles } = await import('./lib/skills')
function makeCtx(partial: Record<string, unknown>) {
@@ -33,7 +33,7 @@ describe('httpApi handlers', () => {
expect(await response.json()).toEqual({ results: [] })
})
it('searchSkillsHttp forwards args', async () => {
it('searchSkillsHttp forwards args (approvedOnly alias)', async () => {
const runAction = vi.fn().mockResolvedValue([
{
score: 1,
@@ -48,14 +48,27 @@ describe('httpApi handlers', () => {
expect(runAction).toHaveBeenCalledWith(expect.anything(), {
query: 'test',
limit: 5,
approvedOnly: true,
highlightedOnly: true,
})
expect(response.status).toBe(200)
const json = await response.json()
expect(json.results[0].slug).toBe('a')
})
it('searchSkillsHttp omits approvedOnly when false', async () => {
it('searchSkillsHttp forwards highlightedOnly', async () => {
const runAction = vi.fn().mockResolvedValue([])
await __handlers.searchSkillsHandler(
makeCtx({ runAction }),
new Request('https://example.com/api/search?q=test&highlightedOnly=true'),
)
expect(runAction).toHaveBeenCalledWith(expect.anything(), {
query: 'test',
limit: undefined,
highlightedOnly: true,
})
})
it('searchSkillsHttp omits highlightedOnly when approvedOnly is false', async () => {
const runAction = vi.fn().mockResolvedValue([])
await __handlers.searchSkillsHandler(
makeCtx({ runAction }),
@@ -64,7 +77,7 @@ describe('httpApi handlers', () => {
expect(runAction).toHaveBeenCalledWith(expect.anything(), {
query: 'test',
limit: undefined,
approvedOnly: undefined,
highlightedOnly: undefined,
})
})
@@ -416,13 +429,14 @@ describe('httpApi handlers', () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'user1' } as never)
const runMutation = vi.fn().mockResolvedValue({ ok: true })
const response = await cliSkillUndeleteHttp(
const response = await __handlers.cliSkillDeleteHandler(
makeCtx({ runMutation }),
new Request('https://x/api/cli/skill/undelete', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ slug: 'demo' }),
}),
false,
)
expect(response.status).toBe(200)
expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
@@ -437,13 +451,14 @@ describe('httpApi handlers', () => {
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'user1' } as never)
const runMutation = vi.fn().mockResolvedValue({ ok: true })
const response = await cliSkillDeleteHttp(
const response = await __handlers.cliSkillDeleteHandler(
makeCtx({ runMutation }),
new Request('https://x/api/cli/skill/delete', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ slug: 'demo' }),
}),
true,
)
expect(response.status).toBe(200)
expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+3 -2
View File
@@ -5,7 +5,7 @@ import {
CliSkillDeleteRequestSchema,
CliTelemetrySyncRequestSchema,
parseArk,
} from 'clawdhub-schema'
} from 'clawhub-schema'
import { api, internal } from './_generated/api'
import type { Id } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
@@ -44,13 +44,14 @@ async function searchSkillsHandler(ctx: ActionCtx, request: Request) {
const query = url.searchParams.get('q')?.trim() ?? ''
const limit = toOptionalNumber(url.searchParams.get('limit'))
const approvedOnly = url.searchParams.get('approvedOnly') === 'true'
const highlightedOnly = url.searchParams.get('highlightedOnly') === 'true' || approvedOnly
if (!query) return json({ results: [] })
const results = (await ctx.runAction(api.search.searchSkills, {
query,
limit,
approvedOnly: approvedOnly || undefined,
highlightedOnly: highlightedOnly || undefined,
})) as SearchSkillEntry[]
return json({
+83
View File
@@ -158,6 +158,31 @@ describe('httpApiV1 handlers', () => {
expect(json.items[0].tags.latest).toBe('1.0.0')
})
it('lists skills supports sort aliases', async () => {
const checks: Array<[string, string]> = [
['rating', 'stars'],
['installs', 'installsCurrent'],
['installs-all-time', 'installsAllTime'],
['trending', 'trending'],
]
for (const [input, expected] of checks) {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('sort' in args || 'cursor' in args || 'limit' in args) {
expect(args.sort).toBe(expected)
return { items: [], nextCursor: null }
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.listSkillsV1Handler(
makeCtx({ runQuery, runMutation }),
new Request(`https://example.com/api/v1/skills?sort=${input}`),
)
expect(response.status).toBe(200)
}
})
it('get skill returns 404 when missing', async () => {
const runQuery = vi.fn().mockResolvedValue(null)
const runMutation = vi.fn().mockResolvedValue(okRate())
@@ -498,4 +523,62 @@ describe('httpApiV1 handlers', () => {
)
expect(response2.status).toBe(200)
})
it('stars require auth', async () => {
vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.starsPostRouterV1Handler(
makeCtx({ runMutation }),
new Request('https://example.com/api/v1/stars/demo', { method: 'POST' }),
)
expect(response.status).toBe(401)
})
it('stars add succeeds', async () => {
vi.mocked(requireApiTokenUser).mockResolvedValue({
userId: 'users:1',
user: { handle: 'p' },
} as never)
const runQuery = vi.fn().mockResolvedValue({ _id: 'skills:1' })
const runMutation = vi
.fn()
.mockResolvedValueOnce(okRate())
.mockResolvedValueOnce(okRate())
.mockResolvedValueOnce({ ok: true, starred: true, alreadyStarred: false })
const response = await __handlers.starsPostRouterV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/stars/demo', {
method: 'POST',
headers: { Authorization: 'Bearer clh_test' },
}),
)
expect(response.status).toBe(200)
const json = await response.json()
expect(json.ok).toBe(true)
expect(json.starred).toBe(true)
})
it('stars delete succeeds', async () => {
vi.mocked(requireApiTokenUser).mockResolvedValue({
userId: 'users:1',
user: { handle: 'p' },
} as never)
const runQuery = vi.fn().mockResolvedValue({ _id: 'skills:1' })
const runMutation = vi
.fn()
.mockResolvedValueOnce(okRate())
.mockResolvedValueOnce(okRate())
.mockResolvedValueOnce({ ok: true, unstarred: true, alreadyUnstarred: false })
const response = await __handlers.starsDeleteRouterV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/stars/demo', {
method: 'DELETE',
headers: { Authorization: 'Bearer clh_test' },
}),
)
expect(response.status).toBe(200)
const json = await response.json()
expect(json.ok).toBe(true)
expect(json.unstarred).toBe(true)
})
})
+115 -7
View File
@@ -1,4 +1,4 @@
import { CliPublishRequestSchema, parseArk } from 'clawdhub-schema'
import { CliPublishRequestSchema, parseArk } from 'clawhub-schema'
import { api, internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
@@ -44,6 +44,9 @@ type ListSkillsResult = {
nextCursor: string | null
}
type SkillFile = Doc<'skillVersions'>['files'][number]
type SoulFile = Doc<'soulVersions'>['files'][number]
type GetBySlugResult = {
skill: {
_id: Id<'skills'>
@@ -56,7 +59,7 @@ type GetBySlugResult = {
updatedAt: number
} | null
latestVersion: Doc<'skillVersions'> | null
owner: { handle?: string; displayName?: string; image?: string } | null
owner: { _id: Id<'users'>; handle?: string; displayName?: string; image?: string } | null
} | null
type ListVersionsResult = {
@@ -191,11 +194,14 @@ async function listSkillsV1Handler(ctx: ActionCtx, request: Request) {
const url = new URL(request.url)
const limit = toOptionalNumber(url.searchParams.get('limit'))
const cursor = url.searchParams.get('cursor')?.trim() || undefined
const rawCursor = url.searchParams.get('cursor')?.trim() || undefined
const sort = parseListSort(url.searchParams.get('sort'))
const cursor = sort === 'updated' ? rawCursor : undefined
const result = (await ctx.runQuery(api.skills.listPublicPage, {
limit,
cursor,
sort,
})) as ListSkillsResult
const items = await Promise.all(
@@ -261,6 +267,7 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
owner: result.owner
? {
handle: result.owner.handle ?? null,
userId: result.owner._id,
displayName: result.owner.displayName ?? null,
image: result.owner.image ?? null,
}
@@ -315,7 +322,7 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
createdAt: version.createdAt,
changelog: version.changelog,
changelogSource: version.changelogSource ?? null,
files: version.files.map((file) => ({
files: version.files.map((file: SkillFile) => ({
path: file.path,
size: file.size,
sha256: file.sha256,
@@ -368,6 +375,9 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
if (!blob) return text('File missing in storage', 410, rate.headers)
const textContent = await blob.text()
const isSvg =
file.contentType?.toLowerCase().includes('svg') || file.path.toLowerCase().endsWith('.svg')
const headers = mergeHeaders(rate.headers, {
'Content-Type': file.contentType
? `${file.contentType}; charset=utf-8`
@@ -376,6 +386,14 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
ETag: file.sha256,
'X-Content-SHA256': file.sha256,
'X-Content-Size': String(file.size),
'X-Content-Type-Options': 'nosniff',
'X-Frame-Options': 'DENY',
// For any text response that a browser might try to render, lock it down.
// In particular, this prevents SVG <foreignObject> script execution from
// reading localStorage tokens on this origin.
'Content-Security-Policy':
"default-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'",
...(isSvg ? { 'Content-Disposition': 'attachment' } : {}),
})
return new Response(textContent, { status: 200, headers })
}
@@ -753,9 +771,36 @@ function toOptionalNumber(value: string | null) {
return Number.isFinite(parsed) ? parsed : undefined
}
type SkillListSort =
| 'updated'
| 'downloads'
| 'stars'
| 'installsCurrent'
| 'installsAllTime'
| 'trending'
function parseListSort(value: string | null): SkillListSort {
const normalized = value?.trim().toLowerCase()
if (normalized === 'downloads') return 'downloads'
if (normalized === 'stars' || normalized === 'rating') return 'stars'
if (
normalized === 'installs' ||
normalized === 'install' ||
normalized === 'installscurrent' ||
normalized === 'installs-current'
) {
return 'installsCurrent'
}
if (normalized === 'installsalltime' || normalized === 'installs-all-time') {
return 'installsAllTime'
}
if (normalized === 'trending') return 'trending'
return 'updated'
}
async function sha256Hex(bytes: Uint8Array) {
const buffer = bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength)
const digest = await crypto.subtle.digest('SHA-256', buffer)
const data = new Uint8Array(bytes)
const digest = await crypto.subtle.digest('SHA-256', data)
return toHex(new Uint8Array(digest))
}
@@ -895,7 +940,7 @@ async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
createdAt: version.createdAt,
changelog: version.changelog,
changelogSource: version.changelogSource ?? null,
files: version.files.map((file) => ({
files: version.files.map((file: SoulFile) => ({
path: file.path,
size: file.size,
sha256: file.sha256,
@@ -950,6 +995,9 @@ async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
void ctx.runMutation(api.soulDownloads.increment, { soulId: soulResult.soul._id })
const isSvg =
file.contentType?.toLowerCase().includes('svg') || file.path.toLowerCase().endsWith('.svg')
const headers = mergeHeaders(rate.headers, {
'Content-Type': file.contentType
? `${file.contentType}; charset=utf-8`
@@ -958,6 +1006,14 @@ async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
ETag: file.sha256,
'X-Content-SHA256': file.sha256,
'X-Content-Size': String(file.size),
'X-Content-Type-Options': 'nosniff',
'X-Frame-Options': 'DENY',
// For any text response that a browser might try to render, lock it down.
// In particular, this prevents SVG <foreignObject> script execution from
// reading localStorage tokens on this origin.
'Content-Security-Policy':
"default-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'",
...(isSvg ? { 'Content-Disposition': 'attachment' } : {}),
})
return new Response(textContent, { status: 200, headers })
}
@@ -1047,6 +1103,56 @@ async function soulsDeleteRouterV1Handler(ctx: ActionCtx, request: Request) {
}
export const soulsDeleteRouterV1Http = httpAction(soulsDeleteRouterV1Handler)
async function starsPostRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'write')
if (!rate.ok) return rate.response
const segments = getPathSegments(request, '/api/v1/stars/')
if (segments.length !== 1) return text('Not found', 404, rate.headers)
const slug = segments[0]?.trim().toLowerCase() ?? ''
try {
const { userId } = await requireApiTokenUser(ctx, request)
const skill = await ctx.runQuery(internal.skills.getSkillBySlugInternal, { slug })
if (!skill) return text('Skill not found', 404, rate.headers)
const result = await ctx.runMutation(internal.stars.addStarInternal, {
userId,
skillId: skill._id,
})
return json(result, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
}
}
export const starsPostRouterV1Http = httpAction(starsPostRouterV1Handler)
async function starsDeleteRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'write')
if (!rate.ok) return rate.response
const segments = getPathSegments(request, '/api/v1/stars/')
if (segments.length !== 1) return text('Not found', 404, rate.headers)
const slug = segments[0]?.trim().toLowerCase() ?? ''
try {
const { userId } = await requireApiTokenUser(ctx, request)
const skill = await ctx.runQuery(internal.skills.getSkillBySlugInternal, { slug })
if (!skill) return text('Skill not found', 404, rate.headers)
const result = await ctx.runMutation(internal.stars.removeStarInternal, {
userId,
skillId: skill._id,
})
return json(result, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
}
}
export const starsDeleteRouterV1Http = httpAction(starsDeleteRouterV1Handler)
export const __handlers = {
searchSkillsV1Handler,
resolveSkillVersionV1Handler,
@@ -1060,5 +1166,7 @@ export const __handlers = {
publishSoulV1Handler,
soulsPostRouterV1Handler,
soulsDeleteRouterV1Handler,
starsPostRouterV1Handler,
starsDeleteRouterV1Handler,
whoamiV1Handler,
}
+39
View File
@@ -0,0 +1,39 @@
import { v } from 'convex/values'
import { internalMutation } from './_generated/server'
import { buildTrendingLeaderboard } from './lib/leaderboards'
const MAX_TRENDING_LIMIT = 200
const KEEP_LEADERBOARD_ENTRIES = 3
export const rebuildTrendingLeaderboardInternal = internalMutation({
args: { limit: v.optional(v.number()) },
handler: async (ctx, args) => {
const limit = clampInt(args.limit ?? MAX_TRENDING_LIMIT, 1, MAX_TRENDING_LIMIT)
const now = Date.now()
const { startDay, endDay, items } = await buildTrendingLeaderboard(ctx, { limit, now })
await ctx.db.insert('skillLeaderboards', {
kind: 'trending',
generatedAt: now,
rangeStartDay: startDay,
rangeEndDay: endDay,
items,
})
const recent = await ctx.db
.query('skillLeaderboards')
.withIndex('by_kind', (q) => q.eq('kind', 'trending'))
.order('desc')
.take(KEEP_LEADERBOARD_ENTRIES + 5)
for (const entry of recent.slice(KEEP_LEADERBOARD_ENTRIES)) {
await ctx.db.delete(entry._id)
}
return { ok: true as const, count: items.length }
},
})
function clampInt(value: number, min: number, max: number) {
return Math.min(Math.max(value, min), max)
}
+10 -2
View File
@@ -1,5 +1,5 @@
import { getAuthUserId } from '@convex-dev/auth/server'
import { api } from '../_generated/api'
import { internal } from '../_generated/api'
import type { Doc } from '../_generated/dataModel'
import type { ActionCtx, MutationCtx, QueryCtx } from '../_generated/server'
@@ -16,7 +16,7 @@ export async function requireUser(ctx: MutationCtx | QueryCtx) {
export async function requireUserFromAction(ctx: ActionCtx) {
const userId = await getAuthUserId(ctx)
if (!userId) throw new Error('Unauthorized')
const user = await ctx.runQuery(api.users.getById, { userId })
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
if (!user || user.deletedAt) throw new Error('User not found')
return { userId, user: user as Doc<'users'> }
}
@@ -26,3 +26,11 @@ export function assertRole(user: Doc<'users'>, allowed: Role[]) {
throw new Error('Forbidden')
}
}
export function assertAdmin(user: Doc<'users'>) {
assertRole(user, ['admin'])
}
export function assertModerator(user: Doc<'users'>) {
assertRole(user, ['admin', 'moderator'])
}
+50
View File
@@ -0,0 +1,50 @@
import type { Doc, Id } from '../_generated/dataModel'
import type { QueryCtx } from '../_generated/server'
type BadgeKind = Doc<'skillBadges'>['kind']
export type SkillBadgeMap = Partial<Record<BadgeKind, { byUserId: Id<'users'>; at: number }>>
export type SkillBadgeSource = { badges?: SkillBadgeMap | null }
type BadgeCtx = Pick<QueryCtx, 'db'>
export function isSkillHighlighted(skill: SkillBadgeSource) {
return Boolean(skill.badges?.highlighted)
}
export function isSkillOfficial(skill: SkillBadgeSource) {
return Boolean(skill.badges?.official)
}
export function isSkillDeprecated(skill: SkillBadgeSource) {
return Boolean(skill.badges?.deprecated)
}
export function buildBadgeMap(records: Doc<'skillBadges'>[]): SkillBadgeMap {
return records.reduce<SkillBadgeMap>((acc, record) => {
acc[record.kind] = { byUserId: record.byUserId, at: record.at }
return acc
}, {})
}
export async function getSkillBadgeMap(
ctx: BadgeCtx,
skillId: Id<'skills'>,
): Promise<SkillBadgeMap> {
const records = await ctx.db
.query('skillBadges')
.withIndex('by_skill', (q) => q.eq('skillId', skillId))
.collect()
return buildBadgeMap(records)
}
export async function getSkillBadgeMaps(
ctx: BadgeCtx,
skillIds: Array<Id<'skills'>>,
): Promise<Map<Id<'skills'>, SkillBadgeMap>> {
const entries = await Promise.all(
skillIds.map(async (skillId) => [skillId, await getSkillBadgeMap(ctx, skillId)] as const),
)
return new Map(entries)
}
+1 -1
View File
@@ -8,7 +8,7 @@ const GITHUB_API = 'https://api.github.com'
const DEFAULT_REPO = 'clawdbot/skills'
const DEFAULT_ROOT = 'skills'
const META_FILENAME = '_meta.json'
const USER_AGENT = 'clawdhub/skills-backup'
const USER_AGENT = 'clawhub/skills-backup'
type BackupFile = {
path: string
+3 -3
View File
@@ -1,4 +1,4 @@
import { TEXT_FILE_EXTENSION_SET } from 'clawdhub-schema'
import { TEXT_FILE_EXTENSION_SET } from 'clawhub-schema'
import { zipSync } from 'fflate'
import semver from 'semver'
import { parseFrontmatter } from './skills'
@@ -118,7 +118,7 @@ async function resolveRefCommit(parsed: GitHubImportUrl, ref: string, fetcher: t
const response = await fetcher(apiUrl, {
headers: {
Accept: 'application/vnd.github+json',
'User-Agent': 'clawdhub/github-import',
'User-Agent': 'clawhub/github-import',
},
})
if (!response.ok) throw new Error('GitHub ref not found')
@@ -156,7 +156,7 @@ export async function fetchGitHubZipBytes(
const maxZipBytes = limits?.maxZipBytes ?? 25 * 1024 * 1024
const url = `https://${CODELOAD_HOST}/${resolved.owner}/${resolved.repo}/zip/${resolved.commit}`
const response = await fetcher(url, {
headers: { 'User-Agent': 'clawdhub/github-import' },
headers: { 'User-Agent': 'clawhub/github-import' },
})
if (!response.ok) throw new Error('GitHub archive download failed')
+1 -1
View File
@@ -8,7 +8,7 @@ const GITHUB_API = 'https://api.github.com'
const DEFAULT_REPO = 'clawdbot/souls'
const DEFAULT_ROOT = 'souls'
const META_FILENAME = '_meta.json'
const USER_AGENT = 'clawdhub/souls-backup'
const USER_AGENT = 'clawhub/souls-backup'
type BackupFile = {
path: string
+103
View File
@@ -0,0 +1,103 @@
import type { Id } from '../_generated/dataModel'
import type { MutationCtx, QueryCtx } from '../_generated/server'
const DAY_MS = 24 * 60 * 60 * 1000
export const TRENDING_DAYS = 7
type LeaderboardEntry = {
skillId: Id<'skills'>
score: number
installs: number
downloads: number
}
export function toDayKey(timestamp: number) {
return Math.floor(timestamp / DAY_MS)
}
export function getTrendingRange(now: number) {
const endDay = toDayKey(now)
const startDay = endDay - (TRENDING_DAYS - 1)
return { startDay, endDay }
}
export async function buildTrendingLeaderboard(
ctx: QueryCtx | MutationCtx,
params: { limit: number; now?: number },
) {
const now = params.now ?? Date.now()
const { startDay, endDay } = getTrendingRange(now)
const rows = await ctx.db
.query('skillDailyStats')
.withIndex('by_day', (q) => q.gte('day', startDay).lte('day', endDay))
.collect()
const totals = new Map<Id<'skills'>, { installs: number; downloads: number }>()
for (const row of rows) {
const current = totals.get(row.skillId) ?? { installs: 0, downloads: 0 }
current.installs += row.installs
current.downloads += row.downloads
totals.set(row.skillId, current)
}
const entries = Array.from(totals, ([skillId, totalsEntry]) => ({
skillId,
installs: totalsEntry.installs,
downloads: totalsEntry.downloads,
score: totalsEntry.installs,
}))
const items = topN(entries, params.limit, compareTrendingEntries).sort((a, b) =>
compareTrendingEntries(b, a),
)
return { startDay, endDay, items }
}
function compareTrendingEntries(a: LeaderboardEntry, b: LeaderboardEntry) {
if (a.score !== b.score) return a.score - b.score
if (a.downloads !== b.downloads) return a.downloads - b.downloads
return 0
}
function topN<T>(entries: T[], limit: number, compare: (a: T, b: T) => number) {
if (entries.length <= limit) return entries.slice()
const heap: T[] = []
for (const entry of entries) {
if (heap.length < limit) {
heap.push(entry)
siftUp(heap, heap.length - 1, compare)
continue
}
if (compare(entry, heap[0]) <= 0) continue
heap[0] = entry
siftDown(heap, 0, compare)
}
return heap
}
function siftUp<T>(heap: T[], index: number, compare: (a: T, b: T) => number) {
let current = index
while (current > 0) {
const parent = Math.floor((current - 1) / 2)
if (compare(heap[current], heap[parent]) >= 0) break
;[heap[current], heap[parent]] = [heap[parent], heap[current]]
current = parent
}
}
function siftDown<T>(heap: T[], index: number, compare: (a: T, b: T) => number) {
let current = index
const length = heap.length
while (true) {
const left = current * 2 + 1
const right = current * 2 + 2
let smallest = current
if (left < length && compare(heap[left], heap[smallest]) < 0) smallest = left
if (right < length && compare(heap[right], heap[smallest]) < 0) smallest = right
if (smallest === current) break
;[heap[current], heap[smallest]] = [heap[smallest], heap[current]]
current = smallest
}
}
+49
View File
@@ -0,0 +1,49 @@
import type { Doc } from '../_generated/dataModel'
const FLAG_RULES: Array<{ flag: string; pattern: RegExp }> = [
// Known-bad / known-suspicious identifiers.
// NOTE: keep these narrowly scoped; use staff review to confirm removals.
{
flag: 'blocked.malware',
pattern: /(keepcold131\/ClawdAuthenticatorTool|ClawdAuthenticatorTool)/i,
},
{ flag: 'suspicious.keyword', pattern: /(malware|stealer|phish|phishing|keylogger)/i },
{ flag: 'suspicious.secrets', pattern: /(api[-_ ]?key|token|password|private key|secret)/i },
{ flag: 'suspicious.crypto', pattern: /(wallet|seed phrase|mnemonic|crypto)/i },
{ flag: 'suspicious.webhook', pattern: /(discord\.gg|webhook|hooks\.slack)/i },
{ flag: 'suspicious.script', pattern: /(curl[^\n]+\|\s*(sh|bash))/i },
{ flag: 'suspicious.url_shortener', pattern: /(bit\.ly|tinyurl\.com|t\.co|goo\.gl|is\.gd)/i },
]
export function deriveModerationFlags({
skill,
parsed,
files,
}: {
skill: Pick<Doc<'skills'>, 'slug' | 'displayName' | 'summary'>
parsed: Doc<'skillVersions'>['parsed']
files: Doc<'skillVersions'>['files']
}) {
const text = [
skill.slug,
skill.displayName,
skill.summary ?? '',
JSON.stringify(parsed?.frontmatter ?? {}),
JSON.stringify(parsed?.metadata ?? {}),
JSON.stringify((parsed as { moltbot?: unknown } | undefined)?.moltbot ?? {}),
...files.map((file) => file.path),
]
.filter(Boolean)
.join('\n')
const flags = new Set<string>()
for (const rule of FLAG_RULES) {
if (rule.pattern.test(text)) {
flags.add(rule.flag)
}
}
return Array.from(flags)
}
+91
View File
@@ -0,0 +1,91 @@
import type { Doc } from '../_generated/dataModel'
export type PublicUser = Pick<
Doc<'users'>,
'_id' | '_creationTime' | 'handle' | 'name' | 'displayName' | 'image' | 'bio'
>
export type PublicSkill = Pick<
Doc<'skills'>,
| '_id'
| '_creationTime'
| 'slug'
| 'displayName'
| 'summary'
| 'ownerUserId'
| 'canonicalSkillId'
| 'forkOf'
| 'latestVersionId'
| 'tags'
| 'badges'
| 'stats'
| 'createdAt'
| 'updatedAt'
>
export type PublicSoul = Pick<
Doc<'souls'>,
| '_id'
| '_creationTime'
| 'slug'
| 'displayName'
| 'summary'
| 'ownerUserId'
| 'latestVersionId'
| 'tags'
| 'stats'
| 'createdAt'
| 'updatedAt'
>
export function toPublicUser(user: Doc<'users'> | null | undefined): PublicUser | null {
if (!user || user.deletedAt) return null
return {
_id: user._id,
_creationTime: user._creationTime,
handle: user.handle,
name: user.name,
displayName: user.displayName,
image: user.image,
bio: user.bio,
}
}
export function toPublicSkill(skill: Doc<'skills'> | null | undefined): PublicSkill | null {
if (!skill || skill.softDeletedAt) return null
if (skill.moderationStatus && skill.moderationStatus !== 'active') return null
if (skill.moderationFlags?.includes('blocked.malware')) return null
return {
_id: skill._id,
_creationTime: skill._creationTime,
slug: skill.slug,
displayName: skill.displayName,
summary: skill.summary,
ownerUserId: skill.ownerUserId,
canonicalSkillId: skill.canonicalSkillId,
forkOf: skill.forkOf,
latestVersionId: skill.latestVersionId,
tags: skill.tags,
badges: skill.badges,
stats: skill.stats,
createdAt: skill.createdAt,
updatedAt: skill.updatedAt,
}
}
export function toPublicSoul(soul: Doc<'souls'> | null | undefined): PublicSoul | null {
if (!soul || soul.softDeletedAt) return null
return {
_id: soul._id,
_creationTime: soul._creationTime,
slug: soul.slug,
displayName: soul.displayName,
summary: soul.summary,
ownerUserId: soul.ownerUserId,
latestVersionId: soul.latestVersionId,
tags: soul.tags,
stats: soul.stats,
createdAt: soul.createdAt,
updatedAt: soul.updatedAt,
}
}
+46
View File
@@ -0,0 +1,46 @@
/* @vitest-environment node */
import { describe, expect, it } from 'vitest'
import { __test, matchesExactTokens, tokenize } from './searchText'
describe('searchText', () => {
it('tokenize lowercases and splits on punctuation', () => {
expect(tokenize('Minimax Usage /minimax-usage')).toEqual([
'minimax',
'usage',
'minimax',
'usage',
])
})
it('matchesExactTokens requires at least one query token to prefix-match', () => {
const queryTokens = tokenize('Remind Me')
expect(matchesExactTokens(queryTokens, ['Remind Me', '/remind-me', 'Short summary'])).toBe(true)
// "Reminder" starts with "remind", so it matches with prefix matching
expect(matchesExactTokens(queryTokens, ['Reminder tool', '/reminder', 'Short summary'])).toBe(
true,
)
// Matches because "remind" token is present
expect(matchesExactTokens(queryTokens, ['Remind tool', '/remind', 'Short summary'])).toBe(true)
// No matching tokens at all
expect(matchesExactTokens(queryTokens, ['Other tool', '/other', 'Short summary'])).toBe(false)
})
it('matchesExactTokens supports prefix matching for partial queries', () => {
// "go" should match "gohome" because "gohome" starts with "go"
expect(matchesExactTokens(['go'], ['GoHome', '/gohome', 'Navigate home'])).toBe(true)
// "pad" should match "padel"
expect(matchesExactTokens(['pad'], ['Padel', '/padel', 'Tennis-like sport'])).toBe(true)
// "xyz" should not match anything
expect(matchesExactTokens(['xyz'], ['GoHome', '/gohome', 'Navigate home'])).toBe(false)
})
it('matchesExactTokens ignores empty inputs', () => {
expect(matchesExactTokens([], ['text'])).toBe(false)
expect(matchesExactTokens(['token'], [' ', null, undefined])).toBe(false)
})
it('normalize uses lowercase', () => {
expect(__test.normalize('AbC')).toBe('abc')
})
})
+27
View File
@@ -0,0 +1,27 @@
const WORD_RE = /[a-z0-9]+/g
function normalize(value: string) {
return value.toLowerCase()
}
export function tokenize(value: string): string[] {
if (!value) return []
return normalize(value).match(WORD_RE) ?? []
}
export function matchesExactTokens(
queryTokens: string[],
parts: Array<string | null | undefined>,
): boolean {
if (queryTokens.length === 0) return false
const text = parts.filter((part) => Boolean(part?.trim())).join(' ')
if (!text) return false
const textTokens = tokenize(text)
if (textTokens.length === 0) return false
// Require at least one token to prefix-match, allowing vector similarity to determine relevance
return queryTokens.some((queryToken) =>
textTokens.some((textToken) => textToken.includes(queryToken)),
)
}
export const __test = { normalize, tokenize, matchesExactTokens }
+22 -13
View File
@@ -3,8 +3,10 @@ import semver from 'semver'
import { api, internal } from '../_generated/api'
import type { Doc, Id } from '../_generated/dataModel'
import type { ActionCtx, MutationCtx } from '../_generated/server'
import { getSkillBadgeMap, isSkillHighlighted } from './badges'
import { generateChangelogForPublish } from './changelog'
import { generateEmbedding } from './embeddings'
import type { PublicUser } from './public'
import {
buildEmbeddingText,
getFrontmatterMetadata,
@@ -75,16 +77,20 @@ export async function publishVersionForUser(
if (sanitizedFiles.some((file) => !file.path)) {
throw new ConvexError('Invalid file paths')
}
if (sanitizedFiles.some((file) => !isTextFile(file.path ?? '', file.contentType ?? undefined))) {
const safeFiles = sanitizedFiles.map((file) => ({
...file,
path: file.path as string,
}))
if (safeFiles.some((file) => !isTextFile(file.path, file.contentType ?? undefined))) {
throw new ConvexError('Only text-based files are allowed')
}
const totalBytes = sanitizedFiles.reduce((sum, file) => sum + file.size, 0)
const totalBytes = safeFiles.reduce((sum, file) => sum + file.size, 0)
if (totalBytes > MAX_TOTAL_BYTES) {
throw new ConvexError('Skill bundle exceeds 50MB limit')
}
const readmeFile = sanitizedFiles.find(
const readmeFile = safeFiles.find(
(file) => file.path?.toLowerCase() === 'skill.md' || file.path?.toLowerCase() === 'skills.md',
)
if (!readmeFile) throw new ConvexError('SKILL.md is required')
@@ -95,7 +101,7 @@ export async function publishVersionForUser(
const metadata = mergeSourceIntoMetadata(getFrontmatterMetadata(frontmatter), args.source)
const otherFiles = [] as Array<{ path: string; content: string }>
for (const file of sanitizedFiles) {
for (const file of safeFiles) {
if (!file.path || file.path.toLowerCase().endsWith('.md')) continue
if (!isTextFile(file.path, file.contentType ?? undefined)) continue
const content = await fetchText(ctx, file.storageId)
@@ -110,7 +116,7 @@ export async function publishVersionForUser(
})
const fingerprintPromise = hashSkillFiles(
sanitizedFiles.map((file) => ({ path: file.path ?? '', sha256: file.sha256 })),
safeFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
)
const changelogPromise =
@@ -120,7 +126,7 @@ export async function publishVersionForUser(
slug,
version,
readmeText,
files: sanitizedFiles.map((file) => ({ path: file.path ?? '', sha256: file.sha256 })),
files: safeFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
})
const embeddingPromise = generateEmbedding(embeddingText)
@@ -148,9 +154,9 @@ export async function publishVersionForUser(
version: args.forkOf.version?.trim() || undefined,
}
: undefined,
files: sanitizedFiles.map((file) => ({
files: safeFiles.map((file) => ({
...file,
path: file.path ?? '',
path: file.path,
})),
parsed: {
frontmatter,
@@ -160,7 +166,9 @@ export async function publishVersionForUser(
embedding,
})) as PublishResult
const owner = (await ctx.runQuery(api.users.getById, { userId })) as Doc<'users'> | null
const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
userId,
})) as Doc<'users'> | null
const ownerHandle = owner?.handle ?? owner?.displayName ?? owner?.name ?? 'unknown'
void ctx.scheduler
@@ -169,7 +177,7 @@ export async function publishVersionForUser(
version,
displayName,
ownerHandle,
files: sanitizedFiles,
files: safeFiles,
publishedAt: Date.now(),
})
.catch((error) => {
@@ -212,13 +220,14 @@ export async function queueHighlightedWebhook(ctx: MutationCtx, skillId: Id<'ski
const owner = await ctx.db.get(skill.ownerUserId)
const latestVersion = skill.latestVersionId ? await ctx.db.get(skill.latestVersionId) : null
const badges = await getSkillBadgeMap(ctx, skillId)
const payload: WebhookSkillPayload = {
slug: skill.slug,
displayName: skill.displayName,
summary: skill.summary ?? undefined,
version: latestVersion?.version ?? undefined,
ownerHandle: owner?.handle ?? owner?.name ?? undefined,
batch: skill.batch ?? undefined,
highlighted: isSkillHighlighted({ badges }),
tags: Object.keys(skill.tags ?? {}),
}
@@ -255,7 +264,7 @@ async function schedulePublishWebhook(
) {
const result = (await ctx.runQuery(api.skills.getBySlug, {
slug: params.slug,
})) as { skill: Doc<'skills'>; owner: Doc<'users'> | null } | null
})) as { skill: Doc<'skills'>; owner: PublicUser | null } | null
if (!result?.skill) return
const payload: WebhookSkillPayload = {
@@ -264,7 +273,7 @@ async function schedulePublishWebhook(
summary: result.skill.summary ?? undefined,
version: params.version,
ownerHandle: result.owner?.handle ?? result.owner?.name ?? undefined,
batch: result.skill.batch ?? undefined,
highlighted: isSkillHighlighted(result.skill),
tags: Object.keys(result.skill.tags ?? {}),
}
+80
View File
@@ -0,0 +1,80 @@
import type { Doc, Id } from '../_generated/dataModel'
import type { MutationCtx } from '../_generated/server'
import { toDayKey } from './leaderboards'
type SkillStatDeltas = {
downloads?: number
stars?: number
installsCurrent?: number
installsAllTime?: number
}
export function applySkillStatDeltas(skill: Doc<'skills'>, deltas: SkillStatDeltas) {
const currentDownloads =
typeof skill.statsDownloads === 'number' ? skill.statsDownloads : skill.stats.downloads
const currentStars = typeof skill.statsStars === 'number' ? skill.statsStars : skill.stats.stars
const currentInstallsCurrent =
typeof skill.statsInstallsCurrent === 'number'
? skill.statsInstallsCurrent
: (skill.stats.installsCurrent ?? 0)
const currentInstallsAllTime =
typeof skill.statsInstallsAllTime === 'number'
? skill.statsInstallsAllTime
: (skill.stats.installsAllTime ?? 0)
const nextDownloads = Math.max(0, currentDownloads + (deltas.downloads ?? 0))
const nextStars = Math.max(0, currentStars + (deltas.stars ?? 0))
const nextInstallsCurrent = Math.max(0, currentInstallsCurrent + (deltas.installsCurrent ?? 0))
const nextInstallsAllTime = Math.max(0, currentInstallsAllTime + (deltas.installsAllTime ?? 0))
return {
statsDownloads: nextDownloads,
statsStars: nextStars,
statsInstallsCurrent: nextInstallsCurrent,
statsInstallsAllTime: nextInstallsAllTime,
stats: {
...skill.stats,
downloads: nextDownloads,
stars: nextStars,
installsCurrent: nextInstallsCurrent,
installsAllTime: nextInstallsAllTime,
},
}
}
export async function bumpDailySkillStats(
ctx: MutationCtx,
params: {
skillId: Id<'skills'>
now: number
downloads?: number
installs?: number
},
) {
const downloads = params.downloads ?? 0
const installs = params.installs ?? 0
if (downloads === 0 && installs === 0) return
const day = toDayKey(params.now)
const existing = await ctx.db
.query('skillDailyStats')
.withIndex('by_skill_day', (q) => q.eq('skillId', params.skillId).eq('day', day))
.unique()
if (existing) {
await ctx.db.patch(existing._id, {
downloads: Math.max(0, existing.downloads + downloads),
installs: Math.max(0, existing.installs + installs),
updatedAt: params.now,
})
return
}
await ctx.db.insert('skillDailyStats', {
skillId: params.skillId,
day,
downloads: Math.max(0, downloads),
installs: Math.max(0, installs),
updatedAt: params.now,
})
}
+1 -1
View File
@@ -7,7 +7,7 @@ import {
parseArk,
type SkillInstallSpec,
TEXT_FILE_EXTENSION_SET,
} from 'clawdhub-schema'
} from 'clawhub-schema'
import { parse as parseYaml } from 'yaml'
export type ParsedSkillFrontmatter = Record<string, unknown>
+4 -2
View File
@@ -1,6 +1,6 @@
import { ConvexError } from 'convex/values'
import semver from 'semver'
import { api, internal } from '../_generated/api'
import { internal } from '../_generated/api'
import type { Doc, Id } from '../_generated/dataModel'
import type { ActionCtx } from '../_generated/server'
import { generateEmbedding } from './embeddings'
@@ -171,7 +171,9 @@ export async function publishSoulVersionForUser(
embedding,
})) as PublishResult
const owner = (await ctx.runQuery(api.users.getById, { userId })) as Doc<'users'> | null
const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
userId,
})) as Doc<'users'> | null
const ownerHandle = owner?.handle ?? owner?.name ?? userId
void ctx.scheduler
+8 -8
View File
@@ -20,7 +20,7 @@ describe('webhook config', () => {
delete process.env.SITE_URL
process.env.DISCORD_WEBHOOK_URL = 'https://example.com'
const config = getWebhookConfig()
expect(config.siteUrl).toBe('https://clawdhub.com')
expect(config.siteUrl).toBe('https://clawhub.ai')
})
})
@@ -36,11 +36,11 @@ describe('webhook filtering', () => {
const config = {
url: 'https://example.com',
highlightedOnly: true,
siteUrl: 'https://clawdhub.com',
siteUrl: 'https://clawhub.ai',
}
const allowed = shouldSendWebhook(
'skill.publish',
{ slug: 'demo', displayName: 'Demo', batch: 'latest' },
{ slug: 'demo', displayName: 'Demo', highlighted: false },
config,
)
expect(allowed).toBe(false)
@@ -50,11 +50,11 @@ describe('webhook filtering', () => {
const config = {
url: 'https://example.com',
highlightedOnly: true,
siteUrl: 'https://clawdhub.com',
siteUrl: 'https://clawhub.ai',
}
const allowed = shouldSendWebhook(
'skill.highlighted',
{ slug: 'demo', displayName: 'Demo', batch: 'latest' },
{ slug: 'demo', displayName: 'Demo', highlighted: true },
config,
)
expect(allowed).toBe(true)
@@ -65,9 +65,9 @@ describe('payload building', () => {
it('builds canonical url with owner', () => {
const url = buildSkillUrl(
{ slug: 'beeper', displayName: 'Beeper', ownerHandle: 'KrauseFx' },
'https://clawdhub.com',
'https://clawhub.ai',
)
expect(url).toBe('https://clawdhub.com/KrauseFx/beeper')
expect(url).toBe('https://clawhub.ai/KrauseFx/beeper')
})
it('builds a publish embed', () => {
@@ -81,7 +81,7 @@ describe('payload building', () => {
ownerHandle: 'steipete',
tags: ['latest', 'discord'],
},
{ url: 'https://example.com', highlightedOnly: false, siteUrl: 'https://clawdhub.com' },
{ url: 'https://example.com', highlightedOnly: false, siteUrl: 'https://clawhub.ai' },
)
const embed = payload.embeds[0]
expect(embed.title).toBe('Demo Skill')
+7 -7
View File
@@ -6,7 +6,7 @@ export type WebhookSkillPayload = {
summary?: string
version?: string
ownerHandle?: string
batch?: string
highlighted?: boolean
tags?: string[]
}
@@ -16,7 +16,7 @@ export type WebhookConfig = {
siteUrl: string
}
const DEFAULT_SITE_URL = 'https://clawdhub.com'
const DEFAULT_SITE_URL = 'https://clawhub.ai'
export function getWebhookConfig(env: NodeJS.ProcessEnv = process.env): WebhookConfig {
const url = env.DISCORD_WEBHOOK_URL?.trim() || null
@@ -33,7 +33,7 @@ export function shouldSendWebhook(
if (!config.url) return false
if (!config.highlightedOnly) return true
if (event === 'skill.highlighted') return true
return skill.batch === 'highlighted'
return Boolean(skill.highlighted)
}
export function buildDiscordPayload(
@@ -72,7 +72,7 @@ export function buildDiscordPayload(
},
],
footer: {
text: 'ClawdHub',
text: 'OpenClaw',
},
timestamp: new Date().toISOString(),
},
@@ -89,9 +89,9 @@ export function buildSkillUrl(skill: WebhookSkillPayload, siteUrl: string) {
function buildDescription(event: WebhookEvent, skill: WebhookSkillPayload) {
const summary = (skill.summary ?? '').trim()
if (summary) return truncate(summary, 200)
if (event === 'skill.highlighted') return 'Newly highlighted skill on ClawdHub.'
if (skill.version) return `New version v${skill.version} published on ClawdHub.`
return 'New skill published on ClawdHub.'
if (event === 'skill.highlighted') return 'Newly highlighted skill on OpenClaw.'
if (skill.version) return `New version v${skill.version} published on OpenClaw.`
return 'New skill published on OpenClaw.'
}
function parseBoolean(value?: string) {
+366
View File
@@ -265,6 +265,34 @@ type FingerprintBackfillPageResult = {
isDone: boolean
}
type BadgeBackfillStats = {
skillsScanned: number
skillsPatched: number
highlightsPatched: number
}
type SkillBadgeTableBackfillStats = {
skillsScanned: number
recordsInserted: number
}
type BadgeBackfillPageItem = {
skillId: Id<'skills'>
ownerUserId: Id<'users'>
createdAt?: number
updatedAt?: number
batch?: string
badges?: Doc<'skills'>['badges']
}
type BadgeBackfillPageResult = {
items: BadgeBackfillPageItem[]
cursor: string | null
isDone: boolean
}
type BadgeKind = Doc<'skillBadges'>['kind']
export const getSkillFingerprintBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
@@ -467,6 +495,344 @@ export const scheduleBackfillSkillFingerprints: ReturnType<typeof action> = acti
},
})
export const getSkillBadgeBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args): Promise<BadgeBackfillPageResult> => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const { page, isDone, continueCursor } = await ctx.db
.query('skills')
.order('asc')
.paginate({ cursor: args.cursor ?? null, numItems: batchSize })
const items: BadgeBackfillPageItem[] = page.map((skill) => ({
skillId: skill._id,
ownerUserId: skill.ownerUserId,
createdAt: skill.createdAt ?? undefined,
updatedAt: skill.updatedAt ?? undefined,
batch: skill.batch ?? undefined,
badges: skill.badges ?? undefined,
}))
return { items, cursor: continueCursor, isDone }
},
})
export const applySkillBadgeBackfillPatchInternal = internalMutation({
args: {
skillId: v.id('skills'),
badges: v.optional(
v.object({
redactionApproved: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
highlighted: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
official: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
deprecated: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
}),
),
},
handler: async (ctx, args) => {
await ctx.db.patch(args.skillId, { badges: args.badges ?? undefined, updatedAt: Date.now() })
return { ok: true as const }
},
})
export const upsertSkillBadgeRecordInternal = internalMutation({
args: {
skillId: v.id('skills'),
kind: v.union(
v.literal('highlighted'),
v.literal('official'),
v.literal('deprecated'),
v.literal('redactionApproved'),
),
byUserId: v.id('users'),
at: v.number(),
},
handler: async (ctx, args) => {
const existing = await ctx.db
.query('skillBadges')
.withIndex('by_skill_kind', (q) => q.eq('skillId', args.skillId).eq('kind', args.kind))
.unique()
if (existing) return { inserted: false as const }
await ctx.db.insert('skillBadges', {
skillId: args.skillId,
kind: args.kind,
byUserId: args.byUserId,
at: args.at,
})
return { inserted: true as const }
},
})
export type BadgeBackfillActionArgs = {
dryRun?: boolean
batchSize?: number
maxBatches?: number
}
export type BadgeBackfillActionResult = { ok: true; stats: BadgeBackfillStats }
export async function backfillSkillBadgesInternalHandler(
ctx: ActionCtx,
args: BadgeBackfillActionArgs,
): Promise<BadgeBackfillActionResult> {
const dryRun = Boolean(args.dryRun)
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
const totals: BadgeBackfillStats = {
skillsScanned: 0,
skillsPatched: 0,
highlightsPatched: 0,
}
let cursor: string | null = null
let isDone = false
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(internal.maintenance.getSkillBadgeBackfillPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as BadgeBackfillPageResult
cursor = page.cursor
isDone = page.isDone
for (const item of page.items) {
totals.skillsScanned++
const shouldHighlight = item.batch === 'highlighted' && !item.badges?.highlighted
if (!shouldHighlight) continue
totals.skillsPatched++
totals.highlightsPatched++
if (dryRun) continue
const at = item.updatedAt ?? item.createdAt ?? Date.now()
await ctx.runMutation(internal.maintenance.applySkillBadgeBackfillPatchInternal, {
skillId: item.skillId,
badges: {
...item.badges,
highlighted: {
byUserId: item.ownerUserId,
at,
},
},
})
}
if (isDone) break
}
if (!isDone) {
throw new ConvexError('Backfill incomplete (maxBatches reached)')
}
return { ok: true as const, stats: totals }
}
export const backfillSkillBadgesInternal = internalAction({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
},
handler: backfillSkillBadgesInternalHandler,
})
export const backfillSkillBadges: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
},
handler: async (ctx, args): Promise<BadgeBackfillActionResult> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
return ctx.runAction(
internal.maintenance.backfillSkillBadgesInternal,
args,
) as Promise<BadgeBackfillActionResult>
},
})
export const scheduleBackfillSkillBadges: ReturnType<typeof action> = action({
args: { dryRun: v.optional(v.boolean()) },
handler: async (ctx, args) => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
await ctx.scheduler.runAfter(0, internal.maintenance.backfillSkillBadgesInternal, {
dryRun: Boolean(args.dryRun),
batchSize: DEFAULT_BATCH_SIZE,
maxBatches: DEFAULT_MAX_BATCHES,
})
return { ok: true as const }
},
})
export type SkillBadgeTableBackfillActionResult = {
ok: true
stats: SkillBadgeTableBackfillStats
}
export async function backfillSkillBadgeTableInternalHandler(
ctx: ActionCtx,
args: BadgeBackfillActionArgs,
): Promise<SkillBadgeTableBackfillActionResult> {
const dryRun = Boolean(args.dryRun)
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
const totals: SkillBadgeTableBackfillStats = {
skillsScanned: 0,
recordsInserted: 0,
}
let cursor: string | null = null
let isDone = false
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(internal.maintenance.getSkillBadgeBackfillPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as BadgeBackfillPageResult
cursor = page.cursor
isDone = page.isDone
for (const item of page.items) {
totals.skillsScanned++
const badges = item.badges ?? {}
const entries: Array<{ kind: BadgeKind; byUserId: Id<'users'>; at: number }> = []
if (badges.redactionApproved) {
entries.push({
kind: 'redactionApproved',
byUserId: badges.redactionApproved.byUserId,
at: badges.redactionApproved.at,
})
}
if (badges.official) {
entries.push({
kind: 'official',
byUserId: badges.official.byUserId,
at: badges.official.at,
})
}
if (badges.deprecated) {
entries.push({
kind: 'deprecated',
byUserId: badges.deprecated.byUserId,
at: badges.deprecated.at,
})
}
const highlighted =
badges.highlighted ??
(item.batch === 'highlighted'
? {
byUserId: item.ownerUserId,
at: item.updatedAt ?? item.createdAt ?? Date.now(),
}
: undefined)
if (highlighted) {
entries.push({
kind: 'highlighted',
byUserId: highlighted.byUserId,
at: highlighted.at,
})
}
if (dryRun) continue
for (const entry of entries) {
const result = await ctx.runMutation(internal.maintenance.upsertSkillBadgeRecordInternal, {
skillId: item.skillId,
kind: entry.kind,
byUserId: entry.byUserId,
at: entry.at,
})
if (result.inserted) {
totals.recordsInserted++
}
}
}
if (isDone) break
}
if (!isDone) {
throw new ConvexError('Backfill incomplete (maxBatches reached)')
}
return { ok: true as const, stats: totals }
}
export const backfillSkillBadgeTableInternal = internalAction({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
},
handler: backfillSkillBadgeTableInternalHandler,
})
export const backfillSkillBadgeTable: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
},
handler: async (ctx, args): Promise<SkillBadgeTableBackfillActionResult> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
return ctx.runAction(
internal.maintenance.backfillSkillBadgeTableInternal,
args,
) as Promise<SkillBadgeTableBackfillActionResult>
},
})
export const scheduleBackfillSkillBadgeTable: ReturnType<typeof action> = action({
args: { dryRun: v.optional(v.boolean()) },
handler: async (ctx, args) => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
await ctx.scheduler.runAfter(0, internal.maintenance.backfillSkillBadgeTableInternal, {
dryRun: Boolean(args.dryRun),
batchSize: DEFAULT_BATCH_SIZE,
maxBatches: DEFAULT_MAX_BATCHES,
})
return { ok: true as const }
},
})
function clampInt(value: number, min: number, max: number) {
const rounded = Math.trunc(value)
if (!Number.isFinite(rounded)) return min
+137 -1
View File
@@ -3,6 +3,8 @@ import { defineSchema, defineTable } from 'convex/server'
import { v } from 'convex/values'
import { EMBEDDING_DIMENSIONS } from './lib/embeddings'
const authSchema = authTables as unknown as Record<string, ReturnType<typeof defineTable>>
const users = defineTable({
name: v.optional(v.string()),
image: v.optional(v.string()),
@@ -47,8 +49,41 @@ const skills = defineTable({
at: v.number(),
}),
),
highlighted: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
official: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
deprecated: v.optional(
v.object({
byUserId: v.id('users'),
at: v.number(),
}),
),
}),
moderationStatus: v.optional(
v.union(v.literal('active'), v.literal('hidden'), v.literal('removed')),
),
moderationNotes: v.optional(v.string()),
moderationReason: v.optional(v.string()),
moderationFlags: v.optional(v.array(v.string())),
lastReviewedAt: v.optional(v.number()),
hiddenAt: v.optional(v.number()),
hiddenBy: v.optional(v.id('users')),
reportCount: v.optional(v.number()),
lastReportedAt: v.optional(v.number()),
batch: v.optional(v.string()),
statsDownloads: v.optional(v.number()),
statsStars: v.optional(v.number()),
statsInstallsCurrent: v.optional(v.number()),
statsInstallsAllTime: v.optional(v.number()),
stats: v.object({
downloads: v.number(),
installsCurrent: v.optional(v.number()),
@@ -63,7 +98,12 @@ const skills = defineTable({
.index('by_slug', ['slug'])
.index('by_owner', ['ownerUserId'])
.index('by_updated', ['updatedAt'])
.index('by_stats_downloads', ['statsDownloads', 'updatedAt'])
.index('by_stats_stars', ['statsStars', 'updatedAt'])
.index('by_stats_installs_current', ['statsInstallsCurrent', 'updatedAt'])
.index('by_stats_installs_all_time', ['statsInstallsAllTime', 'updatedAt'])
.index('by_batch', ['batch'])
.index('by_active_updated', ['softDeletedAt', 'updatedAt'])
const souls = defineTable({
slug: v.string(),
@@ -105,6 +145,7 @@ const skillVersions = defineTable({
frontmatter: v.record(v.string(), v.any()),
metadata: v.optional(v.any()),
clawdis: v.optional(v.any()),
moltbot: v.optional(v.any()),
}),
createdBy: v.id('users'),
createdAt: v.number(),
@@ -131,6 +172,8 @@ const soulVersions = defineTable({
parsed: v.object({
frontmatter: v.record(v.string(), v.any()),
metadata: v.optional(v.any()),
clawdis: v.optional(v.any()),
moltbot: v.optional(v.any()),
}),
createdBy: v.id('users'),
createdAt: v.number(),
@@ -149,6 +192,21 @@ const skillVersionFingerprints = defineTable({
.index('by_fingerprint', ['fingerprint'])
.index('by_skill_fingerprint', ['skillId', 'fingerprint'])
const skillBadges = defineTable({
skillId: v.id('skills'),
kind: v.union(
v.literal('highlighted'),
v.literal('official'),
v.literal('deprecated'),
v.literal('redactionApproved'),
),
byUserId: v.id('users'),
at: v.number(),
})
.index('by_skill', ['skillId'])
.index('by_skill_kind', ['skillId', 'kind'])
.index('by_kind_at', ['kind', 'at'])
const soulVersionFingerprints = defineTable({
soulId: v.id('souls'),
versionId: v.id('soulVersions'),
@@ -177,6 +235,67 @@ const skillEmbeddings = defineTable({
filterFields: ['visibility'],
})
const skillDailyStats = defineTable({
skillId: v.id('skills'),
day: v.number(),
downloads: v.number(),
installs: v.number(),
updatedAt: v.number(),
})
.index('by_skill_day', ['skillId', 'day'])
.index('by_day', ['day'])
const skillLeaderboards = defineTable({
kind: v.string(),
generatedAt: v.number(),
rangeStartDay: v.number(),
rangeEndDay: v.number(),
items: v.array(
v.object({
skillId: v.id('skills'),
score: v.number(),
installs: v.number(),
downloads: v.number(),
}),
),
}).index('by_kind', ['kind', 'generatedAt'])
const skillStatBackfillState = defineTable({
key: v.string(),
cursor: v.optional(v.string()),
doneAt: v.optional(v.number()),
updatedAt: v.number(),
}).index('by_key', ['key'])
const skillStatEvents = defineTable({
skillId: v.id('skills'),
kind: v.union(
v.literal('download'),
v.literal('star'),
v.literal('unstar'),
v.literal('install_new'),
v.literal('install_reactivate'),
v.literal('install_deactivate'),
v.literal('install_clear'),
),
delta: v.optional(
v.object({
allTime: v.number(),
current: v.number(),
}),
),
occurredAt: v.number(),
processedAt: v.optional(v.number()),
})
.index('by_unprocessed', ['processedAt'])
.index('by_skill', ['skillId'])
const skillStatUpdateCursors = defineTable({
key: v.string(),
cursorCreationTime: v.optional(v.number()),
updatedAt: v.number(),
}).index('by_key', ['key'])
const soulEmbeddings = defineTable({
soulId: v.id('souls'),
versionId: v.id('soulVersions'),
@@ -206,6 +325,16 @@ const comments = defineTable({
.index('by_skill', ['skillId'])
.index('by_user', ['userId'])
const skillReports = defineTable({
skillId: v.id('skills'),
userId: v.id('users'),
reason: v.optional(v.string()),
createdAt: v.number(),
})
.index('by_skill', ['skillId'])
.index('by_user', ['userId'])
.index('by_skill_user', ['skillId', 'userId'])
const soulComments = defineTable({
soulId: v.id('souls'),
userId: v.id('users'),
@@ -313,17 +442,24 @@ const userSkillRootInstalls = defineTable({
.index('by_skill', ['skillId'])
export default defineSchema({
...authTables,
...authSchema,
users,
skills,
souls,
skillVersions,
soulVersions,
skillVersionFingerprints,
skillBadges,
soulVersionFingerprints,
skillEmbeddings,
soulEmbeddings,
skillDailyStats,
skillLeaderboards,
skillStatBackfillState,
skillStatEvents,
skillStatUpdateCursors,
comments,
skillReports,
soulComments,
stars,
soulStars,
+12
View File
@@ -0,0 +1,12 @@
/* @vitest-environment node */
import { describe, expect, it } from 'vitest'
import { __test } from './search'
describe('search helpers', () => {
it('advances candidate limit until max', () => {
expect(__test.getNextCandidateLimit(50, 1000)).toBe(100)
expect(__test.getNextCandidateLimit(800, 1000)).toBe(1000)
expect(__test.getNextCandidateLimit(1000, 1000)).toBeNull()
})
})
+169 -41
View File
@@ -2,16 +2,25 @@ import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import { action, internalQuery } from './_generated/server'
import { getSkillBadgeMaps, isSkillHighlighted, type SkillBadgeMap } from './lib/badges'
import { generateEmbedding } from './lib/embeddings'
import { toPublicSkill, toPublicSoul } from './lib/public'
import { matchesExactTokens, tokenize } from './lib/searchText'
type HydratedEntry = {
embeddingId: Id<'skillEmbeddings'>
skill: Doc<'skills'> | null
skill: NonNullable<ReturnType<typeof toPublicSkill>>
version: Doc<'skillVersions'> | null
ownerHandle: string | null
}
type SearchResult = HydratedEntry & { score: number }
function getNextCandidateLimit(current: number, max: number) {
const next = Math.min(current * 2, max)
return next > current ? next : null
}
export const searchSkills: ReturnType<typeof action> = action({
args: {
query: v.string(),
@@ -21,55 +30,127 @@ export const searchSkills: ReturnType<typeof action> = action({
handler: async (ctx, args): Promise<SearchResult[]> => {
const query = args.query.trim()
if (!query) return []
const vector = await generateEmbedding(query)
const results = await ctx.vectorSearch('skillEmbeddings', 'by_embedding', {
vector,
limit: args.limit ?? 10,
filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
})
const queryTokens = tokenize(query)
if (queryTokens.length === 0) return []
let vector: number[]
try {
vector = await generateEmbedding(query)
} catch (error) {
console.warn('Search embedding generation failed', error)
return []
}
const limit = args.limit ?? 10
// Convex vectorSearch max limit is 256; clamp candidate sizes accordingly.
const maxCandidate = Math.min(Math.max(limit * 10, 200), 256)
let candidateLimit = Math.min(Math.max(limit * 3, 50), 256)
let hydrated: HydratedEntry[] = []
let scoreById = new Map<Id<'skillEmbeddings'>, number>()
let exactMatches: HydratedEntry[] = []
const hydrated = (await ctx.runQuery(internal.search.hydrateResults, {
embeddingIds: results.map((result) => result._id),
})) as HydratedEntry[]
while (candidateLimit <= maxCandidate) {
const results = await ctx.vectorSearch('skillEmbeddings', 'by_embedding', {
vector,
limit: candidateLimit,
filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
})
const scoreById = new Map<Id<'skillEmbeddings'>, number>(
results.map((result) => [result._id, result._score]),
)
hydrated = (await ctx.runQuery(internal.search.hydrateResults, {
embeddingIds: results.map((result) => result._id),
})) as HydratedEntry[]
const filtered = args.highlightedOnly
? hydrated.filter((entry) => entry.skill?.batch === 'highlighted')
: hydrated
scoreById = new Map<Id<'skillEmbeddings'>, number>(
results.map((result) => [result._id, result._score]),
)
return filtered
const badgeMapEntries = (await ctx.runQuery(internal.search.getSkillBadgeMapsInternal, {
skillIds: hydrated.map((entry) => entry.skill._id),
})) as Array<[Id<'skills'>, SkillBadgeMap]>
const badgeMapBySkillId = new Map(badgeMapEntries)
const hydratedWithBadges = hydrated.map((entry) => ({
...entry,
skill: {
...entry.skill,
badges: badgeMapBySkillId.get(entry.skill._id) ?? {},
},
}))
const filtered = args.highlightedOnly
? hydratedWithBadges.filter((entry) => isSkillHighlighted(entry.skill))
: hydratedWithBadges
exactMatches = filtered.filter((entry) =>
matchesExactTokens(queryTokens, [
entry.skill.displayName,
entry.skill.slug,
entry.skill.summary,
]),
)
if (exactMatches.length >= limit || results.length < candidateLimit) {
break
}
const nextLimit = getNextCandidateLimit(candidateLimit, maxCandidate)
if (!nextLimit) break
candidateLimit = nextLimit
}
return exactMatches
.map((entry) => ({
...entry,
score: scoreById.get(entry.embeddingId) ?? 0,
}))
.filter((entry) => entry.skill)
.slice(0, limit)
},
})
export const getBadgeMapsForSkills = internalQuery({
args: { skillIds: v.array(v.id('skills')) },
handler: async (ctx, args): Promise<Array<[Id<'skills'>, SkillBadgeMap]>> => {
const badgeMap = await getSkillBadgeMaps(ctx, args.skillIds)
return Array.from(badgeMap.entries())
},
})
export const hydrateResults = internalQuery({
args: { embeddingIds: v.array(v.id('skillEmbeddings')) },
handler: async (ctx, args): Promise<HydratedEntry[]> => {
const entries: HydratedEntry[] = []
const ownerHandleCache = new Map<Id<'users'>, Promise<string | null>>()
for (const embeddingId of args.embeddingIds) {
const embedding = await ctx.db.get(embeddingId)
if (!embedding) continue
const skill = await ctx.db.get(embedding.skillId)
if (skill?.softDeletedAt) continue
const version = await ctx.db.get(embedding.versionId)
entries.push({ embeddingId, skill, version })
const getOwnerHandle = (ownerUserId: Id<'users'>) => {
const cached = ownerHandleCache.get(ownerUserId)
if (cached) return cached
const handlePromise = ctx.db
.get(ownerUserId)
.then((owner) => owner?.handle ?? owner?._id ?? null)
ownerHandleCache.set(ownerUserId, handlePromise)
return handlePromise
}
return entries
const entries = await Promise.all(
args.embeddingIds.map(async (embeddingId) => {
const embedding = await ctx.db.get(embeddingId)
if (!embedding) return null
const skill = await ctx.db.get(embedding.skillId)
if (!skill || skill.softDeletedAt) return null
const [version, ownerHandle] = await Promise.all([
ctx.db.get(embedding.versionId),
getOwnerHandle(skill.ownerUserId),
])
const publicSkill = toPublicSkill(skill)
if (!publicSkill) return null
return { embeddingId, skill: publicSkill, version, ownerHandle }
}),
)
return entries.filter((entry): entry is HydratedEntry => entry !== null)
},
})
type HydratedSoulEntry = {
embeddingId: Id<'soulEmbeddings'>
soul: Doc<'souls'> | null
soul: NonNullable<ReturnType<typeof toPublicSoul>>
version: Doc<'soulVersions'> | null
}
@@ -83,27 +164,62 @@ export const searchSouls: ReturnType<typeof action> = action({
handler: async (ctx, args): Promise<SoulSearchResult[]> => {
const query = args.query.trim()
if (!query) return []
const vector = await generateEmbedding(query)
const results = await ctx.vectorSearch('soulEmbeddings', 'by_embedding', {
vector,
limit: args.limit ?? 10,
filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
})
const queryTokens = tokenize(query)
if (queryTokens.length === 0) return []
let vector: number[]
try {
vector = await generateEmbedding(query)
} catch (error) {
console.warn('Search embedding generation failed', error)
return []
}
const limit = args.limit ?? 10
// Convex vectorSearch max limit is 256; clamp candidate sizes accordingly.
const maxCandidate = Math.min(Math.max(limit * 10, 200), 256)
let candidateLimit = Math.min(Math.max(limit * 3, 50), 256)
let hydrated: HydratedSoulEntry[] = []
let scoreById = new Map<Id<'soulEmbeddings'>, number>()
let exactMatches: HydratedSoulEntry[] = []
const hydrated = (await ctx.runQuery(internal.search.hydrateSoulResults, {
embeddingIds: results.map((result) => result._id),
})) as HydratedSoulEntry[]
while (candidateLimit <= maxCandidate) {
const results = await ctx.vectorSearch('soulEmbeddings', 'by_embedding', {
vector,
limit: candidateLimit,
filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
})
const scoreById = new Map<Id<'soulEmbeddings'>, number>(
results.map((result) => [result._id, result._score]),
)
hydrated = (await ctx.runQuery(internal.search.hydrateSoulResults, {
embeddingIds: results.map((result) => result._id),
})) as HydratedSoulEntry[]
return hydrated
scoreById = new Map<Id<'soulEmbeddings'>, number>(
results.map((result) => [result._id, result._score]),
)
exactMatches = hydrated.filter((entry) =>
matchesExactTokens(queryTokens, [
entry.soul.displayName,
entry.soul.slug,
entry.soul.summary,
]),
)
if (exactMatches.length >= limit || results.length < candidateLimit) {
break
}
const nextLimit = getNextCandidateLimit(candidateLimit, maxCandidate)
if (!nextLimit) break
candidateLimit = nextLimit
}
return exactMatches
.map((entry) => ({
...entry,
score: scoreById.get(entry.embeddingId) ?? 0,
}))
.filter((entry) => entry.soul)
.slice(0, limit)
},
})
@@ -118,9 +234,21 @@ export const hydrateSoulResults = internalQuery({
const soul = await ctx.db.get(embedding.soulId)
if (soul?.softDeletedAt) continue
const version = await ctx.db.get(embedding.versionId)
entries.push({ embeddingId, soul, version })
const publicSoul = toPublicSoul(soul)
if (!publicSoul) continue
entries.push({ embeddingId, soul: publicSoul, version })
}
return entries
},
})
export const getSkillBadgeMapsInternal = internalQuery({
args: { skillIds: v.array(v.id('skills')) },
handler: async (ctx, args) => {
const badgeMap = await getSkillBadgeMaps(ctx, args.skillIds)
return Array.from(badgeMap.entries())
},
})
export const __test = { getNextCandidateLimit }
+2 -1
View File
@@ -242,7 +242,8 @@ export const ensureSeedUserInternal = internalMutation({
})
async function sha256Hex(bytes: Uint8Array) {
const digest = await crypto.subtle.digest('SHA-256', bytes)
const data = new Uint8Array(bytes)
const digest = await crypto.subtle.digest('SHA-256', data)
return toHex(new Uint8Array(digest))
}
+3 -3
View File
File diff suppressed because one or more lines are too long
+568
View File
@@ -0,0 +1,568 @@
/**
* Skill Stat Events - Event-sourced stats processing for skills
*
* Instead of updating skill stats synchronously in the hot path (which can cause
* contention when multiple users download/star/install the same skill), we insert
* lightweight event records and process them in batches via a cron job.
*
* Flow:
* 1. User action (download, star, install) → insertStatEvent() writes to skillStatEvents table
* 2. Cron job runs every 5 minutes → processSkillStatEventsInternal() processes batches
* 3. Events are aggregated per-skill to minimize database operations
* 4. Stats are applied to skill documents and daily stats tables
* 5. Events are marked as processed (kept forever for auditing)
*/
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import type { MutationCtx } from './_generated/server'
import { internalAction, internalMutation, internalQuery } from './_generated/server'
import { applySkillStatDeltas, bumpDailySkillStats } from './lib/skillStats'
/**
* Event types that affect skill stats:
*
* - download: User downloaded skill as zip (+1 downloads)
* - star: User starred the skill (+1 stars)
* - unstar: User removed their star (-1 stars)
* - install_new: First time this user installed this skill (+1 installsAllTime, +1 installsCurrent)
* - install_reactivate: User re-added skill after removing it (+1 installsCurrent only)
* - install_deactivate: User removed skill from all projects (-1 installsCurrent)
* - install_clear: User cleared all telemetry data (custom delta for both allTime and current)
*/
export type StatEventKind =
| 'download'
| 'star'
| 'unstar'
| 'install_new'
| 'install_reactivate'
| 'install_deactivate'
| 'install_clear'
/**
* Insert a stat event to be processed later by the cron job.
*
* This is called from the hot path (downloads, stars, telemetry) instead of
* directly updating skill stats. It's a single insert with no read-modify-write
* cycle, so it's fast and doesn't contend with other operations on the same skill.
*
* @param ctx - Mutation context
* @param params.skillId - The skill being affected
* @param params.kind - Type of event (download, star, install_new, etc.)
* @param params.occurredAt - When the event happened (defaults to now). Important for
* daily stats bucketing - we want downloads at 11:55 PM Monday
* to count toward Monday's stats even if processed on Tuesday.
* @param params.delta - Only used for install_clear events, specifies exact delta amounts
*/
export async function insertStatEvent(
ctx: MutationCtx,
params: {
skillId: Id<'skills'>
kind: StatEventKind
occurredAt?: number
delta?: { allTime: number; current: number }
},
) {
await ctx.db.insert('skillStatEvents', {
skillId: params.skillId,
kind: params.kind,
delta: params.delta,
occurredAt: params.occurredAt ?? Date.now(),
processedAt: undefined,
})
}
/**
* Aggregated deltas for a single skill after processing multiple events.
*
* When we process a batch of 100 events, many might be for the same skill.
* Instead of updating the skill document once per event, we aggregate all
* events for each skill and apply a single update.
*
* The downloadEvents and installNewEvents arrays store the original timestamps
* so we can update daily stats with the correct day bucket for each event.
*/
type AggregatedDeltas = {
downloads: number
stars: number
installsAllTime: number
installsCurrent: number
/** Original timestamps for each download event (for daily stats bucketing) */
downloadEvents: number[]
/** Original timestamps for each new install event (for daily stats bucketing) */
installNewEvents: number[]
}
/**
* Aggregate multiple events for a single skill into net deltas.
*
* Example: If a skill has these events in the batch:
* - download (Mon 11pm)
* - download (Tue 1am)
* - star
* - unstar
* - star
*
* The result would be:
* - downloads: 2
* - stars: 1 (net: +1 -1 +1 = +1)
* - downloadEvents: [<Mon 11pm timestamp>, <Tue 1am timestamp>]
*
* This aggregation reduces the number of database operations from N events
* to 1 skill update + N daily stat updates (which themselves may coalesce
* if multiple events fall on the same day).
*/
function aggregateEvents(events: Doc<'skillStatEvents'>[]): AggregatedDeltas {
const result: AggregatedDeltas = {
downloads: 0,
stars: 0,
installsAllTime: 0,
installsCurrent: 0,
downloadEvents: [],
installNewEvents: [],
}
for (const event of events) {
switch (event.kind) {
case 'download':
result.downloads += 1
result.downloadEvents.push(event.occurredAt)
break
case 'star':
result.stars += 1
break
case 'unstar':
result.stars -= 1
break
case 'install_new':
// New user installing for the first time: count toward both lifetime and current
result.installsAllTime += 1
result.installsCurrent += 1
result.installNewEvents.push(event.occurredAt)
break
case 'install_reactivate':
// User re-added skill after removing: only affects current count
result.installsCurrent += 1
break
case 'install_deactivate':
// User removed skill from all projects: only affects current count
result.installsCurrent -= 1
break
case 'install_clear':
// User cleared telemetry: uses custom delta values (typically negative)
if (event.delta) {
result.installsAllTime += event.delta.allTime
result.installsCurrent += event.delta.current
}
break
}
}
return result
}
/**
* Process a batch of unprocessed stat events.
*
* Called by cron every 5 minutes. Processes up to batchSize events (default 100).
* If the batch is full, schedules an immediate follow-up run to drain the queue.
*
* Processing steps:
* 1. Query unprocessed events (processedAt is undefined)
* 2. Group events by skillId to minimize skill document fetches
* 3. For each skill:
* a. Fetch the skill document once
* b. Aggregate all events for this skill into net deltas
* c. Apply deltas to skill stats (downloads, stars, installs)
* d. Update daily stats for trending (using original event timestamps)
* e. Mark all events as processed
* 4. If batch was full, schedule another run immediately
*
* Aggregation levels:
* - Level 1: Batch of 100 events from the queue
* - Level 2: Group by skillId (e.g., 100 events → 30 unique skills)
* - Level 3: Aggregate events per skill (e.g., 5 events → 1 skill update)
* - Level 4: Daily stats may coalesce (e.g., 3 downloads same day → 1 upsert)
*/
export const processSkillStatEventsInternal = internalMutation({
args: { batchSize: v.optional(v.number()) },
handler: async (ctx, args) => {
const batchSize = args.batchSize ?? 100
const now = Date.now()
// Level 1: Fetch a batch of unprocessed events
const events = await ctx.db
.query('skillStatEvents')
.withIndex('by_unprocessed', (q) => q.eq('processedAt', undefined))
.take(batchSize)
if (events.length === 0) {
return { processed: 0 }
}
// Level 2: Group events by skillId to minimize database reads
// Instead of fetching the same skill document multiple times,
// we fetch it once and process all its events together
const eventsBySkill = new Map<Id<'skills'>, Doc<'skillStatEvents'>[]>()
for (const event of events) {
const existing = eventsBySkill.get(event.skillId) ?? []
existing.push(event)
eventsBySkill.set(event.skillId, existing)
}
// Process each skill's events
for (const [skillId, skillEvents] of eventsBySkill) {
const skill = await ctx.db.get(skillId)
// Skill was deleted - just mark events as processed
if (!skill) {
for (const event of skillEvents) {
await ctx.db.patch(event._id, { processedAt: now })
}
continue
}
// Level 3: Aggregate all events for this skill into net deltas
// e.g., 3 downloads + 2 stars - 1 unstar → { downloads: 3, stars: 1 }
const deltas = aggregateEvents(skillEvents)
// Apply aggregated deltas to skill stats (single update per skill)
if (
deltas.downloads !== 0 ||
deltas.stars !== 0 ||
deltas.installsAllTime !== 0 ||
deltas.installsCurrent !== 0
) {
const patch = applySkillStatDeltas(skill, {
downloads: deltas.downloads,
stars: deltas.stars,
installsAllTime: deltas.installsAllTime,
installsCurrent: deltas.installsCurrent,
})
await ctx.db.patch(skill._id, {
...patch,
updatedAt: now,
})
}
// Update daily stats for trending/leaderboards
// We use the ORIGINAL event timestamp (occurredAt) so that:
// - A download at Mon 11:55 PM counts toward Monday's stats
// - Even if the cron processes it on Tuesday
//
// Level 4: bumpDailySkillStats does its own coalescing - multiple
// events on the same day will update the same daily record
for (const occurredAt of deltas.downloadEvents) {
await bumpDailySkillStats(ctx, { skillId, now: occurredAt, downloads: 1 })
}
for (const occurredAt of deltas.installNewEvents) {
await bumpDailySkillStats(ctx, { skillId, now: occurredAt, installs: 1 })
}
// Mark all events for this skill as processed
for (const event of skillEvents) {
await ctx.db.patch(event._id, { processedAt: now })
}
}
// If we hit the batch limit, there may be more events waiting.
// Schedule an immediate follow-up run to drain the queue.
// This ensures high-volume periods don't create a backlog.
if (events.length === batchSize) {
await ctx.scheduler.runAfter(0, internal.skillStatEvents.processSkillStatEventsInternal, {
batchSize,
})
}
return { processed: events.length }
},
})
// ============================================================================
// Action-based processing (cursor-based, runs outside transaction window)
// ============================================================================
const CURSOR_KEY = 'skill_stat_events'
const EVENT_BATCH_SIZE = 500
const MAX_SKILLS_PER_RUN = 500
/**
* Fetch a batch of events after the given cursor (by _creationTime).
* Returns events sorted by _creationTime ascending.
*/
export const getUnprocessedEventBatch = internalQuery({
args: {
cursorCreationTime: v.optional(v.number()),
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const limit = args.limit ?? EVENT_BATCH_SIZE
const cursor = args.cursorCreationTime
// Query events after the cursor using the built-in creation time index
const events = await ctx.db
.query('skillStatEvents')
.withIndex('by_creation_time', (q) =>
cursor !== undefined ? q.gt('_creationTime', cursor) : q,
)
.take(limit)
return events
},
})
/**
* Get the current cursor position from the cursors table.
*/
export const getStatEventCursor = internalQuery({
args: {},
handler: async (ctx) => {
const cursor = await ctx.db
.query('skillStatUpdateCursors')
.withIndex('by_key', (q) => q.eq('key', CURSOR_KEY))
.unique()
return cursor?.cursorCreationTime
},
})
/**
* Validator for skill deltas passed to the mutation.
*/
const skillDeltaValidator = v.object({
skillId: v.id('skills'),
downloads: v.number(),
stars: v.number(),
installsAllTime: v.number(),
installsCurrent: v.number(),
downloadEvents: v.array(v.number()),
installNewEvents: v.array(v.number()),
})
/**
* Apply aggregated stats to skills and update the cursor.
* This is a single atomic mutation that:
* 1. Updates all affected skills with their aggregated deltas
* 2. Updates daily stats for trending
* 3. Advances the cursor to the new position
*/
export const applyAggregatedStatsAndUpdateCursor = internalMutation({
args: {
skillDeltas: v.array(skillDeltaValidator),
newCursor: v.number(),
},
handler: async (ctx, args) => {
const now = Date.now()
// Process each skill's aggregated deltas
for (const delta of args.skillDeltas) {
const skill = await ctx.db.get(delta.skillId)
// Skill was deleted - skip
if (!skill) {
continue
}
// Apply aggregated deltas to skill stats
if (
delta.downloads !== 0 ||
delta.stars !== 0 ||
delta.installsAllTime !== 0 ||
delta.installsCurrent !== 0
) {
const patch = applySkillStatDeltas(skill, {
downloads: delta.downloads,
stars: delta.stars,
installsAllTime: delta.installsAllTime,
installsCurrent: delta.installsCurrent,
})
await ctx.db.patch(skill._id, {
...patch,
updatedAt: now,
})
}
// Update daily stats for trending/leaderboards
for (const occurredAt of delta.downloadEvents) {
await bumpDailySkillStats(ctx, { skillId: delta.skillId, now: occurredAt, downloads: 1 })
}
for (const occurredAt of delta.installNewEvents) {
await bumpDailySkillStats(ctx, { skillId: delta.skillId, now: occurredAt, installs: 1 })
}
}
// Update cursor position (upsert)
const existingCursor = await ctx.db
.query('skillStatUpdateCursors')
.withIndex('by_key', (q) => q.eq('key', CURSOR_KEY))
.unique()
if (existingCursor) {
await ctx.db.patch(existingCursor._id, {
cursorCreationTime: args.newCursor,
updatedAt: now,
})
} else {
await ctx.db.insert('skillStatUpdateCursors', {
key: CURSOR_KEY,
cursorCreationTime: args.newCursor,
updatedAt: now,
})
}
return { skillsUpdated: args.skillDeltas.length }
},
})
/**
* Action that processes skill stat events in batches outside the transaction window.
*
* Algorithm:
* 1. Get current cursor position
* 2. Fetch events in batches of 500, aggregating as we go
* 3. Stop when we have >= 500 unique skills OR run out of events
* 4. Call mutation to apply all deltas and update cursor atomically
* 5. Self-schedule if we stopped due to skill limit (not exhaustion)
*/
export const processSkillStatEventsAction = internalAction({
args: {},
handler: async (ctx) => {
// Get current cursor position (convert null to undefined for consistency)
const cursorResult = await ctx.runQuery(internal.skillStatEvents.getStatEventCursor)
let cursor: number | undefined = cursorResult ?? undefined
console.log(`[STAT-AGG] Starting aggregation, cursor=${cursor ?? 'none'}`)
// Aggregated deltas per skill
const aggregatedBySkill = new Map<
Id<'skills'>,
{
downloads: number
stars: number
installsAllTime: number
installsCurrent: number
downloadEvents: number[]
installNewEvents: number[]
}
>()
let maxCreationTime: number | undefined = cursor
let exhausted = false
let totalEventsFetched = 0
// Fetch and aggregate until we have enough skills or run out of events
while (aggregatedBySkill.size < MAX_SKILLS_PER_RUN) {
const events = await ctx.runQuery(internal.skillStatEvents.getUnprocessedEventBatch, {
cursorCreationTime: cursor,
limit: EVENT_BATCH_SIZE,
})
if (events.length === 0) {
exhausted = true
break
}
totalEventsFetched += events.length
const skillsBefore = aggregatedBySkill.size
// Aggregate events into per-skill deltas
for (const event of events) {
let skillDelta = aggregatedBySkill.get(event.skillId)
if (!skillDelta) {
skillDelta = {
downloads: 0,
stars: 0,
installsAllTime: 0,
installsCurrent: 0,
downloadEvents: [],
installNewEvents: [],
}
aggregatedBySkill.set(event.skillId, skillDelta)
}
// Apply event to aggregated deltas
switch (event.kind) {
case 'download':
skillDelta.downloads += 1
skillDelta.downloadEvents.push(event.occurredAt)
break
case 'star':
skillDelta.stars += 1
break
case 'unstar':
skillDelta.stars -= 1
break
case 'install_new':
skillDelta.installsAllTime += 1
skillDelta.installsCurrent += 1
skillDelta.installNewEvents.push(event.occurredAt)
break
case 'install_reactivate':
skillDelta.installsCurrent += 1
break
case 'install_deactivate':
skillDelta.installsCurrent -= 1
break
case 'install_clear':
if (event.delta) {
skillDelta.installsAllTime += event.delta.allTime
skillDelta.installsCurrent += event.delta.current
}
break
}
// Track highest _creationTime seen
if (maxCreationTime === undefined || event._creationTime > maxCreationTime) {
maxCreationTime = event._creationTime
}
}
// Update cursor for next batch fetch
cursor = events[events.length - 1]._creationTime
console.log(
`[STAT-AGG] Fetched ${events.length} events, ${aggregatedBySkill.size - skillsBefore} new skills (${aggregatedBySkill.size} total)`,
)
// If we got fewer than requested, we've exhausted the events
if (events.length < EVENT_BATCH_SIZE) {
exhausted = true
break
}
}
// If we have nothing to process, we're done
if (aggregatedBySkill.size === 0 || maxCreationTime === undefined) {
console.log('[STAT-AGG] No events to process, done')
return { processed: 0, skillsUpdated: 0, exhausted: true }
}
// Convert map to array for mutation
const skillDeltas = Array.from(aggregatedBySkill.entries()).map(([skillId, delta]) => ({
skillId,
...delta,
}))
console.log(
`[STAT-AGG] Running mutation for ${skillDeltas.length} skills (${totalEventsFetched} total events)`,
)
// Apply all deltas and update cursor atomically
await ctx.runMutation(internal.skillStatEvents.applyAggregatedStatsAndUpdateCursor, {
skillDeltas,
newCursor: maxCreationTime,
})
// Self-schedule if we stopped because of skill limit, not exhaustion
if (!exhausted) {
console.log('[STAT-AGG] More events remaining, self-scheduling')
await ctx.scheduler.runAfter(0, internal.skillStatEvents.processSkillStatEventsAction, {})
} else {
console.log('[STAT-AGG] All events processed, done')
}
return {
skillsUpdated: skillDeltas.length,
exhausted,
}
},
})
+896 -54
View File
File diff suppressed because it is too large Load Diff
+5 -4
View File
@@ -1,7 +1,8 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { assertRole, requireUser } from './lib/access'
import { assertModerator, requireUser } from './lib/access'
import { type PublicUser, toPublicUser } from './lib/public'
export const listBySoul = query({
args: { soulId: v.id('souls'), limit: v.optional(v.number()) },
@@ -13,10 +14,10 @@ export const listBySoul = query({
.order('desc')
.take(limit)
const results: Array<{ comment: Doc<'soulComments'>; user: Doc<'users'> | null }> = []
const results: Array<{ comment: Doc<'soulComments'>; user: PublicUser | null }> = []
for (const comment of comments) {
if (comment.softDeletedAt) continue
const user = await ctx.db.get(comment.userId)
const user = toPublicUser(await ctx.db.get(comment.userId))
results.push({ comment, user })
}
return results
@@ -59,7 +60,7 @@ export const remove = mutation({
const isOwner = comment.userId === user._id
if (!isOwner) {
assertRole(user, ['admin', 'moderator'])
assertModerator(user)
}
await ctx.db.patch(comment._id, {
+5 -3
View File
@@ -1,7 +1,7 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { requireUser } from './lib/access'
import { toPublicSoul } from './lib/public'
export const isStarred = query({
args: { soulId: v.id('souls') },
@@ -59,10 +59,12 @@ export const listByUser = query({
.order('desc')
.take(limit)
const souls: Doc<'souls'>[] = []
const souls: NonNullable<ReturnType<typeof toPublicSoul>>[] = []
for (const star of stars) {
const soul = await ctx.db.get(star.soulId)
if (soul) souls.push(soul)
const publicSoul = toPublicSoul(soul)
if (!publicSoul) continue
souls.push(publicSoul)
}
return souls
},
+26 -10
View File
@@ -2,7 +2,8 @@ import { ConvexError, v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import { action, internalMutation, internalQuery, mutation, query } from './_generated/server'
import { assertRole, requireUser, requireUserFromAction } from './lib/access'
import { assertModerator, requireUser, requireUserFromAction } from './lib/access'
import { toPublicSoul, toPublicUser } from './lib/public'
import { getFrontmatterValue, hashSkillFiles } from './lib/skills'
import { generateSoulChangelogPreview } from './lib/soulChangelog'
import { fetchText, type PublishResult, publishSoulVersionForUser } from './lib/soulPublish'
@@ -27,9 +28,11 @@ export const getBySlug = query({
const soul = matches[0] ?? null
if (!soul || soul.softDeletedAt) return null
const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
const owner = await ctx.db.get(soul.ownerUserId)
const owner = toPublicUser(await ctx.db.get(soul.ownerUserId))
const publicSoul = toPublicSoul(soul)
if (!publicSoul) return null
return { soul, latestVersion, owner }
return { soul: publicSoul, latestVersion, owner }
},
})
@@ -59,13 +62,21 @@ export const list = query({
.withIndex('by_owner', (q) => q.eq('ownerUserId', ownerUserId))
.order('desc')
.take(limit * 5)
return entries.filter((soul) => !soul.softDeletedAt).slice(0, limit)
return entries
.filter((soul) => !soul.softDeletedAt)
.slice(0, limit)
.map((soul) => toPublicSoul(soul))
.filter((soul): soul is NonNullable<typeof soul> => Boolean(soul))
}
const entries = await ctx.db
.query('souls')
.order('desc')
.take(limit * 5)
return entries.filter((soul) => !soul.softDeletedAt).slice(0, limit)
return entries
.filter((soul) => !soul.softDeletedAt)
.slice(0, limit)
.map((soul) => toPublicSoul(soul))
.filter((soul): soul is NonNullable<typeof soul> => Boolean(soul))
},
})
@@ -82,12 +93,17 @@ export const listPublicPage = query({
.order('desc')
.paginate({ cursor: args.cursor ?? null, numItems: limit })
const items: Array<{ soul: Doc<'souls'>; latestVersion: Doc<'soulVersions'> | null }> = []
const items: Array<{
soul: NonNullable<ReturnType<typeof toPublicSoul>>
latestVersion: Doc<'soulVersions'> | null
}> = []
for (const soul of page) {
if (soul.softDeletedAt) continue
const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
items.push({ soul, latestVersion })
const publicSoul = toPublicSoul(soul)
if (!publicSoul) continue
items.push({ soul: publicSoul, latestVersion })
}
return { items, nextCursor: isDone ? null : continueCursor }
@@ -309,7 +325,7 @@ export const updateTags = mutation({
const soul = await ctx.db.get(args.soulId)
if (!soul) throw new Error('Soul not found')
if (soul.ownerUserId !== user._id) {
assertRole(user, ['admin', 'moderator'])
assertModerator(user)
}
const nextTags = { ...soul.tags }
@@ -377,7 +393,7 @@ export const insertVersion = internalMutation({
.withIndex('by_slug', (q) => q.eq('slug', args.slug))
.order('desc')
.take(2)
let soul = soulMatches[0] ?? null
let soul: Doc<'souls'> | null = soulMatches[0] ?? null
if (soul && soul.ownerUserId !== userId) {
throw new Error('Only the owner can publish updates')
@@ -506,7 +522,7 @@ export const setSoulSoftDeletedInternal = internalMutation({
if (!soul) throw new Error('Soul not found')
if (soul.ownerUserId !== args.userId) {
assertRole(user, ['admin', 'moderator'])
assertModerator(user)
}
const now = Date.now()
+50 -12
View File
@@ -1,7 +1,8 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { internalMutation, mutation, query } from './_generated/server'
import { requireUser } from './lib/access'
import { toPublicSkill } from './lib/public'
import { insertStatEvent } from './skillStatEvents'
export const isStarred = query({
args: { skillId: v.id('skills') },
@@ -29,10 +30,7 @@ export const toggle = mutation({
if (existing) {
await ctx.db.delete(existing._id)
await ctx.db.patch(skill._id, {
stats: { ...skill.stats, stars: Math.max(0, skill.stats.stars - 1) },
updatedAt: Date.now(),
})
await insertStatEvent(ctx, { skillId: skill._id, kind: 'unstar' })
return { starred: false }
}
@@ -42,10 +40,7 @@ export const toggle = mutation({
createdAt: Date.now(),
})
await ctx.db.patch(skill._id, {
stats: { ...skill.stats, stars: skill.stats.stars + 1 },
updatedAt: Date.now(),
})
await insertStatEvent(ctx, { skillId: skill._id, kind: 'star' })
return { starred: true }
},
@@ -60,11 +55,54 @@ export const listByUser = query({
.withIndex('by_user', (q) => q.eq('userId', args.userId))
.order('desc')
.take(limit)
const skills: Doc<'skills'>[] = []
const skills: NonNullable<ReturnType<typeof toPublicSkill>>[] = []
for (const star of stars) {
const skill = await ctx.db.get(star.skillId)
if (skill) skills.push(skill)
const publicSkill = toPublicSkill(skill)
if (!publicSkill) continue
skills.push(publicSkill)
}
return skills
},
})
export const addStarInternal = internalMutation({
args: { userId: v.id('users'), skillId: v.id('skills') },
handler: async (ctx, args) => {
const skill = await ctx.db.get(args.skillId)
if (!skill) throw new Error('Skill not found')
const existing = await ctx.db
.query('stars')
.withIndex('by_skill_user', (q) => q.eq('skillId', args.skillId).eq('userId', args.userId))
.unique()
if (existing) return { ok: true as const, starred: true, alreadyStarred: true }
await ctx.db.insert('stars', {
skillId: args.skillId,
userId: args.userId,
createdAt: Date.now(),
})
await insertStatEvent(ctx, { skillId: skill._id, kind: 'star' })
return { ok: true as const, starred: true, alreadyStarred: false }
},
})
export const removeStarInternal = internalMutation({
args: { userId: v.id('users'), skillId: v.id('skills') },
handler: async (ctx, args) => {
const skill = await ctx.db.get(args.skillId)
if (!skill) throw new Error('Skill not found')
const existing = await ctx.db
.query('stars')
.withIndex('by_skill_user', (q) => q.eq('skillId', args.skillId).eq('userId', args.userId))
.unique()
if (!existing) return { ok: true as const, unstarred: false, alreadyUnstarred: true }
await ctx.db.delete(existing._id)
await insertStatEvent(ctx, { skillId: skill._id, kind: 'unstar' })
return { ok: true as const, unstarred: true, alreadyUnstarred: false }
},
})
+205
View File
@@ -0,0 +1,205 @@
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
import { internalAction, internalMutation, internalQuery } from './_generated/server'
const DEFAULT_BATCH_SIZE = 200
const MAX_BATCH_SIZE = 1000
const DEFAULT_MAX_BATCHES = 5
const MAX_MAX_BATCHES = 50
const BACKFILL_STATE_KEY = 'default'
export const backfillSkillStatFieldsInternal = internalMutation({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args) => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const { page, isDone, continueCursor } = await ctx.db
.query('skills')
.order('asc')
.paginate({ cursor: args.cursor ?? null, numItems: batchSize })
let patched = 0
for (const skill of page) {
const next = buildSkillStatPatch(skill)
if (!next) continue
await ctx.db.patch(skill._id, next)
patched += 1
}
return {
ok: true as const,
scanned: page.length,
patched,
cursor: isDone ? null : continueCursor,
isDone,
}
},
})
type BackfillState = {
cursor: string | null
doneAt?: number
}
type BackfillActionArgs = {
batchSize?: number
maxBatches?: number
resetCursor?: boolean
}
type BackfillStats = {
scanned: number
patched: number
batches: number
}
type BackfillActionResult = {
ok: true
isDone: boolean
cursor: string | null
stats: BackfillStats
}
export const getSkillStatBackfillStateInternal = internalQuery({
args: {},
handler: async (ctx): Promise<BackfillState> => {
const state = await ctx.db
.query('skillStatBackfillState')
.withIndex('by_key', (q) => q.eq('key', BACKFILL_STATE_KEY))
.unique()
return { cursor: state?.cursor ?? null, doneAt: state?.doneAt }
},
})
export const setSkillStatBackfillStateInternal = internalMutation({
args: {
cursor: v.optional(v.string()),
doneAt: v.optional(v.number()),
},
handler: async (ctx, args) => {
const now = Date.now()
const state = await ctx.db
.query('skillStatBackfillState')
.withIndex('by_key', (q) => q.eq('key', BACKFILL_STATE_KEY))
.unique()
if (!state) {
await ctx.db.insert('skillStatBackfillState', {
key: BACKFILL_STATE_KEY,
cursor: args.cursor,
doneAt: args.doneAt,
updatedAt: now,
})
return { ok: true as const }
}
await ctx.db.patch(state._id, {
cursor: args.cursor,
doneAt: args.doneAt,
updatedAt: now,
})
return { ok: true as const }
},
})
async function runSkillStatBackfillInternalHandler(
ctx: ActionCtx,
args: BackfillActionArgs,
): Promise<BackfillActionResult> {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
if (args.resetCursor) {
await ctx.runMutation(internal.statsMaintenance.setSkillStatBackfillStateInternal, {
cursor: undefined,
doneAt: undefined,
})
}
const state = (await ctx.runQuery(
internal.statsMaintenance.getSkillStatBackfillStateInternal,
{},
)) as BackfillState
if (state.doneAt && !args.resetCursor) {
return {
ok: true,
isDone: true,
cursor: null,
stats: { scanned: 0, patched: 0, batches: 0 },
}
}
let cursor: string | null = state.cursor ?? null
const stats: BackfillStats = { scanned: 0, patched: 0, batches: 0 }
for (let i = 0; i < maxBatches; i += 1) {
const result = (await ctx.runMutation(
internal.statsMaintenance.backfillSkillStatFieldsInternal,
{
cursor: cursor ?? undefined,
batchSize,
},
)) as { scanned: number; patched: number; cursor: string | null; isDone: boolean }
stats.scanned += result.scanned
stats.patched += result.patched
stats.batches += 1
cursor = result.cursor
if (result.isDone) {
await ctx.runMutation(internal.statsMaintenance.setSkillStatBackfillStateInternal, {
cursor: undefined,
doneAt: Date.now(),
})
return { ok: true, isDone: true, cursor: null, stats }
}
await ctx.runMutation(internal.statsMaintenance.setSkillStatBackfillStateInternal, {
cursor: cursor ?? undefined,
doneAt: undefined,
})
}
return { ok: true, isDone: false, cursor, stats }
}
export const runSkillStatBackfillInternal: ReturnType<typeof internalAction> = internalAction({
args: {
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
resetCursor: v.optional(v.boolean()),
},
handler: runSkillStatBackfillInternalHandler,
})
function buildSkillStatPatch(skill: Doc<'skills'>) {
const stats = skill.stats
const nextDownloads = stats.downloads
const nextStars = stats.stars
const nextInstallsCurrent = stats.installsCurrent ?? 0
const nextInstallsAllTime = stats.installsAllTime ?? 0
if (
skill.statsDownloads === nextDownloads &&
skill.statsStars === nextStars &&
skill.statsInstallsCurrent === nextInstallsCurrent &&
skill.statsInstallsAllTime === nextInstallsAllTime
) {
return null
}
return {
statsDownloads: nextDownloads,
statsStars: nextStars,
statsInstallsCurrent: nextInstallsCurrent,
statsInstallsAllTime: nextInstallsAllTime,
}
}
function clampInt(value: number, min: number, max: number) {
return Math.min(Math.max(value, min), max)
}
+28 -37
View File
@@ -4,6 +4,7 @@ import type { Id } from './_generated/dataModel'
import type { MutationCtx, QueryCtx } from './_generated/server'
import { internalMutation, mutation, query } from './_generated/server'
import { requireUser } from './lib/access'
import { insertStatEvent } from './skillStatEvents'
const TELEMETRY_STALE_MS = 120 * 24 * 60 * 60 * 1000
@@ -87,7 +88,13 @@ export const getMyInstalled = query({
lastSeenAt: number
expiredAt?: number
skills: Array<{
skill: { slug: string; displayName: string; summary?: string; stats: unknown }
skill: {
slug: string
displayName: string
summary?: string
stats: unknown
ownerUserId: Id<'users'>
}
firstSeenAt: number
lastSeenAt: number
lastVersion?: string
@@ -104,7 +111,13 @@ export const getMyInstalled = query({
const filtered = includeRemoved ? installs : installs.filter((entry) => !entry.removedAt)
const skills: Array<{
skill: { slug: string; displayName: string; summary?: string; stats: unknown }
skill: {
slug: string
displayName: string
summary?: string
stats: unknown
ownerUserId: Id<'users'>
}
firstSeenAt: number
lastSeenAt: number
lastVersion?: string
@@ -120,6 +133,7 @@ export const getMyInstalled = query({
displayName: skill.displayName,
summary: skill.summary,
stats: skill.stats,
ownerUserId: skill.ownerUserId,
},
firstSeenAt: entry.firstSeenAt,
lastSeenAt: entry.lastSeenAt,
@@ -157,24 +171,13 @@ async function clearTelemetryForUser(ctx: MutationCtx, params: { userId: Id<'use
await ctx.db.delete(entry._id)
continue
}
const stats = skill.stats as {
downloads: number
installsCurrent?: number
installsAllTime?: number
stars: number
versions: number
comments: number
}
await ctx.db.patch(skill._id, {
stats: {
...stats,
installsCurrent: Math.max(
0,
(stats.installsCurrent ?? 0) - (entry.activeRoots > 0 ? 1 : 0),
),
installsAllTime: Math.max(0, (stats.installsAllTime ?? 0) - 1),
await insertStatEvent(ctx, {
skillId: skill._id,
kind: 'install_clear',
delta: {
allTime: -1,
current: entry.activeRoots > 0 ? -1 : 0,
},
updatedAt: Date.now(),
})
await ctx.db.delete(entry._id)
}
@@ -381,25 +384,13 @@ async function bumpSkillInstallCounts(
ctx: MutationCtx,
params: { skillId: Id<'skills'>; deltaAllTime: number; deltaCurrent: number },
) {
const skill = await ctx.db.get(params.skillId)
if (!skill) return
const stats = skill.stats as {
downloads: number
installsCurrent?: number
installsAllTime?: number
stars: number
versions: number
comments: number
if (params.deltaAllTime === 1 && params.deltaCurrent === 1) {
await insertStatEvent(ctx, { skillId: params.skillId, kind: 'install_new' })
} else if (params.deltaAllTime === 0 && params.deltaCurrent === 1) {
await insertStatEvent(ctx, { skillId: params.skillId, kind: 'install_reactivate' })
} else if (params.deltaAllTime === 0 && params.deltaCurrent === -1) {
await insertStatEvent(ctx, { skillId: params.skillId, kind: 'install_deactivate' })
}
await ctx.db.patch(skill._id, {
stats: {
...stats,
installsAllTime: Math.max(0, (stats.installsAllTime ?? 0) + params.deltaAllTime),
installsCurrent: Math.max(0, (stats.installsCurrent ?? 0) + params.deltaCurrent),
},
updatedAt: Date.now(),
})
}
async function expireStaleRoots(
+12 -5
View File
@@ -1,13 +1,19 @@
import { getAuthUserId } from '@convex-dev/auth/server'
import { v } from 'convex/values'
import { internal } from './_generated/api'
import { mutation, query } from './_generated/server'
import { assertRole, requireUser } from './lib/access'
import { internalQuery, mutation, query } from './_generated/server'
import { assertAdmin, requireUser } from './lib/access'
import { toPublicUser } from './lib/public'
const DEFAULT_ROLE = 'user'
const ADMIN_HANDLE = 'steipete'
export const getById = query({
args: { userId: v.id('users') },
handler: async (ctx, args) => toPublicUser(await ctx.db.get(args.userId)),
})
export const getByIdInternal = internalQuery({
args: { userId: v.id('users') },
handler: async (ctx, args) => ctx.db.get(args.userId),
})
@@ -78,7 +84,7 @@ export const list = query({
args: { limit: v.optional(v.number()) },
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
assertRole(user, ['admin'])
assertAdmin(user)
const limit = args.limit ?? 50
return ctx.db.query('users').order('desc').take(limit)
},
@@ -87,10 +93,11 @@ export const list = query({
export const getByHandle = query({
args: { handle: v.string() },
handler: async (ctx, args) => {
return ctx.db
const user = await ctx.db
.query('users')
.withIndex('handle', (q) => q.eq('handle', args.handle))
.unique()
return toPublicUser(user)
},
})
@@ -101,7 +108,7 @@ export const setRole = mutation({
},
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
assertRole(user, ['admin'])
assertAdmin(user)
await ctx.db.patch(args.userId, { role: args.role, updatedAt: Date.now() })
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
+2 -2
View File
@@ -11,7 +11,7 @@ export const sendDiscordWebhook = internalAction({
summary: v.optional(v.string()),
version: v.optional(v.string()),
ownerHandle: v.optional(v.string()),
batch: v.optional(v.string()),
highlighted: v.optional(v.boolean()),
tags: v.optional(v.array(v.string())),
}),
},
@@ -21,7 +21,7 @@ export const sendDiscordWebhook = internalAction({
event: args.event,
slug: args.skill.slug,
version: args.skill.version ?? null,
batch: args.skill.batch ?? null,
highlighted: args.skill.highlighted ?? false,
highlightedOnly: config.highlightedOnly,
}
if (!shouldSendWebhook(args.event, args.skill, config)) {
+1 -1
View File
@@ -22,7 +22,7 @@ Reading order (new contributor):
Feature/ops docs (already present):
- `docs/spec.md`: product + implementation spec (data model + flows).
- `docs/telemetry.md`: what `clawdhub sync` reports; opt-out.
- `docs/telemetry.md`: what `clawhub sync` reports; opt-out.
- `docs/webhook.md`: Discord webhook events/payload.
- `docs/diffing.md`: version-to-version diff UI spec.
- `docs/manual-testing.md`: CLI smoke scripts.
+3 -2
View File
@@ -7,7 +7,7 @@ read_when:
# API v1
Base: `https://clawdhub.com`
Base: `https://clawhub.ai`
OpenAPI: `/api/v1/openapi.json`
@@ -30,7 +30,8 @@ Headers: `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `Ret
Public read:
- `GET /api/v1/search?q=...`
- `GET /api/v1/skills?limit=&cursor=`
- `GET /api/v1/skills?limit=&cursor=&sort=`
- `sort`: `updated` (default), `downloads`, `stars` (`rating`), `installsCurrent` (`installs`), `installsAllTime`, `trending`
- `GET /api/v1/skills/{slug}`
- `GET /api/v1/skills/{slug}/versions?limit=&cursor=`
- `GET /api/v1/skills/{slug}/versions/{version}`
+4 -4
View File
@@ -11,8 +11,8 @@ read_when:
- Web app: TanStack Start (React) under `src/`.
- Backend: Convex under `convex/` (DB, storage, actions, HTTP routes).
- CLI: `packages/clawdhub/` (published as `clawdhub`).
- Shared schemas/routes: `packages/schema/` (`clawdhub-schema`).
- CLI: `packages/clawdhub/` (published as `clawhub`, legacy `clawdhub`).
- Shared schemas/routes: `packages/schema/` (`clawhub-schema`).
## Data + storage
@@ -38,8 +38,8 @@ read_when:
- Download zip via `/api/v1/download?slug=...&version=...`.
- Extract into `./skills/<slug>` (default).
- Persist install state:
- `./.clawdhub/lock.json` (per workdir)
- `./skills/<slug>/.clawdhub/origin.json` (per skill folder)
- `./.clawhub/lock.json` (per workdir, legacy `.clawdhub`)
- `./skills/<slug>/.clawhub/origin.json` (per skill folder, legacy `.clawdhub`)
### Update (CLI)
+4 -4
View File
@@ -23,7 +23,7 @@ The CLI uses a long-lived API token (Bearer token) for publish/sync/delete.
### Browser flow (default)
`clawdhub login` does:
`clawhub login` does:
1. Starts a loopback HTTP server on `127.0.0.1` (random port).
2. Opens `<site>/cli/auth?redirect_uri=http://127.0.0.1:<port>/callback&state=...`.
@@ -35,18 +35,18 @@ The CLI uses a long-lived API token (Bearer token) for publish/sync/delete.
Create a token in the web UI (Settings → API tokens) and paste it:
```bash
clawdhub login --token clh_...
clawhub login --token clh_...
```
### Token storage
Default global config path:
- macOS: `~/Library/Application Support/clawdhub/config.json`
- macOS: `~/Library/Application Support/clawhub/config.json`
Override:
- `CLAWDHUB_CONFIG_PATH=/path/to/config.json`
- `CLAWHUB_CONFIG_PATH=/path/to/config.json` (legacy `CLAWDHUB_CONFIG_PATH`)
### Revocation
+30 -15
View File
@@ -7,62 +7,77 @@ read_when:
# CLI
CLI package: `packages/clawdhub/` (bin: `clawdhub`).
CLI package: `packages/clawdhub/` (published as `clawhub`, bin: `clawhub`).
From this repo you can run it via the wrapper script:
```bash
bun clawdhub --help
bun clawhub --help
```
## Global flags
- `--workdir <dir>`: working directory (default: cwd; falls back to Clawdbot workspace if configured)
- `--dir <dir>`: install dir under workdir (default: `skills`)
- `--site <url>`: base URL for browser login (default: `https://clawdhub.com`)
- `--registry <url>`: API base URL (default: discovered, else `https://clawdhub.com`)
- `--site <url>`: base URL for browser login (default: `https://clawhub.ai`)
- `--registry <url>`: API base URL (default: discovered, else `https://clawhub.ai`)
- `--no-input`: disable prompts
Env equivalents:
- `CLAWDHUB_SITE`
- `CLAWDHUB_REGISTRY`
- `CLAWDHUB_WORKDIR`
- `CLAWHUB_SITE` (legacy `CLAWDHUB_SITE`)
- `CLAWHUB_REGISTRY` (legacy `CLAWDHUB_REGISTRY`)
- `CLAWHUB_WORKDIR` (legacy `CLAWDHUB_WORKDIR`)
## Config file
Stores your API token + cached registry URL.
- macOS: `~/Library/Application Support/clawdhub/config.json`
- override: `CLAWDHUB_CONFIG_PATH`
- macOS: `~/Library/Application Support/clawhub/config.json`
- override: `CLAWHUB_CONFIG_PATH` (legacy `CLAWDHUB_CONFIG_PATH`)
## Commands
### `login` / `auth login`
- Default: opens browser to `<site>/cli/auth` and completes via loopback callback.
- Headless: `clawdhub login --token clh_...`
- Headless: `clawhub login --token clh_...`
### `whoami`
- Verifies the stored token via `/api/v1/whoami`.
### `star <slug>` / `unstar <slug>`
- Adds/removes a skill from your highlights.
- Calls `POST /api/v1/stars/<slug>` and `DELETE /api/v1/stars/<slug>`.
- `--yes` skips confirmation.
### `search <query...>`
- Calls `/api/v1/search?q=...`.
### `explore`
- Lists latest updated skills via `/api/v1/skills?limit=...` (sorted by `updatedAt` desc).
- Flags:
- `--limit <n>` (1200, default: 25)
- `--sort newest|downloads|rating|installs|installsAllTime|trending` (default: newest)
- `--json` (machine-readable output)
- Output: `<slug> v<version> <age> <summary>` (summary truncated to 50 chars).
### `install <slug>`
- Resolves latest version via `/api/v1/skills/<slug>`.
- Downloads zip via `/api/v1/download`.
- Extracts into `<workdir>/<dir>/<slug>`.
- Writes:
- `<workdir>/.clawdhub/lock.json`
- `<skill>/.clawdhub/origin.json`
- `<workdir>/.clawhub/lock.json` (legacy `.clawdhub`)
- `<skill>/.clawhub/origin.json` (legacy `.clawdhub`)
### `list`
- Reads `<workdir>/.clawdhub/lock.json`.
- Reads `<workdir>/.clawhub/lock.json` (legacy `.clawdhub`).
### `update [slug]` / `update --all`
@@ -86,7 +101,7 @@ Stores your API token + cached registry URL.
- `routing.agents.*.workspace/skills` (per-agent)
- `~/.clawdbot/skills` (shared)
- `skills.load.extraDirs` (shared packs)
- Respects `CLAWDBOT_CONFIG_PATH` and `CLAWDBOT_STATE_DIR`.
- Respects `CLAWDBOT_CONFIG_PATH` / `CLAWDBOT_STATE_DIR` and `OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR`.
- Flags:
- `--root <dir...>` extra scan roots
- `--all` upload without prompting
@@ -98,5 +113,5 @@ Stores your API token + cached registry URL.
Telemetry:
- Sent during `sync` when logged in, unless `CLAWDHUB_DISABLE_TELEMETRY=1`.
- Sent during `sync` when logged in, unless `CLAWHUB_DISABLE_TELEMETRY=1` (legacy `CLAWDHUB_DISABLE_TELEMETRY=1`).
- Details: `docs/telemetry.md`.
+6 -5
View File
@@ -7,7 +7,7 @@ read_when:
# Deploy
ClawdHub is two deployables:
OpenClaw is two deployables:
- Web app (TanStack Start) → typically Vercel.
- Convex backend → Convex deployment (serves `/api/...` routes).
@@ -55,12 +55,13 @@ For self-host:
The CLI can discover the API base from:
- `/.well-known/clawdhub.json`
- `/.well-known/clawhub.json` (preferred)
- `/.well-known/clawdhub.json` (legacy)
If you dont serve that file, users must set:
```bash
export CLAWDHUB_REGISTRY=https://your-site.example
export CLAWHUB_REGISTRY=https://your-site.example
```
## 5) Post-deploy checks
@@ -73,6 +74,6 @@ curl -i "https://<site>/api/v1/skills/gifgrep"
Then:
```bash
clawdhub login --site https://<site>
clawdhub whoami
clawhub login --site https://<site>
clawhub whoami
```
+2 -2
View File
@@ -10,7 +10,7 @@ read_when:
## Goals
- Compare any file between two versions.
- Default compare: `latest` vs `previous` (SemVer precedence).
- UX feels native to ClawdHub (theme + typography + motion).
- UX feels native to OpenClaw (theme + typography + motion).
- Inline or side-by-side toggle.
- Public access.
@@ -57,7 +57,7 @@ Optional helper action:
- Feed into Monaco diff editor.
## Monaco theming
- Define `clawdhub-light` / `clawdhub-dark` via `monaco.editor.defineTheme`.
- Define `clawhub-light` / `clawhub-dark` via `monaco.editor.defineTheme`.
- Derive colors from CSS variables on `document.documentElement`:
- `--surface`, `--surface-muted`, `--ink`, `--ink-soft`, `--line`, `--accent`.
- Apply theme on load + when theme changes (`data-theme`).
+27 -7
View File
@@ -7,7 +7,7 @@ read_when:
# HTTP API
Base URL: `https://clawdhub.com` (default).
Base URL: `https://clawhub.ai` (default).
All v1 paths are under `/api/v1/...` and implemented by Convex HTTP routes (`convex/http.ts`).
Legacy `/api/...` and `/api/cli/...` remain for compatibility (see `DEPRECATIONS.md`).
@@ -44,8 +44,13 @@ Response:
Query params:
- `limit` (optional): integer
- `cursor` (optional): pagination cursor
- `limit` (optional): integer (1200)
- `cursor` (optional): pagination cursor (only for `sort=updated`)
- `sort` (optional): `updated` (default), `downloads`, `stars` (alias: `rating`), `installsCurrent` (alias: `installs`), `installsAllTime`, `trending`
Notes:
- `trending` ranks by installs in the last 7 days (telemetry-based).
Response:
@@ -140,6 +145,20 @@ Publishes a new version.
Soft-delete / restore a skill (owner/admin only).
### `POST /api/v1/stars/{slug}` / `DELETE /api/v1/stars/{slug}`
Add/remove a star (highlights). Both endpoints are idempotent.
Responses:
```json
{ "ok": true, "starred": true, "alreadyStarred": false }
```
```json
{ "ok": true, "unstarred": true, "alreadyUnstarred": false }
```
## Legacy CLI endpoints (deprecated)
Still supported for older CLI versions:
@@ -153,16 +172,17 @@ Still supported for older CLI versions:
See `DEPRECATIONS.md` for removal plan.
## Registry discovery (`/.well-known/clawdhub.json`)
## Registry discovery (`/.well-known/clawhub.json`)
The CLI can discover registry/auth settings from the site:
- `/.well-known/clawdhub.json` (JSON)
- `/.well-known/clawhub.json` (JSON, preferred)
- `/.well-known/clawdhub.json` (legacy)
Schema:
```json
{ "apiBase": "https://clawdhub.com", "authBase": "https://clawdhub.com", "minCliVersion": "0.0.5" }
{ "apiBase": "https://clawhub.ai", "authBase": "https://clawhub.ai", "minCliVersion": "0.0.5" }
```
If you self-host, serve this file (or set `CLAWDHUB_REGISTRY` explicitly).
If you self-host, serve this file (or set `CLAWHUB_REGISTRY` explicitly; legacy `CLAWDHUB_REGISTRY`).
+20 -20
View File
@@ -8,53 +8,53 @@ read_when:
# Manual testing (CLI)
## Setup
- Ensure logged in: `bun clawdhub whoami` (or `bun clawdhub login`).
- Ensure logged in: `bun clawhub whoami` (or `bun clawhub login`).
- Optional: set env
- `CLAWDHUB_SITE=https://clawdhub.com`
- `CLAWDHUB_REGISTRY=https://clawdhub.com`
- `CLAWHUB_SITE=https://clawhub.ai`
- `CLAWHUB_REGISTRY=https://clawhub.ai`
## Smoke
- `bun clawdhub --help`
- `bun clawdhub --cli-version`
- `bun clawdhub whoami`
- `bun clawhub --help`
- `bun clawhub --cli-version`
- `bun clawhub whoami`
## Search
- `bun clawdhub search gif --limit 5`
- `bun clawhub search gif --limit 5`
## Install / list / update
- `mkdir -p /tmp/clawdhub-manual && cd /tmp/clawdhub-manual`
- `bunx clawdhub@beta install gifgrep --force`
- `bunx clawdhub@beta list`
- `bunx clawdhub@beta update gifgrep --force`
- `mkdir -p /tmp/clawhub-manual && cd /tmp/clawhub-manual`
- `bunx clawhub@beta install gifgrep --force`
- `bunx clawhub@beta list`
- `bunx clawhub@beta update gifgrep --force`
## Publish (changelog optional)
- `mkdir -p /tmp/clawdhub-skill-demo/SKILL && cd /tmp/clawdhub-skill-demo`
- `mkdir -p /tmp/clawhub-skill-demo/SKILL && cd /tmp/clawhub-skill-demo`
- Create files:
- `SKILL.md`
- `notes.md`
- Publish:
- `bun clawdhub publish . --slug clawdhub-manual-<ts> --name "Manual <ts>" --version 1.0.0 --tags latest`
- `bun clawhub publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.0 --tags latest`
- Publish update with empty changelog:
- `bun clawdhub publish . --slug clawdhub-manual-<ts> --name "Manual <ts>" --version 1.0.1 --tags latest`
- `bun clawhub publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.1 --tags latest`
## Delete / undelete (owner/admin)
- `bun clawdhub delete clawdhub-manual-<ts> --yes`
- `bun clawhub delete clawhub-manual-<ts> --yes`
- Verify hidden:
- `curl -i "https://clawdhub.com/api/v1/skills/clawdhub-manual-<ts>"`
- `curl -i "https://clawhub.ai/api/v1/skills/clawhub-manual-<ts>"`
- Restore:
- `bun clawdhub undelete clawdhub-manual-<ts> --yes`
- `bun clawhub undelete clawhub-manual-<ts> --yes`
- Cleanup:
- `bun clawdhub delete clawdhub-manual-<ts> --yes`
- `bun clawhub delete clawhub-manual-<ts> --yes`
## Sync
- `bun clawdhub sync --dry-run --all`
- `bun clawhub sync --dry-run --all`
## Playwright (menu smoke)
Run against prod:
```
PLAYWRIGHT_BASE_URL=https://clawdhub.com bun run test:pw
PLAYWRIGHT_BASE_URL=https://clawhub.ai bun run test:pw
```
Run against a local preview server:
+1 -1
View File
@@ -20,7 +20,7 @@ Example (starter):
```json
{
"name": "ClawdHub",
"name": "OpenClaw",
"logo": "public/logo.svg",
"navigation": [
{ "group": "Start", "pages": ["docs/README", "docs/quickstart"] },
+13 -13
View File
@@ -51,29 +51,29 @@ Then paste the printed `JWT_PRIVATE_KEY` + `JWKS` into `.env.local` (and ensure
From this repo:
```bash
bun clawdhub --help
bun clawdhub login
bun clawdhub whoami
bun clawdhub search gif --limit 5
bun clawhub --help
bun clawhub login
bun clawhub whoami
bun clawhub search gif --limit 5
```
Install a skill into `./skills/<slug>` (if Clawdbot is configured, installs into that workspace instead):
```bash
bun clawdhub install <slug>
bun clawdhub list
bun clawhub install <slug>
bun clawhub list
```
You can also install into any folder:
```bash
bun clawdhub install <slug> --workdir /tmp/clawdhub-demo --dir skills
bun clawhub install <slug> --workdir /tmp/clawhub-demo --dir skills
```
Update:
```bash
bun clawdhub update --all
bun clawhub update --all
```
## 4) Publish a skill
@@ -81,7 +81,7 @@ bun clawdhub update --all
Create a folder containing `SKILL.md` (required) plus any supporting text files:
```bash
mkdir -p /tmp/clawdhub-skill-demo && cd /tmp/clawdhub-skill-demo
mkdir -p /tmp/clawhub-skill-demo && cd /tmp/clawhub-skill-demo
cat > SKILL.md <<'EOF'
---
name: Demo Skill
@@ -97,8 +97,8 @@ EOF
Publish:
```bash
bun clawdhub publish . \
--slug clawdhub-demo-$(date +%s) \
bun clawhub publish . \
--slug clawhub-demo-$(date +%s) \
--name "Demo $(date +%s)" \
--version 1.0.0 \
--tags latest \
@@ -110,11 +110,11 @@ bun clawdhub publish . \
`sync` scans for local skill folders and publishes the ones that arent “synced” yet.
```bash
bun clawdhub sync
bun clawhub sync
```
Dry run + non-interactive:
```bash
bun clawdhub sync --all --dry-run --no-input
bun clawhub sync --all --dry-run --no-input
```
+3 -3
View File
@@ -18,16 +18,16 @@ Required:
Optional:
- any supporting *text-based* files (see “Allowed files”)
- `.clawdhubignore` (ignore patterns for publish/sync)
- `.clawhubignore` (ignore patterns for publish/sync, legacy `.clawdhubignore`)
- `.gitignore` (also honored)
Local install metadata (written by the CLI):
- `<skill>/.clawdhub/origin.json`
- `<skill>/.clawhub/origin.json` (legacy `.clawdhub`)
Workdir install state (written by the CLI):
- `<workdir>/.clawdhub/lock.json`
- `<workdir>/.clawhub/lock.json` (legacy `.clawdhub`)
## `SKILL.md`
+14 -8
View File
@@ -1,12 +1,12 @@
---
summary: "ClawdHub spec: skills registry, versioning, vector search, moderation"
summary: "OpenClaw spec: skills registry, versioning, vector search, moderation"
read_when:
- Bootstrapping ClawdHub
- Bootstrapping OpenClaw
- Implementing schema/auth/search/versioning
- Reviewing API and upload/download flows
---
# ClawdHub — product + implementation spec (v1)
# OpenClaw — product + implementation spec (v1)
## Goals
- onlycrabs.ai mode for sharing `SOUL.md` bundles (host-based entry point).
@@ -29,7 +29,7 @@ read_when:
- `handle` (GitHub login)
- `name`, `bio`
- `avatarUrl` (GitHub, fallback gravatar)
- `role`: `admin | moderator | user`
- `role`: `admin | moderator | user` (moderators can soft-delete and flag; admins can hard-delete + change owners)
- `createdAt`, `updatedAt`
### Skill
@@ -40,9 +40,14 @@ read_when:
- `latestVersionId`
- `latestTagVersionId` (for `latest` tag)
- `tags` map: `{ tag -> versionId }`
- `badges`: `{ redactionApproved?: { byUserId, at } }`
- `badges`: `{ redactionApproved?: { byUserId, at }, highlighted?: { byUserId, at }, official?: { byUserId, at }, deprecated?: { byUserId, at } }`
- `official` marks admin-verified/official skills.
- `deprecated` marks skills that should not be used for new integrations.
- `moderationStatus`: `active | hidden | removed`
- `moderationFlags`: `string[]` (automatic detection)
- `moderationNotes`, `moderationReason`
- `hiddenAt`, `hiddenBy`, `lastReviewedAt`, `reportCount`
- `stats`: `{ downloads, stars, versions, comments }`
- `status`: `active` only (soft-delete on version/comment only)
- `createdAt`, `updatedAt`
### SkillVersion
@@ -118,7 +123,8 @@ From SKILL.md frontmatter + AgentSkills + Clawdis extensions:
## Auth + roles
- Convex Auth with GitHub OAuth App.
- Default role `user`; bootstrap `steipete` to `admin` on first login.
- Admin UI to promote/demote roles; all changes logged.
- Management console: moderators can hide/restore skills + mark duplicates; admins can change owners, approve badges, and hard-delete.
- Role changes are admin-only and audited.
## Upload flow (50MB per version)
1) Client requests upload session.
@@ -151,7 +157,7 @@ Seed data lives in `convex/seed.ts` for local dev.
- Soft-delete versions; downloads remain for non-deleted versions only.
## UI (SPA)
- Home: search + filters + trending/featured + “Highlighted” batch.
- Home: search + filters + trending/featured + “Highlighted” badge.
- Skill detail: README render, files list, version history, tags, stats, badges.
- Upload/edit: file picker + version + tag + changelog.
- Account settings: name + delete account (soft delete).
+8 -8
View File
@@ -1,5 +1,5 @@
---
summary: 'Install telemetry collected via `clawdhub sync` + opt-out.'
summary: 'Install telemetry collected via `clawhub sync` + opt-out.'
read_when:
- Working on telemetry / privacy controls
- Questions about what data is collected
@@ -7,22 +7,22 @@ read_when:
# Telemetry
ClawdHub uses **minimal telemetry** to compute **install counts** (whats actually in use) and to power better sorting/filtering.
This is based on the CLI `clawdhub sync` command.
OpenClaw uses **minimal telemetry** to compute **install counts** (whats actually in use) and to power better sorting/filtering.
This is based on the CLI `clawhub sync` command.
## When telemetry is collected
Telemetry is only sent when:
- You are **logged in** in the CLI (we already require auth for sync/publish flows).
- You run `clawdhub sync`.
- You run `clawhub sync`.
- Telemetry is **not disabled** (see “How to disable” below).
If you are not logged in, nothing is reported.
## What we collect
On each `clawdhub sync`, the CLI reports a **full snapshot** of what it found, grouped by scan root (“folder/root”).
On each `clawhub sync`, the CLI reports a **full snapshot** of what it found, grouped by scan root (“folder/root”).
For each root we store:
@@ -70,7 +70,7 @@ This is evaluated lazily (on the next telemetry report) to avoid background jobs
## Transparency + user controls
ClawdHub provides a private “Installed” tab on your own profile:
OpenClaw provides a private “Installed” tab on your own profile:
- Shows the exact roots + installed skills we store.
- Includes a **JSON export** view.
@@ -85,7 +85,7 @@ Deleting your account also deletes your telemetry data.
Set the environment variable:
```bash
export CLAWDHUB_DISABLE_TELEMETRY=1
export CLAWHUB_DISABLE_TELEMETRY=1
```
With this set, the CLI will not send telemetry during `clawdhub sync`.
With this set, the CLI will not send telemetry during `clawhub sync`.
+6 -6
View File
@@ -6,16 +6,16 @@ read_when:
# Troubleshooting
## `clawdhub login` opens browser but never completes
## `clawhub login` opens browser but never completes
- Ensure your browser can reach `http://127.0.0.1:<port>/callback` (local firewalls/VPNs can interfere).
- Use headless mode:
- create a token in the web UI (Settings → API tokens)
- `clawdhub login --token clh_...`
- `clawhub login --token clh_...`
## `whoami` / `publish` returns `Unauthorized` (401)
- Token missing or revoked: check your config file (`CLAWDHUB_CONFIG_PATH` override?).
- Token missing or revoked: check your config file (`CLAWHUB_CONFIG_PATH` override?).
- Ensure requests include `Authorization: Bearer ...` (CLI does this automatically).
## `publish` fails with `OPENAI_API_KEY is not configured`
@@ -32,7 +32,7 @@ read_when:
- Provide explicit roots:
```bash
clawdhub sync --root /path/to/skills
clawhub sync --root /path/to/skills
```
## `update` refuses due to “local changes (no match)”
@@ -40,8 +40,8 @@ clawdhub sync --root /path/to/skills
- Your local files dont match any published fingerprint.
- Options:
- keep local edits; skip updating
- overwrite: `clawdhub update <slug> --force`
- publish as fork: copy to new folder/slug then `clawdhub publish ... --fork-of upstream@version`
- overwrite: `clawhub update <slug> --force`
- publish as fork: copy to new folder/slug then `clawhub publish ... --fork-of upstream@version`
## `GET /api/*` works locally but not on Vercel
+4 -4
View File
@@ -6,7 +6,7 @@ read_when:
# Webhooks (Discord)
ClawdHub can post Discord embeds when skills are published or highlighted.
OpenClaw can post Discord embeds when skills are published or highlighted.
## Setup
@@ -14,7 +14,7 @@ Set the webhook URL in the Convex environment:
- `DISCORD_WEBHOOK_URL` (required): Discord webhook URL.
- `DISCORD_WEBHOOK_HIGHLIGHTED_ONLY` (optional): `true` to only send for highlighted skills.
- `SITE_URL` (optional): Base site URL for links (default `https://clawdhub.com`).
- `SITE_URL` (optional): Base site URL for links (default `https://clawhub.ai`).
## Events
@@ -38,13 +38,13 @@ Discord receives a JSON payload with a single embed:
{
"title": "Demo Skill",
"description": "Nice skill",
"url": "https://clawdhub.com/owner/demo-skill",
"url": "https://clawhub.ai/owner/demo-skill",
"fields": [
{ "name": "Version", "value": "v1.2.3", "inline": true },
{ "name": "Owner", "value": "@owner", "inline": true },
{ "name": "Tags", "value": "latest, discord", "inline": false }
],
"footer": { "text": "ClawdHub" }
"footer": { "text": "OpenClaw" }
}
]
}
+98 -57
View File
@@ -9,19 +9,47 @@ import {
ApiV1SearchResponseSchema,
ApiV1WhoamiResponseSchema,
parseArk,
} from 'clawdhub-schema'
} from 'clawhub-schema'
import { unzipSync } from 'fflate'
import { Agent, setGlobalDispatcher } from 'undici'
import { describe, expect, it } from 'vitest'
import { readGlobalConfig } from '../packages/clawdhub/src/config'
const REQUEST_TIMEOUT_MS = 15_000
try {
setGlobalDispatcher(
new Agent({
allowH2: true,
connect: { timeout: REQUEST_TIMEOUT_MS },
}),
)
} catch {
// ignore dispatcher setup failures
}
function mustGetToken() {
const fromEnv = process.env.CLAWDHUB_E2E_TOKEN?.trim()
const fromEnv = process.env.CLAWHUB_E2E_TOKEN?.trim() || process.env.CLAWDHUB_E2E_TOKEN?.trim()
if (fromEnv) return fromEnv
return null
}
function getRegistry() {
return (
process.env.CLAWHUB_REGISTRY?.trim() ||
process.env.CLAWDHUB_REGISTRY?.trim() ||
'https://clawhub.ai'
)
}
function getSite() {
return (
process.env.CLAWHUB_SITE?.trim() || process.env.CLAWDHUB_SITE?.trim() || 'https://clawhub.ai'
)
}
async function makeTempConfig(registry: string, token: string | null) {
const dir = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-'))
const dir = await mkdtemp(join(tmpdir(), 'clawhub-e2e-'))
const path = join(dir, 'config.json')
await writeFile(
path,
@@ -31,9 +59,19 @@ async function makeTempConfig(registry: string, token: string | null) {
return { dir, path }
}
describe('clawdhub e2e', () => {
async function fetchWithTimeout(input: RequestInfo | URL, init?: RequestInit) {
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
try {
return await fetch(input, { ...init, signal: controller.signal })
} finally {
clearTimeout(timeout)
}
}
describe('clawhub e2e', () => {
it('prints CLI version via --cli-version', async () => {
const result = spawnSync('bun', ['clawdhub', '--cli-version'], {
const result = spawnSync('bun', ['clawhub', '--cli-version'], {
cwd: process.cwd(),
encoding: 'utf8',
})
@@ -42,12 +80,14 @@ describe('clawdhub e2e', () => {
})
it('search endpoint returns a results array (schema parse)', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const registry = getRegistry()
const url = new URL(ApiRoutes.search, registry)
url.searchParams.set('q', 'gif')
url.searchParams.set('limit', '5')
const response = await fetch(url.toString(), { headers: { Accept: 'application/json' } })
const response = await fetchWithTimeout(url.toString(), {
headers: { Accept: 'application/json' },
})
expect(response.ok).toBe(true)
const json = (await response.json()) as unknown
const parsed = parseArk(ApiV1SearchResponseSchema, json, 'API response')
@@ -55,17 +95,17 @@ describe('clawdhub e2e', () => {
})
it('cli search does not error on multi-result responses', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const site = process.env.CLAWDHUB_SITE?.trim() || 'https://clawdhub.com'
const registry = getRegistry()
const site = getSite()
const token = mustGetToken() ?? (await readGlobalConfig())?.token ?? null
const cfg = await makeTempConfig(registry, token)
try {
const workdir = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-workdir-'))
const workdir = await mkdtemp(join(tmpdir(), 'clawhub-e2e-workdir-'))
const result = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'search',
'gif',
'--limit',
@@ -79,7 +119,7 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -93,17 +133,17 @@ describe('clawdhub e2e', () => {
})
it('assumes a logged-in user (whoami succeeds)', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const site = process.env.CLAWDHUB_SITE?.trim() || 'https://clawdhub.com'
const registry = getRegistry()
const site = getSite()
const token = mustGetToken() ?? (await readGlobalConfig())?.token ?? null
if (!token) {
throw new Error('Missing token. Set CLAWDHUB_E2E_TOKEN or run: bun clawdhub auth login')
throw new Error('Missing token. Set CLAWHUB_E2E_TOKEN or run: bun clawhub auth login')
}
const cfg = await makeTempConfig(registry, token)
try {
const whoamiUrl = new URL(ApiRoutes.whoami, registry)
const whoamiRes = await fetch(whoamiUrl.toString(), {
const whoamiRes = await fetchWithTimeout(whoamiUrl.toString(), {
headers: { Accept: 'application/json', Authorization: `Bearer ${token}` },
})
expect(whoamiRes.ok).toBe(true)
@@ -116,10 +156,10 @@ describe('clawdhub e2e', () => {
const result = spawnSync(
'bun',
['clawdhub', 'whoami', '--site', site, '--registry', registry],
['clawhub', 'whoami', '--site', site, '--registry', registry],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -131,15 +171,15 @@ describe('clawdhub e2e', () => {
})
it('sync dry-run finds skills from an explicit root', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const site = process.env.CLAWDHUB_SITE?.trim() || 'https://clawdhub.com'
const registry = getRegistry()
const site = getSite()
const token = mustGetToken() ?? (await readGlobalConfig())?.token ?? null
if (!token) {
throw new Error('Missing token. Set CLAWDHUB_E2E_TOKEN or run: bun clawdhub auth login')
throw new Error('Missing token. Set CLAWHUB_E2E_TOKEN or run: bun clawhub auth login')
}
const cfg = await makeTempConfig(registry, token)
const root = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-sync-'))
const root = await mkdtemp(join(tmpdir(), 'clawhub-e2e-sync-'))
try {
const skillDir = join(root, 'cool-skill')
await mkdir(skillDir, { recursive: true })
@@ -148,7 +188,7 @@ describe('clawdhub e2e', () => {
const result = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'sync',
'--dry-run',
'--all',
@@ -161,7 +201,7 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -175,15 +215,15 @@ describe('clawdhub e2e', () => {
})
it('sync dry-run finds skills from clawdbot.json roots', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const site = process.env.CLAWDHUB_SITE?.trim() || 'https://clawdhub.com'
const registry = getRegistry()
const site = getSite()
const token = mustGetToken() ?? (await readGlobalConfig())?.token ?? null
if (!token) {
throw new Error('Missing token. Set CLAWDHUB_E2E_TOKEN or run: bun clawdhub auth login')
throw new Error('Missing token. Set CLAWHUB_E2E_TOKEN or run: bun clawhub auth login')
}
const cfg = await makeTempConfig(registry, token)
const root = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-clawdbot-'))
const root = await mkdtemp(join(tmpdir(), 'clawhub-e2e-clawdbot-'))
const stateDir = join(root, 'state')
const configPath = join(root, 'clawdbot.json')
const workspace = join(root, 'clawd-work')
@@ -206,13 +246,13 @@ describe('clawdhub e2e', () => {
const result = spawnSync(
'bun',
['clawdhub', 'sync', '--dry-run', '--all', '--site', site, '--registry', registry],
['clawhub', 'sync', '--dry-run', '--all', '--site', site, '--registry', registry],
{
cwd: process.cwd(),
env: {
...process.env,
CLAWDHUB_CONFIG_PATH: cfg.path,
CLAWDHUB_DISABLE_TELEMETRY: '1',
CLAWHUB_CONFIG_PATH: cfg.path,
CLAWHUB_DISABLE_TELEMETRY: '1',
CLAWDBOT_CONFIG_PATH: configPath,
CLAWDBOT_STATE_DIR: stateDir,
},
@@ -230,16 +270,16 @@ describe('clawdhub e2e', () => {
})
it('publishes, deletes, and undeletes a skill (logged-in)', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const site = process.env.CLAWDHUB_SITE?.trim() || 'https://clawdhub.com'
const registry = getRegistry()
const site = getSite()
const token = mustGetToken() ?? (await readGlobalConfig())?.token ?? null
if (!token) {
throw new Error('Missing token. Set CLAWDHUB_E2E_TOKEN or run: bun clawdhub auth login')
throw new Error('Missing token. Set CLAWHUB_E2E_TOKEN or run: bun clawhub auth login')
}
const cfg = await makeTempConfig(registry, token)
const workdir = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-publish-'))
const installWorkdir = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-install-'))
const workdir = await mkdtemp(join(tmpdir(), 'clawhub-e2e-publish-'))
const installWorkdir = await mkdtemp(join(tmpdir(), 'clawhub-e2e-install-'))
const slug = `e2e-${Date.now()}`
const skillDir = join(workdir, slug)
@@ -250,7 +290,7 @@ describe('clawdhub e2e', () => {
const publish1 = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'publish',
skillDir,
'--slug',
@@ -270,7 +310,7 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -280,7 +320,7 @@ describe('clawdhub e2e', () => {
const publish2 = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'publish',
skillDir,
'--slug',
@@ -300,7 +340,7 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -310,7 +350,7 @@ describe('clawdhub e2e', () => {
const downloadUrl = new URL(ApiRoutes.download, registry)
downloadUrl.searchParams.set('slug', slug)
downloadUrl.searchParams.set('version', '1.0.1')
const zipRes = await fetch(downloadUrl.toString())
const zipRes = await fetchWithTimeout(downloadUrl.toString())
expect(zipRes.ok).toBe(true)
const zipBytes = new Uint8Array(await zipRes.arrayBuffer())
const unzipped = unzipSync(zipBytes)
@@ -319,7 +359,7 @@ describe('clawdhub e2e', () => {
const install = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'install',
slug,
'--version',
@@ -334,7 +374,7 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -342,10 +382,10 @@ describe('clawdhub e2e', () => {
const list = spawnSync(
'bun',
['clawdhub', 'list', '--site', site, '--registry', registry, '--workdir', installWorkdir],
['clawhub', 'list', '--site', site, '--registry', registry, '--workdir', installWorkdir],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
@@ -355,7 +395,7 @@ describe('clawdhub e2e', () => {
const update = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'update',
slug,
'--force',
@@ -368,13 +408,14 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
expect(update.status).toBe(0)
const metaRes = await fetch(`${registry}${ApiRoutes.skills}/${slug}`, {
const metaUrl = new URL(`${ApiRoutes.skills}/${slug}`, registry)
const metaRes = await fetchWithTimeout(metaUrl.toString(), {
headers: { Accept: 'application/json' },
})
expect(metaRes.status).toBe(200)
@@ -382,7 +423,7 @@ describe('clawdhub e2e', () => {
const del = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'delete',
slug,
'--yes',
@@ -395,24 +436,24 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
expect(del.status).toBe(0)
const metaAfterDelete = await fetch(metaUrl.toString(), {
const metaAfterDelete = await fetchWithTimeout(metaUrl.toString(), {
headers: { Accept: 'application/json' },
})
expect(metaAfterDelete.status).toBe(404)
const downloadAfterDelete = await fetch(downloadUrl.toString())
const downloadAfterDelete = await fetchWithTimeout(downloadUrl.toString())
expect(downloadAfterDelete.status).toBe(404)
const undelete = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'undelete',
slug,
'--yes',
@@ -425,13 +466,13 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
expect(undelete.status).toBe(0)
const metaAfterUndelete = await fetch(metaUrl.toString(), {
const metaAfterUndelete = await fetchWithTimeout(metaUrl.toString(), {
headers: { Accept: 'application/json' },
})
expect(metaAfterUndelete.status).toBe(200)
@@ -439,7 +480,7 @@ describe('clawdhub e2e', () => {
const cleanup = spawnSync(
'bun',
[
'clawdhub',
'clawhub',
'delete',
slug,
'--yes',
@@ -452,7 +493,7 @@ describe('clawdhub e2e', () => {
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
env: { ...process.env, CLAWHUB_CONFIG_PATH: cfg.path, CLAWHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
+1 -1
View File
@@ -43,7 +43,7 @@ test('header menu routes render', async ({ page }) => {
if (label === 'Search') {
await expect(page).toHaveURL(/\/?(\?|$)/)
await expect(page.locator('h1', { hasText: 'ClawdHub' })).toBeVisible()
await expect(page.locator('h1', { hasText: 'OpenClaw' })).toBeVisible()
}
}
})
+97
View File
@@ -0,0 +1,97 @@
import { expect, test } from '@playwright/test'
test('skills search paginates exact results', async ({ page }) => {
await page.addInitScript(() => {
const makeSearchResults = (count: number) =>
Array.from({ length: count }, (_, index) => ({
score: 0.9,
skill: {
_id: `skill_${index}`,
slug: `skill-${index}`,
displayName: `Skill ${index}`,
summary: `Summary ${index}`,
tags: {},
stats: {
downloads: 0,
installsCurrent: 0,
installsAllTime: 0,
stars: 0,
versions: 1,
comments: 0,
},
createdAt: 0,
updatedAt: 0,
},
version: null,
}))
class MockWebSocket {
url: string
readyState = 0
onopen?: () => void
onmessage?: (event: { data: string }) => void
onclose?: (event: { code: number; reason: string }) => void
onerror?: () => void
constructor(url: string) {
this.url = url
window.setTimeout(() => {
this.readyState = 1
this.onopen?.()
}, 0)
}
send(data: string) {
try {
const message = JSON.parse(data) as {
type?: string
requestId?: number
udfPath?: string
args?: Array<Record<string, unknown>>
}
if (message.type === 'Action' && message.udfPath?.includes('searchSkills')) {
const [args] = message.args ?? []
const limit = typeof args?.limit === 'number' ? args.limit : 10
const limits = (window as typeof window & { __searchLimits: number[] }).__searchLimits
limits.push(limit)
const response = {
type: 'ActionResponse',
requestId: message.requestId,
success: true,
result: makeSearchResults(limit),
logLines: [],
}
window.setTimeout(() => {
this.onmessage?.({ data: JSON.stringify(response) })
}, 0)
}
} catch {
this.onerror?.()
}
}
close(code = 1000, reason = 'closed') {
this.readyState = 3
this.onclose?.({ code, reason })
}
}
;(window as typeof window & { __searchLimits: number[] }).__searchLimits = []
window.WebSocket = MockWebSocket as unknown as typeof WebSocket
})
await page.goto('/skills', { waitUntil: 'domcontentloaded' })
await expect(page.getByRole('heading', { name: 'Skills' })).toBeVisible()
const input = page.getByPlaceholder('Filter by name, slug, or summary…')
await input.fill('remind')
await expect(page.getByText('Skill 0')).toBeVisible()
await expect(page.getByText('Scroll to load more')).toBeVisible()
await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight))
await expect(page.getByText('Skill 75')).toBeVisible()
const limits = await page.evaluate(
() => (window as typeof window & { __searchLimits: number[] }).__searchLimits,
)
expect(limits).toEqual([50, 100])
})
+23 -19
View File
@@ -1,15 +1,17 @@
{
"name": "clawdhub",
"name": "clawhub",
"private": true,
"type": "module",
"workspaces": [
"packages/*"
],
"scripts": {
"preinstall": "bunx only-allow bun",
"dev": "bun --bun vite dev --port 3000",
"build": "bun --bun vite build",
"preview": "bun --bun vite preview",
"docs:list": "bun scripts/docs-list.ts",
"check:peers": "bun scripts/check-peer-deps.ts",
"test": "vitest run",
"test:watch": "vitest",
"test:e2e": "vitest run -c vitest.e2e.config.ts",
@@ -23,7 +25,7 @@
"format": "biome format --write ."
},
"dependencies": {
"@auth/core": "^0.41.1",
"@auth/core": "^0.37.4",
"@convex-dev/auth": "^0.0.90",
"@fontsource/bricolage-grotesque": "^5.2.10",
"@fontsource/ibm-plex-mono": "^5.2.7",
@@ -33,17 +35,18 @@
"@radix-ui/react-toggle-group": "^1.1.11",
"@resvg/resvg-wasm": "^2.6.2",
"@tailwindcss/vite": "^4.1.18",
"@tanstack/react-devtools": "^0.9.0",
"@tanstack/react-router": "^1.144.0",
"@tanstack/react-router-devtools": "^1.144.0",
"@tanstack/react-start": "^1.145.3",
"@tanstack/router-plugin": "^1.145.2",
"@tanstack/react-devtools": "^0.9.2",
"@tanstack/react-router": "^1.151.6",
"@tanstack/react-router-devtools": "^1.151.6",
"@tanstack/react-start": "^1.152.0",
"@tanstack/router-plugin": "^1.151.6",
"@vercel/analytics": "^1.6.1",
"clawdhub-schema": "^0.0.2",
"clawhub-schema": "workspace:*",
"clsx": "^2.1.1",
"convex": "^1.31.2",
"convex": "^1.31.6",
"convex-helpers": "^0.1.111",
"fflate": "^0.8.2",
"h3": "2.0.1-rc.5",
"h3": "2.0.1-rc.8",
"lucide-react": "^0.562.0",
"monaco-editor": "^0.55.1",
"nitro": "^3.0.1-alpha.1",
@@ -54,26 +57,27 @@
"semver": "^7.7.3",
"tailwind-merge": "^3.4.0",
"tailwindcss": "^4.1.18",
"vite-tsconfig-paths": "^6.0.3",
"vite-tsconfig-paths": "^6.0.4",
"yaml": "^2.8.2"
},
"devDependencies": {
"@biomejs/biome": "^2.3.11",
"@playwright/test": "^1.57.0",
"@tanstack/devtools-vite": "^0.4.0",
"@tanstack/devtools-vite": "^0.4.1",
"@testing-library/dom": "^10.4.1",
"@testing-library/react": "^16.3.1",
"@types/node": "^25.0.3",
"@types/react": "^19.2.7",
"@types/node": "^25.0.9",
"@types/react": "^19.2.8",
"@types/react-dom": "^19.2.3",
"@types/semver": "^7.7.1",
"@vitejs/plugin-react": "^5.1.2",
"@vitest/coverage-v8": "^4.0.16",
"@vitest/coverage-v8": "^4.0.17",
"jsdom": "^27.4.0",
"oxlint": "^1.36.0",
"oxlint-tsgolint": "^0.10.1",
"only-allow": "^1.2.1",
"oxlint": "^1.39.0",
"oxlint-tsgolint": "^0.11.1",
"typescript": "^5.9.3",
"vite": "^7.3.0",
"vitest": "^4.0.16"
"vite": "^7.3.1",
"vitest": "^4.0.17"
}
}
+20 -20
View File
@@ -1,57 +1,57 @@
# `clawdhub`
# `clawhub`
ClawdHub CLI — install, update, search, and publish agent skills as folders.
OpenClaw CLI — install, update, search, and publish agent skills as folders.
## Install
```bash
# From this repo (shortcut script at repo root)
bun clawdhub --help
bun clawhub --help
# Once published to npm
# npm i -g clawdhub
# npm i -g clawhub
```
## Auth (publish)
```bash
clawdhub login
clawhub login
# or
clawdhub auth login
clawhub auth login
# Headless / token paste
# or (token paste / headless)
clawdhub login --token clh_...
clawhub login --token clh_...
```
Notes:
- Browser login opens `https://clawdhub.com/cli/auth` and completes via a loopback callback.
- Token stored in `~/Library/Application Support/clawdhub/config.json` on macOS (override via `CLAWDHUB_CONFIG_PATH`).
- Browser login opens `https://clawhub.ai/cli/auth` and completes via a loopback callback.
- Token stored in `~/Library/Application Support/clawhub/config.json` on macOS (override via `CLAWHUB_CONFIG_PATH`, legacy `CLAWDHUB_CONFIG_PATH`).
## Examples
```bash
clawdhub search "postgres backups"
clawdhub install my-skill-pack
clawdhub update --all
clawdhub update --all --no-input --force
clawdhub publish ./my-skill-pack --slug my-skill-pack --name "My Skill Pack" --version 1.2.0 --changelog "Fixes + docs"
clawhub search "postgres backups"
clawhub install my-skill-pack
clawhub update --all
clawhub update --all --no-input --force
clawhub publish ./my-skill-pack --slug my-skill-pack --name "My Skill Pack" --version 1.2.0 --changelog "Fixes + docs"
```
## Sync (upload local skills)
```bash
# Start anywhere; scans workdir first, then legacy Clawdis/Clawd locations.
clawdhub sync
# Start anywhere; scans workdir first, then legacy Clawdis/Clawd/OpenClaw/Moltbot locations.
clawhub sync
# Explicit roots + non-interactive dry-run
clawdhub sync --root ../clawdis/skills --all --dry-run
clawhub sync --root ../clawdis/skills --all --dry-run
```
## Defaults
- Site: `https://clawdhub.com` (override via `--site` or `CLAWDHUB_SITE`)
- Registry: discovered from `/.well-known/clawdhub.json` on the site (override via `--registry` or `CLAWDHUB_REGISTRY`)
- Workdir: current directory (falls back to Clawdbot workspace if configured; override via `--workdir` or `CLAWDHUB_WORKDIR`)
- Site: `https://clawhub.ai` (override via `--site` or `CLAWHUB_SITE`, legacy `CLAWDHUB_SITE`)
- Registry: discovered from `/.well-known/clawhub.json` on the site (legacy `/.well-known/clawdhub.json`; override via `--registry` or `CLAWHUB_REGISTRY`)
- Workdir: current directory (falls back to Clawdbot workspace if configured; override via `--workdir` or `CLAWHUB_WORKDIR`)
- Install dir: `./skills` under workdir (override via `--dir`)
+7 -5
View File
@@ -1,10 +1,11 @@
{
"name": "clawdhub",
"version": "0.2.0",
"description": "ClawdHub CLI \u2014 install, update, search, and publish agent skills.",
"name": "clawhub",
"version": "0.4.0",
"description": "OpenClaw CLI \\u2014 install, update, search, and publish agent skills.",
"license": "MIT",
"type": "module",
"bin": {
"clawhub": "bin/clawdhub.js",
"clawdhub": "bin/clawdhub.js"
},
"files": [
@@ -28,10 +29,11 @@
"mime": "^4.1.0",
"ora": "^9.0.0",
"p-retry": "^7.1.1",
"semver": "^7.7.3"
"semver": "^7.7.3",
"undici": "^7.16.0"
},
"devDependencies": {
"@types/node": "^25.0.3",
"@types/node": "^25.0.9",
"typescript": "^5.9.3"
},
"engines": {
+1 -1
View File
@@ -60,7 +60,7 @@ describe('browserAuth', () => {
const response = await fetch(server.redirectUri)
expect(response.status).toBe(200)
const text = await response.text()
expect(text).toContain('ClawdHub CLI Login')
expect(text).toContain('OpenClaw CLI Login')
server.close()
})
+1 -1
View File
@@ -127,7 +127,7 @@ const CALLBACK_HTML = `<!doctype html>
<html lang="en">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>ClawdHub CLI Login</title>
<title>OpenClaw CLI Login</title>
<style>
:root { color-scheme: light dark; }
body { font-family: ui-sans-serif, system-ui, -apple-system, Segoe UI, sans-serif; padding: 24px; }
+71 -13
View File
@@ -7,8 +7,10 @@ import { resolveClawdbotDefaultWorkspace } from './cli/clawdbotConfig.js'
import { cmdLoginFlow, cmdLogout, cmdWhoami } from './cli/commands/auth.js'
import { cmdDeleteSkill, cmdUndeleteSkill } from './cli/commands/delete.js'
import { cmdPublish } from './cli/commands/publish.js'
import { cmdInstall, cmdList, cmdSearch, cmdUpdate } from './cli/commands/skills.js'
import { cmdExplore, cmdInstall, cmdList, cmdSearch, cmdUpdate } from './cli/commands/skills.js'
import { cmdStarSkill } from './cli/commands/star.js'
import { cmdSync } from './cli/commands/sync.js'
import { cmdUnstarSkill } from './cli/commands/unstar.js'
import { configureCommanderHelp, styleEnvBlock, styleTitle } from './cli/helpStyle.js'
import { DEFAULT_REGISTRY, DEFAULT_SITE } from './cli/registry.js'
import type { GlobalOpts } from './cli/types.js'
@@ -16,9 +18,9 @@ import { fail } from './cli/ui.js'
import { readGlobalConfig } from './config.js'
const program = new Command()
.name('clawdhub')
.name('clawhub')
.description(
`${styleTitle(`ClawdHub CLI ${getCliBuildLabel()}`)}\n${styleEnvBlock(
`${styleTitle(`OpenClaw CLI ${getCliBuildLabel()}`)}\n${styleEnvBlock(
'install, update, search, and publish agent skills.',
)}`,
)
@@ -32,7 +34,9 @@ const program = new Command()
.showSuggestionAfterError()
.addHelpText(
'after',
styleEnvBlock('\nEnv:\n CLAWDHUB_SITE\n CLAWDHUB_REGISTRY\n CLAWDHUB_WORKDIR\n'),
styleEnvBlock(
'\nEnv:\n CLAWHUB_SITE\n CLAWHUB_REGISTRY\n CLAWHUB_WORKDIR\n (CLAWDHUB_* supported)\n',
),
)
configureCommanderHelp(program)
@@ -41,9 +45,17 @@ async function resolveGlobalOpts(): Promise<GlobalOpts> {
const raw = program.opts<{ workdir?: string; dir?: string; site?: string; registry?: string }>()
const workdir = await resolveWorkdir(raw.workdir)
const dir = resolve(workdir, raw.dir ?? 'skills')
const site = raw.site ?? process.env.CLAWDHUB_SITE ?? DEFAULT_SITE
const registrySource = raw.registry ? 'cli' : process.env.CLAWDHUB_REGISTRY ? 'env' : 'default'
const registry = raw.registry ?? process.env.CLAWDHUB_REGISTRY ?? DEFAULT_REGISTRY
const site = raw.site ?? process.env.CLAWHUB_SITE ?? process.env.CLAWDHUB_SITE ?? DEFAULT_SITE
const registrySource = raw.registry
? 'cli'
: process.env.CLAWHUB_REGISTRY || process.env.CLAWDHUB_REGISTRY
? 'env'
: 'default'
const registry =
raw.registry ??
process.env.CLAWHUB_REGISTRY ??
process.env.CLAWDHUB_REGISTRY ??
DEFAULT_REGISTRY
return { workdir, dir, site, registry, registrySource }
}
@@ -54,22 +66,26 @@ function isInputAllowed() {
async function resolveWorkdir(explicit?: string) {
if (explicit?.trim()) return resolve(explicit.trim())
const envWorkdir = process.env.CLAWDHUB_WORKDIR?.trim()
const envWorkdir = process.env.CLAWHUB_WORKDIR?.trim() ?? process.env.CLAWDHUB_WORKDIR?.trim()
if (envWorkdir) return resolve(envWorkdir)
const cwd = resolve(process.cwd())
const hasMarker = await hasClawdhubMarker(cwd)
const hasMarker = await hasClawhubMarker(cwd)
if (hasMarker) return cwd
const clawdbotWorkspace = await resolveClawdbotDefaultWorkspace()
return clawdbotWorkspace ? resolve(clawdbotWorkspace) : cwd
}
async function hasClawdhubMarker(workdir: string) {
const lockfile = join(workdir, '.clawdhub', 'lock.json')
async function hasClawhubMarker(workdir: string) {
const lockfile = join(workdir, '.clawhub', 'lock.json')
if (await pathExists(lockfile)) return true
const markerDir = join(workdir, '.clawdhub')
return pathExists(markerDir)
const markerDir = join(workdir, '.clawhub')
if (await pathExists(markerDir)) return true
const legacyLockfile = join(workdir, '.clawdhub', 'lock.json')
if (await pathExists(legacyLockfile)) return true
const legacyMarkerDir = join(workdir, '.clawdhub')
return pathExists(legacyMarkerDir)
}
async function pathExists(path: string) {
@@ -183,6 +199,28 @@ program
await cmdList(opts)
})
program
.command('explore')
.description('Browse latest updated skills from the registry')
.option(
'--limit <n>',
'Number of skills to show (max 200)',
(value) => Number.parseInt(value, 10),
25,
)
.option(
'--sort <order>',
'Sort by newest, downloads, rating, installs, installsAllTime, or trending',
'newest',
)
.option('--json', 'Output JSON')
.action(async (options) => {
const opts = await resolveGlobalOpts()
const limit =
typeof options.limit === 'number' && Number.isFinite(options.limit) ? options.limit : 25
await cmdExplore(opts, { limit, sort: options.sort, json: options.json })
})
program
.command('publish')
.description('Publish skill from folder')
@@ -218,6 +256,26 @@ program
await cmdUndeleteSkill(opts, slug, options, isInputAllowed())
})
program
.command('star')
.description('Add a skill to your highlights')
.argument('<slug>', 'Skill slug')
.option('--yes', 'Skip confirmation')
.action(async (slug, options) => {
const opts = await resolveGlobalOpts()
await cmdStarSkill(opts, slug, options, isInputAllowed())
})
program
.command('unstar')
.description('Remove a skill from your highlights')
.argument('<slug>', 'Skill slug')
.option('--yes', 'Skip confirmation')
.action(async (slug, options) => {
const opts = await resolveGlobalOpts()
await cmdUnstarSkill(opts, slug, options, isInputAllowed())
})
program
.command('sync')
.description('Scan local skills and publish new/updated ones')
+1
View File
@@ -24,6 +24,7 @@ function shortCommit(value: string) {
export function getCliCommit() {
const candidates = [
process.env.CLAWHUB_COMMIT,
process.env.CLAWDHUB_COMMIT,
process.env.VERCEL_GIT_COMMIT_SHA,
process.env.GITHUB_SHA,
@@ -13,14 +13,17 @@ afterEach(() => {
describe('resolveClawdbotSkillRoots', () => {
it('reads JSON5 config and resolves per-agent + shared skill roots', async () => {
const base = await mkdtemp(join(tmpdir(), 'clawdhub-clawdbot-'))
const base = await mkdtemp(join(tmpdir(), 'clawhub-clawdbot-'))
const home = join(base, 'home')
const stateDir = join(base, 'state')
const configPath = join(base, 'clawdbot.json')
const openclawStateDir = join(base, 'openclaw-state')
process.env.HOME = home
process.env.CLAWDBOT_STATE_DIR = stateDir
process.env.CLAWDBOT_CONFIG_PATH = configPath
process.env.OPENCLAW_STATE_DIR = openclawStateDir
process.env.OPENCLAW_CONFIG_PATH = join(openclawStateDir, 'openclaw.json')
const config = `{
// JSON5 comments + trailing commas supported
@@ -49,6 +52,7 @@ describe('resolveClawdbotSkillRoots', () => {
const expectedRoots = [
resolve(stateDir, 'skills'),
resolve(openclawStateDir, 'skills'),
resolve(home, 'clawd-main', 'skills'),
resolve(home, 'clawd-work', 'skills'),
resolve(home, 'clawd-family', 'skills'),
@@ -58,6 +62,7 @@ describe('resolveClawdbotSkillRoots', () => {
expect(roots).toEqual(expect.arrayContaining(expectedRoots))
expect(labels[resolve(stateDir, 'skills')]).toBe('Shared skills')
expect(labels[resolve(openclawStateDir, 'skills')]).toBe('OpenClaw: Shared skills')
expect(labels[resolve(home, 'clawd-main', 'skills')]).toBe('Agent: main')
expect(labels[resolve(home, 'clawd-work', 'skills')]).toBe('Agent: Work Bot')
expect(labels[resolve(home, 'clawd-family', 'skills')]).toBe('Agent: family')
@@ -66,16 +71,19 @@ describe('resolveClawdbotSkillRoots', () => {
})
it('resolves default workspace from agents.defaults and agents.list', async () => {
const base = await mkdtemp(join(tmpdir(), 'clawdhub-clawdbot-default-'))
const base = await mkdtemp(join(tmpdir(), 'clawhub-clawdbot-default-'))
const home = join(base, 'home')
const stateDir = join(base, 'state')
const configPath = join(base, 'clawdbot.json')
const workspaceMain = join(base, 'workspace-main')
const workspaceList = join(base, 'workspace-list')
const openclawStateDir = join(base, 'openclaw-state')
process.env.HOME = home
process.env.CLAWDBOT_STATE_DIR = stateDir
process.env.CLAWDBOT_CONFIG_PATH = configPath
process.env.OPENCLAW_STATE_DIR = openclawStateDir
process.env.OPENCLAW_CONFIG_PATH = join(openclawStateDir, 'openclaw.json')
const config = `{
agents: {
@@ -92,14 +100,17 @@ describe('resolveClawdbotSkillRoots', () => {
})
it('falls back to default agent in agents.list when defaults missing', async () => {
const base = await mkdtemp(join(tmpdir(), 'clawdhub-clawdbot-list-'))
const base = await mkdtemp(join(tmpdir(), 'clawhub-clawdbot-list-'))
const home = join(base, 'home')
const configPath = join(base, 'clawdbot.json')
const workspaceMain = join(base, 'workspace-main')
const workspaceWork = join(base, 'workspace-work')
const openclawStateDir = join(base, 'openclaw-state')
process.env.HOME = home
process.env.CLAWDBOT_CONFIG_PATH = configPath
process.env.OPENCLAW_STATE_DIR = openclawStateDir
process.env.OPENCLAW_CONFIG_PATH = join(openclawStateDir, 'openclaw.json')
const config = `{
agents: {
@@ -116,14 +127,17 @@ describe('resolveClawdbotSkillRoots', () => {
})
it('respects CLAWDBOT_STATE_DIR and CLAWDBOT_CONFIG_PATH overrides', async () => {
const base = await mkdtemp(join(tmpdir(), 'clawdhub-clawdbot-override-'))
const base = await mkdtemp(join(tmpdir(), 'clawhub-clawdbot-override-'))
const home = join(base, 'home')
const stateDir = join(base, 'custom-state')
const configPath = join(base, 'config', 'clawdbot.json')
const openclawStateDir = join(base, 'openclaw-state')
process.env.HOME = home
process.env.CLAWDBOT_STATE_DIR = stateDir
process.env.CLAWDBOT_CONFIG_PATH = configPath
process.env.OPENCLAW_STATE_DIR = openclawStateDir
process.env.OPENCLAW_CONFIG_PATH = join(openclawStateDir, 'openclaw.json')
const config = `{
agent: { workspace: "${join(base, 'workspace-main')}" },
@@ -136,24 +150,54 @@ describe('resolveClawdbotSkillRoots', () => {
expect(roots).toEqual(
expect.arrayContaining([
resolve(stateDir, 'skills'),
resolve(openclawStateDir, 'skills'),
resolve(join(base, 'workspace-main'), 'skills'),
]),
)
expect(labels[resolve(stateDir, 'skills')]).toBe('Shared skills')
expect(labels[resolve(openclawStateDir, 'skills')]).toBe('OpenClaw: Shared skills')
expect(labels[resolve(join(base, 'workspace-main'), 'skills')]).toBe('Agent: main')
})
it('returns shared skills root when config is missing', async () => {
const base = await mkdtemp(join(tmpdir(), 'clawdhub-clawdbot-missing-'))
const base = await mkdtemp(join(tmpdir(), 'clawhub-clawdbot-missing-'))
const stateDir = join(base, 'state')
const configPath = join(base, 'missing', 'clawdbot.json')
const openclawStateDir = join(base, 'openclaw-state')
process.env.CLAWDBOT_STATE_DIR = stateDir
process.env.CLAWDBOT_CONFIG_PATH = configPath
process.env.OPENCLAW_STATE_DIR = openclawStateDir
process.env.OPENCLAW_CONFIG_PATH = join(openclawStateDir, 'openclaw.json')
const { roots, labels } = await resolveClawdbotSkillRoots()
expect(roots).toEqual([resolve(stateDir, 'skills')])
expect(roots).toEqual([resolve(stateDir, 'skills'), resolve(openclawStateDir, 'skills')])
expect(labels[resolve(stateDir, 'skills')]).toBe('Shared skills')
expect(labels[resolve(openclawStateDir, 'skills')]).toBe('OpenClaw: Shared skills')
})
it('supports OpenClaw configuration files', async () => {
const base = await mkdtemp(join(tmpdir(), 'clawhub-openclaw-'))
const stateDir = join(base, 'openclaw-state')
const workspace = join(base, 'openclaw-main')
const configPath = join(stateDir, 'openclaw.json')
process.env.OPENCLAW_STATE_DIR = stateDir
await mkdir(stateDir, { recursive: true })
const config = `{
agents: {
defaults: { workspace: "${workspace}", },
},
}`
await writeFile(configPath, config, 'utf8')
const { roots, labels } = await resolveClawdbotSkillRoots()
expect(roots).toEqual(
expect.arrayContaining([resolve(stateDir, 'skills'), resolve(workspace, 'skills')]),
)
expect(labels[resolve(stateDir, 'skills')]).toBe('OpenClaw: Shared skills')
expect(labels[resolve(workspace, 'skills')]).toBe('OpenClaw: Agent: main')
})
})
+93 -37
View File
@@ -39,42 +39,25 @@ export async function resolveClawdbotSkillRoots(): Promise<ClawdbotSkillRoots> {
const roots: string[] = []
const labels: Record<string, string> = {}
const stateDir = resolveClawdbotStateDir()
const sharedSkills = resolveUserPath(join(stateDir, 'skills'))
const clawdbotStateDir = resolveClawdbotStateDir()
const sharedSkills = resolveUserPath(join(clawdbotStateDir, 'skills'))
pushRoot(roots, labels, sharedSkills, 'Shared skills')
const config = await readClawdbotConfig()
if (!config) return { roots, labels }
const openclawStateDir = resolveOpenclawStateDir()
const openclawShared = resolveUserPath(join(openclawStateDir, 'skills'))
pushRoot(roots, labels, openclawShared, 'OpenClaw: Shared skills')
const mainWorkspace = resolveUserPath(
config.agents?.defaults?.workspace ?? config.agent?.workspace ?? '',
)
if (mainWorkspace) {
pushRoot(roots, labels, join(mainWorkspace, 'skills'), 'Agent: main')
const [clawdbotConfig, openclawConfig] = await Promise.all([
readClawdbotConfig(),
readOpenclawConfig(),
])
if (!clawdbotConfig && !openclawConfig) return { roots, labels }
if (clawdbotConfig) {
addConfigRoots(clawdbotConfig, roots, labels)
}
const listedAgents = config.agents?.list ?? []
for (const entry of listedAgents) {
const workspace = resolveUserPath(entry?.workspace ?? '')
if (!workspace) continue
const name = entry?.name?.trim() || entry?.id?.trim() || 'agent'
pushRoot(roots, labels, join(workspace, 'skills'), `Agent: ${name}`)
}
const agents = config.routing?.agents ?? {}
for (const [agentId, entry] of Object.entries(agents)) {
const workspace = resolveUserPath(entry?.workspace ?? '')
if (!workspace) continue
const name = entry?.name?.trim() || agentId
pushRoot(roots, labels, join(workspace, 'skills'), `Agent: ${name}`)
}
const extraDirs = config.skills?.load?.extraDirs ?? []
for (const dir of extraDirs) {
const resolved = resolveUserPath(String(dir))
if (!resolved) continue
const label = `Extra: ${basename(resolved) || resolved}`
pushRoot(roots, labels, resolved, label)
if (openclawConfig) {
addConfigRoots(openclawConfig, roots, labels, 'OpenClaw')
}
return { roots, labels }
@@ -82,18 +65,31 @@ export async function resolveClawdbotSkillRoots(): Promise<ClawdbotSkillRoots> {
export async function resolveClawdbotDefaultWorkspace(): Promise<string | null> {
const config = await readClawdbotConfig()
if (!config) return null
const openclawConfig = await readOpenclawConfig()
if (!config && !openclawConfig) return null
const defaultsWorkspace = resolveUserPath(
config.agents?.defaults?.workspace ?? config.agent?.workspace ?? '',
config?.agents?.defaults?.workspace ?? config?.agent?.workspace ?? '',
)
if (defaultsWorkspace) return defaultsWorkspace
const listedAgents = config.agents?.list ?? []
const listedAgents = config?.agents?.list ?? []
const defaultAgent =
listedAgents.find((entry) => entry.default) ?? listedAgents.find((entry) => entry.id === 'main')
const listWorkspace = resolveUserPath(defaultAgent?.workspace ?? '')
return listWorkspace || null
if (listWorkspace) return listWorkspace
if (!openclawConfig) return null
const openclawDefaults = resolveUserPath(
openclawConfig.agents?.defaults?.workspace ?? openclawConfig.agent?.workspace ?? '',
)
if (openclawDefaults) return openclawDefaults
const openclawAgents = openclawConfig.agents?.list ?? []
const openclawDefaultAgent =
openclawAgents.find((entry) => entry.default) ??
openclawAgents.find((entry) => entry.id === 'main')
const openclawWorkspace = resolveUserPath(openclawDefaultAgent?.workspace ?? '')
return openclawWorkspace || null
}
function resolveClawdbotStateDir() {
@@ -108,6 +104,18 @@ function resolveClawdbotConfigPath() {
return join(resolveClawdbotStateDir(), 'clawdbot.json')
}
function resolveOpenclawStateDir() {
const override = process.env.OPENCLAW_STATE_DIR?.trim()
if (override) return resolveUserPath(override)
return join(homedir(), '.openclaw')
}
function resolveOpenclawConfigPath() {
const override = process.env.OPENCLAW_CONFIG_PATH?.trim()
if (override) return resolveUserPath(override)
return join(resolveOpenclawStateDir(), 'openclaw.json')
}
function resolveUserPath(input: string) {
const trimmed = input.trim()
if (!trimmed) return ''
@@ -118,8 +126,16 @@ function resolveUserPath(input: string) {
}
async function readClawdbotConfig(): Promise<ClawdbotConfig | null> {
return readConfigFile(resolveClawdbotConfigPath())
}
async function readOpenclawConfig(): Promise<ClawdbotConfig | null> {
return readConfigFile(resolveOpenclawConfigPath())
}
async function readConfigFile(path: string): Promise<ClawdbotConfig | null> {
try {
const raw = await readFile(resolveClawdbotConfigPath(), 'utf8')
const raw = await readFile(path, 'utf8')
const parsed = JSON5.parse(raw)
if (!parsed || typeof parsed !== 'object') return null
return parsed as ClawdbotConfig
@@ -128,6 +144,46 @@ async function readClawdbotConfig(): Promise<ClawdbotConfig | null> {
}
}
function addConfigRoots(
config: ClawdbotConfig,
roots: string[],
labels: Record<string, string>,
labelPrefix?: string,
) {
const prefix = labelPrefix ? `${labelPrefix}: ` : ''
const mainWorkspace = resolveUserPath(
config.agents?.defaults?.workspace ?? config.agent?.workspace ?? '',
)
if (mainWorkspace) {
pushRoot(roots, labels, join(mainWorkspace, 'skills'), `${prefix}Agent: main`)
}
const listedAgents = config.agents?.list ?? []
for (const entry of listedAgents) {
const workspace = resolveUserPath(entry?.workspace ?? '')
if (!workspace) continue
const name = entry?.name?.trim() || entry?.id?.trim() || 'agent'
pushRoot(roots, labels, join(workspace, 'skills'), `${prefix}Agent: ${name}`)
}
const agents = config.routing?.agents ?? {}
for (const [agentId, entry] of Object.entries(agents)) {
const workspace = resolveUserPath(entry?.workspace ?? '')
if (!workspace) continue
const name = entry?.name?.trim() || agentId
pushRoot(roots, labels, join(workspace, 'skills'), `${prefix}Agent: ${name}`)
}
const extraDirs = config.skills?.load?.extraDirs ?? []
for (const dir of extraDirs) {
const resolved = resolveUserPath(String(dir))
if (!resolved) continue
const label = `${prefix}Extra: ${basename(resolved) || resolved}`
pushRoot(roots, labels, resolved, label)
}
}
function pushRoot(roots: string[], labels: Record<string, string>, root: string, label?: string) {
const resolved = resolveUserPath(root)
if (!resolved) return
+2 -2
View File
@@ -47,7 +47,7 @@ export async function cmdLogin(
) {
if (!tokenFlag && !inputAllowed) fail('Token required (use --token or remove --no-input)')
const token = tokenFlag || (await promptHidden('ClawdHub token: '))
const token = tokenFlag || (await promptHidden('OpenClaw token: '))
if (!token) fail('Token required')
const registry = await getRegistry(opts, { cache: true })
@@ -79,7 +79,7 @@ export async function cmdLogout(opts: GlobalOpts) {
export async function cmdWhoami(opts: GlobalOpts) {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawdhub login')
if (!token) fail('Not logged in. Run: clawhub login')
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner('Checking token')
@@ -4,11 +4,11 @@ import { afterEach, describe, expect, it, vi } from 'vitest'
import type { GlobalOpts } from '../types'
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawdhub.com', token: 'tkn' })),
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
}))
vi.mock('../registry.js', () => ({
getRegistry: vi.fn(async () => 'https://clawdhub.com'),
getRegistry: vi.fn(async () => 'https://clawhub.ai'),
}))
const mockApiRequest = vi.fn()
@@ -35,8 +35,8 @@ function makeOpts(): GlobalOpts {
return {
workdir: '/work',
dir: '/work/skills',
site: 'https://clawdhub.com',
registry: 'https://clawdhub.com',
site: 'https://clawhub.ai',
registry: 'https://clawhub.ai',
registrySource: 'default',
}
}
+1 -1
View File
@@ -8,7 +8,7 @@ import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawdhub login')
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
@@ -7,10 +7,10 @@ import { afterEach, describe, expect, it, vi } from 'vitest'
import type { GlobalOpts } from '../types'
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawdhub.com', token: 'tkn' })),
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
}))
const mockGetRegistry = vi.fn(async (_opts: unknown, _params?: unknown) => 'https://clawdhub.com')
const mockGetRegistry = vi.fn(async (_opts: unknown, _params?: unknown) => 'https://clawhub.ai')
vi.mock('../registry.js', () => ({
getRegistry: (opts: unknown, params?: unknown) => mockGetRegistry(opts, params),
}))
@@ -34,7 +34,7 @@ vi.mock('../ui.js', () => ({
const { cmdPublish } = await import('./publish')
async function makeTmpWorkdir() {
const root = await mkdtemp(join(tmpdir(), 'clawdhub-publish-'))
const root = await mkdtemp(join(tmpdir(), 'clawhub-publish-'))
return root
}
@@ -42,8 +42,8 @@ function makeOpts(workdir: string): GlobalOpts {
return {
workdir,
dir: join(workdir, 'skills'),
site: 'https://clawdhub.com',
registry: 'https://clawdhub.com',
site: 'https://clawhub.ai',
registry: 'https://clawhub.ai',
registrySource: 'default',
}
}
@@ -29,7 +29,7 @@ export async function cmdPublish(
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawdhub login')
if (!token) fail('Not logged in. Run: clawhub login')
const registry = await getRegistry(opts, { cache: true })
const slug = options.slug ?? sanitizeSlug(basename(folder))
@@ -0,0 +1,191 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from 'vitest'
import { ApiRoutes } from '../../schema/index.js'
import type { GlobalOpts } from '../types'
const mockApiRequest = vi.fn()
const mockDownloadZip = vi.fn()
vi.mock('../../http.js', () => ({
apiRequest: (...args: unknown[]) => mockApiRequest(...args),
downloadZip: (...args: unknown[]) => mockDownloadZip(...args),
}))
const mockGetRegistry = vi.fn(async () => 'https://clawhub.ai')
vi.mock('../registry.js', () => ({
getRegistry: () => mockGetRegistry(),
}))
const mockSpinner = {
stop: vi.fn(),
fail: vi.fn(),
start: vi.fn(),
succeed: vi.fn(),
isSpinning: false,
text: '',
}
vi.mock('../ui.js', () => ({
createSpinner: vi.fn(() => mockSpinner),
fail: (message: string) => {
throw new Error(message)
},
formatError: (error: unknown) => (error instanceof Error ? error.message : String(error)),
isInteractive: () => false,
promptConfirm: vi.fn(async () => false),
}))
vi.mock('../../skills.js', () => ({
extractZipToDir: vi.fn(),
hashSkillFiles: vi.fn(),
listTextFiles: vi.fn(),
readLockfile: vi.fn(),
readSkillOrigin: vi.fn(),
writeLockfile: vi.fn(),
writeSkillOrigin: vi.fn(),
}))
vi.mock('node:fs/promises', () => ({
mkdir: vi.fn(),
rm: vi.fn(),
stat: vi.fn(),
}))
const { clampLimit, cmdExplore, cmdUpdate, formatExploreLine } = await import('./skills')
const {
extractZipToDir,
hashSkillFiles,
listTextFiles,
readLockfile,
readSkillOrigin,
writeLockfile,
writeSkillOrigin,
} = await import('../../skills.js')
const { rm, stat } = await import('node:fs/promises')
const mockLog = vi.spyOn(console, 'log').mockImplementation(() => {})
function makeOpts(): GlobalOpts {
return {
workdir: '/work',
dir: '/work/skills',
site: 'https://clawhub.ai',
registry: 'https://clawhub.ai',
registrySource: 'default',
}
}
afterEach(() => {
vi.clearAllMocks()
})
describe('explore helpers', () => {
it('clamps explore limits and handles non-finite values', () => {
expect(clampLimit(-5)).toBe(1)
expect(clampLimit(0)).toBe(1)
expect(clampLimit(1)).toBe(1)
expect(clampLimit(50)).toBe(50)
expect(clampLimit(99)).toBe(99)
expect(clampLimit(200)).toBe(200)
expect(clampLimit(250)).toBe(200)
expect(clampLimit(Number.NaN)).toBe(25)
expect(clampLimit(Number.POSITIVE_INFINITY)).toBe(25)
expect(clampLimit(Number.NaN, 10)).toBe(10)
})
it('formats explore lines with relative time and truncation', () => {
const now = 4 * 60 * 60 * 1000
const nowSpy = vi.spyOn(Date, 'now').mockReturnValue(now)
const summary = 'a'.repeat(60)
const line = formatExploreLine({
slug: 'weather',
summary,
updatedAt: now - 2 * 60 * 60 * 1000,
latestVersion: null,
})
expect(line).toBe(`weather v? 2h ago ${'a'.repeat(49)}`)
nowSpy.mockRestore()
})
})
describe('cmdExplore', () => {
it('clamps limit and handles empty results', async () => {
mockApiRequest.mockResolvedValue({ items: [] })
await cmdExplore(makeOpts(), { limit: 0 })
const [, args] = mockApiRequest.mock.calls[0] ?? []
const url = new URL(String(args?.url))
expect(url.searchParams.get('limit')).toBe('1')
expect(mockLog).toHaveBeenCalledWith('No skills found.')
})
it('prints formatted results', async () => {
const now = 10 * 60 * 1000
const nowSpy = vi.spyOn(Date, 'now').mockReturnValue(now)
const item = {
slug: 'gog',
summary: 'Google Workspace CLI for Gmail, Calendar, Drive and more.',
updatedAt: now - 90 * 1000,
latestVersion: { version: '1.2.3' },
}
mockApiRequest.mockResolvedValue({ items: [item] })
await cmdExplore(makeOpts(), { limit: 250 })
const [, args] = mockApiRequest.mock.calls[0] ?? []
const url = new URL(String(args?.url))
expect(url.searchParams.get('limit')).toBe('200')
expect(mockLog).toHaveBeenCalledWith(formatExploreLine(item))
nowSpy.mockRestore()
})
it('supports sort and json output', async () => {
const payload = { items: [], nextCursor: null }
mockApiRequest.mockResolvedValue(payload)
await cmdExplore(makeOpts(), { limit: 10, sort: 'installs', json: true })
const [, args] = mockApiRequest.mock.calls[0] ?? []
const url = new URL(String(args?.url))
expect(url.searchParams.get('limit')).toBe('10')
expect(url.searchParams.get('sort')).toBe('installsCurrent')
expect(mockLog).toHaveBeenCalledWith(JSON.stringify(payload, null, 2))
})
it('supports all-time installs and trending sorts', async () => {
mockApiRequest.mockResolvedValue({ items: [], nextCursor: null })
await cmdExplore(makeOpts(), { limit: 5, sort: 'installsAllTime' })
await cmdExplore(makeOpts(), { limit: 5, sort: 'trending' })
const first = new URL(String(mockApiRequest.mock.calls[0]?.[1]?.url))
const second = new URL(String(mockApiRequest.mock.calls[1]?.[1]?.url))
expect(first.searchParams.get('sort')).toBe('installsAllTime')
expect(second.searchParams.get('sort')).toBe('trending')
})
})
describe('cmdUpdate', () => {
it('uses path-based skill lookup when no local fingerprint is available', async () => {
mockApiRequest.mockResolvedValue({ latestVersion: { version: '1.0.0' } })
mockDownloadZip.mockResolvedValue(new Uint8Array([1, 2, 3]))
vi.mocked(readLockfile).mockResolvedValue({
version: 1,
skills: { demo: { version: '0.1.0', installedAt: 123 } },
})
vi.mocked(writeLockfile).mockResolvedValue()
vi.mocked(readSkillOrigin).mockResolvedValue(null)
vi.mocked(writeSkillOrigin).mockResolvedValue()
vi.mocked(extractZipToDir).mockResolvedValue()
vi.mocked(listTextFiles).mockResolvedValue([])
vi.mocked(hashSkillFiles).mockReturnValue({ fingerprint: 'hash', files: [] })
vi.mocked(stat).mockRejectedValue(new Error('missing'))
vi.mocked(rm).mockResolvedValue()
await cmdUpdate(makeOpts(), 'demo', {}, false)
const [, args] = mockApiRequest.mock.calls[0] ?? []
expect(args?.path).toBe(`${ApiRoutes.skills}/${encodeURIComponent('demo')}`)
expect(args?.url).toBeUndefined()
})
})
+119 -1
View File
@@ -5,6 +5,7 @@ import { apiRequest, downloadZip } from '../../http.js'
import {
ApiRoutes,
ApiV1SearchResponseSchema,
ApiV1SkillListResponseSchema,
ApiV1SkillResolveResponseSchema,
ApiV1SkillResponseSchema,
} from '../../schema/index.js'
@@ -152,7 +153,7 @@ export async function cmdUpdate(
} else {
const meta = await apiRequest(
registry,
{ method: 'GET', url: `${ApiRoutes.skills}/${encodeURIComponent(entry)}` },
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(entry)}` },
ApiV1SkillResponseSchema,
)
resolveResult = { match: null, latestVersion: meta.latestVersion ?? null }
@@ -241,6 +242,123 @@ export async function cmdList(opts: GlobalOpts) {
}
}
type ExploreSort = 'newest' | 'downloads' | 'rating' | 'installs' | 'installsAllTime' | 'trending'
type ApiExploreSort =
| 'updated'
| 'downloads'
| 'stars'
| 'installsCurrent'
| 'installsAllTime'
| 'trending'
export async function cmdExplore(
opts: GlobalOpts,
options: { limit?: number; sort?: string; json?: boolean } = {},
) {
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner('Fetching latest skills')
try {
const url = new URL(ApiRoutes.skills, registry)
const boundedLimit = clampLimit(options.limit ?? 25)
const { apiSort } = resolveExploreSort(options.sort)
url.searchParams.set('limit', String(boundedLimit))
if (apiSort !== 'updated') url.searchParams.set('sort', apiSort)
const result = await apiRequest(
registry,
{ method: 'GET', url: url.toString() },
ApiV1SkillListResponseSchema,
)
spinner.stop()
if (options.json) {
console.log(JSON.stringify(result, null, 2))
return
}
if (result.items.length === 0) {
console.log('No skills found.')
return
}
for (const item of result.items) {
console.log(formatExploreLine(item))
}
} catch (error) {
spinner.fail(formatError(error))
throw error
}
}
export function formatExploreLine(item: {
slug: string
summary?: string | null
updatedAt: number
latestVersion?: { version: string } | null
}) {
const version = item.latestVersion?.version ?? '?'
const age = formatRelativeTime(item.updatedAt)
const summary = item.summary ? ` ${truncate(item.summary, 50)}` : ''
return `${item.slug} v${version} ${age}${summary}`
}
export function clampLimit(limit: number, fallback = 25) {
if (!Number.isFinite(limit)) return fallback
return Math.min(Math.max(1, limit), 200)
}
function formatRelativeTime(timestamp: number): string {
const now = Date.now()
const diff = now - timestamp
const seconds = Math.floor(diff / 1000)
const minutes = Math.floor(seconds / 60)
const hours = Math.floor(minutes / 60)
const days = Math.floor(hours / 24)
if (days > 30) {
const months = Math.floor(days / 30)
return `${months}mo ago`
}
if (days > 0) return `${days}d ago`
if (hours > 0) return `${hours}h ago`
if (minutes > 0) return `${minutes}m ago`
return 'just now'
}
function truncate(str: string, maxLen: number): string {
if (str.length <= maxLen) return str
return `${str.slice(0, maxLen - 1)}`
}
function resolveExploreSort(raw?: string): { sort: ExploreSort; apiSort: ApiExploreSort } {
const normalized = raw?.trim().toLowerCase()
if (!normalized || normalized === 'newest' || normalized === 'updated') {
return { sort: 'newest', apiSort: 'updated' }
}
if (normalized === 'downloads' || normalized === 'download') {
return { sort: 'downloads', apiSort: 'downloads' }
}
if (normalized === 'rating' || normalized === 'stars' || normalized === 'star') {
return { sort: 'rating', apiSort: 'stars' }
}
if (
normalized === 'installs' ||
normalized === 'install' ||
normalized === 'installscurrent' ||
normalized === 'installs-current' ||
normalized === 'current'
) {
return { sort: 'installs', apiSort: 'installsCurrent' }
}
if (normalized === 'installsalltime' || normalized === 'installs-all-time') {
return { sort: 'installsAllTime', apiSort: 'installsAllTime' }
}
if (normalized === 'trending') {
return { sort: 'trending', apiSort: 'trending' }
}
fail(
`Invalid sort "${raw}". Use newest, downloads, rating, installs, installsAllTime, or trending.`,
)
}
async function resolveSkillVersion(registry: string, slug: string, hash: string) {
const url = new URL(ApiRoutes.resolve, registry)
url.searchParams.set('slug', slug)
@@ -0,0 +1,46 @@
import { readGlobalConfig } from '../../config.js'
import { apiRequest } from '../../http.js'
import { ApiRoutes, ApiV1StarResponseSchema } from '../../schema/index.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
export async function cmdStarSkill(
opts: GlobalOpts,
slugArg: string,
options: { yes?: boolean },
inputAllowed: boolean,
) {
const slug = slugArg.trim().toLowerCase()
if (!slug) fail('Slug required')
const allowPrompt = isInteractive() && inputAllowed !== false
if (!options.yes) {
if (!allowPrompt) fail('Pass --yes (no input)')
const ok = await promptConfirm(`Star ${slug}?`)
if (!ok) return
}
const token = await requireToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner(`Starring ${slug}`)
try {
const result = await apiRequest(
registry,
{ method: 'POST', path: `${ApiRoutes.stars}/${encodeURIComponent(slug)}`, token },
ApiV1StarResponseSchema,
)
spinner.succeed(result.alreadyStarred ? `OK. ${slug} already starred.` : `OK. Starred ${slug}`)
return result
} catch (error) {
spinner.fail(formatError(error))
throw error
}
}
@@ -27,10 +27,10 @@ vi.mock('@clack/prompts', () => ({
}))
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawdhub.com', token: 'tkn' })),
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
}))
const mockGetRegistry = vi.fn(async () => 'https://clawdhub.com')
const mockGetRegistry = vi.fn(async () => 'https://clawhub.ai')
vi.mock('../registry.js', () => ({
getRegistry: () => mockGetRegistry(),
}))
@@ -89,8 +89,8 @@ function makeOpts(): GlobalOpts {
return {
workdir: '/work',
dir: '/work/skills',
site: 'https://clawdhub.com',
registry: 'https://clawdhub.com',
site: 'https://clawhub.ai',
registry: 'https://clawhub.ai',
registrySource: 'default',
}
}
@@ -290,9 +290,9 @@ describe('cmdSync', () => {
expect(update.changelog).toBe('')
})
it('skips telemetry when CLAWDHUB_DISABLE_TELEMETRY is set', async () => {
it('skips telemetry when CLAWHUB_DISABLE_TELEMETRY is set', async () => {
interactive = false
process.env.CLAWDHUB_DISABLE_TELEMETRY = '1'
process.env.CLAWHUB_DISABLE_TELEMETRY = '1'
mockApiRequest.mockImplementation(async (_registry: string, args: { path: string }) => {
if (args.path === '/api/v1/whoami') return { user: { handle: 'steipete' } }
if (args.path.startsWith('/api/v1/resolve?')) {
@@ -305,6 +305,6 @@ describe('cmdSync', () => {
expect(
mockApiRequest.mock.calls.some((call) => call[1]?.path === '/api/cli/telemetry/sync'),
).toBe(false)
delete process.env.CLAWDHUB_DISABLE_TELEMETRY
delete process.env.CLAWHUB_DISABLE_TELEMETRY
})
})
+2 -2
View File
@@ -30,11 +30,11 @@ import type { Candidate, LocalSkill, SyncOptions } from './syncTypes.js'
export async function cmdSync(opts: GlobalOpts, options: SyncOptions, inputAllowed: boolean) {
const allowPrompt = isInteractive() && inputAllowed !== false
intro('ClawdHub sync')
intro('OpenClaw sync')
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawdhub login')
if (!token) fail('Not logged in. Run: clawhub login')
const registry = await getRegistryWithAuth(opts, token)
const selectedRoots = buildScanRoots(opts, options.root)
@@ -58,7 +58,7 @@ export async function reportTelemetryIfEnabled(params: {
}
function isTelemetryDisabled() {
const raw = process.env.CLAWDHUB_DISABLE_TELEMETRY
const raw = process.env.CLAWHUB_DISABLE_TELEMETRY ?? process.env.CLAWDHUB_DISABLE_TELEMETRY
if (!raw) return false
return ['1', 'true', 'yes', 'on'].includes(raw.trim().toLowerCase())
}
@@ -0,0 +1,48 @@
import { readGlobalConfig } from '../../config.js'
import { apiRequest } from '../../http.js'
import { ApiRoutes, ApiV1UnstarResponseSchema } from '../../schema/index.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
export async function cmdUnstarSkill(
opts: GlobalOpts,
slugArg: string,
options: { yes?: boolean },
inputAllowed: boolean,
) {
const slug = slugArg.trim().toLowerCase()
if (!slug) fail('Slug required')
const allowPrompt = isInteractive() && inputAllowed !== false
if (!options.yes) {
if (!allowPrompt) fail('Pass --yes (no input)')
const ok = await promptConfirm(`Unstar ${slug}?`)
if (!ok) return
}
const token = await requireToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner(`Unstarring ${slug}`)
try {
const result = await apiRequest(
registry,
{ method: 'DELETE', path: `${ApiRoutes.stars}/${encodeURIComponent(slug)}`, token },
ApiV1UnstarResponseSchema,
)
spinner.succeed(
result.alreadyUnstarred ? `OK. ${slug} already unstarred.` : `OK. Unstarred ${slug}`,
)
return result
} catch (error) {
spinner.fail(formatError(error))
throw error
}
}
+5 -5
View File
@@ -22,7 +22,7 @@ function makeOpts(overrides: Partial<GlobalOpts> = {}): GlobalOpts {
return {
workdir: '/work',
dir: '/work/skills',
site: 'https://clawdhub.com',
site: 'https://clawhub.ai',
registry: DEFAULT_REGISTRY,
registrySource: 'default',
...overrides,
@@ -38,7 +38,7 @@ beforeEach(() => {
describe('registry resolution', () => {
it('prefers explicit registry over discovery/cache', async () => {
readGlobalConfig.mockResolvedValue({ registry: 'https://auth.clawdhub.com' })
discoverRegistryFromSite.mockResolvedValue({ apiBase: 'https://clawdhub.com' })
discoverRegistryFromSite.mockResolvedValue({ apiBase: 'https://clawhub.ai' })
const registry = await resolveRegistry(
makeOpts({ registry: 'https://custom.example', registrySource: 'cli' }),
@@ -50,13 +50,13 @@ describe('registry resolution', () => {
it('ignores legacy registry and updates cache from discovery', async () => {
readGlobalConfig.mockResolvedValue({ registry: 'https://auth.clawdhub.com', token: 'tkn' })
discoverRegistryFromSite.mockResolvedValue({ apiBase: 'https://clawdhub.com' })
discoverRegistryFromSite.mockResolvedValue({ apiBase: 'https://clawhub.ai' })
const registry = await getRegistry(makeOpts(), { cache: true })
expect(registry).toBe('https://clawdhub.com')
expect(registry).toBe('https://clawhub.ai')
expect(writeGlobalConfig).toHaveBeenCalledWith({
registry: 'https://clawdhub.com',
registry: 'https://clawhub.ai',
token: 'tkn',
})
})
+3 -3
View File
@@ -2,9 +2,9 @@ import { readGlobalConfig, writeGlobalConfig } from '../config.js'
import { discoverRegistryFromSite } from '../discovery.js'
import type { GlobalOpts } from './types.js'
export const DEFAULT_SITE = 'https://clawdhub.com'
export const DEFAULT_REGISTRY = 'https://clawdhub.com'
const LEGACY_REGISTRY_HOSTS = new Set(['auth.clawdhub.com'])
export const DEFAULT_SITE = 'https://clawhub.ai'
export const DEFAULT_REGISTRY = 'https://clawhub.ai'
const LEGACY_REGISTRY_HOSTS = new Set(['auth.clawdhub.com', 'auth.clawhub.com', 'auth.clawhub.ai'])
export async function resolveRegistry(opts: GlobalOpts) {
const explicit = opts.registrySource !== 'default' ? opts.registry.trim() : ''
+3 -1
View File
@@ -7,7 +7,7 @@ import { describe, expect, it } from 'vitest'
import { findSkillFolders, getFallbackSkillRoots } from './scanSkills'
async function makeTmpDir() {
return mkdtemp(join(tmpdir(), 'clawdhub-scan-'))
return mkdtemp(join(tmpdir(), 'clawhub-scan-'))
}
describe('scanSkills', () => {
@@ -60,5 +60,7 @@ describe('scanSkills', () => {
expect(roots.some((p) => p.endsWith('/clawdis/skills'))).toBe(true)
expect(roots.some((p) => p.endsWith('/clawd/skills'))).toBe(true)
expect(roots.some((p) => p.endsWith('/clawdbot/skills'))).toBe(true)
expect(roots.some((p) => p.endsWith('/openclaw/skills'))).toBe(true)
expect(roots.some((p) => p.endsWith('/moltbot/skills'))).toBe(true)
})
})
+18
View File
@@ -37,6 +37,10 @@ export function getFallbackSkillRoots(workdir: string) {
resolve(workdir, '..', 'clawdis', 'Skills'),
resolve(workdir, '..', 'clawdbot', 'skills'),
resolve(workdir, '..', 'clawdbot', 'Skills'),
resolve(workdir, '..', 'openclaw', 'skills'),
resolve(workdir, '..', 'openclaw', 'Skills'),
resolve(workdir, '..', 'moltbot', 'skills'),
resolve(workdir, '..', 'moltbot', 'Skills'),
// legacy locations
resolve(home, 'clawd', 'skills'),
@@ -54,11 +58,25 @@ export function getFallbackSkillRoots(workdir: string) {
resolve(home, '.clawdis', 'skills'),
resolve(home, '.clawdis', 'Skills'),
resolve(home, 'openclaw', 'skills'),
resolve(home, 'openclaw', 'Skills'),
resolve(home, '.openclaw', 'skills'),
resolve(home, '.openclaw', 'Skills'),
resolve(home, 'moltbot', 'skills'),
resolve(home, 'moltbot', 'Skills'),
resolve(home, '.moltbot', 'skills'),
resolve(home, '.moltbot', 'Skills'),
// macOS App Support legacy
resolve(home, 'Library', 'Application Support', 'clawdbot', 'skills'),
resolve(home, 'Library', 'Application Support', 'clawdbot', 'Skills'),
resolve(home, 'Library', 'Application Support', 'clawdis', 'skills'),
resolve(home, 'Library', 'Application Support', 'clawdis', 'Skills'),
resolve(home, 'Library', 'Application Support', 'openclaw', 'skills'),
resolve(home, 'Library', 'Application Support', 'openclaw', 'Skills'),
resolve(home, 'Library', 'Application Support', 'moltbot', 'skills'),
resolve(home, 'Library', 'Application Support', 'moltbot', 'Skills'),
]
return Array.from(new Set(roots))
}
+27 -5
View File
@@ -1,22 +1,44 @@
import { existsSync } from 'node:fs'
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { homedir } from 'node:os'
import { dirname, join, resolve } from 'node:path'
import { type GlobalConfig, GlobalConfigSchema, parseArk } from './schema/index.js'
export function getGlobalConfigPath() {
const override = process.env.CLAWDHUB_CONFIG_PATH?.trim()
const override =
process.env.CLAWHUB_CONFIG_PATH?.trim() ?? process.env.CLAWDHUB_CONFIG_PATH?.trim()
if (override) return resolve(override)
const home = homedir()
if (process.platform === 'darwin') {
return join(home, 'Library', 'Application Support', 'clawdhub', 'config.json')
const clawhubPath = join(home, 'Library', 'Application Support', 'clawhub', 'config.json')
const clawdhubPath = join(home, 'Library', 'Application Support', 'clawdhub', 'config.json')
if (existsSync(clawhubPath)) return clawhubPath
if (existsSync(clawdhubPath)) return clawdhubPath
return clawhubPath
}
const xdg = process.env.XDG_CONFIG_HOME
if (xdg) return join(xdg, 'clawdhub', 'config.json')
if (xdg) {
const clawhubPath = join(xdg, 'clawhub', 'config.json')
const clawdhubPath = join(xdg, 'clawdhub', 'config.json')
if (existsSync(clawhubPath)) return clawhubPath
if (existsSync(clawdhubPath)) return clawdhubPath
return clawhubPath
}
if (process.platform === 'win32') {
const appData = process.env.APPDATA
if (appData) return join(appData, 'clawdhub', 'config.json')
if (appData) {
const clawhubPath = join(appData, 'clawhub', 'config.json')
const clawdhubPath = join(appData, 'clawdhub', 'config.json')
if (existsSync(clawhubPath)) return clawhubPath
if (existsSync(clawdhubPath)) return clawdhubPath
return clawhubPath
}
}
return join(home, '.config', 'clawdhub', 'config.json')
const clawhubPath = join(home, '.config', 'clawhub', 'config.json')
const clawdhubPath = join(home, '.config', 'clawdhub', 'config.json')
if (existsSync(clawhubPath)) return clawhubPath
if (existsSync(clawdhubPath)) return clawdhubPath
return clawhubPath
}
export async function readGlobalConfig(): Promise<GlobalConfig | null> {
+18 -14
View File
@@ -1,19 +1,23 @@
import { parseArk, WellKnownConfigSchema } from './schema/index.js'
export async function discoverRegistryFromSite(siteUrl: string) {
const url = new URL('/.well-known/clawdhub.json', siteUrl)
const response = await fetch(url.toString(), {
method: 'GET',
headers: { Accept: 'application/json' },
})
if (!response.ok) return null
const raw = (await response.json()) as unknown
const parsed = parseArk(WellKnownConfigSchema, raw, 'WellKnown config')
const apiBase = 'apiBase' in parsed ? parsed.apiBase : parsed.registry
if (!apiBase) return null
return {
apiBase,
authBase: parsed.authBase,
minCliVersion: parsed.minCliVersion,
const paths = ['/.well-known/clawhub.json', '/.well-known/clawdhub.json']
for (const path of paths) {
const url = new URL(path, siteUrl)
const response = await fetch(url.toString(), {
method: 'GET',
headers: { Accept: 'application/json' },
})
if (!response.ok) continue
const raw = (await response.json()) as unknown
const parsed = parseArk(WellKnownConfigSchema, raw, 'WellKnown config')
const apiBase = 'apiBase' in parsed ? parsed.apiBase : parsed.registry
if (!apiBase) return null
return {
apiBase,
authBase: parsed.authBase,
minCliVersion: parsed.minCliVersion,
}
}
return null
}
+202 -3
View File
@@ -1,7 +1,29 @@
import { spawnSync } from 'node:child_process'
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import pRetry, { AbortError } from 'p-retry'
import { Agent, setGlobalDispatcher } from 'undici'
import type { ArkValidator } from './schema/index.js'
import { ApiRoutes, parseArk } from './schema/index.js'
const REQUEST_TIMEOUT_MS = 15_000
const REQUEST_TIMEOUT_SECONDS = Math.ceil(REQUEST_TIMEOUT_MS / 1000)
const isBun = typeof process !== 'undefined' && Boolean(process.versions?.bun)
if (typeof process !== 'undefined' && process.versions?.node) {
try {
setGlobalDispatcher(
new Agent({
allowH2: true,
connect: { timeout: REQUEST_TIMEOUT_MS },
}),
)
} catch {
// ignore dispatcher setup failures in non-node runtimes
}
}
type RequestArgs =
| { method: 'GET' | 'POST' | 'DELETE'; path: string; token?: string; body?: unknown }
| { method: 'GET' | 'POST' | 'DELETE'; url: string; token?: string; body?: unknown }
@@ -20,6 +42,10 @@ export async function apiRequest<T>(
const url = 'url' in args ? args.url : new URL(args.path, registry).toString()
const json = await pRetry(
async () => {
if (isBun) {
return await fetchJsonViaCurl(url, args)
}
const headers: Record<string, string> = { Accept: 'application/json' }
if (args.token) headers.Authorization = `Bearer ${args.token}`
let body: string | undefined
@@ -27,7 +53,15 @@ export async function apiRequest<T>(
headers['Content-Type'] = 'application/json'
body = JSON.stringify(args.body ?? {})
}
const response = await fetch(url, { method: args.method, headers, body })
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url, {
method: args.method,
headers,
body,
signal: controller.signal,
})
clearTimeout(timeout)
if (!response.ok) {
const text = await response.text().catch(() => '')
const message = text || `HTTP ${response.status}`
@@ -62,9 +96,21 @@ export async function apiRequestForm<T>(
const url = 'url' in args ? args.url : new URL(args.path, registry).toString()
const json = await pRetry(
async () => {
if (isBun) {
return await fetchJsonFormViaCurl(url, args)
}
const headers: Record<string, string> = { Accept: 'application/json' }
if (args.token) headers.Authorization = `Bearer ${args.token}`
const response = await fetch(url, { method: args.method, headers, body: args.form })
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url, {
method: args.method,
headers,
body: args.form,
signal: controller.signal,
})
clearTimeout(timeout)
if (!response.ok) {
const text = await response.text().catch(() => '')
const message = text || `HTTP ${response.status}`
@@ -87,7 +133,14 @@ export async function downloadZip(registry: string, args: { slug: string; versio
if (args.version) url.searchParams.set('version', args.version)
return pRetry(
async () => {
const response = await fetch(url.toString(), { method: 'GET' })
if (isBun) {
return await fetchBinaryViaCurl(url.toString())
}
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url.toString(), { method: 'GET', signal: controller.signal })
clearTimeout(timeout)
if (!response.ok) {
const message = (await response.text().catch(() => '')) || `HTTP ${response.status}`
if (response.status === 429 || response.status >= 500) {
@@ -100,3 +153,149 @@ export async function downloadZip(registry: string, args: { slug: string; versio
{ retries: 2 },
)
}
async function fetchJsonViaCurl(url: string, args: RequestArgs) {
const headers = ['-H', 'Accept: application/json']
if (args.token) {
headers.push('-H', `Authorization: Bearer ${args.token}`)
}
const curlArgs = [
'--silent',
'--show-error',
'--location',
'--max-time',
String(REQUEST_TIMEOUT_SECONDS),
'--write-out',
'\n%{http_code}',
'-X',
args.method,
...headers,
url,
]
if (args.method === 'POST') {
curlArgs.push('-H', 'Content-Type: application/json')
curlArgs.push('--data-binary', JSON.stringify(args.body ?? {}))
}
const result = spawnSync('curl', curlArgs, { encoding: 'utf8' })
if (result.status !== 0) {
throw new Error(result.stderr || 'curl failed')
}
const output = result.stdout ?? ''
const splitAt = output.lastIndexOf('\n')
if (splitAt === -1) throw new Error('curl response missing status')
const body = output.slice(0, splitAt)
const status = Number(output.slice(splitAt + 1).trim())
if (!Number.isFinite(status)) throw new Error('curl response missing status')
if (status < 200 || status >= 300) {
if (status === 429 || status >= 500) {
throw new Error(body || `HTTP ${status}`)
}
throw new AbortError(body || `HTTP ${status}`)
}
return JSON.parse(body || 'null') as unknown
}
async function fetchJsonFormViaCurl(url: string, args: FormRequestArgs) {
const headers = ['-H', 'Accept: application/json']
if (args.token) {
headers.push('-H', `Authorization: Bearer ${args.token}`)
}
const tempDir = await mkdtemp(join(tmpdir(), 'clawhub-upload-'))
try {
const formArgs: string[] = []
for (const [key, value] of args.form.entries()) {
if (value instanceof Blob) {
const filename = typeof (value as File).name === 'string' ? (value as File).name : 'file'
const filePath = join(tempDir, filename)
const bytes = new Uint8Array(await value.arrayBuffer())
await writeFile(filePath, bytes)
formArgs.push('-F', `${key}=@${filePath};filename=${filename}`)
} else {
formArgs.push('-F', `${key}=${String(value)}`)
}
}
const curlArgs = [
'--silent',
'--show-error',
'--location',
'--max-time',
String(REQUEST_TIMEOUT_SECONDS),
'--write-out',
'\n%{http_code}',
'-X',
args.method,
...headers,
...formArgs,
url,
]
const result = spawnSync('curl', curlArgs, { encoding: 'utf8' })
if (result.status !== 0) {
throw new Error(result.stderr || 'curl failed')
}
const output = result.stdout ?? ''
const splitAt = output.lastIndexOf('\n')
if (splitAt === -1) throw new Error('curl response missing status')
const body = output.slice(0, splitAt)
const status = Number(output.slice(splitAt + 1).trim())
if (!Number.isFinite(status)) throw new Error('curl response missing status')
if (status < 200 || status >= 300) {
if (status === 429 || status >= 500) {
throw new Error(body || `HTTP ${status}`)
}
throw new AbortError(body || `HTTP ${status}`)
}
return JSON.parse(body || 'null') as unknown
} finally {
await rm(tempDir, { recursive: true, force: true })
}
}
async function fetchBinaryViaCurl(url: string) {
const tempDir = await mkdtemp(join(tmpdir(), 'clawhub-download-'))
const filePath = join(tempDir, 'payload.bin')
try {
const curlArgs = [
'--silent',
'--show-error',
'--location',
'--max-time',
String(REQUEST_TIMEOUT_SECONDS),
'-o',
filePath,
'--write-out',
'%{http_code}',
url,
]
const result = spawnSync('curl', curlArgs, { encoding: 'utf8' })
if (result.status !== 0) {
throw new Error(result.stderr || 'curl failed')
}
const status = Number((result.stdout ?? '').trim())
if (!Number.isFinite(status)) throw new Error('curl response missing status')
if (status < 200 || status >= 300) {
const body = await readFileSafe(filePath)
const message = body ? new TextDecoder().decode(body) : `HTTP ${status}`
if (status === 429 || status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
}
const bytes = await readFileSafe(filePath)
return bytes ? new Uint8Array(bytes) : new Uint8Array()
} finally {
await rm(tempDir, { recursive: true, force: true })
}
}
async function readFileSafe(path: string) {
try {
const { readFile } = await import('node:fs/promises')
return await readFile(path)
} catch {
return null
}
}
+1
View File
@@ -16,6 +16,7 @@ export const ApiRoutes = {
resolve: '/api/v1/resolve',
download: '/api/v1/download',
skills: '/api/v1/skills',
stars: '/api/v1/stars',
souls: '/api/v1/souls',
whoami: '/api/v1/whoami',
} as const
+12
View File
@@ -215,6 +215,18 @@ export const ApiV1DeleteResponseSchema = type({
ok: 'true',
})
export const ApiV1StarResponseSchema = type({
ok: 'true',
starred: 'boolean',
alreadyStarred: 'boolean',
})
export const ApiV1UnstarResponseSchema = type({
ok: 'true',
unstarred: 'boolean',
alreadyUnstarred: 'boolean',
})
export const SkillInstallSpecSchema = type({
id: 'string?',
kind: '"brew"|"node"|"go"|"uv"',

Some files were not shown because too many files have changed in this diff Show More