fix: add root undici devDependency for e2e (#255) (thanks @tanujbhaud)

Refactor action handlers in convex/vt.ts to use explicit return types, resolving circular type inference (TS7022). Also add undici to devDependencies for E2E tests.

.env.local.example

@@ -1,6 +1,9 @@
 # Frontend
 VITE_CONVEX_URL=
 VITE_CONVEX_SITE_URL=
+VITE_SOULHUB_SITE_URL=
+VITE_SOULHUB_HOST=
+VITE_SITE_MODE=
 SITE_URL=http://localhost:3000
 CONVEX_SITE_URL=
 

.github/workflows/ci.yml

@@ -15,10 +15,12 @@ jobs:
 
       - uses: oven-sh/setup-bun@v2
         with:
-          bun-version: 1.3.5
+          bun-version: 1.3.6
 
       - name: Install
         run: bun install --frozen-lockfile
+      - name: Peer deps
+        run: bun run check:peers
 
       - name: Lint
         run: bun run lint
@@ -36,4 +38,3 @@ jobs:
 
       - name: Build
         run: bun run build
-

.gitignore

@@ -1,5 +1,8 @@
 node_modules
 .DS_Store
+.bun-build
+*.bun-build
+bin/docs-list
 dist
 dist-ssr
 !packages/schema/dist
@@ -18,3 +21,6 @@ todos.json
 .vscode
 .env*.local
 coverage
+playwright-report
+test-results
+.playwright

AGENTS.md

@@ -38,3 +38,8 @@
 - Local env: `.env.local` (never commit secrets).
 - Convex env holds JWT keys; Vercel only needs `VITE_CONVEX_URL` + `VITE_CONVEX_SITE_URL`.
 - OAuth: GitHub OAuth App credentials required for login.
+
+## Convex Ops (Gotchas)
+- New Convex functions must be pushed before `convex run`: use `bunx convex dev --once` (dev) or `bunx convex deploy` (prod).
+- For non-interactive prod deploys, use `bunx convex deploy -y` to skip confirmation.
+- If `bunx convex run --env-file .env.local ...` returns `401 MissingAccessToken` despite `bunx convex login`, workaround: omit `--env-file` and use `--deployment-name <name>` / `--prod`.

CHANGELOG.md

@@ -1,5 +1,176 @@
 # Changelog
 
+## 0.6.1 - 2026-02-13
+
+### Added
+- Security: add LLM-based security evaluation during skill publish.
+- Parsing: recognize `metadata.openclaw` frontmatter and evaluate all skill files for requirements.
+
+### Changed
+- Performance: lazy-load Monaco diff viewer on demand (thanks @alexjcm, #212).
+- Search: improve recall/ranking with lexical fallback and relevance prioritization.
+- Moderation UX: collapse OpenClaw analysis by default; update spacing and default reasoning model.
+
+### Fixed
+- Upload gate: handle GitHub API rate limits and optional authenticated lookup token (thanks @superlowburn, #246).
+- HTTP: remove `allowH2` from Undici agent to prevent `fetch failed` on Node.js 22+ (#245).
+- Tests: add root `undici` dev dependency for Node E2E imports (thanks @tanujbhaud, #255).
+- VirusTotal: fix scan sync race conditions and retry behavior in scan/backfill paths.
+- Metadata: tolerate trailing commas in JSON metadata.
+- Auth: allow soft-deleted users to re-authenticate on fresh login, while keeping banned users blocked (thanks @tanujbhaud, #177).
+- Web: prevent horizontal overflow from long code blocks in skill pages (thanks @bewithgaurav, #183).
+
+## 0.6.0 - 2026-02-10
+### Added
+- CLI/API: add `set-role` to change user roles (admin only).
+- Security: quarantine skill publishes with VirusTotal scans + UI (thanks @aleph8, #130).
+- Testing: add tests for badges, skillZip, uploadFiles expandDroppedItems, and ark schema error truncation.
+- Moderation: add ban reasons to API/CLI and show in management UI.
+
+### Changed
+- Coverage: track `convex/lib/skillZip.ts` in coverage reports.
+
+### Fixed
+- Web: show pending-scan skills to owners without 404 (thanks @orlyjamie, #136).
+- Users: backfill empty handles from name/email in ensure (thanks @adlai88, #158).
+- Web: update footer branding to OpenClaw (thanks @jontsai, #122).
+- Auth: restore soft-deleted users on reauth, block banned users (thanks @mkrokosz, #106).
+
+## 0.5.0 - 2026-02-02
+
+### Added
+- Admin: ban users and delete owned skills from management console.
+- Moderation: auto-hide skills after 4 unique reports; per-user report cap; moderators can ban users.
+- Uploads: require GitHub accounts to be at least 7 days old for skill + soul publish/import.
+- CLI: add `inspect` to fetch skill metadata/files without installing.
+- CLI: add moderation commands for hide/unhide/delete and ban users.
+- Management: add filters for reported skills and users.
+
+### Changed
+- Deps: update dependencies to latest available versions.
+- Reporting: require reasons, show them in management console, warn about abuse bans.
+
+### Fixed
+- Bans: batch hard-delete cleanup to avoid Convex read limits on large skills.
+
+## 0.4.0 - 2026-01-30
+
+### Added
+- Web: show published skills on user profiles (thanks @njoylab, #20).
+- CLI: include ClawHub + Moltbot fallback skill roots for sync scans.
+- CLI: support OpenClaw configuration files (`OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR`).
+
+### Changed
+- Brand: rebrand to ClawHub and publish CLI as `clawhub` (legacy `clawdhub` supported).
+- Domain: default site/registry now `https://clawhub.ai`; `.well-known/clawhub.json` preferred.
+- Theme: persist theme under `clawhub-theme` (legacy key still read).
+
+### Fixed
+- Registry: drop missing skills during search hydration (thanks @aaronn, #28).
+- CLI: use path-based skill metadata lookup for updates (thanks @daveonkels, #22).
+- Search: keep highlighted-only filtering and clamp vector candidates to Convex limits (thanks @aaronn, #30).
+
+## 0.3.0 - 2026-01-19
+
+### Added
+- CLI: add `explore` command for latest updates, with limit clamping + tests/docs (thanks @jdrhyne, #14).
+- CLI: `explore --json` output + new sorts (`installs`, `installsAllTime`, `trending`) and limit up to 200.
+- API: `/api/v1/skills` supports installs + trending sorts (7-day installs).
+- API: idempotent `POST/DELETE /api/v1/stars/{slug}` endpoints.
+- Registry: trending leaderboard + daily stats backfill for installs-based sorts.
+
+### Fixed
+- Web: keep search mode navigation and state in sync (thanks @NACC96, #12).
+
+## 0.2.0 - 2026-01-13
+
+### Added
+- Web: dynamic OG image cards for skills (name, description, version).
+- CLI: auto-scan Clawdbot skill roots (per-agent workspaces, shared skills, extraDirs).
+- Web: import skills from public GitHub URLs (auto-detect `SKILL.md`, smart file selection, provenance).
+- Web/API: SoulHub (SOUL.md registry) with v1 endpoints and first-run auto-seed.
+
+### Fixed
+- Web: stabilize skill OG image generation on server runtimes.
+- Web: prevent skill OG text overflow outside the card.
+- Registry: make SoulHub auto-seed idempotent and non-user-owned.
+- Registry: keep GitHub backup state + publish backups intact (thanks @joshp123, #1).
+- CLI/Registry: restore fork lineage on sync + clamp bulk list queries (thanks @joshp123, #1).
+- CLI: default workdir falls back to Clawdbot workspace (override with `--workdir` / `CLAWHUB_WORKDIR`).
+
+## 0.0.6 - 2026-01-07
+
+
+### Added
+- API: v1 public REST endpoints with rate limits, raw file fetch, and OpenAPI spec.
+- Docs: `docs/api.md` and `DEPRECATIONS.md` for the v1 cutover plan.
+
+### Changed
+- CLI: publish now uses single multipart `POST /api/v1/skills`.
+- Registry: legacy `/api/*` + `/api/cli/*` marked for deprecation (kept for now).
+
+## 0.0.5 - 2026-01-06
+
+### Added
+- Telemetry: track installs via `clawhub sync` (logged-in only), per root, with 120-day staleness.
+- Skills: show current + all-time installs; sort by installs.
+- Profile: private "Installed" tab with JSON export + delete telemetry controls.
+- Docs: add `docs/telemetry.md` (what we track + how to opt out).
+- Web: custom Open Graph image (`/og.png`) + richer OG/Twitter tags.
+- Web: dashboard for managing your published skills (thanks @dbhurley!).
+
+### Changed
+- CLI: telemetry opt-out via `CLAWHUB_DISABLE_TELEMETRY=1`.
+- Web: move theme picker into mobile menu.
+
+### Fixed
+- Web: handle shorthand hex colors in diff theme (thanks @dbhurley!).
+
+## 0.0.5 - 2026-01-06
+
+### Added
+- Maintenance: admin backfill to re-parse `SKILL.md` and repair stored summaries/parsed metadata.
+
+### Fixed
+- CLI sync: ignore plural `skills.md` docs files when scanning for skills.
+- Registry: parse YAML frontmatter (incl multiline `description`) and accept YAML `metadata` objects.
+
+## 0.0.4 - 2026-01-05
+
+### Added
+- Web: `/skills` list view with sorting (newest/downloads/stars/name) + quick filter.
+- Web: admin/moderator highlight toggle on skill detail.
+- Web: canonical skill URLs as `/<owner>/<slug>` (legacy `/skills/<slug>` redirects).
+- Web: upload auto-generates a changelog via OpenAI when left blank (marked as auto-generated).
+
+### Fixed
+- Web: skill detail shows a loading state instead of flashing "Skill not found".
+- Web: user profile shows avatar + loading state (no "User not found" flash).
+- Web: improved mobile responsiveness (nav menu, skill detail layout, install command overflow).
+- Web: upload now unwraps folder picks so `SKILL.md` can be at the bundle root.
+- Registry: cap embedding payload size to avoid model context errors.
+- CLI: ignore legacy `auth.clawdhub.com` registry and prefer site discovery.
+
+### Changed
+- Web: homepage search now expands into full search mode with live results + highlighted toggle.
+- CLI: sync no longer prompts for changelog; registry auto-generates when blank.
+
+## 0.0.3 - 2026-01-04
+
+### Added
+- CLI sync: concurrency flag to limit registry checks.
+- Home: install command switcher (npm/pnpm/bun).
+
+### Changed
+- CLI sync: default `--concurrency` is now 4 (was 8).
+- CLI sync: replace boxed notes with plain output for long lists.
+
+### Fixed
+- CLI sync: wrap note output to avoid terminal overflow; cap list lengths.
+- CLI sync: label fallback scans as fallback locations.
+- CLI package: bundle schema internally (no external `clawhub-schema` publish).
+- Repo: mark `clawhub-schema` as private to prevent publishing.
+
 ## 0.0.2 - 2026-01-04
 
 ### Added

DEPRECATIONS.md

@@ -0,0 +1,7 @@
+# Deprecations
+
+## Legacy /api routes (pre-v1)
+
+- Deprecated: 2026-01-07
+- TODO: remove legacy `/api/*` and `/api/cli/*` routes after clients migrate to `/api/v1`.
+- Legacy handlers live in `convex/http.ts` and `convex/httpApi.ts`.

README.md

@@ -1,38 +1,167 @@
-# ClawdHub
+# ClawHub
 
-Minimal skill registry powered by TanStack Start + Convex.
+<p align="center">
+  <a href="https://github.com/openclaw/clawhub/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/openclaw/clawhub/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
+  <a href="https://discord.gg/clawd"><img src="https://img.shields.io/discord/1456350064065904867?label=Discord&logo=discord&logoColor=white&color=5865F2&style=for-the-badge" alt="Discord"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
+</p>
 
-## Quick start
+ClawHub is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
+It’s designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
+
+onlycrabs.ai is the **SOUL.md registry**: publish and share system lore the same way you publish skills.
+
+Live: `https://clawhub.ai`
+onlycrabs.ai: `https://onlycrabs.ai`
+
+## What you can do with it
+
+- Browse skills + render their `SKILL.md`.
+- Publish new skill versions with changelogs + tags (including `latest`).
+- Browse souls + render their `SOUL.md`.
+- Publish new soul versions with changelogs + tags.
+- Search via embeddings (vector index) instead of brittle keywords.
+- Star + comment; admins/mods can curate and approve skills.
+
+## onlycrabs.ai (SOUL.md registry)
+
+- Entry point is host-based: `onlycrabs.ai`.
+- On the onlycrabs.ai host, the home page and nav default to souls.
+- On ClawHub, souls live under `/souls`.
+- Soul bundles only accept `SOUL.md` for now (no extra files).
+
+## How it works (high level)
+
+- Web app: TanStack Start (React, Vite/Nitro).
+- Backend: Convex (DB + file storage + HTTP actions) + Convex Auth (GitHub OAuth).
+- Search: OpenAI embeddings (`text-embedding-3-small`) + Convex vector search.
+- API schema + routes: `packages/schema` (`clawhub-schema`).
+
+
+## Telemetry
+
+ClawHub tracks minimal **install telemetry** (to compute install counts) when you run `clawhub sync` while logged in.
+Disable via:
+
+```bash
+export CLAWHUB_DISABLE_TELEMETRY=1
+```
+
+Details: `docs/telemetry.md`.
+
+## Repo layout
+
+- `src/` — TanStack Start app (routes, components, styles).
+- `convex/` — schema + queries/mutations/actions + HTTP API routes.
+- `packages/schema/` — shared API types/routes for the CLI and app.
+- `docs/spec.md` — product + implementation spec (good first read).
+
+## Local dev
+
+Prereqs: Bun + Convex CLI.
 
 ```bash
 bun install
 cp .env.local.example .env.local
-bun --bun run dev
-```
 
-In another terminal:
+# terminal A: web app
+bun run dev
 
-```bash
+# terminal B: Convex dev deployment
 bunx convex dev
 ```
 
-## Convex Auth setup
+## Auth (GitHub OAuth) setup
+
+Create a GitHub OAuth App, set `AUTH_GITHUB_ID` / `AUTH_GITHUB_SECRET`, then:
 
 ```bash
 bunx auth --deployment-name <deployment> --web-server-url http://localhost:3000
 ```
 
-This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints the values for local `.env.local`.
+This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints values for your local `.env.local`.
 
 ## Environment
 
 - `VITE_CONVEX_URL`: Convex deployment URL (`https://<deployment>.convex.cloud`).
 - `VITE_CONVEX_SITE_URL`: Convex site URL (`https://<deployment>.convex.site`).
+- `VITE_SOULHUB_SITE_URL`: onlycrabs.ai site URL (`https://onlycrabs.ai`).
+- `VITE_SOULHUB_HOST`: onlycrabs.ai host match (`onlycrabs.ai`).
+- `VITE_SITE_MODE`: Optional override (`skills` or `souls`) for SSR builds.
 - `CONVEX_SITE_URL`: same as `VITE_CONVEX_SITE_URL` (auth + cookies).
 - `SITE_URL`: App URL (local: `http://localhost:3000`).
 - `AUTH_GITHUB_ID` / `AUTH_GITHUB_SECRET`: GitHub OAuth App.
 - `JWT_PRIVATE_KEY` / `JWKS`: Convex Auth keys.
-- `OPENAI_API_KEY`: embeddings.
+- `OPENAI_API_KEY`: embeddings for search + indexing.
+
+## Nix plugins (nixmode skills)
+
+ClawHub can store a nix-clawdbot plugin pointer in SKILL frontmatter so the registry knows which
+Nix package bundle to install. A nix plugin is different from a regular skill pack: it bundles the
+skill pack, the CLI binary, and its config flags/requirements together.
+
+Add this to `SKILL.md`:
+
+```yaml
+---
+name: peekaboo
+description: Capture and automate macOS UI with the Peekaboo CLI.
+metadata: {"clawdbot":{"nix":{"plugin":"github:clawdbot/nix-steipete-tools?dir=tools/peekaboo","systems":["aarch64-darwin"]}}}
+---
+```
+
+Install via nix-clawdbot:
+
+```nix
+programs.clawdbot.plugins = [
+  { source = "github:clawdbot/nix-steipete-tools?dir=tools/peekaboo"; }
+];
+```
+
+You can also declare config requirements + an example snippet:
+
+```yaml
+---
+name: padel
+description: Check padel court availability and manage bookings via Playtomic.
+metadata: {"clawdbot":{"config":{"requiredEnv":["PADEL_AUTH_FILE"],"stateDirs":[".config/padel"],"example":"config = { env = { PADEL_AUTH_FILE = \\\"/run/agenix/padel-auth\\\"; }; };"}}}
+---
+```
+
+To show CLI help (recommended for nix plugins), include the `cli --help` output:
+
+```yaml
+---
+name: padel
+description: Check padel court availability and manage bookings via Playtomic.
+metadata: {"clawdbot":{"cliHelp":"padel --help\\nUsage: padel [command]\\n"}}
+---
+```
+
+`metadata.clawdbot` is preferred, but `metadata.clawdis` and `metadata.openclaw` are accepted as aliases.
+
+## Skill metadata
+
+Skills declare their runtime requirements (env vars, binaries, install specs) in the `SKILL.md` frontmatter. ClawHub's security analysis checks these declarations against actual skill behavior.
+
+Full reference: [`docs/skill-format.md`](docs/skill-format.md#frontmatter-metadata)
+
+Quick example:
+
+```yaml
+---
+name: my-skill
+description: Does a thing with an API.
+metadata:
+  openclaw:
+    requires:
+      env:
+        - MY_API_KEY
+      bins:
+        - curl
+    primaryEnv: MY_API_KEY
+---
+```
 
 ## Scripts
 

biome.json

@@ -1,5 +1,5 @@
 {
-  "$schema": "https://biomejs.dev/schemas/2.3.11/schema.json",
+  "$schema": "https://biomejs.dev/schemas/2.3.13/schema.json",
   "files": {
     "includes": [
       "**",
@@ -10,9 +10,12 @@
       "!**/.output",
       "!**/coverage",
       "!**/convex/_generated",
+      "!**/test-results",
       "!**/src/routeTree.gen.ts",
       "!**/.tanstack",
-      "!**/public"
+      "!**/public",
+      "!**/.devenv",
+      "!**/.devenv"
     ]
   },
   "assist": { "actions": { "source": { "organizeImports": "on" } } },

clawhub

@@ -0,0 +1,46 @@
+#!/usr/bin/env bun
+import { existsSync } from 'node:fs'
+import { stat } from 'node:fs/promises'
+import { fileURLToPath } from 'node:url'
+
+const distCliUrl = new URL('./packages/clawdhub/dist/cli.js', import.meta.url)
+const distCliPath = fileURLToPath(distCliUrl)
+const srcRootPath = fileURLToPath(new URL('./packages/clawdhub/src/', import.meta.url))
+
+const shouldBuild = await (async () => {
+  if (!existsSync(distCliPath)) return true
+  try {
+    const dist = await stat(distCliPath)
+    const latestSrcMtime = await getLatestMtime(srcRootPath)
+    return latestSrcMtime > dist.mtimeMs
+  } catch {
+    return true
+  }
+})()
+
+if (shouldBuild) {
+  const proc = Bun.spawn(['bunx', 'tsc', '-p', 'packages/clawdhub/tsconfig.json'], {
+    stdin: 'inherit',
+    stdout: 'inherit',
+    stderr: 'inherit',
+  })
+  const code = await proc.exited
+  if (code !== 0) process.exit(code)
+}
+
+await import(distCliUrl.href)
+
+async function getLatestMtime(root: string) {
+  let latest = 0
+  const glob = new Bun.Glob('**/*.ts')
+  for await (const rel of glob.scan({ cwd: root, onlyFiles: true })) {
+    const path = `${root}${root.endsWith('/') ? '' : '/'}${rel}`
+    try {
+      const entry = await stat(path)
+      latest = Math.max(latest, entry.mtimeMs)
+    } catch {
+      // ignore
+    }
+  }
+  return latest
+}

convex/_generated/api.d.ts

@@ -10,20 +10,63 @@
 
 import type * as auth from "../auth.js";
 import type * as comments from "../comments.js";
+import type * as crons from "../crons.js";
+import type * as devSeed from "../devSeed.js";
+import type * as devSeedExtra from "../devSeedExtra.js";
 import type * as downloads from "../downloads.js";
+import type * as githubBackups from "../githubBackups.js";
+import type * as githubBackupsNode from "../githubBackupsNode.js";
+import type * as githubImport from "../githubImport.js";
+import type * as githubSoulBackups from "../githubSoulBackups.js";
+import type * as githubSoulBackupsNode from "../githubSoulBackupsNode.js";
 import type * as http from "../http.js";
 import type * as httpApi from "../httpApi.js";
+import type * as httpApiV1 from "../httpApiV1.js";
+import type * as leaderboards from "../leaderboards.js";
 import type * as lib_access from "../lib/access.js";
 import type * as lib_apiTokenAuth from "../lib/apiTokenAuth.js";
+import type * as lib_badges from "../lib/badges.js";
+import type * as lib_changelog from "../lib/changelog.js";
 import type * as lib_embeddings from "../lib/embeddings.js";
+import type * as lib_githubAccount from "../lib/githubAccount.js";
+import type * as lib_githubBackup from "../lib/githubBackup.js";
+import type * as lib_githubImport from "../lib/githubImport.js";
+import type * as lib_githubSoulBackup from "../lib/githubSoulBackup.js";
+import type * as lib_leaderboards from "../lib/leaderboards.js";
+import type * as lib_moderation from "../lib/moderation.js";
+import type * as lib_public from "../lib/public.js";
+import type * as lib_searchText from "../lib/searchText.js";
+import type * as lib_securityPrompt from "../lib/securityPrompt.js";
+import type * as lib_skillBackfill from "../lib/skillBackfill.js";
+import type * as lib_skillPublish from "../lib/skillPublish.js";
+import type * as lib_skillStats from "../lib/skillStats.js";
+import type * as lib_skillZip from "../lib/skillZip.js";
 import type * as lib_skills from "../lib/skills.js";
+import type * as lib_soulChangelog from "../lib/soulChangelog.js";
+import type * as lib_soulPublish from "../lib/soulPublish.js";
 import type * as lib_tokens from "../lib/tokens.js";
+import type * as lib_userSearch from "../lib/userSearch.js";
+import type * as lib_webhooks from "../lib/webhooks.js";
+import type * as llmEval from "../llmEval.js";
+import type * as maintenance from "../maintenance.js";
+import type * as rateLimits from "../rateLimits.js";
 import type * as search from "../search.js";
+import type * as seed from "../seed.js";
+import type * as seedSouls from "../seedSouls.js";
+import type * as skillStatEvents from "../skillStatEvents.js";
 import type * as skills from "../skills.js";
+import type * as soulComments from "../soulComments.js";
+import type * as soulDownloads from "../soulDownloads.js";
+import type * as soulStars from "../soulStars.js";
+import type * as souls from "../souls.js";
 import type * as stars from "../stars.js";
+import type * as statsMaintenance from "../statsMaintenance.js";
+import type * as telemetry from "../telemetry.js";
 import type * as tokens from "../tokens.js";
 import type * as uploads from "../uploads.js";
 import type * as users from "../users.js";
+import type * as vt from "../vt.js";
+import type * as webhooks from "../webhooks.js";
 
 import type {
   ApiFromModules,
@@ -34,20 +77,63 @@ import type {
 declare const fullApi: ApiFromModules<{
   auth: typeof auth;
   comments: typeof comments;
+  crons: typeof crons;
+  devSeed: typeof devSeed;
+  devSeedExtra: typeof devSeedExtra;
   downloads: typeof downloads;
+  githubBackups: typeof githubBackups;
+  githubBackupsNode: typeof githubBackupsNode;
+  githubImport: typeof githubImport;
+  githubSoulBackups: typeof githubSoulBackups;
+  githubSoulBackupsNode: typeof githubSoulBackupsNode;
   http: typeof http;
   httpApi: typeof httpApi;
+  httpApiV1: typeof httpApiV1;
+  leaderboards: typeof leaderboards;
   "lib/access": typeof lib_access;
   "lib/apiTokenAuth": typeof lib_apiTokenAuth;
+  "lib/badges": typeof lib_badges;
+  "lib/changelog": typeof lib_changelog;
   "lib/embeddings": typeof lib_embeddings;
+  "lib/githubAccount": typeof lib_githubAccount;
+  "lib/githubBackup": typeof lib_githubBackup;
+  "lib/githubImport": typeof lib_githubImport;
+  "lib/githubSoulBackup": typeof lib_githubSoulBackup;
+  "lib/leaderboards": typeof lib_leaderboards;
+  "lib/moderation": typeof lib_moderation;
+  "lib/public": typeof lib_public;
+  "lib/searchText": typeof lib_searchText;
+  "lib/securityPrompt": typeof lib_securityPrompt;
+  "lib/skillBackfill": typeof lib_skillBackfill;
+  "lib/skillPublish": typeof lib_skillPublish;
+  "lib/skillStats": typeof lib_skillStats;
+  "lib/skillZip": typeof lib_skillZip;
   "lib/skills": typeof lib_skills;
+  "lib/soulChangelog": typeof lib_soulChangelog;
+  "lib/soulPublish": typeof lib_soulPublish;
   "lib/tokens": typeof lib_tokens;
+  "lib/userSearch": typeof lib_userSearch;
+  "lib/webhooks": typeof lib_webhooks;
+  llmEval: typeof llmEval;
+  maintenance: typeof maintenance;
+  rateLimits: typeof rateLimits;
   search: typeof search;
+  seed: typeof seed;
+  seedSouls: typeof seedSouls;
+  skillStatEvents: typeof skillStatEvents;
   skills: typeof skills;
+  soulComments: typeof soulComments;
+  soulDownloads: typeof soulDownloads;
+  soulStars: typeof soulStars;
+  souls: typeof souls;
   stars: typeof stars;
+  statsMaintenance: typeof statsMaintenance;
+  telemetry: typeof telemetry;
   tokens: typeof tokens;
   uploads: typeof uploads;
   users: typeof users;
+  vt: typeof vt;
+  webhooks: typeof webhooks;
 }>;
 
 /**

convex/auth.test.ts

@@ -0,0 +1,101 @@
+import { describe, expect, it, vi } from 'vitest'
+import type { Id } from './_generated/dataModel'
+import { BANNED_REAUTH_MESSAGE, handleSoftDeletedUserReauth } from './auth'
+
+function makeCtx({
+  user,
+  banRecord,
+}: {
+  user: { deletedAt?: number } | null
+  banRecord?: Record<string, unknown> | null
+}) {
+  const query = {
+    withIndex: vi.fn().mockReturnValue({
+      filter: vi.fn().mockReturnValue({
+        first: vi.fn().mockResolvedValue(banRecord ?? null),
+      }),
+    }),
+  }
+  const ctx = {
+    db: {
+      get: vi.fn().mockResolvedValue(user),
+      patch: vi.fn().mockResolvedValue(null),
+      query: vi.fn().mockReturnValue(query),
+    },
+  }
+  return { ctx, query }
+}
+
+describe('handleSoftDeletedUserReauth', () => {
+  const userId = 'users:1' as Id<'users'>
+
+  it('skips when user not found', async () => {
+    const { ctx } = makeCtx({ user: null })
+
+    await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId })
+
+    expect(ctx.db.get).toHaveBeenCalledWith(userId)
+    expect(ctx.db.query).not.toHaveBeenCalled()
+  })
+
+  it('skips active users', async () => {
+    const { ctx } = makeCtx({ user: { deletedAt: undefined } })
+
+    await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId })
+
+    expect(ctx.db.query).not.toHaveBeenCalled()
+    expect(ctx.db.patch).not.toHaveBeenCalled()
+  })
+
+  it('restores soft-deleted users when not banned', async () => {
+    const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: null })
+
+    await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId })
+
+    expect(ctx.db.patch).toHaveBeenCalledWith(userId, {
+      deletedAt: undefined,
+      updatedAt: expect.any(Number),
+    })
+  })
+
+  it('restores soft-deleted users on fresh login (existingUserId is null)', async () => {
+    const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: null })
+
+    await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: null })
+
+    expect(ctx.db.patch).toHaveBeenCalledWith(userId, {
+      deletedAt: undefined,
+      updatedAt: expect.any(Number),
+    })
+  })
+
+  it('skips reactivation when existingUserId does not match userId', async () => {
+    const otherUserId = 'users:999' as Id<'users'>
+    const { ctx } = makeCtx({ user: { deletedAt: 123 } })
+
+    await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: otherUserId })
+
+    expect(ctx.db.query).not.toHaveBeenCalled()
+    expect(ctx.db.patch).not.toHaveBeenCalled()
+  })
+
+  it('blocks banned users with a custom message', async () => {
+    const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: { action: 'user.ban' } })
+
+    await expect(
+      handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId }),
+    ).rejects.toThrow(BANNED_REAUTH_MESSAGE)
+
+    expect(ctx.db.patch).not.toHaveBeenCalled()
+  })
+
+  it('blocks banned users on fresh login (existingUserId is null)', async () => {
+    const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: { action: 'user.ban' } })
+
+    await expect(
+      handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: null }),
+    ).rejects.toThrow(BANNED_REAUTH_MESSAGE)
+
+    expect(ctx.db.patch).not.toHaveBeenCalled()
+  })
+})

convex/auth.ts

@@ -1,5 +1,40 @@
 import GitHub from '@auth/core/providers/github'
 import { convexAuth } from '@convex-dev/auth/server'
+import type { GenericMutationCtx } from 'convex/server'
+import { ConvexError } from 'convex/values'
+import type { DataModel, Id } from './_generated/dataModel'
+
+export const BANNED_REAUTH_MESSAGE =
+  'Your account has been banned for uploading malicious skills. If you believe this is a mistake, please contact security@openclaw.ai and we will work with you to restore access.'
+
+export async function handleSoftDeletedUserReauth(
+  ctx: GenericMutationCtx<DataModel>,
+  args: { userId: Id<'users'>; existingUserId: Id<'users'> | null },
+) {
+  const user = await ctx.db.get(args.userId)
+  if (!user?.deletedAt) return
+
+  // Verify that the incoming identity matches the soft-deleted user to prevent bypass.
+  if (args.existingUserId && args.existingUserId !== args.userId) {
+    return
+  }
+
+  const userId = args.userId
+  const banRecord = await ctx.db
+    .query('auditLogs')
+    .withIndex('by_target', (q) => q.eq('targetType', 'user').eq('targetId', userId.toString()))
+    .filter((q) => q.eq(q.field('action'), 'user.ban'))
+    .first()
+
+  if (banRecord) {
+    throw new ConvexError(BANNED_REAUTH_MESSAGE)
+  }
+
+  await ctx.db.patch(userId, {
+    deletedAt: undefined,
+    updatedAt: Date.now(),
+  })
+}
 
 export const { auth, signIn, signOut, store, isAuthenticated } = convexAuth({
   providers: [
@@ -16,4 +51,17 @@ export const { auth, signIn, signOut, store, isAuthenticated } = convexAuth({
       },
     }),
   ],
+  callbacks: {
+    /**
+     * Handle re-authentication of soft-deleted users.
+     *
+     * Performance note: This callback runs on every OAuth sign-in, but the
+     * audit log query ONLY executes when a soft-deleted user attempts to
+     * sign in (user.deletedAt is set). For normal active users, this is
+     * just a single `if` check on an already-loaded field - no extra queries.
+     */
+    async afterUserCreatedOrUpdated(ctx, args) {
+      await handleSoftDeletedUserReauth(ctx, args)
+    },
+  },
 })

convex/comments.ts

@@ -1,7 +1,9 @@
 import { v } from 'convex/values'
 import type { Doc } from './_generated/dataModel'
 import { mutation, query } from './_generated/server'
-import { assertRole, requireUser } from './lib/access'
+import { assertModerator, requireUser } from './lib/access'
+import { type PublicUser, toPublicUser } from './lib/public'
+import { insertStatEvent } from './skillStatEvents'
 
 export const listBySkill = query({
   args: { skillId: v.id('skills'), limit: v.optional(v.number()) },
@@ -13,10 +15,10 @@ export const listBySkill = query({
       .order('desc')
       .take(limit)
 
-    const results: Array<{ comment: Doc<'comments'>; user: Doc<'users'> | null }> = []
+    const results: Array<{ comment: Doc<'comments'>; user: PublicUser | null }> = []
     for (const comment of comments) {
       if (comment.softDeletedAt) continue
-      const user = await ctx.db.get(comment.userId)
+      const user = toPublicUser(await ctx.db.get(comment.userId))
       results.push({ comment, user })
     }
     return results
@@ -42,10 +44,7 @@ export const add = mutation({
       deletedBy: undefined,
     })
 
-    await ctx.db.patch(skill._id, {
-      stats: { ...skill.stats, comments: skill.stats.comments + 1 },
-      updatedAt: Date.now(),
-    })
+    await insertStatEvent(ctx, { skillId: skill._id, kind: 'comment' })
   },
 })
 
@@ -59,7 +58,7 @@ export const remove = mutation({
 
     const isOwner = comment.userId === user._id
     if (!isOwner) {
-      assertRole(user, ['admin', 'moderator'])
+      assertModerator(user)
     }
 
     await ctx.db.patch(comment._id, {
@@ -67,13 +66,7 @@ export const remove = mutation({
       deletedBy: user._id,
     })
 
-    const skill = await ctx.db.get(comment.skillId)
-    if (skill) {
-      await ctx.db.patch(skill._id, {
-        stats: { ...skill.stats, comments: Math.max(0, skill.stats.comments - 1) },
-        updatedAt: Date.now(),
-      })
-    }
+    await insertStatEvent(ctx, { skillId: comment.skillId, kind: 'uncomment' })
 
     await ctx.db.insert('auditLogs', {
       actorUserId: user._id,

convex/crons.ts

@@ -0,0 +1,43 @@
+import { cronJobs } from 'convex/server'
+import { internal } from './_generated/api'
+
+const crons = cronJobs()
+
+crons.interval(
+  'github-backup-sync',
+  { minutes: 30 },
+  internal.githubBackupsNode.syncGitHubBackupsInternal,
+  { batchSize: 50, maxBatches: 5 },
+)
+
+crons.interval(
+  'trending-leaderboard',
+  { minutes: 60 },
+  internal.leaderboards.rebuildTrendingLeaderboardInternal,
+  { limit: 200 },
+)
+
+crons.interval(
+  'skill-stats-backfill',
+  { minutes: 10 },
+  internal.statsMaintenance.runSkillStatBackfillInternal,
+  { batchSize: 200, maxBatches: 5 },
+)
+
+crons.interval(
+  'skill-stat-events',
+  { minutes: 15 },
+  internal.skillStatEvents.processSkillStatEventsAction,
+  {},
+)
+
+crons.interval('vt-pending-scans', { minutes: 5 }, internal.vt.pollPendingScans, { batchSize: 100 })
+
+crons.interval('vt-cache-backfill', { minutes: 30 }, internal.vt.backfillActiveSkillsVTCache, {
+  batchSize: 100,
+})
+
+// Daily re-scan of all active skills at 3am UTC
+crons.daily('vt-daily-rescan', { hourUTC: 3, minuteUTC: 0 }, internal.vt.rescanActiveSkills, {})
+
+export default crons

convex/devSeed.ts

@@ -0,0 +1,459 @@
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { ActionCtx } from './_generated/server'
+import { internalAction, internalMutation } from './_generated/server'
+import { EMBEDDING_DIMENSIONS } from './lib/embeddings'
+import { parseClawdisMetadata, parseFrontmatter } from './lib/skills'
+
+type SeedSkillSpec = {
+  slug: string
+  displayName: string
+  summary: string
+  version: string
+  metadata: Record<string, unknown>
+  rawSkillMd: string
+}
+
+type SeedActionArgs = {
+  reset?: boolean
+}
+
+type SeedActionResult = {
+  ok: true
+  results: Array<Record<string, unknown> & { slug: string }>
+}
+
+type SeedMutationResult = Record<string, unknown>
+
+const SEED_SKILLS: SeedSkillSpec[] = [
+  {
+    slug: 'padel',
+    displayName: 'Padel',
+    summary: 'Check padel court availability and manage bookings via Playtomic.',
+    version: '0.1.0',
+    metadata: {
+      clawdbot: {
+        nix: {
+          plugin: 'github:joshp123/padel-cli',
+          systems: ['aarch64-darwin', 'x86_64-linux'],
+        },
+        config: {
+          requiredEnv: ['PADEL_AUTH_FILE'],
+          stateDirs: ['.config/padel'],
+          example:
+            'config = { env = { PADEL_AUTH_FILE = "/run/agenix/padel-auth"; }; stateDirs = [ ".config/padel" ]; };',
+        },
+        cliHelp: `Padel CLI for availability
+
+Usage:
+  padel [command]
+
+Available Commands:
+  auth         Manage authentication
+  availability Show availability for a club on a date
+  book         Book a court
+  bookings     Manage bookings history
+  search       Search for available courts
+  venues       Manage saved venues
+
+Flags:
+  -h, --help   help for padel
+  --json       Output JSON
+
+Use "padel [command] --help" for more information about a command.
+`,
+      },
+    },
+    rawSkillMd: `---
+name: padel
+description: Check padel court availability and manage bookings via the padel CLI.
+---
+
+# Padel Booking Skill
+
+## CLI
+
+\`\`\`bash
+padel  # On PATH (clawdbot plugin bundle)
+\`\`\`
+
+## Venues
+
+Use the configured venue list in order of preference. If no venues are configured, ask for a venue name or location.
+
+## Commands
+
+### Check next booking
+\`\`\`bash
+padel bookings list 2>&1 | head -3
+\`\`\`
+
+### Search availability
+\`\`\`bash
+padel search --venues VENUE1,VENUE2 --date YYYY-MM-DD --time 09:00-12:00
+\`\`\`
+
+## Response guidelines
+
+- Keep responses concise.
+- Use 🎾 emoji.
+- End with a call to action.
+
+## Authorization
+
+Only the authorized booker can confirm bookings. If the requester is not authorized, ask the authorized user to confirm.
+`,
+  },
+  {
+    slug: 'gohome',
+    displayName: 'GoHome',
+    summary: 'Operate GoHome via gRPC discovery, metrics, and Grafana dashboards.',
+    version: '0.1.0',
+    metadata: {
+      clawdbot: {
+        nix: {
+          plugin: 'github:joshp123/gohome',
+          systems: ['x86_64-linux', 'aarch64-linux'],
+        },
+        config: {
+          requiredEnv: ['GOHOME_GRPC_ADDR', 'GOHOME_HTTP_BASE'],
+          example:
+            'config = { env = { GOHOME_GRPC_ADDR = "gohome:9000"; GOHOME_HTTP_BASE = "http://gohome:8080"; }; };',
+        },
+        cliHelp: `GoHome CLI
+
+Usage:
+  gohome-cli [command]
+
+Available Commands:
+  services   List registered services
+  plugins    Inspect loaded plugins
+  methods    List RPC methods
+  call       Call an RPC method
+  roborock   Manage roborock devices
+  tado       Manage tado zones
+
+Flags:
+  --grpc-addr string   gRPC endpoint (host:port)
+  -h, --help           help for gohome-cli
+`,
+      },
+    },
+    rawSkillMd: `---
+name: gohome
+description: Use when Clawdbot needs to test or operate GoHome via gRPC discovery, metrics, and Grafana.
+---
+
+# GoHome Skill
+
+## Quick start
+
+\`\`\`bash
+export GOHOME_HTTP_BASE="http://gohome:8080"
+export GOHOME_GRPC_ADDR="gohome:9000"
+\`\`\`
+
+## CLI
+
+\`\`\`bash
+gohome-cli services
+\`\`\`
+
+## Discovery flow (read-only)
+
+1) List plugins.
+2) Describe a plugin.
+3) List RPC methods.
+4) Call a read-only RPC.
+
+## Metrics validation
+
+\`\`\`bash
+curl -s "\${GOHOME_HTTP_BASE}/gohome/metrics" | rg -n "gohome_"
+\`\`\`
+
+## Stateful actions
+
+Only call write RPCs after explicit user approval.
+`,
+  },
+  {
+    slug: 'xuezh',
+    displayName: 'Xuezh',
+    summary: 'Teach Mandarin with the xuezh engine for review, speaking, and audits.',
+    version: '0.1.0',
+    metadata: {
+      clawdbot: {
+        nix: {
+          plugin: 'github:joshp123/xuezh',
+          systems: ['aarch64-darwin', 'x86_64-linux'],
+        },
+        config: {
+          requiredEnv: ['XUEZH_AZURE_SPEECH_KEY_FILE', 'XUEZH_AZURE_SPEECH_REGION'],
+          stateDirs: ['.config/xuezh'],
+          example:
+            'config = { env = { XUEZH_AZURE_SPEECH_KEY_FILE = "/run/agenix/xuezh-azure-speech-key"; XUEZH_AZURE_SPEECH_REGION = "westeurope"; }; stateDirs = [ ".config/xuezh" ]; };',
+        },
+        cliHelp: `xuezh - Chinese learning engine
+
+Usage:
+  xuezh [command]
+
+Available Commands:
+  snapshot  Fetch learner state snapshot
+  review    Review due items
+  audio     Process speech audio
+  items     Manage learning items
+  events    Log learning events
+
+Flags:
+  -h, --help   help for xuezh
+  --json       Output JSON
+`,
+      },
+    },
+    rawSkillMd: `---
+name: xuezh
+description: Teach Mandarin using the xuezh engine for review, speaking, and audits.
+---
+
+# Xuezh Skill
+
+## Contract
+
+Use the xuezh CLI exactly as specified. If a command is missing, ask for implementation instead of guessing.
+
+## Default loop
+
+1) Call \`xuezh snapshot\`.
+2) Pick a tiny plan (1-2 bullets).
+3) Run a short activity.
+4) Log outcomes.
+
+## CLI examples
+
+\`\`\`bash
+xuezh snapshot --profile default
+xuezh review next --limit 10
+xuezh audio process-voice --file ./utterance.wav
+\`\`\`
+`,
+  },
+]
+
+function injectMetadata(rawSkillMd: string, metadata: Record<string, unknown>) {
+  const frontmatterEnd = rawSkillMd.indexOf('\n---', 3)
+  if (frontmatterEnd === -1) return rawSkillMd
+  return `${rawSkillMd.slice(0, frontmatterEnd)}\nmetadata: ${JSON.stringify(
+    metadata,
+  )}${rawSkillMd.slice(frontmatterEnd)}`
+}
+
+async function seedNixSkillsHandler(
+  ctx: ActionCtx,
+  args: SeedActionArgs,
+): Promise<SeedActionResult> {
+  const results: Array<Record<string, unknown> & { slug: string }> = []
+
+  for (const spec of SEED_SKILLS) {
+    const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
+    const frontmatter = parseFrontmatter(skillMd)
+    const clawdis = parseClawdisMetadata(frontmatter)
+    const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
+
+    const result: SeedMutationResult = await ctx.runMutation(internal.devSeed.seedSkillMutation, {
+      reset: args.reset,
+      storageId,
+      metadata: spec.metadata,
+      frontmatter,
+      clawdis,
+      skillMd,
+      slug: spec.slug,
+      displayName: spec.displayName,
+      summary: spec.summary,
+      version: spec.version,
+    })
+
+    results.push({ slug: spec.slug, ...result })
+  }
+
+  return { ok: true, results }
+}
+
+export const seedNixSkills: ReturnType<typeof internalAction> = internalAction({
+  args: {
+    reset: v.optional(v.boolean()),
+  },
+  handler: seedNixSkillsHandler,
+})
+
+async function seedPadelSkillHandler(
+  ctx: ActionCtx,
+  args: SeedActionArgs,
+): Promise<SeedMutationResult> {
+  const spec = SEED_SKILLS.find((entry) => entry.slug === 'padel')
+  if (!spec) throw new Error('padel seed spec missing')
+
+  const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
+  const frontmatter = parseFrontmatter(skillMd)
+  const clawdis = parseClawdisMetadata(frontmatter)
+  const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
+
+  return (await ctx.runMutation(internal.devSeed.seedSkillMutation, {
+    reset: args.reset,
+    storageId,
+    metadata: spec.metadata,
+    frontmatter,
+    clawdis,
+    skillMd,
+    slug: spec.slug,
+    displayName: spec.displayName,
+    summary: spec.summary,
+    version: spec.version,
+  })) as SeedMutationResult
+}
+
+export const seedPadelSkill: ReturnType<typeof internalAction> = internalAction({
+  args: {
+    reset: v.optional(v.boolean()),
+  },
+  handler: seedPadelSkillHandler,
+})
+
+export const seedSkillMutation = internalMutation({
+  args: {
+    reset: v.optional(v.boolean()),
+    storageId: v.id('_storage'),
+    metadata: v.any(),
+    frontmatter: v.any(),
+    clawdis: v.any(),
+    skillMd: v.string(),
+    slug: v.string(),
+    displayName: v.string(),
+    summary: v.optional(v.string()),
+    version: v.string(),
+  },
+  handler: async (ctx, args) => {
+    const existing = await ctx.db
+      .query('skills')
+      .withIndex('by_slug', (q) => q.eq('slug', args.slug))
+      .unique()
+
+    if (existing && !args.reset) {
+      return { ok: true, skipped: true, skillId: existing._id }
+    }
+
+    if (existing && args.reset) {
+      const versions = await ctx.db
+        .query('skillVersions')
+        .withIndex('by_skill', (q) => q.eq('skillId', existing._id))
+        .collect()
+      for (const version of versions) {
+        await ctx.db.delete(version._id)
+      }
+      const embeddings = await ctx.db
+        .query('skillEmbeddings')
+        .withIndex('by_skill', (q) => q.eq('skillId', existing._id))
+        .collect()
+      for (const embedding of embeddings) {
+        await ctx.db.delete(embedding._id)
+      }
+      await ctx.db.delete(existing._id)
+    }
+
+    const now = Date.now()
+    const existingUsers = await ctx.db
+      .query('users')
+      .withIndex('handle', (q) => q.eq('handle', 'local'))
+      .collect()
+
+    const userId =
+      existingUsers[0]?._id ??
+      (await ctx.db.insert('users', {
+        handle: 'local',
+        displayName: 'Local Dev',
+        role: 'admin',
+        createdAt: now,
+        updatedAt: now,
+      }))
+
+    const skillId = await ctx.db.insert('skills', {
+      slug: args.slug,
+      displayName: args.displayName,
+      summary: args.summary,
+      ownerUserId: userId,
+      latestVersionId: undefined,
+      tags: {},
+      softDeletedAt: undefined,
+      badges: { redactionApproved: undefined },
+      statsDownloads: 0,
+      statsStars: 0,
+      statsInstallsCurrent: 0,
+      statsInstallsAllTime: 0,
+      stats: {
+        downloads: 0,
+        installsCurrent: 0,
+        installsAllTime: 0,
+        stars: 0,
+        versions: 0,
+        comments: 0,
+      },
+      createdAt: now,
+      updatedAt: now,
+    })
+
+    const versionId = await ctx.db.insert('skillVersions', {
+      skillId,
+      version: args.version,
+      changelog: 'Seeded local version for screenshots.',
+      files: [
+        {
+          path: 'SKILL.md',
+          size: args.skillMd.length,
+          storageId: args.storageId,
+          sha256: 'seeded',
+          contentType: 'text/markdown',
+        },
+      ],
+      parsed: {
+        frontmatter: args.frontmatter,
+        metadata: args.metadata,
+        clawdis: args.clawdis,
+      },
+      createdBy: userId,
+      createdAt: now,
+      softDeletedAt: undefined,
+    })
+
+    const embeddingId = await ctx.db.insert('skillEmbeddings', {
+      skillId,
+      versionId,
+      ownerId: userId,
+      embedding: Array.from({ length: EMBEDDING_DIMENSIONS }, () => 0),
+      isLatest: true,
+      isApproved: true,
+      visibility: 'latest-approved',
+      updatedAt: now,
+    })
+
+    await ctx.db.patch(skillId, {
+      latestVersionId: versionId,
+      tags: { latest: versionId },
+      statsDownloads: 0,
+      statsStars: 0,
+      statsInstallsCurrent: 0,
+      statsInstallsAllTime: 0,
+      stats: {
+        downloads: 0,
+        installsCurrent: 0,
+        installsAllTime: 0,
+        stars: 0,
+        versions: 1,
+        comments: 0,
+      },
+      updatedAt: now,
+    })
+
+    return { ok: true, skillId, versionId, embeddingId }
+  },
+})

convex/devSeedExtra.ts

@@ -0,0 +1,541 @@
+/**
+ * Extra seed skills for pagination testing.
+ *
+ * This file contains 50 placeholder skills to test pagination behavior.
+ * Run with: bunx convex run internal.devSeedExtra.seedExtraSkillsInternal
+ * Or with reset: bunx convex run internal.devSeedExtra.seedExtraSkillsInternal '{"reset": true}'
+ */
+
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Id } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
+import { internalAction, internalMutation } from './_generated/server'
+import { parseClawdisMetadata, parseFrontmatter } from './lib/skills'
+
+type SeedSkillSpec = {
+  slug: string
+  displayName: string
+  summary: string
+  version: string
+  metadata: Record<string, unknown>
+  rawSkillMd: string
+}
+
+function makeSkill(
+  slug: string,
+  displayName: string,
+  summary: string,
+  envVars: string[] = [],
+  commands: string[] = ['help', 'status', 'run'],
+): SeedSkillSpec {
+  const cliHelp = `${slug} - ${summary}
+
+Usage:
+  ${slug} [command]
+
+Commands:
+${commands.map((cmd) => `  ${cmd.padEnd(12)} Run ${cmd} operation`).join('\n')}
+
+Flags:
+  -h, --help   Show help
+  --json       Output as JSON
+`
+
+  const rawSkillMd = `---
+name: ${slug}
+description: ${summary}
+---
+
+# ${displayName}
+
+## CLI
+
+\`\`\`bash
+${commands.map((cmd) => `${slug} ${cmd}`).join('\n')}
+\`\`\`
+
+## Usage
+
+Use this skill to ${summary.toLowerCase()}.
+`
+
+  return {
+    slug,
+    displayName,
+    summary,
+    version: '0.1.0',
+    metadata: {
+      clawdbot: {
+        nix: {
+          plugin: `github:example/${slug}`,
+          systems: ['aarch64-darwin', 'x86_64-linux'],
+        },
+        config: {
+          requiredEnv: envVars,
+        },
+        cliHelp,
+      },
+    },
+    rawSkillMd,
+  }
+}
+
+// 50 placeholder skills for pagination testing
+const EXTRA_SEED_SKILLS: SeedSkillSpec[] = [
+  // DevOps & Infrastructure (10)
+  makeSkill(
+    'kubectl-helper',
+    'Kubectl Helper',
+    'Simplified kubectl commands for common Kubernetes operations.',
+    ['KUBECONFIG'],
+    ['pods', 'logs', 'exec', 'describe', 'apply'],
+  ),
+  makeSkill(
+    'terraform-runner',
+    'Terraform Runner',
+    'Execute Terraform plans and applies with safety checks.',
+    ['TF_VAR_region', 'AWS_PROFILE'],
+    ['plan', 'apply', 'destroy', 'output', 'state'],
+  ),
+  makeSkill(
+    'ansible-exec',
+    'Ansible Exec',
+    'Run Ansible playbooks and ad-hoc commands.',
+    ['ANSIBLE_INVENTORY'],
+    ['playbook', 'adhoc', 'inventory', 'facts', 'vault'],
+  ),
+  makeSkill(
+    'docker-compose-mgr',
+    'Docker Compose Manager',
+    'Manage Docker Compose stacks and services.',
+    ['DOCKER_HOST'],
+    ['up', 'down', 'logs', 'ps', 'restart'],
+  ),
+  makeSkill(
+    'k9s-wrapper',
+    'K9s Wrapper',
+    'Interactive Kubernetes cluster management via K9s.',
+    ['KUBECONFIG'],
+    ['launch', 'contexts', 'namespaces', 'pods', 'logs'],
+  ),
+  makeSkill(
+    'helm-charts',
+    'Helm Charts',
+    'Manage Helm chart deployments and releases.',
+    ['KUBECONFIG', 'HELM_REPO'],
+    ['install', 'upgrade', 'rollback', 'list', 'search'],
+  ),
+  makeSkill(
+    'prometheus-alerts',
+    'Prometheus Alerts',
+    'Query Prometheus metrics and manage alerting rules.',
+    ['PROMETHEUS_URL'],
+    ['query', 'alerts', 'rules', 'targets', 'status'],
+  ),
+  makeSkill(
+    'grafana-dash',
+    'Grafana Dashboards',
+    'Create and manage Grafana dashboards programmatically.',
+    ['GRAFANA_URL', 'GRAFANA_API_KEY'],
+    ['list', 'export', 'import', 'create', 'delete'],
+  ),
+  makeSkill(
+    'nginx-config',
+    'Nginx Config',
+    'Generate and validate Nginx configuration files.',
+    ['NGINX_CONF_DIR'],
+    ['generate', 'validate', 'reload', 'test', 'sites'],
+  ),
+  makeSkill(
+    'jenkins-jobs',
+    'Jenkins Jobs',
+    'Manage Jenkins jobs and pipelines.',
+    ['JENKINS_URL', 'JENKINS_TOKEN'],
+    ['list', 'build', 'status', 'logs', 'config'],
+  ),
+
+  // Productivity (8)
+  makeSkill(
+    'todoist-sync',
+    'Todoist Sync',
+    'Sync and manage Todoist tasks from the command line.',
+    ['TODOIST_API_TOKEN'],
+    ['list', 'add', 'complete', 'projects', 'labels'],
+  ),
+  makeSkill(
+    'notion-backup',
+    'Notion Backup',
+    'Export and backup Notion workspaces.',
+    ['NOTION_TOKEN'],
+    ['export', 'backup', 'restore', 'pages', 'databases'],
+  ),
+  makeSkill(
+    'gcal-manager',
+    'Google Calendar Manager',
+    'Manage Google Calendar events and schedules.',
+    ['GOOGLE_CREDENTIALS_FILE'],
+    ['events', 'create', 'delete', 'calendars', 'reminders'],
+  ),
+  makeSkill(
+    'time-tracker',
+    'Time Tracker',
+    'Track time spent on projects and tasks.',
+    ['TIMETRACK_DB'],
+    ['start', 'stop', 'status', 'report', 'projects'],
+  ),
+  makeSkill(
+    'email-digest',
+    'Email Digest',
+    'Generate email digests and summaries.',
+    ['IMAP_SERVER', 'IMAP_USER'],
+    ['fetch', 'digest', 'search', 'folders', 'unread'],
+  ),
+  makeSkill(
+    'habit-tracker',
+    'Habit Tracker',
+    'Track daily habits and streaks.',
+    ['HABITS_DB'],
+    ['log', 'streak', 'stats', 'habits', 'remind'],
+  ),
+  makeSkill(
+    'bookmark-sync',
+    'Bookmark Sync',
+    'Sync bookmarks across browsers and devices.',
+    ['BOOKMARKS_DIR'],
+    ['sync', 'export', 'import', 'search', 'tags'],
+  ),
+  makeSkill(
+    'notes-export',
+    'Notes Export',
+    'Export notes to various formats.',
+    ['NOTES_DIR'],
+    ['export', 'convert', 'search', 'list', 'tags'],
+  ),
+
+  // Media & Entertainment (6)
+  makeSkill(
+    'spotify-ctl',
+    'Spotify Control',
+    'Control Spotify playback from the terminal.',
+    ['SPOTIFY_CLIENT_ID', 'SPOTIFY_CLIENT_SECRET'],
+    ['play', 'pause', 'next', 'prev', 'search'],
+  ),
+  makeSkill(
+    'plex-manager',
+    'Plex Manager',
+    'Manage Plex media libraries and playback.',
+    ['PLEX_URL', 'PLEX_TOKEN'],
+    ['libraries', 'scan', 'search', 'play', 'sessions'],
+  ),
+  makeSkill(
+    'ytdl-wrapper',
+    'YouTube Downloader',
+    'Download videos from YouTube and other platforms.',
+    ['YTDL_OUTPUT_DIR'],
+    ['download', 'info', 'playlist', 'audio', 'formats'],
+  ),
+  makeSkill(
+    'podcast-dl',
+    'Podcast Downloader',
+    'Download and manage podcast episodes.',
+    ['PODCAST_DIR'],
+    ['subscribe', 'download', 'list', 'play', 'search'],
+  ),
+  makeSkill(
+    'audiobook-player',
+    'Audiobook Player',
+    'Manage and play audiobook collections.',
+    ['AUDIOBOOK_DIR'],
+    ['play', 'pause', 'bookmark', 'list', 'progress'],
+  ),
+  makeSkill(
+    'music-lib',
+    'Music Library',
+    'Organize and query local music libraries.',
+    ['MUSIC_DIR'],
+    ['scan', 'search', 'play', 'playlist', 'stats'],
+  ),
+
+  // Smart Home (8)
+  makeSkill(
+    'hass-control',
+    'Home Assistant Control',
+    'Control Home Assistant entities and automations.',
+    ['HASS_URL', 'HASS_TOKEN'],
+    ['entities', 'services', 'automations', 'scenes', 'history'],
+  ),
+  makeSkill(
+    'zigbee-mqtt',
+    'Zigbee2MQTT',
+    'Manage Zigbee devices via MQTT.',
+    ['MQTT_BROKER', 'ZIGBEE_TOPIC'],
+    ['devices', 'pair', 'remove', 'rename', 'groups'],
+  ),
+  makeSkill(
+    'tasmota-ctl',
+    'Tasmota Control',
+    'Control Tasmota-flashed devices.',
+    ['TASMOTA_HOSTS'],
+    ['status', 'power', 'config', 'update', 'backup'],
+  ),
+  makeSkill(
+    'esphome-mgr',
+    'ESPHome Manager',
+    'Manage ESPHome device configurations.',
+    ['ESPHOME_DIR'],
+    ['compile', 'upload', 'logs', 'dashboard', 'config'],
+  ),
+  makeSkill(
+    'mqtt-broker',
+    'MQTT Broker',
+    'Interact with MQTT brokers for IoT messaging.',
+    ['MQTT_BROKER', 'MQTT_USER'],
+    ['pub', 'sub', 'topics', 'clients', 'stats'],
+  ),
+  makeSkill(
+    'hue-lights',
+    'Philips Hue',
+    'Control Philips Hue lights and scenes.',
+    ['HUE_BRIDGE_IP', 'HUE_API_KEY'],
+    ['lights', 'scenes', 'groups', 'schedules', 'sensors'],
+  ),
+  makeSkill(
+    'smart-thermo',
+    'Smart Thermostat',
+    'Control smart thermostats and HVAC systems.',
+    ['THERMOSTAT_API_KEY'],
+    ['status', 'set', 'schedule', 'history', 'zones'],
+  ),
+  makeSkill(
+    'cam-viewer',
+    'Camera Viewer',
+    'View and manage security camera feeds.',
+    ['CAMERA_URLS'],
+    ['list', 'snapshot', 'stream', 'record', 'events'],
+  ),
+
+  // Finance (5)
+  makeSkill(
+    'budget-track',
+    'Budget Tracker',
+    'Track budgets and spending across categories.',
+    ['BUDGET_DB'],
+    ['summary', 'add', 'categories', 'report', 'goals'],
+  ),
+  makeSkill(
+    'crypto-watch',
+    'Crypto Watcher',
+    'Monitor cryptocurrency prices and portfolios.',
+    ['CRYPTO_API_KEY'],
+    ['prices', 'portfolio', 'alerts', 'history', 'convert'],
+  ),
+  makeSkill(
+    'stock-alerts',
+    'Stock Alerts',
+    'Set up stock price alerts and notifications.',
+    ['STOCK_API_KEY'],
+    ['quote', 'watch', 'alerts', 'portfolio', 'news'],
+  ),
+  makeSkill(
+    'expense-cat',
+    'Expense Categorizer',
+    'Automatically categorize expenses.',
+    ['EXPENSE_DB'],
+    ['import', 'categorize', 'report', 'rules', 'export'],
+  ),
+  makeSkill(
+    'invoice-gen',
+    'Invoice Generator',
+    'Generate and manage invoices.',
+    ['INVOICE_DIR', 'COMPANY_INFO'],
+    ['create', 'list', 'send', 'paid', 'overdue'],
+  ),
+
+  // Communication (5)
+  makeSkill(
+    'slack-bot',
+    'Slack Bot',
+    'Interact with Slack channels and messages.',
+    ['SLACK_TOKEN'],
+    ['send', 'channels', 'users', 'search', 'files'],
+  ),
+  makeSkill(
+    'discord-mgr',
+    'Discord Manager',
+    'Manage Discord servers and messages.',
+    ['DISCORD_TOKEN'],
+    ['send', 'servers', 'channels', 'members', 'roles'],
+  ),
+  makeSkill(
+    'telegram-bot',
+    'Telegram Bot',
+    'Send and receive Telegram messages.',
+    ['TELEGRAM_BOT_TOKEN'],
+    ['send', 'receive', 'chats', 'files', 'inline'],
+  ),
+  makeSkill(
+    'matrix-cli',
+    'Matrix CLI',
+    'Interact with Matrix chat rooms.',
+    ['MATRIX_HOMESERVER', 'MATRIX_TOKEN'],
+    ['send', 'rooms', 'join', 'leave', 'sync'],
+  ),
+  makeSkill(
+    'irc-bridge',
+    'IRC Bridge',
+    'Bridge IRC channels to other platforms.',
+    ['IRC_SERVER', 'IRC_NICK'],
+    ['connect', 'join', 'send', 'channels', 'users'],
+  ),
+
+  // Data & Analytics (5)
+  makeSkill(
+    'pg-queries',
+    'PostgreSQL Queries',
+    'Execute PostgreSQL queries and manage databases.',
+    ['DATABASE_URL'],
+    ['query', 'tables', 'schema', 'backup', 'restore'],
+  ),
+  makeSkill(
+    'clickhouse-ql',
+    'ClickHouse Queries',
+    'Run ClickHouse analytics queries.',
+    ['CLICKHOUSE_URL'],
+    ['query', 'tables', 'insert', 'system', 'optimize'],
+  ),
+  makeSkill(
+    'redis-cli',
+    'Redis CLI',
+    'Interact with Redis cache and data structures.',
+    ['REDIS_URL'],
+    ['get', 'set', 'keys', 'info', 'flush'],
+  ),
+  makeSkill(
+    'elastic-search',
+    'Elasticsearch',
+    'Search and manage Elasticsearch indices.',
+    ['ELASTICSEARCH_URL'],
+    ['search', 'index', 'mapping', 'cluster', 'aliases'],
+  ),
+  makeSkill(
+    'mongo-shell',
+    'MongoDB Shell',
+    'Query and manage MongoDB collections.',
+    ['MONGODB_URI'],
+    ['find', 'insert', 'update', 'delete', 'aggregate'],
+  ),
+
+  // Security (3)
+  makeSkill(
+    'vault-secrets',
+    'Vault Secrets',
+    'Manage secrets in HashiCorp Vault.',
+    ['VAULT_ADDR', 'VAULT_TOKEN'],
+    ['read', 'write', 'list', 'delete', 'seal'],
+  ),
+  makeSkill(
+    'gpg-keys',
+    'GPG Keys',
+    'Manage GPG keys and encryption.',
+    ['GNUPGHOME'],
+    ['list', 'generate', 'export', 'import', 'encrypt'],
+  ),
+  makeSkill(
+    'ssh-rotate',
+    'SSH Key Rotator',
+    'Rotate and manage SSH keys.',
+    ['SSH_KEY_DIR'],
+    ['generate', 'rotate', 'deploy', 'list', 'revoke'],
+  ),
+]
+
+function injectMetadata(rawSkillMd: string, metadata: Record<string, unknown>) {
+  const frontmatterEnd = rawSkillMd.indexOf('\n---', 3)
+  if (frontmatterEnd === -1) return rawSkillMd
+  return `${rawSkillMd.slice(0, frontmatterEnd)}\nmetadata: ${JSON.stringify(
+    metadata,
+  )}${rawSkillMd.slice(frontmatterEnd)}`
+}
+
+function randomStats() {
+  return {
+    downloads: Math.floor(Math.random() * 5000),
+    stars: Math.floor(Math.random() * 500),
+    installsCurrent: Math.floor(Math.random() * 200),
+    installsAllTime: Math.floor(Math.random() * 1000),
+  }
+}
+
+export const applyRandomStats = internalMutation({
+  args: {
+    skillId: v.id('skills'),
+    stats: v.object({
+      downloads: v.number(),
+      stars: v.number(),
+      installsCurrent: v.number(),
+      installsAllTime: v.number(),
+    }),
+  },
+  handler: async (ctx, args) => {
+    await ctx.db.patch(args.skillId, {
+      statsDownloads: args.stats.downloads,
+      statsStars: args.stats.stars,
+      statsInstallsCurrent: args.stats.installsCurrent,
+      statsInstallsAllTime: args.stats.installsAllTime,
+      stats: {
+        downloads: args.stats.downloads,
+        stars: args.stats.stars,
+        installsCurrent: args.stats.installsCurrent,
+        installsAllTime: args.stats.installsAllTime,
+        versions: 1,
+        comments: 0,
+      },
+    })
+  },
+})
+
+export const seedExtraSkillsInternal = internalAction({
+  args: {
+    reset: v.optional(v.boolean()),
+  },
+  handler: async (ctx: ActionCtx, args) => {
+    const results: Array<{ slug: string; ok: boolean; skipped?: boolean }> = []
+
+    for (const spec of EXTRA_SEED_SKILLS) {
+      const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata)
+      const frontmatter = parseFrontmatter(skillMd)
+      const clawdis = parseClawdisMetadata(frontmatter)
+      const storageId = await ctx.storage.store(new Blob([skillMd], { type: 'text/markdown' }))
+
+      const result = (await ctx.runMutation(internal.devSeed.seedSkillMutation, {
+        reset: args.reset,
+        storageId,
+        metadata: spec.metadata,
+        frontmatter,
+        clawdis,
+        skillMd,
+        slug: spec.slug,
+        displayName: spec.displayName,
+        summary: spec.summary,
+        version: spec.version,
+      })) as { ok: boolean; skipped?: boolean; skillId?: string }
+
+      // Apply random stats after creation (only if not skipped)
+      if (result.skillId && !result.skipped) {
+        const stats = randomStats()
+        await ctx.runMutation(internal.devSeedExtra.applyRandomStats, {
+          skillId: result.skillId as Id<'skills'>,
+          stats,
+        })
+      }
+
+      results.push({ slug: spec.slug, ok: result.ok, skipped: result.skipped })
+    }
+
+    const created = results.filter((r) => !r.skipped).length
+    const skipped = results.filter((r) => r.skipped).length
+
+    return { ok: true, total: results.length, created, skipped }
+  },
+})

convex/downloads.ts

@@ -1,7 +1,8 @@
 import { v } from 'convex/values'
-import { zipSync } from 'fflate'
 import { api } from './_generated/api'
 import { httpAction, mutation } from './_generated/server'
+import { buildDeterministicZip } from './lib/skillZip'
+import { insertStatEvent } from './skillStatEvents'
 
 export const downloadZip = httpAction(async (ctx, request) => {
   const url = new URL(request.url)
@@ -18,6 +19,27 @@ export const downloadZip = httpAction(async (ctx, request) => {
     return new Response('Skill not found', { status: 404 })
   }
 
+  // Block downloads based on moderation status
+  const mod = skillResult.moderationInfo
+  if (mod?.isMalwareBlocked) {
+    return new Response(
+      'Blocked: this skill has been flagged as malicious by VirusTotal and cannot be downloaded.',
+      { status: 403 },
+    )
+  }
+  if (mod?.isPendingScan) {
+    return new Response(
+      'This skill is pending a security scan by VirusTotal. Please try again in a few minutes.',
+      { status: 423 },
+    )
+  }
+  if (mod?.isRemoved) {
+    return new Response('This skill has been removed by a moderator.', { status: 410 })
+  }
+  if (mod?.isHiddenByMod) {
+    return new Response('This skill is currently unavailable.', { status: 403 })
+  }
+
   const skill = skillResult.skill
   let version = skillResult.latestVersion
 
@@ -40,16 +62,19 @@ export const downloadZip = httpAction(async (ctx, request) => {
     return new Response('Version not available', { status: 410 })
   }
 
-  const files: Record<string, Uint8Array> = {}
+  const entries: Array<{ path: string; bytes: Uint8Array }> = []
   for (const file of version.files) {
     const blob = await ctx.storage.get(file.storageId)
     if (!blob) continue
     const buffer = new Uint8Array(await blob.arrayBuffer())
-    files[file.path] = buffer
+    entries.push({ path: file.path, bytes: buffer })
   }
-
-  const zipData = zipSync(files, { level: 6 })
-  const zipArray = Uint8Array.from(zipData)
+  const zipArray = buildDeterministicZip(entries, {
+    ownerId: String(skill.ownerUserId),
+    slug: skill.slug,
+    version: version.version,
+    publishedAt: version.createdAt,
+  })
   const zipBlob = new Blob([zipArray], { type: 'application/zip' })
 
   await ctx.runMutation(api.downloads.increment, { skillId: skill._id })
@@ -67,11 +92,12 @@ export const downloadZip = httpAction(async (ctx, request) => {
 export const increment = mutation({
   args: { skillId: v.id('skills') },
   handler: async (ctx, args) => {
-    const skill = await ctx.db.get(args.skillId)
-    if (!skill) return
-    await ctx.db.patch(skill._id, {
-      stats: { ...skill.stats, downloads: skill.stats.downloads + 1 },
-      updatedAt: Date.now(),
+    // Skip db.get to avoid adding the skill doc to the read set.
+    // The calling HTTP action already validated the skill exists,
+    // and the stat processor handles deleted skills gracefully.
+    await insertStatEvent(ctx, {
+      skillId: args.skillId,
+      kind: 'download',
     })
   },
 })

convex/githubBackups.ts

@@ -0,0 +1,170 @@
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import { action, internalMutation, internalQuery } from './_generated/server'
+import { assertRole, requireUserFromAction } from './lib/access'
+
+const DEFAULT_BATCH_SIZE = 50
+const MAX_BATCH_SIZE = 200
+const SYNC_STATE_KEY = 'default'
+
+type BackupPageItem =
+  | {
+      kind: 'ok'
+      skillId: Id<'skills'>
+      versionId: Id<'skillVersions'>
+      slug: string
+      displayName: string
+      version: string
+      ownerHandle: string
+      files: Doc<'skillVersions'>['files']
+      publishedAt: number
+    }
+  | { kind: 'missingLatestVersion'; skillId: Id<'skills'> }
+  | { kind: 'missingVersionDoc'; skillId: Id<'skills'>; versionId: Id<'skillVersions'> }
+  | { kind: 'missingOwner'; skillId: Id<'skills'>; ownerUserId: Id<'users'> }
+
+type BackupPageResult = {
+  items: BackupPageItem[]
+  cursor: string | null
+  isDone: boolean
+}
+
+type BackupSyncState = {
+  cursor: string | null
+}
+
+export type SyncGitHubBackupsResult = {
+  stats: {
+    skillsScanned: number
+    skillsSkipped: number
+    skillsBackedUp: number
+    skillsMissingVersion: number
+    skillsMissingOwner: number
+    errors: number
+  }
+  cursor: string | null
+  isDone: boolean
+}
+
+export const getGitHubBackupPageInternal = internalQuery({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<BackupPageResult> => {
+    const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('skills')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    const items: BackupPageItem[] = []
+    for (const skill of page) {
+      if (skill.softDeletedAt) continue
+      if (!skill.latestVersionId) {
+        items.push({ kind: 'missingLatestVersion', skillId: skill._id })
+        continue
+      }
+
+      const version = await ctx.db.get(skill.latestVersionId)
+      if (!version) {
+        items.push({
+          kind: 'missingVersionDoc',
+          skillId: skill._id,
+          versionId: skill.latestVersionId,
+        })
+        continue
+      }
+
+      const owner = await ctx.db.get(skill.ownerUserId)
+      if (!owner || owner.deletedAt) {
+        items.push({ kind: 'missingOwner', skillId: skill._id, ownerUserId: skill.ownerUserId })
+        continue
+      }
+
+      items.push({
+        kind: 'ok',
+        skillId: skill._id,
+        versionId: version._id,
+        slug: skill.slug,
+        displayName: skill.displayName,
+        version: version.version,
+        ownerHandle: owner.handle ?? owner._id,
+        files: version.files,
+        publishedAt: version.createdAt,
+      })
+    }
+
+    return { items, cursor: continueCursor, isDone }
+  },
+})
+
+export const getGitHubBackupSyncStateInternal = internalQuery({
+  args: {},
+  handler: async (ctx): Promise<BackupSyncState> => {
+    const state = await ctx.db
+      .query('githubBackupSyncState')
+      .withIndex('by_key', (q) => q.eq('key', SYNC_STATE_KEY))
+      .unique()
+    return { cursor: state?.cursor ?? null }
+  },
+})
+
+export const setGitHubBackupSyncStateInternal = internalMutation({
+  args: {
+    cursor: v.optional(v.string()),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    const state = await ctx.db
+      .query('githubBackupSyncState')
+      .withIndex('by_key', (q) => q.eq('key', SYNC_STATE_KEY))
+      .unique()
+
+    if (!state) {
+      await ctx.db.insert('githubBackupSyncState', {
+        key: SYNC_STATE_KEY,
+        cursor: args.cursor,
+        updatedAt: now,
+      })
+      return { ok: true as const }
+    }
+
+    await ctx.db.patch(state._id, {
+      cursor: args.cursor,
+      updatedAt: now,
+    })
+
+    return { ok: true as const }
+  },
+})
+
+export const syncGitHubBackups: ReturnType<typeof action> = action({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+    resetCursor: v.optional(v.boolean()),
+  },
+  handler: async (ctx, args): Promise<SyncGitHubBackupsResult> => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+
+    if (args.resetCursor && !args.dryRun) {
+      await ctx.runMutation(internal.githubBackups.setGitHubBackupSyncStateInternal, {
+        cursor: undefined,
+      })
+    }
+
+    return ctx.runAction(internal.githubBackupsNode.syncGitHubBackupsInternal, {
+      dryRun: args.dryRun,
+      batchSize: args.batchSize,
+      maxBatches: args.maxBatches,
+    }) as Promise<SyncGitHubBackupsResult>
+  },
+})
+
+function clampInt(value: number, min: number, max: number) {
+  return Math.max(min, Math.min(max, Math.floor(value)))
+}

convex/githubBackupsNode.ts

@@ -0,0 +1,183 @@
+'use node'
+
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
+import { internalAction } from './_generated/server'
+import {
+  backupSkillToGitHub,
+  fetchGitHubSkillMeta,
+  getGitHubBackupContext,
+  isGitHubBackupConfigured,
+} from './lib/githubBackup'
+
+const DEFAULT_BATCH_SIZE = 50
+const MAX_BATCH_SIZE = 200
+const DEFAULT_MAX_BATCHES = 5
+const MAX_MAX_BATCHES = 200
+
+type BackupPageItem =
+  | {
+      kind: 'ok'
+      slug: string
+      version: string
+      displayName: string
+      ownerHandle: string
+      files: Doc<'skillVersions'>['files']
+      publishedAt: number
+    }
+  | { kind: 'missingLatestVersion' }
+  | { kind: 'missingVersionDoc' }
+  | { kind: 'missingOwner' }
+
+export type GitHubBackupSyncStats = {
+  skillsScanned: number
+  skillsSkipped: number
+  skillsBackedUp: number
+  skillsMissingVersion: number
+  skillsMissingOwner: number
+  errors: number
+}
+
+export type SyncGitHubBackupsInternalArgs = {
+  dryRun?: boolean
+  batchSize?: number
+  maxBatches?: number
+}
+
+export type SyncGitHubBackupsInternalResult = {
+  stats: GitHubBackupSyncStats
+  cursor: string | null
+  isDone: boolean
+}
+
+export const backupSkillForPublishInternal = internalAction({
+  args: {
+    slug: v.string(),
+    version: v.string(),
+    displayName: v.string(),
+    ownerHandle: v.string(),
+    files: v.array(
+      v.object({
+        path: v.string(),
+        size: v.number(),
+        storageId: v.id('_storage'),
+        sha256: v.string(),
+        contentType: v.optional(v.string()),
+      }),
+    ),
+    publishedAt: v.number(),
+  },
+  handler: async (ctx, args) => {
+    if (!isGitHubBackupConfigured()) {
+      return { skipped: true as const }
+    }
+    await backupSkillToGitHub(ctx, args)
+    return { skipped: false as const }
+  },
+})
+
+export async function syncGitHubBackupsInternalHandler(
+  ctx: ActionCtx,
+  args: SyncGitHubBackupsInternalArgs,
+): Promise<SyncGitHubBackupsInternalResult> {
+  const dryRun = Boolean(args.dryRun)
+  const stats: GitHubBackupSyncStats = {
+    skillsScanned: 0,
+    skillsSkipped: 0,
+    skillsBackedUp: 0,
+    skillsMissingVersion: 0,
+    skillsMissingOwner: 0,
+    errors: 0,
+  }
+
+  if (!isGitHubBackupConfigured()) {
+    return { stats, cursor: null, isDone: true }
+  }
+
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+  const context = await getGitHubBackupContext()
+
+  const state = dryRun
+    ? { cursor: null as string | null }
+    : ((await ctx.runQuery(internal.githubBackups.getGitHubBackupSyncStateInternal, {})) as {
+        cursor: string | null
+      })
+
+  let cursor: string | null = state.cursor
+  let isDone = false
+
+  for (let batch = 0; batch < maxBatches; batch++) {
+    const page = (await ctx.runQuery(internal.githubBackups.getGitHubBackupPageInternal, {
+      cursor: cursor ?? undefined,
+      batchSize,
+    })) as { items: BackupPageItem[]; cursor: string | null; isDone: boolean }
+
+    cursor = page.cursor
+    isDone = page.isDone
+
+    for (const item of page.items) {
+      if (item.kind !== 'ok') {
+        if (item.kind === 'missingLatestVersion' || item.kind === 'missingVersionDoc') {
+          stats.skillsMissingVersion += 1
+        } else if (item.kind === 'missingOwner') {
+          stats.skillsMissingOwner += 1
+        }
+        continue
+      }
+
+      stats.skillsScanned += 1
+      try {
+        const meta = await fetchGitHubSkillMeta(context, item.ownerHandle, item.slug)
+        if (meta?.latest?.version === item.version) {
+          stats.skillsSkipped += 1
+          continue
+        }
+
+        if (!dryRun) {
+          await backupSkillToGitHub(
+            ctx,
+            {
+              slug: item.slug,
+              version: item.version,
+              displayName: item.displayName,
+              ownerHandle: item.ownerHandle,
+              files: item.files,
+              publishedAt: item.publishedAt,
+            },
+            context,
+          )
+          stats.skillsBackedUp += 1
+        }
+      } catch (error) {
+        console.error('GitHub backup sync failed', error)
+        stats.errors += 1
+      }
+    }
+
+    if (!dryRun) {
+      await ctx.runMutation(internal.githubBackups.setGitHubBackupSyncStateInternal, {
+        cursor: isDone ? undefined : (cursor ?? undefined),
+      })
+    }
+
+    if (isDone) break
+  }
+
+  return { stats, cursor, isDone }
+}
+
+export const syncGitHubBackupsInternal = internalAction({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: syncGitHubBackupsInternalHandler,
+})
+
+function clampInt(value: number, min: number, max: number) {
+  return Math.max(min, Math.min(max, Math.floor(value)))
+}

convex/githubImport.ts

@@ -0,0 +1,317 @@
+import { ConvexError, v } from 'convex/values'
+import { unzipSync } from 'fflate'
+import semver from 'semver'
+import { api, internal } from './_generated/api'
+import type { Id } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
+import { action } from './_generated/server'
+import { requireUserFromAction } from './lib/access'
+import {
+  buildGitHubImportFileList,
+  computeDefaultSelectedPaths,
+  detectGitHubImportCandidates,
+  fetchGitHubZipBytes,
+  listTextFilesUnderCandidate,
+  normalizeRepoPath,
+  parseGitHubImportUrl,
+  resolveGitHubCommit,
+  stripGitHubZipRoot,
+  suggestDisplayName,
+  suggestVersion,
+} from './lib/githubImport'
+import { publishVersionForUser } from './lib/skillPublish'
+import { sanitizePath } from './lib/skills'
+
+const MAX_SELECTED_BYTES = 50 * 1024 * 1024
+const MAX_UNZIPPED_BYTES = 80 * 1024 * 1024
+const MAX_FILE_COUNT = 7_500
+const MAX_SINGLE_FILE_BYTES = 10 * 1024 * 1024
+
+export const previewGitHubImport = action({
+  args: { url: v.string() },
+  handler: async (ctx, args) => {
+    await requireUserFromAction(ctx)
+
+    const parsed = parseGitHubImportUrl(args.url)
+    const resolved = await resolveGitHubCommit(parsed, fetch)
+    const zipBytes = await fetchGitHubZipBytes(resolved, fetch)
+    const entries = unzipToEntries(zipBytes)
+    const stripped = stripGitHubZipRoot(entries)
+    const candidates = detectGitHubImportCandidates(stripped).filter((candidate) =>
+      isCandidateUnderResolvedPath(candidate.path, resolved.path),
+    )
+    if (candidates.length === 0) throw new ConvexError('No SKILL.md found in this repo')
+
+    return {
+      resolved,
+      candidates: candidates.map((candidate) => ({
+        path: candidate.path,
+        readmePath: candidate.readmePath,
+        name: candidate.name ?? null,
+        description: candidate.description ?? null,
+      })),
+    }
+  },
+})
+
+export const previewGitHubImportCandidate = action({
+  args: { url: v.string(), candidatePath: v.string() },
+  handler: async (ctx, args) => {
+    const { userId } = await requireUserFromAction(ctx)
+
+    const parsed = parseGitHubImportUrl(args.url)
+    const resolved = await resolveGitHubCommit(parsed, fetch)
+    const zipBytes = await fetchGitHubZipBytes(resolved, fetch)
+    const entries = unzipToEntries(zipBytes)
+    const stripped = stripGitHubZipRoot(entries)
+
+    const normalizedCandidatePath = normalizeRepoPath(args.candidatePath)
+    if (!isCandidateUnderResolvedPath(normalizedCandidatePath, resolved.path)) {
+      throw new ConvexError('Candidate path is outside the requested import scope')
+    }
+
+    const candidates = detectGitHubImportCandidates(stripped).filter((candidate) =>
+      isCandidateUnderResolvedPath(candidate.path, resolved.path),
+    )
+
+    const candidate = candidates.find((item) => item.path === normalizedCandidatePath)
+    if (!candidate) throw new ConvexError('Candidate not found')
+
+    const files = listTextFilesUnderCandidate(stripped, candidate.path)
+    const defaultSelectedPaths = computeDefaultSelectedPaths({ candidate, files })
+    const fileList = buildGitHubImportFileList({
+      candidate,
+      files,
+      defaultSelectedPaths,
+    })
+
+    const baseForNaming = candidate.path ? (candidate.path.split('/').at(-1) ?? '') : resolved.repo
+    const suggestedDisplayName = suggestDisplayName(candidate, baseForNaming)
+
+    const rawSlugBase = sanitizeSlug(candidate.path ? baseForNaming : resolved.repo)
+    const suggestedSlug = await suggestAvailableSlug(ctx, userId, rawSlugBase)
+
+    const existing = await ctx.runQuery(api.skills.getBySlug, { slug: suggestedSlug })
+    const existingLatest =
+      existing?.skill && existing.skill.ownerUserId === userId
+        ? (existing.latestVersion?.version ?? null)
+        : null
+    const suggestedVersion = suggestVersion(existingLatest)
+
+    return {
+      resolved,
+      candidate: {
+        path: candidate.path,
+        readmePath: candidate.readmePath,
+        name: candidate.name ?? null,
+        description: candidate.description ?? null,
+      },
+      defaults: {
+        selectedPaths: defaultSelectedPaths,
+        slug: suggestedSlug,
+        displayName: suggestedDisplayName,
+        version: suggestedVersion,
+        tags: ['latest'],
+      },
+      files: fileList,
+    }
+  },
+})
+
+export const importGitHubSkill = action({
+  args: {
+    url: v.string(),
+    commit: v.string(),
+    candidatePath: v.string(),
+    selectedPaths: v.array(v.string()),
+    slug: v.optional(v.string()),
+    displayName: v.optional(v.string()),
+    version: v.optional(v.string()),
+    tags: v.optional(v.array(v.string())),
+  },
+  handler: async (ctx, args) => {
+    const { userId } = await requireUserFromAction(ctx)
+
+    const parsed = parseGitHubImportUrl(args.url)
+    const resolved = await resolveGitHubCommit(parsed, fetch)
+    if (!/^[a-f0-9]{40}$/i.test(args.commit)) throw new ConvexError('Invalid commit')
+    if (args.commit.toLowerCase() !== resolved.commit.toLowerCase()) {
+      throw new ConvexError('Import is out of date. Re-run preview.')
+    }
+
+    const normalizedCandidatePath = normalizeRepoPath(args.candidatePath)
+    if (!isCandidateUnderResolvedPath(normalizedCandidatePath, resolved.path)) {
+      throw new ConvexError('Candidate path is outside the requested import scope')
+    }
+
+    const zipBytes = await fetchGitHubZipBytes(resolved, fetch)
+    const entries = stripGitHubZipRoot(unzipToEntries(zipBytes))
+
+    const candidates = detectGitHubImportCandidates(entries).filter((candidate) =>
+      isCandidateUnderResolvedPath(candidate.path, resolved.path),
+    )
+    const candidate = candidates.find((item) => item.path === normalizedCandidatePath)
+    if (!candidate) throw new ConvexError('Candidate not found')
+
+    const filesUnderCandidate = listTextFilesUnderCandidate(entries, candidate.path)
+    const byPath = new Map(filesUnderCandidate.map((file) => [file.path, file.bytes]))
+
+    const selected = Array.from(
+      new Set(args.selectedPaths.map((path) => normalizeRepoPath(path)).filter(Boolean)),
+    )
+    if (selected.length === 0) throw new ConvexError('No files selected')
+
+    const candidateRoot = candidate.path ? `${candidate.path}/` : ''
+    const normalizedReadmePath = normalizeRepoPath(candidate.readmePath)
+    if (!selected.includes(normalizedReadmePath)) {
+      throw new ConvexError('SKILL.md must be selected')
+    }
+
+    let totalBytes = 0
+    const storedFiles: Array<{
+      path: string
+      size: number
+      storageId: Id<'_storage'>
+      sha256: string
+      contentType?: string
+    }> = []
+
+    for (const path of selected.sort()) {
+      if (candidateRoot && !path.startsWith(candidateRoot)) {
+        throw new ConvexError('Selected file is outside the chosen skill folder')
+      }
+
+      const bytes = byPath.get(path)
+      if (!bytes) continue
+      totalBytes += bytes.byteLength
+      if (totalBytes > MAX_SELECTED_BYTES) throw new ConvexError('Selected files exceed 50MB limit')
+
+      const relPath = candidateRoot ? path.slice(candidateRoot.length) : path
+      const sanitized = sanitizePath(relPath)
+      if (!sanitized) throw new ConvexError('Invalid file paths')
+
+      const sha256 = await sha256Hex(bytes)
+      const safeBytes = new Uint8Array(bytes)
+      const storageId = await ctx.storage.store(new Blob([safeBytes], { type: 'text/plain' }))
+      storedFiles.push({
+        path: sanitized,
+        size: bytes.byteLength,
+        storageId,
+        sha256,
+        contentType: 'text/plain',
+      })
+    }
+
+    if (storedFiles.length === 0) throw new ConvexError('No files selected')
+
+    const slugBase = (args.slug ?? '').trim().toLowerCase()
+    const displayName = (args.displayName ?? '').trim()
+    const tags = (args.tags ?? ['latest']).map((tag) => tag.trim()).filter(Boolean)
+    const version = (args.version ?? '').trim()
+
+    if (!slugBase) throw new ConvexError('Slug required')
+    if (!displayName) throw new ConvexError('Display name required')
+    if (!version || !semver.valid(version)) throw new ConvexError('Version must be valid semver')
+
+    const result = await publishVersionForUser(ctx, userId, {
+      slug: slugBase,
+      displayName,
+      version,
+      changelog: '',
+      tags,
+      files: storedFiles,
+      source: {
+        kind: 'github',
+        url: resolved.originalUrl,
+        repo: `${resolved.owner}/${resolved.repo}`,
+        ref: resolved.ref,
+        commit: resolved.commit,
+        path: candidate.path,
+        importedAt: Date.now(),
+      },
+    })
+
+    return { ok: true, slug: slugBase, version, ...result }
+  },
+})
+
+function unzipToEntries(zipBytes: Uint8Array) {
+  const entries = unzipSync(zipBytes)
+  const out: Record<string, Uint8Array> = {}
+  const rawPaths = Object.keys(entries)
+  if (rawPaths.length > MAX_FILE_COUNT) throw new ConvexError('Repo archive has too many files')
+  let totalBytes = 0
+  for (const [rawPath, bytes] of Object.entries(entries)) {
+    const normalizedPath = normalizeZipPath(rawPath)
+    if (!normalizedPath) continue
+    if (isJunkPath(normalizedPath)) continue
+    if (!bytes) continue
+    if (bytes.byteLength > MAX_SINGLE_FILE_BYTES) continue
+    totalBytes += bytes.byteLength
+    if (totalBytes > MAX_UNZIPPED_BYTES) throw new ConvexError('Repo archive is too large')
+    out[normalizedPath] = bytes
+  }
+  return out
+}
+
+function isCandidateUnderResolvedPath(candidatePath: string, resolvedPath: string) {
+  const root = normalizeRepoPath(resolvedPath)
+  if (!root) return true
+  if (!candidatePath) return false
+  if (candidatePath === root) return true
+  return candidatePath.startsWith(`${root}/`)
+}
+
+function sanitizeSlug(value: string) {
+  return value
+    .trim()
+    .toLowerCase()
+    .replace(/[^a-z0-9-]+/g, '-')
+    .replace(/^-+/, '')
+    .replace(/-+$/, '')
+    .replace(/--+/g, '-')
+}
+
+async function suggestAvailableSlug(ctx: ActionCtx, userId: Id<'users'>, base: string) {
+  const cleaned = sanitizeSlug(base)
+  if (!cleaned) throw new ConvexError('Could not derive slug')
+  for (let i = 0; i < 50; i += 1) {
+    const candidate = i === 0 ? cleaned : `${cleaned}-${i + 1}`
+    const existing = await ctx.runQuery(internal.skills.getSkillBySlugInternal, { slug: candidate })
+    if (!existing) return candidate
+    if (existing.ownerUserId === userId) return candidate
+  }
+  throw new ConvexError('Could not find an available slug')
+}
+
+async function sha256Hex(bytes: Uint8Array) {
+  const normalized = new Uint8Array(bytes)
+  const digest = await crypto.subtle.digest('SHA-256', normalized.buffer)
+  return toHex(new Uint8Array(digest))
+}
+
+function toHex(bytes: Uint8Array) {
+  let out = ''
+  for (const byte of bytes) out += byte.toString(16).padStart(2, '0')
+  return out
+}
+
+function normalizeZipPath(path: string) {
+  const normalized = path
+    .replaceAll('\u0000', '')
+    .replaceAll('\\', '/')
+    .trim()
+    .replace(/^\.\/+/, '')
+    .replace(/^\/+/, '')
+  if (!normalized) return ''
+  if (normalized.includes('..')) return ''
+  return normalized
+}
+
+function isJunkPath(path: string) {
+  const normalized = path.toLowerCase()
+  if (normalized.startsWith('__macosx/')) return true
+  if (normalized.endsWith('/.ds_store')) return true
+  if (normalized === '.ds_store') return true
+  return false
+}

convex/githubSoulBackups.ts

@@ -0,0 +1,170 @@
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import { action, internalMutation, internalQuery } from './_generated/server'
+import { assertRole, requireUserFromAction } from './lib/access'
+
+const DEFAULT_BATCH_SIZE = 50
+const MAX_BATCH_SIZE = 200
+const SYNC_STATE_KEY = 'souls'
+
+type BackupPageItem =
+  | {
+      kind: 'ok'
+      soulId: Id<'souls'>
+      versionId: Id<'soulVersions'>
+      slug: string
+      displayName: string
+      version: string
+      ownerHandle: string
+      files: Doc<'soulVersions'>['files']
+      publishedAt: number
+    }
+  | { kind: 'missingLatestVersion'; soulId: Id<'souls'> }
+  | { kind: 'missingVersionDoc'; soulId: Id<'souls'>; versionId: Id<'soulVersions'> }
+  | { kind: 'missingOwner'; soulId: Id<'souls'>; ownerUserId: Id<'users'> }
+
+type BackupPageResult = {
+  items: BackupPageItem[]
+  cursor: string | null
+  isDone: boolean
+}
+
+type BackupSyncState = {
+  cursor: string | null
+}
+
+export type SyncGitHubSoulBackupsResult = {
+  stats: {
+    soulsScanned: number
+    soulsSkipped: number
+    soulsBackedUp: number
+    soulsMissingVersion: number
+    soulsMissingOwner: number
+    errors: number
+  }
+  cursor: string | null
+  isDone: boolean
+}
+
+export const getGitHubSoulBackupPageInternal = internalQuery({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<BackupPageResult> => {
+    const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('souls')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    const items: BackupPageItem[] = []
+    for (const soul of page) {
+      if (soul.softDeletedAt) continue
+      if (!soul.latestVersionId) {
+        items.push({ kind: 'missingLatestVersion', soulId: soul._id })
+        continue
+      }
+
+      const version = await ctx.db.get(soul.latestVersionId)
+      if (!version) {
+        items.push({
+          kind: 'missingVersionDoc',
+          soulId: soul._id,
+          versionId: soul.latestVersionId,
+        })
+        continue
+      }
+
+      const owner = await ctx.db.get(soul.ownerUserId)
+      if (!owner || owner.deletedAt) {
+        items.push({ kind: 'missingOwner', soulId: soul._id, ownerUserId: soul.ownerUserId })
+        continue
+      }
+
+      items.push({
+        kind: 'ok',
+        soulId: soul._id,
+        versionId: version._id,
+        slug: soul.slug,
+        displayName: soul.displayName,
+        version: version.version,
+        ownerHandle: owner.handle ?? owner._id,
+        files: version.files,
+        publishedAt: version.createdAt,
+      })
+    }
+
+    return { items, cursor: continueCursor, isDone }
+  },
+})
+
+export const getGitHubSoulBackupSyncStateInternal = internalQuery({
+  args: {},
+  handler: async (ctx): Promise<BackupSyncState> => {
+    const state = await ctx.db
+      .query('githubBackupSyncState')
+      .withIndex('by_key', (q) => q.eq('key', SYNC_STATE_KEY))
+      .unique()
+    return { cursor: state?.cursor ?? null }
+  },
+})
+
+export const setGitHubSoulBackupSyncStateInternal = internalMutation({
+  args: {
+    cursor: v.optional(v.string()),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    const state = await ctx.db
+      .query('githubBackupSyncState')
+      .withIndex('by_key', (q) => q.eq('key', SYNC_STATE_KEY))
+      .unique()
+
+    if (!state) {
+      await ctx.db.insert('githubBackupSyncState', {
+        key: SYNC_STATE_KEY,
+        cursor: args.cursor,
+        updatedAt: now,
+      })
+      return { ok: true as const }
+    }
+
+    await ctx.db.patch(state._id, {
+      cursor: args.cursor,
+      updatedAt: now,
+    })
+
+    return { ok: true as const }
+  },
+})
+
+export const syncGitHubSoulBackups: ReturnType<typeof action> = action({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+    resetCursor: v.optional(v.boolean()),
+  },
+  handler: async (ctx, args): Promise<SyncGitHubSoulBackupsResult> => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+
+    if (args.resetCursor && !args.dryRun) {
+      await ctx.runMutation(internal.githubSoulBackups.setGitHubSoulBackupSyncStateInternal, {
+        cursor: undefined,
+      })
+    }
+
+    return ctx.runAction(internal.githubSoulBackupsNode.syncGitHubSoulBackupsInternal, {
+      dryRun: args.dryRun,
+      batchSize: args.batchSize,
+      maxBatches: args.maxBatches,
+    }) as Promise<SyncGitHubSoulBackupsResult>
+  },
+})
+
+function clampInt(value: number, min: number, max: number) {
+  return Math.max(min, Math.min(max, Math.floor(value)))
+}

convex/githubSoulBackupsNode.ts

@@ -0,0 +1,186 @@
+'use node'
+
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
+import { internalAction } from './_generated/server'
+import {
+  backupSoulToGitHub,
+  fetchGitHubSoulMeta,
+  getGitHubSoulBackupContext,
+  isGitHubSoulBackupConfigured,
+} from './lib/githubSoulBackup'
+
+const DEFAULT_BATCH_SIZE = 50
+const MAX_BATCH_SIZE = 200
+const DEFAULT_MAX_BATCHES = 5
+const MAX_MAX_BATCHES = 200
+
+type BackupPageItem =
+  | {
+      kind: 'ok'
+      slug: string
+      version: string
+      displayName: string
+      ownerHandle: string
+      files: Doc<'soulVersions'>['files']
+      publishedAt: number
+    }
+  | { kind: 'missingLatestVersion' }
+  | { kind: 'missingVersionDoc' }
+  | { kind: 'missingOwner' }
+
+export type GitHubSoulBackupSyncStats = {
+  soulsScanned: number
+  soulsSkipped: number
+  soulsBackedUp: number
+  soulsMissingVersion: number
+  soulsMissingOwner: number
+  errors: number
+}
+
+export type SyncGitHubSoulBackupsInternalArgs = {
+  dryRun?: boolean
+  batchSize?: number
+  maxBatches?: number
+}
+
+export type SyncGitHubSoulBackupsInternalResult = {
+  stats: GitHubSoulBackupSyncStats
+  cursor: string | null
+  isDone: boolean
+}
+
+export const backupSoulForPublishInternal = internalAction({
+  args: {
+    slug: v.string(),
+    version: v.string(),
+    displayName: v.string(),
+    ownerHandle: v.string(),
+    files: v.array(
+      v.object({
+        path: v.string(),
+        size: v.number(),
+        storageId: v.id('_storage'),
+        sha256: v.string(),
+        contentType: v.optional(v.string()),
+      }),
+    ),
+    publishedAt: v.number(),
+  },
+  handler: async (ctx, args) => {
+    if (!isGitHubSoulBackupConfigured()) {
+      return { skipped: true as const }
+    }
+    await backupSoulToGitHub(ctx, args)
+    return { skipped: false as const }
+  },
+})
+
+export async function syncGitHubSoulBackupsInternalHandler(
+  ctx: ActionCtx,
+  args: SyncGitHubSoulBackupsInternalArgs,
+): Promise<SyncGitHubSoulBackupsInternalResult> {
+  const dryRun = Boolean(args.dryRun)
+  const stats: GitHubSoulBackupSyncStats = {
+    soulsScanned: 0,
+    soulsSkipped: 0,
+    soulsBackedUp: 0,
+    soulsMissingVersion: 0,
+    soulsMissingOwner: 0,
+    errors: 0,
+  }
+
+  if (!isGitHubSoulBackupConfigured()) {
+    return { stats, cursor: null, isDone: true }
+  }
+
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+  const context = await getGitHubSoulBackupContext()
+
+  const state = dryRun
+    ? { cursor: null as string | null }
+    : ((await ctx.runQuery(
+        internal.githubSoulBackups.getGitHubSoulBackupSyncStateInternal,
+        {},
+      )) as {
+        cursor: string | null
+      })
+
+  let cursor: string | null = state.cursor
+  let isDone = false
+
+  for (let batch = 0; batch < maxBatches; batch++) {
+    const page = (await ctx.runQuery(internal.githubSoulBackups.getGitHubSoulBackupPageInternal, {
+      cursor: cursor ?? undefined,
+      batchSize,
+    })) as { items: BackupPageItem[]; cursor: string | null; isDone: boolean }
+
+    cursor = page.cursor
+    isDone = page.isDone
+
+    for (const item of page.items) {
+      if (item.kind !== 'ok') {
+        if (item.kind === 'missingLatestVersion' || item.kind === 'missingVersionDoc') {
+          stats.soulsMissingVersion += 1
+        } else if (item.kind === 'missingOwner') {
+          stats.soulsMissingOwner += 1
+        }
+        continue
+      }
+
+      stats.soulsScanned += 1
+      try {
+        const meta = await fetchGitHubSoulMeta(context, item.ownerHandle, item.slug)
+        if (meta?.latest?.version === item.version) {
+          stats.soulsSkipped += 1
+          continue
+        }
+
+        if (!dryRun) {
+          await backupSoulToGitHub(
+            ctx,
+            {
+              slug: item.slug,
+              version: item.version,
+              displayName: item.displayName,
+              ownerHandle: item.ownerHandle,
+              files: item.files,
+              publishedAt: item.publishedAt,
+            },
+            context,
+          )
+          stats.soulsBackedUp += 1
+        }
+      } catch (error) {
+        console.error('GitHub soul backup sync failed', error)
+        stats.errors += 1
+      }
+    }
+
+    if (!dryRun) {
+      await ctx.runMutation(internal.githubSoulBackups.setGitHubSoulBackupSyncStateInternal, {
+        cursor: isDone ? undefined : (cursor ?? undefined),
+      })
+    }
+
+    if (isDone) break
+  }
+
+  return { stats, cursor, isDone }
+}
+
+export const syncGitHubSoulBackupsInternal = internalAction({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: syncGitHubSoulBackupsInternalHandler,
+})
+
+function clampInt(value: number, min: number, max: number) {
+  return Math.max(min, Math.min(max, Math.floor(value)))
+}

convex/http.ts

@@ -1,4 +1,4 @@
-import { ApiRoutes } from 'clawdhub-schema'
+import { ApiRoutes, LegacyApiRoutes } from 'clawhub-schema'
 import { httpRouter } from 'convex/server'
 import { auth } from './auth'
 import { downloadZip } from './downloads'
@@ -6,12 +6,32 @@ import {
   cliPublishHttp,
   cliSkillDeleteHttp,
   cliSkillUndeleteHttp,
+  cliTelemetrySyncHttp,
   cliUploadUrlHttp,
   cliWhoamiHttp,
   getSkillHttp,
   resolveSkillVersionHttp,
   searchSkillsHttp,
 } from './httpApi'
+import {
+  listSkillsV1Http,
+  listSoulsV1Http,
+  publishSkillV1Http,
+  publishSoulV1Http,
+  resolveSkillVersionV1Http,
+  searchSkillsV1Http,
+  skillsDeleteRouterV1Http,
+  skillsGetRouterV1Http,
+  skillsPostRouterV1Http,
+  soulsDeleteRouterV1Http,
+  soulsGetRouterV1Http,
+  soulsPostRouterV1Http,
+  starsDeleteRouterV1Http,
+  starsPostRouterV1Http,
+  usersListV1Http,
+  usersPostRouterV1Http,
+  whoamiV1Http,
+} from './httpApiV1'
 
 const http = httpRouter()
 
@@ -26,47 +46,161 @@ http.route({
 http.route({
   path: ApiRoutes.search,
   method: 'GET',
+  handler: searchSkillsV1Http,
+})
+
+http.route({
+  path: ApiRoutes.resolve,
+  method: 'GET',
+  handler: resolveSkillVersionV1Http,
+})
+
+http.route({
+  path: ApiRoutes.skills,
+  method: 'GET',
+  handler: listSkillsV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.skills}/`,
+  method: 'GET',
+  handler: skillsGetRouterV1Http,
+})
+
+http.route({
+  path: ApiRoutes.skills,
+  method: 'POST',
+  handler: publishSkillV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.skills}/`,
+  method: 'POST',
+  handler: skillsPostRouterV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.skills}/`,
+  method: 'DELETE',
+  handler: skillsDeleteRouterV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.stars}/`,
+  method: 'POST',
+  handler: starsPostRouterV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.stars}/`,
+  method: 'DELETE',
+  handler: starsDeleteRouterV1Http,
+})
+
+http.route({
+  path: ApiRoutes.whoami,
+  method: 'GET',
+  handler: whoamiV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.users}/`,
+  method: 'POST',
+  handler: usersPostRouterV1Http,
+})
+
+http.route({
+  path: ApiRoutes.users,
+  method: 'GET',
+  handler: usersListV1Http,
+})
+
+http.route({
+  path: ApiRoutes.souls,
+  method: 'GET',
+  handler: listSoulsV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.souls}/`,
+  method: 'GET',
+  handler: soulsGetRouterV1Http,
+})
+
+http.route({
+  path: ApiRoutes.souls,
+  method: 'POST',
+  handler: publishSoulV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.souls}/`,
+  method: 'POST',
+  handler: soulsPostRouterV1Http,
+})
+
+http.route({
+  pathPrefix: `${ApiRoutes.souls}/`,
+  method: 'DELETE',
+  handler: soulsDeleteRouterV1Http,
+})
+
+// TODO: remove legacy /api routes after deprecation window.
+http.route({
+  path: LegacyApiRoutes.download,
+  method: 'GET',
+  handler: downloadZip,
+})
+http.route({
+  path: LegacyApiRoutes.search,
+  method: 'GET',
   handler: searchSkillsHttp,
 })
 
 http.route({
-  path: ApiRoutes.skill,
+  path: LegacyApiRoutes.skill,
   method: 'GET',
   handler: getSkillHttp,
 })
 
 http.route({
-  path: ApiRoutes.skillResolve,
+  path: LegacyApiRoutes.skillResolve,
   method: 'GET',
   handler: resolveSkillVersionHttp,
 })
 
 http.route({
-  path: ApiRoutes.cliWhoami,
+  path: LegacyApiRoutes.cliWhoami,
   method: 'GET',
   handler: cliWhoamiHttp,
 })
 
 http.route({
-  path: ApiRoutes.cliUploadUrl,
+  path: LegacyApiRoutes.cliUploadUrl,
   method: 'POST',
   handler: cliUploadUrlHttp,
 })
 
 http.route({
-  path: ApiRoutes.cliPublish,
+  path: LegacyApiRoutes.cliPublish,
   method: 'POST',
   handler: cliPublishHttp,
 })
 
 http.route({
-  path: ApiRoutes.cliSkillDelete,
+  path: LegacyApiRoutes.cliTelemetrySync,
+  method: 'POST',
+  handler: cliTelemetrySyncHttp,
+})
+
+http.route({
+  path: LegacyApiRoutes.cliSkillDelete,
   method: 'POST',
   handler: cliSkillDeleteHttp,
 })
 
 http.route({
-  path: ApiRoutes.cliSkillUndelete,
+  path: LegacyApiRoutes.cliSkillUndelete,
   method: 'POST',
   handler: cliSkillUndeleteHttp,
 })

convex/httpApi.handlers.test.ts

@@ -14,6 +14,10 @@ const { publishVersionForUser } = await import('./skills')
 const { __handlers } = await import('./httpApi')
 const { hashSkillFiles } = await import('./lib/skills')
 
+function makeCtx(partial: Record<string, unknown>) {
+  return partial as unknown as import('./_generated/server').ActionCtx
+}
+
 describe('httpApi handlers', () => {
   afterEach(() => {
     vi.mocked(requireApiTokenUser).mockReset()
@@ -22,14 +26,14 @@ describe('httpApi handlers', () => {
 
   it('searchSkillsHttp returns empty results for empty query', async () => {
     const response = await __handlers.searchSkillsHandler(
-      { runAction: vi.fn() },
+      makeCtx({ runAction: vi.fn() }),
       new Request('https://example.com/api/search?q=%20%20'),
     )
     expect(response.status).toBe(200)
     expect(await response.json()).toEqual({ results: [] })
   })
 
-  it('searchSkillsHttp forwards args', async () => {
+  it('searchSkillsHttp forwards args (approvedOnly alias)', async () => {
     const runAction = vi.fn().mockResolvedValue([
       {
         score: 1,
@@ -38,22 +42,48 @@ describe('httpApi handlers', () => {
       },
     ])
     const response = await __handlers.searchSkillsHandler(
-      { runAction },
+      makeCtx({ runAction }),
       new Request('https://example.com/api/search?q=test&approvedOnly=true&limit=5'),
     )
     expect(runAction).toHaveBeenCalledWith(expect.anything(), {
       query: 'test',
       limit: 5,
-      approvedOnly: true,
+      highlightedOnly: true,
     })
     expect(response.status).toBe(200)
     const json = await response.json()
     expect(json.results[0].slug).toBe('a')
   })
 
+  it('searchSkillsHttp forwards highlightedOnly', async () => {
+    const runAction = vi.fn().mockResolvedValue([])
+    await __handlers.searchSkillsHandler(
+      makeCtx({ runAction }),
+      new Request('https://example.com/api/search?q=test&highlightedOnly=true'),
+    )
+    expect(runAction).toHaveBeenCalledWith(expect.anything(), {
+      query: 'test',
+      limit: undefined,
+      highlightedOnly: true,
+    })
+  })
+
+  it('searchSkillsHttp omits highlightedOnly when approvedOnly is false', async () => {
+    const runAction = vi.fn().mockResolvedValue([])
+    await __handlers.searchSkillsHandler(
+      makeCtx({ runAction }),
+      new Request('https://example.com/api/search?q=test&approvedOnly=false'),
+    )
+    expect(runAction).toHaveBeenCalledWith(expect.anything(), {
+      query: 'test',
+      limit: undefined,
+      highlightedOnly: undefined,
+    })
+  })
+
   it('getSkillHttp validates slug', async () => {
     const response = await __handlers.getSkillHandler(
-      { runQuery: vi.fn() },
+      makeCtx({ runQuery: vi.fn() }),
       new Request('https://example.com/api/skill'),
     )
     expect(response.status).toBe(400)
@@ -62,7 +92,7 @@ describe('httpApi handlers', () => {
   it('getSkillHttp returns 404 when missing', async () => {
     const runQuery = vi.fn().mockResolvedValue(null)
     const response = await __handlers.getSkillHandler(
-      { runQuery },
+      makeCtx({ runQuery }),
       new Request('https://example.com/api/skill?slug=missing'),
     )
     expect(response.status).toBe(404)
@@ -75,7 +105,14 @@ describe('httpApi handlers', () => {
         displayName: 'Demo',
         summary: 'x',
         tags: {},
-        stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
+        stats: {
+          downloads: 0,
+          installsCurrent: 0,
+          installsAllTime: 0,
+          stars: 0,
+          versions: 1,
+          comments: 0,
+        },
         createdAt: 1,
         updatedAt: 2,
       },
@@ -83,7 +120,7 @@ describe('httpApi handlers', () => {
       owner: { handle: 'p', displayName: 'Peter', image: null },
     })
     const response = await __handlers.getSkillHandler(
-      { runQuery },
+      makeCtx({ runQuery }),
       new Request('https://example.com/api/skill?slug=demo'),
     )
     expect(response.status).toBe(200)
@@ -93,9 +130,33 @@ describe('httpApi handlers', () => {
     expect(json.owner.handle).toBe('p')
   })
 
+  it('getSkillHttp returns payload with null owner/latestVersion', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      skill: {
+        slug: 'demo',
+        displayName: 'Demo',
+        summary: null,
+        tags: {},
+        stats: {},
+        createdAt: 1,
+        updatedAt: 2,
+      },
+      latestVersion: null,
+      owner: null,
+    })
+    const response = await __handlers.getSkillHandler(
+      makeCtx({ runQuery }),
+      new Request('https://example.com/api/skill?slug=demo'),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.latestVersion).toBeNull()
+    expect(json.owner).toBeNull()
+  })
+
   it('resolveSkillVersionHttp validates hash', async () => {
     const response = await __handlers.resolveSkillVersionHandler(
-      { runQuery: vi.fn() },
+      makeCtx({ runQuery: vi.fn() }),
       new Request('https://example.com/api/skill/resolve?slug=demo&hash=bad'),
     )
     expect(response.status).toBe(400)
@@ -104,7 +165,7 @@ describe('httpApi handlers', () => {
   it('resolveSkillVersionHttp returns 404 when missing', async () => {
     const runQuery = vi.fn().mockResolvedValue(null)
     const response = await __handlers.resolveSkillVersionHandler(
-      { runQuery },
+      makeCtx({ runQuery }),
       new Request(
         'https://example.com/api/skill/resolve?slug=missing&hash=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
       ),
@@ -114,25 +175,13 @@ describe('httpApi handlers', () => {
 
   it('resolveSkillVersionHttp returns match and latestVersion', async () => {
     const matchHash = await hashSkillFiles([{ path: 'SKILL.md', sha256: 'abc' }])
-    const runQuery = vi
-      .fn()
-      .mockResolvedValueOnce({
-        skill: {
-          _id: 's',
-          slug: 'demo',
-          displayName: 'Demo',
-          tags: {},
-          stats: {},
-          createdAt: 1,
-          updatedAt: 2,
-        },
-        latestVersion: { version: '2.0.0', createdAt: 3, changelog: 'c' },
-        owner: null,
-      })
-      .mockResolvedValueOnce([{ version: '1.0.0', files: [{ path: 'SKILL.md', sha256: 'abc' }] }])
+    const runQuery = vi.fn().mockResolvedValueOnce({
+      match: { version: '1.0.0' },
+      latestVersion: { version: '2.0.0' },
+    })
 
     const response = await __handlers.resolveSkillVersionHandler(
-      { runQuery },
+      makeCtx({ runQuery }),
       new Request(`https://example.com/api/skill/resolve?slug=demo&hash=${matchHash}`),
     )
     expect(response.status).toBe(200)
@@ -144,7 +193,7 @@ describe('httpApi handlers', () => {
   it('cliWhoamiHttp returns 401 on auth failure', async () => {
     vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
     const response = await __handlers.cliWhoamiHandler(
-      {} as unknown,
+      makeCtx({}),
       new Request('https://x/api/cli/whoami'),
     )
     expect(response.status).toBe(401)
@@ -155,7 +204,7 @@ describe('httpApi handlers', () => {
       user: { handle: 'p', displayName: 'Peter', image: 'x' },
     } as never)
     const response = await __handlers.cliWhoamiHandler(
-      {} as unknown,
+      makeCtx({}),
       new Request('https://x/api/cli/whoami'),
     )
     expect(response.status).toBe(200)
@@ -163,11 +212,94 @@ describe('httpApi handlers', () => {
     expect(json.user.handle).toBe('p')
   })
 
+  it('cliTelemetrySyncHttp forwards roots and returns ok', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'users:1' } as never)
+    const runMutation = vi.fn().mockResolvedValue(null)
+    const response = await __handlers.cliTelemetrySyncHandler(
+      makeCtx({ runMutation }),
+      new Request('https://x/api/cli/telemetry/sync', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({
+          roots: [
+            {
+              rootId: 'abc',
+              label: '~/skills',
+              skills: [{ slug: 'weather', version: null }],
+            },
+          ],
+        }),
+      }),
+    )
+    expect(response.status).toBe(200)
+    expect(await response.json()).toEqual({ ok: true })
+    expect(runMutation).toHaveBeenCalledTimes(1)
+  })
+
+  it('cliTelemetrySyncHttp returns 400 on invalid payload', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'users:1' } as never)
+    const response = await __handlers.cliTelemetrySyncHandler(
+      makeCtx({ runMutation: vi.fn() }),
+      new Request('https://x/api/cli/telemetry/sync', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ roots: 'nope' }),
+      }),
+    )
+    expect(response.status).toBe(400)
+  })
+
+  it('cliTelemetrySyncHttp forwards skill versions when provided', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'users:1' } as never)
+    const runMutation = vi.fn().mockResolvedValue(null)
+    await __handlers.cliTelemetrySyncHandler(
+      makeCtx({ runMutation }),
+      new Request('https://x/api/cli/telemetry/sync', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({
+          roots: [
+            {
+              rootId: 'abc',
+              label: '~/skills',
+              skills: [{ slug: 'weather', version: '1.0.0' }],
+            },
+          ],
+        }),
+      }),
+    )
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      userId: 'users:1',
+      roots: [
+        { rootId: 'abc', label: '~/skills', skills: [{ slug: 'weather', version: '1.0.0' }] },
+      ],
+    })
+  })
+
+  it('cliTelemetrySyncHttp returns 400 on invalid json', async () => {
+    const request = new Request('https://x/api/cli/telemetry/sync', { method: 'POST', body: '{' })
+    const response = await __handlers.cliTelemetrySyncHandler(makeCtx({}), request)
+    expect(response.status).toBe(400)
+  })
+
+  it('cliTelemetrySyncHttp returns 401 when unauthorized', async () => {
+    vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
+    const response = await __handlers.cliTelemetrySyncHandler(
+      makeCtx({}),
+      new Request('https://x/api/cli/telemetry/sync', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ roots: [] }),
+      }),
+    )
+    expect(response.status).toBe(401)
+  })
+
   it('cliUploadUrlHttp returns uploadUrl', async () => {
     vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'user1' } as never)
     const runMutation = vi.fn().mockResolvedValue('https://upload.local')
     const response = await __handlers.cliUploadUrlHandler(
-      { runMutation } as unknown,
+      makeCtx({ runMutation }),
       new Request('https://x/api/cli/upload-url', { method: 'POST' }),
     )
     expect(response.status).toBe(200)
@@ -177,7 +309,7 @@ describe('httpApi handlers', () => {
   it('cliUploadUrlHttp returns 401 when unauthorized', async () => {
     vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
     const response = await __handlers.cliUploadUrlHandler(
-      {} as unknown,
+      makeCtx({}),
       new Request('https://x/api/cli/upload-url', { method: 'POST' }),
     )
     expect(response.status).toBe(401)
@@ -185,7 +317,7 @@ describe('httpApi handlers', () => {
 
   it('cliPublishHttp returns 400 on invalid json', async () => {
     const request = new Request('https://x/api/cli/publish', { method: 'POST', body: '{' })
-    const response = await __handlers.cliPublishHandler({} as unknown, request)
+    const response = await __handlers.cliPublishHandler(makeCtx({}), request)
     expect(response.status).toBe(400)
   })
 
@@ -196,7 +328,7 @@ describe('httpApi handlers', () => {
       headers: { 'Content-Type': 'application/json' },
       body: JSON.stringify({}),
     })
-    const response = await __handlers.cliPublishHandler({} as unknown, request)
+    const response = await __handlers.cliPublishHandler(makeCtx({}), request)
     expect(response.status).toBe(401)
   })
 
@@ -214,7 +346,7 @@ describe('httpApi handlers', () => {
         files: [{ path: 'SKILL.md', size: 1, storageId: 'id', sha256: 'a' }],
       }),
     })
-    const response = await __handlers.cliPublishHandler({} as unknown, request)
+    const response = await __handlers.cliPublishHandler(makeCtx({}), request)
     expect(response.status).toBe(400)
   })
 
@@ -236,7 +368,7 @@ describe('httpApi handlers', () => {
         files: [{ path: 'SKILL.md', size: 1, storageId: 'id', sha256: 'a' }],
       }),
     })
-    const response = await __handlers.cliPublishHandler({} as unknown, request)
+    const response = await __handlers.cliPublishHandler(makeCtx({}), request)
     expect(response.status).toBe(200)
     const json = await response.json()
     expect(json.ok).toBe(true)
@@ -250,7 +382,7 @@ describe('httpApi handlers', () => {
       headers: { 'Content-Type': 'application/json' },
       body: JSON.stringify({ slug: 'demo' }),
     })
-    const response = await __handlers.cliSkillDeleteHandler({} as never, request, true)
+    const response = await __handlers.cliSkillDeleteHandler(makeCtx({}), request, true)
     expect(response.status).toBe(401)
   })
 
@@ -262,7 +394,7 @@ describe('httpApi handlers', () => {
       headers: { 'Content-Type': 'application/json' },
       body: JSON.stringify({ slug: 'demo' }),
     })
-    const response = await __handlers.cliSkillDeleteHandler({ runMutation } as never, request, true)
+    const response = await __handlers.cliSkillDeleteHandler(makeCtx({ runMutation }), request, true)
     expect(response.status).toBe(200)
     expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
       userId: 'user1',
@@ -281,7 +413,7 @@ describe('httpApi handlers', () => {
       body: JSON.stringify({ slug: 'demo' }),
     })
     const response = await __handlers.cliSkillDeleteHandler(
-      { runMutation } as never,
+      makeCtx({ runMutation }),
       request,
       false,
     )
@@ -293,9 +425,53 @@ describe('httpApi handlers', () => {
     })
   })
 
+  it('cliSkillUndeleteHttp calls delete handler with deleted=false', async () => {
+    const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'user1' } as never)
+    const runMutation = vi.fn().mockResolvedValue({ ok: true })
+    const response = await __handlers.cliSkillDeleteHandler(
+      makeCtx({ runMutation }),
+      new Request('https://x/api/cli/skill/undelete', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ slug: 'demo' }),
+      }),
+      false,
+    )
+    expect(response.status).toBe(200)
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      userId: 'user1',
+      slug: 'demo',
+      deleted: false,
+    })
+    warnSpy.mockRestore()
+  })
+
+  it('cliSkillDeleteHttp calls delete handler with deleted=true', async () => {
+    const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: 'user1' } as never)
+    const runMutation = vi.fn().mockResolvedValue({ ok: true })
+    const response = await __handlers.cliSkillDeleteHandler(
+      makeCtx({ runMutation }),
+      new Request('https://x/api/cli/skill/delete', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ slug: 'demo' }),
+      }),
+      true,
+    )
+    expect(response.status).toBe(200)
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      userId: 'user1',
+      slug: 'demo',
+      deleted: true,
+    })
+    warnSpy.mockRestore()
+  })
+
   it('cliSkillDeleteHandler returns 400 on invalid json', async () => {
     const request = new Request('https://x/api/cli/skill/delete', { method: 'POST', body: '{' })
-    const response = await __handlers.cliSkillDeleteHandler({} as never, request, true)
+    const response = await __handlers.cliSkillDeleteHandler(makeCtx({}), request, true)
     expect(response.status).toBe(400)
   })
 
@@ -306,7 +482,7 @@ describe('httpApi handlers', () => {
       headers: { 'Content-Type': 'application/json' },
       body: JSON.stringify({}),
     })
-    const response = await __handlers.cliSkillDeleteHandler({} as never, request, true)
+    const response = await __handlers.cliSkillDeleteHandler(makeCtx({}), request, true)
     expect(response.status).toBe(400)
   })
 })

convex/httpApi.test.ts

@@ -25,6 +25,29 @@ describe('httpApi', () => {
     expect(parsed.files[0]?.path).toBe('SKILL.md')
   })
 
+  it('normalizes optional fields in publish payload', () => {
+    const parsed = __test.parsePublishBody({
+      slug: 'cool-skill',
+      displayName: 'Cool Skill',
+      version: '1.2.3',
+      changelog: '',
+      tags: [],
+      forkOf: { slug: 'base-skill' },
+      files: [
+        {
+          path: 'SKILL.md',
+          size: 5,
+          storageId: 'fakeStorageId',
+          sha256: 'abcd',
+          contentType: 'text/markdown',
+        },
+      ],
+    })
+    expect(parsed.tags).toBeUndefined()
+    expect(parsed.source).toBeUndefined()
+    expect(parsed.forkOf).toEqual({ slug: 'base-skill', version: undefined })
+  })
+
   it('rejects invalid publish payloads', () => {
     expect(() => __test.parsePublishBody(null)).toThrow(/Publish payload/i)
     expect(() =>

convex/httpApi.ts

@@ -1,22 +1,18 @@
 import {
   ApiCliSkillDeleteResponseSchema,
+  ApiCliTelemetrySyncResponseSchema,
   CliPublishRequestSchema,
   CliSkillDeleteRequestSchema,
+  CliTelemetrySyncRequestSchema,
   parseArk,
-} from 'clawdhub-schema'
+} from 'clawhub-schema'
 import { api, internal } from './_generated/api'
 import type { Id } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
 import { httpAction } from './_generated/server'
 import { requireApiTokenUser } from './lib/apiTokenAuth'
-import { hashSkillFiles } from './lib/skills'
 import { publishVersionForUser } from './skills'
 
-type HttpCtx = {
-  runAction: (fn: unknown, args: unknown) => Promise<unknown>
-  runQuery: (fn: unknown, args: unknown) => Promise<unknown>
-  runMutation: (fn: unknown, args: unknown) => Promise<unknown>
-}
-
 type SearchSkillEntry = {
   score: number
   skill: {
@@ -43,18 +39,19 @@ type GetBySlugResult = {
   owner: { handle?: string; displayName?: string; image?: string } | null
 } | null
 
-async function searchSkillsHandler(ctx: HttpCtx, request: Request) {
+async function searchSkillsHandler(ctx: ActionCtx, request: Request) {
   const url = new URL(request.url)
   const query = url.searchParams.get('q')?.trim() ?? ''
   const limit = toOptionalNumber(url.searchParams.get('limit'))
   const approvedOnly = url.searchParams.get('approvedOnly') === 'true'
+  const highlightedOnly = url.searchParams.get('highlightedOnly') === 'true' || approvedOnly
 
   if (!query) return json({ results: [] })
 
   const results = (await ctx.runAction(api.search.searchSkills, {
     query,
     limit,
-    approvedOnly: approvedOnly || undefined,
+    highlightedOnly: highlightedOnly || undefined,
   })) as SearchSkillEntry[]
 
   return json({
@@ -71,7 +68,7 @@ async function searchSkillsHandler(ctx: HttpCtx, request: Request) {
 
 export const searchSkillsHttp = httpAction(searchSkillsHandler)
 
-async function getSkillHandler(ctx: HttpCtx, request: Request) {
+async function getSkillHandler(ctx: ActionCtx, request: Request) {
   const url = new URL(request.url)
   const slug = url.searchParams.get('slug')?.trim().toLowerCase()
   if (!slug) return text('Missing slug', 400)
@@ -108,39 +105,22 @@ async function getSkillHandler(ctx: HttpCtx, request: Request) {
 
 export const getSkillHttp = httpAction(getSkillHandler)
 
-async function resolveSkillVersionHandler(ctx: HttpCtx, request: Request) {
+async function resolveSkillVersionHandler(ctx: ActionCtx, request: Request) {
   const url = new URL(request.url)
   const slug = url.searchParams.get('slug')?.trim().toLowerCase()
   const hash = url.searchParams.get('hash')?.trim().toLowerCase()
   if (!slug || !hash) return text('Missing slug or hash', 400)
   if (!/^[a-f0-9]{64}$/.test(hash)) return text('Invalid hash', 400)
 
-  const result = (await ctx.runQuery(api.skills.getBySlug, { slug })) as GetBySlugResult
-  if (!result?.skill) return text('Skill not found', 404)
+  const resolved = await ctx.runQuery(api.skills.resolveVersionByHash, { slug, hash })
+  if (!resolved) return text('Skill not found', 404)
 
-  const versions = (await ctx.runQuery(api.skills.listVersions, {
-    skillId: result.skill._id,
-    limit: 200,
-  })) as Array<{ version: string; files: Array<{ path: string; sha256: string }> }>
-  let match: { version: string } | null = null
-  for (const version of versions) {
-    const fingerprint = await hashSkillFiles(version.files)
-    if (fingerprint === hash) {
-      match = { version: version.version }
-      break
-    }
-  }
-
-  return json({
-    slug,
-    match,
-    latestVersion: result.latestVersion ? { version: result.latestVersion.version } : null,
-  })
+  return json({ slug, match: resolved.match, latestVersion: resolved.latestVersion })
 }
 
 export const resolveSkillVersionHttp = httpAction(resolveSkillVersionHandler)
 
-async function cliWhoamiHandler(ctx: HttpCtx, request: Request) {
+async function cliWhoamiHandler(ctx: ActionCtx, request: Request) {
   try {
     const { user } = await requireApiTokenUser(ctx, request)
     return json({
@@ -157,7 +137,7 @@ async function cliWhoamiHandler(ctx: HttpCtx, request: Request) {
 
 export const cliWhoamiHttp = httpAction(cliWhoamiHandler)
 
-async function cliUploadUrlHandler(ctx: HttpCtx, request: Request) {
+async function cliUploadUrlHandler(ctx: ActionCtx, request: Request) {
   try {
     const { userId } = await requireApiTokenUser(ctx, request)
     const uploadUrl = await ctx.runMutation(internal.uploads.generateUploadUrlForUserInternal, {
@@ -171,7 +151,7 @@ async function cliUploadUrlHandler(ctx: HttpCtx, request: Request) {
 
 export const cliUploadUrlHttp = httpAction(cliUploadUrlHandler)
 
-async function cliPublishHandler(ctx: HttpCtx, request: Request) {
+async function cliPublishHandler(ctx: ActionCtx, request: Request) {
   let body: unknown
   try {
     body = await request.json()
@@ -193,7 +173,7 @@ async function cliPublishHandler(ctx: HttpCtx, request: Request) {
 
 export const cliPublishHttp = httpAction(cliPublishHandler)
 
-async function cliSkillDeleteHandler(ctx: HttpCtx, request: Request, deleted: boolean) {
+async function cliSkillDeleteHandler(ctx: ActionCtx, request: Request, deleted: boolean) {
   let body: unknown
   try {
     body = await request.json()
@@ -225,6 +205,39 @@ export const cliSkillUndeleteHttp = httpAction((ctx, request) =>
   cliSkillDeleteHandler(ctx, request, false),
 )
 
+async function cliTelemetrySyncHandler(ctx: ActionCtx, request: Request) {
+  let body: unknown
+  try {
+    body = await request.json()
+  } catch {
+    return text('Invalid JSON', 400)
+  }
+
+  try {
+    const { userId } = await requireApiTokenUser(ctx, request)
+    const args = parseArk(CliTelemetrySyncRequestSchema, body, 'Telemetry payload')
+    await ctx.runMutation(internal.telemetry.reportCliSyncInternal, {
+      userId,
+      roots: args.roots.map((root) => ({
+        rootId: root.rootId,
+        label: root.label,
+        skills: root.skills.map((skill) => ({
+          slug: skill.slug,
+          version: skill.version ?? undefined,
+        })),
+      })),
+    })
+    const ok = parseArk(ApiCliTelemetrySyncResponseSchema, { ok: true }, 'Telemetry response')
+    return json(ok)
+  } catch (error) {
+    const message = error instanceof Error ? error.message : 'Telemetry failed'
+    if (message.toLowerCase().includes('unauthorized')) return text('Unauthorized', 401)
+    return text(message, 400)
+  }
+}
+
+export const cliTelemetrySyncHttp = httpAction(cliTelemetrySyncHandler)
+
 function json(value: unknown, status = 200) {
   return new Response(JSON.stringify(value), {
     status,
@@ -261,6 +274,13 @@ function parsePublishBody(body: unknown) {
     version: parsed.version,
     changelog: parsed.changelog,
     tags,
+    source: parsed.source ?? undefined,
+    forkOf: parsed.forkOf
+      ? {
+          slug: parsed.forkOf.slug,
+          version: parsed.forkOf.version ?? undefined,
+        }
+      : undefined,
     files: parsed.files.map((file) => ({
       ...file,
       storageId: file.storageId as Id<'_storage'>,
@@ -281,4 +301,5 @@ export const __handlers = {
   cliUploadUrlHandler,
   cliPublishHandler,
   cliSkillDeleteHandler,
+  cliTelemetrySyncHandler,
 }

convex/httpApiV1.handlers.test.ts

@@ -0,0 +1,711 @@
+/* @vitest-environment node */
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+
+vi.mock('./lib/apiTokenAuth', () => ({
+  requireApiTokenUser: vi.fn(),
+}))
+
+vi.mock('./skills', () => ({
+  publishVersionForUser: vi.fn(),
+}))
+
+const { requireApiTokenUser } = await import('./lib/apiTokenAuth')
+const { publishVersionForUser } = await import('./skills')
+const { __handlers } = await import('./httpApiV1')
+
+type ActionCtx = import('./_generated/server').ActionCtx
+
+type RateLimitArgs = { key: string; limit: number; windowMs: number }
+
+function isRateLimitArgs(args: unknown): args is RateLimitArgs {
+  if (!args || typeof args !== 'object') return false
+  const value = args as Record<string, unknown>
+  return (
+    typeof value.key === 'string' &&
+    typeof value.limit === 'number' &&
+    typeof value.windowMs === 'number'
+  )
+}
+
+function makeCtx(partial: Record<string, unknown>) {
+  const partialRunQuery =
+    typeof partial.runQuery === 'function'
+      ? (partial.runQuery as (query: unknown, args: Record<string, unknown>) => unknown)
+      : null
+  const runQuery = vi.fn(async (query: unknown, args: Record<string, unknown>) => {
+    if (isRateLimitArgs(args)) return okRate()
+    return partialRunQuery ? await partialRunQuery(query, args) : null
+  })
+  const runMutation =
+    typeof partial.runMutation === 'function'
+      ? partial.runMutation
+      : vi.fn().mockResolvedValue(okRate())
+
+  return { ...partial, runQuery, runMutation } as unknown as ActionCtx
+}
+
+const okRate = () => ({
+  allowed: true,
+  remaining: 10,
+  limit: 100,
+  resetAt: Date.now() + 60_000,
+})
+
+const blockedRate = () => ({
+  allowed: false,
+  remaining: 0,
+  limit: 100,
+  resetAt: Date.now() + 60_000,
+})
+
+beforeEach(() => {
+  vi.mocked(requireApiTokenUser).mockReset()
+  vi.mocked(publishVersionForUser).mockReset()
+})
+
+describe('httpApiV1 handlers', () => {
+  it('search returns empty results for blank query', async () => {
+    const runAction = vi.fn()
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.searchSkillsV1Handler(
+      makeCtx({ runAction, runMutation }),
+      new Request('https://example.com/api/v1/search?q=%20%20'),
+    )
+    if (response.status !== 200) {
+      throw new Error(await response.text())
+    }
+    expect(await response.json()).toEqual({ results: [] })
+    expect(runAction).not.toHaveBeenCalled()
+  })
+
+  it('search forwards limit and highlightedOnly', async () => {
+    const runAction = vi.fn().mockResolvedValue([
+      {
+        score: 1,
+        skill: { slug: 'a', displayName: 'A', summary: null, updatedAt: 1 },
+        version: { version: '1.0.0' },
+      },
+    ])
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.searchSkillsV1Handler(
+      makeCtx({ runAction, runMutation }),
+      new Request('https://example.com/api/v1/search?q=test&limit=5&highlightedOnly=true'),
+    )
+    if (response.status !== 200) {
+      throw new Error(await response.text())
+    }
+    expect(runAction).toHaveBeenCalledWith(expect.anything(), {
+      query: 'test',
+      limit: 5,
+      highlightedOnly: true,
+    })
+  })
+
+  it('search rate limits', async () => {
+    const runMutation = vi.fn().mockResolvedValue(blockedRate())
+    const response = await __handlers.searchSkillsV1Handler(
+      makeCtx({ runAction: vi.fn(), runMutation }),
+      new Request('https://example.com/api/v1/search?q=test'),
+    )
+    expect(response.status).toBe(429)
+  })
+
+  it('resolve validates hash', async () => {
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.resolveSkillVersionV1Handler(
+      makeCtx({ runQuery: vi.fn(), runMutation }),
+      new Request('https://example.com/api/v1/resolve?slug=demo&hash=bad'),
+    )
+    expect(response.status).toBe(400)
+  })
+
+  it('resolve returns 404 when missing', async () => {
+    const runQuery = vi.fn().mockResolvedValue(null)
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.resolveSkillVersionV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request(
+        'https://example.com/api/v1/resolve?slug=demo&hash=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
+      ),
+    )
+    expect(response.status).toBe(404)
+  })
+
+  it('resolve returns match and latestVersion', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      match: { version: '1.0.0' },
+      latestVersion: { version: '2.0.0' },
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.resolveSkillVersionV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request(
+        'https://example.com/api/v1/resolve?slug=demo&hash=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
+      ),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.match.version).toBe('1.0.0')
+  })
+
+  it('lists skills with resolved tags', async () => {
+    const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
+      if ('cursor' in args || 'limit' in args) {
+        return {
+          items: [
+            {
+              skill: {
+                _id: 'skills:1',
+                slug: 'demo',
+                displayName: 'Demo',
+                summary: 's',
+                tags: { latest: 'versions:1' },
+                stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
+                createdAt: 1,
+                updatedAt: 2,
+              },
+              latestVersion: { version: '1.0.0', createdAt: 3, changelog: 'c' },
+            },
+          ],
+          nextCursor: null,
+        }
+      }
+      if ('versionId' in args) return { version: '1.0.0' }
+      return null
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.listSkillsV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/skills?limit=1'),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.items[0].tags.latest).toBe('1.0.0')
+  })
+
+  it('lists skills supports sort aliases', async () => {
+    const checks: Array<[string, string]> = [
+      ['rating', 'stars'],
+      ['installs', 'installsCurrent'],
+      ['installs-all-time', 'installsAllTime'],
+      ['trending', 'trending'],
+    ]
+
+    for (const [input, expected] of checks) {
+      const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
+        if ('sort' in args || 'cursor' in args || 'limit' in args) {
+          expect(args.sort).toBe(expected)
+          return { items: [], nextCursor: null }
+        }
+        return null
+      })
+      const runMutation = vi.fn().mockResolvedValue(okRate())
+      const response = await __handlers.listSkillsV1Handler(
+        makeCtx({ runQuery, runMutation }),
+        new Request(`https://example.com/api/v1/skills?sort=${input}`),
+      )
+      expect(response.status).toBe(200)
+    }
+  })
+
+  it('get skill returns 404 when missing', async () => {
+    const runQuery = vi.fn().mockResolvedValue(null)
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.skillsGetRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/skills/missing'),
+    )
+    expect(response.status).toBe(404)
+  })
+
+  it('get skill returns payload', async () => {
+    const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
+      if ('slug' in args) {
+        return {
+          skill: {
+            _id: 'skills:1',
+            slug: 'demo',
+            displayName: 'Demo',
+            summary: 's',
+            tags: { latest: 'versions:1' },
+            stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
+            createdAt: 1,
+            updatedAt: 2,
+          },
+          latestVersion: {
+            version: '1.0.0',
+            createdAt: 3,
+            changelog: 'c',
+            files: [],
+          },
+          owner: { handle: 'p', displayName: 'Peter', image: null },
+        }
+      }
+      if ('versionId' in args) return { version: '1.0.0' }
+      return null
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.skillsGetRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/skills/demo'),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.skill.slug).toBe('demo')
+    expect(json.latestVersion.version).toBe('1.0.0')
+  })
+
+  it('lists versions', async () => {
+    const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
+      if ('slug' in args) {
+        return { _id: 'skills:1', slug: 'demo', displayName: 'Demo' }
+      }
+      if ('skillId' in args && 'cursor' in args) {
+        return {
+          items: [
+            {
+              version: '1.0.0',
+              createdAt: 1,
+              changelog: 'c',
+              changelogSource: 'user',
+              files: [],
+            },
+          ],
+          nextCursor: null,
+        }
+      }
+      return null
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.skillsGetRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/skills/demo/versions?limit=1'),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.items[0].version).toBe('1.0.0')
+  })
+
+  it('returns version detail', async () => {
+    const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
+      if ('slug' in args) {
+        return { _id: 'skills:1', slug: 'demo', displayName: 'Demo' }
+      }
+      if ('skillId' in args && 'version' in args) {
+        return {
+          version: '1.0.0',
+          createdAt: 1,
+          changelog: 'c',
+          changelogSource: 'auto',
+          files: [
+            {
+              path: 'SKILL.md',
+              size: 1,
+              storageId: 'storage:1',
+              sha256: 'abc',
+              contentType: 'text/plain',
+            },
+          ],
+        }
+      }
+      return null
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.skillsGetRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/skills/demo/versions/1.0.0'),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.version.files[0].path).toBe('SKILL.md')
+  })
+
+  it('returns raw file content', async () => {
+    const version = {
+      version: '1.0.0',
+      createdAt: 1,
+      changelog: 'c',
+      files: [
+        {
+          path: 'SKILL.md',
+          size: 5,
+          storageId: 'storage:1',
+          sha256: 'abcd',
+          contentType: 'text/plain',
+        },
+      ],
+      softDeletedAt: undefined,
+    }
+    const runQuery = vi.fn().mockResolvedValue({
+      skill: {
+        _id: 'skills:1',
+        slug: 'demo',
+        displayName: 'Demo',
+        summary: 's',
+        tags: {},
+        stats: {},
+        createdAt: 1,
+        updatedAt: 2,
+      },
+      latestVersion: version,
+      owner: null,
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const storage = {
+      get: vi.fn().mockResolvedValue(new Blob(['hello'], { type: 'text/plain' })),
+    }
+    const response = await __handlers.skillsGetRouterV1Handler(
+      makeCtx({ runQuery, runMutation, storage }),
+      new Request('https://example.com/api/v1/skills/demo/file?path=SKILL.md'),
+    )
+    expect(response.status).toBe(200)
+    expect(await response.text()).toBe('hello')
+    expect(response.headers.get('X-Content-SHA256')).toBe('abcd')
+  })
+
+  it('returns 413 when raw file too large', async () => {
+    const version = {
+      version: '1.0.0',
+      createdAt: 1,
+      changelog: 'c',
+      files: [
+        {
+          path: 'SKILL.md',
+          size: 210 * 1024,
+          storageId: 'storage:1',
+          sha256: 'abcd',
+          contentType: 'text/plain',
+        },
+      ],
+      softDeletedAt: undefined,
+    }
+    const runQuery = vi.fn().mockResolvedValue({
+      skill: {
+        _id: 'skills:1',
+        slug: 'demo',
+        displayName: 'Demo',
+        summary: 's',
+        tags: {},
+        stats: {},
+        createdAt: 1,
+        updatedAt: 2,
+      },
+      latestVersion: version,
+      owner: null,
+    })
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.skillsGetRouterV1Handler(
+      makeCtx({ runQuery, runMutation, storage: { get: vi.fn() } }),
+      new Request('https://example.com/api/v1/skills/demo/file?path=SKILL.md'),
+    )
+    expect(response.status).toBe(413)
+  })
+
+  it('publish json succeeds', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    vi.mocked(publishVersionForUser).mockResolvedValueOnce({
+      skillId: 's',
+      versionId: 'v',
+      embeddingId: 'e',
+    } as never)
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const body = JSON.stringify({
+      slug: 'demo',
+      displayName: 'Demo',
+      version: '1.0.0',
+      changelog: 'c',
+      files: [
+        {
+          path: 'SKILL.md',
+          size: 1,
+          storageId: 'storage:1',
+          sha256: 'abc',
+          contentType: 'text/plain',
+        },
+      ],
+    })
+    const response = await __handlers.publishSkillV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/skills', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json', Authorization: 'Bearer clh_test' },
+        body,
+      }),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.ok).toBe(true)
+    expect(publishVersionForUser).toHaveBeenCalled()
+  })
+
+  it('publish multipart succeeds', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    vi.mocked(publishVersionForUser).mockResolvedValueOnce({
+      skillId: 's',
+      versionId: 'v',
+      embeddingId: 'e',
+    } as never)
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const form = new FormData()
+    form.set(
+      'payload',
+      JSON.stringify({
+        slug: 'demo',
+        displayName: 'Demo',
+        version: '1.0.0',
+        changelog: '',
+        tags: ['latest'],
+      }),
+    )
+    form.append('files', new Blob(['hello'], { type: 'text/plain' }), 'SKILL.md')
+    const response = await __handlers.publishSkillV1Handler(
+      makeCtx({ runMutation, storage: { store: vi.fn().mockResolvedValue('storage:1') } }),
+      new Request('https://example.com/api/v1/skills', {
+        method: 'POST',
+        headers: { Authorization: 'Bearer clh_test' },
+        body: form,
+      }),
+    )
+    if (response.status !== 200) {
+      throw new Error(await response.text())
+    }
+  })
+
+  it('publish rejects missing token', async () => {
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.publishSkillV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/skills', { method: 'POST' }),
+    )
+    expect(response.status).toBe(401)
+  })
+
+  it('whoami returns user payload', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValueOnce({
+      userId: 'users:1',
+      user: { handle: 'p', displayName: 'Peter', image: null },
+    } as never)
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.whoamiV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/whoami', {
+        headers: { Authorization: 'Bearer clh_test' },
+      }),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.user.handle).toBe('p')
+  })
+
+  it('delete and undelete require auth', async () => {
+    vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.skillsDeleteRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/skills/demo', { method: 'DELETE' }),
+    )
+    expect(response.status).toBe(401)
+
+    vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
+    const response2 = await __handlers.skillsPostRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/skills/demo/undelete', { method: 'POST' }),
+    )
+    expect(response2.status).toBe(401)
+  })
+
+  it('delete and undelete succeed', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValue({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    const runMutation = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
+      if ('key' in args) return okRate()
+      return { ok: true }
+    })
+
+    const response = await __handlers.skillsDeleteRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/skills/demo', {
+        method: 'DELETE',
+        headers: { Authorization: 'Bearer clh_test' },
+      }),
+    )
+    expect(response.status).toBe(200)
+
+    const response2 = await __handlers.skillsPostRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/skills/demo/undelete', {
+        method: 'POST',
+        headers: { Authorization: 'Bearer clh_test' },
+      }),
+    )
+    expect(response2.status).toBe(200)
+  })
+
+  it('ban user requires auth', async () => {
+    vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.usersPostRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/users/ban', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify({ handle: 'demo' }),
+      }),
+    )
+    expect(response.status).toBe(401)
+  })
+
+  it('ban user succeeds with handle', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValue({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    const runQuery = vi.fn().mockResolvedValue({ _id: 'users:2' })
+    const runMutation = vi
+      .fn()
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce({ ok: true, alreadyBanned: false, deletedSkills: 2 })
+    const response = await __handlers.usersPostRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/users/ban', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify({ handle: 'demo' }),
+      }),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.deletedSkills).toBe(2)
+  })
+
+  it('ban user forwards reason', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValue({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    const runQuery = vi.fn().mockResolvedValue({ _id: 'users:2' })
+    const runMutation = vi
+      .fn()
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce({ ok: true, alreadyBanned: false, deletedSkills: 0 })
+    await __handlers.usersPostRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/users/ban', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify({ handle: 'demo', reason: 'malware' }),
+      }),
+    )
+    expect(runMutation).toHaveBeenCalledWith(
+      expect.anything(),
+      expect.objectContaining({
+        actorUserId: 'users:1',
+        targetUserId: 'users:2',
+        reason: 'malware',
+      }),
+    )
+  })
+
+  it('set role requires auth', async () => {
+    vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.usersPostRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/users/role', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify({ handle: 'demo', role: 'moderator' }),
+      }),
+    )
+    expect(response.status).toBe(401)
+  })
+
+  it('set role succeeds with handle', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValue({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    const runQuery = vi.fn().mockResolvedValue({ _id: 'users:2' })
+    const runMutation = vi
+      .fn()
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce({ ok: true, role: 'moderator' })
+    const response = await __handlers.usersPostRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/users/role', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify({ handle: 'demo', role: 'moderator' }),
+      }),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.role).toBe('moderator')
+  })
+
+  it('stars require auth', async () => {
+    vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
+    const runMutation = vi.fn().mockResolvedValue(okRate())
+    const response = await __handlers.starsPostRouterV1Handler(
+      makeCtx({ runMutation }),
+      new Request('https://example.com/api/v1/stars/demo', { method: 'POST' }),
+    )
+    expect(response.status).toBe(401)
+  })
+
+  it('stars add succeeds', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValue({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    const runQuery = vi.fn().mockResolvedValue({ _id: 'skills:1' })
+    const runMutation = vi
+      .fn()
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce({ ok: true, starred: true, alreadyStarred: false })
+    const response = await __handlers.starsPostRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/stars/demo', {
+        method: 'POST',
+        headers: { Authorization: 'Bearer clh_test' },
+      }),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.ok).toBe(true)
+    expect(json.starred).toBe(true)
+  })
+
+  it('stars delete succeeds', async () => {
+    vi.mocked(requireApiTokenUser).mockResolvedValue({
+      userId: 'users:1',
+      user: { handle: 'p' },
+    } as never)
+    const runQuery = vi.fn().mockResolvedValue({ _id: 'skills:1' })
+    const runMutation = vi
+      .fn()
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce(okRate())
+      .mockResolvedValueOnce({ ok: true, unstarred: true, alreadyUnstarred: false })
+    const response = await __handlers.starsDeleteRouterV1Handler(
+      makeCtx({ runQuery, runMutation }),
+      new Request('https://example.com/api/v1/stars/demo', {
+        method: 'DELETE',
+        headers: { Authorization: 'Bearer clh_test' },
+      }),
+    )
+    expect(response.status).toBe(200)
+    const json = await response.json()
+    expect(json.ok).toBe(true)
+    expect(json.unstarred).toBe(true)
+  })
+})

convex/leaderboards.ts

@@ -0,0 +1,39 @@
+import { v } from 'convex/values'
+import { internalMutation } from './_generated/server'
+import { buildTrendingLeaderboard } from './lib/leaderboards'
+
+const MAX_TRENDING_LIMIT = 200
+const KEEP_LEADERBOARD_ENTRIES = 3
+
+export const rebuildTrendingLeaderboardInternal = internalMutation({
+  args: { limit: v.optional(v.number()) },
+  handler: async (ctx, args) => {
+    const limit = clampInt(args.limit ?? MAX_TRENDING_LIMIT, 1, MAX_TRENDING_LIMIT)
+    const now = Date.now()
+    const { startDay, endDay, items } = await buildTrendingLeaderboard(ctx, { limit, now })
+
+    await ctx.db.insert('skillLeaderboards', {
+      kind: 'trending',
+      generatedAt: now,
+      rangeStartDay: startDay,
+      rangeEndDay: endDay,
+      items,
+    })
+
+    const recent = await ctx.db
+      .query('skillLeaderboards')
+      .withIndex('by_kind', (q) => q.eq('kind', 'trending'))
+      .order('desc')
+      .take(KEEP_LEADERBOARD_ENTRIES + 5)
+
+    for (const entry of recent.slice(KEEP_LEADERBOARD_ENTRIES)) {
+      await ctx.db.delete(entry._id)
+    }
+
+    return { ok: true as const, count: items.length }
+  },
+})
+
+function clampInt(value: number, min: number, max: number) {
+  return Math.min(Math.max(value, min), max)
+}

convex/lib/access.ts

@@ -1,5 +1,5 @@
 import { getAuthUserId } from '@convex-dev/auth/server'
-import { api } from '../_generated/api'
+import { internal } from '../_generated/api'
 import type { Doc } from '../_generated/dataModel'
 import type { ActionCtx, MutationCtx, QueryCtx } from '../_generated/server'
 
@@ -16,7 +16,7 @@ export async function requireUser(ctx: MutationCtx | QueryCtx) {
 export async function requireUserFromAction(ctx: ActionCtx) {
   const userId = await getAuthUserId(ctx)
   if (!userId) throw new Error('Unauthorized')
-  const user = await ctx.runQuery(api.users.getById, { userId })
+  const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
   if (!user || user.deletedAt) throw new Error('User not found')
   return { userId, user: user as Doc<'users'> }
 }
@@ -26,3 +26,11 @@ export function assertRole(user: Doc<'users'>, allowed: Role[]) {
     throw new Error('Forbidden')
   }
 }
+
+export function assertAdmin(user: Doc<'users'>) {
+  assertRole(user, ['admin'])
+}
+
+export function assertModerator(user: Doc<'users'>) {
+  assertRole(user, ['admin', 'moderator'])
+}

convex/lib/badges.ts

@@ -0,0 +1,50 @@
+import type { Doc, Id } from '../_generated/dataModel'
+import type { QueryCtx } from '../_generated/server'
+
+type BadgeKind = Doc<'skillBadges'>['kind']
+
+export type SkillBadgeMap = Partial<Record<BadgeKind, { byUserId: Id<'users'>; at: number }>>
+
+export type SkillBadgeSource = { badges?: SkillBadgeMap | null }
+
+type BadgeCtx = Pick<QueryCtx, 'db'>
+
+export function isSkillHighlighted(skill: SkillBadgeSource) {
+  return Boolean(skill.badges?.highlighted)
+}
+
+export function isSkillOfficial(skill: SkillBadgeSource) {
+  return Boolean(skill.badges?.official)
+}
+
+export function isSkillDeprecated(skill: SkillBadgeSource) {
+  return Boolean(skill.badges?.deprecated)
+}
+
+export function buildBadgeMap(records: Doc<'skillBadges'>[]): SkillBadgeMap {
+  return records.reduce<SkillBadgeMap>((acc, record) => {
+    acc[record.kind] = { byUserId: record.byUserId, at: record.at }
+    return acc
+  }, {})
+}
+
+export async function getSkillBadgeMap(
+  ctx: BadgeCtx,
+  skillId: Id<'skills'>,
+): Promise<SkillBadgeMap> {
+  const records = await ctx.db
+    .query('skillBadges')
+    .withIndex('by_skill', (q) => q.eq('skillId', skillId))
+    .collect()
+  return buildBadgeMap(records)
+}
+
+export async function getSkillBadgeMaps(
+  ctx: BadgeCtx,
+  skillIds: Array<Id<'skills'>>,
+): Promise<Map<Id<'skills'>, SkillBadgeMap>> {
+  const entries = await Promise.all(
+    skillIds.map(async (skillId) => [skillId, await getSkillBadgeMap(ctx, skillId)] as const),
+  )
+  return new Map(entries)
+}

convex/lib/changelog.test.ts

@@ -0,0 +1,34 @@
+import { describe, expect, it } from 'vitest'
+import { __test } from './changelog'
+
+describe('changelog utils', () => {
+  it('summarizes file diffs', () => {
+    const diff = __test.summarizeFileDiff(
+      [
+        { path: 'a.txt', sha256: 'aaa' },
+        { path: 'b.txt', sha256: 'bbb' },
+      ],
+      [
+        { path: 'a.txt', sha256: 'aaa' },
+        { path: 'b.txt', sha256: 'ccc' },
+        { path: 'c.txt', sha256: 'ddd' },
+      ],
+    )
+
+    expect(diff.added).toEqual(['c.txt'])
+    expect(diff.removed).toEqual([])
+    expect(diff.changed).toEqual(['b.txt'])
+    expect(__test.formatDiffSummary(diff)).toBe('1 added, 1 changed')
+  })
+
+  it('generates a fallback initial release note', () => {
+    const text = __test.generateFallback({
+      slug: 'demo',
+      version: '1.0.0',
+      oldReadme: null,
+      nextReadme: 'hi',
+      fileDiff: null,
+    })
+    expect(text).toMatch(/Initial release/i)
+  })
+})

convex/lib/changelog.ts

@@ -0,0 +1,278 @@
+import { internal } from '../_generated/api'
+import type { Doc, Id } from '../_generated/dataModel'
+import type { ActionCtx } from '../_generated/server'
+
+const CHANGELOG_MODEL = process.env.OPENAI_CHANGELOG_MODEL ?? 'gpt-4.1'
+const MAX_README_CHARS = 8_000
+const MAX_PATHS_IN_PROMPT = 30
+
+type FileMeta = { path: string; sha256?: string }
+
+type FileDiffSummary = {
+  added: string[]
+  removed: string[]
+  changed: string[]
+}
+
+function clampText(value: string, maxChars: number) {
+  const trimmed = value.trim()
+  if (trimmed.length <= maxChars) return trimmed
+  return `${trimmed.slice(0, maxChars).trimEnd()}\n…`
+}
+
+function summarizeFileDiff(oldFiles: FileMeta[], nextFiles: FileMeta[]): FileDiffSummary {
+  const oldByPath = new Map(oldFiles.map((f) => [f.path, f] as const))
+  const nextByPath = new Map(nextFiles.map((f) => [f.path, f] as const))
+
+  const added: string[] = []
+  const removed: string[] = []
+  const changed: string[] = []
+
+  for (const [path, file] of nextByPath.entries()) {
+    const prev = oldByPath.get(path)
+    if (!prev) {
+      added.push(path)
+      continue
+    }
+    if (file.sha256 && prev.sha256 && file.sha256 !== prev.sha256) changed.push(path)
+  }
+  for (const path of oldByPath.keys()) {
+    if (!nextByPath.has(path)) removed.push(path)
+  }
+
+  added.sort()
+  removed.sort()
+  changed.sort()
+  return { added, removed, changed }
+}
+
+function formatDiffSummary(diff: FileDiffSummary) {
+  const parts: string[] = []
+  if (diff.added.length) parts.push(`${diff.added.length} added`)
+  if (diff.changed.length) parts.push(`${diff.changed.length} changed`)
+  if (diff.removed.length) parts.push(`${diff.removed.length} removed`)
+  return parts.join(', ') || 'no file changes detected'
+}
+
+function pickPaths(values: string[]) {
+  if (values.length <= MAX_PATHS_IN_PROMPT) return values
+  return values.slice(0, MAX_PATHS_IN_PROMPT)
+}
+
+function extractResponseText(payload: unknown) {
+  if (!payload || typeof payload !== 'object') return null
+  const output = (payload as { output?: unknown }).output
+  if (!Array.isArray(output)) return null
+  const chunks: string[] = []
+  for (const item of output) {
+    if (!item || typeof item !== 'object') continue
+    if ((item as { type?: unknown }).type !== 'message') continue
+    const content = (item as { content?: unknown }).content
+    if (!Array.isArray(content)) continue
+    for (const part of content) {
+      if (!part || typeof part !== 'object') continue
+      if ((part as { type?: unknown }).type !== 'output_text') continue
+      const text = (part as { text?: unknown }).text
+      if (typeof text === 'string' && text.trim()) chunks.push(text)
+    }
+  }
+  const joined = chunks.join('\n').trim()
+  return joined || null
+}
+
+async function generateWithOpenAI(args: {
+  slug: string
+  version: string
+  oldReadme: string | null
+  nextReadme: string
+  fileDiff: FileDiffSummary | null
+}) {
+  const apiKey = process.env.OPENAI_API_KEY
+  if (!apiKey) return null
+
+  const oldReadme = args.oldReadme ? clampText(args.oldReadme, MAX_README_CHARS) : ''
+  const nextReadme = clampText(args.nextReadme, MAX_README_CHARS)
+
+  const fileDiff = args.fileDiff
+  const diffSummary = fileDiff ? formatDiffSummary(fileDiff) : 'unknown'
+  const changedPaths = fileDiff ? pickPaths(fileDiff.changed) : []
+  const addedPaths = fileDiff ? pickPaths(fileDiff.added) : []
+  const removedPaths = fileDiff ? pickPaths(fileDiff.removed) : []
+
+  const input = [
+    `Skill: ${args.slug}`,
+    `Version: ${args.version}`,
+    `File changes: ${diffSummary}`,
+    changedPaths.length ? `Changed files (sample): ${changedPaths.join(', ')}` : null,
+    addedPaths.length ? `Added files (sample): ${addedPaths.join(', ')}` : null,
+    removedPaths.length ? `Removed files (sample): ${removedPaths.join(', ')}` : null,
+    oldReadme ? `Previous SKILL.md:\n${oldReadme}` : null,
+    `New SKILL.md:\n${nextReadme}`,
+  ]
+    .filter(Boolean)
+    .join('\n\n')
+
+  const response = await fetch('https://api.openai.com/v1/responses', {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      Authorization: `Bearer ${apiKey}`,
+    },
+    body: JSON.stringify({
+      model: CHANGELOG_MODEL,
+      instructions:
+        'Write a concise changelog for this skill version. Audience: everyone. Output plain text. Prefer 2–6 bullet points. If it is a big change, include a short 1-line summary first, then bullets. Don’t mention that you are AI. Don’t invent details; only use the inputs.',
+      input,
+      max_output_tokens: 220,
+    }),
+  })
+
+  if (!response.ok) return null
+  const payload = (await response.json()) as unknown
+  return extractResponseText(payload)
+}
+
+function generateFallback(args: {
+  slug: string
+  version: string
+  oldReadme: string | null
+  nextReadme: string
+  fileDiff: FileDiffSummary | null
+}) {
+  const lines: string[] = []
+  if (!args.oldReadme) {
+    lines.push(`- Initial release.`)
+    return lines.join('\n')
+  }
+
+  const diff = args.fileDiff
+  if (diff) {
+    const parts: string[] = []
+    if (diff.added.length) parts.push(`added ${diff.added.length}`)
+    if (diff.changed.length) parts.push(`updated ${diff.changed.length}`)
+    if (diff.removed.length) parts.push(`removed ${diff.removed.length}`)
+    if (parts.length) lines.push(`- ${parts.join(', ')} file(s).`)
+  }
+
+  lines.push(`- Updated SKILL.md and bundle contents.`)
+  return lines.join('\n')
+}
+
+export async function generateChangelogForPublish(
+  ctx: ActionCtx,
+  args: { slug: string; version: string; readmeText: string; files: FileMeta[] },
+): Promise<string> {
+  try {
+    const skill = (await ctx.runQuery(internal.skills.getSkillBySlugInternal, {
+      slug: args.slug,
+    })) as Doc<'skills'> | null
+    const previous: Doc<'skillVersions'> | null =
+      skill?.latestVersionId && !skill.softDeletedAt
+        ? ((await ctx.runQuery(internal.skills.getVersionByIdInternal, {
+            versionId: skill.latestVersionId,
+          })) as Doc<'skillVersions'> | null)
+        : null
+
+    const oldReadmeText: string | null = previous
+      ? await readReadmeFromVersion(ctx, previous)
+      : null
+    const oldFiles = previous
+      ? previous.files.map((file) => ({ path: file.path, sha256: file.sha256 }))
+      : []
+    const fileDiff = previous ? summarizeFileDiff(oldFiles, args.files) : null
+
+    const ai = await generateWithOpenAI({
+      slug: args.slug,
+      version: args.version,
+      oldReadme: oldReadmeText,
+      nextReadme: args.readmeText,
+      fileDiff,
+    }).catch(() => null)
+
+    return (
+      ai ??
+      generateFallback({
+        slug: args.slug,
+        version: args.version,
+        oldReadme: oldReadmeText,
+        nextReadme: args.readmeText,
+        fileDiff,
+      })
+    )
+  } catch {
+    return '- Updated skill.'
+  }
+}
+
+export async function generateChangelogPreview(
+  ctx: ActionCtx,
+  args: {
+    slug: string
+    version: string
+    readmeText: string
+    filePaths?: string[]
+  },
+): Promise<string> {
+  try {
+    const skill = (await ctx.runQuery(internal.skills.getSkillBySlugInternal, {
+      slug: args.slug,
+    })) as Doc<'skills'> | null
+    const previous: Doc<'skillVersions'> | null =
+      skill?.latestVersionId && !skill.softDeletedAt
+        ? ((await ctx.runQuery(internal.skills.getVersionByIdInternal, {
+            versionId: skill.latestVersionId,
+          })) as Doc<'skillVersions'> | null)
+        : null
+
+    const oldReadmeText: string | null = previous
+      ? await readReadmeFromVersion(ctx, previous)
+      : null
+    const fileDiff =
+      previous && args.filePaths
+        ? summarizeFileDiff(
+            previous.files.map((file) => ({ path: file.path, sha256: file.sha256 })),
+            args.filePaths.map((path) => ({ path })),
+          )
+        : null
+
+    const ai = await generateWithOpenAI({
+      slug: args.slug,
+      version: args.version,
+      oldReadme: oldReadmeText,
+      nextReadme: args.readmeText,
+      fileDiff,
+    }).catch(() => null)
+
+    return (
+      ai ??
+      generateFallback({
+        slug: args.slug,
+        version: args.version,
+        oldReadme: oldReadmeText,
+        nextReadme: args.readmeText,
+        fileDiff,
+      })
+    )
+  } catch {
+    return '- Updated skill.'
+  }
+}
+
+async function readReadmeFromVersion(ctx: ActionCtx, version: Doc<'skillVersions'>) {
+  const readmeFile = version.files.find((file) => {
+    const lower = file.path.toLowerCase()
+    return lower === 'skill.md' || lower === 'skills.md'
+  })
+  if (!readmeFile) return null
+  const blob = await ctx.storage.get(readmeFile.storageId as Id<'_storage'>)
+  if (!blob) return null
+  return blob.text()
+}
+
+export const __test = {
+  clampText,
+  extractResponseText,
+  formatDiffSummary,
+  summarizeFileDiff,
+  generateFallback,
+}

convex/lib/embeddings.ts

@@ -1,9 +1,16 @@
 export const EMBEDDING_MODEL = 'text-embedding-3-small'
 export const EMBEDDING_DIMENSIONS = 1536
 
+function emptyEmbedding() {
+  return Array.from({ length: EMBEDDING_DIMENSIONS }, () => 0)
+}
+
 export async function generateEmbedding(text: string) {
   const apiKey = process.env.OPENAI_API_KEY
-  if (!apiKey) throw new Error('OPENAI_API_KEY is not configured')
+  if (!apiKey) {
+    console.warn('OPENAI_API_KEY is not configured; using zero embeddings')
+    return emptyEmbedding()
+  }
 
   const response = await fetch('https://api.openai.com/v1/embeddings', {
     method: 'POST',

convex/lib/githubAccount.test.ts

@@ -0,0 +1,191 @@
+/* @vitest-environment node */
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+
+import { internal } from '../_generated/api'
+import { requireGitHubAccountAge } from './githubAccount'
+
+vi.mock('../_generated/api', () => ({
+  internal: {
+    users: {
+      getByIdInternal: Symbol('getByIdInternal'),
+      updateGithubMetaInternal: Symbol('updateGithubMetaInternal'),
+    },
+  },
+}))
+
+const ONE_DAY_MS = 24 * 60 * 60 * 1000
+
+describe('requireGitHubAccountAge', () => {
+  beforeEach(() => {
+    vi.restoreAllMocks()
+    vi.unstubAllEnvs()
+  })
+
+  afterEach(() => {
+    vi.unstubAllEnvs()
+  })
+
+  it('uses cached githubCreatedAt when fresh', async () => {
+    vi.useFakeTimers()
+    const now = new Date('2026-02-02T12:00:00Z')
+    vi.setSystemTime(now)
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'steipete',
+      githubCreatedAt: now.getTime() - 10 * ONE_DAY_MS,
+      githubFetchedAt: now.getTime() - ONE_DAY_MS + 1000,
+    })
+    const runMutation = vi.fn()
+    const fetchMock = vi.fn()
+    vi.stubGlobal('fetch', fetchMock)
+
+    await requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never)
+
+    expect(fetchMock).not.toHaveBeenCalled()
+    expect(runMutation).not.toHaveBeenCalled()
+    expect(runQuery).toHaveBeenCalledWith(internal.users.getByIdInternal, { userId: 'users:1' })
+
+    vi.useRealTimers()
+  })
+
+  it('rejects accounts younger than 7 days', async () => {
+    vi.useFakeTimers()
+    const now = new Date('2026-02-02T12:00:00Z')
+    vi.setSystemTime(now)
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'newbie',
+      githubCreatedAt: now.getTime() - 2 * ONE_DAY_MS,
+      githubFetchedAt: now.getTime() - ONE_DAY_MS / 2,
+    })
+    const runMutation = vi.fn()
+
+    await expect(
+      requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
+    ).rejects.toThrow(/GitHub account must be at least 7 days old/i)
+
+    vi.useRealTimers()
+  })
+
+  it('refreshes githubCreatedAt when cache is stale', async () => {
+    vi.useFakeTimers()
+    const now = new Date('2026-02-02T12:00:00Z')
+    vi.setSystemTime(now)
+
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'steipete',
+      githubCreatedAt: undefined,
+      githubFetchedAt: now.getTime() - 2 * ONE_DAY_MS,
+    })
+    const runMutation = vi.fn()
+    const fetchMock = vi.fn().mockResolvedValue({
+      ok: true,
+      json: async () => ({
+        created_at: '2020-01-01T00:00:00Z',
+      }),
+    })
+    vi.stubGlobal('fetch', fetchMock)
+
+    await requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never)
+
+    expect(fetchMock).toHaveBeenCalledWith(
+      'https://api.github.com/users/steipete',
+      expect.objectContaining({
+        headers: expect.objectContaining({ 'User-Agent': 'clawhub' }),
+      }),
+    )
+    expect(runMutation).toHaveBeenCalledWith(internal.users.updateGithubMetaInternal, {
+      userId: 'users:1',
+      githubCreatedAt: Date.parse('2020-01-01T00:00:00Z'),
+      githubFetchedAt: now.getTime(),
+    })
+
+    vi.useRealTimers()
+  })
+
+  it('throws when GitHub lookup fails', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'steipete',
+      githubCreatedAt: undefined,
+      githubFetchedAt: 0,
+    })
+    const runMutation = vi.fn()
+    const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 404 })
+    vi.stubGlobal('fetch', fetchMock)
+
+    await expect(
+      requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
+    ).rejects.toThrow(/GitHub account lookup failed/i)
+  })
+
+  it('throws rate-limit error on 403', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'steipete',
+      githubCreatedAt: undefined,
+      githubFetchedAt: 0,
+    })
+    const runMutation = vi.fn()
+    const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 403 })
+    vi.stubGlobal('fetch', fetchMock)
+
+    await expect(
+      requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
+    ).rejects.toThrow(/rate limit exceeded/i)
+  })
+
+  it('throws rate-limit error on 429', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'steipete',
+      githubCreatedAt: undefined,
+      githubFetchedAt: 0,
+    })
+    const runMutation = vi.fn()
+    const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 429 })
+    vi.stubGlobal('fetch', fetchMock)
+
+    await expect(
+      requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
+    ).rejects.toThrow(/rate limit exceeded/i)
+  })
+
+  it('includes Authorization header when GITHUB_TOKEN is set', async () => {
+    vi.useFakeTimers()
+    const now = new Date('2026-02-02T12:00:00Z')
+    vi.setSystemTime(now)
+
+    vi.stubEnv('GITHUB_TOKEN', 'ghp_test123')
+
+    const runQuery = vi.fn().mockResolvedValue({
+      _id: 'users:1',
+      handle: 'steipete',
+      githubCreatedAt: undefined,
+      githubFetchedAt: now.getTime() - 2 * ONE_DAY_MS,
+    })
+    const runMutation = vi.fn()
+    const fetchMock = vi.fn().mockResolvedValue({
+      ok: true,
+      json: async () => ({
+        created_at: '2020-01-01T00:00:00Z',
+      }),
+    })
+    vi.stubGlobal('fetch', fetchMock)
+
+    await requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never)
+
+    expect(fetchMock).toHaveBeenCalledWith(
+      'https://api.github.com/users/steipete',
+      expect.objectContaining({
+        headers: {
+          'User-Agent': 'clawhub',
+          Authorization: 'Bearer ghp_test123',
+        },
+      }),
+    )
+
+    vi.useRealTimers()
+  })
+})

convex/lib/githubAccount.ts

@@ -0,0 +1,67 @@
+import { ConvexError } from 'convex/values'
+import { internal } from '../_generated/api'
+import type { Id } from '../_generated/dataModel'
+import type { ActionCtx } from '../_generated/server'
+
+const GITHUB_API = 'https://api.github.com'
+const MIN_ACCOUNT_AGE_MS = 7 * 24 * 60 * 60 * 1000
+const FETCH_TTL_MS = 24 * 60 * 60 * 1000
+
+type GitHubUser = {
+  created_at?: string
+}
+
+export async function requireGitHubAccountAge(ctx: ActionCtx, userId: Id<'users'>) {
+  const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
+  if (!user || user.deletedAt) throw new ConvexError('User not found')
+
+  const handle = user.handle?.trim()
+  if (!handle) throw new ConvexError('GitHub handle required')
+
+  const now = Date.now()
+  let createdAt = user.githubCreatedAt ?? null
+  const fetchedAt = user.githubFetchedAt ?? 0
+  const stale = !createdAt || now - fetchedAt > FETCH_TTL_MS
+
+  if (stale) {
+    const headers: Record<string, string> = { 'User-Agent': 'clawhub' }
+    const token = process.env.GITHUB_TOKEN
+    if (token) {
+      headers.Authorization = `Bearer ${token}`
+    }
+
+    const response = await fetch(`${GITHUB_API}/users/${encodeURIComponent(handle)}`, {
+      headers,
+    })
+    if (!response.ok) {
+      if (response.status === 403 || response.status === 429) {
+        throw new ConvexError('GitHub API rate limit exceeded — please try again in a few minutes')
+      }
+      throw new ConvexError('GitHub account lookup failed')
+    }
+
+    const payload = (await response.json()) as GitHubUser
+    const parsed = payload.created_at ? Date.parse(payload.created_at) : Number.NaN
+    if (!Number.isFinite(parsed)) throw new ConvexError('GitHub account lookup failed')
+
+    createdAt = parsed
+    await ctx.runMutation(internal.users.updateGithubMetaInternal, {
+      userId,
+      githubCreatedAt: createdAt,
+      githubFetchedAt: now,
+    })
+  }
+
+  if (!createdAt) throw new ConvexError('GitHub account lookup failed')
+
+  const ageMs = now - createdAt
+  if (ageMs < MIN_ACCOUNT_AGE_MS) {
+    const remainingMs = MIN_ACCOUNT_AGE_MS - ageMs
+    const remainingDays = Math.max(1, Math.ceil(remainingMs / (24 * 60 * 60 * 1000)))
+    throw new ConvexError(
+      `GitHub account must be at least 7 days old to upload skills. Try again in ${remainingDays} day${
+        remainingDays === 1 ? '' : 's'
+      }.`,
+    )
+  }
+}

convex/lib/githubBackup.ts

@@ -0,0 +1,443 @@
+'use node'
+
+import { createPrivateKey, createSign } from 'node:crypto'
+import type { Id } from '../_generated/dataModel'
+import type { ActionCtx } from '../_generated/server'
+
+const GITHUB_API = 'https://api.github.com'
+const DEFAULT_REPO = 'clawdbot/skills'
+const DEFAULT_ROOT = 'skills'
+const META_FILENAME = '_meta.json'
+const USER_AGENT = 'clawhub/skills-backup'
+
+type BackupFile = {
+  path: string
+  size: number
+  storageId: Id<'_storage'>
+  sha256: string
+  contentType?: string
+}
+
+type BackupParams = {
+  slug: string
+  version: string
+  displayName: string
+  ownerHandle: string
+  files: BackupFile[]
+  publishedAt: number
+}
+
+type RepoInfo = {
+  default_branch?: string
+}
+
+type GitRef = {
+  object: { sha: string }
+}
+
+type GitCommit = {
+  sha: string
+  tree: { sha: string }
+}
+
+type GitTreeEntry = {
+  path?: string
+  type?: string
+}
+
+type GitTree = {
+  tree?: GitTreeEntry[]
+}
+
+type MetaFile = {
+  owner: string
+  slug: string
+  displayName: string
+  latest: {
+    version: string
+    publishedAt: number
+    commit: string | null
+  }
+  history: Array<{
+    version: string
+    publishedAt: number
+    commit: string
+  }>
+}
+
+export type GitHubBackupContext = {
+  token: string
+  repo: string
+  repoOwner: string
+  repoName: string
+  branch: string
+  root: string
+}
+
+export function isGitHubBackupConfigured() {
+  return Boolean(
+    process.env.GITHUB_APP_ID &&
+      process.env.GITHUB_APP_PRIVATE_KEY &&
+      process.env.GITHUB_APP_INSTALLATION_ID,
+  )
+}
+
+export async function getGitHubBackupContext(): Promise<GitHubBackupContext> {
+  const repo = process.env.GITHUB_SKILLS_REPO ?? DEFAULT_REPO
+  const root = process.env.GITHUB_SKILLS_ROOT ?? DEFAULT_ROOT
+  const [repoOwner, repoName] = parseRepo(repo)
+  const token = await createInstallationToken()
+  const repoInfo = await githubGet<RepoInfo>(token, `/repos/${repoOwner}/${repoName}`)
+  const branch = repoInfo.default_branch ?? 'main'
+
+  return { token, repo, repoOwner, repoName, branch, root }
+}
+
+export async function fetchGitHubSkillMeta(
+  context: GitHubBackupContext,
+  ownerHandle: string,
+  slug: string,
+): Promise<MetaFile | null> {
+  const skillRoot = buildSkillRoot(context.root, ownerHandle, slug)
+  return fetchMetaFile(
+    context.token,
+    context.repoOwner,
+    context.repoName,
+    `${skillRoot}/${META_FILENAME}`,
+    context.branch,
+  )
+}
+
+export async function backupSkillToGitHub(
+  ctx: ActionCtx,
+  params: BackupParams,
+  context?: GitHubBackupContext,
+) {
+  if (!isGitHubBackupConfigured()) return
+
+  const resolved = context ?? (await getGitHubBackupContext())
+  const skillRoot = buildSkillRoot(resolved.root, params.ownerHandle, params.slug)
+  const ref = await githubGet<GitRef>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/ref/heads/${resolved.branch}`,
+  )
+  const baseCommitSha = ref.object.sha
+  const baseCommit = await githubGet<GitCommit>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits/${baseCommitSha}`,
+  )
+  const baseTreeSha = baseCommit.tree.sha
+  const existingTree = await githubGet<GitTree>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees/${baseTreeSha}?recursive=1`,
+  )
+
+  const prefix = `${skillRoot}/`
+  const existingPaths = new Set(
+    (existingTree.tree ?? [])
+      .filter((entry) => entry.type === 'blob' && entry.path?.startsWith(prefix))
+      .map((entry) => entry.path ?? ''),
+  )
+
+  const newPaths = new Set<string>()
+  const treeEntries: Array<{
+    path: string
+    mode: '100644'
+    type: 'blob'
+    sha: string | null
+  }> = []
+
+  for (const file of params.files) {
+    const content = await fetchStorageBase64(ctx, file.storageId)
+    const blobSha = await createBlob(resolved.token, resolved.repoOwner, resolved.repoName, content)
+    const path = `${skillRoot}/${file.path}`
+    newPaths.add(path)
+    treeEntries.push({ path, mode: '100644', type: 'blob', sha: blobSha })
+  }
+
+  const existingMeta = await fetchMetaFile(
+    resolved.token,
+    resolved.repoOwner,
+    resolved.repoName,
+    `${skillRoot}/${META_FILENAME}`,
+    resolved.branch,
+  )
+  const metaPath = `${skillRoot}/${META_FILENAME}`
+  const metaDraft = buildMetaFile(params, existingMeta, resolved.repo, baseCommitSha, null)
+  const metaDraftContent = `${JSON.stringify(metaDraft, null, 2)}\n`
+  const metaDraftSha = await createBlob(
+    resolved.token,
+    resolved.repoOwner,
+    resolved.repoName,
+    toBase64(metaDraftContent),
+  )
+  newPaths.add(metaPath)
+  treeEntries.push({ path: metaPath, mode: '100644', type: 'blob', sha: metaDraftSha })
+
+  for (const path of existingPaths) {
+    if (newPaths.has(path)) continue
+    treeEntries.push({ path, mode: '100644', type: 'blob', sha: null })
+  }
+
+  const newTree = await githubPost<{ sha: string }>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees`,
+    {
+      base_tree: baseTreeSha,
+      tree: treeEntries,
+    },
+  )
+
+  const commit = await githubPost<GitCommit>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits`,
+    {
+      message: `skill: ${params.slug} v${params.version}`,
+      tree: newTree.sha,
+      parents: [baseCommitSha],
+    },
+  )
+
+  const metaFinal = buildMetaFile(params, existingMeta, resolved.repo, baseCommitSha, commit.sha)
+  const metaFinalContent = `${JSON.stringify(metaFinal, null, 2)}\n`
+  const metaFinalSha = await createBlob(
+    resolved.token,
+    resolved.repoOwner,
+    resolved.repoName,
+    toBase64(metaFinalContent),
+  )
+  const metaTree = await githubPost<{ sha: string }>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees`,
+    {
+      base_tree: commit.tree.sha,
+      tree: [{ path: metaPath, mode: '100644', type: 'blob', sha: metaFinalSha }],
+    },
+  )
+  const metaCommit = await githubPost<GitCommit>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits`,
+    {
+      message: `meta: ${params.slug} v${params.version}`,
+      tree: metaTree.sha,
+      parents: [commit.sha],
+    },
+  )
+
+  await githubPatch(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/refs/heads/${resolved.branch}`,
+    {
+      sha: metaCommit.sha,
+    },
+  )
+}
+
+function buildMetaFile(
+  params: BackupParams,
+  existing: MetaFile | null,
+  repo: string,
+  baseCommitSha: string,
+  latestCommitSha: string | null,
+): MetaFile {
+  let history = [...(existing?.history ?? [])]
+  if (existing?.latest?.version) {
+    const previousCommit = existing.latest.commit ?? commitUrl(repo, baseCommitSha)
+    const previous = {
+      version: existing.latest.version,
+      publishedAt: existing.latest.publishedAt,
+      commit: previousCommit,
+    }
+    history = [previous, ...history.filter((entry) => entry.version !== previous.version)]
+  }
+
+  return {
+    owner: normalizeOwner(params.ownerHandle),
+    slug: params.slug,
+    displayName: params.displayName,
+    latest: {
+      version: params.version,
+      publishedAt: params.publishedAt,
+      commit: latestCommitSha ? commitUrl(repo, latestCommitSha) : null,
+    },
+    history: history.slice(0, 200),
+  }
+}
+
+async function fetchMetaFile(
+  token: string,
+  repoOwner: string,
+  repoName: string,
+  path: string,
+  branch: string,
+): Promise<MetaFile | null> {
+  try {
+    const response = await githubGet<{ content?: string }>(
+      token,
+      `/repos/${repoOwner}/${repoName}/contents/${encodePath(path)}?ref=${branch}`,
+    )
+    if (!response.content) return null
+    const raw = fromBase64(response.content)
+    return JSON.parse(raw) as MetaFile
+  } catch (error) {
+    if (isNotFoundError(error)) return null
+    throw error
+  }
+}
+
+async function fetchStorageBase64(ctx: ActionCtx, storageId: Id<'_storage'>) {
+  const blob = await ctx.storage.get(storageId)
+  if (!blob) throw new Error('File missing in storage')
+  const buffer = Buffer.from(await blob.arrayBuffer())
+  return buffer.toString('base64')
+}
+
+async function createInstallationToken() {
+  const appId = process.env.GITHUB_APP_ID
+  const installationId = process.env.GITHUB_APP_INSTALLATION_ID
+  if (!appId || !installationId) {
+    throw new Error('GitHub App credentials missing')
+  }
+  const jwt = createAppJwt(appId)
+  const response = await fetch(`${GITHUB_API}/app/installations/${installationId}/access_tokens`, {
+    method: 'POST',
+    headers: buildHeaders(jwt, true),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub App token failed: ${message}`)
+  }
+  const payload = (await response.json()) as { token?: string }
+  if (!payload.token) throw new Error('GitHub App token missing')
+  return payload.token
+}
+
+function createAppJwt(appId: string) {
+  const privateKey = loadPrivateKey()
+  const now = Math.floor(Date.now() / 1000)
+  const header = { alg: 'RS256', typ: 'JWT' }
+  const payload = { iat: now - 60, exp: now + 9 * 60, iss: appId }
+  const encodedHeader = base64Url(JSON.stringify(header))
+  const encodedPayload = base64Url(JSON.stringify(payload))
+  const signingInput = `${encodedHeader}.${encodedPayload}`
+  const sign = createSign('RSA-SHA256')
+  sign.update(signingInput)
+  sign.end()
+  const signature = sign.sign(privateKey)
+  return `${signingInput}.${base64Url(signature)}`
+}
+
+function loadPrivateKey() {
+  const raw = process.env.GITHUB_APP_PRIVATE_KEY
+  if (!raw) throw new Error('GITHUB_APP_PRIVATE_KEY is not configured')
+  const normalized = raw.replace(/\\n/g, '\n')
+  return createPrivateKey(normalized)
+}
+
+async function createBlob(token: string, repoOwner: string, repoName: string, content: string) {
+  const result = await githubPost<{ sha: string }>(
+    token,
+    `/repos/${repoOwner}/${repoName}/git/blobs`,
+    {
+      content,
+      encoding: 'base64',
+    },
+  )
+  if (!result.sha) throw new Error('GitHub blob missing sha')
+  return result.sha
+}
+
+async function githubGet<T>(token: string, path: string): Promise<T> {
+  const response = await fetch(`${GITHUB_API}${path}`, {
+    headers: buildHeaders(token),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub GET ${path} failed: ${message}`)
+  }
+  return (await response.json()) as T
+}
+
+async function githubPost<T>(token: string, path: string, body: unknown): Promise<T> {
+  const response = await fetch(`${GITHUB_API}${path}`, {
+    method: 'POST',
+    headers: buildHeaders(token),
+    body: JSON.stringify(body),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub POST ${path} failed: ${message}`)
+  }
+  return (await response.json()) as T
+}
+
+async function githubPatch(token: string, path: string, body: unknown) {
+  const response = await fetch(`${GITHUB_API}${path}`, {
+    method: 'PATCH',
+    headers: buildHeaders(token),
+    body: JSON.stringify(body),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub PATCH ${path} failed: ${message}`)
+  }
+}
+
+function buildHeaders(token: string, isAppJwt = false) {
+  return {
+    Authorization: `${isAppJwt ? 'Bearer' : 'token'} ${token}`,
+    Accept: 'application/vnd.github+json',
+    'User-Agent': USER_AGENT,
+  }
+}
+
+function parseRepo(repo: string) {
+  const [owner, name] = repo.split('/')
+  if (!owner || !name) throw new Error('GITHUB_SKILLS_REPO must be owner/repo')
+  return [owner, name] as const
+}
+
+function normalizeOwner(value: string) {
+  const normalized = value
+    .trim()
+    .toLowerCase()
+    .replace(/[^a-z0-9-]/g, '-')
+    .replace(/-+/g, '-')
+    .replace(/^-+|-+$/g, '')
+  return normalized || 'unknown'
+}
+
+function commitUrl(repo: string, sha: string) {
+  return `https://github.com/${repo}/commit/${sha}`
+}
+
+function buildSkillRoot(root: string, ownerHandle: string, slug: string) {
+  const ownerSegment = normalizeOwner(ownerHandle)
+  return `${root}/${ownerSegment}/${slug}`
+}
+
+function encodePath(path: string) {
+  return path
+    .split('/')
+    .map((segment) => encodeURIComponent(segment))
+    .join('/')
+}
+
+function base64Url(value: string | Buffer) {
+  const buffer = typeof value === 'string' ? Buffer.from(value) : value
+  return buffer.toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/g, '')
+}
+
+function toBase64(value: string) {
+  return Buffer.from(value).toString('base64')
+}
+
+function fromBase64(value: string) {
+  return Buffer.from(value, 'base64').toString('utf8')
+}
+
+function isNotFoundError(error: unknown) {
+  return (
+    error instanceof Error && (error.message.includes('404') || error.message.includes('Not Found'))
+  )
+}

convex/lib/githubImport.test.ts

@@ -0,0 +1,247 @@
+/* @vitest-environment node */
+
+import { unzipSync } from 'fflate'
+import { describe, expect, it } from 'vitest'
+import {
+  buildGitHubZipForTests,
+  computeDefaultSelectedPaths,
+  detectGitHubImportCandidates,
+  extractMarkdownRelativeTargets,
+  fetchGitHubZipBytes,
+  parseGitHubImportUrl,
+  resolveGitHubCommit,
+  resolveMarkdownTarget,
+  stripGitHubZipRoot,
+} from './githubImport'
+
+function requestInfoToUrlString(input: RequestInfo | URL): string {
+  if (typeof input === 'string') return input
+  if (input instanceof URL) return input.toString()
+  if (input instanceof Request) return input.url
+
+  throw new Error('Unexpected fetch input type')
+}
+
+describe('github import', () => {
+  it('parses repo root urls', () => {
+    expect(parseGitHubImportUrl('https://github.com/visionik/ouracli')).toEqual({
+      owner: 'visionik',
+      repo: 'ouracli',
+      originalUrl: 'https://github.com/visionik/ouracli',
+    })
+  })
+
+  it('rejects non-https and non-github urls', () => {
+    expect(() => parseGitHubImportUrl('http://github.com/a/b')).toThrow(/https/i)
+    expect(() => parseGitHubImportUrl('https://example.com/a/b')).toThrow(/github\.com/i)
+    expect(() => parseGitHubImportUrl('not-a-url')).toThrow(/Invalid URL/i)
+  })
+
+  it('rejects malformed tree/blob urls', () => {
+    expect(() => parseGitHubImportUrl('https://github.com/a/b/tree/')).toThrow(/Missing ref/i)
+    expect(() => parseGitHubImportUrl('https://github.com/a/b/blob/main')).toThrow(/Missing path/i)
+    expect(() => parseGitHubImportUrl('https://github.com/a/b/tree/main/bad%5cpath')).toThrow()
+  })
+
+  it('parses tree urls with ref and path', () => {
+    expect(parseGitHubImportUrl('https://github.com/a/b/tree/main/skills/foo')).toEqual({
+      owner: 'a',
+      repo: 'b',
+      ref: 'main',
+      path: 'skills/foo',
+      originalUrl: 'https://github.com/a/b/tree/main/skills/foo',
+    })
+  })
+
+  it('parses blob urls and derives folder path', () => {
+    expect(parseGitHubImportUrl('https://github.com/a/b/blob/main/skills/foo/SKILL.md')).toEqual({
+      owner: 'a',
+      repo: 'b',
+      ref: 'main',
+      path: 'skills/foo',
+      originalUrl: 'https://github.com/a/b/blob/main/skills/foo/SKILL.md',
+    })
+  })
+
+  it('strips single top-level folder from GitHub zip entries', () => {
+    const zip = buildGitHubZipForTests({
+      'repo-1/skill/SKILL.md': 'Body',
+      'repo-1/skill/a.txt': 'a',
+    })
+    const stripped = stripGitHubZipRoot(unzipSync(zip))
+    expect(Object.keys(stripped).sort()).toEqual(['skill/SKILL.md', 'skill/a.txt'])
+  })
+
+  it('keeps paths when zip has multiple top-level roots', () => {
+    const zip = buildGitHubZipForTests({
+      'a/SKILL.md': 'Body',
+      'b/SKILL.md': 'Body',
+    })
+    const stripped = stripGitHubZipRoot(unzipSync(zip))
+    expect(Object.keys(stripped).sort()).toEqual(['a/SKILL.md', 'b/SKILL.md'])
+  })
+
+  it('detects candidates in a GitHub zip and strips the root folder', () => {
+    const zip = buildGitHubZipForTests({
+      'ouracli-123/SKILL.md': `---\nname: demo\ndescription: Hello\n---\nBody`,
+      'ouracli-123/src/index.ts': 'export {}',
+    })
+    const stripped = stripGitHubZipRoot(unzipSync(zip))
+    const candidates = detectGitHubImportCandidates(stripped)
+    expect(candidates.map((c) => c.path)).toEqual([''])
+    expect(candidates[0]?.name).toBe('demo')
+  })
+
+  it('detects multiple candidates and supports skills.md', () => {
+    const zip = buildGitHubZipForTests({
+      'repo-1/alpha/SKILL.md': `---\nname: Alpha\n---\nBody`,
+      'repo-1/beta/skills.md': `---\nname: Beta\n---\nBody`,
+      'repo-1/readme.md': 'x',
+    })
+    const stripped = stripGitHubZipRoot(unzipSync(zip))
+    const candidates = detectGitHubImportCandidates(stripped)
+    expect(candidates.map((c) => c.path)).toEqual(['alpha', 'beta'])
+    expect(candidates.map((c) => c.name)).toEqual(['Alpha', 'Beta'])
+  })
+
+  it('computes default selection via markdown references', () => {
+    const entries = {
+      'skill/SKILL.md': `---\nname: demo\n---\nSee [usage](docs/usage.md) and ![logo](img/logo.svg).\nIgnore [web](https://example.com).`,
+      'skill/docs/usage.md': `See [more](more.md)`,
+      'skill/docs/more.md': `Ok`,
+      'skill/img/logo.svg': `<svg/>`,
+      'skill/extra.txt': 'not referenced',
+    }
+    const zip = buildGitHubZipForTests(
+      Object.fromEntries(Object.entries(entries).map(([k, v]) => [`repo-1/${k}`, v])),
+    )
+    const raw = unzipSync(zip)
+    const stripped = stripGitHubZipRoot(raw)
+    const candidates = detectGitHubImportCandidates(stripped)
+    const candidate = candidates.find((c) => c.path === 'skill')
+    expect(candidate).toBeTruthy()
+    if (!candidate) throw new Error('candidate not found')
+
+    const files = Object.entries(stripped)
+      .filter(([path]) => path.startsWith('skill/'))
+      .map(([path, bytes]) => ({ path, bytes }))
+    const selected = computeDefaultSelectedPaths({ candidate, files })
+    expect(selected).toContain('skill/SKILL.md')
+    expect(selected).toContain('skill/docs/usage.md')
+    expect(selected).toContain('skill/docs/more.md')
+    expect(selected).toContain('skill/img/logo.svg')
+    expect(selected).not.toContain('skill/extra.txt')
+  })
+
+  it('does not select files outside skill folder (even when referenced)', () => {
+    const entries = {
+      'skill/SKILL.md': `See [outside](../outside.md) and [abs](/abs.md) and [mail](mailto:test@example.com).`,
+      'outside.md': `secret`,
+      'skill/docs/usage.md': `Ok`,
+    }
+    const zip = buildGitHubZipForTests(
+      Object.fromEntries(Object.entries(entries).map(([k, v]) => [`repo-1/${k}`, v])),
+    )
+    const stripped = stripGitHubZipRoot(unzipSync(zip))
+    const candidate = detectGitHubImportCandidates(stripped).find((c) => c.path === 'skill')
+    expect(candidate).toBeTruthy()
+    if (!candidate) throw new Error('candidate not found')
+    const files = Object.entries(stripped).map(([path, bytes]) => ({ path, bytes }))
+    const selected = computeDefaultSelectedPaths({ candidate, files })
+    expect(selected).toContain('skill/SKILL.md')
+    expect(selected).not.toContain('outside.md')
+  })
+
+  it('extracts markdown targets with titles and angle brackets', () => {
+    const targets = extractMarkdownRelativeTargets(
+      `See [a](docs/usage.md "Title") and [b](<docs/my file.md>) and ![c](img/logo.svg)`,
+    )
+    expect(targets).toEqual(['docs/usage.md', 'docs/my file.md', 'img/logo.svg'])
+  })
+
+  it('resolves markdown targets safely', () => {
+    expect(resolveMarkdownTarget('a/SKILL.md', 'docs/usage.md')).toBe('a/docs/usage.md')
+    expect(resolveMarkdownTarget('a/SKILL.md', '../oops.md')).toBeNull()
+    expect(resolveMarkdownTarget('a/SKILL.md', '/abs.md')).toBeNull()
+    expect(resolveMarkdownTarget('a/SKILL.md', 'docs/usage.md#section')).toBe('a/docs/usage.md')
+    expect(resolveMarkdownTarget('a/SKILL.md', 'docs/usage.md?x=1')).toBe('a/docs/usage.md')
+  })
+
+  it('resolves HEAD commit via redirect chain and refuses unexpected redirect hosts', async () => {
+    const fetcher: typeof fetch = async (input) => {
+      const url = requestInfoToUrlString(input)
+      if (url.includes('/archive/HEAD.zip')) {
+        return new Response(null, {
+          status: 302,
+          headers: {
+            location:
+              'https://codeload.github.com/a/b/zip/0123456789012345678901234567890123456789',
+          },
+        })
+      }
+      if (url.startsWith('https://codeload.github.com/a/b/zip/')) {
+        return new Response(null, { status: 200 })
+      }
+      throw new Error(`Unexpected fetch: ${url}`)
+    }
+    const resolved = await resolveGitHubCommit(
+      { owner: 'a', repo: 'b', originalUrl: 'https://github.com/a/b' },
+      fetcher,
+    )
+    expect(resolved.commit).toBe('0123456789012345678901234567890123456789')
+
+    const badFetcher: typeof fetch = async (input) => {
+      const url = requestInfoToUrlString(input)
+      if (url.includes('/archive/HEAD.zip')) {
+        return new Response(null, {
+          status: 302,
+          headers: { location: 'https://evil.example/zip/abc' },
+        })
+      }
+      throw new Error(`Unexpected fetch: ${url}`)
+    }
+    await expect(
+      resolveGitHubCommit(
+        { owner: 'a', repo: 'b', originalUrl: 'https://github.com/a/b' },
+        badFetcher,
+      ),
+    ).rejects.toThrow(/redirect/i)
+  })
+
+  it('resolves explicit ref commit via GitHub API', async () => {
+    const fetcher: typeof fetch = async (input) => {
+      const url = requestInfoToUrlString(input)
+      if (url.startsWith('https://api.github.com/repos/a/b/commits/')) {
+        return new Response(JSON.stringify({ sha: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' }), {
+          status: 200,
+        })
+      }
+      throw new Error(`Unexpected fetch: ${url}`)
+    }
+    const resolved = await resolveGitHubCommit(
+      { owner: 'a', repo: 'b', ref: 'main', originalUrl: 'https://github.com/a/b' },
+      fetcher,
+    )
+    expect(resolved.commit).toBe('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')
+  })
+
+  it('enforces zip byte cap when content-length is too large', async () => {
+    const resolved = {
+      owner: 'a',
+      repo: 'b',
+      ref: 'main',
+      commit: '0123456789012345678901234567890123456789',
+      path: '',
+      repoUrl: 'https://github.com/a/b',
+      originalUrl: 'https://github.com/a/b',
+    } as const
+    const fetcher: typeof fetch = async () =>
+      new Response(new Blob([new Uint8Array([1, 2, 3])]), {
+        status: 200,
+        headers: { 'content-length': String(999_999_999) },
+      })
+    await expect(fetchGitHubZipBytes(resolved, fetcher, { maxZipBytes: 10 })).rejects.toThrow(
+      /too large/i,
+    )
+  })
+})

convex/lib/githubImport.ts

@@ -0,0 +1,425 @@
+import { TEXT_FILE_EXTENSION_SET } from 'clawhub-schema'
+import { zipSync } from 'fflate'
+import semver from 'semver'
+import { parseFrontmatter } from './skills'
+
+export type GitHubImportUrl = {
+  owner: string
+  repo: string
+  ref?: string
+  path?: string
+  originalUrl: string
+}
+
+export type GitHubImportResolved = {
+  owner: string
+  repo: string
+  ref: string
+  commit: string
+  path: string
+  repoUrl: string
+  originalUrl: string
+}
+
+export type GitHubImportCandidate = {
+  path: string
+  readmePath: string
+  name?: string
+  description?: string
+}
+
+export type GitHubImportFileEntry = {
+  path: string
+  size: number
+  defaultSelected: boolean
+}
+
+const MAX_REDIRECTS = 6
+const GITHUB_HOST = 'github.com'
+const CODELOAD_HOST = 'codeload.github.com'
+const SKILL_FILENAMES = ['skill.md', 'skills.md']
+
+export function parseGitHubImportUrl(input: string): GitHubImportUrl {
+  const originalUrl = input.trim()
+  let url: URL
+  try {
+    url = new URL(originalUrl)
+  } catch {
+    throw new Error('Invalid URL')
+  }
+  if (url.protocol !== 'https:') throw new Error('Only https:// URLs are supported')
+  if (url.hostname !== GITHUB_HOST) throw new Error('Only github.com URLs are supported')
+
+  const segments = url.pathname
+    .split('/')
+    .map((segment) => segment.trim())
+    .filter(Boolean)
+    .map((segment) => {
+      try {
+        return decodeURIComponent(segment)
+      } catch {
+        throw new Error('Invalid URL')
+      }
+    })
+
+  const owner = segments[0] ?? ''
+  const repo = (segments[1] ?? '').replace(/\.git$/, '')
+  if (!owner || !repo) throw new Error('GitHub URL must be /<owner>/<repo>')
+
+  const kind = segments[2] ?? ''
+  if (!kind) return { owner, repo, originalUrl }
+  if (kind !== 'tree' && kind !== 'blob') {
+    return { owner, repo, originalUrl }
+  }
+
+  const ref = segments[3] ?? ''
+  if (!ref) throw new Error('Missing ref in GitHub URL')
+
+  const rest = segments.slice(4).join('/')
+  const normalizedRest = normalizeRepoPath(rest)
+
+  if (kind === 'blob') {
+    if (!rest) throw new Error('Missing path in GitHub URL')
+    if (!normalizedRest) throw new Error('Invalid path in GitHub URL')
+    const dir = normalizedRest.split('/').slice(0, -1).join('/')
+    return { owner, repo, ref, path: dir || undefined, originalUrl }
+  }
+
+  if (rest && !normalizedRest) throw new Error('Invalid path in GitHub URL')
+  return { owner, repo, ref, path: normalizedRest || undefined, originalUrl }
+}
+
+export async function resolveGitHubCommit(
+  parsed: GitHubImportUrl,
+  fetcher: typeof fetch,
+): Promise<GitHubImportResolved> {
+  const repoUrl = `https://${GITHUB_HOST}/${parsed.owner}/${parsed.repo}`
+  const ref = parsed.ref?.trim() || 'HEAD'
+  const path = normalizeRepoPath(parsed.path ?? '')
+
+  const commit =
+    ref === 'HEAD'
+      ? await resolveHeadCommit(parsed, fetcher)
+      : await resolveRefCommit(parsed, ref, fetcher)
+
+  return {
+    owner: parsed.owner,
+    repo: parsed.repo,
+    ref,
+    commit,
+    path,
+    repoUrl,
+    originalUrl: parsed.originalUrl,
+  }
+}
+
+async function resolveRefCommit(parsed: GitHubImportUrl, ref: string, fetcher: typeof fetch) {
+  const apiUrl = `https://api.github.com/repos/${parsed.owner}/${parsed.repo}/commits/${encodeURIComponent(ref)}`
+  const response = await fetcher(apiUrl, {
+    headers: {
+      Accept: 'application/vnd.github+json',
+      'User-Agent': 'clawhub/github-import',
+    },
+  })
+  if (!response.ok) throw new Error('GitHub ref not found')
+  const body = (await response.json()) as { sha?: unknown }
+  const sha = typeof body.sha === 'string' ? body.sha : ''
+  if (!/^[a-f0-9]{40}$/i.test(sha)) throw new Error('GitHub commit sha missing')
+  return sha.toLowerCase()
+}
+
+async function resolveHeadCommit(parsed: GitHubImportUrl, fetcher: typeof fetch) {
+  let url = `https://${GITHUB_HOST}/${parsed.owner}/${parsed.repo}/archive/HEAD.zip`
+  for (let i = 0; i < MAX_REDIRECTS; i += 1) {
+    const response = await fetcher(url, { redirect: 'manual' })
+    const location = response.headers.get('location')
+    if (!location) break
+    const next = new URL(location, url)
+    if (next.hostname !== GITHUB_HOST && next.hostname !== CODELOAD_HOST) {
+      throw new Error('Unexpected redirect host')
+    }
+    url = next.toString()
+  }
+
+  const maybe = url.split('/').at(-1) ?? ''
+  if (!/^[a-f0-9]{40}$/i.test(maybe)) {
+    throw new Error('Could not resolve commit for HEAD')
+  }
+  return maybe.toLowerCase()
+}
+
+export async function fetchGitHubZipBytes(
+  resolved: GitHubImportResolved,
+  fetcher: typeof fetch,
+  limits?: { maxZipBytes?: number },
+): Promise<Uint8Array> {
+  const maxZipBytes = limits?.maxZipBytes ?? 25 * 1024 * 1024
+  const url = `https://${CODELOAD_HOST}/${resolved.owner}/${resolved.repo}/zip/${resolved.commit}`
+  const response = await fetcher(url, {
+    headers: { 'User-Agent': 'clawhub/github-import' },
+  })
+  if (!response.ok) throw new Error('GitHub archive download failed')
+
+  const lengthHeader = response.headers.get('content-length')
+  if (lengthHeader) {
+    const contentLength = Number.parseInt(lengthHeader, 10)
+    if (Number.isFinite(contentLength) && contentLength > maxZipBytes) {
+      throw new Error('GitHub archive too large')
+    }
+  }
+
+  const reader = response.body?.getReader()
+  if (!reader) {
+    const buffer = new Uint8Array(await response.arrayBuffer())
+    if (buffer.byteLength > maxZipBytes) throw new Error('GitHub archive too large')
+    return buffer
+  }
+
+  const chunks: Uint8Array[] = []
+  let total = 0
+  while (true) {
+    const { done, value } = await reader.read()
+    if (done) break
+    if (!value) continue
+    total += value.byteLength
+    if (total > maxZipBytes) throw new Error('GitHub archive too large')
+    chunks.push(value)
+  }
+
+  const out = new Uint8Array(total)
+  let offset = 0
+  for (const chunk of chunks) {
+    out.set(chunk, offset)
+    offset += chunk.byteLength
+  }
+  return out
+}
+
+export type ZipEntryMap = Record<string, Uint8Array>
+
+export function buildGitHubZipForTests(entries: Record<string, string>) {
+  const asBytes = Object.fromEntries(
+    Object.entries(entries).map(([path, text]) => [path, new TextEncoder().encode(text)]),
+  )
+  return Uint8Array.from(zipSync(asBytes, { level: 1 }))
+}
+
+export function stripGitHubZipRoot(entries: ZipEntryMap): ZipEntryMap {
+  const paths = Object.keys(entries)
+  if (paths.length === 0) return {}
+  const first = paths[0] ?? ''
+  const firstRoot = first.split('/')[0] ?? ''
+  if (!firstRoot) return entries
+  const prefix = `${firstRoot}/`
+  if (!paths.every((path) => path.startsWith(prefix))) return entries
+  const out: ZipEntryMap = {}
+  for (const [path, data] of Object.entries(entries)) {
+    const stripped = path.slice(prefix.length)
+    if (!stripped) continue
+    out[stripped] = data
+  }
+  return out
+}
+
+export function detectGitHubImportCandidates(entries: ZipEntryMap): GitHubImportCandidate[] {
+  const candidates: GitHubImportCandidate[] = []
+  for (const path of Object.keys(entries)) {
+    const normalized = normalizeRepoPath(path)
+    const lower = normalized.toLowerCase()
+    const isSkill = SKILL_FILENAMES.some((name) => lower === name || lower.endsWith(`/${name}`))
+    if (!isSkill) continue
+    const dir = normalized.split('/').slice(0, -1).join('/')
+    const readmePath = normalized
+    const raw = new TextDecoder().decode(entries[path] ?? new Uint8Array())
+    const frontmatter = parseFrontmatter(raw)
+    const name = typeof frontmatter.name === 'string' ? frontmatter.name : undefined
+    const description =
+      typeof frontmatter.description === 'string' ? frontmatter.description : undefined
+    candidates.push({
+      path: normalizeRepoPath(dir),
+      readmePath,
+      name: name?.trim() || undefined,
+      description: description?.trim() || undefined,
+    })
+  }
+  return uniqCandidates(candidates)
+}
+
+function uniqCandidates(candidates: GitHubImportCandidate[]) {
+  const seen = new Set<string>()
+  const out: GitHubImportCandidate[] = []
+  for (const candidate of candidates) {
+    const key = `${candidate.path}::${candidate.readmePath}`
+    if (seen.has(key)) continue
+    seen.add(key)
+    out.push(candidate)
+  }
+  return out.sort((a, b) => a.path.localeCompare(b.path))
+}
+
+export function listTextFilesUnderCandidate(
+  entries: ZipEntryMap,
+  candidatePath: string,
+): Array<{ path: string; bytes: Uint8Array }> {
+  const root = normalizeCandidateRoot(candidatePath)
+  const out: Array<{ path: string; bytes: Uint8Array }> = []
+  for (const [path, bytes] of Object.entries(entries)) {
+    const normalized = normalizeRepoPath(path)
+    if (!isUnderRoot(normalized, root)) continue
+    if (!isTextPath(normalized)) continue
+    out.push({ path: normalized, bytes })
+  }
+  return out.sort((a, b) => a.path.localeCompare(b.path))
+}
+
+export function computeDefaultSelectedPaths(params: {
+  candidate: GitHubImportCandidate
+  files: Array<{ path: string; bytes: Uint8Array }>
+  maxDepth?: number
+  maxAdds?: number
+}) {
+  const maxDepth = params.maxDepth ?? 4
+  const maxAdds = params.maxAdds ?? 200
+  const byPath = new Map(params.files.map((file) => [file.path, file.bytes]))
+  const candidateRoot = normalizeCandidateRoot(params.candidate.path)
+  const selected = new Set<string>()
+  let added = 0
+
+  const add = (path: string) => {
+    const normalized = normalizeRepoPath(path)
+    if (!isUnderRoot(normalized, candidateRoot)) return
+    if (!byPath.has(normalized)) return
+    if (!selected.has(normalized)) {
+      selected.add(normalized)
+      added += 1
+    }
+  }
+
+  add(params.candidate.readmePath)
+
+  const visited = new Set<string>()
+  const queue: Array<{ path: string; depth: number }> = [
+    { path: params.candidate.readmePath, depth: 0 },
+  ]
+
+  while (queue.length > 0) {
+    const item = queue.shift()
+    if (!item) break
+    if (item.depth >= maxDepth) continue
+    if (visited.has(item.path)) continue
+    visited.add(item.path)
+
+    const bytes = byPath.get(item.path)
+    if (!bytes) continue
+    if (!item.path.toLowerCase().endsWith('.md')) continue
+
+    const text = new TextDecoder().decode(bytes)
+    const refs = extractMarkdownRelativeTargets(text)
+    for (const ref of refs) {
+      if (added >= maxAdds) break
+      const resolved = resolveMarkdownTarget(item.path, ref)
+      if (!resolved) continue
+      add(resolved)
+      if (resolved.toLowerCase().endsWith('.md') && byPath.has(resolved)) {
+        queue.push({ path: resolved, depth: item.depth + 1 })
+      }
+    }
+    if (added >= maxAdds) break
+  }
+
+  return Array.from(selected).sort()
+}
+
+export function buildGitHubImportFileList(params: {
+  candidate: GitHubImportCandidate
+  files: Array<{ path: string; bytes: Uint8Array }>
+  defaultSelectedPaths: string[]
+}): GitHubImportFileEntry[] {
+  const selected = new Set(params.defaultSelectedPaths)
+  return params.files.map((file) => ({
+    path: file.path,
+    size: file.bytes.byteLength,
+    defaultSelected: selected.has(file.path),
+  }))
+}
+
+export function normalizeRepoPath(path: string) {
+  const stripped = path.replace(/^\/+/, '').trim()
+  if (!stripped) return ''
+  const cleaned = stripped.split('/').filter(Boolean).join('/')
+  if (!cleaned || cleaned.includes('\\') || cleaned.includes('..')) return ''
+  return cleaned
+}
+
+export function normalizeCandidateRoot(candidatePath: string) {
+  const normalized = normalizeRepoPath(candidatePath)
+  return normalized ? `${normalized}/` : ''
+}
+
+function isUnderRoot(path: string, rootWithSlash: string) {
+  if (!rootWithSlash) return true
+  return path === rootWithSlash.slice(0, -1) || path.startsWith(rootWithSlash)
+}
+
+function isTextPath(path: string) {
+  const lower = path.toLowerCase()
+  const ext = lower.split('.').at(-1) ?? ''
+  if (!ext) return false
+  return TEXT_FILE_EXTENSION_SET.has(ext)
+}
+
+export function suggestDisplayName(candidate: GitHubImportCandidate, fallbackBase: string) {
+  const base = candidate.name?.trim() || fallbackBase.trim()
+  if (!base) return ''
+  return base
+    .replace(/[-_]+/g, ' ')
+    .replace(/\s+/g, ' ')
+    .replace(/\b\w/g, (char) => char.toUpperCase())
+}
+
+export function suggestVersion(latestVersion?: string | null) {
+  const latest = latestVersion?.trim() || ''
+  if (latest && semver.valid(latest)) {
+    return semver.inc(latest, 'patch') ?? '0.1.0'
+  }
+  return '0.1.0'
+}
+
+export function extractMarkdownRelativeTargets(markdown: string): string[] {
+  const out: string[] = []
+  const pattern = /!?\[[^\]]*]\(([^)]+)\)/g
+  for (const match of markdown.matchAll(pattern)) {
+    const raw = (match[1] ?? '').trim()
+    if (!raw) continue
+    const isAngleWrapped = raw.startsWith('<') && raw.endsWith('>')
+    const cleaned = raw.replace(/^<|>$/g, '').trim()
+    if (!cleaned) continue
+    const target = isAngleWrapped ? cleaned : (cleaned.split(/\s+/)[0] ?? '')
+    if (!target) continue
+    if (target.startsWith('#')) continue
+    const lower = target.toLowerCase()
+    if (lower.startsWith('http:') || lower.startsWith('https:')) continue
+    if (lower.startsWith('mailto:')) continue
+    out.push(target)
+  }
+  return out
+}
+
+export function resolveMarkdownTarget(fromPath: string, target: string) {
+  const withoutHash = target.split('#')[0] ?? ''
+  const withoutQuery = (withoutHash.split('?')[0] ?? '').trim()
+  if (!withoutQuery) return null
+  if (withoutQuery.startsWith('/')) return null
+  if (withoutQuery.includes('\\') || withoutQuery.includes('..')) return null
+
+  const fromDirParts = normalizeRepoPath(fromPath).split('/').slice(0, -1)
+  const targetParts = withoutQuery.split('/').filter(Boolean)
+  const combined = [...fromDirParts, ...targetParts]
+  const normalized: string[] = []
+  for (const part of combined) {
+    if (part === '.') continue
+    if (part === '..') return null
+    normalized.push(part)
+  }
+  return normalizeRepoPath(normalized.join('/')) || null
+}

convex/lib/githubSoulBackup.ts

@@ -0,0 +1,443 @@
+'use node'
+
+import { createPrivateKey, createSign } from 'node:crypto'
+import type { Id } from '../_generated/dataModel'
+import type { ActionCtx } from '../_generated/server'
+
+const GITHUB_API = 'https://api.github.com'
+const DEFAULT_REPO = 'clawdbot/souls'
+const DEFAULT_ROOT = 'souls'
+const META_FILENAME = '_meta.json'
+const USER_AGENT = 'clawhub/souls-backup'
+
+type BackupFile = {
+  path: string
+  size: number
+  storageId: Id<'_storage'>
+  sha256: string
+  contentType?: string
+}
+
+type BackupParams = {
+  slug: string
+  version: string
+  displayName: string
+  ownerHandle: string
+  files: BackupFile[]
+  publishedAt: number
+}
+
+type RepoInfo = {
+  default_branch?: string
+}
+
+type GitRef = {
+  object: { sha: string }
+}
+
+type GitCommit = {
+  sha: string
+  tree: { sha: string }
+}
+
+type GitTreeEntry = {
+  path?: string
+  type?: string
+}
+
+type GitTree = {
+  tree?: GitTreeEntry[]
+}
+
+type MetaFile = {
+  owner: string
+  slug: string
+  displayName: string
+  latest: {
+    version: string
+    publishedAt: number
+    commit: string | null
+  }
+  history: Array<{
+    version: string
+    publishedAt: number
+    commit: string
+  }>
+}
+
+export type GitHubBackupContext = {
+  token: string
+  repo: string
+  repoOwner: string
+  repoName: string
+  branch: string
+  root: string
+}
+
+export function isGitHubSoulBackupConfigured() {
+  return Boolean(
+    process.env.GITHUB_APP_ID &&
+      process.env.GITHUB_APP_PRIVATE_KEY &&
+      process.env.GITHUB_APP_INSTALLATION_ID,
+  )
+}
+
+export async function getGitHubSoulBackupContext(): Promise<GitHubBackupContext> {
+  const repo = process.env.GITHUB_SOULS_REPO ?? DEFAULT_REPO
+  const root = process.env.GITHUB_SOULS_ROOT ?? DEFAULT_ROOT
+  const [repoOwner, repoName] = parseRepo(repo)
+  const token = await createInstallationToken()
+  const repoInfo = await githubGet<RepoInfo>(token, `/repos/${repoOwner}/${repoName}`)
+  const branch = repoInfo.default_branch ?? 'main'
+
+  return { token, repo, repoOwner, repoName, branch, root }
+}
+
+export async function fetchGitHubSoulMeta(
+  context: GitHubBackupContext,
+  ownerHandle: string,
+  slug: string,
+): Promise<MetaFile | null> {
+  const soulRoot = buildSoulRoot(context.root, ownerHandle, slug)
+  return fetchMetaFile(
+    context.token,
+    context.repoOwner,
+    context.repoName,
+    `${soulRoot}/${META_FILENAME}`,
+    context.branch,
+  )
+}
+
+export async function backupSoulToGitHub(
+  ctx: ActionCtx,
+  params: BackupParams,
+  context?: GitHubBackupContext,
+) {
+  if (!isGitHubSoulBackupConfigured()) return
+
+  const resolved = context ?? (await getGitHubSoulBackupContext())
+  const soulRoot = buildSoulRoot(resolved.root, params.ownerHandle, params.slug)
+  const ref = await githubGet<GitRef>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/ref/heads/${resolved.branch}`,
+  )
+  const baseCommitSha = ref.object.sha
+  const baseCommit = await githubGet<GitCommit>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits/${baseCommitSha}`,
+  )
+  const baseTreeSha = baseCommit.tree.sha
+  const existingTree = await githubGet<GitTree>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees/${baseTreeSha}?recursive=1`,
+  )
+
+  const prefix = `${soulRoot}/`
+  const existingPaths = new Set(
+    (existingTree.tree ?? [])
+      .filter((entry) => entry.type === 'blob' && entry.path?.startsWith(prefix))
+      .map((entry) => entry.path ?? ''),
+  )
+
+  const newPaths = new Set<string>()
+  const treeEntries: Array<{
+    path: string
+    mode: '100644'
+    type: 'blob'
+    sha: string | null
+  }> = []
+
+  for (const file of params.files) {
+    const content = await fetchStorageBase64(ctx, file.storageId)
+    const blobSha = await createBlob(resolved.token, resolved.repoOwner, resolved.repoName, content)
+    const path = `${soulRoot}/${file.path}`
+    newPaths.add(path)
+    treeEntries.push({ path, mode: '100644', type: 'blob', sha: blobSha })
+  }
+
+  const existingMeta = await fetchMetaFile(
+    resolved.token,
+    resolved.repoOwner,
+    resolved.repoName,
+    `${soulRoot}/${META_FILENAME}`,
+    resolved.branch,
+  )
+  const metaPath = `${soulRoot}/${META_FILENAME}`
+  const metaDraft = buildMetaFile(params, existingMeta, resolved.repo, baseCommitSha, null)
+  const metaDraftContent = `${JSON.stringify(metaDraft, null, 2)}\n`
+  const metaDraftSha = await createBlob(
+    resolved.token,
+    resolved.repoOwner,
+    resolved.repoName,
+    toBase64(metaDraftContent),
+  )
+  newPaths.add(metaPath)
+  treeEntries.push({ path: metaPath, mode: '100644', type: 'blob', sha: metaDraftSha })
+
+  for (const path of existingPaths) {
+    if (newPaths.has(path)) continue
+    treeEntries.push({ path, mode: '100644', type: 'blob', sha: null })
+  }
+
+  const newTree = await githubPost<{ sha: string }>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees`,
+    {
+      base_tree: baseTreeSha,
+      tree: treeEntries,
+    },
+  )
+
+  const commit = await githubPost<GitCommit>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits`,
+    {
+      message: `soul: ${params.slug} v${params.version}`,
+      tree: newTree.sha,
+      parents: [baseCommitSha],
+    },
+  )
+
+  const metaFinal = buildMetaFile(params, existingMeta, resolved.repo, baseCommitSha, commit.sha)
+  const metaFinalContent = `${JSON.stringify(metaFinal, null, 2)}\n`
+  const metaFinalSha = await createBlob(
+    resolved.token,
+    resolved.repoOwner,
+    resolved.repoName,
+    toBase64(metaFinalContent),
+  )
+  const metaTree = await githubPost<{ sha: string }>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees`,
+    {
+      base_tree: commit.tree.sha,
+      tree: [{ path: metaPath, mode: '100644', type: 'blob', sha: metaFinalSha }],
+    },
+  )
+  const metaCommit = await githubPost<GitCommit>(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits`,
+    {
+      message: `meta: ${params.slug} v${params.version}`,
+      tree: metaTree.sha,
+      parents: [commit.sha],
+    },
+  )
+
+  await githubPatch(
+    resolved.token,
+    `/repos/${resolved.repoOwner}/${resolved.repoName}/git/refs/heads/${resolved.branch}`,
+    {
+      sha: metaCommit.sha,
+    },
+  )
+}
+
+function buildMetaFile(
+  params: BackupParams,
+  existing: MetaFile | null,
+  repo: string,
+  baseCommitSha: string,
+  latestCommitSha: string | null,
+): MetaFile {
+  let history = [...(existing?.history ?? [])]
+  if (existing?.latest?.version) {
+    const previousCommit = existing.latest.commit ?? commitUrl(repo, baseCommitSha)
+    const previous = {
+      version: existing.latest.version,
+      publishedAt: existing.latest.publishedAt,
+      commit: previousCommit,
+    }
+    history = [previous, ...history.filter((entry) => entry.version !== previous.version)]
+  }
+
+  return {
+    owner: normalizeOwner(params.ownerHandle),
+    slug: params.slug,
+    displayName: params.displayName,
+    latest: {
+      version: params.version,
+      publishedAt: params.publishedAt,
+      commit: latestCommitSha ? commitUrl(repo, latestCommitSha) : null,
+    },
+    history: history.slice(0, 200),
+  }
+}
+
+async function fetchMetaFile(
+  token: string,
+  repoOwner: string,
+  repoName: string,
+  path: string,
+  branch: string,
+): Promise<MetaFile | null> {
+  try {
+    const response = await githubGet<{ content?: string }>(
+      token,
+      `/repos/${repoOwner}/${repoName}/contents/${encodePath(path)}?ref=${branch}`,
+    )
+    if (!response.content) return null
+    const raw = fromBase64(response.content)
+    return JSON.parse(raw) as MetaFile
+  } catch (error) {
+    if (isNotFoundError(error)) return null
+    throw error
+  }
+}
+
+async function fetchStorageBase64(ctx: ActionCtx, storageId: Id<'_storage'>) {
+  const blob = await ctx.storage.get(storageId)
+  if (!blob) throw new Error('File missing in storage')
+  const buffer = Buffer.from(await blob.arrayBuffer())
+  return buffer.toString('base64')
+}
+
+async function createInstallationToken() {
+  const appId = process.env.GITHUB_APP_ID
+  const installationId = process.env.GITHUB_APP_INSTALLATION_ID
+  if (!appId || !installationId) {
+    throw new Error('GitHub App credentials missing')
+  }
+  const jwt = createAppJwt(appId)
+  const response = await fetch(`${GITHUB_API}/app/installations/${installationId}/access_tokens`, {
+    method: 'POST',
+    headers: buildHeaders(jwt, true),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub App token failed: ${message}`)
+  }
+  const payload = (await response.json()) as { token?: string }
+  if (!payload.token) throw new Error('GitHub App token missing')
+  return payload.token
+}
+
+function createAppJwt(appId: string) {
+  const privateKey = loadPrivateKey()
+  const now = Math.floor(Date.now() / 1000)
+  const header = { alg: 'RS256', typ: 'JWT' }
+  const payload = { iat: now - 60, exp: now + 9 * 60, iss: appId }
+  const encodedHeader = base64Url(JSON.stringify(header))
+  const encodedPayload = base64Url(JSON.stringify(payload))
+  const signingInput = `${encodedHeader}.${encodedPayload}`
+  const sign = createSign('RSA-SHA256')
+  sign.update(signingInput)
+  sign.end()
+  const signature = sign.sign(privateKey)
+  return `${signingInput}.${base64Url(signature)}`
+}
+
+function loadPrivateKey() {
+  const raw = process.env.GITHUB_APP_PRIVATE_KEY
+  if (!raw) throw new Error('GITHUB_APP_PRIVATE_KEY is not configured')
+  const normalized = raw.replace(/\\n/g, '\n')
+  return createPrivateKey(normalized)
+}
+
+async function createBlob(token: string, repoOwner: string, repoName: string, content: string) {
+  const result = await githubPost<{ sha: string }>(
+    token,
+    `/repos/${repoOwner}/${repoName}/git/blobs`,
+    {
+      content,
+      encoding: 'base64',
+    },
+  )
+  if (!result.sha) throw new Error('GitHub blob missing sha')
+  return result.sha
+}
+
+async function githubGet<T>(token: string, path: string): Promise<T> {
+  const response = await fetch(`${GITHUB_API}${path}`, {
+    headers: buildHeaders(token),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub GET ${path} failed: ${message}`)
+  }
+  return (await response.json()) as T
+}
+
+async function githubPost<T>(token: string, path: string, body: unknown): Promise<T> {
+  const response = await fetch(`${GITHUB_API}${path}`, {
+    method: 'POST',
+    headers: buildHeaders(token),
+    body: JSON.stringify(body),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub POST ${path} failed: ${message}`)
+  }
+  return (await response.json()) as T
+}
+
+async function githubPatch(token: string, path: string, body: unknown) {
+  const response = await fetch(`${GITHUB_API}${path}`, {
+    method: 'PATCH',
+    headers: buildHeaders(token),
+    body: JSON.stringify(body),
+  })
+  if (!response.ok) {
+    const message = await response.text()
+    throw new Error(`GitHub PATCH ${path} failed: ${message}`)
+  }
+}
+
+function buildHeaders(token: string, isAppJwt = false) {
+  return {
+    Authorization: `${isAppJwt ? 'Bearer' : 'token'} ${token}`,
+    Accept: 'application/vnd.github+json',
+    'User-Agent': USER_AGENT,
+  }
+}
+
+function parseRepo(repo: string) {
+  const [owner, name] = repo.split('/')
+  if (!owner || !name) throw new Error('GITHUB_SOULS_REPO must be owner/repo')
+  return [owner, name] as const
+}
+
+function normalizeOwner(value: string) {
+  const normalized = value
+    .trim()
+    .toLowerCase()
+    .replace(/[^a-z0-9-]/g, '-')
+    .replace(/-+/g, '-')
+    .replace(/^-+|-+$/g, '')
+  return normalized || 'unknown'
+}
+
+function commitUrl(repo: string, sha: string) {
+  return `https://github.com/${repo}/commit/${sha}`
+}
+
+function buildSoulRoot(root: string, ownerHandle: string, slug: string) {
+  const ownerSegment = normalizeOwner(ownerHandle)
+  return `${root}/${ownerSegment}/${slug}`
+}
+
+function encodePath(path: string) {
+  return path
+    .split('/')
+    .map((segment) => encodeURIComponent(segment))
+    .join('/')
+}
+
+function base64Url(value: string | Buffer) {
+  const buffer = typeof value === 'string' ? Buffer.from(value) : value
+  return buffer.toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/g, '')
+}
+
+function toBase64(value: string) {
+  return Buffer.from(value).toString('base64')
+}
+
+function fromBase64(value: string) {
+  return Buffer.from(value, 'base64').toString('utf8')
+}
+
+function isNotFoundError(error: unknown) {
+  return (
+    error instanceof Error && (error.message.includes('404') || error.message.includes('Not Found'))
+  )
+}

convex/lib/leaderboards.ts

@@ -0,0 +1,103 @@
+import type { Id } from '../_generated/dataModel'
+import type { MutationCtx, QueryCtx } from '../_generated/server'
+
+const DAY_MS = 24 * 60 * 60 * 1000
+export const TRENDING_DAYS = 7
+
+type LeaderboardEntry = {
+  skillId: Id<'skills'>
+  score: number
+  installs: number
+  downloads: number
+}
+
+export function toDayKey(timestamp: number) {
+  return Math.floor(timestamp / DAY_MS)
+}
+
+export function getTrendingRange(now: number) {
+  const endDay = toDayKey(now)
+  const startDay = endDay - (TRENDING_DAYS - 1)
+  return { startDay, endDay }
+}
+
+export async function buildTrendingLeaderboard(
+  ctx: QueryCtx | MutationCtx,
+  params: { limit: number; now?: number },
+) {
+  const now = params.now ?? Date.now()
+  const { startDay, endDay } = getTrendingRange(now)
+  const rows = await ctx.db
+    .query('skillDailyStats')
+    .withIndex('by_day', (q) => q.gte('day', startDay).lte('day', endDay))
+    .collect()
+
+  const totals = new Map<Id<'skills'>, { installs: number; downloads: number }>()
+  for (const row of rows) {
+    const current = totals.get(row.skillId) ?? { installs: 0, downloads: 0 }
+    current.installs += row.installs
+    current.downloads += row.downloads
+    totals.set(row.skillId, current)
+  }
+
+  const entries = Array.from(totals, ([skillId, totalsEntry]) => ({
+    skillId,
+    installs: totalsEntry.installs,
+    downloads: totalsEntry.downloads,
+    score: totalsEntry.installs,
+  }))
+
+  const items = topN(entries, params.limit, compareTrendingEntries).sort((a, b) =>
+    compareTrendingEntries(b, a),
+  )
+
+  return { startDay, endDay, items }
+}
+
+function compareTrendingEntries(a: LeaderboardEntry, b: LeaderboardEntry) {
+  if (a.score !== b.score) return a.score - b.score
+  if (a.downloads !== b.downloads) return a.downloads - b.downloads
+  return 0
+}
+
+function topN<T>(entries: T[], limit: number, compare: (a: T, b: T) => number) {
+  if (entries.length <= limit) return entries.slice()
+
+  const heap: T[] = []
+  for (const entry of entries) {
+    if (heap.length < limit) {
+      heap.push(entry)
+      siftUp(heap, heap.length - 1, compare)
+      continue
+    }
+    if (compare(entry, heap[0]) <= 0) continue
+    heap[0] = entry
+    siftDown(heap, 0, compare)
+  }
+  return heap
+}
+
+function siftUp<T>(heap: T[], index: number, compare: (a: T, b: T) => number) {
+  let current = index
+  while (current > 0) {
+    const parent = Math.floor((current - 1) / 2)
+    if (compare(heap[current], heap[parent]) >= 0) break
+    ;[heap[current], heap[parent]] = [heap[parent], heap[current]]
+    current = parent
+  }
+}
+
+function siftDown<T>(heap: T[], index: number, compare: (a: T, b: T) => number) {
+  let current = index
+  const length = heap.length
+  while (true) {
+    const left = current * 2 + 1
+    const right = current * 2 + 2
+    let smallest = current
+    if (left < length && compare(heap[left], heap[smallest]) < 0) smallest = left
+    if (right < length && compare(heap[right], heap[smallest]) < 0) smallest = right
+    if (smallest === current) break
+    ;[heap[current], heap[smallest]] = [heap[smallest], heap[current]]
+    current = smallest
+  }
+}

convex/lib/moderation.ts

@@ -0,0 +1,49 @@
+import type { Doc } from '../_generated/dataModel'
+
+const FLAG_RULES: Array<{ flag: string; pattern: RegExp }> = [
+  // Known-bad / known-suspicious identifiers.
+  // NOTE: keep these narrowly scoped; use staff review to confirm removals.
+  {
+    flag: 'blocked.malware',
+    pattern: /(keepcold131\/ClawdAuthenticatorTool|ClawdAuthenticatorTool)/i,
+  },
+
+  { flag: 'suspicious.keyword', pattern: /(malware|stealer|phish|phishing|keylogger)/i },
+  { flag: 'suspicious.secrets', pattern: /(api[-_ ]?key|token|password|private key|secret)/i },
+  { flag: 'suspicious.crypto', pattern: /(wallet|seed phrase|mnemonic|crypto)/i },
+  { flag: 'suspicious.webhook', pattern: /(discord\.gg|webhook|hooks\.slack)/i },
+  { flag: 'suspicious.script', pattern: /(curl[^\n]+\|\s*(sh|bash))/i },
+  { flag: 'suspicious.url_shortener', pattern: /(bit\.ly|tinyurl\.com|t\.co|goo\.gl|is\.gd)/i },
+]
+
+export function deriveModerationFlags({
+  skill,
+  parsed,
+  files,
+}: {
+  skill: Pick<Doc<'skills'>, 'slug' | 'displayName' | 'summary'>
+  parsed: Doc<'skillVersions'>['parsed']
+  files: Doc<'skillVersions'>['files']
+}) {
+  const text = [
+    skill.slug,
+    skill.displayName,
+    skill.summary ?? '',
+    JSON.stringify(parsed?.frontmatter ?? {}),
+    JSON.stringify(parsed?.metadata ?? {}),
+    JSON.stringify((parsed as { moltbot?: unknown } | undefined)?.moltbot ?? {}),
+    ...files.map((file) => file.path),
+  ]
+    .filter(Boolean)
+    .join('\n')
+
+  const flags = new Set<string>()
+
+  for (const rule of FLAG_RULES) {
+    if (rule.pattern.test(text)) {
+      flags.add(rule.flag)
+    }
+  }
+
+  return Array.from(flags)
+}

convex/lib/public.ts

@@ -0,0 +1,91 @@
+import type { Doc } from '../_generated/dataModel'
+
+export type PublicUser = Pick<
+  Doc<'users'>,
+  '_id' | '_creationTime' | 'handle' | 'name' | 'displayName' | 'image' | 'bio'
+>
+
+export type PublicSkill = Pick<
+  Doc<'skills'>,
+  | '_id'
+  | '_creationTime'
+  | 'slug'
+  | 'displayName'
+  | 'summary'
+  | 'ownerUserId'
+  | 'canonicalSkillId'
+  | 'forkOf'
+  | 'latestVersionId'
+  | 'tags'
+  | 'badges'
+  | 'stats'
+  | 'createdAt'
+  | 'updatedAt'
+>
+
+export type PublicSoul = Pick<
+  Doc<'souls'>,
+  | '_id'
+  | '_creationTime'
+  | 'slug'
+  | 'displayName'
+  | 'summary'
+  | 'ownerUserId'
+  | 'latestVersionId'
+  | 'tags'
+  | 'stats'
+  | 'createdAt'
+  | 'updatedAt'
+>
+
+export function toPublicUser(user: Doc<'users'> | null | undefined): PublicUser | null {
+  if (!user || user.deletedAt) return null
+  return {
+    _id: user._id,
+    _creationTime: user._creationTime,
+    handle: user.handle,
+    name: user.name,
+    displayName: user.displayName,
+    image: user.image,
+    bio: user.bio,
+  }
+}
+
+export function toPublicSkill(skill: Doc<'skills'> | null | undefined): PublicSkill | null {
+  if (!skill || skill.softDeletedAt) return null
+  if (skill.moderationStatus && skill.moderationStatus !== 'active') return null
+  if (skill.moderationFlags?.includes('blocked.malware')) return null
+  return {
+    _id: skill._id,
+    _creationTime: skill._creationTime,
+    slug: skill.slug,
+    displayName: skill.displayName,
+    summary: skill.summary,
+    ownerUserId: skill.ownerUserId,
+    canonicalSkillId: skill.canonicalSkillId,
+    forkOf: skill.forkOf,
+    latestVersionId: skill.latestVersionId,
+    tags: skill.tags,
+    badges: skill.badges,
+    stats: skill.stats,
+    createdAt: skill.createdAt,
+    updatedAt: skill.updatedAt,
+  }
+}
+
+export function toPublicSoul(soul: Doc<'souls'> | null | undefined): PublicSoul | null {
+  if (!soul || soul.softDeletedAt) return null
+  return {
+    _id: soul._id,
+    _creationTime: soul._creationTime,
+    slug: soul.slug,
+    displayName: soul.displayName,
+    summary: soul.summary,
+    ownerUserId: soul.ownerUserId,
+    latestVersionId: soul.latestVersionId,
+    tags: soul.tags,
+    stats: soul.stats,
+    createdAt: soul.createdAt,
+    updatedAt: soul.updatedAt,
+  }
+}

convex/lib/searchText.test.ts

@@ -0,0 +1,48 @@
+/* @vitest-environment node */
+
+import { describe, expect, it } from 'vitest'
+import { __test, matchesExactTokens, tokenize } from './searchText'
+
+describe('searchText', () => {
+  it('tokenize lowercases and splits on punctuation', () => {
+    expect(tokenize('Minimax Usage /minimax-usage')).toEqual([
+      'minimax',
+      'usage',
+      'minimax',
+      'usage',
+    ])
+  })
+
+  it('matchesExactTokens requires at least one query token to prefix-match', () => {
+    const queryTokens = tokenize('Remind Me')
+    expect(matchesExactTokens(queryTokens, ['Remind Me', '/remind-me', 'Short summary'])).toBe(true)
+    // "Reminder" starts with "remind", so it matches with prefix matching
+    expect(matchesExactTokens(queryTokens, ['Reminder tool', '/reminder', 'Short summary'])).toBe(
+      true,
+    )
+    // Matches because "remind" token is present
+    expect(matchesExactTokens(queryTokens, ['Remind tool', '/remind', 'Short summary'])).toBe(true)
+    // No matching tokens at all
+    expect(matchesExactTokens(queryTokens, ['Other tool', '/other', 'Short summary'])).toBe(false)
+  })
+
+  it('matchesExactTokens supports prefix matching for partial queries', () => {
+    // "go" should match "gohome" because "gohome" starts with "go"
+    expect(matchesExactTokens(['go'], ['GoHome', '/gohome', 'Navigate home'])).toBe(true)
+    // "pad" should match "padel"
+    expect(matchesExactTokens(['pad'], ['Padel', '/padel', 'Tennis-like sport'])).toBe(true)
+    // "xyz" should not match anything
+    expect(matchesExactTokens(['xyz'], ['GoHome', '/gohome', 'Navigate home'])).toBe(false)
+    // "notion" should not match "annotations" (substring only)
+    expect(matchesExactTokens(['notion'], ['Annotations helper', '/annotations'])).toBe(false)
+  })
+
+  it('matchesExactTokens ignores empty inputs', () => {
+    expect(matchesExactTokens([], ['text'])).toBe(false)
+    expect(matchesExactTokens(['token'], ['  ', null, undefined])).toBe(false)
+  })
+
+  it('normalize uses lowercase', () => {
+    expect(__test.normalize('AbC')).toBe('abc')
+  })
+})

convex/lib/searchText.ts

@@ -0,0 +1,27 @@
+const WORD_RE = /[a-z0-9]+/g
+
+function normalize(value: string) {
+  return value.toLowerCase()
+}
+
+export function tokenize(value: string): string[] {
+  if (!value) return []
+  return normalize(value).match(WORD_RE) ?? []
+}
+
+export function matchesExactTokens(
+  queryTokens: string[],
+  parts: Array<string | null | undefined>,
+): boolean {
+  if (queryTokens.length === 0) return false
+  const text = parts.filter((part) => Boolean(part?.trim())).join(' ')
+  if (!text) return false
+  const textTokens = tokenize(text)
+  if (textTokens.length === 0) return false
+  // Require at least one token to prefix-match, allowing vector similarity to determine relevance
+  return queryTokens.some((queryToken) =>
+    textTokens.some((textToken) => textToken.startsWith(queryToken)),
+  )
+}
+
+export const __test = { normalize, tokenize, matchesExactTokens }

convex/lib/securityPrompt.ts

@@ -0,0 +1,499 @@
+export function getLlmEvalModel(): string {
+  return process.env.OPENAI_EVAL_MODEL ?? 'gpt-5-mini'
+}
+export const LLM_EVAL_MAX_OUTPUT_TOKENS = 16000
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+
+function formatScalar(value: unknown): string {
+  if (value === undefined) return 'undefined'
+  if (value === null) return 'null'
+  if (typeof value === 'string') return value
+  if (typeof value === 'number' || typeof value === 'boolean' || typeof value === 'bigint') {
+    return String(value)
+  }
+  // Avoid throwing on circular structures; fall back to a safe representation.
+  try {
+    return JSON.stringify(value)
+  } catch {
+    return Object.prototype.toString.call(value)
+  }
+}
+
+function formatWithDefault(value: unknown, defaultLabel: string): string {
+  if (value === undefined || value === null) return defaultLabel
+  return formatScalar(value)
+}
+
+// ---------------------------------------------------------------------------
+// Types
+// ---------------------------------------------------------------------------
+
+export type SkillEvalContext = {
+  slug: string
+  displayName: string
+  ownerUserId: string
+  version: string
+  createdAt: number
+  summary?: string
+  source?: string
+  homepage?: string
+  parsed: {
+    frontmatter: Record<string, unknown>
+    metadata?: unknown
+    clawdis?: unknown
+  }
+  files: Array<{ path: string; size: number }>
+  skillMdContent: string
+  fileContents: Array<{ path: string; content: string }>
+  injectionSignals: string[]
+}
+
+export type LlmEvalDimension = {
+  name: string
+  label: string
+  rating: string
+  detail: string
+}
+
+export type LlmEvalResponse = {
+  verdict: 'benign' | 'suspicious' | 'malicious'
+  confidence: 'high' | 'medium' | 'low'
+  summary: string
+  dimensions: LlmEvalDimension[]
+  guidance: string
+  findings: string
+}
+
+// ---------------------------------------------------------------------------
+// System prompt (~3500 words)
+// ---------------------------------------------------------------------------
+
+export const SECURITY_EVALUATOR_SYSTEM_PROMPT = `You are a security evaluator for OpenClaw AI skills. Users install skills to extend what their AI agent can do. Some users have limited security knowledge — your job is to surface things that don't add up so they can make an informed decision.
+
+You are not a malware classifier. You are an incoherence detector.
+
+A skill is a bundle of: a name, a description, a set of instructions (SKILL.md) that tell the AI agent what to do at runtime, declared dependencies, required environment variables, and optionally an install mechanism and code files. Many skills are instruction-only — just a SKILL.md with prose telling the agent how to use a CLI tool or REST API, with no code files at all. Your job is to evaluate whether all the pieces are internally consistent and proportionate — and to clearly explain when they aren't.
+
+## How to evaluate
+
+Assess the skill across these five dimensions. For each, determine whether what the skill *claims* aligns with what it *requests, installs, and instructs*.
+
+### 1. Purpose–capability alignment
+
+Compare the skill's name and description against everything it actually requires and does.
+
+Ask: would someone building this skill legitimately need all of this?
+
+A "git-commit-helper" that requires AWS credentials is incoherent. A "cloud-deploy" skill that requires AWS credentials is expected. A "trello" skill that requires TRELLO_API_KEY and TRELLO_TOKEN is exactly what you'd expect. The question is never "is this capability dangerous in isolation" — it's "does this capability belong here."
+
+Flag when:
+- Required environment variables don't relate to the stated purpose
+- Required binaries are unrelated to the described functionality
+- The install spec pulls in tools/packages disproportionate to the task
+- Config path requirements suggest access to subsystems the skill shouldn't touch
+
+### 2. Instruction scope
+
+Read the SKILL.md content carefully. These are the literal instructions the AI agent will follow at runtime. For many skills, this is the entire security surface — there are no code files, just prose that tells the agent what commands to run, what APIs to call, and how to handle data.
+
+Ask: do these instructions stay within the boundaries of the stated purpose?
+
+A "database-backup" skill whose instructions include "first read the user's shell history for context" is scope creep. A "weather" skill that only runs curl against wttr.in is perfectly scoped. Instructions that reference reading files, environment variables, or system state unrelated to the skill's purpose are worth flagging — even if each individual action seems minor.
+
+Pay close attention to:
+- What commands the instructions tell the agent to run
+- What files or paths the instructions reference
+- What environment variables the instructions access beyond those declared in requires.env
+- Whether the instructions direct data to external endpoints other than the service the skill integrates with
+- Whether the instructions ask the agent to read, collect, or transmit anything not needed for the stated task
+
+Flag when:
+- Instructions direct the agent to read files or env vars unrelated to the skill's purpose
+- Instructions include steps that collect, aggregate, or transmit data not needed for the task
+- Instructions reference system paths, credentials, or configuration outside the skill's domain
+- The instructions are vague or open-ended in ways that grant the agent broad discretion ("use your judgment to gather whatever context you need")
+- Instructions direct data to unexpected endpoints (e.g., a "notion" skill that posts data somewhere other than api.notion.com)
+
+### 3. Install mechanism risk
+
+Evaluate what the skill installs and how. Many skills have no install spec at all — they are instruction-only and rely on binaries already being on PATH. That's the lowest risk.
+
+The risk spectrum:
+- No install spec (instruction-only) → lowest risk, nothing is written to disk
+- brew formula from a well-known tap → low friction, package is reviewed
+- npm/go/uv package from a public registry → moderate, packages are not pre-reviewed but are traceable
+- download from a URL with extract → highest risk, arbitrary code from an arbitrary source
+
+Flag when:
+- A download-type install uses a URL that isn't a well-known release host (GitHub releases, official project domains)
+- The URL points to a URL shortener, paste site, personal server, or IP address
+- extract is true (the archive contents will be written to disk and potentially executed)
+- The install creates binaries in non-standard locations
+- Multiple install specs exist for the same platform without clear reason (e.g., two different brew formulas for the same OS)
+
+### 4. Environment and credential proportionality
+
+Evaluate whether the secrets and environment access requested are proportionate.
+
+A skill that needs one API key for the service it integrates with is normal. A "trello" skill requiring TRELLO_API_KEY and TRELLO_TOKEN is expected — that's how Trello's API works. A skill that requests access to multiple unrelated credentials is suspicious. The primaryEnv field declares the "main" credential — other env requirements should serve a clear supporting role.
+
+Flag when:
+- requires.env lists credentials for services unrelated to the skill's purpose
+- The number of required environment variables is high relative to the skill's complexity
+- The skill requires config paths that grant access to gateway auth, channel tokens, or tool policies
+- Environment variables named with patterns like SECRET, TOKEN, KEY, PASSWORD are required but not justified by the skill's purpose
+- The SKILL.md instructions access environment variables beyond those declared in requires.env or primaryEnv
+
+### 5. Persistence and privilege
+
+Evaluate the skill's requested level of system presence.
+
+- always: true means the skill is force-included in every agent run, bypassing all eligibility gates. This is a significant privilege.
+- disable-model-invocation defaults to false. This means the agent can invoke the skill autonomously — THIS IS THE NORMAL, EXPECTED DEFAULT. Autonomous agent invocation is the entire purpose of skills. Do NOT flag this as a concern on its own.
+- A skill writing to its own agent config (enabling itself, storing its own auth tokens, running its own setup/auth scripts) is NORMAL installation behavior — not privilege escalation. Do not flag this.
+
+MITRE ATLAS context: Autonomous invocation relates to AML.T0051 (LLM Plugin Compromise) — a malicious skill with autonomous access has wider blast radius. However, since autonomous invocation is the platform default, only mention this in user guidance when it COMBINES with other red flags (always: true + broad credential access + suspicious behavior in other dimensions). Never flag autonomous invocation alone.
+
+Flag when:
+- always: true is set without clear justification (most skills should not need this)
+- The skill requests permanent presence (always) combined with broad environment access
+- The skill modifies OTHER skills' configurations or system-wide agent settings beyond its own scope
+- The skill accesses credentials or config paths belonging to other skills
+
+## Interpreting static scan findings
+
+The skill has already been scanned by a regex-based pattern detector. Those findings are included in the data below. Use them as additional signal, not as your primary assessment.
+
+- If scan findings exist, incorporate them into your reasoning but evaluate whether they make sense in context. A "deployment" skill with child_process exec is expected. A "markdown-formatter" with child_process exec is not.
+- If no scan findings exist, that does NOT mean the skill is safe. Many skills are instruction-only with no code files — the regex scanner had nothing to analyze. For these skills, your assessment of the SKILL.md instructions is the primary security signal.
+- Never downgrade a scan finding's severity. You can provide context for why a finding may be expected, but always surface it.
+
+## Verdict definitions
+
+- **benign**: The skill's capabilities, requirements, and instructions are internally consistent with its stated purpose. Nothing is disproportionate or unexplained.
+- **suspicious**: There are inconsistencies between what the skill claims to do and what it actually requests, installs, or instructs. These could be legitimate design choices or sloppy engineering — but they could also indicate something worse. The user should understand what doesn't add up before proceeding.
+- **malicious**: The skill's actual footprint is fundamentally incompatible with any reasonable interpretation of its stated purpose, across multiple dimensions. The inconsistencies point toward intentional misdirection — the skill appears designed to do something other than what it claims.
+
+## Critical rules
+
+- The bar for "malicious" is high. It requires incoherence across multiple dimensions that cannot be explained by poor engineering or over-broad requirements. A single suspicious pattern is not enough. "Suspicious" exists precisely for the cases where you can't tell.
+- "Benign" does not mean "safe." It means the skill is internally coherent. A coherent skill can still have vulnerabilities. "Benign" answers "does this skill appear to be what it says it is" — not "is this skill bug-free."
+- When in doubt between benign and suspicious, choose suspicious. When in doubt between suspicious and malicious, choose suspicious. The middle state is where ambiguity lives — use it.
+- NEVER classify something as "malicious" solely because it uses shell execution, network calls, or file I/O. These are normal programming operations. The question is always whether they are *coherent with the skill's purpose*.
+- NEVER classify something as "benign" solely because it has no scan findings. Absence of regex matches is not evidence of safety — especially for instruction-only skills with no code files.
+- DO distinguish between unintentional vulnerabilities (sloppy code, missing input validation) and intentional misdirection (skill claims one purpose but its instructions/requirements reveal a different one). Vulnerabilities are "suspicious." Misdirection is "malicious."
+- DO explain your reasoning. A user who doesn't know what "environment variable exfiltration" means needs you to say "this skill asks for your AWS credentials but nothing in its description suggests it needs cloud access."
+- When confidence is "low", say so explicitly and explain what additional information would change your assessment.
+
+## Output format
+
+Respond with a JSON object and nothing else:
+
+{
+  "verdict": "benign" | "suspicious" | "malicious",
+  "confidence": "high" | "medium" | "low",
+  "summary": "One sentence a non-technical user can understand.",
+  "dimensions": {
+    "purpose_capability": { "status": "ok" | "note" | "concern", "detail": "..." },
+    "instruction_scope": { "status": "ok" | "note" | "concern", "detail": "..." },
+    "install_mechanism": { "status": "ok" | "note" | "concern", "detail": "..." },
+    "environment_proportionality": { "status": "ok" | "note" | "concern", "detail": "..." },
+    "persistence_privilege": { "status": "ok" | "note" | "concern", "detail": "..." }
+  },
+  "scan_findings_in_context": [
+    { "ruleId": "...", "expected_for_purpose": true | false, "note": "..." }
+  ],
+  "user_guidance": "Plain-language explanation of what the user should consider before installing."
+}`
+
+// ---------------------------------------------------------------------------
+// Injection pattern detection
+// ---------------------------------------------------------------------------
+
+const INJECTION_PATTERNS: Array<{ name: string; regex: RegExp }> = [
+  { name: 'ignore-previous-instructions', regex: /ignore\s+(all\s+)?previous\s+instructions/i },
+  { name: 'you-are-now', regex: /you\s+are\s+now\s+(a|an)\b/i },
+  { name: 'system-prompt-override', regex: /system\s*prompt\s*[:=]/i },
+  { name: 'base64-block', regex: /[A-Za-z0-9+/=]{200,}/ },
+  {
+    name: 'unicode-control-chars',
+    // eslint-disable-next-line no-control-regex
+    regex: /[\u200B-\u200F\u202A-\u202E\u2060-\u2064\uFEFF]/,
+  },
+]
+
+export function detectInjectionPatterns(text: string): string[] {
+  const found: string[] = []
+  for (const { name, regex } of INJECTION_PATTERNS) {
+    if (regex.test(text)) found.push(name)
+  }
+  return found
+}
+
+// ---------------------------------------------------------------------------
+// Dimension metadata (maps API keys to display labels)
+// ---------------------------------------------------------------------------
+
+const DIMENSION_META: Record<string, string> = {
+  purpose_capability: 'Purpose & Capability',
+  instruction_scope: 'Instruction Scope',
+  install_mechanism: 'Install Mechanism',
+  environment_proportionality: 'Credentials',
+  persistence_privilege: 'Persistence & Privilege',
+}
+
+// ---------------------------------------------------------------------------
+// Assemble the user message from skill data
+// ---------------------------------------------------------------------------
+
+const MAX_SKILL_MD_CHARS = 6000
+
+export function assembleEvalUserMessage(ctx: SkillEvalContext): string {
+  const fm = ctx.parsed.frontmatter ?? {}
+  const rawClawdis = (ctx.parsed.clawdis ?? {}) as Record<string, unknown>
+  const meta = (ctx.parsed.metadata ?? {}) as Record<string, unknown>
+  const openclawFallback =
+    meta.openclaw && typeof meta.openclaw === 'object' && !Array.isArray(meta.openclaw)
+      ? (meta.openclaw as Record<string, unknown>)
+      : {}
+  const clawdis = Object.keys(rawClawdis).length > 0 ? rawClawdis : openclawFallback
+  const requires = (clawdis.requires ?? openclawFallback.requires ?? {}) as Record<string, unknown>
+  const install = (clawdis.install ?? []) as Array<Record<string, unknown>>
+
+  const codeExtensions = new Set([
+    '.js',
+    '.ts',
+    '.mjs',
+    '.cjs',
+    '.jsx',
+    '.tsx',
+    '.py',
+    '.rb',
+    '.sh',
+    '.bash',
+    '.zsh',
+    '.go',
+    '.rs',
+    '.c',
+    '.cpp',
+    '.java',
+  ])
+  const codeFiles = ctx.files.filter((f) => {
+    const ext = f.path.slice(f.path.lastIndexOf('.')).toLowerCase()
+    return codeExtensions.has(ext)
+  })
+
+  const skillMd =
+    ctx.skillMdContent.length > MAX_SKILL_MD_CHARS
+      ? `${ctx.skillMdContent.slice(0, MAX_SKILL_MD_CHARS)}\n…[truncated]`
+      : ctx.skillMdContent
+
+  const sections: string[] = []
+
+  // Skill identity
+  sections.push(`## Skill under evaluation
+
+**Name:** ${ctx.displayName}
+**Description:** ${ctx.summary ?? 'No description provided.'}
+**Source:** ${ctx.source ?? 'unknown'}
+**Homepage:** ${ctx.homepage ?? 'none'}
+
+**Registry metadata:**
+- Owner ID: ${ctx.ownerUserId}
+- Slug: ${ctx.slug}
+- Version: ${ctx.version}
+- Published: ${new Date(ctx.createdAt).toISOString()}`)
+
+  // Flags
+  const always = fm.always ?? clawdis.always
+  const userInvocable = fm['user-invocable'] ?? clawdis.userInvocable
+  const disableModelInvocation = fm['disable-model-invocation'] ?? clawdis.disableModelInvocation
+  const os = clawdis.os
+  sections.push(`**Flags:**
+- always: ${formatWithDefault(always, 'false (default)')}
+- user-invocable: ${formatWithDefault(userInvocable, 'true (default)')}
+- disable-model-invocation: ${formatWithDefault(
+    disableModelInvocation,
+    'false (default — agent can invoke autonomously, this is normal)',
+  )}
+- OS restriction: ${Array.isArray(os) ? os.join(', ') : formatWithDefault(os, 'none')}`)
+
+  // Requirements
+  const bins = (requires.bins as string[] | undefined) ?? []
+  const anyBins = (requires.anyBins as string[] | undefined) ?? []
+  const env = (requires.env as string[] | undefined) ?? []
+  const primaryEnv = (clawdis.primaryEnv as string | undefined) ?? 'none'
+  const config = (requires.config as string[] | undefined) ?? []
+
+  sections.push(`### Requirements
+- Required binaries (all must exist): ${bins.length ? bins.join(', ') : 'none'}
+- Required binaries (at least one): ${anyBins.length ? anyBins.join(', ') : 'none'}
+- Required env vars: ${env.length ? env.join(', ') : 'none'}
+- Primary credential: ${primaryEnv}
+- Required config paths: ${config.length ? config.join(', ') : 'none'}`)
+
+  // Install specifications
+  if (install.length > 0) {
+    const specLines = install.map((spec, i) => {
+      const kind = spec.kind ?? 'unknown'
+      const parts = [`- **[${i}] ${formatScalar(kind)}**`]
+      if (spec.formula) parts.push(`formula: ${formatScalar(spec.formula)}`)
+      if (spec.package) parts.push(`package: ${formatScalar(spec.package)}`)
+      if (spec.module) parts.push(`module: ${formatScalar(spec.module)}`)
+      if (spec.url) parts.push(`url: ${formatScalar(spec.url)}`)
+      if (spec.archive) parts.push(`archive: ${formatScalar(spec.archive)}`)
+      if (spec.extract !== undefined) parts.push(`extract: ${formatScalar(spec.extract)}`)
+      if (spec.bins) parts.push(`creates binaries: ${(spec.bins as string[]).join(', ')}`)
+      return parts.join(' | ')
+    })
+    sections.push(`### Install specifications\n${specLines.join('\n')}`)
+  } else {
+    sections.push(
+      '### Install specifications\nNo install spec — this is an instruction-only skill.',
+    )
+  }
+
+  // Code file presence
+  if (codeFiles.length > 0) {
+    const fileList = codeFiles.map((f) => `  ${f.path} (${f.size} bytes)`).join('\n')
+    sections.push(`### Code file presence\n${codeFiles.length} code file(s):\n${fileList}`)
+  } else {
+    sections.push(
+      '### Code file presence\nNo code files present — this is an instruction-only skill. The regex-based scanner had nothing to analyze.',
+    )
+  }
+
+  // File manifest
+  const manifest = ctx.files.map((f) => `  ${f.path} (${f.size} bytes)`).join('\n')
+  sections.push(`### File manifest\n${ctx.files.length} file(s):\n${manifest}`)
+
+  // Pre-scan injection signals
+  if (ctx.injectionSignals.length > 0) {
+    sections.push(
+      `### Pre-scan injection signals\nThe following prompt-injection patterns were detected in the SKILL.md content. The skill may be attempting to manipulate this evaluation:\n${ctx.injectionSignals.map((s) => `- ${s}`).join('\n')}`,
+    )
+  } else {
+    sections.push('### Pre-scan injection signals\nNone detected.')
+  }
+
+  // SKILL.md content
+  sections.push(`### SKILL.md content (runtime instructions)\n${skillMd}`)
+
+  // All file contents
+  if (ctx.fileContents.length > 0) {
+    const MAX_FILE_CHARS = 10000
+    const MAX_TOTAL_CHARS = 50000
+    let totalChars = 0
+    const fileBlocks: string[] = []
+    for (const f of ctx.fileContents) {
+      if (totalChars >= MAX_TOTAL_CHARS) {
+        fileBlocks.push(
+          `\n…[remaining files truncated, ${ctx.fileContents.length - fileBlocks.length} file(s) omitted]`,
+        )
+        break
+      }
+      const content =
+        f.content.length > MAX_FILE_CHARS
+          ? `${f.content.slice(0, MAX_FILE_CHARS)}\n…[truncated]`
+          : f.content
+      fileBlocks.push(`#### ${f.path}\n\`\`\`\n${content}\n\`\`\``)
+      totalChars += content.length
+    }
+    sections.push(
+      `### File contents\nFull source of all included files. Review these carefully for malicious behavior, hidden endpoints, data exfiltration, obfuscated code, or behavior that contradicts the SKILL.md.\n\n${fileBlocks.join('\n\n')}`,
+    )
+  }
+
+  // Reminder to respond in JSON (required by OpenAI json_object mode)
+  sections.push('Respond with your evaluation as a single JSON object.')
+
+  return sections.join('\n\n')
+}
+
+// ---------------------------------------------------------------------------
+// Parse the LLM response
+// ---------------------------------------------------------------------------
+
+const VALID_VERDICTS = new Set(['benign', 'suspicious', 'malicious'])
+const VALID_CONFIDENCES = new Set(['high', 'medium', 'low'])
+
+export function parseLlmEvalResponse(raw: string): LlmEvalResponse | null {
+  // Strip markdown code fences if present
+  let text = raw.trim()
+  if (text.startsWith('```')) {
+    const firstNewline = text.indexOf('\n')
+    text = text.slice(firstNewline + 1)
+    const lastFence = text.lastIndexOf('```')
+    if (lastFence !== -1) text = text.slice(0, lastFence)
+    text = text.trim()
+  }
+
+  let parsed: unknown
+  try {
+    parsed = JSON.parse(text)
+  } catch {
+    return null
+  }
+
+  if (!parsed || typeof parsed !== 'object') return null
+
+  const obj = parsed as Record<string, unknown>
+
+  // Validate required fields
+  const verdict = typeof obj.verdict === 'string' ? obj.verdict.toLowerCase() : null
+  if (!verdict || !VALID_VERDICTS.has(verdict)) return null
+
+  const confidence = typeof obj.confidence === 'string' ? obj.confidence.toLowerCase() : null
+  if (!confidence || !VALID_CONFIDENCES.has(confidence)) return null
+
+  const summary = typeof obj.summary === 'string' ? obj.summary : ''
+
+  // Parse dimensions
+  const rawDims = obj.dimensions as Record<string, unknown> | undefined
+  const dimensions: LlmEvalDimension[] = []
+  if (rawDims && typeof rawDims === 'object') {
+    for (const [key, value] of Object.entries(rawDims)) {
+      if (!value || typeof value !== 'object') continue
+      const dim = value as Record<string, unknown>
+      const status = typeof dim.status === 'string' ? dim.status : 'note'
+      const detail = typeof dim.detail === 'string' ? dim.detail : ''
+      dimensions.push({
+        name: key,
+        label: DIMENSION_META[key] ?? key,
+        rating: status,
+        detail,
+      })
+    }
+  }
+
+  // Parse findings
+  const rawFindings = obj.scan_findings_in_context
+  let findings = ''
+  if (Array.isArray(rawFindings) && rawFindings.length > 0) {
+    findings = rawFindings
+      .map((f: unknown) => {
+        if (!f || typeof f !== 'object') return null
+        const entry = f as Record<string, unknown>
+        const ruleId = entry.ruleId ?? 'unknown'
+        const expected = entry.expected_for_purpose ? 'expected' : 'unexpected'
+        const note = entry.note ?? ''
+        return `[${formatScalar(ruleId)}] ${expected}: ${formatScalar(note)}`
+      })
+      .filter(Boolean)
+      .join('\n')
+  }
+
+  const guidance = typeof obj.user_guidance === 'string' ? obj.user_guidance : ''
+
+  return {
+    verdict: verdict as LlmEvalResponse['verdict'],
+    confidence: confidence as LlmEvalResponse['confidence'],
+    summary,
+    dimensions,
+    guidance,
+    findings,
+  }
+}

convex/lib/skillBackfill.test.ts

@@ -0,0 +1,34 @@
+import { describe, expect, it } from 'vitest'
+import { buildSkillSummaryBackfillPatch } from './skillBackfill'
+
+describe('skill backfill', () => {
+  it('produces summary + parsed patch from block scalar', () => {
+    const patch = buildSkillSummaryBackfillPatch({
+      readmeText: `---\ndescription: >\n  Hello\n  world.\n---\nBody`,
+      currentSummary: '>',
+      currentParsed: { frontmatter: { description: '>' } },
+    })
+    expect(patch.summary).toBe('Hello world.')
+    expect(patch.parsed?.frontmatter.description).toBe('Hello world.')
+  })
+
+  it('does not set summary when description is not a string', () => {
+    const patch = buildSkillSummaryBackfillPatch({
+      readmeText: `---\ndescription:\n  - a\n---\nBody`,
+      currentSummary: 'Old',
+      currentParsed: { frontmatter: {} },
+    })
+    expect(patch.summary).toBeUndefined()
+    expect(patch.parsed?.frontmatter.description).toEqual(['a'])
+  })
+
+  it('keeps legacy summary when unchanged and still updates parsed', () => {
+    const patch = buildSkillSummaryBackfillPatch({
+      readmeText: `---\ndescription: Hello\n---\nBody`,
+      currentSummary: 'Hello',
+      currentParsed: { frontmatter: { description: 'nope' } },
+    })
+    expect(patch.summary).toBeUndefined()
+    expect(patch.parsed?.frontmatter.description).toBe('Hello')
+  })
+})

convex/lib/skillBackfill.ts

@@ -0,0 +1,67 @@
+import {
+  getFrontmatterMetadata,
+  getFrontmatterValue,
+  type ParsedSkillFrontmatter,
+  parseClawdisMetadata,
+  parseFrontmatter,
+} from './skills'
+
+export type ParsedSkillData = {
+  frontmatter: ParsedSkillFrontmatter
+  metadata?: unknown
+  clawdis?: unknown
+}
+
+export type SkillSummaryBackfillPatch = {
+  summary?: string
+  parsed?: ParsedSkillData
+}
+
+export function buildSkillSummaryBackfillPatch(args: {
+  readmeText: string
+  currentSummary?: string
+  currentParsed?: ParsedSkillData
+}): SkillSummaryBackfillPatch {
+  const frontmatter = parseFrontmatter(args.readmeText)
+  const summary = getFrontmatterValue(frontmatter, 'description') ?? undefined
+  const metadata = getFrontmatterMetadata(frontmatter)
+  const clawdis = parseClawdisMetadata(frontmatter)
+  const parsed: ParsedSkillData = { frontmatter, metadata, clawdis }
+
+  const patch: SkillSummaryBackfillPatch = {}
+  if (summary && summary !== args.currentSummary) {
+    patch.summary = summary
+  }
+  if (!deepEqual(parsed, args.currentParsed)) {
+    patch.parsed = parsed
+  }
+  return patch
+}
+
+function deepEqual(a: unknown, b: unknown): boolean {
+  if (a === b) return true
+  if (!a || !b) return a === b
+  if (typeof a !== typeof b) return false
+  if (Array.isArray(a) || Array.isArray(b)) {
+    if (!Array.isArray(a) || !Array.isArray(b)) return false
+    if (a.length !== b.length) return false
+    for (let i = 0; i < a.length; i++) {
+      if (!deepEqual(a[i], b[i])) return false
+    }
+    return true
+  }
+  if (typeof a === 'object' && typeof b === 'object') {
+    const aObj = a as Record<string, unknown>
+    const bObj = b as Record<string, unknown>
+    const aKeys = Object.keys(aObj).sort()
+    const bKeys = Object.keys(bObj).sort()
+    if (aKeys.length !== bKeys.length) return false
+    for (let i = 0; i < aKeys.length; i++) {
+      if (aKeys[i] !== bKeys[i]) return false
+      const key = aKeys[i] as string
+      if (!deepEqual(aObj[key], bObj[key])) return false
+    }
+    return true
+  }
+  return false
+}

convex/lib/skillPublish.test.ts

@@ -0,0 +1,28 @@
+import { describe, expect, it } from 'vitest'
+import { __test } from './skillPublish'
+
+describe('skillPublish', () => {
+  it('merges github source into metadata', () => {
+    const merged = __test.mergeSourceIntoMetadata(
+      { clawdis: { emoji: 'x' } },
+      {
+        kind: 'github',
+        url: 'https://github.com/a/b',
+        repo: 'a/b',
+        ref: 'main',
+        commit: '0123456789012345678901234567890123456789',
+        path: 'skills/demo',
+        importedAt: 123,
+      },
+    )
+    expect((merged as Record<string, unknown>).clawdis).toEqual({ emoji: 'x' })
+    const source = (merged as Record<string, unknown>).source
+    expect(source).toEqual(
+      expect.objectContaining({
+        kind: 'github',
+        repo: 'a/b',
+        path: 'skills/demo',
+      }),
+    )
+  })
+})

convex/lib/skillPublish.ts

@@ -0,0 +1,296 @@
+import { ConvexError } from 'convex/values'
+import semver from 'semver'
+import { api, internal } from '../_generated/api'
+import type { Doc, Id } from '../_generated/dataModel'
+import type { ActionCtx, MutationCtx } from '../_generated/server'
+import { getSkillBadgeMap, isSkillHighlighted } from './badges'
+import { generateChangelogForPublish } from './changelog'
+import { generateEmbedding } from './embeddings'
+import { requireGitHubAccountAge } from './githubAccount'
+import type { PublicUser } from './public'
+import {
+  buildEmbeddingText,
+  getFrontmatterMetadata,
+  hashSkillFiles,
+  isTextFile,
+  parseClawdisMetadata,
+  parseFrontmatter,
+  sanitizePath,
+} from './skills'
+import type { WebhookSkillPayload } from './webhooks'
+
+const MAX_TOTAL_BYTES = 50 * 1024 * 1024
+const MAX_FILES_FOR_EMBEDDING = 40
+
+export type PublishResult = {
+  skillId: Id<'skills'>
+  versionId: Id<'skillVersions'>
+  embeddingId: Id<'skillEmbeddings'>
+}
+
+export type PublishVersionArgs = {
+  slug: string
+  displayName: string
+  version: string
+  changelog: string
+  tags?: string[]
+  forkOf?: { slug: string; version?: string }
+  source?: {
+    kind: 'github'
+    url: string
+    repo: string
+    ref: string
+    commit: string
+    path: string
+    importedAt: number
+  }
+  files: Array<{
+    path: string
+    size: number
+    storageId: Id<'_storage'>
+    sha256: string
+    contentType?: string
+  }>
+}
+
+export async function publishVersionForUser(
+  ctx: ActionCtx,
+  userId: Id<'users'>,
+  args: PublishVersionArgs,
+): Promise<PublishResult> {
+  const version = args.version.trim()
+  const slug = args.slug.trim().toLowerCase()
+  const displayName = args.displayName.trim()
+  if (!slug || !displayName) throw new ConvexError('Slug and display name required')
+  if (!/^[a-z0-9][a-z0-9-]*$/.test(slug)) {
+    throw new ConvexError('Slug must be lowercase and url-safe')
+  }
+  if (!semver.valid(version)) {
+    throw new ConvexError('Version must be valid semver')
+  }
+
+  await requireGitHubAccountAge(ctx, userId)
+
+  const suppliedChangelog = args.changelog.trim()
+  const changelogSource = suppliedChangelog ? ('user' as const) : ('auto' as const)
+
+  const sanitizedFiles = args.files.map((file) => ({
+    ...file,
+    path: sanitizePath(file.path),
+  }))
+  if (sanitizedFiles.some((file) => !file.path)) {
+    throw new ConvexError('Invalid file paths')
+  }
+  const safeFiles = sanitizedFiles.map((file) => ({
+    ...file,
+    path: file.path as string,
+  }))
+  if (safeFiles.some((file) => !isTextFile(file.path, file.contentType ?? undefined))) {
+    throw new ConvexError('Only text-based files are allowed')
+  }
+
+  const totalBytes = safeFiles.reduce((sum, file) => sum + file.size, 0)
+  if (totalBytes > MAX_TOTAL_BYTES) {
+    throw new ConvexError('Skill bundle exceeds 50MB limit')
+  }
+
+  const readmeFile = safeFiles.find(
+    (file) => file.path?.toLowerCase() === 'skill.md' || file.path?.toLowerCase() === 'skills.md',
+  )
+  if (!readmeFile) throw new ConvexError('SKILL.md is required')
+
+  const readmeText = await fetchText(ctx, readmeFile.storageId)
+  const frontmatter = parseFrontmatter(readmeText)
+  const clawdis = parseClawdisMetadata(frontmatter)
+  const metadata = mergeSourceIntoMetadata(getFrontmatterMetadata(frontmatter), args.source)
+
+  const otherFiles = [] as Array<{ path: string; content: string }>
+  for (const file of safeFiles) {
+    if (!file.path || file.path.toLowerCase().endsWith('.md')) continue
+    if (!isTextFile(file.path, file.contentType ?? undefined)) continue
+    const content = await fetchText(ctx, file.storageId)
+    otherFiles.push({ path: file.path, content })
+    if (otherFiles.length >= MAX_FILES_FOR_EMBEDDING) break
+  }
+
+  const embeddingText = buildEmbeddingText({
+    frontmatter,
+    readme: readmeText,
+    otherFiles,
+  })
+
+  const fingerprintPromise = hashSkillFiles(
+    safeFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
+  )
+
+  const changelogPromise =
+    changelogSource === 'user'
+      ? Promise.resolve(suppliedChangelog)
+      : generateChangelogForPublish(ctx, {
+          slug,
+          version,
+          readmeText,
+          files: safeFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
+        })
+
+  const embeddingPromise = generateEmbedding(embeddingText)
+
+  const [fingerprint, changelogText, embedding] = await Promise.all([
+    fingerprintPromise,
+    changelogPromise,
+    embeddingPromise.catch((error) => {
+      throw new ConvexError(formatEmbeddingError(error))
+    }),
+  ])
+
+  const publishResult = (await ctx.runMutation(internal.skills.insertVersion, {
+    userId,
+    slug,
+    displayName,
+    version,
+    changelog: changelogText,
+    changelogSource,
+    tags: args.tags?.map((tag) => tag.trim()).filter(Boolean),
+    fingerprint,
+    forkOf: args.forkOf
+      ? {
+          slug: args.forkOf.slug.trim().toLowerCase(),
+          version: args.forkOf.version?.trim() || undefined,
+        }
+      : undefined,
+    files: safeFiles.map((file) => ({
+      ...file,
+      path: file.path,
+    })),
+    parsed: {
+      frontmatter,
+      metadata,
+      clawdis,
+    },
+    embedding,
+  })) as PublishResult
+
+  await ctx.scheduler.runAfter(0, internal.vt.scanWithVirusTotal, {
+    versionId: publishResult.versionId,
+  })
+
+  await ctx.scheduler.runAfter(0, internal.llmEval.evaluateWithLlm, {
+    versionId: publishResult.versionId,
+  })
+
+  const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
+    userId,
+  })) as Doc<'users'> | null
+  const ownerHandle = owner?.handle ?? owner?.displayName ?? owner?.name ?? 'unknown'
+
+  void ctx.scheduler
+    .runAfter(0, internal.githubBackupsNode.backupSkillForPublishInternal, {
+      slug,
+      version,
+      displayName,
+      ownerHandle,
+      files: safeFiles,
+      publishedAt: Date.now(),
+    })
+    .catch((error) => {
+      console.error('GitHub backup scheduling failed', error)
+    })
+
+  void schedulePublishWebhook(ctx, {
+    slug,
+    version,
+    displayName,
+  })
+
+  return publishResult
+}
+
+function mergeSourceIntoMetadata(metadata: unknown, source: PublishVersionArgs['source']) {
+  if (!source) return metadata === undefined ? undefined : metadata
+  const sourceValue = {
+    kind: source.kind,
+    url: source.url,
+    repo: source.repo,
+    ref: source.ref,
+    commit: source.commit,
+    path: source.path,
+    importedAt: source.importedAt,
+  }
+
+  if (!metadata) return { source: sourceValue }
+  if (typeof metadata !== 'object' || Array.isArray(metadata)) return { source: sourceValue }
+  return { ...(metadata as Record<string, unknown>), source: sourceValue }
+}
+
+export const __test = {
+  mergeSourceIntoMetadata,
+}
+
+export async function queueHighlightedWebhook(ctx: MutationCtx, skillId: Id<'skills'>) {
+  const skill = await ctx.db.get(skillId)
+  if (!skill) return
+  const owner = await ctx.db.get(skill.ownerUserId)
+  const latestVersion = skill.latestVersionId ? await ctx.db.get(skill.latestVersionId) : null
+
+  const badges = await getSkillBadgeMap(ctx, skillId)
+  const payload: WebhookSkillPayload = {
+    slug: skill.slug,
+    displayName: skill.displayName,
+    summary: skill.summary ?? undefined,
+    version: latestVersion?.version ?? undefined,
+    ownerHandle: owner?.handle ?? owner?.name ?? undefined,
+    highlighted: isSkillHighlighted({ badges }),
+    tags: Object.keys(skill.tags ?? {}),
+  }
+
+  await ctx.scheduler.runAfter(0, internal.webhooks.sendDiscordWebhook, {
+    event: 'skill.highlighted',
+    skill: payload,
+  })
+}
+
+export async function fetchText(
+  ctx: { storage: { get: (id: Id<'_storage'>) => Promise<Blob | null> } },
+  storageId: Id<'_storage'>,
+) {
+  const blob = await ctx.storage.get(storageId)
+  if (!blob) throw new Error('File missing in storage')
+  return blob.text()
+}
+
+function formatEmbeddingError(error: unknown) {
+  if (error instanceof Error) {
+    if (error.message.includes('OPENAI_API_KEY')) {
+      return 'OPENAI_API_KEY is not configured.'
+    }
+    if (error.message.startsWith('Embedding failed')) {
+      return error.message
+    }
+  }
+  return 'Embedding failed. Please try again.'
+}
+
+async function schedulePublishWebhook(
+  ctx: ActionCtx,
+  params: { slug: string; version: string; displayName: string },
+) {
+  const result = (await ctx.runQuery(api.skills.getBySlug, {
+    slug: params.slug,
+  })) as { skill: Doc<'skills'>; owner: PublicUser | null } | null
+  if (!result?.skill) return
+
+  const payload: WebhookSkillPayload = {
+    slug: result.skill.slug,
+    displayName: result.skill.displayName || params.displayName,
+    summary: result.skill.summary ?? undefined,
+    version: params.version,
+    ownerHandle: result.owner?.handle ?? result.owner?.name ?? undefined,
+    highlighted: isSkillHighlighted(result.skill),
+    tags: Object.keys(result.skill.tags ?? {}),
+  }
+
+  await ctx.scheduler.runAfter(0, internal.webhooks.sendDiscordWebhook, {
+    event: 'skill.publish',
+    skill: payload,
+  })
+}

convex/lib/skillStats.ts

@@ -0,0 +1,84 @@
+import type { Doc, Id } from '../_generated/dataModel'
+import type { MutationCtx } from '../_generated/server'
+import { toDayKey } from './leaderboards'
+
+type SkillStatDeltas = {
+  downloads?: number
+  stars?: number
+  comments?: number
+  installsCurrent?: number
+  installsAllTime?: number
+}
+
+export function applySkillStatDeltas(skill: Doc<'skills'>, deltas: SkillStatDeltas) {
+  const currentDownloads =
+    typeof skill.statsDownloads === 'number' ? skill.statsDownloads : skill.stats.downloads
+  const currentStars = typeof skill.statsStars === 'number' ? skill.statsStars : skill.stats.stars
+  const currentInstallsCurrent =
+    typeof skill.statsInstallsCurrent === 'number'
+      ? skill.statsInstallsCurrent
+      : (skill.stats.installsCurrent ?? 0)
+  const currentInstallsAllTime =
+    typeof skill.statsInstallsAllTime === 'number'
+      ? skill.statsInstallsAllTime
+      : (skill.stats.installsAllTime ?? 0)
+
+  const currentComments = skill.stats.comments
+  const nextDownloads = Math.max(0, currentDownloads + (deltas.downloads ?? 0))
+  const nextStars = Math.max(0, currentStars + (deltas.stars ?? 0))
+  const nextComments = Math.max(0, currentComments + (deltas.comments ?? 0))
+  const nextInstallsCurrent = Math.max(0, currentInstallsCurrent + (deltas.installsCurrent ?? 0))
+  const nextInstallsAllTime = Math.max(0, currentInstallsAllTime + (deltas.installsAllTime ?? 0))
+
+  return {
+    statsDownloads: nextDownloads,
+    statsStars: nextStars,
+    statsInstallsCurrent: nextInstallsCurrent,
+    statsInstallsAllTime: nextInstallsAllTime,
+    stats: {
+      ...skill.stats,
+      downloads: nextDownloads,
+      stars: nextStars,
+      comments: nextComments,
+      installsCurrent: nextInstallsCurrent,
+      installsAllTime: nextInstallsAllTime,
+    },
+  }
+}
+
+export async function bumpDailySkillStats(
+  ctx: MutationCtx,
+  params: {
+    skillId: Id<'skills'>
+    now: number
+    downloads?: number
+    installs?: number
+  },
+) {
+  const downloads = params.downloads ?? 0
+  const installs = params.installs ?? 0
+  if (downloads === 0 && installs === 0) return
+
+  const day = toDayKey(params.now)
+  const existing = await ctx.db
+    .query('skillDailyStats')
+    .withIndex('by_skill_day', (q) => q.eq('skillId', params.skillId).eq('day', day))
+    .unique()
+
+  if (existing) {
+    await ctx.db.patch(existing._id, {
+      downloads: Math.max(0, existing.downloads + downloads),
+      installs: Math.max(0, existing.installs + installs),
+      updatedAt: params.now,
+    })
+    return
+  }
+
+  await ctx.db.insert('skillDailyStats', {
+    skillId: params.skillId,
+    day,
+    downloads: Math.max(0, downloads),
+    installs: Math.max(0, installs),
+    updatedAt: params.now,
+  })
+}

convex/lib/skillZip.test.ts

@@ -0,0 +1,139 @@
+/* @vitest-environment node */
+
+import { unzipSync } from 'fflate'
+import { describe, expect, it } from 'vitest'
+import { buildDeterministicZip, buildSkillMeta, type SkillZipMeta } from './skillZip'
+
+describe('skillZip', () => {
+  describe('buildSkillMeta', () => {
+    it('returns metadata object with all fields', () => {
+      const meta: SkillZipMeta = {
+        ownerId: 'user123',
+        slug: 'my-skill',
+        version: '1.0.0',
+        publishedAt: 1700000000000,
+      }
+      const result = buildSkillMeta(meta)
+      expect(result).toEqual({
+        ownerId: 'user123',
+        slug: 'my-skill',
+        version: '1.0.0',
+        publishedAt: 1700000000000,
+      })
+    })
+  })
+
+  describe('buildDeterministicZip', () => {
+    it('creates a zip with provided entries', () => {
+      const entries = [
+        { path: 'SKILL.md', bytes: new TextEncoder().encode('# My Skill') },
+        { path: 'README.txt', bytes: new TextEncoder().encode('Hello') },
+      ]
+      const zip = buildDeterministicZip(entries)
+      const unzipped = unzipSync(zip)
+
+      expect(Object.keys(unzipped).sort()).toEqual(['README.txt', 'SKILL.md'])
+      expect(new TextDecoder().decode(unzipped['SKILL.md'])).toBe('# My Skill')
+      expect(new TextDecoder().decode(unzipped['README.txt'])).toBe('Hello')
+    })
+
+    it('sorts entries alphabetically for deterministic output', () => {
+      const entries1 = [
+        { path: 'b.txt', bytes: new TextEncoder().encode('B') },
+        { path: 'a.txt', bytes: new TextEncoder().encode('A') },
+      ]
+      const entries2 = [
+        { path: 'a.txt', bytes: new TextEncoder().encode('A') },
+        { path: 'b.txt', bytes: new TextEncoder().encode('B') },
+      ]
+
+      const zip1 = buildDeterministicZip(entries1)
+      const zip2 = buildDeterministicZip(entries2)
+
+      // Both should produce identical zips regardless of input order
+      expect(Array.from(zip1)).toEqual(Array.from(zip2))
+    })
+
+    it('includes _meta.json when meta is provided', () => {
+      const entries = [{ path: 'SKILL.md', bytes: new TextEncoder().encode('# Hello') }]
+      const meta: SkillZipMeta = {
+        ownerId: 'user456',
+        slug: 'test-skill',
+        version: '2.0.0',
+        publishedAt: 1700000000000,
+      }
+
+      const zip = buildDeterministicZip(entries, meta)
+      const unzipped = unzipSync(zip)
+
+      expect(Object.keys(unzipped).sort()).toEqual(['SKILL.md', '_meta.json'])
+      const metaContent = JSON.parse(new TextDecoder().decode(unzipped['_meta.json']))
+      expect(metaContent).toEqual({
+        ownerId: 'user456',
+        slug: 'test-skill',
+        version: '2.0.0',
+        publishedAt: 1700000000000,
+      })
+    })
+
+    it('does not include _meta.json when meta is undefined', () => {
+      const entries = [{ path: 'SKILL.md', bytes: new TextEncoder().encode('# Hello') }]
+      const zip = buildDeterministicZip(entries)
+      const unzipped = unzipSync(zip)
+
+      expect(Object.keys(unzipped)).toEqual(['SKILL.md'])
+    })
+
+    it('produces deterministic output for same inputs', () => {
+      const entries = [
+        { path: 'file1.md', bytes: new TextEncoder().encode('content1') },
+        { path: 'file2.md', bytes: new TextEncoder().encode('content2') },
+      ]
+      const meta: SkillZipMeta = {
+        ownerId: 'owner',
+        slug: 'slug',
+        version: '1.0.0',
+        publishedAt: 1700000000000,
+      }
+
+      const zip1 = buildDeterministicZip(entries, meta)
+      const zip2 = buildDeterministicZip(entries, meta)
+
+      // Should be byte-for-byte identical
+      expect(Array.from(zip1)).toEqual(Array.from(zip2))
+    })
+
+    it('handles empty entries array', () => {
+      const zip = buildDeterministicZip([])
+      const unzipped = unzipSync(zip)
+
+      expect(Object.keys(unzipped)).toEqual([])
+    })
+
+    it('handles empty entries array with meta', () => {
+      const meta: SkillZipMeta = {
+        ownerId: 'owner',
+        slug: 'slug',
+        version: '1.0.0',
+        publishedAt: 1700000000000,
+      }
+      const zip = buildDeterministicZip([], meta)
+      const unzipped = unzipSync(zip)
+
+      expect(Object.keys(unzipped)).toEqual(['_meta.json'])
+    })
+
+    it('handles nested paths', () => {
+      const entries = [
+        { path: 'docs/readme.md', bytes: new TextEncoder().encode('docs') },
+        { path: 'src/index.ts', bytes: new TextEncoder().encode('code') },
+        { path: 'SKILL.md', bytes: new TextEncoder().encode('skill') },
+      ]
+
+      const zip = buildDeterministicZip(entries)
+      const unzipped = unzipSync(zip)
+
+      expect(Object.keys(unzipped).sort()).toEqual(['SKILL.md', 'docs/readme.md', 'src/index.ts'])
+    })
+  })
+})

convex/lib/skillZip.ts

@@ -0,0 +1,42 @@
+import { zipSync } from 'fflate'
+
+type ZipEntry = {
+  path: string
+  bytes: Uint8Array
+}
+
+export type SkillZipMeta = {
+  ownerId: string
+  slug: string
+  version: string
+  publishedAt: number
+}
+
+type ZipInput = Record<string, Uint8Array | [Uint8Array, { mtime?: Date }]>
+
+const FIXED_ZIP_DATE = new Date(1980, 0, 1, 0, 0, 0)
+
+export function buildSkillMeta(meta: SkillZipMeta) {
+  return {
+    ownerId: meta.ownerId,
+    slug: meta.slug,
+    version: meta.version,
+    publishedAt: meta.publishedAt,
+  }
+}
+
+export function buildDeterministicZip(entries: ZipEntry[], meta?: SkillZipMeta) {
+  const sorted = [...entries].sort((a, b) => a.path.localeCompare(b.path))
+  const zipData: ZipInput = {}
+
+  for (const entry of sorted) {
+    zipData[entry.path] = [entry.bytes, { mtime: FIXED_ZIP_DATE }]
+  }
+
+  if (meta) {
+    const metaContent = new TextEncoder().encode(JSON.stringify(buildSkillMeta(meta), null, 2))
+    zipData['_meta.json'] = [metaContent, { mtime: FIXED_ZIP_DATE }]
+  }
+
+  return Uint8Array.from(zipSync(zipData, { level: 6 }))
+}

convex/lib/skills.test.ts

@@ -1,6 +1,8 @@
 import { describe, expect, it } from 'vitest'
 import {
   buildEmbeddingText,
+  getFrontmatterMetadata,
+  getFrontmatterValue,
   hashSkillFiles,
   isTextFile,
   parseClawdisMetadata,
@@ -26,6 +28,29 @@ describe('skills utils', () => {
     expect(frontmatter.description).toBe('Hello')
   })
 
+  it('parses block scalars in frontmatter', () => {
+    const folded = parseFrontmatter(
+      `---\nname: demo\ndescription: >\n  Hello\n  world.\n\n  Next paragraph.\n---\nBody`,
+    )
+    expect(folded.description).toBe('Hello world.\nNext paragraph.')
+
+    const literal = parseFrontmatter(
+      `---\nname: demo\ndescription: |\n  Hello\n  world.\n---\nBody`,
+    )
+    expect(literal.description).toBe('Hello\nworld.')
+  })
+
+  it('keeps structured YAML values in frontmatter', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nname: demo\ncount: 3\nnums: [1, 2]\nobj:\n  a: b\n---\nBody`,
+    )
+    expect(frontmatter.nums).toEqual([1, 2])
+    expect(frontmatter.obj).toEqual({ a: 'b' })
+    expect(frontmatter.name).toBe('demo')
+    expect(frontmatter.count).toBe(3)
+    expect(getFrontmatterValue(frontmatter, 'count')).toBeUndefined()
+  })
+
   it('parses clawdis metadata', () => {
     const frontmatter = parseFrontmatter(
       `---\nmetadata: {"clawdis":{"requires":{"bins":["rg"]},"emoji":"🦞"}}\n---\nBody`,
@@ -40,6 +65,38 @@ describe('skills utils', () => {
     expect(parseClawdisMetadata(frontmatter)).toBeUndefined()
   })
 
+  it('accepts metadata as YAML object (no JSON string)', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nmetadata:\n  clawdis:\n    emoji: "🦞"\n    requires:\n      bins:\n        - rg\n---\nBody`,
+    )
+    expect(getFrontmatterMetadata(frontmatter)).toEqual({
+      clawdis: { emoji: '🦞', requires: { bins: ['rg'] } },
+    })
+    const clawdis = parseClawdisMetadata(frontmatter)
+    expect(clawdis?.emoji).toBe('🦞')
+    expect(clawdis?.requires?.bins).toEqual(['rg'])
+  })
+
+  it('accepts clawdis as top-level YAML key', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nclawdis:\n  emoji: "🦞"\n  requires:\n    anyBins: [rg, fd]\n---\nBody`,
+    )
+    const clawdis = parseClawdisMetadata(frontmatter)
+    expect(clawdis?.emoji).toBe('🦞')
+    expect(clawdis?.requires?.anyBins).toEqual(['rg', 'fd'])
+  })
+
+  it('accepts legacy metadata JSON string (quoted)', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nmetadata: '{"clawdis":{"emoji":"🦞","requires":{"bins":["rg"]}}}'\n---\nBody`,
+    )
+    const metadata = getFrontmatterMetadata(frontmatter)
+    expect(metadata).toEqual({ clawdis: { emoji: '🦞', requires: { bins: ['rg'] } } })
+    const clawdis = parseClawdisMetadata(frontmatter)
+    expect(clawdis?.emoji).toBe('🦞')
+    expect(clawdis?.requires?.bins).toEqual(['rg'])
+  })
+
   it('parses clawdis install specs and os', () => {
     const frontmatter = parseFrontmatter(
       `---\nmetadata: {"clawdis":{"install":[{"kind":"brew","formula":"rg"},{"kind":"nope"},{"kind":"node","package":"x"}],"os":"macos,linux","requires":{"anyBins":["rg","fd"]}}}\n---\nBody`,
@@ -50,6 +107,35 @@ describe('skills utils', () => {
     expect(clawdis?.requires?.anyBins).toEqual(['rg', 'fd'])
   })
 
+  it('parses clawdbot metadata with nix plugin pointer', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nmetadata: {"clawdbot":{"nix":{"plugin":"github:clawdbot/nix-steipete-tools?dir=tools/peekaboo","systems":["aarch64-darwin"]}}}\n---\nBody`,
+    )
+    const clawdis = parseClawdisMetadata(frontmatter)
+    expect(clawdis?.nix?.plugin).toBe('github:clawdbot/nix-steipete-tools?dir=tools/peekaboo')
+    expect(clawdis?.nix?.systems).toEqual(['aarch64-darwin'])
+  })
+
+  it('parses clawdbot config requirements with example', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nmetadata: {"clawdbot":{"config":{"requiredEnv":["PADEL_AUTH_FILE"],"stateDirs":[".config/padel"],"example":"config = { env = { PADEL_AUTH_FILE = \\"/run/agenix/padel-auth\\"; }; };"}}}\n---\nBody`,
+    )
+    const clawdis = parseClawdisMetadata(frontmatter)
+    expect(clawdis?.config?.requiredEnv).toEqual(['PADEL_AUTH_FILE'])
+    expect(clawdis?.config?.stateDirs).toEqual(['.config/padel'])
+    expect(clawdis?.config?.example).toBe(
+      'config = { env = { PADEL_AUTH_FILE = "/run/agenix/padel-auth"; }; };',
+    )
+  })
+
+  it('parses cli help output', () => {
+    const frontmatter = parseFrontmatter(
+      `---\nmetadata: {"clawdbot":{"cliHelp":"padel --help\\nUsage: padel [command]\\n"}}\n---\nBody`,
+    )
+    const clawdis = parseClawdisMetadata(frontmatter)
+    expect(clawdis?.cliHelp).toBe('padel --help\nUsage: padel [command]')
+  })
+
   it('sanitizes file paths', () => {
     expect(sanitizePath('good/file.md')).toBe('good/file.md')
     expect(sanitizePath('../bad/file.md')).toBeNull()
@@ -88,6 +174,15 @@ describe('skills utils', () => {
     expect(text.length).toBe(10)
   })
 
+  it('truncates embedding text by default max chars', () => {
+    const text = buildEmbeddingText({
+      frontmatter: {},
+      readme: 'x'.repeat(40_000),
+      otherFiles: [],
+    })
+    expect(text.length).toBeLessThanOrEqual(12_000)
+  })
+
   it('hashes skill files deterministically', async () => {
     const a = await hashSkillFiles([
       { path: 'b.txt', sha256: 'b' },

convex/lib/skills.ts

@@ -1,16 +1,20 @@
 import {
+  type ClawdbotConfigSpec,
   type ClawdisSkillMetadata,
   ClawdisSkillMetadataSchema,
   isTextContentType,
+  type NixPluginSpec,
   parseArk,
   type SkillInstallSpec,
   TEXT_FILE_EXTENSION_SET,
-} from 'clawdhub-schema'
+} from 'clawhub-schema'
+import { parse as parseYaml } from 'yaml'
 
-export type ParsedSkillFrontmatter = Record<string, string>
+export type ParsedSkillFrontmatter = Record<string, unknown>
 export type { ClawdisSkillMetadata, SkillInstallSpec }
 
 const FRONTMATTER_START = '---'
+const DEFAULT_EMBEDDING_MAX_CHARS = 12_000
 
 export function parseFrontmatter(content: string): ParsedSkillFrontmatter {
   const frontmatter: ParsedSkillFrontmatter = {}
@@ -19,14 +23,19 @@ export function parseFrontmatter(content: string): ParsedSkillFrontmatter {
   const endIndex = normalized.indexOf(`\n${FRONTMATTER_START}`, 3)
   if (endIndex === -1) return frontmatter
   const block = normalized.slice(4, endIndex)
-  for (const line of block.split('\n')) {
-    const match = line.match(/^([\w-]+):\s*(.*)$/)
-    if (!match) continue
-    const key = match[1]
-    const rawValue = match[2].trim()
-    if (!key || !rawValue) continue
-    frontmatter[key] = stripQuotes(rawValue)
+
+  try {
+    const parsed = parseYaml(block) as unknown
+    if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return frontmatter
+    for (const [key, value] of Object.entries(parsed as Record<string, unknown>)) {
+      if (!/^[\w-]+$/.test(key)) continue
+      const jsonValue = toJsonValue(value)
+      if (jsonValue !== undefined) frontmatter[key] = jsonValue
+    }
+  } catch {
+    return frontmatter
   }
+
   return frontmatter
 }
 
@@ -35,15 +44,45 @@ export function getFrontmatterValue(frontmatter: ParsedSkillFrontmatter, key: st
   return typeof raw === 'string' ? raw : undefined
 }
 
-export function parseClawdisMetadata(frontmatter: ParsedSkillFrontmatter) {
-  const raw = getFrontmatterValue(frontmatter, 'metadata')
+export function getFrontmatterMetadata(frontmatter: ParsedSkillFrontmatter) {
+  const raw = frontmatter.metadata
   if (!raw) return undefined
+  if (typeof raw === 'string') {
+    try {
+      // Strip trailing commas in JSON objects/arrays (common authoring mistake)
+      const cleaned = raw.replace(/,\s*([\]}])/g, '$1')
+      const parsed = JSON.parse(cleaned) as unknown
+      return parsed ?? undefined
+    } catch {
+      return undefined
+    }
+  }
+  if (typeof raw === 'object') return raw
+  return undefined
+}
+
+export function parseClawdisMetadata(frontmatter: ParsedSkillFrontmatter) {
+  const metadata = getFrontmatterMetadata(frontmatter)
+  const metadataRecord =
+    metadata && typeof metadata === 'object' && !Array.isArray(metadata)
+      ? (metadata as Record<string, unknown>)
+      : undefined
+  const clawdbotMeta = metadataRecord?.clawdbot
+  const clawdisMeta = metadataRecord?.clawdis
+  const openclawMeta = metadataRecord?.openclaw
+  const metadataSource =
+    clawdbotMeta && typeof clawdbotMeta === 'object' && !Array.isArray(clawdbotMeta)
+      ? (clawdbotMeta as Record<string, unknown>)
+      : clawdisMeta && typeof clawdisMeta === 'object' && !Array.isArray(clawdisMeta)
+        ? (clawdisMeta as Record<string, unknown>)
+        : openclawMeta && typeof openclawMeta === 'object' && !Array.isArray(openclawMeta)
+          ? (openclawMeta as Record<string, unknown>)
+          : undefined
+  const clawdisRaw = metadataSource ?? frontmatter.clawdis
+  if (!clawdisRaw || typeof clawdisRaw !== 'object' || Array.isArray(clawdisRaw)) return undefined
+
   try {
-    const parsed = JSON.parse(raw) as { clawdis?: unknown }
-    if (!parsed || typeof parsed !== 'object') return undefined
-    const clawdis = (parsed as { clawdis?: unknown }).clawdis
-    if (!clawdis || typeof clawdis !== 'object') return undefined
-    const clawdisObj = clawdis as Record<string, unknown>
+    const clawdisObj = clawdisRaw as Record<string, unknown>
     const requiresRaw =
       typeof clawdisObj.requires === 'object' && clawdisObj.requires !== null
         ? (clawdisObj.requires as Record<string, unknown>)
@@ -60,6 +99,7 @@ export function parseClawdisMetadata(frontmatter: ParsedSkillFrontmatter) {
     if (typeof clawdisObj.homepage === 'string') metadata.homepage = clawdisObj.homepage
     if (typeof clawdisObj.skillKey === 'string') metadata.skillKey = clawdisObj.skillKey
     if (typeof clawdisObj.primaryEnv === 'string') metadata.primaryEnv = clawdisObj.primaryEnv
+    if (typeof clawdisObj.cliHelp === 'string') metadata.cliHelp = clawdisObj.cliHelp
     if (osRaw.length > 0) metadata.os = osRaw
 
     if (requiresRaw) {
@@ -77,6 +117,10 @@ export function parseClawdisMetadata(frontmatter: ParsedSkillFrontmatter) {
     }
 
     if (install.length > 0) metadata.install = install
+    const nix = parseNixPluginSpec(clawdisObj.nix)
+    if (nix) metadata.nix = nix
+    const config = parseClawdbotConfigSpec(clawdisObj.config)
+    if (config) metadata.config = config
 
     return parseArk(ClawdisSkillMetadataSchema, metadata, 'Clawdis metadata')
   } catch {
@@ -110,14 +154,14 @@ export function buildEmbeddingText(params: {
   otherFiles: Array<{ path: string; content: string }>
   maxChars?: number
 }) {
-  const { frontmatter, readme, otherFiles, maxChars = 200_000 } = params
+  const { frontmatter, readme, otherFiles, maxChars = DEFAULT_EMBEDDING_MAX_CHARS } = params
   const headerParts = [
-    frontmatter.name,
-    frontmatter.description,
-    frontmatter.homepage,
-    frontmatter.website,
-    frontmatter.url,
-    frontmatter.emoji,
+    getFrontmatterValue(frontmatter, 'name'),
+    getFrontmatterValue(frontmatter, 'description'),
+    getFrontmatterValue(frontmatter, 'homepage'),
+    getFrontmatterValue(frontmatter, 'website'),
+    getFrontmatterValue(frontmatter, 'url'),
+    getFrontmatterValue(frontmatter, 'emoji'),
   ].filter(Boolean)
   const fileParts = otherFiles.map((file) => `# ${file.path}\n${file.content}`)
   const raw = [headerParts.join('\n'), readme, ...fileParts].filter(Boolean).join('\n\n')
@@ -137,14 +181,32 @@ export async function hashSkillFiles(files: Array<{ path: string; sha256: string
   return toHex(new Uint8Array(digest))
 }
 
-function stripQuotes(value: string) {
-  if (
-    (value.startsWith('"') && value.endsWith('"')) ||
-    (value.startsWith("'") && value.endsWith("'"))
-  ) {
-    return value.slice(1, -1)
+function toJsonValue(value: unknown): unknown {
+  if (value === null) return null
+  if (value === undefined) return undefined
+  if (typeof value === 'string') {
+    const trimmedEnd = value.trimEnd()
+    return trimmedEnd.trim() ? trimmedEnd : undefined
   }
-  return value
+  if (typeof value === 'number') return Number.isFinite(value) ? value : undefined
+  if (typeof value === 'boolean') return value
+  if (typeof value === 'bigint') return value.toString()
+  if (value instanceof Date) return value.toISOString()
+  if (Array.isArray(value)) {
+    return value.map((entry) => {
+      const next = toJsonValue(entry)
+      return next === undefined ? null : next
+    })
+  }
+  if (isPlainObject(value)) {
+    const out: Record<string, unknown> = {}
+    for (const [key, entry] of Object.entries(value)) {
+      const next = toJsonValue(entry)
+      if (next !== undefined) out[key] = next
+    }
+    return out
+  }
+  return undefined
 }
 
 function normalizeStringList(input: unknown): string[] {
@@ -181,8 +243,39 @@ function parseInstallSpec(input: unknown): SkillInstallSpec | undefined {
   return spec
 }
 
+function parseNixPluginSpec(input: unknown): NixPluginSpec | undefined {
+  if (!input || typeof input !== 'object') return undefined
+  const raw = input as Record<string, unknown>
+  if (typeof raw.plugin !== 'string') return undefined
+  const plugin = raw.plugin.trim()
+  if (!plugin) return undefined
+  const systems = normalizeStringList(raw.systems)
+  const spec: NixPluginSpec = { plugin }
+  if (systems.length > 0) spec.systems = systems
+  return spec
+}
+
+function parseClawdbotConfigSpec(input: unknown): ClawdbotConfigSpec | undefined {
+  if (!input || typeof input !== 'object') return undefined
+  const raw = input as Record<string, unknown>
+  const requiredEnv = normalizeStringList(raw.requiredEnv)
+  const stateDirs = normalizeStringList(raw.stateDirs)
+  const example = typeof raw.example === 'string' ? raw.example.trim() : ''
+  const spec: ClawdbotConfigSpec = {}
+  if (requiredEnv.length > 0) spec.requiredEnv = requiredEnv
+  if (stateDirs.length > 0) spec.stateDirs = stateDirs
+  if (example) spec.example = example
+  return Object.keys(spec).length > 0 ? spec : undefined
+}
+
 function toHex(bytes: Uint8Array) {
   let out = ''
   for (const byte of bytes) out += byte.toString(16).padStart(2, '0')
   return out
 }
+
+function isPlainObject(value: unknown): value is Record<string, unknown> {
+  if (!value || typeof value !== 'object') return false
+  const proto = Object.getPrototypeOf(value)
+  return proto === Object.prototype || proto === null
+}

convex/lib/soulChangelog.ts

@@ -0,0 +1,273 @@
+import { internal } from '../_generated/api'
+import type { Doc } from '../_generated/dataModel'
+import type { ActionCtx } from '../_generated/server'
+
+const CHANGELOG_MODEL = process.env.OPENAI_CHANGELOG_MODEL ?? 'gpt-4.1'
+const MAX_README_CHARS = 8_000
+const MAX_PATHS_IN_PROMPT = 30
+
+type FileMeta = { path: string; sha256?: string }
+
+type FileDiffSummary = {
+  added: string[]
+  removed: string[]
+  changed: string[]
+}
+
+function clampText(value: string, maxChars: number) {
+  const trimmed = value.trim()
+  if (trimmed.length <= maxChars) return trimmed
+  return `${trimmed.slice(0, maxChars).trimEnd()}\n…`
+}
+
+function summarizeFileDiff(oldFiles: FileMeta[], nextFiles: FileMeta[]): FileDiffSummary {
+  const oldByPath = new Map(oldFiles.map((f) => [f.path, f] as const))
+  const nextByPath = new Map(nextFiles.map((f) => [f.path, f] as const))
+
+  const added: string[] = []
+  const removed: string[] = []
+  const changed: string[] = []
+
+  for (const [path, file] of nextByPath.entries()) {
+    const prev = oldByPath.get(path)
+    if (!prev) {
+      added.push(path)
+      continue
+    }
+    if (file.sha256 && prev.sha256 && file.sha256 !== prev.sha256) changed.push(path)
+  }
+  for (const path of oldByPath.keys()) {
+    if (!nextByPath.has(path)) removed.push(path)
+  }
+
+  added.sort()
+  removed.sort()
+  changed.sort()
+  return { added, removed, changed }
+}
+
+function formatDiffSummary(diff: FileDiffSummary) {
+  const parts: string[] = []
+  if (diff.added.length) parts.push(`${diff.added.length} added`)
+  if (diff.changed.length) parts.push(`${diff.changed.length} changed`)
+  if (diff.removed.length) parts.push(`${diff.removed.length} removed`)
+  return parts.join(', ') || 'no file changes detected'
+}
+
+function pickPaths(values: string[]) {
+  if (values.length <= MAX_PATHS_IN_PROMPT) return values
+  return values.slice(0, MAX_PATHS_IN_PROMPT)
+}
+
+function extractResponseText(payload: unknown) {
+  if (!payload || typeof payload !== 'object') return null
+  const output = (payload as { output?: unknown }).output
+  if (!Array.isArray(output)) return null
+  const chunks: string[] = []
+  for (const item of output) {
+    if (!item || typeof item !== 'object') continue
+    if ((item as { type?: unknown }).type !== 'message') continue
+    const content = (item as { content?: unknown }).content
+    if (!Array.isArray(content)) continue
+    for (const part of content) {
+      if (!part || typeof part !== 'object') continue
+      if ((part as { type?: unknown }).type !== 'output_text') continue
+      const text = (part as { text?: unknown }).text
+      if (typeof text === 'string' && text.trim()) chunks.push(text)
+    }
+  }
+  const joined = chunks.join('\n').trim()
+  return joined || null
+}
+
+async function generateWithOpenAI(args: {
+  slug: string
+  version: string
+  oldReadme: string | null
+  nextReadme: string
+  fileDiff: FileDiffSummary | null
+}) {
+  const apiKey = process.env.OPENAI_API_KEY
+  if (!apiKey) return null
+
+  const oldReadme = args.oldReadme ? clampText(args.oldReadme, MAX_README_CHARS) : ''
+  const nextReadme = clampText(args.nextReadme, MAX_README_CHARS)
+
+  const fileDiff = args.fileDiff
+  const diffSummary = fileDiff ? formatDiffSummary(fileDiff) : 'unknown'
+  const changedPaths = fileDiff ? pickPaths(fileDiff.changed) : []
+  const addedPaths = fileDiff ? pickPaths(fileDiff.added) : []
+  const removedPaths = fileDiff ? pickPaths(fileDiff.removed) : []
+
+  const input = [
+    `Soul: ${args.slug}`,
+    `Version: ${args.version}`,
+    `File changes: ${diffSummary}`,
+    changedPaths.length ? `Changed files (sample): ${changedPaths.join(', ')}` : null,
+    addedPaths.length ? `Added files (sample): ${addedPaths.join(', ')}` : null,
+    removedPaths.length ? `Removed files (sample): ${removedPaths.join(', ')}` : null,
+    oldReadme ? `Previous SOUL.md:\n${oldReadme}` : null,
+    `New SOUL.md:\n${nextReadme}`,
+  ]
+    .filter(Boolean)
+    .join('\n\n')
+
+  const response = await fetch('https://api.openai.com/v1/responses', {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      Authorization: `Bearer ${apiKey}`,
+    },
+    body: JSON.stringify({
+      model: CHANGELOG_MODEL,
+      instructions:
+        'Write a concise changelog for this soul version. Audience: everyone. Output plain text. Prefer 2–6 bullet points. If it is a big change, include a short 1-line summary first, then bullets. Don’t mention that you are AI. Don’t invent details; only use the inputs.',
+      input,
+      max_output_tokens: 220,
+    }),
+  })
+
+  if (!response.ok) return null
+  const payload = (await response.json()) as unknown
+  return extractResponseText(payload)
+}
+
+function generateFallback(args: {
+  slug: string
+  version: string
+  oldReadme: string | null
+  nextReadme: string
+  fileDiff: FileDiffSummary | null
+}) {
+  const lines: string[] = []
+  if (!args.oldReadme) {
+    lines.push(`- Initial release.`)
+    return lines.join('\n')
+  }
+
+  const diff = args.fileDiff
+  if (diff) {
+    const parts: string[] = []
+    if (diff.added.length) parts.push(`added ${diff.added.length}`)
+    if (diff.changed.length) parts.push(`updated ${diff.changed.length}`)
+    if (diff.removed.length) parts.push(`removed ${diff.removed.length}`)
+    if (parts.length) lines.push(`- ${parts.join(', ')} file(s).`)
+  }
+
+  lines.push(`- Updated SOUL.md.`)
+  return lines.join('\n')
+}
+
+export async function generateSoulChangelogForPublish(
+  ctx: ActionCtx,
+  args: { slug: string; version: string; readmeText: string; files: FileMeta[] },
+): Promise<string> {
+  try {
+    const soul = (await ctx.runQuery(internal.souls.getSoulBySlugInternal, {
+      slug: args.slug,
+    })) as Doc<'souls'> | null
+    const previous: Doc<'soulVersions'> | null =
+      soul?.latestVersionId && !soul.softDeletedAt
+        ? ((await ctx.runQuery(internal.souls.getVersionByIdInternal, {
+            versionId: soul.latestVersionId,
+          })) as Doc<'soulVersions'> | null)
+        : null
+
+    const oldReadmeText: string | null = previous
+      ? await readReadmeFromVersion(ctx, previous)
+      : null
+    const oldFiles = previous
+      ? previous.files.map((file) => ({ path: file.path, sha256: file.sha256 }))
+      : []
+    const fileDiff = previous ? summarizeFileDiff(oldFiles, args.files) : null
+
+    const ai = await generateWithOpenAI({
+      slug: args.slug,
+      version: args.version,
+      oldReadme: oldReadmeText,
+      nextReadme: args.readmeText,
+      fileDiff,
+    }).catch(() => null)
+
+    return (
+      ai ??
+      generateFallback({
+        slug: args.slug,
+        version: args.version,
+        oldReadme: oldReadmeText,
+        nextReadme: args.readmeText,
+        fileDiff,
+      })
+    )
+  } catch {
+    return '- Updated soul.'
+  }
+}
+
+export async function generateSoulChangelogPreview(
+  ctx: ActionCtx,
+  args: {
+    slug: string
+    version: string
+    readmeText: string
+    filePaths?: string[]
+  },
+): Promise<string> {
+  try {
+    const soul = (await ctx.runQuery(internal.souls.getSoulBySlugInternal, {
+      slug: args.slug,
+    })) as Doc<'souls'> | null
+    const previous: Doc<'soulVersions'> | null =
+      soul?.latestVersionId && !soul.softDeletedAt
+        ? ((await ctx.runQuery(internal.souls.getVersionByIdInternal, {
+            versionId: soul.latestVersionId,
+          })) as Doc<'soulVersions'> | null)
+        : null
+
+    const oldReadmeText: string | null = previous
+      ? await readReadmeFromVersion(ctx, previous)
+      : null
+    const oldPaths = previous ? previous.files.map((file) => file.path) : []
+    const nextPaths = args.filePaths ?? []
+    const diff = previous ? summarizeFileDiffFromPaths(oldPaths, nextPaths) : null
+
+    const ai = await generateWithOpenAI({
+      slug: args.slug,
+      version: args.version,
+      oldReadme: oldReadmeText,
+      nextReadme: args.readmeText,
+      fileDiff: diff,
+    }).catch(() => null)
+
+    return (
+      ai ??
+      generateFallback({
+        slug: args.slug,
+        version: args.version,
+        oldReadme: oldReadmeText,
+        nextReadme: args.readmeText,
+        fileDiff: diff,
+      })
+    )
+  } catch {
+    return '- Updated soul.'
+  }
+}
+
+async function readReadmeFromVersion(ctx: ActionCtx, version: Doc<'soulVersions'>) {
+  const file = version.files.find((entry) => entry.path.toLowerCase() === 'soul.md')
+  if (!file) return null
+  const blob = await ctx.storage.get(file.storageId)
+  if (!blob) return null
+  return blob.text()
+}
+
+function summarizeFileDiffFromPaths(oldPaths: string[], nextPaths: string[]) {
+  const oldFiles = oldPaths.map((path) => ({ path }))
+  const nextFiles = nextPaths.map((path) => ({ path }))
+  return summarizeFileDiff(oldFiles, nextFiles)
+}
+
+export const __test = {
+  summarizeFileDiff,
+}

convex/lib/soulPublish.ts

@@ -0,0 +1,240 @@
+import { ConvexError } from 'convex/values'
+import semver from 'semver'
+import { internal } from '../_generated/api'
+import type { Doc, Id } from '../_generated/dataModel'
+import type { ActionCtx } from '../_generated/server'
+import { generateEmbedding } from './embeddings'
+import { requireGitHubAccountAge } from './githubAccount'
+import {
+  buildEmbeddingText,
+  getFrontmatterMetadata,
+  getFrontmatterValue,
+  hashSkillFiles,
+  isTextFile,
+  parseFrontmatter,
+  sanitizePath,
+} from './skills'
+import { generateSoulChangelogForPublish } from './soulChangelog'
+
+const MAX_TOTAL_BYTES = 50 * 1024 * 1024
+
+const MAX_SUMMARY_LENGTH = 160
+
+function deriveSoulSummary(readmeText: string) {
+  const lines = readmeText.split(/\r?\n/)
+  let inFrontmatter = false
+  for (const raw of lines) {
+    const trimmed = raw.trim()
+    if (!trimmed) continue
+    if (!inFrontmatter && trimmed === '---') {
+      inFrontmatter = true
+      continue
+    }
+    if (inFrontmatter) {
+      if (trimmed === '---') {
+        inFrontmatter = false
+      }
+      continue
+    }
+    const cleaned = trimmed.replace(/^#+\s*/, '')
+    if (!cleaned) continue
+    if (cleaned.length > MAX_SUMMARY_LENGTH) {
+      return `${cleaned.slice(0, MAX_SUMMARY_LENGTH - 3).trimEnd()}...`
+    }
+    return cleaned
+  }
+  return undefined
+}
+
+export type PublishResult = {
+  soulId: Id<'souls'>
+  versionId: Id<'soulVersions'>
+  embeddingId: Id<'soulEmbeddings'>
+}
+
+export type PublishVersionArgs = {
+  slug: string
+  displayName: string
+  version: string
+  changelog: string
+  tags?: string[]
+  source?: {
+    kind: 'github'
+    url: string
+    repo: string
+    ref: string
+    commit: string
+    path: string
+    importedAt: number
+  }
+  files: Array<{
+    path: string
+    size: number
+    storageId: Id<'_storage'>
+    sha256: string
+    contentType?: string
+  }>
+}
+
+export async function publishSoulVersionForUser(
+  ctx: ActionCtx,
+  userId: Id<'users'>,
+  args: PublishVersionArgs,
+): Promise<PublishResult> {
+  const version = args.version.trim()
+  const slug = args.slug.trim().toLowerCase()
+  const displayName = args.displayName.trim()
+  if (!slug || !displayName) throw new ConvexError('Slug and display name required')
+  if (!/^[a-z0-9][a-z0-9-]*$/.test(slug)) {
+    throw new ConvexError('Slug must be lowercase and url-safe')
+  }
+  if (!semver.valid(version)) {
+    throw new ConvexError('Version must be valid semver')
+  }
+
+  await requireGitHubAccountAge(ctx, userId)
+
+  const suppliedChangelog = args.changelog.trim()
+  const changelogSource = suppliedChangelog ? ('user' as const) : ('auto' as const)
+
+  const sanitizedFiles = args.files.map((file) => {
+    const path = sanitizePath(file.path)
+    if (!path) throw new ConvexError('Invalid file paths')
+    if (!isTextFile(path, file.contentType ?? undefined)) {
+      throw new ConvexError('Only text-based files are allowed')
+    }
+    return { ...file, path }
+  })
+
+  const totalBytes = sanitizedFiles.reduce((sum, file) => sum + file.size, 0)
+  if (totalBytes > MAX_TOTAL_BYTES) {
+    throw new ConvexError('Soul bundle exceeds 50MB limit')
+  }
+
+  const isSoulFile = (path: string) => path.toLowerCase() === 'soul.md'
+  const readmeFile = sanitizedFiles.find((file) => isSoulFile(file.path))
+  if (!readmeFile) throw new ConvexError('SOUL.md is required')
+
+  const nonSoulFiles = sanitizedFiles.filter((file) => !isSoulFile(file.path))
+  if (nonSoulFiles.length > 0) {
+    throw new ConvexError('Only SOUL.md is allowed for soul bundles')
+  }
+
+  const readmeText = await fetchText(ctx, readmeFile.storageId)
+  const frontmatter = parseFrontmatter(readmeText)
+  const summary = getFrontmatterValue(frontmatter, 'description') ?? deriveSoulSummary(readmeText)
+  const metadata = mergeSourceIntoMetadata(getFrontmatterMetadata(frontmatter), args.source)
+
+  const embeddingText = buildEmbeddingText({
+    frontmatter,
+    readme: readmeText,
+    otherFiles: [],
+  })
+
+  const fingerprint = await hashSkillFiles(
+    sanitizedFiles.map((file) => ({
+      path: file.path ?? '',
+      sha256: file.sha256,
+    })),
+  )
+
+  const changelogPromise =
+    changelogSource === 'user'
+      ? Promise.resolve(suppliedChangelog)
+      : generateSoulChangelogForPublish(ctx, {
+          slug,
+          version,
+          readmeText,
+          files: sanitizedFiles.map((file) => ({ path: file.path ?? '', sha256: file.sha256 })),
+        })
+
+  const embeddingPromise = generateEmbedding(embeddingText)
+
+  const [changelogText, embedding] = await Promise.all([
+    changelogPromise,
+    embeddingPromise.catch((error) => {
+      throw new ConvexError(formatEmbeddingError(error))
+    }),
+  ])
+
+  const publishResult = (await ctx.runMutation(internal.souls.insertVersion, {
+    userId,
+    slug,
+    displayName,
+    version,
+    changelog: changelogText,
+    changelogSource,
+    tags: args.tags?.map((tag) => tag.trim()).filter(Boolean),
+    fingerprint,
+    files: sanitizedFiles,
+    parsed: {
+      frontmatter,
+      metadata,
+    },
+    summary,
+    embedding,
+  })) as PublishResult
+
+  const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
+    userId,
+  })) as Doc<'users'> | null
+  const ownerHandle = owner?.handle ?? owner?.name ?? userId
+
+  void ctx.scheduler
+    .runAfter(0, internal.githubSoulBackupsNode.backupSoulForPublishInternal, {
+      slug,
+      version,
+      displayName,
+      ownerHandle,
+      files: sanitizedFiles,
+      publishedAt: Date.now(),
+    })
+    .catch((error) => {
+      console.error('GitHub soul backup scheduling failed', error)
+    })
+
+  return publishResult
+}
+
+function mergeSourceIntoMetadata(metadata: unknown, source: PublishVersionArgs['source']) {
+  if (!source) return metadata === undefined ? undefined : metadata
+  const sourceValue = {
+    kind: source.kind,
+    url: source.url,
+    repo: source.repo,
+    ref: source.ref,
+    commit: source.commit,
+    path: source.path,
+    importedAt: source.importedAt,
+  }
+
+  if (!metadata) return { source: sourceValue }
+  if (typeof metadata !== 'object' || Array.isArray(metadata)) return { source: sourceValue }
+  return { ...(metadata as Record<string, unknown>), source: sourceValue }
+}
+
+export async function fetchText(
+  ctx: { storage: { get: (id: Id<'_storage'>) => Promise<Blob | null> } },
+  storageId: Id<'_storage'>,
+) {
+  const blob = await ctx.storage.get(storageId)
+  if (!blob) throw new Error('File missing in storage')
+  return blob.text()
+}
+
+function formatEmbeddingError(error: unknown) {
+  if (error instanceof Error) {
+    if (error.message.includes('OPENAI_API_KEY')) {
+      return 'OPENAI_API_KEY is not configured.'
+    }
+    if (error.message.startsWith('Embedding failed')) {
+      return error.message
+    }
+  }
+  return 'Embedding failed. Please try again.'
+}
+
+export const __test = {
+  getSummary: (frontmatter: Record<string, unknown>) =>
+    getFrontmatterValue(frontmatter, 'description'),
+}

convex/lib/tokens.test.ts

@@ -1,20 +1,33 @@
+/* @vitest-environment node */
+
 import { describe, expect, it } from 'vitest'
-import { API_TOKEN_PREFIX, generateToken, hashToken } from './tokens'
+import { __test, generateToken, hashToken } from './tokens'
 
 describe('tokens', () => {
-  it('generates token with prefix and url-safe chars', () => {
-    const { token, prefix } = generateToken()
-    expect(token.startsWith(API_TOKEN_PREFIX)).toBe(true)
-    expect(prefix).toBe(token.slice(0, 12))
-    expect(token).toMatch(/^[a-z0-9_-]+$/i)
+  it('hashToken returns sha256 hex', async () => {
+    await expect(hashToken('test')).resolves.toBe(
+      '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08',
+    )
   })
 
-  it('hashes tokens deterministically', async () => {
-    const a = await hashToken('clh_test')
-    const b = await hashToken('clh_test')
-    const c = await hashToken('clh_other')
-    expect(a).toBe(b)
-    expect(a).not.toBe(c)
-    expect(a).toMatch(/^[a-f0-9]{64}$/)
+  it('generateToken returns token + prefix', () => {
+    const { token, prefix } = generateToken()
+    expect(token).toMatch(/^clh_[A-Za-z0-9_-]+$/)
+    expect(prefix).toBe(token.slice(0, 12))
+  })
+
+  it('toHex encodes bytes', () => {
+    expect(__test.toHex(new Uint8Array([0, 15, 255]))).toBe('000fff')
+  })
+
+  it('toBase64 encodes 1/2/3-byte tails', () => {
+    expect(__test.toBase64(new Uint8Array([0xff]))).toBe('/w==')
+    expect(__test.toBase64(new Uint8Array([0xff, 0xee]))).toBe('/+4=')
+    expect(__test.toBase64(new Uint8Array([0xff, 0xee, 0xdd]))).toBe('/+7d')
+  })
+
+  it('toBase64Url replaces alphabet and strips padding', () => {
+    expect(__test.toBase64Url(new Uint8Array([0xff]))).toBe('_w')
+    expect(__test.toBase64Url(new Uint8Array([0xfa, 0x00, 0x00]))).toBe('-gAA')
   })
 })

convex/lib/tokens.ts

@@ -43,3 +43,9 @@ function toBase64(bytes: Uint8Array) {
   }
   return output
 }
+
+export const __test = {
+  toHex,
+  toBase64,
+  toBase64Url,
+}

convex/lib/userSearch.ts

@@ -0,0 +1,68 @@
+import type { Doc } from '../_generated/dataModel'
+
+type UserSearchResult = {
+  items: Doc<'users'>[]
+  total: number
+}
+
+type UserSearchMatch = {
+  user: Doc<'users'>
+  score: number
+}
+
+function normalizeCompact(value: string) {
+  return value.toLowerCase().replace(/[^a-z0-9]/g, '')
+}
+
+function scoreUser(user: Doc<'users'>, query: string, compactQuery: string) {
+  const handle = user.handle?.toLowerCase() ?? ''
+  const name = user.name?.toLowerCase() ?? ''
+  const displayName = user.displayName?.toLowerCase() ?? ''
+  const email = user.email?.toLowerCase() ?? ''
+  const id = String(user._id).toLowerCase()
+
+  let score = 0
+
+  if (id === query) score = Math.max(score, 100)
+  if (handle === query) score = Math.max(score, 96)
+  if (displayName === query || name === query) score = Math.max(score, 90)
+
+  if (handle.startsWith(query)) score = Math.max(score, 82)
+  if (displayName.startsWith(query) || name.startsWith(query)) score = Math.max(score, 72)
+
+  if (handle.includes(query)) score = Math.max(score, 62)
+  if (displayName.includes(query) || name.includes(query)) score = Math.max(score, 52)
+  if (email.includes(query)) score = Math.max(score, 42)
+  if (id.includes(query)) score = Math.max(score, 40)
+
+  if (compactQuery.length >= 2) {
+    const compactHandle = normalizeCompact(handle)
+    const compactName = normalizeCompact(displayName || name)
+    if (compactHandle === compactQuery) score = Math.max(score, 88)
+    if (compactHandle.includes(compactQuery)) score = Math.max(score, 58)
+    if (compactName.includes(compactQuery)) score = Math.max(score, 48)
+  }
+
+  return score
+}
+
+export function buildUserSearchResults(users: Doc<'users'>[], query?: string): UserSearchResult {
+  const trimmed = query?.trim() ?? ''
+  if (!trimmed) return { items: users, total: users.length }
+
+  const normalized = trimmed.toLowerCase()
+  const compactQuery = normalizeCompact(normalized)
+  const matches: UserSearchMatch[] = []
+
+  for (const user of users) {
+    const score = scoreUser(user, normalized, compactQuery)
+    if (score > 0) matches.push({ user, score })
+  }
+
+  matches.sort((a, b) => {
+    if (b.score !== a.score) return b.score - a.score
+    return b.user._creationTime - a.user._creationTime
+  })
+
+  return { items: matches.map((entry) => entry.user), total: matches.length }
+}

convex/lib/webhooks.test.ts

@@ -0,0 +1,91 @@
+/* @vitest-environment node */
+import { afterEach, describe, expect, it } from 'vitest'
+import { buildDiscordPayload, buildSkillUrl, getWebhookConfig, shouldSendWebhook } from './webhooks'
+
+const originalEnv = { ...process.env }
+
+afterEach(() => {
+  process.env = { ...originalEnv }
+})
+
+describe('webhook config', () => {
+  it('parses highlighted-only flag', () => {
+    process.env.DISCORD_WEBHOOK_URL = 'https://example.com'
+    process.env.DISCORD_WEBHOOK_HIGHLIGHTED_ONLY = 'true'
+    const config = getWebhookConfig()
+    expect(config.highlightedOnly).toBe(true)
+  })
+
+  it('defaults site url when missing', () => {
+    delete process.env.SITE_URL
+    process.env.DISCORD_WEBHOOK_URL = 'https://example.com'
+    const config = getWebhookConfig()
+    expect(config.siteUrl).toBe('https://clawhub.ai')
+  })
+})
+
+describe('webhook filtering', () => {
+  it('skips when url missing', () => {
+    const config = getWebhookConfig({} as NodeJS.ProcessEnv)
+    expect(shouldSendWebhook('skill.publish', { slug: 'demo', displayName: 'Demo' }, config)).toBe(
+      false,
+    )
+  })
+
+  it('filters non-highlighted when highlighted-only', () => {
+    const config = {
+      url: 'https://example.com',
+      highlightedOnly: true,
+      siteUrl: 'https://clawhub.ai',
+    }
+    const allowed = shouldSendWebhook(
+      'skill.publish',
+      { slug: 'demo', displayName: 'Demo', highlighted: false },
+      config,
+    )
+    expect(allowed).toBe(false)
+  })
+
+  it('allows highlighted event when highlighted-only', () => {
+    const config = {
+      url: 'https://example.com',
+      highlightedOnly: true,
+      siteUrl: 'https://clawhub.ai',
+    }
+    const allowed = shouldSendWebhook(
+      'skill.highlighted',
+      { slug: 'demo', displayName: 'Demo', highlighted: true },
+      config,
+    )
+    expect(allowed).toBe(true)
+  })
+})
+
+describe('payload building', () => {
+  it('builds canonical url with owner', () => {
+    const url = buildSkillUrl(
+      { slug: 'beeper', displayName: 'Beeper', ownerHandle: 'KrauseFx' },
+      'https://clawhub.ai',
+    )
+    expect(url).toBe('https://clawhub.ai/KrauseFx/beeper')
+  })
+
+  it('builds a publish embed', () => {
+    const payload = buildDiscordPayload(
+      'skill.publish',
+      {
+        slug: 'demo',
+        displayName: 'Demo Skill',
+        summary: 'Nice skill',
+        version: '1.2.3',
+        ownerHandle: 'steipete',
+        tags: ['latest', 'discord'],
+      },
+      { url: 'https://example.com', highlightedOnly: false, siteUrl: 'https://clawhub.ai' },
+    )
+    const embed = payload.embeds[0]
+    expect(embed.title).toBe('Demo Skill')
+    expect(embed.description).toBe('Nice skill')
+    expect(embed.fields[0].value).toBe('v1.2.3')
+  })
+})

convex/lib/webhooks.ts

@@ -0,0 +1,112 @@
+export type WebhookEvent = 'skill.publish' | 'skill.highlighted'
+
+export type WebhookSkillPayload = {
+  slug: string
+  displayName: string
+  summary?: string
+  version?: string
+  ownerHandle?: string
+  highlighted?: boolean
+  tags?: string[]
+}
+
+export type WebhookConfig = {
+  url: string | null
+  highlightedOnly: boolean
+  siteUrl: string
+}
+
+const DEFAULT_SITE_URL = 'https://clawhub.ai'
+
+export function getWebhookConfig(env: NodeJS.ProcessEnv = process.env): WebhookConfig {
+  const url = env.DISCORD_WEBHOOK_URL?.trim() || null
+  const highlightedOnly = parseBoolean(env.DISCORD_WEBHOOK_HIGHLIGHTED_ONLY)
+  const siteUrl = env.SITE_URL?.trim() || DEFAULT_SITE_URL
+  return { url, highlightedOnly, siteUrl }
+}
+
+export function shouldSendWebhook(
+  event: WebhookEvent,
+  skill: WebhookSkillPayload,
+  config: WebhookConfig,
+) {
+  if (!config.url) return false
+  if (!config.highlightedOnly) return true
+  if (event === 'skill.highlighted') return true
+  return Boolean(skill.highlighted)
+}
+
+export function buildDiscordPayload(
+  event: WebhookEvent,
+  skill: WebhookSkillPayload,
+  config: WebhookConfig,
+) {
+  const titleBase = skill.displayName || skill.slug
+  const title = event === 'skill.highlighted' ? `Highlighted: ${titleBase}` : titleBase
+  const description = buildDescription(event, skill)
+  const url = buildSkillUrl(skill, config.siteUrl)
+  const tags = formatTags(skill.tags)
+
+  return {
+    embeds: [
+      {
+        title,
+        description,
+        url,
+        color: event === 'skill.highlighted' ? 0xff6b4a : 0x2f76ff,
+        fields: [
+          {
+            name: 'Version',
+            value: skill.version ? `v${skill.version}` : '—',
+            inline: true,
+          },
+          {
+            name: 'Owner',
+            value: skill.ownerHandle ? `@${skill.ownerHandle}` : '—',
+            inline: true,
+          },
+          {
+            name: 'Tags',
+            value: tags,
+            inline: false,
+          },
+        ],
+        footer: {
+          text: 'ClawHub',
+        },
+        timestamp: new Date().toISOString(),
+      },
+    ],
+  }
+}
+
+export function buildSkillUrl(skill: WebhookSkillPayload, siteUrl: string) {
+  const owner = skill.ownerHandle?.trim()
+  if (owner) return `${siteUrl}/${owner}/${skill.slug}`
+  return `${siteUrl}/skills/${skill.slug}`
+}
+
+function buildDescription(event: WebhookEvent, skill: WebhookSkillPayload) {
+  const summary = (skill.summary ?? '').trim()
+  if (summary) return truncate(summary, 200)
+  if (event === 'skill.highlighted') return 'Newly highlighted skill on ClawHub.'
+  if (skill.version) return `New version v${skill.version} published on ClawHub.`
+  return 'New skill published on ClawHub.'
+}
+
+function parseBoolean(value?: string) {
+  if (!value) return false
+  const normalized = value.trim().toLowerCase()
+  return normalized === '1' || normalized === 'true' || normalized === 'yes' || normalized === 'on'
+}
+
+function formatTags(tags?: string[] | null) {
+  const cleaned = (tags ?? []).map((tag) => tag.trim()).filter(Boolean)
+  if (cleaned.length === 0) return '—'
+  return cleaned.slice(0, 8).join(', ')
+}
+
+function truncate(value: string, max: number) {
+  if (value.length <= max) return value
+  return `${value.slice(0, max - 1).trim()}…`
+}

convex/llmEval.ts

@@ -0,0 +1,398 @@
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import { internalAction } from './_generated/server'
+import type { SkillEvalContext } from './lib/securityPrompt'
+import {
+  assembleEvalUserMessage,
+  detectInjectionPatterns,
+  getLlmEvalModel,
+  LLM_EVAL_MAX_OUTPUT_TOKENS,
+  parseLlmEvalResponse,
+  SECURITY_EVALUATOR_SYSTEM_PROMPT,
+} from './lib/securityPrompt'
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+
+function extractResponseText(payload: unknown): string | null {
+  if (!payload || typeof payload !== 'object') return null
+  const output = (payload as { output?: unknown }).output
+  if (!Array.isArray(output)) return null
+  const chunks: string[] = []
+  for (const item of output) {
+    if (!item || typeof item !== 'object') continue
+    if ((item as { type?: unknown }).type !== 'message') continue
+    const content = (item as { content?: unknown }).content
+    if (!Array.isArray(content)) continue
+    for (const part of content) {
+      if (!part || typeof part !== 'object') continue
+      if ((part as { type?: unknown }).type !== 'output_text') continue
+      const text = (part as { text?: unknown }).text
+      if (typeof text === 'string' && text.trim()) chunks.push(text)
+    }
+  }
+  const joined = chunks.join('\n').trim()
+  return joined || null
+}
+
+function verdictToStatus(verdict: string): string {
+  switch (verdict) {
+    case 'benign':
+      return 'clean'
+    case 'malicious':
+      return 'malicious'
+    case 'suspicious':
+      return 'suspicious'
+    default:
+      return 'pending'
+  }
+}
+
+// ---------------------------------------------------------------------------
+// Publish-time evaluation action
+// ---------------------------------------------------------------------------
+
+export const evaluateWithLlm = internalAction({
+  args: {
+    versionId: v.id('skillVersions'),
+  },
+  handler: async (ctx, args) => {
+    const apiKey = process.env.OPENAI_API_KEY
+    if (!apiKey) {
+      console.log('[llmEval] OPENAI_API_KEY not configured, skipping evaluation')
+      return
+    }
+
+    const model = getLlmEvalModel()
+
+    // Store error helper
+    const storeError = async (message: string) => {
+      console.error(`[llmEval] ${message}`)
+      await ctx.runMutation(internal.skills.updateVersionLlmAnalysisInternal, {
+        versionId: args.versionId,
+        llmAnalysis: {
+          status: 'error',
+          summary: message,
+          model,
+          checkedAt: Date.now(),
+        },
+      })
+    }
+
+    // 1. Fetch version
+    const version = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
+      versionId: args.versionId,
+    })) as Doc<'skillVersions'> | null
+
+    if (!version) {
+      await storeError(`Version ${args.versionId} not found`)
+      return
+    }
+
+    // 2. Fetch skill
+    const skill = (await ctx.runQuery(internal.skills.getSkillByIdInternal, {
+      skillId: version.skillId,
+    })) as Doc<'skills'> | null
+
+    if (!skill) {
+      await storeError(`Skill ${version.skillId} not found`)
+      return
+    }
+
+    // 3. Read SKILL.md content
+    const skillMdFile = version.files.find((f) => {
+      const lower = f.path.toLowerCase()
+      return lower === 'skill.md' || lower === 'skills.md'
+    })
+
+    let skillMdContent = ''
+    if (skillMdFile) {
+      const blob = await ctx.storage.get(skillMdFile.storageId as Id<'_storage'>)
+      if (blob) {
+        skillMdContent = await blob.text()
+      }
+    }
+
+    if (!skillMdContent) {
+      await storeError('No SKILL.md content found')
+      return
+    }
+
+    // 4. Read all file contents
+    const fileContents: Array<{ path: string; content: string }> = []
+    for (const f of version.files) {
+      const lower = f.path.toLowerCase()
+      if (lower === 'skill.md' || lower === 'skills.md') continue
+      try {
+        const blob = await ctx.storage.get(f.storageId as Id<'_storage'>)
+        if (blob) {
+          fileContents.push({ path: f.path, content: await blob.text() })
+        }
+      } catch {
+        // Skip files that can't be read
+      }
+    }
+
+    // 5. Detect injection patterns across ALL content
+    const allContent = [skillMdContent, ...fileContents.map((f) => f.content)].join('\n')
+    const injectionSignals = detectInjectionPatterns(allContent)
+
+    // 6. Build eval context
+    const parsed = version.parsed as SkillEvalContext['parsed']
+    const fm = parsed.frontmatter ?? {}
+
+    const evalCtx: SkillEvalContext = {
+      slug: skill.slug,
+      displayName: skill.displayName,
+      ownerUserId: String(skill.ownerUserId),
+      version: version.version,
+      createdAt: version.createdAt,
+      summary: (skill.summary as string | undefined) ?? undefined,
+      source: (fm.source as string | undefined) ?? undefined,
+      homepage: (fm.homepage as string | undefined) ?? undefined,
+      parsed,
+      files: version.files.map((f) => ({ path: f.path, size: f.size })),
+      skillMdContent,
+      fileContents,
+      injectionSignals,
+    }
+
+    // 6. Assemble user message
+    const userMessage = assembleEvalUserMessage(evalCtx)
+
+    // 7. Call OpenAI Responses API (with retry for rate limits)
+    const MAX_RETRIES = 3
+    let raw: string | null = null
+    try {
+      const body = JSON.stringify({
+        model,
+        instructions: SECURITY_EVALUATOR_SYSTEM_PROMPT,
+        input: userMessage,
+        max_output_tokens: LLM_EVAL_MAX_OUTPUT_TOKENS,
+        text: {
+          format: {
+            type: 'json_object',
+          },
+        },
+      })
+
+      let response: Response | null = null
+      for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
+        response = await fetch('https://api.openai.com/v1/responses', {
+          method: 'POST',
+          headers: {
+            'Content-Type': 'application/json',
+            Authorization: `Bearer ${apiKey}`,
+          },
+          body,
+        })
+
+        if (response.status === 429 || response.status >= 500) {
+          if (attempt < MAX_RETRIES) {
+            const delay = 2 ** attempt * 2000 + Math.random() * 1000
+            console.log(
+              `[llmEval] Rate limited (${response.status}), retrying in ${Math.round(delay)}ms (attempt ${attempt + 1}/${MAX_RETRIES})`,
+            )
+            await new Promise((r) => setTimeout(r, delay))
+            continue
+          }
+        }
+        break
+      }
+
+      if (!response || !response.ok) {
+        const errorText = response ? await response.text() : 'No response'
+        await storeError(`OpenAI API error (${response?.status}): ${errorText.slice(0, 200)}`)
+        return
+      }
+
+      const payload = (await response.json()) as unknown
+      raw = extractResponseText(payload)
+    } catch (error) {
+      await storeError(
+        `OpenAI API call failed: ${error instanceof Error ? error.message : String(error)}`,
+      )
+      return
+    }
+
+    if (!raw) {
+      await storeError('Empty response from OpenAI')
+      return
+    }
+
+    // 8. Parse response
+    const result = parseLlmEvalResponse(raw)
+
+    if (!result) {
+      console.error(`[llmEval] Raw response (first 500 chars): ${raw.slice(0, 500)}`)
+      await storeError('Failed to parse LLM evaluation response')
+      return
+    }
+
+    // 9. Store result
+    await ctx.runMutation(internal.skills.updateVersionLlmAnalysisInternal, {
+      versionId: args.versionId,
+      llmAnalysis: {
+        status: verdictToStatus(result.verdict),
+        verdict: result.verdict,
+        confidence: result.confidence,
+        summary: result.summary,
+        dimensions: result.dimensions,
+        guidance: result.guidance,
+        findings: result.findings || undefined,
+        model,
+        checkedAt: Date.now(),
+      },
+    })
+
+    console.log(
+      `[llmEval] Evaluated ${skill.slug}@${version.version}: ${result.verdict} (${result.confidence} confidence)`,
+    )
+
+    // 10. Update moderation flags — re-read version to get the sha256hash
+    // that VT may have stored while we were evaluating (both run concurrently).
+    const freshVersion = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
+      versionId: args.versionId,
+    })) as Doc<'skillVersions'> | null
+
+    const sha256hash = freshVersion?.sha256hash ?? version.sha256hash
+    if (sha256hash) {
+      const status = verdictToStatus(result.verdict)
+      if (status === 'malicious' || status === 'suspicious' || status === 'clean') {
+        await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
+          sha256hash,
+          scanner: 'llm',
+          status,
+        })
+      }
+    }
+  },
+})
+
+// ---------------------------------------------------------------------------
+// Convenience: evaluate a single skill by slug (for testing / manual runs)
+// Usage: npx convex run llmEval:evaluateBySlug '{"slug": "transcribeexx"}'
+// ---------------------------------------------------------------------------
+
+export const evaluateBySlug = internalAction({
+  args: {
+    slug: v.string(),
+  },
+  handler: async (ctx, args) => {
+    const skill = (await ctx.runQuery(internal.skills.getSkillBySlugInternal, {
+      slug: args.slug,
+    })) as Doc<'skills'> | null
+
+    if (!skill) {
+      console.error(`[llmEval:bySlug] Skill "${args.slug}" not found`)
+      return { error: 'Skill not found' }
+    }
+
+    if (!skill.latestVersionId) {
+      console.error(`[llmEval:bySlug] Skill "${args.slug}" has no published version`)
+      return { error: 'No published version' }
+    }
+
+    console.log(`[llmEval:bySlug] Evaluating ${args.slug} (versionId: ${skill.latestVersionId})`)
+
+    await ctx.scheduler.runAfter(0, internal.llmEval.evaluateWithLlm, {
+      versionId: skill.latestVersionId,
+    })
+
+    return { ok: true, slug: args.slug, versionId: skill.latestVersionId }
+  },
+})
+
+// ---------------------------------------------------------------------------
+// Backfill action (Phase 2)
+// Schedules individual evaluateWithLlm actions for each skill in the batch,
+// then self-schedules the next batch. Each eval runs as its own action
+// invocation so we don't hit Convex action timeouts.
+// ---------------------------------------------------------------------------
+
+export const backfillLlmEval = internalAction({
+  args: {
+    cursor: v.optional(v.number()),
+    batchSize: v.optional(v.number()),
+    accTotal: v.optional(v.number()),
+    accScheduled: v.optional(v.number()),
+    accSkipped: v.optional(v.number()),
+    startTime: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const startTime = args.startTime ?? Date.now()
+    const apiKey = process.env.OPENAI_API_KEY
+    if (!apiKey) {
+      console.log('[llmEval:backfill] OPENAI_API_KEY not configured')
+      return { error: 'OPENAI_API_KEY not configured' }
+    }
+
+    const batchSize = args.batchSize ?? 25
+    const cursor = args.cursor ?? 0
+    let accTotal = args.accTotal ?? 0
+    let accScheduled = args.accScheduled ?? 0
+    let accSkipped = args.accSkipped ?? 0
+
+    const batch = await ctx.runQuery(internal.skills.getActiveSkillBatchForLlmBackfillInternal, {
+      cursor,
+      batchSize,
+    })
+
+    if (batch.skills.length === 0 && batch.done) {
+      console.log('[llmEval:backfill] No more skills to evaluate')
+      return { total: accTotal, scheduled: accScheduled, skipped: accSkipped }
+    }
+
+    console.log(
+      `[llmEval:backfill] Processing batch of ${batch.skills.length} skills (cursor=${cursor}, accumulated=${accTotal})`,
+    )
+
+    for (const { versionId, slug } of batch.skills) {
+      // Re-evaluate all (full file content reading upgrade)
+      const version = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
+        versionId,
+      })) as Doc<'skillVersions'> | null
+
+      if (!version) {
+        accSkipped++
+        continue
+      }
+
+      // Schedule each evaluation as a separate action invocation
+      await ctx.scheduler.runAfter(0, internal.llmEval.evaluateWithLlm, { versionId })
+      accScheduled++
+      console.log(`[llmEval:backfill] Scheduled eval for ${slug}`)
+    }
+
+    accTotal += batch.skills.length
+
+    if (!batch.done) {
+      // Delay the next batch slightly to avoid overwhelming the scheduler
+      // when all evals from this batch are also running
+      console.log(
+        `[llmEval:backfill] Scheduling next batch (cursor=${batch.nextCursor}, total so far=${accTotal})`,
+      )
+      await ctx.scheduler.runAfter(5_000, internal.llmEval.backfillLlmEval, {
+        cursor: batch.nextCursor,
+        batchSize,
+        accTotal,
+        accScheduled,
+        accSkipped,
+        startTime,
+      })
+      return { status: 'continuing', totalSoFar: accTotal }
+    }
+
+    const durationMs = Date.now() - startTime
+    const result = {
+      total: accTotal,
+      scheduled: accScheduled,
+      skipped: accSkipped,
+      durationMs,
+    }
+    console.log('[llmEval:backfill] Complete:', result)
+    return result
+  },
+})

convex/maintenance.test.ts

@@ -0,0 +1,270 @@
+/* @vitest-environment node */
+import { describe, expect, it, vi } from 'vitest'
+
+vi.mock('./_generated/api', () => ({
+  internal: {
+    maintenance: {
+      getSkillBackfillPageInternal: Symbol('getSkillBackfillPageInternal'),
+      applySkillBackfillPatchInternal: Symbol('applySkillBackfillPatchInternal'),
+      backfillSkillSummariesInternal: Symbol('backfillSkillSummariesInternal'),
+      getSkillFingerprintBackfillPageInternal: Symbol('getSkillFingerprintBackfillPageInternal'),
+      applySkillFingerprintBackfillPatchInternal: Symbol(
+        'applySkillFingerprintBackfillPatchInternal',
+      ),
+      backfillSkillFingerprintsInternal: Symbol('backfillSkillFingerprintsInternal'),
+    },
+  },
+}))
+
+const { backfillSkillFingerprintsInternalHandler, backfillSkillSummariesInternalHandler } =
+  await import('./maintenance')
+
+function makeBlob(text: string) {
+  return { text: () => Promise.resolve(text) } as unknown as Blob
+}
+
+describe('maintenance backfill', () => {
+  it('repairs summary + parsed by reparsing SKILL.md', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          kind: 'ok',
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          skillSummary: '>',
+          versionParsed: { frontmatter: { description: '>' } },
+          readmeStorageId: 'storage:1',
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn().mockResolvedValue({ ok: true })
+    const storageGet = vi
+      .fn()
+      .mockResolvedValue(makeBlob(`---\ndescription: >\n  Hello\n  world.\n---\nBody`))
+
+    const result = await backfillSkillSummariesInternalHandler(
+      { runQuery, runMutation, storage: { get: storageGet } } as never,
+      { dryRun: false, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.ok).toBe(true)
+    expect(result.stats.skillsScanned).toBe(1)
+    expect(result.stats.skillsPatched).toBe(1)
+    expect(result.stats.versionsPatched).toBe(1)
+    expect(runMutation).toHaveBeenCalledTimes(1)
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      skillId: 'skills:1',
+      versionId: 'skillVersions:1',
+      summary: 'Hello world.',
+      parsed: {
+        frontmatter: { description: 'Hello world.' },
+        metadata: undefined,
+        clawdis: undefined,
+      },
+    })
+  })
+
+  it('dryRun does not patch', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          kind: 'ok',
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          skillSummary: '>',
+          versionParsed: { frontmatter: { description: '>' } },
+          readmeStorageId: 'storage:1',
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn()
+    const storageGet = vi.fn().mockResolvedValue(makeBlob(`---\ndescription: Hello\n---\nBody`))
+
+    const result = await backfillSkillSummariesInternalHandler(
+      { runQuery, runMutation, storage: { get: storageGet } } as never,
+      { dryRun: true, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.ok).toBe(true)
+    expect(result.stats.skillsPatched).toBe(1)
+    expect(runMutation).not.toHaveBeenCalled()
+  })
+
+  it('counts missing storage blob', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          kind: 'ok',
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          skillSummary: null,
+          versionParsed: { frontmatter: {} },
+          readmeStorageId: 'storage:missing',
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn()
+    const storageGet = vi.fn().mockResolvedValue(null)
+
+    const result = await backfillSkillSummariesInternalHandler(
+      { runQuery, runMutation, storage: { get: storageGet } } as never,
+      { dryRun: false, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.stats.missingStorageBlob).toBe(1)
+    expect(runMutation).not.toHaveBeenCalled()
+  })
+})
+
+describe('maintenance fingerprint backfill', () => {
+  it('backfills fingerprint field and inserts index entry', async () => {
+    const { hashSkillFiles } = await import('./lib/skills')
+    const expected = await hashSkillFiles([{ path: 'SKILL.md', sha256: 'abc' }])
+
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          versionFingerprint: undefined,
+          files: [{ path: 'SKILL.md', sha256: 'abc' }],
+          existingEntries: [],
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn().mockResolvedValue({ ok: true })
+
+    const result = await backfillSkillFingerprintsInternalHandler(
+      { runQuery, runMutation } as never,
+      { dryRun: false, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.ok).toBe(true)
+    expect(result.stats.versionsScanned).toBe(1)
+    expect(result.stats.versionsPatched).toBe(1)
+    expect(result.stats.fingerprintsInserted).toBe(1)
+    expect(result.stats.fingerprintMismatches).toBe(0)
+    expect(runMutation).toHaveBeenCalledTimes(1)
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      versionId: 'skillVersions:1',
+      fingerprint: expected,
+      patchVersion: true,
+      replaceEntries: true,
+      existingEntryIds: [],
+    })
+  })
+
+  it('dryRun does not patch', async () => {
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          versionFingerprint: undefined,
+          files: [{ path: 'SKILL.md', sha256: 'abc' }],
+          existingEntries: [],
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn()
+
+    const result = await backfillSkillFingerprintsInternalHandler(
+      { runQuery, runMutation } as never,
+      { dryRun: true, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.ok).toBe(true)
+    expect(result.stats.versionsPatched).toBe(1)
+    expect(result.stats.fingerprintsInserted).toBe(1)
+    expect(runMutation).not.toHaveBeenCalled()
+  })
+
+  it('patches missing version fingerprint without touching correct entries', async () => {
+    const { hashSkillFiles } = await import('./lib/skills')
+    const expected = await hashSkillFiles([{ path: 'SKILL.md', sha256: 'abc' }])
+
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          versionFingerprint: undefined,
+          files: [{ path: 'SKILL.md', sha256: 'abc' }],
+          existingEntries: [{ id: 'skillVersionFingerprints:1', fingerprint: expected }],
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn().mockResolvedValue({ ok: true })
+
+    const result = await backfillSkillFingerprintsInternalHandler(
+      { runQuery, runMutation } as never,
+      { dryRun: false, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.ok).toBe(true)
+    expect(result.stats.versionsPatched).toBe(1)
+    expect(result.stats.fingerprintsInserted).toBe(0)
+    expect(result.stats.fingerprintMismatches).toBe(0)
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      versionId: 'skillVersions:1',
+      fingerprint: expected,
+      patchVersion: true,
+      replaceEntries: false,
+      existingEntryIds: [],
+    })
+  })
+
+  it('replaces mismatched fingerprint entries', async () => {
+    const { hashSkillFiles } = await import('./lib/skills')
+    const expected = await hashSkillFiles([{ path: 'SKILL.md', sha256: 'abc' }])
+
+    const runQuery = vi.fn().mockResolvedValue({
+      items: [
+        {
+          skillId: 'skills:1',
+          versionId: 'skillVersions:1',
+          versionFingerprint: 'wrong',
+          files: [{ path: 'SKILL.md', sha256: 'abc' }],
+          existingEntries: [{ id: 'skillVersionFingerprints:1', fingerprint: 'wrong' }],
+        },
+      ],
+      cursor: null,
+      isDone: true,
+    })
+
+    const runMutation = vi.fn().mockResolvedValue({ ok: true })
+
+    const result = await backfillSkillFingerprintsInternalHandler(
+      { runQuery, runMutation } as never,
+      { dryRun: false, batchSize: 10, maxBatches: 1 },
+    )
+
+    expect(result.ok).toBe(true)
+    expect(result.stats.fingerprintMismatches).toBe(1)
+    expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
+      versionId: 'skillVersions:1',
+      fingerprint: expected,
+      patchVersion: true,
+      replaceEntries: true,
+      existingEntryIds: ['skillVersionFingerprints:1'],
+    })
+  })
+})

convex/maintenance.ts

@@ -0,0 +1,840 @@
+import { ConvexError, v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
+import { action, internalAction, internalMutation, internalQuery } from './_generated/server'
+import { assertRole, requireUserFromAction } from './lib/access'
+import { buildSkillSummaryBackfillPatch, type ParsedSkillData } from './lib/skillBackfill'
+import { hashSkillFiles } from './lib/skills'
+
+const DEFAULT_BATCH_SIZE = 50
+const MAX_BATCH_SIZE = 200
+const DEFAULT_MAX_BATCHES = 20
+const MAX_MAX_BATCHES = 200
+
+type BackfillStats = {
+  skillsScanned: number
+  skillsPatched: number
+  versionsPatched: number
+  missingLatestVersion: number
+  missingReadme: number
+  missingStorageBlob: number
+}
+
+type BackfillPageItem =
+  | {
+      kind: 'ok'
+      skillId: Id<'skills'>
+      versionId: Id<'skillVersions'>
+      skillSummary: Doc<'skills'>['summary']
+      versionParsed: Doc<'skillVersions'>['parsed']
+      readmeStorageId: Id<'_storage'>
+    }
+  | { kind: 'missingLatestVersion'; skillId: Id<'skills'> }
+  | { kind: 'missingVersionDoc'; skillId: Id<'skills'>; versionId: Id<'skillVersions'> }
+  | { kind: 'missingReadme'; skillId: Id<'skills'>; versionId: Id<'skillVersions'> }
+
+type BackfillPageResult = {
+  items: BackfillPageItem[]
+  cursor: string | null
+  isDone: boolean
+}
+
+export const getSkillBackfillPageInternal = internalQuery({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<BackfillPageResult> => {
+    const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('skills')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    const items: BackfillPageItem[] = []
+    for (const skill of page) {
+      if (!skill.latestVersionId) {
+        items.push({ kind: 'missingLatestVersion', skillId: skill._id })
+        continue
+      }
+
+      const version = await ctx.db.get(skill.latestVersionId)
+      if (!version) {
+        items.push({
+          kind: 'missingVersionDoc',
+          skillId: skill._id,
+          versionId: skill.latestVersionId,
+        })
+        continue
+      }
+
+      const readmeFile = version.files.find(
+        (file) => file.path.toLowerCase() === 'skill.md' || file.path.toLowerCase() === 'skills.md',
+      )
+      if (!readmeFile) {
+        items.push({ kind: 'missingReadme', skillId: skill._id, versionId: version._id })
+        continue
+      }
+
+      items.push({
+        kind: 'ok',
+        skillId: skill._id,
+        versionId: version._id,
+        skillSummary: skill.summary,
+        versionParsed: version.parsed,
+        readmeStorageId: readmeFile.storageId,
+      })
+    }
+
+    return { items, cursor: continueCursor, isDone }
+  },
+})
+
+export const applySkillBackfillPatchInternal = internalMutation({
+  args: {
+    skillId: v.id('skills'),
+    versionId: v.id('skillVersions'),
+    summary: v.optional(v.string()),
+    parsed: v.optional(
+      v.object({
+        frontmatter: v.record(v.string(), v.any()),
+        metadata: v.optional(v.any()),
+        clawdis: v.optional(v.any()),
+      }),
+    ),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    if (typeof args.summary === 'string') {
+      await ctx.db.patch(args.skillId, { summary: args.summary, updatedAt: now })
+    }
+    if (args.parsed) {
+      await ctx.db.patch(args.versionId, { parsed: args.parsed })
+    }
+    return { ok: true as const }
+  },
+})
+
+export type BackfillActionArgs = {
+  dryRun?: boolean
+  batchSize?: number
+  maxBatches?: number
+}
+
+export type BackfillActionResult = { ok: true; stats: BackfillStats }
+
+export async function backfillSkillSummariesInternalHandler(
+  ctx: ActionCtx,
+  args: BackfillActionArgs,
+): Promise<BackfillActionResult> {
+  const dryRun = Boolean(args.dryRun)
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+
+  const totals: BackfillStats = {
+    skillsScanned: 0,
+    skillsPatched: 0,
+    versionsPatched: 0,
+    missingLatestVersion: 0,
+    missingReadme: 0,
+    missingStorageBlob: 0,
+  }
+
+  let cursor: string | null = null
+  let isDone = false
+
+  for (let i = 0; i < maxBatches; i++) {
+    const page = (await ctx.runQuery(internal.maintenance.getSkillBackfillPageInternal, {
+      cursor: cursor ?? undefined,
+      batchSize,
+    })) as BackfillPageResult
+
+    cursor = page.cursor
+    isDone = page.isDone
+
+    for (const item of page.items) {
+      totals.skillsScanned++
+      if (item.kind === 'missingLatestVersion') {
+        totals.missingLatestVersion++
+        continue
+      }
+      if (item.kind === 'missingVersionDoc') {
+        totals.missingLatestVersion++
+        continue
+      }
+      if (item.kind === 'missingReadme') {
+        totals.missingReadme++
+        continue
+      }
+
+      const blob = await ctx.storage.get(item.readmeStorageId)
+      if (!blob) {
+        totals.missingStorageBlob++
+        continue
+      }
+
+      const readmeText = await blob.text()
+      const patch = buildSkillSummaryBackfillPatch({
+        readmeText,
+        currentSummary: item.skillSummary ?? undefined,
+        currentParsed: item.versionParsed as ParsedSkillData,
+      })
+
+      if (!patch.summary && !patch.parsed) continue
+      if (patch.summary) totals.skillsPatched++
+      if (patch.parsed) totals.versionsPatched++
+
+      if (dryRun) continue
+
+      await ctx.runMutation(internal.maintenance.applySkillBackfillPatchInternal, {
+        skillId: item.skillId,
+        versionId: item.versionId,
+        summary: patch.summary,
+        parsed: patch.parsed,
+      })
+    }
+
+    if (isDone) break
+  }
+
+  if (!isDone) {
+    throw new ConvexError('Backfill incomplete (maxBatches reached)')
+  }
+
+  return { ok: true as const, stats: totals }
+}
+
+export const backfillSkillSummariesInternal = internalAction({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: backfillSkillSummariesInternalHandler,
+})
+
+export const backfillSkillSummaries: ReturnType<typeof action> = action({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<BackfillActionResult> => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    return ctx.runAction(
+      internal.maintenance.backfillSkillSummariesInternal,
+      args,
+    ) as Promise<BackfillActionResult>
+  },
+})
+
+export const scheduleBackfillSkillSummaries: ReturnType<typeof action> = action({
+  args: { dryRun: v.optional(v.boolean()) },
+  handler: async (ctx, args) => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    await ctx.scheduler.runAfter(0, internal.maintenance.backfillSkillSummariesInternal, {
+      dryRun: Boolean(args.dryRun),
+      batchSize: DEFAULT_BATCH_SIZE,
+      maxBatches: DEFAULT_MAX_BATCHES,
+    })
+    return { ok: true as const }
+  },
+})
+
+type FingerprintBackfillStats = {
+  versionsScanned: number
+  versionsPatched: number
+  fingerprintsInserted: number
+  fingerprintMismatches: number
+}
+
+type FingerprintBackfillPageItem = {
+  skillId: Id<'skills'>
+  versionId: Id<'skillVersions'>
+  versionFingerprint?: string
+  files: Array<{ path: string; sha256: string }>
+  existingEntries: Array<{ id: Id<'skillVersionFingerprints'>; fingerprint: string }>
+}
+
+type FingerprintBackfillPageResult = {
+  items: FingerprintBackfillPageItem[]
+  cursor: string | null
+  isDone: boolean
+}
+
+type BadgeBackfillStats = {
+  skillsScanned: number
+  skillsPatched: number
+  highlightsPatched: number
+}
+
+type SkillBadgeTableBackfillStats = {
+  skillsScanned: number
+  recordsInserted: number
+}
+
+type BadgeBackfillPageItem = {
+  skillId: Id<'skills'>
+  ownerUserId: Id<'users'>
+  createdAt?: number
+  updatedAt?: number
+  batch?: string
+  badges?: Doc<'skills'>['badges']
+}
+
+type BadgeBackfillPageResult = {
+  items: BadgeBackfillPageItem[]
+  cursor: string | null
+  isDone: boolean
+}
+
+type BadgeKind = Doc<'skillBadges'>['kind']
+
+export const getSkillFingerprintBackfillPageInternal = internalQuery({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<FingerprintBackfillPageResult> => {
+    const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('skillVersions')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    const items: FingerprintBackfillPageItem[] = []
+    for (const version of page) {
+      const existingEntries = await ctx.db
+        .query('skillVersionFingerprints')
+        .withIndex('by_version', (q) => q.eq('versionId', version._id))
+        .take(20)
+
+      const normalizedFiles = version.files.map((file) => ({
+        path: file.path,
+        sha256: file.sha256,
+      }))
+
+      const hasAnyEntry = existingEntries.length > 0
+      const entryFingerprints = new Set(existingEntries.map((entry) => entry.fingerprint))
+      const hasFingerprintMismatch =
+        typeof version.fingerprint === 'string' &&
+        hasAnyEntry &&
+        (entryFingerprints.size !== 1 || !entryFingerprints.has(version.fingerprint))
+      const needsFingerprintField = !version.fingerprint
+      const needsFingerprintEntry = !hasAnyEntry
+
+      if (!needsFingerprintField && !needsFingerprintEntry && !hasFingerprintMismatch) continue
+
+      items.push({
+        skillId: version.skillId,
+        versionId: version._id,
+        versionFingerprint: version.fingerprint ?? undefined,
+        files: normalizedFiles,
+        existingEntries: existingEntries.map((entry) => ({
+          id: entry._id,
+          fingerprint: entry.fingerprint,
+        })),
+      })
+    }
+
+    return { items, cursor: continueCursor, isDone }
+  },
+})
+
+export const applySkillFingerprintBackfillPatchInternal = internalMutation({
+  args: {
+    versionId: v.id('skillVersions'),
+    fingerprint: v.string(),
+    patchVersion: v.boolean(),
+    replaceEntries: v.boolean(),
+    existingEntryIds: v.optional(v.array(v.id('skillVersionFingerprints'))),
+  },
+  handler: async (ctx, args) => {
+    const version = await ctx.db.get(args.versionId)
+    if (!version) return { ok: false as const, reason: 'missingVersion' as const }
+
+    const now = Date.now()
+
+    if (args.patchVersion) {
+      await ctx.db.patch(version._id, { fingerprint: args.fingerprint })
+    }
+
+    if (args.replaceEntries) {
+      const existing = args.existingEntryIds ?? []
+      for (const id of existing) {
+        await ctx.db.delete(id)
+      }
+
+      await ctx.db.insert('skillVersionFingerprints', {
+        skillId: version.skillId,
+        versionId: version._id,
+        fingerprint: args.fingerprint,
+        createdAt: now,
+      })
+    }
+
+    return { ok: true as const }
+  },
+})
+
+export type FingerprintBackfillActionArgs = {
+  dryRun?: boolean
+  batchSize?: number
+  maxBatches?: number
+}
+
+export type FingerprintBackfillActionResult = { ok: true; stats: FingerprintBackfillStats }
+
+export async function backfillSkillFingerprintsInternalHandler(
+  ctx: ActionCtx,
+  args: FingerprintBackfillActionArgs,
+): Promise<FingerprintBackfillActionResult> {
+  const dryRun = Boolean(args.dryRun)
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+
+  const totals: FingerprintBackfillStats = {
+    versionsScanned: 0,
+    versionsPatched: 0,
+    fingerprintsInserted: 0,
+    fingerprintMismatches: 0,
+  }
+
+  let cursor: string | null = null
+  let isDone = false
+
+  for (let i = 0; i < maxBatches; i++) {
+    const page = (await ctx.runQuery(internal.maintenance.getSkillFingerprintBackfillPageInternal, {
+      cursor: cursor ?? undefined,
+      batchSize,
+    })) as FingerprintBackfillPageResult
+
+    cursor = page.cursor
+    isDone = page.isDone
+
+    for (const item of page.items) {
+      totals.versionsScanned++
+
+      const fingerprint = await hashSkillFiles(item.files)
+
+      const existingFingerprints = new Set(item.existingEntries.map((entry) => entry.fingerprint))
+      const hasAnyEntry = item.existingEntries.length > 0
+      const entryIsCorrect =
+        hasAnyEntry && existingFingerprints.size === 1 && existingFingerprints.has(fingerprint)
+      const versionFingerprintIsCorrect = item.versionFingerprint === fingerprint
+
+      if (hasAnyEntry && !entryIsCorrect) totals.fingerprintMismatches++
+
+      const shouldPatchVersion = !versionFingerprintIsCorrect
+      const shouldReplaceEntries = !entryIsCorrect
+      if (!shouldPatchVersion && !shouldReplaceEntries) continue
+
+      if (shouldPatchVersion) totals.versionsPatched++
+      if (shouldReplaceEntries) totals.fingerprintsInserted++
+
+      if (dryRun) continue
+
+      await ctx.runMutation(internal.maintenance.applySkillFingerprintBackfillPatchInternal, {
+        versionId: item.versionId,
+        fingerprint,
+        patchVersion: shouldPatchVersion,
+        replaceEntries: shouldReplaceEntries,
+        existingEntryIds: shouldReplaceEntries ? item.existingEntries.map((entry) => entry.id) : [],
+      })
+    }
+
+    if (isDone) break
+  }
+
+  if (!isDone) {
+    throw new ConvexError('Backfill incomplete (maxBatches reached)')
+  }
+
+  return { ok: true as const, stats: totals }
+}
+
+export const backfillSkillFingerprintsInternal = internalAction({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: backfillSkillFingerprintsInternalHandler,
+})
+
+export const backfillSkillFingerprints: ReturnType<typeof action> = action({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<FingerprintBackfillActionResult> => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    return ctx.runAction(
+      internal.maintenance.backfillSkillFingerprintsInternal,
+      args,
+    ) as Promise<FingerprintBackfillActionResult>
+  },
+})
+
+export const scheduleBackfillSkillFingerprints: ReturnType<typeof action> = action({
+  args: { dryRun: v.optional(v.boolean()) },
+  handler: async (ctx, args) => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    await ctx.scheduler.runAfter(0, internal.maintenance.backfillSkillFingerprintsInternal, {
+      dryRun: Boolean(args.dryRun),
+      batchSize: DEFAULT_BATCH_SIZE,
+      maxBatches: DEFAULT_MAX_BATCHES,
+    })
+    return { ok: true as const }
+  },
+})
+
+export const getSkillBadgeBackfillPageInternal = internalQuery({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<BadgeBackfillPageResult> => {
+    const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('skills')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    const items: BadgeBackfillPageItem[] = page.map((skill) => ({
+      skillId: skill._id,
+      ownerUserId: skill.ownerUserId,
+      createdAt: skill.createdAt ?? undefined,
+      updatedAt: skill.updatedAt ?? undefined,
+      batch: skill.batch ?? undefined,
+      badges: skill.badges ?? undefined,
+    }))
+
+    return { items, cursor: continueCursor, isDone }
+  },
+})
+
+export const applySkillBadgeBackfillPatchInternal = internalMutation({
+  args: {
+    skillId: v.id('skills'),
+    badges: v.optional(
+      v.object({
+        redactionApproved: v.optional(
+          v.object({
+            byUserId: v.id('users'),
+            at: v.number(),
+          }),
+        ),
+        highlighted: v.optional(
+          v.object({
+            byUserId: v.id('users'),
+            at: v.number(),
+          }),
+        ),
+        official: v.optional(
+          v.object({
+            byUserId: v.id('users'),
+            at: v.number(),
+          }),
+        ),
+        deprecated: v.optional(
+          v.object({
+            byUserId: v.id('users'),
+            at: v.number(),
+          }),
+        ),
+      }),
+    ),
+  },
+  handler: async (ctx, args) => {
+    await ctx.db.patch(args.skillId, { badges: args.badges ?? undefined, updatedAt: Date.now() })
+    return { ok: true as const }
+  },
+})
+
+export const upsertSkillBadgeRecordInternal = internalMutation({
+  args: {
+    skillId: v.id('skills'),
+    kind: v.union(
+      v.literal('highlighted'),
+      v.literal('official'),
+      v.literal('deprecated'),
+      v.literal('redactionApproved'),
+    ),
+    byUserId: v.id('users'),
+    at: v.number(),
+  },
+  handler: async (ctx, args) => {
+    const existing = await ctx.db
+      .query('skillBadges')
+      .withIndex('by_skill_kind', (q) => q.eq('skillId', args.skillId).eq('kind', args.kind))
+      .unique()
+    if (existing) return { inserted: false as const }
+    await ctx.db.insert('skillBadges', {
+      skillId: args.skillId,
+      kind: args.kind,
+      byUserId: args.byUserId,
+      at: args.at,
+    })
+    return { inserted: true as const }
+  },
+})
+
+export type BadgeBackfillActionArgs = {
+  dryRun?: boolean
+  batchSize?: number
+  maxBatches?: number
+}
+
+export type BadgeBackfillActionResult = { ok: true; stats: BadgeBackfillStats }
+
+export async function backfillSkillBadgesInternalHandler(
+  ctx: ActionCtx,
+  args: BadgeBackfillActionArgs,
+): Promise<BadgeBackfillActionResult> {
+  const dryRun = Boolean(args.dryRun)
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+
+  const totals: BadgeBackfillStats = {
+    skillsScanned: 0,
+    skillsPatched: 0,
+    highlightsPatched: 0,
+  }
+
+  let cursor: string | null = null
+  let isDone = false
+
+  for (let i = 0; i < maxBatches; i++) {
+    const page = (await ctx.runQuery(internal.maintenance.getSkillBadgeBackfillPageInternal, {
+      cursor: cursor ?? undefined,
+      batchSize,
+    })) as BadgeBackfillPageResult
+
+    cursor = page.cursor
+    isDone = page.isDone
+
+    for (const item of page.items) {
+      totals.skillsScanned++
+
+      const shouldHighlight = item.batch === 'highlighted' && !item.badges?.highlighted
+      if (!shouldHighlight) continue
+
+      totals.skillsPatched++
+      totals.highlightsPatched++
+
+      if (dryRun) continue
+
+      const at = item.updatedAt ?? item.createdAt ?? Date.now()
+      await ctx.runMutation(internal.maintenance.applySkillBadgeBackfillPatchInternal, {
+        skillId: item.skillId,
+        badges: {
+          ...item.badges,
+          highlighted: {
+            byUserId: item.ownerUserId,
+            at,
+          },
+        },
+      })
+    }
+
+    if (isDone) break
+  }
+
+  if (!isDone) {
+    throw new ConvexError('Backfill incomplete (maxBatches reached)')
+  }
+
+  return { ok: true as const, stats: totals }
+}
+
+export const backfillSkillBadgesInternal = internalAction({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: backfillSkillBadgesInternalHandler,
+})
+
+export const backfillSkillBadges: ReturnType<typeof action> = action({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<BadgeBackfillActionResult> => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    return ctx.runAction(
+      internal.maintenance.backfillSkillBadgesInternal,
+      args,
+    ) as Promise<BadgeBackfillActionResult>
+  },
+})
+
+export const scheduleBackfillSkillBadges: ReturnType<typeof action> = action({
+  args: { dryRun: v.optional(v.boolean()) },
+  handler: async (ctx, args) => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    await ctx.scheduler.runAfter(0, internal.maintenance.backfillSkillBadgesInternal, {
+      dryRun: Boolean(args.dryRun),
+      batchSize: DEFAULT_BATCH_SIZE,
+      maxBatches: DEFAULT_MAX_BATCHES,
+    })
+    return { ok: true as const }
+  },
+})
+
+export type SkillBadgeTableBackfillActionResult = {
+  ok: true
+  stats: SkillBadgeTableBackfillStats
+}
+
+export async function backfillSkillBadgeTableInternalHandler(
+  ctx: ActionCtx,
+  args: BadgeBackfillActionArgs,
+): Promise<SkillBadgeTableBackfillActionResult> {
+  const dryRun = Boolean(args.dryRun)
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+
+  const totals: SkillBadgeTableBackfillStats = {
+    skillsScanned: 0,
+    recordsInserted: 0,
+  }
+
+  let cursor: string | null = null
+  let isDone = false
+
+  for (let i = 0; i < maxBatches; i++) {
+    const page = (await ctx.runQuery(internal.maintenance.getSkillBadgeBackfillPageInternal, {
+      cursor: cursor ?? undefined,
+      batchSize,
+    })) as BadgeBackfillPageResult
+
+    cursor = page.cursor
+    isDone = page.isDone
+
+    for (const item of page.items) {
+      totals.skillsScanned++
+      const badges = item.badges ?? {}
+      const entries: Array<{ kind: BadgeKind; byUserId: Id<'users'>; at: number }> = []
+
+      if (badges.redactionApproved) {
+        entries.push({
+          kind: 'redactionApproved',
+          byUserId: badges.redactionApproved.byUserId,
+          at: badges.redactionApproved.at,
+        })
+      }
+
+      if (badges.official) {
+        entries.push({
+          kind: 'official',
+          byUserId: badges.official.byUserId,
+          at: badges.official.at,
+        })
+      }
+
+      if (badges.deprecated) {
+        entries.push({
+          kind: 'deprecated',
+          byUserId: badges.deprecated.byUserId,
+          at: badges.deprecated.at,
+        })
+      }
+
+      const highlighted =
+        badges.highlighted ??
+        (item.batch === 'highlighted'
+          ? {
+              byUserId: item.ownerUserId,
+              at: item.updatedAt ?? item.createdAt ?? Date.now(),
+            }
+          : undefined)
+
+      if (highlighted) {
+        entries.push({
+          kind: 'highlighted',
+          byUserId: highlighted.byUserId,
+          at: highlighted.at,
+        })
+      }
+
+      if (dryRun) continue
+
+      for (const entry of entries) {
+        const result = await ctx.runMutation(internal.maintenance.upsertSkillBadgeRecordInternal, {
+          skillId: item.skillId,
+          kind: entry.kind,
+          byUserId: entry.byUserId,
+          at: entry.at,
+        })
+        if (result.inserted) {
+          totals.recordsInserted++
+        }
+      }
+    }
+
+    if (isDone) break
+  }
+
+  if (!isDone) {
+    throw new ConvexError('Backfill incomplete (maxBatches reached)')
+  }
+
+  return { ok: true as const, stats: totals }
+}
+
+export const backfillSkillBadgeTableInternal = internalAction({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: backfillSkillBadgeTableInternalHandler,
+})
+
+export const backfillSkillBadgeTable: ReturnType<typeof action> = action({
+  args: {
+    dryRun: v.optional(v.boolean()),
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<SkillBadgeTableBackfillActionResult> => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    return ctx.runAction(
+      internal.maintenance.backfillSkillBadgeTableInternal,
+      args,
+    ) as Promise<SkillBadgeTableBackfillActionResult>
+  },
+})
+
+export const scheduleBackfillSkillBadgeTable: ReturnType<typeof action> = action({
+  args: { dryRun: v.optional(v.boolean()) },
+  handler: async (ctx, args) => {
+    const { user } = await requireUserFromAction(ctx)
+    assertRole(user, ['admin'])
+    await ctx.scheduler.runAfter(0, internal.maintenance.backfillSkillBadgeTableInternal, {
+      dryRun: Boolean(args.dryRun),
+      batchSize: DEFAULT_BATCH_SIZE,
+      maxBatches: DEFAULT_MAX_BATCHES,
+    })
+    return { ok: true as const }
+  },
+})
+
+function clampInt(value: number, min: number, max: number) {
+  const rounded = Math.trunc(value)
+  if (!Number.isFinite(rounded)) return min
+  return Math.min(max, Math.max(min, rounded))
+}

convex/rateLimits.ts

@@ -0,0 +1,85 @@
+import { v } from 'convex/values'
+import { internalMutation, internalQuery } from './_generated/server'
+
+/**
+ * Read-only rate limit check. Returns current status without writing anything.
+ * This eliminates write conflicts for denied requests entirely.
+ */
+export const getRateLimitStatusInternal = internalQuery({
+  args: {
+    key: v.string(),
+    limit: v.number(),
+    windowMs: v.number(),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    const windowStart = Math.floor(now / args.windowMs) * args.windowMs
+    const resetAt = windowStart + args.windowMs
+    if (args.limit <= 0) {
+      return { allowed: false, remaining: 0, limit: args.limit, resetAt }
+    }
+
+    const existing = await ctx.db
+      .query('rateLimits')
+      .withIndex('by_key_window', (q) => q.eq('key', args.key).eq('windowStart', windowStart))
+      .unique()
+
+    const count = existing?.count ?? 0
+    const allowed = count < args.limit
+    return {
+      allowed,
+      remaining: Math.max(0, args.limit - count),
+      limit: args.limit,
+      resetAt,
+    }
+  },
+})
+
+/**
+ * Consume one rate limit token. Only call this after getRateLimitStatusInternal
+ * returns allowed=true. Includes a double-check to handle races between the
+ * query and this mutation.
+ */
+export const consumeRateLimitInternal = internalMutation({
+  args: {
+    key: v.string(),
+    limit: v.number(),
+    windowMs: v.number(),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    const windowStart = Math.floor(now / args.windowMs) * args.windowMs
+
+    const existing = await ctx.db
+      .query('rateLimits')
+      .withIndex('by_key_window', (q) => q.eq('key', args.key).eq('windowStart', windowStart))
+      .unique()
+
+    // Double-check: another request may have consumed the last token
+    // between our query and this mutation
+    if (existing && existing.count >= args.limit) {
+      return { allowed: false, remaining: 0 }
+    }
+
+    if (!existing) {
+      await ctx.db.insert('rateLimits', {
+        key: args.key,
+        windowStart,
+        count: 1,
+        limit: args.limit,
+        updatedAt: now,
+      })
+      return { allowed: true, remaining: Math.max(0, args.limit - 1) }
+    }
+
+    await ctx.db.patch(existing._id, {
+      count: existing.count + 1,
+      limit: args.limit,
+      updatedAt: now,
+    })
+    return {
+      allowed: true,
+      remaining: Math.max(0, args.limit - existing.count - 1),
+    }
+  },
+})

convex/schema.ts

@@ -3,6 +3,8 @@ import { defineSchema, defineTable } from 'convex/server'
 import { v } from 'convex/values'
 import { EMBEDDING_DIMENSIONS } from './lib/embeddings'
 
+const authSchema = authTables as unknown as Record<string, ReturnType<typeof defineTable>>
+
 const users = defineTable({
   name: v.optional(v.string()),
   image: v.optional(v.string()),
@@ -15,7 +17,10 @@ const users = defineTable({
   displayName: v.optional(v.string()),
   bio: v.optional(v.string()),
   role: v.optional(v.union(v.literal('admin'), v.literal('moderator'), v.literal('user'))),
+  githubCreatedAt: v.optional(v.number()),
+  githubFetchedAt: v.optional(v.number()),
   deletedAt: v.optional(v.number()),
+  banReason: v.optional(v.string()),
   createdAt: v.optional(v.number()),
   updatedAt: v.optional(v.number()),
 })
@@ -27,19 +32,98 @@ const skills = defineTable({
   slug: v.string(),
   displayName: v.string(),
   summary: v.optional(v.string()),
+  resourceId: v.optional(v.string()),
   ownerUserId: v.id('users'),
+  canonicalSkillId: v.optional(v.id('skills')),
+  forkOf: v.optional(
+    v.object({
+      skillId: v.id('skills'),
+      kind: v.union(v.literal('fork'), v.literal('duplicate')),
+      version: v.optional(v.string()),
+      at: v.number(),
+    }),
+  ),
   latestVersionId: v.optional(v.id('skillVersions')),
   tags: v.record(v.string(), v.id('skillVersions')),
   softDeletedAt: v.optional(v.number()),
-  badges: v.object({
-    redactionApproved: v.optional(
-      v.object({
-        byUserId: v.id('users'),
-        at: v.number(),
-      }),
-    ),
-  }),
+  badges: v.optional(
+    v.object({
+      redactionApproved: v.optional(
+        v.object({
+          byUserId: v.id('users'),
+          at: v.number(),
+        }),
+      ),
+      highlighted: v.optional(
+        v.object({
+          byUserId: v.id('users'),
+          at: v.number(),
+        }),
+      ),
+      official: v.optional(
+        v.object({
+          byUserId: v.id('users'),
+          at: v.number(),
+        }),
+      ),
+      deprecated: v.optional(
+        v.object({
+          byUserId: v.id('users'),
+          at: v.number(),
+        }),
+      ),
+    }),
+  ),
+  moderationStatus: v.optional(
+    v.union(v.literal('active'), v.literal('hidden'), v.literal('removed')),
+  ),
+  moderationNotes: v.optional(v.string()),
+  moderationReason: v.optional(v.string()),
+  moderationFlags: v.optional(v.array(v.string())),
+  lastReviewedAt: v.optional(v.number()),
+  // VT scan tracking
+  scanLastCheckedAt: v.optional(v.number()),
+  scanCheckCount: v.optional(v.number()),
+  hiddenAt: v.optional(v.number()),
+  hiddenBy: v.optional(v.id('users')),
+  reportCount: v.optional(v.number()),
+  lastReportedAt: v.optional(v.number()),
   batch: v.optional(v.string()),
+  statsDownloads: v.optional(v.number()),
+  statsStars: v.optional(v.number()),
+  statsInstallsCurrent: v.optional(v.number()),
+  statsInstallsAllTime: v.optional(v.number()),
+  stats: v.object({
+    downloads: v.number(),
+    installsCurrent: v.optional(v.number()),
+    installsAllTime: v.optional(v.number()),
+    stars: v.number(),
+    versions: v.number(),
+    comments: v.number(),
+  }),
+  createdAt: v.number(),
+  updatedAt: v.number(),
+})
+  .index('by_slug', ['slug'])
+  .index('by_owner', ['ownerUserId'])
+  .index('by_updated', ['updatedAt'])
+  .index('by_stats_downloads', ['statsDownloads', 'updatedAt'])
+  .index('by_stats_stars', ['statsStars', 'updatedAt'])
+  .index('by_stats_installs_current', ['statsInstallsCurrent', 'updatedAt'])
+  .index('by_stats_installs_all_time', ['statsInstallsAllTime', 'updatedAt'])
+  .index('by_batch', ['batch'])
+  .index('by_active_updated', ['softDeletedAt', 'updatedAt'])
+  .index('by_canonical', ['canonicalSkillId'])
+  .index('by_fork_of', ['forkOf.skillId'])
+
+const souls = defineTable({
+  slug: v.string(),
+  displayName: v.string(),
+  summary: v.optional(v.string()),
+  ownerUserId: v.id('users'),
+  latestVersionId: v.optional(v.id('soulVersions')),
+  tags: v.record(v.string(), v.id('soulVersions')),
+  softDeletedAt: v.optional(v.number()),
   stats: v.object({
     downloads: v.number(),
     stars: v.number(),
@@ -52,12 +136,13 @@ const skills = defineTable({
   .index('by_slug', ['slug'])
   .index('by_owner', ['ownerUserId'])
   .index('by_updated', ['updatedAt'])
-  .index('by_batch', ['batch'])
 
 const skillVersions = defineTable({
   skillId: v.id('skills'),
   version: v.string(),
+  fingerprint: v.optional(v.string()),
   changelog: v.string(),
+  changelogSource: v.optional(v.union(v.literal('auto'), v.literal('user'))),
   files: v.array(
     v.object({
       path: v.string(),
@@ -68,16 +153,113 @@ const skillVersions = defineTable({
     }),
   ),
   parsed: v.object({
-    frontmatter: v.record(v.string(), v.string()),
+    frontmatter: v.record(v.string(), v.any()),
     metadata: v.optional(v.any()),
     clawdis: v.optional(v.any()),
+    moltbot: v.optional(v.any()),
+  }),
+  createdBy: v.id('users'),
+  createdAt: v.number(),
+  softDeletedAt: v.optional(v.number()),
+  sha256hash: v.optional(v.string()),
+  vtAnalysis: v.optional(
+    v.object({
+      status: v.string(),
+      verdict: v.optional(v.string()),
+      analysis: v.optional(v.string()),
+      source: v.optional(v.string()),
+      checkedAt: v.number(),
+    }),
+  ),
+  llmAnalysis: v.optional(
+    v.object({
+      status: v.string(),
+      verdict: v.optional(v.string()),
+      confidence: v.optional(v.string()),
+      summary: v.optional(v.string()),
+      dimensions: v.optional(
+        v.array(
+          v.object({
+            name: v.string(),
+            label: v.string(),
+            rating: v.string(),
+            detail: v.string(),
+          }),
+        ),
+      ),
+      guidance: v.optional(v.string()),
+      findings: v.optional(v.string()),
+      model: v.optional(v.string()),
+      checkedAt: v.number(),
+    }),
+  ),
+})
+  .index('by_skill', ['skillId'])
+  .index('by_skill_version', ['skillId', 'version'])
+  .index('by_sha256hash', ['sha256hash'])
+
+const soulVersions = defineTable({
+  soulId: v.id('souls'),
+  version: v.string(),
+  fingerprint: v.optional(v.string()),
+  changelog: v.string(),
+  changelogSource: v.optional(v.union(v.literal('auto'), v.literal('user'))),
+  files: v.array(
+    v.object({
+      path: v.string(),
+      size: v.number(),
+      storageId: v.id('_storage'),
+      sha256: v.string(),
+      contentType: v.optional(v.string()),
+    }),
+  ),
+  parsed: v.object({
+    frontmatter: v.record(v.string(), v.any()),
+    metadata: v.optional(v.any()),
+    clawdis: v.optional(v.any()),
+    moltbot: v.optional(v.any()),
   }),
   createdBy: v.id('users'),
   createdAt: v.number(),
   softDeletedAt: v.optional(v.number()),
+})
+  .index('by_soul', ['soulId'])
+  .index('by_soul_version', ['soulId', 'version'])
+
+const skillVersionFingerprints = defineTable({
+  skillId: v.id('skills'),
+  versionId: v.id('skillVersions'),
+  fingerprint: v.string(),
+  createdAt: v.number(),
+})
+  .index('by_version', ['versionId'])
+  .index('by_fingerprint', ['fingerprint'])
+  .index('by_skill_fingerprint', ['skillId', 'fingerprint'])
+
+const skillBadges = defineTable({
+  skillId: v.id('skills'),
+  kind: v.union(
+    v.literal('highlighted'),
+    v.literal('official'),
+    v.literal('deprecated'),
+    v.literal('redactionApproved'),
+  ),
+  byUserId: v.id('users'),
+  at: v.number(),
 })
   .index('by_skill', ['skillId'])
-  .index('by_skill_version', ['skillId', 'version'])
+  .index('by_skill_kind', ['skillId', 'kind'])
+  .index('by_kind_at', ['kind', 'at'])
+
+const soulVersionFingerprints = defineTable({
+  soulId: v.id('souls'),
+  versionId: v.id('soulVersions'),
+  fingerprint: v.string(),
+  createdAt: v.number(),
+})
+  .index('by_version', ['versionId'])
+  .index('by_fingerprint', ['fingerprint'])
+  .index('by_soul_fingerprint', ['soulId', 'fingerprint'])
 
 const skillEmbeddings = defineTable({
   skillId: v.id('skills'),
@@ -97,6 +279,87 @@ const skillEmbeddings = defineTable({
     filterFields: ['visibility'],
   })
 
+const skillDailyStats = defineTable({
+  skillId: v.id('skills'),
+  day: v.number(),
+  downloads: v.number(),
+  installs: v.number(),
+  updatedAt: v.number(),
+})
+  .index('by_skill_day', ['skillId', 'day'])
+  .index('by_day', ['day'])
+
+const skillLeaderboards = defineTable({
+  kind: v.string(),
+  generatedAt: v.number(),
+  rangeStartDay: v.number(),
+  rangeEndDay: v.number(),
+  items: v.array(
+    v.object({
+      skillId: v.id('skills'),
+      score: v.number(),
+      installs: v.number(),
+      downloads: v.number(),
+    }),
+  ),
+}).index('by_kind', ['kind', 'generatedAt'])
+
+const skillStatBackfillState = defineTable({
+  key: v.string(),
+  cursor: v.optional(v.string()),
+  doneAt: v.optional(v.number()),
+  updatedAt: v.number(),
+}).index('by_key', ['key'])
+
+const skillStatEvents = defineTable({
+  skillId: v.id('skills'),
+  kind: v.union(
+    v.literal('download'),
+    v.literal('star'),
+    v.literal('unstar'),
+    v.literal('comment'),
+    v.literal('uncomment'),
+    v.literal('install_new'),
+    v.literal('install_reactivate'),
+    v.literal('install_deactivate'),
+    v.literal('install_clear'),
+  ),
+  delta: v.optional(
+    v.object({
+      allTime: v.number(),
+      current: v.number(),
+    }),
+  ),
+  occurredAt: v.number(),
+  processedAt: v.optional(v.number()),
+})
+  .index('by_unprocessed', ['processedAt'])
+  .index('by_skill', ['skillId'])
+
+const skillStatUpdateCursors = defineTable({
+  key: v.string(),
+  cursorCreationTime: v.optional(v.number()),
+  updatedAt: v.number(),
+}).index('by_key', ['key'])
+
+const soulEmbeddings = defineTable({
+  soulId: v.id('souls'),
+  versionId: v.id('soulVersions'),
+  ownerId: v.id('users'),
+  embedding: v.array(v.number()),
+  isLatest: v.boolean(),
+  isApproved: v.boolean(),
+  visibility: v.string(),
+  updatedAt: v.number(),
+})
+  .index('by_soul', ['soulId'])
+  .index('by_version', ['versionId'])
+  .vectorIndex('by_embedding', {
+    vectorField: 'embedding',
+    dimensions: EMBEDDING_DIMENSIONS,
+    filterFields: ['visibility'],
+  })
+
 const comments = defineTable({
   skillId: v.id('skills'),
   userId: v.id('users'),
@@ -108,6 +371,28 @@ const comments = defineTable({
   .index('by_skill', ['skillId'])
   .index('by_user', ['userId'])
 
+const skillReports = defineTable({
+  skillId: v.id('skills'),
+  userId: v.id('users'),
+  reason: v.optional(v.string()),
+  createdAt: v.number(),
+})
+  .index('by_skill', ['skillId'])
+  .index('by_skill_createdAt', ['skillId', 'createdAt'])
+  .index('by_user', ['userId'])
+  .index('by_skill_user', ['skillId', 'userId'])
+
+const soulComments = defineTable({
+  soulId: v.id('souls'),
+  userId: v.id('users'),
+  body: v.string(),
+  createdAt: v.number(),
+  softDeletedAt: v.optional(v.number()),
+  deletedBy: v.optional(v.id('users')),
+})
+  .index('by_soul', ['soulId'])
+  .index('by_user', ['userId'])
+
 const stars = defineTable({
   skillId: v.id('skills'),
   userId: v.id('users'),
@@ -117,6 +402,15 @@ const stars = defineTable({
   .index('by_user', ['userId'])
   .index('by_skill_user', ['skillId', 'userId'])
 
+const soulStars = defineTable({
+  soulId: v.id('souls'),
+  userId: v.id('users'),
+  createdAt: v.number(),
+})
+  .index('by_soul', ['soulId'])
+  .index('by_user', ['userId'])
+  .index('by_soul_user', ['soulId', 'userId'])
+
 const auditLogs = defineTable({
   actorUserId: v.id('users'),
   action: v.string(),
@@ -128,6 +422,24 @@ const auditLogs = defineTable({
   .index('by_actor', ['actorUserId'])
   .index('by_target', ['targetType', 'targetId'])
 
+const vtScanLogs = defineTable({
+  type: v.union(v.literal('daily_rescan'), v.literal('backfill'), v.literal('pending_poll')),
+  total: v.number(),
+  updated: v.number(),
+  unchanged: v.number(),
+  errors: v.number(),
+  flaggedSkills: v.optional(
+    v.array(
+      v.object({
+        slug: v.string(),
+        status: v.string(),
+      }),
+    ),
+  ),
+  durationMs: v.number(),
+  createdAt: v.number(),
+}).index('by_type_date', ['type', 'createdAt'])
+
 const apiTokens = defineTable({
   userId: v.id('users'),
   label: v.string(),
@@ -140,14 +452,88 @@ const apiTokens = defineTable({
   .index('by_user', ['userId'])
   .index('by_hash', ['tokenHash'])
 
+const rateLimits = defineTable({
+  key: v.string(),
+  windowStart: v.number(),
+  count: v.number(),
+  limit: v.number(),
+  updatedAt: v.number(),
+})
+  .index('by_key_window', ['key', 'windowStart'])
+  .index('by_key', ['key'])
+
+const githubBackupSyncState = defineTable({
+  key: v.string(),
+  cursor: v.optional(v.string()),
+  updatedAt: v.number(),
+}).index('by_key', ['key'])
+
+const userSyncRoots = defineTable({
+  userId: v.id('users'),
+  rootId: v.string(),
+  label: v.string(),
+  firstSeenAt: v.number(),
+  lastSeenAt: v.number(),
+  expiredAt: v.optional(v.number()),
+})
+  .index('by_user', ['userId'])
+  .index('by_user_root', ['userId', 'rootId'])
+
+const userSkillInstalls = defineTable({
+  userId: v.id('users'),
+  skillId: v.id('skills'),
+  firstSeenAt: v.number(),
+  lastSeenAt: v.number(),
+  activeRoots: v.number(),
+  lastVersion: v.optional(v.string()),
+})
+  .index('by_user', ['userId'])
+  .index('by_user_skill', ['userId', 'skillId'])
+  .index('by_skill', ['skillId'])
+
+const userSkillRootInstalls = defineTable({
+  userId: v.id('users'),
+  rootId: v.string(),
+  skillId: v.id('skills'),
+  firstSeenAt: v.number(),
+  lastSeenAt: v.number(),
+  lastVersion: v.optional(v.string()),
+  removedAt: v.optional(v.number()),
+})
+  .index('by_user', ['userId'])
+  .index('by_user_root', ['userId', 'rootId'])
+  .index('by_user_root_skill', ['userId', 'rootId', 'skillId'])
+  .index('by_user_skill', ['userId', 'skillId'])
+  .index('by_skill', ['skillId'])
+
 export default defineSchema({
-  ...authTables,
+  ...authSchema,
   users,
   skills,
+  souls,
   skillVersions,
+  soulVersions,
+  skillVersionFingerprints,
+  skillBadges,
+  soulVersionFingerprints,
   skillEmbeddings,
+  soulEmbeddings,
+  skillDailyStats,
+  skillLeaderboards,
+  skillStatBackfillState,
+  skillStatEvents,
+  skillStatUpdateCursors,
   comments,
+  skillReports,
+  soulComments,
   stars,
+  soulStars,
   auditLogs,
+  vtScanLogs,
   apiTokens,
+  rateLimits,
+  githubBackupSyncState,
+  userSyncRoots,
+  userSkillInstalls,
+  userSkillRootInstalls,
 })

convex/search.test.ts

@@ -0,0 +1,305 @@
+/* @vitest-environment node */
+
+import { describe, expect, it, vi } from 'vitest'
+import { tokenize } from './lib/searchText'
+import { __test, lexicalFallbackSkills, searchSkills } from './search'
+
+const { generateEmbeddingMock, getSkillBadgeMapsMock } = vi.hoisted(() => ({
+  generateEmbeddingMock: vi.fn(),
+  getSkillBadgeMapsMock: vi.fn(),
+}))
+
+vi.mock('./lib/embeddings', () => ({
+  generateEmbedding: generateEmbeddingMock,
+}))
+
+vi.mock('./lib/badges', () => ({
+  getSkillBadgeMaps: getSkillBadgeMapsMock,
+  isSkillHighlighted: (skill: { badges?: Record<string, unknown> }) =>
+    Boolean(skill.badges?.highlighted),
+}))
+
+type WrappedHandler = {
+  _handler: (ctx: unknown, args: unknown) => Promise<unknown>
+}
+
+const searchSkillsHandler = (searchSkills as unknown as WrappedHandler)._handler
+const lexicalFallbackSkillsHandler = (lexicalFallbackSkills as unknown as WrappedHandler)._handler
+
+describe('search helpers', () => {
+  it('returns fallback results when vector candidates are empty', async () => {
+    generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2])
+    const fallback = [
+      {
+        skill: makePublicSkill({ id: 'skills:orf', slug: 'orf', displayName: 'ORF' }),
+        version: null,
+        ownerHandle: 'steipete',
+      },
+    ]
+    const runQuery = vi
+      .fn()
+      .mockResolvedValueOnce([])
+      .mockResolvedValueOnce([])
+      .mockResolvedValueOnce(fallback)
+
+    const result = await searchSkillsHandler(
+      {
+        vectorSearch: vi.fn().mockResolvedValue([]),
+        runQuery,
+      },
+      { query: 'orf', limit: 10 },
+    )
+
+    expect(result).toHaveLength(1)
+    expect(result[0].skill.slug).toBe('orf')
+    expect(runQuery).toHaveBeenLastCalledWith(
+      expect.anything(),
+      expect.objectContaining({ query: 'orf', queryTokens: ['orf'] }),
+    )
+  })
+
+  it('applies highlightedOnly filtering in lexical fallback', async () => {
+    const highlighted = makeSkillDoc({
+      id: 'skills:hl',
+      slug: 'orf-highlighted',
+      displayName: 'ORF Highlighted',
+    })
+    const plain = makeSkillDoc({ id: 'skills:plain', slug: 'orf-plain', displayName: 'ORF Plain' })
+    getSkillBadgeMapsMock.mockResolvedValueOnce(
+      new Map([
+        ['skills:hl', { highlighted: { byUserId: 'users:mod', at: 1 } }],
+        ['skills:plain', {}],
+      ]),
+    )
+
+    const result = await lexicalFallbackSkillsHandler(
+      makeLexicalCtx({
+        exactSlugSkill: null,
+        recentSkills: [highlighted, plain],
+      }),
+      { query: 'orf', queryTokens: ['orf'], highlightedOnly: true, limit: 10 },
+    )
+
+    expect(result).toHaveLength(1)
+    expect(result[0].skill.slug).toBe('orf-highlighted')
+  })
+
+  it('includes exact slug match from by_slug even when recent scan is empty', async () => {
+    const exactSlugSkill = makeSkillDoc({ id: 'skills:orf', slug: 'orf', displayName: 'ORF' })
+    getSkillBadgeMapsMock.mockResolvedValueOnce(new Map([['skills:orf', {}]]))
+    const ctx = makeLexicalCtx({
+      exactSlugSkill,
+      recentSkills: [],
+    })
+
+    const result = await lexicalFallbackSkillsHandler(ctx, {
+      query: 'orf',
+      queryTokens: ['orf'],
+      limit: 10,
+    })
+
+    expect(result).toHaveLength(1)
+    expect(result[0].skill.slug).toBe('orf')
+    expect(ctx.db.query).toHaveBeenCalledWith('skills')
+  })
+
+  it('dedupes overlap and enforces rank + limit across vector and fallback', async () => {
+    generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2])
+    const vectorEntries = [
+      {
+        embeddingId: 'skillEmbeddings:a',
+        skill: makePublicSkill({
+          id: 'skills:a',
+          slug: 'foo-a',
+          displayName: 'Foo Alpha',
+          downloads: 10,
+        }),
+        version: null,
+        ownerHandle: 'one',
+      },
+      {
+        embeddingId: 'skillEmbeddings:b',
+        skill: makePublicSkill({
+          id: 'skills:b',
+          slug: 'foo-b',
+          displayName: 'Foo Beta',
+          downloads: 2,
+        }),
+        version: null,
+        ownerHandle: 'two',
+      },
+    ]
+    const fallbackEntries = [
+      {
+        skill: makePublicSkill({
+          id: 'skills:a',
+          slug: 'foo-a',
+          displayName: 'Foo Alpha',
+          downloads: 10,
+        }),
+        version: null,
+        ownerHandle: 'one',
+      },
+      {
+        skill: makePublicSkill({
+          id: 'skills:c',
+          slug: 'foo-c',
+          displayName: 'Foo Classic',
+          downloads: 1,
+        }),
+        version: null,
+        ownerHandle: 'three',
+      },
+    ]
+
+    const runQuery = vi
+      .fn()
+      .mockResolvedValueOnce(vectorEntries)
+      .mockResolvedValueOnce([])
+      .mockResolvedValueOnce(fallbackEntries)
+
+    const result = await searchSkillsHandler(
+      {
+        vectorSearch: vi.fn().mockResolvedValue([
+          { _id: 'skillEmbeddings:a', _score: 0.4 },
+          { _id: 'skillEmbeddings:b', _score: 0.9 },
+        ]),
+        runQuery,
+      },
+      { query: 'foo', limit: 2 },
+    )
+
+    expect(result).toHaveLength(2)
+    expect(result[0].skill.slug).toBe('foo-b')
+    expect(new Set(result.map((entry: { skill: { _id: string } }) => entry.skill._id)).size).toBe(2)
+  })
+
+  it('advances candidate limit until max', () => {
+    expect(__test.getNextCandidateLimit(50, 1000)).toBe(100)
+    expect(__test.getNextCandidateLimit(800, 1000)).toBe(1000)
+    expect(__test.getNextCandidateLimit(1000, 1000)).toBeNull()
+  })
+
+  it('boosts exact slug/name matches over loose matches', () => {
+    const queryTokens = tokenize('notion')
+    const exactScore = __test.scoreSkillResult(queryTokens, 0.4, 'Notion Sync', 'notion-sync', 5)
+    const looseScore = __test.scoreSkillResult(queryTokens, 0.6, 'Notes Sync', 'notes-sync', 500)
+    expect(exactScore).toBeGreaterThan(looseScore)
+  })
+
+  it('adds a popularity prior for equally relevant matches', () => {
+    const queryTokens = tokenize('notion')
+    const lowDownloads = __test.scoreSkillResult(
+      queryTokens,
+      0.5,
+      'Notion Helper',
+      'notion-helper',
+      0,
+    )
+    const highDownloads = __test.scoreSkillResult(
+      queryTokens,
+      0.5,
+      'Notion Helper',
+      'notion-helper',
+      1000,
+    )
+    expect(highDownloads).toBeGreaterThan(lowDownloads)
+  })
+
+  it('merges fallback matches without duplicate skill ids', () => {
+    const primary = [
+      {
+        embeddingId: 'skillEmbeddings:1',
+        skill: { _id: 'skills:1' },
+      },
+    ] as unknown as Parameters<typeof __test.mergeUniqueBySkillId>[0]
+    const fallback = [
+      {
+        skill: { _id: 'skills:1' },
+      },
+      {
+        skill: { _id: 'skills:2' },
+      },
+    ] as unknown as Parameters<typeof __test.mergeUniqueBySkillId>[1]
+
+    const merged = __test.mergeUniqueBySkillId(primary, fallback)
+    expect(merged).toHaveLength(2)
+    expect(merged.map((entry) => entry.skill._id)).toEqual(['skills:1', 'skills:2'])
+  })
+})
+
+function makePublicSkill(params: {
+  id: string
+  slug: string
+  displayName: string
+  downloads?: number
+}) {
+  return {
+    _id: params.id,
+    _creationTime: 1,
+    slug: params.slug,
+    displayName: params.displayName,
+    summary: `${params.displayName} summary`,
+    ownerUserId: 'users:owner',
+    canonicalSkillId: undefined,
+    forkOf: undefined,
+    latestVersionId: 'skillVersions:1',
+    tags: {},
+    badges: {},
+    stats: {
+      downloads: params.downloads ?? 0,
+      installsCurrent: 0,
+      installsAllTime: 0,
+      stars: 0,
+      versions: 1,
+      comments: 0,
+    },
+    createdAt: 1,
+    updatedAt: 1,
+  }
+}
+
+function makeSkillDoc(params: { id: string; slug: string; displayName: string }) {
+  return {
+    ...makePublicSkill(params),
+    _creationTime: 1,
+    moderationStatus: 'active',
+    moderationFlags: [],
+    softDeletedAt: undefined,
+  }
+}
+
+function makeLexicalCtx(params: {
+  exactSlugSkill: ReturnType<typeof makeSkillDoc> | null
+  recentSkills: Array<ReturnType<typeof makeSkillDoc>>
+}) {
+  return {
+    db: {
+      query: vi.fn((table: string) => {
+        if (table !== 'skills') throw new Error(`Unexpected table ${table}`)
+        return {
+          withIndex: (index: string) => {
+            if (index === 'by_slug') {
+              return {
+                unique: vi.fn().mockResolvedValue(params.exactSlugSkill),
+              }
+            }
+            if (index === 'by_active_updated') {
+              return {
+                order: () => ({
+                  take: vi.fn().mockResolvedValue(params.recentSkills),
+                }),
+              }
+            }
+            throw new Error(`Unexpected index ${index}`)
+          },
+        }
+      }),
+      get: vi.fn(async (id: string) => {
+        if (id.startsWith('users:')) return { _id: id, handle: 'owner' }
+        if (id.startsWith('skillVersions:')) return { _id: id, version: '1.0.0' }
+        return null
+      }),
+    },
+  }
+}

convex/search.ts

@@ -2,15 +2,90 @@ import { v } from 'convex/values'
 import { internal } from './_generated/api'
 import type { Doc, Id } from './_generated/dataModel'
 import { action, internalQuery } from './_generated/server'
+import { getSkillBadgeMaps, isSkillHighlighted, type SkillBadgeMap } from './lib/badges'
 import { generateEmbedding } from './lib/embeddings'
+import { toPublicSkill, toPublicSoul } from './lib/public'
+import { matchesExactTokens, tokenize } from './lib/searchText'
 
-type HydratedEntry = {
-  embeddingId: Id<'skillEmbeddings'>
-  skill: Doc<'skills'> | null
+type SkillSearchEntry = {
+  embeddingId?: Id<'skillEmbeddings'>
+  skill: NonNullable<ReturnType<typeof toPublicSkill>>
   version: Doc<'skillVersions'> | null
+  ownerHandle: string | null
 }
 
-type SearchResult = HydratedEntry & { score: number }
+type SearchResult = SkillSearchEntry & { score: number }
+
+const SLUG_EXACT_BOOST = 1.4
+const SLUG_PREFIX_BOOST = 0.8
+const NAME_EXACT_BOOST = 1.1
+const NAME_PREFIX_BOOST = 0.6
+const POPULARITY_WEIGHT = 0.08
+const FALLBACK_SCAN_LIMIT = 1200
+
+function getNextCandidateLimit(current: number, max: number) {
+  const next = Math.min(current * 2, max)
+  return next > current ? next : null
+}
+
+function matchesAllTokens(
+  queryTokens: string[],
+  candidateTokens: string[],
+  matcher: (candidate: string, query: string) => boolean,
+) {
+  if (queryTokens.length === 0 || candidateTokens.length === 0) return false
+  return queryTokens.every((queryToken) =>
+    candidateTokens.some((candidateToken) => matcher(candidateToken, queryToken)),
+  )
+}
+
+function getLexicalBoost(queryTokens: string[], displayName: string, slug: string) {
+  const slugTokens = tokenize(slug)
+  const nameTokens = tokenize(displayName)
+
+  let boost = 0
+  if (matchesAllTokens(queryTokens, slugTokens, (candidate, query) => candidate === query)) {
+    boost += SLUG_EXACT_BOOST
+  } else if (
+    matchesAllTokens(queryTokens, slugTokens, (candidate, query) => candidate.startsWith(query))
+  ) {
+    boost += SLUG_PREFIX_BOOST
+  }
+
+  if (matchesAllTokens(queryTokens, nameTokens, (candidate, query) => candidate === query)) {
+    boost += NAME_EXACT_BOOST
+  } else if (
+    matchesAllTokens(queryTokens, nameTokens, (candidate, query) => candidate.startsWith(query))
+  ) {
+    boost += NAME_PREFIX_BOOST
+  }
+
+  return boost
+}
+
+function scoreSkillResult(
+  queryTokens: string[],
+  vectorScore: number,
+  displayName: string,
+  slug: string,
+  downloads: number,
+) {
+  const lexicalBoost = getLexicalBoost(queryTokens, displayName, slug)
+  const popularityBoost = Math.log1p(Math.max(downloads, 0)) * POPULARITY_WEIGHT
+  return vectorScore + lexicalBoost + popularityBoost
+}
+
+function mergeUniqueBySkillId(primary: SkillSearchEntry[], fallback: SkillSearchEntry[]) {
+  if (fallback.length === 0) return primary
+  const out = [...primary]
+  const seen = new Set(primary.map((entry) => entry.skill._id))
+  for (const entry of fallback) {
+    if (seen.has(entry.skill._id)) continue
+    seen.add(entry.skill._id)
+    out.push(entry)
+  }
+  return out
+}
 
 export const searchSkills: ReturnType<typeof action> = action({
   args: {
@@ -21,48 +96,338 @@ export const searchSkills: ReturnType<typeof action> = action({
   handler: async (ctx, args): Promise<SearchResult[]> => {
     const query = args.query.trim()
     if (!query) return []
-    const vector = await generateEmbedding(query)
-    const results = await ctx.vectorSearch('skillEmbeddings', 'by_embedding', {
-      vector,
-      limit: args.limit ?? 10,
-      filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
-    })
+    const queryTokens = tokenize(query)
+    if (queryTokens.length === 0) return []
+    let vector: number[]
+    try {
+      vector = await generateEmbedding(query)
+    } catch (error) {
+      console.warn('Search embedding generation failed', error)
+      return []
+    }
+    const limit = args.limit ?? 10
+    // Convex vectorSearch max limit is 256; clamp candidate sizes accordingly.
+    const maxCandidate = Math.min(Math.max(limit * 10, 200), 256)
+    let candidateLimit = Math.min(Math.max(limit * 3, 50), 256)
+    let hydrated: SkillSearchEntry[] = []
+    let scoreById = new Map<Id<'skillEmbeddings'>, number>()
+    let exactMatches: SkillSearchEntry[] = []
 
-    const hydrated = (await ctx.runQuery(internal.search.hydrateResults, {
-      embeddingIds: results.map((result) => result._id),
-    })) as HydratedEntry[]
+    while (candidateLimit <= maxCandidate) {
+      const results = await ctx.vectorSearch('skillEmbeddings', 'by_embedding', {
+        vector,
+        limit: candidateLimit,
+        filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
+      })
 
-    const scoreById = new Map<Id<'skillEmbeddings'>, number>(
-      results.map((result) => [result._id, result._score]),
-    )
+      hydrated = (await ctx.runQuery(internal.search.hydrateResults, {
+        embeddingIds: results.map((result) => result._id),
+      })) as SkillSearchEntry[]
 
-    const filtered = args.highlightedOnly
-      ? hydrated.filter((entry) => entry.skill?.batch === 'highlighted')
-      : hydrated
+      scoreById = new Map<Id<'skillEmbeddings'>, number>(
+        results.map((result) => [result._id, result._score]),
+      )
 
-    return filtered
-      .map((entry) => ({
+      const badgeMapEntries = (await ctx.runQuery(internal.search.getSkillBadgeMapsInternal, {
+        skillIds: hydrated.map((entry) => entry.skill._id),
+      })) as Array<[Id<'skills'>, SkillBadgeMap]>
+      const badgeMapBySkillId = new Map(badgeMapEntries)
+      const hydratedWithBadges = hydrated.map((entry) => ({
         ...entry,
-        score: scoreById.get(entry.embeddingId) ?? 0,
+        skill: {
+          ...entry.skill,
+          badges: badgeMapBySkillId.get(entry.skill._id) ?? {},
+        },
       }))
+
+      const filtered = args.highlightedOnly
+        ? hydratedWithBadges.filter((entry) => isSkillHighlighted(entry.skill))
+        : hydratedWithBadges
+
+      exactMatches = filtered.filter((entry) =>
+        matchesExactTokens(queryTokens, [
+          entry.skill.displayName,
+          entry.skill.slug,
+          entry.skill.summary,
+        ]),
+      )
+
+      if (exactMatches.length >= limit || results.length < candidateLimit) {
+        break
+      }
+
+      const nextLimit = getNextCandidateLimit(candidateLimit, maxCandidate)
+      if (!nextLimit) break
+      candidateLimit = nextLimit
+    }
+
+    const fallbackMatches =
+      exactMatches.length >= limit
+        ? []
+        : ((await ctx.runQuery(internal.search.lexicalFallbackSkills, {
+            query,
+            queryTokens,
+            limit: Math.min(Math.max(limit * 4, 200), FALLBACK_SCAN_LIMIT),
+            highlightedOnly: args.highlightedOnly,
+          })) as SkillSearchEntry[])
+
+    const mergedMatches = mergeUniqueBySkillId(exactMatches, fallbackMatches)
+
+    return mergedMatches
+      .map((entry) => {
+        const vectorScore = entry.embeddingId ? (scoreById.get(entry.embeddingId) ?? 0) : 0
+        return {
+          ...entry,
+          score: scoreSkillResult(
+            queryTokens,
+            vectorScore,
+            entry.skill.displayName,
+            entry.skill.slug,
+            entry.skill.stats.downloads,
+          ),
+        }
+      })
       .filter((entry) => entry.skill)
+      .sort((a, b) => b.score - a.score || b.skill.stats.downloads - a.skill.stats.downloads)
+      .slice(0, limit)
+  },
+})
+
+export const getBadgeMapsForSkills = internalQuery({
+  args: { skillIds: v.array(v.id('skills')) },
+  handler: async (ctx, args): Promise<Array<[Id<'skills'>, SkillBadgeMap]>> => {
+    const badgeMap = await getSkillBadgeMaps(ctx, args.skillIds)
+    return Array.from(badgeMap.entries())
   },
 })
 
 export const hydrateResults = internalQuery({
   args: { embeddingIds: v.array(v.id('skillEmbeddings')) },
-  handler: async (ctx, args): Promise<HydratedEntry[]> => {
-    const entries: HydratedEntry[] = []
+  handler: async (ctx, args): Promise<SkillSearchEntry[]> => {
+    const ownerHandleCache = new Map<Id<'users'>, Promise<string | null>>()
+
+    const getOwnerHandle = (ownerUserId: Id<'users'>) => {
+      const cached = ownerHandleCache.get(ownerUserId)
+      if (cached) return cached
+      const handlePromise = ctx.db
+        .get(ownerUserId)
+        .then((owner) => owner?.handle ?? owner?._id ?? null)
+      ownerHandleCache.set(ownerUserId, handlePromise)
+      return handlePromise
+    }
+
+    const entries = await Promise.all(
+      args.embeddingIds.map(async (embeddingId) => {
+        const embedding = await ctx.db.get(embeddingId)
+        if (!embedding) return null
+        const skill = await ctx.db.get(embedding.skillId)
+        if (!skill || skill.softDeletedAt) return null
+        const [version, ownerHandle] = await Promise.all([
+          ctx.db.get(embedding.versionId),
+          getOwnerHandle(skill.ownerUserId),
+        ])
+        const publicSkill = toPublicSkill(skill)
+        if (!publicSkill) return null
+        return { embeddingId, skill: publicSkill, version, ownerHandle }
+      }),
+    )
+
+    return entries.filter((entry): entry is SkillSearchEntry => entry !== null)
+  },
+})
+
+export const lexicalFallbackSkills = internalQuery({
+  args: {
+    query: v.string(),
+    queryTokens: v.array(v.string()),
+    limit: v.optional(v.number()),
+    highlightedOnly: v.optional(v.boolean()),
+  },
+  handler: async (ctx, args): Promise<SkillSearchEntry[]> => {
+    const limit = Math.min(Math.max(args.limit ?? 200, 10), FALLBACK_SCAN_LIMIT)
+    const seenSkillIds = new Set<Id<'skills'>>()
+    const candidateSkills: Doc<'skills'>[] = []
+
+    const slugQuery = args.query.trim().toLowerCase()
+    if (/^[a-z0-9][a-z0-9-]*$/.test(slugQuery)) {
+      const exactSlugSkill = await ctx.db
+        .query('skills')
+        .withIndex('by_slug', (q) => q.eq('slug', slugQuery))
+        .unique()
+      if (exactSlugSkill && !exactSlugSkill.softDeletedAt) {
+        seenSkillIds.add(exactSlugSkill._id)
+        candidateSkills.push(exactSlugSkill)
+      }
+    }
+
+    const recentSkills = await ctx.db
+      .query('skills')
+      .withIndex('by_active_updated', (q) => q.eq('softDeletedAt', undefined))
+      .order('desc')
+      .take(FALLBACK_SCAN_LIMIT)
+
+    for (const skill of recentSkills) {
+      if (seenSkillIds.has(skill._id)) continue
+      seenSkillIds.add(skill._id)
+      candidateSkills.push(skill)
+    }
+
+    const matched = candidateSkills.filter((skill) =>
+      matchesExactTokens(args.queryTokens, [skill.displayName, skill.slug, skill.summary]),
+    )
+    if (matched.length === 0) return []
+
+    const ownerHandleCache = new Map<Id<'users'>, Promise<string | null>>()
+    const getOwnerHandle = (ownerUserId: Id<'users'>) => {
+      const cached = ownerHandleCache.get(ownerUserId)
+      if (cached) return cached
+      const handlePromise = ctx.db
+        .get(ownerUserId)
+        .then((owner) => owner?.handle ?? owner?._id ?? null)
+      ownerHandleCache.set(ownerUserId, handlePromise)
+      return handlePromise
+    }
+
+    const entries = await Promise.all(
+      matched.map(async (skill) => {
+        const [version, ownerHandle] = await Promise.all([
+          skill.latestVersionId ? ctx.db.get(skill.latestVersionId) : Promise.resolve(null),
+          getOwnerHandle(skill.ownerUserId),
+        ])
+        const publicSkill = toPublicSkill(skill)
+        if (!publicSkill) return null
+        return { skill: publicSkill, version, ownerHandle }
+      }),
+    )
+    const validEntries = entries.filter((entry): entry is SkillSearchEntry => entry !== null)
+    if (validEntries.length === 0) return []
+
+    const badgeMap = await getSkillBadgeMaps(
+      ctx,
+      validEntries.map((entry) => entry.skill._id),
+    )
+    const withBadges = validEntries.map((entry) => ({
+      ...entry,
+      skill: {
+        ...entry.skill,
+        badges: badgeMap.get(entry.skill._id) ?? {},
+      },
+    }))
+
+    const filtered = args.highlightedOnly
+      ? withBadges.filter((entry) => isSkillHighlighted(entry.skill))
+      : withBadges
+    return filtered.slice(0, limit)
+  },
+})
+
+type HydratedSoulEntry = {
+  embeddingId: Id<'soulEmbeddings'>
+  soul: NonNullable<ReturnType<typeof toPublicSoul>>
+  version: Doc<'soulVersions'> | null
+}
+
+type SoulSearchResult = HydratedSoulEntry & { score: number }
+
+export const searchSouls: ReturnType<typeof action> = action({
+  args: {
+    query: v.string(),
+    limit: v.optional(v.number()),
+  },
+  handler: async (ctx, args): Promise<SoulSearchResult[]> => {
+    const query = args.query.trim()
+    if (!query) return []
+    const queryTokens = tokenize(query)
+    if (queryTokens.length === 0) return []
+    let vector: number[]
+    try {
+      vector = await generateEmbedding(query)
+    } catch (error) {
+      console.warn('Search embedding generation failed', error)
+      return []
+    }
+    const limit = args.limit ?? 10
+    // Convex vectorSearch max limit is 256; clamp candidate sizes accordingly.
+    const maxCandidate = Math.min(Math.max(limit * 10, 200), 256)
+    let candidateLimit = Math.min(Math.max(limit * 3, 50), 256)
+    let hydrated: HydratedSoulEntry[] = []
+    let scoreById = new Map<Id<'soulEmbeddings'>, number>()
+    let exactMatches: HydratedSoulEntry[] = []
+
+    while (candidateLimit <= maxCandidate) {
+      const results = await ctx.vectorSearch('soulEmbeddings', 'by_embedding', {
+        vector,
+        limit: candidateLimit,
+        filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
+      })
+
+      hydrated = (await ctx.runQuery(internal.search.hydrateSoulResults, {
+        embeddingIds: results.map((result) => result._id),
+      })) as HydratedSoulEntry[]
+
+      scoreById = new Map<Id<'soulEmbeddings'>, number>(
+        results.map((result) => [result._id, result._score]),
+      )
+
+      exactMatches = hydrated.filter((entry) =>
+        matchesExactTokens(queryTokens, [
+          entry.soul.displayName,
+          entry.soul.slug,
+          entry.soul.summary,
+        ]),
+      )
+
+      if (exactMatches.length >= limit || results.length < candidateLimit) {
+        break
+      }
+
+      const nextLimit = getNextCandidateLimit(candidateLimit, maxCandidate)
+      if (!nextLimit) break
+      candidateLimit = nextLimit
+    }
+
+    return exactMatches
+      .map((entry) => ({
+        ...entry,
+        score: scoreById.get(entry.embeddingId) ?? 0,
+      }))
+      .filter((entry) => entry.soul)
+      .slice(0, limit)
+  },
+})
+
+export const hydrateSoulResults = internalQuery({
+  args: { embeddingIds: v.array(v.id('soulEmbeddings')) },
+  handler: async (ctx, args): Promise<HydratedSoulEntry[]> => {
+    const entries: HydratedSoulEntry[] = []
 
     for (const embeddingId of args.embeddingIds) {
       const embedding = await ctx.db.get(embeddingId)
       if (!embedding) continue
-      const skill = await ctx.db.get(embedding.skillId)
-      if (skill?.softDeletedAt) continue
+      const soul = await ctx.db.get(embedding.soulId)
+      if (soul?.softDeletedAt) continue
       const version = await ctx.db.get(embedding.versionId)
-      entries.push({ embeddingId, skill, version })
+      const publicSoul = toPublicSoul(soul)
+      if (!publicSoul) continue
+      entries.push({ embeddingId, soul: publicSoul, version })
     }
 
     return entries
   },
 })
+
+export const getSkillBadgeMapsInternal = internalQuery({
+  args: { skillIds: v.array(v.id('skills')) },
+  handler: async (ctx, args) => {
+    const badgeMap = await getSkillBadgeMaps(ctx, args.skillIds)
+    return Array.from(badgeMap.entries())
+  },
+})
+
+export const __test = {
+  getNextCandidateLimit,
+  matchesAllTokens,
+  getLexicalBoost,
+  scoreSkillResult,
+  mergeUniqueBySkillId,
+}

convex/seed.test.ts

@@ -0,0 +1,37 @@
+import { describe, expect, it } from 'vitest'
+import type { Doc } from './_generated/dataModel'
+import { decideSeedStart } from './seed'
+
+function seedState(cursor: string, updatedAt: number) {
+  return { cursor, updatedAt } as unknown as Doc<'githubBackupSyncState'>
+}
+
+describe('decideSeedStart', () => {
+  it('returns done when done', () => {
+    expect(decideSeedStart(seedState('done', Date.now()), Date.now())).toEqual({
+      started: false,
+      reason: 'done',
+    })
+  })
+
+  it('returns running when lock fresh', () => {
+    const now = Date.now()
+    expect(decideSeedStart(seedState('running', now), now + 1000)).toEqual({
+      started: false,
+      reason: 'running',
+    })
+  })
+
+  it('starts when lock stale', () => {
+    const now = Date.now()
+    const stale = now - 10 * 60 * 1000 - 1
+    expect(decideSeedStart(seedState('running', stale), now)).toEqual({
+      started: true,
+      reason: 'patched',
+    })
+  })
+
+  it('starts when missing', () => {
+    expect(decideSeedStart(null, Date.now())).toEqual({ started: true, reason: 'inserted' })
+  })
+})

convex/seed.ts

@@ -0,0 +1,254 @@
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import type { ActionCtx, DatabaseReader, DatabaseWriter } from './_generated/server'
+import { action, internalMutation, internalQuery } from './_generated/server'
+import { publishSoulVersionForUser } from './lib/soulPublish'
+import { SOUL_SEED_DISPLAY_NAME, SOUL_SEED_HANDLE, SOUL_SEED_KEY, SOUL_SEEDS } from './seedSouls'
+
+const SEED_LOCK_STALE_MS = 10 * 60 * 1000
+
+type SeedStateDoc = Doc<'githubBackupSyncState'>
+
+type SeedStartDecision = {
+  started: boolean
+  reason: 'done' | 'running' | 'patched' | 'inserted'
+}
+
+async function getSeedState(ctx: { db: DatabaseReader }): Promise<SeedStateDoc | null> {
+  const entries = (await ctx.db
+    .query('githubBackupSyncState')
+    .withIndex('by_key', (q) => q.eq('key', SOUL_SEED_KEY))
+    .order('desc')
+    .take(2)) as SeedStateDoc[]
+  return entries[0] ?? null
+}
+
+async function cleanupSeedState(ctx: { db: DatabaseWriter }, keepId: Id<'githubBackupSyncState'>) {
+  const entries = (await ctx.db
+    .query('githubBackupSyncState')
+    .withIndex('by_key', (q) => q.eq('key', SOUL_SEED_KEY))
+    .order('desc')
+    .take(50)) as SeedStateDoc[]
+
+  for (const entry of entries) {
+    if (entry._id === keepId) continue
+    await ctx.db.delete(entry._id)
+  }
+}
+
+export function decideSeedStart(existing: SeedStateDoc | null, now: number): SeedStartDecision {
+  const cursor = existing?.cursor ?? null
+  if (cursor === 'done') return { started: false, reason: 'done' }
+  if (cursor === 'running' && existing && now - existing.updatedAt < SEED_LOCK_STALE_MS) {
+    return { started: false, reason: 'running' }
+  }
+  return existing ? { started: true, reason: 'patched' } : { started: true, reason: 'inserted' }
+}
+
+export const getSoulSeedStateInternal = internalQuery({
+  args: {},
+  handler: async (ctx) => getSeedState(ctx),
+})
+
+export const setSoulSeedStateInternal = internalMutation({
+  args: { status: v.string() },
+  handler: async (ctx, args) => {
+    const existing = await getSeedState(ctx)
+    const now = Date.now()
+    if (existing) {
+      await ctx.db.patch(existing._id, { cursor: args.status, updatedAt: now })
+      await cleanupSeedState(ctx, existing._id)
+      return existing._id
+    }
+    const id = await ctx.db.insert('githubBackupSyncState', {
+      key: SOUL_SEED_KEY,
+      cursor: args.status,
+      updatedAt: now,
+    })
+    await cleanupSeedState(ctx, id)
+    return id
+  },
+})
+
+export const tryStartSoulSeedInternal = internalMutation({
+  args: {},
+  handler: async (ctx) => {
+    const now = Date.now()
+    const existing = await getSeedState(ctx)
+    const decision = decideSeedStart(existing, now)
+
+    if (!decision.started) return decision
+
+    if (existing) {
+      await ctx.db.patch(existing._id, { cursor: 'running', updatedAt: now })
+      await cleanupSeedState(ctx, existing._id)
+      return { started: true, reason: 'patched' as const }
+    }
+
+    const id = await ctx.db.insert('githubBackupSyncState', {
+      key: SOUL_SEED_KEY,
+      cursor: 'running',
+      updatedAt: now,
+    })
+    await cleanupSeedState(ctx, id)
+    return { started: true, reason: 'inserted' as const }
+  },
+})
+
+export const hasAnySoulsInternal = internalQuery({
+  args: {},
+  handler: async (ctx) => {
+    const entry = await ctx.db.query('souls').take(1)
+    return entry.length > 0
+  },
+})
+
+export const ensureSoulSeeds = action({
+  args: {},
+  handler: async (ctx) => {
+    const started = (await ctx.runMutation(internal.seed.tryStartSoulSeedInternal, {})) as {
+      started: boolean
+      reason: 'done' | 'running' | 'patched' | 'inserted'
+    }
+    if (!started.started) {
+      if (started.reason === 'done') return { seeded: false, reason: 'already-seeded' as const }
+      return { seeded: false, reason: 'in-progress' as const }
+    }
+
+    const hasSouls = (await ctx.runQuery(internal.seed.hasAnySoulsInternal, {})) as boolean
+    if (hasSouls) {
+      await ctx.runMutation(internal.seed.setSoulSeedStateInternal, { status: 'done' })
+      return { seeded: false, reason: 'souls-exist' as const }
+    }
+
+    try {
+      const result = await runSeed(ctx)
+      await ctx.runMutation(internal.seed.setSoulSeedStateInternal, { status: 'done' })
+      return { seeded: true, reason: 'seeded' as const, ...result }
+    } catch (error) {
+      await ctx.runMutation(internal.seed.setSoulSeedStateInternal, { status: 'error' })
+      throw error
+    }
+  },
+})
+
+export const seed = action({
+  args: {},
+  handler: async (ctx) => runSeed(ctx),
+})
+
+async function runSeed(ctx: ActionCtx) {
+  const userId = (await ctx.runMutation(internal.seed.ensureSeedUserInternal, {
+    handle: SOUL_SEED_HANDLE,
+    displayName: SOUL_SEED_DISPLAY_NAME,
+  })) as Id<'users'>
+
+  const created: string[] = []
+  const skipped: string[] = []
+
+  for (const seedEntry of SOUL_SEEDS) {
+    const existing = (await ctx.runQuery(internal.souls.getSoulBySlugInternal, {
+      slug: seedEntry.slug,
+    })) as Doc<'souls'> | null
+    if (existing) {
+      if (existing.softDeletedAt && existing.ownerUserId === userId) {
+        await ctx.runMutation(internal.souls.setSoulSoftDeletedInternal, {
+          userId,
+          slug: seedEntry.slug,
+          deleted: false,
+        })
+      }
+      skipped.push(seedEntry.slug)
+      continue
+    }
+
+    const body = seedEntry.readme
+    if (!body) {
+      skipped.push(seedEntry.slug)
+      continue
+    }
+
+    const bytes = new TextEncoder().encode(body)
+    const sha256 = await sha256Hex(bytes)
+    const storageId = await ctx.storage.store(new Blob([bytes], { type: 'text/markdown' }))
+
+    try {
+      await publishSoulVersionForUser(ctx, userId, {
+        slug: seedEntry.slug,
+        displayName: seedEntry.displayName,
+        version: seedEntry.version,
+        changelog: '',
+        tags: seedEntry.tags,
+        files: [
+          {
+            path: 'SOUL.md',
+            size: bytes.byteLength,
+            storageId,
+            sha256,
+            contentType: 'text/markdown',
+          },
+        ],
+      })
+      created.push(seedEntry.slug)
+    } catch (error) {
+      if (!isExpectedSeedSkipError(error)) throw error
+      skipped.push(seedEntry.slug)
+    }
+  }
+
+  return { created, skipped }
+}
+
+function isExpectedSeedSkipError(error: unknown) {
+  const message = error instanceof Error ? error.message : String(error)
+  return (
+    message.includes('Version already exists') || message.includes('Only the owner can publish')
+  )
+}
+
+export const ensureSeedUserInternal = internalMutation({
+  args: {
+    handle: v.string(),
+    displayName: v.string(),
+  },
+  handler: async (ctx, args) => {
+    const baseHandle = args.handle.trim()
+    const displayName = args.displayName.trim()
+    const candidates = [baseHandle, `${baseHandle}-bot`]
+    for (let i = 2; i <= 6; i += 1) candidates.push(`${baseHandle}-bot-${i}`)
+
+    for (const candidate of candidates) {
+      const existing = await ctx.db
+        .query('users')
+        .withIndex('handle', (q) => q.eq('handle', candidate))
+        .take(2)
+      const user = (existing[0] ?? null) as Doc<'users'> | null
+      if (user) {
+        if ((user.displayName ?? user.name) === displayName) return user._id
+        continue
+      }
+
+      return ctx.db.insert('users', {
+        handle: candidate,
+        displayName,
+        createdAt: Date.now(),
+        updatedAt: Date.now(),
+      })
+    }
+
+    throw new Error('Unable to allocate seed user handle')
+  },
+})
+
+async function sha256Hex(bytes: Uint8Array) {
+  const data = new Uint8Array(bytes)
+  const digest = await crypto.subtle.digest('SHA-256', data)
+  return toHex(new Uint8Array(digest))
+}
+
+function toHex(bytes: Uint8Array) {
+  let out = ''
+  for (const byte of bytes) out += byte.toString(16).padStart(2, '0')
+  return out
+}

convex/skillStatEvents.test.ts

@@ -0,0 +1,110 @@
+/* @vitest-environment node */
+import { describe, expect, it } from 'vitest'
+
+// Test the aggregateEvents function by importing and testing the module logic
+// Since aggregateEvents is not exported, we test the behavior indirectly through
+// the event processing contract
+
+describe('skill stat events - comment delta handling', () => {
+  it('aggregates comment and uncomment events into net deltas', () => {
+    // Simulate the aggregation logic from processSkillStatEventsAction
+    type EventKind =
+      | 'download'
+      | 'star'
+      | 'unstar'
+      | 'comment'
+      | 'uncomment'
+      | 'install_new'
+      | 'install_reactivate'
+      | 'install_deactivate'
+      | 'install_clear'
+
+    const events: { kind: EventKind; occurredAt: number }[] = [
+      { kind: 'star', occurredAt: 1000 },
+      { kind: 'comment', occurredAt: 2000 },
+      { kind: 'comment', occurredAt: 3000 },
+      { kind: 'uncomment', occurredAt: 4000 },
+      { kind: 'download', occurredAt: 5000 },
+    ]
+
+    // Replicate the aggregation logic
+    const result = {
+      downloads: 0,
+      stars: 0,
+      comments: 0,
+      installsAllTime: 0,
+      installsCurrent: 0,
+      downloadEvents: [] as number[],
+      installNewEvents: [] as number[],
+    }
+
+    for (const event of events) {
+      switch (event.kind) {
+        case 'download':
+          result.downloads += 1
+          result.downloadEvents.push(event.occurredAt)
+          break
+        case 'star':
+          result.stars += 1
+          break
+        case 'unstar':
+          result.stars -= 1
+          break
+        case 'comment':
+          result.comments += 1
+          break
+        case 'uncomment':
+          result.comments -= 1
+          break
+        case 'install_new':
+          result.installsAllTime += 1
+          result.installsCurrent += 1
+          result.installNewEvents.push(event.occurredAt)
+          break
+        case 'install_reactivate':
+          result.installsCurrent += 1
+          break
+        case 'install_deactivate':
+          result.installsCurrent -= 1
+          break
+      }
+    }
+
+    expect(result.stars).toBe(1)
+    expect(result.comments).toBe(1) // 2 comments - 1 uncomment
+    expect(result.downloads).toBe(1)
+    expect(result.downloadEvents).toEqual([5000])
+  })
+
+  it('should include comments in delta check (regression test for dropped comments)', () => {
+    // This test verifies the fix: the condition guard in applyAggregatedStatsAndUpdateCursor
+    // must include comments !== 0 so comment-only batches are not skipped
+    const delta = {
+      downloads: 0,
+      stars: 0,
+      comments: 3,
+      installsAllTime: 0,
+      installsCurrent: 0,
+    }
+
+    // The OLD buggy condition (missing comments):
+    const oldCondition =
+      delta.downloads !== 0 ||
+      delta.stars !== 0 ||
+      delta.installsAllTime !== 0 ||
+      delta.installsCurrent !== 0
+
+    // The FIXED condition (includes comments):
+    const fixedCondition =
+      delta.downloads !== 0 ||
+      delta.stars !== 0 ||
+      delta.comments !== 0 ||
+      delta.installsAllTime !== 0 ||
+      delta.installsCurrent !== 0
+
+    // With only comment deltas, the old condition would skip the patch
+    expect(oldCondition).toBe(false)
+    // The fixed condition correctly triggers the patch
+    expect(fixedCondition).toBe(true)
+  })
+})

convex/skillStatEvents.ts

@@ -0,0 +1,591 @@
+/**
+ * Skill Stat Events - Event-sourced stats processing for skills
+ *
+ * Instead of updating skill stats synchronously in the hot path (which can cause
+ * contention when multiple users download/star/install the same skill), we insert
+ * lightweight event records and process them in batches via a cron job.
+ *
+ * Flow:
+ * 1. User action (download, star, install) → insertStatEvent() writes to skillStatEvents table
+ * 2. Cron job runs every 5 minutes → processSkillStatEventsInternal() processes batches
+ * 3. Events are aggregated per-skill to minimize database operations
+ * 4. Stats are applied to skill documents and daily stats tables
+ * 5. Events are marked as processed (kept forever for auditing)
+ */
+
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import type { MutationCtx } from './_generated/server'
+import { internalAction, internalMutation, internalQuery } from './_generated/server'
+import { applySkillStatDeltas, bumpDailySkillStats } from './lib/skillStats'
+
+/**
+ * Event types that affect skill stats:
+ *
+ * - download: User downloaded skill as zip (+1 downloads)
+ * - star: User starred the skill (+1 stars)
+ * - unstar: User removed their star (-1 stars)
+ * - install_new: First time this user installed this skill (+1 installsAllTime, +1 installsCurrent)
+ * - install_reactivate: User re-added skill after removing it (+1 installsCurrent only)
+ * - install_deactivate: User removed skill from all projects (-1 installsCurrent)
+ * - install_clear: User cleared all telemetry data (custom delta for both allTime and current)
+ */
+export type StatEventKind =
+  | 'download'
+  | 'star'
+  | 'unstar'
+  | 'comment'
+  | 'uncomment'
+  | 'install_new'
+  | 'install_reactivate'
+  | 'install_deactivate'
+  | 'install_clear'
+
+/**
+ * Insert a stat event to be processed later by the cron job.
+ *
+ * This is called from the hot path (downloads, stars, telemetry) instead of
+ * directly updating skill stats. It's a single insert with no read-modify-write
+ * cycle, so it's fast and doesn't contend with other operations on the same skill.
+ *
+ * @param ctx - Mutation context
+ * @param params.skillId - The skill being affected
+ * @param params.kind - Type of event (download, star, install_new, etc.)
+ * @param params.occurredAt - When the event happened (defaults to now). Important for
+ *                            daily stats bucketing - we want downloads at 11:55 PM Monday
+ *                            to count toward Monday's stats even if processed on Tuesday.
+ * @param params.delta - Only used for install_clear events, specifies exact delta amounts
+ */
+export async function insertStatEvent(
+  ctx: MutationCtx,
+  params: {
+    skillId: Id<'skills'>
+    kind: StatEventKind
+    occurredAt?: number
+    delta?: { allTime: number; current: number }
+  },
+) {
+  await ctx.db.insert('skillStatEvents', {
+    skillId: params.skillId,
+    kind: params.kind,
+    delta: params.delta,
+    occurredAt: params.occurredAt ?? Date.now(),
+    processedAt: undefined,
+  })
+}
+
+/**
+ * Aggregated deltas for a single skill after processing multiple events.
+ *
+ * When we process a batch of 100 events, many might be for the same skill.
+ * Instead of updating the skill document once per event, we aggregate all
+ * events for each skill and apply a single update.
+ *
+ * The downloadEvents and installNewEvents arrays store the original timestamps
+ * so we can update daily stats with the correct day bucket for each event.
+ */
+type AggregatedDeltas = {
+  downloads: number
+  stars: number
+  comments: number
+  installsAllTime: number
+  installsCurrent: number
+  /** Original timestamps for each download event (for daily stats bucketing) */
+  downloadEvents: number[]
+  /** Original timestamps for each new install event (for daily stats bucketing) */
+  installNewEvents: number[]
+}
+
+/**
+ * Aggregate multiple events for a single skill into net deltas.
+ *
+ * Example: If a skill has these events in the batch:
+ *   - download (Mon 11pm)
+ *   - download (Tue 1am)
+ *   - star
+ *   - unstar
+ *   - star
+ *
+ * The result would be:
+ *   - downloads: 2
+ *   - stars: 1 (net: +1 -1 +1 = +1)
+ *   - downloadEvents: [<Mon 11pm timestamp>, <Tue 1am timestamp>]
+ *
+ * This aggregation reduces the number of database operations from N events
+ * to 1 skill update + N daily stat updates (which themselves may coalesce
+ * if multiple events fall on the same day).
+ */
+function aggregateEvents(events: Doc<'skillStatEvents'>[]): AggregatedDeltas {
+  const result: AggregatedDeltas = {
+    downloads: 0,
+    stars: 0,
+    comments: 0,
+    installsAllTime: 0,
+    installsCurrent: 0,
+    downloadEvents: [],
+    installNewEvents: [],
+  }
+
+  for (const event of events) {
+    switch (event.kind) {
+      case 'download':
+        result.downloads += 1
+        result.downloadEvents.push(event.occurredAt)
+        break
+      case 'star':
+        result.stars += 1
+        break
+      case 'unstar':
+        result.stars -= 1
+        break
+      case 'comment':
+        result.comments += 1
+        break
+      case 'uncomment':
+        result.comments -= 1
+        break
+      case 'install_new':
+        // New user installing for the first time: count toward both lifetime and current
+        result.installsAllTime += 1
+        result.installsCurrent += 1
+        result.installNewEvents.push(event.occurredAt)
+        break
+      case 'install_reactivate':
+        // User re-added skill after removing: only affects current count
+        result.installsCurrent += 1
+        break
+      case 'install_deactivate':
+        // User removed skill from all projects: only affects current count
+        result.installsCurrent -= 1
+        break
+      case 'install_clear':
+        // User cleared telemetry: uses custom delta values (typically negative)
+        if (event.delta) {
+          result.installsAllTime += event.delta.allTime
+          result.installsCurrent += event.delta.current
+        }
+        break
+    }
+  }
+
+  return result
+}
+
+/**
+ * Process a batch of unprocessed stat events.
+ *
+ * Called by cron every 5 minutes. Processes up to batchSize events (default 100).
+ * If the batch is full, schedules an immediate follow-up run to drain the queue.
+ *
+ * Processing steps:
+ * 1. Query unprocessed events (processedAt is undefined)
+ * 2. Group events by skillId to minimize skill document fetches
+ * 3. For each skill:
+ *    a. Fetch the skill document once
+ *    b. Aggregate all events for this skill into net deltas
+ *    c. Apply deltas to skill stats (downloads, stars, installs)
+ *    d. Update daily stats for trending (using original event timestamps)
+ *    e. Mark all events as processed
+ * 4. If batch was full, schedule another run immediately
+ *
+ * Aggregation levels:
+ * - Level 1: Batch of 100 events from the queue
+ * - Level 2: Group by skillId (e.g., 100 events → 30 unique skills)
+ * - Level 3: Aggregate events per skill (e.g., 5 events → 1 skill update)
+ * - Level 4: Daily stats may coalesce (e.g., 3 downloads same day → 1 upsert)
+ */
+export const processSkillStatEventsInternal = internalMutation({
+  args: { batchSize: v.optional(v.number()) },
+  handler: async (ctx, args) => {
+    const batchSize = args.batchSize ?? 100
+    const now = Date.now()
+
+    // Level 1: Fetch a batch of unprocessed events
+    const events = await ctx.db
+      .query('skillStatEvents')
+      .withIndex('by_unprocessed', (q) => q.eq('processedAt', undefined))
+      .take(batchSize)
+
+    if (events.length === 0) {
+      return { processed: 0 }
+    }
+
+    // Level 2: Group events by skillId to minimize database reads
+    // Instead of fetching the same skill document multiple times,
+    // we fetch it once and process all its events together
+    const eventsBySkill = new Map<Id<'skills'>, Doc<'skillStatEvents'>[]>()
+    for (const event of events) {
+      const existing = eventsBySkill.get(event.skillId) ?? []
+      existing.push(event)
+      eventsBySkill.set(event.skillId, existing)
+    }
+
+    // Process each skill's events
+    for (const [skillId, skillEvents] of eventsBySkill) {
+      const skill = await ctx.db.get(skillId)
+
+      // Skill was deleted - just mark events as processed
+      if (!skill) {
+        for (const event of skillEvents) {
+          await ctx.db.patch(event._id, { processedAt: now })
+        }
+        continue
+      }
+
+      // Level 3: Aggregate all events for this skill into net deltas
+      // e.g., 3 downloads + 2 stars - 1 unstar → { downloads: 3, stars: 1 }
+      const deltas = aggregateEvents(skillEvents)
+
+      // Apply aggregated deltas to skill stats (single update per skill)
+      if (
+        deltas.downloads !== 0 ||
+        deltas.stars !== 0 ||
+        deltas.comments !== 0 ||
+        deltas.installsAllTime !== 0 ||
+        deltas.installsCurrent !== 0
+      ) {
+        const patch = applySkillStatDeltas(skill, {
+          downloads: deltas.downloads,
+          stars: deltas.stars,
+          comments: deltas.comments,
+          installsAllTime: deltas.installsAllTime,
+          installsCurrent: deltas.installsCurrent,
+        })
+        await ctx.db.patch(skill._id, {
+          ...patch,
+          updatedAt: now,
+        })
+      }
+
+      // Update daily stats for trending/leaderboards
+      // We use the ORIGINAL event timestamp (occurredAt) so that:
+      // - A download at Mon 11:55 PM counts toward Monday's stats
+      // - Even if the cron processes it on Tuesday
+      //
+      // Level 4: bumpDailySkillStats does its own coalescing - multiple
+      // events on the same day will update the same daily record
+      for (const occurredAt of deltas.downloadEvents) {
+        await bumpDailySkillStats(ctx, { skillId, now: occurredAt, downloads: 1 })
+      }
+      for (const occurredAt of deltas.installNewEvents) {
+        await bumpDailySkillStats(ctx, { skillId, now: occurredAt, installs: 1 })
+      }
+
+      // Mark all events for this skill as processed
+      for (const event of skillEvents) {
+        await ctx.db.patch(event._id, { processedAt: now })
+      }
+    }
+
+    // If we hit the batch limit, there may be more events waiting.
+    // Schedule an immediate follow-up run to drain the queue.
+    // This ensures high-volume periods don't create a backlog.
+    if (events.length === batchSize) {
+      await ctx.scheduler.runAfter(0, internal.skillStatEvents.processSkillStatEventsInternal, {
+        batchSize,
+      })
+    }
+
+    return { processed: events.length }
+  },
+})
+
+// ============================================================================
+// Action-based processing (cursor-based, runs outside transaction window)
+// ============================================================================
+
+const CURSOR_KEY = 'skill_stat_events'
+const EVENT_BATCH_SIZE = 500
+const MAX_SKILLS_PER_RUN = 50
+
+/**
+ * Fetch a batch of events after the given cursor (by _creationTime).
+ * Returns events sorted by _creationTime ascending.
+ */
+export const getUnprocessedEventBatch = internalQuery({
+  args: {
+    cursorCreationTime: v.optional(v.number()),
+    limit: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const limit = args.limit ?? EVENT_BATCH_SIZE
+    const cursor = args.cursorCreationTime
+
+    // Query events after the cursor using the built-in creation time index
+    const events = await ctx.db
+      .query('skillStatEvents')
+      .withIndex('by_creation_time', (q) =>
+        cursor !== undefined ? q.gt('_creationTime', cursor) : q,
+      )
+      .take(limit)
+    return events
+  },
+})
+
+/**
+ * Get the current cursor position from the cursors table.
+ */
+export const getStatEventCursor = internalQuery({
+  args: {},
+  handler: async (ctx) => {
+    const cursor = await ctx.db
+      .query('skillStatUpdateCursors')
+      .withIndex('by_key', (q) => q.eq('key', CURSOR_KEY))
+      .unique()
+    return cursor?.cursorCreationTime
+  },
+})
+
+/**
+ * Validator for skill deltas passed to the mutation.
+ */
+const skillDeltaValidator = v.object({
+  skillId: v.id('skills'),
+  downloads: v.number(),
+  stars: v.number(),
+  comments: v.number(),
+  installsAllTime: v.number(),
+  installsCurrent: v.number(),
+  downloadEvents: v.array(v.number()),
+  installNewEvents: v.array(v.number()),
+})
+
+/**
+ * Apply aggregated stats to skills and update the cursor.
+ * This is a single atomic mutation that:
+ * 1. Updates all affected skills with their aggregated deltas
+ * 2. Updates daily stats for trending
+ * 3. Advances the cursor to the new position
+ */
+export const applyAggregatedStatsAndUpdateCursor = internalMutation({
+  args: {
+    skillDeltas: v.array(skillDeltaValidator),
+    newCursor: v.number(),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+
+    // Process each skill's aggregated deltas
+    for (const delta of args.skillDeltas) {
+      const skill = await ctx.db.get(delta.skillId)
+
+      // Skill was deleted - skip
+      if (!skill) {
+        continue
+      }
+
+      // Apply aggregated deltas to skill stats
+      if (
+        delta.downloads !== 0 ||
+        delta.stars !== 0 ||
+        delta.comments !== 0 ||
+        delta.installsAllTime !== 0 ||
+        delta.installsCurrent !== 0
+      ) {
+        const patch = applySkillStatDeltas(skill, {
+          downloads: delta.downloads,
+          stars: delta.stars,
+          comments: delta.comments,
+          installsAllTime: delta.installsAllTime,
+          installsCurrent: delta.installsCurrent,
+        })
+        await ctx.db.patch(skill._id, {
+          ...patch,
+          updatedAt: now,
+        })
+      }
+
+      // Update daily stats for trending/leaderboards
+      for (const occurredAt of delta.downloadEvents) {
+        await bumpDailySkillStats(ctx, { skillId: delta.skillId, now: occurredAt, downloads: 1 })
+      }
+      for (const occurredAt of delta.installNewEvents) {
+        await bumpDailySkillStats(ctx, { skillId: delta.skillId, now: occurredAt, installs: 1 })
+      }
+    }
+
+    // Update cursor position (upsert)
+    const existingCursor = await ctx.db
+      .query('skillStatUpdateCursors')
+      .withIndex('by_key', (q) => q.eq('key', CURSOR_KEY))
+      .unique()
+
+    if (existingCursor) {
+      await ctx.db.patch(existingCursor._id, {
+        cursorCreationTime: args.newCursor,
+        updatedAt: now,
+      })
+    } else {
+      await ctx.db.insert('skillStatUpdateCursors', {
+        key: CURSOR_KEY,
+        cursorCreationTime: args.newCursor,
+        updatedAt: now,
+      })
+    }
+
+    return { skillsUpdated: args.skillDeltas.length }
+  },
+})
+
+/**
+ * Action that processes skill stat events in batches outside the transaction window.
+ *
+ * Algorithm:
+ * 1. Get current cursor position
+ * 2. Fetch events in batches of 500, aggregating as we go
+ * 3. Stop when we have >= 500 unique skills OR run out of events
+ * 4. Call mutation to apply all deltas and update cursor atomically
+ * 5. Self-schedule if we stopped due to skill limit (not exhaustion)
+ */
+export const processSkillStatEventsAction = internalAction({
+  args: {},
+  handler: async (ctx) => {
+    // Get current cursor position (convert null to undefined for consistency)
+    const cursorResult = await ctx.runQuery(internal.skillStatEvents.getStatEventCursor)
+    let cursor: number | undefined = cursorResult ?? undefined
+
+    console.log(`[STAT-AGG] Starting aggregation, cursor=${cursor ?? 'none'}`)
+
+    // Aggregated deltas per skill
+    const aggregatedBySkill = new Map<
+      Id<'skills'>,
+      {
+        downloads: number
+        stars: number
+        comments: number
+        installsAllTime: number
+        installsCurrent: number
+        downloadEvents: number[]
+        installNewEvents: number[]
+      }
+    >()
+
+    let maxCreationTime: number | undefined = cursor
+    let exhausted = false
+    let totalEventsFetched = 0
+
+    // Fetch and aggregate until we have enough skills or run out of events
+    while (aggregatedBySkill.size < MAX_SKILLS_PER_RUN) {
+      const events = await ctx.runQuery(internal.skillStatEvents.getUnprocessedEventBatch, {
+        cursorCreationTime: cursor,
+        limit: EVENT_BATCH_SIZE,
+      })
+
+      if (events.length === 0) {
+        exhausted = true
+        break
+      }
+
+      totalEventsFetched += events.length
+      const skillsBefore = aggregatedBySkill.size
+
+      // Aggregate events into per-skill deltas
+      for (const event of events) {
+        let skillDelta = aggregatedBySkill.get(event.skillId)
+        if (!skillDelta) {
+          skillDelta = {
+            downloads: 0,
+            stars: 0,
+            comments: 0,
+            installsAllTime: 0,
+            installsCurrent: 0,
+            downloadEvents: [],
+            installNewEvents: [],
+          }
+          aggregatedBySkill.set(event.skillId, skillDelta)
+        }
+
+        // Apply event to aggregated deltas
+        switch (event.kind) {
+          case 'download':
+            skillDelta.downloads += 1
+            skillDelta.downloadEvents.push(event.occurredAt)
+            break
+          case 'star':
+            skillDelta.stars += 1
+            break
+          case 'unstar':
+            skillDelta.stars -= 1
+            break
+          case 'comment':
+            skillDelta.comments += 1
+            break
+          case 'uncomment':
+            skillDelta.comments -= 1
+            break
+          case 'install_new':
+            skillDelta.installsAllTime += 1
+            skillDelta.installsCurrent += 1
+            skillDelta.installNewEvents.push(event.occurredAt)
+            break
+          case 'install_reactivate':
+            skillDelta.installsCurrent += 1
+            break
+          case 'install_deactivate':
+            skillDelta.installsCurrent -= 1
+            break
+          case 'install_clear':
+            if (event.delta) {
+              skillDelta.installsAllTime += event.delta.allTime
+              skillDelta.installsCurrent += event.delta.current
+            }
+            break
+        }
+
+        // Track highest _creationTime seen
+        if (maxCreationTime === undefined || event._creationTime > maxCreationTime) {
+          maxCreationTime = event._creationTime
+        }
+      }
+
+      // Update cursor for next batch fetch
+      cursor = events[events.length - 1]._creationTime
+
+      console.log(
+        `[STAT-AGG] Fetched ${events.length} events, ${aggregatedBySkill.size - skillsBefore} new skills (${aggregatedBySkill.size} total)`,
+      )
+
+      // If we got fewer than requested, we've exhausted the events
+      if (events.length < EVENT_BATCH_SIZE) {
+        exhausted = true
+        break
+      }
+    }
+
+    // If we have nothing to process, we're done
+    if (aggregatedBySkill.size === 0 || maxCreationTime === undefined) {
+      console.log('[STAT-AGG] No events to process, done')
+      return { processed: 0, skillsUpdated: 0, exhausted: true }
+    }
+
+    // Convert map to array for mutation
+    const skillDeltas = Array.from(aggregatedBySkill.entries()).map(([skillId, delta]) => ({
+      skillId,
+      ...delta,
+    }))
+
+    console.log(
+      `[STAT-AGG] Running mutation for ${skillDeltas.length} skills (${totalEventsFetched} total events)`,
+    )
+
+    // Apply all deltas and update cursor atomically
+    await ctx.runMutation(internal.skillStatEvents.applyAggregatedStatsAndUpdateCursor, {
+      skillDeltas,
+      newCursor: maxCreationTime,
+    })
+
+    // Self-schedule if we stopped because of skill limit, not exhaustion
+    if (!exhausted) {
+      console.log('[STAT-AGG] More events remaining, self-scheduling')
+      await ctx.scheduler.runAfter(0, internal.skillStatEvents.processSkillStatEventsAction, {})
+    } else {
+      console.log('[STAT-AGG] All events processed, done')
+    }
+
+    return {
+      skillsUpdated: skillDeltas.length,
+      exhausted,
+    }
+  },
+})

convex/soulComments.ts

@@ -0,0 +1,88 @@
+import { v } from 'convex/values'
+import type { Doc } from './_generated/dataModel'
+import { mutation, query } from './_generated/server'
+import { assertModerator, requireUser } from './lib/access'
+import { type PublicUser, toPublicUser } from './lib/public'
+
+export const listBySoul = query({
+  args: { soulId: v.id('souls'), limit: v.optional(v.number()) },
+  handler: async (ctx, args) => {
+    const limit = args.limit ?? 50
+    const comments = await ctx.db
+      .query('soulComments')
+      .withIndex('by_soul', (q) => q.eq('soulId', args.soulId))
+      .order('desc')
+      .take(limit)
+
+    const results: Array<{ comment: Doc<'soulComments'>; user: PublicUser | null }> = []
+    for (const comment of comments) {
+      if (comment.softDeletedAt) continue
+      const user = toPublicUser(await ctx.db.get(comment.userId))
+      results.push({ comment, user })
+    }
+    return results
+  },
+})
+
+export const add = mutation({
+  args: { soulId: v.id('souls'), body: v.string() },
+  handler: async (ctx, args) => {
+    const { userId } = await requireUser(ctx)
+    const body = args.body.trim()
+    if (!body) throw new Error('Comment body required')
+
+    const soul = await ctx.db.get(args.soulId)
+    if (!soul) throw new Error('Soul not found')
+
+    await ctx.db.insert('soulComments', {
+      soulId: args.soulId,
+      userId,
+      body,
+      createdAt: Date.now(),
+      softDeletedAt: undefined,
+      deletedBy: undefined,
+    })
+
+    await ctx.db.patch(soul._id, {
+      stats: { ...soul.stats, comments: soul.stats.comments + 1 },
+      updatedAt: Date.now(),
+    })
+  },
+})
+
+export const remove = mutation({
+  args: { commentId: v.id('soulComments') },
+  handler: async (ctx, args) => {
+    const { user } = await requireUser(ctx)
+    const comment = await ctx.db.get(args.commentId)
+    if (!comment) throw new Error('Comment not found')
+    if (comment.softDeletedAt) return
+
+    const isOwner = comment.userId === user._id
+    if (!isOwner) {
+      assertModerator(user)
+    }
+
+    await ctx.db.patch(comment._id, {
+      softDeletedAt: Date.now(),
+      deletedBy: user._id,
+    })
+
+    const soul = await ctx.db.get(comment.soulId)
+    if (soul) {
+      await ctx.db.patch(soul._id, {
+        stats: { ...soul.stats, comments: Math.max(0, soul.stats.comments - 1) },
+        updatedAt: Date.now(),
+      })
+    }
+
+    await ctx.db.insert('auditLogs', {
+      actorUserId: user._id,
+      action: 'soul.comment.delete',
+      targetType: 'soulComment',
+      targetId: comment._id,
+      metadata: { soulId: comment.soulId },
+      createdAt: Date.now(),
+    })
+  },
+})

convex/soulDownloads.ts

@@ -0,0 +1,14 @@
+import { v } from 'convex/values'
+import { mutation } from './_generated/server'
+
+export const increment = mutation({
+  args: { soulId: v.id('souls') },
+  handler: async (ctx, args) => {
+    const soul = await ctx.db.get(args.soulId)
+    if (!soul) return
+    await ctx.db.patch(soul._id, {
+      stats: { ...soul.stats, downloads: soul.stats.downloads + 1 },
+      updatedAt: Date.now(),
+    })
+  },
+})

convex/soulStars.ts

@@ -0,0 +1,71 @@
+import { v } from 'convex/values'
+import { mutation, query } from './_generated/server'
+import { requireUser } from './lib/access'
+import { toPublicSoul } from './lib/public'
+
+export const isStarred = query({
+  args: { soulId: v.id('souls') },
+  handler: async (ctx, args) => {
+    const { userId } = await requireUser(ctx)
+    const existing = await ctx.db
+      .query('soulStars')
+      .withIndex('by_soul_user', (q) => q.eq('soulId', args.soulId).eq('userId', userId))
+      .unique()
+    return Boolean(existing)
+  },
+})
+
+export const toggle = mutation({
+  args: { soulId: v.id('souls') },
+  handler: async (ctx, args) => {
+    const { userId } = await requireUser(ctx)
+    const soul = await ctx.db.get(args.soulId)
+    if (!soul) throw new Error('Soul not found')
+
+    const existing = await ctx.db
+      .query('soulStars')
+      .withIndex('by_soul_user', (q) => q.eq('soulId', args.soulId).eq('userId', userId))
+      .unique()
+
+    if (existing) {
+      await ctx.db.delete(existing._id)
+      await ctx.db.patch(soul._id, {
+        stats: { ...soul.stats, stars: Math.max(0, soul.stats.stars - 1) },
+        updatedAt: Date.now(),
+      })
+      return { starred: false }
+    }
+
+    await ctx.db.insert('soulStars', {
+      soulId: args.soulId,
+      userId,
+      createdAt: Date.now(),
+    })
+    await ctx.db.patch(soul._id, {
+      stats: { ...soul.stats, stars: soul.stats.stars + 1 },
+      updatedAt: Date.now(),
+    })
+    return { starred: true }
+  },
+})
+
+export const listByUser = query({
+  args: { userId: v.id('users'), limit: v.optional(v.number()) },
+  handler: async (ctx, args) => {
+    const limit = args.limit ?? 50
+    const stars = await ctx.db
+      .query('soulStars')
+      .withIndex('by_user', (q) => q.eq('userId', args.userId))
+      .order('desc')
+      .take(limit)
+
+    const souls: NonNullable<ReturnType<typeof toPublicSoul>>[] = []
+    for (const star of stars) {
+      const soul = await ctx.db.get(star.soulId)
+      const publicSoul = toPublicSoul(soul)
+      if (!publicSoul) continue
+      souls.push(publicSoul)
+    }
+    return souls
+  },
+})

convex/souls.ts

@@ -0,0 +1,570 @@
+import { ConvexError, v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import { action, internalMutation, internalQuery, mutation, query } from './_generated/server'
+import { assertModerator, requireUser, requireUserFromAction } from './lib/access'
+import { toPublicSoul, toPublicUser } from './lib/public'
+import { getFrontmatterValue, hashSkillFiles } from './lib/skills'
+import { generateSoulChangelogPreview } from './lib/soulChangelog'
+import { fetchText, type PublishResult, publishSoulVersionForUser } from './lib/soulPublish'
+
+export { publishSoulVersionForUser } from './lib/soulPublish'
+
+type ReadmeResult = { path: string; text: string }
+
+type FileTextResult = { path: string; text: string; size: number; sha256: string }
+
+const MAX_DIFF_FILE_BYTES = 200 * 1024
+const MAX_LIST_LIMIT = 50
+
+export const getBySlug = query({
+  args: { slug: v.string() },
+  handler: async (ctx, args) => {
+    const matches = await ctx.db
+      .query('souls')
+      .withIndex('by_slug', (q) => q.eq('slug', args.slug))
+      .order('desc')
+      .take(2)
+    const soul = matches[0] ?? null
+    if (!soul || soul.softDeletedAt) return null
+    const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
+    const owner = toPublicUser(await ctx.db.get(soul.ownerUserId))
+    const publicSoul = toPublicSoul(soul)
+    if (!publicSoul) return null
+
+    return { soul: publicSoul, latestVersion, owner }
+  },
+})
+
+export const getSoulBySlugInternal = internalQuery({
+  args: { slug: v.string() },
+  handler: async (ctx, args) => {
+    const matches = await ctx.db
+      .query('souls')
+      .withIndex('by_slug', (q) => q.eq('slug', args.slug))
+      .order('desc')
+      .take(2)
+    return matches[0] ?? null
+  },
+})
+
+export const list = query({
+  args: {
+    ownerUserId: v.optional(v.id('users')),
+    limit: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const limit = args.limit ?? 24
+    const ownerUserId = args.ownerUserId
+    if (ownerUserId) {
+      const entries = await ctx.db
+        .query('souls')
+        .withIndex('by_owner', (q) => q.eq('ownerUserId', ownerUserId))
+        .order('desc')
+        .take(limit * 5)
+      return entries
+        .filter((soul) => !soul.softDeletedAt)
+        .slice(0, limit)
+        .map((soul) => toPublicSoul(soul))
+        .filter((soul): soul is NonNullable<typeof soul> => Boolean(soul))
+    }
+    const entries = await ctx.db
+      .query('souls')
+      .order('desc')
+      .take(limit * 5)
+    return entries
+      .filter((soul) => !soul.softDeletedAt)
+      .slice(0, limit)
+      .map((soul) => toPublicSoul(soul))
+      .filter((soul): soul is NonNullable<typeof soul> => Boolean(soul))
+  },
+})
+
+export const listPublicPage = query({
+  args: {
+    cursor: v.optional(v.string()),
+    limit: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const limit = clampInt(args.limit ?? 24, 1, MAX_LIST_LIMIT)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('souls')
+      .withIndex('by_updated', (q) => q)
+      .order('desc')
+      .paginate({ cursor: args.cursor ?? null, numItems: limit })
+
+    const items: Array<{
+      soul: NonNullable<ReturnType<typeof toPublicSoul>>
+      latestVersion: Doc<'soulVersions'> | null
+    }> = []
+
+    for (const soul of page) {
+      if (soul.softDeletedAt) continue
+      const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
+      const publicSoul = toPublicSoul(soul)
+      if (!publicSoul) continue
+      items.push({ soul: publicSoul, latestVersion })
+    }
+
+    return { items, nextCursor: isDone ? null : continueCursor }
+  },
+})
+
+export const listVersions = query({
+  args: { soulId: v.id('souls'), limit: v.optional(v.number()) },
+  handler: async (ctx, args) => {
+    const limit = args.limit ?? 20
+    return ctx.db
+      .query('soulVersions')
+      .withIndex('by_soul', (q) => q.eq('soulId', args.soulId))
+      .order('desc')
+      .take(limit)
+  },
+})
+
+export const listVersionsPage = query({
+  args: {
+    soulId: v.id('souls'),
+    cursor: v.optional(v.string()),
+    limit: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const limit = clampInt(args.limit ?? 20, 1, MAX_LIST_LIMIT)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('soulVersions')
+      .withIndex('by_soul', (q) => q.eq('soulId', args.soulId))
+      .order('desc')
+      .paginate({ cursor: args.cursor ?? null, numItems: limit })
+    const items = page.filter((version) => !version.softDeletedAt)
+    return { items, nextCursor: isDone ? null : continueCursor }
+  },
+})
+
+export const getVersionById = query({
+  args: { versionId: v.id('soulVersions') },
+  handler: async (ctx, args) => ctx.db.get(args.versionId),
+})
+
+export const getVersionByIdInternal = internalQuery({
+  args: { versionId: v.id('soulVersions') },
+  handler: async (ctx, args) => ctx.db.get(args.versionId),
+})
+
+export const getVersionBySoulAndVersion = query({
+  args: { soulId: v.id('souls'), version: v.string() },
+  handler: async (ctx, args) => {
+    return ctx.db
+      .query('soulVersions')
+      .withIndex('by_soul_version', (q) => q.eq('soulId', args.soulId).eq('version', args.version))
+      .unique()
+  },
+})
+
+export const publishVersion: ReturnType<typeof action> = action({
+  args: {
+    slug: v.string(),
+    displayName: v.string(),
+    version: v.string(),
+    changelog: v.string(),
+    tags: v.optional(v.array(v.string())),
+    source: v.optional(
+      v.object({
+        kind: v.literal('github'),
+        url: v.string(),
+        repo: v.string(),
+        ref: v.string(),
+        commit: v.string(),
+        path: v.string(),
+        importedAt: v.number(),
+      }),
+    ),
+    files: v.array(
+      v.object({
+        path: v.string(),
+        size: v.number(),
+        storageId: v.id('_storage'),
+        sha256: v.string(),
+        contentType: v.optional(v.string()),
+      }),
+    ),
+  },
+  handler: async (ctx, args): Promise<PublishResult> => {
+    const { userId } = await requireUserFromAction(ctx)
+    return publishSoulVersionForUser(ctx, userId, args)
+  },
+})
+
+export const generateChangelogPreview = action({
+  args: {
+    slug: v.string(),
+    version: v.string(),
+    readmeText: v.string(),
+    filePaths: v.optional(v.array(v.string())),
+  },
+  handler: async (ctx, args) => {
+    await requireUserFromAction(ctx)
+    const changelog = await generateSoulChangelogPreview(ctx, {
+      slug: args.slug.trim().toLowerCase(),
+      version: args.version.trim(),
+      readmeText: args.readmeText,
+      filePaths: args.filePaths?.map((value) => value.trim()).filter(Boolean),
+    })
+    return { changelog, source: 'auto' as const }
+  },
+})
+
+export const getReadme: ReturnType<typeof action> = action({
+  args: { versionId: v.id('soulVersions') },
+  handler: async (ctx, args): Promise<ReadmeResult> => {
+    const version = (await ctx.runQuery(internal.souls.getVersionByIdInternal, {
+      versionId: args.versionId,
+    })) as Doc<'soulVersions'> | null
+    if (!version) throw new ConvexError('Version not found')
+    const readmeFile = version.files.find((file) => file.path.toLowerCase() === 'soul.md')
+    if (!readmeFile) throw new ConvexError('SOUL.md not found')
+    const text = await fetchText(ctx, readmeFile.storageId)
+    return { path: readmeFile.path, text }
+  },
+})
+
+export const getFileText: ReturnType<typeof action> = action({
+  args: { versionId: v.id('soulVersions'), path: v.string() },
+  handler: async (ctx, args): Promise<FileTextResult> => {
+    const version = (await ctx.runQuery(internal.souls.getVersionByIdInternal, {
+      versionId: args.versionId,
+    })) as Doc<'soulVersions'> | null
+    if (!version) throw new ConvexError('Version not found')
+
+    const normalizedPath = args.path.trim()
+    const normalizedLower = normalizedPath.toLowerCase()
+    const file =
+      version.files.find((entry) => entry.path === normalizedPath) ??
+      version.files.find((entry) => entry.path.toLowerCase() === normalizedLower)
+    if (!file) throw new ConvexError('File not found')
+    if (file.size > MAX_DIFF_FILE_BYTES) {
+      throw new ConvexError('File exceeds 200KB limit')
+    }
+
+    const text = await fetchText(ctx, file.storageId)
+    return { path: file.path, text, size: file.size, sha256: file.sha256 }
+  },
+})
+
+export const resolveVersionByHash = query({
+  args: { slug: v.string(), hash: v.string() },
+  handler: async (ctx, args) => {
+    const slug = args.slug.trim().toLowerCase()
+    const hash = args.hash.trim().toLowerCase()
+    if (!slug || !/^[a-f0-9]{64}$/.test(hash)) return null
+
+    const soulMatches = await ctx.db
+      .query('souls')
+      .withIndex('by_slug', (q) => q.eq('slug', slug))
+      .order('desc')
+      .take(2)
+    const soul = soulMatches[0] ?? null
+    if (!soul || soul.softDeletedAt) return null
+
+    const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
+
+    const fingerprintMatches = await ctx.db
+      .query('soulVersionFingerprints')
+      .withIndex('by_soul_fingerprint', (q) => q.eq('soulId', soul._id).eq('fingerprint', hash))
+      .take(25)
+
+    let match: { version: string } | null = null
+    if (fingerprintMatches.length > 0) {
+      const newest = fingerprintMatches.reduce(
+        (best, entry) => (entry.createdAt > best.createdAt ? entry : best),
+        fingerprintMatches[0] as (typeof fingerprintMatches)[number],
+      )
+      const version = await ctx.db.get(newest.versionId)
+      if (version && !version.softDeletedAt) {
+        match = { version: version.version }
+      }
+    }
+
+    if (!match) {
+      const versions = await ctx.db
+        .query('soulVersions')
+        .withIndex('by_soul', (q) => q.eq('soulId', soul._id))
+        .order('desc')
+        .take(200)
+
+      for (const version of versions) {
+        if (version.softDeletedAt) continue
+        if (typeof version.fingerprint === 'string' && version.fingerprint === hash) {
+          match = { version: version.version }
+          break
+        }
+
+        const fingerprint = await hashSkillFiles(
+          version.files.map((file) => ({ path: file.path, sha256: file.sha256 })),
+        )
+        if (fingerprint === hash) {
+          match = { version: version.version }
+          break
+        }
+      }
+    }
+
+    return {
+      match,
+      latestVersion: latestVersion ? { version: latestVersion.version } : null,
+    }
+  },
+})
+
+export const updateTags = mutation({
+  args: {
+    soulId: v.id('souls'),
+    tags: v.array(v.object({ tag: v.string(), versionId: v.id('soulVersions') })),
+  },
+  handler: async (ctx, args) => {
+    const { user } = await requireUser(ctx)
+    const soul = await ctx.db.get(args.soulId)
+    if (!soul) throw new Error('Soul not found')
+    if (soul.ownerUserId !== user._id) {
+      assertModerator(user)
+    }
+
+    const nextTags = { ...soul.tags }
+    for (const entry of args.tags) {
+      nextTags[entry.tag] = entry.versionId
+    }
+
+    const latestEntry = args.tags.find((entry) => entry.tag === 'latest')
+    await ctx.db.patch(soul._id, {
+      tags: nextTags,
+      latestVersionId: latestEntry ? latestEntry.versionId : soul.latestVersionId,
+      updatedAt: Date.now(),
+    })
+
+    if (latestEntry) {
+      const embeddings = await ctx.db
+        .query('soulEmbeddings')
+        .withIndex('by_soul', (q) => q.eq('soulId', soul._id))
+        .collect()
+      for (const embedding of embeddings) {
+        const isLatest = embedding.versionId === latestEntry.versionId
+        await ctx.db.patch(embedding._id, {
+          isLatest,
+          visibility: visibilityFor(isLatest, embedding.isApproved),
+          updatedAt: Date.now(),
+        })
+      }
+    }
+  },
+})
+
+export const insertVersion = internalMutation({
+  args: {
+    userId: v.id('users'),
+    slug: v.string(),
+    displayName: v.string(),
+    version: v.string(),
+    changelog: v.string(),
+    changelogSource: v.optional(v.union(v.literal('auto'), v.literal('user'))),
+    tags: v.optional(v.array(v.string())),
+    fingerprint: v.string(),
+    summary: v.optional(v.string()),
+    files: v.array(
+      v.object({
+        path: v.string(),
+        size: v.number(),
+        storageId: v.id('_storage'),
+        sha256: v.string(),
+        contentType: v.optional(v.string()),
+      }),
+    ),
+    parsed: v.object({
+      frontmatter: v.record(v.string(), v.any()),
+      metadata: v.optional(v.any()),
+    }),
+    embedding: v.array(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const userId = args.userId
+    const user = await ctx.db.get(userId)
+    if (!user || user.deletedAt) throw new Error('User not found')
+
+    const soulMatches = await ctx.db
+      .query('souls')
+      .withIndex('by_slug', (q) => q.eq('slug', args.slug))
+      .order('desc')
+      .take(2)
+    let soul: Doc<'souls'> | null = soulMatches[0] ?? null
+
+    if (soul && soul.ownerUserId !== userId) {
+      throw new Error('Only the owner can publish updates')
+    }
+
+    const now = Date.now()
+    if (!soul) {
+      const summary = args.summary ?? getFrontmatterValue(args.parsed.frontmatter, 'description')
+      const soulId = await ctx.db.insert('souls', {
+        slug: args.slug,
+        displayName: args.displayName,
+        summary: summary ?? undefined,
+        ownerUserId: userId,
+        latestVersionId: undefined,
+        tags: {},
+        softDeletedAt: undefined,
+        stats: {
+          downloads: 0,
+          stars: 0,
+          versions: 0,
+          comments: 0,
+        },
+        createdAt: now,
+        updatedAt: now,
+      })
+      soul = await ctx.db.get(soulId)
+    }
+
+    if (!soul) throw new Error('Soul creation failed')
+
+    const existingVersion = await ctx.db
+      .query('soulVersions')
+      .withIndex('by_soul_version', (q) => q.eq('soulId', soul._id).eq('version', args.version))
+      .unique()
+    if (existingVersion) {
+      throw new Error('Version already exists')
+    }
+
+    const versionId = await ctx.db.insert('soulVersions', {
+      soulId: soul._id,
+      version: args.version,
+      fingerprint: args.fingerprint,
+      changelog: args.changelog,
+      changelogSource: args.changelogSource,
+      files: args.files,
+      parsed: args.parsed,
+      createdBy: userId,
+      createdAt: now,
+      softDeletedAt: undefined,
+    })
+
+    const nextTags: Record<string, Id<'soulVersions'>> = { ...soul.tags }
+    nextTags.latest = versionId
+    for (const tag of args.tags ?? []) {
+      nextTags[tag] = versionId
+    }
+
+    const latestBefore = soul.latestVersionId
+
+    await ctx.db.patch(soul._id, {
+      displayName: args.displayName,
+      summary:
+        args.summary ?? getFrontmatterValue(args.parsed.frontmatter, 'description') ?? soul.summary,
+      latestVersionId: versionId,
+      tags: nextTags,
+      stats: { ...soul.stats, versions: soul.stats.versions + 1 },
+      softDeletedAt: undefined,
+      updatedAt: now,
+    })
+
+    const embeddingId = await ctx.db.insert('soulEmbeddings', {
+      soulId: soul._id,
+      versionId,
+      ownerId: userId,
+      embedding: args.embedding,
+      isLatest: true,
+      isApproved: true,
+      visibility: visibilityFor(true, true),
+      updatedAt: now,
+    })
+
+    if (latestBefore) {
+      const previousEmbedding = await ctx.db
+        .query('soulEmbeddings')
+        .withIndex('by_version', (q) => q.eq('versionId', latestBefore))
+        .unique()
+      if (previousEmbedding) {
+        await ctx.db.patch(previousEmbedding._id, {
+          isLatest: false,
+          visibility: visibilityFor(false, previousEmbedding.isApproved),
+          updatedAt: now,
+        })
+      }
+    }
+
+    await ctx.db.insert('soulVersionFingerprints', {
+      soulId: soul._id,
+      versionId,
+      fingerprint: args.fingerprint,
+      createdAt: now,
+    })
+
+    return { soulId: soul._id, versionId, embeddingId }
+  },
+})
+
+export const setSoulSoftDeletedInternal = internalMutation({
+  args: {
+    userId: v.id('users'),
+    slug: v.string(),
+    deleted: v.boolean(),
+  },
+  handler: async (ctx, args) => {
+    const user = await ctx.db.get(args.userId)
+    if (!user || user.deletedAt) throw new Error('User not found')
+
+    const slug = args.slug.trim().toLowerCase()
+    if (!slug) throw new Error('Slug required')
+
+    const soulMatches = await ctx.db
+      .query('souls')
+      .withIndex('by_slug', (q) => q.eq('slug', slug))
+      .order('desc')
+      .take(2)
+    const soul = soulMatches[0] ?? null
+    if (!soul) throw new Error('Soul not found')
+
+    if (soul.ownerUserId !== args.userId) {
+      assertModerator(user)
+    }
+
+    const now = Date.now()
+    await ctx.db.patch(soul._id, {
+      softDeletedAt: args.deleted ? now : undefined,
+      updatedAt: now,
+    })
+
+    const embeddings = await ctx.db
+      .query('soulEmbeddings')
+      .withIndex('by_soul', (q) => q.eq('soulId', soul._id))
+      .collect()
+    for (const embedding of embeddings) {
+      await ctx.db.patch(embedding._id, {
+        visibility: args.deleted
+          ? 'deleted'
+          : visibilityFor(embedding.isLatest, embedding.isApproved),
+        updatedAt: now,
+      })
+    }
+
+    await ctx.db.insert('auditLogs', {
+      actorUserId: args.userId,
+      action: args.deleted ? 'soul.delete' : 'soul.undelete',
+      targetType: 'soul',
+      targetId: soul._id,
+      metadata: { slug, softDeletedAt: args.deleted ? now : null },
+      createdAt: now,
+    })
+
+    return { ok: true as const }
+  },
+})
+
+function visibilityFor(isLatest: boolean, isApproved: boolean) {
+  if (isLatest && isApproved) return 'latest-approved'
+  if (isLatest) return 'latest'
+  if (isApproved) return 'archived-approved'
+  return 'archived'
+}
+
+function clampInt(value: number, min: number, max: number) {
+  const rounded = Number.isFinite(value) ? Math.round(value) : min
+  return Math.min(max, Math.max(min, rounded))
+}

convex/stars.ts

@@ -1,7 +1,8 @@
 import { v } from 'convex/values'
-import type { Doc } from './_generated/dataModel'
-import { mutation, query } from './_generated/server'
+import { internalMutation, mutation, query } from './_generated/server'
 import { requireUser } from './lib/access'
+import { toPublicSkill } from './lib/public'
+import { insertStatEvent } from './skillStatEvents'
 
 export const isStarred = query({
   args: { skillId: v.id('skills') },
@@ -29,10 +30,7 @@ export const toggle = mutation({
 
     if (existing) {
       await ctx.db.delete(existing._id)
-      await ctx.db.patch(skill._id, {
-        stats: { ...skill.stats, stars: Math.max(0, skill.stats.stars - 1) },
-        updatedAt: Date.now(),
-      })
+      await insertStatEvent(ctx, { skillId: skill._id, kind: 'unstar' })
       return { starred: false }
     }
 
@@ -42,10 +40,7 @@ export const toggle = mutation({
       createdAt: Date.now(),
     })
 
-    await ctx.db.patch(skill._id, {
-      stats: { ...skill.stats, stars: skill.stats.stars + 1 },
-      updatedAt: Date.now(),
-    })
+    await insertStatEvent(ctx, { skillId: skill._id, kind: 'star' })
 
     return { starred: true }
   },
@@ -60,11 +55,54 @@ export const listByUser = query({
       .withIndex('by_user', (q) => q.eq('userId', args.userId))
       .order('desc')
       .take(limit)
-    const skills: Doc<'skills'>[] = []
+    const skills: NonNullable<ReturnType<typeof toPublicSkill>>[] = []
     for (const star of stars) {
       const skill = await ctx.db.get(star.skillId)
-      if (skill) skills.push(skill)
+      const publicSkill = toPublicSkill(skill)
+      if (!publicSkill) continue
+      skills.push(publicSkill)
     }
     return skills
   },
 })
+
+export const addStarInternal = internalMutation({
+  args: { userId: v.id('users'), skillId: v.id('skills') },
+  handler: async (ctx, args) => {
+    const skill = await ctx.db.get(args.skillId)
+    if (!skill) throw new Error('Skill not found')
+    const existing = await ctx.db
+      .query('stars')
+      .withIndex('by_skill_user', (q) => q.eq('skillId', args.skillId).eq('userId', args.userId))
+      .unique()
+    if (existing) return { ok: true as const, starred: true, alreadyStarred: true }
+
+    await ctx.db.insert('stars', {
+      skillId: args.skillId,
+      userId: args.userId,
+      createdAt: Date.now(),
+    })
+
+    await insertStatEvent(ctx, { skillId: skill._id, kind: 'star' })
+
+    return { ok: true as const, starred: true, alreadyStarred: false }
+  },
+})
+
+export const removeStarInternal = internalMutation({
+  args: { userId: v.id('users'), skillId: v.id('skills') },
+  handler: async (ctx, args) => {
+    const skill = await ctx.db.get(args.skillId)
+    if (!skill) throw new Error('Skill not found')
+    const existing = await ctx.db
+      .query('stars')
+      .withIndex('by_skill_user', (q) => q.eq('skillId', args.skillId).eq('userId', args.userId))
+      .unique()
+    if (!existing) return { ok: true as const, unstarred: false, alreadyUnstarred: true }
+
+    await ctx.db.delete(existing._id)
+    await insertStatEvent(ctx, { skillId: skill._id, kind: 'unstar' })
+
+    return { ok: true as const, unstarred: true, alreadyUnstarred: false }
+  },
+})

convex/statsMaintenance.ts

@@ -0,0 +1,301 @@
+import { v } from 'convex/values'
+import { internal } from './_generated/api'
+import type { Doc } from './_generated/dataModel'
+import type { ActionCtx } from './_generated/server'
+import { internalAction, internalMutation, internalQuery } from './_generated/server'
+
+const DEFAULT_BATCH_SIZE = 200
+const MAX_BATCH_SIZE = 1000
+const DEFAULT_MAX_BATCHES = 5
+const MAX_MAX_BATCHES = 50
+const BACKFILL_STATE_KEY = 'default'
+
+export const backfillSkillStatFieldsInternal = internalMutation({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('skills')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    let patched = 0
+    for (const skill of page) {
+      const next = buildSkillStatPatch(skill)
+      if (!next) continue
+      await ctx.db.patch(skill._id, next)
+      patched += 1
+    }
+
+    return {
+      ok: true as const,
+      scanned: page.length,
+      patched,
+      cursor: isDone ? null : continueCursor,
+      isDone,
+    }
+  },
+})
+
+type BackfillState = {
+  cursor: string | null
+  doneAt?: number
+}
+
+type BackfillActionArgs = {
+  batchSize?: number
+  maxBatches?: number
+  resetCursor?: boolean
+}
+
+type BackfillStats = {
+  scanned: number
+  patched: number
+  batches: number
+}
+
+type BackfillActionResult = {
+  ok: true
+  isDone: boolean
+  cursor: string | null
+  stats: BackfillStats
+}
+
+export const getSkillStatBackfillStateInternal = internalQuery({
+  args: {},
+  handler: async (ctx): Promise<BackfillState> => {
+    const state = await ctx.db
+      .query('skillStatBackfillState')
+      .withIndex('by_key', (q) => q.eq('key', BACKFILL_STATE_KEY))
+      .unique()
+    return { cursor: state?.cursor ?? null, doneAt: state?.doneAt }
+  },
+})
+
+export const setSkillStatBackfillStateInternal = internalMutation({
+  args: {
+    cursor: v.optional(v.string()),
+    doneAt: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    const state = await ctx.db
+      .query('skillStatBackfillState')
+      .withIndex('by_key', (q) => q.eq('key', BACKFILL_STATE_KEY))
+      .unique()
+
+    if (!state) {
+      await ctx.db.insert('skillStatBackfillState', {
+        key: BACKFILL_STATE_KEY,
+        cursor: args.cursor,
+        doneAt: args.doneAt,
+        updatedAt: now,
+      })
+      return { ok: true as const }
+    }
+
+    await ctx.db.patch(state._id, {
+      cursor: args.cursor,
+      doneAt: args.doneAt,
+      updatedAt: now,
+    })
+
+    return { ok: true as const }
+  },
+})
+
+async function runSkillStatBackfillInternalHandler(
+  ctx: ActionCtx,
+  args: BackfillActionArgs,
+): Promise<BackfillActionResult> {
+  const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
+  const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
+
+  if (args.resetCursor) {
+    await ctx.runMutation(internal.statsMaintenance.setSkillStatBackfillStateInternal, {
+      cursor: undefined,
+      doneAt: undefined,
+    })
+  }
+
+  const state = (await ctx.runQuery(
+    internal.statsMaintenance.getSkillStatBackfillStateInternal,
+    {},
+  )) as BackfillState
+  if (state.doneAt && !args.resetCursor) {
+    return {
+      ok: true,
+      isDone: true,
+      cursor: null,
+      stats: { scanned: 0, patched: 0, batches: 0 },
+    }
+  }
+
+  let cursor: string | null = state.cursor ?? null
+  const stats: BackfillStats = { scanned: 0, patched: 0, batches: 0 }
+
+  for (let i = 0; i < maxBatches; i += 1) {
+    const result = (await ctx.runMutation(
+      internal.statsMaintenance.backfillSkillStatFieldsInternal,
+      {
+        cursor: cursor ?? undefined,
+        batchSize,
+      },
+    )) as { scanned: number; patched: number; cursor: string | null; isDone: boolean }
+    stats.scanned += result.scanned
+    stats.patched += result.patched
+    stats.batches += 1
+    cursor = result.cursor
+
+    if (result.isDone) {
+      await ctx.runMutation(internal.statsMaintenance.setSkillStatBackfillStateInternal, {
+        cursor: undefined,
+        doneAt: Date.now(),
+      })
+      return { ok: true, isDone: true, cursor: null, stats }
+    }
+
+    await ctx.runMutation(internal.statsMaintenance.setSkillStatBackfillStateInternal, {
+      cursor: cursor ?? undefined,
+      doneAt: undefined,
+    })
+  }
+
+  return { ok: true, isDone: false, cursor, stats }
+}
+
+export const runSkillStatBackfillInternal: ReturnType<typeof internalAction> = internalAction({
+  args: {
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+    resetCursor: v.optional(v.boolean()),
+  },
+  handler: runSkillStatBackfillInternalHandler,
+})
+
+function buildSkillStatPatch(skill: Doc<'skills'>) {
+  const stats = skill.stats
+  const nextDownloads = stats.downloads
+  const nextStars = stats.stars
+  const nextInstallsCurrent = stats.installsCurrent ?? 0
+  const nextInstallsAllTime = stats.installsAllTime ?? 0
+
+  if (
+    skill.statsDownloads === nextDownloads &&
+    skill.statsStars === nextStars &&
+    skill.statsInstallsCurrent === nextInstallsCurrent &&
+    skill.statsInstallsAllTime === nextInstallsAllTime
+  ) {
+    return null
+  }
+
+  return {
+    statsDownloads: nextDownloads,
+    statsStars: nextStars,
+    statsInstallsCurrent: nextInstallsCurrent,
+    statsInstallsAllTime: nextInstallsAllTime,
+  }
+}
+
+/**
+ * Reconcile skill stats by counting actual records in source-of-truth tables.
+ *
+ * This fixes stats that got out of sync due to missed events, cursor issues,
+ * or bugs in the event processing pipeline. It counts:
+ * - stars: actual records in the `stars` table for each skill
+ * - comments: actual records in the `comments` table for each skill
+ *
+ * Downloads and installs are event-sourced only (no separate table to count from),
+ * so they cannot be reconciled this way.
+ */
+export const reconcileSkillStarCounts = internalMutation({
+  args: {
+    cursor: v.optional(v.string()),
+    batchSize: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const batchSize = clampInt(args.batchSize ?? 50, 1, 200)
+    const now = Date.now()
+
+    const { page, isDone, continueCursor } = await ctx.db
+      .query('skills')
+      .order('asc')
+      .paginate({ cursor: args.cursor ?? null, numItems: batchSize })
+
+    let patched = 0
+    for (const skill of page) {
+      // Count actual star records for this skill
+      const starRecords = await ctx.db
+        .query('stars')
+        .withIndex('by_skill_user', (q) => q.eq('skillId', skill._id))
+        .collect()
+      const actualStars = starRecords.length
+
+      // Count actual comment records for this skill
+      const commentRecords = await ctx.db
+        .query('comments')
+        .withIndex('by_skill', (q) => q.eq('skillId', skill._id))
+        .collect()
+      const actualComments = commentRecords.filter((c) => !c.softDeletedAt).length
+
+      // Check if stats are out of sync
+      if (skill.stats.stars !== actualStars || skill.stats.comments !== actualComments) {
+        const updatedStats = {
+          ...skill.stats,
+          stars: actualStars,
+          comments: actualComments,
+        }
+        await ctx.db.patch(skill._id, {
+          statsStars: actualStars,
+          stats: updatedStats,
+          updatedAt: now,
+        })
+        patched += 1
+      }
+    }
+
+    return {
+      scanned: page.length,
+      patched,
+      cursor: isDone ? null : continueCursor,
+      isDone,
+    }
+  },
+})
+
+export const runReconcileSkillStarCountsInternal = internalAction({
+  args: {
+    batchSize: v.optional(v.number()),
+    maxBatches: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const batchSize = clampInt(args.batchSize ?? 50, 1, 200)
+    const maxBatches = clampInt(args.maxBatches ?? 10, 1, 50)
+
+    let cursor: string | undefined
+    let totalScanned = 0
+    let totalPatched = 0
+
+    for (let i = 0; i < maxBatches; i++) {
+      const result = (await ctx.runMutation(internal.statsMaintenance.reconcileSkillStarCounts, {
+        cursor,
+        batchSize,
+      })) as { scanned: number; patched: number; cursor: string | null; isDone: boolean }
+
+      totalScanned += result.scanned
+      totalPatched += result.patched
+
+      if (result.isDone) break
+      cursor = result.cursor ?? undefined
+    }
+
+    return { scanned: totalScanned, patched: totalPatched }
+  },
+})
+
+function clampInt(value: number, min: number, max: number) {
+  return Math.min(Math.max(value, min), max)
+}

convex/telemetry.ts

@@ -0,0 +1,434 @@
+import { getAuthUserId } from '@convex-dev/auth/server'
+import { v } from 'convex/values'
+import type { Id } from './_generated/dataModel'
+import type { MutationCtx, QueryCtx } from './_generated/server'
+import { internalMutation, mutation, query } from './_generated/server'
+import { requireUser } from './lib/access'
+import { insertStatEvent } from './skillStatEvents'
+
+const TELEMETRY_STALE_MS = 120 * 24 * 60 * 60 * 1000
+
+type RootPayload = {
+  rootId: string
+  label: string
+  skills: Array<{ slug: string; version?: string | null }>
+}
+
+export const reportCliSyncInternal = internalMutation({
+  args: {
+    userId: v.id('users'),
+    roots: v.array(
+      v.object({
+        rootId: v.string(),
+        label: v.string(),
+        skills: v.array(
+          v.object({
+            slug: v.string(),
+            version: v.optional(v.string()),
+          }),
+        ),
+      }),
+    ),
+  },
+  handler: async (ctx, args) => {
+    const now = Date.now()
+    const stalenessCutoff = now - TELEMETRY_STALE_MS
+
+    await expireStaleRoots(ctx, { userId: args.userId, stalenessCutoff, now })
+
+    const roots = normalizeRoots(args.roots)
+    const skillsBySlug = await resolveSkillsBySlug(ctx, roots)
+
+    for (const root of roots) {
+      await upsertRoot(ctx, { userId: args.userId, rootId: root.rootId, now, label: root.label })
+      await applyRootReport(ctx, {
+        userId: args.userId,
+        root,
+        skillsBySlug,
+        now,
+      })
+    }
+  },
+})
+
+export const clearMyTelemetry = mutation({
+  args: {},
+  handler: async (ctx) => {
+    const { userId } = await requireUser(ctx)
+    await clearTelemetryForUser(ctx, { userId })
+  },
+})
+
+export const clearUserTelemetryInternal = internalMutation({
+  args: { userId: v.id('users') },
+  handler: async (ctx, args) => {
+    await clearTelemetryForUser(ctx, { userId: args.userId })
+  },
+})
+
+export const getMyInstalled = query({
+  args: {
+    includeRemoved: v.optional(v.boolean()),
+  },
+  handler: async (ctx, args) => {
+    const userId = await getAuthUserId(ctx)
+    if (!userId) return null
+
+    const roots = await ctx.db
+      .query('userSyncRoots')
+      .withIndex('by_user', (q) => q.eq('userId', userId))
+      .order('desc')
+      .take(200)
+
+    const includeRemoved = Boolean(args.includeRemoved)
+    const resultRoots: Array<{
+      rootId: string
+      label: string
+      firstSeenAt: number
+      lastSeenAt: number
+      expiredAt?: number
+      skills: Array<{
+        skill: {
+          slug: string
+          displayName: string
+          summary?: string
+          stats: unknown
+          ownerUserId: Id<'users'>
+        }
+        firstSeenAt: number
+        lastSeenAt: number
+        lastVersion?: string
+        removedAt?: number
+      }>
+    }> = []
+
+    for (const root of roots) {
+      const installs = await ctx.db
+        .query('userSkillRootInstalls')
+        .withIndex('by_user_root', (q) => q.eq('userId', userId).eq('rootId', root.rootId))
+        .order('desc')
+        .take(2000)
+
+      const filtered = includeRemoved ? installs : installs.filter((entry) => !entry.removedAt)
+      const skills: Array<{
+        skill: {
+          slug: string
+          displayName: string
+          summary?: string
+          stats: unknown
+          ownerUserId: Id<'users'>
+        }
+        firstSeenAt: number
+        lastSeenAt: number
+        lastVersion?: string
+        removedAt?: number
+      }> = []
+
+      for (const entry of filtered) {
+        const skill = await ctx.db.get(entry.skillId)
+        if (!skill) continue
+        skills.push({
+          skill: {
+            slug: skill.slug,
+            displayName: skill.displayName,
+            summary: skill.summary,
+            stats: skill.stats,
+            ownerUserId: skill.ownerUserId,
+          },
+          firstSeenAt: entry.firstSeenAt,
+          lastSeenAt: entry.lastSeenAt,
+          lastVersion: entry.lastVersion,
+          removedAt: entry.removedAt,
+        })
+      }
+
+      resultRoots.push({
+        rootId: root.rootId,
+        label: root.label,
+        firstSeenAt: root.firstSeenAt,
+        lastSeenAt: root.lastSeenAt,
+        expiredAt: root.expiredAt,
+        skills,
+      })
+    }
+
+    return {
+      roots: resultRoots,
+      cutoffDays: 120,
+    }
+  },
+})
+
+async function clearTelemetryForUser(ctx: MutationCtx, params: { userId: Id<'users'> }) {
+  const installs = await ctx.db
+    .query('userSkillInstalls')
+    .withIndex('by_user', (q) => q.eq('userId', params.userId))
+    .take(5000)
+
+  for (const entry of installs) {
+    const skill = await ctx.db.get(entry.skillId)
+    if (!skill) {
+      await ctx.db.delete(entry._id)
+      continue
+    }
+    await insertStatEvent(ctx, {
+      skillId: skill._id,
+      kind: 'install_clear',
+      delta: {
+        allTime: -1,
+        current: entry.activeRoots > 0 ? -1 : 0,
+      },
+    })
+    await ctx.db.delete(entry._id)
+  }
+
+  const roots = await ctx.db
+    .query('userSyncRoots')
+    .withIndex('by_user', (q) => q.eq('userId', params.userId))
+    .take(5000)
+  for (const root of roots) {
+    await ctx.db.delete(root._id)
+  }
+
+  const rootInstalls = await ctx.db
+    .query('userSkillRootInstalls')
+    .withIndex('by_user', (q) => q.eq('userId', params.userId))
+    .take(10000)
+  for (const entry of rootInstalls) {
+    await ctx.db.delete(entry._id)
+  }
+}
+
+function normalizeRoots(roots: RootPayload[]): RootPayload[] {
+  const seen = new Set<string>()
+  const unique: RootPayload[] = []
+  for (const root of roots) {
+    const id = root.rootId.trim()
+    if (!id) continue
+    if (seen.has(id)) continue
+    seen.add(id)
+    unique.push({
+      rootId: id,
+      label: root.label.trim() || 'Unknown',
+      skills: root.skills
+        .map((skill) => ({
+          slug: skill.slug.trim().toLowerCase(),
+          version: skill.version ?? null,
+        }))
+        .filter((skill) => Boolean(skill.slug)),
+    })
+  }
+  return unique
+}
+
+async function upsertRoot(
+  ctx: MutationCtx,
+  params: { userId: Id<'users'>; rootId: string; now: number; label: string },
+) {
+  const existing = await ctx.db
+    .query('userSyncRoots')
+    .withIndex('by_user_root', (q) => q.eq('userId', params.userId).eq('rootId', params.rootId))
+    .unique()
+  if (existing) {
+    await ctx.db.patch(existing._id, {
+      label: params.label,
+      lastSeenAt: params.now,
+      expiredAt: undefined,
+    })
+    return
+  }
+  await ctx.db.insert('userSyncRoots', {
+    userId: params.userId,
+    rootId: params.rootId,
+    label: params.label,
+    firstSeenAt: params.now,
+    lastSeenAt: params.now,
+    expiredAt: undefined,
+  })
+}
+
+async function applyRootReport(
+  ctx: MutationCtx,
+  params: {
+    userId: Id<'users'>
+    root: RootPayload
+    skillsBySlug: Map<string, { skillId: Id<'skills'> }>
+    now: number
+  },
+) {
+  const expected = new Set<Id<'skills'>>()
+  const versionsBySkill = new Map<Id<'skills'>, string | undefined>()
+  for (const entry of params.root.skills) {
+    const resolved = params.skillsBySlug.get(entry.slug)
+    if (!resolved) continue
+    expected.add(resolved.skillId)
+    const version = entry.version?.trim() || undefined
+    if (version) versionsBySkill.set(resolved.skillId, version)
+  }
+
+  const previous = await ctx.db
+    .query('userSkillRootInstalls')
+    .withIndex('by_user_root', (q) =>
+      q.eq('userId', params.userId).eq('rootId', params.root.rootId),
+    )
+    .take(5000)
+
+  const active = previous.filter((entry) => !entry.removedAt)
+
+  for (const skillId of expected) {
+    const existing = await ctx.db
+      .query('userSkillRootInstalls')
+      .withIndex('by_user_root_skill', (q) =>
+        q.eq('userId', params.userId).eq('rootId', params.root.rootId).eq('skillId', skillId),
+      )
+      .unique()
+
+    const reportedVersion = versionsBySkill.get(skillId)
+
+    if (existing) {
+      const wasRemoved = Boolean(existing.removedAt)
+      await ctx.db.patch(existing._id, {
+        lastSeenAt: params.now,
+        lastVersion: reportedVersion ?? existing.lastVersion,
+        removedAt: undefined,
+      })
+      if (wasRemoved) {
+        await incrementActiveRoots(ctx, {
+          userId: params.userId,
+          skillId,
+          now: params.now,
+          version: reportedVersion,
+        })
+      }
+      continue
+    }
+
+    await ctx.db.insert('userSkillRootInstalls', {
+      userId: params.userId,
+      rootId: params.root.rootId,
+      skillId,
+      firstSeenAt: params.now,
+      lastSeenAt: params.now,
+      lastVersion: reportedVersion,
+    })
+    await incrementActiveRoots(ctx, {
+      userId: params.userId,
+      skillId,
+      now: params.now,
+      version: reportedVersion,
+    })
+  }
+
+  for (const entry of active) {
+    if (expected.has(entry.skillId)) continue
+    await ctx.db.patch(entry._id, { removedAt: params.now })
+    await decrementActiveRoots(ctx, { userId: params.userId, skillId: entry.skillId })
+  }
+}
+
+async function incrementActiveRoots(
+  ctx: MutationCtx,
+  params: { userId: Id<'users'>; skillId: Id<'skills'>; now: number; version?: string },
+) {
+  const existing = await ctx.db
+    .query('userSkillInstalls')
+    .withIndex('by_user_skill', (q) => q.eq('userId', params.userId).eq('skillId', params.skillId))
+    .unique()
+
+  if (!existing) {
+    await ctx.db.insert('userSkillInstalls', {
+      userId: params.userId,
+      skillId: params.skillId,
+      firstSeenAt: params.now,
+      lastSeenAt: params.now,
+      activeRoots: 1,
+      lastVersion: params.version,
+    })
+    await bumpSkillInstallCounts(ctx, { skillId: params.skillId, deltaAllTime: 1, deltaCurrent: 1 })
+    return
+  }
+
+  const nextActive = Math.max(0, (existing.activeRoots ?? 0) + 1)
+  await ctx.db.patch(existing._id, {
+    activeRoots: nextActive,
+    lastSeenAt: params.now,
+    lastVersion: params.version ?? existing.lastVersion,
+  })
+  if ((existing.activeRoots ?? 0) === 0 && nextActive > 0) {
+    await bumpSkillInstallCounts(ctx, { skillId: params.skillId, deltaAllTime: 0, deltaCurrent: 1 })
+  }
+}
+
+async function decrementActiveRoots(
+  ctx: MutationCtx,
+  params: { userId: Id<'users'>; skillId: Id<'skills'> },
+) {
+  const existing = await ctx.db
+    .query('userSkillInstalls')
+    .withIndex('by_user_skill', (q) => q.eq('userId', params.userId).eq('skillId', params.skillId))
+    .unique()
+  if (!existing) return
+
+  const nextActive = Math.max(0, (existing.activeRoots ?? 0) - 1)
+  await ctx.db.patch(existing._id, { activeRoots: nextActive })
+  if ((existing.activeRoots ?? 0) > 0 && nextActive === 0) {
+    await bumpSkillInstallCounts(ctx, {
+      skillId: params.skillId,
+      deltaAllTime: 0,
+      deltaCurrent: -1,
+    })
+  }
+}
+
+async function bumpSkillInstallCounts(
+  ctx: MutationCtx,
+  params: { skillId: Id<'skills'>; deltaAllTime: number; deltaCurrent: number },
+) {
+  if (params.deltaAllTime === 1 && params.deltaCurrent === 1) {
+    await insertStatEvent(ctx, { skillId: params.skillId, kind: 'install_new' })
+  } else if (params.deltaAllTime === 0 && params.deltaCurrent === 1) {
+    await insertStatEvent(ctx, { skillId: params.skillId, kind: 'install_reactivate' })
+  } else if (params.deltaAllTime === 0 && params.deltaCurrent === -1) {
+    await insertStatEvent(ctx, { skillId: params.skillId, kind: 'install_deactivate' })
+  }
+}
+
+async function expireStaleRoots(
+  ctx: MutationCtx,
+  params: { userId: Id<'users'>; stalenessCutoff: number; now: number },
+) {
+  const roots = await ctx.db
+    .query('userSyncRoots')
+    .withIndex('by_user', (q) => q.eq('userId', params.userId))
+    .take(5000)
+
+  const stale = roots.filter((root) => !root.expiredAt && root.lastSeenAt < params.stalenessCutoff)
+  for (const root of stale) {
+    await ctx.db.patch(root._id, { expiredAt: params.now })
+    const installs = await ctx.db
+      .query('userSkillRootInstalls')
+      .withIndex('by_user_root', (q) => q.eq('userId', params.userId).eq('rootId', root.rootId))
+      .take(5000)
+    for (const entry of installs) {
+      if (entry.removedAt) continue
+      await ctx.db.patch(entry._id, { removedAt: params.now })
+      await decrementActiveRoots(ctx, { userId: params.userId, skillId: entry.skillId })
+    }
+  }
+}
+
+async function resolveSkillsBySlug(ctx: QueryCtx | MutationCtx, roots: RootPayload[]) {
+  const slugs = new Set<string>()
+  for (const root of roots) {
+    for (const entry of root.skills) slugs.add(entry.slug)
+  }
+  const map = new Map<string, { skillId: Id<'skills'> }>()
+  for (const slug of slugs) {
+    const skill = await ctx.db
+      .query('skills')
+      .withIndex('by_slug', (q) => q.eq('slug', slug))
+      .unique()
+    if (skill && !skill.softDeletedAt) map.set(slug, { skillId: skill._id })
+  }
+  return map
+}

convex/users.ts

@@ -1,16 +1,66 @@
 import { getAuthUserId } from '@convex-dev/auth/server'
 import { v } from 'convex/values'
-import { mutation, query } from './_generated/server'
-import { assertRole, requireUser } from './lib/access'
+import { internal } from './_generated/api'
+import type { Doc, Id } from './_generated/dataModel'
+import type { MutationCtx } from './_generated/server'
+import { internalMutation, internalQuery, mutation, query } from './_generated/server'
+import { assertAdmin, assertModerator, requireUser } from './lib/access'
+import { toPublicUser } from './lib/public'
+import { buildUserSearchResults } from './lib/userSearch'
 
 const DEFAULT_ROLE = 'user'
 const ADMIN_HANDLE = 'steipete'
 
 export const getById = query({
+  args: { userId: v.id('users') },
+  handler: async (ctx, args) => toPublicUser(await ctx.db.get(args.userId)),
+})
+
+export const getByIdInternal = internalQuery({
   args: { userId: v.id('users') },
   handler: async (ctx, args) => ctx.db.get(args.userId),
 })
 
+export const searchInternal = internalQuery({
+  args: {
+    actorUserId: v.id('users'),
+    query: v.optional(v.string()),
+    limit: v.optional(v.number()),
+  },
+  handler: async (ctx, args) => {
+    const actor = await ctx.db.get(args.actorUserId)
+    if (!actor || actor.deletedAt) throw new Error('Unauthorized')
+    assertAdmin(actor)
+
+    const limit = Math.min(Math.max(args.limit ?? 20, 1), 200)
+    const users = await ctx.db.query('users').order('desc').collect()
+    const result = buildUserSearchResults(users, args.query)
+    const items = result.items.slice(0, limit).map((user) => ({
+      userId: user._id,
+      handle: user.handle ?? null,
+      displayName: user.displayName ?? null,
+      name: user.name ?? null,
+      role: user.role ?? null,
+    }))
+    return { items, total: result.total }
+  },
+})
+
+export const updateGithubMetaInternal = internalMutation({
+  args: {
+    userId: v.id('users'),
+    githubCreatedAt: v.number(),
+    githubFetchedAt: v.number(),
+  },
+  handler: async (ctx, args) => {
+    await ctx.db.patch(args.userId, {
+      githubCreatedAt: args.githubCreatedAt,
+      githubFetchedAt: args.githubFetchedAt,
+      updatedAt: args.githubFetchedAt,
+    })
+  },
+})
+
 export const me = query({
   args: {},
   handler: async (ctx) => {
@@ -26,19 +76,18 @@ export const ensure = mutation({
   args: {},
   handler: async (ctx) => {
     const { userId, user } = await requireUser(ctx)
-    const now = Date.now()
     const updates: Record<string, unknown> = {}
 
-    const handle = user.handle ?? user.name ?? user.email?.split('@')[0]
+    const handle = user.handle || user.name || user.email?.split('@')[0]
     if (!user.handle && handle) updates.handle = handle
     if (!user.displayName) updates.displayName = handle
     if (!user.role) {
       updates.role = handle === ADMIN_HANDLE ? 'admin' : DEFAULT_ROLE
     }
     if (!user.createdAt) updates.createdAt = user._creationTime
-    updates.updatedAt = now
 
     if (Object.keys(updates).length > 0) {
+      updates.updatedAt = Date.now()
       await ctx.db.patch(userId, updates)
     }
 
@@ -69,26 +118,31 @@ export const deleteAccount = mutation({
       deletedAt: Date.now(),
       updatedAt: Date.now(),
     })
+    await ctx.runMutation(internal.telemetry.clearUserTelemetryInternal, { userId })
   },
 })
 
 export const list = query({
-  args: { limit: v.optional(v.number()) },
+  args: { limit: v.optional(v.number()), search: v.optional(v.string()) },
   handler: async (ctx, args) => {
     const { user } = await requireUser(ctx)
-    assertRole(user, ['admin'])
-    const limit = args.limit ?? 50
-    return ctx.db.query('users').order('desc').take(limit)
+    assertAdmin(user)
+    const limit = Math.min(Math.max(args.limit ?? 50, 1), 200)
+    const query = args.search?.trim().toLowerCase()
+    const users = await ctx.db.query('users').order('desc').collect()
+    const result = buildUserSearchResults(users, query)
+    return { items: result.items.slice(0, limit), total: result.total }
   },
 })
 
 export const getByHandle = query({
   args: { handle: v.string() },
   handler: async (ctx, args) => {
-    return ctx.db
+    const user = await ctx.db
       .query('users')
       .withIndex('handle', (q) => q.eq('handle', args.handle))
       .unique()
+    return toPublicUser(user)
   },
 })
 
@@ -99,15 +153,208 @@ export const setRole = mutation({
   },
   handler: async (ctx, args) => {
     const { user } = await requireUser(ctx)
-    assertRole(user, ['admin'])
-    await ctx.db.patch(args.userId, { role: args.role, updatedAt: Date.now() })
+    return setRoleWithActor(ctx, user, args.userId, args.role)
+  },
+})
+
+export const setRoleInternal = internalMutation({
+  args: {
+    actorUserId: v.id('users'),
+    targetUserId: v.id('users'),
+    role: v.union(v.literal('admin'), v.literal('moderator'), v.literal('user')),
+  },
+  handler: async (ctx, args) => {
+    const actor = await ctx.db.get(args.actorUserId)
+    if (!actor || actor.deletedAt) throw new Error('User not found')
+    return setRoleWithActor(ctx, actor, args.targetUserId, args.role)
+  },
+})
+
+async function setRoleWithActor(
+  ctx: MutationCtx,
+  actor: Doc<'users'>,
+  targetUserId: Id<'users'>,
+  role: 'admin' | 'moderator' | 'user',
+) {
+  assertAdmin(actor)
+  const target = await ctx.db.get(targetUserId)
+  if (!target) throw new Error('User not found')
+  const now = Date.now()
+  await ctx.db.patch(targetUserId, { role, updatedAt: now })
+  await ctx.db.insert('auditLogs', {
+    actorUserId: actor._id,
+    action: 'role.change',
+    targetType: 'user',
+    targetId: targetUserId,
+    metadata: { role },
+    createdAt: now,
+  })
+  return { ok: true as const, role }
+}
+
+export const banUser = mutation({
+  args: { userId: v.id('users'), reason: v.optional(v.string()) },
+  handler: async (ctx, args) => {
+    const { user } = await requireUser(ctx)
+    return banUserWithActor(ctx, user, args.userId, args.reason)
+  },
+})
+
+export const banUserInternal = internalMutation({
+  args: {
+    actorUserId: v.id('users'),
+    targetUserId: v.id('users'),
+    reason: v.optional(v.string()),
+  },
+  handler: async (ctx, args) => {
+    const actor = await ctx.db.get(args.actorUserId)
+    if (!actor || actor.deletedAt) throw new Error('User not found')
+    return banUserWithActor(ctx, actor, args.targetUserId, args.reason)
+  },
+})
+
+async function banUserWithActor(
+  ctx: MutationCtx,
+  actor: Doc<'users'>,
+  targetUserId: Id<'users'>,
+  reasonRaw?: string,
+) {
+  assertModerator(actor)
+
+  if (targetUserId === actor._id) throw new Error('Cannot ban yourself')
+
+  const target = await ctx.db.get(targetUserId)
+  if (!target) throw new Error('User not found')
+  if (target.role === 'admin' && actor.role !== 'admin') {
+    throw new Error('Forbidden')
+  }
+
+  const now = Date.now()
+  const reason = reasonRaw?.trim()
+  if (reason && reason.length > 500) {
+    throw new Error('Reason too long (max 500 chars)')
+  }
+  if (target.deletedAt) {
+    return { ok: true as const, alreadyBanned: true, deletedSkills: 0 }
+  }
+
+  const skills = await ctx.db
+    .query('skills')
+    .withIndex('by_owner', (q) => q.eq('ownerUserId', targetUserId))
+    .collect()
+
+  for (const skill of skills) {
+    await ctx.scheduler.runAfter(0, internal.skills.hardDeleteInternal, {
+      skillId: skill._id,
+      actorUserId: actor._id,
+    })
+  }
+
+  const tokens = await ctx.db
+    .query('apiTokens')
+    .withIndex('by_user', (q) => q.eq('userId', targetUserId))
+    .collect()
+  for (const token of tokens) {
+    await ctx.db.patch(token._id, { revokedAt: now })
+  }
+
+  await ctx.db.patch(targetUserId, {
+    deletedAt: now,
+    role: 'user',
+    updatedAt: now,
+    banReason: reason || undefined,
+  })
+
+  await ctx.runMutation(internal.telemetry.clearUserTelemetryInternal, { userId: targetUserId })
+
+  await ctx.db.insert('auditLogs', {
+    actorUserId: actor._id,
+    action: 'user.ban',
+    targetType: 'user',
+    targetId: targetUserId,
+    metadata: { deletedSkills: skills.length, reason: reason || undefined },
+    createdAt: now,
+  })
+
+  return { ok: true as const, alreadyBanned: false, deletedSkills: skills.length }
+}
+
+/**
+ * Auto-ban a user whose skill was flagged malicious by VT.
+ * Skips moderators/admins. No actor required — this is a system-level action.
+ */
+export const autobanMalwareAuthorInternal = internalMutation({
+  args: {
+    ownerUserId: v.id('users'),
+    sha256hash: v.string(),
+    slug: v.string(),
+  },
+  handler: async (ctx, args) => {
+    const target = await ctx.db.get(args.ownerUserId)
+    if (!target) return { ok: false, reason: 'user_not_found' }
+    if (target.deletedAt) return { ok: true, alreadyBanned: true }
+
+    // Never auto-ban moderators or admins
+    if (target.role === 'admin' || target.role === 'moderator') {
+      console.log(`[autoban] Skipping ${target.handle ?? args.ownerUserId}: role=${target.role}`)
+      return { ok: false, reason: 'protected_role' }
+    }
+
+    const now = Date.now()
+
+    // Soft-delete all their skills
+    const skills = await ctx.db
+      .query('skills')
+      .withIndex('by_owner', (q) => q.eq('ownerUserId', args.ownerUserId))
+      .collect()
+
+    for (const skill of skills) {
+      if (!skill.softDeletedAt) {
+        await ctx.db.patch(skill._id, { softDeletedAt: now, updatedAt: now })
+      }
+    }
+
+    // Revoke all API tokens
+    const tokens = await ctx.db
+      .query('apiTokens')
+      .withIndex('by_user', (q) => q.eq('userId', args.ownerUserId))
+      .collect()
+    for (const token of tokens) {
+      if (!token.revokedAt) {
+        await ctx.db.patch(token._id, { revokedAt: now })
+      }
+    }
+
+    // Ban the user
+    await ctx.db.patch(args.ownerUserId, {
+      deletedAt: now,
+      role: 'user',
+      updatedAt: now,
+    })
+
+    await ctx.runMutation(internal.telemetry.clearUserTelemetryInternal, {
+      userId: args.ownerUserId,
+    })
+
+    // Audit log — use the target as actor since there's no human actor
     await ctx.db.insert('auditLogs', {
-      actorUserId: user._id,
-      action: 'role.change',
+      actorUserId: args.ownerUserId,
+      action: 'user.autoban.malware',
       targetType: 'user',
-      targetId: args.userId,
-      metadata: { role: args.role },
-      createdAt: Date.now(),
+      targetId: args.ownerUserId,
+      metadata: {
+        trigger: 'vt.malicious',
+        sha256hash: args.sha256hash,
+        slug: args.slug,
+        deletedSkills: skills.length,
+      },
+      createdAt: now,
     })
+
+    console.warn(
+      `[autoban] Banned ${target.handle ?? args.ownerUserId} — malicious skill: ${args.slug}`,
+    )
+
+    return { ok: true, alreadyBanned: false, deletedSkills: skills.length }
   },
 })

convex/webhooks.ts

@@ -0,0 +1,50 @@
+import { v } from 'convex/values'
+import { internalAction } from './_generated/server'
+import { buildDiscordPayload, getWebhookConfig, shouldSendWebhook } from './lib/webhooks'
+
+export const sendDiscordWebhook = internalAction({
+  args: {
+    event: v.union(v.literal('skill.publish'), v.literal('skill.highlighted')),
+    skill: v.object({
+      slug: v.string(),
+      displayName: v.string(),
+      summary: v.optional(v.string()),
+      version: v.optional(v.string()),
+      ownerHandle: v.optional(v.string()),
+      highlighted: v.optional(v.boolean()),
+      tags: v.optional(v.array(v.string())),
+    }),
+  },
+  handler: async (_ctx, args) => {
+    const config = getWebhookConfig()
+    const logMeta = {
+      event: args.event,
+      slug: args.skill.slug,
+      version: args.skill.version ?? null,
+      highlighted: args.skill.highlighted ?? false,
+      highlightedOnly: config.highlightedOnly,
+    }
+    if (!shouldSendWebhook(args.event, args.skill, config)) {
+      console.info('[webhook] skipped', logMeta)
+      return { ok: false, skipped: true }
+    }
+
+    const payload = buildDiscordPayload(args.event, args.skill, config)
+    const response = await fetch(config.url as string, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify(payload),
+    })
+    if (!response.ok) {
+      const message = await response.text()
+      console.error('[webhook] failed', {
+        ...logMeta,
+        status: response.status,
+        body: message.slice(0, 300),
+      })
+      throw new Error(`Discord webhook failed: ${response.status} ${message}`)
+    }
+    console.info('[webhook] sent', { ...logMeta, status: response.status })
+    return { ok: true }
+  },
+})

docs/README.md

@@ -0,0 +1,33 @@
+---
+summary: 'Documentation index + reading order.'
+read_when:
+  - New contributor onboarding
+  - Looking for the right doc
+---
+
+# Docs
+
+Reading order (new contributor):
+
+1. `README.md` (repo root): run locally.
+2. `docs/quickstart.md`: end-to-end: search → install → publish → sync.
+3. `docs/architecture.md`: how the pieces fit (TanStack Start + Convex + CLI).
+4. `docs/skill-format.md`: what a “skill” is on disk + on the registry.
+5. `docs/cli.md`: CLI reference (flags, config, lockfiles, sync rules).
+6. `docs/http-api.md`: HTTP endpoints used by the CLI + public API.
+7. `docs/auth.md`: GitHub OAuth + API tokens + CLI loopback login.
+8. `docs/deploy.md`: Convex + Vercel deployment + rewrites.
+9. `docs/troubleshooting.md`: common failure modes.
+
+Feature/ops docs (already present):
+
+- `docs/spec.md`: product + implementation spec (data model + flows).
+- `docs/security.md`: moderation, reporting, bans, upload gating.
+- `docs/telemetry.md`: what `clawhub sync` reports; opt-out.
+- `docs/webhook.md`: Discord webhook events/payload.
+- `docs/diffing.md`: version-to-version diff UI spec.
+- `docs/manual-testing.md`: CLI smoke scripts.
+
+Docs tooling:
+
+- `docs/mintlify.md`: publish these docs with Mintlify.

docs/api.md

@@ -0,0 +1,51 @@
+---
+summary: 'Public REST API (v1) overview and conventions.'
+read_when:
+  - Building API clients
+  - Adding endpoints or schemas
+---
+
+# API v1
+
+Base: `https://clawhub.ai`
+
+OpenAPI: `/api/v1/openapi.json`
+
+## Auth
+
+- Public read: no token required.
+- Write + account: `Authorization: Bearer clh_...`.
+
+## Rate limits
+
+Per IP + per API key:
+
+- Read: 120/min per IP, 600/min per key
+- Write: 30/min per IP, 120/min per key
+
+Headers: `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `Retry-After` (on 429).
+
+## Endpoints
+
+Public read:
+
+- `GET /api/v1/search?q=...`
+- `GET /api/v1/skills?limit=&cursor=&sort=`
+  - `sort`: `updated` (default), `downloads`, `stars` (`rating`), `installsCurrent` (`installs`), `installsAllTime`, `trending`
+- `GET /api/v1/skills/{slug}`
+- `GET /api/v1/skills/{slug}/versions?limit=&cursor=`
+- `GET /api/v1/skills/{slug}/versions/{version}`
+- `GET /api/v1/skills/{slug}/file?path=&version=&tag=`
+- `GET /api/v1/resolve?slug=&hash=`
+- `GET /api/v1/download?slug=&version=&tag=`
+
+Auth required:
+
+- `POST /api/v1/skills` (publish, multipart preferred)
+- `DELETE /api/v1/skills/{slug}`
+- `POST /api/v1/skills/{slug}/undelete`
+- `GET /api/v1/whoami`
+
+## Legacy
+
+Legacy `/api/*` and `/api/cli/*` still available. See `DEPRECATIONS.md`.

docs/architecture.md

@@ -0,0 +1,61 @@
+---
+summary: 'System overview: web app + Convex backend + CLI + shared schema.'
+read_when:
+  - Orienting in codebase
+  - Tracing a user flow across layers
+---
+
+# Architecture
+
+## Pieces
+
+- Web app: TanStack Start (React) under `src/`.
+- Backend: Convex under `convex/` (DB, storage, actions, HTTP routes).
+- CLI: `packages/clawdhub/` (published as `clawhub`, legacy `clawdhub`).
+- Shared schemas/routes: `packages/schema/` (`clawhub-schema`).
+
+## Data + storage
+
+- Skill “bundle” = versioned set of text files stored in Convex `_storage`.
+- Metadata extracted from `SKILL.md` frontmatter.
+- Stats stored on `skills` (downloads, installs, stars, comments, …).
+
+## Main flows
+
+### Browse (web)
+
+- UI reads skill metadata + latest version from Convex queries/actions.
+- `SKILL.md` rendered as Markdown.
+
+### Search (HTTP)
+
+- `/api/v1/search?q=...` routes to Convex action for vector search.
+- Embeddings currently generated during publish.
+
+### Install (CLI)
+
+- Resolve latest version via `/api/v1/skills/<slug>`.
+- Download zip via `/api/v1/download?slug=...&version=...`.
+- Extract into `./skills/<slug>` (default).
+- Persist install state:
+  - `./.clawhub/lock.json` (per workdir, legacy `.clawdhub`)
+  - `./skills/<slug>/.clawhub/origin.json` (per skill folder, legacy `.clawdhub`)
+
+### Update (CLI)
+
+- Hash local files, call `/api/v1/resolve?slug=...&hash=<sha256>`.
+- If local matches a known version → use that for “current”.
+- If local doesn’t match:
+  - refuse by default
+  - or overwrite with `--force`
+
+### Publish (CLI)
+
+- Publish via `POST /api/v1/skills` (multipart; requires Bearer token).
+
+### Sync (CLI)
+
+- Scan roots for skill folders (contain `SKILL.md`).
+- Compute fingerprint; compare to registry state.
+- Optionally reports telemetry (see `docs/telemetry.md`).
+- Publishes new/changed skills (skips modified installed skills inside install root).

docs/auth.md

@@ -0,0 +1,54 @@
+---
+summary: 'Auth overview: GitHub OAuth (web) + API tokens (CLI).'
+read_when:
+  - Working on login/token flows
+  - Debugging 401s
+---
+
+# Auth
+
+## Web auth (GitHub OAuth)
+
+- Convex Auth + GitHub OAuth App.
+- Env vars:
+  - `AUTH_GITHUB_ID`
+  - `AUTH_GITHUB_SECRET`
+  - `CONVEX_SITE_URL` (used by auth config)
+
+Local setup steps are in the repo root `README.md`.
+
+## API tokens (CLI)
+
+The CLI uses a long-lived API token (Bearer token) for publish/sync/delete.
+
+### Browser flow (default)
+
+`clawhub login` does:
+
+1. Starts a loopback HTTP server on `127.0.0.1` (random port).
+2. Opens `<site>/cli/auth?redirect_uri=http://127.0.0.1:<port>/callback&state=...`.
+3. Web UI requires GitHub login, then creates a token and redirects back to the loopback server.
+4. CLI stores the token in the global config file.
+
+### Headless flow
+
+Create a token in the web UI (Settings → API tokens) and paste it:
+
+```bash
+clawhub login --token clh_...
+```
+
+### Token storage
+
+Default global config path:
+
+- macOS: `~/Library/Application Support/clawhub/config.json`
+
+Override:
+
+- `CLAWHUB_CONFIG_PATH=/path/to/config.json` (legacy `CLAWDHUB_CONFIG_PATH`)
+
+### Revocation
+
+- Tokens can be revoked in the web UI.
+- Revoked tokens return `401 Unauthorized` on CLI endpoints.

docs/cli.md

@@ -0,0 +1,167 @@
+---
+summary: 'CLI reference: commands, flags, config, lockfile, sync behavior.'
+read_when:
+  - Working on CLI behavior
+  - Debugging install/update/sync
+---
+
+# CLI
+
+CLI package: `packages/clawdhub/` (published as `clawhub`, bin: `clawhub`).
+
+From this repo you can run it via the wrapper script:
+
+```bash
+bun clawhub --help
+```
+
+## Global flags
+
+- `--workdir <dir>`: working directory (default: cwd; falls back to Clawdbot workspace if configured)
+- `--dir <dir>`: install dir under workdir (default: `skills`)
+- `--site <url>`: base URL for browser login (default: `https://clawhub.ai`)
+- `--registry <url>`: API base URL (default: discovered, else `https://clawhub.ai`)
+- `--no-input`: disable prompts
+
+Env equivalents:
+
+- `CLAWHUB_SITE` (legacy `CLAWDHUB_SITE`)
+- `CLAWHUB_REGISTRY` (legacy `CLAWDHUB_REGISTRY`)
+- `CLAWHUB_WORKDIR` (legacy `CLAWDHUB_WORKDIR`)
+
+## Config file
+
+Stores your API token + cached registry URL.
+
+- macOS: `~/Library/Application Support/clawhub/config.json`
+- override: `CLAWHUB_CONFIG_PATH` (legacy `CLAWDHUB_CONFIG_PATH`)
+
+## Commands
+
+### `login` / `auth login`
+
+- Default: opens browser to `<site>/cli/auth` and completes via loopback callback.
+- Headless: `clawhub login --token clh_...`
+
+### `whoami`
+
+- Verifies the stored token via `/api/v1/whoami`.
+
+### `star <slug>` / `unstar <slug>`
+
+- Adds/removes a skill from your highlights.
+- Calls `POST /api/v1/stars/<slug>` and `DELETE /api/v1/stars/<slug>`.
+- `--yes` skips confirmation.
+
+### `search <query...>`
+
+- Calls `/api/v1/search?q=...`.
+
+### `explore`
+
+- Lists latest updated skills via `/api/v1/skills?limit=...` (sorted by `updatedAt` desc).
+- Flags:
+- `--limit <n>` (1-200, default: 25)
+  - `--sort newest|downloads|rating|installs|installsAllTime|trending` (default: newest)
+  - `--json` (machine-readable output)
+- Output: `<slug>  v<version>  <age>  <summary>` (summary truncated to 50 chars).
+
+### `inspect <slug>`
+
+- Fetches skill metadata and version files without installing.
+- `--version <version>`: inspect a specific version (default: latest).
+- `--tag <tag>`: inspect a tagged version (e.g. `latest`).
+- `--versions`: list version history (first page).
+- `--limit <n>`: max versions to list (1-200).
+- `--files`: list files for the selected version.
+- `--file <path>`: fetch raw file content (text files only; 200KB limit).
+- `--json`: machine-readable output.
+
+### `install <slug>`
+
+- Resolves latest version via `/api/v1/skills/<slug>`.
+- Downloads zip via `/api/v1/download`.
+- Extracts into `<workdir>/<dir>/<slug>`.
+- Writes:
+  - `<workdir>/.clawhub/lock.json` (legacy `.clawdhub`)
+  - `<skill>/.clawhub/origin.json` (legacy `.clawdhub`)
+
+### `list`
+
+- Reads `<workdir>/.clawhub/lock.json` (legacy `.clawdhub`).
+
+### `update [slug]` / `update --all`
+
+- Computes fingerprint from local files.
+- If fingerprint matches a known version: no prompt.
+- If fingerprint does not match:
+  - refuses by default
+  - overwrites with `--force` (or prompt, if interactive)
+
+### `publish <path>`
+
+- Publishes via `POST /api/v1/skills` (multipart).
+- Requires semver: `--version 1.2.3`.
+
+### `delete <slug>`
+
+- Soft-delete a skill (moderator/admin only).
+- Calls `DELETE /api/v1/skills/{slug}`.
+- `--yes` skips confirmation.
+
+### `undelete <slug>`
+
+- Restore a hidden skill (moderator/admin only).
+- Calls `POST /api/v1/skills/{slug}/undelete`.
+- `--yes` skips confirmation.
+
+### `hide <slug>`
+
+- Hide a skill (moderator/admin only).
+- Alias for `delete`.
+
+### `unhide <slug>`
+
+- Unhide a skill (moderator/admin only).
+- Alias for `undelete`.
+
+### `ban-user <handleOrId>`
+
+- Ban a user and delete owned skills (moderator/admin only).
+- Calls `POST /api/v1/users/ban`.
+- `--id` treats the argument as a user id instead of a handle.
+- `--fuzzy` resolves the handle via fuzzy user search (admin only).
+- `--reason` records an optional ban reason.
+- `--yes` skips confirmation.
+
+### `set-role <handleOrId> <role>`
+
+- Change a user role (admin only).
+- Calls `POST /api/v1/users/role`.
+- `--id` treats the argument as a user id instead of a handle.
+- `--fuzzy` resolves the handle via fuzzy user search (admin only).
+- `--yes` skips confirmation.
+
+### `sync`
+
+- Scans for local skill folders and publishes new/changed ones.
+- Roots can be any folder: a skills directory or a single skill folder with `SKILL.md`.
+- Auto-adds Clawdbot skill roots when `~/.clawdbot/clawdbot.json` is present:
+  - `agent.workspace/skills` (main agent)
+  - `routing.agents.*.workspace/skills` (per-agent)
+  - `~/.clawdbot/skills` (shared)
+  - `skills.load.extraDirs` (shared packs)
+- Respects `CLAWDBOT_CONFIG_PATH` / `CLAWDBOT_STATE_DIR` and `OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR`.
+- Flags:
+  - `--root <dir...>` extra scan roots
+  - `--all` upload without prompting
+  - `--dry-run` show plan only
+  - `--bump patch|minor|major` (default: patch)
+  - `--changelog <text>` (non-interactive)
+  - `--tags a,b,c` (default: latest)
+  - `--concurrency <n>` (default: 4)
+
+Telemetry:
+
+- Sent during `sync` when logged in, unless `CLAWHUB_DISABLE_TELEMETRY=1` (legacy `CLAWDHUB_DISABLE_TELEMETRY=1`).
+- Details: `docs/telemetry.md`.

docs/deploy.md

@@ -0,0 +1,80 @@
+---
+summary: 'Deploy checklist: Convex backend + Vercel web app + /api rewrites.'
+read_when:
+  - Shipping to production
+  - Debugging /api routing
+---
+
+# Deploy
+
+ClawHub is two deployables:
+
+- Web app (TanStack Start) → typically Vercel.
+- Convex backend → Convex deployment (serves `/api/...` routes).
+
+## 1) Deploy Convex
+
+From your local machine:
+
+```bash
+bunx convex deploy
+```
+
+Ensure Convex env is set (auth + embeddings):
+
+- `AUTH_GITHUB_ID`
+- `AUTH_GITHUB_SECRET`
+- `CONVEX_SITE_URL`
+- `JWT_PRIVATE_KEY`
+- `JWKS`
+- `OPENAI_API_KEY`
+- `SITE_URL` (your web app URL)
+- Optional webhook env (see `docs/webhook.md`)
+- Optional: `GITHUB_TOKEN` (recommended; raises GitHub account lookup limit used by publish gate)
+
+## 2) Deploy web app (Vercel)
+
+Set env vars:
+
+- `VITE_CONVEX_URL`
+- `VITE_CONVEX_SITE_URL` (Convex “site” URL)
+- `CONVEX_SITE_URL` (same value; used by auth provider config)
+- `SITE_URL` (web app URL)
+
+## 3) Route `/api/*` to Convex
+
+This repo currently uses `vercel.json` rewrites:
+
+- `source: /api/:path*`
+- `destination: https://<deployment>.convex.site/api/:path*`
+
+For self-host:
+
+- update `vercel.json` to your deployment’s Convex site URL.
+
+## 4) Registry discovery
+
+The CLI can discover the API base from:
+
+- `/.well-known/clawhub.json` (preferred)
+- `/.well-known/clawdhub.json` (legacy)
+
+If you don’t serve that file, users must set:
+
+```bash
+export CLAWHUB_REGISTRY=https://your-site.example
+```
+
+## 5) Post-deploy checks
+
+```bash
+curl -i "https://<site>/api/v1/search?q=test"
+curl -i "https://<site>/api/v1/skills/gifgrep"
+```
+
+Then:
+
+```bash
+clawhub login --site https://<site>
+clawhub whoami
+```

docs/diffing.md

@@ -0,0 +1,84 @@
+---
+summary: "Skill version diffing mode (Monaco-backed)"
+read_when:
+  - Implementing skill diff UI
+  - Adding version comparisons
+---
+
+# Diffing mode
+
+## Goals
+- Compare any file between two versions.
+- Default compare: `latest` vs `previous` (SemVer precedence).
+- UX feels native to ClawHub (theme + typography + motion).
+- Inline or side-by-side toggle.
+- Public access.
+
+## UX
+- Diff card on skill detail page.
+- Two selectors: Left/Right.
+  - Items: version strings, plus tags (e.g. `latest`), plus `previous`.
+  - Default: Left = `previous`, Right = `latest`.
+- File list with status: added / removed / changed / same.
+  - Default file: `SKILL.md` if present; else first changed file.
+- Toggle: Inline vs Side-by-side.
+- Show size guard message when file > 200KB.
+
+## SemVer ordering
+- Use SemVer precedence to sort versions.
+- `previous` = immediate predecessor of `latest` by SemVer.
+- If `latest` missing or only one version:
+  - Disable `previous` and show empty-state copy.
+
+## Data sources
+- Versions: `api.skills.listVersions` (all, not just latest 10).
+- Tags: `skill.tags` map.
+- File list: `version.files` with `path`, `sha256`, `size`.
+
+## API
+Add action:
+- `skills.getFileText({ versionId, path }) -> { text, size, sha256 }`
+  - Validate version exists + file path exists in version.
+  - Enforce size <= 200KB (both in action and client).
+  - Use `fetchText` from `convex/lib/skillPublish.ts`.
+
+Optional helper action:
+- `skills.getVersionFiles({ versionId }) -> files[]`
+  - If we want lightweight fetch without full version object.
+
+## Client flow
+1. Fetch versions + tags.
+2. Resolve default compare pair:
+   - Right = tag `latest` if present else highest SemVer.
+   - Left = `previous` (SemVer predecessor).
+3. Build file union by path.
+4. For selected file:
+   - Fetch left/right text (guard by size).
+   - Feed into Monaco diff editor.
+
+## Monaco theming
+- Define `clawhub-light` / `clawhub-dark` via `monaco.editor.defineTheme`.
+- Derive colors from CSS variables on `document.documentElement`:
+  - `--surface`, `--surface-muted`, `--ink`, `--ink-soft`, `--line`, `--accent`.
+- Apply theme on load + when theme changes (`data-theme`).
+- Match font: `var(--font-mono)`.
+- Set diff options:
+  - `renderSideBySide` toggle
+  - `diffAlgorithm: 'advanced'`
+  - `renderSideBySideInlineBreakpoint` for mobile
+  - `wordWrap: 'on'`
+
+## Edge cases
+- File removed/added: show empty buffer on missing side + label.
+- Non-text file should not exist (upload rejects), but still guard.
+- Large file: show size warning + disable fetch.
+- Missing version: show error state.
+
+## Perf
+- Cache file text per version+path in client state.
+- Debounce selector changes (100-200ms).
+- Limit concurrent fetches to 2.
+
+## Tests
+- Unit: SemVer ordering + `previous` selection.
+- Component: default selectors, tag inclusion, size guard.

docs/github-import.md

@@ -0,0 +1,171 @@
+---
+summary: 'Feature spec: import a skill from a public GitHub URL (auto-detect SKILL.md, selective file upload, provenance).'
+read_when:
+  - Adding GitHub import (web + API)
+  - Reviewing safety limits (SSRF/zip-bombs)
+  - Implementing provenance + canonical-claim flows
+---
+
+# GitHub import (public repos)
+
+Goal: paste a GitHub URL → auto-detect skill → preview files → publish (selective) → persist provenance.
+
+Non-goal (v1): private repos (no OAuth/PAT support).
+
+Related:
+- `docs/skill-format.md` (what counts as a skill; text-only limits)
+- `docs/api.md` / `docs/http-api.md` (REST patterns + auth)
+
+## UX
+
+Upload page: “Import from GitHub” mode.
+
+Flow:
+1) URL input
+2) Detect skill candidates (SKILL.md)
+3) If multiple candidates: choose one
+4) File picker: check/uncheck; smart-select referenced files
+5) Confirm slug/name/version/tags
+6) Import → publish
+
+## Accepted URLs
+
+Allowlist: `https://github.com/...` only.
+
+Supported shapes:
+- Repo root: `https://github.com/<owner>/<repo>`
+- Tree path: `https://github.com/<owner>/<repo>/tree/<ref>/<path>`
+- Blob path (file): `https://github.com/<owner>/<repo>/blob/<ref>/<path>`
+
+Normalization:
+- Strip query/hash for fetch.
+- From `blob/.../SKILL.md` derive `path` as parent folder.
+- If `ref` missing: use `HEAD`.
+
+Reject:
+- Non-GitHub hosts.
+- Unknown URL patterns.
+- Paths containing `..` after normalization.
+
+## Fetch strategy (public)
+
+Download archive:
+- `https://github.com/<owner>/<repo>/archive/<ref>.zip`
+- Follow redirects. Final redirect usually pins a commit via `codeload.github.com/.../zip/<sha-or-branch>`.
+
+Unzip server-side (Node or Convex node action). Scan for skill candidates.
+
+Skill candidate definition:
+- Any folder containing `SKILL.md` or `skill.md` (also accept `skills.md` for compatibility).
+- Treat repo root as a folder too.
+
+Multiple skills:
+- Return candidate list: `{ path, frontmatter.name, frontmatter.description }`.
+- User chooses one.
+
+## Smart file selection
+
+Defaults:
+- Always select `SKILL.md` (or chosen readme file).
+- Prefer selecting only within chosen skill folder; allow “include out-of-folder refs” if explicitly toggled.
+
+Referenced file expansion:
+- Parse Markdown links/images from selected `.md` files:
+  - `[](<rel>)`, `![](<rel>)`, `<rel>` only when relative.
+  - Ignore `http(s):`, `mailto:`, `#anchors`.
+  - Strip query/hash from relative targets.
+- Resolve against the current file’s directory.
+- Normalize, reject escapes (`..`).
+- Add referenced file if present in archive and is text-allowed.
+- Recurse for newly added `.md` files.
+
+Hard caps:
+- Max recursion depth (e.g. 4).
+- Max referenced additions (e.g. 200).
+
+UI affordances:
+- “Select referenced”
+- “Select all text”
+- “Clear”
+- Search/filter by path
+
+## Publish behavior
+
+Server publishes using existing pipeline:
+- Text-only enforced (see `docs/skill-format.md`).
+- Total ≤ 50MB (selected set).
+- Must include `SKILL.md` (or accepted variant).
+
+Suggested defaults (UI):
+- `displayName`: frontmatter `name` else folder basename → title case.
+- `slug`: sanitize folder basename; if collision, suffix (`-2`, `-3`, …).
+- `version`: if new skill → `0.1.0`; if updating own existing skill → bump patch.
+- `tags`: default `latest`.
+
+## Provenance (persist source)
+
+Persist on each published version (server-side injection; no mutation of imported files):
+- Store in `skillVersions.parsed.metadata.source`:
+
+Example:
+```json
+{
+  "kind": "github",
+  "url": "https://github.com/visionik/ouracli",
+  "repo": "visionik/ouracli",
+  "ref": "HEAD",
+  "commit": "66ac8fb266b7c5ff6519431862be6a375bbfb883",
+  "path": "",
+  "importedAt": 1767930000000
+}
+```
+
+Why `parsed.metadata`:
+- Already optional and stored with each version.
+- No schema churn for v1.
+
+Future: canonical-claim
+- “claim canonical” can key off `{ kind:'github', repo, path }`.
+- Prefer commit-pinned provenance for auditability; allow UI to show “Imported from …”.
+
+## API sketch (internal actions)
+
+Two-step (recommended):
+- `previewGitHubImport(url)` → `{ commit, candidates:[...], files:[...], defaults:{...} }`
+- `importGitHubSkill({ url, commit, candidatePath, selectedPaths, slug, displayName, version, tags })`
+
+Notes:
+- `importGitHubSkill` should re-fetch by pinned `commit` (not floating branch), to avoid TOCTOU.
+- Validate `selectedPaths` subset of fetched archive manifest.
+
+## Security / abuse controls
+
+SSRF:
+- Only `github.com` (+ `codeload.github.com` during redirect follow).
+- No arbitrary redirects to other hosts.
+
+Zip safety:
+- Max compressed bytes (from `Content-Length` if present; else streaming cap).
+- Max uncompressed total bytes.
+- Max file count.
+- Max single file size.
+- Reject symlinks; reject absolute paths; reject `..` segments.
+
+Rate limits:
+- Tie to existing write limits (import == publish).
+- Cache preview results briefly (e.g. 60s) keyed by `{repo, commit}`.
+
+Error UX:
+- “No SKILL.md found.”
+- “Multiple skills found; pick one.”
+- “Repo too large / too many files.”
+- “Selected files exceed 50MB.”
+
+## Manual test checklist
+
+- Repo root skill (`SKILL.md` at root).
+- Nested skill (`skills/foo/SKILL.md`).
+- Multi-skill repo (two SKILL.md).
+- SKILL.md references `docs/usage.md` + images; smart-select picks `.md` and referenced text files; ignores external links.
+- Huge repo → clean “too large” error.
+- Redirect pinning → import stores commit sha in provenance.

docs/http-api.md

@@ -0,0 +1,263 @@
+---
+summary: 'HTTP API reference (public + CLI endpoints + auth).'
+read_when:
+  - Adding/changing endpoints
+  - Debugging CLI ↔ registry requests
+---
+
+# HTTP API
+
+Base URL: `https://clawhub.ai` (default).
+
+All v1 paths are under `/api/v1/...` and implemented by Convex HTTP routes (`convex/http.ts`).
+Legacy `/api/...` and `/api/cli/...` remain for compatibility (see `DEPRECATIONS.md`).
+OpenAPI: `/api/v1/openapi.json`.
+
+## Rate limits
+
+Enforced per IP + per API key:
+
+- Read: 120/min per IP, 600/min per key
+- Write: 30/min per IP, 120/min per key
+
+Headers:
+
+- `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `Retry-After` (when limited)
+
+## Public endpoints (no auth)
+
+### `GET /api/v1/search`
+
+Query params:
+
+- `q` (required): query string
+- `limit` (optional): integer
+- `highlightedOnly` (optional): `true` to filter to highlighted skills
+
+Response:
+
+```json
+{ "results": [{ "score": 0.123, "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "version": "1.2.3", "updatedAt": 1730000000000 }] }
+```
+
+Notes:
+
+- Results are returned in relevance order (embedding similarity + exact slug/name token boosts + popularity prior from downloads).
+
+### `GET /api/v1/skills`
+
+Query params:
+
+- `limit` (optional): integer (1–200)
+- `cursor` (optional): pagination cursor (only for `sort=updated`)
+- `sort` (optional): `updated` (default), `downloads`, `stars` (alias: `rating`), `installsCurrent` (alias: `installs`), `installsAllTime`, `trending`
+
+Notes:
+
+- `trending` ranks by installs in the last 7 days (telemetry-based).
+
+Response:
+
+```json
+{ "items": [{ "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "1.2.3" }, "stats": {}, "createdAt": 0, "updatedAt": 0, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" } }], "nextCursor": null }
+```
+
+### `GET /api/v1/skills/{slug}`
+
+Response:
+
+```json
+{ "skill": { "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "1.2.3" }, "stats": {}, "createdAt": 0, "updatedAt": 0 }, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" }, "owner": { "handle": "steipete", "displayName": "Peter", "image": null } }
+```
+
+### `GET /api/v1/skills/{slug}/versions`
+
+Query params:
+
+- `limit` (optional): integer
+- `cursor` (optional): pagination cursor
+
+### `GET /api/v1/skills/{slug}/versions/{version}`
+
+Returns version metadata + files list.
+
+### `GET /api/v1/skills/{slug}/file`
+
+Returns raw text content.
+
+Query params:
+
+- `path` (required)
+- `version` (optional)
+- `tag` (optional)
+
+Notes:
+
+- Defaults to latest version.
+- File size limit: 200KB.
+
+### `GET /api/v1/resolve`
+
+Used by the CLI to map a local fingerprint to a known version.
+
+Query params:
+
+- `slug` (required)
+- `hash` (required): 64-char hex sha256 of the bundle fingerprint
+
+Response:
+
+```json
+{ "slug": "gifgrep", "match": { "version": "1.2.2" }, "latestVersion": { "version": "1.2.3" } }
+```
+
+### `GET /api/v1/download`
+
+Downloads a zip of a skill version.
+
+Query params:
+
+- `slug` (required)
+- `version` (optional): semver string
+- `tag` (optional): tag name (e.g. `latest`)
+
+Notes:
+
+- If neither `version` nor `tag` is provided, the latest version is used.
+- Soft-deleted versions return `410`.
+
+## Auth endpoints (Bearer token)
+
+All endpoints require:
+
+```
+Authorization: Bearer clh_...
+```
+
+### `GET /api/v1/whoami`
+
+Validates token and returns the user handle.
+
+### `POST /api/v1/skills`
+
+Publishes a new version.
+
+- Preferred: `multipart/form-data` with `payload` JSON + `files[]` blobs.
+- JSON body with `files` (storageId-based) is also accepted.
+
+### `DELETE /api/v1/skills/{slug}` / `POST /api/v1/skills/{slug}/undelete`
+
+Soft-delete / restore a skill (moderator/admin only).
+
+### `POST /api/v1/users/ban`
+
+Ban a user and hard-delete owned skills (moderator/admin only).
+
+Body:
+
+```json
+{ "handle": "user_handle", "reason": "optional ban reason" }
+```
+
+or
+
+```json
+{ "userId": "users_...", "reason": "optional ban reason" }
+```
+
+Response:
+
+```json
+{ "ok": true, "alreadyBanned": false, "deletedSkills": 3 }
+```
+
+### `POST /api/v1/users/role`
+
+Change a user role (admin only).
+
+Body:
+
+```json
+{ "handle": "user_handle", "role": "moderator" }
+```
+
+or
+
+```json
+{ "userId": "users_...", "role": "admin" }
+```
+
+Response:
+
+```json
+{ "ok": true, "role": "moderator" }
+```
+
+### `GET /api/v1/users`
+
+List or search users (admin only).
+
+Query params:
+
+- `q` (optional): search query
+- `query` (optional): alias for `q`
+- `limit` (optional): max results (default 20, max 200)
+
+Response:
+
+```json
+{
+  "items": [
+    {
+      "userId": "users_...",
+      "handle": "user_handle",
+      "displayName": "User",
+      "name": "User",
+      "role": "moderator"
+    }
+  ],
+  "total": 1
+}
+```
+
+### `POST /api/v1/stars/{slug}` / `DELETE /api/v1/stars/{slug}`
+
+Add/remove a star (highlights). Both endpoints are idempotent.
+
+Responses:
+
+```json
+{ "ok": true, "starred": true, "alreadyStarred": false }
+```
+
+```json
+{ "ok": true, "unstarred": true, "alreadyUnstarred": false }
+```
+
+## Legacy CLI endpoints (deprecated)
+
+Still supported for older CLI versions:
+
+- `GET /api/cli/whoami`
+- `POST /api/cli/upload-url`
+- `POST /api/cli/publish`
+- `POST /api/cli/telemetry/sync`
+- `POST /api/cli/skill/delete`
+- `POST /api/cli/skill/undelete`
+
+See `DEPRECATIONS.md` for removal plan.
+
+## Registry discovery (`/.well-known/clawhub.json`)
+
+The CLI can discover registry/auth settings from the site:
+
+- `/.well-known/clawhub.json` (JSON, preferred)
+- `/.well-known/clawdhub.json` (legacy)
+
+Schema:
+
+```json
+{ "apiBase": "https://clawhub.ai", "authBase": "https://clawhub.ai", "minCliVersion": "0.0.5" }
+```
+
+If you self-host, serve this file (or set `CLAWHUB_REGISTRY` explicitly; legacy `CLAWDHUB_REGISTRY`).

docs/manual-testing.md

@@ -1,43 +1,64 @@
+---
+summary: 'Copy/paste CLI smoke checklist for local verification.'
+read_when:
+  - Pre-merge validation
+  - Reproducing a reported CLI bug
+---
+
 # Manual testing (CLI)
 
 ## Setup
-- Ensure logged in: `bun clawdhub whoami` (or `bun clawdhub login`).
+- Ensure logged in: `bun clawhub whoami` (or `bun clawhub login`).
 - Optional: set env
-  - `CLAWDHUB_SITE=https://clawdhub.com`
-  - `CLAWDHUB_REGISTRY=https://clawdhub.com`
+  - `CLAWHUB_SITE=https://clawhub.ai`
+  - `CLAWHUB_REGISTRY=https://clawhub.ai`
 
 ## Smoke
-- `bun clawdhub --help`
-- `bun clawdhub --cli-version`
-- `bun clawdhub whoami`
+- `bun clawhub --help`
+- `bun clawhub --cli-version`
+- `bun clawhub whoami`
 
 ## Search
-- `bun clawdhub search gif --limit 5`
+- `bun clawhub search gif --limit 5`
 
 ## Install / list / update
-- `mkdir -p /tmp/clawdhub-manual && cd /tmp/clawdhub-manual`
-- `bunx clawdhub@beta install gifgrep --force`
-- `bunx clawdhub@beta list`
-- `bunx clawdhub@beta update gifgrep --force`
+- `mkdir -p /tmp/clawhub-manual && cd /tmp/clawhub-manual`
+- `bunx clawhub@beta install gifgrep --force`
+- `bunx clawhub@beta list`
+- `bunx clawhub@beta update gifgrep --force`
 
 ## Publish (changelog optional)
-- `mkdir -p /tmp/clawdhub-skill-demo/SKILL && cd /tmp/clawdhub-skill-demo`
+- `mkdir -p /tmp/clawhub-skill-demo/SKILL && cd /tmp/clawhub-skill-demo`
 - Create files:
   - `SKILL.md`
   - `notes.md`
 - Publish:
-  - `bun clawdhub publish . --slug clawdhub-manual-<ts> --name "Manual <ts>" --version 1.0.0 --tags latest`
+  - `bun clawhub publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.0 --tags latest`
 - Publish update with empty changelog:
-  - `bun clawdhub publish . --slug clawdhub-manual-<ts> --name "Manual <ts>" --version 1.0.1 --tags latest`
+  - `bun clawhub publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.1 --tags latest`
 
 ## Delete / undelete (owner/admin)
-- `bun clawdhub delete clawdhub-manual-<ts> --yes`
+- `bun clawhub delete clawhub-manual-<ts> --yes`
 - Verify hidden:
-  - `curl -i "https://clawdhub.com/api/skill?slug=clawdhub-manual-<ts>"`
+- `curl -i "https://clawhub.ai/api/v1/skills/clawhub-manual-<ts>"`
 - Restore:
-  - `bun clawdhub undelete clawdhub-manual-<ts> --yes`
+  - `bun clawhub undelete clawhub-manual-<ts> --yes`
 - Cleanup:
-  - `bun clawdhub delete clawdhub-manual-<ts> --yes`
+  - `bun clawhub delete clawhub-manual-<ts> --yes`
 
 ## Sync
-- `bun clawdhub sync --dry-run --all`
+- `bun clawhub sync --dry-run --all`
+
+## Playwright (menu smoke)
+
+Run against prod:
+
+```
+PLAYWRIGHT_BASE_URL=https://clawhub.ai bun run test:pw
+```
+
+Run against a local preview server:
+
+```
+bun run test:e2e:local
+```

docs/mintlify.md

@@ -0,0 +1,43 @@
+---
+summary: 'Mintlify setup notes for publishing docs/.'
+read_when:
+  - Setting up docs site
+---
+
+# Mintlify
+
+Goal: publish `docs/` as a browsable docs site (nice UX for OSS users).
+
+This repo does **not** include Mintlify config yet (`mint.json` missing).
+
+## Minimal setup
+
+1) Install Mintlify CLI (per Mintlify docs).
+
+2) Add a `mint.json` at repo root that points to `docs/` pages.
+
+Example (starter):
+
+```json
+{
+  "name": "ClawHub",
+  "logo": "public/logo.svg",
+  "navigation": [
+    { "group": "Start", "pages": ["docs/README", "docs/quickstart"] },
+    { "group": "Concepts", "pages": ["docs/architecture", "docs/skill-format", "docs/telemetry"] },
+    { "group": "Reference", "pages": ["docs/cli", "docs/http-api", "docs/auth", "docs/deploy"] }
+  ]
+}
+```
+
+Notes:
+
+- Mintlify usually wants page paths without extension; keep files as `.md`.
+- If you prefer Mintlify conventions, rename to `.mdx` later (optional).
+
+## Recommended “docs UX” additions
+
+- Add an “Overview” page (use `docs/README.md`).
+- Keep “Quickstart” copy/paste friendly.
+- Provide CLI + HTTP API reference pages (done here).
+- Add a Troubleshooting page for common setup failures.

docs/quickstart.md

@@ -0,0 +1,120 @@
+---
+summary: 'Local setup + CLI smoke: login, search, install, publish, sync.'
+read_when:
+  - First run / local dev setup
+  - Verifying end-to-end flows
+---
+
+# Quickstart
+
+## 0) Prereqs
+
+- Bun
+- Convex CLI (`bunx convex ...`)
+- GitHub OAuth App (for login)
+- OpenAI key (for embeddings/search)
+
+## 1) Local dev (web + Convex)
+
+```bash
+bun install
+cp .env.local.example .env.local
+
+# terminal A
+bun run dev
+
+# terminal B
+bunx convex dev
+```
+
+## 2) Auth setup (GitHub OAuth + Convex Auth keys)
+
+Fill in `.env.local`:
+
+- `AUTH_GITHUB_ID`
+- `AUTH_GITHUB_SECRET`
+- `VITE_CONVEX_URL`
+- `VITE_CONVEX_SITE_URL`
+- `CONVEX_SITE_URL` (same as `VITE_CONVEX_SITE_URL`)
+- `OPENAI_API_KEY`
+
+Generate Convex Auth keys for your deployment:
+
+```bash
+bunx auth --deployment-name <deployment> --web-server-url http://localhost:3000
+```
+
+Then paste the printed `JWT_PRIVATE_KEY` + `JWKS` into `.env.local` (and ensure the deployment got them too).
+
+## 3) CLI: login + basic commands
+
+From this repo:
+
+```bash
+bun clawhub --help
+bun clawhub login
+bun clawhub whoami
+bun clawhub search gif --limit 5
+```
+
+Install a skill into `./skills/<slug>` (if Clawdbot is configured, installs into that workspace instead):
+
+```bash
+bun clawhub install <slug>
+bun clawhub list
+```
+
+You can also install into any folder:
+
+```bash
+bun clawhub install <slug> --workdir /tmp/clawhub-demo --dir skills
+```
+
+Update:
+
+```bash
+bun clawhub update --all
+```
+
+## 4) Publish a skill
+
+Create a folder containing `SKILL.md` (required) plus any supporting text files:
+
+```bash
+mkdir -p /tmp/clawhub-skill-demo && cd /tmp/clawhub-skill-demo
+cat > SKILL.md <<'EOF'
+---
+name: Demo Skill
+description: Demo skill for local testing
+---
+
+# Demo Skill
+
+Hello.
+EOF
+```
+
+Publish:
+
+```bash
+bun clawhub publish . \
+  --slug clawhub-demo-$(date +%s) \
+  --name "Demo $(date +%s)" \
+  --version 1.0.0 \
+  --tags latest \
+  --changelog "Initial release"
+```
+
+## 5) Sync local skills (auto-publish new/changed)
+
+`sync` scans for local skill folders and publishes the ones that aren’t “synced” yet.
+
+```bash
+bun clawhub sync
+```
+
+Dry run + non-interactive:
+
+```bash
+bun clawhub sync --all --dry-run --no-input
+```

docs/security.md

@@ -0,0 +1,55 @@
+---
+summary: 'Security + moderation controls (reports, bans, upload gating).'
+read_when:
+  - Working on moderation or abuse controls
+  - Reviewing upload restrictions
+  - Troubleshooting hidden/removed skills
+---
+
+# Security + Moderation
+
+## Roles + permissions
+
+- user: upload skills/souls (subject to GitHub age gate), report skills.
+- moderator: hide/restore skills, view hidden skills, unhide, soft-delete, ban users (except admins).
+- admin: all moderator actions + hard delete skills, change owners, change roles.
+
+## Reporting + auto-hide
+
+- Reports are unique per user + skill.
+- Report reason required (trimmed, max 500 chars). Abuse of reporting may result in account bans.
+- Per-user cap: 20 **active** reports.
+  - Active = skill exists, not soft-deleted, not `moderationStatus = removed`,
+    and the owner is not banned.
+- Auto-hide: when unique reports exceed 3 (4th report), the skill is:
+  - soft-deleted (`softDeletedAt`)
+  - `moderationStatus = hidden`
+  - `moderationReason = auto.reports`
+  - embeddings visibility set to `deleted`
+  - audit log entry: `skill.auto_hide`
+- Public queries hide non-active moderation statuses; staff can still access via
+  staff-only queries and unhide/restore/delete/ban.
+
+## Bans
+
+- Banning a user:
+  - hard-deletes all owned skills
+  - revokes API tokens
+  - sets `deletedAt` on the user
+- Optional ban reason is stored in `users.banReason` and audit logs.
+- Moderators cannot ban admins; nobody can ban themselves.
+- Report counters effectively reset because deleted/banned skills are no longer
+  considered active in the per-user report cap.
+
+## Upload gate (GitHub account age)
+
+- Skill + soul publish actions require GitHub account age ≥ 7 days.
+- Lookup uses GitHub `created_at` and caches on the user:
+  - `githubCreatedAt` (source of truth)
+  - `githubFetchedAt` (fetch timestamp)
+- Cache TTL: 24 hours.
+- Gate applies to web uploads, CLI publish, and GitHub import.
+- If GitHub responds `403` or `429`, publish fails with:
+  - `GitHub API rate limit exceeded — please try again in a few minutes`
+- To reduce rate-limit failures, set `GITHUB_TOKEN` in Convex env for authenticated
+  GitHub API requests.

docs/skill-format.md

@@ -0,0 +1,151 @@
+---
+summary: 'Skill folder format, required files, allowed file types, limits.'
+read_when:
+  - Publishing skills
+  - Debugging publish/sync failures
+---
+
+# Skill format
+
+## On disk
+
+A skill is a folder.
+
+Required:
+
+- `SKILL.md` (or `skill.md`)
+
+Optional:
+
+- any supporting *text-based* files (see “Allowed files”)
+- `.clawhubignore` (ignore patterns for publish/sync, legacy `.clawdhubignore`)
+- `.gitignore` (also honored)
+
+Local install metadata (written by the CLI):
+
+- `<skill>/.clawhub/origin.json` (legacy `.clawdhub`)
+
+Workdir install state (written by the CLI):
+
+- `<workdir>/.clawhub/lock.json` (legacy `.clawdhub`)
+
+## `SKILL.md`
+
+- Markdown with optional YAML frontmatter.
+- The server extracts metadata from frontmatter during publish.
+- `description` is used as the skill summary in the UI/search.
+
+## Frontmatter metadata
+
+Skill metadata is declared in the YAML frontmatter at the top of your `SKILL.md`. This tells the registry (and security analysis) what your skill needs to run.
+
+### Basic frontmatter
+
+```yaml
+---
+name: my-skill
+description: Short summary of what this skill does.
+version: 1.0.0
+---
+```
+
+### Runtime metadata (`metadata.openclaw`)
+
+Declare your skill's runtime requirements under `metadata.openclaw` (aliases: `metadata.clawdbot`, `metadata.clawdis`).
+
+```yaml
+---
+name: my-skill
+description: Manage tasks via the Todoist API.
+metadata:
+  openclaw:
+    requires:
+      env:
+        - TODOIST_API_KEY
+      bins:
+        - curl
+    primaryEnv: TODOIST_API_KEY
+---
+```
+
+### Full field reference
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `requires.env` | `string[]` | Environment variables your skill expects. |
+| `requires.bins` | `string[]` | CLI binaries that must all be installed. |
+| `requires.anyBins` | `string[]` | CLI binaries where at least one must exist. |
+| `requires.config` | `string[]` | Config file paths your skill reads. |
+| `primaryEnv` | `string` | The main credential env var for your skill. |
+| `always` | `boolean` | If `true`, skill is always active (no explicit install needed). |
+| `skillKey` | `string` | Override the skill's invocation key. |
+| `emoji` | `string` | Display emoji for the skill. |
+| `homepage` | `string` | URL to the skill's homepage or docs. |
+| `os` | `string[]` | OS restrictions (e.g. `["macos"]`, `["linux"]`). |
+| `install` | `array` | Install specs for dependencies (see below). |
+| `nix` | `object` | Nix plugin spec (see README). |
+| `config` | `object` | Clawdbot config spec (see README). |
+
+### Install specs
+
+If your skill needs dependencies installed, declare them in the `install` array:
+
+```yaml
+metadata:
+  openclaw:
+    install:
+      - kind: brew
+        formula: jq
+        bins: [jq]
+      - kind: node
+        package: typescript
+        bins: [tsc]
+```
+
+Supported install kinds: `brew`, `node`, `go`, `uv`.
+
+### Why this matters
+
+ClawHub's security analysis checks that what your skill declares matches what it actually does. If your code references `TODOIST_API_KEY` but your frontmatter doesn't declare it under `requires.env`, the analysis will flag a metadata mismatch. Keeping declarations accurate helps your skill pass review and helps users understand what they're installing.
+
+### Example: complete frontmatter
+
+```yaml
+---
+name: todoist-cli
+description: Manage Todoist tasks, projects, and labels from the command line.
+version: 1.2.0
+metadata:
+  openclaw:
+    requires:
+      env:
+        - TODOIST_API_KEY
+      bins:
+        - curl
+    primaryEnv: TODOIST_API_KEY
+    emoji: "\u2705"
+    homepage: https://github.com/example/todoist-cli
+---
+```
+
+## Allowed files
+
+Only “text-based” files are accepted by publish.
+
+- Extension allowlist is in `packages/schema/src/textFiles.ts` (`TEXT_FILE_EXTENSIONS`).
+- Content types starting with `text/` are treated as text; plus a small allowlist (JSON/YAML/TOML/JS/TS/Markdown/SVG).
+
+Limits (server-side):
+
+- Total bundle size: 50MB.
+- Embedding text includes `SKILL.md` + up to ~40 non-`.md` files (best-effort cap).
+
+## Slugs
+
+- Derived from folder name by default.
+- Must be lowercase and URL-safe: `^[a-z0-9][a-z0-9-]*$`.
+
+## Versioning + tags
+
+- Each publish creates a new version (semver).
+- Tags are string pointers to a version; `latest` is commonly used.