Compare commits

...
50 Commits
Author SHA1 Message Date
Patrick Erichsen 821494b9dd fix(cli): accept legacy verify json flag
Accept legacy skill verify --json usage as a hidden compatibility no-op, add a regression e2e for the flattened verifier response, and prepare clawhub@0.19.2.
2026-06-05 16:30:19 -07:00
Patrick Erichsen 94413a60fb feat: show plugin downloads in lists
Show plugin download counts in lists, add downloads sorting, and remove plugin family badges from plugin list/grid rows.
2026-06-05 16:18:00 -07:00
Patrick Erichsen 9cbf98297c fix: exclude official orgs from publisher abuse (#2527) 2026-06-05 15:59:10 -07:00
Patrick Erichsen 94ded18dec feat: add org deletion cascade
Closes #2516.
2026-06-05 14:52:31 -07:00
Patrick Erichsen 9897850074 fix: add targeted legacy publisher ownership repair
Adds a targeted legacy owner-publisher projection repair mutation for one user and covers the repair path in maintenance tests.
2026-06-05 14:24:29 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 159118dc59 build(deps): bump the github-actions group with 3 updates (#2473)
Bumps the github-actions group with 3 updates: [useblacksmith/begin-testbox](https://github.com/useblacksmith/begin-testbox), [github/codeql-action](https://github.com/github/codeql-action) and [trufflesecurity/trufflehog](https://github.com/trufflesecurity/trufflehog).


Updates `useblacksmith/begin-testbox` from d0e04585c26905fdd92c94a09c159544c7ee1b67 to 233448af4bfdc6fca509a7f0974411ac6d8a8043
- [Commits](https://github.com/useblacksmith/begin-testbox/compare/d0e04585c26905fdd92c94a09c159544c7ee1b67...233448af4bfdc6fca509a7f0974411ac6d8a8043)

Updates `github/codeql-action` from 4.36.0 to 4.36.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/7211b7c8077ea37d8641b6271f6a365a22a5fbfa...87557b9c84dde89fdd9b10e88954ac2f4248e463)

Updates `trufflesecurity/trufflehog` from 3.95.3 to 3.95.5
- [Release notes](https://github.com/trufflesecurity/trufflehog/releases)
- [Commits](https://github.com/trufflesecurity/trufflehog/compare/v3.95.3...v3.95.5)

---
updated-dependencies:
- dependency-name: useblacksmith/begin-testbox
  dependency-version: 233448af4bfdc6fca509a7f0974411ac6d8a8043
  dependency-type: direct:production
  dependency-group: github-actions
- dependency-name: github/codeql-action
  dependency-version: 4.36.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: trufflesecurity/trufflehog
  dependency-version: 3.95.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-05 14:11:39 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 843f19e640 build(deps): bump commander from 14.0.3 to 15.0.0 (#2476)
Bumps [commander](https://github.com/tj/commander.js) from 14.0.3 to 15.0.0.
- [Release notes](https://github.com/tj/commander.js/releases)
- [Changelog](https://github.com/tj/commander.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tj/commander.js/compare/v14.0.3...v15.0.0)

---
updated-dependencies:
- dependency-name: commander
  dependency-version: 15.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-05 14:07:07 -07:00
dependabot[bot]dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Peter Steinberger
4b882b2f43 build(deps): bump the production-minor-and-patch group across 1 directory with 12 updates (#2510)
* build(deps): bump the production-minor-and-patch group across 1 directory with 12 updates

Bumps the production-minor-and-patch group with 12 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [@convex-dev/auth](https://github.com/get-convex/convex-auth) | `0.0.92` | `0.0.93` |
| [@shikijs/rehype](https://github.com/shikijs/shiki/tree/HEAD/packages/rehype) | `4.1.0` | `4.2.0` |
| [@tanstack/react-router](https://github.com/TanStack/router/tree/HEAD/packages/react-router) | `1.170.8` | `1.170.12` |
| [@tanstack/react-start](https://github.com/TanStack/router/tree/HEAD/packages/react-start) | `1.168.13` | `1.168.21` |
| [convex](https://github.com/get-convex/convex-backend/tree/HEAD/npm-packages/convex) | `1.39.1` | `1.40.0` |
| [lucide-react](https://github.com/lucide-icons/lucide/tree/HEAD/packages/lucide-react) | `1.16.0` | `1.17.0` |
| [react](https://github.com/facebook/react/tree/HEAD/packages/react) | `19.2.6` | `19.2.7` |
| [react-dom](https://github.com/facebook/react/tree/HEAD/packages/react-dom) | `19.2.6` | `19.2.7` |
| [semver](https://github.com/npm/node-semver) | `7.8.1` | `7.8.2` |
| [shiki](https://github.com/shikijs/shiki/tree/HEAD/packages/shiki) | `4.1.0` | `4.2.0` |
| [undici](https://github.com/nodejs/undici) | `7.26.0` | `7.27.1` |
| [@clack/prompts](https://github.com/bombshell-dev/clack/tree/HEAD/packages/prompts) | `1.4.0` | `1.5.1` |



Updates `@convex-dev/auth` from 0.0.92 to 0.0.93
- [Changelog](https://github.com/get-convex/convex-auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/get-convex/convex-auth/compare/v0.0.92...v0.0.93)

Updates `@shikijs/rehype` from 4.1.0 to 4.2.0
- [Release notes](https://github.com/shikijs/shiki/releases)
- [Commits](https://github.com/shikijs/shiki/commits/v4.2.0/packages/rehype)

Updates `@tanstack/react-router` from 1.170.8 to 1.170.12
- [Release notes](https://github.com/TanStack/router/releases)
- [Changelog](https://github.com/TanStack/router/blob/main/packages/react-router/CHANGELOG.md)
- [Commits](https://github.com/TanStack/router/commits/@tanstack/react-router@1.170.12/packages/react-router)

Updates `@tanstack/react-start` from 1.168.13 to 1.168.21
- [Release notes](https://github.com/TanStack/router/releases)
- [Changelog](https://github.com/TanStack/router/blob/main/packages/react-start/CHANGELOG.md)
- [Commits](https://github.com/TanStack/router/commits/@tanstack/react-start@1.168.21/packages/react-start)

Updates `convex` from 1.39.1 to 1.40.0
- [Release notes](https://github.com/get-convex/convex-backend/releases)
- [Changelog](https://github.com/get-convex/convex-backend/blob/main/npm-packages/convex/CHANGELOG.md)
- [Commits](https://github.com/get-convex/convex-backend/commits/HEAD/npm-packages/convex)

Updates `lucide-react` from 1.16.0 to 1.17.0
- [Release notes](https://github.com/lucide-icons/lucide/releases)
- [Commits](https://github.com/lucide-icons/lucide/commits/1.17.0/packages/lucide-react)

Updates `react` from 19.2.6 to 19.2.7
- [Release notes](https://github.com/facebook/react/releases)
- [Changelog](https://github.com/facebook/react/blob/main/CHANGELOG.md)
- [Commits](https://github.com/facebook/react/commits/v19.2.7/packages/react)

Updates `react-dom` from 19.2.6 to 19.2.7
- [Release notes](https://github.com/facebook/react/releases)
- [Changelog](https://github.com/facebook/react/blob/main/CHANGELOG.md)
- [Commits](https://github.com/facebook/react/commits/v19.2.7/packages/react-dom)

Updates `semver` from 7.8.1 to 7.8.2
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v7.8.1...v7.8.2)

Updates `shiki` from 4.1.0 to 4.2.0
- [Release notes](https://github.com/shikijs/shiki/releases)
- [Commits](https://github.com/shikijs/shiki/commits/v4.2.0/packages/shiki)

Updates `undici` from 7.26.0 to 7.27.1
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v7.26.0...v7.27.1)

Updates `@clack/prompts` from 1.4.0 to 1.5.1
- [Release notes](https://github.com/bombshell-dev/clack/releases)
- [Changelog](https://github.com/bombshell-dev/clack/blob/main/packages/prompts/CHANGELOG.md)
- [Commits](https://github.com/bombshell-dev/clack/commits/@clack/prompts@1.5.1/packages/prompts)

---
updated-dependencies:
- dependency-name: "@clack/prompts"
  dependency-version: 1.5.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: production-minor-and-patch
- dependency-name: "@convex-dev/auth"
  dependency-version: 0.0.93
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: production-minor-and-patch
- dependency-name: "@shikijs/rehype"
  dependency-version: 4.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: production-minor-and-patch
- dependency-name: "@tanstack/react-router"
  dependency-version: 1.170.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: production-minor-and-patch
- dependency-name: "@tanstack/react-start"
  dependency-version: 1.168.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: production-minor-and-patch
- dependency-name: convex
  dependency-version: 1.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: production-minor-and-patch
- dependency-name: lucide-react
  dependency-version: 1.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: production-minor-and-patch
- dependency-name: react
  dependency-version: 19.2.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: production-minor-and-patch
- dependency-name: react-dom
  dependency-version: 19.2.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: production-minor-and-patch
- dependency-name: semver
  dependency-version: 7.8.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: production-minor-and-patch
- dependency-name: shiki
  dependency-version: 4.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: production-minor-and-patch
- dependency-name: undici
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: production-minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* ci: update convex local auth runner

* ci: keep local auth deployment marker prefixed

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-05 13:50:43 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 634667c2c8 build(deps-dev): bump the development-minor-and-patch group across 1 directory with 5 updates (#2475)
Bumps the development-minor-and-patch group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) | `4.1.7` | `4.1.8` |
| [oxfmt](https://github.com/oxc-project/oxc/tree/HEAD/npm/oxfmt) | `0.51.0` | `0.53.0` |
| [oxlint](https://github.com/oxc-project/oxc/tree/HEAD/npm/oxlint) | `1.66.0` | `1.68.0` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.0.14` | `8.0.16` |
| [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) | `4.1.7` | `4.1.8` |



Updates `@vitest/coverage-v8` from 4.1.7 to 4.1.8
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.8/packages/coverage-v8)

Updates `oxfmt` from 0.51.0 to 0.53.0
- [Release notes](https://github.com/oxc-project/oxc/releases)
- [Changelog](https://github.com/oxc-project/oxc/blob/main/npm/oxfmt/CHANGELOG.md)
- [Commits](https://github.com/oxc-project/oxc/commits/oxfmt_v0.53.0/npm/oxfmt)

Updates `oxlint` from 1.66.0 to 1.68.0
- [Release notes](https://github.com/oxc-project/oxc/releases)
- [Changelog](https://github.com/oxc-project/oxc/blob/main/npm/oxlint/CHANGELOG.md)
- [Commits](https://github.com/oxc-project/oxc/commits/oxlint_v1.68.0/npm/oxlint)

Updates `vite` from 8.0.14 to 8.0.16
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.16/packages/vite)

Updates `vitest` from 4.1.7 to 4.1.8
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.8/packages/vitest)

---
updated-dependencies:
- dependency-name: "@vitest/coverage-v8"
  dependency-version: 4.1.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: development-minor-and-patch
- dependency-name: oxfmt
  dependency-version: 0.53.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: development-minor-and-patch
- dependency-name: oxlint
  dependency-version: 1.68.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: development-minor-and-patch
- dependency-name: vite
  dependency-version: 8.0.16
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: development-minor-and-patch
- dependency-name: vitest
  dependency-version: 4.1.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: development-minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-05 13:26:14 -07:00
Patrick Erichsen bd30b182d7 feat: show skill scan queue progress 2026-06-05 11:35:08 -07:00
Patrick Erichsen 2d719feef5 fix: sync github status through skill digests 2026-06-04 21:02:54 -07:00
Patrick Erichsen d3e1059266 fix: remove org badge from publisher profiles 2026-06-04 21:02:54 -07:00
Patrick Erichsen 1a968d8243 fix: clean up publisher catalog rows (#2513)
* fix: clean up publisher catalog rows

* fix: type github sync soft deletes
2026-06-04 20:21:59 -07:00
Patrick Erichsen 59667cc1a5 fix: keep pending github skills visible (#2514) 2026-06-04 20:20:13 -07:00
Patrick Erichsen 5b0dd96530 fix: soft delete removed github skills (#2512) 2026-06-04 19:46:57 -07:00
Patrick Erichsen 8e3858a31d chore: release clawhub cli 0.19.1
Release clawhub CLI 0.19.1 so npm includes GitHub-backed skill install support.
2026-06-04 19:26:34 -07:00
Patrick Erichsen fe4acc8e10 feat: surface github sync issues (#2509) 2026-06-04 19:15:39 -07:00
Patrick Erichsen f2e6e87756 test: gate public routes in ci smoke (#2508) 2026-06-04 19:09:39 -07:00
Patrick Erichsen 3de69919de Add official org selector to GitHub sync settings (#2505)
* feat: select official org for github sync sources

* fix: rename github sync publisher field
2026-06-04 19:08:23 -07:00
Patrick Erichsen 7629433248 fix: skip publisher handle conflicts in ownership repair (#2507) 2026-06-04 19:06:38 -07:00
Patrick Erichsen b82ad43eae fix: align org publisher catalog tabs (#2506) 2026-06-04 19:05:33 -07:00
Patrick Erichsen b7ee5c8e62 fix: backfill legacy publisher ownership (#2503)
* fix: backfill legacy publisher ownership

* test: update publisher official mocks
2026-06-04 18:57:42 -07:00
Patrick Erichsen 1c8543ade6 feat: add official org mod commands (#2504) 2026-06-04 18:47:12 -07:00
Patrick Erichsen 7cd0ef362d fix: select publisher previews by installs (#2502) 2026-06-04 18:39:47 -07:00
Patrick Erichsen 9cf9e12bdd GitHub-backed skill source sync (#2498)
* feat: add github-backed skill seed schema

* feat: render github-backed skill manifests

* feat: add github-backed skill install resolver

* feat: add github-backed skill source sync

* fix: handle github resolver blocks

* fix: harden github source sync

* fix: expose github install blocks

* feat: support GitHub-backed skill sync

* fix: handle blocked github skill content

* fix: harden github sync rollout

* fix: bound github sync payloads

* feat: allow forcing pending github skill installs

* fix: remove official publisher seed rollout

* chore: format clawhub e2e test
2026-06-04 18:20:43 -07:00
Patrick Erichsen 112e25e28c fix: retry incomplete github sign-ins (#2501) 2026-06-04 17:18:32 -07:00
Jesse Merhi d786374b77 feat: add shared download metering (#2485) 2026-06-04 16:48:08 -07:00
Jesse Merhi 3fbe27560e feat: add publisher abuse review dashboard (#2471) 2026-06-05 09:05:50 +10:00
Patrick Erichsen 33539b6df3 fix: point account recovery to github issues (#2499) 2026-06-04 14:41:52 -07:00
clawsweeper[bot]clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>momothemage
354f12a033 fix(httpApiV1): surface 400-class delete validation errors instead of opaque 500 (#2496)
Summary:
- The branch narrows the soft-delete bad-request substring whitelist from any `reserved` message to the ClawHu ... rvation phrase and adds regression tests for the intended 400 path and an unrelated reserved-word 500 path.
- Reproducibility: yes. by source inspection: current main maps any cleaned soft-delete error containing `rese ...  while preserving the package route-reservation case. I did not execute tests during this read-only review.

Automerge notes:
- No ClawSweeper repair was needed after automerge opt-in.

Validation:
- ClawSweeper review passed for head a58687c54e.
- Required merge gates passed before the squash merge.

Prepared head SHA: a58687c54e
Review: https://github.com/openclaw/clawhub/pull/2496#issuecomment-4620039541

Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: momothemage
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
2026-06-04 07:43:22 +00:00
Momo b0ea80df6c fix(httpApiV1): surface 400-class delete validation errors instead of opaque 500 (#2488)
Merged via squash.

Prepared head SHA: cc6ac9d31a
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-06-04 15:32:30 +08:00
Patrick Erichsen 909a47106e chore: bump clawhub cli to 0.19.0 (#2486) 2026-06-03 16:19:42 -07:00
Patrick Erichsen e8cfbddf17 feat: add clawhub scan command (#2479)
* docs: design clawhub scan command

* docs: plan clawhub scan command

* feat: add clawhub scan command

* fix: prune ephemeral scan uploads

* fix: avoid scan route slug collision
2026-06-03 16:05:11 -07:00
Patrick Erichsen 162528abe4 feat: show plugin download counts (#2484) 2026-06-03 14:46:20 -07:00
Vyctor H. Brzezowski 74aa61086e fix: gate local Codex workers (#2472) 2026-06-03 12:26:20 -07:00
Momo 858a121d33 fix(plugins): rewrite relative README image URLs to source-host raw URLs (#2412)
Merged via squash.

Prepared head SHA: 545db16f01
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-06-03 17:05:58 +08:00
Patrick Erichsen 953358a322 fix: use GitHub App auth for GitHub account lookups
* fix: use GitHub App auth for GitHub account lookups

* style: format GitHub account age backfill
2026-06-03 00:24:23 -07:00
Patrick Erichsen 0a79612fe5 feat: add security audit download export (#2477) 2026-06-02 22:20:05 -07:00
Peter Steinberger 9b5d2e088d fix: speed GitHub account-age backfill scan 2026-06-02 13:01:59 +01:00
Peter Steinberger ce62df9d08 fix: batch GitHub account-age backfill writes 2026-06-02 11:52:21 +01:00
Peter Steinberger 0abdbf4a50 fix: harden GitHub account age lookup 2026-06-02 11:32:16 +01:00
Jesse Merhi dcbc38999f fix: require multipart package publishes
Require multipart package publishes so public clients cannot provide trusted file or artifact metadata.

Preserve released CLI multipart field names (`files` and `clawpack`), derive stored file and npm-pack artifact metadata from uploaded bytes, and keep large ClawPack tarballs working through ticketed staged uploads.

Tests:
- bunx vitest run convex/uploads.test.ts --reporter=dot
- bunx vitest run convex/httpApi.handlers.test.ts convex/httpApiV1.handlers.test.ts --testNamePattern "package publish|multipart package publish|multipart ClawPack|staged ClawPack|browser session auth|cliUploadUrl|upload tickets" --reporter=dot
- bun run --cwd packages/clawhub test:src -- src/cli/commands/packages.test.ts --testNamePattern "ClawPack tarballs over|stages ClawPack|publishes a ClawPack|cleans generated ClawPack"
- bun run format:check
- bun run lint
- bunx tsc -p packages/schema/tsconfig.json --noEmit --pretty false
- bunx tsc -p packages/clawhub/tsconfig.json --noEmit --pretty false
- bunx tsc --noEmit --pretty false
- bun run ci:unit
- git diff --check origin/main
2026-06-02 09:59:53 +10:00
Jesse Merhi 01aa28ccda feat: make recommended the default skill ranking
Make Recommended the default public skill ranking while preserving the v1 API no-sort default. Adds recommended/default API support, digest rank indexes/backfill safety, OpenAPI/docs updates, and regression coverage.
2026-06-02 08:20:37 +10:00
Jesse Merhi cb6ced7906 fix: keep skill card metadata grouped (#2465)
Move skill card tag/platform pills below the description and keep author/update/stat metadata grouped at the bottom of grid cards.

Thanks @jesse-merhi.
2026-06-02 05:02:40 +10:00
Vyctor H. Brzezowski ded9ff4235 fix: stabilize auth loading states (#2447) 2026-06-01 10:01:04 -07:00
Val AlexanderandBunsDev 9fc2da4dc4 fix: clarify paginated plugin count (#2393)
* fix: clarify paginated plugin count

* fix: correct shown-count pluralization

Agent-Logs-Url: https://github.com/openclaw/clawhub/sessions/32575bd4-9dc7-4b8f-9536-bdc96f008783

Co-authored-by: BunsDev <68980965+BunsDev@users.noreply.github.com>
2026-05-31 01:00:02 -05:00
Patrick Erichsen 05d5fc1151 docs: explain skill catalog publishing (#2455)
* docs: explain skill publishing workflow

* docs: polish ClawHub publishing guide
2026-05-30 22:14:59 -05:00
Patrick Erichsen 9aaab158cb feat(cli): print stored auth token (#2454) 2026-05-30 21:11:51 -05:00
Patrick Erichsen 6fc5bb7cd8 feat: add reusable skill publish workflow (#2450)
* feat: add reusable skill publish workflow

* test: cover exact skill sync roots

* fix: keep skill workflow scans exact
2026-05-30 20:26:51 -05:00
Patrick Erichsen 9aa3f37ee1 feat: export skill bundle content for security datasets (#2448) 2026-05-30 12:11:19 -05:00
278 changed files with 37023 additions and 2827 deletions
+6
View File
@@ -16,5 +16,11 @@ AUTH_GITHUB_SECRET=
JWT_PRIVATE_KEY=
JWKS=
# Local dev personas
DEV_AUTH_ENABLED=
DEV_AUTH_CONVEX_DEPLOYMENT=
DEV_AUTH_SITE_URL=
DEV_AUTH_SECRET=
# Embeddings
OPENAI_API_KEY=
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Begin Testbox
uses: useblacksmith/begin-testbox@d0e04585c26905fdd92c94a09c159544c7ee1b67
uses: useblacksmith/begin-testbox@233448af4bfdc6fca509a7f0974411ac6d8a8043
with:
testbox_id: ${{ inputs.testbox_id }}
+2 -2
View File
@@ -88,13 +88,13 @@ jobs:
- name: Initialize CodeQL
if: ${{ github.event_name != 'workflow_dispatch' || inputs.profile == 'all' || inputs.profile == matrix.category }}
uses: github/codeql-action/init@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4
uses: github/codeql-action/init@87557b9c84dde89fdd9b10e88954ac2f4248e463 # v4
with:
languages: ${{ matrix.language }}
config-file: ${{ matrix.config_file }}
- name: Analyze
if: ${{ github.event_name != 'workflow_dispatch' || inputs.profile == 'all' || inputs.profile == matrix.category }}
uses: github/codeql-action/analyze@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4
uses: github/codeql-action/analyze@87557b9c84dde89fdd9b10e88954ac2f4248e463 # v4
with:
category: "/codeql-light/${{ matrix.category }}"
@@ -0,0 +1,84 @@
name: ClawHub Scheduled Live Checks
on:
schedule:
- cron: "17 5 * * *"
workflow_dispatch:
inputs:
github-repo:
description: GitHub skills repo to use for the source-backed canary
required: false
default: openclaw/agent-skills
github-skill:
description: Skill slug to verify from the GitHub skills repo
required: false
default: handoff
concurrency:
group: clawhub-scheduled-live-checks-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
env:
VITE_CONVEX_URL: https://example.invalid
jobs:
github-backed-skills:
name: GitHub-backed skills canary
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/setup-bun
- name: Run GitHub-backed skills live canary
env:
CLAWHUB_LIVE_GITHUB_CANARY: "1"
CLAWHUB_LIVE_GITHUB_REPO: ${{ inputs.github-repo || 'openclaw/agent-skills' }}
CLAWHUB_LIVE_GITHUB_SKILL: ${{ inputs.github-skill || 'handoff' }}
GITHUB_TOKEN: ${{ github.token }}
run: bunx vitest run convex/githubSkillSync.live.test.ts
open-failure-issue:
name: Open failure issue
needs: github-backed-skills
if: ${{ always() && needs.github-backed-skills.result == 'failure' }}
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Open or update failure issue
env:
GH_TOKEN: ${{ github.token }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
WORKFLOW_NAME: ${{ github.workflow }}
run: |
set -euo pipefail
marker_token="clawhub-scheduled-live-checks-failure"
marker="<!-- $marker_token -->"
title="ClawHub scheduled live checks failing"
issue_number="$(gh issue list \
--repo "$GITHUB_REPOSITORY" \
--state open \
--search "$marker_token in:body" \
--json number \
--jq '.[0].number // empty')"
body_file="$(mktemp)"
cat > "$body_file" <<EOF
$marker
The scheduled ClawHub live checks failed.
Workflow: $WORKFLOW_NAME
Run: $RUN_URL
EOF
if [[ -n "$issue_number" ]]; then
gh issue comment "$issue_number" --repo "$GITHUB_REPOSITORY" --body-file "$body_file"
else
gh issue create --repo "$GITHUB_REPOSITORY" --title "$title" --body-file "$body_file"
fi
+1 -1
View File
@@ -51,7 +51,7 @@ jobs:
id: trufflehog
# Use a concrete released ref that resolves in upstream action registry.
# v3 (major tag) is not published by trufflesecurity/trufflehog.
uses: trufflesecurity/trufflehog@v3.95.3
uses: trufflesecurity/trufflehog@v3.95.5
with:
path: ./
base: ${{ steps.scan_range.outputs.base }}
+292
View File
@@ -0,0 +1,292 @@
name: Skill Publish
on:
workflow_call:
inputs:
skill_path:
description: Optional path to one skill folder. When set, only this skill is processed.
required: false
type: string
default: ""
root:
description: Directory containing skill folders for bulk catalog publishing.
required: false
type: string
default: skills
dry_run:
description: Preview only. When true, no publish mutation is performed.
required: false
type: boolean
default: true
owner:
description: Optional owner/publisher handle for org publishing.
required: false
type: string
default: ""
tags:
description: Optional comma-separated tags override.
required: false
type: string
default: latest
bump:
description: Version bump for updated skills. One of patch, minor, or major.
required: false
type: string
default: patch
registry:
description: ClawHub registry URL.
required: false
type: string
default: https://clawhub.ai
site:
description: ClawHub site URL.
required: false
type: string
default: https://clawhub.ai
ref:
description: Optional caller repository ref to check out.
required: false
type: string
default: ""
secrets:
clawhub_token:
required: false
outputs:
publish_json:
description: Structured JSON output from clawhub sync.
value: ${{ jobs.publish.outputs.publish_json }}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
permissions: {}
jobs:
publish:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
id-token: write
outputs:
publish_json: ${{ steps.capture.outputs.publish_json }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.sha }}
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: 1.3.10
- name: Resolve ClawHub workflow source
id: clawhub_source
run: |
python3 - <<'PY'
import base64
import json
import os
from pathlib import Path
from urllib.request import Request, urlopen
request_token = os.environ.get("ACTIONS_ID_TOKEN_REQUEST_TOKEN", "").strip()
request_url = os.environ.get("ACTIONS_ID_TOKEN_REQUEST_URL", "").strip()
if not request_token or not request_url:
raise SystemExit("GitHub OIDC token request env vars are missing; id-token: write is required.")
audience = "clawhub-workflow-source"
joiner = "&" if "?" in request_url else "?"
token_url = f"{request_url}{joiner}audience={audience}"
request = Request(token_url, headers={"Authorization": f"Bearer {request_token}"})
with urlopen(request) as response:
payload = json.load(response)
token = str(payload.get("value", "")).strip()
if not token:
raise SystemExit("GitHub OIDC token response did not include a token value.")
try:
encoded_payload = token.split(".")[1]
except IndexError as exc:
raise SystemExit("GitHub OIDC token was not a valid JWT.") from exc
padding = "=" * (-len(encoded_payload) % 4)
claims = json.loads(base64.urlsafe_b64decode(encoded_payload + padding).decode("utf-8"))
workflow_ref = str(claims.get("job_workflow_ref", "")).strip()
workflow_sha = str(claims.get("job_workflow_sha", "")).strip()
repo, marker, _ = workflow_ref.partition("/.github/workflows/")
if not marker or not repo or not workflow_sha:
raise SystemExit(
"Unable to resolve reusable workflow source from GitHub OIDC claims: "
f"job_workflow_ref={workflow_ref!r} job_workflow_sha={workflow_sha!r}"
)
output_path = Path(os.environ["GITHUB_OUTPUT"])
with output_path.open("a", encoding="utf-8") as fh:
fh.write(f"repository={repo}\n")
fh.write(f"ref={workflow_sha}\n")
PY
- uses: actions/checkout@v6
with:
repository: ${{ steps.clawhub_source.outputs.repository }}
ref: ${{ steps.clawhub_source.outputs.ref }}
path: clawhub-source
- name: Install ClawHub CLI dependencies
working-directory: clawhub-source
run: bun install --frozen-lockfile
- name: Validate publish mode inputs
env:
DRY_RUN: ${{ inputs.dry_run }}
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
run: |
if [[ "$DRY_RUN" == "true" ]]; then
exit 0
fi
if [[ -n "$CLAWHUB_TOKEN" ]]; then
exit 0
fi
echo "::error::Real skill publishes need secrets.clawhub_token. GitHub OIDC trusted publishing for skills is not supported yet."
exit 1
- name: Write ClawHub config
env:
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
CLAWHUB_REGISTRY: ${{ inputs.registry }}
run: |
if [[ -z "$CLAWHUB_TOKEN" ]]; then
echo "No ClawHub token provided, skipping config file creation."
exit 0
fi
python3 - <<'PY'
import json
import os
from pathlib import Path
path = Path(os.environ["RUNNER_TEMP"]) / "clawhub-config.json"
path.write_text(
json.dumps(
{
"registry": os.environ["CLAWHUB_REGISTRY"],
"token": os.environ["CLAWHUB_TOKEN"],
},
indent=2,
)
+ "\n",
encoding="utf-8",
)
print(path)
PY
echo "CLAWHUB_CONFIG_PATH=$RUNNER_TEMP/clawhub-config.json" >> "$GITHUB_ENV"
- name: Resolve sync command
env:
INPUT_SKILL_PATH: ${{ inputs.skill_path }}
INPUT_ROOT: ${{ inputs.root }}
INPUT_DRY_RUN: ${{ inputs.dry_run }}
INPUT_OWNER: ${{ inputs.owner }}
INPUT_TAGS: ${{ inputs.tags }}
INPUT_BUMP: ${{ inputs.bump }}
INPUT_SITE: ${{ inputs.site }}
INPUT_REGISTRY: ${{ inputs.registry }}
INPUT_REF: ${{ inputs.ref }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_REF: ${{ github.ref }}
run: |
python3 - <<'PY'
import os
import shlex
import subprocess
from pathlib import Path
skill_path = os.environ["INPUT_SKILL_PATH"].strip()
root = os.environ["INPUT_ROOT"].strip() or "skills"
scan_root = skill_path or root
source_commit = subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip()
source_ref = os.environ["INPUT_REF"].strip() or os.environ["GITHUB_REF"].strip()
cli_entry = (
Path(os.environ["GITHUB_WORKSPACE"])
/ "clawhub-source"
/ "packages"
/ "clawhub"
/ "src"
/ "cli.ts"
)
if not cli_entry.exists():
raise SystemExit(f"Missing ClawHub CLI entrypoint at {cli_entry}")
cmd = [
"bun",
str(cli_entry),
"--workdir",
scan_root,
"--dir",
".",
"sync",
"--all",
"--json",
"--no-clawdbot-roots",
"--site",
os.environ["INPUT_SITE"],
"--registry",
os.environ["INPUT_REGISTRY"],
"--bump",
os.environ["INPUT_BUMP"].strip() or "patch",
"--source-repo",
os.environ["GITHUB_REPOSITORY"],
"--source-commit",
source_commit,
]
if os.environ["INPUT_DRY_RUN"] == "true":
cmd.append("--dry-run")
owner = os.environ["INPUT_OWNER"].strip()
tags = os.environ["INPUT_TAGS"].strip()
if owner:
cmd += ["--owner", owner]
if tags:
cmd += ["--tags", tags]
if source_ref:
cmd += ["--source-ref", source_ref]
path = Path(os.environ["RUNNER_TEMP"]) / "clawhub-skill-publish-command.sh"
shell_line = " ".join(shlex.quote(part) for part in cmd)
path.write_text("#!/usr/bin/env bash\nset -euo pipefail\n" + shell_line + "\n", encoding="utf-8")
path.chmod(0o755)
print(shell_line)
PY
- name: Run skill sync
run: |
set -euo pipefail
"$RUNNER_TEMP/clawhub-skill-publish-command.sh" | tee "$RUNNER_TEMP/skill-publish.json"
- name: Capture workflow outputs
id: capture
run: |
python3 - <<'PY'
import json
import os
from pathlib import Path
output_path = Path(os.environ["RUNNER_TEMP"]) / "skill-publish.json"
raw = output_path.read_text(encoding="utf-8").strip()
parsed = json.loads(raw)
github_output = Path(os.environ["GITHUB_OUTPUT"])
with github_output.open("a", encoding="utf-8") as fh:
fh.write("publish_json<<__CLAWHUB_JSON__\n")
fh.write(json.dumps(parsed, indent=2))
fh.write("\n__CLAWHUB_JSON__\n")
PY
- name: Upload publish JSON artifact
uses: actions/upload-artifact@v7
with:
name: clawhub-skill-publish-json
path: ${{ runner.temp }}/skill-publish.json
if-no-files-found: error
+1
View File
@@ -3,6 +3,7 @@ node_modules
.bun-build
*.bun-build
.artifacts/
artifacts/
.cache/
.data/
bin/docs-list
+1
View File
@@ -51,6 +51,7 @@ Specialized corpus, scanner, security-worker, UI proof, proof publishing, Crabbo
- Tests live in `src/**` and `convex/lib/**`.
- Coverage threshold: 80% global (lines/functions/branches/statements).
- Example: `convex/lib/skills.test.ts`.
- For local UI state testing, prefer creating realistic backend state through seed logic plus a DevPersonaFab entry for the associated test user. Avoid one-off manual DB edits when the state is likely to be reused, such as org membership, official publisher access, moderation holds, or publishing permissions.
## Commit & Pull Request Guidelines
+19
View File
@@ -2,8 +2,27 @@
## Unreleased
## 0.19.2 - 2026-06-05
### Fixes
- CLI: accept the legacy `clawhub skill verify --json` flag as a hidden compatibility no-op while continuing to print JSON by default.
## 0.19.1 - 2026-06-05
### Fixes
- CLI: install source-backed GitHub skills from the deployed `/api/v1/skills/:slug/install` resolver so `clawhub install` works for skills without hosted ClawHub versions.
## 0.19.0 - 2026-06-03
### Changes
- CLI/API: add authenticated `clawhub scan` submit/poll support for ephemeral local skill bundles and owner-authorized published skill scans, including JSON output and report ZIP downloads (#2479).
### Fixes
- Auth/Ops: keep GitHub account-age lookups on immutable numeric IDs, retry without auth when a configured GitHub token is rejected, and add an operator backfill for missing cached account ages.
- API/CLI: report Skill Card verification with flattened skill/version metadata, ClawScan verdict fields at `security.*`, and supporting scanner evidence under `security.signals`.
## 0.18.0 - 2026-05-25
+18
View File
@@ -103,6 +103,24 @@ CLAWHUB_WORKTREE_SOURCE=/path/to/source/worktree bun run setup:worktree
The detached server writes runtime state under `.codex/runtime/`. Stop it with `wt --yes stop` before removing the worktree.
### Local Codex workers
Local dev does not start Codex-backed workers by default, so `dev:worktree` does
not spend Codex quota.
To process local ClawScan or Skill Card jobs, opt in for that shell:
```bash
CLAWHUB_ALLOW_LOCAL_CODEX_SCAN=1 bun run dev:workers -- --workers security-scan --once
CLAWHUB_ALLOW_LOCAL_CODEX_SCAN=1 bun run dev:workers -- --workers skill-card --once
```
Opted-in local runs use an ignored worktree-local `CODEX_HOME` unless you provide
one.
Without those workers, local ClawScan and Skill Card jobs stay pending until you
opt in, seed/mock results, or use the production workflows.
### Seed the database
Populate local QA fixtures and the committed public corpus so the UI isn't empty:
+127 -133
View File
@@ -6,7 +6,7 @@
"name": "clawhub",
"dependencies": {
"@auth/core": "0.37.4",
"@convex-dev/auth": "0.0.92",
"@convex-dev/auth": "0.0.93",
"@fontsource/bricolage-grotesque": "5.2.10",
"@fontsource/ibm-plex-mono": "5.2.7",
"@fontsource/manrope": "5.2.8",
@@ -22,29 +22,29 @@
"@radix-ui/react-toggle-group": "1.1.11",
"@radix-ui/react-tooltip": "1.2.8",
"@resvg/resvg-wasm": "2.6.2",
"@shikijs/rehype": "4.1.0",
"@tanstack/react-router": "1.170.8",
"@tanstack/react-start": "1.168.13",
"@shikijs/rehype": "4.2.0",
"@tanstack/react-router": "1.170.12",
"@tanstack/react-start": "1.168.21",
"@vercel/analytics": "2.0.1",
"class-variance-authority": "0.7.1",
"clawhub-schema": "workspace:0.0.2",
"clsx": "2.1.1",
"convex": "1.39.1",
"convex": "1.40.0",
"convex-helpers": "0.1.118",
"fflate": "0.8.3",
"h3": "2.0.1-rc.22",
"ignore": "7.0.5",
"lucide-react": "1.16.0",
"lucide-react": "1.17.0",
"mime": "4.1.0",
"monaco-editor": "0.55.1",
"react": "19.2.6",
"react-dom": "19.2.6",
"react": "19.2.7",
"react-dom": "19.2.7",
"react-markdown": "10.1.0",
"rehype-raw": "7.0.0",
"rehype-sanitize": "6.0.0",
"remark-gfm": "4.0.1",
"semver": "7.8.1",
"shiki": "4.1.0",
"semver": "7.8.2",
"shiki": "4.2.0",
"sonner": "2.0.7",
"tailwind-merge": "3.6.0",
"tailwindcss": "4.3.0",
@@ -62,42 +62,42 @@
"@testing-library/dom": "10.4.1",
"@testing-library/react": "16.3.2",
"@types/node": "25.9.1",
"@types/react": "19.2.15",
"@types/react": "19.2.17",
"@types/react-dom": "19.2.3",
"@types/semver": "7.7.1",
"@vitejs/plugin-react": "6.0.2",
"@vitest/coverage-v8": "4.1.7",
"@vitest/coverage-v8": "4.1.8",
"jsdom": "29.1.1",
"nitro": "3.0.260429-beta",
"only-allow": "1.2.2",
"oxfmt": "0.51.0",
"oxlint": "1.66.0",
"oxfmt": "0.53.0",
"oxlint": "1.68.0",
"oxlint-tsgolint": "0.23.0",
"typescript": "6.0.3",
"undici": "7.26.0",
"vite": "8.0.14",
"vitest": "4.1.7",
"undici": "7.27.1",
"vite": "8.0.16",
"vitest": "4.1.8",
},
},
"packages/clawhub": {
"name": "clawhub",
"version": "0.18.0",
"version": "0.19.2",
"bin": {
"clawdhub": "bin/clawdhub.js",
"clawhub": "bin/clawdhub.js",
},
"dependencies": {
"@clack/prompts": "1.4.0",
"@clack/prompts": "1.5.1",
"arktype": "2.2.0",
"commander": "14.0.3",
"commander": "15.0.0",
"fflate": "0.8.3",
"ignore": "7.0.5",
"json5": "2.2.3",
"mime": "4.1.0",
"ora": "9.4.0",
"p-retry": "8.0.0",
"semver": "7.8.1",
"undici": "7.26.0",
"semver": "7.8.2",
"undici": "7.27.1",
},
"devDependencies": {
"@types/node": "25.9.1",
@@ -111,17 +111,17 @@
"clawhub-mod": "bin/clawhub-mod.js",
},
"dependencies": {
"@clack/prompts": "1.4.0",
"@clack/prompts": "1.5.1",
"arktype": "2.2.0",
"commander": "14.0.3",
"commander": "15.0.0",
"fflate": "0.8.3",
"ignore": "7.0.5",
"json5": "2.2.3",
"mime": "4.1.0",
"ora": "9.4.0",
"p-retry": "8.0.0",
"semver": "7.8.1",
"undici": "7.26.0",
"semver": "7.8.2",
"undici": "7.27.1",
},
"devDependencies": {
"@types/node": "25.9.1",
@@ -176,8 +176,6 @@
"@babel/helper-module-transforms": ["@babel/helper-module-transforms@7.28.6", "", { "dependencies": { "@babel/helper-module-imports": "^7.28.6", "@babel/helper-validator-identifier": "^7.28.5", "@babel/traverse": "^7.28.6" }, "peerDependencies": { "@babel/core": "^7.0.0" } }, "sha512-67oXFAYr2cDLDVGLXTEABjdBJZ6drElUSI7WKp70NrpyISso3plG9SAGEF6y7zbha/wOzUByWWTJvEDVNIUGcA=="],
"@babel/helper-plugin-utils": ["@babel/helper-plugin-utils@7.28.6", "", {}, "sha512-S9gzZ/bz83GRysI7gAD4wPT/AI3uCnY+9xn+Mx/KPs2JwHJIz1W8PZkg2cqyt3RNOBM8ejcXhV6y8Og7ly/Dug=="],
"@babel/helper-string-parser": ["@babel/helper-string-parser@7.27.1", "", {}, "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA=="],
"@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.28.5", "", {}, "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q=="],
@@ -188,10 +186,6 @@
"@babel/parser": ["@babel/parser@7.29.3", "", { "dependencies": { "@babel/types": "^7.29.0" }, "bin": "./bin/babel-parser.js" }, "sha512-b3ctpQwp+PROvU/cttc4OYl4MzfJUWy6FZg+PMXfzmt/+39iHVF0sDfqay8TQM3JA2EUOyKcFZt75jWriQijsA=="],
"@babel/plugin-syntax-jsx": ["@babel/plugin-syntax-jsx@7.28.6", "", { "dependencies": { "@babel/helper-plugin-utils": "^7.28.6" }, "peerDependencies": { "@babel/core": "^7.0.0-0" } }, "sha512-wgEmr06G6sIpqr8YDwA2dSRTE3bJ+V0IfpzfSY3Lfgd7YWOaAdlykvJi13ZKBt8cZHfgH1IXN+CL656W3uUa4w=="],
"@babel/plugin-syntax-typescript": ["@babel/plugin-syntax-typescript@7.28.6", "", { "dependencies": { "@babel/helper-plugin-utils": "^7.28.6" }, "peerDependencies": { "@babel/core": "^7.0.0-0" } }, "sha512-+nDNmQye7nlnuuHDboPbGm00Vqg3oO8niRRL27/4LYHUsHYh0zJ1xWOz0uRwNFmM1Avzk8wZbc6rdiYhomzv/A=="],
"@babel/runtime": ["@babel/runtime@7.29.2", "", {}, "sha512-JiDShH45zKHWyGe4ZNVRrCjBz8Nh9TMmZG1kh4QTK8hCBTWBi8Da+i7s1fJw7/lYpM4ccepSNfqzZ/QvABBi5g=="],
"@babel/template": ["@babel/template@7.28.6", "", { "dependencies": { "@babel/code-frame": "^7.28.6", "@babel/parser": "^7.28.6", "@babel/types": "^7.28.6" } }, "sha512-YA6Ma2KsCdGb+WC6UpBVFJGXL58MDA6oyONbjyF/+5sBgxY/dwkhLogbMT2GXXyU84/IhRw/2D1Os1B/giz+BQ=="],
@@ -204,11 +198,11 @@
"@bramus/specificity": ["@bramus/specificity@2.4.2", "", { "dependencies": { "css-tree": "^3.0.0" }, "bin": { "specificity": "bin/cli.js" } }, "sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw=="],
"@clack/core": ["@clack/core@1.3.1", "", { "dependencies": { "fast-wrap-ansi": "^0.2.0", "sisteransi": "^1.0.5" } }, "sha512-fT1qHVGAag4IEkrupZ6lRRbNCs1vS9P01KB/sG8zKgvUztbYtFBtQpjSITNwooDZ83tpsPzP0mRNs1/KVszCRA=="],
"@clack/core": ["@clack/core@1.4.1", "", { "dependencies": { "fast-wrap-ansi": "^0.2.0", "sisteransi": "^1.0.5" } }, "sha512-FILJa1gGKEFTGZAJE9RpVhrjKz3c3h4ar60dSv6cGuDqufQ84YEIS3GAGvZiN+H6yaLbbvTFNejjCC4tXpZEuw=="],
"@clack/prompts": ["@clack/prompts@1.4.0", "", { "dependencies": { "@clack/core": "1.3.1", "fast-string-width": "^3.0.2", "fast-wrap-ansi": "^0.2.0", "sisteransi": "^1.0.5" } }, "sha512-S0My7XPGIgpRWMDG8uRqalbgT+a6FmCUdOW+HaIOVVpUPHOb7RrpvjTjiODadKp06fsrVDJZlIzc6yCTp4AnxA=="],
"@clack/prompts": ["@clack/prompts@1.5.1", "", { "dependencies": { "@clack/core": "1.4.1", "fast-string-width": "^3.0.2", "fast-wrap-ansi": "^0.2.0", "sisteransi": "^1.0.5" } }, "sha512-zccHj2z2oCCO4yrDiRSlFOxWerGqRiysP7a5jPK6uoI9URKAquwY42Dd/iUP8JWHxEzdRe4TlbvZCo8z1/mhrw=="],
"@convex-dev/auth": ["@convex-dev/auth@0.0.92", "", { "dependencies": { "@oslojs/crypto": "^1.0.1", "@oslojs/encoding": "^1.1.0", "cookie": "^1.0.1", "is-network-error": "^1.1.0", "jose": "^5.2.2", "jwt-decode": "^4.0.0", "lucia": "^3.2.0", "oauth4webapi": "^3.1.2", "path-to-regexp": "^6.3.0", "server-only": "^0.0.1" }, "peerDependencies": { "@auth/core": "^0.37.0", "convex": "^1.17.0", "react": "^18.2.0 || ^19.0.0-0" }, "optionalPeers": ["react"], "bin": { "auth": "dist/bin.cjs" } }, "sha512-tNRIMTDxi2vrbT+3vz1FgNR1321IfIBDDBy59zul7E1DyzWQKoU0OzgFqWbiVm3o8gn0eQsYTU3UHNRX9kp3wQ=="],
"@convex-dev/auth": ["@convex-dev/auth@0.0.93", "", { "dependencies": { "@oslojs/crypto": "^1.0.1", "@oslojs/encoding": "^1.1.0", "cookie": "^1.0.1", "is-network-error": "^1.1.0", "jose": "^5.2.2", "jwt-decode": "^4.0.0", "lucia": "^3.2.0", "oauth4webapi": "^3.1.2", "path-to-regexp": "^6.3.0", "server-only": "^0.0.1" }, "peerDependencies": { "@auth/core": "^0.37.0", "convex": "^1.17.0", "react": "^18.2.0 || ^19.0.0-0" }, "optionalPeers": ["react"], "bin": { "auth": "dist/bin.cjs" } }, "sha512-z7g1lxcNz1Yck238i79rCIubT+rCM1V9sDUdUkZfthPmOarDx0uchut/Gy78xvjQTTSy1oW1XHECKxdclmdJQA=="],
"@csstools/color-helpers": ["@csstools/color-helpers@6.0.2", "", {}, "sha512-LMGQLS9EuADloEFkcTBR3BwV/CGHV7zyDxVRtVDTwdI2Ca4it0CCVTT9wCkxSgokjE5Ho41hEPgb8OEUwoXr6Q=="],
@@ -376,43 +370,43 @@
"@oxc-project/types": ["@oxc-project/types@0.120.0", "", {}, "sha512-k1YNu55DuvAip/MGE1FTsIuU3FUCn6v/ujG9V7Nq5Df/kX2CWb13hhwD0lmJGMGqE+bE1MXvv9SZVnMzEXlWcg=="],
"@oxfmt/binding-android-arm-eabi": ["@oxfmt/binding-android-arm-eabi@0.51.0", "", { "os": "android", "cpu": "arm" }, "sha512-Ni0sCqg5CIHaLIYFGj+ncbcumylvNC6FE4rfD0KfdmnWHbPJ+zev0qZCXKxy2hFVa0fYRK0yPzf5nzPbkZou7g=="],
"@oxfmt/binding-android-arm-eabi": ["@oxfmt/binding-android-arm-eabi@0.53.0", "", { "os": "android", "cpu": "arm" }, "sha512-XfVM8AmIovBTKXCt14Op5wbfcoM8418nttd+nhMgM3RAVaJg1MtJc73FyWfUt0oxLyBGVwfniNVUsbV/b3VmPg=="],
"@oxfmt/binding-android-arm64": ["@oxfmt/binding-android-arm64@0.51.0", "", { "os": "android", "cpu": "arm64" }, "sha512-eu5lAZjuo0KAkp+M24EhDqfOwA8owQ8d7wyBlOUUGRbDLHpU3IRlDHp8Dif+YqGlxs6jra7yS6WQu/NkPhAxeg=="],
"@oxfmt/binding-android-arm64": ["@oxfmt/binding-android-arm64@0.53.0", "", { "os": "android", "cpu": "arm64" }, "sha512-btHDfXckwdf9zgyAVznfZkf+GVyB0I1m1hlvaOMRx2xoyz3hphfPX97s89J3wfCN8QBETLtk4lQUaeOkrMuQOg=="],
"@oxfmt/binding-darwin-arm64": ["@oxfmt/binding-darwin-arm64@0.51.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-6LsUNIdURhhcIfIn8+xsOb61mSTa9msAHTeSGx9Jf4rsP/gN8PGCF+SKWPAQZbND2w/WBkqQ6303jqEEIXzMdQ=="],
"@oxfmt/binding-darwin-arm64": ["@oxfmt/binding-darwin-arm64@0.53.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-k2RjMcSTkHjoOlsVGbL35JVzXL+oQco3GHPl/5kjebVF4oHNfE24In8F5isqBh9LBJucycWHKDXdGrCchdWcHQ=="],
"@oxfmt/binding-darwin-x64": ["@oxfmt/binding-darwin-x64@0.51.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-9aUMGmVxdHjYMsEAW1tNRoieTJXlVNDFkRvIR1J7LttJXWjVYCu2ekclLij2KJtxBxSQOYSHd12ME/adVGVbZg=="],
"@oxfmt/binding-darwin-x64": ["@oxfmt/binding-darwin-x64@0.53.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-65jIBE2H1l5SSs16fmv6/7b6sAx/WpvnsgDhVWK9qSjNFDUro7MPQ6q5UhpY7kl46yltfR046iAnxy/Bzqbiew=="],
"@oxfmt/binding-freebsd-x64": ["@oxfmt/binding-freebsd-x64@0.51.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-mkY1nhZTqYb+NHaAWxOCKISN6FwdrwMNsu17vTUA3wzUV2VJ+Paq15ZokRcsMU/2PUdHO73prxyeJpjXQ3MPpQ=="],
"@oxfmt/binding-freebsd-x64": ["@oxfmt/binding-freebsd-x64@0.53.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-oYe1gkz7U49PCYrS9147d2fJZj8mDI4Di6AvlsU5fu9p+Tq8S7qqOMSZjUiVTLX8bXuSA9Lk/tIxuegVjkNYRA=="],
"@oxfmt/binding-linux-arm-gnueabihf": ["@oxfmt/binding-linux-arm-gnueabihf@0.51.0", "", { "os": "linux", "cpu": "arm" }, "sha512-wtFwNwE4+YCNuPaWoGDZeGsKvD6D1YSUNBJNn/rJBh7CrDBThFE+TBI5kY7vRW9rIOQRsbW2IpyyL3Du4Zqwiw=="],
"@oxfmt/binding-linux-arm-gnueabihf": ["@oxfmt/binding-linux-arm-gnueabihf@0.53.0", "", { "os": "linux", "cpu": "arm" }, "sha512-ailB2vLzGi629tymdAb2VYJyEHref7oqGxP+tRBrtRBxQrb6NV55JMT7xtGZ8uTeG2+Y9zojqW4LhJYxQnz9Pg=="],
"@oxfmt/binding-linux-arm-musleabihf": ["@oxfmt/binding-linux-arm-musleabihf@0.51.0", "", { "os": "linux", "cpu": "arm" }, "sha512-rnOaNx86G7iRKM6lsCIQMux0SMGNC/TEbFR+r7lpruJ12bnrIWgxd5w1PLqOvgR9r8ZJbpK/zfRKctJnh8/Jfg=="],
"@oxfmt/binding-linux-arm-musleabihf": ["@oxfmt/binding-linux-arm-musleabihf@0.53.0", "", { "os": "linux", "cpu": "arm" }, "sha512-abh4mWBvOvD966sobqF7r103y2yYx7Rb4WGHLOS4+5igGqLbbPxS9aK5+45D6iUY7dWMsk3Muz9a8gUtufvqJA=="],
"@oxfmt/binding-linux-arm64-gnu": ["@oxfmt/binding-linux-arm64-gnu@0.51.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-jOgDzSqWcICGRjsp4mc08FxKMN8vzP2Kgs4E0d2HUP99F+nJDQKklRV4Zuj+0gcBgjrzx2CbpqaIdUVPepCojA=="],
"@oxfmt/binding-linux-arm64-gnu": ["@oxfmt/binding-linux-arm64-gnu@0.53.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-z73PvuhJ8qA+cDbaiqbtopHglA91U4+y5wn2sTJJrnpB957d5P33FEuyP3DQIFd7ofljmDmfVT4G0CVGHZaJWg=="],
"@oxfmt/binding-linux-arm64-musl": ["@oxfmt/binding-linux-arm64-musl@0.51.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-KBUCdrH5bwVrAvI9gU/1S55oH6fzXjr++J/oVocdu7bYTks1l7DNNT+rLd/1TDdAEjObGwmfWamn7LC1m8A0DQ=="],
"@oxfmt/binding-linux-arm64-musl": ["@oxfmt/binding-linux-arm64-musl@0.53.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-I6bhOTroqc3ThrwZ89l2k3ivKuELhdPLbAcJhRNyjWvlgwb0vjRgEnVL1XLx5Jud04/ypNRZBykAWrSk6l/D+g=="],
"@oxfmt/binding-linux-ppc64-gnu": ["@oxfmt/binding-linux-ppc64-gnu@0.51.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-NapfjYsABFqTJ1Dn9Efq6sN5esaHconVKwVLbDGNQLrwpOx/g17mkwErHzU72PutL67nf3wNAkbq122H+zLxag=="],
"@oxfmt/binding-linux-ppc64-gnu": ["@oxfmt/binding-linux-ppc64-gnu@0.53.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-w0p3JzB/PkkQjXALMJMqP9YfP3yq4w6zGsu5kezQmUnxRkN3b/Theg2l/nDgBsOcczxS3gL6Gam5XNAVrO6QJQ=="],
"@oxfmt/binding-linux-riscv64-gnu": ["@oxfmt/binding-linux-riscv64-gnu@0.51.0", "", { "os": "linux", "cpu": "none" }, "sha512-5dlDt1dUZCVi6elIhiK1PWg9wpTzTcIuj0IZnSurvIoMrhOWqqTcc1dSTxcSkNaBZhfsNqRZdINI1zAgbKkJNQ=="],
"@oxfmt/binding-linux-riscv64-gnu": ["@oxfmt/binding-linux-riscv64-gnu@0.53.0", "", { "os": "linux", "cpu": "none" }, "sha512-mzBhF6k1Yq1K/dqDmVe/AAafnlJfEpx7yfUiksyeWXJk5iSzZqBSxcsa02zIytYgQFRZ7h6WPZfwHg/DoOE1Kw=="],
"@oxfmt/binding-linux-riscv64-musl": ["@oxfmt/binding-linux-riscv64-musl@0.51.0", "", { "os": "linux", "cpu": "none" }, "sha512-pgdWUJn0S5nulyiVdlFV8DzCUnGXkU99W5PSkkmbaZW+LrZBPxpezun4G0DDHbQaVYuJeCuKsXsGKGo77CkUTQ=="],
"@oxfmt/binding-linux-riscv64-musl": ["@oxfmt/binding-linux-riscv64-musl@0.53.0", "", { "os": "linux", "cpu": "none" }, "sha512-AlFCpnRQhogQFzZXWbO6xB6/Udy745L+eQNmDPGg7G/OeWsYmJc4jZYfUN5pQg0reOPWSED2mOQqKZOJM1U8cA=="],
"@oxfmt/binding-linux-s390x-gnu": ["@oxfmt/binding-linux-s390x-gnu@0.51.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-2XTFUe97CbDGAI8vjwDfZ1HdakO0XIADyJ24idEg64SC4/K4in/OisXVnrW4NMK7I6TgC7EqRhC0Ln/nKhAemA=="],
"@oxfmt/binding-linux-s390x-gnu": ["@oxfmt/binding-linux-s390x-gnu@0.53.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-XD4ulY4f1DWbuuZXAqxhVn+gdPmrhnmojWtFN78ctVoupmS845fGhsUrk1HZXKQI+iymbaiz9vAjPsghHNQ7Ag=="],
"@oxfmt/binding-linux-x64-gnu": ["@oxfmt/binding-linux-x64-gnu@0.51.0", "", { "os": "linux", "cpu": "x64" }, "sha512-kQ1OuCqqt/yyf0ZN9VFxW1/JnlgJgii3Dr7pWf9vNBvrX1hv6g39/+mc5oGRHRGJFZtl3zsGDWR9c5N2B/gwBw=="],
"@oxfmt/binding-linux-x64-gnu": ["@oxfmt/binding-linux-x64-gnu@0.53.0", "", { "os": "linux", "cpu": "x64" }, "sha512-xg8KWX0QnxmYWRe60CgHYWXI0ZOtBbqTsXvWiWrcl2XUHJ3fht2QerOk2iWvylzX3zNT2GpvBRxGoR4d3sxPRQ=="],
"@oxfmt/binding-linux-x64-musl": ["@oxfmt/binding-linux-x64-musl@0.51.0", "", { "os": "linux", "cpu": "x64" }, "sha512-ARTYqxHF475o96Gbn41hvSWSSRygPlRDXZZgZ9I2scU1y0qiWpCQyZCoefaQa0mwv+wwtZ+luS4YOzsRzM/izg=="],
"@oxfmt/binding-linux-x64-musl": ["@oxfmt/binding-linux-x64-musl@0.53.0", "", { "os": "linux", "cpu": "x64" }, "sha512-MWExpYBGvl+pIvVB/gj/CcWlN2al8AizT7rUbtaYaWNoQkhWARM6W3qpgoCr72CYSN9PborzPmM5MIRe2BrNdA=="],
"@oxfmt/binding-openharmony-arm64": ["@oxfmt/binding-openharmony-arm64@0.51.0", "", { "os": "none", "cpu": "arm64" }, "sha512-QiC1XrCl6a6BmqMzduO8hdIRMf1m44hCkt2Q68KWkTvUB/E7fd2iomyNh6KnnRca5w6eBrRAAtLFqTh+xjsjJA=="],
"@oxfmt/binding-openharmony-arm64": ["@oxfmt/binding-openharmony-arm64@0.53.0", "", { "os": "none", "cpu": "arm64" }, "sha512-u4sajgO4nxgmJIgc/y2AqPhkdbOkQH8WugXpA1+pW0ESQhvGZ1oGq61Q4xMbJHJU1hFgtO18QNrcFYDPYH0gwQ=="],
"@oxfmt/binding-win32-arm64-msvc": ["@oxfmt/binding-win32-arm64-msvc@0.51.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-NC/hJb9dtU23Zf8L7IVK95xnFjiQ7AfcLO2l5pb69TDEr958qxrtnB2CveeeNSCBFNIkgaTCfd/vHNSoG78l9g=="],
"@oxfmt/binding-win32-arm64-msvc": ["@oxfmt/binding-win32-arm64-msvc@0.53.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-Yq9sOZoIOJ5xPjO0qOyHJS4CiPuTkB2en9auxZz7Ar2p5RaC7BzLyVVmAA7zz9/L9YnjjY1DwNxN+ivKXimN/A=="],
"@oxfmt/binding-win32-ia32-msvc": ["@oxfmt/binding-win32-ia32-msvc@0.51.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-2C45za4Rj36n8YIbhRL1PQbxmXJYf81WEcAgvj5I4ptRROG+A+81hREEN5bmCHADE1UfYaN312U6tkILoZZy6w=="],
"@oxfmt/binding-win32-ia32-msvc": ["@oxfmt/binding-win32-ia32-msvc@0.53.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-es1fVNZEkBqEcQtBpn19SYFgZF7FawlkCjkT/iImfEAus4gun8fBwB1E9hpV5LcR9B0DBNvRIXhW8BQk3JaE+Q=="],
"@oxfmt/binding-win32-x64-msvc": ["@oxfmt/binding-win32-x64-msvc@0.51.0", "", { "os": "win32", "cpu": "x64" }, "sha512-73RqdAuVKQTkjZIDw08JaDHUM4lav5Qu+CaPwg4QbbA7k8o7LEW0p3UsfZ/F8dsO/pwVYh3RzFcanwLRTTahbQ=="],
"@oxfmt/binding-win32-x64-msvc": ["@oxfmt/binding-win32-x64-msvc@0.53.0", "", { "os": "win32", "cpu": "x64" }, "sha512-QFmJs2bEu9AO4O6qsmEaZNGi6dFq8N+rT8EHAAnZIq/B9SeJDUbc4DzVxQ48MfDsL7D3sCZzo37zuTuspcURgg=="],
"@oxlint-tsgolint/darwin-arm64": ["@oxlint-tsgolint/darwin-arm64@0.23.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-gOs9PVr2wEg4ox9z0aJo+RKhhImW86YL5N6yav8BK/rgPsIrwN/igSZ+pbRr723NFvUNKde9fgMhRA6JrXAOZw=="],
@@ -426,43 +420,43 @@
"@oxlint-tsgolint/win32-x64": ["@oxlint-tsgolint/win32-x64@0.23.0", "", { "os": "win32", "cpu": "x64" }, "sha512-5MyjFuqf+g8OUPJBSGWHJtmoWnzFJYyOg4To9WMQshZYEWig/vtu7JtJ03VWnzHv9LJkAUeApY0gVCOywFR/iQ=="],
"@oxlint/binding-android-arm-eabi": ["@oxlint/binding-android-arm-eabi@1.66.0", "", { "os": "android", "cpu": "arm" }, "sha512-f7kq8N51T4phpzqfBpA2qaVTI/KrkCmNwaj3t/97I/WLTDI+UhlP5GL9eER+zVxBhtlx5rKXWByJU1/zDAvyaw=="],
"@oxlint/binding-android-arm-eabi": ["@oxlint/binding-android-arm-eabi@1.68.0", "", { "os": "android", "cpu": "arm" }, "sha512-wEdsIspexXLLMCPAEOcCuFLMt6aE3AzTuA/nQKLPRnoJ+EQTturmGheDkhHuuVHx0GbutjQ3JKmEn+Gz6Ag28Q=="],
"@oxlint/binding-android-arm64": ["@oxlint/binding-android-arm64@1.66.0", "", { "os": "android", "cpu": "arm64" }, "sha512-xu6QO71tdDS9mjmLZ3AqhtaVHBvdmsOKkYnReNNDgh+XiwnsipeQOIxbiYOOO0iAXycJ+GK0wdMSZP/2j/AmSg=="],
"@oxlint/binding-android-arm64": ["@oxlint/binding-android-arm64@1.68.0", "", { "os": "android", "cpu": "arm64" }, "sha512-6aZRNNXQTsYtgaus8HTb9nuCcsrQTlKXGnktwvwW0n/SooRWNxNb3925grDkC63aEYZuCIyOVLV16IdYIoC2aQ=="],
"@oxlint/binding-darwin-arm64": ["@oxlint/binding-darwin-arm64@1.66.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-HZ24VimSOC7mxuEA99e0H2FS0C1yO3+iW13jPRAk+e2njsUs3QeAXsafCDyaIrV/MirdOVez+etQNQsJE43zNQ=="],
"@oxlint/binding-darwin-arm64": ["@oxlint/binding-darwin-arm64@1.68.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-lVTbsE3kO4bLpZELgjRZuAJc8kP98wb83yMXWH8gaPaFZ+cM2IDeZto4ByoUAYj0Mxv2rvw+A1ssZequSepVSg=="],
"@oxlint/binding-darwin-x64": ["@oxlint/binding-darwin-x64@1.66.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-awhj8ZvJrrRSnXj7V++rpZvTmnl99L6mi0B7gg7Cp7BN6cKpzuI481bHNLvXGA9GB1/oEgA3ponuyoAc6Md12A=="],
"@oxlint/binding-darwin-x64": ["@oxlint/binding-darwin-x64@1.68.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-nCmw2XrmQskjBUh/sfP5yKs93V68LijQgjd1cuuZ/q4SCARngLYs60/qqyzuMsg8QQ9KArDI98hxs/RDGE4KRQ=="],
"@oxlint/binding-freebsd-x64": ["@oxlint/binding-freebsd-x64@1.66.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-KQF0oVV21/FjIqkRuL8Q1vh8ECsE5+ocdH5tcqTQ4ZnYuDVoYibQUNfqBjQaUsP6UIIda5Y75Wpm5p4RgQWiWw=="],
"@oxlint/binding-freebsd-x64": ["@oxlint/binding-freebsd-x64@1.68.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-TI4ovQJliYE9V6e06cEv+qEI9uj7Ao65fmif4er4HD+aouyYyh0P31q2jh3KtqsOHHcQqv2PZ61TjJFLpBDGWQ=="],
"@oxlint/binding-linux-arm-gnueabihf": ["@oxlint/binding-linux-arm-gnueabihf@1.66.0", "", { "os": "linux", "cpu": "arm" }, "sha512-9u1rgwZSEXWb30vbFZzQ78HVXBo0WCKNwJ3a2InRUTNMRng+PUDIoSFmA+m4HdUfBaIqftShq8J8qHc+eE/Vig=="],
"@oxlint/binding-linux-arm-gnueabihf": ["@oxlint/binding-linux-arm-gnueabihf@1.68.0", "", { "os": "linux", "cpu": "arm" }, "sha512-LcNnEi9g71Cmry5ZpLbKT+oVv+/zYG3hYVAbBBB5X85nOQZSk8l92CnDkxJMcxUg0NCnMCOFZuaVDlMyv4tYJw=="],
"@oxlint/binding-linux-arm-musleabihf": ["@oxlint/binding-linux-arm-musleabihf@1.66.0", "", { "os": "linux", "cpu": "arm" }, "sha512-Ynot2HR1bHxUaNWoC280MVTDfZuaWuP3XfSMRDhyuZrVjhzoaBCVFlw8h8qeZjWKVUBhPWFIxB7AQTlK8Z2WWg=="],
"@oxlint/binding-linux-arm-musleabihf": ["@oxlint/binding-linux-arm-musleabihf@1.68.0", "", { "os": "linux", "cpu": "arm" }, "sha512-OovHahL3FX4UaK+hgSf11llUx2vszqjSdQQ61Ck9InOEI/ptZoC4XSQJurITqItVvd53JSlmkLMeaNjM1PoQew=="],
"@oxlint/binding-linux-arm64-gnu": ["@oxlint/binding-linux-arm64-gnu@1.66.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-xCbgzciGgo+A4aQZEknsNrNiIwY7sU5SfRuMmRjPIvZAgdF34cIHiKvwOsS5XRLjlTVSFwitmq6YclTtHTfU+g=="],
"@oxlint/binding-linux-arm64-gnu": ["@oxlint/binding-linux-arm64-gnu@1.68.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-YbzTglnHLzzi9zv5or8Ztz5fykAoZE8W9iM42/bOrF4HBSB6rJTqdLQWuoP76EHQw9DuKl76K1QmFlG29sPJXQ=="],
"@oxlint/binding-linux-arm64-musl": ["@oxlint/binding-linux-arm64-musl@1.66.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-hmo+ZB/lHkR1HdDmnziNpzSLmulnUSu10VEqX2Yex7OwvoBAbjJQLvy4gIBRV3AAwWnCvAxKp5Nv1GE6LU1QMg=="],
"@oxlint/binding-linux-arm64-musl": ["@oxlint/binding-linux-arm64-musl@1.68.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-qVKtCZNic+OoNnOr/hCQAu22HSQzflI7Fsq/Blzkw02SnLuv163k3kfmrVpZjSBlUHgsRKj6WgQiw30d3SX02Q=="],
"@oxlint/binding-linux-ppc64-gnu": ["@oxlint/binding-linux-ppc64-gnu@1.66.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-2Invd4Uyy81mVooQC5FBtfxSNrvcX1OxbMlVQ6M2erRrNI2awFYF26YNW2yFxdVFZ4ffNOWKghtMjhnUPsXsVA=="],
"@oxlint/binding-linux-ppc64-gnu": ["@oxlint/binding-linux-ppc64-gnu@1.68.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-zExyZ8ZOUuAyQ0y9jpTcyjKUz62YY9JhKPyVxzvjTpXzZ3ujdqiVwfPWDdnA1SsIOrxdtxHn7KErDHLWskFjXg=="],
"@oxlint/binding-linux-riscv64-gnu": ["@oxlint/binding-linux-riscv64-gnu@1.66.0", "", { "os": "linux", "cpu": "none" }, "sha512-s0iXPDQVdgayE3RGa/N2DZF7tjgg0TwEtD1sGoDxqPDGrIXgo45H0yHknT0f9A0yteASsweYZtDyTuVlM4aSag=="],
"@oxlint/binding-linux-riscv64-gnu": ["@oxlint/binding-linux-riscv64-gnu@1.68.0", "", { "os": "linux", "cpu": "none" }, "sha512-6C4MPuwewyDavA7sxM14wzgRi5GGL68HPIxRCdVyS75U4MDbpFVYzKO9WNR6KLKTMPq2pcz3THwo1sK2uiqngw=="],
"@oxlint/binding-linux-riscv64-musl": ["@oxlint/binding-linux-riscv64-musl@1.66.0", "", { "os": "linux", "cpu": "none" }, "sha512-OekL4XFiu7RPK0JIZi8VeHgtIXPREf42t8Cy/rKEsC+P3gcqDgNAAGiyuUOpdbG4wwbfue1q4CHcCO7spSve6w=="],
"@oxlint/binding-linux-riscv64-musl": ["@oxlint/binding-linux-riscv64-musl@1.68.0", "", { "os": "linux", "cpu": "none" }, "sha512-bnZooVeHAcvA+dH0EDLgx+7HY/DRi6e0hFszg3P+OBatuUjV6EvfIyNIzWOusmqAVh4L6r21GGTZtiKE4iqM4Q=="],
"@oxlint/binding-linux-s390x-gnu": ["@oxlint/binding-linux-s390x-gnu@1.66.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-Ga1D0kj1SFslm34ThA/BdkUlyAYEnTsXyRC4pF0C5agZSwtGdHYWMTQWemUfBGp4RCG4QWXgdO+HmmmKqOtlBg=="],
"@oxlint/binding-linux-s390x-gnu": ["@oxlint/binding-linux-s390x-gnu@1.68.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-dIqnZnJSmHCMOUpUcWQOiV14o3DDPVx1DSsMaSzvdhNjC1tB1iEPZbdiMSCIEYbkgbsYznHXWqFdKL8WUB3F8g=="],
"@oxlint/binding-linux-x64-gnu": ["@oxlint/binding-linux-x64-gnu@1.66.0", "", { "os": "linux", "cpu": "x64" }, "sha512-p5jfP1wUZe/IC3qpQO84n9DRnf9g3lKRtLBlQq23ykyrDglHcVx7sWmVTlPuU6SBw8mNnPzyOn022G3XZHnlww=="],
"@oxlint/binding-linux-x64-gnu": ["@oxlint/binding-linux-x64-gnu@1.68.0", "", { "os": "linux", "cpu": "x64" }, "sha512-zc9lEnfV/HreDTY6gdMlZe+irkwHSxQ4/B1pS9GyK7RVaA5LxhoZY/w6/o2vIwLLEYiXQ5ujGxOM1ZazeFAAIA=="],
"@oxlint/binding-linux-x64-musl": ["@oxlint/binding-linux-x64-musl@1.66.0", "", { "os": "linux", "cpu": "x64" }, "sha512-vUB/sYlYZorDL1ZD+o9mRv7zbsykrrFRtmgS6R8musZqLtrPRQn1gc1eGpuX+sfdccz42STl/AqldY6XRb2upQ=="],
"@oxlint/binding-linux-x64-musl": ["@oxlint/binding-linux-x64-musl@1.68.0", "", { "os": "linux", "cpu": "x64" }, "sha512-Dl5QEX0TCo/40Cdh1o1JdPS//+YiWqjC+Hrrya5OQmStZZr4svAFtdlqcpCrU9yq2Mo3vRVyO9B3h0dzD8s36Q=="],
"@oxlint/binding-openharmony-arm64": ["@oxlint/binding-openharmony-arm64@1.66.0", "", { "os": "none", "cpu": "arm64" }, "sha512-yde+6p/F59xRkGR9H1HfngWRif1QRJjynZK349l+UI0H6w9hL3G8/AVaTHFyTtLVQ56qtNbX2/5Dc77n1ovnOg=="],
"@oxlint/binding-openharmony-arm64": ["@oxlint/binding-openharmony-arm64@1.68.0", "", { "os": "none", "cpu": "arm64" }, "sha512-/qy6dOvi4S3/LeXq0l5BT5pRKPYA7oj3uKwJOAZOr5HRLL+HK6jdBynvWuXIA2wwfE01RzNYmbBdM7vwYx00sA=="],
"@oxlint/binding-win32-arm64-msvc": ["@oxlint/binding-win32-arm64-msvc@1.66.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-O9GLucgoTdmOrbBX+EjzNe7o/Ze5TFOvXcib6bzUOtBOmj6cV+zw18NgB+cGKAkDw1Pdqs8vGkfHbbsLuDtXWg=="],
"@oxlint/binding-win32-arm64-msvc": ["@oxlint/binding-win32-arm64-msvc@1.68.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-fHNtVqPHSYE7UFDSLVFUjxQjnSVXxseNJmRW+XuP4pXXDwePdPda43NL7/BBCFTxHjycOc44JNDaOPtFDNui9A=="],
"@oxlint/binding-win32-ia32-msvc": ["@oxlint/binding-win32-ia32-msvc@1.66.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-m3Pjwc2MfTcom4E4gOv7DyuGyt7OfGNCbmqDHd+N7EzXmP+ppHuudm2NjcA3AjV5TSeGxaguVF4SbTKHe1USYA=="],
"@oxlint/binding-win32-ia32-msvc": ["@oxlint/binding-win32-ia32-msvc@1.68.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-NnKXr4Wgo4nps3erhrE0f8shBvBPZMHg72nDsvX0JyrRvsNiP3f1JNvbCKh+A6VFvpF7ZoJxu904P3cKMhvZnA=="],
"@oxlint/binding-win32-x64-msvc": ["@oxlint/binding-win32-x64-msvc@1.66.0", "", { "os": "win32", "cpu": "x64" }, "sha512-/DbBvw8UFBhja6PqudUjV4UtfsJr0Oa7jUjWVKB0g86lj/VwnPrkngn0sFql3c9RDA0O16dh7ozsXb6GjNAzBQ=="],
"@oxlint/binding-win32-x64-msvc": ["@oxlint/binding-win32-x64-msvc@1.68.0", "", { "os": "win32", "cpu": "x64" }, "sha512-zg5pA+84AlU6XHJ3ruiRxziO71QTrz8nLsk6u01JGS5+tL9/bnlakFiklFrcy4R1/V7ktWtaNitN3JZWmKnf6g=="],
"@panva/hkdf": ["@panva/hkdf@1.2.1", "", {}, "sha512-6oclG6Y3PiDFcoyk8srjLfVKyMfVCKJ27JwNPViuXziFpmdz+MZnZN/aKY0JGXgYuO/VghU0jcOAZgWXZ1Dmrw=="],
@@ -578,21 +572,21 @@
"@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.1", "", {}, "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw=="],
"@shikijs/core": ["@shikijs/core@4.1.0", "", { "dependencies": { "@shikijs/primitive": "4.1.0", "@shikijs/types": "4.1.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4", "hast-util-to-html": "^9.0.5" } }, "sha512-jLJtSJeuFffqX6/inRE1zqU5aFv2hrszvYgq3OjbAgFRZiWv7abKMDdQzYxuSDfmUPQozZvI/kuy6VMTvnvqTQ=="],
"@shikijs/core": ["@shikijs/core@4.2.0", "", { "dependencies": { "@shikijs/primitive": "4.2.0", "@shikijs/types": "4.2.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4", "hast-util-to-html": "^9.0.5" } }, "sha512-Hc87Ab1Ld/vEbZRCbwx344I5v+4RU8CVToUTRkqXL1+TjbuOp9U5Xa0M23V4GEWHxVn+yO5otb+HkQVm3ptWQQ=="],
"@shikijs/engine-javascript": ["@shikijs/engine-javascript@4.1.0", "", { "dependencies": { "@shikijs/types": "4.1.0", "@shikijs/vscode-textmate": "^10.0.2", "oniguruma-to-es": "^4.3.6" } }, "sha512-YquhawCUgaBfhsS72e2Y/dI59gCBNPHu3fEO/tvLaXrTssxZrY5ddjtNLTwndrMgPo8b3IscE+xoICDzpTmlFQ=="],
"@shikijs/engine-javascript": ["@shikijs/engine-javascript@4.2.0", "", { "dependencies": { "@shikijs/types": "4.2.0", "@shikijs/vscode-textmate": "^10.0.2", "oniguruma-to-es": "^4.3.6" } }, "sha512-fjETeq1k5ffyXqRgS6+3hpvqseLalp1kjNfRbXpUgWR8FpZ1CmQfiNHovc5lncYjt/Vg5JK/WJEmLahjwMa0og=="],
"@shikijs/engine-oniguruma": ["@shikijs/engine-oniguruma@4.1.0", "", { "dependencies": { "@shikijs/types": "4.1.0", "@shikijs/vscode-textmate": "^10.0.2" } }, "sha512-axLpjVs45YBvvINa+dJF+NPW+KtFkNXsFr4SDw2BMj9GdeMnGxVB9PQb2xXlJYovslt/nz6giedAyOANkfc7hg=="],
"@shikijs/engine-oniguruma": ["@shikijs/engine-oniguruma@4.2.0", "", { "dependencies": { "@shikijs/types": "4.2.0", "@shikijs/vscode-textmate": "^10.0.2" } }, "sha512-hTorK1dffPkpbMUk6Z+828PgRo7d07HbnizoP0hNPFjhxMHctj0Px/qoHeGMYafc6ju+u9iMldN4JbVzNQM++g=="],
"@shikijs/langs": ["@shikijs/langs@4.1.0", "", { "dependencies": { "@shikijs/types": "4.1.0" } }, "sha512-nwOMruEkbgdZfQ/b8CgpNBVOpvG1k0N5tbmgiFeqsan401+x3ILqlzZJowSla4Agmq4hG2Uf2wh5jLTEhR8VSg=="],
"@shikijs/langs": ["@shikijs/langs@4.2.0", "", { "dependencies": { "@shikijs/types": "4.2.0" } }, "sha512-bwrVRlJ0wUhZxAbVdvBbv2TTC9yLsh4C/IO5Ofz0T8MQntgDvyVnkbjw9vi50r1kx7RCIJdnJnjZAwmAsXFLZQ=="],
"@shikijs/primitive": ["@shikijs/primitive@4.1.0", "", { "dependencies": { "@shikijs/types": "4.1.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-zx2/2Uwj2q9X3KSyYREEhXO23xBw5WUhP4orK2lE4r+t9JGITmEe0JH+wPmJhqHpOT2bRRs6lAL945+LDvOAGw=="],
"@shikijs/primitive": ["@shikijs/primitive@4.2.0", "", { "dependencies": { "@shikijs/types": "4.2.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-NOq+DtUkVBJtZMVXL5A0vI0Xk8nvDYaXetFHSJFlOqjDZIVhIPRYFdGkSoElDqNuegikcc3A76SNUa8dTqtAYA=="],
"@shikijs/rehype": ["@shikijs/rehype@4.1.0", "", { "dependencies": { "@shikijs/types": "4.1.0", "@types/hast": "^3.0.4", "hast-util-to-string": "^3.0.1", "shiki": "4.1.0", "unified": "^11.0.5", "unist-util-visit": "^5.1.0" } }, "sha512-HQwltCcO2/UiFz44/8whyji4rP1VghLu++MgvQn+lQA8/gvuycGkay8DH8o8VAOvLBDKGOkBEw7cC1Cm33GObQ=="],
"@shikijs/rehype": ["@shikijs/rehype@4.2.0", "", { "dependencies": { "@shikijs/types": "4.2.0", "@types/hast": "^3.0.4", "hast-util-to-string": "^3.0.1", "shiki": "4.2.0", "unified": "^11.0.5", "unist-util-visit": "^5.1.0" } }, "sha512-ST3EWye/dwF1gWskczJNBnwFtDzEQ9ceytXZtyc/GfwR5V0qJrkoSGZO55O3SAKDDsXkTDcsfwd9pVe7ROlAHg=="],
"@shikijs/themes": ["@shikijs/themes@4.1.0", "", { "dependencies": { "@shikijs/types": "4.1.0" } }, "sha512-emCcTnUM7yO2wltYbaxm+yLvcCI4+h8XBKc4KmJ7EZUXoSGjcCHifkI//R4OFit9ewpg7H2/9tjOuXrT2v/Knw=="],
"@shikijs/themes": ["@shikijs/themes@4.2.0", "", { "dependencies": { "@shikijs/types": "4.2.0" } }, "sha512-RX8IHYeLv8Cu2W6ruc3RxUqWn0IYCqSrMBzi/uRGAmfyDNOnNO5BF/Px7o97n4XTpmFTo5GbRaazuOWj+2ak2w=="],
"@shikijs/types": ["@shikijs/types@4.1.0", "", { "dependencies": { "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-3EQWX54fMpniOrDblzAhiwiJwpiTMW6+B9DWyUd9ska483tbayFYuw47UxwuPknI31bKnySfVQ/QW+jFL4rFdA=="],
"@shikijs/types": ["@shikijs/types@4.2.0", "", { "dependencies": { "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-VT/MKtlpOhEPZloSH3Pb9WCZEBDoQVMa9jedp5UAwmJOar1DVc9DRODAxmYPW9M93IK4ryuqRejFfmlvlVDemw=="],
"@shikijs/vscode-textmate": ["@shikijs/vscode-textmate@10.0.2", "", {}, "sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg=="],
@@ -638,35 +632,35 @@
"@tanstack/history": ["@tanstack/history@1.162.0", "", {}, "sha512-79pf/RkhteYZTRgcR4F9kbk84P2N8rugQJswxfIqovlbRiT3yI7eBE+5QorIrZaOKktsgzRlXh1l/du/xpl4iA=="],
"@tanstack/react-router": ["@tanstack/react-router@1.170.8", "", { "dependencies": { "@tanstack/history": "1.162.0", "@tanstack/react-store": "^0.9.3", "@tanstack/router-core": "1.171.6", "isbot": "^5.1.22" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-Qw2ju6jjnIsMpuW+VrnHZWHuugqs592PWsnI56sG28qNhg14CgRLahOcNajfuJR9P4MxKGP94WVzmFKSYUz/ig=="],
"@tanstack/react-router": ["@tanstack/react-router@1.170.12", "", { "dependencies": { "@tanstack/history": "1.162.0", "@tanstack/react-store": "^0.9.3", "@tanstack/router-core": "1.171.10", "isbot": "^5.1.22" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-3QSX1kTBFCHV1pMePdLhcmxl2E7io8W59a5f+jDQs+BZjZuRLUDtOuGFiHUGaUCsYHxv6z2dCqWvwxaU6WJpyA=="],
"@tanstack/react-start": ["@tanstack/react-start@1.168.13", "", { "dependencies": { "@tanstack/react-router": "1.170.8", "@tanstack/react-start-client": "1.168.4", "@tanstack/react-start-rsc": "0.1.13", "@tanstack/react-start-server": "1.167.9", "@tanstack/router-utils": "1.162.1", "@tanstack/start-client-core": "1.170.4", "@tanstack/start-plugin-core": "1.171.6", "@tanstack/start-server-core": "1.169.4", "pathe": "^2.0.3" }, "peerDependencies": { "@rsbuild/core": "^2.0.0", "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0", "vite": ">=7.0.0" }, "optionalPeers": ["@rsbuild/core", "vite"] }, "sha512-E2pHQ92NiND1/HiD5Ax71xFXxiRZ2reOfU5W4BqxUL5plap3p8xSw1c6L8Np1E60vsxknuPCYRZESKkRy/LkOA=="],
"@tanstack/react-start": ["@tanstack/react-start@1.168.21", "", { "dependencies": { "@tanstack/react-router": "1.170.12", "@tanstack/react-start-client": "1.168.9", "@tanstack/react-start-rsc": "0.1.20", "@tanstack/react-start-server": "1.167.15", "@tanstack/router-utils": "1.162.2", "@tanstack/start-client-core": "1.170.8", "@tanstack/start-plugin-core": "1.171.13", "@tanstack/start-server-core": "1.169.10", "pathe": "^2.0.3" }, "peerDependencies": { "@rsbuild/core": "^2.0.0", "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0", "vite": ">=7.0.0" }, "optionalPeers": ["@rsbuild/core", "vite"] }, "sha512-vEdTgtH4xaj19XFStSU9MG40JdjGjmro0l9rxvuY2/Ns1spmHbOhIJGoNm/xH7HAYNvGByxny4/QMzay7vZ9Xw=="],
"@tanstack/react-start-client": ["@tanstack/react-start-client@1.168.4", "", { "dependencies": { "@tanstack/react-router": "1.170.8", "@tanstack/router-core": "1.171.6", "@tanstack/start-client-core": "1.170.4" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-PDJ7xEuUKrlBiQz2PrVN9pD2ErmWeFpckYW1WUE8JCAeVi8U7C6rQNTQe4hQxBhycRfRdD53M6UfdWdQODIxyg=="],
"@tanstack/react-start-client": ["@tanstack/react-start-client@1.168.9", "", { "dependencies": { "@tanstack/react-router": "1.170.12", "@tanstack/router-core": "1.171.10", "@tanstack/start-client-core": "1.170.8" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-VLKTu2rmn7C13g4y9W18mGer5fx0klx4YHxYVKR+da4P2pogjumAMMlRRvTMqoruBCSVGDp9VxprU7MEsomoFQ=="],
"@tanstack/react-start-rsc": ["@tanstack/react-start-rsc@0.1.13", "", { "dependencies": { "@tanstack/react-router": "1.170.8", "@tanstack/react-start-server": "1.167.9", "@tanstack/router-core": "1.171.6", "@tanstack/router-utils": "1.162.1", "@tanstack/start-client-core": "1.170.4", "@tanstack/start-fn-stubs": "1.162.0", "@tanstack/start-plugin-core": "1.171.6", "@tanstack/start-server-core": "1.169.4", "@tanstack/start-storage-context": "1.167.8", "pathe": "^2.0.3" }, "peerDependencies": { "@rspack/core": ">=2.0.0-0", "@vitejs/plugin-rsc": ">=0.5.20", "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0", "react-server-dom-rspack": ">=0.0.2" }, "optionalPeers": ["@rspack/core", "@vitejs/plugin-rsc", "react-server-dom-rspack"] }, "sha512-nl5pKkxy1RnRxOLjy/c3g/RKdQSQYWzK5iuLlsRaO9TbLuMhQlNAn255xQgVXG56G9xCtDg8/nD0ZycxSlSkWA=="],
"@tanstack/react-start-rsc": ["@tanstack/react-start-rsc@0.1.20", "", { "dependencies": { "@tanstack/react-router": "1.170.12", "@tanstack/router-core": "1.171.10", "@tanstack/router-utils": "1.162.2", "@tanstack/start-client-core": "1.170.8", "@tanstack/start-fn-stubs": "1.162.0", "@tanstack/start-plugin-core": "1.171.13", "@tanstack/start-server-core": "1.169.10", "@tanstack/start-storage-context": "1.167.12", "pathe": "^2.0.3" }, "peerDependencies": { "@rspack/core": ">=2.0.0-0", "@vitejs/plugin-rsc": ">=0.5.20", "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0", "react-server-dom-rspack": ">=0.0.2" }, "optionalPeers": ["@rspack/core", "@vitejs/plugin-rsc", "react-server-dom-rspack"] }, "sha512-0gXLJQEc2ltL0t/+jbyQV1TRmGVkRre9c+pFQ/41pjHmX8Cj9j1EwSrENYkJF0ghFSpeP7T8YzBQcNew6o1o7g=="],
"@tanstack/react-start-server": ["@tanstack/react-start-server@1.167.9", "", { "dependencies": { "@tanstack/history": "1.162.0", "@tanstack/react-router": "1.170.8", "@tanstack/router-core": "1.171.6", "@tanstack/start-client-core": "1.170.4", "@tanstack/start-server-core": "1.169.4" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-a1SGeeoIEg411vEN6DThB2Bm5tiYBb0tCC/RaG8BSjRVtsY6kxD9cP1+LOpZwjRSgfdyqtSbe1v78ZDB9z0/uw=="],
"@tanstack/react-start-server": ["@tanstack/react-start-server@1.167.15", "", { "dependencies": { "@tanstack/react-router": "1.170.12", "@tanstack/router-core": "1.171.10", "@tanstack/start-server-core": "1.169.10" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-HaraKuBsPYj2i9deu7cofpDWv8hXMQrIosZfAq/ARRMjmwAEO/Im+dzrDRFusJxDQ6h/8QqznTYBT0t7KIUQjQ=="],
"@tanstack/react-store": ["@tanstack/react-store@0.9.3", "", { "dependencies": { "@tanstack/store": "0.9.3", "use-sync-external-store": "^1.6.0" }, "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", "react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-y2iHd/N9OkoQbFJLUX1T9vbc2O9tjH0pQRgTcx1/Nz4IlwLvkgpuglXUx+mXt0g5ZDFrEeDnONPqkbfxXJKwRg=="],
"@tanstack/router-core": ["@tanstack/router-core@1.171.6", "", { "dependencies": { "@tanstack/history": "1.162.0", "cookie-es": "^3.0.0", "seroval": "^1.5.4", "seroval-plugins": "^1.5.4" } }, "sha512-Ol6DQ+j6rf/rPVELIzo8LHwOQV2KL+zry3b+39kL/GKrt7YId52WJRAFMzuseY4XceSW+PU7sG/Cc1QkwJr0hg=="],
"@tanstack/router-core": ["@tanstack/router-core@1.171.10", "", { "dependencies": { "@tanstack/history": "1.162.0", "cookie-es": "^3.0.0", "seroval": "^1.5.4", "seroval-plugins": "^1.5.4" } }, "sha512-EOOvTUBUS2W/DtyqG0A0HW6RLbsbZBVTu3nsWdFpUoAfDxHPjOqRNjz3xwq5KhjMlYuqtHNgaajAd18aebd0ZQ=="],
"@tanstack/router-generator": ["@tanstack/router-generator@1.167.10", "", { "dependencies": { "@babel/types": "^7.28.5", "@tanstack/router-core": "1.171.6", "@tanstack/router-utils": "1.162.1", "@tanstack/virtual-file-routes": "1.162.0", "jiti": "^2.7.0", "magic-string": "^0.30.21", "prettier": "^3.5.0", "zod": "^4.4.3" } }, "sha512-CjbjWRSo6djLU/C7ncb9IbKUcf4IwpdqhLGngkwKkXaVFXGxEAafA/uhvOCv/UEUVR7NI3tJqqQmxYXGcJPbjw=="],
"@tanstack/router-generator": ["@tanstack/router-generator@1.167.14", "", { "dependencies": { "@babel/types": "^7.28.5", "@tanstack/router-core": "1.171.10", "@tanstack/router-utils": "1.162.2", "@tanstack/virtual-file-routes": "1.162.0", "jiti": "^2.7.0", "magic-string": "^0.30.21", "prettier": "^3.5.0", "zod": "^4.4.3" } }, "sha512-63KL39u6H8qr8A9KtUs9IH3CE4mYErk1zTAlmdbHJ5qpsV1FrRb7kLjRR0BzzgwqPPfMe52Gr/X6beBOLcQq/g=="],
"@tanstack/router-plugin": ["@tanstack/router-plugin@1.168.11", "", { "dependencies": { "@babel/core": "^7.28.5", "@babel/plugin-syntax-jsx": "^7.27.1", "@babel/plugin-syntax-typescript": "^7.27.1", "@babel/template": "^7.27.2", "@babel/traverse": "^7.28.5", "@babel/types": "^7.28.5", "@tanstack/router-core": "1.171.6", "@tanstack/router-generator": "1.167.10", "@tanstack/router-utils": "1.162.1", "@tanstack/virtual-file-routes": "1.162.0", "chokidar": "^5.0.0", "unplugin": "^3.0.0", "zod": "^4.4.3" }, "peerDependencies": { "@rsbuild/core": ">=1.0.2 || ^2.0.0", "@tanstack/react-router": "^1.170.8", "vite": ">=5.0.0 || >=6.0.0 || >=7.0.0 || >=8.0.0", "vite-plugin-solid": "^2.11.10 || ^3.0.0-0", "webpack": ">=5.92.0" }, "optionalPeers": ["@rsbuild/core", "@tanstack/react-router", "vite", "vite-plugin-solid", "webpack"] }, "sha512-b2eom/8xCWL/OiWxKub8kYsr8p+kvmB/eXwYGqCWG8vilcJo+eQCSyp54nKt0AZ5k/ET1+eINc+4mwL3bVeAgg=="],
"@tanstack/router-plugin": ["@tanstack/router-plugin@1.168.15", "", { "dependencies": { "@babel/core": "^7.28.5", "@babel/template": "^7.27.2", "@babel/types": "^7.28.5", "@tanstack/router-core": "1.171.10", "@tanstack/router-generator": "1.167.14", "@tanstack/router-utils": "1.162.2", "chokidar": "^5.0.0", "unplugin": "^3.0.0", "zod": "^4.4.3" }, "peerDependencies": { "@rsbuild/core": ">=1.0.2 || ^2.0.0", "@tanstack/react-router": "^1.170.12", "vite": ">=5.0.0 || >=6.0.0 || >=7.0.0 || >=8.0.0", "vite-plugin-solid": "^2.11.10 || ^3.0.0-0", "webpack": ">=5.92.0" }, "optionalPeers": ["@rsbuild/core", "@tanstack/react-router", "vite", "vite-plugin-solid", "webpack"] }, "sha512-Gb+29gWahpi9nqCCYswsADsFJzPLAfWWQ0BfYmY2XUr2pZZYG+RY2w6ZXQFjyvUPvDX6Xm2CqCq9HU0grxASqA=="],
"@tanstack/router-utils": ["@tanstack/router-utils@1.162.1", "", { "dependencies": { "@babel/core": "^7.28.5", "@babel/generator": "^7.28.5", "@babel/parser": "^7.28.5", "@babel/types": "^7.28.5", "ansis": "^4.1.0", "babel-dead-code-elimination": "^1.0.12", "diff": "^8.0.2", "pathe": "^2.0.3", "tinyglobby": "^0.2.15" } }, "sha512-62layyTGmclHDQS/eidwKRfN1hhCKwViG7iEBcVmL0MXgcAB3OOucWCEcDDGd9Cu11H6b4QQ5oOo47MWIqwz0A=="],
"@tanstack/router-utils": ["@tanstack/router-utils@1.162.2", "", { "dependencies": { "@babel/generator": "^7.28.5", "@babel/parser": "^7.28.5", "@babel/types": "^7.28.5", "ansis": "^4.1.0", "babel-dead-code-elimination": "^1.0.12", "diff": "^8.0.2", "pathe": "^2.0.3", "tinyglobby": "^0.2.15" } }, "sha512-hTWqJtqIFFdvuCl8WXNyrodp2L9zo2G37xKRrcVmVRWpAB2h+U1LuRAfS4tsFTiWOIoE/B+WDVFB8JpoEdw6jQ=="],
"@tanstack/start-client-core": ["@tanstack/start-client-core@1.170.4", "", { "dependencies": { "@tanstack/router-core": "1.171.6", "@tanstack/start-fn-stubs": "1.162.0", "@tanstack/start-storage-context": "1.167.8", "seroval": "^1.5.4" } }, "sha512-j/Deupf0zR7P5QObN38xTHufCRZkWTb6a/7aauu8eBmzOzDVggvuEdYHRZWiwJ9HRKbR2/SIJASVKeTtj1OcWw=="],
"@tanstack/start-client-core": ["@tanstack/start-client-core@1.170.8", "", { "dependencies": { "@tanstack/router-core": "1.171.10", "@tanstack/start-fn-stubs": "1.162.0", "@tanstack/start-storage-context": "1.167.12", "seroval": "^1.5.4" } }, "sha512-v1V6TGWdMW7Yu5/11k1ODvKe0lfTDR4iw7QIxv75P73u66mqmqfheKRXlp5tFfb3yR1kX9xS38jjdeVDdnWC1Q=="],
"@tanstack/start-fn-stubs": ["@tanstack/start-fn-stubs@1.162.0", "", {}, "sha512-QWfUZ3Yo923tdQn38LyKMU8rcTw69zc+T4dAvgTWV4O56SqFRsGfS0lSWIMhJRwXIx/bvdi7nTUBDdZtTHtpTQ=="],
"@tanstack/start-plugin-core": ["@tanstack/start-plugin-core@1.171.6", "", { "dependencies": { "@babel/code-frame": "7.27.1", "@babel/core": "^7.28.5", "@babel/types": "^7.28.5", "@rolldown/pluginutils": "1.0.1", "@tanstack/router-core": "1.171.6", "@tanstack/router-generator": "1.167.10", "@tanstack/router-plugin": "1.168.11", "@tanstack/router-utils": "1.162.1", "@tanstack/start-client-core": "1.170.4", "@tanstack/start-server-core": "1.169.4", "exsolve": "^1.0.7", "lightningcss": "^1.32.0", "pathe": "^2.0.3", "picomatch": "^4.0.3", "seroval": "^1.5.4", "source-map": "^0.7.6", "srvx": "^0.11.9", "tinyglobby": "^0.2.15", "ufo": "^1.5.4", "vitefu": "^1.1.1", "xmlbuilder2": "^4.0.3", "zod": "^4.4.3" }, "peerDependencies": { "@rsbuild/core": "^2.0.0", "vite": ">=7.0.0" }, "optionalPeers": ["@rsbuild/core", "vite"] }, "sha512-e0AUN+omib0qLgs0r3zoKRSeHEkwL8qs8skvbl8zgDQXw9zF73K7ZXE7QarSzbqfLAiehVqlv0iPETp8ogUftQ=="],
"@tanstack/start-plugin-core": ["@tanstack/start-plugin-core@1.171.13", "", { "dependencies": { "@babel/code-frame": "7.27.1", "@babel/core": "^7.28.5", "@babel/types": "^7.28.5", "@tanstack/router-core": "1.171.10", "@tanstack/router-generator": "1.167.14", "@tanstack/router-plugin": "1.168.15", "@tanstack/router-utils": "1.162.2", "@tanstack/start-server-core": "1.169.10", "exsolve": "^1.0.7", "lightningcss": "^1.32.0", "pathe": "^2.0.3", "picomatch": "^4.0.3", "seroval": "^1.5.4", "source-map": "^0.7.6", "srvx": "^0.11.9", "tinyglobby": "^0.2.15", "ufo": "^1.5.4", "vitefu": "^1.1.1", "xmlbuilder2": "^4.0.3", "zod": "^4.4.3" }, "peerDependencies": { "@rsbuild/core": "^2.0.0", "vite": ">=7.0.0" }, "optionalPeers": ["@rsbuild/core", "vite"] }, "sha512-1l71rj8vIUfaazmzZXzaFkfEvS5kYIxm7a8bmPStuejO9+kfWh8oja/mzA4UMfRst+MQwfFYsin8eYoHwqgagg=="],
"@tanstack/start-server-core": ["@tanstack/start-server-core@1.169.4", "", { "dependencies": { "@tanstack/history": "1.162.0", "@tanstack/router-core": "1.171.6", "@tanstack/start-client-core": "1.170.4", "@tanstack/start-storage-context": "1.167.8", "fetchdts": "^0.1.6", "h3-v2": "npm:h3@2.0.1-rc.20", "seroval": "^1.5.4" } }, "sha512-iM3HamWRQPROuAb+22frV/+GkqG2a3rL0X14N+Y0Dt5OajrIumPuprOn9ldUXsbdg89RTBf1KoJNDPeYGOqH4g=="],
"@tanstack/start-server-core": ["@tanstack/start-server-core@1.169.10", "", { "dependencies": { "@tanstack/history": "1.162.0", "@tanstack/router-core": "1.171.10", "@tanstack/start-client-core": "1.170.8", "@tanstack/start-storage-context": "1.167.12", "fetchdts": "^0.1.6", "h3-v2": "npm:h3@2.0.1-rc.20", "seroval": "^1.5.4" } }, "sha512-wnTrG3wha06J29x4h1G0GaEhG9/ckzitMwqUJy9aHajweCFlq1RZ2VWDXjpmeHpd4SLunzik7ixdZaOBKBN6tg=="],
"@tanstack/start-storage-context": ["@tanstack/start-storage-context@1.167.8", "", { "dependencies": { "@tanstack/router-core": "1.171.6" } }, "sha512-y9T+bIIp1ihLAXyS2+r+UovSupfu4KydSXpnoeRsw/14/E0huJsX7xB/n6XXOdmDYAaJ2WGOrG9wYjzeIDuBAw=="],
"@tanstack/start-storage-context": ["@tanstack/start-storage-context@1.167.12", "", { "dependencies": { "@tanstack/router-core": "1.171.10" } }, "sha512-f5Z5nqCRlDvuHCXVDYEmFX9RAjXrdCK6pjmGUh/U9qeZdBCg4hIqJ17l1YuHQx+Jsi6su5dihTBs1NfvINpL9A=="],
"@tanstack/store": ["@tanstack/store@0.9.3", "", {}, "sha512-8reSzl/qGWGGVKhBoxXPMWzATSbZLZFWhwBAFO9NAyp0TxzfBP0mIrGb8CP8KrQTmvzXlR/vFPPUrHTLBGyFyw=="],
@@ -698,7 +692,7 @@
"@types/node": ["@types/node@25.9.1", "", { "dependencies": { "undici-types": ">=7.24.0 <7.24.7" } }, "sha512-xfrlY7UD5rMJk3ZVJP8BNzS28J36YJg+xp+LPXV1TdWxr8uMH5A860QNxYDGQe/ylDSgjxE52Q9VnO7p75tJxg=="],
"@types/react": ["@types/react@19.2.15", "", { "dependencies": { "csstype": "^3.2.2" } }, "sha512-eRwcGNHve+E8qtEQSSRl6urh+rFop4v8gm6O8rGv25CodbvFdLjA1vVQ1KkiFE0w0UPOnb8tDiFKL5lp0rtY5Q=="],
"@types/react": ["@types/react@19.2.17", "", { "dependencies": { "csstype": "^3.2.2" } }, "sha512-MXfmqaVPEVgkBT/aY0aGCkRWWtByiYQXo3xdQ8r5RzuFrPiRn8Gar2tQdXSUQ2GKV3bkXckek89V8wQBY2Q/Aw=="],
"@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="],
@@ -714,21 +708,21 @@
"@vitejs/plugin-react": ["@vitejs/plugin-react@6.0.2", "", { "dependencies": { "@rolldown/pluginutils": "^1.0.0" }, "peerDependencies": { "@rolldown/plugin-babel": "^0.1.7 || ^0.2.0", "babel-plugin-react-compiler": "^1.0.0", "vite": "^8.0.0" }, "optionalPeers": ["@rolldown/plugin-babel", "babel-plugin-react-compiler"] }, "sha512-DlSMqo4WhThw4vB8Mpn0Woe9J+Jfq1geJ61AKW0QEgLzGMNwtIMdxbDUzLxcun8W7NbJO0e2Jg/Nxm3cCSVzzg=="],
"@vitest/coverage-v8": ["@vitest/coverage-v8@4.1.7", "", { "dependencies": { "@bcoe/v8-coverage": "^1.0.2", "@vitest/utils": "4.1.7", "ast-v8-to-istanbul": "^1.0.0", "istanbul-lib-coverage": "^3.2.2", "istanbul-lib-report": "^3.0.1", "istanbul-reports": "^3.2.0", "magicast": "^0.5.2", "obug": "^2.1.1", "std-env": "^4.0.0-rc.1", "tinyrainbow": "^3.1.0" }, "peerDependencies": { "@vitest/browser": "4.1.7", "vitest": "4.1.7" }, "optionalPeers": ["@vitest/browser"] }, "sha512-qsYPeXc5Q9dFLd1i8Ap+Bx8sQgcp+rFVQo4R0dDsWNBzl26ldVF1qOO+RL24K7FDrR6pA+50XedRLSoSG24bVQ=="],
"@vitest/coverage-v8": ["@vitest/coverage-v8@4.1.8", "", { "dependencies": { "@bcoe/v8-coverage": "^1.0.2", "@vitest/utils": "4.1.8", "ast-v8-to-istanbul": "^1.0.0", "istanbul-lib-coverage": "^3.2.2", "istanbul-lib-report": "^3.0.1", "istanbul-reports": "^3.2.0", "magicast": "^0.5.2", "obug": "^2.1.1", "std-env": "^4.0.0-rc.1", "tinyrainbow": "^3.1.0" }, "peerDependencies": { "@vitest/browser": "4.1.8", "vitest": "4.1.8" }, "optionalPeers": ["@vitest/browser"] }, "sha512-lt3kovsyHwYe00wq4D1ti0Z974fWj4NLp6siqiyEufUpyFwK9Yhi7rBhac9JL5aA0zoMrJqc4vYPZRUnI7l7nw=="],
"@vitest/expect": ["@vitest/expect@4.1.7", "", { "dependencies": { "@standard-schema/spec": "^1.1.0", "@types/chai": "^5.2.2", "@vitest/spy": "4.1.7", "@vitest/utils": "4.1.7", "chai": "^6.2.2", "tinyrainbow": "^3.1.0" } }, "sha512-1R+tw0ortHEbZDGMymm+pN7/AFQ/RkFFdtd7EN+VBpynKmLbP8A3rpEXdshBJ7+8hQ9zBJh/i1s0yKNtxAnU7w=="],
"@vitest/expect": ["@vitest/expect@4.1.8", "", { "dependencies": { "@standard-schema/spec": "^1.1.0", "@types/chai": "^5.2.2", "@vitest/spy": "4.1.8", "@vitest/utils": "4.1.8", "chai": "^6.2.2", "tinyrainbow": "^3.1.0" } }, "sha512-h3nDO677RDLEGlBxyQ5CW8RlMThSKSRLUePLOx09gNIWRL40edgA1GCZSZgf1W55MFAG6/Sw14KeaAnqv0NKdQ=="],
"@vitest/mocker": ["@vitest/mocker@4.1.7", "", { "dependencies": { "@vitest/spy": "4.1.7", "estree-walker": "^3.0.3", "magic-string": "^0.30.21" }, "peerDependencies": { "msw": "^2.4.9", "vite": "^6.0.0 || ^7.0.0 || ^8.0.0" }, "optionalPeers": ["msw", "vite"] }, "sha512-vY7nuamKgfvpA1Koa3oYIw/k7D6kZnpGyNMZW8loow2bsBYla1TFdqTaXncWdRn4pgwNs+90RhnXhJScDwQeJA=="],
"@vitest/mocker": ["@vitest/mocker@4.1.8", "", { "dependencies": { "@vitest/spy": "4.1.8", "estree-walker": "^3.0.3", "magic-string": "^0.30.21" }, "peerDependencies": { "msw": "^2.4.9", "vite": "^6.0.0 || ^7.0.0 || ^8.0.0" }, "optionalPeers": ["msw", "vite"] }, "sha512-LEiN/xe4OSIbKe9HQIp5OC24agGD9J5CnmMgsLohVVoOPWL9a2sBoR6VBx43jQZb7Kr1l4RCuyCJzcAa0+dojw=="],
"@vitest/pretty-format": ["@vitest/pretty-format@4.1.7", "", { "dependencies": { "tinyrainbow": "^3.1.0" } }, "sha512-umgCarTOYQWIaDMvGDRZij+6b9oVeLIyJzfN+AS88e0ZOU3QTgNNSTtjQOpcvWr3np1N0j4WgZj+sb3oYBDscw=="],
"@vitest/pretty-format": ["@vitest/pretty-format@4.1.8", "", { "dependencies": { "tinyrainbow": "^3.1.0" } }, "sha512-9GasEBxpZ1VYIpqHf/0+YGg121uSNwCKOJqIrTwWP/TB7DmFCiaBpNl3aPZzoLWfWkuqhbH8vJIVobZkvdo2cA=="],
"@vitest/runner": ["@vitest/runner@4.1.7", "", { "dependencies": { "@vitest/utils": "4.1.7", "pathe": "^2.0.3" } }, "sha512-BapjmAQ2aI78WdMEfeUWivnfVzB+VPGwWRQcJE0OUq7qEeEcBsCSf+0T5iREBNE5nBb4wA5Ya0W6IA+sghdEFw=="],
"@vitest/runner": ["@vitest/runner@4.1.8", "", { "dependencies": { "@vitest/utils": "4.1.8", "pathe": "^2.0.3" } }, "sha512-EmVxeBAfMJvycdjd6Hm+RbFBbA9fKvo0Kx37hNpBYoYeavH3RNsBXWDooR1mgD52dCrxIIuP7UotpfiwOikvcg=="],
"@vitest/snapshot": ["@vitest/snapshot@4.1.7", "", { "dependencies": { "@vitest/pretty-format": "4.1.7", "@vitest/utils": "4.1.7", "magic-string": "^0.30.21", "pathe": "^2.0.3" } }, "sha512-ZacLzja+TmJeZ1h14xW2FB/WpeimUD3haBXQPyJqxvo8jQTmfeA8zv58mtjN2C7EHXZDYVcVYdYmAxjkWVvKCw=="],
"@vitest/snapshot": ["@vitest/snapshot@4.1.8", "", { "dependencies": { "@vitest/pretty-format": "4.1.8", "@vitest/utils": "4.1.8", "magic-string": "^0.30.21", "pathe": "^2.0.3" } }, "sha512-acfZboRmAIf05DEKcBQy33VXojFJjtUdLyo7oOmV9kebb2xdU01UknNiPuPZoJZQyO7DF0gZdTGTpeAzET9QPQ=="],
"@vitest/spy": ["@vitest/spy@4.1.7", "", {}, "sha512-kbkI5LMWakyuTIvs6fUJ5qdIVb1XVKsYJAT4OJ938cHMROYMSfmoQdZy0aaAnjbbc8F61vkoTqz/Az+/HiIu5Q=="],
"@vitest/spy": ["@vitest/spy@4.1.8", "", {}, "sha512-6EevtBp6OZOPF7bmz36HrGMeP3txgVSrgebWxHOafDXGkhIzfXK14f8KF6MuFfgXXUeHxmpD3BQxkV00/3s5mA=="],
"@vitest/utils": ["@vitest/utils@4.1.7", "", { "dependencies": { "@vitest/pretty-format": "4.1.7", "convert-source-map": "^2.0.0", "tinyrainbow": "^3.1.0" } }, "sha512-T532WBu791cBxJlCl6SO+J14l81DQx6uQHm1bQbmCDY7nqlEIgkza/UFnSBNaUtSf41unldDFjdOBYEQC4b5Hw=="],
"@vitest/utils": ["@vitest/utils@4.1.8", "", { "dependencies": { "@vitest/pretty-format": "4.1.8", "convert-source-map": "^2.0.0", "tinyrainbow": "^3.1.0" } }, "sha512-uOJamYALNhfJ6iolExyQM40yIQwDqYnkKtQ5VCiSe17E33H0aQ/u+1GlRuz4LZBk6Mm3sg90G9hEbmEt37C1Zg=="],
"ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
@@ -798,13 +792,13 @@
"comma-separated-tokens": ["comma-separated-tokens@2.0.3", "", {}, "sha512-Fu4hJdvzeylCfQPp9SGWidpzrMs7tTrlu6Vb8XGaRGck8QSNZJJp538Wrb60Lax4fPwR64ViY468OIUTbRlGZg=="],
"commander": ["commander@14.0.3", "", {}, "sha512-H+y0Jo/T1RZ9qPP4Eh1pkcQcLRglraJaSLoyOtHxu6AapkjWVCy2Sit1QQ4x3Dng8qDlSsZEet7g5Pq06MvTgw=="],
"commander": ["commander@15.0.0", "", {}, "sha512-z67u4ZhzCL/Tydu1lJARtEZYWbWaN7oYLHbsuzocr6y4N6WZAagG3RQ4FW61V1/0+jImpj293XfrcYnd1qxtPg=="],
"consola": ["consola@3.4.2", "", {}, "sha512-5IKcdX0nnYavi6G7TtOhwkYzyjfJlatbjMjuLSfE2kYT5pMDOilZ4OvMhi637CcDICTmz3wARPoyhqyX1Y+XvA=="],
"convert-source-map": ["convert-source-map@2.0.0", "", {}, "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg=="],
"convex": ["convex@1.39.1", "", { "dependencies": { "esbuild": "0.27.0", "prettier": "^3.0.0", "ws": "8.18.0" }, "peerDependencies": { "@auth0/auth0-react": "^2.0.1", "@clerk/clerk-react": "^4.12.8 || ^5.0.0", "@clerk/react": "^6.4.3", "react": "^18.0.0 || ^19.0.0-0 || ^19.0.0" }, "optionalPeers": ["@auth0/auth0-react", "@clerk/clerk-react", "@clerk/react", "react"], "bin": { "convex": "bin/main.js" } }, "sha512-W+gVXA7BpRF1xLlS1kGTtKVaqd5yonqbGESKiPtIUXjV744GdDz8IG7RVsSY5KzHbgxuJBHKaJYk+92OIHTskQ=="],
"convex": ["convex@1.40.0", "", { "dependencies": { "esbuild": "0.27.0", "prettier": "^3.0.0", "ws": "8.20.1" }, "peerDependencies": { "@auth0/auth0-react": "^2.0.1", "@clerk/clerk-react": "^4.12.8 || ^5.0.0", "@clerk/react": "^6.4.3", "react": "^18.0.0 || ^19.0.0-0 || ^19.0.0" }, "optionalPeers": ["@auth0/auth0-react", "@clerk/clerk-react", "@clerk/react", "react"], "bin": { "convex": "bin/main.js" } }, "sha512-jChWEB45q+9Ibryc7hg0l6hB1xA4zwE2y6ZhkhGP6oJkqYeiURkMagA2ZQZYMy1/T8PZ9ztoVJJtbL/+Ob851Q=="],
"convex-helpers": ["convex-helpers@0.1.118", "", { "peerDependencies": { "@standard-schema/spec": "^1.0.0", "convex": "^1.32.0", "hono": "^4.0.5", "react": "^17.0.2 || ^18.0.0 || ^19.0.0", "typescript": "^5.5 || ^6.0.0", "zod": "^3.25.0 || ^4.0.0" }, "optionalPeers": ["@standard-schema/spec", "hono", "react", "typescript", "zod"], "bin": { "convex-helpers": "bin.cjs" } }, "sha512-07t10n8CZG/YCDzOy5/WDdNNQYL+mP7VU76BLJCZrB2dvJTH7UZJxPqNrhPH+pZbW52joQ91eQHSksdcgOXebQ=="],
@@ -1020,7 +1014,7 @@
"lucia": ["lucia@3.2.2", "", { "dependencies": { "@oslojs/crypto": "^1.0.1", "@oslojs/encoding": "^1.1.0" } }, "sha512-P1FlFBGCMPMXu+EGdVD9W4Mjm0DqsusmKgO7Xc33mI5X1bklmsQb0hfzPhXomQr9waWIBDsiOjvr1e6BTaUqpA=="],
"lucide-react": ["lucide-react@1.16.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-dYwyPzb4MEKpGUmNYk3WKWPnMrHs3FKM+q94kAnJrcDIqqn1hq2xY8scaS2ovsOCM5D51ey2gaRG3PBb1vgoYQ=="],
"lucide-react": ["lucide-react@1.17.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-9FA9evdox/JQL5PT57fdA1x/yg8T7knJ98+zjTL3UfKza6pflQUUh3XtaQIHKvnsJw1lmsEyHVlt5jchYxOQ5w=="],
"lz-string": ["lz-string@1.5.0", "", { "bin": { "lz-string": "bin/bin.js" } }, "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ=="],
@@ -1162,9 +1156,9 @@
"oxc-parser": ["oxc-parser@0.120.0", "", { "dependencies": { "@oxc-project/types": "^0.120.0" }, "optionalDependencies": { "@oxc-parser/binding-android-arm-eabi": "0.120.0", "@oxc-parser/binding-android-arm64": "0.120.0", "@oxc-parser/binding-darwin-arm64": "0.120.0", "@oxc-parser/binding-darwin-x64": "0.120.0", "@oxc-parser/binding-freebsd-x64": "0.120.0", "@oxc-parser/binding-linux-arm-gnueabihf": "0.120.0", "@oxc-parser/binding-linux-arm-musleabihf": "0.120.0", "@oxc-parser/binding-linux-arm64-gnu": "0.120.0", "@oxc-parser/binding-linux-arm64-musl": "0.120.0", "@oxc-parser/binding-linux-ppc64-gnu": "0.120.0", "@oxc-parser/binding-linux-riscv64-gnu": "0.120.0", "@oxc-parser/binding-linux-riscv64-musl": "0.120.0", "@oxc-parser/binding-linux-s390x-gnu": "0.120.0", "@oxc-parser/binding-linux-x64-gnu": "0.120.0", "@oxc-parser/binding-linux-x64-musl": "0.120.0", "@oxc-parser/binding-openharmony-arm64": "0.120.0", "@oxc-parser/binding-wasm32-wasi": "0.120.0", "@oxc-parser/binding-win32-arm64-msvc": "0.120.0", "@oxc-parser/binding-win32-ia32-msvc": "0.120.0", "@oxc-parser/binding-win32-x64-msvc": "0.120.0" } }, "sha512-WyPWZlcIm+Fkte63FGfgFB8mAAk33aH9h5N9lphXVOHSXEBFFsmYdOBedVKly363aWABjZdaj/m9lBfEY4wt+w=="],
"oxfmt": ["oxfmt@0.51.0", "", { "dependencies": { "tinypool": "2.1.0" }, "optionalDependencies": { "@oxfmt/binding-android-arm-eabi": "0.51.0", "@oxfmt/binding-android-arm64": "0.51.0", "@oxfmt/binding-darwin-arm64": "0.51.0", "@oxfmt/binding-darwin-x64": "0.51.0", "@oxfmt/binding-freebsd-x64": "0.51.0", "@oxfmt/binding-linux-arm-gnueabihf": "0.51.0", "@oxfmt/binding-linux-arm-musleabihf": "0.51.0", "@oxfmt/binding-linux-arm64-gnu": "0.51.0", "@oxfmt/binding-linux-arm64-musl": "0.51.0", "@oxfmt/binding-linux-ppc64-gnu": "0.51.0", "@oxfmt/binding-linux-riscv64-gnu": "0.51.0", "@oxfmt/binding-linux-riscv64-musl": "0.51.0", "@oxfmt/binding-linux-s390x-gnu": "0.51.0", "@oxfmt/binding-linux-x64-gnu": "0.51.0", "@oxfmt/binding-linux-x64-musl": "0.51.0", "@oxfmt/binding-openharmony-arm64": "0.51.0", "@oxfmt/binding-win32-arm64-msvc": "0.51.0", "@oxfmt/binding-win32-ia32-msvc": "0.51.0", "@oxfmt/binding-win32-x64-msvc": "0.51.0" }, "peerDependencies": { "svelte": "^5.0.0" }, "optionalPeers": ["svelte"], "bin": { "oxfmt": "bin/oxfmt" } }, "sha512-l/AoAnaEOV7Q5/Z9kHOMDehVJnCgYN7wRoooWCTUMBMi16BJhLZqd9cmCnwcVFfVlzkt53zK2KLPFNp8vSsoDg=="],
"oxfmt": ["oxfmt@0.53.0", "", { "dependencies": { "tinypool": "2.1.0" }, "optionalDependencies": { "@oxfmt/binding-android-arm-eabi": "0.53.0", "@oxfmt/binding-android-arm64": "0.53.0", "@oxfmt/binding-darwin-arm64": "0.53.0", "@oxfmt/binding-darwin-x64": "0.53.0", "@oxfmt/binding-freebsd-x64": "0.53.0", "@oxfmt/binding-linux-arm-gnueabihf": "0.53.0", "@oxfmt/binding-linux-arm-musleabihf": "0.53.0", "@oxfmt/binding-linux-arm64-gnu": "0.53.0", "@oxfmt/binding-linux-arm64-musl": "0.53.0", "@oxfmt/binding-linux-ppc64-gnu": "0.53.0", "@oxfmt/binding-linux-riscv64-gnu": "0.53.0", "@oxfmt/binding-linux-riscv64-musl": "0.53.0", "@oxfmt/binding-linux-s390x-gnu": "0.53.0", "@oxfmt/binding-linux-x64-gnu": "0.53.0", "@oxfmt/binding-linux-x64-musl": "0.53.0", "@oxfmt/binding-openharmony-arm64": "0.53.0", "@oxfmt/binding-win32-arm64-msvc": "0.53.0", "@oxfmt/binding-win32-ia32-msvc": "0.53.0", "@oxfmt/binding-win32-x64-msvc": "0.53.0" }, "peerDependencies": { "svelte": "^5.0.0", "vite-plus": "*" }, "optionalPeers": ["svelte", "vite-plus"], "bin": { "oxfmt": "bin/oxfmt" } }, "sha512-9cB5glS3Ip6NMuZ+6NYTao9FCWkDhRtPYCtR3QBu/NxHoFbgzzTvi41N4jxz/GqGfuLKspui1qb/LlSu2IbMcw=="],
"oxlint": ["oxlint@1.66.0", "", { "optionalDependencies": { "@oxlint/binding-android-arm-eabi": "1.66.0", "@oxlint/binding-android-arm64": "1.66.0", "@oxlint/binding-darwin-arm64": "1.66.0", "@oxlint/binding-darwin-x64": "1.66.0", "@oxlint/binding-freebsd-x64": "1.66.0", "@oxlint/binding-linux-arm-gnueabihf": "1.66.0", "@oxlint/binding-linux-arm-musleabihf": "1.66.0", "@oxlint/binding-linux-arm64-gnu": "1.66.0", "@oxlint/binding-linux-arm64-musl": "1.66.0", "@oxlint/binding-linux-ppc64-gnu": "1.66.0", "@oxlint/binding-linux-riscv64-gnu": "1.66.0", "@oxlint/binding-linux-riscv64-musl": "1.66.0", "@oxlint/binding-linux-s390x-gnu": "1.66.0", "@oxlint/binding-linux-x64-gnu": "1.66.0", "@oxlint/binding-linux-x64-musl": "1.66.0", "@oxlint/binding-openharmony-arm64": "1.66.0", "@oxlint/binding-win32-arm64-msvc": "1.66.0", "@oxlint/binding-win32-ia32-msvc": "1.66.0", "@oxlint/binding-win32-x64-msvc": "1.66.0" }, "peerDependencies": { "oxlint-tsgolint": ">=0.22.1" }, "optionalPeers": ["oxlint-tsgolint"], "bin": { "oxlint": "bin/oxlint" } }, "sha512-N4LLxYLd94KEBqXDMDM5f+2PUpItTjDLreXe2Gn5KhjhCK4Qp2YUXaBi8Yu325ryOgKwt22m45fpD7nPOn69Yw=="],
"oxlint": ["oxlint@1.68.0", "", { "optionalDependencies": { "@oxlint/binding-android-arm-eabi": "1.68.0", "@oxlint/binding-android-arm64": "1.68.0", "@oxlint/binding-darwin-arm64": "1.68.0", "@oxlint/binding-darwin-x64": "1.68.0", "@oxlint/binding-freebsd-x64": "1.68.0", "@oxlint/binding-linux-arm-gnueabihf": "1.68.0", "@oxlint/binding-linux-arm-musleabihf": "1.68.0", "@oxlint/binding-linux-arm64-gnu": "1.68.0", "@oxlint/binding-linux-arm64-musl": "1.68.0", "@oxlint/binding-linux-ppc64-gnu": "1.68.0", "@oxlint/binding-linux-riscv64-gnu": "1.68.0", "@oxlint/binding-linux-riscv64-musl": "1.68.0", "@oxlint/binding-linux-s390x-gnu": "1.68.0", "@oxlint/binding-linux-x64-gnu": "1.68.0", "@oxlint/binding-linux-x64-musl": "1.68.0", "@oxlint/binding-openharmony-arm64": "1.68.0", "@oxlint/binding-win32-arm64-msvc": "1.68.0", "@oxlint/binding-win32-ia32-msvc": "1.68.0", "@oxlint/binding-win32-x64-msvc": "1.68.0" }, "peerDependencies": { "oxlint-tsgolint": ">=0.22.1", "vite-plus": "*" }, "optionalPeers": ["oxlint-tsgolint", "vite-plus"], "bin": { "oxlint": "bin/oxlint" } }, "sha512-dXcbq+xsmLrMy6T8d0euf3IYUfLmjHIE11pOxiUSi5LHkFZaYPv568R6sEjcavVpUxoaQe66UBuK4HEi74NxpA=="],
"oxlint-tsgolint": ["oxlint-tsgolint@0.23.0", "", { "optionalDependencies": { "@oxlint-tsgolint/darwin-arm64": "0.23.0", "@oxlint-tsgolint/darwin-x64": "0.23.0", "@oxlint-tsgolint/linux-arm64": "0.23.0", "@oxlint-tsgolint/linux-x64": "0.23.0", "@oxlint-tsgolint/win32-arm64": "0.23.0", "@oxlint-tsgolint/win32-x64": "0.23.0" }, "bin": { "tsgolint": "bin/tsgolint.js" } }, "sha512-3mBv3CoPbh8dFbzfDGIWa2ytZjn2v+3EX4aKRXjIhsoGFzG8GCjfRirz3rwZf1wYbZzsNLTSgpw8VjQuWdp/jA=="],
@@ -1200,9 +1194,9 @@
"punycode": ["punycode@2.3.1", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="],
"react": ["react@19.2.6", "", {}, "sha512-sfWGGfavi0xr8Pg0sVsyHMAOziVYKgPLNrS7ig+ivMNb3wbCBw3KxtflsGBAwD3gYQlE/AEZsTLgToRrSCjb0Q=="],
"react": ["react@19.2.7", "", {}, "sha512-HNe9WslTbXmFK8o8cmwgAeJFSBvt1bPdHCVKtaaV+WlAN36mpT4hcRpwbf3fY56ar2oIXzsBpOAiIRHAdY0OlQ=="],
"react-dom": ["react-dom@19.2.6", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.6" } }, "sha512-0prMI+hvBbPjsWnxDLxlCGyM8PN6UuWjEUCYmZhO67xIV9Xasa/r/vDnq+Xyq4Lo27g8QSbO5YzARu0D1Sps3g=="],
"react-dom": ["react-dom@19.2.7", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.7" } }, "sha512-t0BRVXvbiE/o20Hfw669rLbMCDWtYZLvmJigy2f0MxsXF+71pxhR3xOkspmsO8h3ZlNzyibAmtCa3l4lYKk6gQ=="],
"react-is": ["react-is@17.0.2", "", {}, "sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w=="],
@@ -1246,7 +1240,7 @@
"scheduler": ["scheduler@0.27.0", "", {}, "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q=="],
"semver": ["semver@7.8.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-rkVq3IXh+4FDGch+KwzX3aV9W3kO54GyEgpvBzSyctDA6Xtd7RJQV1xmXbeQp5v7+VzLOfVqiutSE6GICgPFvg=="],
"semver": ["semver@7.8.2", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-c8jsqUZm3omBOI66G90z1Dyw5z622G8oLG+omfsHBJf3CWQTlOcwOjvOG6wtiNfW6anKm/eA39LMwMtMez2TiQ=="],
"seroval": ["seroval@1.5.4", "", {}, "sha512-46uFvgrXTVxZcUorgSSRZ4y+ieqLLQRMlG4bnCZKW3qI6BZm7Rg4ntMW4p1mILEEBZWrFlcpp0AyIIlM6jD9iw=="],
@@ -1256,7 +1250,7 @@
"shell-quote": ["shell-quote@1.8.3", "", {}, "sha512-ObmnIF4hXNg1BqhnHmgbDETF8dLPCggZWBjkQfhZpbszZnYur5DUljTcCHii5LC3J5E0yeO/1LIMyH+UvHQgyw=="],
"shiki": ["shiki@4.1.0", "", { "dependencies": { "@shikijs/core": "4.1.0", "@shikijs/engine-javascript": "4.1.0", "@shikijs/engine-oniguruma": "4.1.0", "@shikijs/langs": "4.1.0", "@shikijs/themes": "4.1.0", "@shikijs/types": "4.1.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-l/ABZPUR5v70jI10EzqfMS/I96vjSGv2y0ihUV+WYFzv0EfvW4s54m0Lg8wCrrL+2IkwBzFTuxkZjPf8b2NX9Q=="],
"shiki": ["shiki@4.2.0", "", { "dependencies": { "@shikijs/core": "4.2.0", "@shikijs/engine-javascript": "4.2.0", "@shikijs/engine-oniguruma": "4.2.0", "@shikijs/langs": "4.2.0", "@shikijs/themes": "4.2.0", "@shikijs/types": "4.2.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-hjNax6o/ylDy9lefQEaSDtzaT3iVNtZ3WmpQnbuQNoG4xvnSKf2kSKbihZVO4JRG1TTMejs7CmNRYlWgAL66pQ=="],
"siginfo": ["siginfo@2.0.0", "", {}, "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g=="],
@@ -1306,7 +1300,7 @@
"tinyexec": ["tinyexec@1.1.2", "", {}, "sha512-dAqSqE/RabpBKI8+h26GfLq6Vb3JVXs30XYQjdMjaj/c2tS8IYYMbIzP599KtRj7c57/wYApb3QjgRgXmrCukA=="],
"tinyglobby": ["tinyglobby@0.2.16", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg=="],
"tinyglobby": ["tinyglobby@0.2.17", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g=="],
"tinypool": ["tinypool@2.1.0", "", {}, "sha512-Pugqs6M0m7Lv1I7FtxN4aoyToKg1C4tu+/381vH35y8oENM/Ai7f7C4StcoK4/+BSw9ebcS8jRiVrORFKCALLw=="],
@@ -1334,7 +1328,7 @@
"ufo": ["ufo@1.6.4", "", {}, "sha512-JFNbkD1Svwe0KvGi8GOeLcP4kAWQ609twvCdcHxq1oSL8svv39ZuSvajcD8B+5D0eL4+s1Is2D/O6KN3qcTeRA=="],
"undici": ["undici@7.26.0", "", {}, "sha512-3O9Tf67pGhgOv9jM35AbhkXAKi13f3oy3aE4CSgr+TckGeY+/iu97ZXN+J7DpHPzLbVApFd1IFhcnBjREYXYcg=="],
"undici": ["undici@7.27.1", "", {}, "sha512-UDdpiex+mzigiyrXrGbiUaF4HzTNhKbh2vRNFaTMzcqmLIPrZxaCtwo/1TMSuWoM1Xz3WiTo9KdgI3kRqYzJGg=="],
"undici-types": ["undici-types@7.24.6", "", {}, "sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg=="],
@@ -1370,11 +1364,11 @@
"vfile-message": ["vfile-message@4.0.3", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-stringify-position": "^4.0.0" } }, "sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw=="],
"vite": ["vite@8.0.14", "", { "dependencies": { "lightningcss": "^1.32.0", "picomatch": "^4.0.4", "postcss": "^8.5.15", "rolldown": "1.0.2", "tinyglobby": "^0.2.16" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "@vitejs/devtools": "^0.1.18", "esbuild": "^0.27.0 || ^0.28.0", "jiti": ">=1.21.0", "less": "^4.0.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "@vitejs/devtools", "esbuild", "jiti", "less", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-s4BJJ+5y1pYL6Otw51FHhVJQhPnuRinKig64g/1+EUNaJsd3gCKdD31IPFvswUgW9/60QT9oFHbZHbQK5imcxw=="],
"vite": ["vite@8.0.16", "", { "dependencies": { "lightningcss": "^1.32.0", "picomatch": "^4.0.4", "postcss": "^8.5.15", "rolldown": "1.0.3", "tinyglobby": "^0.2.17" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "@vitejs/devtools": "^0.1.18", "esbuild": "^0.27.0 || ^0.28.0", "jiti": ">=1.21.0", "less": "^4.0.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "@vitejs/devtools", "esbuild", "jiti", "less", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-h9bXPmJichP5fLmVQo3PyaGSDE2n3aPuomeAlVRm0JLmt4rY6zmPKd59HYI4LNW8oTK7tlTsuC7l/m7awx9Jcw=="],
"vitefu": ["vitefu@1.1.3", "", { "peerDependencies": { "vite": "^3.0.0 || ^4.0.0 || ^5.0.0 || ^6.0.0 || ^7.0.0 || ^8.0.0" }, "optionalPeers": ["vite"] }, "sha512-ub4okH7Z5KLjb6hDyjqrGXqWtWvoYdU3IGm/NorpgHncKoLTCfRIbvlhBm7r0YstIaQRYlp4yEbFqDcKSzXSSg=="],
"vitest": ["vitest@4.1.7", "", { "dependencies": { "@vitest/expect": "4.1.7", "@vitest/mocker": "4.1.7", "@vitest/pretty-format": "4.1.7", "@vitest/runner": "4.1.7", "@vitest/snapshot": "4.1.7", "@vitest/spy": "4.1.7", "@vitest/utils": "4.1.7", "es-module-lexer": "^2.0.0", "expect-type": "^1.3.0", "magic-string": "^0.30.21", "obug": "^2.1.1", "pathe": "^2.0.3", "picomatch": "^4.0.3", "std-env": "^4.0.0-rc.1", "tinybench": "^2.9.0", "tinyexec": "^1.0.2", "tinyglobby": "^0.2.15", "tinyrainbow": "^3.1.0", "vite": "^6.0.0 || ^7.0.0 || ^8.0.0", "why-is-node-running": "^2.3.0" }, "peerDependencies": { "@edge-runtime/vm": "*", "@opentelemetry/api": "^1.9.0", "@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0", "@vitest/browser-playwright": "4.1.7", "@vitest/browser-preview": "4.1.7", "@vitest/browser-webdriverio": "4.1.7", "@vitest/coverage-istanbul": "4.1.7", "@vitest/coverage-v8": "4.1.7", "@vitest/ui": "4.1.7", "happy-dom": "*", "jsdom": "*" }, "optionalPeers": ["@edge-runtime/vm", "@opentelemetry/api", "@types/node", "@vitest/browser-playwright", "@vitest/browser-preview", "@vitest/browser-webdriverio", "@vitest/coverage-istanbul", "@vitest/coverage-v8", "@vitest/ui", "happy-dom", "jsdom"], "bin": { "vitest": "vitest.mjs" } }, "sha512-flYyaFd2CgoCoU+0UKt3pxksgC+S02iTDN0n3LtqaMeXsI9SBcdNujc2k0DeFLzUn/0k538yNjOSdwgCqcrwJA=="],
"vitest": ["vitest@4.1.8", "", { "dependencies": { "@vitest/expect": "4.1.8", "@vitest/mocker": "4.1.8", "@vitest/pretty-format": "4.1.8", "@vitest/runner": "4.1.8", "@vitest/snapshot": "4.1.8", "@vitest/spy": "4.1.8", "@vitest/utils": "4.1.8", "es-module-lexer": "^2.0.0", "expect-type": "^1.3.0", "magic-string": "^0.30.21", "obug": "^2.1.1", "pathe": "^2.0.3", "picomatch": "^4.0.3", "std-env": "^4.0.0-rc.1", "tinybench": "^2.9.0", "tinyexec": "^1.0.2", "tinyglobby": "^0.2.15", "tinyrainbow": "^3.1.0", "vite": "^6.0.0 || ^7.0.0 || ^8.0.0", "why-is-node-running": "^2.3.0" }, "peerDependencies": { "@edge-runtime/vm": "*", "@opentelemetry/api": "^1.9.0", "@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0", "@vitest/browser-playwright": "4.1.8", "@vitest/browser-preview": "4.1.8", "@vitest/browser-webdriverio": "4.1.8", "@vitest/coverage-istanbul": "4.1.8", "@vitest/coverage-v8": "4.1.8", "@vitest/ui": "4.1.8", "happy-dom": "*", "jsdom": "*" }, "optionalPeers": ["@edge-runtime/vm", "@opentelemetry/api", "@types/node", "@vitest/browser-playwright", "@vitest/browser-preview", "@vitest/browser-webdriverio", "@vitest/coverage-istanbul", "@vitest/coverage-v8", "@vitest/ui", "happy-dom", "jsdom"], "bin": { "vitest": "vitest.mjs" } }, "sha512-flY6ScbCIt9HThs+C5HS7jvGOB560DJtk/Z15IQROTA6zEy49Nh8T/dofWTQL+n3vswqn87sbJNiuqw1SDp5Ig=="],
"w3c-xmlserializer": ["w3c-xmlserializer@5.0.0", "", { "dependencies": { "xml-name-validator": "^5.0.0" } }, "sha512-o8qghlI8NZHU1lLPrpi2+Uq7abh4GGPpYANlalzWxyWteJOCsr/P+oPBA49TOLu5FTZO4d3F9MnWJfiMo4BkmA=="],
@@ -1510,7 +1504,7 @@
"strip-ansi/ansi-regex": ["ansi-regex@6.2.2", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="],
"vite/rolldown": ["rolldown@1.0.2", "", { "dependencies": { "@oxc-project/types": "=0.132.0", "@rolldown/pluginutils": "^1.0.0" }, "optionalDependencies": { "@rolldown/binding-android-arm64": "1.0.2", "@rolldown/binding-darwin-arm64": "1.0.2", "@rolldown/binding-darwin-x64": "1.0.2", "@rolldown/binding-freebsd-x64": "1.0.2", "@rolldown/binding-linux-arm-gnueabihf": "1.0.2", "@rolldown/binding-linux-arm64-gnu": "1.0.2", "@rolldown/binding-linux-arm64-musl": "1.0.2", "@rolldown/binding-linux-ppc64-gnu": "1.0.2", "@rolldown/binding-linux-s390x-gnu": "1.0.2", "@rolldown/binding-linux-x64-gnu": "1.0.2", "@rolldown/binding-linux-x64-musl": "1.0.2", "@rolldown/binding-openharmony-arm64": "1.0.2", "@rolldown/binding-wasm32-wasi": "1.0.2", "@rolldown/binding-win32-arm64-msvc": "1.0.2", "@rolldown/binding-win32-x64-msvc": "1.0.2" }, "bin": { "rolldown": "./bin/cli.mjs" } }, "sha512-oZx5zVDtVB44AW3eaifgDml1gWRDZGvjcfdxonE4swNPG98PrrXjaO/KrnUjzlMnztCCRVlUueA1kCXhARGk6g=="],
"vite/rolldown": ["rolldown@1.0.3", "", { "dependencies": { "@oxc-project/types": "=0.133.0", "@rolldown/pluginutils": "^1.0.0" }, "optionalDependencies": { "@rolldown/binding-android-arm64": "1.0.3", "@rolldown/binding-darwin-arm64": "1.0.3", "@rolldown/binding-darwin-x64": "1.0.3", "@rolldown/binding-freebsd-x64": "1.0.3", "@rolldown/binding-linux-arm-gnueabihf": "1.0.3", "@rolldown/binding-linux-arm64-gnu": "1.0.3", "@rolldown/binding-linux-arm64-musl": "1.0.3", "@rolldown/binding-linux-ppc64-gnu": "1.0.3", "@rolldown/binding-linux-s390x-gnu": "1.0.3", "@rolldown/binding-linux-x64-gnu": "1.0.3", "@rolldown/binding-linux-x64-musl": "1.0.3", "@rolldown/binding-openharmony-arm64": "1.0.3", "@rolldown/binding-wasm32-wasi": "1.0.3", "@rolldown/binding-win32-arm64-msvc": "1.0.3", "@rolldown/binding-win32-x64-msvc": "1.0.3" }, "bin": { "rolldown": "./bin/cli.mjs" } }, "sha512-i00lAJ2ks1BYr7rjNjKC7BcqAS7nVfiT3QX1SI5aY+AFHblCmaUf9OE9dbdzDvW6dJxbi2ZCZiy9v3CcwOiX3g=="],
"@radix-ui/react-arrow/@radix-ui/react-primitive/@radix-ui/react-slot": ["@radix-ui/react-slot@1.2.3", "", { "dependencies": { "@radix-ui/react-compose-refs": "1.1.2" }, "peerDependencies": { "@types/react": "*", "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-aeNmHnBxbi2St0au6VBVC7JXFlhLlOnvIIlePNniyUNAClzmtAUEY8/pBiK3iHjufOlwA+c20/8jngo7xcrg8A=="],
@@ -1536,36 +1530,36 @@
"hast-util-raw/parse5/entities": ["entities@6.0.1", "", {}, "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g=="],
"vite/rolldown/@oxc-project/types": ["@oxc-project/types@0.132.0", "", {}, "sha512-FESMOxil5Se014ui/Eq8fT5uHJo6nIRwH0PfJrZJXs6Gek3ZVFOrpUv3YIZT20m+extU98Hg1Ym72U58rlsxUQ=="],
"vite/rolldown/@oxc-project/types": ["@oxc-project/types@0.133.0", "", {}, "sha512-KzkdCd6Uxqnf6l3HOw1xfatAlUURA0g14cvBYFyJ5SaNOQbOUvBr9PKArcPcrNIeRsBdgcUzOGrhKveVpvOIGA=="],
"vite/rolldown/@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.0.2", "", { "os": "android", "cpu": "arm64" }, "sha512-ZS4D1JPGn/MYQN/SYDWftIE/nVsM8j/AFOYEzAoOE2O3NktQOZru+/vYXGbR/qtdLdIfGCP0lcoJiYVzsEz+iQ=="],
"vite/rolldown/@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.0.3", "", { "os": "android", "cpu": "arm64" }, "sha512-454rs7jHngixp/NMxd5srYD57OnzSlZ/eFTETjORQHLwJG1lRtmNOJcBerZlfu4GjKqeq8aCCIQrMdHyhI51Hw=="],
"vite/rolldown/@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.0.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-vdFA9+C/rekyGce7WqHs/xoT0ioZEWaOFyZLIV1mEeNFaFDUQrPIo8Vs2GvJ6eetb3rzDUtUBgzto3ExpXJB3w=="],
"vite/rolldown/@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.0.3", "", { "os": "darwin", "cpu": "arm64" }, "sha512-PcAhP+ynjURNyy8SKGl5DQP94aGuB/7JrXJb/t7P+hanXvQVMWzUvRRhBAcg/lNRadBhoUPqSoP4xw5tR/KBEA=="],
"vite/rolldown/@rolldown/binding-darwin-x64": ["@rolldown/binding-darwin-x64@1.0.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-BewSOwTHazv77DTYiAZXSqqKZ4KP/KonFisDMVU7PImxoWfB2aepnPhd2E4SWz3zDzYgDNbs6jBmTdgNnF02GA=="],
"vite/rolldown/@rolldown/binding-darwin-x64": ["@rolldown/binding-darwin-x64@1.0.3", "", { "os": "darwin", "cpu": "x64" }, "sha512-9YpfeUvSE2RS7wysJ81uOZkXJz7f7Q55H2Gvp3VEw/EsahqDtrphrZ0EwDLK5vvKOzaCrBsjF8JmnMLcUt78Gg=="],
"vite/rolldown/@rolldown/binding-freebsd-x64": ["@rolldown/binding-freebsd-x64@1.0.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-m41o7M0YWtUdqk61Tb+jnKb2rN++iRdIASlExkUoKfIAH30DOHCB8fVLzSUpbWHHU8esmEioY62PxzexE8MBuA=="],
"vite/rolldown/@rolldown/binding-freebsd-x64": ["@rolldown/binding-freebsd-x64@1.0.3", "", { "os": "freebsd", "cpu": "x64" }, "sha512-yB1IlAsSNHncV6SCTL27/MVGR5htvQsoGxIv5KMGXALp+Ll1wYsn+x98M9MW7qa+NdSbvrrY7ANI4wLJ0n1e6g=="],
"vite/rolldown/@rolldown/binding-linux-arm-gnueabihf": ["@rolldown/binding-linux-arm-gnueabihf@1.0.2", "", { "os": "linux", "cpu": "arm" }, "sha512-jcojB9H7W/jS29pMKWAK1N+fU99vXodHDTatS3b3y/XSOCiHo0kkA74pL3jJmkoQtYpOCxDvaKs1fo2Ij/1X5w=="],
"vite/rolldown/@rolldown/binding-linux-arm-gnueabihf": ["@rolldown/binding-linux-arm-gnueabihf@1.0.3", "", { "os": "linux", "cpu": "arm" }, "sha512-Yi30IVAAfLUCy2MseFjbB1jAMDl1VMCAas5StnYp8da9+CKvMd2H2cbEjWcw5NPaPqzvYkVIaF1nNUG+b7u/sw=="],
"vite/rolldown/@rolldown/binding-linux-arm64-gnu": ["@rolldown/binding-linux-arm64-gnu@1.0.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-1jn6qDU5iiOgFgygDzKUuKP0maTi0/f1+sBLgvij/76C77Nm3ts6ufz9Bjg5q5dduxiUIxtq86JIoBvo1xQ4Ig=="],
"vite/rolldown/@rolldown/binding-linux-arm64-gnu": ["@rolldown/binding-linux-arm64-gnu@1.0.3", "", { "os": "linux", "cpu": "arm64" }, "sha512-jsO7R8To+AdlYgUmN5sHSCZbfhtMBkO0WUx8iORQnPcMMdgr7qM2DQmMwgabs3GhNztdmoKkMKQFHD6DTMCIQw=="],
"vite/rolldown/@rolldown/binding-linux-arm64-musl": ["@rolldown/binding-linux-arm64-musl@1.0.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-QVLO/czFMdoMFSqlX3bcswcJNm/23r+qoa/jgtmFc/qEp6/jXmIkDjF/XIo8dPfGaiwy1xfQn8o77L79GeXFgw=="],
"vite/rolldown/@rolldown/binding-linux-arm64-musl": ["@rolldown/binding-linux-arm64-musl@1.0.3", "", { "os": "linux", "cpu": "arm64" }, "sha512-VWkUHwWriDciit80wleYwKILoR/KMvxh/IdwS/paX+ZgpuRpCrKLUdadJbc0NpBEiyhpYawsJ73j9aCvOH+f7Q=="],
"vite/rolldown/@rolldown/binding-linux-ppc64-gnu": ["@rolldown/binding-linux-ppc64-gnu@1.0.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-hgO5Abm0w5UL6FEa2iFnZqo2KlK7TQ5QhV5x09hujBf7t5KzHQ1VmfPuTpqRy/rNlSxua3eWH374xxiVrP+lcA=="],
"vite/rolldown/@rolldown/binding-linux-ppc64-gnu": ["@rolldown/binding-linux-ppc64-gnu@1.0.3", "", { "os": "linux", "cpu": "ppc64" }, "sha512-5f1laC0SlIR0yDbFCd8acUhvJIag6N3zC5P7oUPN6wX0aOma+uKJ0wBDH5aq7I1PVI2ttTlhJwzwRIBnLiSGEg=="],
"vite/rolldown/@rolldown/binding-linux-s390x-gnu": ["@rolldown/binding-linux-s390x-gnu@1.0.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-fy8rXxuYEu602abC8MUNaPjYLIFzReOaEIEMKMUa0rFEUxNpVXhs15KSSQ4qlqSaM7B6rcj9rDZgADh/IGDzLQ=="],
"vite/rolldown/@rolldown/binding-linux-s390x-gnu": ["@rolldown/binding-linux-s390x-gnu@1.0.3", "", { "os": "linux", "cpu": "s390x" }, "sha512-Iq4ko0r4XsgbrF/LunNgHtAGLRRVE2kXonAXQ/MV0mC6jQpMOhW1SvtZja2EhC/kd05++bP78dsqBeIQyYJ6Yg=="],
"vite/rolldown/@rolldown/binding-linux-x64-gnu": ["@rolldown/binding-linux-x64-gnu@1.0.2", "", { "os": "linux", "cpu": "x64" }, "sha512-0+bOkiQ779+r1WpoHOWHqncvyySci0vKph+myNDYb+im6meJAzHQXay6oEgnkHuUGouM1LKTZwqKpBow6Kj7CQ=="],
"vite/rolldown/@rolldown/binding-linux-x64-gnu": ["@rolldown/binding-linux-x64-gnu@1.0.3", "", { "os": "linux", "cpu": "x64" }, "sha512-B8m6tD5+/N5FeNQFbKlLA/2yVq9ycQP1SeedyEYYKWBNR3ZQbkvIUcNnDNM03lO1l5F2roiiFJGgvoLLyZXtSg=="],
"vite/rolldown/@rolldown/binding-linux-x64-musl": ["@rolldown/binding-linux-x64-musl@1.0.2", "", { "os": "linux", "cpu": "x64" }, "sha512-mjSkrzZK5Qsl0a9d1JgILOiuZOSDTVdKENcSXBoqbzSrspLR/4/IRVDo5wd2GgZjNss/viBFJdeq+j7qH2nypw=="],
"vite/rolldown/@rolldown/binding-linux-x64-musl": ["@rolldown/binding-linux-x64-musl@1.0.3", "", { "os": "linux", "cpu": "x64" }, "sha512-pSdpdUJHkuCxun9LE7jvgUB9qsRgaiyNNCX7m/AvHTcq67AiT/Yhoxvw5zPfhrM8k/BfP8ce/hMOpthKDpEUow=="],
"vite/rolldown/@rolldown/binding-openharmony-arm64": ["@rolldown/binding-openharmony-arm64@1.0.2", "", { "os": "none", "cpu": "arm64" }, "sha512-1v5vHasdfQAZoEHakBV72LIFAC9JjnymsiKxp+GEr/ma3+NJCPSaYK+qavInOovJkgwFrs7GccX2d6IgDA3Z5w=="],
"vite/rolldown/@rolldown/binding-openharmony-arm64": ["@rolldown/binding-openharmony-arm64@1.0.3", "", { "os": "none", "cpu": "arm64" }, "sha512-OXXS3RKJgX2uLwM+gYyuH5omcH8fL1LJs96pZGgtetVCahON57+d4SJHzTgZiOjxgGkSnpXpOsWuPDGAKAigEg=="],
"vite/rolldown/@rolldown/binding-wasm32-wasi": ["@rolldown/binding-wasm32-wasi@1.0.2", "", { "dependencies": { "@emnapi/core": "1.10.0", "@emnapi/runtime": "1.10.0", "@napi-rs/wasm-runtime": "^1.1.4" }, "cpu": "none" }, "sha512-mb1VobWn6NheziTk5/WEaR6AKVbrwT5sOi6C7zk3gy/pD1qtJfU1j4PgTo2NJnOtbL9Dl3Aeei8w9jJ7qC2jZQ=="],
"vite/rolldown/@rolldown/binding-wasm32-wasi": ["@rolldown/binding-wasm32-wasi@1.0.3", "", { "dependencies": { "@emnapi/core": "1.10.0", "@emnapi/runtime": "1.10.0", "@napi-rs/wasm-runtime": "^1.1.4" }, "cpu": "none" }, "sha512-JTtb8BWFynicNSoPrehsCzBtOKjZ6jhMiPFEmOiuXg1Fl8dn2KHQob+GuPSGR0dryQa1PQJbzjF3dqO/whhjLg=="],
"vite/rolldown/@rolldown/binding-win32-arm64-msvc": ["@rolldown/binding-win32-arm64-msvc@1.0.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-SqKonF56vA/L2yHwHYcEp2P34URpOZ7d1fS635cTkpDnUtEGdUbhI6NzsPdqeSWvAAeGDrxjWjNmibDIdFf9/A=="],
"vite/rolldown/@rolldown/binding-win32-arm64-msvc": ["@rolldown/binding-win32-arm64-msvc@1.0.3", "", { "os": "win32", "cpu": "arm64" }, "sha512-gEdFFEN70A/jxb2svrWsN3aDL7OUtmvlOy+6fa2jxG8K0wQ1ZbdeLGnidov6Yu5/733dI5ySfzFlQ/cb0bSz1g=="],
"vite/rolldown/@rolldown/binding-win32-x64-msvc": ["@rolldown/binding-win32-x64-msvc@1.0.2", "", { "os": "win32", "cpu": "x64" }, "sha512-v7qRI7gXLRINcOGXt+7YmAZ6iFuyZVMIoXAxhd8oP+DR9dLfL9GfNIx7PLMxmhZdvq8waUJBQiWN9EKNy+TRBQ=="],
"vite/rolldown/@rolldown/binding-win32-x64-msvc": ["@rolldown/binding-win32-x64-msvc@1.0.3", "", { "os": "win32", "cpu": "x64" }, "sha512-eXB7CHuaQdqmJcc3koCNtNPmT/bj2gc999kUFgBxG8Ac0NdgXc4rkCHhqrgrhN3zddvvvrgzj1e90SuSfmyIXA=="],
}
}
+26
View File
@@ -17,14 +17,18 @@ import type * as crons from "../crons.js";
import type * as depRegistryScan from "../depRegistryScan.js";
import type * as devSeed from "../devSeed.js";
import type * as devSeedExtra from "../devSeedExtra.js";
import type * as downloadMetrics from "../downloadMetrics.js";
import type * as downloads from "../downloads.js";
import type * as functions from "../functions.js";
import type * as githubAccountAgeBackfill from "../githubAccountAgeBackfill.js";
import type * as githubBackups from "../githubBackups.js";
import type * as githubBackupsNode from "../githubBackupsNode.js";
import type * as githubIdentity from "../githubIdentity.js";
import type * as githubImport from "../githubImport.js";
import type * as githubRestore from "../githubRestore.js";
import type * as githubRestoreMutations from "../githubRestoreMutations.js";
import type * as githubSkillSources from "../githubSkillSources.js";
import type * as githubSkillSync from "../githubSkillSync.js";
import type * as githubSoulBackups from "../githubSoulBackups.js";
import type * as githubSoulBackupsNode from "../githubSoulBackupsNode.js";
import type * as http from "../http.js";
@@ -54,25 +58,30 @@ import type * as lib_commentScamPrompt from "../lib/commentScamPrompt.js";
import type * as lib_contentTypes from "../lib/contentTypes.js";
import type * as lib_depRegistryScan from "../lib/depRegistryScan.js";
import type * as lib_devAuth from "../lib/devAuth.js";
import type * as lib_devSeed from "../lib/devSeed.js";
import type * as lib_embeddingVisibility from "../lib/embeddingVisibility.js";
import type * as lib_embeddings from "../lib/embeddings.js";
import type * as lib_githubAccount from "../lib/githubAccount.js";
import type * as lib_githubActionsOidc from "../lib/githubActionsOidc.js";
import type * as lib_githubAuth from "../lib/githubAuth.js";
import type * as lib_githubBackup from "../lib/githubBackup.js";
import type * as lib_githubIdentity from "../lib/githubIdentity.js";
import type * as lib_githubImport from "../lib/githubImport.js";
import type * as lib_githubProfileSync from "../lib/githubProfileSync.js";
import type * as lib_githubRestoreHelpers from "../lib/githubRestoreHelpers.js";
import type * as lib_githubSkillSync from "../lib/githubSkillSync.js";
import type * as lib_githubSoulBackup from "../lib/githubSoulBackup.js";
import type * as lib_globalStats from "../lib/globalStats.js";
import type * as lib_httpHeaders from "../lib/httpHeaders.js";
import type * as lib_httpRateLimit from "../lib/httpRateLimit.js";
import type * as lib_httpUtils from "../lib/httpUtils.js";
import type * as lib_installResolver from "../lib/installResolver.js";
import type * as lib_leaderboards from "../lib/leaderboards.js";
import type * as lib_manualOverrides from "../lib/manualOverrides.js";
import type * as lib_moderation from "../lib/moderation.js";
import type * as lib_moderationEngine from "../lib/moderationEngine.js";
import type * as lib_moderationReasonCodes from "../lib/moderationReasonCodes.js";
import type * as lib_observabilityEvents from "../lib/observabilityEvents.js";
import type * as lib_officialPublishers from "../lib/officialPublishers.js";
import type * as lib_openaiResponse from "../lib/openaiResponse.js";
import type * as lib_packageRegistry from "../lib/packageRegistry.js";
@@ -83,6 +92,7 @@ import type * as lib_public from "../lib/public.js";
import type * as lib_publicRouteReservations from "../lib/publicRouteReservations.js";
import type * as lib_publishLimits from "../lib/publishLimits.js";
import type * as lib_publisherAbuseScoring from "../lib/publisherAbuseScoring.js";
import type * as lib_publisherCatalogDisplay from "../lib/publisherCatalogDisplay.js";
import type * as lib_publisherStats from "../lib/publisherStats.js";
import type * as lib_publishers from "../lib/publishers.js";
import type * as lib_reporting from "../lib/reporting.js";
@@ -93,6 +103,7 @@ import type * as lib_securityPrompt from "../lib/securityPrompt.js";
import type * as lib_skillBackfill from "../lib/skillBackfill.js";
import type * as lib_skillCapabilityTags from "../lib/skillCapabilityTags.js";
import type * as lib_skillCards from "../lib/skillCards.js";
import type * as lib_skillFileAccess from "../lib/skillFileAccess.js";
import type * as lib_skillIcon from "../lib/skillIcon.js";
import type * as lib_skillPublish from "../lib/skillPublish.js";
import type * as lib_skillQuality from "../lib/skillQuality.js";
@@ -112,9 +123,11 @@ import type * as lib_userSkillStats from "../lib/userSkillStats.js";
import type * as lib_webhooks from "../lib/webhooks.js";
import type * as llmEval from "../llmEval.js";
import type * as maintenance from "../maintenance.js";
import type * as managementDevSeed from "../managementDevSeed.js";
import type * as packagePublishTokens from "../packagePublishTokens.js";
import type * as packages from "../packages.js";
import type * as publisherAbuse from "../publisherAbuse.js";
import type * as publisherAbuseDevSeed from "../publisherAbuseDevSeed.js";
import type * as publishers from "../publishers.js";
import type * as rateLimits from "../rateLimits.js";
import type * as search from "../search.js";
@@ -156,14 +169,18 @@ declare const fullApi: ApiFromModules<{
depRegistryScan: typeof depRegistryScan;
devSeed: typeof devSeed;
devSeedExtra: typeof devSeedExtra;
downloadMetrics: typeof downloadMetrics;
downloads: typeof downloads;
functions: typeof functions;
githubAccountAgeBackfill: typeof githubAccountAgeBackfill;
githubBackups: typeof githubBackups;
githubBackupsNode: typeof githubBackupsNode;
githubIdentity: typeof githubIdentity;
githubImport: typeof githubImport;
githubRestore: typeof githubRestore;
githubRestoreMutations: typeof githubRestoreMutations;
githubSkillSources: typeof githubSkillSources;
githubSkillSync: typeof githubSkillSync;
githubSoulBackups: typeof githubSoulBackups;
githubSoulBackupsNode: typeof githubSoulBackupsNode;
http: typeof http;
@@ -193,25 +210,30 @@ declare const fullApi: ApiFromModules<{
"lib/contentTypes": typeof lib_contentTypes;
"lib/depRegistryScan": typeof lib_depRegistryScan;
"lib/devAuth": typeof lib_devAuth;
"lib/devSeed": typeof lib_devSeed;
"lib/embeddingVisibility": typeof lib_embeddingVisibility;
"lib/embeddings": typeof lib_embeddings;
"lib/githubAccount": typeof lib_githubAccount;
"lib/githubActionsOidc": typeof lib_githubActionsOidc;
"lib/githubAuth": typeof lib_githubAuth;
"lib/githubBackup": typeof lib_githubBackup;
"lib/githubIdentity": typeof lib_githubIdentity;
"lib/githubImport": typeof lib_githubImport;
"lib/githubProfileSync": typeof lib_githubProfileSync;
"lib/githubRestoreHelpers": typeof lib_githubRestoreHelpers;
"lib/githubSkillSync": typeof lib_githubSkillSync;
"lib/githubSoulBackup": typeof lib_githubSoulBackup;
"lib/globalStats": typeof lib_globalStats;
"lib/httpHeaders": typeof lib_httpHeaders;
"lib/httpRateLimit": typeof lib_httpRateLimit;
"lib/httpUtils": typeof lib_httpUtils;
"lib/installResolver": typeof lib_installResolver;
"lib/leaderboards": typeof lib_leaderboards;
"lib/manualOverrides": typeof lib_manualOverrides;
"lib/moderation": typeof lib_moderation;
"lib/moderationEngine": typeof lib_moderationEngine;
"lib/moderationReasonCodes": typeof lib_moderationReasonCodes;
"lib/observabilityEvents": typeof lib_observabilityEvents;
"lib/officialPublishers": typeof lib_officialPublishers;
"lib/openaiResponse": typeof lib_openaiResponse;
"lib/packageRegistry": typeof lib_packageRegistry;
@@ -222,6 +244,7 @@ declare const fullApi: ApiFromModules<{
"lib/publicRouteReservations": typeof lib_publicRouteReservations;
"lib/publishLimits": typeof lib_publishLimits;
"lib/publisherAbuseScoring": typeof lib_publisherAbuseScoring;
"lib/publisherCatalogDisplay": typeof lib_publisherCatalogDisplay;
"lib/publisherStats": typeof lib_publisherStats;
"lib/publishers": typeof lib_publishers;
"lib/reporting": typeof lib_reporting;
@@ -232,6 +255,7 @@ declare const fullApi: ApiFromModules<{
"lib/skillBackfill": typeof lib_skillBackfill;
"lib/skillCapabilityTags": typeof lib_skillCapabilityTags;
"lib/skillCards": typeof lib_skillCards;
"lib/skillFileAccess": typeof lib_skillFileAccess;
"lib/skillIcon": typeof lib_skillIcon;
"lib/skillPublish": typeof lib_skillPublish;
"lib/skillQuality": typeof lib_skillQuality;
@@ -251,9 +275,11 @@ declare const fullApi: ApiFromModules<{
"lib/webhooks": typeof lib_webhooks;
llmEval: typeof llmEval;
maintenance: typeof maintenance;
managementDevSeed: typeof managementDevSeed;
packagePublishTokens: typeof packagePublishTokens;
packages: typeof packages;
publisherAbuse: typeof publisherAbuse;
publisherAbuseDevSeed: typeof publisherAbuseDevSeed;
publishers: typeof publishers;
rateLimits: typeof rateLimits;
search: typeof search;
+9 -4
View File
@@ -9,12 +9,12 @@ import { isLocalDevAuthEnabled } from "./lib/devAuth";
import { shouldScheduleGitHubProfileSync } from "./lib/githubProfileSync";
export const BANNED_REAUTH_MESSAGE =
"This account has been banned and cannot sign in. If you believe this is a mistake, please contact security@openclaw.ai and we will review it.";
"This account has been banned and cannot sign in. If you believe this is a mistake, open a GitHub issue: https://github.com/openclaw/clawhub/issues/new.";
export const DELETED_ACCOUNT_REAUTH_MESSAGE =
"This account has been permanently deleted and cannot be restored.";
const REAUTH_BLOCKING_BAN_ACTIONS = new Set(["user.ban", "user.autoban.malware"]);
const DEV_PERSONAS = new Set(["owner", "user", "admin"]);
const DEV_PERSONAS = new Set(["owner", "user", "admin", "officialOrgMember"]);
function getBannedReauthMessage(reason: string | undefined) {
const normalizedReason = reason?.trim();
@@ -90,11 +90,16 @@ export const { auth, signIn, signOut, store, isAuthenticated } = convexAuth({
ConvexCredentials({
id: "dev-persona",
authorize: async (credentials, ctx) => {
if (!isLocalDevAuthEnabled()) throw new Error("Dev auth is disabled");
const devAuthSecret =
typeof credentials.devAuthSecret === "string" ? credentials.devAuthSecret : undefined;
if (!isLocalDevAuthEnabled(process.env, devAuthSecret)) {
throw new Error("Dev auth is disabled");
}
const persona = typeof credentials.persona === "string" ? credentials.persona : "";
if (!DEV_PERSONAS.has(persona)) throw new Error("Unknown dev persona");
const userId: Id<"users"> = await ctx.runMutation(internal.users.upsertDevPersonaInternal, {
persona: persona as "owner" | "user" | "admin",
persona: persona as "owner" | "user" | "admin" | "officialOrgMember",
devAuthSecret,
});
return { userId };
},
+58
View File
@@ -0,0 +1,58 @@
/* @vitest-environment node */
import { describe, expect, it, vi } from "vitest";
const mocks = vi.hoisted(() => {
const interval = vi.fn();
const githubSkillSyncRef = Symbol("github-skill-source-sync");
return { interval, githubSkillSyncRef };
});
vi.mock("convex/server", () => ({
cronJobs: () => ({
interval: mocks.interval,
}),
}));
vi.mock("./_generated/api", () => ({
internal: {
githubBackupsNode: { syncGitHubBackupsInternal: Symbol("github-backup-sync") },
githubSkillSync: { syncGitHubSkillSourcesInternal: mocks.githubSkillSyncRef },
leaderboards: { rebuildTrendingLeaderboardAction: Symbol("trending-leaderboard") },
statsMaintenance: {
runSkillStatBackfillInternal: Symbol("skill-stats-backfill"),
updateGlobalStatsAction: Symbol("global-stats-update"),
},
skillStatEvents: { processSkillStatEventsAction: Symbol("skill-stat-events") },
packages: {
processPackageStatEventsInternal: Symbol("package-stat-events"),
backfillPackageReleaseScansInternal: Symbol("package-scan-backfill"),
},
publisherAbuse: {
runPublisherAbuseScoreRunInternal: Symbol("publisher-abuse-score-refresh"),
},
vt: {
pollPendingScans: Symbol("vt-pending-scans"),
backfillActiveSkillsVTCache: Symbol("vt-cache-backfill"),
},
securityScan: {
pruneExpiredSkillScanRequestsInternal: Symbol("skill-scan-request-prune"),
},
downloads: { pruneDownloadDedupesInternal: Symbol("download-dedupe-prune") },
downloadMetrics: {
pruneDownloadMetricDedupesInternal: Symbol("download-metric-dedupe-prune"),
},
},
}));
describe("crons", () => {
it("runs GitHub skill source sync every 15 minutes", async () => {
await import("./crons");
expect(mocks.interval).toHaveBeenCalledWith(
"github-skill-source-sync",
{ minutes: 15 },
mocks.githubSkillSyncRef,
{},
);
});
});
+21
View File
@@ -10,6 +10,13 @@ crons.interval(
{ batchSize: 50, maxBatches: 5 },
);
crons.interval(
"github-skill-source-sync",
{ minutes: 15 },
internal.githubSkillSync.syncGitHubSkillSourcesInternal,
{},
);
crons.interval(
"trending-leaderboard",
{ minutes: 60 },
@@ -79,6 +86,13 @@ crons.interval(
{ batchSize: 100 },
);
crons.interval(
"skill-scan-request-prune",
{ hours: 6 },
internal.securityScan.pruneExpiredSkillScanRequestsInternal,
{ batchSize: 250 },
);
crons.interval(
"download-dedupe-prune",
{ hours: 24 },
@@ -86,4 +100,11 @@ crons.interval(
{},
);
crons.interval(
"download-metric-dedupe-prune",
{ hours: 24 },
internal.downloadMetrics.pruneDownloadMetricDedupesInternal,
{},
);
export default crons;
+205
View File
@@ -4,6 +4,8 @@ import {
currentUserSeedPackageName,
currentUserSeedSkillSlug,
seedFeaturedPluginPackagesMutation,
seedGitHubBackedSkillSourceMutation,
seedLocalFixtures,
seedLocalModerationFixturesHandler,
seedSkillMutation,
} from "./devSeed";
@@ -18,6 +20,12 @@ const seedSkillMutationHandler = (
const seedFeaturedPluginPackagesHandler = (
seedFeaturedPluginPackagesMutation as unknown as WrappedHandler<Record<string, unknown>>
)._handler;
const seedGitHubBackedSkillSourceHandler = (
seedGitHubBackedSkillSourceMutation as unknown as WrappedHandler<Record<string, unknown>>
)._handler;
const seedLocalFixturesHandler = (
seedLocalFixtures as unknown as WrappedHandler<{ reset?: boolean }>
)._handler;
function chainEq(constraints: Record<string, unknown>) {
return {
@@ -146,6 +154,33 @@ function seedSkillArgs(storageId: string) {
}
describe("devSeed local fixtures", () => {
it("does not preconfigure GitHub-backed source fixtures in the local seed action", async () => {
const mutationCalls: Array<{ args: Record<string, unknown> }> = [];
let storageCounter = 0;
const ctx = {
storage: {
store: async () => `storage:${++storageCounter}`,
},
runMutation: async (_ref: unknown, args: Record<string, unknown>) => {
mutationCalls.push({ args });
return { ok: true, seeded: ["local-moderation-fixtures"], skipped: [] };
},
};
const result = await seedLocalFixturesHandler(ctx as never, { reset: true });
expect(mutationCalls).toHaveLength(1);
expect(mutationCalls[0]?.args).toMatchObject({
reset: true,
});
expect(result).toEqual(
expect.objectContaining({
ok: true,
results: [expect.objectContaining({ slug: "local-moderation-fixtures" })],
}),
);
});
it("seeds core skill fixtures for an explicit local user without creating @local", async () => {
const { db, tables } = createDb();
const userId = (await db.insert("users", {
@@ -190,6 +225,176 @@ describe("devSeed local fixtures", () => {
);
});
it("seeds a GitHub-backed source and skills without creating mirrored versions", async () => {
const { db, tables } = createDb();
const userId = (await db.insert("users", {
handle: "nvidia-dev",
displayName: "NVIDIA Dev",
role: "user",
createdAt: 1,
updatedAt: 1,
})) as Id<"users">;
const result = await seedGitHubBackedSkillSourceHandler(
createMutationCtx(db) as never,
{
ownerUserId: userId,
repo: "NVIDIA/skills",
defaultBranch: "main",
displayManifestKind: "skills.sh",
displayManifestHash: "manifest-sha256",
displayManifestCommit: "0".repeat(40),
displayManifestFetchedAt: 123,
displayManifestStatus: "ok",
displayManifest: {
notGrouped: "bottom",
groupings: [
{
title: "Agentic AI",
description: "Agentic AI skills.",
skills: ["aiq-deploy", "nemoclaw-user-configure-security"],
},
],
},
skills: [
{
slug: "aiq-deploy",
displayName: "AIQ Deploy",
summary: "Deploy AgentIQ workflows.",
githubPath: "skills/aiq-deploy",
githubCurrentCommit: "1".repeat(40),
githubCurrentContentHash: "hash-aiq-deploy",
githubScanStatus: "clean",
githubCurrentCheckedAt: 456,
},
{
slug: "nemoclaw-user-configure-security",
displayName: "NeMoClaw User Configure Security",
summary: "Configure NeMoClaw user security.",
githubPath: "skills/nemoclaw-user-configure-security",
githubCurrentCommit: "2".repeat(40),
githubCurrentContentHash: "hash-nemoclaw",
githubScanStatus: "clean",
githubCurrentCheckedAt: 789,
githubRemovedAt: 900,
},
],
} as never,
);
expect(result).toMatchObject({
ok: true,
seeded: ["aiq-deploy", "nemoclaw-user-configure-security"],
skipped: [],
});
expect(tables.githubSkillSources).toHaveLength(1);
expect(tables.githubSkillSources?.[0]).toEqual(
expect.objectContaining({
repo: "NVIDIA/skills",
ownerPublisherId: tables.publishers?.[0]?._id,
defaultBranch: "main",
displayManifestKind: "skills.sh",
displayManifestHash: "manifest-sha256",
displayManifestCommit: "0".repeat(40),
displayManifestFetchedAt: 123,
displayManifestStatus: "ok",
displayManifest: {
notGrouped: "bottom",
groupings: [
{
title: "Agentic AI",
description: "Agentic AI skills.",
skills: ["aiq-deploy", "nemoclaw-user-configure-security"],
},
],
},
}),
);
expect(tables.skills).toHaveLength(2);
expect(tables.skills).toEqual(
expect.arrayContaining([
expect.objectContaining({
slug: "aiq-deploy",
installKind: "github",
githubSourceId: tables.githubSkillSources?.[0]?._id,
githubPath: "skills/aiq-deploy",
githubCurrentCommit: "1".repeat(40),
githubCurrentContentHash: "hash-aiq-deploy",
githubScanStatus: "clean",
githubCurrentCheckedAt: 456,
latestVersionId: undefined,
latestVersionSummary: undefined,
tags: {},
stats: expect.objectContaining({ versions: 0 }),
}),
expect.objectContaining({
slug: "nemoclaw-user-configure-security",
installKind: "github",
githubSourceId: tables.githubSkillSources?.[0]?._id,
githubPath: "skills/nemoclaw-user-configure-security",
githubCurrentContentHash: "hash-nemoclaw",
githubRemovedAt: 900,
moderationStatus: "hidden",
moderationReason: "github.upstream.removed",
moderationVerdict: undefined,
isSuspicious: false,
latestVersionId: undefined,
tags: {},
}),
]),
);
expect(tables.skillVersions ?? []).toHaveLength(0);
});
it("keeps unscanned GitHub-backed skills hidden from public listings", async () => {
const { db, tables } = createDb();
await seedGitHubBackedSkillSourceHandler(
createMutationCtx(db) as never,
{
repo: "NVIDIA/skills",
displayManifestStatus: "ok",
skills: [
{
slug: "pending-github-skill",
displayName: "Pending GitHub Skill",
githubPath: "skills/pending-github-skill",
githubCurrentCommit: "1".repeat(40),
githubCurrentContentHash: "hash-pending",
githubScanStatus: "pending",
},
{
slug: "failed-scan-github-skill",
displayName: "Failed Scan GitHub Skill",
githubPath: "skills/failed-scan-github-skill",
githubCurrentCommit: "2".repeat(40),
githubCurrentContentHash: "hash-failed-scan",
githubScanStatus: "failed",
},
],
} as never,
);
expect(tables.skills).toEqual(
expect.arrayContaining([
expect.objectContaining({
slug: "pending-github-skill",
moderationStatus: "hidden",
moderationReason: "pending.scan",
moderationVerdict: undefined,
isSuspicious: false,
}),
expect.objectContaining({
slug: "failed-scan-github-skill",
moderationStatus: "hidden",
moderationReason: "scanner.failed",
moderationVerdict: undefined,
isSuspicious: false,
}),
]),
);
});
it("seeds moderation and plugin fixtures for an explicit local user with scoped identifiers", async () => {
const { db, tables } = createDb();
const userId = (await db.insert("users", {
+582 -1
View File
@@ -31,6 +31,67 @@ type SeedActionResult = {
type SeedMutationResult = Record<string, unknown>;
const displayManifestStatusValidator = v.union(
v.literal("ok"),
v.literal("missing"),
v.literal("invalid"),
v.literal("failed"),
);
const displayManifestValidator = v.object({
notGrouped: v.optional(v.union(v.literal("top"), v.literal("bottom"))),
groupings: v.array(
v.object({
title: v.string(),
description: v.optional(v.string()),
skills: v.array(v.string()),
}),
),
});
const githubSkillScanStatusValidator = v.union(
v.literal("clean"),
v.literal("suspicious"),
v.literal("malicious"),
v.literal("pending"),
v.literal("failed"),
);
type GitHubSkillScanStatus = "clean" | "suspicious" | "malicious" | "pending" | "failed";
type SeedGitHubBackedSkillSourceArgs = {
reset?: boolean;
ownerUserId?: Id<"users">;
repo: string;
defaultBranch?: string;
displayManifestKind?: "skills.sh";
displayManifestHash?: string;
displayManifestCommit?: string;
displayManifestFetchedAt?: number;
displayManifestStatus?: "ok" | "missing" | "invalid" | "failed";
displayManifest?: {
notGrouped?: "top" | "bottom";
groupings: Array<{
title: string;
description?: string;
skills: string[];
}>;
};
skills: Array<{
slug: string;
displayName: string;
summary?: string;
githubPath: string;
githubCurrentCommit: string;
githubCurrentContentHash: string;
githubCurrentStatus?: "present" | "missing" | "unknown";
githubCurrentCheckedAt?: number;
githubScanStatus: GitHubSkillScanStatus;
githubRemovedAt?: number;
capabilityTags?: string[];
}>;
};
type PublicCorpusDummyOwner = {
handle: string;
displayName: string;
@@ -734,7 +795,10 @@ async function seedLocalFixturesHandler(
},
);
return { ok: true, results: [{ slug: "local-moderation-fixtures", ...fixtureResult }] };
return {
ok: true,
results: [{ slug: "local-moderation-fixtures", ...fixtureResult }],
};
}
export const seedLocalFixtures: ReturnType<typeof internalAction> = internalAction({
@@ -2432,6 +2496,278 @@ export const seedLocalModerationFixturesMutation = internalMutation({
handler: seedLocalModerationFixturesHandler,
});
function githubBackedSkillModeration(scanStatus: GitHubSkillScanStatus, removedAt?: number) {
if (typeof removedAt === "number") {
return {
moderationStatus: "hidden" as const,
moderationReason: "github.upstream.removed",
moderationVerdict: undefined,
moderationFlags: [],
isSuspicious: false,
};
}
if (scanStatus === "pending") {
return {
moderationStatus: "hidden" as const,
moderationReason: "pending.scan",
moderationVerdict: undefined,
moderationFlags: [],
isSuspicious: false,
};
}
if (scanStatus === "failed") {
return {
moderationStatus: "hidden" as const,
moderationReason: "scanner.failed",
moderationVerdict: undefined,
moderationFlags: [],
isSuspicious: false,
};
}
if (scanStatus === "malicious") {
return {
moderationStatus: "hidden" as const,
moderationReason: "scanner.llm.malicious",
moderationVerdict: "malicious" as const,
moderationFlags: ["blocked.malware"],
isSuspicious: true,
};
}
if (scanStatus === "suspicious") {
return {
moderationStatus: "active" as const,
moderationReason: "scanner.llm.suspicious",
moderationVerdict: "suspicious" as const,
moderationFlags: ["flagged.suspicious"],
isSuspicious: true,
};
}
return {
moderationStatus: "active" as const,
moderationReason: undefined,
moderationVerdict: "clean" as const,
moderationFlags: [],
isSuspicious: false,
};
}
export async function seedGitHubBackedSkillSourceHandler(
ctx: MutationCtx,
args: SeedGitHubBackedSkillSourceArgs,
) {
const now = Date.now();
const { userId, publisherId } = await ensureSeedOwner(ctx, args.ownerUserId);
const existingSource = await ctx.db
.query("githubSkillSources")
.withIndex("by_repo", (q) => q.eq("repo", args.repo))
.unique();
const sourcePatch = {
repo: args.repo,
ownerPublisherId: publisherId,
defaultBranch: args.defaultBranch,
displayManifestKind: args.displayManifestKind,
displayManifestHash: args.displayManifestHash,
displayManifestCommit: args.displayManifestCommit,
displayManifestFetchedAt: args.displayManifestFetchedAt,
displayManifestStatus: args.displayManifestStatus,
displayManifest: args.displayManifest,
updatedAt: now,
};
const sourceId =
existingSource?._id ??
(await ctx.db.insert("githubSkillSources", {
...sourcePatch,
createdAt: now,
}));
if (existingSource) await ctx.db.patch(existingSource._id, sourcePatch);
const seeded: string[] = [];
const skipped: string[] = [];
for (const spec of args.skills) {
const existing = await ctx.db
.query("skills")
.withIndex("by_slug", (q) => q.eq("slug", spec.slug))
.unique();
if (existing && !args.reset) {
skipped.push(spec.slug);
continue;
}
if (existing && args.reset) await deleteSkillAndVersions(ctx, existing._id);
const moderation = githubBackedSkillModeration(spec.githubScanStatus, spec.githubRemovedAt);
const skillId = await ctx.db.insert("skills", {
slug: spec.slug,
displayName: spec.displayName,
summary: spec.summary,
ownerUserId: userId,
ownerPublisherId: publisherId,
installKind: "github",
githubSourceId: sourceId,
githubPath: spec.githubPath,
githubCurrentCommit: spec.githubCurrentCommit,
githubCurrentContentHash: spec.githubCurrentContentHash,
githubCurrentStatus:
spec.githubCurrentStatus ?? (spec.githubRemovedAt ? "missing" : "present"),
githubCurrentCheckedAt: spec.githubCurrentCheckedAt,
githubScanStatus: spec.githubScanStatus,
githubRemovedAt: spec.githubRemovedAt,
latestVersionId: undefined,
latestVersionSummary: undefined,
tags: {},
capabilityTags: spec.capabilityTags ?? [],
softDeletedAt: undefined,
badges: { highlighted: { byUserId: userId, at: now }, redactionApproved: undefined },
moderationStatus: moderation.moderationStatus,
moderationReason: moderation.moderationReason,
moderationVerdict: moderation.moderationVerdict,
moderationFlags: moderation.moderationFlags,
isSuspicious: moderation.isSuspicious,
statsDownloads: 0,
statsStars: 0,
statsInstallsCurrent: 0,
statsInstallsAllTime: 0,
stats: {
downloads: 0,
installsCurrent: 0,
installsAllTime: 0,
stars: 0,
versions: 0,
comments: 0,
},
createdAt: now,
updatedAt: now,
});
await ensureHighlightedSkillBadge(ctx, skillId, userId, now);
seeded.push(spec.slug);
}
return {
ok: true,
sourceId,
ownerUserId: userId,
ownerPublisherId: publisherId,
seeded,
skipped,
};
}
export const seedGitHubBackedSkillSourceMutation = internalMutation({
args: {
reset: v.optional(v.boolean()),
ownerUserId: v.optional(v.id("users")),
repo: v.string(),
defaultBranch: v.optional(v.string()),
displayManifestKind: v.optional(v.literal("skills.sh")),
displayManifestHash: v.optional(v.string()),
displayManifestCommit: v.optional(v.string()),
displayManifestFetchedAt: v.optional(v.number()),
displayManifestStatus: v.optional(displayManifestStatusValidator),
displayManifest: v.optional(displayManifestValidator),
skills: v.array(
v.object({
slug: v.string(),
displayName: v.string(),
summary: v.optional(v.string()),
githubPath: v.string(),
githubCurrentCommit: v.string(),
githubCurrentContentHash: v.string(),
githubCurrentStatus: v.optional(
v.union(v.literal("present"), v.literal("missing"), v.literal("unknown")),
),
githubCurrentCheckedAt: v.optional(v.number()),
githubScanStatus: githubSkillScanStatusValidator,
githubRemovedAt: v.optional(v.number()),
capabilityTags: v.optional(v.array(v.string())),
}),
),
},
handler: seedGitHubBackedSkillSourceHandler,
});
export const seedGitHubSourceInvalidSkillsPreviewMutation = internalMutation({
args: {
repo: v.string(),
},
handler: async (ctx, args) => {
const source = await ctx.db
.query("githubSkillSources")
.withIndex("by_repo", (q) => q.eq("repo", args.repo))
.unique();
if (!source) {
return { ok: false as const, reason: "source_not_found" as const };
}
const overlongSlug = "preview-" + "x".repeat(97);
await ctx.db.patch(source._id, {
lastSyncIssues: [
{
slug: overlongSlug,
path: `skills/${overlongSlug}`,
displayName: "Preview Invalid Skill",
kind: "invalid_slug",
severity: "error",
message: "Slug must be at most 96 characters.",
},
],
lastSyncInvalidSkills: [
{
slug: overlongSlug,
path: `skills/${overlongSlug}`,
displayName: "Preview Invalid Skill",
error: "Slug must be at most 96 characters.",
},
],
updatedAt: Date.now(),
});
return { ok: true as const, sourceId: source._id };
},
});
export const deleteGitHubBackedSkillSourceSeedMutation = internalMutation({
args: {
repo: v.optional(v.string()),
sourceId: v.optional(v.id("githubSkillSources")),
},
handler: async (ctx, args) => {
const source = args.sourceId
? await ctx.db.get(args.sourceId)
: args.repo
? await ctx.db
.query("githubSkillSources")
.withIndex("by_repo", (q) => q.eq("repo", args.repo as string))
.unique()
: null;
const sourceId = source?._id ?? args.sourceId;
if (!sourceId) {
return { ok: true as const, deletedSource: false, deletedSkills: 0, deletedContents: 0 };
}
const contents = await ctx.db
.query("githubSkillContents")
.withIndex("by_github_source", (q) => q.eq("githubSourceId", sourceId))
.collect();
for (const content of contents) await ctx.db.delete(content._id);
const skills = await ctx.db
.query("skills")
.withIndex("by_github_source", (q) => q.eq("githubSourceId", sourceId))
.collect();
for (const skill of skills) await deleteSkillAndVersions(ctx, skill._id);
if (source) await ctx.db.delete(source._id);
return {
ok: true as const,
deletedSource: Boolean(source),
deletedSkills: skills.length,
deletedContents: contents.length,
};
},
});
export const seedFeaturedPluginPackagesMutation = internalMutation({
args: {
reset: v.optional(v.boolean()),
@@ -2764,6 +3100,251 @@ export const seedCliRoleHelpFixtures = rawInternalMutation({
},
});
type OrgDeletionFixtureArgs = {
handle: string;
displayName: string;
skillSlug: string;
skillDisplayName: string;
packageName: string;
packageDisplayName: string;
};
type OrgDeletionFixtureResult = {
ok: true;
publisherId: Id<"publishers">;
skillId: Id<"skills">;
skillVersionId: Id<"skillVersions">;
packageId: Id<"packages">;
packageReleaseId: Id<"packageReleases">;
handle: string;
skillSlug: string;
packageName: string;
};
export const seedOrgDeletionFixture: ReturnType<typeof rawInternalMutation> = rawInternalMutation({
args: {
handle: v.string(),
displayName: v.string(),
skillSlug: v.string(),
skillDisplayName: v.string(),
packageName: v.string(),
packageDisplayName: v.string(),
},
handler: async (ctx, args): Promise<OrgDeletionFixtureResult> => {
return (await ctx.runMutation(
internal.devSeed.seedOrgDeletionFixtureMutation,
args as OrgDeletionFixtureArgs,
)) as OrgDeletionFixtureResult;
},
});
export const seedOrgDeletionFixtureMutation = internalMutation({
args: {
handle: v.string(),
displayName: v.string(),
skillSlug: v.string(),
skillDisplayName: v.string(),
packageName: v.string(),
packageDisplayName: v.string(),
},
handler: async (ctx, args) => {
const now = Date.now();
const { userId } = await ensureLocalSeedOwner(ctx);
const normalizedName = normalizePackageName(args.packageName);
const existingSkill = await ctx.db
.query("skills")
.withIndex("by_slug", (q) => q.eq("slug", args.skillSlug))
.unique();
const existingPackage = await ctx.db
.query("packages")
.withIndex("by_name", (q) => q.eq("normalizedName", normalizedName))
.unique();
if (existingSkill || existingPackage) {
throw new Error("Org deletion fixture names must be unique per run");
}
const publisherId = await ctx.db.insert("publishers", {
kind: "org",
handle: args.handle,
displayName: args.displayName,
bio: "Disposable local-auth fixture for org deletion e2e proof.",
image: undefined,
trustedPublisher: false,
publishedSkills: 1,
publishedPackages: 1,
totalInstalls: 0,
totalDownloads: 0,
totalStars: 0,
skillTotalInstalls: 0,
skillTotalDownloads: 0,
skillTotalStars: 0,
createdAt: now,
updatedAt: now,
});
await ctx.db.insert("publisherMembers", {
publisherId,
userId,
role: "owner",
createdAt: now,
updatedAt: now,
});
const skillId = await ctx.db.insert("skills", {
slug: args.skillSlug,
displayName: args.skillDisplayName,
summary: "Disposable local-auth fixture skill owned by an organization.",
ownerUserId: userId,
ownerPublisherId: publisherId,
latestVersionId: undefined,
latestVersionSummary: undefined,
tags: {},
capabilityTags: ["dev-tools"],
softDeletedAt: undefined,
badges: { highlighted: undefined, redactionApproved: undefined },
moderationStatus: "active",
moderationReason: "clean",
isSuspicious: false,
statsDownloads: 0,
statsStars: 0,
statsInstallsCurrent: 0,
statsInstallsAllTime: 0,
stats: {
downloads: 0,
installsCurrent: 0,
installsAllTime: 0,
stars: 0,
versions: 0,
comments: 0,
},
createdAt: now,
updatedAt: now,
});
const skillVersionId = await ctx.db.insert("skillVersions", {
skillId,
version: "1.0.0",
changelog: "Seeded local-auth org deletion fixture.",
changelogSource: "user",
files: [],
parsed: {
frontmatter: {
name: args.skillSlug,
description: "Disposable local-auth org deletion fixture skill.",
},
metadata: {},
},
capabilityTags: ["dev-tools"],
createdBy: userId,
createdAt: now,
softDeletedAt: undefined,
});
await ctx.db.patch(skillId, {
latestVersionId: skillVersionId,
latestVersionSummary: {
version: "1.0.0",
createdAt: now,
changelog: "Seeded local-auth org deletion fixture.",
changelogSource: "user",
},
tags: { latest: skillVersionId },
stats: {
downloads: 0,
installsCurrent: 0,
installsAllTime: 0,
stars: 0,
versions: 1,
comments: 0,
},
updatedAt: now,
});
const compatibility = { pluginApiRange: ">=0.1.0" };
const capabilities = {
executesCode: true,
runtimeId: normalizedName,
pluginKind: "runtime",
capabilityTags: ["dev-tools"],
};
const verification = {
tier: "structural" as const,
scope: "artifact-only" as const,
summary: "Seeded local-auth org deletion fixture.",
scanStatus: "clean" as const,
};
const packageId = await ctx.db.insert("packages", {
name: args.packageName,
normalizedName,
displayName: args.packageDisplayName,
summary: "Disposable local-auth fixture plugin owned by an organization.",
ownerUserId: userId,
ownerPublisherId: publisherId,
family: "code-plugin",
channel: "community",
isOfficial: false,
runtimeId: normalizedName,
latestReleaseId: undefined,
latestVersionSummary: undefined,
tags: {},
capabilityTags: ["dev-tools"],
executesCode: true,
compatibility,
capabilities,
verification,
scanStatus: "clean",
stats: { downloads: 0, installs: 0, stars: 0, versions: 0 },
softDeletedAt: undefined,
createdAt: now,
updatedAt: now,
});
const packageReleaseId = await ctx.db.insert("packageReleases", {
packageId,
version: "1.0.0",
changelog: "Seeded local-auth org deletion fixture.",
summary: "Disposable local-auth fixture plugin release.",
distTags: ["latest"],
files: [],
integritySha256: `org-delete-fixture-${normalizedName}`,
extractedPackageJson: {
name: args.packageName,
version: "1.0.0",
},
compatibility,
capabilities,
verification,
sha256hash: `org-delete-fixture-${normalizedName}`,
createdBy: userId,
publishActor: { kind: "user", userId },
createdAt: now,
softDeletedAt: undefined,
});
await ctx.db.patch(packageId, {
latestReleaseId: packageReleaseId,
latestVersionSummary: {
version: "1.0.0",
createdAt: now,
changelog: "Seeded local-auth org deletion fixture.",
compatibility,
capabilities,
verification,
},
tags: { latest: packageReleaseId },
stats: { downloads: 0, installs: 0, stars: 0, versions: 1 },
updatedAt: now,
});
return {
ok: true,
publisherId,
skillId,
skillVersionId,
packageId,
packageReleaseId,
handle: args.handle,
skillSlug: args.skillSlug,
packageName: args.packageName,
};
},
});
async function upsertRoleHelpFixtureUser(ctx: MutationCtx, user: RoleHelpFixtureUser) {
const now = Date.now();
const existing = await ctx.db
+249
View File
@@ -0,0 +1,249 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from "vitest";
import {
__test,
pruneDownloadMetricDedupesInternal,
recordDownloadMetricInternal,
} from "./downloadMetrics";
type WrappedHandler<TArgs, TResult> = {
_handler: (ctx: unknown, args: TArgs) => Promise<TResult>;
};
const recordDownloadMetricHandler = (
recordDownloadMetricInternal as unknown as WrappedHandler<
{
target: { kind: "skill"; id: string } | { kind: "package"; id: string };
identityKind: "user" | "ip";
identityHash: string;
dayStart: number;
occurredAt?: number;
},
void
>
)._handler;
const pruneDownloadMetricDedupesHandler = (
pruneDownloadMetricDedupesInternal as unknown as WrappedHandler<
Record<string, never>,
{ deleted: number; hasMore: boolean }
>
)._handler;
function makeQueryBuilder() {
const builder = {
eq: vi.fn(() => builder),
lt: vi.fn(() => builder),
};
return builder;
}
type QueryBuilder = ReturnType<typeof makeQueryBuilder>;
function makeDb(
existingByTable: Record<string, unknown> = {},
rowsByTable: Record<string, Array<{ _id: string }>> = {},
) {
const indexCalls: Array<{ table: string; indexName: string; builder: QueryBuilder }> = [];
const insert = vi.fn();
const unique = vi.fn(async function uniqueForTable(this: { table: string }) {
return existingByTable[this.table] ?? null;
});
const take = vi.fn(async function takeForTable(this: { table: string }, limit: number) {
return (rowsByTable[this.table] ?? []).slice(0, limit);
});
const query = vi.fn((table: string) => ({
withIndex: vi.fn((indexName: string, buildQuery: (q: unknown) => unknown) => {
const builder = makeQueryBuilder();
buildQuery(builder);
indexCalls.push({ table, indexName, builder });
return {
unique: unique.bind({ table }),
take: take.bind({ table }),
};
}),
}));
const delete_ = vi.fn();
return {
db: {
query,
get: vi.fn(),
insert,
patch: vi.fn(),
replace: vi.fn(),
delete: delete_,
normalizeId: vi.fn(),
system: {
get: vi.fn(),
query: vi.fn(),
},
},
insert,
delete_,
take,
indexCalls,
};
}
describe("download metric helpers", () => {
afterEach(() => {
vi.useRealTimers();
});
it("uses a day bucket for download dedupe", () => {
expect(__test.getDayStart(86_400_000 - 1)).toBe(0);
expect(__test.getDayStart(86_400_000)).toBe(86_400_000);
});
it("prefers user identity and falls back to IP identity", () => {
const request = new Request("https://example.com", {
headers: { "cf-connecting-ip": "203.0.113.10" },
});
expect(__test.getDownloadIdentity(request, "users:one")).toEqual({
identityKind: "user",
identityValue: "users:one",
});
expect(__test.getDownloadIdentity(request, null)).toEqual({
identityKind: "ip",
identityValue: "203.0.113.10",
});
});
it("does not create a metering identity when user and IP are missing", () => {
expect(__test.getDownloadIdentity(new Request("https://example.com"), null)).toBeNull();
});
it("records one authenticated skill download and emits the existing skill stat event", async () => {
const { db, insert, indexCalls } = makeDb();
await recordDownloadMetricHandler(
{ db },
{
target: { kind: "skill", id: "skills:one" },
identityKind: "user",
identityHash: "hash-user",
dayStart: 86_400_000,
occurredAt: 86_500_000,
},
);
expect(indexCalls[0]?.table).toBe("downloadMetricDedupes");
expect(indexCalls[0]?.indexName).toBe("by_target_identity_day");
expect(indexCalls[0]?.builder.eq).toHaveBeenCalledWith("targetKind", "skill");
expect(indexCalls[0]?.builder.eq).toHaveBeenCalledWith("targetId", "skills:one");
expect(indexCalls[0]?.builder.eq).toHaveBeenCalledWith("identityKind", "user");
expect(indexCalls[0]?.builder.eq).toHaveBeenCalledWith("identityHash", "hash-user");
expect(indexCalls[0]?.builder.eq).toHaveBeenCalledWith("dayStart", 86_400_000);
expect(insert).toHaveBeenCalledWith(
"downloadMetricDedupes",
expect.objectContaining({
targetKind: "skill",
targetId: "skills:one",
identityKind: "user",
identityHash: "hash-user",
dayStart: 86_400_000,
}),
);
expect(insert).toHaveBeenCalledWith(
"skillStatEvents",
expect.objectContaining({
skillId: "skills:one",
kind: "download",
occurredAt: 86_500_000,
}),
);
expect(insert).not.toHaveBeenCalledWith("packageStatEvents", expect.anything());
});
it("records one anonymous package download and emits the existing package stat event", async () => {
const { db, insert } = makeDb();
await recordDownloadMetricHandler(
{ db },
{
target: { kind: "package", id: "packages:one" },
identityKind: "ip",
identityHash: "hash-ip",
dayStart: 86_400_000,
occurredAt: 86_500_000,
},
);
expect(insert).toHaveBeenCalledWith(
"downloadMetricDedupes",
expect.objectContaining({
targetKind: "package",
targetId: "packages:one",
identityKind: "ip",
identityHash: "hash-ip",
dayStart: 86_400_000,
}),
);
expect(insert).toHaveBeenCalledWith(
"packageStatEvents",
expect.objectContaining({
packageId: "packages:one",
kind: "download",
occurredAt: 86_500_000,
}),
);
expect(insert).not.toHaveBeenCalledWith("skillStatEvents", expect.anything());
});
it("ignores duplicate identities in the same target/day bucket", async () => {
const { db, insert } = makeDb({
downloadMetricDedupes: { _id: "downloadMetricDedupes:existing" },
});
await recordDownloadMetricHandler(
{ db },
{
target: { kind: "skill", id: "skills:one" },
identityKind: "ip",
identityHash: "hash-ip",
dayStart: 86_400_000,
},
);
expect(insert).not.toHaveBeenCalled();
});
it("prunes stale dedupe rows by day bucket", async () => {
vi.setSystemTime(30 * 86_400_000);
const { db, delete_, take, indexCalls } = makeDb(
{},
{
downloadMetricDedupes: [
{ _id: "downloadMetricDedupes:one" },
{ _id: "downloadMetricDedupes:two" },
],
},
);
const result = await pruneDownloadMetricDedupesHandler({ db }, {});
expect(result).toEqual({ deleted: 2, hasMore: false });
expect(indexCalls[0]?.table).toBe("downloadMetricDedupes");
expect(indexCalls[0]?.indexName).toBe("by_day");
expect(take).toHaveBeenCalledWith(200);
expect(delete_).toHaveBeenCalledWith("downloadMetricDedupes:one");
expect(delete_).toHaveBeenCalledWith("downloadMetricDedupes:two");
});
it("reschedules stale dedupe pruning when one bounded batch fills", async () => {
vi.setSystemTime(30 * 86_400_000);
const rows = Array.from({ length: 200 }, (_, index) => ({
_id: `downloadMetricDedupes:${index}`,
}));
const { db, delete_ } = makeDb({}, { downloadMetricDedupes: rows });
const runAfter = vi.fn();
const result = await pruneDownloadMetricDedupesHandler({ db, scheduler: { runAfter } }, {});
expect(result).toEqual({ deleted: 200, hasMore: true });
expect(delete_).toHaveBeenCalledTimes(200);
expect(runAfter).toHaveBeenCalledWith(0, expect.anything(), {});
});
});
+137
View File
@@ -0,0 +1,137 @@
import { v } from "convex/values";
import { internal } from "./_generated/api";
import type { Id } from "./_generated/dataModel";
import { internalMutation } from "./functions";
import { getClientIp } from "./lib/httpRateLimit";
import { hashToken } from "./lib/tokens";
import { insertStatEvent } from "./skillStatEvents";
const DAY_MS = 86_400_000;
const DEDUPE_RETENTION_MS = 14 * DAY_MS;
const PRUNE_BATCH_SIZE = 200;
const identityKindValidator = v.union(v.literal("user"), v.literal("ip"));
const targetValidator = v.union(
v.object({ kind: v.literal("skill"), id: v.id("skills") }),
v.object({ kind: v.literal("package"), id: v.id("packages") }),
);
type DownloadIdentityKind = "user" | "ip";
type DownloadIdentity = {
identityKind: DownloadIdentityKind;
identityValue: string;
};
export function getDownloadIdentity(
request: Request,
userId: string | null,
): DownloadIdentity | null {
if (userId) return { identityKind: "user", identityValue: userId };
const ip = getClientIp(request);
if (!ip) return null;
return { identityKind: "ip", identityValue: ip };
}
export async function buildDownloadMetricArgs(params: {
target: { kind: "skill"; id: Id<"skills"> } | { kind: "package"; id: Id<"packages"> };
identity: DownloadIdentity;
now: number;
}) {
return {
target: params.target,
identityKind: params.identity.identityKind,
identityHash: await hashToken(
`${params.identity.identityKind}:${params.identity.identityValue}`,
),
dayStart: getDayStart(params.now),
occurredAt: params.now,
};
}
export const recordDownloadMetricInternal = internalMutation({
args: {
target: targetValidator,
identityKind: identityKindValidator,
identityHash: v.string(),
dayStart: v.number(),
occurredAt: v.optional(v.number()),
},
handler: async (ctx, args) => {
const targetId = args.target.id;
const existing = await ctx.db
.query("downloadMetricDedupes")
.withIndex("by_target_identity_day", (q) =>
q
.eq("targetKind", args.target.kind)
.eq("targetId", targetId)
.eq("identityKind", args.identityKind)
.eq("identityHash", args.identityHash)
.eq("dayStart", args.dayStart),
)
.unique();
if (existing) return;
const now = Date.now();
await ctx.db.insert("downloadMetricDedupes", {
targetKind: args.target.kind,
targetId,
identityKind: args.identityKind,
identityHash: args.identityHash,
dayStart: args.dayStart,
createdAt: now,
});
if (args.target.kind === "skill") {
await insertStatEvent(ctx, {
skillId: args.target.id,
kind: "download",
occurredAt: args.occurredAt,
});
return;
}
await ctx.db.insert("packageStatEvents", {
packageId: args.target.id,
kind: "download",
occurredAt: args.occurredAt ?? now,
processedAt: undefined,
});
},
});
export const pruneDownloadMetricDedupesInternal = internalMutation({
args: {},
handler: async (ctx) => {
const cutoffDayStart = getDayStart(Date.now() - DEDUPE_RETENTION_MS);
const stale = await ctx.db
.query("downloadMetricDedupes")
.withIndex("by_day", (q) => q.lt("dayStart", cutoffDayStart))
.take(PRUNE_BATCH_SIZE);
for (const entry of stale) {
await ctx.db.delete(entry._id);
}
const hasMore = stale.length === PRUNE_BATCH_SIZE;
if (hasMore) {
await ctx.scheduler.runAfter(
0,
internal.downloadMetrics.pruneDownloadMetricDedupesInternal,
{},
);
}
return { deleted: stale.length, hasMore };
},
});
function getDayStart(timestamp: number) {
return Math.floor(timestamp / DAY_MS) * DAY_MS;
}
export const __test = {
getDayStart,
getDownloadIdentity,
};
+150 -14
View File
@@ -21,6 +21,19 @@ const okRate = () => ({
resetAt: Date.now() + 60_000,
});
function stubZipResponse() {
class MockResponse {
status: number;
headers: Headers;
constructor(_body?: BodyInit | null, init?: ResponseInit) {
this.status = init?.status ?? 200;
this.headers = new Headers(init?.headers);
}
}
vi.stubGlobal("Response", MockResponse as unknown as typeof Response);
}
describe("downloads helpers", () => {
afterEach(() => {
vi.unstubAllEnvs();
@@ -63,16 +76,7 @@ describe("downloads helpers", () => {
});
it("schedules zip download stats outside the response path", async () => {
class MockResponse {
status: number;
headers: Headers;
constructor(_body?: BodyInit | null, init?: ResponseInit) {
this.status = init?.status ?? 200;
this.headers = new Headers(init?.headers);
}
}
vi.stubGlobal("Response", MockResponse as unknown as typeof Response);
stubZipResponse();
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
@@ -127,9 +131,10 @@ describe("downloads helpers", () => {
if (!args || typeof args !== "object") return false;
const value = args as Record<string, unknown>;
return (
value.skillId === "skills:1" &&
typeof value.target === "object" &&
typeof value.identityHash === "string" &&
typeof value.hourStart === "number"
value.identityKind === "ip" &&
typeof value.dayStart === "number"
);
});
expect(recordCalls).toHaveLength(1);
@@ -137,9 +142,11 @@ describe("downloads helpers", () => {
expect(recordCalls[0]?.[0]).toBeGreaterThanOrEqual(0);
expect(recordCalls[0]?.[0]).toBeLessThan(60_000);
expect(recordCalls[0]?.[2]).toEqual({
skillId: "skills:1",
target: { kind: "skill", id: "skills:1" },
identityKind: "ip",
identityHash: expect.any(String),
hourStart: expect.any(Number),
dayStart: expect.any(Number),
occurredAt: expect.any(Number),
});
});
@@ -202,4 +209,133 @@ describe("downloads helpers", () => {
expect(await response.text()).toBe("Version not found");
expect(storageGet).not.toHaveBeenCalled();
});
it("uses API token user identity for zip download stats when present", async () => {
stubZipResponse();
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
if ("tokenHash" in args) {
return { _id: "apiTokens:1", revokedAt: undefined };
}
if ("tokenId" in args) {
return { _id: "users:token", deletedAt: undefined, deactivatedAt: undefined };
}
if ("slug" in args) {
return {
skill: {
_id: "skills:1",
ownerUserId: "users:1",
slug: "demo",
tags: {},
latestVersionId: "skillVersions:1",
},
moderationInfo: null,
};
}
if ("versionId" in args) {
return {
_id: "skillVersions:1",
skillId: "skills:1",
version: "1.0.0",
createdAt: 3,
files: [{ path: "SKILL.md", storageId: "_storage:1" }],
softDeletedAt: undefined,
};
}
return null;
});
const runMutation = vi.fn(async (_mutation: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
return { tokenTouched: "tokenId" in args };
});
const runAfter = vi.fn();
const storageGet = vi.fn().mockResolvedValue(new Blob(["hello"], { type: "text/markdown" }));
const response = await downloadZipHandler(
{
runQuery,
runMutation,
scheduler: { runAfter },
storage: { get: storageGet },
} as unknown as ActionCtx,
new Request("https://example.com/api/v1/download?slug=demo", {
headers: {
authorization: "Bearer clh_test",
"cf-connecting-ip": "1.2.3.4",
},
}),
);
expect(response.status).toBe(200);
expect(runAfter).toHaveBeenCalledWith(
expect.any(Number),
expect.anything(),
expect.objectContaining({
target: { kind: "skill", id: "skills:1" },
identityKind: "user",
identityHash: expect.stringMatching(/^[a-f0-9]{64}$/),
}),
);
});
it("returns zip downloads when download metering is scheduled", async () => {
stubZipResponse();
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
if ("slug" in args) {
return {
skill: {
_id: "skills:1",
ownerUserId: "users:1",
slug: "demo",
tags: {},
latestVersionId: "skillVersions:1",
},
moderationInfo: null,
};
}
if ("versionId" in args) {
return {
_id: "skillVersions:1",
skillId: "skills:1",
version: "1.0.0",
createdAt: 3,
files: [{ path: "SKILL.md", storageId: "_storage:1" }],
softDeletedAt: undefined,
};
}
return null;
});
const runMutation = vi.fn(async (_mutation: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
return { mutationRecorded: true };
});
const runAfter = vi.fn();
const storageGet = vi.fn().mockResolvedValue(new Blob(["hello"], { type: "text/markdown" }));
const response = await downloadZipHandler(
{
runQuery,
runMutation,
scheduler: { runAfter },
storage: { get: storageGet },
} as unknown as ActionCtx,
new Request("https://example.com/api/v1/download?slug=demo", {
headers: { "cf-connecting-ip": "1.2.3.4" },
}),
);
expect(response.status).toBe(200);
expect(runAfter).toHaveBeenCalledWith(
expect.any(Number),
expect.anything(),
expect.objectContaining({
target: { kind: "skill", id: "skills:1" },
identityKind: "ip",
identityHash: expect.stringMatching(/^[a-f0-9]{64}$/),
}),
);
});
});
+20 -9
View File
@@ -1,12 +1,14 @@
import { v } from "convex/values";
import { api, internal } from "./_generated/api";
import type { Id } from "./_generated/dataModel";
import { buildDownloadMetricArgs, getDownloadIdentity } from "./downloadMetrics";
import { httpAction, internalMutation } from "./functions";
import { getOptionalActiveAuthUserIdFromAction } from "./lib/access";
import { getOptionalApiTokenUserId } from "./lib/apiTokenAuth";
import { corsHeaders, mergeHeaders } from "./lib/httpHeaders";
import { applyRateLimit, getClientIp } from "./lib/httpRateLimit";
import { getPublicSkillFileAccessBlock, isSkillVersionForSkill } from "./lib/skillFileAccess";
import { buildDeterministicZip } from "./lib/skillZip";
import { hashToken } from "./lib/tokens";
import { insertStatEvent } from "./skillStatEvents";
const HOUR_MS = 3_600_000;
@@ -98,17 +100,17 @@ export async function downloadZipHandler(
const zipBlob = new Blob([zipArray], { type: "application/zip" });
try {
const userId = await getOptionalApiTokenUserId(ctx, request);
const identity = getDownloadIdentityValue(request, userId ? String(userId) : null);
const userId = await getOptionalDownloadUserId(ctx, request);
const identity = getDownloadIdentity(request, userId ? String(userId) : null);
if (identity) {
await ctx.scheduler.runAfter(
Math.floor(Math.random() * DOWNLOAD_STAT_JITTER_MS),
internal.downloads.recordDownloadInternal,
{
skillId: skill._id,
identityHash: await hashToken(identity),
hourStart: getHourStart(Date.now()),
},
internal.downloadMetrics.recordDownloadMetricInternal,
await buildDownloadMetricArgs({
target: { kind: "skill", id: skill._id },
identity,
now: Date.now(),
}),
);
}
} catch {
@@ -196,6 +198,15 @@ export function getDownloadIdentityValue(request: Request, userId: string | null
return `ip:${ip}`;
}
async function getOptionalDownloadUserId(
ctx: Parameters<Parameters<typeof httpAction>[0]>[0],
request: Request,
): Promise<Id<"users"> | null> {
const apiTokenUserId = await getOptionalApiTokenUserId(ctx, request);
if (apiTokenUserId) return apiTokenUserId;
return (await getOptionalActiveAuthUserIdFromAction(ctx)) ?? null;
}
export const __test = {
getHourStart,
getDownloadIdentityValue,
+186
View File
@@ -2,7 +2,9 @@
import { describe, expect, it, vi } from "vitest";
import { internal } from "./_generated/api";
import type { Doc, Id, TableNames } from "./_generated/dataModel";
import {
internalMutation,
isGitHubMirrorEligibleSkillDoc,
repointPackageLatestRelease,
scheduleGitHubBackupDeletionForSkill,
@@ -13,6 +15,35 @@ import {
syncSkillSearchDigestsForOwnerPublisherId,
} from "./functions";
type WrappedHandler = {
_handler: (ctx: unknown, args: Record<string, never>) => Promise<unknown>;
};
function hasWrappedHandler(value: unknown): value is WrappedHandler {
return typeof value === "function" && "_handler" in value && typeof value._handler === "function";
}
function getWrappedHandler(value: unknown): WrappedHandler["_handler"] {
if (!hasWrappedHandler(value)) {
throw new Error("Expected a Convex function with a test-callable _handler");
}
return value._handler;
}
function testId<TableName extends TableNames>(
tableName: TableName,
value: `${TableName}:${string}`,
): Id<TableName> {
if (!value.startsWith(`${tableName}:`)) {
throw new Error(`Expected ${value} to be a ${tableName} id`);
}
return value as Id<TableName>;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
describe("package digest sync", () => {
it("identifies GitHub mirror eligibility from skill visibility fields", () => {
expect(isGitHubMirrorEligibleSkillDoc({ softDeletedAt: undefined })).toBe(true);
@@ -677,4 +708,159 @@ describe("publisher digest scheduling", () => {
{ ownerPublisherId: "publishers:demo", cursor: "next-skills" },
);
});
it("syncs recommended rank stats into the skill search digest after wrapped skill patches", async () => {
const skillId = testId("skills", "skills:demo");
const ownerUserId = testId("users", "users:owner");
const publisherId = testId("publishers", "publishers:owner");
const digestId = testId("skillSearchDigest", "skillSearchDigest:demo");
const skill = {
_id: skillId,
_creationTime: 1,
slug: "demo-skill",
displayName: "Demo Skill",
summary: "Demo summary",
ownerUserId,
ownerPublisherId: publisherId,
tags: {},
statsDownloads: 3,
statsStars: 2,
statsInstallsCurrent: 4,
statsInstallsAllTime: 5,
stats: {
downloads: 3,
stars: 2,
installsCurrent: 4,
installsAllTime: 5,
versions: 1,
comments: 0,
},
createdAt: 10,
updatedAt: 20,
} satisfies Doc<"skills">;
const publisher = {
_id: publisherId,
_creationTime: 2,
kind: "user",
handle: "owner",
displayName: "Owner",
linkedUserId: ownerUserId,
publishedSkills: 1,
publishedPackages: 0,
totalInstalls: 5,
totalDownloads: 3,
totalStars: 2,
skillTotalInstalls: 5,
skillTotalDownloads: 3,
skillTotalStars: 2,
createdAt: 10,
updatedAt: 20,
} satisfies Doc<"publishers">;
const digest = {
_id: digestId,
_creationTime: 3,
skillId,
slug: "demo-skill",
displayName: "Demo Skill",
summary: "Demo summary",
ownerUserId,
ownerPublisherId: publisherId,
ownerHandle: "owner",
ownerKind: "user",
ownerDisplayName: "Owner",
tags: {},
statsDownloads: 3,
statsStars: 2,
statsInstallsCurrent: 4,
statsInstallsAllTime: 5,
stats: {
downloads: 3,
stars: 2,
installsCurrent: 4,
installsAllTime: 5,
versions: 1,
comments: 0,
},
createdAt: 10,
updatedAt: 20,
} satisfies Doc<"skillSearchDigest">;
const docs = new Map<string, unknown>([
[skillId, skill],
[publisherId, publisher],
[digestId, digest],
]);
const patchSkillRankStats = internalMutation({
args: {},
handler: async (ctx) => {
await ctx.db.patch(skillId, {
statsDownloads: 13,
statsStars: 7,
statsInstallsAllTime: 11,
stats: {
downloads: 13,
stars: 7,
installsCurrent: 4,
installsAllTime: 11,
versions: 1,
comments: 0,
},
});
},
});
const handler = getWrappedHandler(patchSkillRankStats);
const db = {
system: {},
normalizeId: vi.fn((tableName: string, id: string) =>
id.startsWith(`${tableName}:`) ? id : null,
),
get: vi.fn(async (first: string, second?: string) => docs.get(second ?? first) ?? null),
insert: vi.fn(async (tableName: string, value: unknown) => {
if (!isRecord(value))
throw new Error(`Expected inserted ${tableName} value to be an object`);
const insertedId = `${tableName}:inserted`;
docs.set(insertedId, { ...value, _id: insertedId, _creationTime: 0 });
return insertedId;
}),
patch: vi.fn(
async (first: string, second: string | Record<string, unknown>, third?: unknown) => {
const id = typeof second === "string" ? second : first;
const patch = typeof second === "string" ? third : second;
if (!isRecord(patch)) throw new Error(`Expected patch for ${id} to be an object`);
const existing = docs.get(id);
if (!isRecord(existing)) throw new Error(`Missing test doc ${id}`);
docs.set(id, { ...existing, ...patch });
},
),
delete: vi.fn(async (first: string, second?: string) => {
docs.delete(second ?? first);
}),
query: vi.fn((tableName: string) => ({
withIndex: vi.fn(() => ({
unique: vi.fn(async () => {
if (tableName === "skillSearchDigest") return docs.get(digestId) ?? null;
return null;
}),
collect: vi.fn(async () => []),
paginate: vi.fn(async () => ({ page: [], isDone: true, continueCursor: "" })),
take: vi.fn(async () => []),
})),
})),
};
await expect(handler({ db }, {})).resolves.toBeUndefined();
expect(docs.get(digestId)).toEqual(
expect.objectContaining({
statsDownloads: 13,
statsStars: 7,
statsInstallsAllTime: 11,
stats: expect.objectContaining({
downloads: 13,
stars: 7,
installsAllTime: 11,
}),
}),
);
});
});
+1
View File
@@ -331,6 +331,7 @@ export async function repointPackageLatestRelease(
if (!packageId || !affectedReleaseId) return;
const pkg = await ctx.db.get(packageId);
if (!pkg) return;
if (pkg.softDeletedAt) return;
const nextTags = Object.fromEntries(
Object.entries(pkg.tags).filter(([, releaseId]) => releaseId !== affectedReleaseId),
+278
View File
@@ -0,0 +1,278 @@
import { ConvexError, v } from "convex/values";
import { internal } from "./_generated/api";
import type { Id } from "./_generated/dataModel";
import type { ActionCtx } from "./_generated/server";
import { internalAction, internalMutation, internalQuery } from "./functions";
import { fetchGitHubCreatedAtByProviderAccountId } from "./lib/githubAccount";
import { getGitHubProviderAccountId } from "./lib/githubIdentity";
import { getUserByHandleOrPersonalPublisher } from "./lib/publishers";
const DEFAULT_BATCH_SIZE = 25;
const MAX_ACTION_BATCH_SIZE = 50;
const MAX_LIST_BATCH_SIZE = 500;
const DEFAULT_MAX_PAGES = 1;
const MAX_MAX_PAGES = 20;
type BackfillCandidate = {
userId: Id<"users">;
providerAccountId: string;
handle: string | null;
};
type BackfillStats = {
scanned: number;
candidates: number;
fetched: number;
patched: number;
failed: number;
missingHandles: string[];
errors: Array<{ userId: string; handle: string | null; message: string }>;
};
type BackfillPageResult = {
candidates: BackfillCandidate[];
scanned: number;
cursor: string | null;
isDone: boolean;
};
type BackfillHandlesResult = {
candidates: BackfillCandidate[];
missingHandles: string[];
};
type BackfillResult =
| { ok: true; stats: BackfillStats; cursor: string | null; isDone: boolean }
| { ok: false; rateLimited: true; stats: BackfillStats; cursor: string | null; isDone: false };
function clampPositiveInteger(value: number | undefined, fallback: number, max: number) {
if (!value || !Number.isFinite(value)) return fallback;
return Math.max(1, Math.min(max, Math.floor(value)));
}
async function candidateForUser(
ctx: Parameters<typeof getGitHubProviderAccountId>[0],
userId: Id<"users">,
): Promise<BackfillCandidate | null> {
const user = await ctx.db.get(userId);
if (!user || user.deletedAt || user.deactivatedAt || user.githubCreatedAt) return null;
const providerAccountId = await getGitHubProviderAccountId(ctx, userId);
if (!providerAccountId || !/^\d+$/.test(providerAccountId)) return null;
return { userId, providerAccountId, handle: user.handle ?? null };
}
export const listGitHubCreatedAtBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args) => {
const batchSize = clampPositiveInteger(args.batchSize, DEFAULT_BATCH_SIZE, MAX_LIST_BATCH_SIZE);
const page = await ctx.db
.query("authAccounts")
.withIndex("providerAndAccountId", (q) => q.eq("provider", "github"))
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
const candidates: BackfillCandidate[] = [];
for (const account of page.page) {
if (!/^\d+$/.test(account.providerAccountId)) continue;
const user = await ctx.db.get(account.userId);
if (!user || user.deletedAt || user.deactivatedAt || user.githubCreatedAt) continue;
candidates.push({
userId: account.userId,
providerAccountId: account.providerAccountId,
handle: user.handle ?? null,
});
}
return {
candidates,
scanned: page.page.length,
cursor: page.continueCursor,
isDone: page.isDone,
};
},
});
export const listGitHubCreatedAtBackfillHandlesInternal = internalQuery({
args: { handles: v.array(v.string()) },
handler: async (ctx, args) => {
const seen = new Set<string>();
const candidates: BackfillCandidate[] = [];
const missingHandles: string[] = [];
for (const handle of args.handles) {
const user = await getUserByHandleOrPersonalPublisher(ctx, handle);
if (!user) {
missingHandles.push(handle);
continue;
}
if (seen.has(user._id)) continue;
seen.add(user._id);
const candidate = await candidateForUser(ctx, user._id);
if (candidate) candidates.push(candidate);
}
return { candidates, missingHandles };
},
});
export const applyGitHubCreatedAtBackfillInternal = internalMutation({
args: {
userId: v.id("users"),
githubCreatedAt: v.number(),
fetchedAt: v.number(),
dryRun: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const user = await ctx.db.get(args.userId);
if (!user || user.deletedAt || user.deactivatedAt || user.githubCreatedAt) {
return { patched: false };
}
if (args.dryRun) return { patched: false };
await ctx.db.patch(args.userId, {
githubCreatedAt: args.githubCreatedAt,
githubFetchedAt: args.fetchedAt,
updatedAt: Date.now(),
});
return { patched: true };
},
});
export const applyGitHubCreatedAtBackfillBatchInternal = internalMutation({
args: {
items: v.array(
v.object({
userId: v.id("users"),
githubCreatedAt: v.number(),
}),
),
fetchedAt: v.number(),
dryRun: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
let patched = 0;
let skipped = 0;
for (const item of args.items) {
const user = await ctx.db.get(item.userId);
if (!user || user.deletedAt || user.deactivatedAt || user.githubCreatedAt) {
skipped += 1;
continue;
}
if (!args.dryRun) {
await ctx.db.patch(item.userId, {
githubCreatedAt: item.githubCreatedAt,
githubFetchedAt: args.fetchedAt,
updatedAt: Date.now(),
});
}
patched += 1;
}
return { patched, skipped };
},
});
export const backfillGitHubCreatedAtInternal = internalAction({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
maxPages: v.optional(v.number()),
dryRun: v.optional(v.boolean()),
handles: v.optional(v.array(v.string())),
},
handler: async (ctx: ActionCtx, args): Promise<BackfillResult> => {
const batchSize = clampPositiveInteger(
args.batchSize,
DEFAULT_BATCH_SIZE,
MAX_ACTION_BATCH_SIZE,
);
const maxPages = clampPositiveInteger(args.maxPages, DEFAULT_MAX_PAGES, MAX_MAX_PAGES);
const dryRun = args.dryRun ?? false;
const fetchedAt = Date.now();
const stats = {
scanned: 0,
candidates: 0,
fetched: 0,
patched: 0,
failed: 0,
missingHandles: [] as string[],
errors: [] as Array<{ userId: string; handle: string | null; message: string }>,
};
let cursor = args.cursor ?? null;
let isDone = true;
let pages = 0;
while (pages < maxPages) {
pages += 1;
const page: BackfillPageResult | BackfillHandlesResult = args.handles
? ((await ctx.runQuery(
internal.githubAccountAgeBackfill.listGitHubCreatedAtBackfillHandlesInternal,
{
handles: args.handles,
},
)) as BackfillHandlesResult)
: ((await ctx.runQuery(
internal.githubAccountAgeBackfill.listGitHubCreatedAtBackfillPageInternal,
{
cursor: cursor ?? undefined,
batchSize,
},
)) as BackfillPageResult);
const candidates = page.candidates;
stats.scanned += "scanned" in page ? page.scanned : (args.handles?.length ?? 0);
if ("missingHandles" in page) stats.missingHandles.push(...page.missingHandles);
stats.candidates += candidates.length;
for (const candidate of candidates) {
try {
const githubCreatedAt = await fetchGitHubCreatedAtByProviderAccountId(
candidate.providerAccountId,
);
stats.fetched += 1;
const result: { patched: boolean } = await ctx.runMutation(
internal.githubAccountAgeBackfill.applyGitHubCreatedAtBackfillInternal,
{
userId: candidate.userId,
githubCreatedAt,
fetchedAt,
dryRun,
},
);
if (result.patched) stats.patched += 1;
} catch (error) {
stats.failed += 1;
const message = error instanceof ConvexError ? String(error.data) : String(error);
if (stats.errors.length < 10) {
stats.errors.push({
userId: candidate.userId,
handle: candidate.handle,
message,
});
}
if (/rate limit/i.test(message)) {
return { ok: false as const, rateLimited: true as const, stats, cursor, isDone: false };
}
}
}
if (args.handles) return { ok: true as const, stats, cursor: null, isDone: true };
cursor = "cursor" in page ? page.cursor : null;
isDone = "isDone" in page ? page.isDone : true;
if (isDone) break;
}
if (!dryRun && !isDone && cursor) {
await ctx.scheduler.runAfter(
0,
internal.githubAccountAgeBackfill.backfillGitHubCreatedAtInternal,
{
cursor,
batchSize,
maxPages,
},
);
}
return { ok: true as const, stats, cursor, isDone };
},
});
+241
View File
@@ -0,0 +1,241 @@
import { ConvexError } from "convex/values";
import { beforeEach, describe, expect, it, vi } from "vitest";
vi.mock("./lib/access", () => ({
requireUser: vi.fn(),
}));
vi.mock("./lib/publishers", async () => {
const actual = await vi.importActual<typeof import("./lib/publishers")>("./lib/publishers");
return {
...actual,
requirePublisherRole: vi.fn(),
};
});
const { requireUser } = await import("./lib/access");
const { requirePublisherRole } = await import("./lib/publishers");
const { deleteForPublisherHandler } = await import("./githubSkillSources");
const { buildSkillInstallResolution } = await import("./lib/installResolver");
type Row = Record<string, unknown> & { _id: string };
function chainEq(constraints: Record<string, unknown>) {
return {
eq(field: string, value: unknown) {
constraints[field] = value;
return chainEq(constraints);
},
};
}
function matches(doc: Row, constraints: Record<string, unknown>) {
return Object.entries(constraints).every(([key, value]) => doc[key] === value);
}
function createDb(initial: Record<string, Row[]> = {}) {
const tables: Record<string, Row[]> = Object.fromEntries(
Object.entries(initial).map(([table, rows]) => [table, [...rows]]),
);
const list = (table: string) => {
tables[table] ??= [];
return tables[table];
};
const db = {
get: async (id: string) => {
const table = id.split(":")[0] ?? "";
return list(table).find((row) => row._id === id) ?? null;
},
patch: async (id: string, patch: Record<string, unknown>) => {
const table = id.split(":")[0] ?? "";
const row = list(table).find((candidate) => candidate._id === id);
if (!row) return;
for (const [key, value] of Object.entries(patch)) {
if (value === undefined) delete row[key];
else row[key] = value;
}
},
insert: async (table: string, doc: Record<string, unknown>) => {
const id = `${table}:${list(table).length + 1}`;
list(table).push({ _id: id, ...doc });
return id;
},
delete: async (id: string) => {
const table = id.split(":")[0] ?? "";
const rows = list(table);
const index = rows.findIndex((row) => row._id === id);
if (index >= 0) rows.splice(index, 1);
},
query: (table: string) => ({
withIndex: (_indexName: string, build: (q: ReturnType<typeof chainEq>) => unknown) => {
const constraints: Record<string, unknown> = {};
build(chainEq(constraints));
const matched = () => list(table).filter((row) => matches(row, constraints));
return {
collect: async () => matched(),
unique: async () => matched()[0] ?? null,
};
},
}),
};
return { db, tables };
}
describe("githubSkillSources.deleteForPublisherHandler", () => {
beforeEach(() => {
vi.mocked(requireUser).mockResolvedValue({ userId: "users:owner" } as never);
vi.mocked(requirePublisherRole).mockResolvedValue(undefined as never);
});
it("deletes a source and removes only GitHub-backed skills from that source", async () => {
const { db, tables } = createDb({
githubSkillSources: [
{
_id: "githubSkillSources:matt",
repo: "mattpocock/skills",
ownerPublisherId: "publishers:openclaw",
createdAt: 1,
updatedAt: 2,
},
],
githubSkillContents: [
{
_id: "githubSkillContents:one",
skillId: "skills:github",
githubSourceId: "githubSkillSources:matt",
},
],
skills: [
{
_id: "skills:github",
slug: "source-backed",
displayName: "Source Backed",
installKind: "github",
githubSourceId: "githubSkillSources:matt",
githubPath: "skills/source-backed",
githubCurrentCommit: "a".repeat(40),
githubCurrentContentHash: "hash-source-backed",
githubCurrentStatus: "present",
githubScanStatus: "clean",
ownerUserId: "users:owner",
ownerPublisherId: "publishers:openclaw",
forkOf: undefined,
tags: {},
capabilityTags: undefined,
badges: {},
stats: {
comments: 0,
downloads: 0,
installsAllTime: 0,
installsCurrent: 0,
stars: 0,
versions: 0,
},
moderationStatus: "active",
moderationFlags: [],
isSuspicious: false,
createdAt: 1,
updatedAt: 2,
softDeletedAt: undefined,
},
{
_id: "skills:direct",
slug: "direct-upload",
displayName: "Direct Upload",
ownerPublisherId: "publishers:openclaw",
softDeletedAt: undefined,
},
{
_id: "skills:other-source",
slug: "other-source",
displayName: "Other Source",
installKind: "github",
githubSourceId: "githubSkillSources:other",
githubPath: "skills/other-source",
githubCurrentCommit: "b".repeat(40),
githubCurrentContentHash: "hash-other-source",
githubCurrentStatus: "present",
githubScanStatus: "clean",
ownerPublisherId: "publishers:openclaw",
softDeletedAt: undefined,
},
],
});
await expect(
deleteForPublisherHandler({ db } as never, {
ownerPublisherId: "publishers:openclaw" as never,
sourceId: "githubSkillSources:matt" as never,
now: 123,
}),
).resolves.toEqual({ ok: true, deletedSkills: 1 });
expect(requirePublisherRole).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
publisherId: "publishers:openclaw",
userId: "users:owner",
allowed: ["admin"],
}),
);
expect(tables.githubSkillSources).toHaveLength(0);
expect(tables.githubSkillContents).toHaveLength(0);
const deletedSkill = tables.skills.find((skill) => skill._id === "skills:github");
expect(deletedSkill).toMatchObject({
softDeletedAt: 123,
githubRemovedAt: 123,
githubCurrentStatus: "missing",
updatedAt: 123,
});
expect(tables.skillSearchDigest).toEqual([
expect.objectContaining({
skillId: "skills:github",
githubCurrentStatus: "missing",
githubScanStatus: "clean",
softDeletedAt: 123,
}),
]);
expect(
buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: deletedSkill as never,
source: null,
}),
).toMatchObject({
ok: false,
reason: "github_upstream_removed",
status: 410,
});
expect(tables.skills.find((skill) => skill._id === "skills:direct")).toMatchObject({
softDeletedAt: undefined,
});
expect(tables.skills.find((skill) => skill._id === "skills:other-source")).toMatchObject({
githubCurrentStatus: "present",
softDeletedAt: undefined,
});
});
it("rejects deleting a source from another publisher", async () => {
const { db } = createDb({
githubSkillSources: [
{
_id: "githubSkillSources:matt",
repo: "mattpocock/skills",
ownerPublisherId: "publishers:other",
createdAt: 1,
updatedAt: 2,
},
],
});
await expect(
deleteForPublisherHandler({ db } as never, {
ownerPublisherId: "publishers:openclaw" as never,
sourceId: "githubSkillSources:matt" as never,
now: 123,
}),
).rejects.toBeInstanceOf(ConvexError);
});
});
+206
View File
@@ -0,0 +1,206 @@
import { ConvexError, v } from "convex/values";
import type { Doc, Id } from "./_generated/dataModel";
import type { MutationCtx, QueryCtx } from "./_generated/server";
import { internalQuery, mutation, query } from "./functions";
import { requireUser } from "./lib/access";
import { adjustGlobalPublicSkillsCount, getPublicSkillVisibilityDelta } from "./lib/globalStats";
import { isOfficialPublisher } from "./lib/officialPublishers";
import { isPublisherActive, isPublisherRoleAllowed, requirePublisherRole } from "./lib/publishers";
import { syncSkillSearchDigestForSkill } from "./lib/skillSearchDigest";
type PublicGitHubSkillSource = Pick<
Doc<"githubSkillSources">,
| "_id"
| "repo"
| "defaultBranch"
| "lastSyncStatus"
| "lastSyncError"
| "lastSyncErrorAt"
| "displayManifestStatus"
| "displayManifestFetchedAt"
| "displayManifestCommit"
| "lastSyncIssues"
| "lastSyncInvalidSkills"
| "createdAt"
| "updatedAt"
> & {
ownerPublisher: Pick<Doc<"publishers">, "_id" | "handle" | "displayName"> | null;
skills: Array<
Pick<Doc<"skills">, "_id" | "slug" | "displayName" | "githubPath" | "githubCurrentStatus">
>;
};
export const getByIdInternal = internalQuery({
args: { sourceId: v.id("githubSkillSources") },
handler: async (ctx, args) => ctx.db.get(args.sourceId),
});
async function toPublicGitHubSkillSource(
ctx: Pick<QueryCtx, "db">,
source: Doc<"githubSkillSources">,
): Promise<PublicGitHubSkillSource> {
const skills = await ctx.db
.query("skills")
.withIndex("by_github_source", (q) => q.eq("githubSourceId", source._id))
.collect();
const visibleGitHubSkills = skills
.filter((skill) => skill.installKind === "github" && !skill.softDeletedAt)
.sort((a, b) => a.displayName.localeCompare(b.displayName))
.map((skill) => ({
_id: skill._id,
slug: skill.slug,
displayName: skill.displayName,
githubPath: skill.githubPath,
githubCurrentStatus: skill.githubCurrentStatus,
}));
const ownerPublisher = source.ownerPublisherId ? await ctx.db.get(source.ownerPublisherId) : null;
return {
_id: source._id as Id<"githubSkillSources">,
repo: source.repo,
ownerPublisher: ownerPublisher
? {
_id: ownerPublisher._id,
handle: ownerPublisher.handle,
displayName: ownerPublisher.displayName,
}
: null,
defaultBranch: source.defaultBranch,
lastSyncStatus: source.lastSyncStatus,
lastSyncError: source.lastSyncError,
lastSyncErrorAt: source.lastSyncErrorAt,
displayManifestStatus: source.displayManifestStatus,
displayManifestFetchedAt: source.displayManifestFetchedAt,
displayManifestCommit: source.displayManifestCommit,
lastSyncIssues: source.lastSyncIssues,
lastSyncInvalidSkills: source.lastSyncInvalidSkills,
createdAt: source.createdAt,
updatedAt: source.updatedAt,
skills: visibleGitHubSkills,
};
}
export const listForPublisher = query({
args: { ownerPublisherId: v.id("publishers") },
handler: async (ctx, args): Promise<PublicGitHubSkillSource[]> => {
const { userId } = await requireUser(ctx);
await requirePublisherRole(ctx, {
publisherId: args.ownerPublisherId,
userId,
allowed: ["admin"],
});
const sources = await ctx.db
.query("githubSkillSources")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", args.ownerPublisherId))
.collect();
const sortedSources = sources.sort((a, b) => b.updatedAt - a.updatedAt);
return await Promise.all(sortedSources.map((source) => toPublicGitHubSkillSource(ctx, source)));
},
});
export const listForManageableOfficialPublishers = query({
args: {},
handler: async (ctx): Promise<PublicGitHubSkillSource[]> => {
const { userId } = await requireUser(ctx);
const memberships = await ctx.db
.query("publisherMembers")
.withIndex("by_user", (q) => q.eq("userId", userId))
.collect();
const ownerPublisherIds: Id<"publishers">[] = [];
for (const membership of memberships) {
if (!isPublisherRoleAllowed(membership.role, ["admin"])) continue;
const publisher = await ctx.db.get(membership.publisherId);
if (
!publisher ||
publisher.kind !== "org" ||
!isPublisherActive(publisher) ||
!(await isOfficialPublisher(ctx, publisher))
) {
continue;
}
ownerPublisherIds.push(publisher._id);
}
const sourceGroups = await Promise.all(
ownerPublisherIds.map((ownerPublisherId) =>
ctx.db
.query("githubSkillSources")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", ownerPublisherId))
.collect(),
),
);
const sortedSources = sourceGroups.flat().sort((a, b) => b.updatedAt - a.updatedAt);
return await Promise.all(sortedSources.map((source) => toPublicGitHubSkillSource(ctx, source)));
},
});
export async function deleteForPublisherHandler(
ctx: MutationCtx,
args: {
ownerPublisherId: Id<"publishers">;
sourceId: Id<"githubSkillSources">;
now?: number;
},
) {
const { userId } = await requireUser(ctx);
await requirePublisherRole(ctx, {
publisherId: args.ownerPublisherId,
userId,
allowed: ["admin"],
});
const source = await ctx.db.get(args.sourceId);
if (!source || source.ownerPublisherId !== args.ownerPublisherId) {
throw new ConvexError("GitHub source not found.");
}
const now = args.now ?? Date.now();
const contents = await ctx.db
.query("githubSkillContents")
.withIndex("by_github_source", (q) => q.eq("githubSourceId", args.sourceId))
.collect();
for (const content of contents) {
await ctx.db.delete(content._id);
}
const skills = await ctx.db
.query("skills")
.withIndex("by_github_source", (q) => q.eq("githubSourceId", args.sourceId))
.collect();
let deletedSkills = 0;
let publicSkillDelta = 0;
for (const skill of skills) {
if (skill.installKind !== "github") continue;
const nextSkill: Doc<"skills"> = {
...skill,
softDeletedAt: skill.softDeletedAt ?? now,
githubCurrentStatus: "missing",
githubRemovedAt: skill.githubRemovedAt ?? now,
updatedAt: now,
};
publicSkillDelta += getPublicSkillVisibilityDelta(skill, nextSkill);
await ctx.db.patch(skill._id, {
softDeletedAt: nextSkill.softDeletedAt,
githubCurrentStatus: nextSkill.githubCurrentStatus,
githubRemovedAt: nextSkill.githubRemovedAt,
updatedAt: now,
});
await syncSkillSearchDigestForSkill(ctx, nextSkill);
deletedSkills += 1;
}
if (publicSkillDelta !== 0) {
await adjustGlobalPublicSkillsCount(ctx, publicSkillDelta, now);
}
await ctx.db.delete(args.sourceId);
return { ok: true as const, deletedSkills };
}
export const deleteForPublisher: ReturnType<typeof mutation> = mutation({
args: {
ownerPublisherId: v.id("publishers"),
sourceId: v.id("githubSkillSources"),
},
handler: async (ctx, args) => deleteForPublisherHandler(ctx, args),
});
+249
View File
@@ -0,0 +1,249 @@
/* @vitest-environment node */
import { describe, expect, it } from "vitest";
import {
applyGitHubSkillSourceSyncHandler,
applyGitHubSkillVerificationResultHandler,
configurePublicGitHubSkillSourceHandler,
upsertGitHubSkillContentHandler,
verifyGitHubSkillHandler,
} from "./githubSkillSync";
import { buildSkillInstallResolution } from "./lib/installResolver";
type Row = Record<string, unknown> & { _id: string };
function chainEq(constraints: Record<string, unknown>) {
return {
eq(field: string, value: unknown) {
constraints[field] = value;
return chainEq(constraints);
},
};
}
function matches(doc: Row, constraints: Record<string, unknown>) {
return Object.entries(constraints).every(([key, value]) => doc[key] === value);
}
function createDb(initial: Record<string, Row[]> = {}) {
const tables: Record<string, Row[]> = Object.fromEntries(
Object.entries(initial).map(([table, rows]) => [table, [...rows]]),
);
const counters: Record<string, number> = {};
const list = (table: string) => {
tables[table] ??= [];
return tables[table];
};
const db = {
get: async (id: string) => {
const table = id.split(":")[0] ?? "";
return list(table).find((row) => row._id === id) ?? null;
},
insert: async (table: string, doc: Record<string, unknown>) => {
counters[table] = (counters[table] ?? 0) + 1;
const inserted = {
_id: `${table}:new-${counters[table]}`,
_creationTime: counters[table],
...doc,
};
list(table).push(inserted);
return inserted._id;
},
patch: async (id: string, patch: Record<string, unknown>) => {
const table = id.split(":")[0] ?? "";
const row = list(table).find((candidate) => candidate._id === id);
if (!row) return;
for (const [key, value] of Object.entries(patch)) {
if (value === undefined) delete row[key];
else row[key] = value;
}
},
query: (table: string) => ({
withIndex: (_indexName: string, build?: (q: ReturnType<typeof chainEq>) => unknown) => {
const constraints: Record<string, unknown> = {};
build?.(chainEq(constraints));
const matched = () => list(table).filter((row) => matches(row, constraints));
return {
collect: async () => matched(),
unique: async () => matched()[0] ?? null,
};
},
}),
};
return { db, tables };
}
function getSkill(tables: Record<string, Row[]>, slug: string) {
const skill = tables.skills?.find((row) => row.slug === slug);
if (!skill) throw new Error(`Live GitHub canary did not discover skill: ${slug}`);
return skill;
}
function resolveInstallFromTables(tables: Record<string, Row[]>, slug: string) {
const skill = getSkill(tables, slug);
const source =
typeof skill.githubSourceId === "string"
? (tables.githubSkillSources?.find((row) => row._id === skill.githubSourceId) ?? null)
: null;
return buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: skill as never,
source: source as never,
});
}
const liveCanaryEnabled = process.env.CLAWHUB_LIVE_GITHUB_CANARY === "1";
const itIfLive = liveCanaryEnabled ? it : it.skip;
describe("GitHub-backed skills live canary", () => {
itIfLive(
"discovers and verifies an installable skill from a real GitHub repo",
{ timeout: 45_000 },
async () => {
const repo = process.env.CLAWHUB_LIVE_GITHUB_REPO?.trim() || "openclaw/agent-skills";
const skillSlug = process.env.CLAWHUB_LIVE_GITHUB_SKILL?.trim() || "handoff";
const { db, tables } = createDb({
globalStats: [
{
_id: "globalStats:default",
key: "default",
activeSkillsCount: 0,
updatedAt: 1,
},
],
});
const scheduler = { runAfter: async () => undefined };
let now = Date.now();
const actionCtx = {
runQuery: async (_query: unknown, args: Record<string, unknown>) => {
if ("ownerPublisherId" in args && "actorUserId" in args) {
return {
ownerUserId: "users:live-owner",
existingSource:
tables.githubSkillSources?.find((source) => source.repo === repo) ?? null,
official: true,
};
}
if ("skillId" in args) {
const skill = tables.skills?.find((row) => row._id === args.skillId);
const source =
skill && typeof skill.githubSourceId === "string"
? tables.githubSkillSources?.find((row) => row._id === skill.githubSourceId)
: null;
return skill && source ? { skill, source } : null;
}
if ("sourceId" in args) {
return (tables.skills ?? []).flatMap((skill) => {
if (
skill.githubSourceId !== args.sourceId ||
skill.installKind !== "github" ||
skill.githubCurrentStatus !== "present" ||
typeof skill.githubPath !== "string" ||
typeof skill.githubCurrentContentHash !== "string"
) {
return [];
}
return [
{
skillId: skill._id,
githubPath: skill.githubPath,
githubCurrentContentHash: skill.githubCurrentContentHash,
},
];
});
}
throw new Error(`unexpected live canary query args: ${JSON.stringify(args)}`);
},
runMutation: async (_mutation: unknown, args: Record<string, unknown>) => {
if ("snapshot" in args) {
return await applyGitHubSkillSourceSyncHandler(
{ db, scheduler } as never,
{
...args,
now,
} as never,
);
}
if ("scanStatus" in args && "contentHash" in args) {
return await applyGitHubSkillVerificationResultHandler(
{ db } as never,
{
...args,
now,
} as never,
);
}
if ("discovered" in args && "commit" in args) {
return await upsertGitHubSkillContentHandler(
{ db } as never,
{
...args,
now,
} as never,
);
}
throw new Error(`unexpected live canary mutation args: ${JSON.stringify(args)}`);
},
auth: { getUserIdentity: async () => null },
};
const configured = await configurePublicGitHubSkillSourceHandler(
actionCtx as never,
{
ownerPublisherId: "publishers:live" as never,
repo,
},
fetch,
{ userId: "users:live-owner" as never },
);
expect(configured.stats.discovered).toBeGreaterThan(0);
expect(configured.manifestStatus === "missing" || configured.manifestStatus === "ok").toBe(
true,
);
expect(configured.commit).toMatch(/^[a-f0-9]{40}$/);
let skill = getSkill(tables, skillSlug);
expect(skill).toMatchObject({
installKind: "github",
githubPath: `skills/${skillSlug}`,
githubCurrentCommit: configured.commit,
githubCurrentStatus: "present",
githubScanStatus: "pending",
});
expect(resolveInstallFromTables(tables, skillSlug)).toMatchObject({
ok: false,
reason: "github_verification_pending",
});
now = Date.now();
const verified = await verifyGitHubSkillHandler(
actionCtx as never,
{
skillId: skill._id as never,
contentHash: skill.githubCurrentContentHash as string,
},
fetch,
);
expect(verified).toMatchObject({ ok: true, scanStatus: "clean" });
skill = getSkill(tables, skillSlug);
expect(skill).toMatchObject({
githubCurrentCommit: configured.commit,
githubScanStatus: "clean",
moderationStatus: "active",
});
expect(resolveInstallFromTables(tables, skillSlug)).toMatchObject({
ok: true,
installKind: "github",
github: {
repo,
path: `skills/${skillSlug}`,
commit: configured.commit,
contentHash: skill.githubCurrentContentHash,
},
});
},
);
});
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+42
View File
@@ -8,6 +8,7 @@ import {
cliDeviceTokenHttp,
cliSkillDeleteHttp,
cliSkillUndeleteHttp,
cliTelemetryInstallHttp,
cliTelemetrySyncHttp,
cliUploadUrlHttp,
cliWhoamiHttp,
@@ -35,6 +36,10 @@ import {
publishSoulV1Http,
resolveSkillVersionV1Http,
searchSkillsV1Http,
skillScanBatchStatusV1Http,
skillScanBatchSubmitV1Http,
skillScanGetRouterV1Http,
skillScanSubmitV1Http,
skillSecurityVerdictsV1Http,
skillsDeleteRouterV1Http,
skillsGetRouterV1Http,
@@ -46,6 +51,7 @@ import {
starsPostRouterV1Http,
transfersGetRouterV1Http,
banAppealContextV1Http,
usersGetRouterV1Http,
usersListV1Http,
usersPostRouterV1Http,
verifyDocsSessionV1Http,
@@ -87,6 +93,12 @@ http.route({
handler: listSkillsV1Http,
});
http.route({
pathPrefix: `${ApiRoutes.skillScans}/`,
method: "GET",
handler: skillScanGetRouterV1Http,
});
http.route({
path: ApiRoutes.packages,
method: "GET",
@@ -141,6 +153,24 @@ http.route({
handler: publishSkillV1Http,
});
http.route({
path: ApiRoutes.skillScans,
method: "POST",
handler: skillScanSubmitV1Http,
});
http.route({
path: `${ApiRoutes.skillScans}/batch`,
method: "POST",
handler: skillScanBatchSubmitV1Http,
});
http.route({
path: `${ApiRoutes.skillScans}/batch/status`,
method: "POST",
handler: skillScanBatchStatusV1Http,
});
http.route({
path: ApiRoutes.packages,
method: "POST",
@@ -243,6 +273,12 @@ http.route({
handler: banAppealContextV1Http,
});
http.route({
pathPrefix: `${ApiRoutes.users}/`,
method: "GET",
handler: usersGetRouterV1Http,
});
http.route({
path: ApiRoutes.users,
method: "GET",
@@ -327,6 +363,12 @@ http.route({
handler: cliPublishHttp,
});
http.route({
path: LegacyApiRoutes.cliTelemetryInstall,
method: "POST",
handler: cliTelemetryInstallHttp,
});
http.route({
path: LegacyApiRoutes.cliTelemetrySync,
method: "POST",
+61 -9
View File
@@ -4,13 +4,15 @@ import { afterEach, describe, expect, it, vi } from "vitest";
vi.mock("./lib/apiTokenAuth", () => ({
getOptionalApiTokenUser: vi.fn(),
requireApiTokenUser: vi.fn(),
requirePackagePublishAuth: vi.fn(),
}));
vi.mock("./skills", () => ({
publishVersionForUser: vi.fn(),
}));
const { getOptionalApiTokenUser, requireApiTokenUser } = await import("./lib/apiTokenAuth");
const { getOptionalApiTokenUser, requireApiTokenUser, requirePackagePublishAuth } =
await import("./lib/apiTokenAuth");
const { publishVersionForUser } = await import("./skills");
const { __handlers } = await import("./httpApi");
const { hashSkillFiles } = await import("./lib/skills");
@@ -23,6 +25,7 @@ describe("httpApi handlers", () => {
afterEach(() => {
vi.mocked(getOptionalApiTokenUser).mockReset();
vi.mocked(requireApiTokenUser).mockReset();
vi.mocked(requirePackagePublishAuth).mockReset();
vi.mocked(publishVersionForUser).mockReset();
});
@@ -240,7 +243,7 @@ describe("httpApi handlers", () => {
it("cliWhoamiHttp returns 401 on auth failure", async () => {
vi.mocked(requireApiTokenUser).mockRejectedValueOnce(
new Error(
"Unauthorized: This ClawHub account is not in good standing and cannot use API tokens. If you believe this is a mistake, contact security@openclaw.ai.",
"Unauthorized: This ClawHub account is not in good standing and cannot use API tokens. If you believe this is a mistake, open a GitHub issue: https://github.com/openclaw/clawhub/issues/new.",
),
);
const response = await __handlers.cliWhoamiHandler(
@@ -264,12 +267,12 @@ describe("httpApi handlers", () => {
expect(json.user.handle).toBe("p");
});
it("cliTelemetrySyncHttp forwards roots and returns ok", async () => {
it("cliTelemetryInstallHttp forwards roots and returns ok", async () => {
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: "users:1" } as never);
const runMutation = vi.fn().mockResolvedValue(null);
const response = await __handlers.cliTelemetrySyncHandler(
const response = await __handlers.cliTelemetryInstallHandler(
makeCtx({ runMutation }),
new Request("https://x/api/cli/telemetry/sync", {
new Request("https://x/api/cli/telemetry/install", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
@@ -288,6 +291,22 @@ describe("httpApi handlers", () => {
expect(runMutation).toHaveBeenCalledTimes(1);
});
it("cliTelemetrySyncHttp remains a backwards-compatible alias", async () => {
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: "users:1" } as never);
const runMutation = vi.fn().mockResolvedValue(null);
const response = await __handlers.cliTelemetrySyncHandler(
makeCtx({ runMutation }),
new Request("https://x/api/cli/telemetry/sync", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ roots: [] }),
}),
);
expect(response.status).toBe(200);
expect(runMutation).toHaveBeenCalledTimes(1);
});
it("cliTelemetrySyncHttp returns 400 on invalid payload", async () => {
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: "users:1" } as never);
const response = await __handlers.cliTelemetrySyncHandler(
@@ -444,18 +463,51 @@ describe("httpApi handlers", () => {
});
it("cliUploadUrlHttp returns uploadUrl", async () => {
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({ userId: "user1" } as never);
const runMutation = vi.fn().mockResolvedValue("https://upload.local");
vi.mocked(requirePackagePublishAuth).mockResolvedValueOnce({
kind: "user",
userId: "user1",
} as never);
const runMutation = vi.fn().mockResolvedValue({
uploadUrl: "https://upload.local",
uploadTicket: "packagePublishUploadTickets:1",
});
const response = await __handlers.cliUploadUrlHandler(
makeCtx({ runMutation }),
new Request("https://x/api/cli/upload-url", { method: "POST" }),
);
expect(response.status).toBe(200);
expect(await response.json()).toEqual({ uploadUrl: "https://upload.local" });
expect(await response.json()).toEqual({
uploadUrl: "https://upload.local",
uploadTicket: "packagePublishUploadTickets:1",
});
});
it("cliUploadUrlHttp accepts package publish tokens", async () => {
vi.mocked(requirePackagePublishAuth).mockResolvedValueOnce({
kind: "github-actions",
publishToken: { _id: "packagePublishTokens:1" },
} as never);
const runMutation = vi.fn().mockResolvedValue({
uploadUrl: "https://upload.local/package",
uploadTicket: "packagePublishUploadTickets:2",
});
const response = await __handlers.cliUploadUrlHandler(
makeCtx({ runMutation }),
new Request("https://x/api/cli/upload-url", { method: "POST" }),
);
expect(response.status).toBe(200);
expect(await response.json()).toEqual({
uploadUrl: "https://upload.local/package",
uploadTicket: "packagePublishUploadTickets:2",
});
expect(runMutation).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({ publishTokenId: "packagePublishTokens:1" }),
);
});
it("cliUploadUrlHttp returns 401 when unauthorized", async () => {
vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error("Unauthorized"));
vi.mocked(requirePackagePublishAuth).mockRejectedValueOnce(new Error("Unauthorized"));
const response = await __handlers.cliUploadUrlHandler(
makeCtx({}),
new Request("https://x/api/cli/upload-url", { method: "POST" }),
+16 -8
View File
@@ -10,7 +10,7 @@ import { api, internal } from "./_generated/api";
import type { Id } from "./_generated/dataModel";
import type { ActionCtx } from "./_generated/server";
import { httpAction } from "./functions";
import { requireApiTokenUser } from "./lib/apiTokenAuth";
import { requireApiTokenUser, requirePackagePublishAuth } from "./lib/apiTokenAuth";
import { corsHeaders, mergeHeaders } from "./lib/httpHeaders";
import { applyRateLimit } from "./lib/httpRateLimit";
import { parseBooleanQueryParam, resolveBooleanQueryParam } from "./lib/httpUtils";
@@ -148,11 +148,16 @@ export const cliWhoamiHttp = httpAction(cliWhoamiHandler);
async function cliUploadUrlHandler(ctx: ActionCtx, request: Request) {
try {
const { userId } = await requireApiTokenUser(ctx, request);
const uploadUrl = await ctx.runMutation(internal.uploads.generateUploadUrlForUserInternal, {
userId,
});
return json({ uploadUrl });
const auth = await requirePackagePublishAuth(ctx, request);
const upload =
auth.kind === "user"
? await ctx.runMutation(internal.uploads.createPackagePublishUploadForUserInternal, {
userId: auth.userId,
})
: await ctx.runMutation(internal.uploads.createPackagePublishUploadForTokenInternal, {
publishTokenId: auth.publishToken._id,
});
return json(upload);
} catch (error) {
return text(formatAuthFailure(error), 401);
}
@@ -222,7 +227,7 @@ export const cliSkillUndeleteHttp = httpAction((ctx, request) =>
cliSkillDeleteHandler(ctx, request, false),
);
async function cliTelemetrySyncHandler(ctx: ActionCtx, request: Request) {
async function cliTelemetryInstallHandler(ctx: ActionCtx, request: Request) {
let body: unknown;
try {
body = await request.json();
@@ -253,7 +258,9 @@ async function cliTelemetrySyncHandler(ctx: ActionCtx, request: Request) {
}
}
export const cliTelemetrySyncHttp = httpAction(cliTelemetrySyncHandler);
const cliTelemetrySyncHandler = cliTelemetryInstallHandler;
export const cliTelemetryInstallHttp = httpAction(cliTelemetryInstallHandler);
export const cliTelemetrySyncHttp = httpAction(cliTelemetryInstallHandler);
async function cliDeviceCodeHandler(ctx: ActionCtx, request: Request) {
if (request.method !== "POST") return text("Method not allowed", 405);
@@ -387,6 +394,7 @@ export const __handlers = {
cliUploadUrlHandler,
cliPublishHandler,
cliSkillDeleteHandler,
cliTelemetryInstallHandler,
cliTelemetrySyncHandler,
cliDeviceCodeHandler,
cliDeviceTokenHandler,
File diff suppressed because it is too large Load Diff
+78 -1
View File
@@ -2,7 +2,12 @@
import { describe, expect, it, vi } from "vitest";
import type { Id } from "./_generated/dataModel";
import type { ActionCtx } from "./_generated/server";
import { formatUserFacingErrorMessage, resolveVersionTagsBatch } from "./httpApiV1/shared";
import {
formatUserFacingErrorMessage,
parseMultipartSkillScan,
resolveVersionTagsBatch,
softDeleteErrorToResponse,
} from "./httpApiV1/shared";
function makeCtx() {
return {
@@ -28,6 +33,54 @@ describe("http API v1 shared helpers", () => {
).toBe("Publisher not found");
});
it("maps soft-delete validation failures to 400 with cleaned messages", async () => {
const response = softDeleteErrorToResponse(
"package",
new Error(
"[CONVEX M] [Request ID: abc] Server Error Called by client Uncaught ConvexError: Package name must be lowercase and npm-safe (example: @scope/name or plugin-name)",
),
{},
);
expect(response.status).toBe(400);
await expect(response.text()).resolves.toBe(
"Package name must be lowercase and npm-safe (example: @scope/name or plugin-name)",
);
});
it("maps reserved package route validation failures to 400 with cleaned messages", async () => {
const response = softDeleteErrorToResponse(
"package",
new Error(
'[CONVEX M] [Request ID: abc] Server Error Called by client Uncaught ConvexError: Package name "publish" is reserved for ClawHub routes. Use a scoped name or choose a different package name.',
),
{},
);
expect(response.status).toBe(400);
await expect(response.text()).resolves.toBe(
'Package name "publish" is reserved for ClawHub routes. Use a scoped name or choose a different package name.',
);
});
it("keeps unknown soft-delete failures generic 500s", async () => {
const response = softDeleteErrorToResponse("soul", new Error("boom"), {});
expect(response.status).toBe(500);
await expect(response.text()).resolves.toBe("Internal Server Error");
});
it("keeps unrelated reserved-word failures generic 500s", async () => {
const response = softDeleteErrorToResponse(
"package",
new Error("database reserved capacity exceeded"),
{},
);
expect(response.status).toBe(500);
await expect(response.text()).resolves.toBe("Internal Server Error");
});
it("resolves latest tags without reading version documents", async () => {
const ctx = makeCtx();
const versionId = "skillVersions:latest" as Id<"skillVersions">;
@@ -82,4 +135,28 @@ describe("http API v1 shared helpers", () => {
expect(result).toEqual([{ stable: "1.5.0" }]);
});
it("validates skill scan multipart payloads before storing uploaded files", async () => {
const form = new FormData();
form.set("payload", JSON.stringify({ source: { kind: "upload" }, update: true }));
form.append("files", new Blob(["# Demo"], { type: "text/markdown" }), "SKILL.md");
const request = new Request("https://clawhub.ai/api/v1/skills/-/scan", {
method: "POST",
body: form,
});
const store = vi.fn();
const ctx = {
storage: {
store,
delete: vi.fn(),
},
} as unknown as ActionCtx;
await expect(
parseMultipartSkillScan(ctx, request, () => {
throw new Error("update is not valid for uploaded scans");
}),
).rejects.toThrow("update is not valid for uploaded scans");
expect(store).not.toHaveBeenCalled();
});
});
+11
View File
@@ -20,6 +20,10 @@ import {
publishSkillV1Handler,
resolveSkillVersionV1Handler,
searchSkillsV1Handler,
skillScanBatchStatusV1Handler,
skillScanBatchSubmitV1Handler,
skillScanGetRouterV1Handler,
skillScanSubmitV1Handler,
skillSecurityVerdictsV1Handler,
skillsDeleteRouterV1Handler,
skillsGetRouterV1Handler,
@@ -36,6 +40,7 @@ import { starsDeleteRouterV1Handler, starsPostRouterV1Handler } from "./httpApiV
import { transfersGetRouterV1Handler } from "./httpApiV1/transfersV1";
import {
banAppealContextV1Handler,
usersGetRouterV1Handler,
usersListV1Handler,
usersPostRouterV1Handler,
} from "./httpApiV1/usersV1";
@@ -61,6 +66,10 @@ export const listSkillsV1Http = httpAction(listSkillsV1Handler);
export const skillsGetRouterV1Http = httpAction(skillsGetRouterV1Handler);
export const publishSkillV1Http = httpAction(publishSkillV1Handler);
export const skillSecurityVerdictsV1Http = httpAction(skillSecurityVerdictsV1Handler);
export const skillScanSubmitV1Http = httpAction(skillScanSubmitV1Handler);
export const skillScanGetRouterV1Http = httpAction(skillScanGetRouterV1Handler);
export const skillScanBatchSubmitV1Http = httpAction(skillScanBatchSubmitV1Handler);
export const skillScanBatchStatusV1Http = httpAction(skillScanBatchStatusV1Handler);
export const skillsPostRouterV1Http = httpAction(skillsPostRouterV1Handler);
export const skillsDeleteRouterV1Http = httpAction(skillsDeleteRouterV1Handler);
export const exportSkillsV1Http = httpAction(exportSkillsV1Handler);
@@ -76,6 +85,7 @@ export const starsDeleteRouterV1Http = httpAction(starsDeleteRouterV1Handler);
export const transfersGetRouterV1Http = httpAction(transfersGetRouterV1Handler);
export const whoamiV1Http = httpAction(whoamiV1Handler);
export const usersGetRouterV1Http = httpAction(usersGetRouterV1Handler);
export const usersPostRouterV1Http = httpAction(usersPostRouterV1Handler);
export const usersListV1Http = httpAction(usersListV1Handler);
export const banAppealContextV1Http = httpAction(banAppealContextV1Handler);
@@ -112,6 +122,7 @@ export const __handlers = {
starsDeleteRouterV1Handler,
transfersGetRouterV1Handler,
whoamiV1Handler,
usersGetRouterV1Handler,
usersPostRouterV1Handler,
usersListV1Handler,
banAppealContextV1Handler,
+324 -153
View File
@@ -11,20 +11,23 @@ import {
PackageReportRequestSchema,
PackageReportTriageRequestSchema,
PackageReleaseModerationRequestSchema,
PackagePublishRequestSchema,
PackagePublishMetadataSchema,
PackageTransferRequestSchema,
PackageTrustedPublisherUpsertRequestSchema,
PublishTokenMintRequestSchema,
isPluginCategorySlug,
parseArk,
type PackagePublishMetadata,
type PackageAppealListStatus,
type PackageModerationQueueStatus,
type PackageOfficialMigrationListPhase,
type PackageReportListStatus,
type ServerPackagePublishRequest,
} from "clawhub-schema";
import { api, internal } from "../_generated/api";
import type { Doc, Id } from "../_generated/dataModel";
import type { ActionCtx } from "../_generated/server";
import { buildDownloadMetricArgs, getDownloadIdentity } from "../downloadMetrics";
import { getOptionalActiveAuthUserIdFromAction } from "../lib/access";
import { getOptionalApiTokenUserId } from "../lib/apiTokenAuth";
import { parseClawPack, sha256Base64, sha256Hex } from "../lib/clawpack";
@@ -43,9 +46,13 @@ import {
} from "../lib/packageSecurity";
import {
getClawPackSizeError,
getPackageMultipartSizeError,
getPublishFileSizeError,
getPublishTotalSizeError,
isPackageMultipartUploadTooLarge,
MAX_CLAWPACK_BYTES,
MAX_PUBLISH_FILE_BYTES,
MAX_PUBLISH_TOTAL_BYTES,
} from "../lib/publishLimits";
import { getPublicSkillFileAccessBlock, isSkillVersionForSkill } from "../lib/skillFileAccess";
import { isMacJunkPath, isTextFile } from "../lib/skills";
@@ -117,9 +124,15 @@ const internalRefs = internal as unknown as {
backfillPackageArtifactKindsInternal: unknown;
listPackageModerationQueueInternal: unknown;
};
downloadMetrics: {
recordDownloadMetricInternal: unknown;
};
packagePublishTokens: {
createInternal: unknown;
};
uploads: {
consumePackagePublishUploadTicketInternal: unknown;
};
skills: {
getSkillBySlugInternal: unknown;
searchPackageCatalogForHttpInternal: unknown;
@@ -218,6 +231,7 @@ function normalizeCapabilityTagSegment(value: string) {
const PACKAGE_FAMILY_VALUES = ["skill", "code-plugin", "bundle-plugin"] as const;
const PACKAGE_CHANNEL_VALUES = ["official", "community", "private"] as const;
const PACKAGE_LIST_SORT_VALUES = ["updated", "downloads"] as const;
function invalidQueryParamMessage(name: string) {
return `Invalid ${name} query parameter`;
@@ -379,6 +393,7 @@ type PackageListQueryArgs = {
executesCode?: boolean;
capabilityTag?: string;
category?: string;
sort?: (typeof PACKAGE_LIST_SORT_VALUES)[number];
viewerUserId?: Id<"users">;
paginationOpts: { cursor: string | null; numItems: number };
};
@@ -634,9 +649,11 @@ function releaseArtifactUrls(request: Request, packageName: string, release: Rel
async function streamClawPackRelease(
ctx: ActionCtx,
request: Request,
rateHeaders: HeadersInit,
pkg: PublicPackageDocLike,
release: ReleaseLike,
viewerUserId: Id<"users"> | null,
statKind: "download" | "install" = "download",
) {
const securityBlock = getReleaseSecurityBlock(release);
@@ -647,13 +664,24 @@ async function streamClawPackRelease(
const blob = await ctx.storage.get(release.clawpackStorageId);
if (!blob) return text("ClawPack artifact not found", 404, rateHeaders);
try {
const statMutation =
statKind === "install"
? internalRefs.packages.recordPackageInstallInternal
: internalRefs.packages.recordPackageDownloadInternal;
await runMutationRef(ctx, statMutation, {
packageId: pkg._id,
});
if (statKind === "install") {
await runMutationRef(ctx, internalRefs.packages.recordPackageInstallInternal, {
packageId: pkg._id,
});
}
const identity = getDownloadIdentity(request, viewerUserId ? String(viewerUserId) : null);
if (identity) {
await runMutationRef(
ctx,
internalRefs.downloadMetrics.recordDownloadMetricInternal,
await buildDownloadMetricArgs({
target: { kind: "package", id: pkg._id },
identity,
now: Date.now(),
}),
);
}
} catch {
// Best-effort metric path; never fail package downloads.
}
@@ -710,6 +738,7 @@ type CatalogListItem = {
capabilityTags?: string[];
executesCode?: boolean;
verificationTier?: string | null;
stats?: { downloads: number; installs: number; stars: number; versions: number };
};
type CatalogSearchEntry = {
@@ -915,6 +944,18 @@ function compareCatalogItems(a: CatalogListItem, b: CatalogListItem) {
return a.name.localeCompare(b.name);
}
function compareCatalogItemsForSort(
a: CatalogListItem,
b: CatalogListItem,
sort: (typeof PACKAGE_LIST_SORT_VALUES)[number] | undefined,
) {
if (sort === "downloads") {
const downloads = (b.stats?.downloads ?? 0) - (a.stats?.downloads ?? 0);
if (downloads !== 0) return downloads;
}
return compareCatalogItems(a, b);
}
function compareCatalogSearchEntries(a: CatalogSearchEntry, b: CatalogSearchEntry) {
return (
(a.rankTier ?? Number.POSITIVE_INFINITY) - (b.rankTier ?? Number.POSITIVE_INFINITY) ||
@@ -1018,75 +1059,27 @@ function skillVersionTags(tags: Record<string, string>, version: string) {
.map(([tag]) => tag);
}
function parsePackagePublishBody(body: unknown) {
const parsed = parseArk(PackagePublishRequestSchema, body, "Package publish payload") as {
name: string;
displayName?: string;
ownerHandle?: string;
family: "skill" | "code-plugin" | "bundle-plugin";
version: string;
changelog: string;
manualOverrideReason?: string;
channel?: "official" | "community" | "private";
tags?: string[];
source?: Record<string, unknown>;
bundle?: Record<string, unknown>;
files: Array<{
path: string;
size: number;
storageId: string;
sha256: string;
contentType?: string;
}>;
artifact?: {
kind: "npm-pack";
storageId: string;
sha256: string;
size: number;
format: "tgz";
npmIntegrity: string;
npmShasum: string;
npmTarballName: string;
npmUnpackedSize: number;
npmFileCount: number;
type StoredPackagePublishFile = ServerPackagePublishRequest["files"][number];
type PackagePublishTarballArtifact = NonNullable<ServerPackagePublishRequest["artifact"]>;
type ParsedPackageClawPack = Awaited<ReturnType<typeof parseClawPack>>;
type PackagePublishAuth =
| { kind: "user"; userId: Id<"users"> }
| { kind: "github-actions"; publishToken: Doc<"packagePublishTokens"> };
type PackagePublishTarballPart =
| { kind: "file"; file: File }
| {
kind: "storage";
storageId: Id<"_storage">;
uploadTicket: Id<"packagePublishUploadTickets">;
};
};
if (parsed.files.length === 0) throw new Error("files required");
return {
name: parsed.name,
displayName: parsed.displayName ?? undefined,
ownerHandle: parsed.ownerHandle?.trim().replace(/^@+/, "") || undefined,
family: parsed.family,
version: parsed.version,
changelog: parsed.changelog,
manualOverrideReason: parsed.manualOverrideReason?.trim() || undefined,
channel: parsed.channel ?? undefined,
tags: parsed.tags?.filter(Boolean) ?? undefined,
source: parsed.source ?? undefined,
bundle: parsed.bundle ?? undefined,
files: parsed.files.map((file) => ({
...file,
storageId: file.storageId as Id<"_storage">,
})),
artifact: parsed.artifact
? {
...parsed.artifact,
storageId: parsed.artifact.storageId as Id<"_storage">,
}
: undefined,
};
}
function inferStoredPackageContentType(path: string) {
const lower = path.toLowerCase();
if (lower.endsWith(".json")) return "application/json";
if (lower.endsWith(".md") || lower.endsWith(".mdx") || lower.endsWith(".txt")) {
return "text/plain; charset=utf-8";
}
if (lower.endsWith(".js") || lower.endsWith(".mjs") || lower.endsWith(".cjs")) {
return "text/javascript; charset=utf-8";
}
if (lower.endsWith(".ts") || lower.endsWith(".tsx")) return "text/plain; charset=utf-8";
if (isTextFile(path)) return "text/plain; charset=utf-8";
return "application/octet-stream";
}
@@ -1096,9 +1089,10 @@ function bytesToArrayBuffer(bytes: Uint8Array) {
return copy.buffer;
}
async function storeClawPackFile(ctx: ActionCtx, entry: { path: string; bytes: Uint8Array }) {
// npm-pack artifacts are bounded by the tarball and total package limits; the
// legacy per-file cap only applies to raw file uploads.
async function storeClawPackFile(
ctx: ActionCtx,
entry: { path: string; bytes: Uint8Array },
): Promise<StoredPackagePublishFile> {
const contentType = inferStoredPackageContentType(entry.path);
const storageId = await ctx.storage.store(
new Blob([bytesToArrayBuffer(entry.bytes)], { type: contentType }),
@@ -1116,91 +1110,234 @@ async function storeClawPackFiles(
ctx: ActionCtx,
entries: Array<{ path: string; bytes: Uint8Array }>,
) {
const files: Awaited<ReturnType<typeof storeClawPackFile>>[] = [];
// Convex HTTP actions have a tight memory ceiling; concurrent Blob/storage
// work can duplicate large npm-pack entries enough to OOM the action.
const files: StoredPackagePublishFile[] = [];
// Convex HTTP actions have a tight memory ceiling; avoid concurrent Blob work.
for (const entry of entries) {
files.push(await storeClawPackFile(ctx, entry));
}
return files;
}
async function parseMultipartPackagePublish(ctx: ActionCtx, request: Request) {
async function storeUploadedPackageFile(
ctx: ActionCtx,
entry: File,
): Promise<StoredPackagePublishFile> {
if (entry.size > MAX_PUBLISH_FILE_BYTES) {
throw new Error(getPublishFileSizeError(entry.name));
}
const buffer = new Uint8Array(await entry.arrayBuffer());
const contentType = inferStoredPackageContentType(entry.name);
const storageId = await ctx.storage.store(
new Blob([bytesToArrayBuffer(buffer)], { type: contentType }),
);
return {
path: entry.name,
size: entry.size,
storageId,
sha256: await sha256Hex(buffer),
contentType,
};
}
function getFileParts(form: FormData, fields: readonly string[], stringPartError: string) {
const parts = fields.flatMap((field) => form.getAll(field));
if (parts.some((entry) => typeof entry === "string")) {
throw new Error(stringPartError);
}
return parts.filter((entry): entry is File => typeof entry !== "string");
}
function getTarballPart(form: FormData): PackagePublishTarballPart | null {
const parts = form.getAll("clawpack");
if (parts.length > 1) throw new Error("Upload one package tarball");
const ticketParts = form.getAll("clawpackUploadTicket");
if (ticketParts.length > 1) throw new Error("Upload one package tarball ticket");
const ticketPart = ticketParts[0];
if (ticketPart && typeof ticketPart !== "string") {
throw new Error("Package tarball upload ticket must be a string");
}
const part = parts[0];
if (!part) {
if (ticketPart) throw new Error("Package tarball upload ticket requires a staged ClawPack");
return null;
}
if (typeof part !== "string") {
if (ticketPart) throw new Error("Package tarball upload ticket requires a staged ClawPack");
return { kind: "file", file: part };
}
const storageId = part.trim();
if (!storageId) throw new Error("Package tarball storage id required");
const uploadTicket = ticketPart?.trim();
if (!uploadTicket) throw new Error("Package tarball upload ticket required");
return {
kind: "storage",
storageId: storageId as Id<"_storage">,
uploadTicket: uploadTicket as Id<"packagePublishUploadTickets">,
};
}
async function consumePackageTarballUploadTicket(
ctx: ActionCtx,
auth: PackagePublishAuth,
part: Extract<PackagePublishTarballPart, { kind: "storage" }>,
) {
await ctx.runMutation(
internalRefs.uploads.consumePackagePublishUploadTicketInternal as never,
{
uploadTicket: part.uploadTicket,
storageId: part.storageId,
auth:
auth.kind === "user"
? { kind: "user", userId: auth.userId }
: { kind: "github-actions", publishTokenId: auth.publishToken._id },
} as never,
);
}
async function readStoredPackageTarball(ctx: ActionCtx, storageId: Id<"_storage">) {
const blob = await ctx.storage.get(storageId);
if (!blob) throw new Error("Package tarball upload no longer exists");
if (blob.size > MAX_CLAWPACK_BYTES) {
throw new Error(getClawPackSizeError("uploaded ClawPack"));
}
return new Uint8Array(await blob.arrayBuffer());
}
async function buildPackagePublishRequestFromClawPack(
ctx: ActionCtx,
metadata: PackagePublishMetadata,
parsed: ParsedPackageClawPack,
artifactBytes: Uint8Array,
artifactStorageId: Id<"_storage">,
): Promise<ServerPackagePublishRequest> {
if (parsed.unpackedSize > MAX_PUBLISH_TOTAL_BYTES) {
throw new Error(getPublishTotalSizeError("package"));
}
const artifact: PackagePublishTarballArtifact = {
kind: "npm-pack",
storageId: artifactStorageId,
sha256: parsed.artifactSha256,
size: artifactBytes.byteLength,
format: "tgz",
npmIntegrity: parsed.npmIntegrity,
npmShasum: parsed.npmShasum,
npmTarballName: parsed.npmTarballName,
npmUnpackedSize: parsed.unpackedSize,
npmFileCount: parsed.fileCount,
};
const files = await storeClawPackFiles(ctx, parsed.entries);
return { ...metadata, files, artifact };
}
const PACKAGE_PUBLISH_FILE_FIELDS = ["files"] as const;
const PACKAGE_PUBLISH_TARBALL_FIELDS = ["clawpack"] as const;
const PACKAGE_PUBLISH_FORM_FIELDS = new Set([
"payload",
...PACKAGE_PUBLISH_FILE_FIELDS,
...PACKAGE_PUBLISH_TARBALL_FIELDS,
"clawpackUploadTicket",
]);
function multipartUploadPart(file: File) {
return {
name: file.name,
size: file.size,
type: file.type || undefined,
};
}
async function parseMultipartPackagePublish(
ctx: ActionCtx,
auth: PackagePublishAuth,
request: Request,
): Promise<ServerPackagePublishRequest> {
const form = await request.formData();
const payloadRaw = form.get("payload");
if (!payloadRaw || typeof payloadRaw !== "string") throw new Error("Missing payload");
const payload = JSON.parse(payloadRaw) as Record<string, unknown>;
const files: Array<{
path: string;
size: number;
storageId: Id<"_storage">;
sha256: string;
contentType?: string;
}> = [];
let artifact:
| {
kind: "npm-pack";
storageId: Id<"_storage">;
sha256: string;
size: number;
format: "tgz";
npmIntegrity: string;
npmShasum: string;
npmTarballName: string;
npmUnpackedSize: number;
npmFileCount: number;
}
| undefined;
for (const field of form.keys()) {
if (!PACKAGE_PUBLISH_FORM_FIELDS.has(field)) {
throw new Error(`Unsupported package publish form field: ${field}`);
}
}
const clawpackEntry = form.get("clawpack") ?? form.get("artifact");
if (clawpackEntry && typeof clawpackEntry !== "string") {
if (form.getAll("files").some((entry) => typeof entry !== "string")) {
throw new Error("Upload either a ClawPack tarball or individual files, not both");
const payloadParts = form.getAll("payload");
const payloadRaw = payloadParts[0];
if (payloadParts.length !== 1 || typeof payloadRaw !== "string") {
throw new Error("Package publish payload must be one JSON string");
}
const parsedPayload: unknown = JSON.parse(payloadRaw);
const metadata: PackagePublishMetadata = parseArk(
PackagePublishMetadataSchema,
parsedPayload,
"Package publish payload",
);
const tarballPart = getTarballPart(form);
const fileParts = getFileParts(
form,
PACKAGE_PUBLISH_FILE_FIELDS,
"Package publish file uploads must be files",
);
if (tarballPart) {
if (fileParts.length > 0) {
throw new Error("Upload either a package tarball or individual files, not both");
}
if (clawpackEntry.size > MAX_CLAWPACK_BYTES) {
throw new Error(getClawPackSizeError(clawpackEntry.name));
if (tarballPart.kind === "storage") {
await consumePackageTarballUploadTicket(ctx, auth, tarballPart);
const artifactBytes = await readStoredPackageTarball(ctx, tarballPart.storageId);
const parsed = await parseClawPack(artifactBytes);
return await buildPackagePublishRequestFromClawPack(
ctx,
metadata,
parsed,
artifactBytes,
tarballPart.storageId,
);
}
const artifactBytes = new Uint8Array(await clawpackEntry.arrayBuffer());
const tarballEntry = tarballPart.file;
if (tarballEntry.size > MAX_CLAWPACK_BYTES) {
throw new Error(getClawPackSizeError(tarballEntry.name));
}
if (
isPackageMultipartUploadTooLarge({
payloadJson: payloadRaw,
fileFieldName: "clawpack",
files: [multipartUploadPart(tarballEntry)],
})
) {
throw new Error(getPackageMultipartSizeError());
}
const artifactBytes = new Uint8Array(await tarballEntry.arrayBuffer());
const parsed = await parseClawPack(artifactBytes);
const artifactBlob = new Blob([artifactBytes], { type: "application/octet-stream" });
const artifactStorageId = await ctx.storage.store(artifactBlob);
artifact = {
kind: "npm-pack",
storageId: artifactStorageId,
sha256: parsed.artifactSha256,
size: artifactBytes.byteLength,
format: "tgz",
npmIntegrity: parsed.npmIntegrity,
npmShasum: parsed.npmShasum,
npmTarballName: parsed.npmTarballName,
npmUnpackedSize: parsed.unpackedSize,
npmFileCount: parsed.fileCount,
};
files.push(...(await storeClawPackFiles(ctx, parsed.entries)));
return parsePackagePublishBody({ ...payload, files, artifact });
const artifactStorageId = await ctx.storage.store(
new Blob([bytesToArrayBuffer(artifactBytes)], { type: "application/octet-stream" }),
);
return await buildPackagePublishRequestFromClawPack(
ctx,
metadata,
parsed,
artifactBytes,
artifactStorageId,
);
}
for (const entry of form.getAll("files")) {
if (typeof entry === "string") continue;
if (isMacJunkPath(entry.name)) continue;
if (entry.size > MAX_PUBLISH_FILE_BYTES) {
throw new Error(getPublishFileSizeError(entry.name));
}
const buffer = new Uint8Array(await entry.arrayBuffer());
const digest = await crypto.subtle.digest("SHA-256", buffer);
const sha256 = Array.from(new Uint8Array(digest), (byte) =>
byte.toString(16).padStart(2, "0"),
).join("");
const storageId = await ctx.storage.store(entry);
files.push({
path: entry.name,
size: entry.size,
storageId,
sha256,
contentType: entry.type || undefined,
});
if (
isPackageMultipartUploadTooLarge({
payloadJson: payloadRaw,
fileFieldName: "files",
files: fileParts.map(multipartUploadPart),
})
) {
throw new Error(getPackageMultipartSizeError());
}
return parsePackagePublishBody({ ...payload, files });
const packageFileParts = fileParts.filter((entry) => !isMacJunkPath(entry.name));
const files = await Promise.all(
packageFileParts.map((entry) => storeUploadedPackageFile(ctx, entry)),
);
if (files.length === 0) throw new Error("files required");
return { ...metadata, files };
}
async function listPackages(
@@ -1230,6 +1367,8 @@ async function listPackages(
if (!highlightedOnlyParam.ok) return text(highlightedOnlyParam.message, 400, rate.headers);
const executesCode = parseBooleanQueryParam(url.searchParams, "executesCode");
if (!executesCode.ok) return text(executesCode.message, 400, rate.headers);
const sortParam = parseEnumQueryParam(url.searchParams, "sort", PACKAGE_LIST_SORT_VALUES);
if (!sortParam.ok) return text(sortParam.message, 400, rate.headers);
const category = url.searchParams.get("category")?.trim() || undefined;
if (category && !isPluginCategorySlug(category)) {
return text("Invalid plugin category", 400, rate.headers);
@@ -1256,6 +1395,7 @@ async function listPackages(
highlightedOnly: highlightedOnly || undefined,
executesCode: executesCode.value,
capabilityTag,
sort: sortParam.value,
paginationOpts: { cursor, numItems: limit },
});
return json(
@@ -1289,6 +1429,7 @@ async function listPackages(
executesCode: executesCode.value,
capabilityTag,
category,
sort: sortParam.value,
viewerUserId: viewerUserId ?? undefined,
paginationOpts: { cursor: pageCursor, numItems },
});
@@ -1309,6 +1450,7 @@ async function listPackages(
highlightedOnly: highlightedOnly || undefined,
executesCode: executesCode.value,
capabilityTag,
sort: sortParam.value,
paginationOpts: { cursor: pageCursor, numItems },
});
return {
@@ -1322,7 +1464,8 @@ async function listPackages(
if (!packageCandidate && !skillCandidate) break;
if (
!skillCandidate ||
(packageCandidate && compareCatalogItems(packageCandidate, skillCandidate) <= 0)
(packageCandidate &&
compareCatalogItemsForSort(packageCandidate, skillCandidate, sortParam.value) <= 0)
) {
items.push(packageCandidate!);
packageSource.index += 1;
@@ -1374,6 +1517,7 @@ async function listPackages(
executesCode: executesCode.value,
capabilityTag,
category,
sort: sortParam.value,
viewerUserId: viewerUserId ?? undefined,
paginationOpts: { cursor: pageCursor, numItems },
});
@@ -1402,7 +1546,8 @@ async function listPackages(
if (
!bundlePluginCandidate ||
(codePluginCandidate &&
compareCatalogItems(codePluginCandidate, bundlePluginCandidate) <= 0)
compareCatalogItemsForSort(codePluginCandidate, bundlePluginCandidate, sortParam.value) <=
0)
) {
items.push(codePluginCandidate!);
codePluginSource.index += 1;
@@ -1443,6 +1588,7 @@ async function listPackages(
executesCode: executesCode.value,
capabilityTag,
category,
sort: sortParam.value,
viewerUserId: viewerUserId ?? undefined,
paginationOpts: { cursor, numItems: limit },
} satisfies PackageListQueryArgs);
@@ -1481,9 +1627,10 @@ export async function publishPackageV1Handler(ctx: ActionCtx, request: Request)
try {
const contentType = request.headers.get("content-type") ?? "";
const payload = contentType.includes("multipart/form-data")
? await parseMultipartPackagePublish(ctx, request)
: parsePackagePublishBody(await request.json());
if (!contentType.includes("multipart/form-data")) {
return text("Package publish requires multipart/form-data", 415, rate.headers);
}
const payload = await parseMultipartPackagePublish(ctx, auth.auth, request);
const result =
auth.auth.kind === "user"
? await runActionRef(ctx, internalRefs.packages.publishPackageForUserInternal, {
@@ -2800,7 +2947,14 @@ export async function packagesGetRouterV1Handler(ctx: ActionCtx, request: Reques
if (!release) return text("Version not found", 404, rate.headers);
if (packageSegments[3] === "download") {
if (release.artifactKind === "npm-pack") {
return await streamClawPackRelease(ctx, rate.headers, publicPackage!, release);
return await streamClawPackRelease(
ctx,
request,
rate.headers,
publicPackage!,
release,
viewerUserId ?? null,
);
}
const url = new URL(
`/api/v1/packages/${encodePackagePath(publicPackage!.name)}/download`,
@@ -2999,9 +3153,18 @@ export async function packagesGetRouterV1Handler(ctx: ActionCtx, request: Reques
const zip = buildDeterministicPackageZip(entries);
const [zipSha256, zipSha256Base64] = await Promise.all([sha256Hex(zip), sha256Base64(zip)]);
try {
await runMutationRef(ctx, internalRefs.packages.recordPackageDownloadInternal, {
packageId: publicPackage!._id,
});
const identity = getDownloadIdentity(request, viewerUserId ? String(viewerUserId) : null);
if (identity) {
await runMutationRef(
ctx,
internalRefs.downloadMetrics.recordDownloadMetricInternal,
await buildDownloadMetricArgs({
target: { kind: "package", id: publicPackage!._id },
identity,
now: Date.now(),
}),
);
}
} catch {
// Best-effort metric path; never fail package downloads.
}
@@ -3147,7 +3310,15 @@ export async function npmMirrorGetHandler(ctx: ActionCtx, request: Request) {
const tarballName = path.rest[1]!;
const release = releases.find((candidate) => candidate.npmTarballName === tarballName);
if (!release) return text("ClawPack artifact not found", 404, rate.headers);
return await streamClawPackRelease(ctx, rate.headers, detail.package, release, "install");
return await streamClawPackRelease(
ctx,
request,
rate.headers,
detail.package,
release,
viewerUserId ?? null,
"install",
);
}
if (path.rest.length > 0) return text("Not found", 404, rate.headers);
+88 -5
View File
@@ -422,6 +422,71 @@ export async function parseMultipartPublish(
return parsePublishBody(body);
}
export async function parseMultipartSkillScan(
ctx: ActionCtx,
request: Request,
validatePayload?: (payload: Record<string, unknown>) => Record<string, unknown>,
): Promise<{
payload: Record<string, unknown>;
files: Array<{
path: string;
size: number;
storageId: Id<"_storage">;
sha256: string;
contentType?: string;
}>;
}> {
const form = await request.formData();
const payloadRaw = form.get("payload");
if (!payloadRaw || typeof payloadRaw !== "string") {
throw new Error("Missing payload");
}
let payload: Record<string, unknown>;
try {
payload = JSON.parse(payloadRaw) as Record<string, unknown>;
} catch {
throw new Error("Invalid JSON payload");
}
const validatedPayload = validatePayload ? validatePayload(payload) : payload;
const fileEntries = form
.getAll("files")
.map((entry) => toFileLike(entry))
.filter((file): file is FileLikeEntry => Boolean(file))
.filter((file) => !isMacJunkPath(file.name));
if (fileEntries.length === 0) throw new Error("files required");
if (!fileEntries.some((file) => file.name.trim().toLowerCase() === "skill.md")) {
throw new Error("SKILL.md required");
}
const oversized = fileEntries.find((file) => file.size > MAX_PUBLISH_FILE_BYTES);
if (oversized) throw new Error(getPublishFileSizeError(oversized.name));
const files: Array<{
path: string;
size: number;
storageId: Id<"_storage">;
sha256: string;
contentType?: string;
}> = [];
try {
for (const file of fileEntries) {
const path = file.name;
const size = file.size;
const contentType = file.type || undefined;
const buffer = new Uint8Array(await file.arrayBuffer());
const sha256 = await sha256Hex(buffer);
const storageId = await ctx.storage.store(file as Blob);
files.push({ path, size, storageId, sha256, contentType });
}
} catch (error) {
await Promise.allSettled(files.map((file) => ctx.storage.delete(file.storageId)));
throw error;
}
return { payload: validatedPayload, files };
}
export function parsePublishBody(body: unknown) {
const parsed = parseArk(CliPublishRequestSchema, body, "Publish payload");
if (parsed.files.length === 0) throw new Error("files required");
@@ -449,22 +514,40 @@ export function parsePublishBody(body: unknown) {
};
}
// Substrings that indicate user-input validation failures from the underlying
// mutations (e.g. `normalizePackageName` ConvexErrors). These are surfaced as
// 400s with the cleaned message so CLI/API clients can see the actual reason
// instead of an opaque 500.
const SOFT_DELETE_BAD_REQUEST_HINTS = [
"slug required",
"package name required",
"package name must be",
"must be lowercase",
"npm-safe",
"reserved for clawhub routes",
"version required",
] as const;
export function softDeleteErrorToResponse(
entity: "skill" | "soul" | "package",
error: unknown,
headers: HeadersInit,
) {
const message = error instanceof Error ? error.message : `${entity} delete failed`;
const lower = message.toLowerCase();
const rawMessage = error instanceof Error ? error.message : `${entity} delete failed`;
const cleaned = cleanUserFacingErrorMessage(rawMessage) || rawMessage;
const lower = cleaned.toLowerCase();
if (lower.includes("unauthorized"))
return text(formatAuthzMessage(error, "Unauthorized"), 401, headers);
if (lower.includes("forbidden"))
return text(formatAuthzMessage(error, "Forbidden"), 403, headers);
if (lower.includes("not found")) return text(message, 404, headers);
if (lower.includes("slug required")) return text("Slug required", 400, headers);
if (lower.includes("not found")) return text(cleaned, 404, headers);
if (SOFT_DELETE_BAD_REQUEST_HINTS.some((hint) => lower.includes(hint))) {
return text(cleaned, 400, headers);
}
// Unknown: server-side failure. Keep body generic.
// Unknown: server-side failure. Keep the body generic; only known
// user-input validation failures above surface the cleaned mutation message.
return text("Internal Server Error", 500, headers);
}
+352 -2
View File
@@ -1,7 +1,11 @@
import {
ApiRoutes,
ApiV1SkillBulkRescanBatchRequestSchema,
ApiV1SkillBulkRescanStatusRequestSchema,
ApiV1SkillRepairVtPendingRequestSchema,
ApiV1SkillScanBatchRequestSchema,
ApiV1SkillScanBatchStatusRequestSchema,
ApiV1SkillScanSubmitRequestSchema,
SkillAppealRequestSchema,
SkillAppealResolveRequestSchema,
SkillReportTriageRequestSchema,
@@ -17,6 +21,12 @@ import { getOptionalApiTokenUserId, requireApiTokenUser } from "../lib/apiTokenA
import { mergeHeaders } from "../lib/httpHeaders";
import { applyRateLimit } from "../lib/httpRateLimit";
import { parseBooleanQueryParam, resolveBooleanQueryParam } from "../lib/httpUtils";
import {
buildSkillInstallResolution,
type InstallResolverSkill,
type InstallResolverSource,
type SkillInstallResolution,
} from "../lib/installResolver";
import type {
LlmAgenticRiskFinding,
LlmEvalDimension,
@@ -25,6 +35,7 @@ import type {
import { selectGeneratedSkillCardFile, sourceSkillVersionFiles } from "../lib/skillCards";
import { getPublicSkillFileAccessBlock, isSkillVersionForSkill } from "../lib/skillFileAccess";
import {
buildDeterministicZip,
buildMergedExportZip,
type MergedExportManifestEntry,
validateSlug,
@@ -37,6 +48,7 @@ import {
getPathSegments,
json,
parseJsonPayload,
parseMultipartSkillScan,
parseMultipartPublish,
parsePublishBody,
publicApiOrigin,
@@ -278,8 +290,14 @@ type SkillSecuritySnapshot = {
};
const internalRefs = internal as unknown as {
githubSkillSources: {
getByIdInternal: unknown;
};
securityScan: {
createUploadedSkillScanRequestInternal: unknown;
createPublishedSkillScanRequestInternal: unknown;
enqueueBulkSkillRescanBatchForAdminInternal: unknown;
getSkillScanRequestForUserInternal: unknown;
getBulkSkillRescanBatchStatusForAdminInternal: unknown;
requestSkillRescanForUserInternal: unknown;
};
@@ -288,6 +306,7 @@ const internalRefs = internal as unknown as {
};
skills: {
getSecurityVerdictTargetInternal: unknown;
getSkillBySlugInternal: unknown;
reportSkillForUserInternal: unknown;
listSkillReportsInternal: unknown;
triageSkillReportForUserInternal: unknown;
@@ -309,6 +328,138 @@ async function runActionRef<T>(ctx: ActionCtx, ref: unknown, args: unknown): Pro
return (await ctx.runAction(ref as never, args as never)) as T;
}
function isMultipartRequest(request: Request) {
return (
request.headers.get("content-type")?.toLowerCase().includes("multipart/form-data") === true
);
}
async function deleteStoredScanFiles(ctx: ActionCtx, files: Array<{ storageId: Id<"_storage"> }>) {
await Promise.allSettled(files.map((file) => ctx.storage.delete(file.storageId)));
}
function encodeJsonEntry(value: unknown) {
return new TextEncoder().encode(`${JSON.stringify(value, null, 2)}\n`);
}
function encodeTextEntry(value: string) {
return new TextEncoder().encode(value);
}
function scanReportPart(status: Record<string, unknown>, key: string) {
const report = status.report;
if (!report || typeof report !== "object" || Array.isArray(report)) return null;
return (report as Record<string, unknown>)[key] ?? null;
}
function buildSkillScanReportZip(status: Record<string, unknown>) {
const manifest = {
scanId: status.scanId,
sourceKind: status.sourceKind,
update: status.update,
status: status.status,
artifact: status.artifact ?? null,
createdAt: status.createdAt,
updatedAt: status.updatedAt,
completedAt: status.completedAt ?? null,
writtenBack: status.writtenBack === true,
};
const scanIdText = typeof status.scanId === "string" ? status.scanId : "";
const statusText = typeof status.status === "string" ? status.status : "";
const readme = [
"# ClawHub Scan Report",
"",
`Scan ID: ${scanIdText}`,
`Status: ${statusText}`,
"",
"This archive uses the ClawHub security-audit export shape:",
"",
"- manifest.json",
"- clawscan.json",
"- skillspector.json",
"- static-analysis.json",
"- virustotal.json",
"- README.md",
"",
].join("\n");
return buildDeterministicZip([
{ path: "manifest.json", bytes: encodeJsonEntry(manifest) },
{ path: "clawscan.json", bytes: encodeJsonEntry(scanReportPart(status, "clawscan")) },
{ path: "skillspector.json", bytes: encodeJsonEntry(scanReportPart(status, "skillspector")) },
{
path: "static-analysis.json",
bytes: encodeJsonEntry(scanReportPart(status, "staticAnalysis")),
},
{ path: "virustotal.json", bytes: encodeJsonEntry(scanReportPart(status, "virustotal")) },
{ path: "README.md", bytes: encodeTextEntry(readme) },
]);
}
async function handleSkillScanBatchSubmit(ctx: ActionCtx, request: Request, headers: HeadersInit) {
const auth = await requireApiTokenUserOrResponse(ctx, request, headers);
if (!auth.ok) return auth.response;
const admin = requireAdminOrResponse(auth.user, headers);
if (!admin.ok) return admin.response;
try {
const body = parseArk(
ApiV1SkillScanBatchRequestSchema,
await request.json(),
"Skill scan batch payload",
) as {
mode?: "all-active-latest";
cursor?: string | null;
batchSize?: number;
dryRun?: boolean;
};
const result = await runMutationRef(
ctx,
internalRefs.securityScan.enqueueBulkSkillRescanBatchForAdminInternal,
{
actorUserId: auth.userId,
...(body.mode ? { mode: body.mode } : {}),
cursor: body.cursor ?? null,
...(body.batchSize !== undefined ? { batchSize: body.batchSize } : {}),
...(body.dryRun !== undefined ? { dryRun: body.dryRun } : {}),
},
);
return json(result, 200, headers);
} catch (error) {
if (error instanceof SyntaxError) return text("Invalid JSON", 400, headers);
return text(error instanceof Error ? error.message : "Skill scan batch failed", 400, headers);
}
}
async function handleSkillScanBatchStatus(ctx: ActionCtx, request: Request, headers: HeadersInit) {
const auth = await requireApiTokenUserOrResponse(ctx, request, headers);
if (!auth.ok) return auth.response;
const admin = requireAdminOrResponse(auth.user, headers);
if (!admin.ok) return admin.response;
try {
const body = parseArk(
ApiV1SkillScanBatchStatusRequestSchema,
await request.json(),
"Skill scan batch status payload",
) as { jobIds: string[] };
const result = await runQueryRef(
ctx,
internalRefs.securityScan.getBulkSkillRescanBatchStatusForAdminInternal,
{
actorUserId: auth.userId,
jobIds: body.jobIds,
},
);
return json(result, 200, headers);
} catch (error) {
if (error instanceof SyntaxError) return text("Invalid JSON", 400, headers);
return text(
error instanceof Error ? error.message : "Skill scan batch status failed",
400,
headers,
);
}
}
function isDefinitiveSecurityStatus(
status: NormalizedSecurityStatus | null | undefined,
): status is "clean" | "suspicious" | "malicious" {
@@ -964,6 +1115,127 @@ export async function skillSecurityVerdictsV1Handler(ctx: ActionCtx, request: Re
return json({ schema: "clawhub.skill.security-verdicts.v1", items }, 200, rate.headers);
}
export async function skillScanSubmitV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "write");
if (!rate.ok) return rate.response;
const auth = await requireApiTokenUserOrResponse(ctx, request, rate.headers);
if (!auth.ok) return auth.response;
try {
if (isMultipartRequest(request)) {
const multipart = await parseMultipartSkillScan(ctx, request, (payload) => {
const parsed = parseArk(
ApiV1SkillScanSubmitRequestSchema,
payload,
"Skill scan payload",
) as {
source: { kind: "upload" } | { kind: "published"; slug: string; version?: string };
update?: boolean;
};
if (parsed.source.kind !== "upload") {
throw new Error("multipart scan payload must use source.kind=upload");
}
if (parsed.update === true) {
throw new Error("update is not valid for uploaded scans");
}
return parsed;
});
const result = await runMutationRef(
ctx,
internalRefs.securityScan.createUploadedSkillScanRequestInternal,
{
actorUserId: auth.userId,
files: multipart.files,
},
).catch(async (error) => {
await deleteStoredScanFiles(ctx, multipart.files);
throw error;
});
return json(result, 202, rate.headers);
}
const body = parseArk(
ApiV1SkillScanSubmitRequestSchema,
await request.json(),
"Skill scan payload",
) as {
source: { kind: "upload" } | { kind: "published"; slug: string; version?: string };
update?: boolean;
};
if (body.source.kind === "upload") {
return text("uploaded scans must use multipart/form-data", 400, rate.headers);
}
const result = await runMutationRef(
ctx,
internalRefs.securityScan.createPublishedSkillScanRequestInternal,
{
actorUserId: auth.userId,
slug: body.source.slug,
...(body.source.version ? { version: body.source.version } : {}),
update: body.update === true,
},
);
return json(result, 202, rate.headers);
} catch (error) {
if (error instanceof SyntaxError) return text("Invalid JSON", 400, rate.headers);
return text(
error instanceof Error ? error.message : "Skill scan submit failed",
400,
rate.headers,
);
}
}
export async function skillScanGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "read");
if (!rate.ok) return rate.response;
const auth = await requireApiTokenUserOrResponse(ctx, request, rate.headers);
if (!auth.ok) return auth.response;
const segments = getPathSegments(request, `${ApiRoutes.skillScans}/`);
const scanId = segments[0];
if (!scanId) return text("scanId required", 400, rate.headers);
try {
const status = (await runQueryRef(
ctx,
internalRefs.securityScan.getSkillScanRequestForUserInternal,
{
actorUserId: auth.userId,
scanId: scanId as Id<"skillScanRequests">,
},
)) as Record<string, unknown>;
if (segments.length === 1) return json(status, 200, rate.headers);
if (segments.length === 2 && segments[1] === "download") {
if (status.status !== "succeeded") return text("Scan is not complete", 409, rate.headers);
const zip = buildSkillScanReportZip(status);
const headers = mergeHeaders(rate.headers, {
"Content-Type": "application/zip",
"Content-Disposition": `attachment; filename="clawhub-scan-${scanId}.zip"`,
});
return new Response(zip, { status: 200, headers });
}
return text("Not found", 404, rate.headers);
} catch (error) {
return text(error instanceof Error ? error.message : "Skill scan failed", 400, rate.headers);
}
}
export async function skillScanBatchSubmitV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "write");
if (!rate.ok) return rate.response;
return handleSkillScanBatchSubmit(ctx, request, rate.headers);
}
export async function skillScanBatchStatusV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "write");
if (!rate.ok) return rate.response;
return handleSkillScanBatchStatus(ctx, request, rate.headers);
}
export async function searchSkillsV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "read");
if (!rate.ok) return rate.response;
@@ -1034,6 +1306,7 @@ export async function resolveSkillVersionV1Handler(ctx: ActionCtx, request: Requ
}
type SkillListSort =
| "recommended"
| "createdAt"
| "updated"
| "downloads"
@@ -1042,11 +1315,14 @@ type SkillListSort =
| "installsAllTime"
| "trending";
type PublicListSort = "newest" | "updated" | "downloads" | "stars" | "installs";
type PublicListSort = "recommended" | "newest" | "updated" | "downloads" | "stars" | "installs";
function parseListSort(value: string | null): SkillListSort | null {
if (value === null) return "updated";
const normalized = value?.trim().toLowerCase();
const normalized = value.trim().toLowerCase();
if (normalized === "default" || normalized === "recommended") {
return "recommended";
}
if (normalized === "createdat" || normalized === "created-at" || normalized === "newest") {
return "createdAt";
}
@@ -1069,6 +1345,7 @@ function parseListSort(value: string | null): SkillListSort | null {
}
function toPublicListSort(sort: Exclude<SkillListSort, "trending">): PublicListSort {
if (sort === "recommended") return "recommended";
if (sort === "createdAt") return "newest";
if (sort === "updated") return "updated";
if (sort === "downloads" || sort === "stars") return sort;
@@ -1199,6 +1476,25 @@ async function describeOwnerVisibleSkillState(
return null;
}
function shouldExposeHiddenGitHubInstallBlock(
skill: InstallResolverSkill & {
installKind?: "github";
moderationStatus?: "active" | "hidden" | "removed";
moderationReason?: string;
},
resolution: SkillInstallResolution,
) {
if (skill.installKind !== "github" || resolution.ok) return false;
if (skill.moderationStatus !== "hidden") return false;
const reason = skill.moderationReason ?? "";
return (
reason === "pending.scan" ||
reason === "scanner.failed" ||
reason === "scanner.llm.malicious" ||
reason.startsWith("github.")
);
}
export async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "read");
if (!rate.ok) return rate.response;
@@ -1263,6 +1559,60 @@ export async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request)
return json(result, 200, rate.headers);
}
if (second === "install" && segments.length === 2) {
const url = new URL(request.url);
const forceInstall = parseBooleanQueryParam(url.searchParams.get("forceInstall"));
const skill = (await runQueryRef<
| (InstallResolverSkill & {
_id: Id<"skills">;
githubSourceId?: Id<"githubSkillSources">;
softDeletedAt?: number;
moderationStatus?: "active" | "hidden" | "removed";
moderationReason?: string;
moderationFlags?: string[];
})
| null
>(ctx, internalRefs.skills.getSkillBySlugInternal, { slug })) as
| (InstallResolverSkill & {
_id: Id<"skills">;
githubSourceId?: Id<"githubSkillSources">;
softDeletedAt?: number;
moderationStatus?: "active" | "hidden" | "removed";
moderationReason?: string;
moderationFlags?: string[];
})
| null;
if (!skill || skill.softDeletedAt || skill.moderationStatus === "removed") {
return text("Skill not found", 404, rate.headers);
}
const source =
skill.installKind === "github" && skill.githubSourceId
? ((await runQueryRef(ctx, internalRefs.githubSkillSources.getByIdInternal, {
sourceId: skill.githubSourceId,
})) as InstallResolverSource | null)
: null;
const resolution = buildSkillInstallResolution({
origin: publicApiOrigin(request),
skill,
source,
forceInstall,
});
const publicSkillResult = (await ctx.runQuery(api.skills.getBySlug, {
slug,
})) as GetBySlugResult;
const publiclyVisible = publicSkillResult?.skill?._id === skill._id;
if (!publiclyVisible) {
if (!resolution.ok && shouldExposeHiddenGitHubInstallBlock(skill, resolution)) {
return json(resolution, resolution.status, rate.headers);
}
return text("Skill not found", 404, rate.headers);
}
return json(resolution, resolution.ok ? 200 : resolution.status, rate.headers);
}
if (segments.length === 1) {
const result = (await ctx.runQuery(api.skills.getBySlug, { slug })) as GetBySlugResult;
if (!result?.skill) {
+77
View File
@@ -14,7 +14,10 @@ import {
const usersV1InternalRefs = internal as unknown as {
publishers: {
addOfficialPublisherInternal: unknown;
listOfficialPublishersInternal: unknown;
removeOrgPublisherMemberInternal: unknown;
removeOfficialPublisherInternal: unknown;
};
users: {
getBanAppealContextByGitHubProviderAccountIdInternal: unknown;
@@ -84,6 +87,7 @@ export async function usersPostRouterV1Handler(ctx: ActionCtx, request: Request)
action !== "reclaim" &&
action !== "reserve" &&
action !== "publisher" &&
action !== "publisher-official" &&
action !== "publisher-member"
) {
return text("Not found", 404, rate.headers);
@@ -139,6 +143,12 @@ export async function usersPostRouterV1Handler(ctx: ActionCtx, request: Request)
return handleAdminEnsurePublisher(ctx, payload, actorUserId, rate.headers);
}
if (action === "publisher-official") {
const admin = requireAdminOrResponse(actorUser, rate.headers);
if (!admin.ok) return admin.response;
return handleAdminOfficialPublisherPost(ctx, payload, actorUserId, rate.headers);
}
if (action === "publisher-member") {
const admin = requireAdminOrResponse(actorUser, rate.headers);
if (!admin.ok) return admin.response;
@@ -352,6 +362,37 @@ async function handleAdminRemediateAutobans(
}
}
export async function usersGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, "read");
if (!rate.ok) return rate.response;
const segments = getPathSegments(request, "/api/v1/users/");
if (segments.length !== 1 || segments[0] !== "publisher-official") {
return text("Not found", 404, rate.headers);
}
const authResult = await requireApiTokenUserOrResponse(ctx, request, rate.headers);
if (!authResult.ok) return authResult.response;
const admin = requireAdminOrResponse(authResult.user, rate.headers);
if (!admin.ok) return admin.response;
try {
const result = await runUsersV1QueryRef(
ctx,
usersV1InternalRefs.publishers.listOfficialPublishersInternal,
{ actorUserId: authResult.userId },
);
return json(result, 200, rate.headers);
} catch (error) {
const message = error instanceof Error ? error.message : "Official publisher list failed";
if (message.toLowerCase().includes("forbidden")) return text("Forbidden", 403, rate.headers);
if (message.toLowerCase().includes("unauthorized")) {
return text("Unauthorized", 401, rate.headers);
}
return text(message, 400, rate.headers);
}
}
/**
* POST /api/v1/users/restore
* Admin-only: restore skills from GitHub backup for a user.
@@ -532,6 +573,42 @@ async function handleAdminReserve(
return json({ ok: true, results, succeeded, failed }, 200, headers);
}
async function handleAdminOfficialPublisherPost(
ctx: ActionCtx,
payload: Record<string, unknown>,
actorUserId: Id<"users">,
headers: HeadersInit,
) {
const action = typeof payload.action === "string" ? payload.action.trim().toLowerCase() : "";
const handle = typeof payload.handle === "string" ? payload.handle.trim().toLowerCase() : "";
const reason = typeof payload.reason === "string" ? payload.reason.trim() : "";
if (action !== "add" && action !== "remove") return text("Invalid action", 400, headers);
if (!handle) return text("Missing handle", 400, headers);
if (!reason) return text("Missing reason", 400, headers);
if (reason.length > 500) return text("Reason too long (max 500 chars)", 400, headers);
try {
const result = await runUsersV1MutationRef(
ctx,
action === "add"
? usersV1InternalRefs.publishers.addOfficialPublisherInternal
: usersV1InternalRefs.publishers.removeOfficialPublisherInternal,
{
actorUserId,
handle,
reason,
},
);
return json(result, 200, headers);
} catch (error) {
const message = error instanceof Error ? error.message : "Official publisher update failed";
if (message.toLowerCase().includes("forbidden")) return text("Forbidden", 403, headers);
if (message.toLowerCase().includes("unauthorized")) return text("Unauthorized", 401, headers);
if (message.toLowerCase().includes("not found")) return text(message, 404, headers);
return text(message, 400, headers);
}
}
async function handleAdminEnsurePublisher(
ctx: ActionCtx,
payload: Record<string, unknown>,
+97 -1
View File
@@ -1,4 +1,4 @@
import { describe, expect, it, vi } from "vitest";
import { beforeEach, describe, expect, it, vi } from "vitest";
vi.mock("@convex-dev/auth/server", () => ({
getAuthUserId: vi.fn(),
@@ -9,6 +9,10 @@ const { getAuthUserId } = await import("@convex-dev/auth/server");
const { assertAdmin, assertModerator, assertRole, requireUser, requireUserFromAction } =
await import("./access");
beforeEach(() => {
vi.mocked(getAuthUserId).mockReset();
});
describe("access.requireUser", () => {
it("throws when auth is missing", async () => {
vi.mocked(getAuthUserId).mockResolvedValue(null);
@@ -59,6 +63,67 @@ describe("access.requireUser", () => {
expect(dbGet).toHaveBeenCalledWith("users:2");
expect(result).toEqual({ userId: "users:2", user });
});
it("uses the local dev impersonation user before browser auth", async () => {
const previousHandle = process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
const previousEnabled = process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
try {
process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE = "local";
process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION = "1";
vi.mocked(getAuthUserId).mockResolvedValue("users:browser" as never);
const user = { _id: "users:local", handle: "local", role: "admin" };
const unique = vi.fn().mockResolvedValue(user as never);
const withIndex = vi.fn().mockReturnValue({ unique });
const query = vi.fn().mockReturnValue({ withIndex });
const dbGet = vi.fn().mockResolvedValue(user as never);
const result = await requireUser({
db: { get: dbGet, query },
} as never);
expect(query).toHaveBeenCalledWith("users");
expect(withIndex).toHaveBeenCalledWith("handle", expect.any(Function));
expect(dbGet).toHaveBeenCalledWith("users:local");
expect(getAuthUserId).not.toHaveBeenCalled();
expect(result).toEqual({ userId: "users:local", user });
} finally {
if (previousHandle === undefined) delete process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
else process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE = previousHandle;
if (previousEnabled === undefined) delete process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
else process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION = previousEnabled;
}
});
it("does not use local dev impersonation in production deployments", async () => {
const previousHandle = process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
const previousEnabled = process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
const previousDeployment = process.env.CONVEX_DEPLOYMENT;
try {
process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE = "local";
process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION = "1";
process.env.CONVEX_DEPLOYMENT = "prod:wry-manatee-359";
vi.mocked(getAuthUserId).mockResolvedValue("users:browser" as never);
const user = { _id: "users:browser", handle: "browser", role: "user" };
const query = vi.fn();
const dbGet = vi.fn().mockResolvedValue(user as never);
const result = await requireUser({
db: { get: dbGet, query },
} as never);
expect(query).not.toHaveBeenCalled();
expect(getAuthUserId).toHaveBeenCalled();
expect(dbGet).toHaveBeenCalledWith("users:browser");
expect(result).toEqual({ userId: "users:browser", user });
} finally {
if (previousHandle === undefined) delete process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
else process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE = previousHandle;
if (previousEnabled === undefined) delete process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
else process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION = previousEnabled;
if (previousDeployment === undefined) delete process.env.CONVEX_DEPLOYMENT;
else process.env.CONVEX_DEPLOYMENT = previousDeployment;
}
});
});
describe("access.requireUserFromAction", () => {
@@ -111,6 +176,37 @@ describe("access.requireUserFromAction", () => {
expect(runQuery).toHaveBeenCalledTimes(1);
expect(result).toEqual({ userId: "users:9", user });
});
it("uses the local dev impersonation user before action auth", async () => {
const previousHandle = process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
const previousEnabled = process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
try {
process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE = "local";
process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION = "1";
vi.mocked(getAuthUserId).mockResolvedValue("users:browser" as never);
const user = { _id: "users:local", handle: "local", role: "admin" };
const runQuery = vi.fn(async (_query, args: { handle?: string; userId?: string }) => {
if (args.handle === "local") return user;
if (args.userId === "users:local") return user;
return null;
});
const result = await requireUserFromAction({ runQuery } as never);
expect(runQuery).toHaveBeenCalledTimes(2);
expect(runQuery).toHaveBeenNthCalledWith(1, expect.anything(), { handle: "local" });
expect(runQuery).toHaveBeenNthCalledWith(2, expect.anything(), {
userId: "users:local",
});
expect(getAuthUserId).not.toHaveBeenCalled();
expect(result).toEqual({ userId: "users:local", user });
} finally {
if (previousHandle === undefined) delete process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
else process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE = previousHandle;
if (previousEnabled === undefined) delete process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
else process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION = previousEnabled;
}
});
});
describe("access role assertions", () => {
+36 -7
View File
@@ -8,10 +8,23 @@ export type Role = "admin" | "moderator" | "user" | "mirror";
const DEV_IMPERSONATE_LOCAL_HANDLE = "local";
function readEnv(name: string) {
const value = process.env[name]?.trim();
const value = readKnownEnv(name)?.trim();
return value ? value : undefined;
}
function readKnownEnv(name: string) {
if (name === "CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE") {
return process.env.CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE;
}
if (name === "CLAW_HUB_ENABLE_DEV_IMPERSONATION") {
return process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
}
if (name === "CONVEX_DEPLOYMENT") {
return process.env.CONVEX_DEPLOYMENT;
}
return undefined;
}
function isDevImpersonationAllowed() {
const requestedHandle = readEnv("CLAW_HUB_DEV_IMPERSONATE_USER_HANDLE");
if (requestedHandle !== DEV_IMPERSONATE_LOCAL_HANDLE) return false;
@@ -52,39 +65,49 @@ async function getDevImpersonatedUserIdFromAction(
export async function getOptionalActiveAuthUserId(
ctx: MutationCtx | QueryCtx,
): Promise<Id<"users"> | undefined> {
const devUserId = await getDevImpersonatedUserId(ctx);
if (devUserId) return devUserId;
try {
const userId = await getAuthUserId(ctx);
if (!userId) return await getDevImpersonatedUserId(ctx);
if (!userId) return undefined;
const user = await ctx.db.get(userId);
if (!user || user.deletedAt || user.deactivatedAt) return undefined;
return userId;
} catch {
return await getDevImpersonatedUserId(ctx);
return undefined;
}
}
export async function getOptionalActiveAuthUserIdFromAction(
ctx: ActionCtx,
): Promise<Id<"users"> | undefined> {
const devUserId = await getDevImpersonatedUserIdFromAction(ctx);
if (devUserId) return devUserId;
try {
const userId = await getAuthUserId(ctx);
if (!userId) return await getDevImpersonatedUserIdFromAction(ctx);
if (!userId) return undefined;
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId });
if (!user || user.deletedAt || user.deactivatedAt) return undefined;
return userId;
} catch {
return await getDevImpersonatedUserIdFromAction(ctx);
return undefined;
}
}
export async function requireUser(ctx: MutationCtx | QueryCtx) {
const devUserId = await getDevImpersonatedUserId(ctx);
if (devUserId) {
const devUser = await ctx.db.get(devUserId);
if (!devUser || devUser.deletedAt || devUser.deactivatedAt) throw new Error("User not found");
return { userId: devUserId, user: devUser };
}
let userId: Id<"users"> | null | undefined = null;
try {
userId = await getAuthUserId(ctx);
} catch {
userId = null;
}
userId ??= await getDevImpersonatedUserId(ctx);
if (!userId) throw new Error("Unauthorized");
let user: Doc<"users"> | null;
try {
@@ -99,13 +122,19 @@ export async function requireUser(ctx: MutationCtx | QueryCtx) {
export async function requireUserFromAction(
ctx: ActionCtx,
): Promise<{ userId: Id<"users">; user: Doc<"users"> }> {
const devUserId = await getDevImpersonatedUserIdFromAction(ctx);
if (devUserId) {
const devUser = await ctx.runQuery(internal.users.getByIdInternal, { userId: devUserId });
if (!devUser || devUser.deletedAt || devUser.deactivatedAt) throw new Error("User not found");
return { userId: devUserId, user: devUser as Doc<"users"> };
}
let userId: Id<"users"> | null | undefined = null;
try {
userId = await getAuthUserId(ctx);
} catch {
userId = null;
}
userId ??= await getDevImpersonatedUserIdFromAction(ctx);
if (!userId) throw new Error("Unauthorized");
let user: Doc<"users"> | null;
try {
+1 -1
View File
@@ -34,7 +34,7 @@ export const MISSING_API_TOKEN_MESSAGE =
export const INVALID_API_TOKEN_MESSAGE =
"Unauthorized: API token is invalid or revoked. Run `clawhub login` again.";
export const BLOCKED_API_TOKEN_ACCOUNT_MESSAGE =
"Unauthorized: This ClawHub account is not in good standing and cannot use API tokens. If you believe this is a mistake, contact security@openclaw.ai.";
"Unauthorized: This ClawHub account is not in good standing and cannot use API tokens. If you believe this is a mistake, open a GitHub issue: https://github.com/openclaw/clawhub/issues/new.";
export async function requireApiTokenUser(
ctx: ActionCtx,
+18 -2
View File
@@ -37,9 +37,9 @@ function tarFile(path: string, content: string) {
return [header, body];
}
function npmPackFixture(files: Record<string, string>) {
function npmPackFixtureEntries(files: Array<[string, string]>) {
const parts: Uint8Array[] = [];
for (const [path, content] of Object.entries(files)) {
for (const [path, content] of files) {
parts.push(...tarFile(path, content));
}
parts.push(new Uint8Array(BLOCK_SIZE), new Uint8Array(BLOCK_SIZE));
@@ -53,6 +53,10 @@ function npmPackFixture(files: Record<string, string>) {
return gzipSync(tar);
}
function npmPackFixture(files: Record<string, string>) {
return npmPackFixtureEntries(Object.entries(files));
}
describe("clawpack", () => {
it("parses npm pack tarballs and computes npm integrity fields", async () => {
const pack = npmPackFixture({
@@ -95,6 +99,18 @@ describe("clawpack", () => {
await expect(parseClawPack(pack)).rejects.toThrow("rooted under package");
});
it("rejects duplicate normalized archive paths", async () => {
const pack = npmPackFixtureEntries([
["package/package.json", JSON.stringify({ name: "demo", version: "1.0.0" })],
["package/openclaw.plugin.json", JSON.stringify({ id: "demo" })],
["package/package.json", JSON.stringify({ name: "other", version: "9.9.9" })],
]);
await expect(parseClawPack(pack)).rejects.toThrow(
"ClawPack contains duplicate path: package.json",
);
});
it("uses npm-style tarball names", () => {
expect(npmTarballName("demo", "1.0.0")).toBe("demo-1.0.0.tgz");
expect(npmTarballName("@scope/demo", "1.0.0")).toBe("scope-demo-1.0.0.tgz");
+5
View File
@@ -67,6 +67,7 @@ function isZeroBlock(block: Uint8Array) {
function parseTarEntries(bytes: Uint8Array): ClawPackEntry[] {
const entries: ClawPackEntry[] = [];
const paths = new Set<string>();
let offset = 0;
while (offset + TAR_BLOCK_SIZE <= bytes.byteLength) {
@@ -93,6 +94,10 @@ function parseTarEntries(bytes: Uint8Array): ClawPackEntry[] {
offset = nextTarOffset(payloadOffset, size);
continue;
}
if (paths.has(relPath)) {
throw new Error(`ClawPack contains duplicate path: ${relPath}`);
}
paths.add(relPath);
entries.push({
path: relPath,
bytes: Uint8Array.from(tarEntryPayload(bytes, payloadOffset, size)),
+67 -3
View File
@@ -1,6 +1,8 @@
import { describe, expect, it } from "vitest";
import { isLocalDevAuthEnabled } from "./devAuth";
const CLOUD_DEV_AUTH_SECRET = "dev-auth-secret-with-enough-entropy-123";
describe("isLocalDevAuthEnabled", () => {
it("requires the explicit dev auth flag", () => {
expect(
@@ -41,16 +43,78 @@ describe("isLocalDevAuthEnabled", () => {
).toBe(true);
});
it("rejects cloud dev deployments even when the dev auth flag is set", () => {
it("allows cloud dev deployments with an explicit localhost site and matching secret", () => {
expect(
isLocalDevAuthEnabled(
{
CONVEX_SITE_URL: "https://clever-rabbit-123.convex.cloud",
DEV_AUTH_CONVEX_DEPLOYMENT: "dev:clever-rabbit-123",
DEV_AUTH_ENABLED: "1",
DEV_AUTH_SECRET: CLOUD_DEV_AUTH_SECRET,
DEV_AUTH_SITE_URL: "http://127.0.0.1:3211",
},
CLOUD_DEV_AUTH_SECRET,
),
).toBe(true);
});
it("allows cloud dev deployments from the fallback marker when Convex deployment is blank", () => {
expect(
isLocalDevAuthEnabled(
{
CONVEX_DEPLOYMENT: "",
CONVEX_SITE_URL: "https://clever-rabbit-123.convex.cloud",
DEV_AUTH_CONVEX_DEPLOYMENT: "dev:clever-rabbit-123",
DEV_AUTH_ENABLED: "1",
DEV_AUTH_SECRET: CLOUD_DEV_AUTH_SECRET,
DEV_AUTH_SITE_URL: "http://127.0.0.1:3211",
},
CLOUD_DEV_AUTH_SECRET,
),
).toBe(true);
});
it("rejects cloud dev deployments when the secret is missing", () => {
expect(
isLocalDevAuthEnabled({
CONVEX_SITE_URL: "http://127.0.0.1:3211",
CONVEX_SITE_URL: "https://clever-rabbit-123.convex.cloud",
DEV_AUTH_CONVEX_DEPLOYMENT: "dev:clever-rabbit-123",
DEV_AUTH_ENABLED: "1",
CONVEX_DEPLOYMENT: "dev:clever-rabbit-123",
DEV_AUTH_SECRET: CLOUD_DEV_AUTH_SECRET,
DEV_AUTH_SITE_URL: "http://127.0.0.1:3211",
}),
).toBe(false);
});
it("rejects cloud dev deployments when the configured secret is too short", () => {
expect(
isLocalDevAuthEnabled(
{
CONVEX_SITE_URL: "https://clever-rabbit-123.convex.cloud",
DEV_AUTH_CONVEX_DEPLOYMENT: "dev:clever-rabbit-123",
DEV_AUTH_ENABLED: "1",
DEV_AUTH_SECRET: "short",
DEV_AUTH_SITE_URL: "http://127.0.0.1:3211",
},
"short",
),
).toBe(false);
});
it("rejects cloud dev deployments without an explicit localhost dev auth site", () => {
expect(
isLocalDevAuthEnabled(
{
CONVEX_SITE_URL: "http://127.0.0.1:3211",
DEV_AUTH_ENABLED: "1",
DEV_AUTH_SECRET: CLOUD_DEV_AUTH_SECRET,
CONVEX_DEPLOYMENT: "dev:clever-rabbit-123",
},
CLOUD_DEV_AUTH_SECRET,
),
).toBe(false);
});
it("rejects localhost site URLs without a local deployment marker", () => {
expect(
isLocalDevAuthEnabled({
+33 -3
View File
@@ -3,18 +3,48 @@ type DevAuthEnv = {
CONVEX_SITE_URL?: string;
DEV_AUTH_CONVEX_DEPLOYMENT?: string;
DEV_AUTH_ENABLED?: string;
DEV_AUTH_SECRET?: string;
DEV_AUTH_SITE_URL?: string;
};
export function isLocalDevAuthEnabled(env: DevAuthEnv = process.env) {
const MIN_CLOUD_DEV_AUTH_SECRET_LENGTH = 32;
export function isLocalDevAuthEnabled(env: DevAuthEnv = process.env, suppliedSecret?: string) {
if (env.DEV_AUTH_ENABLED !== "1") return false;
const deployment = env.CONVEX_DEPLOYMENT?.trim() || env.DEV_AUTH_CONVEX_DEPLOYMENT?.trim() || "";
return isLocalConvexDeployment(deployment) && isLocalhostUrl(env.CONVEX_SITE_URL);
const convexDeployment = env.CONVEX_DEPLOYMENT?.trim();
const devAuthDeployment = env.DEV_AUTH_CONVEX_DEPLOYMENT?.trim();
const deployment = convexDeployment || devAuthDeployment || "";
if (isLocalConvexDeployment(deployment)) {
return isLocalhostUrl(env.CONVEX_SITE_URL);
}
if (isDevConvexDeployment(deployment)) {
return isLocalhostUrl(env.DEV_AUTH_SITE_URL) && hasValidCloudDevAuthSecret(env, suppliedSecret);
}
return false;
}
function isLocalConvexDeployment(deployment: string) {
return deployment.startsWith("local:") || deployment.startsWith("anonymous:");
}
function isDevConvexDeployment(deployment: string) {
return deployment.startsWith("dev:");
}
function hasValidCloudDevAuthSecret(env: DevAuthEnv, suppliedSecret: string | undefined) {
const expected = env.DEV_AUTH_SECRET?.trim();
const actual = suppliedSecret?.trim();
return Boolean(
expected &&
actual &&
expected.length >= MIN_CLOUD_DEV_AUTH_SECRET_LENGTH &&
actual === expected,
);
}
function isLocalhostUrl(value: string | undefined) {
if (!value) return false;
try {
+18
View File
@@ -0,0 +1,18 @@
export function assertLocalDevSeedAllowed(seedName: string): void {
const deployment =
process.env.CONVEX_DEPLOYMENT?.trim() || process.env.DEV_AUTH_CONVEX_DEPLOYMENT?.trim() || "";
if (
deployment.startsWith("dev:") ||
deployment.startsWith("local:") ||
deployment.startsWith("anonymous:")
) {
return;
}
if (
!deployment &&
(process.env.DEV_AUTH_ENABLED === "1" || process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION === "1")
) {
return;
}
throw new Error(`${seedName} dev seed is disabled outside local/dev deployments`);
}
+117 -2
View File
@@ -282,13 +282,128 @@ describe("requireGitHubAccountAge", () => {
expect(fetchMock).toHaveBeenCalledWith(
"https://api.github.com/user/12345",
expect.objectContaining({
headers: {
headers: expect.objectContaining({
"User-Agent": "clawhub",
Authorization: "Bearer ghp_test123",
},
}),
}),
);
});
it("omits Authorization header when GITHUB_TOKEN is blank", async () => {
vi.useFakeTimers();
const now = new Date("2026-02-02T12:00:00Z");
vi.setSystemTime(now);
vi.stubEnv("GITHUB_TOKEN", " ");
const runQuery = vi
.fn()
.mockResolvedValueOnce({
_id: "users:1",
githubCreatedAt: undefined,
})
.mockResolvedValueOnce("12345");
const runMutation = vi.fn();
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({
created_at: "2020-01-01T00:00:00Z",
}),
});
vi.stubGlobal("fetch", fetchMock);
await requireGitHubAccountAge({ runQuery, runMutation } as never, "users:1" as never);
expect(fetchMock).toHaveBeenCalledWith(
"https://api.github.com/user/12345",
expect.objectContaining({
headers: expect.objectContaining({ "User-Agent": "clawhub" }),
}),
);
expect(fetchMock.mock.calls[0]?.[1]?.headers).not.toHaveProperty("Authorization");
});
it("retries without Authorization when GITHUB_TOKEN is rejected", async () => {
vi.useFakeTimers();
const now = new Date("2026-02-02T12:00:00Z");
vi.setSystemTime(now);
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
vi.stubEnv("GITHUB_TOKEN", "ghp_expired");
const runQuery = vi
.fn()
.mockResolvedValueOnce({
_id: "users:1",
githubCreatedAt: undefined,
})
.mockResolvedValueOnce("12345");
const runMutation = vi.fn();
const fetchMock = vi
.fn()
.mockResolvedValueOnce({ ok: false, status: 401 })
.mockResolvedValueOnce({
ok: true,
json: async () => ({
created_at: "2020-01-01T00:00:00Z",
}),
});
vi.stubGlobal("fetch", fetchMock);
await requireGitHubAccountAge({ runQuery, runMutation } as never, "users:1" as never);
expect(fetchMock).toHaveBeenNthCalledWith(
1,
"https://api.github.com/user/12345",
expect.objectContaining({
headers: expect.objectContaining({
"User-Agent": "clawhub",
Authorization: "Bearer ghp_expired",
}),
}),
);
expect(fetchMock).toHaveBeenNthCalledWith(
2,
"https://api.github.com/user/12345",
expect.objectContaining({
headers: { "User-Agent": "clawhub" },
}),
);
expect(runMutation).toHaveBeenCalledWith(internal.users.setGitHubCreatedAtInternal, {
userId: "users:1",
githubCreatedAt: Date.parse("2020-01-01T00:00:00Z"),
});
expect(warnSpy).toHaveBeenCalledWith(
"[githubAccount] GitHub API auth was rejected; retrying lookup without auth",
);
});
it("does not retry unauthenticated 401 responses", async () => {
const runQuery = vi
.fn()
.mockResolvedValueOnce({
_id: "users:1",
githubCreatedAt: undefined,
})
.mockResolvedValueOnce("12345");
const runMutation = vi.fn();
const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 401 });
vi.stubGlobal("fetch", fetchMock);
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, "users:1" as never),
).rejects.toThrow(/GitHub account lookup failed/i);
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock).toHaveBeenCalledWith(
"https://api.github.com/user/12345",
expect.objectContaining({
headers: expect.objectContaining({ "User-Agent": "clawhub" }),
}),
);
expect(fetchMock.mock.calls[0]?.[1]?.headers).not.toHaveProperty("Authorization");
});
});
describe("syncGitHubProfile", () => {
+30 -26
View File
@@ -2,6 +2,7 @@ import { ConvexError } from "convex/values";
import { internal } from "../_generated/api";
import type { Id } from "../_generated/dataModel";
import type { ActionCtx } from "../_generated/server";
import { buildGitHubApiHeaders } from "./githubAuth";
import { GITHUB_PROFILE_SYNC_WINDOW_MS } from "./githubProfileSync";
const GITHUB_API = "https://api.github.com";
@@ -22,13 +23,34 @@ function assertGitHubNumericId(providerAccountId: string) {
}
}
function buildGitHubHeaders() {
const headers: Record<string, string> = { "User-Agent": "clawhub" };
const token = process.env.GITHUB_TOKEN;
if (token) {
headers.Authorization = `Bearer ${token}`;
async function fetchGitHubUserByNumericId(providerAccountId: string) {
assertGitHubNumericId(providerAccountId);
const url = `${GITHUB_API}/user/${providerAccountId}`;
const headers = await buildGitHubApiHeaders({ userAgent: "clawhub" });
const response = await fetch(url, {
headers,
});
if (response.status !== 401 || !headers.Authorization) return response;
console.warn("[githubAccount] GitHub API auth was rejected; retrying lookup without auth");
return await fetch(url, {
headers: { "User-Agent": "clawhub" },
});
}
export async function fetchGitHubCreatedAtByProviderAccountId(providerAccountId: string) {
const response = await fetchGitHubUserByNumericId(providerAccountId);
if (!response.ok) {
if (response.status === 403 || response.status === 429) {
throw new ConvexError("GitHub API rate limit exceeded — please try again in a few minutes");
}
throw new ConvexError("GitHub account lookup failed");
}
return headers;
const payload = (await response.json()) as GitHubUser;
const parsed = payload.created_at ? Date.parse(payload.created_at) : Number.NaN;
if (!Number.isFinite(parsed)) throw new ConvexError("GitHub account lookup failed");
return parsed;
}
export async function requireGitHubAccountAge(ctx: GitHubAccountGateCtx, userId: Id<"users">) {
@@ -48,24 +70,8 @@ export async function requireGitHubAccountAge(ctx: GitHubAccountGateCtx, userId:
// Invariant: GitHub is our only auth provider, so this should never happen.
throw new ConvexError("GitHub account required");
}
assertGitHubNumericId(providerAccountId);
// Fetch by immutable GitHub numeric ID to avoid username swap attacks entirely.
const response = await fetch(`${GITHUB_API}/user/${providerAccountId}`, {
headers: buildGitHubHeaders(),
});
if (!response.ok) {
if (response.status === 403 || response.status === 429) {
throw new ConvexError("GitHub API rate limit exceeded — please try again in a few minutes");
}
throw new ConvexError("GitHub account lookup failed");
}
const payload = (await response.json()) as GitHubUser;
const parsed = payload.created_at ? Date.parse(payload.created_at) : Number.NaN;
if (!Number.isFinite(parsed)) throw new ConvexError("GitHub account lookup failed");
createdAt = parsed;
createdAt = await fetchGitHubCreatedAtByProviderAccountId(providerAccountId);
await ctx.runMutation(internal.users.setGitHubCreatedAtInternal, {
userId,
githubCreatedAt: createdAt,
@@ -107,9 +113,7 @@ export async function syncGitHubProfile(ctx: ActionCtx, userId: Id<"users">) {
assertGitHubNumericId(providerAccountId);
const response = await fetch(`${GITHUB_API}/user/${providerAccountId}`, {
headers: buildGitHubHeaders(),
});
const response = await fetchGitHubUserByNumericId(providerAccountId);
if (!response.ok) {
// Silently fail - this is a best-effort sync, not critical path
console.warn(`[syncGitHubProfile] GitHub API error for user ${userId}: ${response.status}`);
+28
View File
@@ -76,6 +76,34 @@ describe("fetchGitHubRepositoryIdentity", () => {
);
});
it("does not use GitHub App auth for arbitrary repository lookup", async () => {
vi.stubEnv("GITHUB_APP_ID", "123");
vi.stubEnv("GITHUB_APP_INSTALLATION_ID", "456");
vi.stubEnv("GITHUB_APP_PRIVATE_KEY", "not-needed-for-this-test");
vi.stubEnv("GITHUB_TOKEN", "ghs_test_token");
const fetchMock = vi.fn(async () =>
Response.json({
id: 123,
full_name: "openclaw/clawhub",
owner: { login: "openclaw", id: 456 },
}),
);
await fetchGitHubRepositoryIdentity("openclaw/clawhub", fetchMock);
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock).toHaveBeenCalledWith(
"https://api.github.com/repos/openclaw/clawhub",
expect.objectContaining({
headers: expect.objectContaining({
Accept: "application/vnd.github+json",
Authorization: "Bearer ghs_test_token",
"User-Agent": "clawhub/package-trusted-publisher",
}),
}),
);
});
it("omits Authorization for repository lookup when GITHUB_TOKEN is blank", async () => {
vi.stubEnv("GITHUB_TOKEN", " ");
const fetchMock = vi.fn(async () =>
+13 -11
View File
@@ -1,3 +1,5 @@
import { buildGitHubApiHeaders } from "./githubAuth";
type JwtHeader = {
alg?: unknown;
kid?: unknown;
@@ -217,7 +219,7 @@ export async function fetchGitHubRepositoryIdentity(
throw new Error(`Invalid GitHub repository: ${repository}`);
}
const response = await fetchImpl(`https://api.github.com/repos/${normalizedRepository}`, {
headers: buildGitHubRepositoryLookupHeaders(),
headers: await buildGitHubRepositoryLookupHeaders(fetchImpl),
});
if (!response.ok) {
throw new Error(
@@ -239,16 +241,16 @@ export async function fetchGitHubRepositoryIdentity(
};
}
function buildGitHubRepositoryLookupHeaders() {
const headers: Record<string, string> = {
Accept: "application/vnd.github+json",
"User-Agent": "clawhub/package-trusted-publisher",
};
const token = process.env.GITHUB_TOKEN?.trim();
if (token) {
headers.Authorization = `Bearer ${token}`;
}
return headers;
async function buildGitHubRepositoryLookupHeaders(fetchImpl: typeof fetch) {
return await buildGitHubApiHeaders({
accept: "application/vnd.github+json",
fetchImpl,
userAgent: "clawhub/package-trusted-publisher",
// This lookup accepts arbitrary public repositories. GitHub App installation
// tokens only see repositories where the App is installed, so prefer PAT or
// anonymous auth here.
useGitHubApp: false,
});
}
export function normalizeGitHubRepository(repository: string) {
+100
View File
@@ -0,0 +1,100 @@
/* @vitest-environment node */
import { generateKeyPairSync } from "node:crypto";
import { afterEach, describe, expect, it, vi } from "vitest";
import { buildGitHubApiHeaders, createGitHubAppInstallationToken } from "./githubAuth";
function stubGitHubAppEnv() {
const { privateKey } = generateKeyPairSync("rsa", {
modulusLength: 2048,
privateKeyEncoding: { type: "pkcs1", format: "pem" },
publicKeyEncoding: { type: "spki", format: "pem" },
});
vi.stubEnv("GITHUB_APP_ID", "3536245");
vi.stubEnv("GITHUB_APP_INSTALLATION_ID", "987654");
vi.stubEnv("GITHUB_APP_PRIVATE_KEY", privateKey);
}
describe("githubAuth", () => {
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllEnvs();
});
it("mints a GitHub App installation token from app credentials", async () => {
stubGitHubAppEnv();
const fetchMock = vi.fn(async () =>
Response.json({
token: "ghs_app_token",
expires_at: "2026-02-02T13:00:00Z",
}),
);
await expect(
createGitHubAppInstallationToken({ fetchImpl: fetchMock, userAgent: "clawhub/test" }),
).resolves.toEqual({
token: "ghs_app_token",
expiresAt: Date.parse("2026-02-02T13:00:00Z"),
});
expect(fetchMock).toHaveBeenCalledWith(
"https://api.github.com/app/installations/987654/access_tokens",
expect.objectContaining({
method: "POST",
headers: expect.objectContaining({
Accept: "application/vnd.github+json",
Authorization: expect.stringMatching(/^Bearer [^.]+\.[^.]+\.[^.]+$/),
"User-Agent": "clawhub/test",
}),
}),
);
});
it("builds API headers with GitHub App auth before PAT fallback", async () => {
stubGitHubAppEnv();
vi.stubEnv("GITHUB_TOKEN", "ghp_pat_token");
const fetchMock = vi.fn(async () =>
Response.json({
token: "ghs_app_token",
expires_at: "2026-02-02T13:00:00Z",
}),
);
await expect(
buildGitHubApiHeaders({ fetchImpl: fetchMock, userAgent: "clawhub/test" }),
).resolves.toEqual({
Accept: "application/vnd.github+json",
Authorization: "Bearer ghs_app_token",
"User-Agent": "clawhub/test",
});
});
it("falls back to GITHUB_TOKEN when GitHub App credentials are absent", async () => {
vi.stubEnv("GITHUB_TOKEN", "ghp_pat_token");
await expect(buildGitHubApiHeaders({ userAgent: "clawhub/test" })).resolves.toEqual({
Accept: "application/vnd.github+json",
Authorization: "Bearer ghp_pat_token",
"User-Agent": "clawhub/test",
});
});
it("can skip GitHub App auth for arbitrary public resources", async () => {
stubGitHubAppEnv();
vi.stubEnv("GITHUB_TOKEN", "ghp_pat_token");
const fetchMock = vi.fn();
await expect(
buildGitHubApiHeaders({
fetchImpl: fetchMock,
userAgent: "clawhub/test",
useGitHubApp: false,
}),
).resolves.toEqual({
Accept: "application/vnd.github+json",
Authorization: "Bearer ghp_pat_token",
"User-Agent": "clawhub/test",
});
expect(fetchMock).not.toHaveBeenCalled();
});
});
+272
View File
@@ -0,0 +1,272 @@
const GITHUB_API = "https://api.github.com";
const DEFAULT_ACCEPT = "application/vnd.github+json";
const DEFAULT_USER_AGENT = "clawhub/github-api";
const APP_TOKEN_CACHE_BUFFER_MS = 60 * 1000;
type FetchImpl = typeof fetch;
type GitHubAppConfig = {
appId: string;
installationId: string;
privateKey: string;
};
type InstallationToken = {
token: string;
expiresAt: number;
};
type CachedInstallationToken = InstallationToken & {
cacheKey: string;
};
let cachedInstallationToken: CachedInstallationToken | null = null;
export function isGitHubAppConfigured(env: NodeJS.ProcessEnv = process.env) {
return Boolean(readGitHubAppConfig(env));
}
export async function buildGitHubApiHeaders(options: {
userAgent: string;
accept?: string;
fetchImpl?: FetchImpl;
allowAnonymous?: boolean;
useGitHubApp?: boolean;
}): Promise<Record<string, string>> {
const headers = buildGitHubHeaders({
userAgent: options.userAgent,
accept: options.accept,
});
if (options.useGitHubApp !== false) {
const appToken = await getCachedGitHubAppInstallationToken({
fetchImpl: options.fetchImpl,
userAgent: options.userAgent,
});
if (appToken) {
headers.Authorization = `Bearer ${appToken}`;
return headers;
}
}
const token = process.env.GITHUB_TOKEN?.trim();
if (token) {
headers.Authorization = `Bearer ${token}`;
return headers;
}
if (options.allowAnonymous === false) {
throw new Error("GitHub API authentication is not configured");
}
return headers;
}
export function buildGitHubHeaders(options: {
userAgent: string;
accept?: string;
token?: string;
isAppJwt?: boolean;
}) {
const headers: Record<string, string> = {
Accept: options.accept ?? DEFAULT_ACCEPT,
"User-Agent": options.userAgent,
};
if (options.token) {
headers.Authorization = `Bearer ${options.token}`;
}
return headers;
}
export async function createGitHubAppInstallationToken(
options: {
fetchImpl?: FetchImpl;
userAgent?: string;
env?: NodeJS.ProcessEnv;
now?: number;
} = {},
): Promise<InstallationToken> {
const env = options.env ?? process.env;
const config = readGitHubAppConfig(env);
if (!config) throw new Error("GitHub App credentials missing");
const jwt = await createGitHubAppJwt(config.appId, config.privateKey, options.now ?? Date.now());
const response = await (options.fetchImpl ?? fetch)(
`${GITHUB_API}/app/installations/${config.installationId}/access_tokens`,
{
method: "POST",
headers: buildGitHubHeaders({
userAgent: options.userAgent ?? DEFAULT_USER_AGENT,
token: jwt,
isAppJwt: true,
}),
},
);
if (!response.ok) {
const message = await response.text();
throw new Error(`GitHub App token failed: ${message}`);
}
const payload = (await response.json()) as { token?: string; expires_at?: string };
const token = payload.token?.trim();
if (!token) throw new Error("GitHub App token missing");
const expiresAt = payload.expires_at ? Date.parse(payload.expires_at) : Number.NaN;
if (!Number.isFinite(expiresAt)) throw new Error("GitHub App token expiry missing");
return { token, expiresAt };
}
async function getCachedGitHubAppInstallationToken(options: {
fetchImpl?: FetchImpl;
userAgent: string;
}) {
const config = readGitHubAppConfig(process.env);
if (!config) return null;
const now = Date.now();
const cacheKey = `${config.appId}:${config.installationId}:${hashCacheKey(config.privateKey)}`;
if (
cachedInstallationToken?.cacheKey === cacheKey &&
cachedInstallationToken.expiresAt - APP_TOKEN_CACHE_BUFFER_MS > now
) {
return cachedInstallationToken.token;
}
try {
const next = await createGitHubAppInstallationToken({
fetchImpl: options.fetchImpl,
userAgent: options.userAgent,
now,
});
cachedInstallationToken = { ...next, cacheKey };
return next.token;
} catch (error) {
console.warn(`[githubAuth] GitHub App token unavailable: ${errorMessage(error)}`);
return null;
}
}
function readGitHubAppConfig(env: NodeJS.ProcessEnv): GitHubAppConfig | null {
const appId = env.GITHUB_APP_ID?.trim();
const installationId = env.GITHUB_APP_INSTALLATION_ID?.trim();
const privateKey = env.GITHUB_APP_PRIVATE_KEY?.trim();
if (!appId || !installationId || !privateKey) return null;
return { appId, installationId, privateKey };
}
async function createGitHubAppJwt(appId: string, rawPrivateKey: string, nowMs: number) {
const now = Math.floor(nowMs / 1000);
const header = { alg: "RS256", typ: "JWT" };
const payload = { iat: now - 60, exp: now + 9 * 60, iss: appId };
const signingInput = `${base64UrlString(JSON.stringify(header))}.${base64UrlString(
JSON.stringify(payload),
)}`;
const key = await importPrivateKey(rawPrivateKey);
const signature = await crypto.subtle.sign(
"RSASSA-PKCS1-v1_5",
key,
new TextEncoder().encode(signingInput),
);
return `${signingInput}.${base64UrlBytes(new Uint8Array(signature))}`;
}
async function importPrivateKey(rawPrivateKey: string) {
const { label, der } = parsePem(rawPrivateKey);
const pkcs8 = label === "RSA PRIVATE KEY" ? wrapPkcs1PrivateKeyAsPkcs8(der) : der;
return await crypto.subtle.importKey(
"pkcs8",
pkcs8,
{ name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" },
false,
["sign"],
);
}
function parsePem(raw: string) {
const normalized = raw.replace(/\\n/g, "\n").trim();
const match = /^-----BEGIN ([A-Z0-9 ]+)-----\s*([A-Za-z0-9+/=\s]+)\s*-----END \1-----$/m.exec(
normalized,
);
if (!match) throw new Error("Invalid GitHub App private key");
const label = match[1];
if (label !== "PRIVATE KEY" && label !== "RSA PRIVATE KEY") {
throw new Error(`Unsupported GitHub App private key type: ${label}`);
}
return { label, der: base64ToBytes(match[2]) };
}
function wrapPkcs1PrivateKeyAsPkcs8(pkcs1: Uint8Array) {
const version = derInteger(0);
const rsaEncryptionAlgorithm = derSequence(
new Uint8Array([0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x01]),
new Uint8Array([0x05, 0x00]),
);
return derSequence(version, rsaEncryptionAlgorithm, derOctetString(pkcs1));
}
function derSequence(...parts: Uint8Array[]) {
return derTagged(0x30, concatBytes(parts));
}
function derInteger(value: number) {
return derTagged(0x02, new Uint8Array([value]));
}
function derOctetString(value: Uint8Array) {
return derTagged(0x04, value);
}
function derTagged(tag: number, value: Uint8Array) {
return concatBytes([new Uint8Array([tag]), derLength(value.length), value]);
}
function derLength(length: number) {
if (length < 0x80) return new Uint8Array([length]);
const bytes: number[] = [];
let remaining = length;
while (remaining > 0) {
bytes.unshift(remaining & 0xff);
remaining >>= 8;
}
return new Uint8Array([0x80 | bytes.length, ...bytes]);
}
function concatBytes(parts: Uint8Array[]) {
const total = parts.reduce((sum, part) => sum + part.length, 0);
const out = new Uint8Array(total);
let offset = 0;
for (const part of parts) {
out.set(part, offset);
offset += part.length;
}
return out;
}
function base64UrlString(value: string) {
return base64UrlBytes(new TextEncoder().encode(value));
}
function base64UrlBytes(value: Uint8Array) {
let binary = "";
for (const byte of value) binary += String.fromCharCode(byte);
return btoa(binary).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/g, "");
}
function base64ToBytes(value: string) {
const binary = atob(value.replace(/\s/g, ""));
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i += 1) {
bytes[i] = binary.charCodeAt(i);
}
return bytes;
}
function hashCacheKey(value: string) {
let hash = 0;
for (let i = 0; i < value.length; i += 1) {
hash = (hash * 31 + value.charCodeAt(i)) | 0;
}
return String(hash);
}
function errorMessage(error: unknown) {
return error instanceof Error ? error.message : String(error);
}
+3 -54
View File
@@ -1,8 +1,8 @@
"use node";
import { createPrivateKey, createSign } from "node:crypto";
import type { Id } from "../_generated/dataModel";
import type { ActionCtx } from "../_generated/server";
import { buildGitHubHeaders, createGitHubAppInstallationToken } from "./githubAuth";
const GITHUB_API = "https://api.github.com";
const DEFAULT_REPO = "clawdbot/skills";
@@ -93,7 +93,7 @@ export async function getGitHubBackupContext(): Promise<GitHubBackupContext> {
const repo = process.env.GITHUB_SKILLS_REPO ?? DEFAULT_REPO;
const root = process.env.GITHUB_SKILLS_ROOT ?? DEFAULT_ROOT;
const [repoOwner, repoName] = parseRepo(repo);
const token = await createInstallationToken();
const { token } = await createGitHubAppInstallationToken({ userAgent: USER_AGENT });
const repoInfo = await githubGet<RepoInfo>(token, `/repos/${repoOwner}/${repoName}`);
const branch = repoInfo.default_branch ?? "main";
@@ -439,48 +439,6 @@ async function fetchStorageBase64(ctx: ActionCtx, storageId: Id<"_storage">) {
return buffer.toString("base64");
}
async function createInstallationToken() {
const appId = process.env.GITHUB_APP_ID;
const installationId = process.env.GITHUB_APP_INSTALLATION_ID;
if (!appId || !installationId) {
throw new Error("GitHub App credentials missing");
}
const jwt = createAppJwt(appId);
const response = await fetch(`${GITHUB_API}/app/installations/${installationId}/access_tokens`, {
method: "POST",
headers: buildHeaders(jwt, true),
});
if (!response.ok) {
const message = await response.text();
throw new Error(`GitHub App token failed: ${message}`);
}
const payload = (await response.json()) as { token?: string };
if (!payload.token) throw new Error("GitHub App token missing");
return payload.token;
}
function createAppJwt(appId: string) {
const privateKey = loadPrivateKey();
const now = Math.floor(Date.now() / 1000);
const header = { alg: "RS256", typ: "JWT" };
const payload = { iat: now - 60, exp: now + 9 * 60, iss: appId };
const encodedHeader = base64Url(JSON.stringify(header));
const encodedPayload = base64Url(JSON.stringify(payload));
const signingInput = `${encodedHeader}.${encodedPayload}`;
const sign = createSign("RSA-SHA256");
sign.update(signingInput);
sign.end();
const signature = sign.sign(privateKey);
return `${signingInput}.${base64Url(signature)}`;
}
function loadPrivateKey() {
const raw = process.env.GITHUB_APP_PRIVATE_KEY;
if (!raw) throw new Error("GITHUB_APP_PRIVATE_KEY is not configured");
const normalized = raw.replace(/\\n/g, "\n");
return createPrivateKey(normalized);
}
async function createBlob(token: string, repoOwner: string, repoName: string, content: string) {
const result = await githubPost<{ sha: string }>(
token,
@@ -531,11 +489,7 @@ async function githubPatch(token: string, path: string, body: unknown) {
}
function buildHeaders(token: string, isAppJwt = false) {
return {
Authorization: `${isAppJwt ? "Bearer" : "token"} ${token}`,
Accept: "application/vnd.github+json",
"User-Agent": USER_AGENT,
};
return buildGitHubHeaders({ token, isAppJwt, userAgent: USER_AGENT });
}
function parseRepo(repo: string) {
@@ -570,11 +524,6 @@ function encodePath(path: string) {
.join("/");
}
function base64Url(value: string | Uint8Array) {
const buffer = typeof value === "string" ? Buffer.from(value) : Buffer.from(value);
return buffer.toString("base64").replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/g, "");
}
function toBase64(value: string) {
return Buffer.from(value).toString("base64");
}
+1 -6
View File
@@ -91,12 +91,7 @@ export function parseGitHubImportUrl(input: string): GitHubImportUrl {
}
function canonicalGitHubImportUrl(url: URL) {
const canonical = new URL(url.toString());
canonical.username = "";
canonical.password = "";
canonical.search = "";
canonical.hash = "";
return `${canonical.origin}${canonical.pathname}`;
return `https://${url.hostname}${url.pathname}`;
}
export async function resolveGitHubCommit(
+566
View File
@@ -0,0 +1,566 @@
import { describe, expect, it } from "vitest";
import {
buildGitHubSkillSourceSnapshot,
buildGitHubSkillSyncPlan,
parseSkillsShDisplayManifest,
} from "./githubSkillSync";
const encoder = new TextEncoder();
function bytes(text: string) {
return encoder.encode(text);
}
function repoEntries(entries: Record<string, string>) {
return Object.fromEntries(Object.entries(entries).map(([path, text]) => [path, bytes(text)]));
}
describe("parseSkillsShDisplayManifest", () => {
it("keeps the supported skills.sh rendering fields and drops invalid groups", () => {
const result = parseSkillsShDisplayManifest(
JSON.stringify({
notGrouped: "top",
groupings: [
{
title: "Agentic AI",
description: "Agentic workflows.",
skills: ["aiq-deploy", "nemoclaw-user-configure-security"],
},
{ title: "Broken", skills: [123] },
{ description: "Missing title", skills: ["ignored"] },
],
}),
);
expect(result).toEqual({
status: "ok",
manifest: {
notGrouped: "top",
groupings: [
{
title: "Agentic AI",
description: "Agentic workflows.",
skills: ["aiq-deploy", "nemoclaw-user-configure-security"],
},
],
},
});
});
it("marks missing and invalid manifests so the UI can fall back", () => {
expect(parseSkillsShDisplayManifest(undefined)).toEqual({
status: "missing",
manifest: undefined,
});
expect(parseSkillsShDisplayManifest("{nope")).toEqual({
status: "invalid",
manifest: undefined,
});
expect(parseSkillsShDisplayManifest(JSON.stringify({ groupings: [] }))).toEqual({
status: "invalid",
manifest: undefined,
});
});
});
describe("buildGitHubSkillSourceSnapshot", () => {
it("discovers skill folders, parses SKILL.md metadata, and hashes exact folder bytes", async () => {
const baseEntries = repoEntries({
"skills/aiq-deploy/SKILL.md":
"---\nname: AIQ Deploy\nversion: 0.2.0\ndescription: Deploy AgentIQ workflows.\n---\n# AIQ Deploy\n",
"skills/aiq-deploy/skill-card.md": "# Card\n",
"skills/vision-helper/SKILL.md": "# Vision Helper\n",
"skills.sh.json": JSON.stringify({
groupings: [{ title: "Agentic AI", skills: ["aiq-deploy"] }],
}),
});
const changedEntries = {
...baseEntries,
"skills/aiq-deploy/skill-card.md": bytes("# Card changed\n"),
};
const base = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "1".repeat(40),
entries: baseEntries,
});
const changed = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "2".repeat(40),
entries: changedEntries,
});
expect(base.manifestStatus).toBe("ok");
expect(base.manifest).toEqual({
groupings: [{ title: "Agentic AI", skills: ["aiq-deploy"] }],
});
expect(base.skills).toEqual(
expect.arrayContaining([
expect.objectContaining({
slug: "aiq-deploy",
displayName: "AIQ Deploy",
summary: "Deploy AgentIQ workflows.",
upstreamVersion: "0.2.0",
path: "skills/aiq-deploy",
skillMarkdownPath: "skills/aiq-deploy/SKILL.md",
skillMarkdown:
"---\nname: AIQ Deploy\nversion: 0.2.0\ndescription: Deploy AgentIQ workflows.\n---\n# AIQ Deploy\n",
skillCardMarkdownPath: "skills/aiq-deploy/skill-card.md",
skillCardMarkdown: "# Card\n",
}),
expect.objectContaining({
slug: "vision-helper",
displayName: "Vision Helper",
path: "skills/vision-helper",
skillMarkdownPath: "skills/vision-helper/SKILL.md",
skillMarkdown: "# Vision Helper\n",
}),
]),
);
expect(changed.skills.find((skill) => skill.slug === "aiq-deploy")?.contentHash).not.toBe(
base.skills.find((skill) => skill.slug === "aiq-deploy")?.contentHash,
);
expect(changed.skills.find((skill) => skill.slug === "vision-helper")?.contentHash).toBe(
base.skills.find((skill) => skill.slug === "vision-helper")?.contentHash,
);
});
it("includes valid filenames containing dot-dot text in folder hashes", async () => {
const base = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "1".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
"skills/aiq-deploy/payload..sh": "echo safe\n",
}),
});
const changed = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "2".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
"skills/aiq-deploy/payload..sh": "echo changed\n",
}),
});
expect(changed.skills[0]?.contentHash).not.toBe(base.skills[0]?.contentHash);
});
it("rejects duplicate normalized skill slugs before syncing content", async () => {
await expect(
buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "1".repeat(40),
entries: repoEntries({
"skills/aiq_deploy/SKILL.md": "# AIQ Deploy A\n",
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy B\n",
}),
}),
).rejects.toThrow(/duplicate normalized slug/i);
});
it("prefers the top-level skills catalog folder over duplicate plugin copies", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "1".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
"plugins/nvidia-skills/skills/aiq-deploy/SKILL.md": "# Plugin Copy\n",
"skills.sh.json": JSON.stringify({
groupings: [{ title: "Agentic AI", skills: ["aiq-deploy"] }],
}),
}),
});
expect(snapshot.skills.map((skill) => skill.path)).toEqual(["skills/aiq-deploy"]);
expect(snapshot.skills[0]?.displayName).toBe("AIQ Deploy");
});
it("rejects oversized cached markdown before writing Convex content docs", async () => {
await expect(
buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "1".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": `# AIQ Deploy\n${"x".repeat(513 * 1024)}`,
}),
}),
).rejects.toThrow(/too large to cache/i);
});
});
describe("buildGitHubSkillSyncPlan", () => {
it("marks changed upstream content pending", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "2".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy v2\n",
"skills.sh.json": JSON.stringify({
groupings: [{ title: "Agentic AI", skills: ["aiq-deploy"] }],
}),
}),
});
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "present",
githubCurrentContentHash: "old-hash",
githubScanStatus: "clean",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches).toEqual([
expect.objectContaining({
skillId: "skills:aiq-deploy",
slug: "aiq-deploy",
patch: expect.objectContaining({
githubCurrentCommit: "2".repeat(40),
githubCurrentContentHash: snapshot.skills[0]?.contentHash,
githubScanStatus: "pending",
moderationStatus: "active",
moderationReason: "pending.scan",
}),
}),
]);
expect(plan.skillInserts).toEqual([]);
expect(plan.stats.changed).toBe(1);
});
it("keeps clean scan status when only the repo commit changes", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "2".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "---\nversion: 0.2.0\n---\n# AIQ Deploy\n",
}),
});
const contentHash = snapshot.skills[0]?.contentHash ?? "";
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
latestVersionSummary: {
version: "0.2.0",
createdAt: 7,
},
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "present",
githubCurrentContentHash: contentHash,
githubScanStatus: "clean",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
githubCurrentCommit: "2".repeat(40),
githubCurrentContentHash: contentHash,
githubScanStatus: "clean",
moderationStatus: "active",
moderationVerdict: "clean",
});
expect(plan.skillPatches[0]?.patch).not.toHaveProperty("updatedAt");
expect(plan.skillPatches[0]?.patch).not.toHaveProperty("latestVersionSummary");
expect(plan.stats.unchanged).toBe(1);
});
it("updates existing skill ownership when a source is reassigned", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "2".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
}),
});
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:new-owner",
ownerPublisherId: "publishers:new-owner",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "present",
githubCurrentContentHash: snapshot.skills[0]?.contentHash ?? "",
githubScanStatus: "clean",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
ownerUserId: "users:new-owner",
ownerPublisherId: "publishers:new-owner",
});
});
it("preserves pending scan status for unchanged pending content", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "3".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
}),
});
const contentHash = snapshot.skills[0]?.contentHash ?? "";
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "present",
githubCurrentContentHash: contentHash,
githubScanStatus: "pending",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
githubCurrentCommit: "3".repeat(40),
githubCurrentContentHash: contentHash,
githubScanStatus: "pending",
moderationStatus: "active",
moderationReason: "pending.scan",
});
expect(plan.stats.unchanged).toBe(1);
});
it("preserves terminal scan status for unchanged current content", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "3".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
}),
});
const contentHash = snapshot.skills[0]?.contentHash ?? "";
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "present",
githubCurrentContentHash: contentHash,
githubScanStatus: "malicious",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
githubCurrentContentHash: contentHash,
githubScanStatus: "malicious",
moderationStatus: "hidden",
moderationReason: "scanner.llm.malicious",
});
});
it("preserves terminal scan status for unchanged current bytes", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "3".repeat(40),
entries: repoEntries({
"skills/aiq-deploy/SKILL.md": "# AIQ Deploy\n",
}),
});
const contentHash = snapshot.skills[0]?.contentHash ?? "";
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "present",
githubCurrentContentHash: contentHash,
githubScanStatus: "malicious",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
githubCurrentContentHash: contentHash,
githubScanStatus: "malicious",
moderationStatus: "hidden",
moderationReason: "scanner.llm.malicious",
});
expect(plan.skillPatches[0]?.patch).not.toHaveProperty("updatedAt");
expect(plan.stats.unchanged).toBe(1);
});
it("revives soft-deleted skills when a configured repo is synced again", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "mattpocock/skills",
defaultBranch: "main",
commit: "4".repeat(40),
entries: repoEntries({
"skills/engineering/tdd/SKILL.md": "# TDD\n",
}),
});
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:matt",
ownerUserId: "users:matt",
ownerPublisherId: "publishers:matt",
existingSkills: [
{
_id: "skills:tdd",
slug: "tdd",
displayName: "TDD",
githubPath: "skills/engineering/tdd",
githubCurrentStatus: "missing",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
githubCurrentStatus: "present",
githubRemovedAt: undefined,
softDeletedAt: undefined,
});
});
it("tombstones upstream removals instead of leaving stale installs active", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "2".repeat(40),
entries: repoEntries({
"skills/vision-helper/SKILL.md": "# Vision Helper\n",
}),
});
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubScanStatus: "clean",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches).toEqual([
expect.objectContaining({
skillId: "skills:aiq-deploy",
slug: "aiq-deploy",
patch: expect.objectContaining({
githubCurrentCommit: "2".repeat(40),
githubCurrentStatus: "missing",
githubRemovedAt: 123,
softDeletedAt: 123,
moderationStatus: "hidden",
moderationReason: "github.upstream.removed",
}),
}),
]);
expect(plan.skillInserts).toHaveLength(1);
expect(plan.stats.removed).toBe(1);
});
it("preserves first upstream removal time on later syncs", async () => {
const snapshot = await buildGitHubSkillSourceSnapshot({
repo: "NVIDIA/skills",
defaultBranch: "main",
commit: "3".repeat(40),
entries: repoEntries({
"skills/vision-helper/SKILL.md": "# Vision Helper\n",
}),
});
const plan = buildGitHubSkillSyncPlan({
sourceId: "githubSkillSources:nvidia",
ownerUserId: "users:nvidia",
ownerPublisherId: "publishers:nvidia",
existingSkills: [
{
_id: "skills:aiq-deploy",
slug: "aiq-deploy",
displayName: "AIQ Deploy",
githubPath: "skills/aiq-deploy",
githubCurrentStatus: "missing",
githubRemovedAt: 77,
githubScanStatus: "clean",
},
],
snapshot,
now: 123,
});
expect(plan.skillPatches[0]?.patch).toMatchObject({
githubCurrentCommit: "3".repeat(40),
githubCurrentStatus: "missing",
githubCurrentCheckedAt: 123,
githubRemovedAt: 77,
moderationStatus: "hidden",
moderationReason: "github.upstream.removed",
});
expect(plan.skillPatches[0]?.patch).not.toHaveProperty("updatedAt");
});
});
+671
View File
@@ -0,0 +1,671 @@
import { getFrontmatterValue, parseFrontmatter } from "./skills";
export type GitHubSkillScanStatus = "clean" | "suspicious" | "malicious" | "pending" | "failed";
export type GitHubCurrentStatus = "present" | "missing" | "unknown";
export type DisplayManifestStatus = "ok" | "missing" | "invalid" | "failed";
export type DisplayManifest = {
notGrouped?: "top" | "bottom";
groupings: Array<{
title: string;
description?: string;
skills: string[];
}>;
};
export type GitHubSkillSourceSnapshot = {
repo: string;
defaultBranch: string;
commit: string;
manifestStatus: DisplayManifestStatus;
manifestHash?: string;
manifest?: DisplayManifest;
skills: DiscoveredGitHubSkill[];
};
export type GitHubSkillSourceMetadataSnapshot = Omit<GitHubSkillSourceSnapshot, "skills"> & {
skills: DiscoveredGitHubSkillMetadata[];
};
export type DiscoveredGitHubSkill = {
slug: string;
displayName: string;
summary?: string;
upstreamVersion?: string;
path: string;
skillMarkdownPath: string;
skillMarkdown: string;
skillCardMarkdownPath?: string;
skillCardMarkdown?: string;
contentHash: string;
};
export type DiscoveredGitHubSkillMetadata = Omit<
DiscoveredGitHubSkill,
"skillMarkdown" | "skillCardMarkdown"
>;
export type ExistingGitHubSkillForSync = {
_id: string;
slug: string;
displayName: string;
summary?: string;
latestVersionSummary?: {
version: string;
createdAt: number;
};
githubPath?: string;
githubCurrentCommit?: string;
githubCurrentContentHash?: string;
githubCurrentStatus?: GitHubCurrentStatus;
githubScanStatus?: GitHubSkillScanStatus;
githubRemovedAt?: number;
softDeletedAt?: number;
};
export type GitHubBackedSkillModeration = {
moderationStatus: "active" | "hidden";
moderationReason?: string;
moderationVerdict?: "clean" | "suspicious" | "malicious";
moderationFlags: string[];
isSuspicious: boolean;
};
export type GitHubSkillPatchForSync = {
skillId: string;
slug: string;
patch: Record<string, unknown>;
};
export type GitHubSkillInsertForSync = {
slug: string;
doc: Record<string, unknown>;
};
export type GitHubSkillSyncPlan = {
sourcePatch: Record<string, unknown>;
skillPatches: GitHubSkillPatchForSync[];
skillInserts: GitHubSkillInsertForSync[];
stats: {
discovered: number;
inserted: number;
changed: number;
unchanged: number;
removed: number;
};
};
const SKILL_MARKDOWN_BASENAME = "skill.md";
const SKILL_CARD_MARKDOWN_BASENAME = "skill-card.md";
const MAX_STORED_MARKDOWN_BYTES = 512 * 1024;
const MAX_STORED_SKILL_CONTENT_BYTES = 768 * 1024;
export function parseSkillsShDisplayManifest(raw: string | undefined | null): {
status: DisplayManifestStatus;
manifest?: DisplayManifest;
} {
if (raw === undefined || raw === null) return { status: "missing", manifest: undefined };
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
return { status: "invalid", manifest: undefined };
}
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return { status: "invalid", manifest: undefined };
}
const record = parsed as Record<string, unknown>;
const rawGroups = record.groupings;
if (!Array.isArray(rawGroups)) return { status: "invalid", manifest: undefined };
const groupings = rawGroups.flatMap((group): DisplayManifest["groupings"] => {
if (!group || typeof group !== "object" || Array.isArray(group)) return [];
const groupRecord = group as Record<string, unknown>;
const title = typeof groupRecord.title === "string" ? groupRecord.title.trim() : "";
const description =
typeof groupRecord.description === "string" ? groupRecord.description.trim() : "";
const skills = Array.isArray(groupRecord.skills)
? groupRecord.skills
.filter((value): value is string => typeof value === "string")
.map((value) => value.trim())
.filter(Boolean)
: [];
if (!title || skills.length === 0) return [];
return [
{
title,
...(description ? { description } : {}),
skills,
},
];
});
if (groupings.length === 0) return { status: "invalid", manifest: undefined };
const notGrouped =
record.notGrouped === "top" || record.notGrouped === "bottom" ? record.notGrouped : undefined;
return {
status: "ok",
manifest: {
...(notGrouped ? { notGrouped } : {}),
groupings,
},
};
}
export async function buildGitHubSkillSourceSnapshot({
repo,
defaultBranch,
commit,
entries,
}: {
repo: string;
defaultBranch: string;
commit: string;
entries: Record<string, Uint8Array>;
}): Promise<GitHubSkillSourceSnapshot> {
const normalizedEntries = normalizeEntryMap(entries);
const manifestBytes = normalizedEntries["skills.sh.json"];
const manifestText = manifestBytes ? decodeUtf8(manifestBytes) : undefined;
const parsedManifest = parseSkillsShDisplayManifest(manifestText);
const manifestHash = manifestBytes ? await sha256Hex(manifestBytes) : undefined;
const skillPaths = discoverSkillPaths(normalizedEntries);
const skills: DiscoveredGitHubSkill[] = [];
for (const skillMdPath of skillPaths) {
const path = parentPath(skillMdPath);
const markdownBytes = normalizedEntries[skillMdPath] ?? new Uint8Array();
assertStoredMarkdownSize(skillMdPath, markdownBytes);
const markdown = decodeUtf8(markdownBytes);
const frontmatter = parseFrontmatter(markdown);
const folderName = path.split("/").filter(Boolean).at(-1) ?? "";
const slug = slugFromPathSegment(folderName);
if (!slug) continue;
const frontmatterName = getFrontmatterValue(frontmatter, "name")?.trim();
const frontmatterDescription = getFrontmatterValue(frontmatter, "description")?.trim();
const frontmatterVersion = getFrontmatterValue(frontmatter, "version")?.trim();
const heading = firstMarkdownHeading(markdown);
const skillCardMarkdownPath = findFolderFilePath(
normalizedEntries,
path,
SKILL_CARD_MARKDOWN_BASENAME,
);
const skillCardBytes = skillCardMarkdownPath
? normalizedEntries[skillCardMarkdownPath]
: undefined;
if (skillCardMarkdownPath && skillCardBytes) {
assertStoredMarkdownSize(skillCardMarkdownPath, skillCardBytes);
assertStoredSkillContentSize(markdownBytes.byteLength + skillCardBytes.byteLength);
} else {
assertStoredSkillContentSize(markdownBytes.byteLength);
}
const skillCardMarkdown = skillCardBytes ? decodeUtf8(skillCardBytes) : undefined;
skills.push({
slug,
displayName: frontmatterName || heading || titleizeSlug(slug),
...(frontmatterDescription ? { summary: frontmatterDescription } : {}),
...(frontmatterVersion ? { upstreamVersion: frontmatterVersion } : {}),
path,
skillMarkdownPath: skillMdPath,
skillMarkdown: markdown,
...(skillCardMarkdownPath ? { skillCardMarkdownPath } : {}),
...(skillCardMarkdown !== undefined ? { skillCardMarkdown } : {}),
contentHash: await computeGitHubSkillFolderContentHash(normalizedEntries, path),
});
}
const sortedSkills = skills.sort((a, b) => a.path.localeCompare(b.path));
assertUniqueDiscoveredSlugs(sortedSkills);
return {
repo,
defaultBranch,
commit,
manifestStatus: parsedManifest.status,
...(manifestHash ? { manifestHash } : {}),
...(parsedManifest.manifest ? { manifest: parsedManifest.manifest } : {}),
skills: sortedSkills,
};
}
export async function computeGitHubSkillFolderContentHash(
entries: Record<string, Uint8Array>,
folderPath: string,
) {
const normalizedEntries = normalizeEntryMap(entries);
const root = folderPath ? `${folderPath}/` : "";
const lines: string[] = [];
for (const [path, content] of Object.entries(normalizedEntries).sort(([a], [b]) =>
a.localeCompare(b),
)) {
if (root && path !== folderPath && !path.startsWith(root)) continue;
if (!root && path.includes("/")) continue;
const relativePath = root ? path.slice(root.length) : path;
if (!relativePath) continue;
const fileHash = await sha256Hex(content);
lines.push(`${relativePath}\0${content.byteLength}\0${fileHash}`);
}
return sha256Hex(new TextEncoder().encode(lines.join("\n")));
}
export function buildGitHubSkillSyncPlan({
sourceId,
ownerUserId,
ownerPublisherId,
existingSkills,
snapshot,
now,
}: {
sourceId: string;
ownerUserId: string;
ownerPublisherId?: string;
existingSkills: ExistingGitHubSkillForSync[];
snapshot: GitHubSkillSourceSnapshot | GitHubSkillSourceMetadataSnapshot;
now: number;
}): GitHubSkillSyncPlan {
const sourcePatch = {
repo: snapshot.repo,
defaultBranch: snapshot.defaultBranch,
lastSyncStatus: "ok",
lastSyncError: undefined,
lastSyncErrorAt: undefined,
displayManifestKind: "skills.sh",
displayManifestHash: snapshot.manifestHash,
displayManifestCommit: snapshot.commit,
displayManifestFetchedAt: now,
displayManifestStatus: snapshot.manifestStatus,
displayManifest: snapshot.manifest,
...(ownerPublisherId ? { ownerPublisherId } : {}),
updatedAt: now,
};
const existingByPath = new Map(
existingSkills
.filter((skill) => skill.githubPath)
.map((skill) => [skill.githubPath as string, skill]),
);
const existingBySlug = new Map(existingSkills.map((skill) => [skill.slug, skill]));
const matchedSkillIds = new Set<string>();
const skillPatches: GitHubSkillPatchForSync[] = [];
const skillInserts: GitHubSkillInsertForSync[] = [];
const stats = {
discovered: snapshot.skills.length,
inserted: 0,
changed: 0,
unchanged: 0,
removed: 0,
};
for (const discovered of snapshot.skills) {
const existing = existingByPath.get(discovered.path) ?? existingBySlug.get(discovered.slug);
if (!existing) {
const scanStatus: GitHubSkillScanStatus = "pending";
const moderation = githubBackedSkillModeration(scanStatus);
skillInserts.push({
slug: discovered.slug,
doc: {
slug: discovered.slug,
displayName: discovered.displayName,
summary: discovered.summary,
ownerUserId,
ownerPublisherId,
installKind: "github",
githubSourceId: sourceId,
githubPath: discovered.path,
githubHasSkillCard: Boolean(discovered.skillCardMarkdownPath),
githubCurrentCommit: snapshot.commit,
githubCurrentContentHash: discovered.contentHash,
githubCurrentStatus: "present",
githubCurrentCheckedAt: now,
githubScanStatus: scanStatus,
githubRemovedAt: undefined,
latestVersionId: undefined,
latestVersionSummary: latestVersionSummary(discovered.upstreamVersion, now),
tags: {},
capabilityTags: [],
softDeletedAt: undefined,
badges: undefined,
statsDownloads: 0,
statsStars: 0,
statsInstallsCurrent: 0,
statsInstallsAllTime: 0,
stats: {
downloads: 0,
stars: 0,
installsCurrent: 0,
installsAllTime: 0,
versions: 0,
comments: 0,
},
...moderation,
createdAt: now,
updatedAt: now,
},
});
stats.inserted += 1;
continue;
}
matchedSkillIds.add(existing._id);
const currentContentUnchanged =
existing.githubCurrentStatus === "present" &&
existing.githubCurrentContentHash === discovered.contentHash;
const scanStatus: GitHubSkillScanStatus = currentContentUnchanged
? githubScanStatusForUnchangedContent(existing.githubScanStatus)
: "pending";
const moderation = githubBackedSkillModeration(scanStatus);
const nextLatestVersionSummary = latestVersionSummary(
discovered.upstreamVersion,
existing.latestVersionSummary?.createdAt ?? now,
);
const materialChanged =
!currentContentUnchanged ||
existing.displayName !== discovered.displayName ||
(existing.summary ?? undefined) !== (discovered.summary ?? undefined) ||
(existing.githubPath ?? undefined) !== discovered.path ||
!sameLatestVersionSummary(existing.latestVersionSummary, nextLatestVersionSummary);
const patch = {
displayName: discovered.displayName,
summary: discovered.summary,
ownerUserId,
...(ownerPublisherId ? { ownerPublisherId } : {}),
githubSourceId: sourceId,
githubPath: discovered.path,
githubHasSkillCard: Boolean(discovered.skillCardMarkdownPath),
githubCurrentCommit: snapshot.commit,
githubCurrentContentHash: discovered.contentHash,
githubCurrentStatus: "present",
githubCurrentCheckedAt: now,
githubScanStatus: scanStatus,
githubRemovedAt: undefined,
softDeletedAt: undefined,
...(materialChanged
? {
latestVersionSummary: latestVersionSummary(discovered.upstreamVersion, now),
updatedAt: now,
}
: {}),
...moderation,
};
skillPatches.push({ skillId: existing._id, slug: existing.slug, patch });
if (materialChanged) stats.changed += 1;
else stats.unchanged += 1;
}
for (const existing of existingSkills) {
if (matchedSkillIds.has(existing._id)) continue;
const removedAt = existing.githubRemovedAt ?? now;
const moderation = githubBackedSkillModeration(
existing.githubScanStatus ?? "pending",
removedAt,
);
const wasAlreadyRemoved =
existing.githubCurrentStatus === "missing" && existing.githubRemovedAt !== undefined;
skillPatches.push({
skillId: existing._id,
slug: existing.slug,
patch: {
githubCurrentCommit: snapshot.commit,
githubCurrentStatus: "missing",
githubCurrentCheckedAt: now,
githubRemovedAt: removedAt,
softDeletedAt: existing.softDeletedAt ?? removedAt,
...(wasAlreadyRemoved ? {} : { updatedAt: now }),
...moderation,
},
});
stats.removed += 1;
}
return { sourcePatch, skillPatches, skillInserts, stats };
}
function githubScanStatusForUnchangedContent(
status: GitHubSkillScanStatus | undefined,
): GitHubSkillScanStatus {
if (
status === "clean" ||
status === "failed" ||
status === "malicious" ||
status === "suspicious"
) {
return status;
}
return "pending";
}
export function githubBackedSkillModeration(
scanStatus: GitHubSkillScanStatus,
removedAt?: number,
): GitHubBackedSkillModeration {
if (typeof removedAt === "number") {
return {
moderationStatus: "hidden",
moderationReason: "github.upstream.removed",
moderationVerdict: undefined,
moderationFlags: [],
isSuspicious: false,
};
}
if (scanStatus === "pending") {
return {
moderationStatus: "active",
moderationReason: "pending.scan",
moderationVerdict: undefined,
moderationFlags: [],
isSuspicious: false,
};
}
if (scanStatus === "failed") {
return {
moderationStatus: "hidden",
moderationReason: "scanner.failed",
moderationVerdict: undefined,
moderationFlags: [],
isSuspicious: false,
};
}
if (scanStatus === "malicious") {
return {
moderationStatus: "hidden",
moderationReason: "scanner.llm.malicious",
moderationVerdict: "malicious",
moderationFlags: ["blocked.malware"],
isSuspicious: true,
};
}
if (scanStatus === "suspicious") {
return {
moderationStatus: "active",
moderationReason: "scanner.llm.suspicious",
moderationVerdict: "suspicious",
moderationFlags: ["flagged.suspicious"],
isSuspicious: true,
};
}
return {
moderationStatus: "active",
moderationReason: undefined,
moderationVerdict: "clean",
moderationFlags: [],
isSuspicious: false,
};
}
function latestVersionSummary(version: string | undefined, now: number) {
if (!version) return undefined;
return {
version,
createdAt: now,
changelog: "Synced from GitHub source.",
changelogSource: "auto" as const,
};
}
function sameLatestVersionSummary(
a: ExistingGitHubSkillForSync["latestVersionSummary"] | undefined,
b: ReturnType<typeof latestVersionSummary>,
) {
if (!a && !b) return true;
if (!a || !b) return false;
return a.version === b.version;
}
function assertStoredMarkdownSize(path: string, bytes: Uint8Array) {
if (bytes.byteLength > MAX_STORED_MARKDOWN_BYTES) {
throw new Error(`GitHub skill markdown file is too large to cache: ${path}`);
}
}
function assertStoredSkillContentSize(totalBytes: number) {
if (totalBytes > MAX_STORED_SKILL_CONTENT_BYTES) {
throw new Error("GitHub skill cached markdown is too large");
}
}
function assertUniqueDiscoveredSlugs(skills: DiscoveredGitHubSkill[]) {
const firstPathBySlug = new Map<string, string>();
for (const skill of skills) {
const firstPath = firstPathBySlug.get(skill.slug);
if (firstPath) {
throw duplicateSkillSlugError(skill.slug, firstPath, skill.path);
}
firstPathBySlug.set(skill.slug, skill.path);
}
}
function normalizeEntryMap(entries: Record<string, Uint8Array>) {
const out: Record<string, Uint8Array> = {};
for (const [rawPath, bytes] of Object.entries(entries)) {
const normalized = normalizeRepoPath(rawPath);
if (!normalized) continue;
out[normalized] = new Uint8Array(bytes);
}
return out;
}
function discoverSkillPaths(entries: Record<string, Uint8Array>) {
const candidates = Object.keys(entries)
.filter((path) => path.split("/").at(-1)?.toLowerCase() === SKILL_MARKDOWN_BASENAME)
.sort((a, b) => a.localeCompare(b));
const pathsBySlug = new Map<string, string[]>();
for (const skillMdPath of candidates) {
const path = parentPath(skillMdPath);
const folderName = path.split("/").filter(Boolean).at(-1) ?? "";
const slug = slugFromPathSegment(folderName);
if (!slug) continue;
const paths = pathsBySlug.get(slug) ?? [];
paths.push(skillMdPath);
pathsBySlug.set(slug, paths);
}
const selected: string[] = [];
for (const [slug, paths] of pathsBySlug) {
if (paths.length === 1) {
selected.push(paths[0] as string);
continue;
}
const canonicalPath = `skills/${slug}/${SKILL_MARKDOWN_BASENAME}`;
const exactTopLevelMatches = paths.filter((path) => path.toLowerCase() === canonicalPath);
const topLevelSkillMatches = paths.filter((path) => path.toLowerCase().startsWith("skills/"));
if (exactTopLevelMatches.length === 1 && topLevelSkillMatches.length === 1) {
selected.push(exactTopLevelMatches[0] as string);
continue;
}
throw duplicateSkillSlugError(
slug,
parentPath(paths[0] as string),
parentPath(paths[1] as string),
);
}
return selected.sort((a, b) => a.localeCompare(b));
}
function duplicateSkillSlugError(slug: string, firstPath: string, secondPath: string) {
return new Error(
`GitHub skill source has duplicate normalized slug "${slug}" at ${firstPath} and ${secondPath}`,
);
}
function findFolderFilePath(
entries: Record<string, Uint8Array>,
folderPath: string,
basename: string,
) {
const prefix = folderPath ? `${folderPath}/` : "";
return Object.keys(entries).find((entryPath) => {
if (prefix) {
if (!entryPath.startsWith(prefix)) return false;
const relativePath = entryPath.slice(prefix.length);
return !relativePath.includes("/") && relativePath.toLowerCase() === basename;
}
return !entryPath.includes("/") && entryPath.toLowerCase() === basename;
});
}
function normalizeRepoPath(path: string) {
if (path.includes("\u0000")) return "";
const normalized = path
.replaceAll("\\", "/")
.trim()
.replace(/^\.\/+/, "")
.replace(/^\/+/, "");
if (!normalized) return "";
const segments = normalized.split("/").filter(Boolean);
if (segments.some((segment) => segment === "." || segment === "..")) return "";
return segments.join("/");
}
function parentPath(path: string) {
return path.split("/").slice(0, -1).join("/");
}
function decodeUtf8(bytes: Uint8Array) {
return new TextDecoder().decode(bytes);
}
async function sha256Hex(bytes: Uint8Array) {
const safe = new Uint8Array(bytes);
const buffer = safe.buffer.slice(safe.byteOffset, safe.byteOffset + safe.byteLength);
const digest = await crypto.subtle.digest("SHA-256", buffer);
return toHex(new Uint8Array(digest));
}
function toHex(bytes: Uint8Array) {
let out = "";
for (const byte of bytes) out += byte.toString(16).padStart(2, "0");
return out;
}
function slugFromPathSegment(value: string) {
return value
.trim()
.toLowerCase()
.replace(/[\s_]+/g, "-")
.replace(/[^a-z0-9-]+/g, "-")
.replace(/-+/g, "-")
.replace(/^-|-$/g, "");
}
function titleizeSlug(slug: string) {
return slug
.split("-")
.filter(Boolean)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
}
function firstMarkdownHeading(markdown: string) {
for (const line of markdown.split(/\r?\n/)) {
const match = /^#\s+(.+)$/.exec(line.trim());
if (match?.[1]) return match[1].trim();
}
return undefined;
}
+3 -54
View File
@@ -1,8 +1,8 @@
"use node";
import { createPrivateKey, createSign } from "node:crypto";
import type { Id } from "../_generated/dataModel";
import type { ActionCtx } from "../_generated/server";
import { buildGitHubHeaders, createGitHubAppInstallationToken } from "./githubAuth";
const GITHUB_API = "https://api.github.com";
const DEFAULT_REPO = "clawdbot/souls";
@@ -86,7 +86,7 @@ export async function getGitHubSoulBackupContext(): Promise<GitHubBackupContext>
const repo = process.env.GITHUB_SOULS_REPO ?? DEFAULT_REPO;
const root = process.env.GITHUB_SOULS_ROOT ?? DEFAULT_ROOT;
const [repoOwner, repoName] = parseRepo(repo);
const token = await createInstallationToken();
const { token } = await createGitHubAppInstallationToken({ userAgent: USER_AGENT });
const repoInfo = await githubGet<RepoInfo>(token, `/repos/${repoOwner}/${repoName}`);
const branch = repoInfo.default_branch ?? "main";
@@ -297,48 +297,6 @@ async function fetchStorageBase64(ctx: ActionCtx, storageId: Id<"_storage">) {
return buffer.toString("base64");
}
async function createInstallationToken() {
const appId = process.env.GITHUB_APP_ID;
const installationId = process.env.GITHUB_APP_INSTALLATION_ID;
if (!appId || !installationId) {
throw new Error("GitHub App credentials missing");
}
const jwt = createAppJwt(appId);
const response = await fetch(`${GITHUB_API}/app/installations/${installationId}/access_tokens`, {
method: "POST",
headers: buildHeaders(jwt, true),
});
if (!response.ok) {
const message = await response.text();
throw new Error(`GitHub App token failed: ${message}`);
}
const payload = (await response.json()) as { token?: string };
if (!payload.token) throw new Error("GitHub App token missing");
return payload.token;
}
function createAppJwt(appId: string) {
const privateKey = loadPrivateKey();
const now = Math.floor(Date.now() / 1000);
const header = { alg: "RS256", typ: "JWT" };
const payload = { iat: now - 60, exp: now + 9 * 60, iss: appId };
const encodedHeader = base64Url(JSON.stringify(header));
const encodedPayload = base64Url(JSON.stringify(payload));
const signingInput = `${encodedHeader}.${encodedPayload}`;
const sign = createSign("RSA-SHA256");
sign.update(signingInput);
sign.end();
const signature = sign.sign(privateKey);
return `${signingInput}.${base64Url(signature)}`;
}
function loadPrivateKey() {
const raw = process.env.GITHUB_APP_PRIVATE_KEY;
if (!raw) throw new Error("GITHUB_APP_PRIVATE_KEY is not configured");
const normalized = raw.replace(/\\n/g, "\n");
return createPrivateKey(normalized);
}
async function createBlob(token: string, repoOwner: string, repoName: string, content: string) {
const result = await githubPost<{ sha: string }>(
token,
@@ -389,11 +347,7 @@ async function githubPatch(token: string, path: string, body: unknown) {
}
function buildHeaders(token: string, isAppJwt = false) {
return {
Authorization: `${isAppJwt ? "Bearer" : "token"} ${token}`,
Accept: "application/vnd.github+json",
"User-Agent": USER_AGENT,
};
return buildGitHubHeaders({ token, isAppJwt, userAgent: USER_AGENT });
}
function parseRepo(repo: string) {
@@ -428,11 +382,6 @@ function encodePath(path: string) {
.join("/");
}
function base64Url(value: string | Uint8Array) {
const buffer = typeof value === "string" ? Buffer.from(value) : Buffer.from(value);
return buffer.toString("base64").replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/g, "");
}
function toBase64(value: string) {
return Buffer.from(value).toString("base64");
}
+214
View File
@@ -0,0 +1,214 @@
import { describe, expect, it } from "vitest";
import { buildSkillInstallResolution } from "./installResolver";
const baseSkill = {
slug: "aiq-deploy",
displayName: "AIQ Deploy",
latestVersionSummary: null,
installKind: "github" as const,
githubPath: "skills/aiq-deploy",
githubCurrentCommit: "1".repeat(40),
githubCurrentContentHash: "hash-aiq-deploy",
githubCurrentStatus: "present" as const,
githubScanStatus: "clean" as const,
githubRemovedAt: undefined,
};
const source = {
repo: "NVIDIA/skills",
defaultBranch: "main",
};
describe("buildSkillInstallResolution", () => {
it("returns an archive descriptor for hosted direct uploads", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: {
slug: "direct-skill",
displayName: "Direct Skill",
latestVersionSummary: { version: "1.2.3" },
},
source: null,
});
expect(resolution).toEqual({
ok: true,
slug: "direct-skill",
installKind: "archive",
archive: {
version: "1.2.3",
downloadUrl: "https://clawhub.ai/api/v1/download?slug=direct-skill&version=1.2.3",
},
});
});
it("returns a pinned GitHub descriptor when current upstream state is scan-clean", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: baseSkill,
source,
});
expect(resolution).toEqual({
ok: true,
slug: "aiq-deploy",
installKind: "github",
github: {
repo: "NVIDIA/skills",
path: "skills/aiq-deploy",
commit: "1".repeat(40),
contentHash: "hash-aiq-deploy",
sourceUrl: `https://github.com/NVIDIA/skills/tree/${"1".repeat(40)}/skills/aiq-deploy`,
},
});
});
it("allows GitHub-backed installs when upstream content changed and the current hash is clean", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: {
...baseSkill,
githubCurrentCommit: "2".repeat(40),
githubCurrentContentHash: "hash-aiq-deploy-v2",
},
source,
});
expect(resolution).toMatchObject({
ok: true,
installKind: "github",
github: {
commit: "2".repeat(40),
contentHash: "hash-aiq-deploy-v2",
sourceUrl: `https://github.com/NVIDIA/skills/tree/${"2".repeat(40)}/skills/aiq-deploy`,
},
});
});
it("allows GitHub-backed installs when only unrelated repository content changed", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: {
...baseSkill,
githubCurrentCommit: "2".repeat(40),
githubCurrentContentHash: baseSkill.githubCurrentContentHash,
},
source,
});
expect(resolution).toMatchObject({
ok: true,
installKind: "github",
github: {
commit: "2".repeat(40),
contentHash: "hash-aiq-deploy",
sourceUrl: `https://github.com/NVIDIA/skills/tree/${"2".repeat(40)}/skills/aiq-deploy`,
},
});
});
it.each([
{
name: "upstream path is missing",
patch: { githubCurrentStatus: "missing" as const },
reason: "github_upstream_missing",
status: 410,
},
{
name: "skill was pulled upstream",
patch: { githubRemovedAt: 456 },
reason: "github_upstream_removed",
status: 410,
},
{
name: "scan is pending",
patch: { githubScanStatus: "pending" as const },
reason: "github_verification_pending",
status: 423,
},
{
name: "scan failed",
patch: { githubScanStatus: "failed" as const },
reason: "github_scan_failed",
status: 403,
},
{
name: "scan is suspicious",
patch: { githubScanStatus: "suspicious" as const },
reason: "github_scan_failed",
status: 403,
},
])("blocks GitHub-backed installs when $name", ({ patch, reason, status }) => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: { ...baseSkill, ...patch },
source,
});
expect(resolution).toMatchObject({
ok: false,
slug: "aiq-deploy",
reason,
status,
});
});
it("explains pending GitHub-backed verification clearly", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: {
...baseSkill,
githubScanStatus: "pending",
},
source,
});
expect(resolution).toMatchObject({
ok: false,
slug: "aiq-deploy",
reason: "github_verification_pending",
status: 423,
message:
"GitHub-backed skill security scan is in progress. Try again shortly, or rerun with --force-install to install the unverified upstream commit.",
});
});
it("allows force-install for pending GitHub-backed verification", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: {
...baseSkill,
githubScanStatus: "pending",
},
source,
forceInstall: true,
});
expect(resolution).toMatchObject({
ok: true,
installKind: "github",
github: {
commit: "1".repeat(40),
contentHash: "hash-aiq-deploy",
},
});
});
it("does not force-install failed GitHub-backed scans", () => {
const resolution = buildSkillInstallResolution({
origin: "https://clawhub.ai",
skill: {
...baseSkill,
githubScanStatus: "failed",
},
source,
forceInstall: true,
});
expect(resolution).toMatchObject({
ok: false,
reason: "github_scan_failed",
status: 403,
});
});
});
+180
View File
@@ -0,0 +1,180 @@
export type GitHubSkillScanStatus = "clean" | "suspicious" | "malicious" | "pending" | "failed";
export type GitHubCurrentStatus = "present" | "missing" | "unknown";
export type InstallResolverSkill = {
slug: string;
displayName: string;
latestVersionSummary?: { version: string } | null;
installKind?: "github";
githubPath?: string;
githubCurrentCommit?: string;
githubCurrentContentHash?: string;
githubCurrentStatus?: GitHubCurrentStatus;
githubScanStatus?: GitHubSkillScanStatus;
githubRemovedAt?: number;
};
export type InstallResolverSource = {
repo: string;
defaultBranch?: string | null;
};
export type SkillInstallResolution =
| {
ok: true;
slug: string;
installKind: "archive";
archive: {
version: string;
downloadUrl: string;
};
}
| {
ok: true;
slug: string;
installKind: "github";
github: {
repo: string;
path: string;
commit: string;
contentHash: string;
sourceUrl: string;
};
}
| {
ok: false;
slug: string;
reason:
| "archive_version_missing"
| "github_source_missing"
| "github_upstream_removed"
| "github_upstream_missing"
| "github_upstream_unknown"
| "github_verification_pending"
| "github_scan_failed";
message: string;
status: 403 | 409 | 410 | 423;
};
export function buildSkillInstallResolution({
origin,
skill,
source,
forceInstall = false,
}: {
origin: string;
skill: InstallResolverSkill;
source: InstallResolverSource | null;
forceInstall?: boolean;
}): SkillInstallResolution {
if (skill.installKind !== "github") {
const version = skill.latestVersionSummary?.version;
if (!version) {
return block(skill.slug, "archive_version_missing", 409);
}
const url = new URL("/api/v1/download", origin);
url.searchParams.set("slug", skill.slug);
url.searchParams.set("version", version);
return {
ok: true,
slug: skill.slug,
installKind: "archive",
archive: {
version,
downloadUrl: url.toString(),
},
};
}
if (skill.githubRemovedAt) {
return block(skill.slug, "github_upstream_removed", 410);
}
if (skill.githubCurrentStatus === "missing") {
return block(skill.slug, "github_upstream_missing", 410);
}
if (
skill.githubScanStatus === "failed" ||
skill.githubScanStatus === "malicious" ||
skill.githubScanStatus === "suspicious"
) {
return block(skill.slug, "github_scan_failed", 403);
}
if (!source || !skill.githubPath) {
return block(skill.slug, "github_source_missing", 409);
}
if (
skill.githubCurrentStatus !== "present" ||
!skill.githubCurrentCommit ||
!skill.githubCurrentContentHash
) {
return block(skill.slug, "github_upstream_unknown", 423);
}
if (
skill.githubScanStatus !== "clean" &&
!(forceInstall && skill.githubScanStatus === "pending")
) {
return block(skill.slug, "github_verification_pending", 423);
}
return {
ok: true,
slug: skill.slug,
installKind: "github",
github: {
repo: source.repo,
path: skill.githubPath,
commit: skill.githubCurrentCommit,
contentHash: skill.githubCurrentContentHash,
sourceUrl: buildGitHubTreeUrl(source.repo, skill.githubCurrentCommit, skill.githubPath),
},
};
}
function block(
slug: string,
reason: Extract<SkillInstallResolution, { ok: false }>["reason"],
status: Extract<SkillInstallResolution, { ok: false }>["status"],
): SkillInstallResolution {
return {
ok: false,
slug,
reason,
status,
message: INSTALL_BLOCK_MESSAGES[reason],
};
}
const INSTALL_BLOCK_MESSAGES: Record<
Extract<SkillInstallResolution, { ok: false }>["reason"],
string
> = {
archive_version_missing: "Hosted skill has no downloadable version.",
github_source_missing: "GitHub-backed skill source metadata is incomplete.",
github_upstream_removed: "GitHub-backed skill has been removed upstream.",
github_upstream_missing: "GitHub-backed skill path is missing upstream.",
github_upstream_unknown: "GitHub-backed skill needs an upstream freshness check before install.",
github_verification_pending:
"GitHub-backed skill security scan is in progress. Try again shortly, or rerun with --force-install to install the unverified upstream commit.",
github_scan_failed: "GitHub-backed skill failed ClawHub security scanning.",
};
function buildGitHubTreeUrl(repo: string, commit: string, path: string) {
return `https://github.com/${encodeURIComponentRepo(repo)}/tree/${commit}/${encodeURIComponentPath(
path,
)}`;
}
function encodeURIComponentRepo(repo: string) {
return repo
.split("/")
.map((segment) => encodeURIComponent(segment))
.join("/");
}
function encodeURIComponentPath(path: string) {
return path
.split("/")
.map((segment) => encodeURIComponent(segment))
.join("/");
}
+18
View File
@@ -0,0 +1,18 @@
export const Events = {
GitHubSkillSourceSyncStarted: "github_skill_source_sync.started",
GitHubSkillSourceSyncCompleted: "github_skill_source_sync.completed",
GitHubSkillSourceSyncSourceFailed: "github_skill_source_sync.source_failed",
GitHubSkillSourceSyncFailed: "github_skill_source_sync.failed",
} as const;
export type EventName = (typeof Events)[keyof typeof Events];
type EventPayload = Record<string, unknown>;
export function logEvent(event: EventName, payload: EventPayload = {}) {
console.log(JSON.stringify({ event, ...payload }));
}
export function logErrorEvent(event: EventName, payload: EventPayload = {}) {
console.error(JSON.stringify({ event, ...payload }));
}
+83 -38
View File
@@ -1,6 +1,6 @@
import { describe, expect, it, vi } from "vitest";
import type { Doc } from "../_generated/dataModel";
import { isOfficialPublisher } from "./officialPublishers";
import { hasOfficialPublisherRow, isOfficialPublisher } from "./officialPublishers";
function makePublisher(
overrides: Partial<Record<keyof Doc<"publishers">, unknown>>,
@@ -17,60 +17,105 @@ function makePublisher(
} as Doc<"publishers">;
}
function makeOfficialRow(publisherId: string) {
return {
_id: `officialPublishers:${publisherId}`,
_creationTime: 1,
publisherId,
createdAt: 1,
updatedAt: 1,
};
}
function makeCtx({ officialPublisherIds = [] }: { officialPublisherIds?: string[] } = {}) {
return {
db: {
query: vi.fn((table: string) => {
if (table !== "officialPublishers") {
throw new Error(`Unexpected table ${table}`);
}
return {
withIndex: vi.fn((_indexName: string, buildQuery: (q: unknown) => unknown) => {
let requestedPublisherId: string | undefined;
buildQuery({
eq: vi.fn((field: string, value: string) => {
if (field === "publisherId") requestedPublisherId = value;
return {};
}),
});
return {
unique: vi.fn(async () =>
requestedPublisherId && officialPublisherIds.includes(requestedPublisherId)
? makeOfficialRow(requestedPublisherId)
: null,
),
};
}),
};
}),
},
};
}
describe("isOfficialPublisher", () => {
it("treats the openclaw org publisher as official", async () => {
const ctx = { db: { query: vi.fn() } };
it("treats a publisher with an official row as official", async () => {
const ctx = makeCtx({ officialPublisherIds: ["publishers:acme"] });
await expect(
isOfficialPublisher(ctx as never, makePublisher({ handle: "openclaw" })),
isOfficialPublisher(ctx as never, makePublisher({ _id: "publishers:acme", handle: "acme" })),
).resolves.toBe(true);
});
it("treats the nvidia org publisher as official", async () => {
const ctx = { db: { query: vi.fn() } };
it("treats a personal publisher with an official row as official", async () => {
const ctx = makeCtx({ officialPublisherIds: ["publishers:alice"] });
await expect(
isOfficialPublisher(ctx as never, makePublisher({ handle: "nvidia" })),
isOfficialPublisher(
ctx as never,
makePublisher({
_id: "publishers:alice",
kind: "user",
handle: "alice",
linkedUserId: "users:alice",
}),
),
).resolves.toBe(true);
});
it("treats personal publishers for openclaw org members as official", async () => {
const openclaw = makePublisher({ _id: "publishers:openclaw", handle: "openclaw" });
it("does not treat legacy official handles as official without a row", async () => {
const ctx = makeCtx();
await expect(
isOfficialPublisher(
ctx as never,
makePublisher({ _id: "publishers:openclaw", handle: "openclaw" }),
),
).resolves.toBe(false);
});
it("does not inherit official status from org membership", async () => {
const personal = makePublisher({
_id: "publishers:alice",
kind: "user",
handle: "alice",
linkedUserId: "users:alice",
});
const ctx = {
db: {
query: vi.fn((table: string) => {
if (table === "publishers") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn(async () => openclaw),
})),
};
}
if (table === "publisherMembers") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn(async () => ({
_id: "publisherMembers:alice",
publisherId: "publishers:openclaw",
userId: "users:alice",
role: "publisher",
createdAt: 1,
updatedAt: 1,
})),
})),
};
}
throw new Error(`Unexpected table ${table}`);
}),
},
};
const ctx = makeCtx({ officialPublisherIds: ["publishers:openclaw"] });
await expect(isOfficialPublisher(ctx as never, personal)).resolves.toBe(true);
await expect(isOfficialPublisher(ctx as never, personal)).resolves.toBe(false);
});
it("can check raw official rows independently from active publisher state", async () => {
const ctx = makeCtx({ officialPublisherIds: ["publishers:acme"] });
await expect(
isOfficialPublisher(
ctx as never,
makePublisher({ _id: "publishers:acme", handle: "acme", deactivatedAt: 123 }),
),
).resolves.toBe(false);
await expect(hasOfficialPublisherRow(ctx as never, "publishers:acme" as never)).resolves.toBe(
true,
);
});
});
+12 -35
View File
@@ -1,51 +1,28 @@
import type { Doc } from "../_generated/dataModel";
import type { MutationCtx, QueryCtx } from "../_generated/server";
import { toPublicPublisher, type PublicPublisher } from "./public";
import {
getPublisherByHandle,
getPublisherMembership,
normalizePublisherHandle,
} from "./publishers";
const OFFICIAL_ORG_HANDLES = ["openclaw", "nvidia"] as const;
const OFFICIAL_ORG_HANDLE_SET = new Set<string>(OFFICIAL_ORG_HANDLES);
type DbCtx = Pick<QueryCtx | MutationCtx, "db">;
type OfficialPublisherCandidate = Pick<
Doc<"publishers">,
| "_id"
| "_creationTime"
| "kind"
| "handle"
| "displayName"
| "image"
| "bio"
| "linkedUserId"
| "deletedAt"
| "deactivatedAt"
>;
type OfficialPublisherCandidate = Pick<Doc<"publishers">, "_id" | "deletedAt" | "deactivatedAt">;
export async function isOfficialPublisher(
ctx: DbCtx,
publisher: OfficialPublisherCandidate | null | undefined,
): Promise<boolean> {
if (!publisher || publisher.deletedAt || publisher.deactivatedAt) return false;
if (publisher.kind === "org") {
const handle = normalizePublisherHandle(publisher.handle);
return Boolean(handle && OFFICIAL_ORG_HANDLE_SET.has(handle));
}
if (!publisher.linkedUserId) return false;
return await hasOfficialPublisherRow(ctx, publisher._id);
}
for (const officialOrgHandle of OFFICIAL_ORG_HANDLES) {
const officialOrg = await getPublisherByHandle(ctx, officialOrgHandle);
if (!officialOrg || officialOrg.deletedAt || officialOrg.deactivatedAt) continue;
const membership = await getPublisherMembership(ctx, officialOrg._id, publisher.linkedUserId);
if (membership) return true;
}
return false;
export async function hasOfficialPublisherRow(
ctx: DbCtx,
publisherId: Doc<"publishers">["_id"],
): Promise<boolean> {
const officialPublisher = await ctx.db
.query("officialPublishers")
.withIndex("by_publisher", (q) => q.eq("publisherId", publisherId))
.unique();
return Boolean(officialPublisher);
}
export async function toPublicPublisherWithOfficial(
+10
View File
@@ -271,6 +271,15 @@ function buildVerification(source: SourceInfo | undefined): PackageVerificationS
scanStatus: "not-run",
};
}
// `source.path` is the package directory inside the source repo (e.g.
// "examples/openclaw-plugin"). When the package lives at the repo root the
// CLI sends "." (or empty), and there's nothing useful to serialize. Only
// promote real subpaths into `verification.sourcePath` so consumers can
// build a `raw.githubusercontent.com/<repo>/<sha>/<path>/` base URL for
// resolving relative README asset references.
const rawPath = typeof source.path === "string" ? source.path.trim() : "";
const sourcePath =
rawPath && rawPath !== "." ? rawPath.replace(/^\/+/, "").replace(/\/+$/, "") : undefined;
return {
tier: "source-linked",
scope: "artifact-only",
@@ -278,6 +287,7 @@ function buildVerification(source: SourceInfo | undefined): PackageVerificationS
sourceRepo: source.repo || source.url,
sourceCommit: source.commit,
sourceTag: source.ref,
sourcePath: sourcePath || undefined,
hasProvenance: false,
scanStatus: "not-run",
};
+3
View File
@@ -20,6 +20,7 @@ const SHARED_KEYS = [
"summary",
"capabilityTags",
"executesCode",
"stats",
"runtimeId",
"scanStatus",
"softDeletedAt",
@@ -45,6 +46,7 @@ const CAPABILITY_SHARED_KEYS = [
"capabilityTags",
"executesCode",
"verificationTier",
"stats",
"scanStatus",
"softDeletedAt",
"createdAt",
@@ -70,6 +72,7 @@ const PLUGIN_CATEGORY_SHARED_KEYS = [
"pluginCategoryTags",
"executesCode",
"verificationTier",
"stats",
"scanStatus",
"softDeletedAt",
"createdAt",
+16
View File
@@ -66,6 +66,22 @@ describe("public skill mapping", () => {
});
});
it("exposes GitHub-backed skill source fields", () => {
const mapped = toPublicSkill(
makeSkill({
installKind: "github",
githubPath: "skills/demo",
githubCurrentCommit: "a".repeat(40),
}),
);
expect(mapped).toMatchObject({
installKind: "github",
githubPath: "skills/demo",
githubCurrentCommit: "a".repeat(40),
});
});
it("returns skill when moderationStatus is active", () => {
const skill = makeSkill({ moderationStatus: "active" });
expect(toPublicSkill(skill)).not.toBeNull();
+21 -2
View File
@@ -24,6 +24,12 @@ export type PublicSkill = Pick<
| "canonicalSkillId"
| "forkOf"
| "latestVersionId"
| "installKind"
| "githubPath"
| "githubCurrentCommit"
| "githubCurrentStatus"
| "githubScanStatus"
| "githubHasSkillCard"
| "tags"
| "capabilityTags"
| "badges"
@@ -31,7 +37,9 @@ export type PublicSkill = Pick<
| "isSuspicious"
| "createdAt"
| "updatedAt"
>;
> & {
githubSourceRepo?: string;
};
/**
* Minimum set of fields needed by `hydrateResults` to filter and convert
@@ -52,6 +60,10 @@ export type HydratableSkill = Pick<
| "canonicalSkillId"
| "forkOf"
| "latestVersionId"
| "installKind"
| "githubHasSkillCard"
| "githubCurrentStatus"
| "githubScanStatus"
| "latestVersionSummary"
| "tags"
| "capabilityTags"
@@ -68,7 +80,8 @@ export type HydratableSkill = Pick<
| "isSuspicious"
| "createdAt"
| "updatedAt"
>;
> &
Partial<Pick<Doc<"skills">, "githubPath" | "githubCurrentCommit">>;
export type PublicSoul = Pick<
Doc<"souls">,
@@ -149,6 +162,12 @@ export function toPublicSkill(skill: HydratableSkill | null | undefined): Public
canonicalSkillId: skill.canonicalSkillId,
forkOf: skill.forkOf,
latestVersionId: skill.latestVersionId,
installKind: skill.installKind,
githubPath: skill.githubPath,
githubCurrentCommit: skill.githubCurrentCommit,
githubCurrentStatus: skill.githubCurrentStatus,
githubScanStatus: skill.githubScanStatus,
githubHasSkillCard: skill.githubHasSkillCard,
tags: skill.tags,
capabilityTags: skill.capabilityTags,
badges: skill.badges,
+3 -1
View File
@@ -5,6 +5,7 @@ import {
getPublishFileSizeError,
getPublishTotalSizeError,
MAX_CLAWPACK_BYTES,
MAX_PACKAGE_MULTIPART_BYTES,
MAX_PUBLISH_FILE_BYTES,
} from "./publishLimits";
@@ -31,8 +32,9 @@ describe("publishLimits", () => {
);
});
it("keeps the ClawPack tarball limit separate from legacy file limits", () => {
it("keeps ClawPack capacity above the multipart request budget", () => {
expect(MAX_CLAWPACK_BYTES).toBe(120 * 1024 * 1024);
expect(MAX_CLAWPACK_BYTES).toBeGreaterThan(MAX_PACKAGE_MULTIPART_BYTES);
expect(MAX_CLAWPACK_BYTES).toBeGreaterThan(MAX_PUBLISH_FILE_BYTES);
});
});
+12 -1
View File
@@ -1,6 +1,17 @@
import { MAX_PACKAGE_CLAWPACK_BYTES } from "clawhub-schema";
export {
estimatePackageMultipartUploadBytes,
getPackageMultipartSizeError,
isPackageMultipartUploadTooLarge,
MAX_PACKAGE_MULTIPART_BYTES,
type PackageMultipartUploadField,
type PackageMultipartUploadPart,
} from "clawhub-schema";
export const MAX_PUBLISH_TOTAL_BYTES = 50 * 1024 * 1024;
export const MAX_PUBLISH_FILE_BYTES = 10 * 1024 * 1024;
export const MAX_CLAWPACK_BYTES = 120 * 1024 * 1024;
export const MAX_CLAWPACK_BYTES = MAX_PACKAGE_CLAWPACK_BYTES;
type SizedPathLike = {
path: string;
+196
View File
@@ -0,0 +1,196 @@
import { describe, expect, it } from "vitest";
import { buildGitHubSkillCatalogDisplay } from "./publisherCatalogDisplay";
const baseItem = {
kind: "skill" as const,
summary: null,
icon: null,
href: "/nvidia/example",
downloads: 0,
stars: 0,
isOfficial: true,
updatedAt: 1,
sourceBacked: true,
sourceRepo: "NVIDIA/skills",
sourcePath: null,
sourceVerifiedCommit: null,
};
describe("buildGitHubSkillCatalogDisplay", () => {
it("groups source-backed skills by manifest entries and ignores missing entries", () => {
const display = buildGitHubSkillCatalogDisplay({
sources: [
{
_id: "githubSkillSources:nvidia",
repo: "NVIDIA/skills",
displayManifestStatus: "ok",
displayManifest: {
notGrouped: "bottom",
groupings: [
{
title: "Agentic AI",
description: "Agentic AI skills.",
skills: ["aiq-deploy", "missing-upstream-entry"],
},
{
title: "Vision AI",
skills: ["vision-helper"],
},
],
},
},
],
items: [
{
...baseItem,
_id: "skills:aiq-deploy",
displayName: "AIQ Deploy",
slug: "aiq-deploy",
sourceId: "githubSkillSources:nvidia",
},
{
...baseItem,
_id: "skills:vision-helper",
displayName: "Vision Helper",
slug: "vision-helper",
sourceId: "githubSkillSources:nvidia",
},
],
});
expect(display).toMatchObject({
mode: "grouped",
sourceRepos: ["NVIDIA/skills"],
sections: [
{
title: "Agentic AI",
description: "Agentic AI skills.",
sourceRepo: "NVIDIA/skills",
items: [{ displayName: "AIQ Deploy" }],
},
{
title: "Vision AI",
sourceRepo: "NVIDIA/skills",
items: [{ displayName: "Vision Helper" }],
},
],
});
});
it("matches manifest entries by normalized display name and places unlisted skills at the requested edge", () => {
const display = buildGitHubSkillCatalogDisplay({
sources: [
{
_id: "githubSkillSources:nvidia",
repo: "NVIDIA/skills",
displayManifestStatus: "ok",
displayManifest: {
notGrouped: "top",
groupings: [
{
title: "Physical AI",
skills: ["Isaac Sim Helper"],
},
],
},
},
],
items: [
{
...baseItem,
_id: "skills:isaac-sim-helper",
displayName: "Isaac Sim Helper",
slug: "isaac-sim-helper",
sourceId: "githubSkillSources:nvidia",
},
{
...baseItem,
_id: "skills:extra",
displayName: "Extra Skill",
slug: "extra",
sourceId: "githubSkillSources:nvidia",
},
],
});
expect(display?.sections.map((section) => section.title)).toEqual([
"Other skills",
"Physical AI",
]);
expect(display?.sections[0]?.items.map((item) => item.displayName)).toEqual(["Extra Skill"]);
});
it("falls back to the normal catalog when the source manifest is missing or invalid", () => {
const display = buildGitHubSkillCatalogDisplay({
sources: [
{
_id: "githubSkillSources:nvidia",
repo: "NVIDIA/skills",
displayManifestStatus: "invalid",
},
],
items: [
{
...baseItem,
_id: "skills:aiq-deploy",
displayName: "AIQ Deploy",
slug: "aiq-deploy",
sourceId: "githubSkillSources:nvidia",
},
],
});
expect(display).toBeNull();
});
it("keeps source-backed skills from non-renderable sources in other skills", () => {
const display = buildGitHubSkillCatalogDisplay({
sources: [
{
_id: "githubSkillSources:nvidia",
repo: "NVIDIA/skills",
displayManifestStatus: "ok",
displayManifest: {
notGrouped: "bottom",
groupings: [
{
title: "Agentic AI",
skills: ["aiq-deploy"],
},
],
},
},
{
_id: "githubSkillSources:invalid",
repo: "example/skills",
displayManifestStatus: "invalid",
},
],
items: [
{
...baseItem,
_id: "skills:aiq-deploy",
displayName: "AIQ Deploy",
slug: "aiq-deploy",
sourceId: "githubSkillSources:nvidia",
},
{
...baseItem,
_id: "skills:unlisted",
displayName: "Unlisted Source Skill",
slug: "unlisted-source-skill",
sourceRepo: "example/skills",
sourceId: "githubSkillSources:invalid",
},
],
});
expect(display?.sections.map((section) => section.title)).toEqual([
"Agentic AI",
"Other skills",
]);
expect(display?.sections.at(-1)?.items.map((item) => item.displayName)).toEqual([
"Unlisted Source Skill",
]);
});
});
+157
View File
@@ -0,0 +1,157 @@
type DisplayManifest = {
notGrouped?: "top" | "bottom";
groupings: Array<{
title: string;
description?: string;
skills: string[];
}>;
};
export type GitHubSkillCatalogSource = {
_id: string;
repo: string;
displayManifestStatus?: "ok" | "missing" | "invalid" | "failed";
displayManifest?: DisplayManifest;
};
export type GitHubSkillCatalogItem = {
_id: string;
kind: "skill" | "plugin";
displayName: string;
slug?: string | null;
sourceBacked?: boolean;
sourceId?: string | null;
sourceRepo?: string | null;
sourcePath?: string | null;
sourceVerifiedCommit?: string | null;
summary: string | null;
icon: string | null;
href: string;
downloads: number;
stars: number;
isOfficial: boolean;
updatedAt: number;
};
export type GitHubSkillCatalogSection = {
key: string;
title: string;
description: string | null;
sourceRepo: string | null;
items: GitHubSkillCatalogItem[];
};
export type GitHubSkillCatalogDisplay = {
mode: "grouped";
sourceRepos: string[];
sections: GitHubSkillCatalogSection[];
};
function normalizeManifestSkillKey(value: string) {
return value
.trim()
.toLowerCase()
.replace(/[\s_]+/g, "-")
.replace(/[^a-z0-9-]+/g, "-")
.replace(/-+/g, "-")
.replace(/^-|-$/g, "");
}
function getItemKeys(item: GitHubSkillCatalogItem) {
const keys = new Set<string>();
if (item.slug) keys.add(normalizeManifestSkillKey(item.slug));
keys.add(normalizeManifestSkillKey(item.displayName));
const sourcePathName = item.sourcePath?.split("/").filter(Boolean).at(-1);
if (sourcePathName) keys.add(normalizeManifestSkillKey(sourcePathName));
return keys;
}
function findManifestItem(
candidates: GitHubSkillCatalogItem[],
manifestEntry: string,
usedItemIds: Set<string>,
) {
const key = normalizeManifestSkillKey(manifestEntry);
if (!key) return null;
return (
candidates.find((item) => !usedItemIds.has(item._id) && getItemKeys(item).has(key)) ?? null
);
}
function isRenderableSource(source: GitHubSkillCatalogSource) {
return (
source.displayManifestStatus === "ok" &&
Boolean(source.displayManifest) &&
source.displayManifest!.groupings.length > 0
);
}
export function buildGitHubSkillCatalogDisplay({
sources,
items,
}: {
sources: GitHubSkillCatalogSource[];
items: GitHubSkillCatalogItem[];
}): GitHubSkillCatalogDisplay | null {
const renderableSources = sources.filter(isRenderableSource);
if (renderableSources.length === 0) return null;
const sourceRepos = Array.from(new Set(renderableSources.map((source) => source.repo)));
const usedItemIds = new Set<string>();
const sections: GitHubSkillCatalogSection[] = [];
const otherPosition = renderableSources.some(
(source) => source.displayManifest?.notGrouped === "top",
)
? "top"
: "bottom";
for (const source of renderableSources) {
const sourceItems = items.filter(
(item) => item.kind === "skill" && item.sourceId === source._id,
);
if (sourceItems.length === 0) continue;
for (const [groupIndex, group] of source.displayManifest!.groupings.entries()) {
const groupItems = group.skills
.map((entry) => findManifestItem(sourceItems, entry, usedItemIds))
.filter((item): item is GitHubSkillCatalogItem => Boolean(item));
if (groupItems.length === 0) continue;
for (const item of groupItems) usedItemIds.add(item._id);
sections.push({
key: `${source._id}:${groupIndex}:${group.title}`,
title: group.title,
description: group.description ?? null,
sourceRepo: source.repo,
items: groupItems,
});
}
}
const otherItems = items.filter((item) => item.kind === "skill" && !usedItemIds.has(item._id));
const otherSection =
otherItems.length > 0
? {
key: "other-skills",
title: "Other skills",
description: null,
sourceRepo: null,
items: otherItems,
}
: null;
const orderedSections =
otherPosition === "top" && otherSection
? [otherSection, ...sections]
: [...sections, ...(otherSection ? [otherSection] : [])];
if (orderedSections.length === 0) return null;
return {
mode: "grouped",
sourceRepos,
sections: orderedSections,
};
}
+27
View File
@@ -52,6 +52,13 @@ describe("publisher stat maintenance", () => {
return {
collect: vi.fn(async () => [
makeSkill({ statsDownloads: 11, statsStars: 2, statsInstallsAllTime: 5 }),
makeSkill({
_id: "skills:hidden",
moderationStatus: "hidden",
statsDownloads: 100,
statsStars: 100,
statsInstallsAllTime: 100,
}),
]),
};
}
@@ -136,6 +143,26 @@ describe("publisher stat maintenance", () => {
expect(ctx.db.query).not.toHaveBeenCalled();
});
it("does not count hidden skills in public publisher aggregates", async () => {
const ctx = {
db: {
get: vi.fn(),
patch: vi.fn(),
query: vi.fn(),
},
};
await adjustPublisherStatsForSkillChange(
ctx as never,
null,
makeSkill({ moderationStatus: "hidden", moderationReason: "pending.scan" }),
);
expect(ctx.db.get).not.toHaveBeenCalled();
expect(ctx.db.patch).not.toHaveBeenCalled();
expect(ctx.db.query).not.toHaveBeenCalled();
});
it("keeps legacy aggregate updates bounded when skill-only aggregates are missing", async () => {
const patch = vi.fn();
const ctx = {
+3 -2
View File
@@ -1,5 +1,6 @@
import type { Doc, Id } from "../_generated/dataModel";
import type { MutationCtx } from "../_generated/server";
import { isPublicSkillDoc } from "./globalStats";
import { readCanonicalStat } from "./skillStats";
export type PublisherStatsContribution = {
@@ -27,7 +28,7 @@ export function emptyPublisherStatsContribution(): PublisherStatsContribution {
}
export function getSkillPublisherContribution(skill: Doc<"skills">): PublisherStatsContribution {
if (skill.softDeletedAt) return emptyPublisherStatsContribution();
if (!isPublicSkillDoc(skill)) return emptyPublisherStatsContribution();
const totalInstalls = readCanonicalStat(skill, "installsAllTime");
const totalDownloads = readCanonicalStat(skill, "downloads");
const totalStars = readCanonicalStat(skill, "stars");
@@ -91,7 +92,7 @@ function publisherHasSkillTotalStats(
);
}
async function recomputePublisherStats(
export async function recomputePublisherStats(
ctx: Pick<MutationCtx, "db">,
publisherId: Id<"publishers">,
): Promise<PublisherStatsContribution> {
+1 -1
View File
@@ -32,7 +32,7 @@ function normalizeGeneratedPublisherHandle(handle: string | undefined | null) {
return sanitized || undefined;
}
function derivePersonalPublisherHandle(user: Doc<"users">) {
export function derivePersonalPublisherHandle(user: Doc<"users">) {
const emailLocalPart = user.email?.split("@")[0];
const userIdSuffix = String(user._id).split(":").pop();
return (
+23
View File
@@ -95,6 +95,29 @@ describe("extractDigestFields", () => {
expect(digest.updatedAt).toBe(2000);
});
it("fills digest rank stats from legacy nested stats", () => {
const skill = makeSkillDoc({
statsDownloads: undefined,
statsStars: undefined,
statsInstallsCurrent: undefined,
statsInstallsAllTime: undefined,
stats: {
downloads: 42,
installsCurrent: 10,
installsAllTime: 100,
stars: 5,
versions: 3,
comments: 1,
},
});
const digest = extractDigestFields(skill as never);
expect(digest.statsDownloads).toBe(42);
expect(digest.statsStars).toBe(5);
expect(digest.statsInstallsCurrent).toBe(10);
expect(digest.statsInstallsAllTime).toBe(100);
});
it("omits large fields not needed for search", () => {
const skill = makeSkillDoc({
moderationEvidence: [
+30
View File
@@ -1,7 +1,9 @@
import type { Doc, Id } from "../_generated/dataModel";
import type { MutationCtx } from "../_generated/server";
import type { HydratableSkill, PublicPublisher } from "./public";
import { getOwnerPublisher } from "./publishers";
import { tokenize } from "./searchText";
import { readCanonicalStat } from "./skillStats";
function pick<T extends Record<string, unknown>, K extends keyof T>(obj: T, keys: K[]): Pick<T, K> {
return Object.fromEntries(keys.map((k) => [k, obj[k]])) as Pick<T, K>;
@@ -24,6 +26,10 @@ const SHARED_KEYS = [
"canonicalSkillId",
"forkOf",
"latestVersionId",
"installKind",
"githubHasSkillCard",
"githubCurrentStatus",
"githubScanStatus",
"latestVersionSummary",
"tags",
"capabilityTags",
@@ -61,6 +67,10 @@ export type SkillSearchDigestFields = Pick<Doc<"skills">, (typeof SHARED_KEYS)[n
export function extractDigestFields(skill: Doc<"skills">): SkillSearchDigestFields {
return {
...pick(skill, [...SHARED_KEYS]),
statsDownloads: readCanonicalStat(skill, "downloads"),
statsStars: readCanonicalStat(skill, "stars"),
statsInstallsCurrent: readCanonicalStat(skill, "installsCurrent"),
statsInstallsAllTime: readCanonicalStat(skill, "installsAllTime"),
skillId: skill._id,
normalizedSlug: normalizeSkillSearchText(skill.slug),
normalizedSlugFirstToken: getFirstSearchToken(skill.slug),
@@ -125,6 +135,26 @@ export async function upsertSkillSearchDigest(
}
}
export async function syncSkillSearchDigestForSkill(
ctx: Pick<MutationCtx, "db">,
skill: Doc<"skills"> | null | undefined,
) {
if (!skill) return;
const fields = await extractValidatedDigestFields(ctx, skill);
const owner = await getOwnerPublisher(ctx, {
ownerPublisherId: skill.ownerPublisherId,
ownerUserId: skill.ownerUserId,
});
await upsertSkillSearchDigest(ctx, {
...fields,
ownerHandle: owner?.handle ?? "",
ownerKind: owner?.kind,
ownerName: owner?.linkedUserId ? owner.handle : undefined,
ownerDisplayName: owner?.displayName,
ownerImage: owner?.image,
});
}
/** Compare new fields against existing row. Returns true if any field differs. */
function hasDigestChanged(
existing: Doc<"skillSearchDigest">,
+2 -2
View File
@@ -4,7 +4,7 @@ import { ConvexError } from "convex/values";
// - Lowercase letters, digits, and single hyphens only.
// - Must start and end with a letter or digit.
// - No consecutive hyphens ("--", "---", ...).
// - Length 3..48 (URL/SEO friendly, aligned with publisher handle).
// - Length 3..96 (URL-friendly, but long enough for source-backed upstream slugs).
//
// The pattern enforces first/last char class and forbids consecutive hyphens
// via a negative lookahead. Length bounds are checked separately so we can
@@ -12,7 +12,7 @@ import { ConvexError } from "convex/values";
const SLUG_PATTERN = /^[a-z0-9](?:(?!--)[a-z0-9-])*[a-z0-9]$/;
const MIN_SLUG_LENGTH = 3;
const MAX_SLUG_LENGTH = 48;
const MAX_SLUG_LENGTH = 96;
// Reserved slugs. These are blocked because they would:
// 1. Clash semantically with top-level routes under src/routes/*.
+830
View File
@@ -11,6 +11,9 @@ vi.mock("./_generated/api", () => ({
getUserOwnedSkillsBackfillPageInternal: Symbol("getUserOwnedSkillsBackfillPageInternal"),
applyUserStatsBackfillPatchInternal: Symbol("applyUserStatsBackfillPatchInternal"),
backfillUserStatsInternal: Symbol("backfillUserStatsInternal"),
getPublisherStatsBackfillPageInternal: Symbol("getPublisherStatsBackfillPageInternal"),
recomputePublisherStatsInternal: Symbol("recomputePublisherStatsInternal"),
backfillPublisherStatsInternal: Symbol("backfillPublisherStatsInternal"),
getSkillFingerprintBackfillPageInternal: Symbol("getSkillFingerprintBackfillPageInternal"),
applySkillFingerprintBackfillPatchInternal: Symbol(
"applySkillFingerprintBackfillPatchInternal",
@@ -24,6 +27,7 @@ vi.mock("./_generated/api", () => ({
nominateUserForEmptySkillSpamInternal: Symbol("nominateUserForEmptySkillSpamInternal"),
cleanupEmptySkillsInternal: Symbol("cleanupEmptySkillsInternal"),
nominateEmptySkillSpammersInternal: Symbol("nominateEmptySkillSpammersInternal"),
repairLegacyPublisherOwnership: Symbol("repairLegacyPublisherOwnership"),
},
skills: {
backfillLatestSkillModerationInternal: Symbol("skills.backfillLatestSkillModerationInternal"),
@@ -44,11 +48,15 @@ const {
applySkillCapabilityTagsInternal,
backfillDigestVersionSummary,
backfillLatestVersionSummaryInternal,
backfillPublisherStatsInternalHandler,
backfillSkillSearchDigestInternal,
backfillSkillFingerprintsInternalHandler,
backfillSkillSummariesInternalHandler,
backfillUserStatsInternalHandler,
cleanupEmptySkillsInternalHandler,
nominateEmptySkillSpammersInternalHandler,
repairLegacyPublisherOwnershipForUserHandler,
repairLegacyPublisherOwnershipHandler,
upsertSkillBadgeRecordInternal,
} = await import("./maintenance");
const { internal } = await import("./_generated/api");
@@ -58,7 +66,773 @@ function makeBlob(text: string) {
return { text: () => Promise.resolve(text) } as unknown as Blob;
}
type QueryEq = {
eq: (field: string, value: unknown) => QueryEq;
};
function makeLegacyPublisherOwnershipDb() {
const now = 1_717_456_000_000;
let nextPublisherId = 2;
let nextMemberId = 1;
const users = new Map<string, Record<string, unknown>>([
[
"users:legacy",
{
_id: "users:legacy",
_creationTime: now - 1000,
handle: "legacy-owner",
name: "Legacy Owner",
displayName: "Legacy Owner",
deletedAt: undefined,
deactivatedAt: undefined,
purgedAt: undefined,
},
],
[
"users:deleted",
{
_id: "users:deleted",
_creationTime: now - 1000,
handle: "deleted-owner",
deletedAt: now - 10,
deactivatedAt: undefined,
purgedAt: undefined,
},
],
]);
const publishers = new Map<string, Record<string, unknown>>([
[
"publishers:existing",
{
_id: "publishers:existing",
_creationTime: now - 500,
kind: "user",
handle: "existing-owner",
displayName: "Existing Owner",
linkedUserId: "users:existing",
publishedSkills: 0,
publishedPackages: 0,
totalInstalls: 0,
totalDownloads: 0,
totalStars: 0,
skillTotalInstalls: 0,
skillTotalDownloads: 0,
skillTotalStars: 0,
createdAt: now - 500,
updatedAt: now - 500,
},
],
]);
const publisherMembers = new Map<string, Record<string, unknown>>();
const skills = new Map<string, Record<string, unknown>>([
[
"skills:legacy",
{
_id: "skills:legacy",
_creationTime: now - 400,
slug: "legacy-skill",
displayName: "Legacy Skill",
ownerUserId: "users:legacy",
ownerPublisherId: undefined,
latestVersionId: "skillVersions:legacy",
tags: { latest: "skillVersions:legacy" },
stats: {
downloads: 10,
stars: 3,
installsCurrent: 2,
installsAllTime: 5,
comments: 0,
versions: 1,
},
statsDownloads: 10,
statsStars: 3,
statsInstallsCurrent: 2,
statsInstallsAllTime: 5,
softDeletedAt: undefined,
moderationStatus: "active",
createdAt: now - 300,
updatedAt: now - 200,
},
],
[
"skills:deleted-owner",
{
_id: "skills:deleted-owner",
_creationTime: now - 400,
slug: "deleted-owner-skill",
displayName: "Deleted Owner Skill",
ownerUserId: "users:deleted",
ownerPublisherId: undefined,
latestVersionId: "skillVersions:deleted-owner",
tags: { latest: "skillVersions:deleted-owner" },
stats: {
downloads: 1,
stars: 0,
installsCurrent: 0,
installsAllTime: 0,
comments: 0,
versions: 1,
},
softDeletedAt: undefined,
moderationStatus: "active",
createdAt: now - 300,
updatedAt: now - 200,
},
],
]);
const skillVersions = new Map<string, Record<string, unknown>>([
[
"skillVersions:legacy",
{
_id: "skillVersions:legacy",
skillId: "skills:legacy",
version: "1.0.0",
softDeletedAt: undefined,
},
],
[
"skillVersions:deleted-owner",
{
_id: "skillVersions:deleted-owner",
skillId: "skills:deleted-owner",
version: "1.0.0",
softDeletedAt: undefined,
},
],
]);
const skillSlugAliases = new Map<string, Record<string, unknown>>([
[
"skillSlugAliases:legacy",
{
_id: "skillSlugAliases:legacy",
slug: "old-legacy-skill",
skillId: "skills:legacy",
ownerUserId: "users:legacy",
ownerPublisherId: undefined,
createdAt: now - 250,
updatedAt: now - 250,
},
],
]);
const skillEmbeddings = new Map<string, Record<string, unknown>>([
[
"skillEmbeddings:legacy",
{
_id: "skillEmbeddings:legacy",
skillId: "skills:legacy",
versionId: "skillVersions:legacy",
ownerId: "users:legacy",
ownerPublisherId: undefined,
embedding: [0.1, 0.2],
isLatest: true,
isApproved: true,
visibility: "public",
updatedAt: now - 200,
},
],
]);
const skillSearchDigest = new Map<string, Record<string, unknown>>([
[
"skillSearchDigest:legacy",
{
_id: "skillSearchDigest:legacy",
skillId: "skills:legacy",
slug: "legacy-skill",
displayName: "Legacy Skill",
ownerUserId: "users:legacy",
ownerPublisherId: undefined,
ownerHandle: "legacy-owner",
ownerKind: "user",
stats: {
downloads: 10,
stars: 3,
installsCurrent: 2,
installsAllTime: 5,
comments: 0,
versions: 1,
},
statsDownloads: 10,
statsStars: 3,
statsInstallsCurrent: 2,
statsInstallsAllTime: 5,
softDeletedAt: undefined,
moderationStatus: "active",
createdAt: now - 300,
updatedAt: now - 200,
},
],
]);
const packages = new Map<string, Record<string, unknown>>([
[
"packages:legacy",
{
_id: "packages:legacy",
_creationTime: now - 400,
name: "@legacy-owner/demo-plugin",
normalizedName: "@legacy-owner/demo-plugin",
displayName: "Demo Plugin",
family: "bundle-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:legacy",
ownerPublisherId: undefined,
summary: "Demo package",
latestReleaseId: undefined,
tags: {},
compatibility: undefined,
capabilities: undefined,
verification: undefined,
scanStatus: "clean",
stats: { downloads: 7, installs: 4, stars: 2, versions: 1 },
softDeletedAt: undefined,
createdAt: now - 300,
updatedAt: now - 200,
},
],
]);
const packageSearchDigest = new Map<string, Record<string, unknown>>([
[
"packageSearchDigest:legacy",
{
_id: "packageSearchDigest:legacy",
packageId: "packages:legacy",
name: "@legacy-owner/demo-plugin",
normalizedName: "@legacy-owner/demo-plugin",
displayName: "Demo Plugin",
family: "bundle-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:legacy",
ownerPublisherId: undefined,
ownerHandle: "legacy-owner",
ownerKind: "user",
summary: "Demo package",
scanStatus: "clean",
softDeletedAt: undefined,
createdAt: now - 300,
updatedAt: now - 200,
},
],
]);
const packageCapabilitySearchDigest = new Map<string, Record<string, unknown>>();
const packagePluginCategorySearchDigest = new Map<string, Record<string, unknown>>();
const tableMap: Record<string, Map<string, Record<string, unknown>>> = {
users,
publishers,
publisherMembers,
skills,
skillVersions,
skillSlugAliases,
skillEmbeddings,
skillSearchDigest,
packages,
packageSearchDigest,
packageCapabilitySearchDigest,
packagePluginCategorySearchDigest,
};
const patchCalls: Array<{ id: string; patch: Record<string, unknown> }> = [];
const insertCalls: Array<{ table: string; value: Record<string, unknown> }> = [];
const getRows = (table: string) => Array.from(tableMap[table]?.values() ?? []);
const getTableForId = (id: string) => id.split(":")[0];
const readField = (row: Record<string, unknown>, field: string) =>
field.split(".").reduce<unknown>((value, part) => {
if (!value || typeof value !== "object") return undefined;
return (value as Record<string, unknown>)[part];
}, row);
const makeQuery = (table: string, rows: Record<string, unknown>[]) => ({
collect: vi.fn(async () => rows),
unique: vi.fn(async () => rows[0] ?? null),
take: vi.fn(async (limit: number) => rows.slice(0, limit)),
order: vi.fn(() => ({
take: vi.fn(async (limit: number) => rows.slice(0, limit)),
paginate: vi.fn(async ({ cursor, numItems }: { cursor: string | null; numItems: number }) =>
paginateRows(rows, cursor, numItems),
),
})),
paginate: vi.fn(async ({ cursor, numItems }: { cursor: string | null; numItems: number }) =>
paginateRows(rows, cursor, numItems),
),
withIndex: vi.fn((indexName: string, build?: (q: QueryEq) => unknown) => {
const filters: Array<{ field: string; value: unknown }> = [];
const q: QueryEq = {
eq: (field, value) => {
filters.push({ field, value });
return q;
},
};
build?.(q);
let indexedRows = getRows(table).filter((row) =>
filters.every((filter) => readField(row, filter.field) === filter.value),
);
if (table === "users" && indexName === "by_active_handle") {
indexedRows = indexedRows.filter(
(row) => row.deletedAt === undefined && row.deactivatedAt === undefined,
);
}
return makeQuery(table, indexedRows);
}),
});
const db = {
get: vi.fn(async (id: string) => tableMap[getTableForId(id)]?.get(id) ?? null),
query: vi.fn((table: string) => makeQuery(table, getRows(table))),
patch: vi.fn(async (id: string, patch: Record<string, unknown>) => {
patchCalls.push({ id, patch });
const row = tableMap[getTableForId(id)]?.get(id);
if (row) Object.assign(row, patch);
}),
insert: vi.fn(async (table: string, value: Record<string, unknown>) => {
const id =
table === "publishers"
? `publishers:created${nextPublisherId++}`
: table === "publisherMembers"
? `publisherMembers:created${nextMemberId++}`
: `${table}:created`;
insertCalls.push({ table, value });
tableMap[table].set(id, { _id: id, _creationTime: now, ...value });
return id;
}),
delete: vi.fn(async (id: string) => {
tableMap[getTableForId(id)]?.delete(id);
}),
normalizeId: vi.fn(),
};
return {
db,
patchCalls,
insertCalls,
tableMap,
};
}
function paginateRows(rows: Record<string, unknown>[], cursor: string | null, numItems: number) {
const start = cursor ? Number(cursor) : 0;
const page = rows.slice(start, start + numItems);
const next = start + page.length;
return {
page,
continueCursor: next >= rows.length ? null : String(next),
isDone: next >= rows.length,
};
}
describe("maintenance legacy publisher ownership repair", () => {
it("dry-runs legacy publisher ownership repair without writes", async () => {
const { db, patchCalls, insertCalls } = makeLegacyPublisherOwnershipDb();
const result = await repairLegacyPublisherOwnershipHandler(
{ db, scheduler: { runAfter: vi.fn() } } as never,
{ phase: "users", dryRun: true, batchSize: 10, scheduleNext: false },
);
expect(result).toMatchObject({
phase: "users",
dryRun: true,
scanned: 1,
repaired: 1,
skipped: 0,
isDone: true,
});
expect(patchCalls).toEqual([]);
expect(insertCalls).toEqual([]);
});
it("reports dry-run personal publisher handle conflicts without writes", async () => {
const { db, tableMap, patchCalls, insertCalls } = makeLegacyPublisherOwnershipDb();
tableMap.users.set("users:conflict", {
_id: "users:conflict",
_creationTime: 1_717_456_000_000 - 1000,
handle: "existing-owner",
name: "Conflicting Owner",
displayName: "Conflicting Owner",
deletedAt: undefined,
deactivatedAt: undefined,
purgedAt: undefined,
});
const result = await repairLegacyPublisherOwnershipHandler(
{ db, scheduler: { runAfter: vi.fn() } } as never,
{ phase: "users", dryRun: true, batchSize: 10, scheduleNext: false },
);
expect(result).toMatchObject({
phase: "users",
dryRun: true,
scanned: 2,
repaired: 1,
skipped: 1,
isDone: true,
errors: ['user:users:conflict: Publisher handle "@existing-owner" is already claimed'],
});
expect(patchCalls).toEqual([]);
expect(insertCalls).toEqual([]);
});
it("skips apply-mode personal publisher handle conflicts while repairing other users", async () => {
const { db, tableMap } = makeLegacyPublisherOwnershipDb();
tableMap.users.set("users:conflict", {
_id: "users:conflict",
_creationTime: 1_717_456_000_000 - 1000,
handle: "existing-owner",
name: "Conflicting Owner",
displayName: "Conflicting Owner",
deletedAt: undefined,
deactivatedAt: undefined,
purgedAt: undefined,
});
const result = await repairLegacyPublisherOwnershipHandler(
{ db, scheduler: { runAfter: vi.fn() } } as never,
{ phase: "users", dryRun: false, batchSize: 10, scheduleNext: false },
);
const createdPublisher = Array.from(tableMap.publishers.values()).find(
(publisher) => publisher.handle === "legacy-owner",
);
expect(result).toMatchObject({
phase: "users",
dryRun: false,
scanned: 2,
repaired: 1,
skipped: 1,
isDone: true,
errors: ['user:users:conflict: Publisher handle "@existing-owner" is already claimed'],
});
expect(createdPublisher).toMatchObject({
kind: "user",
linkedUserId: "users:legacy",
});
expect(tableMap.users.get("users:legacy")).toMatchObject({
personalPublisherId: createdPublisher?._id,
});
expect(tableMap.users.get("users:conflict")).not.toHaveProperty("personalPublisherId");
});
it("repairs active legacy users, skills, aliases, embeddings, and packages", async () => {
const { db, tableMap, patchCalls, insertCalls } = makeLegacyPublisherOwnershipDb();
const scheduler = { runAfter: vi.fn() };
const usersResult = await repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "users",
dryRun: false,
batchSize: 10,
scheduleNext: false,
});
const createdPublisher = Array.from(tableMap.publishers.values()).find(
(publisher) => publisher.handle === "legacy-owner",
);
expect(usersResult).toMatchObject({
phase: "users",
dryRun: false,
scanned: 1,
repaired: 1,
skipped: 0,
isDone: true,
});
expect(createdPublisher).toMatchObject({
kind: "user",
handle: "legacy-owner",
displayName: "Legacy Owner",
linkedUserId: "users:legacy",
});
expect(tableMap.users.get("users:legacy")).toMatchObject({
personalPublisherId: createdPublisher?._id,
});
expect(insertCalls.some((call) => call.table === "publisherMembers")).toBe(true);
const skillsResult = await repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "skills",
dryRun: false,
batchSize: 10,
scheduleNext: false,
});
expect(skillsResult).toMatchObject({
phase: "skills",
dryRun: false,
scanned: 2,
repaired: 1,
skipped: 1,
isDone: true,
});
expect(tableMap.skills.get("skills:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
expect(tableMap.skills.get("skills:deleted-owner")).toMatchObject({
ownerPublisherId: undefined,
});
expect(tableMap.skillSlugAliases.get("skillSlugAliases:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
expect(tableMap.skillEmbeddings.get("skillEmbeddings:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
const packagesResult = await repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "packages",
dryRun: false,
batchSize: 10,
scheduleNext: false,
});
expect(packagesResult).toMatchObject({
phase: "packages",
dryRun: false,
scanned: 1,
repaired: 1,
skipped: 0,
isDone: true,
});
expect(tableMap.packages.get("packages:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
expect(patchCalls.some((call) => call.id === "skillSearchDigest:legacy")).toBe(false);
expect(patchCalls.some((call) => call.id === "packageSearchDigest:legacy")).toBe(false);
expect(
patchCalls.some(
(call) =>
call.id === createdPublisher?._id &&
("publishedSkills" in call.patch || "publishedPackages" in call.patch),
),
).toBe(false);
});
it("repairs legacy owner projections for one targeted user by handle", async () => {
const { db, tableMap } = makeLegacyPublisherOwnershipDb();
const scheduler = { runAfter: vi.fn() };
await repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "users",
dryRun: false,
batchSize: 10,
scheduleNext: false,
});
const createdPublisher = Array.from(tableMap.publishers.values()).find(
(publisher) => publisher.handle === "legacy-owner",
);
const skillsResult = await repairLegacyPublisherOwnershipForUserHandler(
{ db, scheduler } as never,
{
handle: "legacy-owner",
phase: "skills",
dryRun: false,
batchSize: 10,
scheduleNext: false,
},
);
expect(skillsResult).toMatchObject({
phase: "skills",
dryRun: false,
userId: "users:legacy",
publisherId: createdPublisher?._id,
scanned: 1,
repaired: 1,
skipped: 0,
isDone: true,
});
expect(tableMap.skills.get("skills:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
expect(tableMap.skills.get("skills:deleted-owner")).toMatchObject({
ownerPublisherId: undefined,
});
expect(tableMap.skillSlugAliases.get("skillSlugAliases:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
expect(tableMap.skillEmbeddings.get("skillEmbeddings:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
const packagesResult = await repairLegacyPublisherOwnershipForUserHandler(
{ db, scheduler } as never,
{
handle: "legacy-owner",
phase: "packages",
dryRun: false,
batchSize: 10,
scheduleNext: false,
},
);
expect(packagesResult).toMatchObject({
phase: "packages",
dryRun: false,
userId: "users:legacy",
publisherId: createdPublisher?._id,
scanned: 1,
repaired: 1,
skipped: 0,
isDone: true,
});
expect(tableMap.packages.get("packages:legacy")).toMatchObject({
ownerPublisherId: createdPublisher?._id,
});
});
it("aborts apply-mode skill repair when owner projection sync fails", async () => {
const { db } = makeLegacyPublisherOwnershipDb();
const scheduler = { runAfter: vi.fn() };
await repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "users",
dryRun: false,
batchSize: 10,
scheduleNext: false,
});
const patch = db.patch;
db.patch = vi.fn(async (id: string, value: Record<string, unknown>) => {
if (id === "skillEmbeddings:legacy") throw new Error("embedding sync failed");
await patch(id, value);
});
await expect(
repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "skills",
dryRun: false,
batchSize: 10,
scheduleNext: false,
}),
).rejects.toThrow("embedding sync failed");
});
it("propagates apply-mode package patch failures", async () => {
const { db } = makeLegacyPublisherOwnershipDb();
const scheduler = { runAfter: vi.fn() };
await repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "users",
dryRun: false,
batchSize: 10,
scheduleNext: false,
});
const patch = db.patch;
db.patch = vi.fn(async (id: string, value: Record<string, unknown>) => {
if (id === "packages:legacy") throw new Error("package patch failed");
await patch(id, value);
});
await expect(
repairLegacyPublisherOwnershipHandler({ db, scheduler } as never, {
phase: "packages",
dryRun: false,
batchSize: 10,
scheduleNext: false,
}),
).rejects.toThrow("package patch failed");
});
});
describe("maintenance backfill", () => {
it("patches stale skill search digest rank stats from legacy skill stats", async () => {
const existingDigest = {
_id: "skillSearchDigest:1",
skillId: "skills:1",
slug: "demo",
displayName: "Demo",
summary: "Old summary",
ownerUserId: "users:owner",
tags: {},
stats: {
downloads: 3,
stars: 2,
installsCurrent: 4,
installsAllTime: 5,
versions: 1,
comments: 0,
},
softDeletedAt: undefined,
createdAt: 100,
updatedAt: 200,
};
const skill = {
_id: "skills:1",
slug: "demo",
displayName: "Demo",
summary: "New summary",
ownerUserId: "users:owner",
tags: {},
stats: {
downloads: 42,
stars: 7,
installsCurrent: 9,
installsAllTime: 100,
versions: 1,
comments: 0,
},
softDeletedAt: undefined,
createdAt: 100,
updatedAt: 300,
};
const paginate = vi.fn().mockResolvedValue({
page: [skill],
continueCursor: null,
isDone: true,
});
const unique = vi.fn().mockResolvedValue(existingDigest);
class TestEqBuilder {
eq(_field: string, _value: unknown) {
return this;
}
}
const withIndex = vi.fn((_indexName: string, build: (q: TestEqBuilder) => unknown) => {
build(new TestEqBuilder());
return { unique };
});
const query = vi.fn((table: string) => {
if (table === "skills") return { paginate };
if (table === "skillSearchDigest") return { withIndex };
throw new Error(`unexpected table ${table}`);
});
const patch = vi.fn().mockResolvedValue(undefined);
const insert = vi.fn().mockResolvedValue("skillSearchDigest:inserted");
const replace = vi.fn().mockResolvedValue(undefined);
const deleteDoc = vi.fn().mockResolvedValue(undefined);
const result = await (
backfillSkillSearchDigestInternal as unknown as { _handler: Function }
)._handler(
{
db: {
get: vi.fn(),
query,
patch,
insert,
replace,
delete: deleteDoc,
normalizeId: vi.fn(),
},
scheduler: {
runAfter: vi.fn(),
},
} as never,
{ batchSize: 10 },
);
expect(result).toEqual({ upserted: 1, isDone: true, scanned: 1 });
expect(paginate).toHaveBeenCalledWith({ cursor: null, numItems: 10 });
expect(withIndex).toHaveBeenCalledWith("by_skill", expect.any(Function));
expect(insert).not.toHaveBeenCalled();
expect(patch).toHaveBeenCalledWith(
"skillSearchDigest:1",
expect.objectContaining({
summary: "New summary",
statsDownloads: 42,
statsStars: 7,
statsInstallsCurrent: 9,
statsInstallsAllTime: 100,
stats: expect.objectContaining({
downloads: 42,
stars: 7,
installsCurrent: 9,
installsAllTime: 100,
}),
}),
);
});
it("repairs summary + parsed by reparsing SKILL.md", async () => {
const runQuery = vi.fn().mockResolvedValue({
items: [
@@ -290,6 +1064,14 @@ describe("maintenance backfill", () => {
_id: "skills:1",
slug: "demo",
displayName: "Demo",
stats: {
downloads: 0,
stars: 0,
installsCurrent: 0,
installsAllTime: 0,
versions: 1,
comments: 0,
},
latestVersionId: "skillVersions:1",
latestVersionSummary: digest.latestVersionSummary,
capabilityTags: ["read-files"],
@@ -398,6 +1180,54 @@ describe("maintenance backfill", () => {
},
);
});
it("backfills denormalized publisher stats through the recompute mutation", async () => {
const runQuery = vi.fn().mockResolvedValue({
items: [{ _id: "publishers:1" }, { _id: "publishers:2" }],
cursor: "next",
isDone: false,
});
const runMutation = vi.fn().mockResolvedValue({ ok: true });
const result = await backfillPublisherStatsInternalHandler({ runQuery, runMutation } as never, {
dryRun: true,
batchSize: 2,
maxBatches: 1,
});
expect(result).toEqual({
ok: true,
stats: {
publishersScanned: 2,
publishersPatched: 0,
},
isDone: false,
cursor: "next",
});
expect(runQuery).toHaveBeenCalledWith(
internal.maintenance.getPublisherStatsBackfillPageInternal,
{
cursor: undefined,
batchSize: 2,
},
);
expect(runMutation).toHaveBeenNthCalledWith(
1,
internal.maintenance.recomputePublisherStatsInternal,
{
publisherId: "publishers:1",
dryRun: true,
},
);
expect(runMutation).toHaveBeenNthCalledWith(
2,
internal.maintenance.recomputePublisherStatsInternal,
{
publisherId: "publishers:2",
dryRun: true,
},
);
});
});
describe("maintenance badge denormalization", () => {
+597 -11
View File
@@ -1,10 +1,20 @@
import { ConvexError, v } from "convex/values";
import { internal } from "./_generated/api";
import type { Doc, Id } from "./_generated/dataModel";
import type { ActionCtx } from "./_generated/server";
import type { ActionCtx, MutationCtx } from "./_generated/server";
import { action, internalAction, internalMutation, internalQuery } from "./functions";
import { assertRole, requireUserFromAction } from "./lib/access";
import { extractPackageDigestFields, upsertPackageSearchDigest } from "./lib/packageSearchDigest";
import {
derivePersonalPublisherHandle,
ensurePersonalPublisherForUser,
getPersonalPublisherForUser,
getPublisherByHandle,
getPublisherMembership,
getUserByHandleOrPersonalPublisher,
isPublisherActive,
} from "./lib/publishers";
import { recomputePublisherStats } from "./lib/publisherStats";
import { buildSkillSummaryBackfillPatch, type ParsedSkillData } from "./lib/skillBackfill";
import { deriveSkillCapabilityTags } from "./lib/skillCapabilityTags";
import { isSkillCardPath } from "./lib/skillCards";
@@ -20,6 +30,7 @@ import {
extractValidatedDigestFields,
getFirstSearchToken,
normalizeSkillSearchText,
upsertSkillSearchDigest,
} from "./lib/skillSearchDigest";
import { generateSkillSummary } from "./lib/skillSummary";
@@ -47,6 +58,11 @@ type UserStatsBackfillStats = {
usersPatched: number;
};
type PublisherStatsBackfillStats = {
publishersScanned: number;
publishersPatched: number;
};
type BackfillPageItem =
| {
kind: "ok";
@@ -74,12 +90,44 @@ type UserStatsBackfillPageResult = {
isDone: boolean;
};
type PublisherStatsBackfillPageResult = {
items: Array<Pick<Doc<"publishers">, "_id">>;
cursor: string | null;
isDone: boolean;
};
type UserOwnedSkillsBackfillPageResult = {
items: Array<Pick<Doc<"skills">, "stats" | "softDeletedAt">>;
cursor: string | null;
isDone: boolean;
};
type LegacyPublisherOwnershipPhase = "users" | "skills" | "packages";
type LegacyPublisherOwnershipTargetPhase = Exclude<LegacyPublisherOwnershipPhase, "users">;
type LegacyPublisherOwnershipRepairResult = {
phase: LegacyPublisherOwnershipPhase;
dryRun: boolean;
scanned: number;
repaired: number;
skipped: number;
errors: string[];
cursor: string | null;
isDone: boolean;
nextPhase?: LegacyPublisherOwnershipPhase;
};
type LegacyPublisherOwnershipForUserRepairResult = Omit<
LegacyPublisherOwnershipRepairResult,
"phase" | "nextPhase"
> & {
phase: LegacyPublisherOwnershipTargetPhase;
userId: Id<"users">;
handle?: string;
publisherId: Id<"publishers"> | null;
nextPhase?: LegacyPublisherOwnershipTargetPhase;
};
export const getSkillBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
@@ -178,6 +226,25 @@ export const getUserStatsBackfillPageInternal = internalQuery({
},
});
export const getPublisherStatsBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args): Promise<PublisherStatsBackfillPageResult> => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
const { page, isDone, continueCursor } = await ctx.db
.query("publishers")
.order("asc")
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
return {
items: page.map((publisher) => ({ _id: publisher._id })),
cursor: continueCursor,
isDone,
};
},
});
export const getUserOwnedSkillsBackfillPageInternal = internalQuery({
args: {
ownerUserId: v.id("users"),
@@ -218,6 +285,20 @@ export const applyUserStatsBackfillPatchInternal = internalMutation({
},
});
export const recomputePublisherStatsInternal = internalMutation({
args: {
publisherId: v.id("publishers"),
dryRun: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const stats = await recomputePublisherStats(ctx, args.publisherId);
if (!args.dryRun) {
await ctx.db.patch(args.publisherId, stats);
}
return { ok: true as const, stats };
},
});
export type BackfillActionArgs = {
dryRun?: boolean;
batchSize?: number;
@@ -247,6 +328,20 @@ export type UserStatsBackfillActionResult = {
cursor: string | null;
};
export type PublisherStatsBackfillActionArgs = {
dryRun?: boolean;
batchSize?: number;
maxBatches?: number;
cursor?: string;
};
export type PublisherStatsBackfillActionResult = {
ok: true;
stats: PublisherStatsBackfillStats;
isDone: boolean;
cursor: string | null;
};
export async function backfillSkillSummariesInternalHandler(
ctx: ActionCtx,
args: BackfillActionArgs,
@@ -409,6 +504,45 @@ export async function backfillUserStatsInternalHandler(
return { ok: true as const, stats: totals, isDone, cursor };
}
export async function backfillPublisherStatsInternalHandler(
ctx: ActionCtx,
args: PublisherStatsBackfillActionArgs,
): Promise<PublisherStatsBackfillActionResult> {
const dryRun = Boolean(args.dryRun);
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES);
const totals: PublisherStatsBackfillStats = {
publishersScanned: 0,
publishersPatched: 0,
};
let cursor: string | null = args.cursor ?? null;
let isDone = false;
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(internal.maintenance.getPublisherStatsBackfillPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as PublisherStatsBackfillPageResult;
cursor = page.cursor;
isDone = page.isDone;
for (const publisher of page.items) {
totals.publishersScanned++;
await ctx.runMutation(internal.maintenance.recomputePublisherStatsInternal, {
publisherId: publisher._id,
dryRun,
});
if (!dryRun) totals.publishersPatched++;
}
if (isDone) break;
}
return { ok: true as const, stats: totals, isDone, cursor };
}
export const backfillSkillSummariesInternal = internalAction({
args: {
dryRun: v.optional(v.boolean()),
@@ -430,6 +564,16 @@ export const backfillUserStatsInternal = internalAction({
handler: backfillUserStatsInternalHandler,
});
export const backfillPublisherStatsInternal = internalAction({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
cursor: v.optional(v.string()),
},
handler: backfillPublisherStatsInternalHandler,
});
export const backfillSkillSummaries: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
@@ -448,6 +592,37 @@ export const backfillSkillSummaries: ReturnType<typeof action> = action({
},
});
export const backfillPublisherStats: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
cursor: v.optional(v.string()),
},
handler: async (ctx, args): Promise<PublisherStatsBackfillActionResult> => {
const { user } = await requireUserFromAction(ctx);
assertRole(user, ["admin"]);
return ctx.runAction(
internal.maintenance.backfillPublisherStatsInternal,
args,
) as Promise<PublisherStatsBackfillActionResult>;
},
});
export const scheduleBackfillPublisherStats: ReturnType<typeof action> = action({
args: { dryRun: v.optional(v.boolean()) },
handler: async (ctx, args) => {
const { user } = await requireUserFromAction(ctx);
assertRole(user, ["admin"]);
await ctx.scheduler.runAfter(0, internal.maintenance.backfillPublisherStatsInternal, {
dryRun: Boolean(args.dryRun),
batchSize: DEFAULT_BATCH_SIZE,
maxBatches: DEFAULT_MAX_BATCHES,
});
return { ok: true as const };
},
});
export const scheduleBackfillSkillSummaries: ReturnType<typeof action> = action({
args: { dryRun: v.optional(v.boolean()), useAi: v.optional(v.boolean()) },
handler: async (ctx, args) => {
@@ -2083,16 +2258,10 @@ export const backfillSkillSearchDigestInternal = internalMutation({
.query("skills")
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
let inserted = 0;
let upserted = 0;
for (const skill of page) {
const existing = await ctx.db
.query("skillSearchDigest")
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
.unique();
if (!existing) {
await ctx.db.insert("skillSearchDigest", await extractValidatedDigestFields(ctx, skill));
inserted++;
}
await upsertSkillSearchDigest(ctx, await extractValidatedDigestFields(ctx, skill));
upserted++;
}
if (!isDone) {
@@ -2102,7 +2271,7 @@ export const backfillSkillSearchDigestInternal = internalMutation({
});
}
return { inserted, isDone, scanned: page.length };
return { upserted, isDone, scanned: page.length };
},
});
@@ -2142,6 +2311,423 @@ export const backfillPackagePluginCategoryDigestsInternal = internalMutation({
},
});
function isActiveLegacyPublisherRepairUser(
user: Doc<"users"> | null | undefined,
): user is Doc<"users"> {
return Boolean(user && !user.deletedAt && !user.deactivatedAt && !user.purgedAt);
}
function nextLegacyPublisherOwnershipPhase(
phase: LegacyPublisherOwnershipPhase,
): LegacyPublisherOwnershipPhase | undefined {
if (phase === "users") return "skills";
if (phase === "skills") return "packages";
return undefined;
}
function nextLegacyPublisherOwnershipTargetPhase(
phase: LegacyPublisherOwnershipTargetPhase,
): LegacyPublisherOwnershipTargetPhase | undefined {
return phase === "skills" ? "packages" : undefined;
}
async function getExistingActivePersonalPublisher(
ctx: Pick<MutationCtx, "db">,
user: Doc<"users">,
) {
if (user.personalPublisherId) {
const publisher = await ctx.db.get(user.personalPublisherId);
if (isPublisherActive(publisher)) return publisher;
}
const publisher = await getPersonalPublisherForUser(ctx, user._id);
return isPublisherActive(publisher) ? publisher : null;
}
async function needsPersonalPublisherRepair(ctx: Pick<MutationCtx, "db">, user: Doc<"users">) {
const publisher = await getExistingActivePersonalPublisher(ctx, user);
if (!publisher) return true;
if (user.personalPublisherId !== publisher._id) return true;
if (publisher.kind !== "user" || publisher.linkedUserId !== user._id) return true;
const member = await getPublisherMembership(ctx, publisher._id, user._id);
return !member;
}
function pushRepairError(errors: string[], label: string, error: unknown) {
if (errors.length >= 10) return;
const message = error instanceof Error ? error.message : String(error);
errors.push(`${label}: ${message}`);
}
function isPublisherHandleConflictError(error: unknown) {
const message = error instanceof Error ? error.message : String(error);
return /Publisher handle "@[^"]+" is already claimed/.test(message);
}
async function resolvePersonalPublisherForOwnershipRepair(
ctx: Pick<MutationCtx, "db">,
user: Doc<"users">,
dryRun: boolean,
) {
if (dryRun) {
const existing = await getExistingActivePersonalPublisher(ctx, user);
if (existing) return existing;
const handle = derivePersonalPublisherHandle(user);
const conflict = await getPublisherByHandle(ctx, handle);
if (conflict && conflict.linkedUserId !== user._id) {
throw new ConvexError(`Publisher handle "@${handle}" is already claimed`);
}
return null;
}
return await ensurePersonalPublisherForUser(ctx, user, {
source: "maintenance.legacy_publisher_ownership",
});
}
async function repairLegacySkillOwnerPublisher(
ctx: Pick<MutationCtx, "db">,
skill: Doc<"skills">,
dryRun: boolean,
) {
if (skill.ownerPublisherId) return "skipped" as const;
const owner = await ctx.db.get(skill.ownerUserId);
if (!isActiveLegacyPublisherRepairUser(owner)) return "skipped" as const;
const publisher = await resolvePersonalPublisherForOwnershipRepair(ctx, owner, dryRun);
if (!dryRun && (!publisher || !isPublisherActive(publisher))) return "skipped" as const;
if (dryRun) return "repaired" as const;
// The trigger-wrapped mutation syncs skill search digest and publisher stats.
// This repair only patches owner projections that triggers do not own.
await ctx.db.patch(skill._id, { ownerPublisherId: publisher!._id });
const aliases = await ctx.db
.query("skillSlugAliases")
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
.collect();
for (const alias of aliases) {
if (alias.ownerPublisherId === publisher!._id) continue;
await ctx.db.patch(alias._id, { ownerPublisherId: publisher!._id });
}
const embeddings = await ctx.db
.query("skillEmbeddings")
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
.collect();
for (const embedding of embeddings) {
if (embedding.ownerPublisherId === publisher!._id) continue;
await ctx.db.patch(embedding._id, { ownerPublisherId: publisher!._id });
}
return "repaired" as const;
}
async function repairLegacyPackageOwnerPublisher(
ctx: Pick<MutationCtx, "db">,
pkg: Doc<"packages">,
dryRun: boolean,
) {
if (pkg.ownerPublisherId) return "skipped" as const;
const owner = await ctx.db.get(pkg.ownerUserId);
if (!isActiveLegacyPublisherRepairUser(owner)) return "skipped" as const;
const publisher = await resolvePersonalPublisherForOwnershipRepair(ctx, owner, dryRun);
if (!dryRun && (!publisher || !isPublisherActive(publisher))) return "skipped" as const;
if (dryRun) return "repaired" as const;
// The trigger-wrapped mutation syncs package search digests and publisher stats.
await ctx.db.patch(pkg._id, { ownerPublisherId: publisher!._id });
return "repaired" as const;
}
async function resolveLegacyPublisherOwnershipTargetUser(
ctx: Pick<MutationCtx, "db">,
args: { userId?: Id<"users">; handle?: string },
) {
const user = args.userId
? await ctx.db.get(args.userId)
: await getUserByHandleOrPersonalPublisher(ctx, args.handle);
if (!user) throw new ConvexError("Target user not found");
if (!isActiveLegacyPublisherRepairUser(user)) throw new ConvexError("Target user is inactive");
return user;
}
async function patchLegacySkillOwnerPublisher(
ctx: Pick<MutationCtx, "db">,
skill: Doc<"skills">,
publisherId: Id<"publishers">,
) {
await ctx.db.patch(skill._id, { ownerPublisherId: publisherId });
const aliases = await ctx.db
.query("skillSlugAliases")
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
.collect();
for (const alias of aliases) {
if (alias.ownerPublisherId === publisherId) continue;
await ctx.db.patch(alias._id, { ownerPublisherId: publisherId });
}
const embeddings = await ctx.db
.query("skillEmbeddings")
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
.collect();
for (const embedding of embeddings) {
if (embedding.ownerPublisherId === publisherId) continue;
await ctx.db.patch(embedding._id, { ownerPublisherId: publisherId });
}
}
async function patchLegacyPackageOwnerPublisher(
ctx: Pick<MutationCtx, "db">,
pkg: Doc<"packages">,
publisherId: Id<"publishers">,
) {
await ctx.db.patch(pkg._id, { ownerPublisherId: publisherId });
}
export async function repairLegacyPublisherOwnershipHandler(
ctx: MutationCtx,
args: {
phase?: LegacyPublisherOwnershipPhase;
cursor?: string;
batchSize?: number;
delayMs?: number;
dryRun?: boolean;
scheduleNext?: boolean;
},
): Promise<LegacyPublisherOwnershipRepairResult> {
const phase = args.phase ?? "users";
const dryRun = args.dryRun === true;
const batchSize = clampInt(args.batchSize ?? 50, 1, 200);
const delayMs = clampInt(args.delayMs ?? 500, 0, 60_000);
const errors: string[] = [];
let scanned = 0;
let repaired = 0;
let skipped = 0;
let continueCursor: string | null = null;
let isDone = true;
if (phase === "users") {
const page = await ctx.db
.query("users")
.withIndex("by_active_handle", (q) =>
q.eq("deletedAt", undefined).eq("deactivatedAt", undefined),
)
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
continueCursor = page.continueCursor;
isDone = page.isDone;
for (const user of page.page) {
scanned++;
if (!isActiveLegacyPublisherRepairUser(user)) {
skipped++;
continue;
}
try {
if (!(await needsPersonalPublisherRepair(ctx, user))) continue;
if (dryRun) {
await resolvePersonalPublisherForOwnershipRepair(ctx, user, true);
} else {
await ensurePersonalPublisherForUser(ctx, user, {
source: "maintenance.legacy_publisher_ownership",
});
}
repaired++;
} catch (error) {
if (!dryRun && !isPublisherHandleConflictError(error)) throw error;
skipped++;
pushRepairError(errors, `user:${user._id}`, error);
}
}
} else if (phase === "skills") {
const page = await ctx.db
.query("skills")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", undefined))
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
continueCursor = page.continueCursor;
isDone = page.isDone;
for (const skill of page.page) {
scanned++;
try {
const result = await repairLegacySkillOwnerPublisher(ctx, skill, dryRun);
if (result === "repaired") repaired++;
else skipped++;
} catch (error) {
if (!dryRun && !isPublisherHandleConflictError(error)) throw error;
skipped++;
pushRepairError(errors, `skill:${skill._id}`, error);
}
}
} else {
const page = await ctx.db
.query("packages")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", undefined))
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
continueCursor = page.continueCursor;
isDone = page.isDone;
for (const pkg of page.page) {
scanned++;
try {
const result = await repairLegacyPackageOwnerPublisher(ctx, pkg, dryRun);
if (result === "repaired") repaired++;
else skipped++;
} catch (error) {
if (!dryRun && !isPublisherHandleConflictError(error)) throw error;
skipped++;
pushRepairError(errors, `package:${pkg._id}`, error);
}
}
}
const nextPhase = isDone ? nextLegacyPublisherOwnershipPhase(phase) : phase;
if (!dryRun && args.scheduleNext !== false && nextPhase) {
await ctx.scheduler.runAfter(delayMs, internal.maintenance.repairLegacyPublisherOwnership, {
phase: nextPhase,
cursor: isDone ? undefined : (continueCursor ?? undefined),
batchSize: args.batchSize,
delayMs: args.delayMs,
scheduleNext: args.scheduleNext,
});
}
return {
phase,
dryRun,
scanned,
repaired,
skipped,
errors,
cursor: continueCursor,
isDone,
...(nextPhase ? { nextPhase } : {}),
};
}
export async function repairLegacyPublisherOwnershipForUserHandler(
ctx: MutationCtx,
args: {
userId?: Id<"users">;
handle?: string;
phase?: LegacyPublisherOwnershipTargetPhase;
cursor?: string;
batchSize?: number;
delayMs?: number;
dryRun?: boolean;
scheduleNext?: boolean;
},
): Promise<LegacyPublisherOwnershipForUserRepairResult> {
const phase = args.phase ?? "skills";
const dryRun = args.dryRun === true;
const batchSize = clampInt(args.batchSize ?? 50, 1, 200);
const delayMs = clampInt(args.delayMs ?? 500, 0, 60_000);
const user = await resolveLegacyPublisherOwnershipTargetUser(ctx, args);
const publisher = await resolvePersonalPublisherForOwnershipRepair(ctx, user, dryRun);
if (!dryRun && !isPublisherActive(publisher)) {
throw new ConvexError("Target personal publisher could not be repaired");
}
let scanned = 0;
let repaired = 0;
let skipped = 0;
const page =
phase === "skills"
? await ctx.db
.query("skills")
.withIndex("by_owner", (q) => q.eq("ownerUserId", user._id))
.paginate({ cursor: args.cursor ?? null, numItems: batchSize })
: await ctx.db
.query("packages")
.withIndex("by_owner", (q) => q.eq("ownerUserId", user._id))
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
for (const item of page.page) {
scanned++;
if (item.ownerPublisherId) {
skipped++;
continue;
}
if (dryRun) {
repaired++;
continue;
}
if (phase === "skills") {
await patchLegacySkillOwnerPublisher(ctx, item as Doc<"skills">, publisher!._id);
} else {
await patchLegacyPackageOwnerPublisher(ctx, item as Doc<"packages">, publisher!._id);
}
repaired++;
}
const nextPhase = page.isDone ? nextLegacyPublisherOwnershipTargetPhase(phase) : phase;
if (!dryRun && args.scheduleNext !== false && nextPhase) {
await ctx.scheduler.runAfter(
delayMs,
internal.maintenance.repairLegacyPublisherOwnershipForUser,
{
userId: user._id,
phase: nextPhase,
cursor: page.isDone ? undefined : (page.continueCursor ?? undefined),
batchSize: args.batchSize,
delayMs: args.delayMs,
scheduleNext: args.scheduleNext,
},
);
}
return {
phase,
dryRun,
userId: user._id,
handle: user.handle,
publisherId: publisher?._id ?? null,
scanned,
repaired,
skipped,
errors: [],
cursor: page.continueCursor,
isDone: page.isDone,
...(nextPhase ? { nextPhase } : {}),
};
}
// Repair legacy personal publisher ownership after the publisher model rollout.
// Dry run one phase:
// npx convex run maintenance:repairLegacyPublisherOwnership '{"phase":"skills","dryRun":true,"scheduleNext":false}' --prod
// Apply all phases, scheduled batch-by-batch:
// npx convex run maintenance:repairLegacyPublisherOwnership '{"phase":"users","batchSize":50}' --prod
export const repairLegacyPublisherOwnership = internalMutation({
args: {
phase: v.optional(v.union(v.literal("users"), v.literal("skills"), v.literal("packages"))),
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
delayMs: v.optional(v.number()),
dryRun: v.optional(v.boolean()),
scheduleNext: v.optional(v.boolean()),
},
handler: repairLegacyPublisherOwnershipHandler,
});
// Targeted variant for production canaries and one-off account repair.
// Example:
// npx convex run maintenance:repairLegacyPublisherOwnershipForUser '{"handle":"harrylabsj","dryRun":true,"scheduleNext":false}' --prod
export const repairLegacyPublisherOwnershipForUser = internalMutation({
args: {
userId: v.optional(v.id("users")),
handle: v.optional(v.string()),
phase: v.optional(v.union(v.literal("skills"), v.literal("packages"))),
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
delayMs: v.optional(v.number()),
dryRun: v.optional(v.boolean()),
scheduleNext: v.optional(v.boolean()),
},
handler: repairLegacyPublisherOwnershipForUserHandler,
});
const DIGEST_OWNER_BACKFILL_KEY = "digest-owner-backfill";
// Start/resume backfill:
+360
View File
@@ -0,0 +1,360 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from "vitest";
vi.mock("./_generated/server", () => ({
internalMutation: (def: { handler: unknown }) => ({ _handler: def.handler }),
}));
const managementDevSeed = await import("./managementDevSeed");
type Handler<TArgs, TResult> = (ctx: unknown, args: TArgs) => Promise<TResult>;
type Wrapped<TArgs, TResult> = { _handler: Handler<TArgs, TResult> };
type TestDoc = Record<string, unknown> & { _id: string };
const seedManagementQueuesHandler = (
managementDevSeed.seedManagementQueues as unknown as Wrapped<
Record<string, never>,
{ reportsInserted: number; reportedSkills: number; duplicatePair: number }
>
)._handler;
const clearManagementQueuesHandler = (
managementDevSeed.clearManagementQueues as unknown as Wrapped<
Record<string, never>,
{ reportsDeleted: number; fingerprintsDeleted: number }
>
)._handler;
const DEMO_REPORT_MARKER = "managementDevSeed:report";
const DEMO_FINGERPRINT = "9f8c2a1b7e4d6c30a5b2f1d089c4e76b";
function chainEq(constraints: Record<string, unknown>) {
return {
eq(field: string, value: unknown) {
constraints[field] = value;
return chainEq(constraints);
},
};
}
function matches(doc: TestDoc, constraints: Record<string, unknown>) {
return Object.entries(constraints).every(([key, value]) => doc[key] === value);
}
function createDb(seedTables: Record<string, TestDoc[]>) {
const tables = Object.fromEntries(
Object.entries(seedTables).map(([name, docs]) => [name, docs.map((doc) => ({ ...doc }))]),
);
const queryCalls: Array<{
table: string;
indexName: string;
constraints: Record<string, unknown>;
}> = [];
const inserts: Array<{ table: string; doc: TestDoc }> = [];
let insertCounter = 0;
const list = (table: string) => {
tables[table] ??= [];
return tables[table];
};
const takeRows = (table: string, numItems: number, constraints?: Record<string, unknown>) => {
const rows = constraints ? list(table).filter((doc) => matches(doc, constraints)) : list(table);
return rows.slice(0, numItems);
};
return {
inserts,
queryCalls,
tables,
db: {
delete: async (id: string) => {
const table = id.split(":")[0] ?? "";
const rows = list(table);
const index = rows.findIndex((doc) => doc._id === id);
if (index !== -1) rows.splice(index, 1);
},
get: async (id: string) => {
const table = id.split(":")[0] ?? "";
return list(table).find((doc) => doc._id === id) ?? null;
},
insert: async (table: string, doc: Record<string, unknown>) => {
const inserted = { ...doc, _id: `${table}:inserted-${insertCounter}` };
insertCounter += 1;
list(table).push(inserted);
inserts.push({ table, doc: inserted });
return inserted._id;
},
patch: async (id: string, patch: Record<string, unknown>) => {
const table = id.split(":")[0] ?? "";
const row = list(table).find((doc) => doc._id === id);
if (row) Object.assign(row, patch);
},
query: (table: string) => ({
order: () => ({
take: async (numItems: number) => takeRows(table, numItems),
}),
take: async (numItems: number) => takeRows(table, numItems),
withIndex: (indexName: string, build: (q: ReturnType<typeof chainEq>) => unknown) => {
const constraints: Record<string, unknown> = {};
build(chainEq(constraints));
queryCalls.push({ table, indexName, constraints });
return {
order: () => ({
take: async (numItems: number) => takeRows(table, numItems, constraints),
}),
take: async (numItems: number) => takeRows(table, numItems, constraints),
};
},
}),
},
};
}
function restoreEnv(name: string, value: string | undefined) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
}
describe("managementDevSeed", () => {
const previousDeployment = process.env.CONVEX_DEPLOYMENT;
const previousDevAuthDeployment = process.env.DEV_AUTH_CONVEX_DEPLOYMENT;
const previousDevAuthEnabled = process.env.DEV_AUTH_ENABLED;
const previousDevImpersonation = process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
afterEach(() => {
restoreEnv("CONVEX_DEPLOYMENT", previousDeployment);
restoreEnv("DEV_AUTH_CONVEX_DEPLOYMENT", previousDevAuthDeployment);
restoreEnv("DEV_AUTH_ENABLED", previousDevAuthEnabled);
restoreEnv("CLAW_HUB_ENABLE_DEV_IMPERSONATION", previousDevImpersonation);
});
it("rejects production deployments before reading tables", async () => {
process.env.CONVEX_DEPLOYMENT = "prod:wry-manatee-359";
const query = vi.fn();
await expect(seedManagementQueuesHandler({ db: { query } }, {})).rejects.toThrow(
"disabled outside local/dev deployments",
);
await expect(clearManagementQueuesHandler({ db: { query } }, {})).rejects.toThrow(
"disabled outside local/dev deployments",
);
expect(query).not.toHaveBeenCalled();
});
it("honors the explicit fallback deployment when the primary marker is blank", async () => {
process.env.CONVEX_DEPLOYMENT = "";
process.env.DEV_AUTH_CONVEX_DEPLOYMENT = "prod:wry-manatee-359";
process.env.DEV_AUTH_ENABLED = "1";
const query = vi.fn();
await expect(seedManagementQueuesHandler({ db: { query } }, {})).rejects.toThrow(
"disabled outside local/dev deployments",
);
await expect(clearManagementQueuesHandler({ db: { query } }, {})).rejects.toThrow(
"disabled outside local/dev deployments",
);
expect(query).not.toHaveBeenCalled();
});
it("seeds content report and duplicate candidate rows for local dashboards", async () => {
process.env.CONVEX_DEPLOYMENT = "";
process.env.DEV_AUTH_CONVEX_DEPLOYMENT = "dev:admired-dodo-615";
const { db, inserts, tables } = createDb({
users: [{ _id: "users:reporter", handle: "local-admin" }],
skills: [
{ _id: "skills:one", latestVersionId: "skillVersions:one" },
{ _id: "skills:two", latestVersionId: "skillVersions:two" },
{ _id: "skills:three", latestVersionId: "skillVersions:three" },
{ _id: "skills:hidden", latestVersionId: "skillVersions:hidden", softDeletedAt: 1 },
],
skillVersions: [
{ _id: "skillVersions:one" },
{ _id: "skillVersions:two" },
{ _id: "skillVersions:three" },
{ _id: "skillVersions:hidden" },
],
skillReports: [],
skillVersionFingerprints: [],
});
await expect(seedManagementQueuesHandler({ db }, {})).resolves.toEqual({
reportsInserted: 6,
reportedSkills: 3,
duplicatePair: 2,
});
expect(tables.skillReports).toHaveLength(6);
expect(tables.skillReports.every((report) => report.triageNote === DEMO_REPORT_MARKER)).toBe(
true,
);
expect(tables.skills.find((skill) => skill._id === "skills:one")).toEqual(
expect.objectContaining({ reportCount: 1 }),
);
expect(tables.skills.find((skill) => skill._id === "skills:two")).toEqual(
expect.objectContaining({ reportCount: 2 }),
);
expect(tables.skills.find((skill) => skill._id === "skills:three")).toEqual(
expect.objectContaining({ reportCount: 3 }),
);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:one")).toEqual(
expect.objectContaining({ fingerprint: DEMO_FINGERPRINT }),
);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:two")).toEqual(
expect.objectContaining({ fingerprint: DEMO_FINGERPRINT }),
);
expect(inserts.filter((insert) => insert.table === "skillVersionFingerprints")).toHaveLength(2);
});
it("does not overwrite existing latest-version fingerprints", async () => {
process.env.CONVEX_DEPLOYMENT = "";
process.env.DEV_AUTH_CONVEX_DEPLOYMENT = "dev:admired-dodo-615";
const { db, inserts, tables } = createDb({
users: [{ _id: "users:reporter", handle: "local-admin" }],
skills: [
{ _id: "skills:one", latestVersionId: "skillVersions:one" },
{ _id: "skills:two", latestVersionId: "skillVersions:two" },
{ _id: "skills:three", latestVersionId: "skillVersions:three" },
{ _id: "skills:four", latestVersionId: "skillVersions:four" },
],
skillVersions: [
{ _id: "skillVersions:one", fingerprint: "real-fingerprint-one" },
{ _id: "skillVersions:two", fingerprint: "real-fingerprint-two" },
{ _id: "skillVersions:three" },
{ _id: "skillVersions:four" },
],
skillReports: [],
skillVersionFingerprints: [
{
_id: "skillVersionFingerprints:real-one",
skillId: "skills:one",
versionId: "skillVersions:one",
fingerprint: "real-fingerprint-one",
},
{
_id: "skillVersionFingerprints:real-two",
skillId: "skills:two",
versionId: "skillVersions:two",
fingerprint: "real-fingerprint-two",
},
],
});
await expect(seedManagementQueuesHandler({ db }, {})).resolves.toEqual({
reportsInserted: 6,
reportedSkills: 3,
duplicatePair: 2,
});
expect(tables.skillVersions.find((version) => version._id === "skillVersions:one")).toEqual(
expect.objectContaining({ fingerprint: "real-fingerprint-one" }),
);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:two")).toEqual(
expect.objectContaining({ fingerprint: "real-fingerprint-two" }),
);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:three")).toEqual(
expect.objectContaining({ fingerprint: DEMO_FINGERPRINT }),
);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:four")).toEqual(
expect.objectContaining({ fingerprint: DEMO_FINGERPRINT }),
);
expect(
inserts
.filter((insert) => insert.table === "skillVersionFingerprints")
.map((insert) => insert.doc.versionId),
).toEqual(["skillVersions:three", "skillVersions:four"]);
});
it("clears only marked demo management rows", async () => {
process.env.CONVEX_DEPLOYMENT = "dev:admired-dodo-615";
const { db, queryCalls, tables } = createDb({
skills: [
{
_id: "skills:demo",
reportCount: 2,
lastReportedAt: 200,
},
{
_id: "skills:real",
reportCount: 1,
lastReportedAt: 200,
},
],
skillReports: [
{
_id: "skillReports:demo",
skillId: "skills:demo",
triageNote: DEMO_REPORT_MARKER,
status: "open",
createdAt: 100,
},
{
_id: "skillReports:demo-real",
skillId: "skills:demo",
triageNote: "real-user-report",
status: "open",
createdAt: 200,
},
{
_id: "skillReports:real",
skillId: "skills:real",
triageNote: "real-user-report",
status: "open",
createdAt: 200,
},
],
skillVersions: [
{ _id: "skillVersions:demo", fingerprint: DEMO_FINGERPRINT },
{ _id: "skillVersions:real", fingerprint: "real-fingerprint" },
],
skillVersionFingerprints: [
{
_id: "skillVersionFingerprints:demo",
versionId: "skillVersions:demo",
fingerprint: DEMO_FINGERPRINT,
},
{
_id: "skillVersionFingerprints:real",
versionId: "skillVersions:real",
fingerprint: "real-fingerprint",
},
],
});
await expect(clearManagementQueuesHandler({ db }, {})).resolves.toEqual({
reportsDeleted: 1,
fingerprintsDeleted: 1,
});
expect(tables.skillReports.map((report) => report._id)).toEqual([
"skillReports:demo-real",
"skillReports:real",
]);
expect(tables.skillVersionFingerprints.map((fingerprint) => fingerprint._id)).toEqual([
"skillVersionFingerprints:real",
]);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:demo")).toEqual(
expect.objectContaining({ fingerprint: undefined }),
);
expect(tables.skillVersions.find((version) => version._id === "skillVersions:real")).toEqual(
expect.objectContaining({ fingerprint: "real-fingerprint" }),
);
expect(tables.skills.find((skill) => skill._id === "skills:demo")).toEqual(
expect.objectContaining({ reportCount: 1, lastReportedAt: 200 }),
);
expect(tables.skills.find((skill) => skill._id === "skills:real")).toEqual(
expect.objectContaining({ reportCount: 1, lastReportedAt: 200 }),
);
expect(queryCalls).toContainEqual({
table: "skillVersionFingerprints",
indexName: "by_fingerprint",
constraints: { fingerprint: DEMO_FINGERPRINT },
});
expect(queryCalls).toContainEqual({
table: "skillReports",
indexName: "by_skill_createdAt",
constraints: { skillId: "skills:demo" },
});
});
});
+181
View File
@@ -0,0 +1,181 @@
import type { Doc, Id } from "./_generated/dataModel";
import type { MutationCtx } from "./_generated/server";
// DEV-ONLY seed for the management Content-reports and Duplicate-candidates queues.
// Uses the un-wrapped mutation builder (not convex/functions.ts) so patching skills
// / versions and inserting report + fingerprint rows does NOT fire table triggers.
// It operates on existing seeded skills rather than creating new ones, so the base
// dev seed must have run first. All demo rows carry a marker so clearDemo can remove
// them precisely.
import { internalMutation } from "./_generated/server";
import { assertLocalDevSeedAllowed } from "./lib/devSeed";
const DEMO_REPORT_MARKER = "managementDevSeed:report";
// Hash-like so the dashboard's fingerprint chip reads like real data; still a
// fixed constant so clearDemo can find and remove the seeded rows.
const DEMO_FINGERPRINT = "9f8c2a1b7e4d6c30a5b2f1d089c4e76b";
const DEMO_REPORT_REASONS = [
"Possible prompt-injection hidden in the skill instructions.",
"Looks like a copy of another publisher's skill.",
"Requests credentials it does not appear to need.",
"Spammy catalog filler with no real functionality.",
];
const HOUR_MS = 60 * 60 * 1000;
const REPORT_SCAN_LIMIT = 500;
const SKILL_SCAN_LIMIT = 50;
type DuplicateDemoTarget = {
skill: Doc<"skills">;
versionId: Id<"skillVersions">;
};
// Remove previously seeded demo reports + duplicate fingerprints so the seed is
// idempotent and the dashboard can be reset.
async function clearDemo(ctx: Pick<MutationCtx, "db">): Promise<{
reportsDeleted: number;
fingerprintsDeleted: number;
}> {
let reportsDeleted = 0;
let fingerprintsDeleted = 0;
const affectedSkillIds = new Set<Id<"skills">>();
const reports = await ctx.db.query("skillReports").order("desc").take(REPORT_SCAN_LIMIT);
for (const report of reports) {
if (report.triageNote !== DEMO_REPORT_MARKER) continue;
affectedSkillIds.add(report.skillId);
await ctx.db.delete(report._id);
reportsDeleted += 1;
}
for (const skillId of affectedSkillIds) {
const skill = await ctx.db.get(skillId);
if (!skill) continue;
await restoreSkillReportSummary(ctx, skillId);
}
const fingerprints = await ctx.db
.query("skillVersionFingerprints")
.withIndex("by_fingerprint", (q) => q.eq("fingerprint", DEMO_FINGERPRINT))
.take(100);
for (const fingerprint of fingerprints) {
const version = await ctx.db.get(fingerprint.versionId);
if (version && version.fingerprint === DEMO_FINGERPRINT) {
await ctx.db.patch(fingerprint.versionId, { fingerprint: undefined });
}
await ctx.db.delete(fingerprint._id);
fingerprintsDeleted += 1;
}
return { reportsDeleted, fingerprintsDeleted };
}
async function restoreSkillReportSummary(
ctx: Pick<MutationCtx, "db">,
skillId: Id<"skills">,
): Promise<void> {
const reports = await ctx.db
.query("skillReports")
.withIndex("by_skill_createdAt", (q) => q.eq("skillId", skillId))
.order("desc")
.take(REPORT_SCAN_LIMIT);
const openReports = reports.filter((report) => (report.status ?? "open") === "open");
await ctx.db.patch(skillId, {
reportCount: openReports.length > 0 ? openReports.length : undefined,
lastReportedAt: openReports[0]?.createdAt,
});
}
async function findDuplicateDemoTargets(
ctx: Pick<MutationCtx, "db">,
skills: Doc<"skills">[],
): Promise<DuplicateDemoTarget[]> {
const targets: DuplicateDemoTarget[] = [];
for (const skill of skills) {
const versionId = skill.latestVersionId;
if (!versionId) continue;
const version = await ctx.db.get(versionId);
if (!version || version.fingerprint) continue;
targets.push({ skill, versionId });
if (targets.length === 2) break;
}
return targets;
}
export const seedManagementQueues = internalMutation({
args: {},
handler: async (
ctx,
): Promise<{
reportsInserted: number;
reportedSkills: number;
duplicatePair: number;
}> => {
assertLocalDevSeedAllowed("Management");
await clearDemo(ctx);
const now = Date.now();
const reporter = (await ctx.db.query("users").take(1))[0];
if (!reporter) {
throw new Error("No users found to attribute demo reports to; run the base dev seed first.");
}
const skills = (await ctx.db.query("skills").order("desc").take(SKILL_SCAN_LIMIT)).filter(
(skill) => !skill.softDeletedAt && skill.latestVersionId,
);
if (skills.length < 2) {
throw new Error("Need at least 2 seeded skills; run the base dev seed first.");
}
// Content reports: flag the first few skills with 1-3 reports each.
const reportTargets = skills.slice(0, Math.min(3, skills.length));
let reportsInserted = 0;
for (let i = 0; i < reportTargets.length; i += 1) {
const skill = reportTargets[i];
const count = 1 + (i % 3);
for (let r = 0; r < count; r += 1) {
await ctx.db.insert("skillReports", {
skillId: skill._id,
userId: reporter._id,
reason: DEMO_REPORT_REASONS[(i + r) % DEMO_REPORT_REASONS.length],
status: "open",
triageNote: DEMO_REPORT_MARKER,
createdAt: now - (i * 3 + r) * HOUR_MS,
});
reportsInserted += 1;
}
await ctx.db.patch(skill._id, {
reportCount: count,
lastReportedAt: now - i * HOUR_MS,
});
}
// Duplicate candidates: give a pair of skills the same latest-version fingerprint
// so each surfaces the other as a near-duplicate.
const duplicatePair = await findDuplicateDemoTargets(ctx, skills);
for (const { skill, versionId } of duplicatePair) {
await ctx.db.patch(versionId, { fingerprint: DEMO_FINGERPRINT });
await ctx.db.insert("skillVersionFingerprints", {
skillId: skill._id,
versionId,
fingerprint: DEMO_FINGERPRINT,
kind: "source",
createdAt: now,
});
}
return {
reportsInserted,
reportedSkills: reportTargets.length,
duplicatePair: duplicatePair.length,
};
},
});
export const clearManagementQueues = internalMutation({
args: {},
handler: async (ctx): Promise<{ reportsDeleted: number; fingerprintsDeleted: number }> => {
assertLocalDevSeedAllowed("Management");
return clearDemo(ctx);
},
});
+303 -24
View File
@@ -10,7 +10,6 @@ import {
getPackageReleaseScanBackfillBatchInternal,
getByName,
list,
publishPackage,
publishPackageForTrustedPublisherInternal,
publishPackageForUserInternal,
listPackageReportsInternal,
@@ -33,6 +32,7 @@ import {
listVersions,
updateReleaseStaticScanInternal,
applyAccountDeletionToOwnedPackagesBatchInternal,
applyPublisherDeletionToOwnedPackagesBatchInternal,
applyBanToOwnedPackagesBatchInternal,
revokePackagePublishTokensForPackageBatchInternal,
restoreOwnedPackagesForUnbanBatchInternal,
@@ -80,6 +80,22 @@ const listHandler = (
}>
>
)._handler;
const applyPublisherDeletionToOwnedPackagesBatchInternalHandler = (
applyPublisherDeletionToOwnedPackagesBatchInternal as unknown as WrappedHandler<
{
ownerPublisherId: string;
actorUserId: string;
deletedAt: number;
cursor?: string;
},
{
deletedCount: number;
revokedTokenCount: number;
scheduled: boolean;
stale?: true;
}
>
)._handler;
const getVersionByNameHandler = (
getVersionByName as unknown as WrappedHandler<
{ name: string; version: string },
@@ -101,6 +117,7 @@ const listPublicPageHandler = (
executesCode?: boolean;
capabilityTag?: string;
category?: string;
sort?: "updated" | "downloads";
paginationOpts: { cursor: string | null; numItems: number };
},
{ page: Array<{ name: string }>; isDone: boolean; continueCursor: string }
@@ -115,6 +132,7 @@ const listPageForViewerInternalHandler = (
executesCode?: boolean;
capabilityTag?: string;
category?: string;
sort?: "updated" | "downloads";
viewerUserId?: string;
paginationOpts: { cursor: string | null; numItems: number };
},
@@ -237,14 +255,6 @@ const searchForViewerInternalHandler = (
Array<{ package: { name: string } }>
>
)._handler;
const publishPackageHandler = (
publishPackage as unknown as WrappedHandler<
{
payload: unknown;
},
unknown
>
)._handler;
const publishPackageForUserInternalHandler = (
publishPackageForUserInternal as unknown as WrappedHandler<
{
@@ -738,6 +748,11 @@ function makeDigestCtx(options: {
isDone: boolean;
continueCursor: string;
}>;
packagePages?: Array<{
page: Array<Record<string, unknown>>;
isDone: boolean;
continueCursor: string;
}>;
exactPackages?: Array<Record<string, unknown>>;
exactDigests?: Array<Record<string, unknown>>;
publisherDocs?: Record<string, Record<string, unknown>>;
@@ -789,6 +804,7 @@ function makeDigestCtx(options: {
setPages("packageSearchDigest", options.pages ?? []);
setPages("packageCapabilitySearchDigest", options.capabilityPages ?? []);
setPages("packagePluginCategorySearchDigest", options.categoryPages ?? []);
setPages("packages", options.packagePages ?? []);
const paginate = vi.fn();
const take = vi.fn();
@@ -848,6 +864,8 @@ function makeDigestCtx(options: {
ctx: {
db: {
get: vi.fn(async (id: string) => {
const exactPackage = (options.exactPackages ?? []).find((pkg) => pkg._id === id);
if (exactPackage) return exactPackage;
if (options.publisherDocs?.[id]) return options.publisherDocs[id];
if (options.publisherMemberships?.[id]) return { _id: id, kind: "org" };
return null;
@@ -882,6 +900,9 @@ function makeDigestCtx(options: {
},
};
builder?.(queryBuilder);
if (indexName === "by_active_downloads") {
return withIndex(table, indexName);
}
if (indexName !== "by_name" && indexName !== "by_runtime_id") {
throw new Error(`Unexpected packages index ${indexName}`);
}
@@ -1151,6 +1172,37 @@ function makeInsertReleaseCtx(
),
};
}
if (table === "officialPublishers") {
return {
withIndex: vi.fn(
(
_indexName: string,
buildQuery?: (q: { eq: (field: string, value: unknown) => unknown }) => unknown,
) => {
const filters = new Map<string, unknown>();
const query = {
eq(field: string, value: unknown) {
filters.set(field, value);
return query;
},
};
buildQuery?.(query);
const rawPublisherId = filters.get("publisherId");
const publisherId = typeof rawPublisherId === "string" ? rawPublisherId : "";
const publisher = recordsById[publisherId];
return {
unique: vi
.fn()
.mockResolvedValue(
publisher?.handle === "openclaw"
? { _id: "officialPublishers:openclaw", publisherId }
: null,
),
};
},
),
};
}
throw new Error(`Unexpected table ${table}`);
}),
insert,
@@ -1265,6 +1317,38 @@ function makeTransferPackageOwnerCtx(options?: {
})),
};
}
if (table === "officialPublishers") {
return {
withIndex: vi.fn((_indexName: string, builder: (q: unknown) => unknown) => {
const terms: Record<string, unknown> = {};
builder({
eq: (field: string, value: unknown) => {
terms[field] = value;
return {};
},
});
const ownerPublisher =
terms.publisherId === "publishers:openclaw"
? (options?.ownerPublisher ?? {
_id: "publishers:openclaw",
kind: "org",
handle: "openclaw",
displayName: "OpenClaw",
trustedPublisher: true,
})
: null;
return {
unique: vi
.fn()
.mockResolvedValue(
ownerPublisher?.handle === "openclaw"
? { _id: "officialPublishers:openclaw", publisherId: terms.publisherId }
: null,
),
};
}),
};
}
if (
table === "packageCapabilitySearchDigest" ||
table === "packagePluginCategorySearchDigest"
@@ -1416,6 +1500,13 @@ function makeUserTransferPackageOwnerCtx(options?: {
}),
};
}
if (table === "officialPublishers") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
})),
};
}
throw new Error(`Unexpected table ${table}`);
}),
insert,
@@ -1689,6 +1780,102 @@ describe("packages public queries", () => {
expect(result.continueCursor).not.toContain("bravo summary");
});
it("includes package stats on public list items", async () => {
const stats = { downloads: 43, installs: 3, stars: 1, versions: 2 };
const { ctx } = makeDigestCtx({
pages: [
{
page: [makeDigest("stats-demo", { stats })],
isDone: true,
continueCursor: "",
},
],
});
const result = await listPublicPageHandler(ctx, {
paginationOpts: { cursor: null, numItems: 10 },
});
expect((result.page[0] as { stats?: unknown }).stats).toEqual(stats);
});
it("uses current package stats when digest stats are stale", async () => {
const currentStats = { downloads: 99, installs: 7, stars: 2, versions: 3 };
const { ctx } = makeDigestCtx({
pages: [
{
page: [
makeDigest("stats-demo", {
stats: { downloads: 1, installs: 0, stars: 0, versions: 1 },
}),
],
isDone: true,
continueCursor: "",
},
],
exactPackages: [
makePackageDoc({
_id: "packages:stats-demo",
name: "stats-demo",
normalizedName: "stats-demo",
displayName: "stats-demo",
stats: currentStats,
}),
],
});
const result = await listPublicPageHandler(ctx, {
paginationOpts: { cursor: null, numItems: 10 },
});
expect((result.page[0] as { stats?: unknown }).stats).toEqual(currentStats);
});
it("continues scanning download-sorted pages until filtered public results are filled", async () => {
const { ctx, paginate } = makeDigestCtx({
packagePages: [
{
page: [
makePackageDoc({
_id: "packages:bundle-plugin",
name: "bundle-plugin",
normalizedName: "bundle-plugin",
displayName: "Bundle Plugin",
family: "bundle-plugin",
stats: { downloads: 500, installs: 0, stars: 0, versions: 1 },
}),
],
isDone: false,
continueCursor: "cursor:next",
},
{
page: [
makePackageDoc({
_id: "packages:code-plugin",
name: "code-plugin",
normalizedName: "code-plugin",
displayName: "Code Plugin",
family: "code-plugin",
stats: { downloads: 200, installs: 0, stars: 0, versions: 1 },
}),
],
isDone: true,
continueCursor: "",
},
],
});
const result = await listPublicPageHandler(ctx, {
family: "code-plugin",
sort: "downloads",
paginationOpts: { cursor: null, numItems: 1 },
});
expect(result.page.map((entry) => entry.name)).toEqual(["code-plugin"]);
expect(result.isDone).toBe(true);
expect(paginate).toHaveBeenCalledTimes(2);
});
it("excludes private packages from public list pages", async () => {
const { ctx } = makeDigestCtx({
pages: [
@@ -2826,6 +3013,62 @@ describe("packages public queries", () => {
});
});
it("derives missing public verification source paths from legacy release provenance", async () => {
const verification = {
tier: "source-linked",
scope: "artifact-only",
sourceRepo: "OpenViking/OpenViking",
sourceCommit: "abcdef0123456789abcdef0123456789abcdef01",
scanStatus: "clean",
};
const latestRelease = makeReleaseDoc({
verification,
source: {
kind: "github",
repo: "OpenViking/OpenViking",
path: "openclaw-plugin",
},
});
const { ctx } = makePackageCtx({
pkg: makePackageDoc({
name: "@openviking/openclaw-plugin",
normalizedName: "@openviking/openclaw-plugin",
verification,
latestVersionSummary: {
version: "1.0.0",
verification,
},
}),
latestRelease,
});
await expect(
getByNameHandler(ctx, {
name: "@openviking/openclaw-plugin",
}),
).resolves.toMatchObject({
package: {
verification: { sourcePath: "openclaw-plugin" },
},
latestRelease: {
verification: { sourcePath: "openclaw-plugin" },
},
});
await expect(
getVersionByNameHandler(ctx, {
name: "@openviking/openclaw-plugin",
version: "1.0.0",
}),
).resolves.toMatchObject({
package: {
verification: { sourcePath: "openclaw-plugin" },
},
version: {
verification: { sourcePath: "openclaw-plugin" },
},
});
});
it("does not mark owner-readable blocked public packages as public download blocked", async () => {
const { ctx } = makePackageCtx({
pkg: makePackageDoc({
@@ -3726,7 +3969,7 @@ describe("packages public queries", () => {
);
});
it("rejects official package transfers to non-OpenClaw publishers", async () => {
it("rejects official package transfers to non-official publishers", async () => {
const { ctx } = makeTransferPackageOwnerCtx({
ownerPublisher: {
_id: "publishers:openclaw",
@@ -6300,20 +6543,6 @@ describe("packages public queries", () => {
expect(result).toEqual([expect.objectContaining({ name: "demo-plugin" })]);
});
it("requires auth inside the public publish action", async () => {
await expect(
publishPackageHandler({ runQuery: vi.fn(), runMutation: vi.fn() } as never, {
payload: {
name: "demo-plugin",
family: "bundle-plugin",
version: "1.0.0",
changelog: "init",
files: [],
},
}),
).rejects.toThrow("Unauthorized");
});
it("records package reports for moderation", async () => {
const insert = vi.fn(async (table: string) =>
table === "packageReports" ? "packageReports:1" : "auditLogs:1",
@@ -8752,6 +8981,56 @@ describe("owned package sanction batches", () => {
});
});
it("soft-deletes packages owned by a deleted publisher", async () => {
const orgPackage = makePackageDoc({
_id: "packages:org-plugin",
ownerUserId: "users:owner",
ownerPublisherId: "publishers:org",
});
const { ctx, patch } = makeOwnedPackageBatchCtx({
publisherPackages: [orgPackage],
releases: [
makeReleaseDoc({
_id: "packageReleases:org-plugin-1",
packageId: "packages:org-plugin",
}),
],
packageTokens: [
{
_id: "packagePublishTokens:org-plugin",
packageId: "packages:org-plugin",
version: "1.0.0",
revokedAt: undefined,
},
],
publishers: {
"publishers:org": {
_id: "publishers:org",
kind: "org",
deletedAt: 3_000,
},
},
});
const result = await applyPublisherDeletionToOwnedPackagesBatchInternalHandler(ctx as never, {
ownerPublisherId: "publishers:org",
actorUserId: "users:owner",
deletedAt: 3_000,
});
expect(result).toMatchObject({ deletedCount: 1, revokedTokenCount: 1, scheduled: false });
expect(patch).toHaveBeenCalledWith(
"packages:org-plugin",
expect.objectContaining({
softDeletedAt: 3_000,
softDeletedReason: "publisher.deleted",
softDeletedBy: "users:owner",
softDeletedByRole: "user",
}),
);
expect(patch).toHaveBeenCalledWith("packagePublishTokens:org-plugin", { revokedAt: 3_000 });
});
it("schedules linked legacy personal publisher scans when the user row lacks the publisher id", async () => {
const { ctx, runAfter } = makeOwnedPackageBatchCtx({
owner: {
+281 -57
View File
@@ -1,5 +1,6 @@
import {
PackagePublishRequestSchema,
ServerPackagePublishRequestSchema,
derivePluginCategoryTags,
getPackageScopeOwnerMismatch,
isPluginCategorySlug,
parseArk,
@@ -11,7 +12,7 @@ import {
type PackageModerationQueueStatus,
type PackageOfficialMigrationListPhase,
type PackageOfficialMigrationPhase,
type PackagePublishRequest,
type ServerPackagePublishRequest,
type PackageVerificationTier,
} from "clawhub-schema";
import { paginationOptsValidator } from "convex/server";
@@ -33,7 +34,6 @@ import {
assertModerator,
getOptionalActiveAuthUserId,
requireUser,
requireUserFromAction,
} from "./lib/access";
import {
assertArtifactAppealFinalAction,
@@ -287,7 +287,7 @@ const packageAutobanRemediationInternalRefs = internal as unknown as {
type DbReaderCtx = Pick<QueryCtx | MutationCtx, "db">;
const BAN_USER_PACKAGES_BATCH_SIZE = 25;
const PACKAGE_PUBLISH_TOKEN_REVOKE_BATCH_SIZE = 25;
type PackageSoftDeletedReason = "user.banned" | "user.deactivated";
type PackageSoftDeletedReason = "user.banned" | "user.deactivated" | "publisher.deleted";
const ownedPackageScanScopeValidator = v.optional(
v.union(v.literal("ownerUserId"), v.literal("personalPublisher")),
);
@@ -332,6 +332,7 @@ type PublicPackageListItem = {
capabilityTags: string[];
executesCode: boolean;
verificationTier: PackageVerificationTier | null;
stats: Doc<"packages">["stats"];
};
type PackageReleaseScanStatus = ReturnType<typeof resolvePackageReleaseScanStatus>;
type PackageReleaseModerationQueueDoc = Omit<Doc<"packageReleases">, "createdAt"> & {
@@ -582,6 +583,7 @@ type PackageDigestLike = Pick<
| "pluginCategoryTags"
| "executesCode"
| "verificationTier"
| "stats"
| "scanStatus"
| "softDeletedAt"
> & {
@@ -765,13 +767,29 @@ function resolvePublicPackageScanStatus(
return pkg.scanStatus;
}
function normalizePublicPackageSourcePath(sourcePath: unknown) {
if (typeof sourcePath !== "string") return undefined;
const trimmed = sourcePath.trim();
if (!trimmed || trimmed === ".") return undefined;
return trimmed.replace(/^\/+/, "").replace(/\/+$/, "") || undefined;
}
function getReleaseSourcePath(release?: Pick<Doc<"packageReleases">, "source"> | null) {
const source = release?.source;
if (!source || typeof source !== "object" || Array.isArray(source)) return undefined;
return normalizePublicPackageSourcePath((source as { path?: unknown }).path);
}
function resolvePublicPackageVerification(
pkg: Pick<Doc<"packages">, "verification" | "latestVersionSummary" | "scanStatus">,
latestRelease?: Doc<"packageReleases"> | null,
) {
const scanStatus = resolvePublicPackageScanStatus(pkg, latestRelease);
const source = pkg.verification ?? pkg.latestVersionSummary?.verification;
return source && scanStatus ? { ...source, scanStatus } : source;
if (!source) return source;
const sourcePath = source.sourcePath ?? getReleaseSourcePath(latestRelease);
const verification = sourcePath ? { ...source, sourcePath } : source;
return scanStatus ? { ...verification, scanStatus } : verification;
}
function toPublicPackage(
@@ -823,6 +841,19 @@ function omitLegacyClawScanNoteFields(release: Doc<"packageReleases">) {
return publicRelease;
}
function toPublicPackageRelease(release: Doc<"packageReleases">) {
const publicRelease = omitLegacyClawScanNoteFields(release);
const sourcePath = release.verification?.sourcePath ?? getReleaseSourcePath(release);
if (!release.verification || !sourcePath) return publicRelease;
return {
...publicRelease,
verification: {
...release.verification,
sourcePath,
},
};
}
function packageArtifactSummary(
release: Pick<
Doc<"packageReleases">,
@@ -940,6 +971,40 @@ function digestMatchesSearchFilters(
return digestMatchesFilters(digest, args);
}
function packageMatchesListFilters(
pkg: Doc<"packages">,
args: {
family?: PackageFamily;
channel?: PackageChannel;
isOfficial?: boolean;
executesCode?: boolean;
capabilityTag?: string;
category?: string;
},
) {
if (args.family && pkg.family !== args.family) return false;
if (args.channel && pkg.channel !== args.channel) return false;
if (typeof args.isOfficial === "boolean" && pkg.isOfficial !== args.isOfficial) return false;
if (typeof args.executesCode === "boolean" && Boolean(pkg.executesCode) !== args.executesCode) {
return false;
}
if (args.capabilityTag && !(pkg.capabilityTags ?? []).includes(args.capabilityTag)) {
return false;
}
if (args.category) {
const categories = derivePluginCategoryTags({
family: pkg.family,
name: pkg.name,
displayName: pkg.displayName,
runtimeId: pkg.runtimeId,
summary: pkg.summary,
capabilityTags: pkg.capabilityTags,
});
if (!categories.includes(args.category as PluginCategorySlug)) return false;
}
return true;
}
async function upsertPackageBadge(
ctx: MutationCtx,
packageId: Id<"packages">,
@@ -975,7 +1040,22 @@ async function removePackageBadge(
if (existing) await ctx.db.delete(existing._id);
}
function toPublicPackageListItem(digest: PackageDigestLike): PublicPackageListItem {
function defaultPackageStats(): Doc<"packages">["stats"] {
return { downloads: 0, installs: 0, stars: 0, versions: 0 };
}
async function resolvePackageListStats(
ctx: DbReaderCtx,
digest: PackageDigestLike,
): Promise<Doc<"packages">["stats"]> {
const pkg = await ctx.db.get(digest.packageId);
return pkg?.stats ?? digest.stats ?? defaultPackageStats();
}
async function toPublicPackageListItem(
ctx: DbReaderCtx,
digest: PackageDigestLike,
): Promise<PublicPackageListItem> {
return {
name: digest.name,
displayName: digest.displayName,
@@ -991,6 +1071,36 @@ function toPublicPackageListItem(digest: PackageDigestLike): PublicPackageListIt
capabilityTags: digest.capabilityTags ?? [],
executesCode: digest.executesCode ?? false,
verificationTier: digest.verificationTier ?? null,
stats: await resolvePackageListStats(ctx, digest),
};
}
async function toPublicPackageListItemFromPackage(
ctx: DbReaderCtx,
pkg: Doc<"packages">,
): Promise<PublicPackageListItem> {
const owner = toPublicPublisher(
await getOwnerPublisher(ctx, {
ownerPublisherId: pkg.ownerPublisherId,
ownerUserId: pkg.ownerUserId,
}),
);
return {
name: pkg.name,
displayName: pkg.displayName,
family: pkg.family,
runtimeId: pkg.runtimeId ?? null,
channel: pkg.channel,
isOfficial: pkg.isOfficial,
summary: pkg.summary ?? null,
ownerHandle: owner?.handle ?? null,
createdAt: pkg.createdAt,
updatedAt: pkg.updatedAt,
latestVersion: pkg.latestVersionSummary?.version ?? null,
capabilityTags: pkg.capabilityTags ?? [],
executesCode: pkg.executesCode ?? false,
verificationTier: pkg.verification?.tier ?? null,
stats: pkg.stats,
};
}
@@ -1803,15 +1913,19 @@ async function fetchHighlightedPackagePage(
},
) {
const digests = await fetchHighlightedPackageDigests(ctx, args);
return digests
const page = digests
.sort(
(a, b) =>
Number(b.isOfficial) - Number(a.isOfficial) ||
b.updatedAt - a.updatedAt ||
a.name.localeCompare(b.name),
)
.slice(0, args.numItems)
.map(toPublicPackageListItem);
.slice(0, args.numItems);
const items: PublicPackageListItem[] = [];
for (const digest of page) {
items.push(await toPublicPackageListItem(ctx, digest));
}
return items;
}
async function getPackageByNormalizedName(ctx: DbReaderCtx, normalizedName: string) {
@@ -1908,7 +2022,7 @@ export const getByName = query({
package: publicPackage,
latestRelease:
latestRelease && !latestRelease.softDeletedAt
? omitLegacyClawScanNoteFields(latestRelease)
? toPublicPackageRelease(latestRelease)
: null,
owner,
};
@@ -1949,7 +2063,7 @@ export const getManageContext = query({
return {
package: pkg,
latestRelease: omitLegacyClawScanNoteFields(latestRelease),
latestRelease: toPublicPackageRelease(latestRelease),
};
},
});
@@ -1979,7 +2093,7 @@ export const getByNameForStaff = query({
package: pkg,
latestRelease:
latestRelease && !latestRelease.softDeletedAt
? omitLegacyClawScanNoteFields(latestRelease)
? toPublicPackageRelease(latestRelease)
: null,
owner,
highlighted: highlighted
@@ -2013,7 +2127,7 @@ export const getByNameForViewerInternal = internalQuery({
package: publicPackage,
latestRelease:
latestRelease && !latestRelease.softDeletedAt
? omitLegacyClawScanNoteFields(latestRelease)
? toPublicPackageRelease(latestRelease)
: null,
owner,
};
@@ -2092,7 +2206,7 @@ export const getVersionByName = query({
if (!publicPackage) return null;
return {
package: publicPackage,
version: omitLegacyClawScanNoteFields(release),
version: toPublicPackageRelease(release),
};
},
});
@@ -2123,7 +2237,7 @@ export const getVersionByNameForViewerInternal = internalQuery({
if (!publicPackage) return null;
return {
package: publicPackage,
version: omitLegacyClawScanNoteFields(release),
version: toPublicPackageRelease(release),
};
},
});
@@ -2160,7 +2274,7 @@ export const getVersionSecurityByNameForViewerInternal = internalQuery({
...publicPackage,
publicDownloadBlocked,
},
version: omitLegacyClawScanNoteFields(release),
version: toPublicPackageRelease(release),
};
},
});
@@ -2203,6 +2317,7 @@ export const listPublicPage = query({
executesCode: v.optional(v.boolean()),
capabilityTag: v.optional(v.string()),
category: v.optional(v.string()),
sort: v.optional(v.union(v.literal("updated"), v.literal("downloads"))),
paginationOpts: paginationOptsValidator,
},
handler: async (ctx, args) => {
@@ -2302,6 +2417,7 @@ export const listPageForViewerInternal = internalQuery({
executesCode: v.optional(v.boolean()),
capabilityTag: v.optional(v.string()),
category: v.optional(v.string()),
sort: v.optional(v.union(v.literal("updated"), v.literal("downloads"))),
viewerUserId: v.optional(v.id("users")),
paginationOpts: paginationOptsValidator,
},
@@ -2320,6 +2436,7 @@ async function listPackagePageImpl(
executesCode?: boolean;
capabilityTag?: string;
category?: string;
sort?: "updated" | "downloads";
viewerUserId?: Id<"users">;
paginationOpts: { cursor: string | null; numItems: number };
},
@@ -2364,6 +2481,68 @@ async function listPackagePageImpl(
const isOfficial = args.isOfficial;
const category = isPluginCategorySlug(args.category) ? args.category : undefined;
if (args.sort === "downloads") {
let cursor = pageCursor;
let pageOffset = offset;
let pageSize: number | null = decodedCursor.pageSize ?? null;
let done = decodedCursor.done;
while ((pageOffset > 0 || !done) && collected.length < targetCount) {
const scanPageSize = Math.min(
MAX_PUBLIC_LIST_PAGE_SIZE,
pageOffset > 0 && pageSize
? Math.max(pageSize, pageOffset + targetCount)
: Math.max(targetCount * 5, targetCount, 50),
);
const currentCursor = cursor;
const page = await ctx.db
.query("packages")
.withIndex("by_active_downloads", (q) => q.eq("softDeletedAt", undefined))
.order("desc")
.paginate({ cursor: currentCursor, numItems: scanPageSize });
for (let index = pageOffset; index < page.page.length; index += 1) {
const pkg = page.page[index];
if (!(await canViewerReadPackage(ctx, pkg, viewerUserId, membershipCache))) continue;
if (!packageMatchesListFilters(pkg, { ...args, category })) continue;
collected.push(await toPublicPackageListItemFromPackage(ctx, pkg));
if (collected.length >= targetCount) {
const nextOffset = index + 1;
const nextState =
nextOffset < page.page.length
? {
cursor: currentCursor,
offset: nextOffset,
pageSize: scanPageSize,
done: page.isDone,
}
: {
cursor: page.continueCursor,
offset: 0,
pageSize: scanPageSize,
done: page.isDone,
};
return {
page: collected,
isDone: nextState.done && nextState.offset === 0,
continueCursor: encodePublicPageCursor(nextState),
};
}
}
done = page.isDone;
cursor = page.continueCursor;
pageOffset = 0;
pageSize = scanPageSize;
}
return {
page: collected,
isDone: done,
continueCursor: encodePublicPageCursor({ cursor, offset: pageOffset, pageSize, done }),
};
}
const builder = category
? buildPackagePluginCategoryDigestQuery(ctx, {
category,
@@ -2399,7 +2578,7 @@ async function listPackagePageImpl(
continue;
}
if (!digestMatchesFilters(digest, args)) continue;
collected.push(toPublicPackageListItem(digest));
collected.push(await toPublicPackageListItem(ctx, digest));
if (collected.length >= targetCount) {
const nextOffset = index + 1;
const nextState =
@@ -2505,7 +2684,7 @@ async function searchPackagesImpl(
await canViewerReadPackage(ctx, digest, viewerUserId, membershipCache);
if (args.highlightedOnly) {
const digests = await fetchHighlightedPackageDigests(ctx, args);
return digests
const entries = digests
.map((digest) => {
const match = packageSearchMatch(digest, queryText);
return match ? { ...match, package: digest } : null;
@@ -2514,12 +2693,16 @@ async function searchPackagesImpl(
Boolean(entry),
)
.sort(comparePackageSearchMatches)
.slice(0, targetCount)
.map((entry) => ({
.slice(0, targetCount);
const results: Array<PackageSearchMatch & { package: PublicPackageListItem }> = [];
for (const entry of entries) {
results.push({
score: entry.score,
rankTier: entry.rankTier,
package: toPublicPackageListItem(entry.package),
}));
package: await toPublicPackageListItem(ctx, entry.package),
});
}
return results;
}
const category = isPluginCategorySlug(args.category) ? args.category : undefined;
@@ -2560,7 +2743,7 @@ async function searchPackagesImpl(
seen.add(digest.packageId);
matches.push({
...match,
package: toPublicPackageListItem(digest),
package: await toPublicPackageListItem(ctx, digest),
});
}
@@ -2578,7 +2761,7 @@ async function searchPackagesImpl(
seen.add(digest.packageId);
matches.push({
...match,
package: toPublicPackageListItem(digest),
package: await toPublicPackageListItem(ctx, digest),
});
if (matches.length >= targetCount) break;
}
@@ -2847,15 +3030,9 @@ async function softDeletePackageDoc(
.query("packageReleases")
.withIndex("by_package", (q) => q.eq("packageId", pkg._id))
.collect();
let releaseCount = 0;
const deletedReleaseIds: Array<Id<"packageReleases">> = [];
for (const release of releases) {
if (release.softDeletedAt) continue;
await ctx.db.patch(release._id, { softDeletedAt: now });
releaseCount += 1;
deletedReleaseIds.push(release._id);
}
const deletedReleaseIds = releases
.filter((release) => !release.softDeletedAt)
.map((release) => release._id);
const packagePatch: Partial<Doc<"packages">> = {
softDeletedAt: now,
softDeletedReason: params.reason,
@@ -2874,6 +3051,9 @@ async function softDeletePackageDoc(
ownerHandle: deleteOwner?.handle ?? "",
ownerKind: deleteOwner?.kind,
});
for (const releaseId of deletedReleaseIds) {
await ctx.db.patch(releaseId, { softDeletedAt: now });
}
await ctx.db.insert("auditLogs", {
actorUserId: params.actorUserId,
action: "package.delete",
@@ -2886,7 +3066,7 @@ async function softDeletePackageDoc(
ownerPublisherId: pkg.ownerPublisherId,
actorRole: params.actorRole ?? "user",
softDeletedReason: params.reason ?? null,
releaseCount,
releaseCount: deletedReleaseIds.length,
releaseIds: deletedReleaseIds,
source: params.source,
},
@@ -2896,7 +3076,7 @@ async function softDeletePackageDoc(
return {
ok: true as const,
packageId: pkg._id,
releaseCount,
releaseCount: deletedReleaseIds.length,
alreadyDeleted: false as const,
};
}
@@ -3448,6 +3628,66 @@ export const applyAccountDeletionToOwnedPackagesBatchInternal = internalMutation
},
});
export const applyPublisherDeletionToOwnedPackagesBatchInternal = internalMutation({
args: {
ownerPublisherId: v.id("publishers"),
actorUserId: v.id("users"),
deletedAt: v.number(),
cursor: v.optional(v.string()),
},
handler: async (ctx, args) => {
const publisher = await ctx.db.get(args.ownerPublisherId);
if (!publisher || publisher.deletedAt !== args.deletedAt) {
return {
ok: true as const,
deletedCount: 0,
revokedTokenCount: 0,
scheduled: false,
stale: true as const,
};
}
const { page, isDone, continueCursor } = await ctx.db
.query("packages")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", args.ownerPublisherId))
.order("desc")
.paginate({
cursor: args.cursor ?? null,
numItems: BAN_USER_PACKAGES_BATCH_SIZE,
});
let deletedCount = 0;
let revokedTokenCount = 0;
for (const pkg of page) {
const revokeResult = await revokePackagePublishTokensForPackage(ctx, pkg._id, args.deletedAt);
revokedTokenCount += revokeResult.revokedCount;
if (pkg.softDeletedAt) continue;
await softDeletePackageDoc(ctx, pkg, {
actorUserId: args.actorUserId,
actorRole: "user",
deletedAt: args.deletedAt,
reason: "publisher.deleted",
source: "dashboard",
});
deletedCount += 1;
}
if (!isDone) {
await ctx.scheduler.runAfter(
0,
internal.packages.applyPublisherDeletionToOwnedPackagesBatchInternal,
{
...args,
cursor: continueCursor,
},
);
}
return { ok: true as const, deletedCount, revokedTokenCount, scheduled: !isDone };
},
});
export const softDeletePackageInternal = internalMutation({
args: {
userId: v.id("users"),
@@ -5027,9 +5267,9 @@ function buildGitHubActionsPublishActor(
}
function resolveTrustedPublishSource(
payload: PackagePublishRequest,
payload: ServerPackagePublishRequest,
publishToken: Doc<"packagePublishTokens">,
): PackagePublishRequest["source"] {
): ServerPackagePublishRequest["source"] {
const source = payload.source;
if (source && source.kind !== "github") {
throw new ConvexError("Trusted publishes only support GitHub source metadata");
@@ -5081,11 +5321,11 @@ async function publishPackageImpl(
auth: PackagePublishAuthContext,
rawPayload: unknown,
) {
const payload = parseArk(
PackagePublishRequestSchema,
const payload = parseArk<ServerPackagePublishRequest>(
ServerPackagePublishRequestSchema,
rawPayload,
"Package publish payload",
) as PackagePublishRequest;
);
if (payload.family === "skill") {
throw new ConvexError("Skill packages must use the skills publish flow");
}
@@ -5204,7 +5444,7 @@ async function publishPackageImpl(
}
const displayName = payload.displayName?.trim() || name;
const files = normalizePublishFiles(payload.files as never);
const files = normalizePublishFiles(payload.files);
if (payload.artifact?.kind !== "npm-pack") {
const oversizedFile = findOversizedPublishFile(files);
if (oversizedFile) {
@@ -5459,14 +5699,6 @@ async function publishPackageImpl(
return publishResult;
}
export const publishPackage = action({
args: { payload: v.any() },
handler: async (ctx, args) => {
const { userId } = await requireUserFromAction(ctx);
return await publishPackageImpl(ctx, { kind: "user", actorUserId: userId }, args.payload);
},
});
export const publishPackageForUserInternal = internalAction({
args: {
actorUserId: v.id("users"),
@@ -5509,14 +5741,6 @@ export const publishPackageForTrustedPublisherInternal = internalAction({
},
});
export const publishRelease = action({
args: { payload: v.any() },
handler: async (ctx, args) => {
const { userId } = await requireUserFromAction(ctx);
return await publishPackageImpl(ctx, { kind: "user", actorUserId: userId }, args.payload);
},
});
export const reservePackageNameInternal = internalMutation({
args: {
actorUserId: v.id("users"),
File diff suppressed because it is too large Load Diff
+593 -10
View File
@@ -1,8 +1,17 @@
import { v } from "convex/values";
import { internal } from "./_generated/api";
import type { Doc, Id } from "./_generated/dataModel";
import type { ActionCtx, MutationCtx } from "./_generated/server";
import { internalAction, internalMutation, internalQuery } from "./functions";
import type { ActionCtx, MutationCtx, QueryCtx } from "./_generated/server";
import {
action,
internalAction,
internalMutation,
internalQuery,
mutation,
query,
} from "./functions";
import { assertModerator, requireUser, requireUserFromAction } from "./lib/access";
import { hasOfficialPublisherRow } from "./lib/officialPublishers";
import {
computePublisherAbuseRawScore,
DEFAULT_PUBLISHER_ABUSE_MODEL_CONFIG,
@@ -20,6 +29,10 @@ const MAX_MAX_PAGES = 50;
const ACTION_CONTINUATION_DELAY_MS = 60_000;
const MAX_ACTIVE_SKILL_FALLBACK_SCAN = 500;
const MAX_ACTIVE_SKILL_FALLBACK_SCANS_PER_PAGE = 20;
const MAX_REVIEW_DASHBOARD_SCAN_MULTIPLIER = 3;
const MAX_REVIEW_DASHBOARD_SCORE_SCAN_MULTIPLIER = 32;
const MAX_REVIEW_DASHBOARD_SCORE_SCAN = 2000;
const MAX_BAN_REASON_LENGTH = 500;
type TriageStatus = Doc<"publisherAbuseReviewNominations">["status"];
type ScoreRun = Doc<"publisherAbuseScoreRuns">;
@@ -42,8 +55,11 @@ type PageResult = RunState & {
type PublisherMetricsDoc = Pick<
Doc<"publishers">,
| "_id"
| "kind"
| "handle"
| "linkedUserId"
| "deletedAt"
| "deactivatedAt"
| "publishedSkills"
| "publishedPackages"
| "totalInstalls"
@@ -54,6 +70,11 @@ type PublisherMetricsDoc = Pick<
| "skillTotalDownloads"
>;
type PublisherAbuseExclusionPublisher = Pick<
Doc<"publishers">,
"_id" | "kind" | "deletedAt" | "deactivatedAt"
>;
type PublisherSkillMetricsOptions =
| {
allowActiveSkillScan: false;
@@ -68,6 +89,195 @@ type ActiveSkillFallbackBudget = {
remainingScans: number;
};
export const listReviewDashboard = query({
args: {
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const { user } = await requireUser(ctx);
assertModerator(user);
const limit = clampInt(args.limit ?? 150, 1, 250);
const latestRun = await getLatestPublisherAbuseScoreRun(ctx);
const scoreRankRunId = latestRun?.status === "completed" ? latestRun._id : undefined;
const pendingPotentialBanCandidateItems = await getPendingPublisherAbuseReviewItemsForLabel(
ctx,
{
status: "pending",
label: "potential_ban_candidate",
limit,
latestCompletedRunId: scoreRankRunId,
},
);
const pendingReviewItems = await getPendingPublisherAbuseReviewItemsForLabel(ctx, {
status: "pending",
label: "review",
limit,
latestCompletedRunId: scoreRankRunId,
});
const pendingItems = [...pendingPotentialBanCandidateItems, ...pendingReviewItems]
.sort(comparePublisherAbuseReviewItemsByLastScoredAt)
.slice(0, limit);
const recentResolvedItems = await getRecentResolvedPublisherAbuseReviewItems(ctx, 30);
return {
latestRun: latestRun ? summarizePublisherAbuseRun(latestRun) : null,
pendingItems,
pendingPotentialBanCandidateItems,
pendingReviewItems,
recentResolvedItems,
};
},
});
export const getReviewNominationDetail = query({
args: {
nominationId: v.id("publisherAbuseReviewNominations"),
},
handler: async (ctx, args) => {
const { user } = await requireUser(ctx);
assertModerator(user);
const nomination = await ctx.db.get(args.nominationId);
if (!nomination) return null;
const item = await summarizePublisherAbuseReviewNomination(ctx, nomination);
if (await isPublisherAbuseExcludedReviewItem(ctx, item)) return null;
const scoreHistory = await ctx.db
.query("publisherAbuseScores")
.withIndex("by_owner_key_and_created_at", (q) => q.eq("ownerKey", nomination.ownerKey))
.order("desc")
.take(5);
const latestScoreRun = item.latestScore ? await ctx.db.get(item.latestScore.runId) : null;
const events = await ctx.db
.query("publisherAbuseReviewEvents")
.withIndex("by_nomination_and_created_at", (q) => q.eq("nominationId", nomination._id))
.order("desc")
.take(20);
return {
item,
latestScoreRun: latestScoreRun ? summarizePublisherAbuseRun(latestScoreRun) : null,
scoreHistory,
events,
};
},
});
export const banPublisherAbuseOwner = mutation({
args: {
nominationId: v.id("publisherAbuseReviewNominations"),
expectedLatestScoreId: v.id("publisherAbuseScores"),
expectedUpdatedAt: v.number(),
reason: v.optional(v.string()),
},
handler: async (ctx, args) => {
const { user } = await requireUser(ctx);
assertModerator(user);
const nomination = await ctx.db.get(args.nominationId);
if (!nomination) throw new Error("Publisher abuse nomination not found");
requireFreshPublisherAbuseReviewNomination(nomination, args);
requireActionablePublisherAbuseReviewNomination(nomination);
if (!nomination.ownerUserId) {
throw new Error("Cannot ban publisher abuse nomination without a linked user");
}
await requirePublisherAbuseNominationNotExcluded(ctx, nomination);
const reason = normalizeBanReason(args.reason);
await ctx.runMutation(internal.users.banUserInternal, {
actorUserId: user._id,
targetUserId: nomination.ownerUserId,
reason,
});
const now = Date.now();
await setPublisherAbuseReviewStatusWithActor(ctx, {
nomination,
status: "banned",
notes: reason,
actorUserId: user._id,
now,
});
return { ok: true, status: "banned" as const };
},
});
async function setPublisherAbuseReviewStatusWithActor(
ctx: Pick<MutationCtx, "db">,
args: {
nomination: Doc<"publisherAbuseReviewNominations">;
status: TriageStatus;
notes: string | undefined;
actorUserId: Id<"users">;
now: number;
},
) {
await ctx.db.patch(args.nomination._id, {
status: args.status,
reviewedByUserId: args.status === "pending" ? undefined : args.actorUserId,
reviewedAt: args.status === "pending" ? undefined : args.now,
notes: args.notes,
updatedAt: args.now,
});
await ctx.db.insert("publisherAbuseReviewEvents", {
nominationId: args.nomination._id,
ownerKey: args.nomination.ownerKey,
actorUserId: args.actorUserId,
scoreId: args.nomination.latestScoreId,
eventType: "triage_status_changed",
previousStatus: args.nomination.status,
nextStatus: args.status,
notes: args.notes,
createdAt: args.now,
});
}
function requireFreshPublisherAbuseReviewNomination(
nomination: Doc<"publisherAbuseReviewNominations">,
expected: { expectedLatestScoreId: Id<"publisherAbuseScores">; expectedUpdatedAt: number },
) {
if (
nomination.latestScoreId !== expected.expectedLatestScoreId ||
nomination.updatedAt !== expected.expectedUpdatedAt
) {
throw new Error("Publisher abuse nomination changed; refresh and try again");
}
}
function requireActionablePublisherAbuseReviewNomination(
nomination: Doc<"publisherAbuseReviewNominations">,
) {
if (nomination.label !== "potential_ban_candidate") {
throw new Error(
"Only potential ban publisher abuse nominations can be manually resolved; review nominations are calibration signals.",
);
}
if (nomination.status !== "pending") {
throw new Error("Only pending publisher abuse nominations can be banned.");
}
}
export const startPublisherAbuseScoreRun = action({
args: {},
handler: async (
ctx,
): Promise<{
ok: true;
runId: Id<"publisherAbuseScoreRuns">;
pages: number;
isDone: boolean;
}> => {
const { userId, user } = await requireUserFromAction(ctx);
assertModerator(user);
return await ctx.runAction(internal.publisherAbuse.runPublisherAbuseScoreRunInternal, {
trigger: "manual",
actorUserId: userId,
});
},
});
export const getOrStartPublisherAbuseScoreRunInternal = internalMutation({
args: {
trigger: v.union(v.literal("cron"), v.literal("manual")),
@@ -136,6 +346,7 @@ export const runPublisherAbuseScoreRunInternal = internalAction({
maxPages: v.optional(v.number()),
forceNew: v.optional(v.boolean()),
trigger: v.optional(v.union(v.literal("cron"), v.literal("manual"))),
actorUserId: v.optional(v.id("users")),
},
handler: runPublisherAbuseScoreRunInternalHandler,
});
@@ -183,6 +394,7 @@ export async function collectPublisherAbuseScoresPageInternalHandler(
activeSkillFallbackBudget,
};
for (const publisher of page.page) {
if (await isPublisherExcludedFromPublisherAbuse(ctx, publisher)) continue;
const input = await publisherInputFromPublisher(ctx, publisher, publisherSkillMetricsOptions);
if (!input) continue;
const rawScore = computePublisherAbuseRawScore(input, modelConfig);
@@ -249,10 +461,11 @@ export async function finalizePublisherAbuseScoresPageInternalHandler(
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
const now = Date.now();
const cohortStats = await summarizePublisherAbuseFinalizationCohort(ctx, run);
const { meanLogPressure, stdDevLogPressure } = summarizePublisherAbuseLogPressure(
run.sumLogPressure,
run.sumSquaredLogPressure,
run.scoredPublishers,
cohortStats.sumLogPressure,
cohortStats.sumSquaredLogPressure,
cohortStats.scoredPublishers,
);
const safeStdDev = stdDevLogPressure === 0 ? 1 : stdDevLogPressure;
const page = await ctx.db
@@ -268,12 +481,19 @@ export async function finalizePublisherAbuseScoresPageInternalHandler(
};
let nominations = 0;
let finalized = 0;
let ranked = 0;
const rankedScoresSoFar = run.passCount + run.reviewCount + run.potentialBanCandidateCount;
const modelConfig = run.modelConfig;
for (const score of page.page) {
if (await isPublisherAbuseScoreExcluded(ctx, score)) {
finalized += 1;
continue;
}
const zScore = (score.logPressure - meanLogPressure) / safeStdDev;
const label = labelForPublisherAbuseZScore(zScore, modelConfig);
const rank = run.finalizedScores + finalized + 1;
const rank = rankedScoresSoFar + ranked + 1;
labelCounts[label] += 1;
ranked += 1;
finalized += 1;
await ctx.db.patch(score._id, { zScore, label, rank });
@@ -348,6 +568,7 @@ export async function runPublisherAbuseScoreRunInternalHandler(
maxPages?: number;
forceNew?: boolean;
trigger?: "cron" | "manual";
actorUserId?: Id<"users">;
},
): Promise<{ ok: true; runId: Id<"publisherAbuseScoreRuns">; pages: number; isDone: boolean }> {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
@@ -358,6 +579,7 @@ export async function runPublisherAbuseScoreRunInternalHandler(
})
: await ctx.runMutation(internal.publisherAbuse.getOrStartPublisherAbuseScoreRunInternal, {
trigger: args.trigger ?? "cron",
actorUserId: args.actorUserId,
forceNew: args.forceNew,
});
let pages = 0;
@@ -493,6 +715,84 @@ async function publisherInputFromPublisher(
};
}
async function isPublisherExcludedFromPublisherAbuse(
ctx: Pick<QueryCtx | MutationCtx, "db">,
publisher: PublisherAbuseExclusionPublisher | null | undefined,
) {
if (!publisher || publisher.kind !== "org") return false;
return await hasOfficialPublisherRow(ctx, publisher._id);
}
async function isPublisherAbuseExcludedReviewItem(
ctx: Pick<QueryCtx | MutationCtx, "db">,
item: PublisherAbuseReviewItem,
) {
return await isPublisherExcludedFromPublisherAbuse(ctx, item.publisher);
}
async function isPublisherAbuseScoreExcluded(
ctx: Pick<QueryCtx | MutationCtx, "db">,
score: Pick<ScoreDoc, "ownerPublisherId">,
) {
if (!score.ownerPublisherId) return false;
const publisher = await ctx.db.get(score.ownerPublisherId);
return await isPublisherExcludedFromPublisherAbuse(ctx, publisher);
}
async function requirePublisherAbuseNominationNotExcluded(
ctx: Pick<MutationCtx, "db">,
nomination: Doc<"publisherAbuseReviewNominations">,
) {
if (!nomination.ownerPublisherId) return;
const publisher = await ctx.db.get(nomination.ownerPublisherId);
if (!(await isPublisherExcludedFromPublisherAbuse(ctx, publisher))) return;
throw new Error("Official org publisher abuse nominations cannot be acted on.");
}
async function summarizePublisherAbuseFinalizationCohort(ctx: MutationCtx, run: ScoreRun) {
const exclusions = await summarizeOfficialPublisherAbuseScoreExclusions(ctx, run);
const scoredPublishers = Math.max(0, run.scoredPublishers - exclusions.scoredPublishers);
if (scoredPublishers === 0) {
return { scoredPublishers, sumLogPressure: 0, sumSquaredLogPressure: 0 };
}
return {
scoredPublishers,
sumLogPressure: run.sumLogPressure - exclusions.sumLogPressure,
sumSquaredLogPressure: run.sumSquaredLogPressure - exclusions.sumSquaredLogPressure,
};
}
async function summarizeOfficialPublisherAbuseScoreExclusions(ctx: MutationCtx, run: ScoreRun) {
let cursor: string | null = null;
let scoredPublishers = 0;
let sumLogPressure = 0;
let sumSquaredLogPressure = 0;
do {
const page = await ctx.db
.query("officialPublishers")
.withIndex("by_created")
.paginate({ cursor, numItems: MAX_BATCH_SIZE });
for (const officialPublisher of page.page) {
const publisher = await ctx.db.get(officialPublisher.publisherId);
if (!publisher || publisher.kind !== "org") continue;
const score = await ctx.db
.query("publisherAbuseScores")
.withIndex("by_run_and_owner_key", (q) =>
q.eq("runId", run._id).eq("ownerKey", `publisher:${officialPublisher.publisherId}`),
)
.first();
if (!score || score.publishedSkills <= 0) continue;
scoredPublishers += 1;
sumLogPressure += score.logPressure;
sumSquaredLogPressure += score.logPressure ** 2;
}
cursor = page.isDone ? null : page.continueCursor;
} while (cursor);
return { scoredPublishers, sumLogPressure, sumSquaredLogPressure };
}
type SkillMetricsForScoring = Pick<
PublisherAbuseInput,
"publishedSkills" | "totalInstalls" | "totalStars" | "totalDownloads"
@@ -604,8 +904,9 @@ async function upsertPublisherAbuseReviewNomination(
if (existing) {
const shouldReopen =
isReviewedNominationStatus(existing.status) &&
isPublisherAbuseLabelEscalation(existing.label, args.score.label);
(isReopenableNominationStatus(existing.status) &&
isPublisherAbuseLabelEscalation(existing.label, args.score.label)) ||
(await isBannedNominationForActiveOwner(ctx, existing, args.score));
await ctx.db.patch(existing._id, {
latestScoreId: args.score._id,
label: args.score.label,
@@ -703,8 +1004,25 @@ async function updateExistingPublisherAbuseReviewNominationForPass(
return existing._id;
}
function isReviewedNominationStatus(status: TriageStatus) {
return status === "reviewed_no_action" || status === "false_positive";
function isReopenableNominationStatus(status: TriageStatus) {
return (
status === "reviewed_no_action" ||
status === "false_positive" ||
status === "needs_policy_discussion" ||
status === "candidate_for_future_action"
);
}
async function isBannedNominationForActiveOwner(
ctx: Pick<MutationCtx, "db">,
nomination: Doc<"publisherAbuseReviewNominations">,
score: ScoreDoc,
) {
if (nomination.status !== "banned") return false;
const ownerUserId = score.ownerUserId ?? nomination.ownerUserId;
if (!ownerUserId) return false;
const ownerUser = await ctx.db.get(ownerUserId);
return Boolean(ownerUser && !ownerUser.deletedAt && !ownerUser.deactivatedAt);
}
function isPublisherAbuseLabelEscalation(
@@ -714,6 +1032,271 @@ function isPublisherAbuseLabelEscalation(
return publisherAbuseLabelSeverity(nextLabel) > publisherAbuseLabelSeverity(previousLabel);
}
type PublisherAbuseReviewItem = Awaited<ReturnType<typeof summarizePublisherAbuseReviewNomination>>;
type PendingPublisherAbuseReviewLabel = Exclude<PublisherAbuseLabel, "pass">;
async function getPendingPublisherAbuseReviewItemsForLabel(
ctx: QueryCtx,
args: {
status: TriageStatus;
label: PendingPublisherAbuseReviewLabel;
limit: number;
latestCompletedRunId: Id<"publisherAbuseScoreRuns"> | undefined;
},
) {
if (!args.latestCompletedRunId) {
return await getPendingPublisherAbuseReviewItemsForLabelFromLastScoredAt(ctx, args);
}
const scoreRankItems = await getPendingPublisherAbuseReviewItemsForLabelFromScoreRank(ctx, {
latestCompletedRunId: args.latestCompletedRunId,
status: args.status,
label: args.label,
limit: args.limit,
});
if (scoreRankItems.length >= args.limit) return scoreRankItems;
const lastScoredItems = await getPendingPublisherAbuseReviewItemsForLabelFromLastScoredAt(
ctx,
args,
);
return mergePublisherAbuseReviewItems(scoreRankItems, lastScoredItems, args.limit);
}
function mergePublisherAbuseReviewItems(
primary: PublisherAbuseReviewItem[],
fallback: PublisherAbuseReviewItem[],
limit: number,
) {
const items = [...primary];
const seen = new Set(primary.map((item) => item.nomination._id));
for (const item of fallback) {
if (seen.has(item.nomination._id)) continue;
items.push(item);
seen.add(item.nomination._id);
if (items.length >= limit) break;
}
return items;
}
function scoreRankScanLimit(limit: number) {
return Math.min(
limit * MAX_REVIEW_DASHBOARD_SCORE_SCAN_MULTIPLIER,
MAX_REVIEW_DASHBOARD_SCORE_SCAN,
);
}
async function getPendingPublisherAbuseReviewItemsForLabelFromScoreRank(
ctx: QueryCtx,
args: {
latestCompletedRunId: Id<"publisherAbuseScoreRuns">;
status: TriageStatus;
label: PendingPublisherAbuseReviewLabel;
limit: number;
},
) {
const items: PublisherAbuseReviewItem[] = [];
const scores = await ctx.db
.query("publisherAbuseScores")
.withIndex("by_run_and_label_and_rank", (q) =>
q.eq("runId", args.latestCompletedRunId).eq("label", args.label),
)
.order("asc")
.take(scoreRankScanLimit(args.limit));
for (const score of scores) {
const nomination = await ctx.db
.query("publisherAbuseReviewNominations")
.withIndex("by_owner_key_and_model_version", (q) =>
q.eq("ownerKey", score.ownerKey).eq("modelVersion", score.modelVersion),
)
.first();
if (
!nomination ||
nomination.status !== args.status ||
nomination.label !== args.label ||
nomination.latestScoreId !== score._id
) {
continue;
}
const item = await summarizePublisherAbuseReviewNomination(ctx, nomination);
if (!(await isVisiblePublisherAbuseReviewItem(ctx, item))) continue;
items.push(item);
if (items.length >= args.limit) break;
}
return items;
}
async function getPendingPublisherAbuseReviewItemsForLabelFromLastScoredAt(
ctx: QueryCtx,
args: { status: TriageStatus; label: PendingPublisherAbuseReviewLabel; limit: number },
) {
const items: PublisherAbuseReviewItem[] = [];
const scanLimit = args.limit * MAX_REVIEW_DASHBOARD_SCAN_MULTIPLIER;
const nominations = await ctx.db
.query("publisherAbuseReviewNominations")
.withIndex("by_status_and_label_and_last_scored_at", (q) =>
q.eq("status", args.status).eq("label", args.label),
)
.order("desc")
.take(scanLimit);
const pageItems = await summarizePublisherAbuseReviewNominations(ctx, nominations);
for (const item of pageItems) {
if (!(await isVisiblePublisherAbuseReviewItem(ctx, item))) continue;
items.push(item);
if (items.length >= args.limit) break;
}
return items;
}
async function getLatestPublisherAbuseScoreRun(ctx: QueryCtx) {
return await ctx.db
.query("publisherAbuseScoreRuns")
.withIndex("by_started_at")
.order("desc")
.first();
}
async function getRecentResolvedPublisherAbuseReviewItems(ctx: QueryCtx, limit: number) {
const resolvedStatuses: TriageStatus[] = [
"banned",
"reviewed_no_action",
"false_positive",
"needs_policy_discussion",
"candidate_for_future_action",
];
const nominations: Doc<"publisherAbuseReviewNominations">[] = [];
for (const status of resolvedStatuses) {
const page = await ctx.db
.query("publisherAbuseReviewNominations")
.withIndex("by_status_and_reviewed_at", (q) => q.eq("status", status))
.order("desc")
.take(limit * MAX_REVIEW_DASHBOARD_SCAN_MULTIPLIER);
nominations.push(...page);
}
nominations.sort((left, right) => (right.reviewedAt ?? 0) - (left.reviewedAt ?? 0));
return await summarizeVisiblePublisherAbuseReviewNominations(ctx, nominations, limit);
}
async function summarizePublisherAbuseReviewNominations(
ctx: QueryCtx,
nominations: Doc<"publisherAbuseReviewNominations">[],
) {
const items = [];
for (const nomination of nominations) {
items.push(await summarizePublisherAbuseReviewNomination(ctx, nomination));
}
return items;
}
async function summarizeVisiblePublisherAbuseReviewNominations(
ctx: QueryCtx,
nominations: Doc<"publisherAbuseReviewNominations">[],
limit?: number,
) {
const items = [];
for (const nomination of nominations) {
const item = await summarizePublisherAbuseReviewNomination(ctx, nomination);
if (await isVisiblePublisherAbuseReviewItem(ctx, item)) items.push(item);
if (limit && items.length >= limit) break;
}
return items;
}
async function summarizePublisherAbuseReviewNomination(
ctx: QueryCtx,
nomination: Doc<"publisherAbuseReviewNominations">,
) {
const score = await ctx.db.get(nomination.latestScoreId);
const publisher = nomination.ownerPublisherId
? await ctx.db.get(nomination.ownerPublisherId)
: null;
const ownerUser = nomination.ownerUserId ? await ctx.db.get(nomination.ownerUserId) : null;
const openedByRun = await ctx.db.get(nomination.openedByRunId);
return {
nomination,
latestScore: score,
publisher: publisher ? summarizePublisherForAbuseReview(publisher) : null,
ownerUser: ownerUser ? summarizeUserForAbuseReview(ownerUser) : null,
openedByRun: openedByRun ? summarizePublisherAbuseRun(openedByRun) : null,
};
}
async function isVisiblePublisherAbuseReviewItem(ctx: QueryCtx, item: PublisherAbuseReviewItem) {
return (
item.nomination.label !== "pass" &&
!item.ownerUser?.deletedAt &&
!item.ownerUser?.deactivatedAt &&
!item.publisher?.deletedAt &&
!item.publisher?.deactivatedAt &&
!(await isPublisherAbuseExcludedReviewItem(ctx, item))
);
}
function comparePublisherAbuseReviewItemsByLastScoredAt(
left: PublisherAbuseReviewItem,
right: PublisherAbuseReviewItem,
) {
if (left.nomination.lastScoredAt !== right.nomination.lastScoredAt) {
return right.nomination.lastScoredAt - left.nomination.lastScoredAt;
}
return right.nomination._id.localeCompare(left.nomination._id);
}
function summarizePublisherAbuseRun(run: Doc<"publisherAbuseScoreRuns">) {
const {
actorUserId: _actorUserId,
collectCursor: _collectCursor,
finalizeCursor: _finalizeCursor,
modelConfig: _modelConfig,
sumLogPressure: _sumLogPressure,
sumSquaredLogPressure: _sumSquaredLogPressure,
...summary
} = run;
return summary;
}
function summarizePublisherForAbuseReview(publisher: Doc<"publishers">) {
return {
_id: publisher._id,
handle: publisher.handle,
displayName: publisher.displayName,
kind: publisher.kind,
linkedUserId: publisher.linkedUserId,
publishedSkills: publisher.publishedSkills,
publishedPackages: publisher.publishedPackages,
totalInstalls: publisher.totalInstalls,
totalStars: publisher.totalStars,
totalDownloads: publisher.totalDownloads,
skillTotalInstalls: publisher.skillTotalInstalls,
skillTotalStars: publisher.skillTotalStars,
skillTotalDownloads: publisher.skillTotalDownloads,
deletedAt: publisher.deletedAt,
deactivatedAt: publisher.deactivatedAt,
};
}
function summarizeUserForAbuseReview(user: Doc<"users">) {
return {
_id: user._id,
handle: user.handle,
name: user.name,
displayName: user.displayName,
role: user.role,
image: user.image,
deletedAt: user.deletedAt,
deactivatedAt: user.deactivatedAt,
banReason: user.banReason,
};
}
function normalizeBanReason(rawReason?: string) {
const reason = rawReason?.trim();
if (!reason) return undefined;
return reason.slice(0, MAX_BAN_REASON_LENGTH);
}
function publisherAbuseLabelSeverity(label: PublisherAbuseLabel) {
if (label === "potential_ban_candidate") return 2;
if (label === "review") return 1;
+329
View File
@@ -0,0 +1,329 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from "vitest";
vi.mock("./_generated/server", () => ({
internalMutation: (def: { handler: unknown }) => ({ _handler: def.handler }),
}));
const publisherAbuseDevSeed = await import("./publisherAbuseDevSeed");
type Handler<TArgs, TResult> = (ctx: unknown, args: TArgs) => Promise<TResult>;
type Wrapped<TArgs, TResult> = { _handler: Handler<TArgs, TResult> };
const clearSeedHandler = (
publisherAbuseDevSeed.clearSeed as unknown as Wrapped<
Record<string, never>,
{
runs: number;
scores: number;
nominations: number;
events: number;
users: number;
hasMore: boolean;
}
>
)._handler;
const seedHandler = (
publisherAbuseDevSeed.seed as unknown as Wrapped<
Record<string, never>,
{ runId: string; inserted: number }
>
)._handler;
type TestDoc = Record<string, unknown> & { _id: string };
function chainEq(constraints: Record<string, unknown>) {
return {
eq(field: string, value: unknown) {
constraints[field] = value;
return chainEq(constraints);
},
};
}
function matches(doc: TestDoc, constraints: Record<string, unknown>) {
return Object.entries(constraints).every(([key, value]) => doc[key] === value);
}
function createDb(seedTables: Record<string, TestDoc[]>) {
const tables = Object.fromEntries(
Object.entries(seedTables).map(([name, docs]) => [name, [...docs]]),
);
let insertCounter = 0;
const queryCalls: Array<{
table: string;
indexName: string;
constraints: Record<string, unknown>;
}> = [];
const list = (table: string) => {
tables[table] ??= [];
return tables[table];
};
return {
tables,
queryCalls,
db: {
get: async (id: string) => {
const table = id.split(":")[0] ?? "";
return list(table).find((doc) => doc._id === id) ?? null;
},
insert: async (table: string, doc: Record<string, unknown>) => {
const id = `${table}:inserted-${insertCounter}`;
insertCounter += 1;
list(table).push({ ...doc, _id: id });
return id;
},
delete: async (id: string) => {
const table = id.split(":")[0] ?? "";
const rows = list(table);
const index = rows.findIndex((doc) => doc._id === id);
if (index !== -1) rows.splice(index, 1);
},
query: (table: string) => ({
withIndex: (indexName: string, build: (q: ReturnType<typeof chainEq>) => unknown) => {
const constraints: Record<string, unknown> = {};
build(chainEq(constraints));
queryCalls.push({ table, indexName, constraints });
const matched = () => list(table).filter((doc) => matches(doc, constraints));
return {
collect: async () => {
throw new Error("clearSeed must not collect whole tables");
},
paginate: async () => {
throw new Error("clearSeed must not use built-in pagination");
},
take: async (numItems: number) => {
return matched().slice(0, numItems);
},
};
},
}),
},
};
}
describe("publisherAbuseDevSeed.clearSeed", () => {
const previousDeployment = process.env.CONVEX_DEPLOYMENT;
const previousDevAuthDeployment = process.env.DEV_AUTH_CONVEX_DEPLOYMENT;
const previousDevAuthEnabled = process.env.DEV_AUTH_ENABLED;
const previousDevImpersonation = process.env.CLAW_HUB_ENABLE_DEV_IMPERSONATION;
afterEach(() => {
restoreEnv("CONVEX_DEPLOYMENT", previousDeployment);
restoreEnv("DEV_AUTH_CONVEX_DEPLOYMENT", previousDevAuthDeployment);
restoreEnv("DEV_AUTH_ENABLED", previousDevAuthEnabled);
restoreEnv("CLAW_HUB_ENABLE_DEV_IMPERSONATION", previousDevImpersonation);
});
it("rejects production deployments before reading tables", async () => {
process.env.CONVEX_DEPLOYMENT = "prod:wry-manatee-359";
const query = vi.fn();
await expect(clearSeedHandler({ db: { query } }, {})).rejects.toThrow(
"disabled outside local/dev deployments",
);
expect(query).not.toHaveBeenCalled();
});
it("honors the explicit fallback deployment when the primary marker is blank", async () => {
process.env.CONVEX_DEPLOYMENT = "";
process.env.DEV_AUTH_CONVEX_DEPLOYMENT = "prod:wry-manatee-359";
process.env.DEV_AUTH_ENABLED = "1";
const query = vi.fn();
await expect(clearSeedHandler({ db: { query } }, {})).rejects.toThrow(
"disabled outside local/dev deployments",
);
expect(query).not.toHaveBeenCalled();
});
it("deletes demo rows through bounded indexed pages", async () => {
process.env.CONVEX_DEPLOYMENT = "dev:admired-dodo-615";
const { db, queryCalls, tables } = createDb({
publisherAbuseScores: [
{
_id: "publisherAbuseScores:demo",
ownerKey: "user:demo-01",
handleSnapshot: "demo-abuse-pub-01",
runId: "publisherAbuseScoreRuns:demo",
},
{
_id: "publisherAbuseScores:real",
ownerKey: "user:real",
handleSnapshot: "real",
runId: "publisherAbuseScoreRuns:real",
},
],
publisherAbuseReviewNominations: [
{
_id: "publisherAbuseReviewNominations:demo",
ownerKey: "user:demo-01",
handleSnapshot: "demo-abuse-pub-01",
openedByRunId: "publisherAbuseScoreRuns:demo",
},
{
_id: "publisherAbuseReviewNominations:real",
ownerKey: "user:real",
handleSnapshot: "real",
openedByRunId: "publisherAbuseScoreRuns:real",
},
],
publisherAbuseScoreRuns: [
{ _id: "publisherAbuseScoreRuns:demo" },
{ _id: "publisherAbuseScoreRuns:real" },
],
publisherAbuseReviewEvents: [
{
_id: "publisherAbuseReviewEvents:demo",
ownerKey: "user:demo-01",
nominationId: "publisherAbuseReviewNominations:demo",
},
{
_id: "publisherAbuseReviewEvents:real",
ownerKey: "user:real",
nominationId: "publisherAbuseReviewNominations:real",
},
],
users: [
{ _id: "users:demo", handle: "demo-abuse-pub-01" },
{ _id: "users:real", handle: "real" },
],
});
await expect(clearSeedHandler({ db }, {})).resolves.toEqual({
runs: 1,
scores: 1,
nominations: 1,
events: 1,
users: 1,
hasMore: false,
});
expect(tables.publisherAbuseScores.map((doc) => doc._id)).toEqual([
"publisherAbuseScores:real",
]);
expect(tables.publisherAbuseReviewNominations.map((doc) => doc._id)).toEqual([
"publisherAbuseReviewNominations:real",
]);
expect(tables.publisherAbuseScoreRuns.map((doc) => doc._id)).toEqual([
"publisherAbuseScoreRuns:real",
]);
expect(tables.publisherAbuseReviewEvents.map((doc) => doc._id)).toEqual([
"publisherAbuseReviewEvents:real",
]);
expect(tables.users.map((doc) => doc._id)).toEqual(["users:real"]);
expect(queryCalls).toContainEqual({
table: "publisherAbuseScores",
indexName: "by_owner_key_and_created_at",
constraints: { ownerKey: "user:demo-01" },
});
expect(queryCalls).toContainEqual({
table: "publisherAbuseReviewNominations",
indexName: "by_owner_key_and_model_version",
constraints: { ownerKey: "user:demo-01" },
});
expect(queryCalls).toContainEqual({
table: "publisherAbuseReviewEvents",
indexName: "by_owner_key_and_created_at",
constraints: { ownerKey: "user:demo-01" },
});
expect(queryCalls).toContainEqual({
table: "users",
indexName: "handle",
constraints: { handle: "demo-abuse-pub-01" },
});
});
});
describe("publisherAbuseDevSeed.seed", () => {
const previousDeployment = process.env.CONVEX_DEPLOYMENT;
const previousDevAuthDeployment = process.env.DEV_AUTH_CONVEX_DEPLOYMENT;
afterEach(() => {
restoreEnv("CONVEX_DEPLOYMENT", previousDeployment);
restoreEnv("DEV_AUTH_CONVEX_DEPLOYMENT", previousDevAuthDeployment);
});
it("seeds a prod-scale nomination distribution across labels", async () => {
process.env.CONVEX_DEPLOYMENT = "";
process.env.DEV_AUTH_CONVEX_DEPLOYMENT = "dev:admired-dodo-615";
const { db, tables } = createDb({});
const result = await seedHandler({ db }, {});
const nominations = tables.publisherAbuseReviewNominations ?? [];
const pendingBan = nominations.filter(
(doc) => doc.label === "potential_ban_candidate" && doc.status === "pending",
);
const pendingReview = nominations.filter(
(doc) => doc.label === "review" && doc.status === "pending",
);
expect(pendingBan).toHaveLength(15);
expect(pendingReview).toHaveLength(124);
expect(result.inserted).toBe(nominations.length);
// Every ban candidate links a demo user so the inspector ban action is
// exercisable; review nominations do not create users.
expect(tables.users ?? []).toHaveLength(15);
});
it("clears existing demo rows before inserting repeatable seed data", async () => {
process.env.CONVEX_DEPLOYMENT = "";
process.env.DEV_AUTH_CONVEX_DEPLOYMENT = "dev:admired-dodo-615";
const { db, tables } = createDb({
publisherAbuseScores: [
{
_id: "publisherAbuseScores:old-demo",
ownerKey: "user:demo-01",
handleSnapshot: "demo-abuse-pub-01",
runId: "publisherAbuseScoreRuns:old-demo",
},
],
publisherAbuseReviewNominations: [
{
_id: "publisherAbuseReviewNominations:old-demo",
ownerKey: "user:demo-01",
handleSnapshot: "demo-abuse-pub-01",
openedByRunId: "publisherAbuseScoreRuns:old-demo",
},
],
publisherAbuseScoreRuns: [{ _id: "publisherAbuseScoreRuns:old-demo" }],
publisherAbuseReviewEvents: [
{
_id: "publisherAbuseReviewEvents:old-demo",
ownerKey: "user:demo-01",
nominationId: "publisherAbuseReviewNominations:old-demo",
},
],
users: [{ _id: "users:old-demo", handle: "demo-abuse-pub-01" }],
});
await seedHandler({ db }, {});
expect(tables.publisherAbuseScores.map((doc) => doc._id)).not.toContain(
"publisherAbuseScores:old-demo",
);
expect(tables.publisherAbuseReviewNominations.map((doc) => doc._id)).not.toContain(
"publisherAbuseReviewNominations:old-demo",
);
expect(tables.publisherAbuseScoreRuns.map((doc) => doc._id)).not.toContain(
"publisherAbuseScoreRuns:old-demo",
);
expect(tables.publisherAbuseReviewEvents.map((doc) => doc._id)).not.toContain(
"publisherAbuseReviewEvents:old-demo",
);
expect(tables.users.map((doc) => doc._id)).not.toContain("users:old-demo");
expect(tables.users.filter((doc) => doc.handle === "demo-abuse-pub-01")).toHaveLength(1);
expect(tables.users).toHaveLength(15);
expect(tables.publisherAbuseReviewNominations).toHaveLength(145);
});
});
function restoreEnv(name: string, value: string | undefined) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
}
+500
View File
@@ -0,0 +1,500 @@
import type { Doc, Id } from "./_generated/dataModel";
import type { MutationCtx } from "./_generated/server";
// DEV-ONLY seed: use the un-wrapped mutation builder (not convex/functions.ts) so
// inserting/deleting demo rows does NOT fire table triggers. The users digest-sync
// trigger runs a paginated query, and Convex allows only one paginated query per
// mutation, so deleting several linked demo users through the wrapped builder fails.
// Demo rows have no real packages/skills, so skipping digest sync is correct here.
import { internalMutation } from "./_generated/server";
import { assertLocalDevSeedAllowed } from "./lib/devSeed";
import {
computePublisherAbuseRawScore,
DEFAULT_PUBLISHER_ABUSE_MODEL_CONFIG,
PUBLISHER_ABUSE_MODEL_VERSION,
type PublisherAbuseLabel,
} from "./lib/publisherAbuseScoring";
// DEV-ONLY seed for the publisher-abuse review dashboard. It inserts one
// completed score run plus a spread of synthetic scores/nominations so every
// dashboard tab renders with realistic rows. All synthetic rows use the
// "demo-" prefix on handle/ownerKey so `clearSeed` can remove them precisely.
const DEMO_HANDLE_PREFIX = "demo-abuse-pub-";
const DEMO_OWNER_KEY_PREFIX = "user:demo-";
const CLEAR_SEED_BATCH_SIZE = 100;
type TriageStatus =
| "pending"
| "reviewed_no_action"
| "false_positive"
| "needs_policy_discussion"
| "candidate_for_future_action";
type SeedPublisher = {
index: number;
label: PublisherAbuseLabel;
status: TriageStatus;
zScore: number;
publishedSkills: number;
totalInstalls: number;
totalStars: number;
totalDownloads: number;
reasonCodes: string[];
notes?: string;
// When true, also create an isolated demo user account and link it so the
// inspector's "Ban user" action is enabled and exercisable in dev.
linkUser?: boolean;
};
// Prod-scale synthetic distribution so every dashboard tab renders with realistic
// volume: 15 potential-ban candidates and 124 review nominations (both pending),
// plus a small resolved/pass set for the Resolved tab. Counts mirror the reported
// production review queue. Rows are deterministic (no randomness) so tests can
// assert the distribution and clearSeed stays reproducible.
const BAN_CANDIDATE_COUNT = 15;
const REVIEW_PENDING_COUNT = 124;
const BAN_CANDIDATE_REASON_CODES = [
"high_catalog_volume",
"extreme_volume_low_engagement",
"low_installs_per_skill",
"low_stars_per_skill",
"low_downloads_per_skill",
];
const REVIEW_REASON_VARIANTS: string[][] = [
["high_catalog_volume", "low_installs_per_skill", "low_stars_per_skill"],
["high_catalog_volume", "low_installs_per_skill"],
["high_catalog_volume", "low_stars_per_skill", "low_downloads_per_skill"],
["high_catalog_volume", "low_installs_per_skill", "low_downloads_per_skill"],
];
// Resolved + pass anchors keep the Resolved tab populated and exercise the
// inspector's notes rendering. None link a demo user, so the only seeded demo
// users are the 15 pending ban candidates.
const RESOLVED_AND_PASS_PUBLISHERS: Array<Omit<SeedPublisher, "index">> = [
{
label: "potential_ban_candidate",
status: "needs_policy_discussion",
zScore: 2.75,
publishedSkills: 2600,
totalInstalls: 210,
totalStars: 28,
totalDownloads: 6400,
reasonCodes: BAN_CANDIDATE_REASON_CODES,
notes: "Escalated to policy: borderline catalog-stuffing pattern, awaiting decision.",
},
{
label: "review",
status: "false_positive",
zScore: 1.8,
publishedSkills: 340,
totalInstalls: 520,
totalStars: 40,
totalDownloads: 48000,
reasonCodes: ["high_catalog_volume", "low_installs_per_skill"],
notes: "Confirmed legitimate bulk publisher; cleared after manual spot-check.",
},
{
label: "review",
status: "candidate_for_future_action",
zScore: 2.0,
publishedSkills: 480,
totalInstalls: 360,
totalStars: 17,
totalDownloads: 29000,
reasonCodes: ["high_catalog_volume", "low_installs_per_skill", "low_stars_per_skill"],
notes: "Watchlist: revisit if catalog keeps growing without engagement.",
},
{
label: "review",
status: "reviewed_no_action",
zScore: 1.6,
publishedSkills: 290,
totalInstalls: 470,
totalStars: 33,
totalDownloads: 31000,
reasonCodes: ["high_catalog_volume", "low_installs_per_skill"],
notes: "Reviewed: engagement within acceptable range for catalog size.",
},
{
label: "pass",
status: "reviewed_no_action",
zScore: 0.4,
publishedSkills: 120,
totalInstalls: 9800,
totalStars: 540,
totalDownloads: 210000,
reasonCodes: [],
notes: "Healthy engagement per skill; no action needed.",
},
{
label: "pass",
status: "reviewed_no_action",
zScore: 0.2,
publishedSkills: 64,
totalInstalls: 7200,
totalStars: 410,
totalDownloads: 150000,
reasonCodes: [],
notes: "Strong installs and stars per skill; clearly legitimate.",
},
];
function roundToTwo(value: number): number {
return Math.round(value * 100) / 100;
}
// Ban candidates carry the highest z-scores (3.9 → 2.55) and link demo users so
// the inspector ban action is exercisable; review nominations span the "on the
// brink" band (2.4 → 1.3). Metrics vary per row so the inspector looks realistic.
function buildSeedPublishers(): SeedPublisher[] {
const publishers: SeedPublisher[] = [];
let index = 1;
for (let i = 0; i < BAN_CANDIDATE_COUNT; i += 1) {
const fraction = i / (BAN_CANDIDATE_COUNT - 1);
publishers.push({
index,
label: "potential_ban_candidate",
status: "pending",
zScore: roundToTwo(3.9 - fraction * 1.35),
publishedSkills: 4200 - i * 170,
totalInstalls: 130 + (i % 6) * 16,
totalStars: 15 + (i % 8) * 2,
totalDownloads: 9800 - i * 300,
reasonCodes: BAN_CANDIDATE_REASON_CODES,
linkUser: true,
});
index += 1;
}
for (let i = 0; i < REVIEW_PENDING_COUNT; i += 1) {
const fraction = i / (REVIEW_PENDING_COUNT - 1);
publishers.push({
index,
label: "review",
status: "pending",
zScore: roundToTwo(2.4 - fraction * 1.1),
publishedSkills: 650 - i * 3,
totalInstalls: 300 + (i % 9) * 30,
totalStars: 14 + (i % 11) * 3,
totalDownloads: 26000 + (i % 13) * 1500,
reasonCodes: REVIEW_REASON_VARIANTS[i % REVIEW_REASON_VARIANTS.length],
});
index += 1;
}
for (const publisher of RESOLVED_AND_PASS_PUBLISHERS) {
publishers.push({ index, ...publisher });
index += 1;
}
return publishers;
}
const SEED_PUBLISHERS: SeedPublisher[] = buildSeedPublishers();
const SCANNED_PUBLISHERS = 194_083;
const SCORED_PUBLISHERS = 10_349;
const DAY_MS = 24 * 60 * 60 * 1000;
const HOUR_MS = 60 * 60 * 1000;
function paddedIndex(index: number): string {
return index.toString().padStart(2, "0");
}
function isDemoHandle(handle: string): boolean {
return handle.startsWith(DEMO_HANDLE_PREFIX);
}
function isDemoOwnerKey(ownerKey: string): boolean {
return ownerKey.startsWith(DEMO_OWNER_KEY_PREFIX);
}
function demoHandle(index: number): string {
return `${DEMO_HANDLE_PREFIX}${paddedIndex(index)}`;
}
function demoOwnerKey(index: number): string {
return `${DEMO_OWNER_KEY_PREFIX}${paddedIndex(index)}`;
}
const DEMO_HANDLES = SEED_PUBLISHERS.map((publisher) => demoHandle(publisher.index));
const DEMO_OWNER_KEYS = SEED_PUBLISHERS.map((publisher) => demoOwnerKey(publisher.index));
type ClearSeedCtx = Pick<MutationCtx, "db">;
type ClearSeedResult = {
runs: number;
scores: number;
nominations: number;
events: number;
users: number;
hasMore: boolean;
};
export const seed = internalMutation({
args: {},
handler: async (ctx): Promise<{ runId: Id<"publisherAbuseScoreRuns">; inserted: number }> => {
assertLocalDevSeedAllowed("Publisher abuse");
await clearDemoRows(ctx);
const now = Date.now();
const startedAt = now - 2 * HOUR_MS;
const completedAt = now - HOUR_MS;
const labelCounts: Record<PublisherAbuseLabel, number> = {
pass: 0,
review: 0,
potential_ban_candidate: 0,
};
let nominatedPublishers = 0;
let sumLogPressure = 0;
let sumSquaredLogPressure = 0;
for (const publisher of SEED_PUBLISHERS) {
labelCounts[publisher.label] += 1;
if (publisher.label !== "pass") nominatedPublishers += 1;
const raw = computePublisherAbuseRawScore({
ownerKey: demoOwnerKey(publisher.index),
handleSnapshot: demoHandle(publisher.index),
publishedSkills: publisher.publishedSkills,
totalInstalls: publisher.totalInstalls,
totalStars: publisher.totalStars,
totalDownloads: publisher.totalDownloads,
});
sumLogPressure += raw.logPressure;
sumSquaredLogPressure += raw.logPressure ** 2;
}
const meanLogPressure = sumLogPressure / SEED_PUBLISHERS.length;
const variance = Math.max(
0,
sumSquaredLogPressure / SEED_PUBLISHERS.length - meanLogPressure ** 2,
);
const stdDevLogPressure = Math.sqrt(variance);
const runId = await ctx.db.insert("publisherAbuseScoreRuns", {
modelVersion: PUBLISHER_ABUSE_MODEL_VERSION,
modelConfig: DEFAULT_PUBLISHER_ABUSE_MODEL_CONFIG,
trigger: "manual",
status: "completed",
phase: "completed",
startedAt,
completedAt,
updatedAt: completedAt,
scannedPublishers: SCANNED_PUBLISHERS,
scoredPublishers: SCORED_PUBLISHERS,
finalizedScores: SCORED_PUBLISHERS,
nominatedPublishers,
passCount: labelCounts.pass,
reviewCount: labelCounts.review,
potentialBanCandidateCount: labelCounts.potential_ban_candidate,
sumLogPressure,
sumSquaredLogPressure,
meanLogPressure,
stdDevLogPressure,
});
let rank = 1;
for (const publisher of SEED_PUBLISHERS) {
const handle = demoHandle(publisher.index);
const ownerKey = demoOwnerKey(publisher.index);
const raw = computePublisherAbuseRawScore({
ownerKey,
handleSnapshot: handle,
publishedSkills: publisher.publishedSkills,
totalInstalls: publisher.totalInstalls,
totalStars: publisher.totalStars,
totalDownloads: publisher.totalDownloads,
});
const lastScoredAt = completedAt;
const openedAt = completedAt;
const reviewed = publisher.status !== "pending";
const reviewedAt = reviewed ? completedAt + publisher.index * 60_000 : undefined;
const updatedAt = reviewedAt ?? completedAt;
const ownerUserId = publisher.linkUser
? await ctx.db.insert("users", {
handle,
name: `Demo Abuse Publisher ${paddedIndex(publisher.index)}`,
role: "user",
createdAt: now - DAY_MS,
updatedAt: now - DAY_MS,
})
: undefined;
const scoreId = await ctx.db.insert("publisherAbuseScores", {
runId,
ownerKey,
ownerPublisherId: undefined,
ownerUserId,
handleSnapshot: handle,
modelVersion: PUBLISHER_ABUSE_MODEL_VERSION,
label: publisher.label,
rank,
pressure: raw.pressure,
logPressure: raw.logPressure,
zScore: publisher.zScore,
publishedSkills: raw.publishedSkills,
totalInstalls: raw.totalInstalls,
totalStars: raw.totalStars,
totalDownloads: raw.totalDownloads,
installsPerSkill: raw.installsPerSkill,
starsPerSkill: raw.starsPerSkill,
downloadsPerSkill: raw.downloadsPerSkill,
reasonCodes: publisher.reasonCodes,
createdAt: now - DAY_MS,
});
rank += 1;
await ctx.db.insert("publisherAbuseReviewNominations", {
ownerKey,
ownerPublisherId: undefined,
ownerUserId,
handleSnapshot: handle,
latestScoreId: scoreId,
modelVersion: PUBLISHER_ABUSE_MODEL_VERSION,
label: publisher.label,
status: publisher.status,
openedAt,
openedByRunId: runId,
lastScoredAt,
reviewedByUserId: undefined,
reviewedAt,
notes: publisher.notes,
updatedAt,
});
}
return { runId, inserted: SEED_PUBLISHERS.length };
},
});
export const clearSeed = internalMutation({
args: {},
handler: async (ctx): Promise<ClearSeedResult> => {
assertLocalDevSeedAllowed("Publisher abuse");
return await clearDemoRows(ctx);
},
});
async function clearDemoRows(ctx: ClearSeedCtx): Promise<ClearSeedResult> {
let runs = 0;
let scores = 0;
let nominations = 0;
let events = 0;
let users = 0;
let hasMore = false;
const demoRunIds = new Set<Id<"publisherAbuseScoreRuns">>();
for (const ownerKey of DEMO_OWNER_KEYS) {
const page = await queryDemoScoresPage(ctx, ownerKey);
hasMore ||= !page.isDone;
for (const score of page.page) {
if (!isDemoOwnerKey(score.ownerKey) && !isDemoHandle(score.handleSnapshot)) continue;
demoRunIds.add(score.runId);
await ctx.db.delete(score._id);
scores += 1;
}
}
for (const ownerKey of DEMO_OWNER_KEYS) {
const page = await queryDemoNominationsPage(ctx, ownerKey);
hasMore ||= !page.isDone;
for (const nomination of page.page) {
if (!isDemoOwnerKey(nomination.ownerKey) && !isDemoHandle(nomination.handleSnapshot)) {
continue;
}
demoRunIds.add(nomination.openedByRunId);
await ctx.db.delete(nomination._id);
nominations += 1;
}
}
for (const ownerKey of DEMO_OWNER_KEYS) {
const page = await queryDemoEventsPage(ctx, ownerKey);
hasMore ||= !page.isDone;
for (const event of page.page) {
if (!isDemoOwnerKey(event.ownerKey)) continue;
await ctx.db.delete(event._id);
events += 1;
}
}
for (const runId of demoRunIds) {
const run = await ctx.db.get(runId);
if (!run) continue;
await ctx.db.delete(runId);
runs += 1;
}
for (const handle of DEMO_HANDLES) {
const page = await queryDemoUsersPage(ctx, handle);
hasMore ||= !page.isDone;
for (const user of page.page) {
if (!user.handle || !isDemoHandle(user.handle)) continue;
await ctx.db.delete(user._id);
users += 1;
}
}
return { runs, scores, nominations, events, users, hasMore };
}
async function queryDemoScoresPage(
ctx: ClearSeedCtx,
ownerKey: string,
): Promise<{ page: Doc<"publisherAbuseScores">[]; isDone: boolean }> {
const rows = await ctx.db
.query("publisherAbuseScores")
.withIndex("by_owner_key_and_created_at", (q) => q.eq("ownerKey", ownerKey))
.take(CLEAR_SEED_BATCH_SIZE + 1);
return {
page: rows.slice(0, CLEAR_SEED_BATCH_SIZE),
isDone: rows.length <= CLEAR_SEED_BATCH_SIZE,
};
}
async function queryDemoNominationsPage(
ctx: ClearSeedCtx,
ownerKey: string,
): Promise<{ page: Doc<"publisherAbuseReviewNominations">[]; isDone: boolean }> {
const rows = await ctx.db
.query("publisherAbuseReviewNominations")
.withIndex("by_owner_key_and_model_version", (q) => q.eq("ownerKey", ownerKey))
.take(CLEAR_SEED_BATCH_SIZE + 1);
return {
page: rows.slice(0, CLEAR_SEED_BATCH_SIZE),
isDone: rows.length <= CLEAR_SEED_BATCH_SIZE,
};
}
async function queryDemoEventsPage(
ctx: ClearSeedCtx,
ownerKey: string,
): Promise<{ page: Doc<"publisherAbuseReviewEvents">[]; isDone: boolean }> {
const rows = await ctx.db
.query("publisherAbuseReviewEvents")
.withIndex("by_owner_key_and_created_at", (q) => q.eq("ownerKey", ownerKey))
.take(CLEAR_SEED_BATCH_SIZE + 1);
return {
page: rows.slice(0, CLEAR_SEED_BATCH_SIZE),
isDone: rows.length <= CLEAR_SEED_BATCH_SIZE,
};
}
async function queryDemoUsersPage(
ctx: ClearSeedCtx,
handle: string,
): Promise<{ page: Doc<"users">[]; isDone: boolean }> {
const rows = await ctx.db
.query("users")
.withIndex("handle", (q) => q.eq("handle", handle))
.take(CLEAR_SEED_BATCH_SIZE + 1);
return {
page: rows.slice(0, CLEAR_SEED_BATCH_SIZE),
isDone: rows.length <= CLEAR_SEED_BATCH_SIZE,
};
}
+777 -23
View File
File diff suppressed because it is too large Load Diff
+434 -8
View File
@@ -1,5 +1,6 @@
import { paginationOptsValidator } from "convex/server";
import { ConvexError, v } from "convex/values";
import { internal } from "./_generated/api";
import type { Doc, Id } from "./_generated/dataModel";
import type { MutationCtx, QueryCtx } from "./_generated/server";
import { internalMutation, internalQuery, mutation, query } from "./functions";
@@ -10,6 +11,12 @@ import {
formatReservedPublicOwnerHandleMessage,
isReservedPublicOwnerHandle,
} from "./lib/publicRouteReservations";
import {
buildGitHubSkillCatalogDisplay,
type GitHubSkillCatalogDisplay,
type GitHubSkillCatalogItem,
type GitHubSkillCatalogSource,
} from "./lib/publisherCatalogDisplay";
import {
canAccessPublisherOwnerScope,
ensurePersonalPublisherForUser,
@@ -46,10 +53,14 @@ type PublisherPublishedItem = {
displayName: string;
downloads: number;
};
type PublisherPublishedPreviewItem = PublisherPublishedItem & {
installs: number;
};
type PublisherCatalogItem = {
_id: Id<"skills"> | Id<"packages">;
kind: "skill" | "plugin";
slug?: string;
displayName: string;
summary: string | null;
// Mirrors `skills.icon` for `kind: "skill"` items so the publisher
@@ -62,6 +73,11 @@ type PublisherCatalogItem = {
stars: number;
isOfficial: boolean;
updatedAt: number;
sourceBacked?: boolean;
sourceId?: Id<"githubSkillSources"> | null;
sourceRepo?: string | null;
sourcePath?: string | null;
sourceVerifiedCommit?: string | null;
};
type PublisherCatalogSort = "downloads" | "recent";
@@ -81,6 +97,10 @@ type PublisherListSummary = {
item: PublisherListItem;
};
function isPublicPublishedSkill(skill: Doc<"skills">) {
return !skill.softDeletedAt && (!skill.moderationStatus || skill.moderationStatus === "active");
}
type PublicPublisherKindFilter = "user" | "org";
type PublisherListCounts = {
all: number;
@@ -171,7 +191,7 @@ async function getPublisherPublishedRows(
)
.collect(),
]);
return { skills, packages };
return { skills: skills.filter(isPublicPublishedSkill), packages };
}
async function getPublisherPublishedPreviewRows(
@@ -181,20 +201,20 @@ async function getPublisherPublishedPreviewRows(
const [skills, packages] = await Promise.all([
ctx.db
.query("skills")
.withIndex("by_owner_publisher_active_downloads", (q) =>
.withIndex("by_owner_publisher_active_installs", (q) =>
q.eq("ownerPublisherId", publisherId).eq("softDeletedAt", undefined),
)
.order("desc")
.take(PUBLISHER_LIST_PREVIEW_LIMIT),
ctx.db
.query("packages")
.withIndex("by_owner_publisher_active_downloads", (q) =>
.withIndex("by_owner_publisher_active_installs", (q) =>
q.eq("ownerPublisherId", publisherId).eq("softDeletedAt", undefined),
)
.order("desc")
.take(PUBLISHER_LIST_PREVIEW_LIMIT),
]);
return { skills, packages };
return { skills: skills.filter(isPublicPublishedSkill), packages };
}
function getIndexedPublisherStatsFromRows(rows: PublisherPublishedRows): PublisherListStats {
@@ -218,20 +238,33 @@ function getIndexedPublisherStatsFromRows(rows: PublisherPublishedRows): Publish
}
function getPublisherPublishedItems(rows: PublisherPublishedRows): PublisherPublishedItem[] {
return [
const items: PublisherPublishedPreviewItem[] = [
...rows.skills.map((skill) => ({
kind: "skill" as const,
displayName: skill.displayName,
downloads: readCanonicalStat(skill, "downloads"),
installs: readCanonicalStat(skill, "installsAllTime"),
})),
...rows.packages.map((pkg) => ({
kind: pkg.family === "skill" ? ("skill" as const) : ("plugin" as const),
displayName: pkg.displayName,
downloads: pkg.stats.downloads,
installs: pkg.stats.installs,
})),
]
.sort((a, b) => b.downloads - a.downloads || a.displayName.localeCompare(b.displayName))
.slice(0, 3);
];
return items
.sort(
(a, b) =>
b.installs - a.installs ||
b.downloads - a.downloads ||
a.displayName.localeCompare(b.displayName),
)
.slice(0, 3)
.map((item) => ({
kind: item.kind,
displayName: item.displayName,
downloads: item.downloads,
}));
}
function buildPluginDetailHref(name: string) {
@@ -277,6 +310,7 @@ function getPublisherCatalogItems(
...rows.skills.map((skill) => ({
_id: skill._id,
kind: "skill" as const,
slug: skill.slug,
displayName: skill.displayName,
summary: skill.summary ?? null,
icon: skill.icon ?? null,
@@ -285,6 +319,10 @@ function getPublisherCatalogItems(
stars: readCanonicalStat(skill, "stars"),
isOfficial: publisherOfficial || Boolean(skill.badges?.official),
updatedAt: skill.updatedAt,
sourceBacked: skill.installKind === "github",
sourceId: skill.githubSourceId ?? null,
sourceRepo: null,
sourcePath: skill.githubPath ?? null,
})),
...rows.packages.map((pkg) => ({
_id: pkg._id,
@@ -301,6 +339,40 @@ function getPublisherCatalogItems(
].sort(comparePublisherCatalogItems(sort));
}
function toGitHubSkillCatalogSource(source: Doc<"githubSkillSources">): GitHubSkillCatalogSource {
return {
_id: source._id,
repo: source.repo,
displayManifestStatus: source.displayManifestStatus,
displayManifest: source.displayManifest,
};
}
function toGitHubSkillCatalogItem(
item: PublisherCatalogItem,
sourceById: Map<string, Doc<"githubSkillSources">>,
): GitHubSkillCatalogItem {
const sourceId = item.sourceId ? String(item.sourceId) : null;
return {
_id: String(item._id),
kind: item.kind,
slug: item.slug ?? null,
displayName: item.displayName,
summary: item.summary,
icon: item.icon,
href: item.href,
downloads: item.downloads,
stars: item.stars,
isOfficial: item.isOfficial,
updatedAt: item.updatedAt,
sourceBacked: item.sourceBacked ?? false,
sourceId,
sourceRepo: sourceId ? (sourceById.get(sourceId)?.repo ?? null) : null,
sourcePath: item.sourcePath ?? null,
sourceVerifiedCommit: item.sourceVerifiedCommit ?? null,
};
}
async function toPublisherListItem(
ctx: Pick<QueryCtx, "db">,
publisher: Doc<"publishers">,
@@ -881,6 +953,80 @@ async function createOrgPublisherForUser(
};
}
async function deleteOrgPublisherForOwner(
ctx: MutationCtx,
args: {
actorUserId: Id<"users">;
publisherId: Id<"publishers">;
deletedAt: number;
source: "settings" | "account.delete";
},
) {
const actor = await ctx.db.get(args.actorUserId);
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new ConvexError("Unauthorized");
const publisher = await ctx.db.get(args.publisherId);
if (!publisher || publisher.kind !== "org" || publisher.deletedAt || publisher.deactivatedAt) {
throw new ConvexError("Publisher not found");
}
const membership = await getPublisherMembership(ctx, publisher._id, args.actorUserId);
if (!membership || membership.role !== "owner") {
throw new ConvexError("Only org owners can delete an organization");
}
await ctx.db.patch(publisher._id, {
deletedAt: args.deletedAt,
deactivatedAt: args.deletedAt,
updatedAt: args.deletedAt,
});
const skillsResult = (await ctx.runMutation(
internal.skills.applyPublisherDeletionToOwnedSkillsBatchInternal,
{
ownerPublisherId: publisher._id,
actorUserId: args.actorUserId,
deletedAt: args.deletedAt,
cursor: undefined,
},
)) as { hiddenCount?: number; scheduled?: boolean };
const packagesResult = (await ctx.runMutation(
internal.packages.applyPublisherDeletionToOwnedPackagesBatchInternal,
{
ownerPublisherId: publisher._id,
actorUserId: args.actorUserId,
deletedAt: args.deletedAt,
cursor: undefined,
},
)) as { deletedCount?: number; revokedTokenCount?: number; scheduled?: boolean };
await ctx.db.insert("auditLogs", {
actorUserId: args.actorUserId,
action: "publisher.org.delete",
targetType: "publisher",
targetId: publisher._id,
metadata: {
handle: publisher.handle,
source: args.source,
hiddenSkills: skillsResult.hiddenCount ?? 0,
deletedPackages: packagesResult.deletedCount ?? 0,
revokedPackageTokens: packagesResult.revokedTokenCount ?? 0,
scheduled: Boolean(skillsResult.scheduled) || Boolean(packagesResult.scheduled) || undefined,
},
createdAt: args.deletedAt,
});
return {
ok: true as const,
publisherId: publisher._id,
handle: publisher.handle,
hiddenSkills: skillsResult.hiddenCount ?? 0,
deletedPackages: packagesResult.deletedCount ?? 0,
revokedPackageTokens: packagesResult.revokedTokenCount ?? 0,
scheduled: Boolean(skillsResult.scheduled) || Boolean(packagesResult.scheduled),
};
}
export const getByIdInternal = internalQuery({
args: { publisherId: v.id("publishers") },
handler: async (ctx, args) => await ctx.db.get(args.publisherId),
@@ -1175,6 +1321,44 @@ export const listPublishedPage = query({
},
});
export const getPublishedDisplayManifest = query({
args: {
handle: v.string(),
kind: v.optional(v.union(v.literal("skill"), v.literal("plugin"))),
sort: v.optional(v.union(v.literal("downloads"), v.literal("recent"))),
},
handler: async (ctx, args): Promise<GitHubSkillCatalogDisplay | null> => {
if (args.kind === "plugin") return null;
const publisher = await getPublisherByHandle(ctx, args.handle);
if (!publisher || publisher.deletedAt || publisher.deactivatedAt) return null;
const sources = await ctx.db
.query("githubSkillSources")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", publisher._id))
.collect();
if (sources.length === 0) return null;
const rows = await getPublisherPublishedRows(ctx, publisher._id);
if (!args.kind && rows.packages.length > 0) return null;
const sourceById = new Map(sources.map((source) => [String(source._id), source]));
const items = getPublisherCatalogItems(
publisher,
rows,
await isOfficialPublisher(ctx, publisher),
args.sort ?? "downloads",
)
.filter((item) => !args.kind || item.kind === args.kind)
.map((item) => toGitHubSkillCatalogItem(item, sourceById));
return buildGitHubSkillCatalogDisplay({
sources: sources.map(toGitHubSkillCatalogSource),
items,
});
},
});
export const listPublic = query({
args: {
limit: v.optional(v.number()),
@@ -1350,6 +1534,21 @@ export const createOrg = mutation({
},
});
export const deleteOrg = mutation({
args: {
publisherId: v.id("publishers"),
},
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx);
return await deleteOrgPublisherForOwner(ctx, {
actorUserId: userId,
publisherId: args.publisherId,
deletedAt: Date.now(),
source: "settings",
});
},
});
export const updateProfile = mutation({
args: {
publisherId: v.id("publishers"),
@@ -1523,6 +1722,174 @@ export const removeOrgPublisherMemberInternal = internalMutation({
},
});
export const listOfficialPublishersInternal = internalQuery({
args: {
actorUserId: v.id("users"),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId);
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new ConvexError("Unauthorized");
assertAdmin(actor);
const rows = await ctx.db
.query("officialPublishers")
.withIndex("by_created", (q) => q)
.order("asc")
.collect();
const items = await Promise.all(
rows.map(async (row) => {
const [publisher, createdBy] = await Promise.all([
ctx.db.get(row.publisherId),
row.createdByUserId ? ctx.db.get(row.createdByUserId) : Promise.resolve(null),
]);
return {
officialPublisherId: row._id,
publisherId: row.publisherId,
handle: publisher?.handle ?? null,
displayName: publisher?.displayName ?? null,
kind: publisher?.kind ?? null,
active: Boolean(publisher && !publisher.deletedAt && !publisher.deactivatedAt),
reason: row.reason ?? null,
createdByUserId: row.createdByUserId ?? null,
createdByHandle: createdBy?.handle ?? null,
createdAt: row.createdAt,
updatedAt: row.updatedAt,
};
}),
);
return { ok: true as const, items };
},
});
export const addOfficialPublisherInternal = internalMutation({
args: {
actorUserId: v.id("users"),
handle: v.string(),
reason: v.string(),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId);
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new ConvexError("Unauthorized");
assertAdmin(actor);
const handle = normalizePublisherHandle(args.handle);
if (!handle) throw new ConvexError("Publisher handle is required");
const reason = args.reason.trim();
if (!reason) throw new ConvexError("Reason is required");
if (reason.length > 500) throw new ConvexError("Reason too long (max 500 chars)");
const publisher = await getPublisherByHandle(ctx, handle);
if (!publisher || publisher.deletedAt || publisher.deactivatedAt) {
throw new ConvexError(`Publisher "@${handle}" not found`);
}
if (publisher.kind !== "org") {
throw new ConvexError("Only org publishers can be marked official");
}
const existing = await ctx.db
.query("officialPublishers")
.withIndex("by_publisher", (q) => q.eq("publisherId", publisher._id))
.unique();
if (existing) {
return {
ok: true as const,
added: false,
publisherId: publisher._id,
handle: publisher.handle,
officialPublisherId: existing._id,
};
}
const now = Date.now();
const officialPublisherId = await ctx.db.insert("officialPublishers", {
publisherId: publisher._id,
reason,
createdByUserId: args.actorUserId,
createdAt: now,
updatedAt: now,
});
await ctx.db.insert("auditLogs", {
actorUserId: args.actorUserId,
action: "publisher.official.add",
targetType: "publisher",
targetId: publisher._id,
metadata: {
handle: publisher.handle,
reason,
},
createdAt: now,
});
return {
ok: true as const,
added: true,
publisherId: publisher._id,
handle: publisher.handle,
officialPublisherId,
};
},
});
export const removeOfficialPublisherInternal = internalMutation({
args: {
actorUserId: v.id("users"),
handle: v.string(),
reason: v.string(),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId);
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new ConvexError("Unauthorized");
assertAdmin(actor);
const handle = normalizePublisherHandle(args.handle);
if (!handle) throw new ConvexError("Publisher handle is required");
const reason = args.reason.trim();
if (!reason) throw new ConvexError("Reason is required");
if (reason.length > 500) throw new ConvexError("Reason too long (max 500 chars)");
const publisher = await getPublisherByHandle(ctx, handle);
if (!publisher || publisher.deletedAt || publisher.deactivatedAt) {
throw new ConvexError(`Publisher "@${handle}" not found`);
}
const existing = await ctx.db
.query("officialPublishers")
.withIndex("by_publisher", (q) => q.eq("publisherId", publisher._id))
.unique();
if (!existing) {
return {
ok: true as const,
removed: false,
publisherId: publisher._id,
handle: publisher.handle,
};
}
const now = Date.now();
await ctx.db.delete(existing._id);
await ctx.db.insert("auditLogs", {
actorUserId: args.actorUserId,
action: "publisher.official.remove",
targetType: "publisher",
targetId: publisher._id,
metadata: {
handle: publisher.handle,
reason,
officialPublisherId: existing._id,
},
createdAt: now,
});
return {
ok: true as const,
removed: true,
publisherId: publisher._id,
handle: publisher.handle,
officialPublisherId: existing._id,
};
},
});
export const createOrgPublisherForUserInternal = internalMutation({
args: {
actorUserId: v.id("users"),
@@ -1532,6 +1899,65 @@ export const createOrgPublisherForUserInternal = internalMutation({
handler: async (ctx, args) => await createOrgPublisherForUser(ctx, args),
});
async function hasOtherActiveOwner(
ctx: MutationCtx,
members: Array<Doc<"publisherMembers">>,
actorUserId: Id<"users">,
) {
for (const member of members) {
if (member.role !== "owner" || member.userId === actorUserId) continue;
const user = await ctx.db.get(member.userId);
if (user && !user.deletedAt && !user.deactivatedAt) return true;
}
return false;
}
export const deleteSoleOwnerOrgsForAccountDeletionInternal = internalMutation({
args: {
actorUserId: v.id("users"),
deletedAt: v.number(),
},
handler: async (ctx, args) => {
const memberships = await ctx.db
.query("publisherMembers")
.withIndex("by_user", (q) => q.eq("userId", args.actorUserId))
.collect();
let deletedOrgs = 0;
let hiddenSkills = 0;
let deletedPackages = 0;
for (const membership of memberships) {
if (membership.role !== "owner") continue;
const publisher = await ctx.db.get(membership.publisherId);
if (
!publisher ||
publisher.kind !== "org" ||
publisher.deletedAt ||
publisher.deactivatedAt
) {
continue;
}
const members = await ctx.db
.query("publisherMembers")
.withIndex("by_publisher", (q) => q.eq("publisherId", publisher._id))
.collect();
if (await hasOtherActiveOwner(ctx, members, args.actorUserId)) continue;
const result = await deleteOrgPublisherForOwner(ctx, {
actorUserId: args.actorUserId,
publisherId: publisher._id,
deletedAt: args.deletedAt,
source: "account.delete",
});
deletedOrgs += 1;
hiddenSkills += result.hiddenSkills;
deletedPackages += result.deletedPackages;
}
return { ok: true as const, deletedOrgs, hiddenSkills, deletedPackages };
},
});
export const addMember = mutation({
args: {
publisherId: v.id("publishers"),
+289 -2
View File
@@ -111,6 +111,53 @@ const llmRiskSummaryBucketValidator = v.object({
highestSeverity: v.optional(v.string()),
});
const llmAnalysisValidator = v.object({
status: v.string(),
verdict: v.optional(v.string()),
confidence: v.optional(v.string()),
summary: v.optional(v.string()),
dimensions: v.optional(
v.array(
v.object({
name: v.string(),
label: v.string(),
rating: v.string(),
detail: v.string(),
}),
),
),
guidance: v.optional(v.string()),
findings: v.optional(v.string()),
agenticRiskFindings: v.optional(v.array(llmAgenticRiskFindingValidator)),
riskSummary: v.optional(
v.object({
abnormal_behavior_control: llmRiskSummaryBucketValidator,
permission_boundary: llmRiskSummaryBucketValidator,
sensitive_data_protection: llmRiskSummaryBucketValidator,
}),
),
model: v.optional(v.string()),
checkedAt: v.number(),
});
const staticScanValidator = v.object({
status: v.union(v.literal("clean"), v.literal("suspicious"), v.literal("malicious")),
reasonCodes: v.array(v.string()),
findings: v.array(
v.object({
code: v.string(),
severity: v.union(v.literal("info"), v.literal("warn"), v.literal("critical")),
file: v.string(),
line: v.number(),
message: v.string(),
evidence: v.string(),
}),
),
summary: v.string(),
engineVersion: v.string(),
checkedAt: v.number(),
});
const users = defineTable({
name: v.optional(v.string()),
image: v.optional(v.string()),
@@ -199,6 +246,94 @@ const publisherMembers = defineTable({
.index("by_user", ["userId"])
.index("by_publisher_user", ["publisherId", "userId"]);
const officialPublishers = defineTable({
publisherId: v.id("publishers"),
reason: v.optional(v.string()),
createdByUserId: v.optional(v.id("users")),
createdAt: v.number(),
updatedAt: v.number(),
})
.index("by_publisher", ["publisherId"])
.index("by_created", ["createdAt"]);
const displayManifestStatusValidator = v.union(
v.literal("ok"),
v.literal("missing"),
v.literal("invalid"),
v.literal("failed"),
);
const displayManifestValidator = v.object({
notGrouped: v.optional(v.union(v.literal("top"), v.literal("bottom"))),
groupings: v.array(
v.object({
title: v.string(),
description: v.optional(v.string()),
skills: v.array(v.string()),
}),
),
});
const githubSkillSourceInvalidSkillValidator = v.object({
slug: v.string(),
path: v.string(),
displayName: v.string(),
error: v.string(),
});
const githubSkillSourceIssueValidator = v.object({
slug: v.string(),
path: v.string(),
displayName: v.string(),
kind: v.union(v.literal("invalid_slug"), v.literal("slug_conflict")),
severity: v.union(v.literal("error"), v.literal("warning")),
message: v.string(),
existingOwnerHandle: v.optional(v.string()),
});
const githubSkillSources = defineTable({
repo: v.string(),
ownerPublisherId: v.optional(v.id("publishers")),
defaultBranch: v.optional(v.string()),
lastSyncStatus: v.optional(v.union(v.literal("ok"), v.literal("failed"), v.literal("skipped"))),
lastSyncError: v.optional(v.string()),
lastSyncErrorAt: v.optional(v.number()),
displayManifestKind: v.optional(v.literal("skills.sh")),
displayManifestHash: v.optional(v.string()),
displayManifestCommit: v.optional(v.string()),
displayManifestFetchedAt: v.optional(v.number()),
displayManifestStatus: v.optional(displayManifestStatusValidator),
displayManifest: v.optional(displayManifestValidator),
lastSyncIssues: v.optional(v.array(githubSkillSourceIssueValidator)),
// Deprecated. Use lastSyncIssues; kept optional for deployed rows and rollback safety.
lastSyncInvalidSkills: v.optional(v.array(githubSkillSourceInvalidSkillValidator)),
createdAt: v.number(),
updatedAt: v.number(),
})
.index("by_repo", ["repo"])
.index("by_owner_publisher", ["ownerPublisherId"])
.index("by_owner_publisher_and_repo", ["ownerPublisherId", "repo"])
.index("by_created", ["createdAt"])
.index("by_updated", ["updatedAt"]);
const githubSkillContents = defineTable({
skillId: v.id("skills"),
githubSourceId: v.id("githubSkillSources"),
githubPath: v.string(),
skillMarkdownPath: v.string(),
skillMarkdown: v.string(),
skillCardMarkdownPath: v.optional(v.string()),
skillCardMarkdown: v.optional(v.string()),
githubCommit: v.string(),
githubContentHash: v.string(),
fetchedAt: v.number(),
createdAt: v.number(),
updatedAt: v.number(),
})
.index("by_skill", ["skillId"])
.index("by_skill_and_content_hash", ["skillId", "githubContentHash"])
.index("by_github_source", ["githubSourceId"]);
// Shared validator fragments used by both `skills` and `skillSearchDigest`.
const forkOfValidator = v.optional(
v.object({
@@ -245,6 +380,20 @@ const moderationStatusValidator = v.optional(
v.union(v.literal("active"), v.literal("hidden"), v.literal("removed")),
);
const githubSkillScanStatusValidator = v.union(
v.literal("clean"),
v.literal("suspicious"),
v.literal("malicious"),
v.literal("pending"),
v.literal("failed"),
);
const githubSkillCurrentStatusValidator = v.union(
v.literal("present"),
v.literal("missing"),
v.literal("unknown"),
);
const packageFamilyValidator = v.union(
v.literal("skill"),
v.literal("code-plugin"),
@@ -277,6 +426,7 @@ const publisherAbuseDryRunLabelValidator = v.union(
const publisherAbuseTriageStatusValidator = v.union(
v.literal("pending"),
v.literal("banned"),
v.literal("reviewed_no_action"),
v.literal("false_positive"),
v.literal("needs_policy_discussion"),
@@ -361,6 +511,7 @@ const packageVerificationValidator = v.optional(
sourceRepo: v.optional(v.string()),
sourceCommit: v.optional(v.string()),
sourceTag: v.optional(v.string()),
sourcePath: v.optional(v.string()),
hasProvenance: v.optional(v.boolean()),
trustedOpenClawPlugin: v.optional(v.boolean()),
scanStatus: v.optional(
@@ -412,6 +563,7 @@ const packageReleaseModerationOverrideValidator = v.object({
const securityScanTargetKindValidator = v.union(
v.literal("skillVersion"),
v.literal("packageRelease"),
v.literal("skillScanRequest"),
);
const securityScanJobStatusValidator = v.union(
v.literal("queued"),
@@ -449,6 +601,8 @@ const packageFilesValidator = v.array(
}),
);
const skillScanRequestSourceKindValidator = v.union(v.literal("upload"), v.literal("published"));
const skills = defineTable({
slug: v.string(),
displayName: v.string(),
@@ -459,6 +613,16 @@ const skills = defineTable({
ownerPublisherId: v.optional(v.id("publishers")),
canonicalSkillId: v.optional(v.id("skills")),
forkOf: forkOfValidator,
installKind: v.optional(v.literal("github")),
githubSourceId: v.optional(v.id("githubSkillSources")),
githubPath: v.optional(v.string()),
githubHasSkillCard: v.optional(v.boolean()),
githubCurrentCommit: v.optional(v.string()),
githubCurrentContentHash: v.optional(v.string()),
githubCurrentStatus: v.optional(githubSkillCurrentStatusValidator),
githubCurrentCheckedAt: v.optional(v.number()),
githubScanStatus: v.optional(githubSkillScanStatusValidator),
githubRemovedAt: v.optional(v.number()),
latestVersionId: v.optional(v.id("skillVersions")),
latestVersionSummary: v.optional(
v.object({
@@ -554,6 +718,12 @@ const skills = defineTable({
"statsDownloads",
"updatedAt",
])
.index("by_owner_publisher_active_installs", [
"ownerPublisherId",
"softDeletedAt",
"statsInstallsAllTime",
"updatedAt",
])
.index("by_updated", ["updatedAt"])
.index("by_stats_downloads", ["statsDownloads", "updatedAt"])
.index("by_stats_stars", ["statsStars", "updatedAt"])
@@ -573,6 +743,7 @@ const skills = defineTable({
.index("by_canonical", ["canonicalSkillId"])
.index("by_fork_of", ["forkOf.skillId"])
.index("by_moderation", ["moderationStatus", "moderationReason"])
.index("by_github_source", ["githubSourceId"])
.index("by_nonsuspicious_updated", ["softDeletedAt", "isSuspicious", "updatedAt"])
.index("by_nonsuspicious_created", ["softDeletedAt", "isSuspicious", "createdAt"])
.index("by_nonsuspicious_name", ["softDeletedAt", "isSuspicious", "displayName"])
@@ -868,6 +1039,10 @@ const skillSearchDigest = defineTable({
forkOf: forkOfValidator,
latestVersionId: v.optional(v.id("skillVersions")),
latestVersionSkillId: v.optional(v.id("skills")),
installKind: v.optional(v.literal("github")),
githubHasSkillCard: v.optional(v.boolean()),
githubCurrentStatus: v.optional(githubSkillCurrentStatusValidator),
githubScanStatus: v.optional(githubSkillScanStatusValidator),
latestVersionSummary: v.optional(
v.object({
version: v.string(),
@@ -913,6 +1088,13 @@ const skillSearchDigest = defineTable({
"statsInstallsAllTime",
"updatedAt",
])
.index("by_active_recommended_rank", [
"softDeletedAt",
"statsStars",
"statsInstallsAllTime",
"statsDownloads",
"updatedAt",
])
.index("by_nonsuspicious_updated", ["softDeletedAt", "isSuspicious", "updatedAt"])
.index("by_nonsuspicious_created", ["softDeletedAt", "isSuspicious", "createdAt"])
.index("by_nonsuspicious_name", ["softDeletedAt", "isSuspicious", "displayName"])
@@ -945,6 +1127,14 @@ const skillSearchDigest = defineTable({
"statsInstallsAllTime",
"updatedAt",
])
.index("by_nonsuspicious_recommended_rank", [
"softDeletedAt",
"isSuspicious",
"statsStars",
"statsInstallsAllTime",
"statsDownloads",
"updatedAt",
])
.searchIndex("search_by_display_name", {
searchField: "displayName",
filterFields: ["softDeletedAt", "isSuspicious"],
@@ -989,7 +1179,13 @@ const packages = defineTable({
reportCount: v.optional(v.number()),
lastReportedAt: v.optional(v.number()),
softDeletedAt: v.optional(v.number()),
softDeletedReason: v.optional(v.union(v.literal("user.banned"), v.literal("user.deactivated"))),
softDeletedReason: v.optional(
v.union(
v.literal("user.banned"),
v.literal("user.deactivated"),
v.literal("publisher.deleted"),
),
),
softDeletedBy: v.optional(v.id("users")),
softDeletedByRole: v.optional(
v.union(v.literal("admin"), v.literal("moderator"), v.literal("user")),
@@ -1007,6 +1203,12 @@ const packages = defineTable({
"stats.downloads",
"updatedAt",
])
.index("by_owner_publisher_active_installs", [
"ownerPublisherId",
"softDeletedAt",
"stats.installs",
"updatedAt",
])
.index("by_family_updated", ["family", "updatedAt"])
.index("by_family_channel_updated", ["family", "channel", "updatedAt"])
.index("by_family_official_updated", ["family", "isOfficial", "updatedAt"])
@@ -1109,6 +1311,7 @@ const securityScanJobs = defineTable({
targetKind: securityScanTargetKindValidator,
skillVersionId: v.optional(v.id("skillVersions")),
packageReleaseId: v.optional(v.id("packageReleases")),
skillScanRequestId: v.optional(v.id("skillScanRequests")),
status: securityScanJobStatusValidator,
source: securityScanJobSourceValidator,
priority: v.number(),
@@ -1132,7 +1335,48 @@ const securityScanJobs = defineTable({
.index("by_status_and_lease_expires_at", ["status", "leaseExpiresAt"])
.index("by_status_malicious_signal_next_run_at", ["status", "hasMaliciousSignal", "nextRunAt"])
.index("by_skill_version", ["skillVersionId"])
.index("by_package_release", ["packageReleaseId"]);
.index("by_package_release", ["packageReleaseId"])
.index("by_skill_scan_request", ["skillScanRequestId"]);
const skillScanRequests = defineTable({
actorUserId: v.id("users"),
sourceKind: skillScanRequestSourceKindValidator,
update: v.boolean(),
writtenBack: v.boolean(),
status: securityScanJobStatusValidator,
securityScanJobId: v.optional(v.id("securityScanJobs")),
slug: v.optional(v.string()),
displayName: v.optional(v.string()),
version: v.optional(v.string()),
skillId: v.optional(v.id("skills")),
skillVersionId: v.optional(v.id("skillVersions")),
files: packageFilesValidator,
parsed: v.optional(
v.object({
frontmatter: v.record(v.string(), v.any()),
metadata: v.optional(v.any()),
clawdis: v.optional(v.any()),
moltbot: v.optional(v.any()),
license: v.optional(v.literal(PLATFORM_SKILL_LICENSE)),
}),
),
sha256hash: v.optional(v.string()),
vtAnalysis: v.optional(vtAnalysisValidator),
skillSpectorAnalysis: v.optional(skillSpectorAnalysisValidator),
llmAnalysis: v.optional(llmAnalysisValidator),
capabilityTags: v.optional(v.array(v.string())),
staticScan: v.optional(staticScanValidator),
lastError: v.optional(v.string()),
runId: v.optional(v.string()),
completedAt: v.optional(v.number()),
expiresAt: v.number(),
createdAt: v.number(),
updatedAt: v.number(),
})
.index("by_actor_user_id_and_created_at", ["actorUserId", "createdAt"])
.index("by_security_scan_job_id", ["securityScanJobId"])
.index("by_skill_version_id_and_created_at", ["skillVersionId", "createdAt"])
.index("by_expires_at", ["expiresAt"]);
const skillCardGenerationJobs = defineTable({
skillId: v.id("skills"),
@@ -1209,6 +1453,16 @@ const packagePublishTokens = defineTable({
.index("by_package", ["packageId", "version", "createdAt"])
.index("by_package_revoked_created", ["packageId", "revokedAt", "createdAt"]);
const packagePublishUploadTickets = defineTable({
kind: v.union(v.literal("user"), v.literal("github-actions")),
userId: v.optional(v.id("users")),
publishTokenId: v.optional(v.id("packagePublishTokens")),
createdAt: v.number(),
expiresAt: v.number(),
usedAt: v.optional(v.number()),
storageId: v.optional(v.id("_storage")),
});
const packageSearchDigest = defineTable({
packageId: v.id("packages"),
name: v.string(),
@@ -1228,6 +1482,7 @@ const packageSearchDigest = defineTable({
pluginCategoryTags: v.optional(v.array(v.string())),
executesCode: v.optional(v.boolean()),
verificationTier: v.optional(packageVerificationTierValidator),
stats: v.optional(packageStatsValidator),
scanStatus: packageScanStatusValidator,
softDeletedAt: v.optional(v.number()),
createdAt: v.number(),
@@ -1314,6 +1569,7 @@ const packageCapabilitySearchDigest = defineTable({
capabilityTag: v.string(),
executesCode: v.optional(v.boolean()),
verificationTier: v.optional(packageVerificationTierValidator),
stats: v.optional(packageStatsValidator),
scanStatus: packageScanStatusValidator,
softDeletedAt: v.optional(v.number()),
createdAt: v.number(),
@@ -1427,6 +1683,7 @@ const packagePluginCategorySearchDigest = defineTable({
pluginCategory: v.string(),
executesCode: v.optional(v.boolean()),
verificationTier: v.optional(packageVerificationTierValidator),
stats: v.optional(packageStatsValidator),
scanStatus: packageScanStatusValidator,
softDeletedAt: v.optional(v.number()),
createdAt: v.number(),
@@ -1887,7 +2144,9 @@ const publisherAbuseScores = defineTable({
createdAt: v.number(),
})
.index("by_run_and_rank", ["runId", "rank"])
.index("by_run_and_label_and_rank", ["runId", "label", "rank"])
.index("by_run_and_pressure", ["runId", "pressure"])
.index("by_run_and_owner_key", ["runId", "ownerKey"])
.index("by_owner_key_and_created_at", ["ownerKey", "createdAt"])
.index("by_owner_key_and_model_version", ["ownerKey", "modelVersion"])
.index("by_label_and_z_score", ["label", "zScore"]);
@@ -1911,6 +2170,8 @@ const publisherAbuseReviewNominations = defineTable({
})
.index("by_owner_key_and_model_version", ["ownerKey", "modelVersion"])
.index("by_status_and_last_scored_at", ["status", "lastScoredAt"])
.index("by_status_and_updated_at", ["status", "updatedAt"])
.index("by_status_and_reviewed_at", ["status", "reviewedAt"])
.index("by_status_and_label_and_last_scored_at", ["status", "label", "lastScoredAt"])
.index("by_label_and_status_and_last_scored_at", ["label", "status", "lastScoredAt"])
.index("by_last_scored_at", ["lastScoredAt"]);
@@ -2022,6 +2283,26 @@ const downloadDedupes = defineTable({
.index("by_skill_identity_hour", ["skillId", "identityHash", "hourStart"])
.index("by_hour", ["hourStart"]);
const downloadMetricTargetKind = v.union(v.literal("skill"), v.literal("package"));
const downloadMetricIdentityKind = v.union(v.literal("user"), v.literal("ip"));
const downloadMetricDedupes = defineTable({
targetKind: downloadMetricTargetKind,
targetId: v.string(),
identityKind: downloadMetricIdentityKind,
identityHash: v.string(),
dayStart: v.number(),
createdAt: v.number(),
})
.index("by_target_identity_day", [
"targetKind",
"targetId",
"identityKind",
"identityHash",
"dayStart",
])
.index("by_day", ["dayStart"]);
const reservedSlugs = defineTable({
slug: v.string(),
originalOwnerUserId: v.id("users"),
@@ -2120,15 +2401,20 @@ export default defineSchema({
users,
publishers,
publisherMembers,
officialPublishers,
githubSkillSources,
githubSkillContents,
skills,
skillSlugAliases,
packages,
packageReleases,
securityScanJobs,
skillScanRequests,
skillCardGenerationJobs,
packageStatEvents,
packageTrustedPublishers,
packagePublishTokens,
packagePublishUploadTickets,
packageBadges,
packageSearchDigest,
packageCapabilitySearchDigest,
@@ -2173,6 +2459,7 @@ export default defineSchema({
rateLimits,
rateLimitShards,
downloadDedupes,
downloadMetricDedupes,
reservedSlugs,
reservedHandles,
githubBackupSyncState,
+104 -13
View File
@@ -544,6 +544,8 @@ describe("search helpers", () => {
slug: "antigravity-image-generator",
displayName: "Antigravity Image Generator",
downloads: 1_000_000_000,
installsAllTime: 1_000,
stars: 100,
}),
version: null,
ownerHandle: "owner",
@@ -563,7 +565,7 @@ describe("search helpers", () => {
vectorSearch: vi.fn().mockResolvedValue(
vectorEntries.map((entry, index) => ({
_id: entry.embeddingId,
_score: 0.5 - index * 0.001,
_score: 0.05 - index * 0.001,
})),
),
runQuery,
@@ -1142,8 +1144,16 @@ describe("search helpers", () => {
it("boosts exact slug/name matches over loose matches", () => {
const queryTokens = tokenize("notion");
const exactScore = __test.scoreSkillResult(queryTokens, 0.4, "Notion Sync", "notion-sync", 5);
const looseScore = __test.scoreSkillResult(queryTokens, 0.6, "Notes Sync", "notes-sync", 500);
const exactScore = __test.scoreSkillResult(queryTokens, 0.4, "Notion Sync", "notion-sync", {
downloads: 5,
installsAllTime: 0,
stars: 0,
});
const looseScore = __test.scoreSkillResult(queryTokens, 0.6, "Notes Sync", "notes-sync", {
downloads: 500,
installsAllTime: 100,
stars: 20,
});
expect(exactScore).toBeGreaterThan(looseScore);
});
@@ -1154,35 +1164,114 @@ describe("search helpers", () => {
0.5,
"Self Improving Agent",
"self-improving-agent",
10,
{ downloads: 10, installsAllTime: 0, stars: 0 },
);
const containingScore = __test.scoreSkillResult(
queryTokens,
0.6,
"Self Improving Agent",
"xiucheng-self-improving-agent",
100,
{ downloads: 100, installsAllTime: 50, stars: 10 },
);
expect(exactScore).toBeGreaterThan(containingScore);
});
it("adds a popularity prior for equally relevant matches", () => {
it("keeps extreme popularity below direct lexical relevance", () => {
const queryTokens = tokenize("needle");
const exactScore = __test.scoreSkillResult(queryTokens, 0, "Unrelated Name", "needle", {
downloads: 0,
installsAllTime: 0,
stars: 0,
});
const popularLooseScore = __test.scoreSkillResult(
queryTokens,
0.9,
"Different Tool",
"different-tool",
{ downloads: 1_000_000, installsAllTime: 25_000, stars: 25_000 },
);
expect(exactScore).toBeGreaterThan(popularLooseScore);
});
it("keeps popularity from flipping a strong name match", () => {
const queryTokens = tokenize("notion");
const lowDownloads = __test.scoreSkillResult(
const nameMatchScore = __test.scoreSkillResult(queryTokens, 0, "Notion Helper", "helper", {
downloads: 0,
installsAllTime: 0,
stars: 0,
});
const popularVectorScore = __test.scoreSkillResult(
queryTokens,
1,
"Different Tool",
"different-tool",
{ downloads: 1_000_000, installsAllTime: 25_000, stars: 25_000 },
);
expect(nameMatchScore).toBeGreaterThan(popularVectorScore);
});
it("adds a stars and installs popularity prior for equally relevant matches", () => {
const queryTokens = tokenize("notion");
const highDownloadsOnly = __test.scoreSkillResult(
queryTokens,
0.5,
"Notion Helper",
"notion-helper",
0,
{ downloads: 1000, installsAllTime: 0, stars: 0 },
);
const highDownloads = __test.scoreSkillResult(
const trustedUsage = __test.scoreSkillResult(
queryTokens,
0.5,
"Notion Helper",
"notion-helper",
1000,
{ downloads: 0, installsAllTime: 20, stars: 5 },
);
expect(highDownloads).toBeGreaterThan(lowDownloads);
expect(trustedUsage).toBeGreaterThan(highDownloadsOnly);
});
it("breaks capped popularity ties by stars and installs before downloads", async () => {
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
const trustedUsage = {
skill: makePublicSkill({
id: "skills:trusted",
slug: "tool-trusted",
displayName: "Tool",
downloads: 0,
installsAllTime: 1_000,
stars: 1_000,
}),
version: null,
ownerHandle: "owner",
owner: null,
};
const downloadedOnly = {
skill: makePublicSkill({
id: "skills:downloaded",
slug: "tool-downloaded",
displayName: "Tool",
downloads: 1_000_000_000,
installsAllTime: 0,
stars: 0,
}),
version: null,
ownerHandle: "owner",
owner: null,
};
const runQuery = vi
.fn()
.mockResolvedValueOnce(null) // getExactSkillSlugMatch
.mockResolvedValueOnce([]) // directPrefixSkillMatches
.mockResolvedValueOnce([downloadedOnly, trustedUsage]); // lexicalFallbackSkills
const result = await searchSkillsHandler(
{
vectorSearch: vi.fn().mockResolvedValue([]),
runQuery,
},
{ query: "tool", limit: 2 },
);
expect(result.map((entry) => entry.skill.slug)).toEqual(["tool-trusted", "tool-downloaded"]);
});
it("uses digest doc instead of full skill doc in hydrateResults but revalidates the owner", async () => {
@@ -1532,6 +1621,8 @@ function makePublicSkill(params: {
slug: string;
displayName: string;
downloads?: number;
installsAllTime?: number;
stars?: number;
capabilityTags?: string[];
}) {
return {
@@ -1550,8 +1641,8 @@ function makePublicSkill(params: {
stats: {
downloads: params.downloads ?? 0,
installsCurrent: 0,
installsAllTime: 0,
stars: 0,
installsAllTime: params.installsAllTime ?? 0,
stars: params.stars ?? 0,
versions: 1,
comments: 0,
},
+45 -7
View File
@@ -75,7 +75,10 @@ const SLUG_TOKEN_BOOST = 1.4;
const SLUG_PREFIX_BOOST = 0.8;
const NAME_EXACT_BOOST = 1.1;
const NAME_PREFIX_BOOST = 0.6;
const POPULARITY_WEIGHT = 0.08;
const STAR_POPULARITY_WEIGHT = 0.12;
const INSTALL_POPULARITY_WEIGHT = 0.04;
const DOWNLOAD_POPULARITY_WEIGHT = 0.005;
const MAX_POPULARITY_BOOST = 0.09;
const FALLBACK_SCAN_LIMIT = 2000;
const MIN_FALLBACK_SCAN_LIMIT = 100;
const FALLBACK_RECALL_MULTIPLIER = 2;
@@ -119,15 +122,29 @@ function getLexicalBoost(queryTokens: string[], displayName: string, slug: strin
return boost;
}
type PopularityStats = {
downloads: number;
installsAllTime?: number;
stars: number;
};
function getPopularityBoost(stats: PopularityStats) {
const rawBoost =
Math.log1p(Math.max(stats.stars, 0)) * STAR_POPULARITY_WEIGHT +
Math.log1p(Math.max(stats.installsAllTime ?? 0, 0)) * INSTALL_POPULARITY_WEIGHT +
Math.log1p(Math.max(stats.downloads, 0)) * DOWNLOAD_POPULARITY_WEIGHT;
return Math.min(rawBoost, MAX_POPULARITY_BOOST);
}
function scoreSkillResult(
queryTokens: string[],
vectorScore: number,
displayName: string,
slug: string,
downloads: number,
stats: PopularityStats,
) {
const lexicalBoost = getLexicalBoost(queryTokens, displayName, slug);
const popularityBoost = Math.log1p(Math.max(downloads, 0)) * POPULARITY_WEIGHT;
const popularityBoost = getPopularityBoost(stats);
return vectorScore + lexicalBoost + popularityBoost;
}
@@ -179,6 +196,14 @@ function classifySkillMatch(
return null;
}
function comparePopularityStats(a: PopularityStats, b: PopularityStats) {
return (
b.stars - a.stars ||
(b.installsAllTime ?? 0) - (a.installsAllTime ?? 0) ||
b.downloads - a.downloads
);
}
function mergeUniqueBySkillId(primary: SkillSearchEntry[], fallback: SkillSearchEntry[]) {
if (fallback.length === 0) return primary;
const out = [...primary];
@@ -353,7 +378,11 @@ export const searchSkills: ReturnType<typeof action> = action({
vectorScore,
entry.skill.displayName,
entry.skill.slug,
entry.skill.stats.downloads,
{
downloads: entry.skill.stats.downloads,
installsAllTime: entry.skill.stats.installsAllTime,
stars: entry.skill.stats.stars,
},
),
};
})
@@ -362,7 +391,8 @@ export const searchSkills: ReturnType<typeof action> = action({
(a, b) =>
a.rankTier - b.rankTier ||
b.score - a.score ||
b.skill.stats.downloads - a.skill.stats.downloads,
comparePopularityStats(a.skill.stats, b.skill.stats) ||
b.skill.updatedAt - a.skill.updatedAt,
)
.slice(0, limit);
return rankedMatches.map(({ rankTier: _rankTier, ...entry }) => entry);
@@ -879,12 +909,20 @@ export const searchSouls: ReturnType<typeof action> = action({
vectorScore,
entry.soul.displayName,
entry.soul.slug,
entry.soul.stats.downloads,
{
downloads: entry.soul.stats.downloads,
stars: entry.soul.stats.stars,
},
),
};
})
.filter((entry) => entry.soul)
.sort((a, b) => b.score - a.score || b.soul.stats.downloads - a.soul.stats.downloads)
.sort(
(a, b) =>
b.score - a.score ||
comparePopularityStats(a.soul.stats, b.soul.stats) ||
b.soul.updatedAt - a.soul.updatedAt,
)
.slice(0, limit);
},
});
+90 -13
View File
@@ -8,7 +8,10 @@ import { getOwnerPublisher } from "./lib/publishers";
const MAX_EXPORT_PAGE_SIZE = 50;
const MAX_EXPORT_BATCH_PAGES = 20;
const REDACTION_POLICY_VERSION = "public-signals-v1";
const MAX_REDACTED_BUNDLE_FILE_BYTES = 192 * 1024;
const MAX_REDACTED_BUNDLE_BYTES_PER_ARTIFACT = 256 * 1024;
const MAX_REDACTED_BUNDLE_BYTES_PER_RESPONSE = 256 * 1024;
const REDACTION_POLICY_VERSION = "public-signals-v2-bundle-files";
const SOURCE_TABLES = ["skillVersions", "packageReleases"] as const;
const SCANNER_SOURCES = [
"static",
@@ -297,17 +300,80 @@ function sanitizeFiles(files: Array<Doc<"skillVersions">["files"][number]>) {
}
async function enrichAndSanitizeArtifactRows(ctx: ActionCtx, rows: ArtifactExportRow[]) {
return await Promise.all(
rows.map(async (row) => {
const skillContent =
row.sourceKind === "skill" ? await readRedactedSkillMdContent(ctx, row.files) : null;
return {
...row,
...(skillContent ? { skillMdContentRedacted: skillContent } : {}),
files: row.files.map(({ storageId: _storageId, ...file }) => file),
};
}),
const enrichedRows = [];
let remainingBundleBytes = MAX_REDACTED_BUNDLE_BYTES_PER_RESPONSE;
for (const row of rows) {
const skillContent =
row.sourceKind === "skill" ? await readRedactedSkillMdContent(ctx, row.files) : null;
const bundleFiles =
row.sourceKind === "skill"
? await readRedactedBundleFiles(ctx, row.files, remainingBundleBytes)
: [];
remainingBundleBytes -= totalBundleBytes(bundleFiles);
enrichedRows.push({
...row,
...(skillContent ? { skillMdContentRedacted: skillContent } : {}),
...(bundleFiles.length > 0 ? { bundleFilesRedacted: bundleFiles } : {}),
files: row.files.map(({ storageId: _storageId, ...file }) => file),
});
}
return enrichedRows;
}
async function readRedactedBundleFiles(
ctx: Pick<ActionCtx, "storage">,
files: Array<{ path: string; size?: number; storageId?: unknown }>,
remainingResponseBytes: number,
) {
const bundleFiles: Array<{ path: string; content: string }> = [];
let remainingArtifactBytes = Math.min(
remainingResponseBytes,
MAX_REDACTED_BUNDLE_BYTES_PER_ARTIFACT,
);
for (const file of files) {
if (isExcludedSkillBundlePath(file.path) || typeof file.storageId !== "string") continue;
if (typeof file.size === "number" && file.size > MAX_REDACTED_BUNDLE_FILE_BYTES) continue;
if (remainingArtifactBytes <= 0) break;
const blob = await ctx.storage.get(file.storageId as never);
if (!blob) continue;
const content = redactBundleContent(await blob.text());
const contentBytes = utf8Bytes(content);
if (contentBytes > MAX_REDACTED_BUNDLE_FILE_BYTES || contentBytes > remainingArtifactBytes) {
continue;
}
bundleFiles.push({ path: file.path, content });
remainingArtifactBytes -= contentBytes;
}
return bundleFiles;
}
function isExcludedSkillBundlePath(path: string) {
return (
isPrimarySkillReadmePath(path) || normalizeBundlePathForComparison(path) === "skill-card.md"
);
}
function isPrimarySkillReadmePath(path: string) {
const normalized = normalizeBundlePathForComparison(path);
return normalized === "skill.md" || normalized === "skills.md";
}
function normalizeBundlePathForComparison(path: string) {
return path
.trim()
.replace(/^\/+/, "")
.split("/")
.filter((segment) => segment && segment !== ".")
.join("/")
.toLowerCase();
}
function totalBundleBytes(files: Array<{ content: string }>) {
return files.reduce((sum, file) => sum + utf8Bytes(file.content), 0);
}
function utf8Bytes(value: string) {
return new TextEncoder().encode(value).byteLength;
}
async function readRedactedSkillMdContent(
@@ -315,8 +381,7 @@ async function readRedactedSkillMdContent(
files: Array<{ path: string; storageId?: unknown }>,
) {
const skillFile = files.find((file) => {
const path = file.path.toLowerCase();
return path === "skill.md" || path.endsWith("/skill.md");
return isPrimarySkillReadmePath(file.path);
});
if (!skillFile || typeof skillFile.storageId !== "string") return null;
const blob = await ctx.storage.get(skillFile.storageId as never);
@@ -336,6 +401,18 @@ function redactSkillContent(value: string) {
return redacted.trim();
}
function redactBundleContent(value: string) {
let redacted = "";
for (let index = 0; index < value.length; index += 1) {
const code = value.charCodeAt(index);
redacted += code < 32 && code !== 9 && code !== 10 && code !== 13 ? " " : value.charAt(index);
}
for (const pattern of SECRET_PATTERNS) {
redacted = redacted.replace(pattern, "[REDACTED_SECRET]");
}
return redacted;
}
function normalizeVtAnalysis(analysis: StoredVtAnalysis) {
if (!analysis) return null;
return {
+105 -18
View File
@@ -8,6 +8,9 @@ import type { ActionCtx } from "./_generated/server";
import { internalAction } from "./functions";
const MAX_EXPORT_BATCH_PAGES = 20;
const MAX_REDACTED_BUNDLE_FILE_BYTES = 192 * 1024;
const MAX_REDACTED_BUNDLE_BYTES_PER_ARTIFACT = 256 * 1024;
const MAX_REDACTED_BUNDLE_BYTES_PER_RESPONSE = 256 * 1024;
type ArtifactExportPage = {
page: unknown[];
@@ -75,30 +78,102 @@ export const listArtifactExportBatchCompressedInternal = internalAction({
});
async function enrichAndSanitizeArtifactRows(ctx: ActionCtx, rows: unknown[]) {
return await Promise.all(
rows.map(async (row) => {
if (!isRecord(row)) return row;
const files = Array.isArray(row.files) ? row.files : [];
const skillContent =
row.sourceKind === "skill" ? await readRedactedSkillMdContent(ctx, files) : null;
return {
...row,
...(skillContent ? { skillMdContentRedacted: skillContent } : {}),
files: files.map((file) => {
if (!isRecord(file)) return file;
const { storageId: _storageId, ...rest } = file;
return rest;
}),
};
}),
const enrichedRows = [];
let remainingBundleBytes = MAX_REDACTED_BUNDLE_BYTES_PER_RESPONSE;
for (const row of rows) {
if (!isRecord(row)) {
enrichedRows.push(row);
continue;
}
const files = Array.isArray(row.files) ? row.files : [];
const skillContent =
row.sourceKind === "skill" ? await readRedactedSkillMdContent(ctx, files) : null;
const bundleFiles =
row.sourceKind === "skill"
? await readRedactedBundleFiles(ctx, files, remainingBundleBytes)
: [];
remainingBundleBytes -= totalBundleBytes(bundleFiles);
enrichedRows.push({
...row,
...(skillContent ? { skillMdContentRedacted: skillContent } : {}),
...(bundleFiles.length > 0 ? { bundleFilesRedacted: bundleFiles } : {}),
files: files.map((file) => {
if (!isRecord(file)) return file;
const { storageId: _storageId, ...rest } = file;
return rest;
}),
});
}
return enrichedRows;
}
async function readRedactedBundleFiles(
ctx: Pick<ActionCtx, "storage">,
files: unknown[],
remainingResponseBytes: number,
) {
const bundleFiles: Array<{ path: string; content: string }> = [];
let remainingArtifactBytes = Math.min(
remainingResponseBytes,
MAX_REDACTED_BUNDLE_BYTES_PER_ARTIFACT,
);
for (const file of files) {
if (
!isRecord(file) ||
typeof file.path !== "string" ||
typeof file.storageId !== "string" ||
isExcludedSkillBundlePath(file.path)
) {
continue;
}
if (typeof file.size === "number" && file.size > MAX_REDACTED_BUNDLE_FILE_BYTES) continue;
if (remainingArtifactBytes <= 0) break;
const blob = await ctx.storage.get(file.storageId as never);
if (!blob) continue;
const content = redactBundleContent(await blob.text());
const contentBytes = utf8Bytes(content);
if (contentBytes > MAX_REDACTED_BUNDLE_FILE_BYTES || contentBytes > remainingArtifactBytes) {
continue;
}
bundleFiles.push({ path: file.path, content });
remainingArtifactBytes -= contentBytes;
}
return bundleFiles;
}
function isExcludedSkillBundlePath(path: string) {
return (
isPrimarySkillReadmePath(path) || normalizeBundlePathForComparison(path) === "skill-card.md"
);
}
function isPrimarySkillReadmePath(path: string) {
const normalized = normalizeBundlePathForComparison(path);
return normalized === "skill.md" || normalized === "skills.md";
}
function normalizeBundlePathForComparison(path: string) {
return path
.trim()
.replace(/^\/+/, "")
.split("/")
.filter((segment) => segment && segment !== ".")
.join("/")
.toLowerCase();
}
function totalBundleBytes(files: Array<{ content: string }>) {
return files.reduce((sum, file) => sum + utf8Bytes(file.content), 0);
}
function utf8Bytes(value: string) {
return new TextEncoder().encode(value).byteLength;
}
async function readRedactedSkillMdContent(ctx: Pick<ActionCtx, "storage">, files: unknown[]) {
const skillFile = files.find((file) => {
if (!isRecord(file) || typeof file.path !== "string") return false;
const path = file.path.toLowerCase();
return path === "skill.md" || path.endsWith("/skill.md");
return isPrimarySkillReadmePath(file.path);
});
if (!isRecord(skillFile) || typeof skillFile.storageId !== "string") return null;
const blob = await ctx.storage.get(skillFile.storageId as never);
@@ -118,6 +193,18 @@ function redactSkillContent(value: string) {
return redacted.trim();
}
function redactBundleContent(value: string) {
let redacted = "";
for (let index = 0; index < value.length; index += 1) {
const code = value.charCodeAt(index);
redacted += code < 32 && code !== 9 && code !== 10 && code !== 13 ? " " : value.charAt(index);
}
for (const pattern of SECRET_PATTERNS) {
redacted = redacted.replace(pattern, "[REDACTED_SECRET]");
}
return redacted;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === "object" && !Array.isArray(value);
}
+352
View File
@@ -9,6 +9,8 @@ import {
enqueueBulkSkillRescanBatchForAdminInternal,
failCodexScanJob,
getBulkSkillRescanBatchStatusForAdminInternal,
getSkillScanRequestForUserInternal,
pruneExpiredSkillScanRequestsInternal,
requestPackageRescanForUserInternal,
requestPackageRescan,
requestSkillRescanForUserInternal,
@@ -101,6 +103,7 @@ type ScanJob = {
targetKind: string;
skillVersionId?: string;
packageReleaseId?: string;
skillScanRequestId?: string;
source: string;
priority: number;
hasMaliciousSignal: boolean;
@@ -123,6 +126,12 @@ const clearQueuedBackfillJobsForLocalDevHandler = (
{ dryRun: boolean; matched: number; deleted: number; sampleDeletedJobIds: string[] }
>
)._handler;
const pruneExpiredSkillScanRequestsInternalHandler = (
pruneExpiredSkillScanRequestsInternal as unknown as WrappedHandler<
{ batchSize?: number },
{ ok: true; deletedRequests: number; deletedJobs: number; deletedFiles: number; done: boolean }
>
)._handler;
const requestSkillRescanHandler = (
requestSkillRescan as unknown as WrappedHandler<
@@ -192,6 +201,26 @@ const getBulkSkillRescanBatchStatusForAdminInternalHandler = (
>
)._handler;
const getSkillScanRequestForUserInternalHandler = (
getSkillScanRequestForUserInternal as unknown as WrappedHandler<
{ actorUserId: string; scanId: string },
{
ok: true;
scanId: string;
jobId?: string;
status: string;
queue: {
queuedAhead: number;
queuedAheadIsEstimate?: boolean;
position: number | null;
running: number;
runningIsEstimate?: boolean;
note: string;
};
}
>
)._handler;
const claimedJob = {
_id: "securityScanJobs:1",
_creationTime: 1,
@@ -579,6 +608,83 @@ function makeClaimCtx(jobs: ScanJob[]) {
};
}
function makeSkillScanStatusCtx(options: {
actor: Record<string, unknown>;
request: Record<string, unknown>;
jobs: ScanJob[];
}) {
const docs = new Map<string, Record<string, unknown>>([
[String(options.actor._id), options.actor],
[String(options.request._id), options.request],
...options.jobs.map((job) => [job._id, job] as const),
]);
const get = vi.fn(async (id: string) => docs.get(id) ?? null);
const query = vi.fn((tableName: string) => {
expect(tableName).toBe("securityScanJobs");
return {
withIndex: vi.fn(
(
indexName: string,
buildRange: (q: {
eq: (field: string, value: unknown) => unknown;
lte: (field: string, value: number) => unknown;
}) => unknown,
) => {
const eqFilters = new Map<string, unknown>();
const lteFilters = new Map<string, number>();
const indexBuilder = {
eq(field: string, value: unknown) {
eqFilters.set(field, value);
return indexBuilder;
},
lte(field: string, value: number) {
lteFilters.set(field, value);
return indexBuilder;
},
};
buildRange(indexBuilder);
const select = () =>
options.jobs
.filter((job) => {
for (const [field, value] of eqFilters) {
if ((job as unknown as Record<string, unknown>)[field] !== value) return false;
}
for (const [field, value] of lteFilters) {
const fieldValue = (job as unknown as Record<string, unknown>)[field];
if (typeof fieldValue !== "number" || fieldValue > value) return false;
}
return true;
})
.sort((a, b) => {
if (indexName.includes("next_run_at")) {
if (a.nextRunAt !== b.nextRunAt) return a.nextRunAt - b.nextRunAt;
if (a._creationTime !== b._creationTime) {
return a._creationTime - b._creationTime;
}
return a._id.localeCompare(b._id);
}
return a.createdAt - b.createdAt;
});
const collect = vi.fn(async () => select());
const take = vi.fn(async (limit: number) => select().slice(0, limit));
return {
collect,
take,
order: vi.fn(() => ({ collect, take })),
};
},
),
};
});
return {
db: {
get,
query,
},
};
}
describe("securityScan", () => {
afterEach(() => {
vi.unstubAllEnvs();
@@ -1206,6 +1312,81 @@ describe("securityScan", () => {
expect(deleted).toEqual(["securityScanJobs:backfill-1", "securityScanJobs:backfill-2"]);
});
it("prunes expired uploaded scan request blobs without deleting published version files", async () => {
const requests = [
{
_id: "skillScanRequests:upload",
sourceKind: "upload",
securityScanJobId: "securityScanJobs:upload",
files: [{ storageId: "storage:upload-1" }, { storageId: "storage:upload-2" }],
},
{
_id: "skillScanRequests:published",
sourceKind: "published",
securityScanJobId: "securityScanJobs:published",
files: [{ storageId: "storage:published-version-file" }],
},
];
const deletedDocs: string[] = [];
const deletedStorage: string[] = [];
const take = vi.fn(async () => requests);
const indexBuilder = {
lt: vi.fn(() => indexBuilder),
};
const withIndex = vi.fn(
(indexName: string, buildRange: (q: typeof indexBuilder) => unknown) => {
expect(indexName).toBe("by_expires_at");
buildRange(indexBuilder);
expect(indexBuilder.lt).toHaveBeenCalledWith("expiresAt", expect.any(Number));
return { take };
},
);
const ctx = {
db: {
query: vi.fn((tableName: string) => {
expect(tableName).toBe("skillScanRequests");
return { withIndex };
}),
insert: vi.fn(async () => "noop"),
patch: vi.fn(async () => undefined),
replace: vi.fn(async () => undefined),
get: vi.fn(async (id: string) => ({
_id: id,
targetKind: "skillScanRequest",
})),
delete: vi.fn(async (id: string) => {
deletedDocs.push(id);
}),
normalizeId: vi.fn(() => null),
system: {},
},
storage: {
delete: vi.fn(async (id: string) => {
deletedStorage.push(id);
}),
},
};
const result = await pruneExpiredSkillScanRequestsInternalHandler(ctx as never, {
batchSize: 10,
});
expect(result).toEqual({
ok: true,
deletedRequests: 2,
deletedJobs: 2,
deletedFiles: 2,
done: true,
});
expect(deletedStorage).toEqual(["storage:upload-1", "storage:upload-2"]);
expect(deletedDocs).toEqual([
"securityScanJobs:upload",
"skillScanRequests:upload",
"securityScanJobs:published",
"skillScanRequests:published",
]);
});
it("fails claimed package jobs when the ClawPack URL is unavailable", async () => {
vi.stubEnv("SECURITY_SCAN_WORKER_TOKEN", "worker-secret");
@@ -1406,6 +1587,177 @@ describe("securityScan", () => {
]);
});
it("reports queued scan position for manual scan requests", async () => {
const targetJob = makeScanJob({
_id: "securityScanJobs:target",
targetKind: "skillScanRequest",
skillScanRequestId: "skillScanRequests:target",
source: "manual",
createdAt: 300,
nextRunAt: 300,
});
const ctx = makeSkillScanStatusCtx({
actor: { _id: "users:owner", role: "user" },
request: {
_id: "skillScanRequests:target",
actorUserId: "users:owner",
sourceKind: "upload",
update: false,
writtenBack: false,
status: "queued",
securityScanJobId: targetJob._id,
files: [],
expiresAt: 1000,
createdAt: 300,
updatedAt: 300,
},
jobs: [
makeScanJob({
_id: "securityScanJobs:older",
source: "manual",
createdAt: 100,
nextRunAt: 100,
}),
makeScanJob({
_id: "securityScanJobs:running",
status: "running",
source: "manual",
createdAt: 200,
nextRunAt: 200,
}),
targetJob,
makeScanJob({
_id: "securityScanJobs:bulk",
source: "bulk-rescan",
createdAt: 1,
nextRunAt: 1,
}),
],
});
const status = await getSkillScanRequestForUserInternalHandler(ctx, {
actorUserId: "users:owner",
scanId: "skillScanRequests:target",
});
expect(status.queue).toEqual({
queuedAhead: 1,
queuedAheadIsEstimate: false,
position: 2,
running: 1,
runningIsEstimate: false,
note: "Scans are asynchronous and may take time to complete.",
});
});
it("uses claim-order tie-breaks for same-timestamp queued scan positions", async () => {
const targetJob = makeScanJob({
_id: "securityScanJobs:target",
_creationTime: 2,
targetKind: "skillScanRequest",
skillScanRequestId: "skillScanRequests:target",
source: "manual",
createdAt: 300,
nextRunAt: 300,
});
const ctx = makeSkillScanStatusCtx({
actor: { _id: "users:owner", role: "user" },
request: {
_id: "skillScanRequests:target",
actorUserId: "users:owner",
sourceKind: "upload",
update: false,
writtenBack: false,
status: "queued",
securityScanJobId: targetJob._id,
files: [],
expiresAt: 1000,
createdAt: 300,
updatedAt: 300,
},
jobs: [
makeScanJob({
_id: "securityScanJobs:first",
_creationTime: 1,
source: "manual",
createdAt: 300,
nextRunAt: 300,
}),
targetJob,
makeScanJob({
_id: "securityScanJobs:last",
_creationTime: 3,
source: "manual",
createdAt: 300,
nextRunAt: 300,
}),
],
});
const status = await getSkillScanRequestForUserInternalHandler(ctx, {
actorUserId: "users:owner",
scanId: "skillScanRequests:target",
});
expect(status.queue).toMatchObject({
queuedAhead: 1,
queuedAheadIsEstimate: false,
position: 2,
});
});
it("bounds large queue position scans and marks the count as estimated", async () => {
const targetJob = makeScanJob({
_id: "securityScanJobs:target",
targetKind: "skillScanRequest",
skillScanRequestId: "skillScanRequests:target",
source: "manual",
createdAt: 1_000,
nextRunAt: 1_000,
});
const ctx = makeSkillScanStatusCtx({
actor: { _id: "users:owner", role: "user" },
request: {
_id: "skillScanRequests:target",
actorUserId: "users:owner",
sourceKind: "upload",
update: false,
writtenBack: false,
status: "queued",
securityScanJobId: targetJob._id,
files: [],
expiresAt: 1000,
createdAt: 1_000,
updatedAt: 1_000,
},
jobs: [
...Array.from({ length: 300 }, (_, index) =>
makeScanJob({
_id: `securityScanJobs:older-${index}`,
source: "manual",
createdAt: index,
nextRunAt: index,
}),
),
targetJob,
],
});
const status = await getSkillScanRequestForUserInternalHandler(ctx, {
actorUserId: "users:owner",
scanId: "skillScanRequests:target",
});
expect(status.queue).toEqual({
queuedAhead: 250,
queuedAheadIsEstimate: true,
position: null,
running: 0,
runningIsEstimate: false,
note: "Scans are asynchronous and may take time to complete.",
});
});
it("caps SkillSpector findings before storing completed scan results", async () => {
vi.stubEnv("SECURITY_SCAN_WORKER_TOKEN", "worker-secret");
const longSnippet = "sensitive SkillSpector artifact text ".repeat(200);
+522 -1
View File
@@ -18,6 +18,8 @@ const DEFAULT_CANCEL_SCAN_LIMIT = 1000;
const DEFAULT_CANCEL_DELETE_LIMIT = 500;
const MAX_CANCEL_SCAN_LIMIT = 5000;
const CANCEL_SAMPLE_LIMIT = 20;
const DEFAULT_PRUNE_SKILL_SCAN_REQUEST_LIMIT = 250;
const MAX_PRUNE_SKILL_SCAN_REQUEST_LIMIT = 1000;
const DEFAULT_BULK_RESCAN_BATCH_SIZE = 50;
const MAX_BULK_RESCAN_BATCH_SIZE = 100;
const MAX_BULK_RESCAN_STATUS_JOB_IDS = 200;
@@ -25,6 +27,10 @@ const BULK_RESCAN_SAMPLE_LIMIT = 10;
const MAX_STORED_SKILLSPECTOR_ISSUES = 25;
const MAX_STORED_SKILLSPECTOR_TEXT_CHARS = 2_000;
const MAX_STORED_SKILLSPECTOR_SHORT_TEXT_CHARS = 512;
const DEFAULT_SKILL_SCAN_REQUEST_RETENTION_MS = 7 * 24 * 60 * 60 * 1000;
const MAX_SKILL_SCAN_QUEUE_POSITION_READS = 250;
const MAX_SKILL_SCAN_RUNNING_COUNT_READS = 512;
const SKILL_SCAN_ASYNC_NOTE = "Scans are asynchronous and may take time to complete.";
const finalLlmAnalysisStatuses = new Set(["clean", "suspicious", "malicious"]);
const artifactBackedLlmAnalysisStatuses = new Set(["clean", "benign", "suspicious", "malicious"]);
@@ -42,8 +48,10 @@ type CancelSkipReason =
type JobTarget = {
job: Doc<"securityScanJobs">;
skill?: Doc<"skills"> | null;
version?: Doc<"skillVersions">;
release?: Doc<"packageReleases">;
scanRequest?: Doc<"skillScanRequests">;
missing?: true;
};
@@ -206,6 +214,14 @@ const skillSpectorAnalysisValidator = v.object({
checkedAt: v.number(),
});
const scanRequestFileValidator = v.object({
path: v.string(),
size: v.number(),
storageId: v.id("_storage"),
sha256: v.string(),
contentType: v.optional(v.string()),
});
const internalRefs = internal as unknown as {
packages: {
getPackageByIdInternal: unknown;
@@ -215,10 +231,15 @@ const internalRefs = internal as unknown as {
};
securityScan: {
claimQueuedJobsInternal: unknown;
createUploadedSkillScanRequestInternal: unknown;
createPublishedSkillScanRequestInternal: unknown;
enqueuePackageReleaseScanInternal: unknown;
enqueueSkillVersionScanInternal: unknown;
failJobInternal: unknown;
getSkillScanRequestForUserInternal: unknown;
getJobTargetInternal: unknown;
recordSkillScanRequestFailedInternal: unknown;
recordSkillScanRequestSucceededInternal: unknown;
succeedJobInternal: unknown;
};
skills: {
@@ -754,6 +775,442 @@ export const requestSkillRescan = mutation({
},
});
function skillScanRequestExpiresAt(now: number) {
return now + DEFAULT_SKILL_SCAN_REQUEST_RETENTION_MS;
}
function skillScanReportFromRequest(request: Doc<"skillScanRequests">) {
return {
clawscan: request.llmAnalysis ?? null,
skillspector: request.skillSpectorAnalysis ?? null,
staticAnalysis: request.staticScan ?? null,
virustotal: request.vtAnalysis
? {
...request.vtAnalysis,
...request.vtAnalysis.engineStats,
}
: null,
};
}
function skillScanArtifactFromRequest(request: Doc<"skillScanRequests">) {
return {
...(request.slug ? { slug: request.slug } : {}),
...(request.displayName ? { displayName: request.displayName } : {}),
...(request.version ? { version: request.version } : {}),
...(request.sha256hash ? { sha256hash: request.sha256hash } : {}),
fileCount: request.files.length,
};
}
async function countSecurityScanJobs(
ctx: QueryCtx | MutationCtx,
status: Doc<"securityScanJobs">["status"],
source: SecurityScanJobSource,
) {
const jobs = await ctx.db
.query("securityScanJobs")
.withIndex("by_status_source_created_at", (q) => q.eq("status", status).eq("source", source))
.take(MAX_SKILL_SCAN_RUNNING_COUNT_READS + 1);
return {
count: Math.min(jobs.length, MAX_SKILL_SCAN_RUNNING_COUNT_READS),
isEstimate: jobs.length > MAX_SKILL_SCAN_RUNNING_COUNT_READS,
};
}
function compareQueuedScanClaimOrder(a: Doc<"securityScanJobs">, b: Doc<"securityScanJobs">) {
if (a.nextRunAt !== b.nextRunAt) return a.nextRunAt - b.nextRunAt;
if (a._creationTime !== b._creationTime) return a._creationTime - b._creationTime;
return a._id.localeCompare(b._id);
}
async function countQueuedJobsAhead(ctx: QueryCtx | MutationCtx, job: Doc<"securityScanJobs">) {
const candidates = await ctx.db
.query("securityScanJobs")
.withIndex("by_status_source_next_run_at", (q) =>
q.eq("status", "queued").eq("source", job.source).lte("nextRunAt", job.nextRunAt),
)
.order("asc")
.take(MAX_SKILL_SCAN_QUEUE_POSITION_READS + 1);
const queuedAhead = candidates.reduce((count, candidate) => {
if (candidate._id === job._id) return count;
return compareQueuedScanClaimOrder(candidate, job) < 0 ? count + 1 : count;
}, 0);
const sawTarget = candidates.some((candidate) => candidate._id === job._id);
const isEstimate =
!sawTarget ||
candidates.length > MAX_SKILL_SCAN_QUEUE_POSITION_READS ||
queuedAhead > MAX_SKILL_SCAN_QUEUE_POSITION_READS;
return {
queuedAhead: Math.min(queuedAhead, MAX_SKILL_SCAN_QUEUE_POSITION_READS),
isEstimate,
};
}
async function skillScanQueueState(
ctx: QueryCtx | MutationCtx,
job: Doc<"securityScanJobs"> | null,
) {
if (!job) {
return {
queuedAhead: 0,
position: null,
running: 0,
note: SKILL_SCAN_ASYNC_NOTE,
};
}
const running = await countSecurityScanJobs(ctx, "running", job.source);
const queuedAhead =
job.status === "queued"
? await countQueuedJobsAhead(ctx, job)
: { queuedAhead: 0, isEstimate: false };
return {
queuedAhead: queuedAhead.queuedAhead,
queuedAheadIsEstimate: queuedAhead.isEstimate,
position:
job.status === "queued" && !queuedAhead.isEstimate ? queuedAhead.queuedAhead + 1 : null,
running: running.count,
runningIsEstimate: running.isEstimate,
note: SKILL_SCAN_ASYNC_NOTE,
};
}
async function skillScanStatusResponse(
ctx: QueryCtx | MutationCtx,
request: Doc<"skillScanRequests">,
job: Doc<"securityScanJobs"> | null,
) {
const status =
request.status === "succeeded" || request.status === "failed"
? request.status
: (job?.status ?? request.status);
return {
ok: true as const,
scanId: request._id,
jobId: request.securityScanJobId,
status,
sourceKind: request.sourceKind,
update: request.update,
writtenBack: request.writtenBack,
artifact: skillScanArtifactFromRequest(request),
report: skillScanReportFromRequest(request),
queue: await skillScanQueueState(ctx, job),
lastError: request.lastError ?? job?.lastError,
createdAt: request.createdAt,
updatedAt: Math.max(request.updatedAt, job?.updatedAt ?? request.updatedAt),
completedAt: request.completedAt ?? job?.completedAt,
};
}
async function enqueueSkillScanRequestJob(ctx: MutationCtx, requestId: Id<"skillScanRequests">) {
const request = await ctx.db.get(requestId);
if (!request) throw new ConvexError("Scan request not found");
const now = Date.now();
const jobId = await ctx.db.insert("securityScanJobs", {
targetKind: "skillScanRequest",
skillScanRequestId: request._id,
status: "queued",
source: "manual",
priority: 100,
hasMaliciousSignal: false,
waitForVtUntil: now,
nextRunAt: now,
attempts: 0,
createdAt: now,
updatedAt: now,
});
await ctx.db.patch(request._id, {
securityScanJobId: jobId,
updatedAt: now,
});
return jobId;
}
export const createUploadedSkillScanRequestInternal = internalMutation({
args: {
actorUserId: v.id("users"),
files: v.array(scanRequestFileValidator),
displayName: v.optional(v.string()),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId);
if (!actor) throw new ConvexError("Unauthorized");
if (args.files.length === 0) throw new ConvexError("files required");
if (
!args.files.some((file) => {
const lower = file.path.trim().toLowerCase();
return lower === "skill.md";
})
) {
throw new ConvexError("SKILL.md required");
}
const now = Date.now();
const scanId = await ctx.db.insert("skillScanRequests", {
actorUserId: actor._id,
sourceKind: "upload",
update: false,
writtenBack: false,
status: "queued",
displayName: args.displayName,
version: "local",
files: args.files,
expiresAt: skillScanRequestExpiresAt(now),
createdAt: now,
updatedAt: now,
});
const jobId = await enqueueSkillScanRequestJob(ctx, scanId);
await ctx.db.insert("auditLogs", {
actorUserId: actor._id,
action: "skill.clawscan.scan_upload",
targetType: "skillScanRequest",
targetId: scanId,
metadata: {
jobId,
fileCount: args.files.length,
},
createdAt: now,
});
return {
ok: true as const,
scanId,
jobId,
status: "queued" as const,
sourceKind: "upload" as const,
update: false,
alreadyQueued: false,
queue: await skillScanQueueState(ctx, await ctx.db.get(jobId)),
};
},
});
export const createPublishedSkillScanRequestInternal = internalMutation({
args: {
actorUserId: v.id("users"),
slug: v.string(),
version: v.optional(v.string()),
update: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId);
if (!actor) throw new ConvexError("Unauthorized");
const slug = args.slug.trim().toLowerCase();
if (!slug) throw new ConvexError("Slug required");
const skill = await ctx.db
.query("skills")
.withIndex("by_slug", (q) => q.eq("slug", slug))
.unique();
if (!skill || skill.softDeletedAt) throw new ConvexError("Skill not found");
await assertCanManageOwnedResource(ctx, {
actor,
ownerUserId: skill.ownerUserId,
ownerPublisherId: skill.ownerPublisherId,
allowPlatformModerator: true,
});
const requestedVersion = args.version?.trim();
const version = requestedVersion
? await ctx.db
.query("skillVersions")
.withIndex("by_skill_version", (q) =>
q.eq("skillId", skill._id).eq("version", requestedVersion),
)
.unique()
: skill.latestVersionId
? await ctx.db.get(skill.latestVersionId)
: null;
if (!version || version.softDeletedAt) throw new ConvexError("Skill version not found");
const fingerprintEntries = await ctx.db
.query("skillVersionFingerprints")
.withIndex("by_version", (q) => q.eq("versionId", version._id))
.collect();
const files = sourceSkillVersionFiles(version.files, {
generatedBundleFingerprints: fingerprintEntries
.filter((entry) => entry.kind === "generated-bundle")
.map((entry) => entry.fingerprint),
});
const now = Date.now();
const update = args.update === true;
const scanId = await ctx.db.insert("skillScanRequests", {
actorUserId: actor._id,
sourceKind: "published",
update,
writtenBack: false,
status: "queued",
slug: skill.slug,
displayName: skill.displayName,
version: version.version,
skillId: skill._id,
skillVersionId: version._id,
files,
parsed: version.parsed,
sha256hash: version.sha256hash,
vtAnalysis: version.vtAnalysis,
capabilityTags: version.capabilityTags,
staticScan: version.staticScan,
expiresAt: skillScanRequestExpiresAt(now),
createdAt: now,
updatedAt: now,
});
const jobId = await enqueueSkillScanRequestJob(ctx, scanId);
await ctx.db.insert("auditLogs", {
actorUserId: actor._id,
action: update ? "skill.clawscan.scan_published_update" : "skill.clawscan.scan_published",
targetType: "skillVersion",
targetId: version._id,
metadata: {
skillId: skill._id,
slug: skill.slug,
version: version.version,
scanId,
jobId,
update,
},
createdAt: now,
});
return {
ok: true as const,
scanId,
jobId,
status: "queued" as const,
sourceKind: "published" as const,
update,
alreadyQueued: false,
queue: await skillScanQueueState(ctx, await ctx.db.get(jobId)),
};
},
});
export const getSkillScanRequestForUserInternal = internalQuery({
args: {
actorUserId: v.id("users"),
scanId: v.id("skillScanRequests"),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId);
if (!actor) throw new ConvexError("Unauthorized");
const request = await ctx.db.get(args.scanId);
if (!request) throw new ConvexError("Scan not found");
if (request.actorUserId !== actor._id && actor.role !== "admin" && actor.role !== "moderator") {
throw new ConvexError("Forbidden");
}
const job = request.securityScanJobId ? await ctx.db.get(request.securityScanJobId) : null;
return await skillScanStatusResponse(ctx, request, job);
},
});
export const recordSkillScanRequestSucceededInternal = internalMutation({
args: {
scanId: v.id("skillScanRequests"),
jobId: v.id("securityScanJobs"),
runId: v.optional(v.string()),
llmAnalysis: llmAnalysisValidator,
skillSpectorAnalysis: v.optional(skillSpectorAnalysisValidator),
writtenBack: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const request = await ctx.db.get(args.scanId);
if (!request) throw new ConvexError("Scan request not found");
const now = Date.now();
await ctx.db.patch(request._id, {
status: "succeeded",
llmAnalysis: args.llmAnalysis,
...(args.skillSpectorAnalysis
? { skillSpectorAnalysis: capSkillSpectorAnalysisForStorage(args.skillSpectorAnalysis) }
: {}),
writtenBack: args.writtenBack === true || request.writtenBack,
runId: args.runId,
completedAt: now,
updatedAt: now,
});
return { ok: true as const };
},
});
export const recordSkillScanRequestFailedInternal = internalMutation({
args: {
scanId: v.id("skillScanRequests"),
error: v.string(),
llmAnalysis: v.optional(llmAnalysisValidator),
},
handler: async (ctx, args) => {
const request = await ctx.db.get(args.scanId);
if (!request) throw new ConvexError("Scan request not found");
const now = Date.now();
await ctx.db.patch(request._id, {
status: "failed",
lastError: args.error.slice(0, 2000),
...(args.llmAnalysis ? { llmAnalysis: args.llmAnalysis } : {}),
completedAt: now,
updatedAt: now,
});
return { ok: true as const };
},
});
export const pruneExpiredSkillScanRequestsInternal = internalMutation({
args: {
batchSize: v.optional(v.number()),
},
handler: async (ctx, args) => {
const batchSize = Math.max(
1,
Math.min(
args.batchSize ?? DEFAULT_PRUNE_SKILL_SCAN_REQUEST_LIMIT,
MAX_PRUNE_SKILL_SCAN_REQUEST_LIMIT,
),
);
const now = Date.now();
const requests = await ctx.db
.query("skillScanRequests")
.withIndex("by_expires_at", (q) => q.lt("expiresAt", now))
.take(batchSize);
let deletedJobs = 0;
let deletedFiles = 0;
for (const request of requests) {
if (request.securityScanJobId) {
const job = await ctx.db.get(request.securityScanJobId);
if (job?.targetKind === "skillScanRequest") {
await ctx.db.delete(job._id);
deletedJobs += 1;
}
}
if (request.sourceKind === "upload") {
for (const file of request.files) {
try {
await ctx.storage.delete(file.storageId);
deletedFiles += 1;
} catch {
// Missing storage objects should not block expiry of the request row.
}
}
}
await ctx.db.delete(request._id);
}
return {
ok: true as const,
deletedRequests: requests.length,
deletedJobs,
deletedFiles,
done: requests.length < batchSize,
};
},
});
async function requestPackageRescanForActor(
ctx: MutationCtx,
args: {
@@ -1168,6 +1625,13 @@ export const claimQueuedJobsInternal = internalMutation({
lastError: undefined,
updatedAt: now,
});
if (job.targetKind === "skillScanRequest" && job.skillScanRequestId) {
await ctx.db.patch(job.skillScanRequestId, {
status: "running",
lastError: undefined,
updatedAt: now,
});
}
claimed.push({
...job,
status: "running" as const,
@@ -1206,6 +1670,15 @@ export const getJobTargetInternal = internalQuery({
trustedOpenClawPlugin: isOpenClawPluginPackage(pkg, ownerPublisher),
};
}
if (job.targetKind === "skillScanRequest" && job.skillScanRequestId) {
const scanRequest = await ctx.db.get(job.skillScanRequestId);
if (!scanRequest) return { job, missing: true as const };
const version = scanRequest.skillVersionId
? await ctx.db.get(scanRequest.skillVersionId)
: null;
const skill = scanRequest.skillId ? await ctx.db.get(scanRequest.skillId) : null;
return { job, skill, version: version ?? undefined, scanRequest };
}
return { job, missing: true as const };
},
});
@@ -1252,6 +1725,14 @@ export const failJobInternal = internalMutation({
workerId: undefined,
updatedAt: now,
});
if (job.targetKind === "skillScanRequest" && job.skillScanRequestId) {
await ctx.db.patch(job.skillScanRequestId, {
status: retry ? "queued" : "failed",
lastError: args.error.slice(0, 2000),
...(retry ? {} : { completedAt: now }),
updatedAt: now,
});
}
return { ok: true as const, retry };
},
});
@@ -1291,6 +1772,7 @@ export const claimCodexScanJobs = action({
continue;
}
const scanRequest = target.scanRequest as Doc<"skillScanRequests"> | undefined;
const version = target.version as Doc<"skillVersions"> | undefined;
const release = target.release as Doc<"packageReleases"> | undefined;
let files: Array<{
@@ -1300,7 +1782,9 @@ export const claimCodexScanJobs = action({
storageId: Id<"_storage">;
contentType?: string;
}> = [];
if (version) {
if (scanRequest) {
files = scanRequest.files;
} else if (version) {
const fingerprintEntries = await runQueryRef<
Array<{ fingerprint: string; kind?: "source" | "generated-bundle" }>
>(ctx, internalRefs.skills.listVersionFingerprintsInternal, {
@@ -1406,6 +1890,33 @@ export const completeCodexScanJob = action({
releaseId: target.release._id,
llmAnalysis: args.llmAnalysis,
});
} else if (target.job.targetKind === "skillScanRequest" && target.scanRequest) {
let writtenBack = false;
if (
target.scanRequest.sourceKind === "published" &&
target.scanRequest.update &&
target.version
) {
if (args.skillSpectorAnalysis) {
await runMutationRef(ctx, internalRefs.skills.updateVersionSkillSpectorAnalysisInternal, {
versionId: target.version._id,
skillSpectorAnalysis: capSkillSpectorAnalysisForStorage(args.skillSpectorAnalysis),
});
}
await runMutationRef(ctx, internalRefs.skills.updateVersionLlmAnalysisInternal, {
versionId: target.version._id,
llmAnalysis: args.llmAnalysis,
});
writtenBack = true;
}
await runMutationRef(ctx, internalRefs.securityScan.recordSkillScanRequestSucceededInternal, {
scanId: target.scanRequest._id,
jobId: args.jobId,
runId: args.runId,
llmAnalysis: args.llmAnalysis,
skillSpectorAnalysis: args.skillSpectorAnalysis,
writtenBack,
});
} else {
throw new ConvexError("Unsupported security scan target");
}
@@ -1462,6 +1973,16 @@ export const failCodexScanJob = action({
llmAnalysis,
});
}
} else if (target.job.targetKind === "skillScanRequest" && target.scanRequest) {
await runMutationRef(
ctx,
internalRefs.securityScan.recordSkillScanRequestFailedInternal,
{
scanId: target.scanRequest._id,
error: args.error,
llmAnalysis,
},
);
}
}
}
+59 -1
View File
@@ -1,6 +1,7 @@
import { describe, expect, it, vi } from "vitest";
import {
applyBanToOwnedSkillsBatchInternal,
applyPublisherDeletionToOwnedSkillsBatchInternal,
restoreOwnedSkillsForUnbanBatchInternal,
} from "./skills";
@@ -22,6 +23,13 @@ const applyBanHandler = (
>
)._handler;
const applyPublisherDeletionHandler = (
applyPublisherDeletionToOwnedSkillsBatchInternal as unknown as WrappedHandler<
{ ownerPublisherId: string; actorUserId: string; deletedAt: number; cursor?: string },
{ hiddenCount: number; scheduled: boolean; stale?: boolean }
>
)._handler;
function makeCtx({
user,
skills = [],
@@ -53,7 +61,11 @@ function makeCtx({
return {
ctx: {
db: {
get: vi.fn(async (id: string) => (id === "users:owner" ? user : null)),
get: vi.fn(async (id: string) => {
if (id === "users:owner") return user;
if (id === "publishers:org") return { _id: id, kind: "org", deletedAt: 3_000 };
return null;
}),
insert: vi.fn(),
patch,
replace: vi.fn(),
@@ -70,6 +82,52 @@ function makeCtx({
}
describe("skills ban/unban batches", () => {
it("soft-deletes active skills for a deleted publisher", async () => {
const { ctx, patch } = makeCtx({
user: { _id: "users:owner", deletedAt: undefined, deactivatedAt: undefined },
skills: [
{
_id: "skills:org-skill",
ownerUserId: "users:owner",
ownerPublisherId: "publishers:org",
softDeletedAt: undefined,
moderationStatus: "active",
moderationFlags: undefined,
stats: {
downloads: 0,
stars: 0,
comments: 0,
versions: 1,
installsCurrent: 0,
installsAllTime: 0,
},
},
],
});
await expect(
applyPublisherDeletionHandler(ctx, {
ownerPublisherId: "publishers:org",
actorUserId: "users:owner",
deletedAt: 3_000,
}),
).resolves.toEqual({
ok: true,
hiddenCount: 1,
scheduled: false,
});
expect(patch).toHaveBeenCalledWith(
"skills:org-skill",
expect.objectContaining({
softDeletedAt: 3_000,
moderationStatus: "hidden",
moderationReason: "publisher.deleted",
hiddenBy: "users:owner",
}),
);
});
it("retimestamps earlier ban-hidden skills during a later ban", async () => {
const { ctx, patch, scheduler } = makeCtx({
user: { _id: "users:owner", deletedAt: 2_000 },
+266
View File
@@ -0,0 +1,266 @@
/* @vitest-environment node */
import { describe, expect, it, vi } from "vitest";
import schema from "./schema";
vi.mock("@convex-dev/auth/server", () => ({
getAuthUserId: vi.fn(),
authTables: {},
}));
const { __test, listPublicPageV4 } = await import("./skills");
const listPublicPageV4Handler = (
listPublicPageV4 as unknown as {
_handler: (
ctx: unknown,
args: unknown,
) => Promise<{ page: Array<{ skill: { slug: string } }> }>;
}
)._handler;
describe("skills.listPublicPageV4", () => {
it("defines recommended rank indexes in contract order", () => {
expect(getSkillSearchDigestIndexFields("by_active_recommended_rank")).toEqual([
"softDeletedAt",
"statsStars",
"statsInstallsAllTime",
"statsDownloads",
"updatedAt",
]);
expect(getSkillSearchDigestIndexFields("by_nonsuspicious_recommended_rank")).toEqual([
"softDeletedAt",
"isSuspicious",
"statsStars",
"statsInstallsAllTime",
"statsDownloads",
"updatedAt",
]);
});
it("forces Recommended ranking to descending for stale URLs", () => {
expect(__test.resolvePublicListDir("recommended", "asc")).toBe("desc");
expect(__test.resolvePublicListDir("default", "asc")).toBe("desc");
});
it("keeps explicit non-default sort directions", () => {
expect(__test.resolvePublicListDir("name", undefined)).toBe("asc");
expect(__test.resolvePublicListDir("downloads", "asc")).toBe("asc");
});
it("keeps recommended-rank cursors on the index that created them", () => {
expect(
__test.resolveRecommendedPublicListSort({
decodedCursor: null,
hasMissingRankStats: false,
}),
).toBe("recommended");
expect(
__test.resolveRecommendedPublicListSort({
decodedCursor: null,
hasMissingRankStats: true,
}),
).toBe("updated");
expect(
__test.resolveRecommendedPublicListSort({
decodedCursor: [undefined, 123, 456, "skillSearchDigest:updated"],
hasMissingRankStats: false,
}),
).toBe("updated");
expect(
__test.resolveRecommendedPublicListSort({
decodedCursor: [undefined, false, 123, 456, "skillSearchDigest:nonsuspicious-updated"],
hasMissingRankStats: false,
}),
).toBe("updated");
expect(
__test.resolveRecommendedPublicListSort({
decodedCursor: [undefined, 10, 20, 30, 123, 456, "skillSearchDigest:recommended"],
hasMissingRankStats: true,
}),
).toBe("recommended");
expect(
__test.resolveRecommendedPublicListSort({
decodedCursor: [
undefined,
false,
10,
20,
30,
123,
456,
"skillSearchDigest:nonsuspicious-recommended",
],
hasMissingRankStats: true,
}),
).toBe("recommended");
});
it("sorts highlighted recommended results by stars, installs, downloads, then updatedAt", async () => {
const result = await listPublicPageV4Handler(
makeHighlightedCtx([
makeDigest({
id: "updated",
slug: "updated-skill",
stars: 2,
installsAllTime: 10,
downloads: 10,
updatedAt: 400,
}),
makeDigest({
id: "downloads",
slug: "downloads-skill",
stars: 2,
installsAllTime: 10,
downloads: 50,
updatedAt: 100,
}),
makeDigest({
id: "installs",
slug: "installs-skill",
stars: 2,
installsAllTime: 20,
downloads: 0,
updatedAt: 100,
}),
makeDigest({
id: "stars",
slug: "stars-skill",
stars: 3,
installsAllTime: 0,
downloads: 0,
updatedAt: 100,
}),
]),
{ highlightedOnly: true, numItems: 10 },
);
expect(result.page.map((entry) => entry.skill.slug)).toEqual([
"stars-skill",
"installs-skill",
"downloads-skill",
"updated-skill",
]);
});
});
function getSkillSearchDigestIndexFields(indexDescriptor: string) {
const index = schema.tables.skillSearchDigest[" indexes"]().find(
(candidate) => candidate.indexDescriptor === indexDescriptor,
);
if (!index) throw new Error(`Missing skillSearchDigest index ${indexDescriptor}`);
return index.fields;
}
type EqBuilder = {
eq: (field: string, value: unknown) => EqBuilder;
getLastValue: () => unknown;
};
function makeEqBuilder(): EqBuilder {
let lastValue: unknown;
const builder: EqBuilder = {
eq: (_field, value) => {
lastValue = value;
return builder;
},
getLastValue: () => lastValue,
};
return builder;
}
function makeHighlightedCtx(digests: Array<ReturnType<typeof makeDigest>>) {
const digestBySkillId = new Map(digests.map((digest) => [digest.skillId, digest]));
return {
db: {
query: vi.fn((table: string) => {
if (table === "skillBadges") {
return {
withIndex: vi.fn((_indexName: string, build: (q: EqBuilder) => unknown) => {
build(makeEqBuilder());
return {
order: vi.fn(() => ({
take: vi.fn().mockResolvedValue(
digests.map((digest) => ({
_id: `skillBadges:${digest.skillId}`,
skillId: digest.skillId,
kind: "highlighted",
awardedAt: digest.updatedAt,
})),
),
})),
};
}),
};
}
if (table === "skillSearchDigest") {
return {
withIndex: vi.fn((_indexName: string, build: (q: EqBuilder) => unknown) => {
const eqBuilder = makeEqBuilder();
build(eqBuilder);
const skillId = eqBuilder.getLastValue();
return {
unique: vi
.fn()
.mockResolvedValue(
typeof skillId === "string" ? (digestBySkillId.get(skillId) ?? null) : null,
),
};
}),
};
}
throw new Error(`Unexpected table ${table}`);
}),
},
};
}
function makeDigest(params: {
id: string;
slug: string;
stars: number;
installsAllTime: number;
downloads: number;
updatedAt: number;
}) {
return {
_id: `skillSearchDigest:${params.id}`,
_creationTime: params.updatedAt,
skillId: `skills:${params.id}`,
slug: params.slug,
displayName: params.slug,
summary: `${params.slug} summary`,
ownerUserId: "users:owner",
ownerPublisherId: undefined,
ownerHandle: "owner",
ownerKind: "user",
ownerName: "owner",
ownerDisplayName: "Owner",
ownerImage: undefined,
canonicalSkillId: undefined,
forkOf: undefined,
latestVersionId: undefined,
latestVersionSummary: undefined,
tags: {},
capabilityTags: [],
badges: undefined,
stats: {
downloads: params.downloads,
installsCurrent: 0,
installsAllTime: params.installsAllTime,
stars: params.stars,
versions: 1,
comments: 0,
},
statsDownloads: params.downloads,
statsStars: params.stars,
statsInstallsCurrent: 0,
statsInstallsAllTime: params.installsAllTime,
softDeletedAt: undefined,
moderationStatus: "active",
moderationFlags: [],
moderationReason: undefined,
isSuspicious: false,
createdAt: 1,
updatedAt: params.updatedAt,
};
}
+11
View File
@@ -63,6 +63,15 @@ function chainEq(constraints: Record<string, unknown>) {
};
}
const defaultSkillStats = {
downloads: 0,
stars: 0,
installsCurrent: 0,
installsAllTime: 0,
versions: 1,
comments: 0,
};
describe("skills ownership", () => {
it("resolves alias slugs to the live target skill", async () => {
const result = await getSkillBySlugInternalHandler(
@@ -430,6 +439,7 @@ describe("skills ownership", () => {
softDeletedAt: undefined,
moderationVerdict: "clean",
moderationReasonCodes: ["suspicious.dynamic_code_execution"],
stats: defaultSkillStats,
};
const aliases = [
{
@@ -688,6 +698,7 @@ describe("skills ownership", () => {
ownerUserId: "users:actor",
ownerPublisherId: "publishers:actor",
softDeletedAt: undefined,
stats: defaultSkillStats,
};
const aliases = [
{
+83 -3
View File
@@ -33,7 +33,15 @@ type WrappedHandler<TArgs, TResult> = {
type PublicListArgs = {
cursor?: string;
numItems?: number;
sort?: "newest" | "updated" | "downloads" | "installs" | "stars" | "name";
sort?:
| "default"
| "recommended"
| "newest"
| "updated"
| "downloads"
| "installs"
| "stars"
| "name";
dir?: "asc" | "desc";
highlightedOnly?: boolean;
nonSuspiciousOnly?: boolean;
@@ -133,12 +141,81 @@ function cursorForIndex(index: string, key: unknown[]): string {
return JSON.stringify({ v: 1, index, key });
}
class TestEqBuilder {
eq(_field: string, _value: unknown) {
return this;
}
}
function makeMissingRecommendedRankStatsCtx() {
const first = vi.fn(async () => makeSearchDigest({ statsStars: undefined }));
const withIndex = vi.fn((_indexName: string, build: (q: TestEqBuilder) => unknown) => {
build(new TestEqBuilder());
return { first };
});
const query = vi.fn((table: string) => {
if (table !== "skillSearchDigest") throw new Error(`unexpected table ${table}`);
return { withIndex };
});
return {
ctx: { db: { query } },
first,
query,
withIndex,
};
}
describe("public skill list deterministic cursors", () => {
beforeEach(() => {
getPageMock.mockReset();
getPageMock.mockResolvedValue({ page: [], hasMore: false, indexKeys: [] });
});
it("falls back to the updated index while default rank stats are missing", async () => {
const { ctx, withIndex } = makeMissingRecommendedRankStatsCtx();
await listPublicPageV4Handler(ctx, {
numItems: 10,
});
expect(withIndex.mock.calls.map(([indexName]) => indexName)).toEqual([
"by_active_stats_stars",
"by_active_stats_installs_all_time",
"by_active_stats_downloads",
]);
expect(getPageMock).toHaveBeenCalledTimes(1);
expect(getPageMock.mock.calls[0]?.[1]).toMatchObject({
index: "by_active_updated",
startIndexKey: [undefined],
endIndexKey: [undefined],
startInclusive: true,
});
});
it("falls back to the non-suspicious updated index while default rank stats are missing", async () => {
const { ctx, withIndex } = makeMissingRecommendedRankStatsCtx();
await listPublicApiPageV1Handler(ctx, {
numItems: 10,
sort: "recommended",
nonSuspiciousOnly: true,
});
expect(withIndex.mock.calls.map(([indexName]) => indexName)).toEqual([
"by_nonsuspicious_stars",
"by_nonsuspicious_installs",
"by_nonsuspicious_downloads",
]);
expect(getPageMock).toHaveBeenCalledTimes(1);
expect(getPageMock.mock.calls[0]?.[1]).toMatchObject({
index: "by_nonsuspicious_updated",
startIndexKey: [undefined, false],
endIndexKey: [undefined, false],
startInclusive: true,
});
});
it("ignores stale legacy cursors that are longer than the selected index", async () => {
const staleDownloadsCursor = legacyCursor([{ __undef: 1 }, false, 100, 200]);
@@ -368,7 +445,10 @@ describe("public skill list deterministic cursors", () => {
indexKeys: [],
});
const result = await listPublicApiPageV1Handler({} as never, { numItems: 10 });
const result = await listPublicApiPageV1Handler({} as never, {
numItems: 10,
sort: "updated",
});
expect(result.items).toHaveLength(1);
expect(result.items[0]).toMatchObject({ latestVersion: null });
@@ -400,7 +480,7 @@ describe("public skill list deterministic cursors", () => {
),
},
} as never,
{ numItems: 10 },
{ numItems: 10, sort: "updated" },
);
expect(result.items).toHaveLength(1);

Some files were not shown because too many files have changed in this diff Show More