Compare commits

...
Author SHA1 Message Date
Peter Steinberger fa3849069d fix(api): centralize v1 soft-delete error mapping 2026-02-14 21:52:43 +01:00
Sergiy Dybskiy ed4c191dfc test: add e2e test for delete error handling
Verifies that deleting a non-existent skill returns a proper 'not found'
error instead of a generic 'Unauthorized' message.
2026-02-14 21:49:21 +01:00
Sergiy Dybskiy 9e704e0a81 fix: return proper HTTP status codes for delete/undelete errors
The delete and undelete handlers for skills and souls were catching all
errors and returning 401 Unauthorized, even for errors like:
- 'Skill not found' (should be 404)
- 'Forbidden' (should be 403)
- Other validation errors (should be 400)

This change updates the error handling to return appropriate status codes:
- 401 Unauthorized: authentication failures
- 403 Forbidden: authorization failures (not owner/admin/moderator)
- 404 Not Found: skill/soul/user not found
- 400 Bad Request: other errors with descriptive message

Fixes #34
2026-02-14 21:48:52 +01:00
Matt Krokosz f05dd556db fix: gate publish by immutable GitHub account ID 2026-02-14 20:25:15 +01:00
964893a622 fix: handle duplicate Convex Auth user records in publish ownership check (#180)
* fix: handle duplicate user records in publish ownership check

* fix: heal publish ownership via GitHub auth identity

---------

Co-authored-by: Emmet Brown <emmet@Emmets-Mac-mini.local>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 19:39:04 +01:00
Peter Steinberger 9a804b951f refactor: batch resolve tags in v1 API (#112) (thanks @mkrokosz) 2026-02-14 19:01:02 +01:00
Matthew KrokoszandClaude Opus 4.5 d699087786 fix: add null guard and short-circuit for empty tags
- Short-circuit when no version IDs to resolve
- Add null coalescing for runQuery response
- Fixes potential crash when tags are empty or query returns null

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 19:01:02 +01:00
Matthew KrokoszandClaude Opus 4.5 26b42727fe perf: batch tag resolution to reduce action→query round-trips
- Add getVersionsByIds batch query to skills.ts and souls.ts
- Replace per-item tag resolution with batch resolution in httpApiV1.ts
- Reduces N action→query round-trips to 1 for list endpoints

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 19:01:02 +01:00
Peter Steinberger 8756b78a4e chore: drop convex-helpers (#302) 2026-02-14 17:54:24 +01:00
e9c771d55d fix(skills): keep global sorting across pagination (#98)
* fix: initial skill sorting

* chore: update unit test

* fix: use correct indexes for skill sorting

* chore: cleanup

* fix(skills): preserve server order for paginated sorting

* chore(lint): apply biome formatting fixes

* chore(convex): bump tsconfig lib to ES2022

* fix(skills): add deterministic tie-breaker for search sorting

* fix(skills): stable sorting across pagination (#98) (thanks @CodeBBakGoSu)

---------

Co-authored-by: Brian Kasper <bkasperr@gmail.com>
Co-authored-by: knox-glorang <knox@glorang.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 17:44:07 +01:00
Peter Steinberger a58f0166fa refactor: centralize CORS + CLI auth token (#297)
* refactor(convex): centralize CORS headers

* refactor(cli): centralize auth token lookup
2026-02-14 15:31:03 +01:00
Peter SteinbergerandGrenghis-Khan 4328d4d700 fix(cors): complete CORS + tokenized CLI reads (#296)
* fix(cors): add Access-Control-Allow-Origin headers to API and downloads

* fix: add CORS to error/raw paths & add CLI install auth

* fix: add OPTIONS handler for CORS preflight

* fix(cors): complete CORS + tokenized CLI reads

* test(cli): fix config mock typing

---------

Co-authored-by: Grenghis-Khan <63885013+Grenghis-Khan@users.noreply.github.com>
2026-02-14 14:45:15 +01:00
Peter Steinberger 28ee2618c1 refactor: simplify user ensure updates 2026-02-14 13:56:15 +01:00
Peter Steinberger a4b850ec33 feat: improve moderation/admin UX + language-aware quality gate
- API: owner-visible responses for hidden/soft-deleted skills\n- Admin: add unban user mutations + docs\n- Quality: Intl.Segmenter tokenization + CJK signal to reduce false rejects\n- Jobs: skill-stat-events interval 15m -> 5m\n- Tests: add coverage for owner-visible states + non-Latin docs\n- Changelog: add Unreleased entry
2026-02-14 13:54:03 +01:00
Peter Steinberger 7e0b21f7c8 fix: sync handle on user ensure (#293) (thanks @christianhpoe) 2026-02-14 13:48:56 +01:00
ChristianHPoe 71c6705ab1 fix: sync handle on user ensure 2026-02-14 13:48:56 +01:00
a57769771f fix: add retry logic for OpenAI embedding API failures (#272)
* fix: add retry logic for OpenAI embedding API failures

Fixes #149

When importing or uploading skills, the OpenAI embedding API call could
fail with transient errors (rate limits, timeouts, network issues),
causing the entire import to fail with a generic "Server Error".

This adds retry logic with exponential backoff (1s, 2s, 4s delays):
- Retries on 429 (rate limit) and 5xx server errors
- Retries on network/fetch errors
- Logs warnings for debugging
- Max 3 retries before failing with clear error message

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct retry count and broaden network error catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address retry loop off-by-one, broaden error catch, preserve original error

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden embeddings retry semantics

* style: format embeddings retry changes

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 04:54:18 +01:00
Peter Steinberger 6a2c131a8a style: remove residual blue accents and warm base palette 2026-02-14 04:40:50 +01:00
Peter Steinberger 67ac157545 fix: keep new skill versions pending until VT verdict 2026-02-14 02:53:02 +01:00
Peter Steinberger ef36cfd698 refactor(cli): centralize HTTP status errors and timeout tests (#286) 2026-02-14 02:39:42 +01:00
Peter SteinbergerandSash Zats e0637ad6aa fix(cli): throw Error for all timeout aborts (#283)
* fix(cli): throw Error on timeout aborts

Users have seen an elevated number of:\n  clawdhub search image\n  ✖ Non-error was thrown: "Timeout". You should only throw errors.\n\nInvestigation shows we were aborting with a string instead of an Error. Switching to controller.abort(new Error('Timeout')) makes retries/formatting treat it as a real error and clears the message.\n\nExample after change:\n  clawdhub search image\n  table-image v1.0.0  Table Image  (0.332)\n  nano-banana-pro v1.0.1  Nano Banana Pro  (0.319)\n  vap-media v1.0.1  AI media generation API - Flux2pro, Veo3.1, Suno Ai  (0.281)\n  clawdbot-meshyai-skill v0.1.0  Meshy AI  (0.276)\n  venice-ai-media v1.0.0  Venice AI Media  (0.274)\n  daily-recap v1.0.2  Daily Recap  (0.260)\n  openai-image-gen v1.0.1  Openai Image Gen  (0.260)\n  bible-votd v1.0.1  Bible Verse of the Day  (0.248)\n  orf v1.0.1  ORF  (0.224)\n  smalltalk v1.0.1  Smalltalk  (0.161)

* fix(http): wrap fetch calls in try-finally to prevent timer leaks

Addresses Vercel review comment: clearTimeout was not called on error paths when fetch throws an exception.

* fix(cli): unify timeout abort handling

---------

Co-authored-by: Sash Zats <sash@zats.io>
2026-02-14 02:27:34 +01:00
Peter Steinberger 09a21a07ff chore: add oxfmt config 2026-02-14 02:15:12 +01:00
Peter Steinberger 65a14dcef3 feat: make account deletion irreversible and migrate lint to oxlint 2026-02-14 02:15:01 +01:00
Peter Steinberger 97c12b2327 refactor(comments): extract handlers and harden mutation tests 2026-02-14 01:56:34 +01:00
a290c81a75 fix(comments): stop updating skills.updatedAt on comment add/remove (#55)
* fix(comments): stop updating skills.updatedAt on comment add/remove

Comments are not content changes, so they shouldn't invalidate skill
list queries that depend on updatedAt. This reduces query invalidation
when users add or remove comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(comments): add updatedAt invalidation regression coverage

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 01:53:40 +01:00
Peter Steinberger cc5d5cfee5 style: restore brown palette and dark-mode CTA tone 2026-02-14 01:18:29 +01:00
Peter Steinberger 03cd710abc fix: dedupe download metrics hourly by user-or-ip identity (#278) 2026-02-14 01:13:52 +01:00
Peter Steinberger 287f639fbc fix: simplify skills CTA label 2026-02-14 00:56:52 +01:00
Peter Steinberger aecf66981c fix: show stars in popular skill cards 2026-02-14 00:55:24 +01:00
Peter Steinberger db8090f287 style: darken hero primary CTA in dark mode 2026-02-14 00:50:52 +01:00
Peter Steinberger ae8614fa98 style: remove remaining warm accent literals 2026-02-14 00:38:21 +01:00
Peter Steinberger e7f78ea5a3 style: shift UI palette to cool blue tones 2026-02-14 00:31:55 +01:00
Peter Steinberger b9355f7a0c docs: thank @GhadiSaab for #53 2026-02-14 00:16:24 +01:00
ghadi saabandPeter Steinberger 9530676f8a fix: resolve search timeout and improve skills page UI alignment (#53)
* fix: resolve search timeout and improve skills page UI alignment

- Added a 10s timeout to OpenAI embedding requests to prevent hanging searches.
- Fixed a TypeScript error in search.ts regarding entry hydration.
- Restructured skills page layout and CSS to ensure consistent alignment between the search toolbar and skill cards.

* fix: resolve search timeout and improve skills page UI alignment

- Added a 10s timeout to OpenAI embedding requests to prevent hanging searches.
- Fixed a TypeScript error in search.ts regarding entry hydration.
- Restructured skills page layout and CSS to ensure consistent alignment between the search toolbar and skill cards.

* style: format skills index layout block

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 00:15:47 +01:00
Peter Steinberger ef23520d22 style: refine global UI theme, borders, and spacing 2026-02-13 23:50:24 +01:00
Peter Steinberger e67a6e6400 fix: render homepage popular cards from nested skill entries 2026-02-13 22:52:21 +01:00
Peter Steinberger e6871b86e1 fix: normalize legacy skill stats to prevent homepage crash 2026-02-13 22:45:34 +01:00
Peter Steinberger 75937e8b53 feat: show popular non-suspicious skills on homepage 2026-02-13 21:38:56 +01:00
Peter Steinberger d2919791d1 style: polish upload page layout and actions 2026-02-13 21:35:25 +01:00
Peter Steinberger 9019cd8462 perf: short-circuit empty skill summary generation 2026-02-13 21:21:09 +01:00
Peter Steinberger 5e58bd459e feat: add self-scheduling skill summary backfill job 2026-02-13 21:10:51 +01:00
Peter Steinberger 3badf0668f fix: make skill summary backfill resumable 2026-02-13 20:43:46 +01:00
Peter Steinberger c719297d70 feat: auto-generate missing skill summaries 2026-02-13 20:35:03 +01:00
Peter Steinberger e19cd23be2 fix: force auth redirects and registry to canonical clawhub host 2026-02-13 20:22:08 +01:00
Peter Steinberger 9266fb7c20 fix: add privileged-owner suspicious flag reconciler 2026-02-13 20:01:39 +01:00
Peter Steinberger 99645d2c27 fix: bypass suspicious flags for privileged owners and polish comment delete UI 2026-02-13 19:56:53 +01:00
Peter Steinberger a1ad7fac85 fix: force canonical downloads sort in skills browse mode 2026-02-13 19:33:26 +01:00
Peter Steinberger ebd2f12cc4 fix: enforce downloads as canonical default skills sort 2026-02-13 19:27:27 +01:00
Peter Steinberger e2ee7b164c feat: default skills sort to downloads 2026-02-13 19:15:19 +01:00
Peter Steinberger 36ed062739 style: polish selected states in skills toolbar 2026-02-13 19:12:48 +01:00
Peter Steinberger d12d6e3926 feat: add non-suspicious skills filter toggle 2026-02-13 19:05:46 +01:00
Peter Steinberger 1851a9c01f fix: make empty-skill cleanup resumable 2026-02-13 17:54:47 +01:00
Peter Steinberger bbeb0be343 feat: add empty-skill cleanup backfill with ban nominations 2026-02-13 17:48:58 +01:00
Peter Steinberger 9c22fb7e54 test: expand reauth ban regression coverage 2026-02-13 17:33:58 +01:00
Peter Steinberger ef2403179b fix: prevent autobanned users from self-reactivating 2026-02-13 17:28:00 +01:00
Peter Steinberger df178d4bfc fix: enforce quality gate and trust-tier spam checks 2026-02-13 17:16:53 +01:00
Peter Steinberger 318cdd33c5 docs: add git local-branch cleanup fallback 2026-02-13 17:03:38 +01:00
Peter Steinberger 9087b037dd fix: add skill publish anti-spam caps and quarantine 2026-02-13 16:58:18 +01:00
Peter Steinberger 6991569a1c fix: replace skill report prompt with modal 2026-02-13 16:47:16 +01:00
Peter Steinberger 32bc600be4 fix: harden skill listing and rate limiting under load 2026-02-13 16:39:57 +01:00
Kevin Kern a52a37d08c fix: harden download rate limiting and dedupe (#43) (thanks @regenrek)
- add download-specific rate limit tier\n- add per-IP/day dedupe + daily pruning\n- keep moderation gating + deterministic zips\n- add optional forwarded-IP trust via TRUST_FORWARDED_IPS
2026-02-13 16:22:05 +01:00
Brian KasperandPeter Steinberger dd58dd0815 Fix initial skill sorting (#92)
* fix: initial skill sorting

* chore: update unit test

* fix: use correct indexes for skill sorting

* chore: cleanup

* fix: land skill sorting update (#92) (thanks @bpk9)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 16:01:37 +01:00
Tanuj BhaudandPeter Steinberger 37a35c955a fix(vt): explicit return types and missing undici dependency (#255)
* fix(vt): explicit return types and missing undici dependency

Refactor action handlers in convex/vt.ts to use explicit return types, resolving circular type inference (TS7022). Also add undici to devDependencies for E2E tests.

* fix: add root undici devDependency for e2e (#255) (thanks @tanujbhaud)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 15:25:03 +01:00
108 changed files with 6467 additions and 1175 deletions
+20
View File
@@ -0,0 +1,20 @@
{
"$schema": "./node_modules/oxfmt/configuration_schema.json",
"experimentalSortImports": {
"newlinesBetween": false,
},
"experimentalSortPackageJson": {
"sortScripts": true,
},
"ignorePatterns": [
".output/",
".tanstack/",
"convex/_generated/",
"coverage/",
"dist/",
"node_modules/",
"public/",
"src/routeTree.gen.ts",
"test-results/",
],
}
+35 -1
View File
@@ -1,3 +1,37 @@
{
"ignorePatterns": ["node_modules", "dist", "coverage", "convex/_generated", ".tanstack", "public"]
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": ["unicorn", "typescript", "oxc"],
"categories": {
"correctness": "error",
"perf": "error",
"suspicious": "error"
},
"rules": {
"curly": "off",
"eslint-plugin-unicorn/prefer-array-find": "off",
"eslint-plugin-unicorn/no-array-sort": "off",
"eslint/no-await-in-loop": "off",
"eslint/no-new": "off",
"oxc/no-accumulating-spread": "off",
"oxc/no-async-endpoint-handlers": "off",
"oxc/no-map-spread": "off",
"typescript/no-explicit-any": "error",
"typescript/no-extraneous-class": "off",
"typescript/no-unnecessary-boolean-literal-compare": "off",
"typescript/no-unnecessary-type-assertion": "off",
"typescript/no-unsafe-type-assertion": "off",
"unicorn/consistent-function-scoping": "off",
"unicorn/require-post-message-target-origin": "off"
},
"ignorePatterns": [
".output/",
".tanstack/",
"convex/_generated/",
"coverage/",
"dist/",
"node_modules/",
"public/",
"src/routeTree.gen.ts",
"test-results/"
]
}
+3
View File
@@ -34,6 +34,9 @@
- Keep changes scoped; avoid repo-wide search/replace.
- PRs: include summary + test commands run. Add screenshots for UI changes.
## Git Notes
- If `git branch -d/-D <branch>` is policy-blocked, delete the local ref directly: `git update-ref -d refs/heads/<branch>`.
## Configuration & Security
- Local env: `.env.local` (never commit secrets).
- Convex env holds JWT keys; Vercel only needs `VITE_CONVEX_URL` + `VITE_CONVEX_SITE_URL`.
+21
View File
@@ -1,5 +1,22 @@
# Changelog
## Unreleased
### Added
- Admin: add manual unban for banned users (clears `deletedAt` + `banReason`, audit log entry). Revoked API tokens stay revoked.
### Changed
- Quality gate: language-aware word counting (`Intl.Segmenter`) and new `cjkChars` signal to reduce false rejects for non-Latin docs.
- Jobs: run skill stat event processing every 5 minutes (was 15).
- API performance: batch resolve skill/soul tags in v1 list/get endpoints (fewer action->query round-trips) (#112) (thanks @mkrokosz).
### Fixed
- Users: sync handle on ensure when GitHub login changes (#293) (thanks @christianhpoe).
- Upload gate: fetch GitHub account age by immutable account ID (prevents username swaps) (#116) (thanks @mkrokosz).
- API: for owners, return clearer status/messages for hidden/soft-deleted skills instead of a generic 404.
- HTTP/CORS: add preflight handler + include CORS headers on API/download errors; CLI: include auth token for owner-visible installs/updates (#146) (thanks @Grenghis-Khan).
- Skills: keep global sorting across pagination on `/skills` (thanks @CodeBBakGoSu, #98).
## 0.6.1 - 2026-02-13
### Added
@@ -12,8 +29,12 @@
- Moderation UX: collapse OpenClaw analysis by default; update spacing and default reasoning model.
### Fixed
- Skills: fix initial `/skills` sort wiring so first page respects selected sort/direction (thanks @bpk9, #92).
- Search/UI: add embedding request timeout and align `/skills` toolbar + list width (thanks @GhadiSaab, #53).
- Upload gate: handle GitHub API rate limits and optional authenticated lookup token (thanks @superlowburn, #246).
- HTTP: remove `allowH2` from Undici agent to prevent `fetch failed` on Node.js 22+ (#245).
- Tests: add root `undici` dev dependency for Node E2E imports (thanks @tanujbhaud, #255).
- Downloads: add download rate limiting + per-IP/day dedupe + scheduled dedupe pruning; preserve moderation gating and deterministic zips (thanks @regenrek, #43).
- VirusTotal: fix scan sync race conditions and retry behavior in scan/backfill paths.
- Metadata: tolerate trailing commas in JSON metadata.
- Auth: allow soft-deleted users to re-authenticate on fresh login, while keeping banned users blocked (thanks @tanujbhaud, #177).
-41
View File
@@ -1,41 +0,0 @@
{
"$schema": "https://biomejs.dev/schemas/2.3.13/schema.json",
"files": {
"includes": [
"**",
"!**/.cta.json",
"!**/.vscode",
"!**/node_modules",
"!**/dist",
"!**/.output",
"!**/coverage",
"!**/convex/_generated",
"!**/test-results",
"!**/src/routeTree.gen.ts",
"!**/.tanstack",
"!**/public",
"!**/.devenv",
"!**/.devenv"
]
},
"assist": { "actions": { "source": { "organizeImports": "on" } } },
"formatter": {
"enabled": true,
"indentStyle": "space",
"indentWidth": 2,
"lineWidth": 100
},
"linter": {
"enabled": true,
"rules": {
"recommended": true
}
},
"javascript": {
"formatter": {
"quoteStyle": "single",
"semicolons": "asNeeded",
"trailingCommas": "all"
}
}
}
+51 -25
View File
@@ -24,7 +24,6 @@
"clawhub-schema": "workspace:*",
"clsx": "^2.1.1",
"convex": "^1.31.7",
"convex-helpers": "^0.1.111",
"fflate": "^0.8.2",
"h3": "2.0.1-rc.11",
"lucide-react": "^0.563.0",
@@ -41,7 +40,6 @@
"yaml": "^2.8.2",
},
"devDependencies": {
"@biomejs/biome": "^2.3.13",
"@playwright/test": "^1.58.1",
"@tanstack/devtools-vite": "^0.5.0",
"@testing-library/dom": "^10.4.1",
@@ -54,16 +52,18 @@
"@vitest/coverage-v8": "^4.0.18",
"jsdom": "^28.0.0",
"only-allow": "^1.2.2",
"oxfmt": "0.32.0",
"oxlint": "^1.42.0",
"oxlint-tsgolint": "^0.11.4",
"typescript": "^5.9.3",
"undici": "^7.19.2",
"vite": "^7.3.1",
"vitest": "^4.0.18",
},
},
"packages/clawdhub": {
"name": "clawhub",
"version": "0.5.0",
"version": "0.6.1",
"bin": {
"clawhub": "bin/clawdhub.js",
"clawdhub": "bin/clawdhub.js",
@@ -158,24 +158,6 @@
"@bcoe/v8-coverage": ["@bcoe/v8-coverage@1.0.2", "", {}, "sha512-6zABk/ECA/QYSCQ1NGiVwwbQerUCZ+TQbp64Q3AgmfNvurHH0j8TtXa1qbShXA6qqkpAj4V5W8pP6mLe1mcMqA=="],
"@biomejs/biome": ["@biomejs/biome@2.3.13", "", { "optionalDependencies": { "@biomejs/cli-darwin-arm64": "2.3.13", "@biomejs/cli-darwin-x64": "2.3.13", "@biomejs/cli-linux-arm64": "2.3.13", "@biomejs/cli-linux-arm64-musl": "2.3.13", "@biomejs/cli-linux-x64": "2.3.13", "@biomejs/cli-linux-x64-musl": "2.3.13", "@biomejs/cli-win32-arm64": "2.3.13", "@biomejs/cli-win32-x64": "2.3.13" }, "bin": { "biome": "bin/biome" } }, "sha512-Fw7UsV0UAtWIBIm0M7g5CRerpu1eKyKAXIazzxhbXYUyMkwNrkX/KLkGI7b+uVDQ5cLUMfOC9vR60q9IDYDstA=="],
"@biomejs/cli-darwin-arm64": ["@biomejs/cli-darwin-arm64@2.3.13", "", { "os": "darwin", "cpu": "arm64" }, "sha512-0OCwP0/BoKzyJHnFdaTk/i7hIP9JHH9oJJq6hrSCPmJPo8JWcJhprK4gQlhFzrwdTBAW4Bjt/RmCf3ZZe59gwQ=="],
"@biomejs/cli-darwin-x64": ["@biomejs/cli-darwin-x64@2.3.13", "", { "os": "darwin", "cpu": "x64" }, "sha512-AGr8OoemT/ejynbIu56qeil2+F2WLkIjn2d8jGK1JkchxnMUhYOfnqc9sVzcRxpG9Ycvw4weQ5sprRvtb7Yhcw=="],
"@biomejs/cli-linux-arm64": ["@biomejs/cli-linux-arm64@2.3.13", "", { "os": "linux", "cpu": "arm64" }, "sha512-xvOiFkrDNu607MPMBUQ6huHmBG1PZLOrqhtK6pXJW3GjfVqJg0Z/qpTdhXfcqWdSZHcT+Nct2fOgewZvytESkw=="],
"@biomejs/cli-linux-arm64-musl": ["@biomejs/cli-linux-arm64-musl@2.3.13", "", { "os": "linux", "cpu": "arm64" }, "sha512-TUdDCSY+Eo/EHjhJz7P2GnWwfqet+lFxBZzGHldrvULr59AgahamLs/N85SC4+bdF86EhqDuuw9rYLvLFWWlXA=="],
"@biomejs/cli-linux-x64": ["@biomejs/cli-linux-x64@2.3.13", "", { "os": "linux", "cpu": "x64" }, "sha512-s+YsZlgiXNq8XkgHs6xdvKDFOj/bwTEevqEY6rC2I3cBHbxXYU1LOZstH3Ffw9hE5tE1sqT7U23C00MzkXztMw=="],
"@biomejs/cli-linux-x64-musl": ["@biomejs/cli-linux-x64-musl@2.3.13", "", { "os": "linux", "cpu": "x64" }, "sha512-0bdwFVSbbM//Sds6OjtnmQGp4eUjOTt6kHvR/1P0ieR9GcTUAlPNvPC3DiavTqq302W34Ae2T6u5VVNGuQtGlQ=="],
"@biomejs/cli-win32-arm64": ["@biomejs/cli-win32-arm64@2.3.13", "", { "os": "win32", "cpu": "arm64" }, "sha512-QweDxY89fq0VvrxME+wS/BXKmqMrOTZlN9SqQ79kQSIc3FrEwvW/PvUegQF6XIVaekncDykB5dzPqjbwSKs9DA=="],
"@biomejs/cli-win32-x64": ["@biomejs/cli-win32-x64@2.3.13", "", { "os": "win32", "cpu": "x64" }, "sha512-trDw2ogdM2lyav9WFQsdsfdVy1dvZALymRpgmWsvSez0BJzBjulhOT/t+wyKeh3pZWvwP3VMs1SoOKwO3wecMQ=="],
"@clack/core": ["@clack/core@0.5.0", "", { "dependencies": { "picocolors": "^1.0.0", "sisteransi": "^1.0.5" } }, "sha512-p3y0FIOwaYRUPRcMO7+dlmLh8PSRcrjuTndsiA0WAFbWES0mLZlrjVoBRZ9DzkPFJZG6KGkJmoEAY0ZcVWTkow=="],
"@clack/prompts": ["@clack/prompts@0.11.0", "", { "dependencies": { "@clack/core": "0.5.0", "picocolors": "^1.0.0", "sisteransi": "^1.0.5" } }, "sha512-pMN5FcrEw9hUkZA4f+zLlzivQSeQf5dRGJjSUbvVYDLvpKCdQx5OaknvKzgbtXOizhP+SJJJjqEbOe55uKKfAw=="],
@@ -380,6 +362,44 @@
"@oxc-transform/binding-win32-x64-msvc": ["@oxc-transform/binding-win32-x64-msvc@0.110.0", "", { "os": "win32", "cpu": "x64" }, "sha512-QROrowwlrApI1fEScMknGWKM6GTM/Z2xwMnDqvSaEmzNazBsDUlE08Jasw610hFEsYAVU2K5sp/YaCa9ORdP4A=="],
"@oxfmt/binding-android-arm-eabi": ["@oxfmt/binding-android-arm-eabi@0.32.0", "", { "os": "android", "cpu": "arm" }, "sha512-DpVyuVzgLH6/MvuB/YD3vXO9CN/o9EdRpA0zXwe/tagP6yfVSFkFWkPqTROdqp0mlzLH5Yl+/m+hOrcM601EbA=="],
"@oxfmt/binding-android-arm64": ["@oxfmt/binding-android-arm64@0.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-w1cmNXf9zs0vKLuNgyUF3hZ9VUAS1hBmQGndYJv1OmcVqStBtRTRNxSWkWM0TMkrA9UbvIvM9gfN+ib4Wy6lkQ=="],
"@oxfmt/binding-darwin-arm64": ["@oxfmt/binding-darwin-arm64@0.32.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-m6wQojz/hn94XdZugFPtdFbOvXbOSYEqPsR2gyLyID3BvcrC2QsJyT1o3gb4BZEGtZrG1NiKVGwDRLM0dHd2mg=="],
"@oxfmt/binding-darwin-x64": ["@oxfmt/binding-darwin-x64@0.32.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-hN966Uh6r3Erkg2MvRcrJWaB6QpBzP15rxWK/QtkUyD47eItJLsAQ2Hrm88zMIpFZ3COXZLuN3hqgSlUtvB0Xw=="],
"@oxfmt/binding-freebsd-x64": ["@oxfmt/binding-freebsd-x64@0.32.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-g5UZPGt8tJj263OfSiDGdS54HPa0KgFfspLVAUivVSdoOgsk6DkwVS9nO16xQTDztzBPGxTvrby8WuufF0g86Q=="],
"@oxfmt/binding-linux-arm-gnueabihf": ["@oxfmt/binding-linux-arm-gnueabihf@0.32.0", "", { "os": "linux", "cpu": "arm" }, "sha512-F4ZY83/PVQo9ZJhtzoMqbmjqEyTVEZjbaw4x1RhzdfUhddB41ZB2Vrt4eZi7b4a4TP85gjPRHgQBeO0c1jbtaw=="],
"@oxfmt/binding-linux-arm-musleabihf": ["@oxfmt/binding-linux-arm-musleabihf@0.32.0", "", { "os": "linux", "cpu": "arm" }, "sha512-olR37eG16Lzdj9OBSvuoT5RxzgM5xfQEHm1OEjB3M7Wm4KWa5TDWIT13Aiy74GvAN77Hq1+kUKcGVJ/0ynf75g=="],
"@oxfmt/binding-linux-arm64-gnu": ["@oxfmt/binding-linux-arm64-gnu@0.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-eZhk6AIjRCDeLoXYBhMW7qq/R1YyVi+tGnGfc3kp7AZQrMsFaWtP/bgdCJCTNXMpbMwymtVz0qhSQvR5w2sKcg=="],
"@oxfmt/binding-linux-arm64-musl": ["@oxfmt/binding-linux-arm64-musl@0.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UYiqO9MlipntFbdbUKOIo84vuyzrK4TVIs7Etat91WNMFSW54F6OnHq08xa5ZM+K9+cyYMgQPXvYCopuP+LyKw=="],
"@oxfmt/binding-linux-ppc64-gnu": ["@oxfmt/binding-linux-ppc64-gnu@0.32.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-IDH/fxMv+HmKsMtsjEbXqhScCKDIYp38sgGEcn0QKeXMxrda67PPZA7HMfoUwEtFUG+jsO1XJxTrQsL+kQ90xQ=="],
"@oxfmt/binding-linux-riscv64-gnu": ["@oxfmt/binding-linux-riscv64-gnu@0.32.0", "", { "os": "linux", "cpu": "none" }, "sha512-bQFGPDa0buYWJFeK2I7ah8wRZjrAgamaG2OAGv+Ua5UMYEnHxmHcv+r8lWUUrwP2oqQGvp1SB8JIVtBbYuAueQ=="],
"@oxfmt/binding-linux-riscv64-musl": ["@oxfmt/binding-linux-riscv64-musl@0.32.0", "", { "os": "linux", "cpu": "none" }, "sha512-3vFp9DW1ItEKWltADzCFqG5N7rYFToT4ztlhg8wALoo2E2VhveLD88uAF4FF9AxD9NhgHDGmPCV+WZl/Qlj8cQ=="],
"@oxfmt/binding-linux-s390x-gnu": ["@oxfmt/binding-linux-s390x-gnu@0.32.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-Fub2y8S9ImuPzAzpbgkoz/EVTWFFBolxFZYCMRhRZc8cJZI2gl/NlZswqhvJd/U0Jopnwgm/OJ2x128vVzFFWA=="],
"@oxfmt/binding-linux-x64-gnu": ["@oxfmt/binding-linux-x64-gnu@0.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-XufwsnV3BF81zO2ofZvhT4FFaMmLTzZEZnC9HpFz/quPeg9C948+kbLlZnsfjmp+1dUxKMCpfmRMqOfF4AOLsA=="],
"@oxfmt/binding-linux-x64-musl": ["@oxfmt/binding-linux-x64-musl@0.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-u2f9tC2qYfikKmA2uGpnEJgManwmk0ZXWs5BB4ga4KDu2JNLdA3i634DGHeMLK9wY9+iRf3t7IYpgN3OVFrvDw=="],
"@oxfmt/binding-openharmony-arm64": ["@oxfmt/binding-openharmony-arm64@0.32.0", "", { "os": "none", "cpu": "arm64" }, "sha512-5ZXb1wrdbZ1YFXuNXNUCePLlmLDy4sUt4evvzD4Cgumbup5wJgS9PIe5BOaLywUg9f1wTH6lwltj3oT7dFpIGA=="],
"@oxfmt/binding-win32-arm64-msvc": ["@oxfmt/binding-win32-arm64-msvc@0.32.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-IGSMm/Agq+IA0++aeAV/AGPfjcBdjrsajB5YpM3j7cMcwoYgUTi/k2YwAmsHH3ueZUE98pSM/Ise2J7HtyRjOA=="],
"@oxfmt/binding-win32-ia32-msvc": ["@oxfmt/binding-win32-ia32-msvc@0.32.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-H/9gsuqXmceWMsVoCPZhtJG2jLbnBeKr7xAXm2zuKpxLVF7/2n0eh7ocOLB6t+L1ARE76iORuUsRMnuGjj8FjQ=="],
"@oxfmt/binding-win32-x64-msvc": ["@oxfmt/binding-win32-x64-msvc@0.32.0", "", { "os": "win32", "cpu": "x64" }, "sha512-fF8VIOeligq+mA6KfKvWtFRXbf0EFy73TdR6ZnNejdJRM8VWN1e3QFhYgIwD7O8jBrQsd7EJbUpkAr/YlUOokg=="],
"@oxlint-tsgolint/darwin-arm64": ["@oxlint-tsgolint/darwin-arm64@0.11.4", "", { "os": "darwin", "cpu": "arm64" }, "sha512-IhdhiC183s5wdFDZSQC8PaFFq1QROiVT5ahz7ysgEKVnkNDjy82ieM7ZKiUfm2ncXNX2RcFGSSZrQO6plR+VAQ=="],
"@oxlint-tsgolint/darwin-x64": ["@oxlint-tsgolint/darwin-x64@0.11.4", "", { "os": "darwin", "cpu": "x64" }, "sha512-KJmBg10Z1uGpJqxDzETXOytYyeVrKUepo8rCXeVkRlZ2QzZqMElgalFN4BI3ccgIPkQpzzu4SVzWNFz7yiKavQ=="],
@@ -770,8 +790,6 @@
"convex": ["convex@1.31.7", "", { "dependencies": { "esbuild": "0.27.0", "prettier": "^3.0.0" }, "peerDependencies": { "@auth0/auth0-react": "^2.0.1", "@clerk/clerk-react": "^4.12.8 || ^5.0.0", "react": "^18.0.0 || ^19.0.0-0 || ^19.0.0" }, "optionalPeers": ["@auth0/auth0-react", "@clerk/clerk-react", "react"], "bin": { "convex": "bin/main.js" } }, "sha512-PtNMe1mAIOvA8Yz100QTOaIdgt2rIuWqencVXrb4McdhxBHZ8IJ1eXTnrgCC9HydyilGT1pOn+KNqT14mqn9fQ=="],
"convex-helpers": ["convex-helpers@0.1.111", "", { "peerDependencies": { "@standard-schema/spec": "^1.0.0", "convex": "^1.25.4", "hono": "^4.0.5", "react": "^17.0.2 || ^18.0.0 || ^19.0.0", "typescript": "^5.5", "zod": "^3.25.0 || ^4.0.0" }, "optionalPeers": ["@standard-schema/spec", "hono", "react", "typescript", "zod"], "bin": { "convex-helpers": "bin.cjs" } }, "sha512-0O59Ohi8HVc3+KULxSC6JHsw8cQJyc8gZ7OAfNRVX7T5Wy6LhPx3l8veYN9avKg7UiPlO7m1eBiQMHKclIyXyQ=="],
"cookie": ["cookie@1.1.1", "", {}, "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ=="],
"cookie-es": ["cookie-es@2.0.0", "", {}, "sha512-RAj4E421UYRgqokKUmotqAwuplYw15qtdXfY+hGzgCJ/MBjCVZcSoHK/kH9kocfjRjcDME7IiDWR/1WX1TM2Pg=="],
@@ -1124,6 +1142,8 @@
"oxc-transform": ["oxc-transform@0.110.0", "", { "optionalDependencies": { "@oxc-transform/binding-android-arm-eabi": "0.110.0", "@oxc-transform/binding-android-arm64": "0.110.0", "@oxc-transform/binding-darwin-arm64": "0.110.0", "@oxc-transform/binding-darwin-x64": "0.110.0", "@oxc-transform/binding-freebsd-x64": "0.110.0", "@oxc-transform/binding-linux-arm-gnueabihf": "0.110.0", "@oxc-transform/binding-linux-arm-musleabihf": "0.110.0", "@oxc-transform/binding-linux-arm64-gnu": "0.110.0", "@oxc-transform/binding-linux-arm64-musl": "0.110.0", "@oxc-transform/binding-linux-ppc64-gnu": "0.110.0", "@oxc-transform/binding-linux-riscv64-gnu": "0.110.0", "@oxc-transform/binding-linux-riscv64-musl": "0.110.0", "@oxc-transform/binding-linux-s390x-gnu": "0.110.0", "@oxc-transform/binding-linux-x64-gnu": "0.110.0", "@oxc-transform/binding-linux-x64-musl": "0.110.0", "@oxc-transform/binding-openharmony-arm64": "0.110.0", "@oxc-transform/binding-wasm32-wasi": "0.110.0", "@oxc-transform/binding-win32-arm64-msvc": "0.110.0", "@oxc-transform/binding-win32-ia32-msvc": "0.110.0", "@oxc-transform/binding-win32-x64-msvc": "0.110.0" } }, "sha512-/fymQNzzUoKZweH0nC5yvbI2eR0yWYusT9TEKDYVgOgYrf9Qmdez9lUFyvxKR9ycx+PTHi/reIOzqf3wkShQsw=="],
"oxfmt": ["oxfmt@0.32.0", "", { "dependencies": { "tinypool": "2.1.0" }, "optionalDependencies": { "@oxfmt/binding-android-arm-eabi": "0.32.0", "@oxfmt/binding-android-arm64": "0.32.0", "@oxfmt/binding-darwin-arm64": "0.32.0", "@oxfmt/binding-darwin-x64": "0.32.0", "@oxfmt/binding-freebsd-x64": "0.32.0", "@oxfmt/binding-linux-arm-gnueabihf": "0.32.0", "@oxfmt/binding-linux-arm-musleabihf": "0.32.0", "@oxfmt/binding-linux-arm64-gnu": "0.32.0", "@oxfmt/binding-linux-arm64-musl": "0.32.0", "@oxfmt/binding-linux-ppc64-gnu": "0.32.0", "@oxfmt/binding-linux-riscv64-gnu": "0.32.0", "@oxfmt/binding-linux-riscv64-musl": "0.32.0", "@oxfmt/binding-linux-s390x-gnu": "0.32.0", "@oxfmt/binding-linux-x64-gnu": "0.32.0", "@oxfmt/binding-linux-x64-musl": "0.32.0", "@oxfmt/binding-openharmony-arm64": "0.32.0", "@oxfmt/binding-win32-arm64-msvc": "0.32.0", "@oxfmt/binding-win32-ia32-msvc": "0.32.0", "@oxfmt/binding-win32-x64-msvc": "0.32.0" }, "bin": { "oxfmt": "bin/oxfmt" } }, "sha512-KArQhGzt/Y8M1eSAX98Y8DLtGYYDQhkR55THUPY5VNcpFQ+9nRZkL3ULXhagHMD2hIvjy8JSeEQEP5/yYJSrLA=="],
"oxlint": ["oxlint@1.42.0", "", { "optionalDependencies": { "@oxlint/darwin-arm64": "1.42.0", "@oxlint/darwin-x64": "1.42.0", "@oxlint/linux-arm64-gnu": "1.42.0", "@oxlint/linux-arm64-musl": "1.42.0", "@oxlint/linux-x64-gnu": "1.42.0", "@oxlint/linux-x64-musl": "1.42.0", "@oxlint/win32-arm64": "1.42.0", "@oxlint/win32-x64": "1.42.0" }, "peerDependencies": { "oxlint-tsgolint": ">=0.11.2" }, "optionalPeers": ["oxlint-tsgolint"], "bin": { "oxlint": "bin/oxlint" } }, "sha512-qnspC/lrp8FgKNaONLLn14dm+W5t0SSlus6V5NJpgI2YNT1tkFYZt4fBf14ESxf9AAh98WBASnW5f0gtw462Lg=="],
"oxlint-tsgolint": ["oxlint-tsgolint@0.11.4", "", { "optionalDependencies": { "@oxlint-tsgolint/darwin-arm64": "0.11.4", "@oxlint-tsgolint/darwin-x64": "0.11.4", "@oxlint-tsgolint/linux-arm64": "0.11.4", "@oxlint-tsgolint/linux-x64": "0.11.4", "@oxlint-tsgolint/win32-arm64": "0.11.4", "@oxlint-tsgolint/win32-x64": "0.11.4" }, "bin": { "tsgolint": "bin/tsgolint.js" } }, "sha512-VyQc+69TxQwUdsEPiVFN7vNZdDVO/FHaEcHltnWs3O6rvwxv67uADlknQQO714sbRdEahOjgO5dFf+K9ili0gg=="],
@@ -1272,6 +1292,8 @@
"tinyglobby": ["tinyglobby@0.2.15", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.3" } }, "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ=="],
"tinypool": ["tinypool@2.1.0", "", {}, "sha512-Pugqs6M0m7Lv1I7FtxN4aoyToKg1C4tu+/381vH35y8oENM/Ai7f7C4StcoK4/+BSw9ebcS8jRiVrORFKCALLw=="],
"tinyrainbow": ["tinyrainbow@3.0.3", "", {}, "sha512-PSkbLUoxOFRzJYjjxHJt9xro7D+iilgMX/C9lawzVuYiIdcihh9DXmVibBe8lmcFrRi/VzlPjBxbN7rH24q8/Q=="],
"tldts": ["tldts@7.0.19", "", { "dependencies": { "tldts-core": "^7.0.19" }, "bin": { "tldts": "bin/cli.js" } }, "sha512-8PWx8tvC4jDB39BQw1m4x8y5MH1BcQ5xHeL2n7UVFulMPH/3Q0uiamahFJ3lXA0zO2SUyRXuVVbWSDmstlt9YA=="],
@@ -1298,7 +1320,7 @@
"ufo": ["ufo@1.6.3", "", {}, "sha512-yDJTmhydvl5lJzBmy/hyOAA0d+aqCBuwl818haVdYCRrWV84o7YyeVm4QlVHStqNrrJSTb6jKuFAVqAFsr+K3Q=="],
"undici": ["undici@7.19.2", "", {}, "sha512-4VQSpGEGsWzk0VYxyB/wVX/Q7qf9t5znLRgs0dzszr9w9Fej/8RVNQ+S20vdXSAyra/bJ7ZQfGv6ZMj7UEbzSg=="],
"undici": ["undici@7.20.0", "", {}, "sha512-MJZrkjyd7DeC+uPZh+5/YaMDxFiiEEaDgbUSVMXayofAkDWF1088CDo+2RPg7B1BuS1qf1vgNE7xqwPxE0DuSQ=="],
"undici-types": ["undici-types@7.16.0", "", {}, "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw=="],
@@ -1402,8 +1424,12 @@
"cheerio/parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="],
"cheerio/undici": ["undici@7.19.2", "", {}, "sha512-4VQSpGEGsWzk0VYxyB/wVX/Q7qf9t5znLRgs0dzszr9w9Fej/8RVNQ+S20vdXSAyra/bJ7ZQfGv6ZMj7UEbzSg=="],
"cheerio/whatwg-mimetype": ["whatwg-mimetype@4.0.0", "", {}, "sha512-QaKxh0eNIi2mE9p2vEdzfagOKHCcj1pJ56EEHGQOVxp8r9/iszLUUV7v89x9O1p/T+NlTM5W7jW6+cz4Fq1YVg=="],
"clawhub/undici": ["undici@7.19.2", "", {}, "sha512-4VQSpGEGsWzk0VYxyB/wVX/Q7qf9t5znLRgs0dzszr9w9Fej/8RVNQ+S20vdXSAyra/bJ7ZQfGv6ZMj7UEbzSg=="],
"convex/esbuild": ["esbuild@0.27.0", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.27.0", "@esbuild/android-arm": "0.27.0", "@esbuild/android-arm64": "0.27.0", "@esbuild/android-x64": "0.27.0", "@esbuild/darwin-arm64": "0.27.0", "@esbuild/darwin-x64": "0.27.0", "@esbuild/freebsd-arm64": "0.27.0", "@esbuild/freebsd-x64": "0.27.0", "@esbuild/linux-arm": "0.27.0", "@esbuild/linux-arm64": "0.27.0", "@esbuild/linux-ia32": "0.27.0", "@esbuild/linux-loong64": "0.27.0", "@esbuild/linux-mips64el": "0.27.0", "@esbuild/linux-ppc64": "0.27.0", "@esbuild/linux-riscv64": "0.27.0", "@esbuild/linux-s390x": "0.27.0", "@esbuild/linux-x64": "0.27.0", "@esbuild/netbsd-arm64": "0.27.0", "@esbuild/netbsd-x64": "0.27.0", "@esbuild/openbsd-arm64": "0.27.0", "@esbuild/openbsd-x64": "0.27.0", "@esbuild/openharmony-arm64": "0.27.0", "@esbuild/sunos-x64": "0.27.0", "@esbuild/win32-arm64": "0.27.0", "@esbuild/win32-ia32": "0.27.0", "@esbuild/win32-x64": "0.27.0" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-jd0f4NHbD6cALCyGElNpGAOtWxSq46l9X/sWB0Nzd5er4Kz2YTm+Vl0qKFT9KUJvD8+fiO8AvoHhFvEatfVixA=="],
"dom-serializer/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="],
@@ -1412,7 +1438,7 @@
"htmlparser2/entities": ["entities@7.0.1", "", {}, "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA=="],
"jsdom/undici": ["undici@7.20.0", "", {}, "sha512-MJZrkjyd7DeC+uPZh+5/YaMDxFiiEEaDgbUSVMXayofAkDWF1088CDo+2RPg7B1BuS1qf1vgNE7xqwPxE0DuSQ=="],
"nitro/undici": ["undici@7.19.2", "", {}, "sha512-4VQSpGEGsWzk0VYxyB/wVX/Q7qf9t5znLRgs0dzszr9w9Fej/8RVNQ+S20vdXSAyra/bJ7ZQfGv6ZMj7UEbzSg=="],
"parse-entities/@types/unist": ["@types/unist@2.0.11", "", {}, "sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA=="],
+14
View File
@@ -22,6 +22,7 @@ import type * as githubSoulBackupsNode from "../githubSoulBackupsNode.js";
import type * as http from "../http.js";
import type * as httpApi from "../httpApi.js";
import type * as httpApiV1 from "../httpApiV1.js";
import type * as httpPreflight from "../httpPreflight.js";
import type * as leaderboards from "../leaderboards.js";
import type * as lib_access from "../lib/access.js";
import type * as lib_apiTokenAuth from "../lib/apiTokenAuth.js";
@@ -30,8 +31,11 @@ import type * as lib_changelog from "../lib/changelog.js";
import type * as lib_embeddings from "../lib/embeddings.js";
import type * as lib_githubAccount from "../lib/githubAccount.js";
import type * as lib_githubBackup from "../lib/githubBackup.js";
import type * as lib_githubIdentity from "../lib/githubIdentity.js";
import type * as lib_githubImport from "../lib/githubImport.js";
import type * as lib_githubSoulBackup from "../lib/githubSoulBackup.js";
import type * as lib_httpHeaders from "../lib/httpHeaders.js";
import type * as lib_httpRateLimit from "../lib/httpRateLimit.js";
import type * as lib_leaderboards from "../lib/leaderboards.js";
import type * as lib_moderation from "../lib/moderation.js";
import type * as lib_public from "../lib/public.js";
@@ -39,7 +43,10 @@ import type * as lib_searchText from "../lib/searchText.js";
import type * as lib_securityPrompt from "../lib/securityPrompt.js";
import type * as lib_skillBackfill from "../lib/skillBackfill.js";
import type * as lib_skillPublish from "../lib/skillPublish.js";
import type * as lib_skillQuality from "../lib/skillQuality.js";
import type * as lib_skillSafety from "../lib/skillSafety.js";
import type * as lib_skillStats from "../lib/skillStats.js";
import type * as lib_skillSummary from "../lib/skillSummary.js";
import type * as lib_skillZip from "../lib/skillZip.js";
import type * as lib_skills from "../lib/skills.js";
import type * as lib_soulChangelog from "../lib/soulChangelog.js";
@@ -89,6 +96,7 @@ declare const fullApi: ApiFromModules<{
http: typeof http;
httpApi: typeof httpApi;
httpApiV1: typeof httpApiV1;
httpPreflight: typeof httpPreflight;
leaderboards: typeof leaderboards;
"lib/access": typeof lib_access;
"lib/apiTokenAuth": typeof lib_apiTokenAuth;
@@ -97,8 +105,11 @@ declare const fullApi: ApiFromModules<{
"lib/embeddings": typeof lib_embeddings;
"lib/githubAccount": typeof lib_githubAccount;
"lib/githubBackup": typeof lib_githubBackup;
"lib/githubIdentity": typeof lib_githubIdentity;
"lib/githubImport": typeof lib_githubImport;
"lib/githubSoulBackup": typeof lib_githubSoulBackup;
"lib/httpHeaders": typeof lib_httpHeaders;
"lib/httpRateLimit": typeof lib_httpRateLimit;
"lib/leaderboards": typeof lib_leaderboards;
"lib/moderation": typeof lib_moderation;
"lib/public": typeof lib_public;
@@ -106,7 +117,10 @@ declare const fullApi: ApiFromModules<{
"lib/securityPrompt": typeof lib_securityPrompt;
"lib/skillBackfill": typeof lib_skillBackfill;
"lib/skillPublish": typeof lib_skillPublish;
"lib/skillQuality": typeof lib_skillQuality;
"lib/skillSafety": typeof lib_skillSafety;
"lib/skillStats": typeof lib_skillStats;
"lib/skillSummary": typeof lib_skillSummary;
"lib/skillZip": typeof lib_skillZip;
"lib/skills": typeof lib_skills;
"lib/soulChangelog": typeof lib_soulChangelog;
+48 -24
View File
@@ -1,19 +1,21 @@
import { describe, expect, it, vi } from 'vitest'
import type { Id } from './_generated/dataModel'
import { BANNED_REAUTH_MESSAGE, handleSoftDeletedUserReauth } from './auth'
import {
BANNED_REAUTH_MESSAGE,
DELETED_ACCOUNT_REAUTH_MESSAGE,
handleDeletedUserSignIn,
} from './auth'
function makeCtx({
user,
banRecord,
banRecords,
}: {
user: { deletedAt?: number } | null
banRecord?: Record<string, unknown> | null
user: { deletedAt?: number; deactivatedAt?: number; purgedAt?: number } | null
banRecords?: Array<Record<string, unknown>>
}) {
const query = {
withIndex: vi.fn().mockReturnValue({
filter: vi.fn().mockReturnValue({
first: vi.fn().mockResolvedValue(banRecord ?? null),
}),
collect: vi.fn().mockResolvedValue(banRecords ?? []),
}),
}
const ctx = {
@@ -26,74 +28,96 @@ function makeCtx({
return { ctx, query }
}
describe('handleSoftDeletedUserReauth', () => {
describe('handleDeletedUserSignIn', () => {
const userId = 'users:1' as Id<'users'>
it('skips when user not found', async () => {
const { ctx } = makeCtx({ user: null })
await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId })
await handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId })
expect(ctx.db.get).toHaveBeenCalledWith(userId)
expect(ctx.db.query).not.toHaveBeenCalled()
})
it('skips active users', async () => {
const { ctx } = makeCtx({ user: { deletedAt: undefined } })
const { ctx } = makeCtx({ user: { deletedAt: undefined, deactivatedAt: undefined } })
await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId })
await handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId })
expect(ctx.db.query).not.toHaveBeenCalled()
expect(ctx.db.patch).not.toHaveBeenCalled()
})
it('restores soft-deleted users when not banned', async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: null })
it('blocks sign-in for deactivated users', async () => {
const { ctx } = makeCtx({ user: { deactivatedAt: 123, purgedAt: 123 } })
await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId })
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(DELETED_ACCOUNT_REAUTH_MESSAGE)
expect(ctx.db.query).not.toHaveBeenCalled()
expect(ctx.db.patch).not.toHaveBeenCalled()
})
it('migrates legacy self-deleted users and blocks sign-in', async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecords: [] })
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(DELETED_ACCOUNT_REAUTH_MESSAGE)
expect(ctx.db.patch).toHaveBeenCalledWith(userId, {
deletedAt: undefined,
deactivatedAt: 123,
purgedAt: 123,
updatedAt: expect.any(Number),
})
})
it('restores soft-deleted users on fresh login (existingUserId is null)', async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: null })
it('migrates legacy users on fresh login (existingUserId is null)', async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecords: [] })
await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: null })
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: null }),
).rejects.toThrow(DELETED_ACCOUNT_REAUTH_MESSAGE)
expect(ctx.db.patch).toHaveBeenCalledWith(userId, {
deletedAt: undefined,
deactivatedAt: 123,
purgedAt: 123,
updatedAt: expect.any(Number),
})
})
it('skips reactivation when existingUserId does not match userId', async () => {
it('skips mutation when existingUserId does not match userId', async () => {
const otherUserId = 'users:999' as Id<'users'>
const { ctx } = makeCtx({ user: { deletedAt: 123 } })
await handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: otherUserId })
await handleDeletedUserSignIn(ctx as never, { userId, existingUserId: otherUserId })
expect(ctx.db.query).not.toHaveBeenCalled()
expect(ctx.db.patch).not.toHaveBeenCalled()
})
it('blocks banned users with a custom message', async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: { action: 'user.ban' } })
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecords: [{ action: 'user.ban' }] })
await expect(
handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: userId }),
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(BANNED_REAUTH_MESSAGE)
expect(ctx.db.patch).not.toHaveBeenCalled()
})
it('blocks banned users on fresh login (existingUserId is null)', async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecord: { action: 'user.ban' } })
it('blocks users auto-banned for malware', async () => {
const { ctx } = makeCtx({
user: { deletedAt: 123 },
banRecords: [{ action: 'user.autoban.malware' }],
})
await expect(
handleSoftDeletedUserReauth(ctx as never, { userId, existingUserId: null }),
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(BANNED_REAUTH_MESSAGE)
expect(ctx.db.patch).not.toHaveBeenCalled()
+27 -12
View File
@@ -6,34 +6,50 @@ import type { DataModel, Id } from './_generated/dataModel'
export const BANNED_REAUTH_MESSAGE =
'Your account has been banned for uploading malicious skills. If you believe this is a mistake, please contact security@openclaw.ai and we will work with you to restore access.'
export const DELETED_ACCOUNT_REAUTH_MESSAGE =
'This account has been permanently deleted and cannot be restored.'
export async function handleSoftDeletedUserReauth(
const REAUTH_BLOCKING_BAN_ACTIONS = new Set(['user.ban', 'user.autoban.malware'])
export async function handleDeletedUserSignIn(
ctx: GenericMutationCtx<DataModel>,
args: { userId: Id<'users'>; existingUserId: Id<'users'> | null },
) {
const user = await ctx.db.get(args.userId)
if (!user?.deletedAt) return
if (!user?.deletedAt && !user?.deactivatedAt) return
// Verify that the incoming identity matches the soft-deleted user to prevent bypass.
// Verify that the incoming identity matches the existing account to prevent bypass.
if (args.existingUserId && args.existingUserId !== args.userId) {
return
}
if (user.deactivatedAt) {
throw new ConvexError(DELETED_ACCOUNT_REAUTH_MESSAGE)
}
const userId = args.userId
const banRecord = await ctx.db
const deletedAt = user.deletedAt ?? Date.now()
const banRecords = await ctx.db
.query('auditLogs')
.withIndex('by_target', (q) => q.eq('targetType', 'user').eq('targetId', userId.toString()))
.filter((q) => q.eq(q.field('action'), 'user.ban'))
.first()
.collect()
if (banRecord) {
const hasBlockingBan = banRecords.some((record) => REAUTH_BLOCKING_BAN_ACTIONS.has(record.action))
if (hasBlockingBan) {
throw new ConvexError(BANNED_REAUTH_MESSAGE)
}
// Migrate legacy self-deleted accounts (stored in deletedAt) to the new
// irreversible state and reject sign-in.
await ctx.db.patch(userId, {
deletedAt: undefined,
deactivatedAt: deletedAt,
purgedAt: user.purgedAt ?? deletedAt,
updatedAt: Date.now(),
})
throw new ConvexError(DELETED_ACCOUNT_REAUTH_MESSAGE)
}
export const { auth, signIn, signOut, store, isAuthenticated } = convexAuth({
@@ -53,15 +69,14 @@ export const { auth, signIn, signOut, store, isAuthenticated } = convexAuth({
],
callbacks: {
/**
* Handle re-authentication of soft-deleted users.
* Block sign-in for deleted/deactivated users.
*
* Performance note: This callback runs on every OAuth sign-in, but the
* audit log query ONLY executes when a soft-deleted user attempts to
* sign in (user.deletedAt is set). For normal active users, this is
* just a single `if` check on an already-loaded field - no extra queries.
* audit log query ONLY executes when a legacy deleted user attempts to sign
* in (user.deletedAt is set). For active users, this is a single field check.
*/
async afterUserCreatedOrUpdated(ctx, args) {
await handleSoftDeletedUserReauth(ctx, args)
await handleDeletedUserSignIn(ctx, args)
},
},
})
+52
View File
@@ -0,0 +1,52 @@
import type { Id } from './_generated/dataModel'
import type { MutationCtx } from './_generated/server'
import { assertModerator, requireUser } from './lib/access'
import { insertStatEvent } from './skillStatEvents'
export async function addHandler(ctx: MutationCtx, args: { skillId: Id<'skills'>; body: string }) {
const { userId } = await requireUser(ctx)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
const skill = await ctx.db.get(args.skillId)
if (!skill) throw new Error('Skill not found')
await ctx.db.insert('comments', {
skillId: args.skillId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
})
await insertStatEvent(ctx, { skillId: skill._id, kind: 'comment' })
}
export async function removeHandler(ctx: MutationCtx, args: { commentId: Id<'comments'> }) {
const { user } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment) throw new Error('Comment not found')
if (comment.softDeletedAt) return
const isOwner = comment.userId === user._id
if (!isOwner) {
assertModerator(user)
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
})
await insertStatEvent(ctx, { skillId: comment.skillId, kind: 'uncomment' })
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
action: 'comment.delete',
targetType: 'comment',
targetId: comment._id,
metadata: { skillId: comment.skillId },
createdAt: Date.now(),
})
}
+127
View File
@@ -0,0 +1,127 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from 'vitest'
vi.mock('./lib/access', () => ({
assertModerator: vi.fn(),
requireUser: vi.fn(),
}))
vi.mock('./skillStatEvents', () => ({
insertStatEvent: vi.fn(),
}))
const { requireUser, assertModerator } = await import('./lib/access')
const { insertStatEvent } = await import('./skillStatEvents')
const { addHandler, removeHandler } = await import('./comments.handlers')
describe('comments mutations', () => {
afterEach(() => {
vi.mocked(assertModerator).mockReset()
vi.mocked(requireUser).mockReset()
vi.mocked(insertStatEvent).mockReset()
})
it('add avoids direct skill patch and records stat event', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const get = vi.fn().mockResolvedValue({
_id: 'skills:1',
})
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await addHandler(ctx, { skillId: 'skills:1', body: ' hello ' } as never)
expect(patch).not.toHaveBeenCalled()
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: 'skills:1',
kind: 'comment',
})
})
it('remove keeps comment soft-delete patch free of updatedAt', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:2',
user: { _id: 'users:2', role: 'moderator' },
} as never)
const comment = {
_id: 'comments:1',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:1') return comment
return null
})
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await removeHandler(ctx, { commentId: 'comments:1' } as never)
expect(patch).toHaveBeenCalledTimes(1)
const deletePatch = vi.mocked(patch).mock.calls[0]?.[1] as Record<string, unknown>
expect(deletePatch.updatedAt).toBeUndefined()
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: 'skills:1',
kind: 'uncomment',
})
})
it('remove rejects non-owner without moderator permission', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:3',
user: { _id: 'users:3', role: 'user' },
} as never)
vi.mocked(assertModerator).mockImplementation(() => {
throw new Error('Moderator role required')
})
const comment = {
_id: 'comments:2',
skillId: 'skills:2',
userId: 'users:9',
softDeletedAt: undefined,
}
const get = vi.fn().mockResolvedValue(comment)
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await expect(removeHandler(ctx, { commentId: 'comments:2' } as never)).rejects.toThrow(
'Moderator role required',
)
expect(patch).not.toHaveBeenCalled()
expect(insertStatEvent).not.toHaveBeenCalled()
})
it('remove no-ops for soft-deleted comment', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:4',
user: { _id: 'users:4', role: 'moderator' },
} as never)
const comment = {
_id: 'comments:3',
skillId: 'skills:3',
userId: 'users:4',
softDeletedAt: 123,
}
const get = vi.fn().mockResolvedValue(comment)
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await removeHandler(ctx, { commentId: 'comments:3' } as never)
expect(patch).not.toHaveBeenCalled()
expect(insert).not.toHaveBeenCalled()
expect(insertStatEvent).not.toHaveBeenCalled()
})
})
+12 -55
View File
@@ -1,9 +1,8 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { assertModerator, requireUser } from './lib/access'
import { addHandler, removeHandler } from './comments.handlers'
import { type PublicUser, toPublicUser } from './lib/public'
import { insertStatEvent } from './skillStatEvents'
export const listBySkill = query({
args: { skillId: v.id('skills'), limit: v.optional(v.number()) },
@@ -15,66 +14,24 @@ export const listBySkill = query({
.order('desc')
.take(limit)
const results: Array<{ comment: Doc<'comments'>; user: PublicUser | null }> = []
for (const comment of comments) {
if (comment.softDeletedAt) continue
const user = toPublicUser(await ctx.db.get(comment.userId))
results.push({ comment, user })
}
return results
const visible = comments.filter((comment) => !comment.softDeletedAt)
return Promise.all(
visible.map(
async (comment): Promise<{ comment: Doc<'comments'>; user: PublicUser | null }> => ({
comment,
user: toPublicUser(await ctx.db.get(comment.userId)),
}),
),
)
},
})
export const add = mutation({
args: { skillId: v.id('skills'), body: v.string() },
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
const skill = await ctx.db.get(args.skillId)
if (!skill) throw new Error('Skill not found')
await ctx.db.insert('comments', {
skillId: args.skillId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
})
await insertStatEvent(ctx, { skillId: skill._id, kind: 'comment' })
},
handler: addHandler,
})
export const remove = mutation({
args: { commentId: v.id('comments') },
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment) throw new Error('Comment not found')
if (comment.softDeletedAt) return
const isOwner = comment.userId === user._id
if (!isOwner) {
assertModerator(user)
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
})
await insertStatEvent(ctx, { skillId: comment.skillId, kind: 'uncomment' })
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
action: 'comment.delete',
targetType: 'comment',
targetId: comment._id,
metadata: { skillId: comment.skillId },
createdAt: Date.now(),
})
},
handler: removeHandler,
})
+8 -1
View File
@@ -26,7 +26,7 @@ crons.interval(
crons.interval(
'skill-stat-events',
{ minutes: 15 },
{ minutes: 5 },
internal.skillStatEvents.processSkillStatEventsAction,
{},
)
@@ -40,4 +40,11 @@ crons.interval('vt-cache-backfill', { minutes: 30 }, internal.vt.backfillActiveS
// Daily re-scan of all active skills at 3am UTC
crons.daily('vt-daily-rescan', { hourUTC: 3, minuteUTC: 0 }, internal.vt.rescanActiveSkills, {})
crons.interval(
'download-dedupe-prune',
{ hours: 24 },
internal.downloads.pruneDownloadDedupesInternal,
{},
)
export default crons
+43
View File
@@ -0,0 +1,43 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { __test } from './downloads'
describe('downloads helpers', () => {
afterEach(() => {
vi.unstubAllEnvs()
})
it('calculates hour start boundaries', () => {
const hour = 3_600_000
expect(__test.getHourStart(0)).toBe(0)
expect(__test.getHourStart(hour - 1)).toBe(0)
expect(__test.getHourStart(hour)).toBe(hour)
expect(__test.getHourStart(hour + 1)).toBe(hour)
})
it('prefers user identity when token user exists', () => {
const request = new Request('https://example.com', {
headers: { 'cf-connecting-ip': '1.2.3.4' },
})
expect(__test.getDownloadIdentityValue(request, 'users_123')).toBe('user:users_123')
})
it('uses cf-connecting-ip for anonymous identity', () => {
const request = new Request('https://example.com', {
headers: { 'cf-connecting-ip': '1.2.3.4' },
})
expect(__test.getDownloadIdentityValue(request, null)).toBe('ip:1.2.3.4')
})
it('falls back to forwarded ip when explicitly enabled', () => {
vi.stubEnv('TRUST_FORWARDED_IPS', 'true')
const request = new Request('https://example.com', {
headers: { 'x-forwarded-for': '10.0.0.1, 10.0.0.2' },
})
expect(__test.getDownloadIdentityValue(request, null)).toBe('ip:10.0.0.1')
})
it('returns null when user and ip are missing', () => {
const request = new Request('https://example.com')
expect(__test.getDownloadIdentityValue(request, null)).toBeNull()
})
})
+139 -17
View File
@@ -1,9 +1,18 @@
import { v } from 'convex/values'
import { api } from './_generated/api'
import { httpAction, mutation } from './_generated/server'
import { api, internal } from './_generated/api'
import { httpAction, internalMutation, mutation } from './_generated/server'
import { getOptionalApiTokenUserId } from './lib/apiTokenAuth'
import { applyRateLimit, getClientIp } from './lib/httpRateLimit'
import { corsHeaders, mergeHeaders } from './lib/httpHeaders'
import { buildDeterministicZip } from './lib/skillZip'
import { hashToken } from './lib/tokens'
import { insertStatEvent } from './skillStatEvents'
const HOUR_MS = 3_600_000
const DEDUPE_RETENTION_MS = 7 * 24 * HOUR_MS
const PRUNE_BATCH_SIZE = 200
const PRUNE_MAX_BATCHES = 50
export const downloadZip = httpAction(async (ctx, request) => {
const url = new URL(request.url)
const slug = url.searchParams.get('slug')?.trim().toLowerCase()
@@ -11,33 +20,54 @@ export const downloadZip = httpAction(async (ctx, request) => {
const tagParam = url.searchParams.get('tag')?.trim()
if (!slug) {
return new Response('Missing slug', { status: 400 })
return new Response('Missing slug', {
status: 400,
headers: corsHeaders(),
})
}
const rate = await applyRateLimit(ctx, request, 'download')
if (!rate.ok) return rate.response
const skillResult = await ctx.runQuery(api.skills.getBySlug, { slug })
if (!skillResult?.skill) {
return new Response('Skill not found', { status: 404 })
return new Response('Skill not found', {
status: 404,
headers: mergeHeaders(rate.headers, corsHeaders()),
})
}
// Block downloads based on moderation status
// Block downloads based on moderation status.
const mod = skillResult.moderationInfo
if (mod?.isMalwareBlocked) {
return new Response(
'Blocked: this skill has been flagged as malicious by VirusTotal and cannot be downloaded.',
{ status: 403 },
{
status: 403,
headers: mergeHeaders(rate.headers, corsHeaders()),
},
)
}
if (mod?.isPendingScan) {
return new Response(
'This skill is pending a security scan by VirusTotal. Please try again in a few minutes.',
{ status: 423 },
{
status: 423,
headers: mergeHeaders(rate.headers, corsHeaders()),
},
)
}
if (mod?.isRemoved) {
return new Response('This skill has been removed by a moderator.', { status: 410 })
return new Response('This skill has been removed by a moderator.', {
status: 410,
headers: mergeHeaders(rate.headers, corsHeaders()),
})
}
if (mod?.isHiddenByMod) {
return new Response('This skill is currently unavailable.', { status: 403 })
return new Response('This skill is currently unavailable.', {
status: 403,
headers: mergeHeaders(rate.headers, corsHeaders()),
})
}
const skill = skillResult.skill
@@ -56,10 +86,16 @@ export const downloadZip = httpAction(async (ctx, request) => {
}
if (!version) {
return new Response('Version not found', { status: 404 })
return new Response('Version not found', {
status: 404,
headers: mergeHeaders(rate.headers, corsHeaders()),
})
}
if (version.softDeletedAt) {
return new Response('Version not available', { status: 410 })
return new Response('Version not available', {
status: 410,
headers: mergeHeaders(rate.headers, corsHeaders()),
})
}
const entries: Array<{ path: string; bytes: Uint8Array }> = []
@@ -77,15 +113,31 @@ export const downloadZip = httpAction(async (ctx, request) => {
})
const zipBlob = new Blob([zipArray], { type: 'application/zip' })
await ctx.runMutation(api.downloads.increment, { skillId: skill._id })
try {
const userId = await getOptionalApiTokenUserId(ctx, request)
const identity = getDownloadIdentityValue(request, userId ? String(userId) : null)
if (identity) {
await ctx.runMutation(internal.downloads.recordDownloadInternal, {
skillId: skill._id,
identityHash: await hashToken(identity),
hourStart: getHourStart(Date.now()),
})
}
} catch {
// Best-effort metric path; do not fail downloads.
}
return new Response(zipBlob, {
status: 200,
headers: {
'Content-Type': 'application/zip',
'Content-Disposition': `attachment; filename="${slug}-${version.version}.zip"`,
'Cache-Control': 'private, max-age=60',
},
headers: mergeHeaders(
rate.headers,
{
'Content-Type': 'application/zip',
'Content-Disposition': `attachment; filename="${slug}-${version.version}.zip"`,
'Cache-Control': 'private, max-age=60',
},
corsHeaders(),
),
})
})
@@ -101,3 +153,73 @@ export const increment = mutation({
})
},
})
export const recordDownloadInternal = internalMutation({
args: {
skillId: v.id('skills'),
identityHash: v.string(),
hourStart: v.number(),
},
handler: async (ctx, args) => {
const existing = await ctx.db
.query('downloadDedupes')
.withIndex('by_skill_identity_hour', (q) =>
q
.eq('skillId', args.skillId)
.eq('identityHash', args.identityHash)
.eq('hourStart', args.hourStart),
)
.unique()
if (existing) return
await ctx.db.insert('downloadDedupes', {
skillId: args.skillId,
identityHash: args.identityHash,
hourStart: args.hourStart,
createdAt: Date.now(),
})
await insertStatEvent(ctx, {
skillId: args.skillId,
kind: 'download',
})
},
})
export const pruneDownloadDedupesInternal = internalMutation({
args: {},
handler: async (ctx) => {
const cutoff = Date.now() - DEDUPE_RETENTION_MS
for (let batches = 0; batches < PRUNE_MAX_BATCHES; batches += 1) {
const stale = await ctx.db
.query('downloadDedupes')
.withIndex('by_hour', (q) => q.lt('hourStart', cutoff))
.take(PRUNE_BATCH_SIZE)
if (stale.length === 0) break
for (const entry of stale) {
await ctx.db.delete(entry._id)
}
if (stale.length < PRUNE_BATCH_SIZE) break
}
},
})
export function getHourStart(timestamp: number) {
return Math.floor(timestamp / HOUR_MS) * HOUR_MS
}
export function getDownloadIdentityValue(request: Request, userId: string | null) {
if (userId) return `user:${userId}`
const ip = getClientIp(request)
if (!ip) return null
return `ip:${ip}`
}
export const __test = {
getHourStart,
getDownloadIdentityValue,
}
+1 -1
View File
@@ -78,7 +78,7 @@ export const getGitHubBackupPageInternal = internalQuery({
}
const owner = await ctx.db.get(skill.ownerUserId)
if (!owner || owner.deletedAt) {
if (!owner || owner.deletedAt || owner.deactivatedAt) {
items.push({ kind: 'missingOwner', skillId: skill._id, ownerUserId: skill.ownerUserId })
continue
}
+1 -1
View File
@@ -78,7 +78,7 @@ export const getGitHubSoulBackupPageInternal = internalQuery({
}
const owner = await ctx.db.get(soul.ownerUserId)
if (!owner || owner.deletedAt) {
if (!owner || owner.deletedAt || owner.deactivatedAt) {
items.push({ kind: 'missingOwner', soulId: soul._id, ownerUserId: soul.ownerUserId })
continue
}
+7
View File
@@ -32,6 +32,7 @@ import {
usersPostRouterV1Http,
whoamiV1Http,
} from './httpApiV1'
import { preflightHandler } from './httpPreflight'
const http = httpRouter()
@@ -145,6 +146,12 @@ http.route({
handler: soulsDeleteRouterV1Http,
})
http.route({
pathPrefix: '/api/',
method: 'OPTIONS',
handler: preflightHandler,
})
// TODO: remove legacy /api routes after deprecation window.
http.route({
path: LegacyApiRoutes.download,
+15 -8
View File
@@ -11,6 +11,7 @@ import type { Id } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
import { httpAction } from './_generated/server'
import { requireApiTokenUser } from './lib/apiTokenAuth'
import { corsHeaders, mergeHeaders } from './lib/httpHeaders'
import { publishVersionForUser } from './skills'
type SearchSkillEntry = {
@@ -241,20 +242,26 @@ export const cliTelemetrySyncHttp = httpAction(cliTelemetrySyncHandler)
function json(value: unknown, status = 200) {
return new Response(JSON.stringify(value), {
status,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
},
headers: mergeHeaders(
{
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
},
corsHeaders(),
),
})
}
function text(value: string, status: number) {
return new Response(value, {
status,
headers: {
'Content-Type': 'text/plain; charset=utf-8',
'Cache-Control': 'no-store',
},
headers: mergeHeaders(
{
'Content-Type': 'text/plain; charset=utf-8',
'Cache-Control': 'no-store',
},
corsHeaders(),
),
})
}
+319 -4
View File
@@ -3,13 +3,14 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'
vi.mock('./lib/apiTokenAuth', () => ({
requireApiTokenUser: vi.fn(),
getOptionalApiTokenUserId: vi.fn(),
}))
vi.mock('./skills', () => ({
publishVersionForUser: vi.fn(),
}))
const { requireApiTokenUser } = await import('./lib/apiTokenAuth')
const { getOptionalApiTokenUserId, requireApiTokenUser } = await import('./lib/apiTokenAuth')
const { publishVersionForUser } = await import('./skills')
const { __handlers } = await import('./httpApiV1')
@@ -59,6 +60,8 @@ const blockedRate = () => ({
})
beforeEach(() => {
vi.mocked(getOptionalApiTokenUserId).mockReset()
vi.mocked(getOptionalApiTokenUserId).mockResolvedValue(null)
vi.mocked(requireApiTokenUser).mockReset()
vi.mocked(publishVersionForUser).mockReset()
})
@@ -148,7 +151,7 @@ describe('httpApiV1 handlers', () => {
expect(json.match.version).toBe('1.0.0')
})
it('lists skills with resolved tags', async () => {
it('lists skills with resolved tags using batch query', async () => {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('cursor' in args || 'limit' in args) {
return {
@@ -170,7 +173,10 @@ describe('httpApiV1 handlers', () => {
nextCursor: null,
}
}
if ('versionId' in args) return { version: '1.0.0' }
// Batch query: versionIds (plural)
if ('versionIds' in args) {
return [{ _id: 'versions:1', version: '1.0.0', softDeletedAt: undefined }]
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
@@ -183,6 +189,209 @@ describe('httpApiV1 handlers', () => {
expect(json.items[0].tags.latest).toBe('1.0.0')
})
it('batches tag resolution across multiple skills into single query', async () => {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('cursor' in args || 'limit' in args) {
return {
items: [
{
skill: {
_id: 'skills:1',
slug: 'skill-a',
displayName: 'Skill A',
summary: 's',
tags: { latest: 'versions:1', stable: 'versions:2' },
stats: { downloads: 0, stars: 0, versions: 2, comments: 0 },
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '2.0.0', createdAt: 3, changelog: 'c' },
},
{
skill: {
_id: 'skills:2',
slug: 'skill-b',
displayName: 'Skill B',
summary: 's',
tags: { latest: 'versions:3' },
stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '1.0.0', createdAt: 3, changelog: 'c' },
},
],
nextCursor: null,
}
}
// Batch query should receive all version IDs from all skills
if ('versionIds' in args) {
const ids = args.versionIds as string[]
expect(ids).toHaveLength(3)
expect(ids).toContain('versions:1')
expect(ids).toContain('versions:2')
expect(ids).toContain('versions:3')
return [
{ _id: 'versions:1', version: '2.0.0', softDeletedAt: undefined },
{ _id: 'versions:2', version: '1.0.0', softDeletedAt: undefined },
{ _id: 'versions:3', version: '1.0.0', softDeletedAt: undefined },
]
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.listSkillsV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/skills'),
)
expect(response.status).toBe(200)
const json = await response.json()
// Verify tags are correctly resolved for each skill
expect(json.items[0].tags.latest).toBe('2.0.0')
expect(json.items[0].tags.stable).toBe('1.0.0')
expect(json.items[1].tags.latest).toBe('1.0.0')
// Verify batch query was called exactly once (not per-tag)
const batchCalls = runQuery.mock.calls.filter(
([, args]) => args && 'versionIds' in (args as Record<string, unknown>),
)
expect(batchCalls).toHaveLength(1)
})
it('lists souls with resolved tags using batch query', async () => {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('cursor' in args || 'limit' in args) {
return {
items: [
{
soul: {
_id: 'souls:1',
slug: 'demo-soul',
displayName: 'Demo Soul',
summary: 's',
tags: { latest: 'soulVersions:1' },
stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '1.0.0', createdAt: 3, changelog: 'c' },
},
],
nextCursor: null,
}
}
if ('versionIds' in args) {
return [{ _id: 'soulVersions:1', version: '1.0.0', softDeletedAt: undefined }]
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.listSoulsV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/souls?limit=1'),
)
expect(response.status).toBe(200)
const json = await response.json()
expect(json.items[0].tags.latest).toBe('1.0.0')
})
it('batches tag resolution across multiple souls into single query', async () => {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('cursor' in args || 'limit' in args) {
return {
items: [
{
soul: {
_id: 'souls:1',
slug: 'soul-a',
displayName: 'Soul A',
summary: 's',
tags: { latest: 'soulVersions:1', stable: 'soulVersions:2' },
stats: { downloads: 0, stars: 0, versions: 2, comments: 0 },
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '2.0.0', createdAt: 3, changelog: 'c' },
},
{
soul: {
_id: 'souls:2',
slug: 'soul-b',
displayName: 'Soul B',
summary: 's',
tags: { latest: 'soulVersions:3' },
stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '1.0.0', createdAt: 3, changelog: 'c' },
},
],
nextCursor: null,
}
}
if ('versionIds' in args) {
const ids = args.versionIds as string[]
expect(ids).toHaveLength(3)
expect(ids).toContain('soulVersions:1')
expect(ids).toContain('soulVersions:2')
expect(ids).toContain('soulVersions:3')
return [
{ _id: 'soulVersions:1', version: '2.0.0', softDeletedAt: undefined },
{ _id: 'soulVersions:2', version: '1.0.0', softDeletedAt: undefined },
{ _id: 'soulVersions:3', version: '1.0.0', softDeletedAt: undefined },
]
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.listSoulsV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/souls'),
)
expect(response.status).toBe(200)
const json = await response.json()
expect(json.items[0].tags.latest).toBe('2.0.0')
expect(json.items[0].tags.stable).toBe('1.0.0')
expect(json.items[1].tags.latest).toBe('1.0.0')
const batchCalls = runQuery.mock.calls.filter(
([, args]) => args && 'versionIds' in (args as Record<string, unknown>),
)
expect(batchCalls).toHaveLength(1)
})
it('souls get resolves tags using batch query', async () => {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('slug' in args) {
return {
soul: {
_id: 'souls:1',
slug: 'demo-soul',
displayName: 'Demo Soul',
summary: 's',
tags: { latest: 'soulVersions:1' },
stats: { downloads: 0, stars: 0, versions: 1, comments: 0 },
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '1.0.0', createdAt: 3, changelog: 'c' },
owner: null,
}
}
if ('versionIds' in args) {
return [{ _id: 'soulVersions:1', version: '1.0.0', softDeletedAt: undefined }]
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.soulsGetRouterV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/souls/demo-soul'),
)
expect(response.status).toBe(200)
const json = await response.json()
expect(json.soul.tags.latest).toBe('1.0.0')
})
it('lists skills supports sort aliases', async () => {
const checks: Array<[string, string]> = [
['rating', 'stars'],
@@ -218,6 +427,52 @@ describe('httpApiV1 handlers', () => {
expect(response.status).toBe(404)
})
it('get skill returns pending-scan message for owner api token', async () => {
vi.mocked(getOptionalApiTokenUserId).mockResolvedValue('users:1' as never)
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('slug' in args) {
return {
_id: 'skills:1',
slug: 'demo',
ownerUserId: 'users:1',
moderationStatus: 'hidden',
moderationReason: 'pending.scan',
}
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.skillsGetRouterV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/skills/demo'),
)
expect(response.status).toBe(423)
expect(await response.text()).toContain('security scan is pending')
})
it('get skill returns undelete hint for owner soft-deleted skill', async () => {
vi.mocked(getOptionalApiTokenUserId).mockResolvedValue('users:1' as never)
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('slug' in args) {
return {
_id: 'skills:1',
slug: 'demo',
ownerUserId: 'users:1',
softDeletedAt: 1,
moderationStatus: 'hidden',
}
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.skillsGetRouterV1Handler(
makeCtx({ runQuery, runMutation }),
new Request('https://example.com/api/v1/skills/demo'),
)
expect(response.status).toBe(410)
expect(await response.text()).toContain('clawhub undelete demo')
})
it('get skill returns payload', async () => {
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('slug' in args) {
@@ -241,7 +496,10 @@ describe('httpApiV1 handlers', () => {
owner: { handle: 'p', displayName: 'Peter', image: null },
}
}
if ('versionId' in args) return { version: '1.0.0' }
// Batch query for tag resolution
if ('versionIds' in args) {
return [{ _id: 'versions:1', version: '1.0.0', softDeletedAt: undefined }]
}
return null
})
const runMutation = vi.fn().mockResolvedValue(okRate())
@@ -549,6 +807,63 @@ describe('httpApiV1 handlers', () => {
expect(response2.status).toBe(200)
})
it('delete/undelete map forbidden/not-found/unknown to 403/404/500', async () => {
vi.mocked(requireApiTokenUser).mockResolvedValue({
userId: 'users:1',
user: { handle: 'p' },
} as never)
const runMutationForbidden = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('key' in args) return okRate()
throw new Error('Forbidden')
})
const forbidden = await __handlers.skillsDeleteRouterV1Handler(
makeCtx({ runMutation: runMutationForbidden }),
new Request('https://example.com/api/v1/skills/demo', {
method: 'DELETE',
headers: { Authorization: 'Bearer clh_test' },
}),
)
expect(forbidden.status).toBe(403)
expect(await forbidden.text()).toBe('Forbidden')
vi.mocked(requireApiTokenUser).mockResolvedValue({
userId: 'users:1',
user: { handle: 'p' },
} as never)
const runMutationNotFound = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('key' in args) return okRate()
throw new Error('Skill not found')
})
const notFound = await __handlers.skillsPostRouterV1Handler(
makeCtx({ runMutation: runMutationNotFound }),
new Request('https://example.com/api/v1/skills/demo/undelete', {
method: 'POST',
headers: { Authorization: 'Bearer clh_test' },
}),
)
expect(notFound.status).toBe(404)
expect(await notFound.text()).toBe('Skill not found')
vi.mocked(requireApiTokenUser).mockResolvedValue({
userId: 'users:1',
user: { handle: 'p' },
} as never)
const runMutationUnknown = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if ('key' in args) return okRate()
throw new Error('boom')
})
const unknown = await __handlers.soulsDeleteRouterV1Handler(
makeCtx({ runMutation: runMutationUnknown }),
new Request('https://example.com/api/v1/souls/demo-soul', {
method: 'DELETE',
headers: { Authorization: 'Bearer clh_test' },
}),
)
expect(unknown.status).toBe(500)
expect(await unknown.text()).toBe('Internal Server Error')
})
it('ban user requires auth', async () => {
vi.mocked(requireApiTokenUser).mockRejectedValueOnce(new Error('Unauthorized'))
const runMutation = vi.fn().mockResolvedValue(okRate())
+195 -190
View File
@@ -3,16 +3,12 @@ import { api, internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
import { httpAction } from './_generated/server'
import { requireApiTokenUser } from './lib/apiTokenAuth'
import { hashToken } from './lib/tokens'
import { getOptionalApiTokenUserId, requireApiTokenUser } from './lib/apiTokenAuth'
import { applyRateLimit, parseBearerToken } from './lib/httpRateLimit'
import { corsHeaders, mergeHeaders } from './lib/httpHeaders'
import { publishVersionForUser } from './skills'
import { publishSoulVersionForUser } from './souls'
const RATE_LIMIT_WINDOW_MS = 60_000
const RATE_LIMITS = {
read: { ip: 120, key: 600 },
write: { ip: 30, key: 120 },
} as const
const MAX_RAW_FILE_BYTES = 200 * 1024
type SearchSkillEntry = {
@@ -212,28 +208,29 @@ async function listSkillsV1Handler(ctx: ActionCtx, request: Request) {
sort,
})) as ListSkillsResult
const items = await Promise.all(
result.items.map(async (item) => {
const tags = await resolveTags(ctx, item.skill.tags)
return {
slug: item.skill.slug,
displayName: item.skill.displayName,
summary: item.skill.summary ?? null,
tags,
stats: item.skill.stats,
createdAt: item.skill.createdAt,
updatedAt: item.skill.updatedAt,
latestVersion: item.latestVersion
? {
version: item.latestVersion.version,
createdAt: item.latestVersion.createdAt,
changelog: item.latestVersion.changelog,
}
: null,
}
}),
// Batch resolve all tags in a single query instead of N queries
const resolvedTagsList = await resolveTagsBatch(
ctx,
result.items.map((item) => item.skill.tags),
)
const items = result.items.map((item, idx) => ({
slug: item.skill.slug,
displayName: item.skill.displayName,
summary: item.skill.summary ?? null,
tags: resolvedTagsList[idx],
stats: item.skill.stats,
createdAt: item.skill.createdAt,
updatedAt: item.skill.updatedAt,
latestVersion: item.latestVersion
? {
version: item.latestVersion.version,
createdAt: item.latestVersion.createdAt,
changelog: item.latestVersion.changelog,
}
: null,
}))
return json({ items, nextCursor: result.nextCursor ?? null }, 200, rate.headers)
}
@@ -251,9 +248,13 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
if (segments.length === 1) {
const result = (await ctx.runQuery(api.skills.getBySlug, { slug })) as GetBySlugResult
if (!result?.skill) return text('Skill not found', 404, rate.headers)
if (!result?.skill) {
const hidden = await describeOwnerVisibleSkillState(ctx, request, slug)
if (hidden) return text(hidden.message, hidden.status, rate.headers)
return text('Skill not found', 404, rate.headers)
}
const tags = await resolveTags(ctx, result.skill.tags)
const [tags] = await resolveTagsBatch(ctx, [result.skill.tags])
return json(
{
skill: {
@@ -392,7 +393,9 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const isSvg =
file.contentType?.toLowerCase().includes('svg') || file.path.toLowerCase().endsWith('.svg')
const headers = mergeHeaders(rate.headers, {
const headers = mergeHeaders(
rate.headers,
{
'Content-Type': file.contentType
? `${file.contentType}; charset=utf-8`
: 'text/plain; charset=utf-8',
@@ -408,13 +411,61 @@ async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
'Content-Security-Policy':
"default-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'",
...(isSvg ? { 'Content-Disposition': 'attachment' } : {}),
})
},
corsHeaders(),
)
return new Response(textContent, { status: 200, headers })
}
return text('Not found', 404, rate.headers)
}
async function describeOwnerVisibleSkillState(
ctx: ActionCtx,
request: Request,
slug: string,
): Promise<{ status: number; message: string } | null> {
const skill = await ctx.runQuery(internal.skills.getSkillBySlugInternal, { slug })
if (!skill) return null
const apiTokenUserId = await getOptionalApiTokenUserId(ctx, request)
const isOwner = Boolean(apiTokenUserId && apiTokenUserId === skill.ownerUserId)
if (!isOwner) return null
if (skill.softDeletedAt) {
return {
status: 410,
message: `Skill is hidden/deleted. Run "clawhub undelete ${slug}" to restore it.`,
}
}
if (skill.moderationStatus === 'hidden') {
if (skill.moderationReason === 'pending.scan' || skill.moderationReason === 'scanner.vt.pending') {
return {
status: 423,
message: 'Skill is hidden while security scan is pending. Try again in a few minutes.',
}
}
if (skill.moderationReason === 'quality.low') {
return {
status: 403,
message:
'Skill is hidden by quality checks. Update SKILL.md content or run "clawhub undelete <slug>" after review.',
}
}
return {
status: 403,
message: `Skill is hidden by moderation${skill.moderationReason ? ` (${skill.moderationReason})` : ''}.`,
}
}
if (skill.moderationStatus === 'removed') {
return { status: 410, message: 'Skill has been removed by moderation.' }
}
return null
}
export const skillsGetRouterV1Http = httpAction(skillsGetRouterV1Handler)
async function publishSkillV1Handler(ctx: ActionCtx, request: Request) {
@@ -487,8 +538,8 @@ async function skillsPostRouterV1Handler(ctx: ActionCtx, request: Request) {
deleted: false,
})
return json({ ok: true }, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
} catch (error) {
return softDeleteErrorToResponse('skill', error, rate.headers)
}
}
@@ -509,8 +560,8 @@ async function skillsDeleteRouterV1Handler(ctx: ActionCtx, request: Request) {
deleted: true,
})
return json({ ok: true }, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
} catch (error) {
return softDeleteErrorToResponse('skill', error, rate.headers)
}
}
@@ -768,132 +819,66 @@ function parsePublishBody(body: unknown) {
}
}
async function resolveSoulTags(
/**
* Batch resolve soul version tags to version strings.
* Collects all version IDs, fetches them in a single query, then maps back.
* Reduces N sequential queries to 1 batch query.
*/
async function resolveSoulTagsBatch(
ctx: ActionCtx,
tags: Record<string, Id<'soulVersions'>>,
): Promise<Record<string, string>> {
const resolved: Record<string, string> = {}
for (const [tag, versionId] of Object.entries(tags)) {
const version = await ctx.runQuery(api.souls.getVersionById, { versionId })
if (version && !version.softDeletedAt) {
resolved[tag] = version.version
tagsList: Array<Record<string, Id<'soulVersions'>>>,
): Promise<Array<Record<string, string>>> {
return resolveVersionTagsBatch(ctx, tagsList, internal.souls.getVersionsByIdsInternal)
}
async function resolveTagsBatch(
ctx: ActionCtx,
tagsList: Array<Record<string, Id<'skillVersions'>>>,
): Promise<Array<Record<string, string>>> {
return resolveVersionTagsBatch(ctx, tagsList, internal.skills.getVersionsByIdsInternal)
}
/**
* Batch resolve version tags to version strings.
* Collects all version IDs, fetches them in a single query, then maps back.
*
* Notes:
* - Uses `internal.*` queries to avoid expanding the public Convex API surface.
* - Sorts ids for stable query args (helps caching/log diffs).
*/
async function resolveVersionTagsBatch<TTable extends 'skillVersions' | 'soulVersions'>(
ctx: ActionCtx,
tagsList: Array<Record<string, Id<TTable>>>,
getVersionsByIdsQuery: unknown,
): Promise<Array<Record<string, string>>> {
const allVersionIds = new Set<Id<TTable>>()
for (const tags of tagsList) {
for (const versionId of Object.values(tags)) allVersionIds.add(versionId)
}
if (allVersionIds.size === 0) return tagsList.map(() => ({}))
const versionIds = [...allVersionIds].sort() as Array<Id<TTable>>
const versions =
((await ctx.runQuery(getVersionsByIdsQuery as never, { versionIds } as never)) as Array<{
_id: Id<TTable>
version: string
softDeletedAt?: unknown
}> | null) ?? []
const versionMap = new Map<Id<TTable>, string>()
for (const v of versions) {
if (!v?.softDeletedAt) versionMap.set(v._id, v.version)
}
return tagsList.map((tags) => {
const resolved: Record<string, string> = {}
for (const [tag, versionId] of Object.entries(tags)) {
const version = versionMap.get(versionId)
if (version) resolved[tag] = version
}
}
return resolved
}
async function resolveTags(
ctx: ActionCtx,
tags: Record<string, Id<'skillVersions'>>,
): Promise<Record<string, string>> {
const resolved: Record<string, string> = {}
for (const [tag, versionId] of Object.entries(tags)) {
const version = await ctx.runQuery(api.skills.getVersionById, { versionId })
if (version && !version.softDeletedAt) {
resolved[tag] = version.version
}
}
return resolved
}
async function applyRateLimit(
ctx: ActionCtx,
request: Request,
kind: 'read' | 'write',
): Promise<{ ok: true; headers: HeadersInit } | { ok: false; response: Response }> {
const ip = getClientIp(request) ?? 'unknown'
const ipResult = await checkRateLimit(ctx, `ip:${ip}`, RATE_LIMITS[kind].ip)
const token = parseBearerToken(request)
const keyResult = token
? await checkRateLimit(ctx, `key:${await hashToken(token)}`, RATE_LIMITS[kind].key)
: null
const chosen = pickMostRestrictive(ipResult, keyResult)
const headers = rateHeaders(chosen)
if (!ipResult.allowed || (keyResult && !keyResult.allowed)) {
return {
ok: false,
response: text('Rate limit exceeded', 429, headers),
}
}
return { ok: true, headers }
}
type RateLimitResult = {
allowed: boolean
remaining: number
limit: number
resetAt: number
}
async function checkRateLimit(
ctx: ActionCtx,
key: string,
limit: number,
): Promise<RateLimitResult> {
// Step 1: Read-only check — no write conflicts for denied requests
const status = (await ctx.runQuery(internal.rateLimits.getRateLimitStatusInternal, {
key,
limit,
windowMs: RATE_LIMIT_WINDOW_MS,
})) as RateLimitResult
if (!status.allowed) {
return status
}
// Step 2: Consume a token (only when allowed, with double-check for races)
const result = (await ctx.runMutation(internal.rateLimits.consumeRateLimitInternal, {
key,
limit,
windowMs: RATE_LIMIT_WINDOW_MS,
})) as { allowed: boolean; remaining: number }
return {
allowed: result.allowed,
remaining: result.remaining,
limit: status.limit,
resetAt: status.resetAt,
}
}
function pickMostRestrictive(primary: RateLimitResult, secondary: RateLimitResult | null) {
if (!secondary) return primary
if (!primary.allowed) return primary
if (!secondary.allowed) return secondary
return secondary.remaining < primary.remaining ? secondary : primary
}
function rateHeaders(result: RateLimitResult): HeadersInit {
const resetSeconds = Math.ceil(result.resetAt / 1000)
return {
'X-RateLimit-Limit': String(result.limit),
'X-RateLimit-Remaining': String(result.remaining),
'X-RateLimit-Reset': String(resetSeconds),
...(result.allowed ? {} : { 'Retry-After': String(resetSeconds) }),
}
}
function getClientIp(request: Request) {
const header =
request.headers.get('cf-connecting-ip') ??
request.headers.get('x-real-ip') ??
request.headers.get('x-forwarded-for') ??
request.headers.get('fly-client-ip')
if (!header) return null
if (header.includes(',')) return header.split(',')[0]?.trim() || null
return header.trim()
}
function parseBearerToken(request: Request) {
const header = request.headers.get('authorization') ?? request.headers.get('Authorization')
if (!header) return null
const trimmed = header.trim()
if (!trimmed.toLowerCase().startsWith('bearer ')) return null
const token = trimmed.slice(7).trim()
return token || null
return resolved
})
}
function json(value: unknown, status = 200, headers?: HeadersInit) {
@@ -905,6 +890,7 @@ function json(value: unknown, status = 200, headers?: HeadersInit) {
'Cache-Control': 'no-store',
},
headers,
corsHeaders(),
),
})
}
@@ -918,14 +904,11 @@ function text(value: string, status: number, headers?: HeadersInit) {
'Cache-Control': 'no-store',
},
headers,
corsHeaders(),
),
})
}
function mergeHeaders(base: HeadersInit, extra?: HeadersInit) {
return { ...(base as Record<string, string>), ...(extra as Record<string, string>) }
}
function getPathSegments(request: Request, prefix: string) {
const pathname = new URL(request.url).pathname
if (!pathname.startsWith(prefix)) return []
@@ -995,28 +978,29 @@ async function listSoulsV1Handler(ctx: ActionCtx, request: Request) {
cursor,
})) as ListSoulsResult
const items = await Promise.all(
result.items.map(async (item) => {
const tags = await resolveSoulTags(ctx, item.soul.tags)
return {
slug: item.soul.slug,
displayName: item.soul.displayName,
summary: item.soul.summary ?? null,
tags,
stats: item.soul.stats,
createdAt: item.soul.createdAt,
updatedAt: item.soul.updatedAt,
latestVersion: item.latestVersion
? {
version: item.latestVersion.version,
createdAt: item.latestVersion.createdAt,
changelog: item.latestVersion.changelog,
}
: null,
}
}),
// Batch resolve all tags in a single query instead of N queries
const resolvedTagsList = await resolveSoulTagsBatch(
ctx,
result.items.map((item) => item.soul.tags),
)
const items = result.items.map((item, idx) => ({
slug: item.soul.slug,
displayName: item.soul.displayName,
summary: item.soul.summary ?? null,
tags: resolvedTagsList[idx],
stats: item.soul.stats,
createdAt: item.soul.createdAt,
updatedAt: item.soul.updatedAt,
latestVersion: item.latestVersion
? {
version: item.latestVersion.version,
createdAt: item.latestVersion.createdAt,
changelog: item.latestVersion.changelog,
}
: null,
}))
return json({ items, nextCursor: result.nextCursor ?? null }, 200, rate.headers)
}
@@ -1036,7 +1020,7 @@ async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const result = (await ctx.runQuery(api.souls.getBySlug, { slug })) as GetSoulBySlugResult
if (!result?.soul) return text('Soul not found', 404, rate.headers)
const tags = await resolveSoulTags(ctx, result.soul.tags)
const [tags] = await resolveSoulTagsBatch(ctx, [result.soul.tags])
return json(
{
soul: {
@@ -1170,7 +1154,9 @@ async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const isSvg =
file.contentType?.toLowerCase().includes('svg') || file.path.toLowerCase().endsWith('.svg')
const headers = mergeHeaders(rate.headers, {
const headers = mergeHeaders(
rate.headers,
{
'Content-Type': file.contentType
? `${file.contentType}; charset=utf-8`
: 'text/plain; charset=utf-8',
@@ -1186,7 +1172,9 @@ async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
'Content-Security-Policy':
"default-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'",
...(isSvg ? { 'Content-Disposition': 'attachment' } : {}),
})
},
corsHeaders(),
)
return new Response(textContent, { status: 200, headers })
}
@@ -1247,8 +1235,8 @@ async function soulsPostRouterV1Handler(ctx: ActionCtx, request: Request) {
deleted: false,
})
return json({ ok: true }, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
} catch (error) {
return softDeleteErrorToResponse('soul', error, rate.headers)
}
}
@@ -1269,13 +1257,30 @@ async function soulsDeleteRouterV1Handler(ctx: ActionCtx, request: Request) {
deleted: true,
})
return json({ ok: true }, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
} catch (error) {
return softDeleteErrorToResponse('soul', error, rate.headers)
}
}
export const soulsDeleteRouterV1Http = httpAction(soulsDeleteRouterV1Handler)
function softDeleteErrorToResponse(
entity: 'skill' | 'soul',
error: unknown,
headers: HeadersInit,
) {
const message = error instanceof Error ? error.message : `${entity} delete failed`
const lower = message.toLowerCase()
if (lower.includes('unauthorized')) return text('Unauthorized', 401, headers)
if (lower.includes('forbidden')) return text('Forbidden', 403, headers)
if (lower.includes('not found')) return text(message, 404, headers)
if (lower.includes('slug required')) return text('Slug required', 400, headers)
// Unknown: server-side failure. Keep body generic.
return text('Internal Server Error', 500, headers)
}
async function starsPostRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'write')
if (!rate.ok) return rate.response
+37
View File
@@ -0,0 +1,37 @@
import { httpAction } from './_generated/server'
import { corsHeaders, mergeHeaders } from './lib/httpHeaders'
function getHeader(request: Request, name: string) {
return request.headers.get(name) ?? request.headers.get(name.toLowerCase())
}
export function buildPreflightHeaders(request: Request) {
const requestedHeaders = getHeader(request, 'Access-Control-Request-Headers')?.trim() || null
const requestedMethod = getHeader(request, 'Access-Control-Request-Method')?.trim() || null
const vary = [
...(requestedMethod ? ['Access-Control-Request-Method'] : []),
...(requestedHeaders ? ['Access-Control-Request-Headers'] : []),
].join(', ')
return mergeHeaders(
corsHeaders(),
{
'Access-Control-Allow-Methods': 'GET, POST, PUT, DELETE, OPTIONS, PATCH, HEAD',
'Access-Control-Allow-Headers':
requestedHeaders ?? 'Content-Type, Authorization, Digest, X-Clawhub-Version',
'Access-Control-Max-Age': '86400',
...(vary ? { Vary: vary } : {}),
},
)
}
export const preflightHandler = httpAction(async (_ctx, request) => {
// No cookies/credentials supported; allow any origin for simple browser access.
// If we ever add cookie auth, this must switch to reflecting origin + Allow-Credentials.
return new Response(null, {
status: 204,
headers: buildPreflightHeaders(request),
})
})
+2 -2
View File
@@ -9,7 +9,7 @@ export async function requireUser(ctx: MutationCtx | QueryCtx) {
const userId = await getAuthUserId(ctx)
if (!userId) throw new Error('Unauthorized')
const user = await ctx.db.get(userId)
if (!user || user.deletedAt) throw new Error('User not found')
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
return { userId, user }
}
@@ -17,7 +17,7 @@ export async function requireUserFromAction(ctx: ActionCtx) {
const userId = await getAuthUserId(ctx)
if (!userId) throw new Error('Unauthorized')
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
if (!user || user.deletedAt) throw new Error('User not found')
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
return { userId, user: user as Doc<'users'> }
}
+110
View File
@@ -0,0 +1,110 @@
import { describe, expect, it, vi } from 'vitest'
import { getOptionalApiTokenUserId } from './apiTokenAuth'
import { hashToken } from './tokens'
describe('getOptionalApiTokenUserId', () => {
it('returns null when auth header is missing', async () => {
const ctx = {
runQuery: vi.fn(),
}
const request = new Request('https://example.com')
const userId = await getOptionalApiTokenUserId(ctx as never, request)
expect(userId).toBeNull()
expect(ctx.runQuery).not.toHaveBeenCalled()
})
it('returns null for unknown token', async () => {
const ctx = {
runQuery: vi.fn().mockResolvedValue(null),
}
const request = new Request('https://example.com', {
headers: { authorization: 'Bearer token-1' },
})
const userId = await getOptionalApiTokenUserId(ctx as never, request)
expect(userId).toBeNull()
expect(ctx.runQuery).toHaveBeenCalledTimes(1)
expect(ctx.runQuery.mock.calls[0]?.[1]).toEqual({
tokenHash: await hashToken('token-1'),
})
})
it('returns user id when token and user are valid', async () => {
const tokenId = 'apiTokens_1'
const expectedUserId = 'users_1'
const ctx = {
runQuery: vi
.fn()
.mockImplementation(async (_fn, args: { tokenHash?: string; tokenId?: string }) => {
if (args.tokenHash) {
return { _id: tokenId, revokedAt: undefined }
}
if (args.tokenId) {
return { _id: expectedUserId, deletedAt: undefined }
}
return null
}),
}
const request = new Request('https://example.com', {
headers: { authorization: 'Bearer token-2' },
})
const userId = await getOptionalApiTokenUserId(ctx as never, request)
expect(userId).toBe(expectedUserId)
expect(ctx.runQuery).toHaveBeenCalledTimes(2)
})
it('returns null when user is deleted', async () => {
const tokenId = 'apiTokens_2'
const ctx = {
runQuery: vi
.fn()
.mockImplementation(async (_fn, args: { tokenHash?: string; tokenId?: string }) => {
if (args.tokenHash) {
return { _id: tokenId, revokedAt: undefined }
}
if (args.tokenId) {
return { _id: 'users_deleted', deletedAt: Date.now() }
}
return null
}),
}
const request = new Request('https://example.com', {
headers: { authorization: 'Bearer token-3' },
})
const userId = await getOptionalApiTokenUserId(ctx as never, request)
expect(userId).toBeNull()
expect(ctx.runQuery).toHaveBeenCalledTimes(2)
})
it('returns null when user is deactivated', async () => {
const tokenId = 'apiTokens_3'
const ctx = {
runQuery: vi
.fn()
.mockImplementation(async (_fn, args: { tokenHash?: string; tokenId?: string }) => {
if (args.tokenHash) {
return { _id: tokenId, revokedAt: undefined }
}
if (args.tokenId) {
return { _id: 'users_deactivated', deactivatedAt: Date.now() }
}
return null
}),
}
const request = new Request('https://example.com', {
headers: { authorization: 'Bearer token-4' },
})
const userId = await getOptionalApiTokenUserId(ctx as never, request)
expect(userId).toBeNull()
expect(ctx.runQuery).toHaveBeenCalledTimes(2)
})
})
+21 -1
View File
@@ -21,12 +21,32 @@ export async function requireApiTokenUser(
const user = await ctx.runQuery(internal.tokens.getUserForTokenInternal, {
tokenId: apiToken._id,
})
if (!user || user.deletedAt) throw new ConvexError('Unauthorized')
if (!user || user.deletedAt || user.deactivatedAt) throw new ConvexError('Unauthorized')
await ctx.runMutation(internal.tokens.touchInternal, { tokenId: apiToken._id })
return { user, userId: user._id }
}
export async function getOptionalApiTokenUserId(
ctx: ActionCtx,
request: Request,
): Promise<Doc<'users'>['_id'] | null> {
const header = request.headers.get('authorization') ?? request.headers.get('Authorization')
const token = parseBearerToken(header)
if (!token) return null
const tokenHash = await hashToken(token)
const apiToken = await ctx.runQuery(internal.tokens.getByHashInternal, { tokenHash })
if (!apiToken || apiToken.revokedAt) return null
const user = await ctx.runQuery(internal.tokens.getUserForTokenInternal, {
tokenId: apiToken._id,
})
if (!user || user.deletedAt || user.deactivatedAt) return null
return user._id
}
function parseBearerToken(header: string | null) {
if (!header) return null
const trimmed = header.trim()
+95
View File
@@ -0,0 +1,95 @@
/* @vitest-environment node */
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { EMBEDDING_DIMENSIONS, generateEmbedding } from './embeddings'
const fetchMock = vi.fn<typeof fetch>()
const consoleWarnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
const originalFetch = globalThis.fetch
const originalApiKey = process.env.OPENAI_API_KEY
function jsonResponse(payload: unknown, init?: ResponseInit) {
return new Response(JSON.stringify(payload), {
status: 200,
headers: {
'content-type': 'application/json',
},
...init,
})
}
beforeEach(() => {
fetchMock.mockReset()
globalThis.fetch = fetchMock as typeof fetch
process.env.OPENAI_API_KEY = 'test-key'
consoleWarnSpy.mockClear()
})
afterEach(() => {
globalThis.fetch = originalFetch
if (originalApiKey === undefined) {
delete process.env.OPENAI_API_KEY
} else {
process.env.OPENAI_API_KEY = originalApiKey
}
vi.useRealTimers()
})
describe('generateEmbedding', () => {
it('returns zero embedding when OPENAI_API_KEY is missing', async () => {
delete process.env.OPENAI_API_KEY
const result = await generateEmbedding('hello world')
expect(result).toHaveLength(EMBEDDING_DIMENSIONS)
expect(result.every((value) => value === 0)).toBe(true)
expect(fetchMock).not.toHaveBeenCalled()
})
it('retries on 429 responses and then succeeds', async () => {
vi.useFakeTimers()
fetchMock.mockResolvedValueOnce(new Response('rate limited', { status: 429 }))
fetchMock.mockResolvedValueOnce(jsonResponse({ data: [{ embedding: [0.25, 0.75] }] }))
const promise = generateEmbedding('retry me')
await vi.runAllTimersAsync()
await expect(promise).resolves.toEqual([0.25, 0.75])
expect(fetchMock).toHaveBeenCalledTimes(2)
})
it('does not retry non-retryable 4xx responses', async () => {
fetchMock.mockResolvedValueOnce(new Response('bad request', { status: 400 }))
await expect(generateEmbedding('bad')).rejects.toThrow('Embedding failed: bad request')
expect(fetchMock).toHaveBeenCalledTimes(1)
})
it('retries on network failures and then succeeds', async () => {
vi.useFakeTimers()
fetchMock.mockRejectedValueOnce(new TypeError('fetch failed'))
fetchMock.mockResolvedValueOnce(jsonResponse({ data: [{ embedding: [1, 2, 3] }] }))
const promise = generateEmbedding('network retry')
await vi.runAllTimersAsync()
await expect(promise).resolves.toEqual([1, 2, 3])
expect(fetchMock).toHaveBeenCalledTimes(2)
})
it('retries timeouts up to max attempts and preserves timeout error', async () => {
vi.useFakeTimers()
fetchMock.mockRejectedValue(new DOMException('aborted', 'AbortError'))
const promise = generateEmbedding('always timeout')
const rejection = expect(promise).rejects.toThrow(
'OpenAI API request timed out after 10 seconds',
)
await vi.runAllTimersAsync()
await rejection
expect(fetchMock).toHaveBeenCalledTimes(3)
})
})
+127 -20
View File
@@ -1,10 +1,67 @@
export const EMBEDDING_MODEL = 'text-embedding-3-small'
export const EMBEDDING_DIMENSIONS = 1536
const EMBEDDING_ENDPOINT = 'https://api.openai.com/v1/embeddings'
const REQUEST_TIMEOUT_MS = 10_000
const MAX_ATTEMPTS = 3
const BASE_RETRY_DELAY_MS = 1_000
class RetryableEmbeddingError extends Error {
constructor(message: string, options?: { cause?: unknown }) {
super(message, options)
this.name = 'RetryableEmbeddingError'
}
}
function emptyEmbedding() {
return Array.from({ length: EMBEDDING_DIMENSIONS }, () => 0)
}
function parseRetryAfterMs(retryAfterHeader: string | null) {
if (!retryAfterHeader) return null
const seconds = Number(retryAfterHeader)
if (Number.isFinite(seconds) && seconds >= 0) {
return Math.round(seconds * 1000)
}
const dateMs = Date.parse(retryAfterHeader)
if (Number.isFinite(dateMs)) {
return Math.max(0, dateMs - Date.now())
}
return null
}
function getRetryDelayMs(attempt: number, retryAfterMs: number | null) {
const exponentialDelayMs = BASE_RETRY_DELAY_MS * 2 ** attempt
if (retryAfterMs == null) return exponentialDelayMs
return Math.max(exponentialDelayMs, retryAfterMs)
}
function normalizeRetryableNetworkError(error: unknown) {
if (!(error instanceof Error)) return null
if (error.name === 'AbortError') {
return new RetryableEmbeddingError(
`OpenAI API request timed out after ${Math.floor(REQUEST_TIMEOUT_MS / 1000)} seconds`,
{ cause: error },
)
}
if (error instanceof TypeError) {
return new RetryableEmbeddingError(`Embedding request failed: ${error.message}`, { cause: error })
}
return null
}
function sleep(ms: number) {
return new Promise<void>((resolve) => {
setTimeout(resolve, ms)
})
}
export async function generateEmbedding(text: string) {
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) {
@@ -12,27 +69,77 @@ export async function generateEmbedding(text: string) {
return emptyEmbedding()
}
const response = await fetch('https://api.openai.com/v1/embeddings', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: EMBEDDING_MODEL,
input: text,
}),
})
let lastRetryableError: RetryableEmbeddingError | null = null
if (!response.ok) {
const message = await response.text()
throw new Error(`Embedding failed: ${message}`)
for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt++) {
const controller = new AbortController()
const timeoutId = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS)
try {
const response = await fetch(EMBEDDING_ENDPOINT, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: EMBEDDING_MODEL,
input: text,
}),
signal: controller.signal,
})
if (!response.ok) {
const message = await response.text()
const isRetryableStatus = response.status === 429 || response.status >= 500
if (isRetryableStatus) {
const retryableError = new RetryableEmbeddingError(
`Embedding failed (${response.status}): ${message}`,
)
lastRetryableError = retryableError
if (attempt < MAX_ATTEMPTS - 1) {
const retryAfterMs = parseRetryAfterMs(response.headers.get('retry-after'))
const delayMs = getRetryDelayMs(attempt, retryAfterMs)
console.warn(
`OpenAI embeddings retry in ${delayMs}ms (attempt ${attempt + 1}/${MAX_ATTEMPTS})`,
)
await sleep(delayMs)
continue
}
throw retryableError
}
throw new Error(`Embedding failed: ${message}`)
}
const payload = (await response.json()) as {
data?: Array<{ embedding: number[] }>
}
const embedding = payload.data?.[0]?.embedding
if (!embedding) throw new Error('Embedding missing from response')
return embedding
} catch (error) {
const retryableNetworkError = normalizeRetryableNetworkError(error)
if (retryableNetworkError) {
lastRetryableError = retryableNetworkError
if (attempt < MAX_ATTEMPTS - 1) {
const delayMs = getRetryDelayMs(attempt, null)
console.warn(
`OpenAI embeddings network retry in ${delayMs}ms (attempt ${attempt + 1}/${MAX_ATTEMPTS})`,
)
await sleep(delayMs)
continue
}
throw retryableNetworkError
}
throw error
} finally {
clearTimeout(timeoutId)
}
}
const payload = (await response.json()) as {
data?: Array<{ embedding: number[] }>
}
const embedding = payload.data?.[0]?.embedding
if (!embedding) throw new Error('Embedding missing from response')
return embedding
throw lastRetryableError ?? new Error('Embedding failed after retries')
}
+122 -43
View File
@@ -8,6 +8,7 @@ vi.mock('../_generated/api', () => ({
internal: {
users: {
getByIdInternal: Symbol('getByIdInternal'),
getGitHubProviderAccountIdInternal: Symbol('getGitHubProviderAccountIdInternal'),
updateGithubMetaInternal: Symbol('updateGithubMetaInternal'),
},
},
@@ -19,21 +20,24 @@ describe('requireGitHubAccountAge', () => {
beforeEach(() => {
vi.restoreAllMocks()
vi.unstubAllEnvs()
vi.unstubAllGlobals()
})
afterEach(() => {
vi.useRealTimers()
vi.unstubAllEnvs()
vi.unstubAllGlobals()
})
it('uses cached githubCreatedAt when fresh', async () => {
it('uses cached githubCreatedAt when present', async () => {
vi.useFakeTimers()
const now = new Date('2026-02-02T12:00:00Z')
vi.setSystemTime(now)
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'steipete',
githubCreatedAt: now.getTime() - 10 * ONE_DAY_MS,
githubFetchedAt: now.getTime() - ONE_DAY_MS + 1000,
githubFetchedAt: now.getTime() - ONE_DAY_MS,
})
const runMutation = vi.fn()
const fetchMock = vi.fn()
@@ -44,17 +48,34 @@ describe('requireGitHubAccountAge', () => {
expect(fetchMock).not.toHaveBeenCalled()
expect(runMutation).not.toHaveBeenCalled()
expect(runQuery).toHaveBeenCalledWith(internal.users.getByIdInternal, { userId: 'users:1' })
expect(runQuery).not.toHaveBeenCalledWith(internal.users.getGitHubProviderAccountIdInternal, {
userId: 'users:1',
})
})
vi.useRealTimers()
it('rejects deactivated users', async () => {
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
deactivatedAt: Date.now(),
})
const runMutation = vi.fn()
const fetchMock = vi.fn()
vi.stubGlobal('fetch', fetchMock)
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
).rejects.toThrow(/User not found/i)
expect(fetchMock).not.toHaveBeenCalled()
})
it('rejects accounts younger than 7 days', async () => {
vi.useFakeTimers()
const now = new Date('2026-02-02T12:00:00Z')
vi.setSystemTime(now)
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'newbie',
githubCreatedAt: now.getTime() - 2 * ONE_DAY_MS,
githubFetchedAt: now.getTime() - ONE_DAY_MS / 2,
})
@@ -63,21 +84,20 @@ describe('requireGitHubAccountAge', () => {
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
).rejects.toThrow(/GitHub account must be at least 7 days old/i)
vi.useRealTimers()
})
it('refreshes githubCreatedAt when cache is stale', async () => {
it('fetches githubCreatedAt when missing (by providerAccountId)', async () => {
vi.useFakeTimers()
const now = new Date('2026-02-02T12:00:00Z')
vi.setSystemTime(now)
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'steipete',
githubCreatedAt: undefined,
githubFetchedAt: now.getTime() - 2 * ONE_DAY_MS,
})
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('12345')
const runMutation = vi.fn()
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
@@ -90,7 +110,7 @@ describe('requireGitHubAccountAge', () => {
await requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never)
expect(fetchMock).toHaveBeenCalledWith(
'https://api.github.com/users/steipete',
'https://api.github.com/user/12345',
expect.objectContaining({
headers: expect.objectContaining({ 'User-Agent': 'clawhub' }),
}),
@@ -100,17 +120,54 @@ describe('requireGitHubAccountAge', () => {
githubCreatedAt: Date.parse('2020-01-01T00:00:00Z'),
githubFetchedAt: now.getTime(),
})
})
vi.useRealTimers()
it('rejects when providerAccountId is missing', async () => {
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce(null)
const runMutation = vi.fn()
const fetchMock = vi.fn()
vi.stubGlobal('fetch', fetchMock)
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
).rejects.toThrow(/GitHub account required/i)
expect(fetchMock).not.toHaveBeenCalled()
})
it('rejects when providerAccountId is invalid', async () => {
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('abc123')
const runMutation = vi.fn()
const fetchMock = vi.fn()
vi.stubGlobal('fetch', fetchMock)
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
).rejects.toThrow(/GitHub account lookup failed/i)
expect(fetchMock).not.toHaveBeenCalled()
})
it('throws when GitHub lookup fails', async () => {
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'steipete',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('12345')
const runMutation = vi.fn()
const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 404 })
vi.stubGlobal('fetch', fetchMock)
@@ -121,12 +178,13 @@ describe('requireGitHubAccountAge', () => {
})
it('throws rate-limit error on 403', async () => {
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'steipete',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('12345')
const runMutation = vi.fn()
const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 403 })
vi.stubGlobal('fetch', fetchMock)
@@ -137,12 +195,13 @@ describe('requireGitHubAccountAge', () => {
})
it('throws rate-limit error on 429', async () => {
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'steipete',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('12345')
const runMutation = vi.fn()
const fetchMock = vi.fn().mockResolvedValue({ ok: false, status: 429 })
vi.stubGlobal('fetch', fetchMock)
@@ -152,6 +211,26 @@ describe('requireGitHubAccountAge', () => {
).rejects.toThrow(/rate limit exceeded/i)
})
it('throws when GitHub returns an invalid payload', async () => {
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('12345')
const runMutation = vi.fn()
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({}),
})
vi.stubGlobal('fetch', fetchMock)
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
).rejects.toThrow(/GitHub account lookup failed/i)
})
it('includes Authorization header when GITHUB_TOKEN is set', async () => {
vi.useFakeTimers()
const now = new Date('2026-02-02T12:00:00Z')
@@ -159,12 +238,13 @@ describe('requireGitHubAccountAge', () => {
vi.stubEnv('GITHUB_TOKEN', 'ghp_test123')
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
handle: 'steipete',
githubCreatedAt: undefined,
githubFetchedAt: now.getTime() - 2 * ONE_DAY_MS,
})
const runQuery = vi.fn()
.mockResolvedValueOnce({
_id: 'users:1',
githubCreatedAt: undefined,
githubFetchedAt: 0,
})
.mockResolvedValueOnce('12345')
const runMutation = vi.fn()
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
@@ -177,7 +257,7 @@ describe('requireGitHubAccountAge', () => {
await requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never)
expect(fetchMock).toHaveBeenCalledWith(
'https://api.github.com/users/steipete',
'https://api.github.com/user/12345',
expect.objectContaining({
headers: {
'User-Agent': 'clawhub',
@@ -185,7 +265,6 @@ describe('requireGitHubAccountAge', () => {
},
}),
)
vi.useRealTimers()
})
})
+15 -9
View File
@@ -5,7 +5,6 @@ import type { ActionCtx } from '../_generated/server'
const GITHUB_API = 'https://api.github.com'
const MIN_ACCOUNT_AGE_MS = 7 * 24 * 60 * 60 * 1000
const FETCH_TTL_MS = 24 * 60 * 60 * 1000
type GitHubUser = {
created_at?: string
@@ -13,24 +12,31 @@ type GitHubUser = {
export async function requireGitHubAccountAge(ctx: ActionCtx, userId: Id<'users'>) {
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
if (!user || user.deletedAt) throw new ConvexError('User not found')
const handle = user.handle?.trim()
if (!handle) throw new ConvexError('GitHub handle required')
if (!user || user.deletedAt || user.deactivatedAt) throw new ConvexError('User not found')
const now = Date.now()
let createdAt = user.githubCreatedAt ?? null
const fetchedAt = user.githubFetchedAt ?? 0
const stale = !createdAt || now - fetchedAt > FETCH_TTL_MS
if (stale) {
if (!createdAt) {
const providerAccountId = await ctx.runQuery(internal.users.getGitHubProviderAccountIdInternal, {
userId,
})
if (!providerAccountId) {
// Invariant: GitHub is our only auth provider, so this should never happen.
throw new ConvexError('GitHub account required')
}
if (!/^[0-9]+$/.test(providerAccountId)) {
throw new ConvexError('GitHub account lookup failed')
}
const headers: Record<string, string> = { 'User-Agent': 'clawhub' }
const token = process.env.GITHUB_TOKEN
if (token) {
headers.Authorization = `Bearer ${token}`
}
const response = await fetch(`${GITHUB_API}/users/${encodeURIComponent(handle)}`, {
// Fetch by immutable GitHub numeric ID to avoid username swap attacks entirely.
const response = await fetch(`${GITHUB_API}/user/${providerAccountId}`, {
headers,
})
if (!response.ok) {
+19
View File
@@ -0,0 +1,19 @@
import { describe, expect, it } from 'vitest'
import { canHealSkillOwnershipByGitHubProviderAccountId } from './githubIdentity'
describe('canHealSkillOwnershipByGitHubProviderAccountId', () => {
it('denies when either providerAccountId is missing', () => {
expect(canHealSkillOwnershipByGitHubProviderAccountId(undefined, undefined)).toBe(false)
expect(canHealSkillOwnershipByGitHubProviderAccountId('123', undefined)).toBe(false)
expect(canHealSkillOwnershipByGitHubProviderAccountId(undefined, '123')).toBe(false)
expect(canHealSkillOwnershipByGitHubProviderAccountId(null, '123')).toBe(false)
})
it('denies when providerAccountId differs', () => {
expect(canHealSkillOwnershipByGitHubProviderAccountId('123', '456')).toBe(false)
})
it('allows when providerAccountId matches', () => {
expect(canHealSkillOwnershipByGitHubProviderAccountId('123', '123')).toBe(true)
})
})
+22
View File
@@ -0,0 +1,22 @@
import type { Id } from '../_generated/dataModel'
import type { QueryCtx } from '../_generated/server'
export function canHealSkillOwnershipByGitHubProviderAccountId(
ownerProviderAccountId: string | null | undefined,
callerProviderAccountId: string | null | undefined,
) {
// Security invariant: missing identity must never grant ownership.
if (!ownerProviderAccountId || !callerProviderAccountId) return false
return ownerProviderAccountId === callerProviderAccountId
}
export async function getGitHubProviderAccountId(
ctx: Pick<QueryCtx, 'db'>,
userId: Id<'users'>,
): Promise<string | null> {
const account = await ctx.db
.query('authAccounts')
.withIndex('userIdAndProvider', (q) => q.eq('userId', userId).eq('provider', 'github'))
.unique()
return account?.providerAccountId ?? null
}
+19
View File
@@ -0,0 +1,19 @@
function toHeaderRecord(init?: HeadersInit): Record<string, string> {
if (!init) return {}
if (init instanceof Headers) return Object.fromEntries(init.entries())
if (Array.isArray(init)) return Object.fromEntries(init)
return { ...(init as Record<string, string>) }
}
export function mergeHeaders(...inits: Array<HeadersInit | undefined>): Record<string, string> {
const out: Record<string, string> = {}
for (const init of inits) {
Object.assign(out, toHeaderRecord(init))
}
return out
}
export function corsHeaders(origin: string = '*'): Record<string, string> {
return { 'Access-Control-Allow-Origin': origin }
}
+46
View File
@@ -0,0 +1,46 @@
/* @vitest-environment node */
import { describe, expect, it } from 'vitest'
import { getClientIp } from './httpRateLimit'
describe('getClientIp', () => {
it('uses forwarded headers by default when cf-connecting-ip is missing', () => {
const request = new Request('https://example.com', {
headers: {
'x-forwarded-for': '203.0.113.9',
},
})
delete process.env.TRUST_FORWARDED_IPS
expect(getClientIp(request)).toBe('203.0.113.9')
})
it('can disable forwarded headers explicitly', () => {
const request = new Request('https://example.com', {
headers: {
'x-forwarded-for': '203.0.113.9',
},
})
process.env.TRUST_FORWARDED_IPS = 'false'
expect(getClientIp(request)).toBeNull()
delete process.env.TRUST_FORWARDED_IPS
})
it('returns first ip from cf-connecting-ip', () => {
const request = new Request('https://example.com', {
headers: {
'cf-connecting-ip': '203.0.113.1, 198.51.100.2',
},
})
expect(getClientIp(request)).toBe('203.0.113.1')
})
it('uses forwarded headers when opt-in enabled', () => {
const request = new Request('https://example.com', {
headers: {
'x-forwarded-for': '203.0.113.9, 198.51.100.2',
},
})
process.env.TRUST_FORWARDED_IPS = 'true'
expect(getClientIp(request)).toBe('203.0.113.9')
delete process.env.TRUST_FORWARDED_IPS
})
})
+162
View File
@@ -0,0 +1,162 @@
import { internal } from '../_generated/api'
import type { ActionCtx } from '../_generated/server'
import { corsHeaders, mergeHeaders } from './httpHeaders'
import { hashToken } from './tokens'
const RATE_LIMIT_WINDOW_MS = 60_000
export const RATE_LIMITS = {
read: { ip: 120, key: 600 },
write: { ip: 30, key: 120 },
download: { ip: 20, key: 120 },
} as const
type RateLimitResult = {
allowed: boolean
remaining: number
limit: number
resetAt: number
}
export async function applyRateLimit(
ctx: ActionCtx,
request: Request,
kind: keyof typeof RATE_LIMITS,
): Promise<{ ok: true; headers: HeadersInit } | { ok: false; response: Response }> {
const ip = getClientIp(request) ?? 'unknown'
const ipResult = await checkRateLimit(ctx, `ip:${ip}`, RATE_LIMITS[kind].ip)
const token = parseBearerToken(request)
const keyResult = token
? await checkRateLimit(ctx, `key:${await hashToken(token)}`, RATE_LIMITS[kind].key)
: null
const chosen = pickMostRestrictive(ipResult, keyResult)
const headers = rateHeaders(chosen)
if (!ipResult.allowed || (keyResult && !keyResult.allowed)) {
return {
ok: false,
response: new Response('Rate limit exceeded', {
status: 429,
headers: mergeHeaders(
{
'Content-Type': 'text/plain; charset=utf-8',
'Cache-Control': 'no-store',
},
headers,
corsHeaders(),
),
}),
}
}
return { ok: true, headers }
}
export function getClientIp(request: Request) {
const cfHeader = request.headers.get('cf-connecting-ip')
if (cfHeader) return splitFirstIp(cfHeader)
if (!shouldTrustForwardedIps()) return null
const forwarded =
request.headers.get('x-real-ip') ??
request.headers.get('x-forwarded-for') ??
request.headers.get('fly-client-ip')
return splitFirstIp(forwarded)
}
async function checkRateLimit(
ctx: ActionCtx,
key: string,
limit: number,
): Promise<RateLimitResult> {
// Step 1: Read-only check to avoid write conflicts on denied requests.
const status = (await ctx.runQuery(internal.rateLimits.getRateLimitStatusInternal, {
key,
limit,
windowMs: RATE_LIMIT_WINDOW_MS,
})) as RateLimitResult
if (!status.allowed) {
return status
}
// Step 2: Consume with a mutation only when still allowed.
let result: { allowed: boolean; remaining: number }
try {
result = (await ctx.runMutation(internal.rateLimits.consumeRateLimitInternal, {
key,
limit,
windowMs: RATE_LIMIT_WINDOW_MS,
})) as { allowed: boolean; remaining: number }
} catch (error) {
if (isRateLimitWriteConflict(error)) {
return {
allowed: false,
remaining: 0,
limit: status.limit,
resetAt: status.resetAt,
}
}
throw error
}
return {
allowed: result.allowed,
remaining: result.remaining,
limit: status.limit,
resetAt: status.resetAt,
}
}
function pickMostRestrictive(primary: RateLimitResult, secondary: RateLimitResult | null) {
if (!secondary) return primary
if (!primary.allowed) return primary
if (!secondary.allowed) return secondary
return secondary.remaining < primary.remaining ? secondary : primary
}
function rateHeaders(result: RateLimitResult): HeadersInit {
const resetSeconds = Math.ceil(result.resetAt / 1000)
return {
'X-RateLimit-Limit': String(result.limit),
'X-RateLimit-Remaining': String(result.remaining),
'X-RateLimit-Reset': String(resetSeconds),
...(result.allowed ? {} : { 'Retry-After': String(resetSeconds) }),
}
}
export function parseBearerToken(request: Request) {
const header = request.headers.get('authorization') ?? request.headers.get('Authorization')
if (!header) return null
const trimmed = header.trim()
if (!trimmed.toLowerCase().startsWith('bearer ')) return null
const token = trimmed.slice(7).trim()
return token || null
}
function splitFirstIp(header: string | null) {
if (!header) return null
if (header.includes(',')) return header.split(',')[0]?.trim() || null
const trimmed = header.trim()
return trimmed || null
}
function shouldTrustForwardedIps() {
const value = String(process.env.TRUST_FORWARDED_IPS ?? '')
.trim()
.toLowerCase()
if (!value) return true
if (value === '1' || value === 'true' || value === 'yes') return true
if (value === '0' || value === 'false' || value === 'no') return false
return false
}
function isRateLimitWriteConflict(error: unknown) {
if (!(error instanceof Error)) return false
return (
error.message.includes('rateLimits') &&
error.message.includes('changed while this mutation was being run')
)
}
+68
View File
@@ -0,0 +1,68 @@
import { describe, expect, it } from 'vitest'
import type { Doc } from '../_generated/dataModel'
import { toPublicSkill } from './public'
function makeSkill(overrides: Partial<Doc<'skills'>> = {}): Doc<'skills'> {
return {
_id: 'skills:1' as Doc<'skills'>['_id'],
_creationTime: 1,
slug: 'demo',
displayName: 'Demo',
summary: 'Demo summary',
ownerUserId: 'users:1' as Doc<'skills'>['ownerUserId'],
canonicalSkillId: undefined,
forkOf: undefined,
latestVersionId: undefined,
tags: {},
badges: {},
moderationStatus: 'active',
moderationReason: undefined,
moderationNotes: undefined,
moderationFlags: undefined,
hiddenAt: undefined,
lastReviewedAt: undefined,
softDeletedAt: undefined,
reportCount: 0,
lastReportedAt: undefined,
quality: undefined,
statsDownloads: 0,
statsStars: 0,
statsInstallsCurrent: 0,
statsInstallsAllTime: 0,
stats: {
downloads: 0,
installsCurrent: 0,
installsAllTime: 0,
stars: 0,
versions: 0,
comments: 0,
},
createdAt: 1,
updatedAt: 1,
...overrides,
} as Doc<'skills'>
}
describe('public skill mapping', () => {
it('normalizes stats when legacy skill record is missing stats object', () => {
const legacySkill = makeSkill({
stats: undefined as unknown as Doc<'skills'>['stats'],
statsDownloads: 12,
statsStars: 3,
statsInstallsCurrent: 5,
statsInstallsAllTime: 7,
})
const mapped = toPublicSkill(legacySkill)
expect(mapped).not.toBeNull()
expect(mapped?.stats).toEqual({
downloads: 12,
stars: 3,
installsCurrent: 5,
installsAllTime: 7,
versions: 0,
comments: 0,
})
})
})
+19 -2
View File
@@ -39,7 +39,7 @@ export type PublicSoul = Pick<
>
export function toPublicUser(user: Doc<'users'> | null | undefined): PublicUser | null {
if (!user || user.deletedAt) return null
if (!user || user.deletedAt || user.deactivatedAt) return null
return {
_id: user._id,
_creationTime: user._creationTime,
@@ -55,6 +55,23 @@ export function toPublicSkill(skill: Doc<'skills'> | null | undefined): PublicSk
if (!skill || skill.softDeletedAt) return null
if (skill.moderationStatus && skill.moderationStatus !== 'active') return null
if (skill.moderationFlags?.includes('blocked.malware')) return null
const stats = {
downloads:
typeof skill.statsDownloads === 'number'
? skill.statsDownloads
: (skill.stats?.downloads ?? 0),
stars: typeof skill.statsStars === 'number' ? skill.statsStars : (skill.stats?.stars ?? 0),
installsCurrent:
typeof skill.statsInstallsCurrent === 'number'
? skill.statsInstallsCurrent
: (skill.stats?.installsCurrent ?? 0),
installsAllTime:
typeof skill.statsInstallsAllTime === 'number'
? skill.statsInstallsAllTime
: (skill.stats?.installsAllTime ?? 0),
versions: skill.stats?.versions ?? 0,
comments: skill.stats?.comments ?? 0,
}
return {
_id: skill._id,
_creationTime: skill._creationTime,
@@ -67,7 +84,7 @@ export function toPublicSkill(skill: Doc<'skills'> | null | undefined): PublicSk
latestVersionId: skill.latestVersionId,
tags: skill.tags,
badges: skill.badges,
stats: skill.stats,
stats,
createdAt: skill.createdAt,
updatedAt: skill.updatedAt,
}
+77
View File
@@ -25,4 +25,81 @@ describe('skillPublish', () => {
}),
)
})
it('rejects thin templated skill content for low-trust publishers', () => {
const signals = __test.computeQualitySignals({
readmeText: `---
description: Expert guidance for sushi-rolls.
---
# Sushi Rolls
## Getting Started
- Step-by-step tutorials
- Tips and techniques
- Project ideas
`,
summary: 'Expert guidance for sushi-rolls.',
})
const quality = __test.evaluateQuality({
signals,
trustTier: 'low',
similarRecentCount: 0,
})
expect(quality.decision).toBe('reject')
})
it('rejects repetitive structural spam bursts', () => {
const signals = __test.computeQualitySignals({
readmeText: `# Kitchen Workflow
## Mise en place
- Gather ingredients and check freshness for each item before prep starts.
- Prepare utensils and containers so every step can be executed smoothly.
- Keep notes on ingredient substitutions and expected flavor impact.
## Rolling flow
- Build rolls in small batches, taste often, and adjust seasoning carefully.
- Track timing, texture, and shape consistency to avoid rushed mistakes.
- Capture what worked and what failed so the next run is more reliable.
## Service checklist
- Plate with clear labels, cleaning steps, and handoff instructions.
- Include safety notes, storage guidance, and quality checkpoints.
- Document outcomes and follow-up improvements for the next iteration.
`,
summary: 'Detailed sushi workflow notes.',
})
const quality = __test.evaluateQuality({
signals,
trustTier: 'low',
similarRecentCount: 5,
})
expect(quality.decision).toBe('reject')
expect(quality.reason).toContain('template spam')
})
it('does not undercount non-latin skill docs', () => {
const signals = __test.computeQualitySignals({
readmeText: `# 飞书图片助手
## 核心能力
- 上传本地图片到飞书并自动返回 image_key,避免重复上传浪费配额。
- 支持群聊与私聊,自动识别目标类型并校验参数,减少调用错误。
- 提供重试与错误分类,方便排查网络问题、权限问题与资源限制。
## 使用说明
先配置应用凭证,然后传入目标会话与文件路径。技能会先检查缓存,再执行上传,并在发送阶段附带日志说明,便于团队追踪。
如果出现失败,输出会包含建议动作,例如补齐权限、检查文件大小、确认机器人是否在群内,以及如何重放请求。
还会记录每一步耗时、返回码与上下文摘要,方便后续做性能分析、告警聚合和批量回放,避免同类问题反复出现。
`,
summary: '上传并发送图片到飞书,支持缓存、重试和错误诊断。',
})
const quality = __test.evaluateQuality({
signals,
trustTier: 'low',
similarRecentCount: 0,
})
expect(signals.bodyWords).toBeGreaterThanOrEqual(45)
expect(quality.decision).toBe('pass')
})
})
+130 -17
View File
@@ -8,6 +8,14 @@ import { generateChangelogForPublish } from './changelog'
import { generateEmbedding } from './embeddings'
import { requireGitHubAccountAge } from './githubAccount'
import type { PublicUser } from './public'
import {
computeQualitySignals,
evaluateQuality,
getTrustTier,
type QualityAssessment,
toStructuralFingerprint,
} from './skillQuality'
import { generateSkillSummary } from './skillSummary'
import {
buildEmbeddingText,
getFrontmatterMetadata,
@@ -21,6 +29,8 @@ import type { WebhookSkillPayload } from './webhooks'
const MAX_TOTAL_BYTES = 50 * 1024 * 1024
const MAX_FILES_FOR_EMBEDDING = 40
const QUALITY_WINDOW_MS = 24 * 60 * 60 * 1000
const QUALITY_ACTIVITY_LIMIT = 60
export type PublishResult = {
skillId: Id<'skills'>
@@ -70,6 +80,10 @@ export async function publishVersionForUser(
}
await requireGitHubAccountAge(ctx, userId)
const existingSkill = (await ctx.runQuery(internal.skills.getSkillBySlugInternal, {
slug,
})) as Doc<'skills'> | null
const isNewSkill = !existingSkill
const suppliedChangelog = args.changelog.trim()
const changelogSource = suppliedChangelog ? ('user' as const) : ('auto' as const)
@@ -102,7 +116,75 @@ export async function publishVersionForUser(
const readmeText = await fetchText(ctx, readmeFile.storageId)
const frontmatter = parseFrontmatter(readmeText)
const clawdis = parseClawdisMetadata(frontmatter)
const metadata = mergeSourceIntoMetadata(getFrontmatterMetadata(frontmatter), args.source)
const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
userId,
})) as Doc<'users'> | null
const ownerCreatedAt = owner?.createdAt ?? owner?._creationTime ?? Date.now()
const now = Date.now()
const frontmatterMetadata = getFrontmatterMetadata(frontmatter)
const summaryFromFrontmatter =
frontmatterMetadata &&
typeof frontmatterMetadata === 'object' &&
!Array.isArray(frontmatterMetadata) &&
typeof (frontmatterMetadata as Record<string, unknown>).description === 'string'
? ((frontmatterMetadata as Record<string, unknown>).description as string)
: undefined
const summary = await generateSkillSummary({
slug,
displayName,
readmeText,
currentSummary: summaryFromFrontmatter ?? existingSkill?.summary ?? undefined,
})
let qualityAssessment: QualityAssessment | null = null
if (isNewSkill) {
const ownerActivity = (await ctx.runQuery(internal.skills.getOwnerSkillActivityInternal, {
ownerUserId: userId,
limit: QUALITY_ACTIVITY_LIMIT,
})) as Array<{
slug: string
summary?: string
createdAt: number
latestVersionId?: Id<'skillVersions'>
}>
const trustTier = getTrustTier(now - ownerCreatedAt, ownerActivity.length)
const qualitySignals = computeQualitySignals({
readmeText,
summary,
})
const recentCandidates = ownerActivity.filter(
(entry) =>
entry.slug !== slug && entry.createdAt >= now - QUALITY_WINDOW_MS && entry.latestVersionId,
)
let similarRecentCount = 0
for (const entry of recentCandidates) {
const version = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
versionId: entry.latestVersionId as Id<'skillVersions'>,
})) as Doc<'skillVersions'> | null
if (!version) continue
const candidateReadmeFile = version.files.find((file) => {
const lower = file.path.toLowerCase()
return lower === 'skill.md' || lower === 'skills.md'
})
if (!candidateReadmeFile) continue
const candidateText = await fetchText(ctx, candidateReadmeFile.storageId)
if (toStructuralFingerprint(candidateText) === qualitySignals.structuralFingerprint) {
similarRecentCount += 1
}
}
qualityAssessment = evaluateQuality({
signals: qualitySignals,
trustTier,
similarRecentCount,
})
if (qualityAssessment.decision === 'reject') {
throw new ConvexError(qualityAssessment.reason)
}
}
const metadata = mergeSourceIntoMetadata(frontmatterMetadata, args.source, qualityAssessment)
const otherFiles = [] as Array<{ path: string; content: string }>
for (const file of safeFiles) {
@@ -167,7 +249,18 @@ export async function publishVersionForUser(
metadata,
clawdis,
},
summary,
embedding,
qualityAssessment: qualityAssessment
? {
decision: qualityAssessment.decision,
score: qualityAssessment.score,
reason: qualityAssessment.reason,
trustTier: qualityAssessment.trustTier,
similarRecentCount: qualityAssessment.similarRecentCount,
signals: qualityAssessment.signals,
}
: undefined,
})) as PublishResult
await ctx.scheduler.runAfter(0, internal.vt.scanWithVirusTotal, {
@@ -178,9 +271,6 @@ export async function publishVersionForUser(
versionId: publishResult.versionId,
})
const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
userId,
})) as Doc<'users'> | null
const ownerHandle = owner?.handle ?? owner?.displayName ?? owner?.name ?? 'unknown'
void ctx.scheduler
@@ -205,25 +295,48 @@ export async function publishVersionForUser(
return publishResult
}
function mergeSourceIntoMetadata(metadata: unknown, source: PublishVersionArgs['source']) {
if (!source) return metadata === undefined ? undefined : metadata
const sourceValue = {
kind: source.kind,
url: source.url,
repo: source.repo,
ref: source.ref,
commit: source.commit,
path: source.path,
importedAt: source.importedAt,
function mergeSourceIntoMetadata(
metadata: unknown,
source: PublishVersionArgs['source'],
qualityAssessment: QualityAssessment | null = null,
) {
const base =
metadata && typeof metadata === 'object' && !Array.isArray(metadata)
? { ...(metadata as Record<string, unknown>) }
: {}
if (source) {
base.source = {
kind: source.kind,
url: source.url,
repo: source.repo,
ref: source.ref,
commit: source.commit,
path: source.path,
importedAt: source.importedAt,
}
}
if (!metadata) return { source: sourceValue }
if (typeof metadata !== 'object' || Array.isArray(metadata)) return { source: sourceValue }
return { ...(metadata as Record<string, unknown>), source: sourceValue }
if (qualityAssessment) {
base._clawhubQuality = {
score: qualityAssessment.score,
decision: qualityAssessment.decision,
trustTier: qualityAssessment.trustTier,
similarRecentCount: qualityAssessment.similarRecentCount,
signals: qualityAssessment.signals,
reason: qualityAssessment.reason,
evaluatedAt: Date.now(),
}
}
return Object.keys(base).length ? base : undefined
}
export const __test = {
mergeSourceIntoMetadata,
computeQualitySignals,
evaluateQuality,
toStructuralFingerprint,
}
export async function queueHighlightedWebhook(ctx: MutationCtx, skillId: Id<'skills'>) {
+234
View File
@@ -0,0 +1,234 @@
const TRUST_TIER_ACCOUNT_AGE_LOW_MS = 30 * 24 * 60 * 60 * 1000
const TRUST_TIER_ACCOUNT_AGE_MEDIUM_MS = 90 * 24 * 60 * 60 * 1000
const TRUST_TIER_SKILLS_LOW = 10
const TRUST_TIER_SKILLS_MEDIUM = 50
const TEMPLATE_MARKERS = [
'expert guidance for',
'practical skill guidance',
'step-by-step tutorials',
'tips and techniques',
'project ideas',
'resource recommendations',
'help with this skill',
'learning guidance',
] as const
export type TrustTier = 'low' | 'medium' | 'trusted'
export type QualitySignals = {
bodyChars: number
bodyWords: number
uniqueWordRatio: number
headingCount: number
bulletCount: number
templateMarkerHits: number
genericSummary: boolean
cjkChars: number
structuralFingerprint: string
}
export type QualityAssessment = {
score: number
decision: 'pass' | 'quarantine' | 'reject'
reason: string
trustTier: TrustTier
similarRecentCount: number
signals: Omit<QualitySignals, 'structuralFingerprint'>
}
function stripFrontmatter(raw: string) {
return raw.replace(/^---\s*\n[\s\S]*?\n---\s*\n?/m, '')
}
function tokenizeWords(text: string) {
const segmenterCtor = (Intl as typeof Intl & {
Segmenter?: new (
locale?: string | string[],
options?: { granularity?: 'grapheme' | 'word' | 'sentence' },
) => {
segment: (
input: string,
) => Iterable<{ segment: string; isWordLike?: boolean }>
}
}).Segmenter
if (segmenterCtor) {
const segmenter = new segmenterCtor(undefined, { granularity: 'word' })
const tokens: string[] = []
for (const entry of segmenter.segment(text)) {
if (!entry.isWordLike) continue
const token = entry.segment.trim().toLowerCase()
if (!token) continue
tokens.push(token)
}
if (tokens.length > 0) return tokens
}
return (text.toLowerCase().match(/[a-z0-9][a-z0-9'-]*/g) ?? []).filter((word) => word.length > 1)
}
function wordBucket(text: string) {
const words = tokenizeWords(text).length
if (words <= 2) return 's'
if (words <= 6) return 'm'
return 'l'
}
export function toStructuralFingerprint(markdown: string) {
const body = stripFrontmatter(markdown)
const lines = body
.split('\n')
.map((line) => line.trim())
.filter(Boolean)
.slice(0, 80)
return lines
.map((line) => {
if (line.startsWith('### ')) return `h3:${wordBucket(line.slice(4))}`
if (line.startsWith('## ')) return `h2:${wordBucket(line.slice(3))}`
if (line.startsWith('# ')) return `h1:${wordBucket(line.slice(2))}`
if (/^[-*]\s+/.test(line)) return `b:${wordBucket(line.replace(/^[-*]\s+/, ''))}`
if (/^\d+\.\s+/.test(line)) return `n:${wordBucket(line.replace(/^\d+\.\s+/, ''))}`
return `p:${wordBucket(line)}`
})
.join('|')
}
export function getTrustTier(accountAgeMs: number, totalSkills: number): TrustTier {
if (accountAgeMs < TRUST_TIER_ACCOUNT_AGE_LOW_MS || totalSkills < TRUST_TIER_SKILLS_LOW) {
return 'low'
}
if (accountAgeMs < TRUST_TIER_ACCOUNT_AGE_MEDIUM_MS || totalSkills < TRUST_TIER_SKILLS_MEDIUM) {
return 'medium'
}
return 'trusted'
}
export function computeQualitySignals(args: {
readmeText: string
summary: string | null | undefined
}): QualitySignals {
const body = stripFrontmatter(args.readmeText)
const bodyChars = body.replace(/\s+/g, '').length
const words = tokenizeWords(body)
const uniqueWordRatio = words.length ? new Set(words).size / words.length : 0
const lines = body.split('\n')
const headingCount = lines.filter((line) => /^#{1,3}\s+/.test(line.trim())).length
const bulletCount = lines.filter((line) => /^[-*]\s+/.test(line.trim())).length
const bodyLower = body.toLowerCase()
const templateMarkerHits = TEMPLATE_MARKERS.filter((marker) => bodyLower.includes(marker)).length
const summary = (args.summary ?? '').trim().toLowerCase()
const genericSummary = /^expert guidance for [a-z0-9-]+\.?$/.test(summary)
const cjkChars = (body.match(/[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}]/gu) ?? []).length
return {
bodyChars,
bodyWords: words.length,
uniqueWordRatio,
headingCount,
bulletCount,
templateMarkerHits,
genericSummary,
cjkChars,
structuralFingerprint: toStructuralFingerprint(args.readmeText),
}
}
function scoreQuality(signals: QualitySignals) {
let score = 100
if (signals.bodyChars < 250) score -= 28
if (signals.bodyWords < 80) score -= 24
if (signals.uniqueWordRatio < 0.45) score -= 14
if (signals.headingCount < 2) score -= 10
if (signals.bulletCount < 3) score -= 8
score -= Math.min(28, signals.templateMarkerHits * 9)
if (signals.genericSummary) score -= 20
return Math.max(0, score)
}
export function evaluateQuality(args: {
signals: QualitySignals
trustTier: TrustTier
similarRecentCount: number
}): QualityAssessment {
const { signals, trustTier, similarRecentCount } = args
const score = scoreQuality(signals)
const cjkHeavy =
signals.cjkChars >= 40 || (signals.bodyChars > 0 && signals.cjkChars / signals.bodyChars >= 0.15)
let rejectWordsThreshold = trustTier === 'low' ? 45 : trustTier === 'medium' ? 35 : 28
let rejectCharsThreshold = trustTier === 'low' ? 260 : trustTier === 'medium' ? 180 : 140
if (cjkHeavy) {
rejectWordsThreshold = Math.max(24, rejectWordsThreshold - 16)
rejectCharsThreshold = Math.max(140, rejectCharsThreshold - 120)
}
const quarantineScoreThreshold = trustTier === 'low' ? 72 : trustTier === 'medium' ? 60 : 50
const similarityRejectThreshold = trustTier === 'low' ? 5 : trustTier === 'medium' ? 8 : 12
const hardReject =
signals.bodyWords < rejectWordsThreshold ||
signals.bodyChars < rejectCharsThreshold ||
(signals.templateMarkerHits >= 3 && signals.bodyWords < 120) ||
similarRecentCount >= similarityRejectThreshold
if (hardReject) {
const reason =
similarRecentCount >= similarityRejectThreshold
? 'Skill appears to be repeated template spam from this account.'
: 'Skill content is too thin or templated. Add meaningful, specific documentation.'
return {
score,
decision: 'reject',
reason,
trustTier,
similarRecentCount,
signals: {
bodyChars: signals.bodyChars,
bodyWords: signals.bodyWords,
uniqueWordRatio: signals.uniqueWordRatio,
headingCount: signals.headingCount,
bulletCount: signals.bulletCount,
templateMarkerHits: signals.templateMarkerHits,
genericSummary: signals.genericSummary,
cjkChars: signals.cjkChars,
},
}
}
if (score < quarantineScoreThreshold) {
return {
score,
decision: 'quarantine',
reason: 'Skill quality is low and requires moderation review before being listed.',
trustTier,
similarRecentCount,
signals: {
bodyChars: signals.bodyChars,
bodyWords: signals.bodyWords,
uniqueWordRatio: signals.uniqueWordRatio,
headingCount: signals.headingCount,
bulletCount: signals.bulletCount,
templateMarkerHits: signals.templateMarkerHits,
genericSummary: signals.genericSummary,
cjkChars: signals.cjkChars,
},
}
}
return {
score,
decision: 'pass',
reason: 'Quality checks passed.',
trustTier,
similarRecentCount,
signals: {
bodyChars: signals.bodyChars,
bodyWords: signals.bodyWords,
uniqueWordRatio: signals.uniqueWordRatio,
headingCount: signals.headingCount,
bulletCount: signals.bulletCount,
templateMarkerHits: signals.templateMarkerHits,
genericSummary: signals.genericSummary,
cjkChars: signals.cjkChars,
},
}
}
+31
View File
@@ -0,0 +1,31 @@
import { describe, expect, it } from 'vitest'
import { isSkillSuspicious } from './skillSafety'
describe('isSkillSuspicious', () => {
it('returns true when suspicious flag is present', () => {
expect(
isSkillSuspicious({
moderationFlags: ['flagged.suspicious'],
moderationReason: undefined,
}),
).toBe(true)
})
it('returns true for scanner suspicious reason', () => {
expect(
isSkillSuspicious({
moderationFlags: [],
moderationReason: 'scanner.vt.suspicious',
}),
).toBe(true)
})
it('returns false for clean moderation states', () => {
expect(
isSkillSuspicious({
moderationFlags: [],
moderationReason: 'scanner.vt.clean',
}),
).toBe(false)
})
})
+13
View File
@@ -0,0 +1,13 @@
import type { Doc } from '../_generated/dataModel'
function isScannerSuspiciousReason(reason: string | undefined) {
if (!reason) return false
return reason.startsWith('scanner.') && reason.endsWith('.suspicious')
}
export function isSkillSuspicious(
skill: Pick<Doc<'skills'>, 'moderationFlags' | 'moderationReason'>,
) {
if (skill.moderationFlags?.includes('flagged.suspicious')) return true
return isScannerSuspiciousReason(skill.moderationReason)
}
+82
View File
@@ -0,0 +1,82 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { __test, generateSkillSummary } from './skillSummary'
const originalFetch = globalThis.fetch
afterEach(() => {
vi.restoreAllMocks()
vi.unstubAllEnvs()
globalThis.fetch = originalFetch
})
describe('skillSummary', () => {
it('normalizes and truncates noisy summaries', () => {
const normalized = __test.normalizeSummary(`" hello\n\nworld "`)
expect(normalized).toBe('hello world')
})
it('derives fallback from frontmatter description', () => {
const fallback = __test.deriveSummaryFallback(`---\ndescription: Crisp summary.\n---\n# Title`)
expect(fallback).toBe('Crisp summary.')
})
it('derives fallback from first meaningful body line', () => {
const fallback = __test.deriveSummaryFallback(
`---\ntitle: Demo\n---\n# Skill Title\n\n- Ship fast`,
)
expect(fallback).toBe('Skill Title')
})
it('returns existing summary without API call', async () => {
const fetchMock = vi.fn()
globalThis.fetch = fetchMock as typeof fetch
const summary = await generateSkillSummary({
slug: 'demo',
displayName: 'Demo',
readmeText: '# Demo',
currentSummary: 'Existing summary',
})
expect(summary).toBe('Existing summary')
expect(fetchMock).not.toHaveBeenCalled()
})
it('uses identity fallback for empty content without API call', async () => {
vi.stubEnv('OPENAI_API_KEY', 'test-key')
const fetchMock = vi.fn()
globalThis.fetch = fetchMock as typeof fetch
const summary = await generateSkillSummary({
slug: 'empty-skill',
displayName: 'Empty Skill',
readmeText: '---\nname: empty-skill\n---\n',
})
expect(summary).toBe('Automation skill for Empty Skill.')
expect(fetchMock).not.toHaveBeenCalled()
})
it('uses OpenAI when key is set and summary missing', async () => {
vi.stubEnv('OPENAI_API_KEY', 'test-key')
globalThis.fetch = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({
output: [
{
type: 'message',
content: [{ type: 'output_text', text: 'AI summary output.' }],
},
],
}),
}) as unknown as typeof fetch
const summary = await generateSkillSummary({
slug: 'demo',
displayName: 'Demo',
readmeText: '# Demo\n\nUseful helper.',
})
expect(summary).toBe('AI summary output.')
})
})
+133
View File
@@ -0,0 +1,133 @@
import { getFrontmatterValue, parseFrontmatter } from './skills'
const SKILL_SUMMARY_MODEL = process.env.OPENAI_SKILL_SUMMARY_MODEL ?? 'gpt-4.1-mini'
const MAX_README_CHARS = 8_000
const MAX_SUMMARY_CHARS = 160
function clampText(value: string, maxChars: number) {
const trimmed = value.trim()
if (trimmed.length <= maxChars) return trimmed
return `${trimmed.slice(0, maxChars).trimEnd()}\n...`
}
function normalizeSummary(value: string | null | undefined) {
if (!value) return undefined
const compact = value
.replace(/\r\n/g, '\n')
.replace(/\r/g, '\n')
.split('\n')
.map((line) => line.trim())
.filter(Boolean)
.join(' ')
.replace(/\s+/g, ' ')
.replace(/^["'`]+|["'`]+$/g, '')
.trim()
if (!compact) return undefined
if (compact.length <= MAX_SUMMARY_CHARS) return compact
return `${compact.slice(0, MAX_SUMMARY_CHARS - 3).trimEnd()}...`
}
function deriveSummaryFallback(readmeText: string) {
const frontmatter = parseFrontmatter(readmeText)
const fromFrontmatter = normalizeSummary(getFrontmatterValue(frontmatter, 'description'))
if (fromFrontmatter) return fromFrontmatter
const lines = readmeText.split(/\r?\n/)
let inFrontmatter = false
for (const raw of lines) {
const trimmed = raw.trim()
if (!trimmed) continue
if (!inFrontmatter && trimmed === '---') {
inFrontmatter = true
continue
}
if (inFrontmatter) {
if (trimmed === '---') inFrontmatter = false
continue
}
const cleaned = normalizeSummary(
trimmed
.replace(/^#+\s*/, '')
.replace(/^[-*]\s+/, '')
.replace(/^\d+\.\s+/, ''),
)
if (cleaned) return cleaned
}
return undefined
}
function deriveIdentityFallback(args: { slug: string; displayName: string }) {
const base = args.displayName.trim() || args.slug.trim()
return normalizeSummary(`Automation skill for ${base}.`)
}
function extractResponseText(payload: unknown) {
if (!payload || typeof payload !== 'object') return null
const output = (payload as { output?: unknown }).output
if (!Array.isArray(output)) return null
const chunks: string[] = []
for (const item of output) {
if (!item || typeof item !== 'object') continue
if ((item as { type?: unknown }).type !== 'message') continue
const content = (item as { content?: unknown }).content
if (!Array.isArray(content)) continue
for (const part of content) {
if (!part || typeof part !== 'object') continue
if ((part as { type?: unknown }).type !== 'output_text') continue
const text = (part as { text?: unknown }).text
if (typeof text === 'string' && text.trim()) chunks.push(text)
}
}
const joined = chunks.join('\n').trim()
return joined || null
}
export async function generateSkillSummary(args: {
slug: string
displayName: string
readmeText: string
currentSummary?: string
}) {
const existing = normalizeSummary(args.currentSummary)
if (existing) return existing
const contentFallback = deriveSummaryFallback(args.readmeText)
const fallback = contentFallback ?? deriveIdentityFallback(args)
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) return fallback
if (!contentFallback) return fallback
const input = [
`Skill slug: ${args.slug}`,
`Display name: ${args.displayName}`,
`SKILL.md:\n${clampText(args.readmeText, MAX_README_CHARS)}`,
].join('\n\n')
try {
const response = await fetch('https://api.openai.com/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: SKILL_SUMMARY_MODEL,
instructions:
'Write a concise public skill description. Return plain text only, one sentence, max 160 characters. No markdown. No quotes. No hype. Be specific and accurate to SKILL.md.',
input,
max_output_tokens: 90,
}),
})
if (!response.ok) return fallback
const payload = (await response.json()) as unknown
return normalizeSummary(extractResponseText(payload)) ?? fallback
} catch {
return fallback
}
}
export const __test = {
clampText,
deriveSummaryFallback,
normalizeSummary,
}
+2 -17
View File
@@ -251,23 +251,8 @@ export const evaluateWithLlm = internalAction({
`[llmEval] Evaluated ${skill.slug}@${version.version}: ${result.verdict} (${result.confidence} confidence)`,
)
// 10. Update moderation flags — re-read version to get the sha256hash
// that VT may have stored while we were evaluating (both run concurrently).
const freshVersion = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
versionId: args.versionId,
})) as Doc<'skillVersions'> | null
const sha256hash = freshVersion?.sha256hash ?? version.sha256hash
if (sha256hash) {
const status = verdictToStatus(result.verdict)
if (status === 'malicious' || status === 'suspicious' || status === 'clean') {
await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
sha256hash,
scanner: 'llm',
status,
})
}
}
// Moderation visibility is finalized by VT results.
// LLM eval only stores analysis payload on the version.
},
})
+282 -2
View File
@@ -12,12 +12,34 @@ vi.mock('./_generated/api', () => ({
'applySkillFingerprintBackfillPatchInternal',
),
backfillSkillFingerprintsInternal: Symbol('backfillSkillFingerprintsInternal'),
getEmptySkillCleanupPageInternal: Symbol('getEmptySkillCleanupPageInternal'),
applyEmptySkillCleanupInternal: Symbol('applyEmptySkillCleanupInternal'),
nominateUserForEmptySkillSpamInternal: Symbol('nominateUserForEmptySkillSpamInternal'),
cleanupEmptySkillsInternal: Symbol('cleanupEmptySkillsInternal'),
nominateEmptySkillSpammersInternal: Symbol('nominateEmptySkillSpammersInternal'),
},
skills: {
getVersionByIdInternal: Symbol('skills.getVersionByIdInternal'),
getOwnerSkillActivityInternal: Symbol('skills.getOwnerSkillActivityInternal'),
},
users: {
getByIdInternal: Symbol('users.getByIdInternal'),
},
},
}))
const { backfillSkillFingerprintsInternalHandler, backfillSkillSummariesInternalHandler } =
await import('./maintenance')
vi.mock('./lib/skillSummary', () => ({
generateSkillSummary: vi.fn(),
}))
const {
backfillSkillFingerprintsInternalHandler,
backfillSkillSummariesInternalHandler,
cleanupEmptySkillsInternalHandler,
nominateEmptySkillSpammersInternalHandler,
} = await import('./maintenance')
const { internal } = await import('./_generated/api')
const { generateSkillSummary } = await import('./lib/skillSummary')
function makeBlob(text: string) {
return { text: () => Promise.resolve(text) } as unknown as Blob
@@ -30,6 +52,8 @@ describe('maintenance backfill', () => {
{
kind: 'ok',
skillId: 'skills:1',
skillSlug: 'skill-1',
skillDisplayName: 'Skill 1',
versionId: 'skillVersions:1',
skillSummary: '>',
versionParsed: { frontmatter: { description: '>' } },
@@ -73,6 +97,8 @@ describe('maintenance backfill', () => {
{
kind: 'ok',
skillId: 'skills:1',
skillSlug: 'skill-1',
skillDisplayName: 'Skill 1',
versionId: 'skillVersions:1',
skillSummary: '>',
versionParsed: { frontmatter: { description: '>' } },
@@ -102,6 +128,8 @@ describe('maintenance backfill', () => {
{
kind: 'ok',
skillId: 'skills:1',
skillSlug: 'skill-1',
skillDisplayName: 'Skill 1',
versionId: 'skillVersions:1',
skillSummary: null,
versionParsed: { frontmatter: {} },
@@ -123,6 +151,49 @@ describe('maintenance backfill', () => {
expect(result.stats.missingStorageBlob).toBe(1)
expect(runMutation).not.toHaveBeenCalled()
})
it('fills empty summary via AI when useAi is enabled', async () => {
vi.mocked(generateSkillSummary).mockResolvedValue('AI generated summary.')
const runQuery = vi.fn().mockResolvedValue({
items: [
{
kind: 'ok',
skillId: 'skills:1',
skillSlug: 'ai-skill',
skillDisplayName: 'AI Skill',
versionId: 'skillVersions:1',
skillSummary: null,
versionParsed: { frontmatter: {} },
readmeStorageId: 'storage:1',
},
],
cursor: null,
isDone: true,
})
const runMutation = vi.fn().mockResolvedValue({ ok: true })
const storageGet = vi.fn().mockResolvedValue(makeBlob('# AI Skill\n\nUseful automation.'))
const result = await backfillSkillSummariesInternalHandler(
{ runQuery, runMutation, storage: { get: storageGet } } as never,
{ dryRun: false, batchSize: 10, maxBatches: 1, useAi: true },
)
expect(result.ok).toBe(true)
expect(result.stats.skillsPatched).toBe(1)
expect(result.stats.aiSummariesPatched).toBe(1)
expect(runMutation).toHaveBeenCalledWith(expect.anything(), {
skillId: 'skills:1',
versionId: 'skillVersions:1',
summary: 'AI generated summary.',
parsed: {
frontmatter: {},
metadata: undefined,
clawdis: undefined,
},
})
})
})
describe('maintenance fingerprint backfill', () => {
@@ -268,3 +339,212 @@ describe('maintenance fingerprint backfill', () => {
})
})
})
describe('maintenance empty skill cleanup', () => {
it('dryRun detects empty skills and returns nominations', async () => {
const runQuery = vi.fn().mockImplementation(async (endpoint: unknown) => {
if (endpoint === internal.maintenance.getEmptySkillCleanupPageInternal) {
return {
items: [
{
skillId: 'skills:1',
slug: 'spam-skill',
ownerUserId: 'users:1',
latestVersionId: 'skillVersions:1',
softDeletedAt: undefined,
summary: 'Expert guidance for spam-skill.',
},
],
cursor: null,
isDone: true,
}
}
if (endpoint === internal.skills.getVersionByIdInternal) {
return {
_id: 'skillVersions:1',
files: [{ path: 'SKILL.md', size: 120, storageId: 'storage:1' }],
}
}
if (endpoint === internal.users.getByIdInternal) {
return { _id: 'users:1', handle: 'spammer', _creationTime: Date.now() }
}
if (endpoint === internal.skills.getOwnerSkillActivityInternal) {
return []
}
throw new Error(`Unexpected endpoint: ${String(endpoint)}`)
})
const runMutation = vi.fn()
const storageGet = vi
.fn()
.mockResolvedValue(
makeBlob(`# Demo\n- Step-by-step tutorials\n- Tips and techniques\n- Project ideas`),
)
const result = await cleanupEmptySkillsInternalHandler(
{ runQuery, runMutation, storage: { get: storageGet } } as never,
{ dryRun: true, batchSize: 10, maxBatches: 1, nominationThreshold: 1 },
)
expect(result.ok).toBe(true)
expect(result.isDone).toBe(true)
expect(result.cursor).toBeNull()
expect(result.stats.emptyDetected).toBe(1)
expect(result.stats.skillsDeleted).toBe(0)
expect(result.nominations).toEqual([
{
userId: 'users:1',
handle: 'spammer',
emptySkillCount: 1,
sampleSlugs: ['spam-skill'],
},
])
expect(runMutation).not.toHaveBeenCalled()
})
it('apply mode deletes empty skills', async () => {
const runQuery = vi.fn().mockImplementation(async (endpoint: unknown) => {
if (endpoint === internal.maintenance.getEmptySkillCleanupPageInternal) {
return {
items: [
{
skillId: 'skills:1',
slug: 'spam-a',
ownerUserId: 'users:1',
latestVersionId: 'skillVersions:1',
summary: 'Expert guidance for spam-a.',
},
{
skillId: 'skills:2',
slug: 'spam-b',
ownerUserId: 'users:1',
latestVersionId: 'skillVersions:2',
summary: 'Expert guidance for spam-b.',
},
],
cursor: null,
isDone: true,
}
}
if (endpoint === internal.skills.getVersionByIdInternal) {
return {
files: [{ path: 'SKILL.md', size: 120, storageId: 'storage:1' }],
}
}
if (endpoint === internal.users.getByIdInternal) {
return { _id: 'users:1', handle: 'spammer', _creationTime: Date.now() }
}
if (endpoint === internal.skills.getOwnerSkillActivityInternal) {
return []
}
throw new Error(`Unexpected endpoint: ${String(endpoint)}`)
})
const runMutation = vi.fn().mockImplementation(async (endpoint: unknown) => {
if (endpoint === internal.maintenance.applyEmptySkillCleanupInternal) {
return { deleted: true }
}
throw new Error(`Unexpected mutation endpoint: ${String(endpoint)}`)
})
const storageGet = vi
.fn()
.mockResolvedValue(
makeBlob(`# Demo\n- Step-by-step tutorials\n- Tips and techniques\n- Project ideas`),
)
const result = await cleanupEmptySkillsInternalHandler(
{ runQuery, runMutation, storage: { get: storageGet } } as never,
{ dryRun: false, batchSize: 10, maxBatches: 1, nominationThreshold: 2 },
)
expect(result.ok).toBe(true)
expect(result.isDone).toBe(true)
expect(result.cursor).toBeNull()
expect(result.stats.emptyDetected).toBe(2)
expect(result.stats.skillsDeleted).toBe(2)
expect(result.nominations).toEqual([
{
userId: 'users:1',
handle: 'spammer',
emptySkillCount: 2,
sampleSlugs: ['spam-a', 'spam-b'],
},
])
})
})
describe('maintenance empty skill nominations', () => {
it('creates ban nominations from backfilled empty deletions', async () => {
const runQuery = vi.fn().mockImplementation(async (endpoint: unknown, args: unknown) => {
if (endpoint === internal.maintenance.getEmptySkillCleanupPageInternal) {
const cursor = (args as { cursor?: string | undefined }).cursor
if (!cursor) {
return {
items: [
{
skillId: 'skills:1',
slug: 'spam-a',
ownerUserId: 'users:1',
softDeletedAt: 1,
moderationReason: 'quality.empty.backfill',
},
{
skillId: 'skills:2',
slug: 'spam-b',
ownerUserId: 'users:1',
softDeletedAt: 1,
moderationReason: 'quality.empty.backfill',
},
],
cursor: 'next',
isDone: false,
}
}
return {
items: [
{
skillId: 'skills:3',
slug: 'valid-hidden',
ownerUserId: 'users:2',
softDeletedAt: 1,
moderationReason: 'scanner.vt.suspicious',
},
],
cursor: null,
isDone: true,
}
}
if (endpoint === internal.users.getByIdInternal) {
return { _id: 'users:1', handle: 'spammer' }
}
throw new Error(`Unexpected query endpoint: ${String(endpoint)}`)
})
const runMutation = vi.fn().mockImplementation(async (endpoint: unknown) => {
if (endpoint === internal.maintenance.nominateUserForEmptySkillSpamInternal) {
return { created: true }
}
throw new Error(`Unexpected mutation endpoint: ${String(endpoint)}`)
})
const result = await nominateEmptySkillSpammersInternalHandler(
{ runQuery, runMutation } as never,
{ batchSize: 10, maxBatches: 2, nominationThreshold: 2 },
)
expect(result.ok).toBe(true)
expect(result.isDone).toBe(true)
expect(result.stats.usersFlagged).toBe(1)
expect(result.stats.nominationsCreated).toBe(1)
expect(result.stats.nominationsExisting).toBe(0)
expect(result.nominations).toEqual([
{
userId: 'users:1',
handle: 'spammer',
emptySkillCount: 2,
sampleSlugs: ['spam-a', 'spam-b'],
},
])
})
})
+589 -11
View File
@@ -5,16 +5,26 @@ import type { ActionCtx } from './_generated/server'
import { action, internalAction, internalMutation, internalQuery } from './_generated/server'
import { assertRole, requireUserFromAction } from './lib/access'
import { buildSkillSummaryBackfillPatch, type ParsedSkillData } from './lib/skillBackfill'
import {
computeQualitySignals,
evaluateQuality,
getTrustTier,
type TrustTier,
} from './lib/skillQuality'
import { generateSkillSummary } from './lib/skillSummary'
import { hashSkillFiles } from './lib/skills'
const DEFAULT_BATCH_SIZE = 50
const MAX_BATCH_SIZE = 200
const DEFAULT_MAX_BATCHES = 20
const MAX_MAX_BATCHES = 200
const DEFAULT_EMPTY_SKILL_MAX_README_BYTES = 8000
const DEFAULT_EMPTY_SKILL_NOMINATION_THRESHOLD = 3
type BackfillStats = {
skillsScanned: number
skillsPatched: number
aiSummariesPatched: number
versionsPatched: number
missingLatestVersion: number
missingReadme: number
@@ -25,6 +35,8 @@ type BackfillPageItem =
| {
kind: 'ok'
skillId: Id<'skills'>
skillSlug: string
skillDisplayName: string
versionId: Id<'skillVersions'>
skillSummary: Doc<'skills'>['summary']
versionParsed: Doc<'skillVersions'>['parsed']
@@ -80,6 +92,8 @@ export const getSkillBackfillPageInternal = internalQuery({
items.push({
kind: 'ok',
skillId: skill._id,
skillSlug: skill.slug,
skillDisplayName: skill.displayName,
versionId: version._id,
skillSummary: skill.summary,
versionParsed: version.parsed,
@@ -120,28 +134,37 @@ export type BackfillActionArgs = {
dryRun?: boolean
batchSize?: number
maxBatches?: number
useAi?: boolean
cursor?: string
}
export type BackfillActionResult = { ok: true; stats: BackfillStats }
export type BackfillActionResult = {
ok: true
stats: BackfillStats
isDone: boolean
cursor: string | null
}
export async function backfillSkillSummariesInternalHandler(
ctx: ActionCtx,
args: BackfillActionArgs,
): Promise<BackfillActionResult> {
const dryRun = Boolean(args.dryRun)
const useAi = Boolean(args.useAi)
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
const totals: BackfillStats = {
skillsScanned: 0,
skillsPatched: 0,
aiSummariesPatched: 0,
versionsPatched: 0,
missingLatestVersion: 0,
missingReadme: 0,
missingStorageBlob: 0,
}
let cursor: string | null = null
let cursor: string | null = args.cursor ?? null
let isDone = false
for (let i = 0; i < maxBatches; i++) {
@@ -181,8 +204,24 @@ export async function backfillSkillSummariesInternalHandler(
currentParsed: item.versionParsed as ParsedSkillData,
})
if (!patch.summary && !patch.parsed) continue
if (patch.summary) totals.skillsPatched++
let nextSummary = patch.summary
const missingSummary = !item.skillSummary?.trim()
if (!nextSummary && useAi && missingSummary) {
nextSummary = await generateSkillSummary({
slug: item.skillSlug,
displayName: item.skillDisplayName,
readmeText,
})
}
const shouldPatchSummary =
typeof nextSummary === 'string' && nextSummary.trim() && nextSummary !== item.skillSummary
if (!shouldPatchSummary && !patch.parsed) continue
if (shouldPatchSummary) {
totals.skillsPatched++
if (!patch.summary) totals.aiSummariesPatched++
}
if (patch.parsed) totals.versionsPatched++
if (dryRun) continue
@@ -190,7 +229,7 @@ export async function backfillSkillSummariesInternalHandler(
await ctx.runMutation(internal.maintenance.applySkillBackfillPatchInternal, {
skillId: item.skillId,
versionId: item.versionId,
summary: patch.summary,
summary: shouldPatchSummary ? nextSummary : undefined,
parsed: patch.parsed,
})
}
@@ -198,11 +237,7 @@ export async function backfillSkillSummariesInternalHandler(
if (isDone) break
}
if (!isDone) {
throw new ConvexError('Backfill incomplete (maxBatches reached)')
}
return { ok: true as const, stats: totals }
return { ok: true as const, stats: totals, isDone, cursor }
}
export const backfillSkillSummariesInternal = internalAction({
@@ -210,6 +245,8 @@ export const backfillSkillSummariesInternal = internalAction({
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
useAi: v.optional(v.boolean()),
cursor: v.optional(v.string()),
},
handler: backfillSkillSummariesInternalHandler,
})
@@ -219,6 +256,8 @@ export const backfillSkillSummaries: ReturnType<typeof action> = action({
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
useAi: v.optional(v.boolean()),
cursor: v.optional(v.string()),
},
handler: async (ctx, args): Promise<BackfillActionResult> => {
const { user } = await requireUserFromAction(ctx)
@@ -231,7 +270,7 @@ export const backfillSkillSummaries: ReturnType<typeof action> = action({
})
export const scheduleBackfillSkillSummaries: ReturnType<typeof action> = action({
args: { dryRun: v.optional(v.boolean()) },
args: { dryRun: v.optional(v.boolean()), useAi: v.optional(v.boolean()) },
handler: async (ctx, args) => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
@@ -239,11 +278,43 @@ export const scheduleBackfillSkillSummaries: ReturnType<typeof action> = action(
dryRun: Boolean(args.dryRun),
batchSize: DEFAULT_BATCH_SIZE,
maxBatches: DEFAULT_MAX_BATCHES,
useAi: Boolean(args.useAi),
})
return { ok: true as const }
},
})
export const continueSkillSummaryBackfillJobInternal = internalAction({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
useAi: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<BackfillActionResult> => {
const result = await backfillSkillSummariesInternalHandler(ctx, {
dryRun: false,
cursor: args.cursor,
batchSize: args.batchSize ?? DEFAULT_BATCH_SIZE,
maxBatches: 1,
useAi: Boolean(args.useAi),
})
if (!result.isDone && result.cursor) {
await ctx.scheduler.runAfter(
0,
internal.maintenance.continueSkillSummaryBackfillJobInternal,
{
cursor: result.cursor,
batchSize: args.batchSize ?? DEFAULT_BATCH_SIZE,
useAi: Boolean(args.useAi),
},
)
}
return result
},
})
type FingerprintBackfillStats = {
versionsScanned: number
versionsPatched: number
@@ -833,6 +904,513 @@ export const scheduleBackfillSkillBadgeTable: ReturnType<typeof action> = action
},
})
type EmptySkillCleanupPageItem = {
skillId: Id<'skills'>
slug: string
ownerUserId: Id<'users'>
latestVersionId?: Id<'skillVersions'>
softDeletedAt?: number
moderationReason?: string
summary?: string
}
type EmptySkillCleanupPageResult = {
items: EmptySkillCleanupPageItem[]
cursor: string | null
isDone: boolean
}
type EmptySkillCleanupStats = {
skillsScanned: number
skillsEvaluated: number
emptyDetected: number
skillsDeleted: number
missingLatestVersion: number
missingVersionDoc: number
missingReadme: number
missingStorageBlob: number
skippedLargeReadme: number
}
type EmptySkillCleanupNomination = {
userId: Id<'users'>
handle: string | null
emptySkillCount: number
sampleSlugs: string[]
}
export type EmptySkillCleanupActionArgs = {
cursor?: string
dryRun?: boolean
batchSize?: number
maxBatches?: number
maxReadmeBytes?: number
nominationThreshold?: number
}
export type EmptySkillCleanupActionResult = {
ok: true
cursor: string | null
isDone: boolean
stats: EmptySkillCleanupStats
nominations: EmptySkillCleanupNomination[]
}
export const getEmptySkillCleanupPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args): Promise<EmptySkillCleanupPageResult> => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const { page, isDone, continueCursor } = await ctx.db
.query('skills')
.order('asc')
.paginate({ cursor: args.cursor ?? null, numItems: batchSize })
return {
items: page.map((skill) => ({
skillId: skill._id,
slug: skill.slug,
ownerUserId: skill.ownerUserId,
latestVersionId: skill.latestVersionId,
softDeletedAt: skill.softDeletedAt,
moderationReason: skill.moderationReason,
summary: skill.summary,
})),
cursor: continueCursor,
isDone,
}
},
})
export const applyEmptySkillCleanupInternal = internalMutation({
args: {
skillId: v.id('skills'),
reason: v.string(),
quality: v.object({
score: v.number(),
trustTier: v.union(v.literal('low'), v.literal('medium'), v.literal('trusted')),
signals: v.object({
bodyChars: v.number(),
bodyWords: v.number(),
uniqueWordRatio: v.number(),
headingCount: v.number(),
bulletCount: v.number(),
templateMarkerHits: v.number(),
genericSummary: v.boolean(),
cjkChars: v.optional(v.number()),
}),
}),
},
handler: async (ctx, args) => {
const skill = await ctx.db.get(args.skillId)
if (!skill) return { deleted: false as const, reason: 'missing_skill' as const }
if (skill.softDeletedAt) return { deleted: false as const, reason: 'already_deleted' as const }
const now = Date.now()
await ctx.db.patch(skill._id, {
softDeletedAt: now,
moderationStatus: 'hidden',
moderationReason: 'quality.empty.backfill',
moderationNotes: args.reason,
quality: {
score: args.quality.score,
decision: 'reject',
trustTier: args.quality.trustTier,
similarRecentCount: 0,
reason: args.reason,
signals: args.quality.signals,
evaluatedAt: now,
},
updatedAt: now,
})
await ctx.db.insert('auditLogs', {
actorUserId: skill.ownerUserId,
action: 'skill.delete.empty.backfill',
targetType: 'skill',
targetId: skill._id,
metadata: {
slug: skill.slug,
score: args.quality.score,
trustTier: args.quality.trustTier,
signals: args.quality.signals,
},
createdAt: now,
})
return {
deleted: true as const,
ownerUserId: skill.ownerUserId,
slug: skill.slug,
}
},
})
export const nominateUserForEmptySkillSpamInternal = internalMutation({
args: {
userId: v.id('users'),
emptySkillCount: v.number(),
sampleSlugs: v.array(v.string()),
},
handler: async (ctx, args) => {
const existing = await ctx.db
.query('auditLogs')
.withIndex('by_target', (q) => q.eq('targetType', 'user').eq('targetId', args.userId))
.filter((q) => q.eq(q.field('action'), 'user.ban.nomination.empty-skill-spam'))
.first()
if (existing) return { created: false as const }
const now = Date.now()
await ctx.db.insert('auditLogs', {
actorUserId: args.userId,
action: 'user.ban.nomination.empty-skill-spam',
targetType: 'user',
targetId: args.userId,
metadata: {
emptySkillCount: args.emptySkillCount,
sampleSlugs: args.sampleSlugs.slice(0, 10),
},
createdAt: now,
})
return { created: true as const }
},
})
export async function cleanupEmptySkillsInternalHandler(
ctx: ActionCtx,
args: EmptySkillCleanupActionArgs,
): Promise<EmptySkillCleanupActionResult> {
const dryRun = args.dryRun !== false
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
const maxReadmeBytes = clampInt(
args.maxReadmeBytes ?? DEFAULT_EMPTY_SKILL_MAX_README_BYTES,
256,
65536,
)
const nominationThreshold = clampInt(
args.nominationThreshold ?? DEFAULT_EMPTY_SKILL_NOMINATION_THRESHOLD,
1,
100,
)
const totals: EmptySkillCleanupStats = {
skillsScanned: 0,
skillsEvaluated: 0,
emptyDetected: 0,
skillsDeleted: 0,
missingLatestVersion: 0,
missingVersionDoc: 0,
missingReadme: 0,
missingStorageBlob: 0,
skippedLargeReadme: 0,
}
const ownerTrustCache = new Map<string, { trustTier: TrustTier; handle: string | null }>()
const emptyByOwner = new Map<string, EmptySkillCleanupNomination>()
let cursor: string | null = args.cursor ?? null
let isDone = false
const now = Date.now()
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(internal.maintenance.getEmptySkillCleanupPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as EmptySkillCleanupPageResult
cursor = page.cursor
isDone = page.isDone
for (const item of page.items) {
totals.skillsScanned++
if (item.softDeletedAt) continue
if (!item.latestVersionId) {
totals.missingLatestVersion++
continue
}
const version = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
versionId: item.latestVersionId,
})) as Doc<'skillVersions'> | null
if (!version) {
totals.missingVersionDoc++
continue
}
const readmeFile = version.files.find((file) => {
const lower = file.path.toLowerCase()
return lower === 'skill.md' || lower === 'skills.md'
})
if (!readmeFile) {
totals.missingReadme++
continue
}
if (readmeFile.size > maxReadmeBytes) {
totals.skippedLargeReadme++
continue
}
const blob = await ctx.storage.get(readmeFile.storageId)
if (!blob) {
totals.missingStorageBlob++
continue
}
const readmeText = await blob.text()
totals.skillsEvaluated++
const ownerKey = String(item.ownerUserId)
let ownerTrust = ownerTrustCache.get(ownerKey)
if (!ownerTrust) {
const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
userId: item.ownerUserId,
})) as Doc<'users'> | null
const ownerActivity = (await ctx.runQuery(internal.skills.getOwnerSkillActivityInternal, {
ownerUserId: item.ownerUserId,
limit: 60,
})) as Array<{
slug: string
summary?: string
createdAt: number
latestVersionId?: Id<'skillVersions'>
}>
const ownerCreatedAt = owner?.createdAt ?? owner?._creationTime ?? now
ownerTrust = {
trustTier: getTrustTier(now - ownerCreatedAt, ownerActivity.length),
handle: owner?.handle ?? null,
}
ownerTrustCache.set(ownerKey, ownerTrust)
}
const qualitySignals = computeQualitySignals({
readmeText,
summary: item.summary ?? undefined,
})
const quality = evaluateQuality({
signals: qualitySignals,
trustTier: ownerTrust.trustTier,
similarRecentCount: 0,
})
if (quality.decision !== 'reject') continue
totals.emptyDetected++
const nomination = emptyByOwner.get(ownerKey) ?? {
userId: item.ownerUserId,
handle: ownerTrust.handle,
emptySkillCount: 0,
sampleSlugs: [],
}
nomination.emptySkillCount += 1
if (nomination.sampleSlugs.length < 10 && !nomination.sampleSlugs.includes(item.slug)) {
nomination.sampleSlugs.push(item.slug)
}
emptyByOwner.set(ownerKey, nomination)
if (dryRun) continue
const result = await ctx.runMutation(internal.maintenance.applyEmptySkillCleanupInternal, {
skillId: item.skillId,
reason: quality.reason,
quality: {
score: quality.score,
trustTier: quality.trustTier,
signals: quality.signals,
},
})
if (result.deleted) totals.skillsDeleted++
}
if (isDone) break
}
const nominations = Array.from(emptyByOwner.values())
.filter((entry) => entry.emptySkillCount >= nominationThreshold)
.sort((a, b) => b.emptySkillCount - a.emptySkillCount)
return {
ok: true as const,
cursor,
isDone,
stats: totals,
nominations: nominations.slice(0, 200),
}
}
export const cleanupEmptySkillsInternal = internalAction({
args: {
cursor: v.optional(v.string()),
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
maxReadmeBytes: v.optional(v.number()),
nominationThreshold: v.optional(v.number()),
},
handler: cleanupEmptySkillsInternalHandler,
})
export const cleanupEmptySkills: ReturnType<typeof action> = action({
args: {
cursor: v.optional(v.string()),
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
maxReadmeBytes: v.optional(v.number()),
nominationThreshold: v.optional(v.number()),
},
handler: async (ctx, args): Promise<EmptySkillCleanupActionResult> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
return ctx.runAction(internal.maintenance.cleanupEmptySkillsInternal, args)
},
})
type EmptySkillBanNominationStats = {
skillsScanned: number
usersFlagged: number
nominationsCreated: number
nominationsExisting: number
}
export type EmptySkillBanNominationActionArgs = {
cursor?: string
batchSize?: number
maxBatches?: number
nominationThreshold?: number
}
export type EmptySkillBanNominationActionResult = {
ok: true
cursor: string | null
isDone: boolean
stats: EmptySkillBanNominationStats
nominations: EmptySkillCleanupNomination[]
}
export async function nominateEmptySkillSpammersInternalHandler(
ctx: ActionCtx,
args: EmptySkillBanNominationActionArgs,
): Promise<EmptySkillBanNominationActionResult> {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
const nominationThreshold = clampInt(
args.nominationThreshold ?? DEFAULT_EMPTY_SKILL_NOMINATION_THRESHOLD,
1,
100,
)
const totals: EmptySkillBanNominationStats = {
skillsScanned: 0,
usersFlagged: 0,
nominationsCreated: 0,
nominationsExisting: 0,
}
const ownerHandleCache = new Map<string, string | null>()
const emptyByOwner = new Map<string, EmptySkillCleanupNomination>()
let cursor: string | null = args.cursor ?? null
let isDone = false
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(internal.maintenance.getEmptySkillCleanupPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as EmptySkillCleanupPageResult
cursor = page.cursor
isDone = page.isDone
for (const item of page.items) {
totals.skillsScanned++
if (!item.softDeletedAt) continue
if (item.moderationReason !== 'quality.empty.backfill') continue
const ownerKey = String(item.ownerUserId)
let handle = ownerHandleCache.get(ownerKey)
if (handle === undefined) {
const owner = (await ctx.runQuery(internal.users.getByIdInternal, {
userId: item.ownerUserId,
})) as Doc<'users'> | null
handle = owner?.handle ?? null
ownerHandleCache.set(ownerKey, handle)
}
const nomination = emptyByOwner.get(ownerKey) ?? {
userId: item.ownerUserId,
handle,
emptySkillCount: 0,
sampleSlugs: [],
}
nomination.emptySkillCount += 1
if (nomination.sampleSlugs.length < 10 && !nomination.sampleSlugs.includes(item.slug)) {
nomination.sampleSlugs.push(item.slug)
}
emptyByOwner.set(ownerKey, nomination)
}
if (isDone) break
}
const nominations = Array.from(emptyByOwner.values())
.filter((entry) => entry.emptySkillCount >= nominationThreshold)
.sort((a, b) => b.emptySkillCount - a.emptySkillCount)
totals.usersFlagged = nominations.length
if (isDone) {
for (const nomination of nominations) {
const result = await ctx.runMutation(
internal.maintenance.nominateUserForEmptySkillSpamInternal,
{
userId: nomination.userId,
emptySkillCount: nomination.emptySkillCount,
sampleSlugs: nomination.sampleSlugs,
},
)
if (result.created) totals.nominationsCreated++
else totals.nominationsExisting++
}
}
return {
ok: true as const,
cursor,
isDone,
stats: totals,
nominations: nominations.slice(0, 200),
}
}
export const nominateEmptySkillSpammersInternal = internalAction({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
nominationThreshold: v.optional(v.number()),
},
handler: nominateEmptySkillSpammersInternalHandler,
})
export const nominateEmptySkillSpammers: ReturnType<typeof action> = action({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
nominationThreshold: v.optional(v.number()),
},
handler: async (ctx, args): Promise<EmptySkillBanNominationActionResult> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
return ctx.runAction(internal.maintenance.nominateEmptySkillSpammersInternal, args)
},
})
function clampInt(value: number, min: number, max: number) {
const rounded = Math.trunc(value)
if (!Number.isFinite(rounded)) return min
+42 -3
View File
@@ -3,8 +3,6 @@ import { defineSchema, defineTable } from 'convex/server'
import { v } from 'convex/values'
import { EMBEDDING_DIMENSIONS } from './lib/embeddings'
const authSchema = authTables as unknown as Record<string, ReturnType<typeof defineTable>>
const users = defineTable({
name: v.optional(v.string()),
image: v.optional(v.string()),
@@ -19,6 +17,8 @@ const users = defineTable({
role: v.optional(v.union(v.literal('admin'), v.literal('moderator'), v.literal('user'))),
githubCreatedAt: v.optional(v.number()),
githubFetchedAt: v.optional(v.number()),
deactivatedAt: v.optional(v.number()),
purgedAt: v.optional(v.number()),
deletedAt: v.optional(v.number()),
banReason: v.optional(v.string()),
createdAt: v.optional(v.number()),
@@ -79,6 +79,26 @@ const skills = defineTable({
),
moderationNotes: v.optional(v.string()),
moderationReason: v.optional(v.string()),
quality: v.optional(
v.object({
score: v.number(),
decision: v.union(v.literal('pass'), v.literal('quarantine'), v.literal('reject')),
trustTier: v.union(v.literal('low'), v.literal('medium'), v.literal('trusted')),
similarRecentCount: v.number(),
reason: v.string(),
signals: v.object({
bodyChars: v.number(),
bodyWords: v.number(),
uniqueWordRatio: v.number(),
headingCount: v.number(),
bulletCount: v.number(),
templateMarkerHits: v.number(),
genericSummary: v.boolean(),
cjkChars: v.optional(v.number()),
}),
evaluatedAt: v.number(),
}),
),
moderationFlags: v.optional(v.array(v.string())),
lastReviewedAt: v.optional(v.number()),
// VT scan tracking
@@ -113,6 +133,15 @@ const skills = defineTable({
.index('by_stats_installs_all_time', ['statsInstallsAllTime', 'updatedAt'])
.index('by_batch', ['batch'])
.index('by_active_updated', ['softDeletedAt', 'updatedAt'])
.index('by_active_created', ['softDeletedAt', 'createdAt'])
.index('by_active_name', ['softDeletedAt', 'displayName'])
.index('by_active_stats_downloads', ['softDeletedAt', 'statsDownloads', 'updatedAt'])
.index('by_active_stats_stars', ['softDeletedAt', 'statsStars', 'updatedAt'])
.index('by_active_stats_installs_all_time', [
'softDeletedAt',
'statsInstallsAllTime',
'updatedAt',
])
.index('by_canonical', ['canonicalSkillId'])
.index('by_fork_of', ['forkOf.skillId'])
@@ -462,6 +491,15 @@ const rateLimits = defineTable({
.index('by_key_window', ['key', 'windowStart'])
.index('by_key', ['key'])
const downloadDedupes = defineTable({
skillId: v.id('skills'),
identityHash: v.string(),
hourStart: v.number(),
createdAt: v.number(),
})
.index('by_skill_identity_hour', ['skillId', 'identityHash', 'hourStart'])
.index('by_hour', ['hourStart'])
const githubBackupSyncState = defineTable({
key: v.string(),
cursor: v.optional(v.string()),
@@ -507,7 +545,7 @@ const userSkillRootInstalls = defineTable({
.index('by_skill', ['skillId'])
export default defineSchema({
...authSchema,
...authTables,
users,
skills,
souls,
@@ -532,6 +570,7 @@ export default defineSchema({
vtScanLogs,
apiTokens,
rateLimits,
downloadDedupes,
githubBackupSyncState,
userSyncRoots,
userSkillInstalls,
+77 -4
View File
@@ -2,7 +2,7 @@
import { describe, expect, it, vi } from 'vitest'
import { tokenize } from './lib/searchText'
import { __test, lexicalFallbackSkills, searchSkills } from './search'
import { __test, hydrateResults, lexicalFallbackSkills, searchSkills } from './search'
const { generateEmbeddingMock, getSkillBadgeMapsMock } = vi.hoisted(() => ({
generateEmbeddingMock: vi.fn(),
@@ -20,11 +20,22 @@ vi.mock('./lib/badges', () => ({
}))
type WrappedHandler = {
_handler: (ctx: unknown, args: unknown) => Promise<unknown>
_handler: (
ctx: unknown,
args: unknown,
) => Promise<Array<{ skill: { slug: string; _id: string } }>>
}
const searchSkillsHandler = (searchSkills as unknown as WrappedHandler)._handler
const lexicalFallbackSkillsHandler = (lexicalFallbackSkills as unknown as WrappedHandler)._handler
const hydrateResultsHandler = (
hydrateResults as unknown as {
_handler: (
ctx: unknown,
args: unknown,
) => Promise<Array<{ skill: { slug: string; _id: string } }>>
}
)._handler
describe('search helpers', () => {
it('returns fallback results when vector candidates are empty', async () => {
@@ -84,6 +95,33 @@ describe('search helpers', () => {
expect(result[0].skill.slug).toBe('orf-highlighted')
})
it('applies nonSuspiciousOnly filtering in lexical fallback', async () => {
const suspicious = makeSkillDoc({
id: 'skills:suspicious',
slug: 'orf-suspicious',
displayName: 'ORF Suspicious',
moderationFlags: ['flagged.suspicious'],
})
const clean = makeSkillDoc({ id: 'skills:clean', slug: 'orf-clean', displayName: 'ORF Clean' })
getSkillBadgeMapsMock.mockResolvedValueOnce(
new Map([
['skills:suspicious', {}],
['skills:clean', {}],
]),
)
const result = await lexicalFallbackSkillsHandler(
makeLexicalCtx({
exactSlugSkill: null,
recentSkills: [suspicious, clean],
}),
{ query: 'orf', queryTokens: ['orf'], nonSuspiciousOnly: true, limit: 10 },
)
expect(result).toHaveLength(1)
expect(result[0].skill.slug).toBe('orf-clean')
})
it('includes exact slug match from by_slug even when recent scan is empty', async () => {
const exactSlugSkill = makeSkillDoc({ id: 'skills:orf', slug: 'orf', displayName: 'ORF' })
getSkillBadgeMapsMock.mockResolvedValueOnce(new Map([['skills:orf', {}]]))
@@ -174,6 +212,34 @@ describe('search helpers', () => {
expect(new Set(result.map((entry: { skill: { _id: string } }) => entry.skill._id)).size).toBe(2)
})
it('filters suspicious vector results in hydrateResults when requested', async () => {
const result = await hydrateResultsHandler(
{
db: {
get: vi.fn(async (id: string) => {
if (id === 'skillEmbeddings:1') {
return { _id: 'skillEmbeddings:1', skillId: 'skills:1', versionId: 'skillVersions:1' }
}
if (id === 'skills:1') {
return makeSkillDoc({
id: 'skills:1',
slug: 'suspicious',
displayName: 'Suspicious',
moderationFlags: ['flagged.suspicious'],
})
}
if (id === 'users:owner') return { _id: 'users:owner', handle: 'owner' }
if (id === 'skillVersions:1') return { _id: 'skillVersions:1', version: '1.0.0' }
return null
}),
},
},
{ embeddingIds: ['skillEmbeddings:1'], nonSuspiciousOnly: true },
)
expect(result).toHaveLength(0)
})
it('advances candidate limit until max', () => {
expect(__test.getNextCandidateLimit(50, 1000)).toBe(100)
expect(__test.getNextCandidateLimit(800, 1000)).toBe(1000)
@@ -259,12 +325,19 @@ function makePublicSkill(params: {
}
}
function makeSkillDoc(params: { id: string; slug: string; displayName: string }) {
function makeSkillDoc(params: {
id: string
slug: string
displayName: string
moderationFlags?: string[]
moderationReason?: string
}) {
return {
...makePublicSkill(params),
_creationTime: 1,
moderationStatus: 'active',
moderationFlags: [],
moderationFlags: params.moderationFlags ?? [],
moderationReason: params.moderationReason,
softDeletedAt: undefined,
}
}
+17 -3
View File
@@ -6,6 +6,7 @@ import { getSkillBadgeMaps, isSkillHighlighted, type SkillBadgeMap } from './lib
import { generateEmbedding } from './lib/embeddings'
import { toPublicSkill, toPublicSoul } from './lib/public'
import { matchesExactTokens, tokenize } from './lib/searchText'
import { isSkillSuspicious } from './lib/skillSafety'
type SkillSearchEntry = {
embeddingId?: Id<'skillEmbeddings'>
@@ -92,6 +93,7 @@ export const searchSkills: ReturnType<typeof action> = action({
query: v.string(),
limit: v.optional(v.number()),
highlightedOnly: v.optional(v.boolean()),
nonSuspiciousOnly: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<SearchResult[]> => {
const query = args.query.trim()
@@ -122,6 +124,7 @@ export const searchSkills: ReturnType<typeof action> = action({
hydrated = (await ctx.runQuery(internal.search.hydrateResults, {
embeddingIds: results.map((result) => result._id),
nonSuspiciousOnly: args.nonSuspiciousOnly,
})) as SkillSearchEntry[]
scoreById = new Map<Id<'skillEmbeddings'>, number>(
@@ -169,6 +172,7 @@ export const searchSkills: ReturnType<typeof action> = action({
queryTokens,
limit: Math.min(Math.max(limit * 4, 200), FALLBACK_SCAN_LIMIT),
highlightedOnly: args.highlightedOnly,
nonSuspiciousOnly: args.nonSuspiciousOnly,
})) as SkillSearchEntry[])
const mergedMatches = mergeUniqueBySkillId(exactMatches, fallbackMatches)
@@ -202,7 +206,10 @@ export const getBadgeMapsForSkills = internalQuery({
})
export const hydrateResults = internalQuery({
args: { embeddingIds: v.array(v.id('skillEmbeddings')) },
args: {
embeddingIds: v.array(v.id('skillEmbeddings')),
nonSuspiciousOnly: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<SkillSearchEntry[]> => {
const ownerHandleCache = new Map<Id<'users'>, Promise<string | null>>()
@@ -216,12 +223,13 @@ export const hydrateResults = internalQuery({
return handlePromise
}
const entries = await Promise.all(
const entries: Array<SkillSearchEntry | null> = await Promise.all(
args.embeddingIds.map(async (embeddingId) => {
const embedding = await ctx.db.get(embeddingId)
if (!embedding) return null
const skill = await ctx.db.get(embedding.skillId)
if (!skill || skill.softDeletedAt) return null
if (args.nonSuspiciousOnly && isSkillSuspicious(skill)) return null
const [version, ownerHandle] = await Promise.all([
ctx.db.get(embedding.versionId),
getOwnerHandle(skill.ownerUserId),
@@ -242,6 +250,7 @@ export const lexicalFallbackSkills = internalQuery({
queryTokens: v.array(v.string()),
limit: v.optional(v.number()),
highlightedOnly: v.optional(v.boolean()),
nonSuspiciousOnly: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<SkillSearchEntry[]> => {
const limit = Math.min(Math.max(args.limit ?? 200, 10), FALLBACK_SCAN_LIMIT)
@@ -254,7 +263,11 @@ export const lexicalFallbackSkills = internalQuery({
.query('skills')
.withIndex('by_slug', (q) => q.eq('slug', slugQuery))
.unique()
if (exactSlugSkill && !exactSlugSkill.softDeletedAt) {
if (
exactSlugSkill &&
!exactSlugSkill.softDeletedAt &&
(!args.nonSuspiciousOnly || !isSkillSuspicious(exactSlugSkill))
) {
seenSkillIds.add(exactSlugSkill._id)
candidateSkills.push(exactSlugSkill)
}
@@ -268,6 +281,7 @@ export const lexicalFallbackSkills = internalQuery({
for (const skill of recentSkills) {
if (seenSkillIds.has(skill._id)) continue
if (args.nonSuspiciousOnly && isSkillSuspicious(skill)) continue
seenSkillIds.add(skill._id)
candidateSkills.push(skill)
}
+349
View File
@@ -0,0 +1,349 @@
import { describe, expect, it, vi } from 'vitest'
import {
approveSkillByHashInternal,
clearOwnerSuspiciousFlagsInternal,
escalateByVtInternal,
insertVersion,
} from './skills'
type WrappedHandler<TArgs> = {
_handler: (ctx: unknown, args: TArgs) => Promise<unknown>
}
const insertVersionHandler = (insertVersion as unknown as WrappedHandler<Record<string, unknown>>)
._handler
const approveSkillByHashHandler = (
approveSkillByHashInternal as unknown as WrappedHandler<Record<string, unknown>>
)._handler
const escalateByVtHandler = (
escalateByVtInternal as unknown as WrappedHandler<Record<string, unknown>>
)._handler
const clearOwnerSuspiciousFlagsHandler = (
clearOwnerSuspiciousFlagsInternal as unknown as WrappedHandler<Record<string, unknown>>
)._handler
function createPublishArgs(overrides?: Partial<Record<string, unknown>>) {
return {
userId: 'users:owner',
slug: 'spam-skill',
displayName: 'Spam Skill',
version: '1.0.0',
changelog: 'Initial release',
changelogSource: 'user',
tags: ['latest'],
fingerprint: 'f'.repeat(64),
files: [
{
path: 'SKILL.md',
size: 128,
storageId: '_storage:1',
sha256: 'a'.repeat(64),
contentType: 'text/markdown',
},
],
parsed: {
frontmatter: { description: 'test' },
metadata: {},
clawdis: {},
},
embedding: [0.1, 0.2],
...overrides,
}
}
describe('skills anti-spam guards', () => {
it('blocks low-trust users after hourly new-skill cap', async () => {
const now = Date.now()
const ownerSkills = Array.from({ length: 5 }, (_, i) => ({
_id: `skills:${i}`,
createdAt: now - i * 10_000,
}))
const db = {
get: vi.fn(async () => ({
_id: 'users:owner',
_creationTime: now - 2 * 24 * 60 * 60 * 1000,
createdAt: now - 2 * 24 * 60 * 60 * 1000,
deletedAt: undefined,
})),
query: vi.fn((table: string) => {
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name === 'by_slug') {
return { unique: async () => null }
}
if (name === 'by_owner') {
return {
order: () => ({
take: async () => ownerSkills,
}),
}
}
throw new Error(`unexpected index ${name}`)
},
}
}
throw new Error(`unexpected table ${table}`)
}),
}
await expect(
insertVersionHandler({ db } as never, createPublishArgs() as never),
).rejects.toThrow(/max 5 new skills per hour/i)
})
it('keeps suspicious skills visible for low-trust publishers', async () => {
const patch = vi.fn(async () => {})
const version = { _id: 'skillVersions:1', skillId: 'skills:1' }
const skill = {
_id: 'skills:1',
slug: 'spam-skill',
ownerUserId: 'users:owner',
moderationFlags: undefined,
moderationReason: undefined,
}
const owner = {
_id: 'users:owner',
_creationTime: Date.now() - 2 * 24 * 60 * 60 * 1000,
createdAt: Date.now() - 2 * 24 * 60 * 60 * 1000,
deletedAt: undefined,
}
const db = {
get: vi.fn(async (id: string) => {
if (id === 'skills:1') return skill
if (id === 'users:owner') return owner
return null
}),
query: vi.fn((table: string) => {
if (table === 'skillVersions') {
return {
withIndex: () => ({
unique: async () => version,
}),
}
}
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name === 'by_owner') {
return {
order: () => ({
take: async () => [],
}),
}
}
throw new Error(`unexpected skills index ${name}`)
},
}
}
throw new Error(`unexpected table ${table}`)
}),
patch,
}
await approveSkillByHashHandler(
{ db, scheduler: { runAfter: vi.fn() } } as never,
{
sha256hash: 'h'.repeat(64),
scanner: 'vt',
status: 'suspicious',
} as never,
)
expect(patch).toHaveBeenCalledWith(
'skills:1',
expect.objectContaining({
moderationStatus: 'active',
moderationReason: 'scanner.vt.suspicious',
moderationFlags: ['flagged.suspicious'],
}),
)
})
it('keeps admin-owned skills non-suspicious for suspicious scanner verdicts', async () => {
const patch = vi.fn(async () => {})
const version = { _id: 'skillVersions:1', skillId: 'skills:1' }
const skill = {
_id: 'skills:1',
slug: 'trusted-skill',
ownerUserId: 'users:owner',
moderationFlags: ['flagged.suspicious'],
moderationReason: 'scanner.vt.suspicious',
}
const owner = {
_id: 'users:owner',
role: 'admin',
_creationTime: Date.now() - 60 * 24 * 60 * 60 * 1000,
createdAt: Date.now() - 60 * 24 * 60 * 60 * 1000,
deletedAt: undefined,
}
const db = {
get: vi.fn(async (id: string) => {
if (id === 'skills:1') return skill
if (id === 'users:owner') return owner
return null
}),
query: vi.fn((table: string) => {
if (table === 'skillVersions') {
return {
withIndex: () => ({
unique: async () => version,
}),
}
}
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name === 'by_owner') {
return {
order: () => ({
take: async () => [],
}),
}
}
throw new Error(`unexpected skills index ${name}`)
},
}
}
throw new Error(`unexpected table ${table}`)
}),
patch,
}
await approveSkillByHashHandler(
{ db, scheduler: { runAfter: vi.fn() } } as never,
{
sha256hash: 'h'.repeat(64),
scanner: 'llm',
status: 'suspicious',
} as never,
)
expect(patch).toHaveBeenCalledWith(
'skills:1',
expect.objectContaining({
moderationStatus: 'active',
moderationReason: 'scanner.llm.clean',
moderationFlags: undefined,
}),
)
})
it('vt suspicious escalation does not keep suspicious flags for admin owners', async () => {
const patch = vi.fn(async () => {})
const version = { _id: 'skillVersions:1', skillId: 'skills:1' }
const skill = {
_id: 'skills:1',
slug: 'trusted-skill',
ownerUserId: 'users:owner',
moderationFlags: ['flagged.suspicious'],
moderationReason: 'scanner.llm.suspicious',
}
const owner = {
_id: 'users:owner',
role: 'admin',
deletedAt: undefined,
}
const db = {
get: vi.fn(async (id: string) => {
if (id === 'skills:1') return skill
if (id === 'users:owner') return owner
return null
}),
query: vi.fn((table: string) => {
if (table === 'skillVersions') {
return {
withIndex: () => ({
unique: async () => version,
}),
}
}
throw new Error(`unexpected table ${table}`)
}),
patch,
}
await escalateByVtHandler(
{ db, scheduler: { runAfter: vi.fn() } } as never,
{
sha256hash: 'h'.repeat(64),
status: 'suspicious',
} as never,
)
expect(patch).toHaveBeenCalledWith(
'skills:1',
expect.objectContaining({
moderationFlags: undefined,
moderationReason: 'scanner.llm.clean',
}),
)
})
it('bulk-clears suspicious flags/reasons for privileged owner skills', async () => {
const patch = vi.fn(async () => {})
const owner = {
_id: 'users:owner',
role: 'admin',
deletedAt: undefined,
}
const skills = [
{
_id: 'skills:1',
moderationFlags: ['flagged.suspicious'],
moderationReason: 'scanner.vt.suspicious',
moderationStatus: 'hidden',
softDeletedAt: undefined,
},
{
_id: 'skills:2',
moderationFlags: undefined,
moderationReason: 'scanner.llm.clean',
moderationStatus: 'active',
softDeletedAt: undefined,
},
]
const db = {
get: vi.fn(async (id: string) => {
if (id === 'users:owner') return owner
return null
}),
query: vi.fn((table: string) => {
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name !== 'by_owner') throw new Error(`unexpected skills index ${name}`)
return {
order: () => ({
take: async () => skills,
}),
}
},
}
}
throw new Error(`unexpected table ${table}`)
}),
patch,
}
const result = await clearOwnerSuspiciousFlagsHandler(
{ db } as never,
{ ownerUserId: 'users:owner', limit: 20 } as never,
)
expect(result).toEqual({ inspected: 2, updated: 1 })
expect(patch).toHaveBeenCalledWith(
'skills:1',
expect.objectContaining({
moderationFlags: undefined,
moderationReason: 'scanner.vt.clean',
moderationStatus: 'active',
}),
)
})
})
+401 -90
View File
@@ -1,7 +1,6 @@
import { getAuthUserId } from '@convex-dev/auth/server'
import { paginationOptsValidator } from 'convex/server'
import { ConvexError, v } from 'convex/values'
import { paginator } from 'convex-helpers/server/pagination'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import type { MutationCtx, QueryCtx } from './_generated/server'
@@ -14,8 +13,17 @@ import {
query,
} from './_generated/server'
import { assertAdmin, assertModerator, requireUser, requireUserFromAction } from './lib/access'
import { getSkillBadgeMap, getSkillBadgeMaps, isSkillHighlighted } from './lib/badges'
import {
getSkillBadgeMap,
getSkillBadgeMaps,
isSkillHighlighted,
type SkillBadgeMap,
} from './lib/badges'
import { generateChangelogPreview as buildChangelogPreview } from './lib/changelog'
import {
canHealSkillOwnershipByGitHubProviderAccountId,
getGitHubProviderAccountId,
} from './lib/githubIdentity'
import { buildTrendingLeaderboard } from './lib/leaderboards'
import { deriveModerationFlags } from './lib/moderation'
import { toPublicSkill, toPublicUser } from './lib/public'
@@ -25,8 +33,8 @@ import {
publishVersionForUser,
queueHighlightedWebhook,
} from './lib/skillPublish'
import { isSkillSuspicious } from './lib/skillSafety'
import { getFrontmatterValue, hashSkillFiles } from './lib/skills'
import schema from './schema'
export { publishVersionForUser } from './lib/skillPublish'
@@ -38,12 +46,32 @@ const MAX_LIST_LIMIT = 50
const MAX_PUBLIC_LIST_LIMIT = 200
const MAX_LIST_BULK_LIMIT = 200
const MAX_LIST_TAKE = 1000
const MAX_BADGE_LOOKUP_SKILLS = 200
const HARD_DELETE_BATCH_SIZE = 100
const HARD_DELETE_VERSION_BATCH_SIZE = 10
const HARD_DELETE_LEADERBOARD_BATCH_SIZE = 25
const MAX_ACTIVE_REPORTS_PER_USER = 20
const AUTO_HIDE_REPORT_THRESHOLD = 3
const MAX_REPORT_REASON_SAMPLE = 5
const RATE_LIMIT_HOUR_MS = 60 * 60 * 1000
const RATE_LIMIT_DAY_MS = 24 * RATE_LIMIT_HOUR_MS
const LOW_TRUST_ACCOUNT_AGE_MS = 30 * RATE_LIMIT_DAY_MS
const TRUSTED_PUBLISHER_SKILL_THRESHOLD = 10
const LOW_TRUST_BURST_THRESHOLD_PER_HOUR = 8
const OWNER_ACTIVITY_SCAN_LIMIT = 500
const NEW_SKILL_RATE_LIMITS = {
lowTrust: { perHour: 5, perDay: 20 },
trusted: { perHour: 20, perDay: 80 },
} as const
const SORT_INDEXES = {
newest: 'by_active_created',
updated: 'by_active_updated',
name: 'by_active_name',
downloads: 'by_active_stats_downloads',
stars: 'by_active_stats_stars',
installs: 'by_active_stats_installs_all_time',
} as const
function isSkillVersionId(
value: Id<'skillVersions'> | null | undefined,
@@ -55,8 +83,81 @@ function isUserId(value: Id<'users'> | null | undefined): value is Id<'users'> {
return typeof value === 'string' && value.startsWith('users:')
}
type OwnerTrustSignals = {
isLowTrust: boolean
skillsLastHour: number
skillsLastDay: number
}
function isPrivilegedOwnerForSuspiciousBypass(owner: Doc<'users'> | null | undefined) {
if (!owner) return false
return owner.role === 'admin' || owner.role === 'moderator'
}
function stripSuspiciousFlag(flags: string[] | undefined) {
if (!flags?.length) return undefined
const next = flags.filter((flag) => flag !== 'flagged.suspicious')
return next.length ? next : undefined
}
function normalizeScannerSuspiciousReason(reason: string | undefined) {
if (!reason) return reason
if (!reason.startsWith('scanner.') || !reason.endsWith('.suspicious')) return reason
return `${reason.slice(0, -'.suspicious'.length)}.clean`
}
async function getOwnerTrustSignals(
ctx: QueryCtx | MutationCtx,
owner: Doc<'users'>,
now: number,
): Promise<OwnerTrustSignals> {
const ownerSkills = await ctx.db
.query('skills')
.withIndex('by_owner', (q) => q.eq('ownerUserId', owner._id))
.order('desc')
.take(OWNER_ACTIVITY_SCAN_LIMIT)
const hourThreshold = now - RATE_LIMIT_HOUR_MS
const dayThreshold = now - RATE_LIMIT_DAY_MS
let skillsLastHour = 0
let skillsLastDay = 0
for (const skill of ownerSkills) {
if (skill.createdAt >= dayThreshold) {
skillsLastDay += 1
if (skill.createdAt >= hourThreshold) {
skillsLastHour += 1
}
}
}
const accountCreatedAt = owner.createdAt ?? owner._creationTime
const accountAgeMs = Math.max(0, now - accountCreatedAt)
const isLowTrust =
accountAgeMs < LOW_TRUST_ACCOUNT_AGE_MS ||
ownerSkills.length < TRUSTED_PUBLISHER_SKILL_THRESHOLD ||
skillsLastHour >= LOW_TRUST_BURST_THRESHOLD_PER_HOUR
return { isLowTrust, skillsLastHour, skillsLastDay }
}
function enforceNewSkillRateLimit(signals: OwnerTrustSignals) {
const limits = signals.isLowTrust ? NEW_SKILL_RATE_LIMITS.lowTrust : NEW_SKILL_RATE_LIMITS.trusted
if (signals.skillsLastHour >= limits.perHour) {
throw new ConvexError(
`Rate limit: max ${limits.perHour} new skills per hour. Please wait before publishing more.`,
)
}
if (signals.skillsLastDay >= limits.perDay) {
throw new ConvexError(
`Rate limit: max ${limits.perDay} new skills per 24 hours. Please wait before publishing more.`,
)
}
}
async function resolveOwnerHandle(ctx: QueryCtx, ownerUserId: Id<'users'>) {
const owner = await ctx.db.get(ownerUserId)
if (!owner || owner.deletedAt || owner.deactivatedAt) return null
return owner?.handle ?? owner?._id ?? null
}
@@ -352,10 +453,23 @@ async function hardDeleteSkillStep(
type PublicSkillEntry = {
skill: NonNullable<ReturnType<typeof toPublicSkill>>
latestVersion: Doc<'skillVersions'> | null
latestVersion: PublicSkillListVersion | null
ownerHandle: string | null
}
type PublicSkillListVersion = Pick<
Doc<'skillVersions'>,
'_id' | '_creationTime' | 'version' | 'createdAt' | 'changelog' | 'changelogSource'
> & {
parsed?: {
clawdis?: {
nix?: {
plugin?: boolean
}
}
}
}
type ManagementSkillEntry = {
skill: Doc<'skills'>
latestVersion: Doc<'skillVersions'> | null
@@ -366,10 +480,13 @@ type BadgeKind = Doc<'skillBadges'>['kind']
async function buildPublicSkillEntries(ctx: QueryCtx, skills: Doc<'skills'>[]) {
const ownerHandleCache = new Map<Id<'users'>, Promise<string | null>>()
const badgeMapBySkillId = await getSkillBadgeMaps(
ctx,
skills.map((skill) => skill._id),
)
const badgeMapBySkillId: Map<Id<'skills'>, SkillBadgeMap> = skills.length <=
MAX_BADGE_LOOKUP_SKILLS
? await getSkillBadgeMaps(
ctx,
skills.map((skill) => skill._id),
)
: new Map()
const getOwnerHandle = (ownerUserId: Id<'users'>) => {
const cached = ownerHandleCache.get(ownerUserId)
@@ -381,13 +498,14 @@ async function buildPublicSkillEntries(ctx: QueryCtx, skills: Doc<'skills'>[]) {
const entries = await Promise.all(
skills.map(async (skill) => {
const [latestVersion, ownerHandle] = await Promise.all([
const [latestVersionDoc, ownerHandle] = await Promise.all([
skill.latestVersionId ? ctx.db.get(skill.latestVersionId) : null,
getOwnerHandle(skill.ownerUserId),
])
const badges = badgeMapBySkillId.get(skill._id) ?? {}
const publicSkill = toPublicSkill({ ...skill, badges })
if (!publicSkill) return null
const latestVersion = toPublicSkillListVersion(latestVersionDoc)
return { skill: publicSkill, latestVersion, ownerHandle }
}),
)
@@ -395,6 +513,21 @@ async function buildPublicSkillEntries(ctx: QueryCtx, skills: Doc<'skills'>[]) {
return entries.filter((entry): entry is PublicSkillEntry => entry !== null)
}
function toPublicSkillListVersion(
version: Doc<'skillVersions'> | null,
): PublicSkillListVersion | null {
if (!version) return null
return {
_id: version._id,
_creationTime: version._creationTime,
version: version.version,
createdAt: version.createdAt,
changelog: version.changelog,
changelogSource: version.changelogSource,
parsed: version.parsed?.clawdis ? { clawdis: version.parsed.clawdis } : undefined,
}
}
async function buildManagementSkillEntries(ctx: QueryCtx, skills: Doc<'skills'>[]) {
const ownerCache = new Map<Id<'users'>, Promise<Doc<'users'> | null>>()
const badgeMapBySkillId = await getSkillBadgeMaps(
@@ -653,6 +786,79 @@ export const getSkillBySlugInternal = internalQuery({
},
})
export const getOwnerSkillActivityInternal = internalQuery({
args: {
ownerUserId: v.id('users'),
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const limit = clampInt(args.limit ?? 60, 1, 500)
const skills = await ctx.db
.query('skills')
.withIndex('by_owner', (q) => q.eq('ownerUserId', args.ownerUserId))
.order('desc')
.take(limit)
return skills.map((skill) => ({
slug: skill.slug,
summary: skill.summary,
createdAt: skill.createdAt,
latestVersionId: skill.latestVersionId,
}))
},
})
export const clearOwnerSuspiciousFlagsInternal = internalMutation({
args: {
ownerUserId: v.id('users'),
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const owner = await ctx.db.get(args.ownerUserId)
if (!owner || owner.deletedAt || owner.deactivatedAt) throw new Error('Owner not found')
if (!isPrivilegedOwnerForSuspiciousBypass(owner)) {
return { inspected: 0, updated: 0, skipped: 'owner_not_privileged' as const }
}
const limit = clampInt(args.limit ?? 500, 1, 5000)
const skills = await ctx.db
.query('skills')
.withIndex('by_owner', (q) => q.eq('ownerUserId', args.ownerUserId))
.order('desc')
.take(limit)
let updated = 0
const now = Date.now()
for (const skill of skills) {
const existingFlags: string[] = (skill.moderationFlags as string[] | undefined) ?? []
const hasSuspiciousFlag = existingFlags.includes('flagged.suspicious')
const hasSuspiciousReason =
skill.moderationReason?.startsWith('scanner.') &&
skill.moderationReason.endsWith('.suspicious')
if (!hasSuspiciousFlag && !hasSuspiciousReason) continue
const patch: Partial<Doc<'skills'>> = { updatedAt: now }
patch.moderationFlags = stripSuspiciousFlag(existingFlags)
if (hasSuspiciousReason) {
patch.moderationReason = normalizeScannerSuspiciousReason(skill.moderationReason)
}
if (
(skill.moderationStatus ?? 'active') === 'hidden' &&
hasSuspiciousReason &&
!skill.softDeletedAt
) {
patch.moderationStatus = 'active'
}
await ctx.db.patch(skill._id, patch)
updated += 1
}
return { inspected: skills.length, updated }
},
})
/**
* Get quick stats without loading versions (fast).
*/
@@ -1153,7 +1359,7 @@ async function countActiveReportsForUser(ctx: MutationCtx, userId: Id<'users'>)
if (skill.softDeletedAt) continue
if (skill.moderationStatus === 'removed') continue
const owner = await ctx.db.get(skill.ownerUserId)
if (!owner || owner.deletedAt) continue
if (!owner || owner.deletedAt || owner.deactivatedAt) continue
count += 1
if (count >= MAX_ACTIVE_REPORTS_PER_USER) break
}
@@ -1301,33 +1507,51 @@ export const listPublicPage = query({
})
/**
* V2 of listPublicPage using convex-helpers paginator for better cache behavior.
* V2 of listPublicPage using standard Convex pagination (paginate + usePaginatedQuery).
*
* Key differences from V1:
* - Uses `paginator` from convex-helpers (doesn't track end-cursor internally, better caching)
* - Uses `by_active_updated` index to filter soft-deleted skills at query level
* - Returns standard pagination shape compatible with usePaginatedQuery
*/
export const listPublicPageV2 = query({
args: {
paginationOpts: paginationOptsValidator,
sort: v.optional(
v.union(
v.literal('newest'),
v.literal('updated'),
v.literal('downloads'),
v.literal('installs'),
v.literal('stars'),
v.literal('name'),
),
),
dir: v.optional(v.union(v.literal('asc'), v.literal('desc'))),
nonSuspiciousOnly: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
// Use the new index to filter out soft-deleted skills at query time.
const sort = args.sort ?? 'newest'
const dir = args.dir ?? (sort === 'name' ? 'asc' : 'desc')
const paginationOpts: { cursor: string | null; numItems: number; id?: number } = {
...args.paginationOpts,
numItems: clampInt(args.paginationOpts.numItems, 1, MAX_PUBLIC_LIST_LIMIT),
}
// Use the index to filter out soft-deleted skills at query time.
// softDeletedAt === undefined means active (non-deleted) skills only.
const result = await paginator(ctx.db, schema)
const result = await ctx.db
.query('skills')
.withIndex('by_active_updated', (q) => q.eq('softDeletedAt', undefined))
.order('desc')
.paginate(args.paginationOpts)
.withIndex(SORT_INDEXES[sort], (q) => q.eq('softDeletedAt', undefined))
.order(dir)
.paginate(paginationOpts)
const filteredPage = args.nonSuspiciousOnly
? result.page.filter((skill) => !isSkillSuspicious(skill))
: result.page
// Build the public skill entries (fetch latestVersion + ownerHandle)
const items = await buildPublicSkillEntries(ctx, result.page)
return {
...result,
page: items,
}
const items = await buildPublicSkillEntries(ctx, filteredPage)
return { ...result, page: items }
},
})
@@ -1403,6 +1627,14 @@ export const getVersionById = query({
handler: async (ctx, args) => ctx.db.get(args.versionId),
})
export const getVersionsByIdsInternal = internalQuery({
args: { versionIds: v.array(v.id('skillVersions')) },
handler: async (ctx, args) => {
const versions = await Promise.all(args.versionIds.map((id) => ctx.db.get(id)))
return versions.filter((v): v is NonNullable<typeof v> => v !== null)
},
})
export const getVersionByIdInternal = internalQuery({
args: { versionId: v.id('skillVersions') },
handler: async (ctx, args) => ctx.db.get(args.versionId),
@@ -1416,56 +1648,32 @@ export const getSkillByIdInternal = internalQuery({
export const getPendingScanSkillsInternal = internalQuery({
args: { limit: v.optional(v.number()), skipRecentMinutes: v.optional(v.number()) },
handler: async (ctx, args) => {
const limit = args.limit ?? 10
const limit = clampInt(args.limit ?? 10, 1, 100)
const skipRecentMinutes = args.skipRecentMinutes ?? 60
const skipThreshold = Date.now() - skipRecentMinutes * 60 * 1000
// Fetch more than needed so we can randomize selection.
// Include newly-published skills (hidden/pending.scan), skills stuck at
// scanner.vt.pending, AND LLM-evaluated skills that still need VT results.
const poolSize = Math.min(limit * 3, 500)
const pendingScan = await ctx.db
// Use an indexed query and bounded scan to avoid full-table reads under spam/high volume.
const poolSize = Math.min(Math.max(limit * 20, 200), 1000)
const allSkills = await ctx.db
.query('skills')
.filter((q) =>
q.and(
q.eq(q.field('moderationStatus'), 'hidden'),
q.eq(q.field('moderationReason'), 'pending.scan'),
),
)
.take(poolSize)
const vtPending = await ctx.db
.query('skills')
.filter((q) =>
q.and(
q.eq(q.field('moderationStatus'), 'active'),
q.eq(q.field('moderationReason'), 'scanner.vt.pending'),
),
)
.take(poolSize)
// LLM-evaluated skills whose VT scan hasn't completed yet
const llmEvaluated = await ctx.db
.query('skills')
.filter((q) =>
q.or(
q.eq(q.field('moderationReason'), 'scanner.llm.clean'),
q.eq(q.field('moderationReason'), 'scanner.llm.suspicious'),
q.eq(q.field('moderationReason'), 'scanner.llm.malicious'),
),
)
.withIndex('by_active_updated', (q) => q.eq('softDeletedAt', undefined))
.order('desc')
.take(poolSize)
// Dedup across pools by skill ID
const seen = new Set<string>()
const allSkills: typeof pendingScan = []
for (const skill of [...pendingScan, ...vtPending, ...llmEvaluated]) {
if (!seen.has(skill._id)) {
seen.add(skill._id)
allSkills.push(skill)
}
}
const candidates = allSkills.filter((skill) => {
const reason = skill.moderationReason
if (skill.moderationStatus === 'hidden' && reason === 'pending.scan') return true
if (skill.moderationStatus === 'hidden' && reason === 'quality.low') return true
if (skill.moderationStatus === 'active' && reason === 'scanner.vt.pending') return true
return (
reason === 'scanner.llm.clean' ||
reason === 'scanner.llm.suspicious' ||
reason === 'scanner.llm.malicious'
)
})
// Filter out recently checked skills
const skills = allSkills.filter(
const skills = candidates.filter(
(s) => !s.scanLastCheckedAt || s.scanLastCheckedAt < skipThreshold,
)
@@ -1760,7 +1968,7 @@ export const getSkillsWithStaleModerationReasonInternal = internalQuery({
const limit = args.limit ?? 100
// Find skills with pending-like moderationReason
const staleReasons = ['scanner.vt.pending', 'pending.scan']
const staleReasons = new Set(['scanner.vt.pending', 'pending.scan'])
const allSkills = await ctx.db
.query('skills')
.filter((q) => q.eq(q.field('moderationStatus'), 'active'))
@@ -1775,7 +1983,7 @@ export const getSkillsWithStaleModerationReasonInternal = internalQuery({
}> = []
for (const skill of allSkills) {
if (!skill.moderationReason || !staleReasons.includes(skill.moderationReason)) continue
if (!skill.moderationReason || !staleReasons.has(skill.moderationReason)) continue
if (!skill.latestVersionId) continue
const version = await ctx.db.get(skill.latestVersionId)
@@ -2093,6 +2301,7 @@ export const approveSkillByHashInternal = internalMutation({
// Update the skill's moderation status based on scan result
const skill = await ctx.db.get(version.skillId)
if (skill) {
const owner = skill.ownerUserId ? await ctx.db.get(skill.ownerUserId) : null
const isMalicious = args.status === 'malicious'
const isSuspicious = args.status === 'suspicious'
const isClean = !isMalicious && !isSuspicious
@@ -2103,13 +2312,15 @@ export const approveSkillByHashInternal = internalMutation({
const existingReason: string | undefined = skill.moderationReason as string | undefined
const alreadyBlocked = existingFlags.includes('blocked.malware')
const alreadyFlagged = existingFlags.includes('flagged.suspicious')
const bypassSuspicious =
isSuspicious && !alreadyBlocked && isPrivilegedOwnerForSuspiciousBypass(owner)
// Determine new flags based on multi-scanner merge
let newFlags: string[] | undefined
if (isMalicious || alreadyBlocked) {
// Malicious from ANY scanner → blocked.malware (upgrade from suspicious)
newFlags = ['blocked.malware']
} else if (isSuspicious || alreadyFlagged) {
} else if ((isSuspicious || alreadyFlagged) && !bypassSuspicious) {
// Suspicious from ANY scanner → flagged.suspicious
newFlags = ['flagged.suspicious']
} else if (isClean) {
@@ -2121,12 +2332,32 @@ export const approveSkillByHashInternal = internalMutation({
!existingReason.endsWith('.pending')
newFlags = otherScannerFlagged ? existingFlags : undefined
}
if (!alreadyBlocked && isPrivilegedOwnerForSuspiciousBypass(owner)) {
newFlags = stripSuspiciousFlag(newFlags ?? existingFlags)
}
const now = Date.now()
const qualityLocked = skill.moderationReason === 'quality.low' && !isMalicious
const nextModerationStatus = qualityLocked ? 'hidden' : 'active'
const nextModerationReason = qualityLocked
? 'quality.low'
: bypassSuspicious
? `scanner.${args.scanner}.clean`
: `scanner.${args.scanner}.${args.status}`
const nextModerationNotes = qualityLocked
? (skill.moderationNotes ??
'Quality gate quarantine is still active. Manual moderation review required.')
: undefined
await ctx.db.patch(skill._id, {
moderationStatus: 'active', // Always visible for transparency
moderationReason: `scanner.${args.scanner}.${args.status}`,
moderationStatus: nextModerationStatus,
moderationReason: nextModerationReason,
moderationFlags: newFlags,
updatedAt: Date.now(),
moderationNotes: nextModerationNotes,
hiddenAt: nextModerationStatus === 'hidden' ? now : undefined,
hiddenBy: undefined,
lastReviewedAt: nextModerationStatus === 'hidden' ? now : undefined,
updatedAt: now,
})
// Auto-ban authors of malicious skills (skips moderators/admins)
@@ -2166,19 +2397,29 @@ export const escalateByVtInternal = internalMutation({
const isMalicious = args.status === 'malicious'
const existingFlags: string[] = (skill.moderationFlags as string[] | undefined) ?? []
const alreadyBlocked = existingFlags.includes('blocked.malware')
const owner = skill.ownerUserId ? await ctx.db.get(skill.ownerUserId) : null
const bypassSuspicious =
!isMalicious && !alreadyBlocked && isPrivilegedOwnerForSuspiciousBypass(owner)
// Determine new flags — stricter verdict always wins
let newFlags: string[]
if (isMalicious || alreadyBlocked) {
newFlags = ['blocked.malware']
} else if (bypassSuspicious) {
newFlags = stripSuspiciousFlag(existingFlags) ?? []
} else {
newFlags = ['flagged.suspicious']
}
const patch: Record<string, unknown> = {
moderationFlags: newFlags,
moderationFlags: newFlags.length ? newFlags : undefined,
updatedAt: Date.now(),
}
if (bypassSuspicious) {
patch.moderationReason = normalizeScannerSuspiciousReason(
skill.moderationReason as string | undefined,
)
}
// Only hide for malicious — suspicious stays visible with a flag
if (isMalicious) {
@@ -2537,7 +2778,8 @@ export const changeOwner = mutation({
if (!skill) throw new Error('Skill not found')
const nextOwner = await ctx.db.get(args.ownerUserId)
if (!nextOwner || nextOwner.deletedAt) throw new Error('User not found')
if (!nextOwner || nextOwner.deletedAt || nextOwner.deactivatedAt)
throw new Error('User not found')
if (skill.ownerUserId === args.ownerUserId) return
@@ -2710,7 +2952,7 @@ export const hardDeleteInternal = internalMutation({
args: { skillId: v.id('skills'), actorUserId: v.id('users'), phase: v.optional(v.string()) },
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId)
if (!actor || actor.deletedAt) throw new Error('User not found')
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new Error('User not found')
assertAdmin(actor)
const skill = await ctx.db.get(args.skillId)
if (!skill) return
@@ -2749,24 +2991,88 @@ export const insertVersion = internalMutation({
metadata: v.optional(v.any()),
clawdis: v.optional(v.any()),
}),
summary: v.optional(v.string()),
qualityAssessment: v.optional(
v.object({
decision: v.union(v.literal('pass'), v.literal('quarantine'), v.literal('reject')),
score: v.number(),
reason: v.string(),
trustTier: v.union(v.literal('low'), v.literal('medium'), v.literal('trusted')),
similarRecentCount: v.number(),
signals: v.object({
bodyChars: v.number(),
bodyWords: v.number(),
uniqueWordRatio: v.number(),
headingCount: v.number(),
bulletCount: v.number(),
templateMarkerHits: v.number(),
genericSummary: v.boolean(),
cjkChars: v.optional(v.number()),
}),
}),
),
embedding: v.array(v.number()),
},
handler: async (ctx, args) => {
const userId = args.userId
const user = await ctx.db.get(userId)
if (!user || user.deletedAt) throw new Error('User not found')
handler: async (ctx, args) => {
const userId = args.userId
const user = await ctx.db.get(userId)
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
let skill = await ctx.db
.query('skills')
.withIndex('by_slug', (q) => q.eq('slug', args.slug))
const now = Date.now()
let skill = await ctx.db
.query('skills')
.withIndex('by_slug', (q) => q.eq('slug', args.slug))
.unique()
if (skill && skill.ownerUserId !== userId) {
throw new Error('Only the owner can publish updates')
}
// Fallback: Convex Auth can create duplicate `users` records. Heal ownership ONLY
// when the underlying GitHub identity matches (authAccounts.providerAccountId).
const owner = await ctx.db.get(skill.ownerUserId)
if (!owner || owner.deletedAt || owner.deactivatedAt) {
throw new Error('Only the owner can publish updates')
}
const [ownerProviderAccountId, callerProviderAccountId] = await Promise.all([
getGitHubProviderAccountId(ctx, skill.ownerUserId),
getGitHubProviderAccountId(ctx, userId),
])
// Deny healing when GitHub identity isn't present/consistent.
if (
!canHealSkillOwnershipByGitHubProviderAccountId(
ownerProviderAccountId,
callerProviderAccountId,
)
) {
throw new Error('Only the owner can publish updates')
}
await ctx.db.patch(skill._id, { ownerUserId: userId, updatedAt: now })
}
const qualityAssessment = args.qualityAssessment
const isQualityQuarantine = qualityAssessment?.decision === 'quarantine'
const moderationReason = isQualityQuarantine ? 'quality.low' : 'pending.scan'
const moderationNotes = isQualityQuarantine
? `Auto-quarantined by quality gate (score=${qualityAssessment.score}, tier=${qualityAssessment.trustTier}, similar=${qualityAssessment.similarRecentCount}).`
: undefined
const qualityRecord = qualityAssessment
? {
score: qualityAssessment.score,
decision: qualityAssessment.decision,
trustTier: qualityAssessment.trustTier,
similarRecentCount: qualityAssessment.similarRecentCount,
reason: qualityAssessment.reason,
signals: qualityAssessment.signals,
evaluatedAt: now,
}
: undefined
if (!skill) {
const ownerTrustSignals = await getOwnerTrustSignals(ctx, user, now)
enforceNewSkillRateLimit(ownerTrustSignals)
const now = Date.now()
if (!skill) {
const forkOfSlug = args.forkOf?.slug.trim().toLowerCase() || ''
const forkOfVersion = args.forkOf?.version?.trim() || undefined
@@ -2805,7 +3111,7 @@ export const insertVersion = internalMutation({
}
}
const summary = getFrontmatterValue(args.parsed.frontmatter, 'description')
const summary = args.summary ?? getFrontmatterValue(args.parsed.frontmatter, 'description')
const summaryValue = summary ?? undefined
const moderationFlags = deriveModerationFlags({
skill: { slug: args.slug, displayName: args.displayName, summary: summaryValue },
@@ -2829,7 +3135,9 @@ export const insertVersion = internalMutation({
deprecated: undefined,
},
moderationStatus: 'hidden',
moderationReason: 'pending.scan',
moderationReason,
moderationNotes,
quality: qualityRecord,
moderationFlags: moderationFlags.length ? moderationFlags : undefined,
reportCount: 0,
lastReportedAt: undefined,
@@ -2882,7 +3190,8 @@ export const insertVersion = internalMutation({
const latestBefore = skill.latestVersionId
const nextSummary = getFrontmatterValue(args.parsed.frontmatter, 'description') ?? skill.summary
const nextSummary =
args.summary ?? getFrontmatterValue(args.parsed.frontmatter, 'description') ?? skill.summary
const moderationFlags = deriveModerationFlags({
skill: { slug: skill.slug, displayName: args.displayName, summary: nextSummary ?? undefined },
parsed: args.parsed,
@@ -2897,7 +3206,9 @@ export const insertVersion = internalMutation({
stats: { ...skill.stats, versions: skill.stats.versions + 1 },
softDeletedAt: undefined,
moderationStatus: 'hidden',
moderationReason: 'pending.scan',
moderationReason,
moderationNotes,
quality: qualityRecord ?? skill.quality,
moderationFlags: moderationFlags.length ? moderationFlags : undefined,
updatedAt: now,
})
@@ -2949,7 +3260,7 @@ export const setSkillSoftDeletedInternal = internalMutation({
},
handler: async (ctx, args) => {
const user = await ctx.db.get(args.userId)
if (!user || user.deletedAt) throw new Error('User not found')
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
const slug = args.slug.trim().toLowerCase()
if (!slug) throw new Error('Slug required')
+10 -2
View File
@@ -145,6 +145,14 @@ export const getVersionById = query({
handler: async (ctx, args) => ctx.db.get(args.versionId),
})
export const getVersionsByIdsInternal = internalQuery({
args: { versionIds: v.array(v.id('soulVersions')) },
handler: async (ctx, args) => {
const versions = await Promise.all(args.versionIds.map((id) => ctx.db.get(id)))
return versions.filter((v): v is NonNullable<typeof v> => v !== null)
},
})
export const getVersionByIdInternal = internalQuery({
args: { versionId: v.id('soulVersions') },
handler: async (ctx, args) => ctx.db.get(args.versionId),
@@ -386,7 +394,7 @@ export const insertVersion = internalMutation({
handler: async (ctx, args) => {
const userId = args.userId
const user = await ctx.db.get(userId)
if (!user || user.deletedAt) throw new Error('User not found')
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
const soulMatches = await ctx.db
.query('souls')
@@ -508,7 +516,7 @@ export const setSoulSoftDeletedInternal = internalMutation({
},
handler: async (ctx, args) => {
const user = await ctx.db.get(args.userId)
if (!user || user.deletedAt) throw new Error('User not found')
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
const slug = args.slug.trim().toLowerCase()
if (!slug) throw new Error('Slug required')
+21 -3
View File
@@ -1,7 +1,25 @@
{
"extends": "../tsconfig.json",
/* This TypeScript project config describes the environment that
* Convex functions run in and is used to typecheck them.
* You can modify it, but some settings are required to use Convex.
*/
"compilerOptions": {
/* These settings are not required by Convex and can be modified. */
"allowJs": true,
"strict": true,
"moduleResolution": "Bundler",
"skipLibCheck": true
}
"jsx": "react-jsx",
"skipLibCheck": true,
"allowSyntheticDefaultImports": true,
/* These compiler options are required by Convex */
"target": "ESNext",
"lib": ["ES2022", "dom", "dom.iterable"],
"forceConsistentCasingInFileNames": true,
"module": "ESNext",
"isolatedModules": true,
"noEmit": true
},
"include": ["./**/*"],
"exclude": ["./_generated"]
}
+1 -1
View File
@@ -14,7 +14,7 @@ export const generateUploadUrlForUserInternal = internalMutation({
args: { userId: v.id('users') },
handler: async (ctx, args) => {
const user = await ctx.db.get(args.userId)
if (!user || user.deletedAt) throw new Error('User not found')
if (!user || user.deletedAt || user.deactivatedAt) throw new Error('User not found')
return ctx.storage.generateUploadUrl()
},
})
+90
View File
@@ -0,0 +1,90 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
vi.mock('./lib/access', async () => {
const actual = await vi.importActual<typeof import('./lib/access')>('./lib/access')
return { ...actual, requireUser: vi.fn() }
})
const { requireUser } = await import('./lib/access')
const { ensureHandler } = await import('./users')
function makeCtx() {
const patch = vi.fn()
const get = vi.fn()
return { ctx: { db: { patch, get } } as never, patch, get }
}
describe('ensureHandler', () => {
afterEach(() => {
vi.mocked(requireUser).mockReset()
})
it('updates handle and display name when GitHub login changes', async () => {
const { ctx, patch } = makeCtx()
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: {
_creationTime: 1,
handle: 'old-handle',
displayName: 'old-handle',
name: 'new-handle',
email: 'old@example.com',
role: 'user',
createdAt: 1,
},
} as never)
await ensureHandler(ctx)
expect(patch).toHaveBeenCalledWith('users:1', {
handle: 'new-handle',
displayName: 'new-handle',
updatedAt: expect.any(Number),
})
})
it('does not override a custom display name when syncing handle', async () => {
const { ctx, patch } = makeCtx()
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:2',
user: {
_creationTime: 1,
handle: 'old-handle',
displayName: 'Custom Name',
name: 'new-handle',
role: 'user',
createdAt: 1,
},
} as never)
await ensureHandler(ctx)
expect(patch).toHaveBeenCalledWith('users:2', {
handle: 'new-handle',
updatedAt: expect.any(Number),
})
})
it('fills display name from existing handle when missing', async () => {
const { ctx, patch } = makeCtx()
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:3',
user: {
_creationTime: 1,
handle: 'steady-handle',
displayName: undefined,
name: undefined,
email: undefined,
role: 'user',
createdAt: 1,
},
} as never)
await ensureHandler(ctx)
expect(patch).toHaveBeenCalledWith('users:3', {
displayName: 'steady-handle',
updatedAt: expect.any(Number),
})
})
})
+169 -28
View File
@@ -21,6 +21,17 @@ export const getByIdInternal = internalQuery({
handler: async (ctx, args) => ctx.db.get(args.userId),
})
export const getGitHubProviderAccountIdInternal = internalQuery({
args: { userId: v.id('users') },
handler: async (ctx, args) => {
const account = await ctx.db
.query('authAccounts')
.withIndex('userIdAndProvider', (q) => q.eq('userId', args.userId).eq('provider', 'github'))
.unique()
return account?.providerAccountId ?? null
},
})
export const searchInternal = internalQuery({
args: {
actorUserId: v.id('users'),
@@ -29,7 +40,7 @@ export const searchInternal = internalQuery({
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId)
if (!actor || actor.deletedAt) throw new Error('Unauthorized')
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new Error('Unauthorized')
assertAdmin(actor)
const limit = Math.min(Math.max(args.limit ?? 20, 1), 200)
@@ -45,7 +56,6 @@ export const searchInternal = internalQuery({
return { items, total: result.total }
},
})
export const updateGithubMetaInternal = internalMutation({
args: {
userId: v.id('users'),
@@ -67,34 +77,72 @@ export const me = query({
const userId = await getAuthUserId(ctx)
if (!userId) return null
const user = await ctx.db.get(userId)
if (!user || user.deletedAt) return null
if (!user || user.deletedAt || user.deactivatedAt) return null
return user
},
})
export const ensure = mutation({
args: {},
handler: async (ctx) => {
const { userId, user } = await requireUser(ctx)
const updates: Record<string, unknown> = {}
const handle = user.handle || user.name || user.email?.split('@')[0]
if (!user.handle && handle) updates.handle = handle
if (!user.displayName) updates.displayName = handle
if (!user.role) {
updates.role = handle === ADMIN_HANDLE ? 'admin' : DEFAULT_ROLE
}
if (!user.createdAt) updates.createdAt = user._creationTime
if (Object.keys(updates).length > 0) {
updates.updatedAt = Date.now()
await ctx.db.patch(userId, updates)
}
return ctx.db.get(userId)
},
handler: ensureHandler,
})
function normalizeHandle(handle: string | undefined) {
const normalized = handle?.trim()
return normalized ? normalized : undefined
}
function deriveHandle(args: { existingHandle?: string; githubLogin?: string; email?: string }) {
// Prefer the GitHub login; only fall back to email-derived handle when we don't already have one.
if (args.githubLogin) return args.githubLogin
if (!args.existingHandle && args.email) return args.email.split('@')[0]?.trim() || undefined
return undefined
}
function computeEnsureUpdates(user: Doc<'users'>) {
const updates: Record<string, unknown> = {}
const existingHandle = normalizeHandle(user.handle)
const githubLogin = normalizeHandle(user.name)
const derivedHandle = deriveHandle({
existingHandle,
githubLogin,
email: user.email,
})
const baseHandle = derivedHandle ?? existingHandle
if (derivedHandle && existingHandle !== derivedHandle) {
updates.handle = derivedHandle
}
const displayName = normalizeHandle(user.displayName)
if (!displayName && baseHandle) {
updates.displayName = baseHandle
} else if (derivedHandle && displayName === existingHandle) {
updates.displayName = derivedHandle
}
if (!user.role) {
updates.role = baseHandle === ADMIN_HANDLE ? 'admin' : DEFAULT_ROLE
}
if (!user.createdAt) updates.createdAt = user._creationTime
return updates
}
export async function ensureHandler(ctx: MutationCtx) {
const { userId, user } = await requireUser(ctx)
const updates = computeEnsureUpdates(user)
if (Object.keys(updates).length > 0) {
updates.updatedAt = Date.now()
await ctx.db.patch(userId, updates)
}
return ctx.db.get(userId)
}
export const updateProfile = mutation({
args: {
displayName: v.string(),
@@ -114,9 +162,37 @@ export const deleteAccount = mutation({
args: {},
handler: async (ctx) => {
const { userId } = await requireUser(ctx)
const now = Date.now()
const tokens = await ctx.db
.query('apiTokens')
.withIndex('by_user', (q) => q.eq('userId', userId))
.collect()
for (const token of tokens) {
if (!token.revokedAt) {
await ctx.db.patch(token._id, { revokedAt: now })
}
}
await ctx.db.patch(userId, {
deletedAt: Date.now(),
updatedAt: Date.now(),
deactivatedAt: now,
purgedAt: now,
deletedAt: undefined,
banReason: undefined,
role: 'user',
handle: undefined,
displayName: undefined,
name: undefined,
image: undefined,
email: undefined,
emailVerificationTime: undefined,
phone: undefined,
phoneVerificationTime: undefined,
isAnonymous: undefined,
bio: undefined,
githubCreatedAt: undefined,
githubFetchedAt: undefined,
updatedAt: now,
})
await ctx.runMutation(internal.telemetry.clearUserTelemetryInternal, { userId })
},
@@ -165,7 +241,7 @@ export const setRoleInternal = internalMutation({
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId)
if (!actor || actor.deletedAt) throw new Error('User not found')
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new Error('User not found')
return setRoleWithActor(ctx, actor, args.targetUserId, args.role)
},
})
@@ -208,11 +284,32 @@ export const banUserInternal = internalMutation({
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId)
if (!actor || actor.deletedAt) throw new Error('User not found')
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new Error('User not found')
return banUserWithActor(ctx, actor, args.targetUserId, args.reason)
},
})
export const unbanUser = mutation({
args: { userId: v.id('users'), reason: v.optional(v.string()) },
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
return unbanUserWithActor(ctx, user, args.userId, args.reason)
},
})
export const unbanUserInternal = internalMutation({
args: {
actorUserId: v.id('users'),
targetUserId: v.id('users'),
reason: v.optional(v.string()),
},
handler: async (ctx, args) => {
const actor = await ctx.db.get(args.actorUserId)
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new Error('User not found')
return unbanUserWithActor(ctx, actor, args.targetUserId, args.reason)
},
})
async function banUserWithActor(
ctx: MutationCtx,
actor: Doc<'users'>,
@@ -234,7 +331,7 @@ async function banUserWithActor(
if (reason && reason.length > 500) {
throw new Error('Reason too long (max 500 chars)')
}
if (target.deletedAt) {
if (target.deletedAt || target.deactivatedAt) {
return { ok: true as const, alreadyBanned: true, deletedSkills: 0 }
}
@@ -279,6 +376,49 @@ async function banUserWithActor(
return { ok: true as const, alreadyBanned: false, deletedSkills: skills.length }
}
async function unbanUserWithActor(
ctx: MutationCtx,
actor: Doc<'users'>,
targetUserId: Id<'users'>,
reasonRaw?: string,
) {
assertAdmin(actor)
if (targetUserId === actor._id) throw new Error('Cannot unban yourself')
const target = await ctx.db.get(targetUserId)
if (!target) throw new Error('User not found')
if (target.deactivatedAt) {
throw new Error('Cannot unban a permanently deleted account')
}
if (!target.deletedAt) {
return { ok: true as const, alreadyUnbanned: true }
}
const reason = reasonRaw?.trim()
if (reason && reason.length > 500) {
throw new Error('Reason too long (max 500 chars)')
}
const now = Date.now()
await ctx.db.patch(targetUserId, {
deletedAt: undefined,
banReason: undefined,
role: 'user',
updatedAt: now,
})
await ctx.db.insert('auditLogs', {
actorUserId: actor._id,
action: 'user.unban',
targetType: 'user',
targetId: targetUserId,
metadata: { reason: reason || undefined },
createdAt: now,
})
return { ok: true as const, alreadyUnbanned: false }
}
/**
* Auto-ban a user whose skill was flagged malicious by VT.
* Skips moderators/admins. No actor required this is a system-level action.
@@ -292,7 +432,7 @@ export const autobanMalwareAuthorInternal = internalMutation({
handler: async (ctx, args) => {
const target = await ctx.db.get(args.ownerUserId)
if (!target) return { ok: false, reason: 'user_not_found' }
if (target.deletedAt) return { ok: true, alreadyBanned: true }
if (target.deletedAt || target.deactivatedAt) return { ok: true, alreadyBanned: true }
// Never auto-ban moderators or admins
if (target.role === 'admin' || target.role === 'moderator') {
@@ -330,6 +470,7 @@ export const autobanMalwareAuthorInternal = internalMutation({
deletedAt: now,
role: 'user',
updatedAt: now,
banReason: 'malware auto-ban',
})
await ctx.runMutation(internal.telemetry.clearUserTelemetryInternal, {
+12 -17
View File
@@ -375,8 +375,6 @@ export const scanWithVirusTotal = internalAction({
// File exists and has AI analysis - use the verdict
const verdict = normalizeVerdict(aiResult.verdict)
const status = verdictToStatus(verdict)
const isSafe = status === 'clean'
console.log(
`Version ${args.versionId} found in VT with AI analysis. Hash: ${sha256hash}. Verdict: ${verdict}`,
)
@@ -393,14 +391,12 @@ export const scanWithVirusTotal = internalAction({
},
})
// VT is supplementary — only escalate (never override LLM verdict)
if (!isSafe && (status === 'malicious' || status === 'suspicious')) {
await ctx.runMutation(internal.skills.escalateByVtInternal, {
sha256hash,
status,
})
}
// Clean VT result: vtAnalysis already written above — don't touch moderation
// VT finalizes moderation visibility for newly published versions.
await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
sha256hash,
scanner: 'vt',
status,
})
return
}
@@ -578,13 +574,12 @@ export const pollPendingScans = internalAction({
},
})
// VT is supplementary — only escalate for malicious/suspicious
if (status === 'malicious' || status === 'suspicious') {
await ctx.runMutation(internal.skills.escalateByVtInternal, {
sha256hash,
status,
})
}
// VT finalizes moderation visibility for newly published versions.
await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
sha256hash,
scanner: 'vt',
status,
})
updated++
} catch (error) {
console.error(`[vt:pollPendingScans] Error checking hash ${sha256hash}:`, error)
+7
View File
@@ -19,11 +19,17 @@ Enforced per IP + per API key:
- Read: 120/min per IP, 600/min per key
- Write: 30/min per IP, 120/min per key
- Download: 20/min per IP, 120/min per key (`/api/v1/download`)
Headers:
- `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `Retry-After` (when limited)
IP source:
- Uses `cf-connecting-ip` first, then falls back to `x-real-ip`, `x-forwarded-for`, or `fly-client-ip`.
- Set `TRUST_FORWARDED_IPS=false` to disable forwarded-header fallback.
## Public endpoints (no auth)
### `GET /api/v1/search`
@@ -125,6 +131,7 @@ Notes:
- If neither `version` nor `tag` is provided, the latest version is used.
- Soft-deleted versions return `410`.
- Download stats are counted as unique identities per hour (`userId` when API token is valid, otherwise IP).
## Auth endpoints (Bearer token)
+25 -3
View File
@@ -29,6 +29,8 @@ read_when:
- audit log entry: `skill.auto_hide`
- Public queries hide non-active moderation statuses; staff can still access via
staff-only queries and unhide/restore/delete/ban.
- Skills directory supports an optional "Hide suspicious" filter to exclude
active-but-flagged (`flagged.suspicious`) entries from browse/search results.
## Bans
@@ -36,20 +38,40 @@ read_when:
- hard-deletes all owned skills
- revokes API tokens
- sets `deletedAt` on the user
- Admins can manually unban (`deletedAt` + `banReason` cleared); revoked API tokens
stay revoked and should be recreated by the user.
- Optional ban reason is stored in `users.banReason` and audit logs.
- Moderators cannot ban admins; nobody can ban themselves.
- Report counters effectively reset because deleted/banned skills are no longer
considered active in the per-user report cap.
## User account deletion
- User-initiated deletion is irreversible.
- Deletion flow:
- sets `deactivatedAt` + `purgedAt`
- revokes API tokens
- clears profile/contact fields
- clears telemetry
- Deleted accounts cannot be restored by logging in again.
- Published skills remain public.
## Upload gate (GitHub account age)
- Skill + soul publish actions require GitHub account age ≥ 7 days.
- Lookup uses GitHub `created_at` and caches on the user:
- Lookup uses GitHub `created_at` fetched by the immutable GitHub numeric ID (`providerAccountId`)
and caches on the user:
- `githubCreatedAt` (source of truth)
- `githubFetchedAt` (fetch timestamp)
- Cache TTL: 24 hours.
- `githubFetchedAt` (fetch timestamp; set when `githubCreatedAt` is populated)
- Gate applies to web uploads, CLI publish, and GitHub import.
- If GitHub responds `403` or `429`, publish fails with:
- `GitHub API rate limit exceeded — please try again in a few minutes`
- To reduce rate-limit failures, set `GITHUB_TOKEN` in Convex env for authenticated
GitHub API requests.
## Empty-skill cleanup (backfill)
- Cleanup uses quality heuristics plus trust tier to identify very thin/templated
skills.
- Word counting is language-aware (`Intl.Segmenter` with fallback), reducing
false positives for non-space-separated languages.
+1 -1
View File
@@ -162,7 +162,7 @@ Seed data lives in `convex/seed.ts` for local dev.
- Home: search + filters + trending/featured + “Highlighted” badge.
- Skill detail: README render, files list, version history, tags, stats, badges.
- Upload/edit: file picker + version + tag + changelog.
- Account settings: name + delete account (soft delete).
- Account settings: name + delete account (permanent, non-recoverable; published skills stay public).
- Admin: user role management + badge approvals + audit log.
## Testing + quality
+46 -1
View File
@@ -60,7 +60,7 @@ async function makeTempConfig(registry: string, token: string | null) {
async function fetchWithTimeout(input: RequestInfo | URL, init?: RequestInit) {
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const timeout = setTimeout(() => controller.abort(new Error('Timeout')), REQUEST_TIMEOUT_MS)
try {
return await fetch(input, { ...init, signal: controller.signal })
} finally {
@@ -504,4 +504,49 @@ describe('clawhub e2e', () => {
await rm(cfg.dir, { recursive: true, force: true })
}
}, 180_000)
it('delete returns proper error for non-existent skill', async () => {
const registry = process.env.CLAWDHUB_REGISTRY?.trim() || 'https://clawdhub.com'
const site = process.env.CLAWDHUB_SITE?.trim() || 'https://clawdhub.com'
const token = mustGetToken() ?? (await readGlobalConfig())?.token ?? null
if (!token) {
throw new Error('Missing token. Set CLAWDHUB_E2E_TOKEN or run: bun clawdhub auth login')
}
const cfg = await makeTempConfig(registry, token)
const workdir = await mkdtemp(join(tmpdir(), 'clawdhub-e2e-delete-'))
const nonExistentSlug = `non-existent-skill-${Date.now()}`
try {
const del = spawnSync(
'bun',
[
'clawdhub',
'delete',
nonExistentSlug,
'--yes',
'--site',
site,
'--registry',
registry,
'--workdir',
workdir,
],
{
cwd: process.cwd(),
env: { ...process.env, CLAWDHUB_CONFIG_PATH: cfg.path, CLAWDHUB_DISABLE_TELEMETRY: '1' },
encoding: 'utf8',
},
)
// Should fail with non-zero exit code
expect(del.status).not.toBe(0)
// Error should mention "not found" - not generic "Unauthorized"
const output = (del.stdout + del.stderr).toLowerCase()
expect(output).toMatch(/not found|404|does not exist/i)
expect(output).not.toMatch(/unauthorized/i)
} finally {
await rm(workdir, { recursive: true, force: true })
await rm(cfg.dir, { recursive: true, force: true })
}
}, 30_000)
})
+14 -14
View File
@@ -1,28 +1,28 @@
{
"name": "clawhub",
"private": true,
"type": "module",
"workspaces": [
"packages/*"
],
"type": "module",
"scripts": {
"preinstall": "bunx only-allow bun",
"dev": "bun --bun vite dev --port 3000",
"build": "bun --bun vite build",
"preview": "bun --bun vite preview",
"docs:list": "bun scripts/docs-list.ts",
"check:peers": "bun scripts/check-peer-deps.ts",
"convex:deploy": "bunx convex deploy --typecheck=disable --yes",
"coverage": "vitest run --coverage",
"dev": "bun --bun vite dev --port 3000",
"docs:list": "bun scripts/docs-list.ts",
"format": "oxfmt --write",
"lint": "bun run lint:oxlint",
"lint:fix": "oxlint --type-aware --tsconfig ./tsconfig.oxlint.json ./src ./convex ./packages/clawdhub/src ./packages/schema/src --fix && bun run format",
"lint:oxlint": "oxlint --type-aware --tsconfig ./tsconfig.oxlint.json ./src ./convex ./packages/clawdhub/src ./packages/schema/src",
"preinstall": "bunx only-allow bun",
"preview": "bun --bun vite preview",
"test": "vitest run",
"test:watch": "vitest",
"test:e2e": "vitest run -c vitest.e2e.config.ts",
"test:e2e:local": "bash scripts/run-playwright-local.sh",
"test:pw": "playwright test",
"coverage": "vitest run --coverage",
"convex:deploy": "bunx convex deploy --typecheck=disable --yes",
"lint": "bun run lint:biome && bun run lint:oxlint",
"lint:biome": "biome check .",
"lint:oxlint": "oxlint --type-aware --tsconfig ./tsconfig.oxlint.json ./src ./convex ./packages/clawdhub/src ./packages/schema/src",
"format": "biome format --write ."
"test:watch": "vitest"
},
"dependencies": {
"@auth/core": "^0.37.4",
@@ -44,7 +44,6 @@
"clawhub-schema": "workspace:*",
"clsx": "^2.1.1",
"convex": "^1.31.7",
"convex-helpers": "^0.1.111",
"fflate": "^0.8.2",
"h3": "2.0.1-rc.11",
"lucide-react": "^0.563.0",
@@ -61,7 +60,6 @@
"yaml": "^2.8.2"
},
"devDependencies": {
"@biomejs/biome": "^2.3.13",
"@playwright/test": "^1.58.1",
"@tanstack/devtools-vite": "^0.5.0",
"@testing-library/dom": "^10.4.1",
@@ -74,9 +72,11 @@
"@vitest/coverage-v8": "^4.0.18",
"jsdom": "^28.0.0",
"only-allow": "^1.2.2",
"oxfmt": "0.32.0",
"oxlint": "^1.42.0",
"oxlint-tsgolint": "^0.11.4",
"typescript": "^5.9.3",
"undici": "^7.19.2",
"vite": "^7.3.1",
"vitest": "^4.0.18"
}
+14
View File
@@ -0,0 +1,14 @@
import { readGlobalConfig } from '../config.js'
import { fail } from './ui.js'
export async function getOptionalAuthToken(): Promise<string | undefined> {
const cfg = await readGlobalConfig()
return cfg?.token ?? undefined
}
export async function requireAuthToken(): Promise<string> {
const token = await getOptionalAuthToken()
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
+2 -3
View File
@@ -3,6 +3,7 @@ import { readGlobalConfig, writeGlobalConfig } from '../../config.js'
import { discoverRegistryFromSite } from '../../discovery.js'
import { apiRequest } from '../../http.js'
import { ApiRoutes, ApiV1WhoamiResponseSchema } from '../../schema/index.js'
import { requireAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, openInBrowser, promptHidden } from '../ui.js'
@@ -78,9 +79,7 @@ export async function cmdLogout(opts: GlobalOpts) {
}
export async function cmdWhoami(opts: GlobalOpts) {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner('Checking token')
@@ -3,8 +3,8 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { GlobalOpts } from '../types'
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
vi.mock('../authToken.js', () => ({
requireAuthToken: vi.fn(async () => 'tkn'),
}))
vi.mock('../registry.js', () => ({
+3 -10
View File
@@ -1,6 +1,6 @@
import { readGlobalConfig } from '../../config.js'
import { apiRequest } from '../../http.js'
import { ApiRoutes, ApiV1DeleteResponseSchema, parseArk } from '../../schema/index.js'
import { requireAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
@@ -40,13 +40,6 @@ const unhideLabels: SkillActionLabels = {
promptSuffix: 'requires moderator/admin',
}
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
export async function cmdDeleteSkill(
opts: GlobalOpts,
slugArg: string,
@@ -64,7 +57,7 @@ export async function cmdDeleteSkill(
if (!ok) return
}
const token = await requireToken()
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner(`${labels.progress} ${slug}`)
try {
@@ -98,7 +91,7 @@ export async function cmdUndeleteSkill(
if (!ok) return
}
const token = await requireToken()
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner(`${labels.progress} ${slug}`)
try {
@@ -16,6 +16,11 @@ vi.mock('../registry.js', () => ({
getRegistry: () => mockGetRegistry(),
}))
const mockGetOptionalAuthToken = vi.fn(async () => undefined as string | undefined)
vi.mock('../authToken.js', () => ({
getOptionalAuthToken: () => mockGetOptionalAuthToken(),
}))
const mockSpinner = {
stop: vi.fn(),
fail: vi.fn(),
@@ -5,6 +5,7 @@ import {
ApiV1SkillVersionListResponseSchema,
ApiV1SkillVersionResponseSchema,
} from '../../schema/index.js'
import { getOptionalAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError } from '../ui.js'
@@ -31,12 +32,13 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
if (!trimmed) fail('Slug required')
if (options.version && options.tag) fail('Use either --version or --tag')
const token = await getOptionalAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner('Fetching skill')
try {
const skillResult = await apiRequest(
registry,
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(trimmed)}` },
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(trimmed)}`, token },
ApiV1SkillResponseSchema,
)
@@ -67,6 +69,7 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
path: `${ApiRoutes.skills}/${encodeURIComponent(trimmed)}/versions/${encodeURIComponent(
targetVersion,
)}`,
token,
},
ApiV1SkillVersionResponseSchema,
)
@@ -80,7 +83,7 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
spinner.text = `Fetching versions (${limit})`
versionsList = await apiRequest(
registry,
{ method: 'GET', url: url.toString() },
{ method: 'GET', url: url.toString(), token },
ApiV1SkillVersionListResponseSchema,
)
}
@@ -97,7 +100,7 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
url.searchParams.set('version', latestVersion)
}
spinner.text = `Fetching ${options.file}`
fileContent = await fetchText(registry, { url: url.toString() })
fileContent = await fetchText(registry, { url: url.toString(), token })
}
spinner.stop()
@@ -3,8 +3,8 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { GlobalOpts } from '../types'
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
vi.mock('../authToken.js', () => ({
requireAuthToken: vi.fn(async () => 'tkn'),
}))
vi.mock('../registry.js', () => ({
@@ -1,5 +1,4 @@
import { isCancel, select } from '@clack/prompts'
import { readGlobalConfig } from '../../config.js'
import { apiRequest } from '../../http.js'
import {
ApiRoutes,
@@ -8,17 +7,11 @@ import {
ApiV1UserSearchResponseSchema,
parseArk,
} from '../../schema/index.js'
import { requireAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
export async function cmdBanUser(
opts: GlobalOpts,
identifierArg: string,
@@ -30,7 +23,7 @@ export async function cmdBanUser(
const reason = options.reason?.trim() || undefined
const token = await requireToken()
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const allowPrompt = isInteractive() && inputAllowed !== false
const resolved = await resolveUserIdentifier(
@@ -87,7 +80,7 @@ export async function cmdSetRole(
if (!raw) fail('Handle or user id required')
const role = normalizeRole(roleArg)
const token = await requireToken()
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const allowPrompt = isInteractive() && inputAllowed !== false
const resolved = await resolveUserIdentifier(
@@ -6,8 +6,8 @@ import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { GlobalOpts } from '../types'
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
vi.mock('../authToken.js', () => ({
requireAuthToken: vi.fn(async () => 'tkn'),
}))
const mockGetRegistry = vi.fn(async (_opts: unknown, _params?: unknown) => 'https://clawhub.ai')
@@ -1,10 +1,10 @@
import { stat } from 'node:fs/promises'
import { basename, resolve } from 'node:path'
import semver from 'semver'
import { readGlobalConfig } from '../../config.js'
import { apiRequestForm } from '../../http.js'
import { ApiRoutes, ApiV1PublishResponseSchema } from '../../schema/index.js'
import { listTextFiles } from '../../skills.js'
import { requireAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import { sanitizeSlug, titleCase } from '../slug.js'
import type { GlobalOpts } from '../types.js'
@@ -27,9 +27,7 @@ export async function cmdPublish(
const folderStat = await stat(folder).catch(() => null)
if (!folderStat || !folderStat.isDirectory()) fail('Path must be a folder')
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const slug = options.slug ?? sanitizeSlug(basename(folder))
@@ -16,6 +16,11 @@ vi.mock('../registry.js', () => ({
getRegistry: () => mockGetRegistry(),
}))
const mockGetOptionalAuthToken = vi.fn(async () => undefined as string | undefined)
vi.mock('../authToken.js', () => ({
getOptionalAuthToken: () => mockGetOptionalAuthToken(),
}))
const mockSpinner = {
stop: vi.fn(),
fail: vi.fn(),
@@ -50,7 +55,7 @@ vi.mock('node:fs/promises', () => ({
stat: vi.fn(),
}))
const { clampLimit, cmdExplore, cmdUpdate, formatExploreLine } = await import('./skills')
const { clampLimit, cmdExplore, cmdInstall, cmdUpdate, formatExploreLine } = await import('./skills')
const {
extractZipToDir,
hashSkillFiles,
@@ -189,3 +194,29 @@ describe('cmdUpdate', () => {
expect(args?.url).toBeUndefined()
})
})
describe('cmdInstall', () => {
it('passes optional auth token to API + download requests', async () => {
mockGetOptionalAuthToken.mockResolvedValue('tkn')
mockApiRequest.mockResolvedValue({
skill: { slug: 'demo', displayName: 'Demo', summary: null, tags: {}, stats: {}, createdAt: 0, updatedAt: 0 },
latestVersion: { version: '1.0.0' },
owner: null,
moderation: null,
})
mockDownloadZip.mockResolvedValue(new Uint8Array([1, 2, 3]))
vi.mocked(readLockfile).mockResolvedValue({ version: 1, skills: {} })
vi.mocked(writeLockfile).mockResolvedValue()
vi.mocked(writeSkillOrigin).mockResolvedValue()
vi.mocked(extractZipToDir).mockResolvedValue()
vi.mocked(stat).mockRejectedValue(new Error('missing'))
vi.mocked(rm).mockResolvedValue()
await cmdInstall(makeOpts(), 'demo')
const [, requestArgs] = mockApiRequest.mock.calls[0] ?? []
expect(requestArgs?.token).toBe('tkn')
const [, zipArgs] = mockDownloadZip.mock.calls[0] ?? []
expect(zipArgs?.token).toBe('tkn')
})
})
+12 -7
View File
@@ -21,6 +21,7 @@ import {
import { getRegistry } from '../registry.js'
import type { GlobalOpts, ResolveResult } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
import { getOptionalAuthToken } from '../authToken.js'
export async function cmdSearch(opts: GlobalOpts, query: string, limit?: number) {
if (!query) fail('Query required')
@@ -61,6 +62,8 @@ export async function cmdInstall(
const trimmed = slug.trim()
if (!trimmed) fail('Slug required')
const token = await getOptionalAuthToken()
const registry = await getRegistry(opts, { cache: true })
await mkdir(opts.dir, { recursive: true })
const target = join(opts.dir, trimmed)
@@ -76,7 +79,7 @@ export async function cmdInstall(
// Fetch skill metadata including moderation status
const skillMeta = await apiRequest(
registry,
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(trimmed)}` },
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(trimmed)}`, token },
ApiV1SkillResponseSchema,
)
@@ -106,7 +109,7 @@ export async function cmdInstall(
if (!resolvedVersion) fail('Could not resolve latest version')
spinner.text = `Downloading ${trimmed}@${resolvedVersion}`
const zip = await downloadZip(registry, { slug: trimmed, version: resolvedVersion })
const zip = await downloadZip(registry, { slug: trimmed, version: resolvedVersion, token })
await extractZipToDir(zip, target)
await writeSkillOrigin(target, {
@@ -144,6 +147,8 @@ export async function cmdUpdate(
if (options.version && !semver.valid(options.version)) fail('--version must be valid semver')
const allowPrompt = isInteractive() && inputAllowed !== false
const token = await getOptionalAuthToken()
const registry = await getRegistry(opts, { cache: true })
const lock = await readLockfile(opts.workdir)
const slugs = slug ? [slug] : Object.keys(lock.skills)
@@ -161,7 +166,7 @@ export async function cmdUpdate(
// Always fetch skill metadata to check moderation status
const skillMeta = await apiRequest(
registry,
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(entry)}` },
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(entry)}`, token },
ApiV1SkillResponseSchema,
)
@@ -202,7 +207,7 @@ export async function cmdUpdate(
let resolveResult: ResolveResult
if (localFingerprint) {
resolveResult = await resolveSkillVersion(registry, entry, localFingerprint)
resolveResult = await resolveSkillVersion(registry, entry, localFingerprint, token)
} else {
resolveResult = { match: null, latestVersion: skillMeta.latestVersion ?? null }
}
@@ -255,7 +260,7 @@ export async function cmdUpdate(
spinner.start(`Updating ${entry} -> ${targetVersion}`)
}
await rm(target, { recursive: true, force: true })
const zip = await downloadZip(registry, { slug: entry, version: targetVersion })
const zip = await downloadZip(registry, { slug: entry, version: targetVersion, token })
await extractZipToDir(zip, target)
const existingOrigin = await readSkillOrigin(target)
@@ -407,13 +412,13 @@ function resolveExploreSort(raw?: string): { sort: ExploreSort; apiSort: ApiExpl
)
}
async function resolveSkillVersion(registry: string, slug: string, hash: string) {
async function resolveSkillVersion(registry: string, slug: string, hash: string, token?: string) {
const url = new URL(ApiRoutes.resolve, registry)
url.searchParams.set('slug', slug)
url.searchParams.set('hash', hash)
return apiRequest(
registry,
{ method: 'GET', url: url.toString() },
{ method: 'GET', url: url.toString(), token },
ApiV1SkillResolveResponseSchema,
)
}
+2 -9
View File
@@ -1,17 +1,10 @@
import { readGlobalConfig } from '../../config.js'
import { apiRequest } from '../../http.js'
import { ApiRoutes, ApiV1StarResponseSchema } from '../../schema/index.js'
import { requireAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
export async function cmdStarSkill(
opts: GlobalOpts,
slugArg: string,
@@ -28,7 +21,7 @@ export async function cmdStarSkill(
if (!ok) return
}
const token = await requireToken()
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner(`Starring ${slug}`)
try {
@@ -26,8 +26,8 @@ vi.mock('@clack/prompts', () => ({
isCancel: () => false,
}))
vi.mock('../../config.js', () => ({
readGlobalConfig: vi.fn(async () => ({ registry: 'https://clawhub.ai', token: 'tkn' })),
vi.mock('../authToken.js', () => ({
requireAuthToken: vi.fn(async () => 'tkn'),
}))
const mockGetRegistry = vi.fn(async () => 'https://clawhub.ai')
+3 -5
View File
@@ -1,7 +1,7 @@
import { intro, outro } from '@clack/prompts'
import { readGlobalConfig } from '../../config.js'
import { hashSkillFiles, listTextFiles, readSkillOrigin } from '../../skills.js'
import { resolveClawdbotSkillRoots } from '../clawdbotConfig.js'
import { requireAuthToken } from '../authToken.js'
import { getFallbackSkillRoots } from '../scanSkills.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive } from '../ui.js'
@@ -32,9 +32,7 @@ export async function cmdSync(opts: GlobalOpts, options: SyncOptions, inputAllow
const allowPrompt = isInteractive() && inputAllowed !== false
intro('ClawHub sync')
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
const token = await requireAuthToken()
const registry = await getRegistryWithAuth(opts, token)
const selectedRoots = buildScanRoots(opts, options.root)
@@ -109,7 +107,7 @@ export async function cmdSync(opts: GlobalOpts, options: SyncOptions, inputAllow
let done = 0
const resolved = await mapWithConcurrency(locals, Math.min(concurrency, 16), async (skill) => {
try {
return await checkRegistrySyncState(registry, skill, resolveSupport)
return await checkRegistrySyncState(registry, skill, resolveSupport, token)
} finally {
done += 1
candidatesSpinner.text = `Checking registry sync state ${done}/${locals.length}`
@@ -100,6 +100,7 @@ export async function checkRegistrySyncState(
registry: string,
skill: LocalSkill,
resolveSupport: { value: boolean | null },
token?: string,
): Promise<Candidate> {
if (resolveSupport.value !== false) {
try {
@@ -108,6 +109,7 @@ export async function checkRegistrySyncState(
{
method: 'GET',
path: `${ApiRoutes.resolve}?slug=${encodeURIComponent(skill.slug)}&hash=${encodeURIComponent(skill.fingerprint)}`,
token,
},
ApiV1SkillResolveResponseSchema,
)
@@ -149,7 +151,7 @@ export async function checkRegistrySyncState(
const meta = await apiRequest(
registry,
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(skill.slug)}` },
{ method: 'GET', path: `${ApiRoutes.skills}/${encodeURIComponent(skill.slug)}`, token },
ApiV1SkillResponseSchema,
).catch(() => null)
@@ -163,7 +165,7 @@ export async function checkRegistrySyncState(
}
}
const zip = await downloadZip(registry, { slug: skill.slug, version: latestVersion })
const zip = await downloadZip(registry, { slug: skill.slug, version: latestVersion, token })
const remote = hashSkillZip(zip).fingerprint
const matchVersion = remote === skill.fingerprint ? latestVersion : null
+2 -9
View File
@@ -1,17 +1,10 @@
import { readGlobalConfig } from '../../config.js'
import { apiRequest } from '../../http.js'
import { ApiRoutes, ApiV1UnstarResponseSchema } from '../../schema/index.js'
import { requireAuthToken } from '../authToken.js'
import { getRegistry } from '../registry.js'
import type { GlobalOpts } from '../types.js'
import { createSpinner, fail, formatError, isInteractive, promptConfirm } from '../ui.js'
async function requireToken() {
const cfg = await readGlobalConfig()
const token = cfg?.token
if (!token) fail('Not logged in. Run: clawhub login')
return token
}
export async function cmdUnstarSkill(
opts: GlobalOpts,
slugArg: string,
@@ -28,7 +21,7 @@ export async function cmdUnstarSkill(
if (!ok) return
}
const token = await requireToken()
const token = await requireAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner(`Unstarring ${slug}`)
try {
+80 -3
View File
@@ -1,9 +1,41 @@
/* @vitest-environment node */
import { describe, expect, it, vi } from 'vitest'
import { apiRequest, apiRequestForm, downloadZip } from './http'
import { apiRequest, apiRequestForm, downloadZip, fetchText } from './http'
import { ApiV1WhoamiResponseSchema } from './schema/index.js'
function mockImmediateTimeouts() {
const setTimeoutMock = vi.fn((callback: () => void) => {
callback()
return 1 as unknown as ReturnType<typeof setTimeout>
})
const clearTimeoutMock = vi.fn()
vi.stubGlobal('setTimeout', setTimeoutMock as unknown as typeof setTimeout)
vi.stubGlobal('clearTimeout', clearTimeoutMock as typeof clearTimeout)
return { setTimeoutMock, clearTimeoutMock }
}
function createAbortingFetchMock() {
return vi.fn(async (_url: string, init?: RequestInit) => {
const signal = init?.signal
if (!signal || !(signal instanceof AbortSignal)) {
throw new Error('Missing abort signal')
}
if (signal.aborted) {
throw signal.reason
}
return await new Promise<Response>((_resolve, reject) => {
signal.addEventListener(
'abort',
() => {
reject(signal.reason)
},
{ once: true },
)
})
})
}
describe('apiRequest', () => {
it('adds bearer token and parses json', async () => {
const fetchMock = vi.fn().mockResolvedValue({
@@ -73,11 +105,16 @@ describe('apiRequest', () => {
arrayBuffer: async () => new Uint8Array([1, 2, 3]).buffer,
})
vi.stubGlobal('fetch', fetchMock)
const bytes = await downloadZip('https://example.com', { slug: 'demo', version: '1.0.0' })
const bytes = await downloadZip('https://example.com', {
slug: 'demo',
version: '1.0.0',
token: 'clh_token',
})
expect(Array.from(bytes)).toEqual([1, 2, 3])
const [url] = fetchMock.mock.calls[0] as [string]
const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit]
expect(url).toContain('slug=demo')
expect(url).toContain('version=1.0.0')
expect((init.headers as Record<string, string>).Authorization).toBe('Bearer clh_token')
vi.unstubAllGlobals()
})
@@ -92,6 +129,25 @@ describe('apiRequest', () => {
expect(fetchMock).toHaveBeenCalledTimes(1)
vi.unstubAllGlobals()
})
it('aborts with Error timeouts and retries', async () => {
const { clearTimeoutMock } = mockImmediateTimeouts()
const fetchMock = createAbortingFetchMock()
vi.stubGlobal('fetch', fetchMock)
let caught: unknown
try {
await apiRequest('https://example.com', { method: 'GET', path: '/x' })
} catch (error) {
caught = error
}
expect(caught).toBeInstanceOf(Error)
expect((caught as Error).message).toBe('Timeout')
expect(fetchMock).toHaveBeenCalledTimes(3)
expect(clearTimeoutMock.mock.calls.length).toBeGreaterThanOrEqual(3)
vi.unstubAllGlobals()
})
})
describe('apiRequestForm', () => {
@@ -154,3 +210,24 @@ describe('apiRequestForm', () => {
vi.unstubAllGlobals()
})
})
describe('fetchText', () => {
it('aborts with Error timeouts and retries', async () => {
const { clearTimeoutMock } = mockImmediateTimeouts()
const fetchMock = createAbortingFetchMock()
vi.stubGlobal('fetch', fetchMock)
let caught: unknown
try {
await fetchText('https://example.com', { path: '/x' })
} catch (error) {
caught = error
}
expect(caught).toBeInstanceOf(Error)
expect((caught as Error).message).toBe('Timeout')
expect(fetchMock).toHaveBeenCalledTimes(3)
expect(clearTimeoutMock.mock.calls.length).toBeGreaterThanOrEqual(3)
vi.unstubAllGlobals()
})
})
+48 -56
View File
@@ -52,22 +52,13 @@ export async function apiRequest<T>(
headers['Content-Type'] = 'application/json'
body = JSON.stringify(args.body ?? {})
}
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url, {
const response = await fetchWithTimeout(url, {
method: args.method,
headers,
body,
signal: controller.signal,
})
clearTimeout(timeout)
if (!response.ok) {
const text = await response.text().catch(() => '')
const message = text || `HTTP ${response.status}`
if (response.status === 429 || response.status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
throwHttpStatusError(response.status, await readResponseTextSafe(response))
}
return (await response.json()) as unknown
},
@@ -101,22 +92,13 @@ export async function apiRequestForm<T>(
const headers: Record<string, string> = { Accept: 'application/json' }
if (args.token) headers.Authorization = `Bearer ${args.token}`
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url, {
const response = await fetchWithTimeout(url, {
method: args.method,
headers,
body: args.form,
signal: controller.signal,
})
clearTimeout(timeout)
if (!response.ok) {
const text = await response.text().catch(() => '')
const message = text || `HTTP ${response.status}`
if (response.status === 429 || response.status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
throwHttpStatusError(response.status, await readResponseTextSafe(response))
}
return (await response.json()) as unknown
},
@@ -138,17 +120,10 @@ export async function fetchText(registry: string, args: TextRequestArgs): Promis
const headers: Record<string, string> = { Accept: 'text/plain' }
if (args.token) headers.Authorization = `Bearer ${args.token}`
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url, { method: 'GET', headers, signal: controller.signal })
clearTimeout(timeout)
const response = await fetchWithTimeout(url, { method: 'GET', headers })
const text = await response.text()
if (!response.ok) {
const message = text || `HTTP ${response.status}`
if (response.status === 429 || response.status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
throwHttpStatusError(response.status, text)
}
return text
},
@@ -156,26 +131,25 @@ export async function fetchText(registry: string, args: TextRequestArgs): Promis
)
}
export async function downloadZip(registry: string, args: { slug: string; version?: string }) {
export async function downloadZip(
registry: string,
args: { slug: string; version?: string; token?: string },
) {
const url = new URL(ApiRoutes.download, registry)
url.searchParams.set('slug', args.slug)
if (args.version) url.searchParams.set('version', args.version)
return pRetry(
async () => {
if (isBun) {
return await fetchBinaryViaCurl(url.toString())
return await fetchBinaryViaCurl(url.toString(), args.token)
}
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort('Timeout'), REQUEST_TIMEOUT_MS)
const response = await fetch(url.toString(), { method: 'GET', signal: controller.signal })
clearTimeout(timeout)
const headers: Record<string, string> = {}
if (args.token) headers.Authorization = `Bearer ${args.token}`
const response = await fetchWithTimeout(url.toString(), { method: 'GET', headers })
if (!response.ok) {
const message = (await response.text().catch(() => '')) || `HTTP ${response.status}`
if (response.status === 429 || response.status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
throwHttpStatusError(response.status, await readResponseTextSafe(response))
}
return new Uint8Array(await response.arrayBuffer())
},
@@ -183,6 +157,28 @@ export async function downloadZip(registry: string, args: { slug: string; versio
)
}
async function fetchWithTimeout(url: string, init: RequestInit): Promise<Response> {
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(new Error('Timeout')), REQUEST_TIMEOUT_MS)
try {
return await fetch(url, { ...init, signal: controller.signal })
} finally {
clearTimeout(timeout)
}
}
async function readResponseTextSafe(response: Response): Promise<string> {
return await response.text().catch(() => '')
}
function throwHttpStatusError(status: number, text: string): never {
const message = text || `HTTP ${status}`
if (status === 429 || status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
}
async function fetchJsonViaCurl(url: string, args: RequestArgs) {
const headers = ['-H', 'Accept: application/json']
if (args.token) {
@@ -217,10 +213,7 @@ async function fetchJsonViaCurl(url: string, args: RequestArgs) {
const status = Number(output.slice(splitAt + 1).trim())
if (!Number.isFinite(status)) throw new Error('curl response missing status')
if (status < 200 || status >= 300) {
if (status === 429 || status >= 500) {
throw new Error(body || `HTTP ${status}`)
}
throw new AbortError(body || `HTTP ${status}`)
throwHttpStatusError(status, body)
}
return JSON.parse(body || 'null') as unknown
}
@@ -272,10 +265,7 @@ async function fetchJsonFormViaCurl(url: string, args: FormRequestArgs) {
const status = Number(output.slice(splitAt + 1).trim())
if (!Number.isFinite(status)) throw new Error('curl response missing status')
if (status < 200 || status >= 300) {
if (status === 429 || status >= 500) {
throw new Error(body || `HTTP ${status}`)
}
throw new AbortError(body || `HTTP ${status}`)
throwHttpStatusError(status, body)
}
return JSON.parse(body || 'null') as unknown
} finally {
@@ -320,16 +310,22 @@ async function fetchTextViaCurl(url: string, args: { token?: string }) {
return body
}
async function fetchBinaryViaCurl(url: string) {
async function fetchBinaryViaCurl(url: string, token?: string) {
const tempDir = await mkdtemp(join(tmpdir(), 'clawhub-download-'))
const filePath = join(tempDir, 'payload.bin')
try {
const headers: string[] = []
if (token) {
headers.push('-H', `Authorization: Bearer ${token}`)
}
const curlArgs = [
'--silent',
'--show-error',
'--location',
'--max-time',
String(REQUEST_TIMEOUT_SECONDS),
...headers,
'-o',
filePath,
'--write-out',
@@ -344,11 +340,7 @@ async function fetchBinaryViaCurl(url: string) {
if (!Number.isFinite(status)) throw new Error('curl response missing status')
if (status < 200 || status >= 300) {
const body = await readFileSafe(filePath)
const message = body ? new TextDecoder().decode(body) : `HTTP ${status}`
if (status === 429 || status >= 500) {
throw new Error(message)
}
throw new AbortError(message)
throwHttpStatusError(status, body ? new TextDecoder().decode(body) : '')
}
const bytes = await readFileSafe(filePath)
return bytes ? new Uint8Array(bytes) : new Uint8Array()
+6 -3
View File
@@ -20,15 +20,18 @@ import {
describe('skills', () => {
it('extracts zip into directory and skips traversal', async () => {
const dir = await mkdtemp(join(tmpdir(), 'clawhub-'))
const parent = await mkdtemp(join(tmpdir(), 'clawhub-zip-'))
const dir = join(parent, 'dir')
await mkdir(dir)
const evilName = `evil-${Date.now()}-${Math.random().toString(16).slice(2)}.txt`
const zip = zipSync({
'SKILL.md': strToU8('hello'),
'../evil.txt': strToU8('nope'),
[`../${evilName}`]: strToU8('nope'),
})
await extractZipToDir(new Uint8Array(zip), dir)
expect((await readFile(join(dir, 'SKILL.md'), 'utf8')).trim()).toBe('hello')
await expect(stat(join(dir, '..', 'evil.txt'))).rejects.toBeTruthy()
await expect(stat(join(parent, evilName))).rejects.toBeTruthy()
})
it('writes and reads lockfile', async () => {
+24 -3
View File
@@ -7,17 +7,20 @@ vi.mock('@tanstack/react-router', () => ({
import { Route } from '../routes/search'
function runBeforeLoad(search: { q?: string; highlighted?: boolean }, hostname = 'clawdhub.com') {
function runBeforeLoad(
search: { q?: string; highlighted?: boolean; nonSuspicious?: boolean },
hostname = 'clawdhub.com',
) {
const route = Route as unknown as {
__config: {
beforeLoad?: (args: {
search: { q?: string; highlighted?: boolean }
search: { q?: string; highlighted?: boolean; nonSuspicious?: boolean }
location: { url: URL }
}) => void
}
}
const beforeLoad = route.__config.beforeLoad as (args: {
search: { q?: string; highlighted?: boolean }
search: { q?: string; highlighted?: boolean; nonSuspicious?: boolean }
location: { url: URL }
}) => void
let thrown: unknown
@@ -41,6 +44,24 @@ describe('search route', () => {
sort: undefined,
dir: undefined,
highlighted: true,
nonSuspicious: undefined,
view: undefined,
},
replace: true,
},
})
})
it('forwards nonSuspicious filter to skills index', () => {
expect(runBeforeLoad({ q: 'crab', nonSuspicious: true }, 'clawdhub.com')).toEqual({
redirect: {
to: '/skills',
search: {
q: 'crab',
sort: undefined,
dir: undefined,
highlighted: undefined,
nonSuspicious: true,
view: undefined,
},
replace: true,
+8 -5
View File
@@ -1,4 +1,4 @@
import { render, screen, waitFor } from '@testing-library/react'
import { fireEvent, render, screen, waitFor } from '@testing-library/react'
import { vi } from 'vitest'
import { SkillDetailPage } from '../components/SkillDetailPage'
@@ -94,7 +94,7 @@ describe('SkillDetailPage', () => {
})
})
it('shows report abuse note for authenticated users', async () => {
it('opens report dialog for authenticated users', async () => {
useAuthStatusMock.mockReturnValue({
isAuthenticated: true,
isLoading: false,
@@ -123,8 +123,11 @@ describe('SkillDetailPage', () => {
render(<SkillDetailPage slug="weather" />)
expect(
await screen.findByText(/Reports require a reason\. Abuse may result in a ban\./i),
).toBeTruthy()
expect(screen.queryByText(/Reports require a reason\. Abuse may result in a ban\./i)).toBeNull()
fireEvent.click(await screen.findByRole('button', { name: /report/i }))
expect(await screen.findByRole('dialog')).toBeTruthy()
expect(screen.getByText(/Report skill/i)).toBeTruthy()
})
})
@@ -15,14 +15,12 @@ vi.mock('@tanstack/react-router', () => ({
useNavigate: () => navigateMock,
useSearch: () => searchMock,
}),
redirect: (options: unknown) => ({ redirect: options }),
Link: (props: { children: ReactNode }) => <a href="/">{props.children}</a>,
}))
vi.mock('convex/react', () => ({
useAction: (...args: unknown[]) => useActionMock(...args),
}))
vi.mock('convex-helpers/react', () => ({
usePaginatedQuery: (...args: unknown[]) => usePaginatedQueryMock(...args),
}))
+72 -5
View File
@@ -15,14 +15,12 @@ vi.mock('@tanstack/react-router', () => ({
useNavigate: () => navigateMock,
useSearch: () => searchMock,
}),
redirect: (options: unknown) => ({ redirect: options }),
Link: (props: { children: ReactNode }) => <a href="/">{props.children}</a>,
}))
vi.mock('convex/react', () => ({
useAction: (...args: unknown[]) => useActionMock(...args),
}))
vi.mock('convex-helpers/react', () => ({
usePaginatedQuery: (...args: unknown[]) => usePaginatedQueryMock(...args),
}))
@@ -48,10 +46,10 @@ describe('SkillsIndex', () => {
it('requests the first skills page', () => {
render(<SkillsIndex />)
// usePaginatedQuery should be called with the API endpoint and empty args
// usePaginatedQuery should be called with the API endpoint and sort/dir args
expect(usePaginatedQueryMock).toHaveBeenCalledWith(
expect.anything(),
{},
{ sort: 'downloads', dir: 'desc', nonSuspiciousOnly: false },
{ initialNumItems: 25 },
)
})
@@ -79,6 +77,7 @@ describe('SkillsIndex', () => {
expect(actionFn).toHaveBeenCalledWith({
query: 'remind',
highlightedOnly: false,
nonSuspiciousOnly: false,
limit: 25,
})
await act(async () => {
@@ -87,6 +86,7 @@ describe('SkillsIndex', () => {
expect(actionFn).toHaveBeenCalledWith({
query: 'remind',
highlightedOnly: false,
nonSuspiciousOnly: false,
limit: 25,
})
})
@@ -115,10 +115,37 @@ describe('SkillsIndex', () => {
expect(actionFn).toHaveBeenLastCalledWith({
query: 'remind',
highlightedOnly: false,
nonSuspiciousOnly: false,
limit: 50,
})
})
it('sorts search results by stars and breaks ties by updatedAt', async () => {
searchMock = { q: 'remind', sort: 'stars', dir: 'desc' }
const actionFn = vi
.fn()
.mockResolvedValue([
makeSearchEntry({ slug: 'skill-a', displayName: 'Skill A', stars: 5, updatedAt: 100 }),
makeSearchEntry({ slug: 'skill-b', displayName: 'Skill B', stars: 5, updatedAt: 200 }),
makeSearchEntry({ slug: 'skill-c', displayName: 'Skill C', stars: 4, updatedAt: 999 }),
])
useActionMock.mockReturnValue(actionFn)
vi.useFakeTimers()
render(<SkillsIndex />)
await act(async () => {
await vi.runAllTimersAsync()
})
await act(async () => {
await vi.runAllTimersAsync()
})
const links = screen.getAllByRole('link')
expect(links[0]?.textContent).toContain('Skill B')
expect(links[1]?.textContent).toContain('Skill A')
expect(links[2]?.textContent).toContain('Skill C')
})
it('uses relevance as default sort when searching', async () => {
searchMock = { q: 'notion' }
const actionFn = vi
@@ -142,6 +169,17 @@ describe('SkillsIndex', () => {
expect(titles[0]).toBe('Older High Score')
expect(titles[1]).toBe('Newer Low Score')
})
it('passes nonSuspiciousOnly to list query when filter is active', () => {
searchMock = { nonSuspicious: true }
render(<SkillsIndex />)
expect(usePaginatedQueryMock).toHaveBeenCalledWith(
expect.anything(),
{ sort: 'downloads', dir: 'desc', nonSuspiciousOnly: true },
{ initialNumItems: 25 },
)
})
})
function makeSearchResults(count: number) {
@@ -191,3 +229,32 @@ function makeSearchResult(slug: string, displayName: string, score: number, crea
version: null,
}
}
function makeSearchEntry(params: {
slug: string
displayName: string
stars: number
updatedAt: number
}) {
return {
score: 0.9,
skill: {
_id: `skill_${params.slug}`,
slug: params.slug,
displayName: params.displayName,
summary: `Summary ${params.slug}`,
tags: {},
stats: {
downloads: 0,
installsCurrent: 0,
installsAllTime: 0,
stars: params.stars,
versions: 1,
comments: 0,
},
createdAt: 0,
updatedAt: params.updatedAt,
},
version: null,
}
}
@@ -0,0 +1,59 @@
import { describe, expect, it, vi } from 'vitest'
vi.mock('@tanstack/react-router', () => ({
createFileRoute:
() =>
(config: {
beforeLoad?: (args: { search: Record<string, unknown> }) => void
component?: unknown
validateSearch?: unknown
}) => ({ __config: config }),
redirect: (options: unknown) => ({ redirect: options }),
Link: () => null,
}))
import { Route } from '../routes/skills/index'
function runBeforeLoad(search: Record<string, unknown>) {
const route = Route as unknown as {
__config: {
beforeLoad?: (args: { search: Record<string, unknown> }) => void
}
}
const beforeLoad = route.__config.beforeLoad as (args: {
search: Record<string, unknown>
}) => void
let thrown: unknown
try {
beforeLoad({ search })
} catch (error) {
thrown = error
}
return thrown
}
describe('skills route default sort', () => {
it('redirects browse view to downloads when sort is missing', () => {
expect(runBeforeLoad({ nonSuspicious: true })).toEqual({
redirect: {
to: '/skills',
search: {
q: undefined,
sort: 'downloads',
dir: undefined,
highlighted: undefined,
nonSuspicious: true,
view: undefined,
focus: undefined,
},
replace: true,
},
})
})
it('does not redirect when query is present', () => {
expect(runBeforeLoad({ q: 'notion' })).toBeUndefined()
})
})
+16 -1
View File
@@ -31,6 +31,7 @@ export default function Header() {
const handle = me?.handle ?? me?.displayName ?? 'user'
const initial = (me?.displayName ?? me?.name ?? handle).charAt(0).toUpperCase()
const isStaff = isModerator(me)
const signInRedirectTo = getCurrentRelativeUrl()
const setTheme = (next: 'system' | 'light' | 'dark') => {
startThemeTransition({
@@ -81,6 +82,7 @@ export default function Header() {
sort: undefined,
dir: undefined,
highlighted: undefined,
nonSuspicious: undefined,
view: undefined,
focus: undefined,
}}
@@ -108,6 +110,7 @@ export default function Header() {
sort: undefined,
dir: undefined,
highlighted: undefined,
nonSuspicious: undefined,
view: undefined,
focus: 'search',
}
@@ -158,6 +161,7 @@ export default function Header() {
sort: undefined,
dir: undefined,
highlighted: undefined,
nonSuspicious: undefined,
view: undefined,
focus: undefined,
}}
@@ -193,6 +197,7 @@ export default function Header() {
sort: undefined,
dir: undefined,
highlighted: undefined,
nonSuspicious: undefined,
view: undefined,
focus: 'search',
}
@@ -282,7 +287,12 @@ export default function Header() {
className="btn btn-primary"
type="button"
disabled={isLoading}
onClick={() => void signIn('github')}
onClick={() =>
void signIn(
'github',
signInRedirectTo ? { redirectTo: signInRedirectTo } : undefined,
)
}
>
<span className="sign-in-label">Sign in</span>
<span className="sign-in-provider">with GitHub</span>
@@ -293,3 +303,8 @@ export default function Header() {
</header>
)
}
function getCurrentRelativeUrl() {
if (typeof window === 'undefined') return '/'
return `${window.location.pathname}${window.location.search}${window.location.hash}`
}
+96 -33
View File
@@ -393,6 +393,10 @@ export function SkillDetailPage({
const [tagName, setTagName] = useState('latest')
const [tagVersionId, setTagVersionId] = useState<Id<'skillVersions'> | ''>('')
const [activeTab, setActiveTab] = useState<'files' | 'compare' | 'versions'>('files')
const [isReportDialogOpen, setIsReportDialogOpen] = useState(false)
const [reportReason, setReportReason] = useState('')
const [reportError, setReportError] = useState<string | null>(null)
const [isSubmittingReport, setIsSubmittingReport] = useState(false)
const isLoadingSkill = result === undefined
const skill = result?.skill
@@ -502,6 +506,12 @@ export function SkillDetailPage({
return stripFrontmatter(readme)
}, [readme])
const latestFiles: SkillFile[] = latestVersion?.files ?? []
const closeReportDialog = () => {
setIsReportDialogOpen(false)
setReportReason('')
setReportError(null)
setIsSubmittingReport(false)
}
useEffect(() => {
if (!latestVersion) return
@@ -685,30 +695,11 @@ export function SkillDetailPage({
<button
className="btn btn-ghost"
type="button"
onClick={async () => {
const reason = window.prompt(
'Report this skill? A reason is required. Abuse may result in a ban.',
)
if (reason === null) return
const trimmedReason = reason.trim()
if (!trimmedReason) {
window.alert('Report reason required.')
return
}
try {
const result = await reportSkill({
skillId: skill._id,
reason: trimmedReason,
})
if (result.reported) {
window.alert('Thanks — your report has been submitted.')
} else {
window.alert('You have already reported this skill.')
}
} catch (error) {
console.error('Failed to report skill', error)
window.alert(formatReportError(error))
}
onClick={() => {
setReportReason('')
setReportError(null)
setIsSubmittingReport(false)
setIsReportDialogOpen(true)
}}
>
Report
@@ -720,11 +711,6 @@ export function SkillDetailPage({
</Link>
) : null}
</div>
{isAuthenticated ? (
<div className="section-subtitle" style={{ margin: '6px 0 0' }}>
Reports require a reason. Abuse may result in a ban.
</div>
) : null}
<SecurityScanResults
sha256hash={latestVersion?.sha256hash}
vtAnalysis={latestVersion?.vtAnalysis}
@@ -1103,14 +1089,14 @@ export function SkillDetailPage({
<div className="stat">No comments yet.</div>
) : (
(comments ?? []).map((entry) => (
<div key={entry.comment._id} className="stat" style={{ alignItems: 'flex-start' }}>
<div>
<div key={entry.comment._id} className="comment-item">
<div className="comment-body">
<strong>@{entry.user?.handle ?? entry.user?.name ?? 'user'}</strong>
<div style={{ color: '#5c554e' }}>{entry.comment.body}</div>
<div className="comment-body-text">{entry.comment.body}</div>
</div>
{isAuthenticated && me && (me._id === entry.comment.userId || isModerator(me)) ? (
<button
className="btn"
className="btn comment-delete"
type="button"
onClick={() => void removeComment({ commentId: entry.comment._id })}
>
@@ -1123,6 +1109,83 @@ export function SkillDetailPage({
</div>
</div>
</div>
{isAuthenticated && isReportDialogOpen ? (
<div className="report-dialog-backdrop">
<div
className="report-dialog"
role="dialog"
aria-modal="true"
aria-labelledby="report-title"
>
<h2
id="report-title"
className="section-title"
style={{ margin: 0, fontSize: '1.1rem' }}
>
Report skill
</h2>
<p className="section-subtitle" style={{ margin: 0 }}>
Describe the issue so moderators can review it quickly.
</p>
<form
className="report-dialog-form"
onSubmit={async (event) => {
event.preventDefault()
const trimmedReason = reportReason.trim()
if (!trimmedReason) {
setReportError('Report reason required.')
return
}
setIsSubmittingReport(true)
setReportError(null)
try {
const result = await reportSkill({
skillId: skill._id,
reason: trimmedReason,
})
closeReportDialog()
if (result.reported) {
window.alert('Thanks — your report has been submitted.')
} else {
window.alert('You have already reported this skill.')
}
} catch (error) {
console.error('Failed to report skill', error)
setReportError(formatReportError(error))
setIsSubmittingReport(false)
}
}}
>
<textarea
className="report-dialog-textarea"
aria-label="Report reason"
placeholder="What should moderators know?"
value={reportReason}
onChange={(event) => setReportReason(event.target.value)}
rows={5}
disabled={isSubmittingReport}
/>
{reportError ? <p className="report-dialog-error">{reportError}</p> : null}
<div className="report-dialog-actions">
<button
type="button"
className="btn btn-ghost"
onClick={() => {
if (!isSubmittingReport) closeReportDialog()
}}
disabled={isSubmittingReport}
>
Cancel
</button>
<button type="submit" className="btn" disabled={isSubmittingReport}>
{isSubmittingReport ? 'Submitting…' : 'Submit report'}
</button>
</div>
</form>
</div>
</div>
) : null}
</main>
)
}
+1 -1
View File
@@ -423,7 +423,7 @@ function applyMonacoTheme(monaco: NonNullable<ReturnType<typeof useMonaco>>) {
const ink = styles.getPropertyValue('--ink').trim() || '#1d1a17'
const inkSoft = styles.getPropertyValue('--ink-soft').trim() || '#4c463f'
const line = styles.getPropertyValue('--line').trim() || 'rgba(29, 26, 23, 0.12)'
const accent = styles.getPropertyValue('--accent').trim() || '#ff6b4a'
const accent = styles.getPropertyValue('--accent').trim() || '#e65c46'
const seafoam = styles.getPropertyValue('--seafoam').trim() || '#2bc6a4'
const diffAdded = styles.getPropertyValue('--diff-added').trim() || seafoam
const diffRemoved = styles.getPropertyValue('--diff-removed').trim() || accent
+4 -4
View File
@@ -230,14 +230,14 @@ export function SoulDetailPage({ slug }: SoulDetailPageProps) {
<div className="stat">No comments yet.</div>
) : (
(comments ?? []).map((entry) => (
<div key={entry.comment._id} className="stat" style={{ alignItems: 'flex-start' }}>
<div>
<div key={entry.comment._id} className="comment-item">
<div className="comment-body">
<strong>@{entry.user?.handle ?? entry.user?.name ?? 'user'}</strong>
<div style={{ color: '#5c554e' }}>{entry.comment.body}</div>
<div className="comment-body-text">{entry.comment.body}</div>
</div>
{isAuthenticated && me && (me._id === entry.comment.userId || isModerator(me)) ? (
<button
className="btn"
className="btn comment-delete"
type="button"
onClick={() => void removeComment({ commentId: entry.comment._id })}
>
+1 -1
View File
@@ -34,7 +34,7 @@ const DropdownMenuItem = React.forwardRef<
<DropdownMenuPrimitive.Item
ref={ref}
className={cn(
'flex cursor-pointer select-none items-center gap-2 rounded-lg px-3 py-2 text-sm font-semibold text-[color:var(--ink)] outline-none transition-colors focus:bg-[color:rgba(255,107,74,0.12)] data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
'flex cursor-pointer select-none items-center gap-2 rounded-lg px-3 py-2 text-sm font-semibold text-[color:var(--ink)] outline-none transition-colors focus:bg-[color:var(--surface-muted)] data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
className,
)}
{...props}
+1 -1
View File
@@ -24,7 +24,7 @@ const ToggleGroupItem = React.forwardRef<
<ToggleGroupPrimitive.Item
ref={ref}
className={cn(
'inline-flex h-9 w-9 items-center justify-center rounded-full text-[color:var(--ink-soft)] transition-colors hover:text-[color:var(--ink)] focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-[color:rgba(255,107,74,0.4)] data-[state=on]:bg-[color:var(--accent)] data-[state=on]:text-white',
'inline-flex h-9 w-9 items-center justify-center rounded-full text-[color:var(--ink-soft)] transition-colors hover:text-[color:var(--ink)] focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-[color:var(--accent)] data-[state=on]:bg-[color:var(--accent)] data-[state=on]:text-white',
className,
)}
{...props}
+6
View File
@@ -46,6 +46,12 @@ describe('site helpers', () => {
withMetaEnv({ VITE_SITE_URL: 'https://example.com' }, () => {
expect(getClawHubSiteUrl()).toBe('https://example.com')
})
withMetaEnv({ VITE_SITE_URL: 'https://clawdhub.com' }, () => {
expect(getClawHubSiteUrl()).toBe('https://clawhub.ai')
})
withMetaEnv({ VITE_SITE_URL: 'https://auth.clawdhub.com' }, () => {
expect(getClawHubSiteUrl()).toBe('https://clawhub.ai')
})
})
it('picks SoulHub URL from explicit env', () => {
+15 -1
View File
@@ -3,9 +3,23 @@ export type SiteMode = 'skills' | 'souls'
const DEFAULT_CLAWHUB_SITE_URL = 'https://clawhub.ai'
const DEFAULT_ONLYCRABS_SITE_URL = 'https://onlycrabs.ai'
const DEFAULT_ONLYCRABS_HOST = 'onlycrabs.ai'
const LEGACY_CLAWDHUB_HOSTS = new Set(['clawdhub.com', 'www.clawdhub.com', 'auth.clawdhub.com'])
export function normalizeClawHubSiteOrigin(value?: string | null) {
if (!value) return null
try {
const url = new URL(value)
if (LEGACY_CLAWDHUB_HOSTS.has(url.hostname.toLowerCase())) {
return DEFAULT_CLAWHUB_SITE_URL
}
return url.origin
} catch {
return null
}
}
export function getClawHubSiteUrl() {
return import.meta.env.VITE_SITE_URL ?? DEFAULT_CLAWHUB_SITE_URL
return normalizeClawHubSiteOrigin(import.meta.env.VITE_SITE_URL) ?? DEFAULT_CLAWHUB_SITE_URL
}
export function getOnlyCrabsSiteUrl() {
+60
View File
@@ -0,0 +1,60 @@
import { describe, expect, it } from 'vitest'
import type { PublicSkill } from './publicUser'
import { mapPublicSkillPageEntries } from './skillPageEntries'
function makeSkill(overrides: Partial<PublicSkill> = {}): PublicSkill {
return {
_id: 'skills:1' as PublicSkill['_id'],
_creationTime: 1,
slug: 'demo',
displayName: 'Demo Skill',
summary: 'summary',
ownerUserId: 'users:1' as PublicSkill['ownerUserId'],
canonicalSkillId: undefined,
forkOf: undefined,
latestVersionId: undefined,
tags: {},
badges: {},
stats: {
downloads: 12,
stars: 3,
installsCurrent: 5,
installsAllTime: 7,
versions: 2,
comments: 1,
},
createdAt: 1,
updatedAt: 1,
...overrides,
}
}
describe('mapPublicSkillPageEntries', () => {
it('extracts nested skill entries from listPublicPageV2 shape', () => {
const skill = makeSkill({ slug: 'popular-skill' })
const result = mapPublicSkillPageEntries([
{
skill,
},
])
expect(result).toHaveLength(1)
expect(result[0]?.slug).toBe('popular-skill')
})
it('normalizes missing stats fields to zero', () => {
const skill = makeSkill({
stats: undefined as unknown as PublicSkill['stats'],
})
const result = mapPublicSkillPageEntries([{ skill }])
expect(result[0]?.stats).toEqual({
downloads: 0,
stars: 0,
installsCurrent: 0,
installsAllTime: 0,
versions: 0,
comments: 0,
})
})
})
+28
View File
@@ -0,0 +1,28 @@
import type { PublicSkill } from './publicUser'
type SkillPageEntry = {
skill?: PublicSkill | null
}
function normalizeSkillStats(skill: PublicSkill): PublicSkill {
const stats = skill.stats
return {
...skill,
stats: {
downloads: stats?.downloads ?? 0,
stars: stats?.stars ?? 0,
installsCurrent: stats?.installsCurrent ?? 0,
installsAllTime: stats?.installsAllTime ?? 0,
versions: stats?.versions ?? 0,
comments: stats?.comments ?? 0,
},
}
}
export function mapPublicSkillPageEntries(page: SkillPageEntry[] | undefined): PublicSkill[] {
if (!page?.length) return []
return page
.map((entry) => entry.skill ?? null)
.filter((skill): skill is PublicSkill => skill !== null)
.map(normalizeSkillStats)
}
+6 -2
View File
@@ -6,8 +6,12 @@ import { expandDroppedItems, expandFiles } from './uploadFiles'
function readWithFileReader(blob: Blob) {
return new Promise<ArrayBuffer>((resolve, reject) => {
const reader = new FileReader()
reader.onerror = () => reject(reader.error ?? new Error('Could not read blob.'))
reader.onload = () => resolve(reader.result as ArrayBuffer)
reader.addEventListener('error', () => {
reject(reader.error ?? new Error('Could not read blob.'))
})
reader.addEventListener('load', () => {
resolve(reader.result as ArrayBuffer)
})
reader.readAsArrayBuffer(blob)
})
}
+6 -2
View File
@@ -137,8 +137,12 @@ async function readArrayBuffer(file: Blob) {
if (typeof FileReader !== 'undefined') {
return new Promise<ArrayBuffer>((resolve, reject) => {
const reader = new FileReader()
reader.onerror = () => reject(reader.error ?? new Error('Could not read file.'))
reader.onload = () => resolve(reader.result as ArrayBuffer)
reader.addEventListener('error', () => {
reject(reader.error ?? new Error('Could not read file.'))
})
reader.addEventListener('load', () => {
resolve(reader.result as ArrayBuffer)
})
reader.readAsArrayBuffer(file)
})
}
+6 -2
View File
@@ -84,8 +84,12 @@ export async function readText(blob: Blob) {
if (typeof FileReader !== 'undefined' && blob instanceof Blob) {
return new Promise<string>((resolve, reject) => {
const reader = new FileReader()
reader.onerror = () => reject(reader.error ?? new Error('Could not read blob.'))
reader.onload = () => resolve(typeof reader.result === 'string' ? reader.result : '')
reader.addEventListener('error', () => {
reject(reader.error ?? new Error('Could not read blob.'))
})
reader.addEventListener('load', () => {
resolve(typeof reader.result === 'string' ? reader.result : '')
})
reader.readAsText(blob)
})
}

Some files were not shown because too many files have changed in this diff Show More