Compare commits

...
Author SHA1 Message Date
Peter Steinberger afa9a6db13 fix: preserve folder upload mode across refresh (#551) (thanks @MunemHashmi) 2026-03-07 18:32:58 +00:00
Munem Hashmi f5944c529e fix(ui): persist folder upload input across hydration and re-renders (#58)
Replace the useEffect + useRef approach for setting webkitdirectory/
directory attributes with a ref callback that sets the attributes
every time the input element is mounted. This ensures folder selection
mode persists after page refresh, where React hydration could strip
the non-standard attributes.

Also removes the @ts-expect-error JSX props since the attributes are
now set imperatively via the ref callback.
2026-03-07 18:31:42 +00:00
Peter Steinberger 531dcc8d26 fix: avoid auth crash in slug availability preflight 2026-03-07 18:29:01 +00:00
Tristan Manchester b1c710e1ea fix: add dedicated slug availability preflight 2026-03-07 15:38:01 +00:00
Tristan Manchester a4a9fc62bc fix: address review comments for slug-collision error handling 2026-03-07 15:38:01 +00:00
Tristan Manchester e58fbc8d1f fix: surface slug-collision publish errors and block conflicts preflight 2026-03-07 15:38:01 +00:00
Peter Steinberger b4b4f266a8 fix: align VT engine fallback verdict mapping (#591) (thanks @Shuai-DaiDai) 2026-03-07 15:33:30 +00:00
帅小呆1号 f9d35cc5e6 fix(vt): sync scan status from AV engines when Code Insight unavailable
When VirusTotal returns scan results with AV engine stats but no Code Insight
AI analysis, the skill status was stuck on 'Pending'. This fix adds fallback
logic to check last_analysis_stats (malicious/suspicious/harmless/undetected)
to determine scan status.

Functions updated:
- pollPendingScans: Check AV engines before requesting rescan
- backfillPendingScans: Check AV engines before marking as no results
- rescanActiveSkills: Check AV engines before keeping as pending
- backfillActiveSkillsVTCache: Check AV engines before skipping

Fixes #33435
2026-03-07 15:33:30 +00:00
Peter Steinberger 1892f72a13 test: lock multipart upload timeout behavior (#550) (thanks @MunemHashmi) 2026-03-07 15:31:20 +00:00
Munem Hashmi fdbc184e0a fix(cli): improve publish timeout handling and error messages (#533)
- Increase upload timeout from 15s to 120s for multipart form uploads
  (apiRequestForm and curl-based form upload). Regular API requests
  remain at 15s.
- Improve timeout error message from bare "Timeout" to
  "Request timed out after Ns" so users know what happened.
- Normalize non-Error throws (e.g. DOMException from AbortController
  across runtimes) into proper Error instances, preventing the
  misleading "Non-error was thrown" message from p-retry.
- Preserve the original error as `cause` on the wrapped Error.
2026-03-07 15:31:20 +00:00
Peter Steinberger 18cbfc6788 test: cover auth token forwarding for search/explore (#608) (thanks @artdaal) 2026-03-07 15:29:37 +00:00
Артемов Даниил Алексеевич c821b0ffc4 fix: pass auth token in search and explore commands
cmdSearch and cmdExplore were not calling getOptionalAuthToken()
and did not pass the token to apiRequest, unlike install/update/uninstall.
This caused 'missing API token' errors on registries that require auth
(e.g. private Hermit instances).
2026-03-07 15:29:37 +00:00
Peter Steinberger 9833a5038d docs: note top-level frontmatter metadata parsing fix (#548) (thanks @MunemHashmi) 2026-03-07 15:28:12 +00:00
Munem Hashmi 40a89e02d5 fix: extract requires.env and homepage from top-level frontmatter (#522)
parseFrontmatterLevelDeclarations did not handle the requires block
(env, bins, anyBins, config) or primaryEnv when declared at the
top level of SKILL.md frontmatter without a metadata.openclaw wrapper.
This caused the security scanner to always show "Required env vars: none"
for skills using that format, triggering false-positive suspicious flags.

Also extends the evalCtx.homepage fallback chain to check
clawdis.homepage and clawdis.links.homepage so skills declaring
homepage inside the metadata block are picked up by the scanner.
2026-03-07 15:28:12 +00:00
Timothy JordanandClaude Opus 4.6 bc06dbffd0 chore: add Vercel attribution in footer (#557)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:06:35 +00:00
Agent f5fa23e0c1 chore: add convex attribution in footer 2026-02-27 23:10:26 +01:00
Vincent Koc e8c3947b21 Merge pull request #547 from openclaw/fix/secret-scan-trufflehog-ref
fix(ci): restore secret scan action reference
2026-02-27 10:39:18 -08:00
Vincent Koc ef26ee0d1f fix(ci): pin trufflehog to published v3.93.6 tag 2026-02-27 10:38:31 -08:00
Vincent Koc 3d006ec663 fix(ci): use resolvable trufflehog action ref 2026-02-27 09:47:17 -08:00
Peter Steinberger 52590e84dd chore: commit local pending changes 2026-02-26 13:03:25 +01:00
Peter Steinberger e3a1c95851 docs(agents): reject skill-in-source PRs; require CLI publish 2026-02-26 13:03:25 +01:00
Mahsum AktaşandPeter Steinberger db4540743f feat(registry): support env vars, dependencies, author, and links in skill manifest (#360)
* feat(registry): support env vars, dependencies, author, and links in skill manifest

Closes #350

Add structured declarations for environment variables, package
dependencies, author identity, and project links to the skill
registry manifest. These fields can be declared in the clawdis
metadata block or as top-level frontmatter keys.

Changes:
- schema: add EnvVarDeclaration, DependencyDeclaration, SkillLinks
  types to ClawdisSkillMetadata
- parser: extract envVars, dependencies, author, links from both
  clawdis block and top-level frontmatter (fallback for skills
  without a clawdis block)
- UI: render env vars with required/optional badges and descriptions,
  dependencies with type/version/links, and project links in the
  skill detail page install card
- security: update evaluator prompt to recognize envVars alongside
  requires.env and primaryEnv
- tests: 7 new test cases covering all declaration formats

* fix(ui): handle unspecified env required state and stable keys

* docs(changelog): credit metadata manifest expansion (#360) (thanks @mahsumaktas)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 12:02:25 +00:00
David Abutbulgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>Peter Steinberger
0cb0963c2b feat(api): expose security evaluation results (#362)
* feat(api): expose security evaluation results

- Add security field to skill version API responses
- Map llmAnalysis database field to public API format
- Display security info in CLI inspect command
- Enable security tools like clawsec-clawhub-checker to access internal security checks

Security field includes:
- status: clean|suspicious|malicious|pending|error
- hasWarnings: boolean
- checkedAt: timestamp
- model: evaluation model name

Backward compatible: optional field, no breaking changes.

* fix: ensure hasWarnings is always boolean

- Add ?? false to coerce undefined to false when dimensions is undefined
- Fixes Greptile comment: hasWarnings can be undefined instead of boolean
- Ensures SecurityStatusSchema validation passes on client side

* Update convex/httpApiV1/skillsV1.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(api-cli): harden security inspect output + tests (#362) (thanks @abutbul)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 12:01:23 +00:00
David AronchickandPeter Steinberger beae065794 fix(cli): handle missing browser opener gracefully (#163)
* fix(cli): handle missing browser opener gracefully

On headless Linux servers without xdg-open, 'clawhub login' crashes with
ENOENT error. This change catches the error and prints the URL for manual
copy-paste instead of crashing.

Fixes crash on:
- VPS/cloud servers
- Docker containers
- CI environments
- WSL without browser integration

* fix(cli): test browser-opener fallback messaging (#163) (thanks @aronchick)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 11:59:24 +00:00
46c5637dae Improve error handling in GitHub import (#512)
* Improve error handling in GitHub import

- Add detailed error messages for storage failures
- Wrap publishVersionForUser in try/catch with helpful context
- Include file size and path in storage error messages
- Guide users to check skill format and slug availability

* fix(github-import): improve failure messaging + coverage (#512) (thanks @vassiliylakhonin)

---------

Co-authored-by: Vassiliy Lakhonin <vassiliy.lakhonin@example.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 11:30:01 +00:00
Tristan Manchester 2217a327e7 fix(upload): ignore macOS junk files during publish (#526) 2026-02-26 11:28:27 +00:00
45d8f0d217 feat: surface platform/architecture labels on skill cards and API (#499)
* feat: surface platform/architecture labels on skill cards and API

Expose existing `os` and `nix.systems` metadata from skill frontmatter
through the HTTP API and render as compact tags on browse/search views.

- Widen `PublicSkillListVersion` and `SkillListEntry` types to include
  `os` and `nix.systems` fields (data already flows through, types were
  artificially narrow)
- Add `metadata: { os, systems }` to `/api/v1/skills/{slug}` and
  `/api/v1/skills` list responses
- Add `formatSystemsList` and `getPlatformLabels` helpers to map nix
  system strings to human-readable labels (e.g. aarch64-darwin → macOS ARM64)
- Add `platformLabels` prop to `SkillCard`, render as `.tag .tag-compact`
- Show platform labels in both card grid and list views
- Update HTTP API docs with new `metadata` field

Coded by Claude Opus 4.6 (Claude Code)
Reviewed and tested by Jason (@asyncjason)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove redundant optional chaining on clawdis

Address greptile-apps review comment — clawdis is already confirmed
truthy by the ternary condition, so `?.` is unnecessary.

Coded by Claude Opus 4.6 (Claude Code)
Reviewed and tested by Jason (@asyncjason)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: include version data in listPublicPageV2 for platform labels

The browse listing passed includeVersion: false, causing latestVersion
to always be null and platform/arch labels to never render.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Jason Separovic <jason@wilma.dog>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 11:27:20 +00:00
Nicolas GreniéandClaude Opus 4.6 add5d83014 docs: add CONTRIBUTING.md and refresh README header (#400)
* docs: add CONTRIBUTING.md and refresh README header

Add a comprehensive CONTRIBUTING.md covering local Convex setup,
env var configuration, GitHub OAuth, JWT keys, database seeding,
CLI development, PR guidelines, and AI-generated code policy.

Refresh the README with a centered logo, quick links row, and
clickable doc references. Condense the Local dev section to link
to CONTRIBUTING.md for full setup details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add #clawhub discord channel

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 11:25:51 +00:00
Abdul B.andCursor 9751199231 fix: prevent filtered skills pagination flicker (#372)
* fix: prevent filtered skills pagination flicker

Skip fully filtered-out pages in public skills pagination so highlighted/non-suspicious filtering doesn't return empty pages with more cursor state, which caused repeated loading-more flicker.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: rely on inferred Convex paginate result type

Remove the custom runPaginate annotation so TypeScript infers the exact Convex paginate result shape and preserves stronger type-safety.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-26 11:25:15 +00:00
Luke 883221f8ec fix(cli): clarify owner delete permissions in command text (#417) 2026-02-26 11:23:53 +00:00
Peter Steinberger 3e45d67e0d fix: delete and hide comments from banned users 2026-02-26 05:52:09 +01:00
Peter Steinberger df346aeea9 feat: require 14-day GitHub age for publish and comments 2026-02-26 05:35:44 +01:00
Peter Steinberger 4317369480 fix: stabilize comment moderation tests without env keys 2026-02-26 02:18:00 +01:00
Peter Steinberger e04d16bdae feat: add ai comment scam backfill and auto-ban flow 2026-02-26 02:16:31 +01:00
Peter Steinberger cb66d8d6f3 feat: add abuse-resistant comment reporting 2026-02-26 01:28:22 +01:00
Peter Steinberger 14a2fa80f6 test: lock 5xx retry behavior in HTTP client (#457) (thanks @YonghaoZhao722) 2026-02-25 13:03:16 +00:00
Peter Steinberger ee788b7af3 test: pin Retry-After relative-delay behavior (#421) (thanks @apoorvdarshan) 2026-02-25 12:19:10 +00:00
Apoorv Darshan 3956ca7e55 fix: use relative delay for Retry-After header on 429 responses
Retry-After was set to an absolute Unix epoch timestamp (e.g. 1771404540),
which violates RFC 9110 §10.2.3. Clients treating it as delay-seconds
would wait ~56 years. Now emits the actual seconds until reset.

Closes #407
2026-02-25 12:19:10 +00:00
Peter Steinberger bc37ec7156 fix: complete registry-url migration with test coverage (#486) (thanks @Liknox) 2026-02-25 12:17:30 +00:00
Nazar Koval f07408eb81 test: url formatter 2026-02-25 12:17:30 +00:00
Nazar Koval cdf5baef7f ref: url entity utilization 2026-02-25 12:17:30 +00:00
Nazar Koval 65b154f36c feat: url formatter entity 2026-02-25 12:17:30 +00:00
Peter Steinberger 6ea7a0792d fix: finalize proxy env support + changelog credits (#363) (thanks @kerrypotter) 2026-02-25 12:14:00 +00:00
Jarvis ed961e459f fix: use EnvHttpProxyAgent for proper proxy support
Address review feedback:
- Use undici's EnvHttpProxyAgent instead of ProxyAgent. This properly
  handles HTTPS_PROXY vs HTTP_PROXY per-scheme, respects NO_PROXY,
  and uses connect.timeout instead of requestTls.
- Update docs to mention NO_PROXY support.
2026-02-25 12:14:00 +00:00
Jarvis 8b5f242f73 fix: respect HTTP_PROXY/HTTPS_PROXY environment variables
The CLI creates a custom undici Agent via setGlobalDispatcher() which
overrides any proxy configuration. Since Node.js native fetch (backed
by undici) does not automatically respect HTTP_PROXY/HTTPS_PROXY env
vars, the CLI fails with 'fetch failed' on systems that require a
proxy for outbound connections.

Import ProxyAgent from undici and use it when any of the standard proxy
environment variables (HTTPS_PROXY, HTTP_PROXY, https_proxy, http_proxy)
is set. When no proxy variable is present, behavior is unchanged.

Also adds proxy documentation to cli.md and a troubleshooting entry.
2026-02-25 12:14:00 +00:00
87 changed files with 5434 additions and 347 deletions
+3 -1
View File
@@ -18,7 +18,9 @@ jobs:
- name: TruffleHog OSS
id: trufflehog
uses: trufflesecurity/trufflehog@e64309e4514a601c7d23f336688782a229a4a754 # Pin to current stable
# Use a concrete released ref that resolves in upstream action registry.
# v3 (major tag) is not published by trufflesecurity/trufflehog.
uses: trufflesecurity/trufflehog@v3.93.6
with:
path: ./
base: ${{ github.event.pull_request.base.sha }} # scope it to the committed files
+1
View File
@@ -33,6 +33,7 @@
- Commit messages: Conventional Commits (`feat:`, `fix:`, `chore:`, `docs:`…).
- Keep changes scoped; avoid repo-wide search/replace.
- PRs: include summary + test commands run. Add screenshots for UI changes.
- Reject PRs that add skills into source code/repo content directly (for example under `skills/` or seed-only additions intended as published skills). Skills must be uploaded/published via CLI.
## Git Notes
- If `git branch -d/-D <branch>` is policy-blocked, delete the local ref directly: `git update-ref -d refs/heads/<branch>`.
+21
View File
@@ -3,6 +3,8 @@
## Unreleased
### Added
- Moderation: add comment reporting with per-user active report caps, unique reporter/target enforcement, and auto-hide on the 4th unique report.
- Moderation: add AI-driven comment scam backfill (`commentModeration:*`) with persisted verdict/confidence/explainer metadata and strict auto-ban for `certain_scam` + `high` confidence.
- Admin: add manual unban for banned users (clears `deletedAt` + `banReason`, audit log entry). Revoked API tokens stay revoked.
- Admin: bulk restore skills from GitHub backup; reclaim squatted slugs via v1 endpoints + internal tooling (#298) (thanks @autogame-17).
- Users: add `trustedPublisher` flag and admin mutations to bypass pending-scan auto-hide for trusted publishers (#298) (thanks @autogame-17).
@@ -12,6 +14,10 @@
- CI/Security: add TruffleHog pull-request scanning for verified leaked credentials (#505) (thanks @akses0).
### Changed
- Security/docs: document comment reporting/auto-hide behavior alongside existing skill reporting rules.
- Security/moderation: add bounded explainable auto-ban reasons for scam comments and protect moderator/admin accounts from automated bans.
- Moderation: banning users now also soft-deletes their authored comments (skill + soul), including legacy cleanup on re-ban.
- Skill metadata: support env vars, dependency declarations, author, and links in parsed manifest metadata + install UI (#360) (thanks @mahsumaktas).
- Quality gate: language-aware word counting (`Intl.Segmenter`) and new `cjkChars` signal to reduce false rejects for non-Latin docs.
- Jobs: run skill stat event processing every 5 minutes (was 15).
- API performance: batch resolve skill/soul tags in v1 list/get endpoints (fewer action->query round-trips) (#112) (thanks @mkrokosz).
@@ -22,8 +28,15 @@
- Search/listing performance: cut embedding hydration and badge read bandwidth via `embeddingSkillMap` + denormalized skill badges; shift stat-doc sync to low-frequency cron (#441) (thanks @sethconvex).
### Fixed
- Upload: keep folder-picking enabled after page refresh by reapplying `webkitdirectory`/`directory` on the file input ref (#551) (thanks @MunemHashmi).
- Skills hard-delete: delete `commentReports` rows during moderation cleanup to avoid orphaned report records.
- Comments: hide entries authored by deleted/deactivated users in `comments:listBySkill`.
- Admin API: `POST /api/v1/users/reclaim` now performs non-destructive root-slug owner transfer
(preserves existing skill versions/stats/metadata) and clears active slug reservations.
- VirusTotal: use shared AV-engine fallback verdict mapping for pending/backfill flows and keep undetected-only results pending (#591) (thanks @Shuai-DaiDai).
- CLI publish: use a longer multipart upload timeout and normalize abort rejections into proper Errors (#550) (thanks @MunemHashmi).
- CLI: forward optional auth tokens for `search` and `explore` against authenticated registries (#608) (thanks @artdaal).
- Skill metadata: parse top-level `requires.*`, `primaryEnv`, and homepage fallbacks for security review accuracy (#548) (thanks @MunemHashmi).
- Users: sync handle on ensure when GitHub login changes (#293) (thanks @christianhpoe).
- Users/Auth: throttle GitHub profile sync on login; also sync avatar when it changes (#312) (thanks @ianalloway).
- Upload gate: fetch GitHub account age by immutable account ID (prevents username swaps) (#116) (thanks @mkrokosz).
@@ -39,6 +52,14 @@
- Skills/Web: prevent filtered pagination dead-ends and loading-state flicker on `/skills`; move highlighted browse filtering into server list query (#339) (thanks @Marvae).
- Web: align `/skills` total count with public visibility and format header count (thanks @rknoche6, #76).
- Skills/Web: centralize public visibility checks and keep `globalStats` skill counts in sync incrementally; remove duplicate `/skills` default-sort fallback and share browse test mocks (thanks @rknoche6, #76).
- Moderation: clear stale `flagged.suspicious` flags when VirusTotal rescans improve to clean verdicts (#418) (thanks @Phineas1500).
- CLI: respect `HTTPS_PROXY`/`HTTP_PROXY`/`NO_PROXY` env vars for outbound registry requests, with troubleshooting docs (#363) (thanks @kerrypotter).
- CLI: preserve registry base paths when composing API URLs for search/inspect/moderation commands (#486) (thanks @Liknox).
- API tests: lock `Retry-After` behavior to relative-delay semantics for v1 search 429s (#421) (thanks @apoorvdarshan).
- CLI tests: assert 5xx HTTP responses still perform retry attempts before surfacing final error (#457) (thanks @YonghaoZhao722).
- GitHub import: improve storage/publish failure errors with actionable context; add regression tests for error formatting (#512) (thanks @vassiliylakhonin).
- CLI: show manual URL guidance when automatic browser opening is unavailable; add regression tests for opener errors (#163) (thanks @aronchick).
- API/CLI: expose skill security status in version inspect output, with schema wiring and CLI regression coverage (#362) (thanks @abutbul).
## 0.6.1 - 2026-02-13
+166
View File
@@ -0,0 +1,166 @@
# Contributing to ClawHub
Welcome! ClawHub is the public skill registry for [OpenClaw](https://github.com/openclaw/openclaw). We appreciate bug fixes, documentation improvements, and feature contributions.
- **Questions?** Ask in [#clawhub on Discord](https://discord.gg/clawd).
- **Bug fixes** — PRs are welcome.
- **New features or architectural changes** — please start with a Discord conversation in #clawhub first so we can align on scope.
## Local Development Setup
### Prerequisites
- [Bun](https://bun.sh/) (Convex CLI runs via `bunx`, no global install needed)
### Install and configure
```bash
bun install
cp .env.local.example .env.local
```
Edit `.env.local` with the following values for **local Convex**:
```bash
# Frontend
VITE_CONVEX_URL=http://127.0.0.1:3210
VITE_CONVEX_SITE_URL=http://127.0.0.1:3210
SITE_URL=http://localhost:3000
CONVEX_SITE_URL=http://127.0.0.1:3210
# Deployment used by `bunx convex dev`
CONVEX_DEPLOYMENT=anonymous:anonymous-clawhub
```
### GitHub OAuth App (for login)
1. Go to [github.com/settings/developers](https://github.com/settings/developers) and create a new OAuth App.
2. Set **Homepage URL** to `http://localhost:3000`.
3. Set **Authorization callback URL** to `http://127.0.0.1:3210/api/auth/callback/github`.
4. Copy the Client ID and generate a Client Secret, then add them to `.env.local`:
```bash
AUTH_GITHUB_ID=<your-client-id>
AUTH_GITHUB_SECRET=<your-client-secret>
```
### JWT keys (for Convex Auth)
Generate the signing keys:
```bash
bunx @convex-dev/auth
```
This outputs `JWT_PRIVATE_KEY` and `JWKS` values — paste them into `.env.local`.
### Run the app
```bash
# Terminal A: local Convex backend
bunx convex dev
# Terminal B: frontend (port 3000)
bun run dev
```
### Seed the database
Populate sample data so the UI isn't empty:
```bash
# 3 sample skills (padel, gohome, xuezh)
bunx convex run --no-push devSeed:seedNixSkills
# 50 extra skills for pagination testing (optional)
bunx convex run --no-push devSeedExtra:seedExtraSkillsInternal
```
To reset and re-seed:
```bash
bunx convex run --no-push devSeed:seedNixSkills '{"reset": true}'
```
### Optional environment variables
These features degrade gracefully without their keys:
| Variable | Purpose |
|----------|---------|
| `OPENAI_API_KEY` | Embeddings and vector search (falls back to zero vectors) |
| `VT_API_KEY` | VirusTotal malware scanning |
| `DISCORD_WEBHOOK_URL` | Discord notifications |
| `GITHUB_APP_ID` / `GITHUB_APP_PRIVATE_KEY` / `GITHUB_APP_INSTALLATION_ID` | GitHub backup sync |
## CLI Development
The CLI source lives in [`packages/clawdhub/`](packages/clawdhub/). Both `clawhub` and `clawdhub` are registered as bin aliases.
To test the CLI against your local instance:
```bash
CLAWHUB_REGISTRY=http://127.0.0.1:3210 CLAWHUB_SITE=http://localhost:3000 clawhub search "padel"
```
Manual smoke tests are documented in [`docs/manual-testing.md`](docs/manual-testing.md).
## Skill & Soul Publishing
- Skill format reference: [`docs/skill-format.md`](docs/skill-format.md)
- Soul format reference: [`docs/soul-format.md`](docs/soul-format.md)
- End-to-end walkthrough (search, install, publish, sync): [`docs/quickstart.md`](docs/quickstart.md)
Quick publish:
```bash
clawhub publish <path-to-skill-directory>
```
## Before Submitting a PR
```bash
bun run lint # oxlint
bun run test # Vitest (80% coverage threshold)
bun run build # Vite + Nitro
```
These are the same checks that run in CI (`.github/workflows/ci.yml`).
**PR guidelines:**
- Keep PRs focused — one concern per PR.
- Use [Conventional Commits](https://www.conventionalcommits.org/): `feat:`, `fix:`, `chore:`, `docs:`, etc.
- Include test commands and screenshots for UI changes.
- Write a clear description of what changed and why.
## AI-Generated Code
AI-assisted contributions are welcome. When submitting AI-generated or AI-assisted code:
- Note it in the PR description.
- Describe the level of testing you applied.
- Include prompts if useful for reviewers.
- Confirm that you understand and can maintain the code.
## Security Reporting
Report vulnerabilities to **security@openclaw.ai** with:
- Severity assessment
- Technical reproduction steps
- Suggested remediation
See [`docs/security.md`](docs/security.md) for moderation and upload gating details.
## Reading Order for New Contributors
1. This file (local setup)
2. [`docs/quickstart.md`](docs/quickstart.md) — end-to-end workflows
3. [`docs/architecture.md`](docs/architecture.md) — system design
4. [`docs/skill-format.md`](docs/skill-format.md) — skill structure
5. [`docs/cli.md`](docs/cli.md) — CLI reference
6. [`docs/http-api.md`](docs/http-api.md) — HTTP endpoints
7. [`docs/auth.md`](docs/auth.md) — authentication
8. [`docs/deploy.md`](docs/deploy.md) — deployment
9. [`docs/troubleshooting.md`](docs/troubleshooting.md) — common issues
+27 -21
View File
@@ -1,4 +1,8 @@
# ClawHub
<p align="center">
<img src="public/clawd-logo.png" alt="ClawHub" width="120">
</p>
<h1 align="center">ClawHub</h1>
<p align="center">
<a href="https://github.com/openclaw/clawhub/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/openclaw/clawhub/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
@@ -7,13 +11,18 @@
</p>
ClawHub is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
Its designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
It's designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
onlycrabs.ai is the **SOUL.md registry**: publish and share system lore the same way you publish skills.
Live: `https://clawhub.ai`
onlycrabs.ai: `https://onlycrabs.ai`
Vision: [`VISION.md`](VISION.md)
<p align="center">
<a href="https://clawhub.ai">ClawHub</a> ·
<a href="https://onlycrabs.ai">onlycrabs.ai</a> ·
<a href="VISION.md">Vision</a> ·
<a href="docs/README.md">Docs</a> ·
<a href="CONTRIBUTING.md">Contributing</a> ·
<a href="https://discord.gg/clawd">Discord</a>
</p>
## What you can do with it
@@ -48,7 +57,7 @@ Common CLI flows:
- Inspect without installing: `clawhub inspect <slug>`
- Publish/sync: `clawhub publish <path>`, `clawhub sync`
Docs: `docs/quickstart.md`, `docs/cli.md`.
Docs: [`docs/quickstart.md`](docs/quickstart.md), [`docs/cli.md`](docs/cli.md).
### Removal permissions
@@ -67,39 +76,36 @@ Disable via:
export CLAWHUB_DISABLE_TELEMETRY=1
```
Details: `docs/telemetry.md`.
Details: [`docs/telemetry.md`](docs/telemetry.md).
## Repo layout
- `src/` — TanStack Start app (routes, components, styles).
- `convex/` — schema + queries/mutations/actions + HTTP API routes.
- `packages/schema/` — shared API types/routes for the CLI and app.
- `docs/spec.md` — product + implementation spec (good first read).
- [`docs/`](docs/README.md) — project documentation (architecture, CLI, auth, deployment, and more).
- [`docs/spec.md`](docs/spec.md) — product + implementation spec (good first read).
## Local dev
Prereqs: Bun + Convex CLI.
Prereqs: [Bun](https://bun.sh/) (Convex runs via `bunx`, no global install needed).
```bash
bun install
cp .env.local.example .env.local
# edit .env.local — see CONTRIBUTING.md for local Convex values
# terminal A: web app
# terminal A: local Convex backend
bunx convex dev
# terminal B: web app (port 3000)
bun run dev
# terminal B: Convex dev deployment
bunx convex dev
# seed sample data
bunx convex run --no-push devSeed:seedNixSkills
```
## Auth (GitHub OAuth) setup
Create a GitHub OAuth App, set `AUTH_GITHUB_ID` / `AUTH_GITHUB_SECRET`, then:
```bash
bunx auth --deployment-name <deployment> --web-server-url http://localhost:3000
```
This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints values for your local `.env.local`.
For full setup instructions (env vars, GitHub OAuth, JWT keys, database seeding), see [CONTRIBUTING.md](CONTRIBUTING.md).
## Environment
+8
View File
@@ -9,6 +9,7 @@
*/
import type * as auth from "../auth.js";
import type * as commentModeration from "../commentModeration.js";
import type * as comments from "../comments.js";
import type * as crons from "../crons.js";
import type * as devSeed from "../devSeed.js";
@@ -38,6 +39,7 @@ import type * as lib_apiTokenAuth from "../lib/apiTokenAuth.js";
import type * as lib_badges from "../lib/badges.js";
import type * as lib_batching from "../lib/batching.js";
import type * as lib_changelog from "../lib/changelog.js";
import type * as lib_commentScamPrompt from "../lib/commentScamPrompt.js";
import type * as lib_contentTypes from "../lib/contentTypes.js";
import type * as lib_embeddingVisibility from "../lib/embeddingVisibility.js";
import type * as lib_embeddings from "../lib/embeddings.js";
@@ -53,7 +55,9 @@ import type * as lib_httpHeaders from "../lib/httpHeaders.js";
import type * as lib_httpRateLimit from "../lib/httpRateLimit.js";
import type * as lib_leaderboards from "../lib/leaderboards.js";
import type * as lib_moderation from "../lib/moderation.js";
import type * as lib_openaiResponse from "../lib/openaiResponse.js";
import type * as lib_public from "../lib/public.js";
import type * as lib_reporting from "../lib/reporting.js";
import type * as lib_reservedSlugs from "../lib/reservedSlugs.js";
import type * as lib_searchText from "../lib/searchText.js";
import type * as lib_securityPrompt from "../lib/securityPrompt.js";
@@ -99,6 +103,7 @@ import type {
declare const fullApi: ApiFromModules<{
auth: typeof auth;
commentModeration: typeof commentModeration;
comments: typeof comments;
crons: typeof crons;
devSeed: typeof devSeed;
@@ -128,6 +133,7 @@ declare const fullApi: ApiFromModules<{
"lib/badges": typeof lib_badges;
"lib/batching": typeof lib_batching;
"lib/changelog": typeof lib_changelog;
"lib/commentScamPrompt": typeof lib_commentScamPrompt;
"lib/contentTypes": typeof lib_contentTypes;
"lib/embeddingVisibility": typeof lib_embeddingVisibility;
"lib/embeddings": typeof lib_embeddings;
@@ -143,7 +149,9 @@ declare const fullApi: ApiFromModules<{
"lib/httpRateLimit": typeof lib_httpRateLimit;
"lib/leaderboards": typeof lib_leaderboards;
"lib/moderation": typeof lib_moderation;
"lib/openaiResponse": typeof lib_openaiResponse;
"lib/public": typeof lib_public;
"lib/reporting": typeof lib_reporting;
"lib/reservedSlugs": typeof lib_reservedSlugs;
"lib/searchText": typeof lib_searchText;
"lib/securityPrompt": typeof lib_securityPrompt;
+285
View File
@@ -0,0 +1,285 @@
/* @vitest-environment node */
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
vi.mock('./_generated/api', () => ({
internal: {
commentModeration: {
getCommentScamBackfillPageInternal: Symbol('commentModeration.getCommentScamBackfillPageInternal'),
applyCommentScamResultInternal: Symbol('commentModeration.applyCommentScamResultInternal'),
backfillCommentScamModerationInternal: Symbol('commentModeration.backfillCommentScamModerationInternal'),
continueCommentScamModerationJobInternal: Symbol(
'commentModeration.continueCommentScamModerationJobInternal',
),
},
llmEval: {
evaluateCommentForScam: Symbol('llmEval.evaluateCommentForScam'),
},
users: {
banUserInternal: Symbol('users.banUserInternal'),
},
},
}))
const {
applyCommentScamResultInternalHandler,
backfillCommentScamModerationInternalHandler,
} = await import('./commentModeration')
const { internal } = await import('./_generated/api')
const previousOpenAiApiKey = process.env.OPENAI_API_KEY
beforeEach(() => {
process.env.OPENAI_API_KEY = 'test-key'
})
afterEach(() => {
if (previousOpenAiApiKey === undefined) {
delete process.env.OPENAI_API_KEY
return
}
process.env.OPENAI_API_KEY = previousOpenAiApiKey
})
describe('commentModeration backfill', () => {
it('evaluates comments and bans on certain/high scams', async () => {
const runQuery = vi
.fn()
.mockResolvedValueOnce({
items: [
{
commentId: 'comments:1',
skillId: 'skills:1',
userId: 'users:2',
body: 'echo "mal" | base64 -D | bash',
softDeletedAt: undefined,
scamScanCheckedAt: undefined,
},
],
cursor: null,
isDone: true,
})
const runAction = vi.fn().mockResolvedValue({
ok: true,
model: 'gpt-5-mini',
verdict: 'certain_scam',
confidence: 'high',
explanation: 'Obfuscated shell execution payload.',
evidence: ['base64 decode piped to bash'],
})
const runMutation = vi.fn().mockResolvedValue({
ok: true,
shouldBan: true,
banned: true,
alreadyBanned: false,
protectedRole: false,
wouldBan: false,
})
const result = await backfillCommentScamModerationInternalHandler(
{ runQuery, runAction, runMutation } as never,
{
actorUserId: 'users:admin',
dryRun: false,
batchSize: 10,
maxBatches: 1,
} as never,
)
expect(result.ok).toBe(true)
expect(result.stats.commentsScanned).toBe(1)
expect(result.stats.commentsEvaluated).toBe(1)
expect(result.stats.certainScams).toBe(1)
expect(result.stats.banCandidates).toBe(1)
expect(result.stats.usersBanned).toBe(1)
expect(runAction).toHaveBeenCalledWith(internal.llmEval.evaluateCommentForScam, {
commentId: 'comments:1',
skillId: 'skills:1',
userId: 'users:2',
body: 'echo "mal" | base64 -D | bash',
})
expect(runMutation).toHaveBeenCalledWith(internal.commentModeration.applyCommentScamResultInternal, {
actorUserId: 'users:admin',
commentId: 'comments:1',
verdict: 'certain_scam',
confidence: 'high',
explanation: 'Obfuscated shell execution payload.',
evidence: ['base64 decode piped to bash'],
model: 'gpt-5-mini',
checkedAt: expect.any(Number),
dryRun: false,
})
})
it('skips previously scanned comments unless rescan=true', async () => {
const runQuery = vi.fn().mockResolvedValue({
items: [
{
commentId: 'comments:1',
skillId: 'skills:1',
userId: 'users:2',
body: 'something',
softDeletedAt: undefined,
scamScanCheckedAt: 123,
},
],
cursor: null,
isDone: true,
})
const runAction = vi.fn()
const runMutation = vi.fn()
const result = await backfillCommentScamModerationInternalHandler(
{ runQuery, runAction, runMutation } as never,
{
actorUserId: 'users:admin',
batchSize: 10,
maxBatches: 1,
} as never,
)
expect(result.stats.commentsScanned).toBe(1)
expect(result.stats.skippedAlreadyScanned).toBe(1)
expect(runAction).not.toHaveBeenCalled()
expect(runMutation).not.toHaveBeenCalled()
})
it('tracks dry-run ban candidates without banning', async () => {
const runQuery = vi.fn().mockResolvedValue({
items: [
{
commentId: 'comments:9',
skillId: 'skills:7',
userId: 'users:5',
body: 'run this update installer from random domain',
softDeletedAt: undefined,
scamScanCheckedAt: undefined,
},
],
cursor: null,
isDone: true,
})
const runAction = vi.fn().mockResolvedValue({
ok: true,
model: 'gpt-5-mini',
verdict: 'certain_scam',
confidence: 'high',
explanation: 'Social-engineering install command.',
evidence: ['unknown update domain'],
})
const runMutation = vi.fn().mockResolvedValue({
ok: true,
shouldBan: true,
banned: false,
alreadyBanned: false,
protectedRole: false,
wouldBan: true,
})
const result = await backfillCommentScamModerationInternalHandler(
{ runQuery, runAction, runMutation } as never,
{
actorUserId: 'users:admin',
dryRun: true,
batchSize: 10,
maxBatches: 1,
} as never,
)
expect(result.stats.usersBanned).toBe(0)
expect(result.stats.usersWouldBeBanned).toBe(1)
})
})
describe('applyCommentScamResultInternalHandler', () => {
it('persists scan metadata and triggers ban with bounded reason', async () => {
const get = vi
.fn()
.mockResolvedValueOnce({
_id: 'comments:1',
skillId: 'skills:1',
userId: 'users:2',
})
.mockResolvedValueOnce({
_id: 'users:2',
role: 'user',
})
const patch = vi.fn()
const insert = vi.fn()
const runMutation = vi.fn().mockResolvedValue({ ok: true, alreadyBanned: false, deletedSkills: 0 })
const result = await applyCommentScamResultInternalHandler(
{ db: { get, patch, insert }, runMutation } as never,
{
actorUserId: 'users:admin',
commentId: 'comments:1',
verdict: 'certain_scam',
confidence: 'high',
explanation: 'X'.repeat(700),
evidence: ['Y'.repeat(280), 'Z'.repeat(280)],
model: 'gpt-5-mini',
checkedAt: 123,
} as never,
)
expect(result.banned).toBe(true)
expect(insert).toHaveBeenCalledWith('auditLogs', {
actorUserId: 'users:admin',
action: 'comment.scam_scan',
targetType: 'comment',
targetId: 'comments:1',
metadata: {
skillId: 'skills:1',
commentAuthorId: 'users:2',
verdict: 'certain_scam',
confidence: 'high',
shouldBan: true,
model: 'gpt-5-mini',
},
createdAt: 123,
})
const banCall = runMutation.mock.calls.find(
(call) => call[0] === internal.users.banUserInternal,
)
expect(banCall).toBeTruthy()
if (!banCall) throw new Error('Expected ban mutation to be called')
expect((banCall[1] as { reason: string }).reason.length).toBeLessThanOrEqual(500)
expect(patch).toHaveBeenCalledWith('comments:1', {
scamBanTriggeredAt: 123,
})
})
it('skips banning moderator/admin accounts', async () => {
const get = vi
.fn()
.mockResolvedValueOnce({
_id: 'comments:2',
skillId: 'skills:2',
userId: 'users:staff',
})
.mockResolvedValueOnce({
_id: 'users:staff',
role: 'moderator',
})
const patch = vi.fn()
const insert = vi.fn()
const runMutation = vi.fn()
const result = await applyCommentScamResultInternalHandler(
{ db: { get, patch, insert }, runMutation } as never,
{
actorUserId: 'users:admin',
commentId: 'comments:2',
verdict: 'certain_scam',
confidence: 'high',
explanation: 'Malicious command spam.',
evidence: ['base64|bash'],
model: 'gpt-5-mini',
checkedAt: 300,
} as never,
)
expect(result.protectedRole).toBe(true)
expect(runMutation).not.toHaveBeenCalled()
})
})
+465
View File
@@ -0,0 +1,465 @@
import { ConvexError, v } from 'convex/values'
import { internal } from './_generated/api'
import type { Id } from './_generated/dataModel'
import type { ActionCtx, MutationCtx } from './_generated/server'
import { action, internalAction, internalMutation, internalQuery } from './_generated/server'
import { assertRole, requireUserFromAction } from './lib/access'
import {
buildCommentScamBanReason,
isCertainScam,
type CommentScamConfidence,
type CommentScamVerdict,
} from './lib/commentScamPrompt'
const DEFAULT_BATCH_SIZE = 25
const MAX_BATCH_SIZE = 100
const DEFAULT_MAX_BATCHES = 10
const MAX_MAX_BATCHES = 200
type CommentBackfillPageItem = {
commentId: Id<'comments'>
skillId: Id<'skills'>
userId: Id<'users'>
body: string
softDeletedAt?: number
scamScanCheckedAt?: number
}
type CommentBackfillPageResult = {
items: CommentBackfillPageItem[]
cursor: string | null
isDone: boolean
}
type ApplyCommentScamResult = {
ok: true
shouldBan: boolean
banned: boolean
alreadyBanned: boolean
protectedRole: boolean
wouldBan: boolean
}
export type CommentScamBackfillStats = {
commentsScanned: number
commentsEvaluated: number
certainScams: number
banCandidates: number
usersBanned: number
usersAlreadyBanned: number
usersWouldBeBanned: number
protectedRoleSkips: number
skippedSoftDeleted: number
skippedAlreadyScanned: number
skippedEmptyBody: number
evalErrors: number
}
export type CommentScamBackfillActionArgs = {
actorUserId: Id<'users'>
dryRun?: boolean
batchSize?: number
maxBatches?: number
cursor?: string
rescan?: boolean
includeSoftDeleted?: boolean
}
export type CommentScamBackfillActionResult = {
ok: true
stats: CommentScamBackfillStats
isDone: boolean
cursor: string | null
}
export const getCommentScamBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args): Promise<CommentBackfillPageResult> => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const { page, isDone, continueCursor } = await ctx.db
.query('comments')
.order('asc')
.paginate({ cursor: args.cursor ?? null, numItems: batchSize })
return {
items: page.map((comment) => ({
commentId: comment._id,
skillId: comment.skillId,
userId: comment.userId,
body: comment.body,
softDeletedAt: comment.softDeletedAt,
scamScanCheckedAt: comment.scamScanCheckedAt,
})),
cursor: continueCursor,
isDone,
}
},
})
export async function applyCommentScamResultInternalHandler(
ctx: MutationCtx,
args: {
actorUserId: Id<'users'>
commentId: Id<'comments'>
verdict: CommentScamVerdict
confidence: CommentScamConfidence
explanation: string
evidence: string[]
model: string
checkedAt: number
dryRun?: boolean
},
): Promise<ApplyCommentScamResult> {
const comment = await ctx.db.get(args.commentId)
if (!comment) {
throw new ConvexError('Comment not found')
}
const user = await ctx.db.get(comment.userId)
if (!user) {
throw new ConvexError('Comment author not found')
}
const dryRun = Boolean(args.dryRun)
const shouldBan = isCertainScam({
verdict: args.verdict,
confidence: args.confidence,
})
const explanation = args.explanation.trim().slice(0, 1200)
const evidence = args.evidence
.map((item) => item.trim())
.filter(Boolean)
.slice(0, 5)
if (!dryRun) {
await ctx.db.patch(comment._id, {
scamScanVerdict: args.verdict,
scamScanConfidence: args.confidence,
scamScanExplanation: explanation,
scamScanEvidence: evidence,
scamScanModel: args.model,
scamScanCheckedAt: args.checkedAt,
})
await ctx.db.insert('auditLogs', {
actorUserId: args.actorUserId,
action: 'comment.scam_scan',
targetType: 'comment',
targetId: comment._id,
metadata: {
skillId: comment.skillId,
commentAuthorId: comment.userId,
verdict: args.verdict,
confidence: args.confidence,
shouldBan,
model: args.model,
},
createdAt: args.checkedAt,
})
}
if (!shouldBan) {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: false,
protectedRole: false,
wouldBan: false,
}
}
if (user.role === 'admin' || user.role === 'moderator') {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: false,
protectedRole: true,
wouldBan: false,
}
}
if (user.deletedAt || user.deactivatedAt) {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: true,
protectedRole: false,
wouldBan: false,
}
}
if (dryRun) {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: false,
protectedRole: false,
wouldBan: true,
}
}
const reason = buildCommentScamBanReason({
commentId: String(comment._id),
skillId: String(comment.skillId),
explanation,
evidence,
})
const banResult = await ctx.runMutation(internal.users.banUserInternal, {
actorUserId: args.actorUserId,
targetUserId: comment.userId,
reason,
})
if (!banResult.alreadyBanned) {
await ctx.db.patch(comment._id, {
scamBanTriggeredAt: args.checkedAt,
})
}
return {
ok: true,
shouldBan,
banned: !banResult.alreadyBanned,
alreadyBanned: Boolean(banResult.alreadyBanned),
protectedRole: false,
wouldBan: false,
}
}
export const applyCommentScamResultInternal = internalMutation({
args: {
actorUserId: v.id('users'),
commentId: v.id('comments'),
verdict: v.union(v.literal('not_scam'), v.literal('likely_scam'), v.literal('certain_scam')),
confidence: v.union(v.literal('low'), v.literal('medium'), v.literal('high')),
explanation: v.string(),
evidence: v.array(v.string()),
model: v.string(),
checkedAt: v.number(),
dryRun: v.optional(v.boolean()),
},
handler: applyCommentScamResultInternalHandler,
})
export async function backfillCommentScamModerationInternalHandler(
ctx: ActionCtx,
args: CommentScamBackfillActionArgs,
): Promise<CommentScamBackfillActionResult> {
if (!process.env.OPENAI_API_KEY) {
throw new ConvexError('OPENAI_API_KEY not configured')
}
const dryRun = Boolean(args.dryRun)
const rescan = Boolean(args.rescan)
const includeSoftDeleted = Boolean(args.includeSoftDeleted)
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
let cursor: string | null = args.cursor ?? null
let isDone = false
const stats: CommentScamBackfillStats = {
commentsScanned: 0,
commentsEvaluated: 0,
certainScams: 0,
banCandidates: 0,
usersBanned: 0,
usersAlreadyBanned: 0,
usersWouldBeBanned: 0,
protectedRoleSkips: 0,
skippedSoftDeleted: 0,
skippedAlreadyScanned: 0,
skippedEmptyBody: 0,
evalErrors: 0,
}
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(internal.commentModeration.getCommentScamBackfillPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as CommentBackfillPageResult
cursor = page.cursor
isDone = page.isDone
for (const comment of page.items) {
stats.commentsScanned++
if (!includeSoftDeleted && comment.softDeletedAt) {
stats.skippedSoftDeleted++
continue
}
if (!rescan && comment.scamScanCheckedAt) {
stats.skippedAlreadyScanned++
continue
}
const body = comment.body.trim()
if (!body) {
stats.skippedEmptyBody++
continue
}
const evalResult = (await ctx.runAction(internal.llmEval.evaluateCommentForScam, {
commentId: comment.commentId,
skillId: comment.skillId,
userId: comment.userId,
body,
})) as
| {
ok: true
model: string
verdict: CommentScamVerdict
confidence: CommentScamConfidence
explanation: string
evidence: string[]
}
| { ok: false; error: string }
if (!evalResult.ok) {
stats.evalErrors++
continue
}
stats.commentsEvaluated++
const shouldBan = isCertainScam(evalResult)
if (evalResult.verdict === 'certain_scam') {
stats.certainScams++
}
if (shouldBan) {
stats.banCandidates++
}
const applyResult = (await ctx.runMutation(internal.commentModeration.applyCommentScamResultInternal, {
actorUserId: args.actorUserId,
commentId: comment.commentId,
verdict: evalResult.verdict,
confidence: evalResult.confidence,
explanation: evalResult.explanation,
evidence: evalResult.evidence,
model: evalResult.model,
checkedAt: Date.now(),
dryRun,
})) as ApplyCommentScamResult
if (applyResult.banned) stats.usersBanned++
if (applyResult.alreadyBanned) stats.usersAlreadyBanned++
if (applyResult.wouldBan) stats.usersWouldBeBanned++
if (applyResult.protectedRole) stats.protectedRoleSkips++
}
if (isDone) break
}
return {
ok: true,
stats,
isDone,
cursor,
}
}
export const backfillCommentScamModerationInternal = internalAction({
args: {
actorUserId: v.id('users'),
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
cursor: v.optional(v.string()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: backfillCommentScamModerationInternalHandler,
})
export const backfillCommentScamModeration: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
cursor: v.optional(v.string()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<CommentScamBackfillActionResult> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin', 'moderator'])
return ctx.runAction(internal.commentModeration.backfillCommentScamModerationInternal, {
actorUserId: user._id,
...args,
}) as Promise<CommentScamBackfillActionResult>
},
})
export const continueCommentScamModerationJobInternal = internalAction({
args: {
actorUserId: v.id('users'),
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
cursor: v.optional(v.string()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const result = await backfillCommentScamModerationInternalHandler(ctx, {
actorUserId: args.actorUserId,
dryRun: args.dryRun,
batchSize: args.batchSize,
cursor: args.cursor,
maxBatches: 1,
rescan: args.rescan,
includeSoftDeleted: args.includeSoftDeleted,
})
if (!result.isDone && result.cursor) {
await ctx.scheduler.runAfter(2_000, internal.commentModeration.continueCommentScamModerationJobInternal, {
actorUserId: args.actorUserId,
dryRun: Boolean(args.dryRun),
batchSize: args.batchSize ?? DEFAULT_BATCH_SIZE,
cursor: result.cursor,
rescan: Boolean(args.rescan),
includeSoftDeleted: Boolean(args.includeSoftDeleted),
})
}
return result
},
})
export const scheduleCommentScamModeration: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<{ ok: true }> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin', 'moderator'])
await ctx.scheduler.runAfter(0, internal.commentModeration.continueCommentScamModerationJobInternal, {
actorUserId: user._id,
dryRun: Boolean(args.dryRun),
batchSize: clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE),
cursor: undefined,
rescan: Boolean(args.rescan),
includeSoftDeleted: Boolean(args.includeSoftDeleted),
})
return { ok: true as const }
},
})
function clampInt(value: number, min: number, max: number) {
return Math.min(Math.max(Math.trunc(value), min), max)
}
+99
View File
@@ -1,10 +1,18 @@
import type { Id } from './_generated/dataModel'
import type { MutationCtx } from './_generated/server'
import { assertModerator, requireUser } from './lib/access'
import { requireGitHubAccountAge } from './lib/githubAccount'
import {
AUTO_HIDE_REPORT_THRESHOLD,
MAX_ACTIVE_REPORTS_PER_USER,
MAX_REPORT_REASON_LENGTH,
} from './lib/reporting'
import { insertStatEvent } from './skillStatEvents'
export async function addHandler(ctx: MutationCtx, args: { skillId: Id<'skills'>; body: string }) {
const { userId } = await requireUser(ctx)
await requireGitHubAccountAge(ctx, userId)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
@@ -50,3 +58,94 @@ export async function removeHandler(ctx: MutationCtx, args: { commentId: Id<'com
createdAt: Date.now(),
})
}
async function countActiveReportsForUser(ctx: MutationCtx, userId: Id<'users'>) {
const reports = await ctx.db
.query('commentReports')
.withIndex('by_user', (q) => q.eq('userId', userId))
.collect()
let count = 0
for (const report of reports) {
const comment = await ctx.db.get(report.commentId)
if (!comment || comment.softDeletedAt) continue
const skill = await ctx.db.get(comment.skillId)
if (!skill || skill.softDeletedAt || skill.moderationStatus === 'removed') continue
const owner = await ctx.db.get(comment.userId)
if (!owner || owner.deletedAt || owner.deactivatedAt) continue
count += 1
if (count >= MAX_ACTIVE_REPORTS_PER_USER) break
}
return count
}
export async function reportHandler(
ctx: MutationCtx,
args: { commentId: Id<'comments'>; reason: string },
) {
const { userId } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment || comment.softDeletedAt) {
throw new Error('Comment not found')
}
const skill = await ctx.db.get(comment.skillId)
if (!skill || skill.softDeletedAt || skill.moderationStatus === 'removed') {
throw new Error('Comment not found')
}
const reason = args.reason.trim()
if (!reason) {
throw new Error('Report reason required.')
}
const existing = await ctx.db
.query('commentReports')
.withIndex('by_comment_user', (q) => q.eq('commentId', args.commentId).eq('userId', userId))
.unique()
if (existing) return { ok: true as const, reported: false, alreadyReported: true }
const activeReports = await countActiveReportsForUser(ctx, userId)
if (activeReports >= MAX_ACTIVE_REPORTS_PER_USER) {
throw new Error('Report limit reached. Please wait for moderation before reporting more.')
}
const now = Date.now()
await ctx.db.insert('commentReports', {
commentId: args.commentId,
skillId: comment.skillId,
userId,
reason: reason.slice(0, MAX_REPORT_REASON_LENGTH),
createdAt: now,
})
const nextReportCount = (comment.reportCount ?? 0) + 1
const shouldAutoHide = nextReportCount > AUTO_HIDE_REPORT_THRESHOLD && !comment.softDeletedAt
const updates: {
reportCount: number
lastReportedAt: number
softDeletedAt?: number
} = {
reportCount: nextReportCount,
lastReportedAt: now,
}
if (shouldAutoHide) {
updates.softDeletedAt = now
}
await ctx.db.patch(comment._id, updates)
if (shouldAutoHide) {
await insertStatEvent(ctx, { skillId: comment.skillId, kind: 'uncomment' })
await ctx.db.insert('auditLogs', {
actorUserId: userId,
action: 'comment.auto_hide',
targetType: 'comment',
targetId: comment._id,
metadata: { skillId: comment.skillId, reportCount: nextReportCount },
createdAt: now,
})
}
return { ok: true as const, reported: true, alreadyReported: false }
}
+127
View File
@@ -0,0 +1,127 @@
/* @vitest-environment node */
import { describe, expect, it } from 'vitest'
import { listBySkill } from './comments'
function makeCtx(args: {
comments: Array<Record<string, unknown>>
usersById: Record<string, Record<string, unknown> | null>
}) {
const get = async (id: string) => args.usersById[id] ?? null
const take = async () => args.comments
const order = () => ({ take })
const withIndex = () => ({ order })
const query = () => ({ withIndex })
return { db: { get, query } } as never
}
describe('comments.listBySkill', () => {
it('skips soft-deleted comments', async () => {
const ctx = makeCtx({
comments: [
{
_id: 'comments:live',
skillId: 'skills:1',
userId: 'users:live',
body: 'hello',
},
{
_id: 'comments:deleted',
skillId: 'skills:1',
userId: 'users:live',
body: 'bye',
softDeletedAt: 123,
},
],
usersById: {
'users:live': {
_id: 'users:live',
_creationTime: 1,
handle: 'live',
name: 'live',
displayName: 'Live',
image: null,
bio: null,
},
},
})
const result = await listBySkill._handler(ctx, {
skillId: 'skills:1',
limit: 50,
} as never)
expect(result).toHaveLength(1)
expect(result[0]?.comment._id).toBe('comments:live')
})
it('skips comments whose author is deleted/deactivated/missing', async () => {
const ctx = makeCtx({
comments: [
{
_id: 'comments:ok',
skillId: 'skills:1',
userId: 'users:ok',
body: 'ok',
},
{
_id: 'comments:deleted-user',
skillId: 'skills:1',
userId: 'users:deleted',
body: 'hidden',
},
{
_id: 'comments:deactivated-user',
skillId: 'skills:1',
userId: 'users:deactivated',
body: 'hidden',
},
{
_id: 'comments:missing-user',
skillId: 'skills:1',
userId: 'users:missing',
body: 'hidden',
},
],
usersById: {
'users:ok': {
_id: 'users:ok',
_creationTime: 1,
handle: 'ok',
name: 'ok',
displayName: 'Ok',
image: null,
bio: null,
},
'users:deleted': {
_id: 'users:deleted',
_creationTime: 1,
handle: 'deleted',
name: 'deleted',
displayName: 'Deleted',
image: null,
bio: null,
deletedAt: 123,
},
'users:deactivated': {
_id: 'users:deactivated',
_creationTime: 1,
handle: 'deactivated',
name: 'deactivated',
displayName: 'Deactivated',
image: null,
bio: null,
deactivatedAt: 456,
},
},
})
const result = await listBySkill._handler(ctx, {
skillId: 'skills:1',
limit: 50,
} as never)
expect(result).toHaveLength(1)
expect(result[0]?.comment._id).toBe('comments:ok')
expect(result[0]?.user._id).toBe('users:ok')
})
})
+460 -1
View File
@@ -10,15 +10,22 @@ vi.mock('./skillStatEvents', () => ({
insertStatEvent: vi.fn(),
}))
vi.mock('./lib/githubAccount', () => ({
requireGitHubAccountAge: vi.fn(),
}))
const { requireUser, assertModerator } = await import('./lib/access')
const { insertStatEvent } = await import('./skillStatEvents')
const { addHandler, removeHandler } = await import('./comments.handlers')
const { requireGitHubAccountAge } = await import('./lib/githubAccount')
const { addHandler, removeHandler, reportHandler } = await import('./comments.handlers')
describe('comments mutations', () => {
afterEach(() => {
vi.mocked(assertModerator).mockReset()
vi.mocked(requireUser).mockReset()
vi.mocked(insertStatEvent).mockReset()
vi.mocked(requireGitHubAccountAge).mockReset()
vi.restoreAllMocks()
})
it('add avoids direct skill patch and records stat event', async () => {
@@ -26,6 +33,7 @@ describe('comments mutations', () => {
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
vi.mocked(requireGitHubAccountAge).mockResolvedValue(undefined as never)
const get = vi.fn().mockResolvedValue({
_id: 'skills:1',
@@ -36,6 +44,7 @@ describe('comments mutations', () => {
await addHandler(ctx, { skillId: 'skills:1', body: ' hello ' } as never)
expect(requireGitHubAccountAge).toHaveBeenCalledWith(ctx, 'users:1')
expect(patch).not.toHaveBeenCalled()
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: 'skills:1',
@@ -43,6 +52,30 @@ describe('comments mutations', () => {
})
})
it('add blocks new comments when github account age gate fails', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:new',
user: { _id: 'users:new', role: 'user' },
} as never)
vi.mocked(requireGitHubAccountAge).mockRejectedValue(
new Error('GitHub account must be at least 14 days old to upload skills. Try again in 3 days.'),
)
const get = vi.fn()
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await expect(addHandler(ctx, { skillId: 'skills:1', body: 'hello' } as never)).rejects.toThrow(
/at least 14 days old/i,
)
expect(get).not.toHaveBeenCalled()
expect(insert).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
expect(insertStatEvent).not.toHaveBeenCalled()
})
it('remove keeps comment soft-delete patch free of updatedAt', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:2',
@@ -57,6 +90,9 @@ describe('comments mutations', () => {
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:1') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
return null
})
const insert = vi.fn()
@@ -124,4 +160,427 @@ describe('comments mutations', () => {
expect(insert).not.toHaveBeenCalled()
expect(insertStatEvent).not.toHaveBeenCalled()
})
it('report increments count and stores reason', async () => {
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000)
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:1',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 1,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:1') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn((table: string) => {
if (table === 'commentReports') {
return {
withIndex: (index: string) => {
if (index === 'by_comment_user') {
return { unique: vi.fn().mockResolvedValue(null) }
}
if (index === 'by_user') {
return { collect: vi.fn().mockResolvedValue([]) }
}
throw new Error(`Unexpected index ${index}`)
},
}
}
throw new Error(`Unexpected table ${table}`)
})
const ctx = { db: { get, insert, patch, query } } as never
const result = await reportHandler(ctx, { commentId: 'comments:1', reason: ' spam ' } as never)
expect(result).toEqual({ ok: true, reported: true, alreadyReported: false })
expect(insert).toHaveBeenCalledWith('commentReports', {
commentId: 'comments:1',
skillId: 'skills:1',
userId: 'users:1',
reason: 'spam',
createdAt: 1_700_000_000_000,
})
expect(patch).toHaveBeenCalledWith('comments:1', {
reportCount: 2,
lastReportedAt: 1_700_000_000_000,
})
expect(insertStatEvent).not.toHaveBeenCalled()
})
it('report returns alreadyReported for duplicate reporter/comment pair', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:dup',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 0,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:dup') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn((table: string) => {
if (table !== 'commentReports') throw new Error(`Unexpected table ${table}`)
return {
withIndex: (index: string) => {
if (index === 'by_comment_user') {
return { unique: vi.fn().mockResolvedValue({ _id: 'commentReports:existing' }) }
}
throw new Error(`Unexpected index ${index}`)
},
}
})
const ctx = { db: { get, insert, patch, query } } as never
const result = await reportHandler(ctx, { commentId: 'comments:dup', reason: 'spam' } as never)
expect(result).toEqual({ ok: true, reported: false, alreadyReported: true })
expect(insert).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
})
it('report rejects empty reason', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:empty',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 0,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:empty') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn()
const ctx = { db: { get, insert, patch, query } } as never
await expect(
reportHandler(ctx, { commentId: 'comments:empty', reason: ' ' } as never),
).rejects.toThrow('Report reason required.')
expect(query).not.toHaveBeenCalled()
expect(insert).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
})
it('report rejects comment when parent skill is hidden/removed', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:hidden-parent',
skillId: 'skills:hidden',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 0,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:hidden-parent') return comment
if (id === 'skills:hidden') {
return { _id: 'skills:hidden', softDeletedAt: 123, moderationStatus: 'removed' }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn()
const ctx = { db: { get, insert, patch, query } } as never
await expect(
reportHandler(ctx, { commentId: 'comments:hidden-parent', reason: 'abuse' } as never),
).rejects.toThrow('Comment not found')
expect(query).not.toHaveBeenCalled()
expect(insert).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
})
it('report truncates long reason to 500 chars', async () => {
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_050)
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:long',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 0,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:long') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn((table: string) => {
if (table !== 'commentReports') throw new Error(`Unexpected table ${table}`)
return {
withIndex: (index: string) => {
if (index === 'by_comment_user') return { unique: vi.fn().mockResolvedValue(null) }
if (index === 'by_user') return { collect: vi.fn().mockResolvedValue([]) }
throw new Error(`Unexpected index ${index}`)
},
}
})
const ctx = { db: { get, insert, patch, query } } as never
await reportHandler(ctx, { commentId: 'comments:long', reason: 'x'.repeat(700) } as never)
const reportInsert = vi.mocked(insert).mock.calls.find((call) => call[0] === 'commentReports')
expect(reportInsert?.[1]).toMatchObject({
commentId: 'comments:long',
reason: 'x'.repeat(500),
})
})
it('report active-count filter ignores stale/non-active report targets', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:target2',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 0,
}
const reports = [
{ _id: 'commentReports:1', commentId: 'comments:deleted', userId: 'users:1', skillId: 'skills:1' },
{ _id: 'commentReports:2', commentId: 'comments:removed-skill', userId: 'users:1', skillId: 'skills:removed' },
{ _id: 'commentReports:3', commentId: 'comments:deleted-owner', userId: 'users:1', skillId: 'skills:active' },
]
const get = vi.fn(async (id: string) => {
if (id === 'comments:target2') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
if (id === 'comments:deleted') {
return { _id: 'comments:deleted', softDeletedAt: 123, skillId: 'skills:1', userId: 'users:2' }
}
if (id === 'comments:removed-skill') {
return {
_id: 'comments:removed-skill',
softDeletedAt: undefined,
skillId: 'skills:removed',
userId: 'users:2',
}
}
if (id === 'skills:removed') {
return { _id: 'skills:removed', softDeletedAt: undefined, moderationStatus: 'removed' }
}
if (id === 'comments:deleted-owner') {
return {
_id: 'comments:deleted-owner',
softDeletedAt: undefined,
skillId: 'skills:active',
userId: 'users:deleted-owner',
}
}
if (id === 'skills:active') {
return { _id: 'skills:active', softDeletedAt: undefined, moderationStatus: 'active' }
}
if (id === 'users:deleted-owner') {
return { _id: 'users:deleted-owner', deletedAt: 1, deactivatedAt: undefined }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn((table: string) => {
if (table !== 'commentReports') throw new Error(`Unexpected table ${table}`)
return {
withIndex: (index: string) => {
if (index === 'by_comment_user') return { unique: vi.fn().mockResolvedValue(null) }
if (index === 'by_user') return { collect: vi.fn().mockResolvedValue(reports) }
throw new Error(`Unexpected index ${index}`)
},
}
})
const ctx = { db: { get, insert, patch, query } } as never
const result = await reportHandler(
ctx,
{ commentId: 'comments:target2', reason: 'still allowed' } as never,
)
expect(result).toEqual({ ok: true, reported: true, alreadyReported: false })
expect(insert).toHaveBeenCalledWith(
'commentReports',
expect.objectContaining({ commentId: 'comments:target2', userId: 'users:1' }),
)
})
it('report rejects when active report limit is reached', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
const comment = {
_id: 'comments:target',
skillId: 'skills:1',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 0,
}
const reportedComment = {
_id: 'comments:reported',
skillId: 'skills:active',
userId: 'users:owner',
softDeletedAt: undefined,
}
const reports = Array.from({ length: 20 }, (_, i) => ({
_id: `commentReports:${i + 1}`,
commentId: `comments:reported-${i + 1}`,
userId: 'users:1',
skillId: 'skills:active',
createdAt: i + 1,
}))
const get = vi.fn(async (id: string) => {
if (id === 'comments:target') return comment
if (id === 'skills:1') {
return { _id: 'skills:1', softDeletedAt: undefined, moderationStatus: 'active' }
}
if (String(id).startsWith('comments:reported-')) return reportedComment
if (id === 'skills:active') {
return { _id: 'skills:active', softDeletedAt: undefined, moderationStatus: 'active' }
}
if (id === 'users:owner') {
return { _id: 'users:owner', deletedAt: undefined, deactivatedAt: undefined }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn((table: string) => {
if (table === 'commentReports') {
return {
withIndex: (index: string) => {
if (index === 'by_comment_user') {
return { unique: vi.fn().mockResolvedValue(null) }
}
if (index === 'by_user') {
return { collect: vi.fn().mockResolvedValue(reports) }
}
throw new Error(`Unexpected index ${index}`)
},
}
}
throw new Error(`Unexpected table ${table}`)
})
const ctx = { db: { get, insert, patch, query } } as never
await expect(
reportHandler(ctx, { commentId: 'comments:target', reason: 'abuse' } as never),
).rejects.toThrow('Report limit reached. Please wait for moderation before reporting more.')
expect(insert).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
})
it('report auto-hides comment after fourth unique report', async () => {
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_100)
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:3',
user: { _id: 'users:3', role: 'user' },
} as never)
const comment = {
_id: 'comments:4',
skillId: 'skills:9',
userId: 'users:2',
softDeletedAt: undefined,
reportCount: 3,
}
const get = vi.fn(async (id: string) => {
if (id === 'comments:4') return comment
if (id === 'skills:9') {
return { _id: 'skills:9', softDeletedAt: undefined, moderationStatus: 'active' }
}
return null
})
const insert = vi.fn()
const patch = vi.fn()
const query = vi.fn((table: string) => {
if (table === 'commentReports') {
return {
withIndex: (index: string) => {
if (index === 'by_comment_user') {
return { unique: vi.fn().mockResolvedValue(null) }
}
if (index === 'by_user') {
return { collect: vi.fn().mockResolvedValue([]) }
}
throw new Error(`Unexpected index ${index}`)
},
}
}
throw new Error(`Unexpected table ${table}`)
})
const ctx = { db: { get, insert, patch, query } } as never
const result = await reportHandler(ctx, { commentId: 'comments:4', reason: ' hate ' } as never)
expect(result).toEqual({ ok: true, reported: true, alreadyReported: false })
expect(patch).toHaveBeenCalledWith('comments:4', {
reportCount: 4,
lastReportedAt: 1_700_000_000_100,
softDeletedAt: 1_700_000_000_100,
})
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: 'skills:9',
kind: 'uncomment',
})
expect(insert).toHaveBeenCalledWith('auditLogs', {
actorUserId: 'users:3',
action: 'comment.auto_hide',
targetType: 'comment',
targetId: 'comments:4',
metadata: { skillId: 'skills:9', reportCount: 4 },
createdAt: 1_700_000_000_100,
})
})
})
+14 -9
View File
@@ -1,7 +1,7 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { addHandler, removeHandler } from './comments.handlers'
import { addHandler, removeHandler, reportHandler } from './comments.handlers'
import { type PublicUser, toPublicUser } from './lib/public'
export const listBySkill = query({
@@ -14,15 +14,15 @@ export const listBySkill = query({
.order('desc')
.take(limit)
const visible = comments.filter((comment) => !comment.softDeletedAt)
return Promise.all(
visible.map(
async (comment): Promise<{ comment: Doc<'comments'>; user: PublicUser | null }> => ({
comment,
user: toPublicUser(await ctx.db.get(comment.userId)),
}),
),
const rows = await Promise.all(
comments.map(async (comment): Promise<{ comment: Doc<'comments'>; user: PublicUser } | null> => {
if (comment.softDeletedAt) return null
const user = toPublicUser(await ctx.db.get(comment.userId))
if (!user) return null
return { comment, user }
}),
)
return rows.filter((row): row is { comment: Doc<'comments'>; user: PublicUser } => row !== null)
},
})
@@ -35,3 +35,8 @@ export const remove = mutation({
args: { commentId: v.id('comments') },
handler: removeHandler,
})
export const report = mutation({
args: { commentId: v.id('comments'), reason: v.string() },
handler: reportHandler,
})
+36
View File
@@ -0,0 +1,36 @@
/* @vitest-environment node */
import { describe, expect, it } from 'vitest'
import { __test } from './githubImport'
import { buildGitHubZipForTests } from './lib/githubImport'
describe('githubImport', () => {
it('formats storage failure message with file context', () => {
const message = __test.buildStoreFailureMessage('skill/SKILL.md', 123, new Error('disk full'))
expect(message).toBe('Failed to store file "skill/SKILL.md" (123 bytes). disk full')
})
it('formats publish failure message with fallback text', () => {
expect(__test.buildPublishFailureMessage(new Error('slug exists'))).toBe(
'Import failed during publish: slug exists. Check skill format, slug availability, and try again.',
)
expect(__test.buildPublishFailureMessage('unexpected')).toBe(
'Import failed during publish: unexpected. Check skill format, slug availability, and try again.',
)
})
it('filters mac junk files while unzipping archive entries', () => {
const zip = buildGitHubZipForTests({
'demo-repo/skill/SKILL.md': '# Demo',
'demo-repo/skill/notes.md': 'notes',
'demo-repo/skill/.DS_Store': 'junk',
'demo-repo/skill/._notes.md': 'junk',
'demo-repo/__MACOSX/._SKILL.md': 'junk',
})
const entries = __test.unzipToEntries(zip)
expect(Object.keys(entries).sort()).toEqual([
'demo-repo/skill/SKILL.md',
'demo-repo/skill/notes.md',
])
})
})
+46 -26
View File
@@ -20,7 +20,7 @@ import {
suggestVersion,
} from './lib/githubImport'
import { publishVersionForUser } from './lib/skillPublish'
import { sanitizePath } from './lib/skills'
import { isMacJunkPath, sanitizePath } from './lib/skills'
const MAX_SELECTED_BYTES = 50 * 1024 * 1024
const MAX_UNZIPPED_BYTES = 80 * 1024 * 1024
@@ -192,7 +192,12 @@ export const importGitHubSkill = action({
const sha256 = await sha256Hex(bytes)
const safeBytes = new Uint8Array(bytes)
const storageId = await ctx.storage.store(new Blob([safeBytes], { type: 'text/plain' }))
let storageId: Id<'_storage'>
try {
storageId = await ctx.storage.store(new Blob([safeBytes], { type: 'text/plain' }))
} catch (error) {
throw new ConvexError(buildStoreFailureMessage(sanitized, bytes.byteLength, error))
}
storedFiles.push({
path: sanitized,
size: bytes.byteLength,
@@ -213,23 +218,28 @@ export const importGitHubSkill = action({
if (!displayName) throw new ConvexError('Display name required')
if (!version || !semver.valid(version)) throw new ConvexError('Version must be valid semver')
const result = await publishVersionForUser(ctx, userId, {
slug: slugBase,
displayName,
version,
changelog: '',
tags,
files: storedFiles,
source: {
kind: 'github',
url: resolved.originalUrl,
repo: `${resolved.owner}/${resolved.repo}`,
ref: resolved.ref,
commit: resolved.commit,
path: candidate.path,
importedAt: Date.now(),
},
})
let result: Awaited<ReturnType<typeof publishVersionForUser>>
try {
result = await publishVersionForUser(ctx, userId, {
slug: slugBase,
displayName,
version,
changelog: '',
tags,
files: storedFiles,
source: {
kind: 'github',
url: resolved.originalUrl,
repo: `${resolved.owner}/${resolved.repo}`,
ref: resolved.ref,
commit: resolved.commit,
path: candidate.path,
importedAt: Date.now(),
},
})
} catch (error) {
throw new ConvexError(buildPublishFailureMessage(error))
}
return { ok: true, slug: slugBase, version, ...result }
},
@@ -244,7 +254,7 @@ function unzipToEntries(zipBytes: Uint8Array) {
for (const [rawPath, bytes] of Object.entries(entries)) {
const normalizedPath = normalizeZipPath(rawPath)
if (!normalizedPath) continue
if (isJunkPath(normalizedPath)) continue
if (isMacJunkPath(normalizedPath)) continue
if (!bytes) continue
if (bytes.byteLength > MAX_SINGLE_FILE_BYTES) continue
totalBytes += bytes.byteLength
@@ -308,10 +318,20 @@ function normalizeZipPath(path: string) {
return normalized
}
function isJunkPath(path: string) {
const normalized = path.toLowerCase()
if (normalized.startsWith('__macosx/')) return true
if (normalized.endsWith('/.ds_store')) return true
if (normalized === '.ds_store') return true
return false
function toErrorMessage(error: unknown) {
return error instanceof Error ? error.message : String(error)
}
function buildStoreFailureMessage(path: string, sizeBytes: number, error: unknown) {
return `Failed to store file "${path}" (${sizeBytes} bytes). ${toErrorMessage(error)}`
}
function buildPublishFailureMessage(error: unknown) {
return `Import failed during publish: ${toErrorMessage(error)}. Check skill format, slug availability, and try again.`
}
export const __test = {
buildPublishFailureMessage,
buildStoreFailureMessage,
unzipToEntries,
}
+57
View File
@@ -237,6 +237,19 @@ describe('httpApiV1 handlers', () => {
expect(response.status).toBe(429)
})
it('429 Retry-After is a relative delay, not an absolute epoch', async () => {
const runMutation = vi.fn().mockResolvedValue(blockedRate())
const response = await __handlers.searchSkillsV1Handler(
makeCtx({ runAction: vi.fn(), runMutation }),
new Request('https://example.com/api/v1/search?q=test'),
)
expect(response.status).toBe(429)
const retryAfter = Number(response.headers.get('Retry-After'))
// Retry-After must be a small relative delay (seconds), not a Unix epoch
expect(retryAfter).toBeGreaterThanOrEqual(1)
expect(retryAfter).toBeLessThanOrEqual(120)
})
it('resolve validates hash', async () => {
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.resolveSkillVersionV1Handler(
@@ -859,6 +872,50 @@ describe('httpApiV1 handlers', () => {
}
})
it('publish multipart ignores mac junk files', async () => {
vi.mocked(requireApiTokenUser).mockResolvedValueOnce({
userId: 'users:1',
user: { handle: 'p' },
} as never)
vi.mocked(publishVersionForUser).mockResolvedValueOnce({
skillId: 's',
versionId: 'v',
embeddingId: 'e',
} as never)
const runMutation = vi.fn().mockResolvedValue(okRate())
const store = vi.fn().mockResolvedValue('storage:1')
const form = new FormData()
form.set(
'payload',
JSON.stringify({
slug: 'demo',
displayName: 'Demo',
version: '1.0.0',
changelog: '',
tags: ['latest'],
}),
)
form.append('files', new Blob(['hello'], { type: 'text/plain' }), 'SKILL.md')
form.append('files', new Blob(['junk'], { type: 'application/octet-stream' }), '.DS_Store')
const response = await __handlers.publishSkillV1Handler(
makeCtx({ runMutation, storage: { store } }),
new Request('https://example.com/api/v1/skills', {
method: 'POST',
headers: { Authorization: 'Bearer clh_test' },
body: form,
}),
)
if (response.status !== 200) {
throw new Error(await response.text())
}
expect(store).toHaveBeenCalledTimes(1)
const publishArgs = vi.mocked(publishVersionForUser).mock.calls[0]?.[2] as
| { files?: Array<{ path: string }> }
| undefined
expect(publishArgs?.files?.map((file) => file.path)).toEqual(['SKILL.md'])
})
it('publish rejects missing token', async () => {
const runMutation = vi.fn().mockResolvedValue(okRate())
const response = await __handlers.publishSkillV1Handler(
+2
View File
@@ -5,6 +5,7 @@ import type { ActionCtx } from '../_generated/server'
import { assertAdmin } from '../lib/access'
import { requireApiTokenUser } from '../lib/apiTokenAuth'
import { corsHeaders, mergeHeaders } from '../lib/httpHeaders'
import { isMacJunkPath } from '../lib/skills'
export const MAX_RAW_FILE_BYTES = 200 * 1024
@@ -259,6 +260,7 @@ export async function parseMultipartPublish(
const file = toFileLike(entry)
if (!file) continue
const path = file.name
if (isMacJunkPath(path)) continue
const size = file.size
const contentType = file.type || undefined
const buffer = new Uint8Array(await file.arrayBuffer())
+56 -1
View File
@@ -41,7 +41,12 @@ type ListSkillsResult = {
updatedAt: number
latestVersionId?: Id<'skillVersions'>
}
latestVersion: { version: string; createdAt: number; changelog: string } | null
latestVersion: {
version: string
createdAt: number
changelog: string
parsed?: { clawdis?: { os?: string[]; nix?: { plugin?: boolean; systems?: string[] } } }
} | null
}>
nextCursor: string | null
}
@@ -202,6 +207,12 @@ export async function listSkillsV1Handler(ctx: ActionCtx, request: Request) {
changelog: item.latestVersion.changelog,
}
: null,
metadata: item.latestVersion?.parsed?.clawdis
? {
os: item.latestVersion.parsed.clawdis.os ?? null,
systems: item.latestVersion.parsed.clawdis.nix?.systems ?? null,
}
: null,
}))
return json({ items, nextCursor: result.nextCursor ?? null }, 200, rate.headers)
@@ -292,6 +303,12 @@ export async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request)
changelog: result.latestVersion.changelog,
}
: null,
metadata: result.latestVersion?.parsed?.clawdis
? {
os: result.latestVersion.parsed.clawdis.os ?? null,
systems: result.latestVersion.parsed.clawdis.nix?.systems ?? null,
}
: null,
owner: result.owner
? {
handle: result.owner.handle ?? null,
@@ -348,6 +365,43 @@ export async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request)
if (!version) return text('Version not found', 404, rate.headers)
if (version.softDeletedAt) return text('Version not available', 410, rate.headers)
// Map llmAnalysis to security status
let security = undefined
if (version.llmAnalysis) {
const analysis = version.llmAnalysis
let status: 'clean' | 'suspicious' | 'malicious' | 'pending' | 'error'
switch (analysis.verdict) {
case 'benign':
status = 'clean'
break
case 'suspicious':
status = 'suspicious'
break
case 'malicious':
status = 'malicious'
break
default:
status = analysis.status === 'error' ? 'error' : 'pending'
}
const hasWarnings =
analysis.verdict === 'suspicious' ||
analysis.verdict === 'malicious' ||
(Array.isArray(analysis.dimensions) &&
analysis.dimensions.some((dimension) => {
if (!dimension || typeof dimension !== 'object') return false
const rating = (dimension as { rating?: unknown }).rating
return typeof rating === 'string' && rating !== 'ok'
}))
security = {
status,
hasWarnings,
checkedAt: analysis.checkedAt ?? null,
model: analysis.model || null,
}
}
return json(
{
skill: { slug: skill.slug, displayName: skill.displayName },
@@ -362,6 +416,7 @@ export async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request)
sha256: file.sha256,
contentType: file.contentType ?? null,
})),
security,
},
},
200,
+4 -2
View File
@@ -1,6 +1,6 @@
import { getAuthUserId } from '@convex-dev/auth/server'
import { internal } from '../_generated/api'
import type { Doc } from '../_generated/dataModel'
import type { Doc, Id } from '../_generated/dataModel'
import type { ActionCtx, MutationCtx, QueryCtx } from '../_generated/server'
export type Role = 'admin' | 'moderator' | 'user'
@@ -13,7 +13,9 @@ export async function requireUser(ctx: MutationCtx | QueryCtx) {
return { userId, user }
}
export async function requireUserFromAction(ctx: ActionCtx) {
export async function requireUserFromAction(
ctx: ActionCtx,
): Promise<{ userId: Id<'users'>; user: Doc<'users'> }> {
const userId = await getAuthUserId(ctx)
if (!userId) throw new Error('Unauthorized')
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
+77
View File
@@ -0,0 +1,77 @@
/* @vitest-environment node */
import { describe, expect, it } from 'vitest'
import {
assembleCommentScamEvalUserMessage,
buildCommentScamBanReason,
isCertainScam,
parseCommentScamEvalResponse,
} from './commentScamPrompt'
describe('commentScamPrompt', () => {
it('parses valid JSON response', () => {
const parsed = parseCommentScamEvalResponse(
JSON.stringify({
verdict: 'certain_scam',
confidence: 'high',
explanation: 'Comment instructs users to decode base64 and pipe to bash.',
evidence: ['echo + base64 -D | bash', 'fake update-service domain'],
}),
)
expect(parsed).toEqual({
verdict: 'certain_scam',
confidence: 'high',
explanation: 'Comment instructs users to decode base64 and pipe to bash.',
evidence: ['echo + base64 -D | bash', 'fake update-service domain'],
})
})
it('parses markdown-fenced JSON', () => {
const parsed = parseCommentScamEvalResponse(`\`\`\`json
{"verdict":"likely_scam","confidence":"medium","explanation":"Suspicious terminal one-liner.","evidence":["curl | bash"]}
\`\`\``)
expect(parsed).toMatchObject({
verdict: 'likely_scam',
confidence: 'medium',
})
})
it('rejects invalid response payloads', () => {
expect(parseCommentScamEvalResponse('{"verdict":"ban"}')).toBeNull()
expect(parseCommentScamEvalResponse('not-json')).toBeNull()
})
it('builds bounded ban reason', () => {
const reason = buildCommentScamBanReason({
commentId: 'comments:1',
skillId: 'skills:1',
explanation: 'A'.repeat(700),
evidence: ['B'.repeat(300), 'C'.repeat(300), 'D'.repeat(300), 'E'.repeat(300)],
})
expect(reason.length).toBeLessThanOrEqual(500)
expect(reason).toContain('commentId=comments:1')
expect(reason).toContain('skillId=skills:1')
})
it('marks certainty only for high-confidence certain_scam', () => {
expect(isCertainScam({ verdict: 'certain_scam', confidence: 'high' })).toBe(true)
expect(isCertainScam({ verdict: 'certain_scam', confidence: 'medium' })).toBe(false)
expect(isCertainScam({ verdict: 'likely_scam', confidence: 'high' })).toBe(false)
})
it('builds compact user message with context', () => {
const message = assembleCommentScamEvalUserMessage({
commentId: 'comments:1',
skillId: 'skills:3',
userId: 'users:9',
body: ' test ',
})
expect(message).toContain('Comment ID: comments:1')
expect(message).toContain('Skill ID: skills:3')
expect(message).toContain('Author User ID: users:9')
expect(message).toContain('test')
})
})
+155
View File
@@ -0,0 +1,155 @@
export type CommentScamVerdict = 'not_scam' | 'likely_scam' | 'certain_scam'
export type CommentScamConfidence = 'low' | 'medium' | 'high'
export type CommentScamEvalResponse = {
verdict: CommentScamVerdict
confidence: CommentScamConfidence
explanation: string
evidence: string[]
}
export const COMMENT_SCAM_EVAL_MAX_OUTPUT_TOKENS = 1200
const MAX_COMMENT_CHARS = 4000
const MAX_EXPLANATION_CHARS = 1200
const MAX_EVIDENCE_ITEMS = 5
const MAX_EVIDENCE_ITEM_CHARS = 160
const MAX_BAN_REASON_CHARS = 500
const VALID_VERDICTS = new Set<CommentScamVerdict>(['not_scam', 'likely_scam', 'certain_scam'])
const VALID_CONFIDENCES = new Set<CommentScamConfidence>(['low', 'medium', 'high'])
export const COMMENT_SCAM_EVALUATOR_SYSTEM_PROMPT = `You are a trust and safety classifier for user comments on a software registry.
Goal: detect comment scams with high precision.
A "certain_scam" verdict is only allowed when the comment clearly attempts fraud, credential theft, malware delivery, or social-engineering abuse.
High-confidence scam patterns include:
- Instructing users to run suspicious shell commands (especially obfuscated/base64/piped-to-bash/curl installer tricks).
- Fake support/update instructions pointing to unknown domains, executables, or terminal one-liners.
- Requests for private keys, seed phrases, passwords, API keys, session tokens, or wallet recovery data.
- Impersonation or urgent pressure language to bypass trust checks.
- Known scam payload structure (e.g. echo+base64+decode+bash, hidden downloader chains).
Important anti-false-positive rules:
- Do NOT mark legitimate troubleshooting or normal install instructions as "certain_scam" unless the malicious intent is explicit.
- If suspicious but ambiguous, use "likely_scam".
- If benign/unclear, use "not_scam".
Output JSON only:
{
"verdict": "not_scam" | "likely_scam" | "certain_scam",
"confidence": "low" | "medium" | "high",
"explanation": "short plain-language rationale",
"evidence": ["short concrete signal", "..."]
}`
export function getCommentScamEvalModel(): string {
return process.env.OPENAI_COMMENT_EVAL_MODEL ?? process.env.OPENAI_EVAL_MODEL ?? 'gpt-5-mini'
}
export function assembleCommentScamEvalUserMessage(args: {
commentId: string
skillId: string
userId: string
body: string
}): string {
const trimmed = args.body.trim()
const body =
trimmed.length > MAX_COMMENT_CHARS
? `${trimmed.slice(0, MAX_COMMENT_CHARS)}\n…[truncated]`
: trimmed
return [
`Comment ID: ${args.commentId}`,
`Skill ID: ${args.skillId}`,
`Author User ID: ${args.userId}`,
'Comment body:',
'```',
body,
'```',
'Respond with a single JSON object.',
].join('\n')
}
function stripCodeFence(raw: string): string {
const text = raw.trim()
if (!text.startsWith('```')) return text
const firstNewline = text.indexOf('\n')
if (firstNewline === -1) return text
const withoutOpening = text.slice(firstNewline + 1)
const lastFence = withoutOpening.lastIndexOf('```')
if (lastFence === -1) return withoutOpening.trim()
return withoutOpening.slice(0, lastFence).trim()
}
function truncate(value: string, max: number): string {
if (value.length <= max) return value
if (max <= 3) return value.slice(0, max)
return `${value.slice(0, max - 3)}...`
}
export function parseCommentScamEvalResponse(raw: string): CommentScamEvalResponse | null {
let parsed: unknown
try {
parsed = JSON.parse(stripCodeFence(raw))
} catch {
return null
}
if (!parsed || typeof parsed !== 'object') return null
const obj = parsed as Record<string, unknown>
const verdict =
typeof obj.verdict === 'string' ? (obj.verdict.toLowerCase() as CommentScamVerdict) : null
if (!verdict || !VALID_VERDICTS.has(verdict)) return null
const confidence =
typeof obj.confidence === 'string'
? (obj.confidence.toLowerCase() as CommentScamConfidence)
: null
if (!confidence || !VALID_CONFIDENCES.has(confidence)) return null
const rawExplanation = typeof obj.explanation === 'string' ? obj.explanation.trim() : ''
if (!rawExplanation) return null
const rawEvidence = Array.isArray(obj.evidence) ? obj.evidence : []
const evidence = rawEvidence
.map((item) => (typeof item === 'string' ? item.trim() : ''))
.filter(Boolean)
.slice(0, MAX_EVIDENCE_ITEMS)
.map((item) => truncate(item, MAX_EVIDENCE_ITEM_CHARS))
return {
verdict,
confidence,
explanation: truncate(rawExplanation, MAX_EXPLANATION_CHARS),
evidence,
}
}
export function isCertainScam(result: {
verdict: CommentScamVerdict
confidence: CommentScamConfidence
}): boolean {
return result.verdict === 'certain_scam' && result.confidence === 'high'
}
export function buildCommentScamBanReason(args: {
commentId: string
skillId: string
explanation: string
evidence: string[]
}): string {
const explanation = args.explanation.trim()
const evidence = args.evidence
.map((item) => item.trim())
.filter(Boolean)
.slice(0, 3)
const suffix = ` commentId=${args.commentId} skillId=${args.skillId}`
const evidenceSegment = evidence.length > 0 ? ` evidence: ${evidence.join('; ')}.` : ''
const core = `comment scam auto-ban. ${explanation}.${evidenceSegment}`
const maxCoreChars = Math.max(0, MAX_BAN_REASON_CHARS - suffix.length)
return `${truncate(core, maxCoreChars)}${suffix}`
}
+3 -3
View File
@@ -39,7 +39,7 @@ describe('requireGitHubAccountAge', () => {
const runQuery = vi.fn().mockResolvedValue({
_id: 'users:1',
githubCreatedAt: now.getTime() - 10 * ONE_DAY_MS,
githubCreatedAt: now.getTime() - 20 * ONE_DAY_MS,
})
const runMutation = vi.fn()
const fetchMock = vi.fn()
@@ -72,7 +72,7 @@ describe('requireGitHubAccountAge', () => {
expect(fetchMock).not.toHaveBeenCalled()
})
it('rejects accounts younger than 7 days', async () => {
it('rejects accounts younger than 14 days', async () => {
vi.useFakeTimers()
const now = new Date('2026-02-02T12:00:00Z')
vi.setSystemTime(now)
@@ -85,7 +85,7 @@ describe('requireGitHubAccountAge', () => {
await expect(
requireGitHubAccountAge({ runQuery, runMutation } as never, 'users:1' as never),
).rejects.toThrow(/GitHub account must be at least 7 days old/i)
).rejects.toThrow(/GitHub account must be at least 14 days old/i)
})
it('fetches githubCreatedAt when missing (by providerAccountId)', async () => {
+5 -3
View File
@@ -5,7 +5,9 @@ import type { ActionCtx } from '../_generated/server'
import { GITHUB_PROFILE_SYNC_WINDOW_MS } from './githubProfileSync'
const GITHUB_API = 'https://api.github.com'
const MIN_ACCOUNT_AGE_MS = 7 * 24 * 60 * 60 * 1000
const MIN_ACCOUNT_AGE_MS = 14 * 24 * 60 * 60 * 1000
type GitHubAccountGateCtx = Pick<ActionCtx, 'runQuery' | 'runMutation'>
type GitHubUser = {
login?: string
@@ -29,7 +31,7 @@ function buildGitHubHeaders() {
return headers
}
export async function requireGitHubAccountAge(ctx: ActionCtx, userId: Id<'users'>) {
export async function requireGitHubAccountAge(ctx: GitHubAccountGateCtx, userId: Id<'users'>) {
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId })
if (!user || user.deletedAt || user.deactivatedAt) throw new ConvexError('User not found')
@@ -76,7 +78,7 @@ export async function requireGitHubAccountAge(ctx: ActionCtx, userId: Id<'users'
const remainingMs = MIN_ACCOUNT_AGE_MS - ageMs
const remainingDays = Math.max(1, Math.ceil(remainingMs / (24 * 60 * 60 * 1000)))
throw new ConvexError(
`GitHub account must be at least 7 days old to upload skills. Try again in ${remainingDays} day${
`GitHub account must be at least 14 days old to publish skills or post comments. Try again in ${remainingDays} day${
remainingDays === 1 ? '' : 's'
}.`,
)
+1 -1
View File
@@ -77,7 +77,7 @@ describe('public skill mapping', () => {
})
it('returns skill when moderationStatus is undefined (legacy)', () => {
const skill = makeSkill({ moderationStatus: undefined as unknown as string })
const skill = makeSkill({ moderationStatus: undefined })
expect(toPublicSkill(skill)).not.toBeNull()
})
+3
View File
@@ -0,0 +1,3 @@
export const MAX_ACTIVE_REPORTS_PER_USER = 20
export const AUTO_HIDE_REPORT_THRESHOLD = 3
export const MAX_REPORT_REASON_LENGTH = 500
+45
View File
@@ -0,0 +1,45 @@
import { describe, expect, it, vi } from 'vitest'
import {
enforceReservedSlugCooldownForNewSkill,
formatReservedSlugCooldownMessage,
} from './reservedSlugs'
describe('reservedSlugs', () => {
it('throws a user-facing error when slug is actively reserved by another user', async () => {
const now = Date.now()
const db = {
query: vi.fn((table: string) => {
if (table !== 'reservedSlugs') throw new Error(`unexpected table ${table}`)
return {
withIndex: (name: string) => {
if (name !== 'by_slug_active_deletedAt') {
throw new Error(`unexpected index ${name}`)
}
return {
order: () => ({
take: async () => [
{
_id: 'reservedSlugs:1',
slug: 'taken-skill',
originalOwnerUserId: 'users:owner',
deletedAt: now - 1000,
expiresAt: now + 60_000,
releasedAt: undefined,
},
],
}),
}
},
}
}),
patch: vi.fn(async () => {}),
}
await expect(
enforceReservedSlugCooldownForNewSkill(
{ db } as never,
{ slug: 'taken-skill', userId: 'users:caller' as never, now },
),
).rejects.toThrow(formatReservedSlugCooldownMessage('taken-skill', now + 60_000))
})
})
+9 -5
View File
@@ -1,3 +1,4 @@
import { ConvexError } from 'convex/values'
import type { Doc, Id } from '../_generated/dataModel'
import type { MutationCtx, QueryCtx } from '../_generated/server'
@@ -5,6 +6,13 @@ type ReservedSlug = Doc<'reservedSlugs'>
const DEFAULT_ACTIVE_LIMIT = 25
export function formatReservedSlugCooldownMessage(slug: string, expiresAt: number) {
return (
`Slug "${slug}" is reserved for its previous owner until ${new Date(expiresAt).toISOString()}. ` +
'Please choose a different slug.'
)
}
function reservedSlugQuery(ctx: QueryCtx | MutationCtx, slug: string) {
return ctx.db
.query('reservedSlugs')
@@ -116,13 +124,9 @@ export async function enforceReservedSlugCooldownForNewSkill(
if (!latest) return
if (latest.expiresAt > params.now && latest.originalOwnerUserId !== params.userId) {
throw new Error(
`Slug "${params.slug}" is reserved for its previous owner until ${new Date(latest.expiresAt).toISOString()}. ` +
'Please choose a different slug.',
)
throw new ConvexError(formatReservedSlugCooldownMessage(params.slug, latest.expiresAt))
}
await ctx.db.patch(latest._id, { releasedAt: params.now })
await releaseDuplicateActiveReservations(ctx, active, latest._id, params.now)
}
+1 -1
View File
@@ -145,7 +145,7 @@ Flag when:
- The number of required environment variables is high relative to the skill's complexity
- The skill requires config paths that grant access to gateway auth, channel tokens, or tool policies
- Environment variables named with patterns like SECRET, TOKEN, KEY, PASSWORD are required but not justified by the skill's purpose
- The SKILL.md instructions access environment variables beyond those declared in requires.env or primaryEnv
- The SKILL.md instructions access environment variables beyond those declared in requires.env, primaryEnv, or envVars
### 5. Persistence and privilege
+10 -8
View File
@@ -21,6 +21,7 @@ import {
getFrontmatterMetadata,
getFrontmatterValue,
hashSkillFiles,
isMacJunkPath,
isTextFile,
parseClawdisMetadata,
parseFrontmatter,
@@ -111,16 +112,17 @@ export async function publishVersionForUser(
...file,
path: file.path as string,
}))
if (safeFiles.some((file) => !isTextFile(file.path, file.contentType ?? undefined))) {
const publishFiles = safeFiles.filter((file) => !isMacJunkPath(file.path))
if (publishFiles.some((file) => !isTextFile(file.path, file.contentType ?? undefined))) {
throw new ConvexError('Only text-based files are allowed')
}
const totalBytes = safeFiles.reduce((sum, file) => sum + file.size, 0)
const totalBytes = publishFiles.reduce((sum, file) => sum + file.size, 0)
if (totalBytes > MAX_TOTAL_BYTES) {
throw new ConvexError('Skill bundle exceeds 50MB limit')
}
const readmeFile = safeFiles.find(
const readmeFile = publishFiles.find(
(file) => file.path?.toLowerCase() === 'skill.md' || file.path?.toLowerCase() === 'skills.md',
)
if (!readmeFile) throw new ConvexError('SKILL.md is required')
@@ -204,7 +206,7 @@ export async function publishVersionForUser(
const metadata = mergeSourceIntoMetadata(frontmatterMetadata, args.source, qualityAssessment)
const otherFiles = [] as Array<{ path: string; content: string }>
for (const file of safeFiles) {
for (const file of publishFiles) {
if (!file.path || file.path.toLowerCase().endsWith('.md')) continue
if (!isTextFile(file.path, file.contentType ?? undefined)) continue
const content = await fetchText(ctx, file.storageId)
@@ -219,7 +221,7 @@ export async function publishVersionForUser(
})
const fingerprintPromise = hashSkillFiles(
safeFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
publishFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
)
const changelogPromise =
@@ -229,7 +231,7 @@ export async function publishVersionForUser(
slug,
version,
readmeText,
files: safeFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
files: publishFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
})
const embeddingPromise = generateEmbedding(embeddingText)
@@ -258,7 +260,7 @@ export async function publishVersionForUser(
}
: undefined,
bypassNewSkillRateLimit: options.bypassNewSkillRateLimit || undefined,
files: safeFiles.map((file) => ({
files: publishFiles.map((file) => ({
...file,
path: file.path,
})),
@@ -298,7 +300,7 @@ export async function publishVersionForUser(
version,
displayName,
ownerHandle,
files: safeFiles,
files: publishFiles,
publishedAt: Date.now(),
})
.catch((error) => {
+168
View File
@@ -4,6 +4,7 @@ import {
getFrontmatterMetadata,
getFrontmatterValue,
hashSkillFiles,
isMacJunkPath,
isTextFile,
parseClawdisMetadata,
parseFrontmatter,
@@ -152,6 +153,15 @@ describe('skills utils', () => {
expect(isTextFile('data.json')).toBe(true)
})
it('detects mac junk paths', () => {
expect(isMacJunkPath('.DS_Store')).toBe(true)
expect(isMacJunkPath('folder/.DS_Store')).toBe(true)
expect(isMacJunkPath('folder/._config.md')).toBe(true)
expect(isMacJunkPath('__MACOSX/._SKILL.md')).toBe(true)
expect(isMacJunkPath('docs/SKILL.md')).toBe(false)
expect(isMacJunkPath('notes.md')).toBe(false)
})
it('builds embedding text', () => {
const frontmatter = { name: 'Demo', description: 'Hello' }
const text = buildEmbeddingText({
@@ -195,3 +205,161 @@ describe('skills utils', () => {
expect(a).toBe(b)
})
})
describe('parseClawdisMetadata — env/deps/author/links (#350)', () => {
it('parses envVars from clawdis block', () => {
const frontmatter = parseFrontmatter(`---
metadata:
clawdis:
envVars:
- name: ANTHROPIC_API_KEY
required: true
description: API key for Claude
- name: MAX_TURNS
required: false
description: Max turns per phase
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.envVars).toHaveLength(2)
expect(meta?.envVars?.[0]).toEqual({
name: 'ANTHROPIC_API_KEY',
required: true,
description: 'API key for Claude',
})
expect(meta?.envVars?.[1]?.required).toBe(false)
})
it('parses dependencies from clawdis block', () => {
const frontmatter = parseFrontmatter(`---
metadata:
clawdis:
dependencies:
- name: securevibes
type: pip
version: ">=0.3.0"
url: https://pypi.org/project/securevibes/
repository: https://github.com/anshumanbh/securevibes
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.dependencies).toHaveLength(1)
expect(meta?.dependencies?.[0]).toEqual({
name: 'securevibes',
type: 'pip',
version: '>=0.3.0',
url: 'https://pypi.org/project/securevibes/',
repository: 'https://github.com/anshumanbh/securevibes',
})
})
it('parses author and links from clawdis block', () => {
const frontmatter = parseFrontmatter(`---
metadata:
clawdis:
author: anshumanbh
links:
homepage: https://securevibes.ai
repository: https://github.com/anshumanbh/securevibes
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.author).toBe('anshumanbh')
expect(meta?.links?.homepage).toBe('https://securevibes.ai')
expect(meta?.links?.repository).toBe('https://github.com/anshumanbh/securevibes')
})
it('parses env/deps/author/links from top-level frontmatter (no clawdis block)', () => {
const frontmatter = parseFrontmatter(`---
env:
- name: MY_API_KEY
required: true
description: Main API key
dependencies:
- name: requests
type: pip
author: someuser
links:
homepage: https://example.com
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.envVars).toHaveLength(1)
expect(meta?.envVars?.[0]?.name).toBe('MY_API_KEY')
expect(meta?.dependencies).toHaveLength(1)
expect(meta?.author).toBe('someuser')
expect(meta?.links?.homepage).toBe('https://example.com')
})
it('handles string-only env arrays as required env vars', () => {
const frontmatter = parseFrontmatter(`---
metadata:
clawdis:
envVars:
- API_KEY
- SECRET_TOKEN
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.envVars).toHaveLength(2)
expect(meta?.envVars?.[0]).toEqual({ name: 'API_KEY', required: true })
})
it('normalizes unknown dependency types to other', () => {
const frontmatter = parseFrontmatter(`---
metadata:
clawdis:
dependencies:
- name: sometool
type: ruby
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.dependencies?.[0]?.type).toBe('other')
})
it('returns undefined when no declarations present', () => {
const frontmatter = parseFrontmatter(`---
name: simple-skill
description: A simple skill
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta).toBeUndefined()
})
it('parses requires.env from top-level frontmatter (no clawdis block) (#522)', () => {
const frontmatter = parseFrontmatter(`---
name: sigil-security
description: Secure AI agent wallets.
homepage: https://sigil.codes
requires:
env:
- SIGIL_API_KEY
- SIGIL_ACCOUNT_ADDRESS
- SIGIL_AGENT_PRIVATE_KEY
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.requires?.env).toEqual([
'SIGIL_API_KEY',
'SIGIL_ACCOUNT_ADDRESS',
'SIGIL_AGENT_PRIVATE_KEY',
])
expect(meta?.homepage).toBe('https://sigil.codes')
})
it('parses requires.bins and requires.anyBins from top-level frontmatter (#522)', () => {
const frontmatter = parseFrontmatter(`---
name: my-tool
description: A tool skill.
requires:
bins:
- curl
- jq
anyBins:
- rg
- fd
config:
- ~/.config/mytool.json
primaryEnv: MY_API_KEY
---`)
const meta = parseClawdisMetadata(frontmatter)
expect(meta?.requires?.bins).toEqual(['curl', 'jq'])
expect(meta?.requires?.anyBins).toEqual(['rg', 'fd'])
expect(meta?.requires?.config).toEqual(['~/.config/mytool.json'])
expect(meta?.primaryEnv).toBe('MY_API_KEY')
})
})
+162 -1
View File
@@ -79,7 +79,12 @@ export function parseClawdisMetadata(frontmatter: ParsedSkillFrontmatter) {
? (openclawMeta as Record<string, unknown>)
: undefined
const clawdisRaw = metadataSource ?? frontmatter.clawdis
if (!clawdisRaw || typeof clawdisRaw !== 'object' || Array.isArray(clawdisRaw)) return undefined
// Support top-level frontmatter env/dependencies/author/links as fallback
// even when no clawdis block exists (per #350)
if (!clawdisRaw || typeof clawdisRaw !== 'object' || Array.isArray(clawdisRaw)) {
return parseFrontmatterLevelDeclarations(frontmatter)
}
try {
const clawdisObj = clawdisRaw as Record<string, unknown>
@@ -122,6 +127,19 @@ export function parseClawdisMetadata(frontmatter: ParsedSkillFrontmatter) {
const config = parseClawdbotConfigSpec(clawdisObj.config)
if (config) metadata.config = config
// Parse env var declarations (detailed env with descriptions)
const envVars = parseEnvVarDeclarations(clawdisObj.envVars ?? clawdisObj.env)
if (envVars.length > 0) metadata.envVars = envVars
// Parse dependency declarations
const dependencies = parseDependencyDeclarations(clawdisObj.dependencies)
if (dependencies.length > 0) metadata.dependencies = dependencies
// Parse author and links
if (typeof clawdisObj.author === 'string') metadata.author = clawdisObj.author
const links = parseSkillLinks(clawdisObj.links)
if (links) metadata.links = links
return parseArk(ClawdisSkillMetadataSchema, metadata, 'Clawdis metadata')
} catch {
return undefined
@@ -140,6 +158,22 @@ export function isTextFile(path: string, contentType?: string | null) {
return false
}
export function isMacJunkPath(path: string) {
const normalized = path
.trim()
.replaceAll('\\', '/')
.replace(/^\/+/, '')
.toLowerCase()
if (!normalized) return false
const segments = normalized.split('/').filter(Boolean)
if (segments.length === 0) return false
if (segments.includes('__macosx')) return true
const basename = segments.at(-1) ?? ''
if (basename === '.ds_store') return true
if (basename.startsWith('._')) return true
return false
}
export function sanitizePath(path: string) {
const trimmed = path.trim().replace(/^\/+/, '')
if (!trimmed || trimmed.includes('..') || trimmed.includes('\\')) {
@@ -279,3 +313,130 @@ function isPlainObject(value: unknown): value is Record<string, unknown> {
const proto = Object.getPrototypeOf(value)
return proto === Object.prototype || proto === null
}
/**
* Parse env var declarations from frontmatter.
* Accepts either an array of {name, required?, description?} objects
* or a simple string array (converted to {name, required: true}).
*/
function parseEnvVarDeclarations(input: unknown): Array<{ name: string; required?: boolean; description?: string }> {
if (!input) return []
if (!Array.isArray(input)) return []
return input
.map((item) => {
if (typeof item === 'string') {
return { name: item.trim(), required: true }
}
if (item && typeof item === 'object' && typeof (item as Record<string, unknown>).name === 'string') {
const obj = item as Record<string, unknown>
const decl: { name: string; required?: boolean; description?: string } = {
name: String(obj.name).trim(),
}
if (typeof obj.required === 'boolean') decl.required = obj.required
if (typeof obj.description === 'string') decl.description = obj.description.trim()
return decl
}
return null
})
.filter((item): item is NonNullable<typeof item> => item !== null && item.name.length > 0)
}
/**
* Parse dependency declarations from frontmatter.
* Accepts an array of {name, type, version?, url?, repository?} objects.
*/
function parseDependencyDeclarations(input: unknown): Array<{
name: string
type: 'pip' | 'npm' | 'brew' | 'go' | 'cargo' | 'apt' | 'other'
version?: string
url?: string
repository?: string
}> {
if (!input || !Array.isArray(input)) return []
const validTypes = new Set(['pip', 'npm', 'brew', 'go', 'cargo', 'apt', 'other'])
return input
.map((item) => {
if (!item || typeof item !== 'object') return null
const obj = item as Record<string, unknown>
if (typeof obj.name !== 'string') return null
const typeStr = typeof obj.type === 'string' ? obj.type.trim().toLowerCase() : 'other'
const depType = validTypes.has(typeStr)
? (typeStr as 'pip' | 'npm' | 'brew' | 'go' | 'cargo' | 'apt' | 'other')
: 'other'
const decl: {
name: string
type: typeof depType
version?: string
url?: string
repository?: string
} = { name: String(obj.name).trim(), type: depType }
if (typeof obj.version === 'string') decl.version = obj.version.trim()
if (typeof obj.url === 'string') decl.url = obj.url.trim()
if (typeof obj.repository === 'string') decl.repository = obj.repository.trim()
return decl
})
.filter((item): item is NonNullable<typeof item> => item !== null && item.name.length > 0)
}
/**
* Parse links object from frontmatter.
*/
function parseSkillLinks(input: unknown): { homepage?: string; repository?: string; documentation?: string; changelog?: string } | undefined {
if (!input || typeof input !== 'object' || Array.isArray(input)) return undefined
const obj = input as Record<string, unknown>
const links: { homepage?: string; repository?: string; documentation?: string; changelog?: string } = {}
if (typeof obj.homepage === 'string') links.homepage = obj.homepage.trim()
if (typeof obj.repository === 'string') links.repository = obj.repository.trim()
if (typeof obj.documentation === 'string') links.documentation = obj.documentation.trim()
if (typeof obj.changelog === 'string') links.changelog = obj.changelog.trim()
return Object.keys(links).length > 0 ? links : undefined
}
/**
* Parse top-level frontmatter env/dependencies/author/links
* when no clawdis block is present (fallback for #350).
*/
function parseFrontmatterLevelDeclarations(frontmatter: ParsedSkillFrontmatter): ClawdisSkillMetadata | undefined {
const metadata: ClawdisSkillMetadata = {}
// Parse requires block (env, bins, anyBins, config) from top-level frontmatter (#522)
const requiresRaw = frontmatter.requires
if (requiresRaw && typeof requiresRaw === 'object' && !Array.isArray(requiresRaw)) {
const req = requiresRaw as Record<string, unknown>
const bins = normalizeStringList(req.bins)
const anyBins = normalizeStringList(req.anyBins)
const env = normalizeStringList(req.env)
const config = normalizeStringList(req.config)
if (bins.length || anyBins.length || env.length || config.length) {
metadata.requires = {}
if (bins.length) metadata.requires.bins = bins
if (anyBins.length) metadata.requires.anyBins = anyBins
if (env.length) metadata.requires.env = env
if (config.length) metadata.requires.config = config
}
}
// Parse primaryEnv from top-level frontmatter
if (typeof frontmatter.primaryEnv === 'string') {
metadata.primaryEnv = String(frontmatter.primaryEnv).trim()
}
const envVars = parseEnvVarDeclarations(frontmatter.env)
if (envVars.length > 0) metadata.envVars = envVars
const dependencies = parseDependencyDeclarations(frontmatter.dependencies)
if (dependencies.length > 0) metadata.dependencies = dependencies
if (typeof frontmatter.author === 'string') metadata.author = String(frontmatter.author).trim()
const links = parseSkillLinks(frontmatter.links)
if (links) metadata.links = links
if (typeof frontmatter.homepage === 'string') {
metadata.homepage = String(frontmatter.homepage).trim()
}
return Object.keys(metadata).length > 0
? parseArk(ClawdisSkillMetadataSchema, metadata, 'Clawdis metadata')
: undefined
}
+13 -11
View File
@@ -10,6 +10,7 @@ import {
getFrontmatterMetadata,
getFrontmatterValue,
hashSkillFiles,
isMacJunkPath,
isTextFile,
parseFrontmatter,
sanitizePath,
@@ -100,22 +101,23 @@ export async function publishSoulVersionForUser(
const sanitizedFiles = args.files.map((file) => {
const path = sanitizePath(file.path)
if (!path) throw new ConvexError('Invalid file paths')
if (!isTextFile(path, file.contentType ?? undefined)) {
throw new ConvexError('Only text-based files are allowed')
}
return { ...file, path }
})
const publishFiles = sanitizedFiles.filter((file) => !isMacJunkPath(file.path))
if (publishFiles.some((file) => !isTextFile(file.path, file.contentType ?? undefined))) {
throw new ConvexError('Only text-based files are allowed')
}
const totalBytes = sanitizedFiles.reduce((sum, file) => sum + file.size, 0)
const totalBytes = publishFiles.reduce((sum, file) => sum + file.size, 0)
if (totalBytes > MAX_TOTAL_BYTES) {
throw new ConvexError('Soul bundle exceeds 50MB limit')
}
const isSoulFile = (path: string) => path.toLowerCase() === 'soul.md'
const readmeFile = sanitizedFiles.find((file) => isSoulFile(file.path))
const readmeFile = publishFiles.find((file) => isSoulFile(file.path))
if (!readmeFile) throw new ConvexError('SOUL.md is required')
const nonSoulFiles = sanitizedFiles.filter((file) => !isSoulFile(file.path))
const nonSoulFiles = publishFiles.filter((file) => !isSoulFile(file.path))
if (nonSoulFiles.length > 0) {
throw new ConvexError('Only SOUL.md is allowed for soul bundles')
}
@@ -132,8 +134,8 @@ export async function publishSoulVersionForUser(
})
const fingerprint = await hashSkillFiles(
sanitizedFiles.map((file) => ({
path: file.path ?? '',
publishFiles.map((file) => ({
path: file.path,
sha256: file.sha256,
})),
)
@@ -145,7 +147,7 @@ export async function publishSoulVersionForUser(
slug,
version,
readmeText,
files: sanitizedFiles.map((file) => ({ path: file.path ?? '', sha256: file.sha256 })),
files: publishFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
})
const embeddingPromise = generateEmbedding(embeddingText)
@@ -166,7 +168,7 @@ export async function publishSoulVersionForUser(
changelogSource,
tags: args.tags?.map((tag) => tag.trim()).filter(Boolean),
fingerprint,
files: sanitizedFiles,
files: publishFiles,
parsed: {
frontmatter,
metadata,
@@ -186,7 +188,7 @@ export async function publishSoulVersionForUser(
version,
displayName,
ownerHandle,
files: sanitizedFiles,
files: publishFiles,
publishedAt: Date.now(),
})
.catch((error) => {
+98 -1
View File
@@ -2,6 +2,13 @@ import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import { internalAction } from './_generated/server'
import {
assembleCommentScamEvalUserMessage,
COMMENT_SCAM_EVALUATOR_SYSTEM_PROMPT,
COMMENT_SCAM_EVAL_MAX_OUTPUT_TOKENS,
getCommentScamEvalModel,
parseCommentScamEvalResponse,
} from './lib/commentScamPrompt'
import type { SkillEvalContext } from './lib/securityPrompt'
import {
assembleEvalUserMessage,
@@ -122,6 +129,8 @@ export const evaluateWithLlm = internalAction({
// 6. Build eval context
const parsed = version.parsed as SkillEvalContext['parsed']
const fm = parsed.frontmatter ?? {}
const clawdisRecord = (parsed.clawdis ?? {}) as Record<string, unknown>
const clawdisLinks = (clawdisRecord.links ?? {}) as Record<string, unknown>
const evalCtx: SkillEvalContext = {
slug: skill.slug,
@@ -131,7 +140,11 @@ export const evaluateWithLlm = internalAction({
createdAt: version.createdAt,
summary: (skill.summary as string | undefined) ?? undefined,
source: (fm.source as string | undefined) ?? undefined,
homepage: (fm.homepage as string | undefined) ?? undefined,
homepage:
(fm.homepage as string | undefined) ??
(clawdisRecord.homepage as string | undefined) ??
(clawdisLinks.homepage as string | undefined) ??
undefined,
parsed,
files: version.files.map((f) => ({ path: f.path, size: f.size })),
skillMdContent,
@@ -361,3 +374,87 @@ export const backfillLlmEval = internalAction({
return result
},
})
export const evaluateCommentForScam = internalAction({
args: {
commentId: v.id('comments'),
skillId: v.id('skills'),
userId: v.id('users'),
body: v.string(),
},
handler: async (_ctx, args) => {
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) {
return { ok: false as const, error: 'OPENAI_API_KEY not configured' }
}
const model = getCommentScamEvalModel()
const input = assembleCommentScamEvalUserMessage({
commentId: String(args.commentId),
skillId: String(args.skillId),
userId: String(args.userId),
body: args.body,
})
const requestBody = JSON.stringify({
model,
instructions: COMMENT_SCAM_EVALUATOR_SYSTEM_PROMPT,
input,
max_output_tokens: COMMENT_SCAM_EVAL_MAX_OUTPUT_TOKENS,
text: {
format: {
type: 'json_object',
},
},
})
const MAX_RETRIES = 3
let response: Response | null = null
for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
response = await fetch('https://api.openai.com/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: requestBody,
})
if ((response.status === 429 || response.status >= 500) && attempt < MAX_RETRIES) {
const delay = 2 ** attempt * 2000 + Math.random() * 1000
await new Promise((resolve) => setTimeout(resolve, delay))
continue
}
break
}
if (!response || !response.ok) {
const errorText = response ? await response.text() : 'No response'
return {
ok: false as const,
error: `OpenAI API error (${response?.status}): ${errorText.slice(0, 200)}`,
}
}
const payload = (await response.json()) as unknown
const raw = extractResponseText(payload)
if (!raw) {
return { ok: false as const, error: 'Empty response from OpenAI' }
}
const parsed = parseCommentScamEvalResponse(raw)
if (!parsed) {
console.error(`[commentScam] Parse failure for ${args.commentId}: ${raw.slice(0, 400)}`)
return { ok: false as const, error: 'Failed to parse scam evaluation response' }
}
return {
ok: true as const,
model,
verdict: parsed.verdict,
confidence: parsed.confidence,
explanation: parsed.explanation,
evidence: parsed.evidence,
}
},
})
+6 -2
View File
@@ -216,7 +216,9 @@ describe('maintenance badge denormalization', () => {
},
} as never
const result = await (upsertSkillBadgeRecordInternal as { _handler: Function })._handler(ctx, {
const result = await (
upsertSkillBadgeRecordInternal as unknown as { _handler: Function }
)._handler(ctx, {
skillId: 'skills:1',
kind: 'highlighted',
byUserId: 'users:1',
@@ -255,7 +257,9 @@ describe('maintenance badge denormalization', () => {
},
} as never
const result = await (upsertSkillBadgeRecordInternal as { _handler: Function })._handler(ctx, {
const result = await (
upsertSkillBadgeRecordInternal as unknown as { _handler: Function }
)._handler(ctx, {
skillId: 'skills:1',
kind: 'official',
byUserId: 'users:2',
+26
View File
@@ -409,12 +409,37 @@ const comments = defineTable({
skillId: v.id('skills'),
userId: v.id('users'),
body: v.string(),
reportCount: v.optional(v.number()),
lastReportedAt: v.optional(v.number()),
scamScanVerdict: v.optional(
v.union(v.literal('not_scam'), v.literal('likely_scam'), v.literal('certain_scam')),
),
scamScanConfidence: v.optional(v.union(v.literal('low'), v.literal('medium'), v.literal('high'))),
scamScanExplanation: v.optional(v.string()),
scamScanEvidence: v.optional(v.array(v.string())),
scamScanModel: v.optional(v.string()),
scamScanCheckedAt: v.optional(v.number()),
scamBanTriggeredAt: v.optional(v.number()),
createdAt: v.number(),
softDeletedAt: v.optional(v.number()),
deletedBy: v.optional(v.id('users')),
})
.index('by_skill', ['skillId'])
.index('by_user', ['userId'])
.index('by_scam_scan_checked', ['scamScanCheckedAt'])
const commentReports = defineTable({
commentId: v.id('comments'),
skillId: v.id('skills'),
userId: v.id('users'),
reason: v.optional(v.string()),
createdAt: v.number(),
})
.index('by_comment', ['commentId'])
.index('by_comment_createdAt', ['commentId', 'createdAt'])
.index('by_skill', ['skillId'])
.index('by_user', ['userId'])
.index('by_comment_user', ['commentId', 'userId'])
const skillReports = defineTable({
skillId: v.id('skills'),
@@ -593,6 +618,7 @@ export default defineSchema({
skillStatEvents,
skillStatUpdateCursors,
comments,
commentReports,
skillReports,
soulComments,
stars,
+57 -5
View File
@@ -104,12 +104,63 @@ describe('skills.listPublicPageV2', () => {
expect(eqMock).toHaveBeenCalledWith('softDeletedAt', undefined)
})
it('preserves pagination cursor when filtering removes the whole page', async () => {
it('skips fully filtered pages until it finds matching skills', async () => {
const plain = makeSkill('skills:plain', 'plain', 'users:1', 'skillVersions:1')
const highlightedClean = makeSkill('skills:hl-clean', 'hl-clean', 'users:2', 'skillVersions:2')
const paginateMock = vi
.fn()
.mockResolvedValueOnce({
page: [plain],
continueCursor: 'next-cursor',
isDone: false,
pageStatus: null,
splitCursor: null,
})
.mockResolvedValueOnce({
page: [highlightedClean],
continueCursor: 'after-highlighted',
isDone: false,
pageStatus: null,
splitCursor: null,
})
const ctx = {
db: {
query: vi.fn(() => ({
withIndex: vi.fn(() => ({
order: vi.fn(() => ({ paginate: paginateMock })),
})),
})),
get: vi.fn(async (id: string) => {
if (id.startsWith('users:')) return makeUser(id)
if (id.startsWith('skillVersions:')) return makeVersion(id)
return null
}),
},
}
const result = await listPublicPageV2Handler(ctx, {
paginationOpts: { cursor: null, numItems: 25 },
sort: 'downloads',
dir: 'desc',
highlightedOnly: true,
nonSuspiciousOnly: false,
})
expect(result.page).toHaveLength(1)
expect(result.page[0]?.skill.slug).toBe('hl-clean')
expect(result.continueCursor).toBe('after-highlighted')
expect(result.isDone).toBe(false)
expect(paginateMock).toHaveBeenCalledTimes(2)
expect(paginateMock).toHaveBeenNthCalledWith(1, { cursor: null, numItems: 25 })
expect(paginateMock).toHaveBeenNthCalledWith(2, { cursor: 'next-cursor', numItems: 25 })
})
it('returns exhausted when filtered pages remain empty to the end', async () => {
const plain = makeSkill('skills:plain', 'plain', 'users:1', 'skillVersions:1')
const paginateMock = vi.fn().mockResolvedValue({
page: [plain],
continueCursor: 'next-cursor',
isDone: false,
continueCursor: null,
isDone: true,
pageStatus: null,
splitCursor: null,
})
@@ -133,8 +184,9 @@ describe('skills.listPublicPageV2', () => {
})
expect(result.page).toEqual([])
expect(result.continueCursor).toBe('next-cursor')
expect(result.isDone).toBe(false)
expect(result.continueCursor).toBeNull()
expect(result.isDone).toBe(true)
expect(paginateMock).toHaveBeenCalledTimes(1)
})
it('restarts pagination from first page when cursor is stale', async () => {
+126
View File
@@ -120,6 +120,132 @@ describe('skills anti-spam guards', () => {
).rejects.toThrow(/max 5 new skills per hour/i)
})
it('returns a user-facing slug-taken message when publishing to another owner slug', async () => {
let authAccountLookupCount = 0
const db = {
get: vi.fn(async (id: string) => {
if (id === 'users:caller') return { _id: 'users:caller', deletedAt: undefined }
if (id === 'users:owner') {
return {
_id: 'users:owner',
handle: 'alice',
deletedAt: undefined,
deactivatedAt: undefined,
}
}
return null
}),
query: vi.fn((table: string) => {
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name !== 'by_slug') throw new Error(`unexpected skills index ${name}`)
return {
unique: async () => ({
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:owner',
softDeletedAt: undefined,
moderationStatus: 'active',
moderationFlags: undefined,
}),
}
},
}
}
if (table === 'authAccounts') {
return {
withIndex: (name: string) => {
if (name !== 'userIdAndProvider') throw new Error(`unexpected auth index ${name}`)
return {
unique: async () => {
authAccountLookupCount += 1
return authAccountLookupCount === 1
? { providerAccountId: 'owner-gh' }
: { providerAccountId: 'caller-gh' }
},
}
},
}
}
throw new Error(`unexpected table ${table}`)
}),
}
await expect(
insertVersionHandler(
{ db } as never,
createPublishArgs({
userId: 'users:caller',
slug: 'taken-skill',
}) as never,
),
).rejects.toThrow('Slug is already taken. Choose a different slug. Existing skill: /alice/taken-skill')
})
it('does not include a URL in slug-taken message when conflicting owner is deleted', async () => {
let authAccountLookupCount = 0
const db = {
get: vi.fn(async (id: string) => {
if (id === 'users:caller') return { _id: 'users:caller', deletedAt: undefined }
if (id === 'users:owner') {
return {
_id: 'users:owner',
handle: 'alice',
deletedAt: Date.now(),
deactivatedAt: undefined,
}
}
return null
}),
query: vi.fn((table: string) => {
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name !== 'by_slug') throw new Error(`unexpected skills index ${name}`)
return {
unique: async () => ({
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:owner',
softDeletedAt: undefined,
moderationStatus: 'active',
moderationFlags: undefined,
}),
}
},
}
}
if (table === 'authAccounts') {
return {
withIndex: (name: string) => {
if (name !== 'userIdAndProvider') throw new Error(`unexpected auth index ${name}`)
return {
unique: async () => {
authAccountLookupCount += 1
return authAccountLookupCount === 1
? { providerAccountId: 'owner-gh' }
: { providerAccountId: 'caller-gh' }
},
}
},
}
}
throw new Error(`unexpected table ${table}`)
}),
}
await expect(
insertVersionHandler(
{ db } as never,
createPublishArgs({
userId: 'users:caller',
slug: 'taken-skill',
}) as never,
),
).rejects.toThrow('Slug is already taken. Choose a different slug.')
})
it('keeps suspicious skills visible for low-trust publishers', async () => {
const patch = vi.fn(async () => {})
const version = { _id: 'skillVersions:1', skillId: 'skills:1' }
+397
View File
@@ -0,0 +1,397 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { formatReservedSlugCooldownMessage } from './lib/reservedSlugs'
vi.mock('@convex-dev/auth/server', () => ({
getAuthUserId: vi.fn(),
}))
import { getAuthUserId } from '@convex-dev/auth/server'
import { checkSlugAvailability } from './skills'
type WrappedHandler<TArgs> = {
_handler: (ctx: unknown, args: TArgs) => Promise<unknown>
}
type SkillDoc = {
_id: string
slug: string
ownerUserId: string
softDeletedAt?: number
moderationStatus?: 'active' | 'hidden' | 'removed'
moderationFlags?: string[]
}
type ReservationDoc = {
_id: string
slug: string
originalOwnerUserId: string
deletedAt: number
expiresAt: number
releasedAt?: number
}
const checkSlugAvailabilityHandler = (
checkSlugAvailability as unknown as WrappedHandler<{ slug: string }>
)._handler
function createCtx(options: {
skill: SkillDoc | null
reservation?: ReservationDoc | null
owner?: { _id: string; handle?: string | null; deletedAt?: number; deactivatedAt?: number } | null
callerId?: string
ownerProviderAccountId?: string | null
callerProviderAccountId?: string | null
}) {
const callerId = options.callerId ?? 'users:caller'
let authAccountLookupCount = 0
const db = {
get: vi.fn(async (id: string) => {
if (id === callerId) {
return { _id: callerId, deletedAt: undefined, deactivatedAt: undefined }
}
if (options.owner && id === options.owner._id) return options.owner
return null
}),
query: vi.fn((table: string) => {
if (table === 'skills') {
return {
withIndex: (name: string) => {
if (name !== 'by_slug') throw new Error(`unexpected skills index ${name}`)
return {
unique: async () => options.skill,
}
},
}
}
if (table === 'reservedSlugs') {
return {
withIndex: (name: string) => {
if (name !== 'by_slug_active_deletedAt') {
throw new Error(`unexpected reservedSlugs index ${name}`)
}
return {
order: () => ({
take: async () => (options.reservation ? [options.reservation] : []),
}),
}
},
}
}
if (table === 'authAccounts') {
return {
withIndex: (name: string) => {
if (name !== 'userIdAndProvider') {
throw new Error(`unexpected authAccounts index ${name}`)
}
return {
unique: async () => {
authAccountLookupCount += 1
if (authAccountLookupCount === 1) {
return options.ownerProviderAccountId
? { providerAccountId: options.ownerProviderAccountId }
: null
}
return options.callerProviderAccountId
? { providerAccountId: options.callerProviderAccountId }
: null
},
}
},
}
}
throw new Error(`unexpected table ${table}`)
}),
}
return { db }
}
describe('skills.checkSlugAvailability', () => {
beforeEach(() => {
vi.clearAllMocks()
})
afterEach(() => {
vi.restoreAllMocks()
})
it('returns taken without URL for non-public collisions', async () => {
vi.mocked(getAuthUserId).mockResolvedValue('users:caller' as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: {
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:owner',
softDeletedAt: 123,
moderationStatus: 'active',
moderationFlags: undefined,
},
owner: {
_id: 'users:owner',
handle: 'alice',
deletedAt: undefined,
deactivatedAt: undefined,
},
ownerProviderAccountId: 'owner-gh',
callerProviderAccountId: 'caller-gh',
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string
url: string | null
}
expect(result).toEqual({
available: false,
reason: 'taken',
message: 'Slug is already taken. Choose a different slug.',
url: null,
})
})
it('returns taken with URL for public collisions', async () => {
vi.mocked(getAuthUserId).mockResolvedValue('users:caller' as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: {
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:owner',
softDeletedAt: undefined,
moderationStatus: 'active',
moderationFlags: undefined,
},
owner: {
_id: 'users:owner',
handle: 'alice',
deletedAt: undefined,
deactivatedAt: undefined,
},
ownerProviderAccountId: 'owner-gh',
callerProviderAccountId: 'caller-gh',
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string
url: string | null
}
expect(result).toEqual({
available: false,
reason: 'taken',
message: 'Slug is already taken. Choose a different slug. Existing skill: /alice/taken-skill',
url: '/alice/taken-skill',
})
})
it('returns taken without requiring auth context', async () => {
vi.mocked(getAuthUserId).mockResolvedValue(null as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: {
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:owner',
softDeletedAt: undefined,
moderationStatus: 'active',
moderationFlags: undefined,
},
owner: {
_id: 'users:owner',
handle: 'alice',
deletedAt: undefined,
deactivatedAt: undefined,
},
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string
url: string | null
}
expect(result).toEqual({
available: false,
reason: 'taken',
message: 'Slug is already taken. Choose a different slug. Existing skill: /alice/taken-skill',
url: '/alice/taken-skill',
})
})
it('returns available when slug belongs to current user', async () => {
vi.mocked(getAuthUserId).mockResolvedValue('users:caller' as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: {
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:caller',
softDeletedAt: undefined,
moderationStatus: 'active',
moderationFlags: undefined,
},
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string | null
url: string | null
}
expect(result).toEqual({
available: true,
reason: 'available',
message: null,
url: null,
})
})
it('returns reserved when active reservation belongs to another user', async () => {
const now = 1_700_000_000_000
vi.spyOn(Date, 'now').mockReturnValue(now)
vi.mocked(getAuthUserId).mockResolvedValue('users:caller' as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: null,
reservation: {
_id: 'reservedSlugs:1',
slug: 'taken-skill',
originalOwnerUserId: 'users:owner',
deletedAt: now - 1_000,
expiresAt: now + 60_000,
releasedAt: undefined,
},
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string
url: string | null
}
expect(result).toEqual({
available: false,
reason: 'reserved',
message: formatReservedSlugCooldownMessage('taken-skill', now + 60_000),
url: null,
})
})
it('returns reserved without requiring auth context', async () => {
const now = 1_700_000_000_000
vi.spyOn(Date, 'now').mockReturnValue(now)
vi.mocked(getAuthUserId).mockResolvedValue(null as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: null,
reservation: {
_id: 'reservedSlugs:1',
slug: 'taken-skill',
originalOwnerUserId: 'users:owner',
deletedAt: now - 1_000,
expiresAt: now + 60_000,
releasedAt: undefined,
},
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string
url: string | null
}
expect(result).toEqual({
available: false,
reason: 'reserved',
message: formatReservedSlugCooldownMessage('taken-skill', now + 60_000),
url: null,
})
})
it('returns available when reservation has expired', async () => {
const now = 1_700_000_000_000
vi.spyOn(Date, 'now').mockReturnValue(now)
vi.mocked(getAuthUserId).mockResolvedValue('users:caller' as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: null,
reservation: {
_id: 'reservedSlugs:1',
slug: 'taken-skill',
originalOwnerUserId: 'users:owner',
deletedAt: now - 120_000,
expiresAt: now - 60_000,
releasedAt: undefined,
},
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string | null
url: string | null
}
expect(result).toEqual({
available: true,
reason: 'available',
message: null,
url: null,
})
})
it('returns available when ownership can be healed via shared GitHub identity', async () => {
vi.mocked(getAuthUserId).mockResolvedValue('users:caller' as never)
const result = (await checkSlugAvailabilityHandler(
createCtx({
skill: {
_id: 'skills:1',
slug: 'taken-skill',
ownerUserId: 'users:owner',
softDeletedAt: undefined,
moderationStatus: 'active',
moderationFlags: undefined,
},
owner: {
_id: 'users:owner',
handle: 'alice',
deletedAt: undefined,
deactivatedAt: undefined,
},
ownerProviderAccountId: 'shared-gh',
callerProviderAccountId: 'shared-gh',
}) as never,
{ slug: 'taken-skill' } as never,
)) as {
available: boolean
reason: string
message: string | null
url: string | null
}
expect(result).toEqual({
available: true,
reason: 'available',
message: null,
url: null,
})
})
})
+160 -19
View File
@@ -27,15 +27,22 @@ import {
adjustGlobalPublicSkillsCount,
countPublicSkillsForGlobalStats,
getPublicSkillVisibilityDelta,
isPublicSkillDoc,
readGlobalPublicSkillsCount,
} from './lib/globalStats'
import { buildTrendingLeaderboard } from './lib/leaderboards'
import { deriveModerationFlags } from './lib/moderation'
import { toPublicSkill, toPublicUser } from './lib/public'
import {
AUTO_HIDE_REPORT_THRESHOLD,
MAX_ACTIVE_REPORTS_PER_USER,
MAX_REPORT_REASON_LENGTH,
} from './lib/reporting'
import { embeddingVisibilityFor } from './lib/embeddingVisibility'
import { scheduleNextBatchIfNeeded } from './lib/batching'
import {
enforceReservedSlugCooldownForNewSkill,
formatReservedSlugCooldownMessage,
getLatestActiveReservedSlug,
listActiveReservedSlugsForSlug,
reserveSlugForHardDeleteFinalize,
@@ -64,8 +71,6 @@ const HARD_DELETE_BATCH_SIZE = 100
const HARD_DELETE_VERSION_BATCH_SIZE = 10
const HARD_DELETE_LEADERBOARD_BATCH_SIZE = 25
const BAN_USER_SKILLS_BATCH_SIZE = 25
const MAX_ACTIVE_REPORTS_PER_USER = 20
const AUTO_HIDE_REPORT_THRESHOLD = 3
const MAX_REPORT_REASON_SAMPLE = 5
const RATE_LIMIT_HOUR_MS = 60 * 60 * 1000
const RATE_LIMIT_DAY_MS = 24 * RATE_LIMIT_HOUR_MS
@@ -116,6 +121,20 @@ function stripSuspiciousFlag(flags: string[] | undefined) {
return next.length ? next : undefined
}
function buildConflictingSkillUrl(skill: Doc<'skills'>, owner: Doc<'users'> | null | undefined) {
if (!owner || owner.deletedAt || owner.deactivatedAt || !isPublicSkillDoc(skill)) return null
const ownerParam = owner.handle?.trim() || String(owner._id)
if (!ownerParam) return null
return `/${encodeURIComponent(ownerParam)}/${encodeURIComponent(skill.slug)}`
}
function buildSlugTakenErrorMessage(skill: Doc<'skills'>, owner: Doc<'users'> | null | undefined) {
const base = 'Slug is already taken. Choose a different slug.'
const url = buildConflictingSkillUrl(skill, owner)
if (!url) return base
return `${base} Existing skill: ${url}`
}
function normalizeScannerSuspiciousReason(reason: string | undefined) {
if (!reason) return reason
if (!reason.startsWith('scanner.') || !reason.endsWith('.suspicious')) return reason
@@ -186,6 +205,7 @@ const HARD_DELETE_PHASES = [
'fingerprints',
'embeddings',
'comments',
'commentReports',
'reports',
'stars',
'badges',
@@ -297,6 +317,21 @@ async function hardDeleteSkillStep(
await scheduleHardDelete(ctx, skill._id, actorUserId, 'comments')
return
}
await scheduleHardDelete(ctx, skill._id, actorUserId, 'commentReports')
return
}
case 'commentReports': {
const commentReports = await ctx.db
.query('commentReports')
.withIndex('by_skill', (q) => q.eq('skillId', skill._id))
.take(HARD_DELETE_BATCH_SIZE)
for (const report of commentReports) {
await ctx.db.delete(report._id)
}
if (commentReports.length === HARD_DELETE_BATCH_SIZE) {
await scheduleHardDelete(ctx, skill._id, actorUserId, 'commentReports')
return
}
await scheduleHardDelete(ctx, skill._id, actorUserId, 'reports')
return
}
@@ -493,8 +528,10 @@ type PublicSkillListVersion = Pick<
> & {
parsed?: {
clawdis?: {
os?: string[]
nix?: {
plugin?: boolean
systems?: string[]
}
}
}
@@ -779,6 +816,99 @@ export const getBySlug = query({
},
})
export const checkSlugAvailability = query({
args: { slug: v.string() },
handler: async (ctx, args) => {
const userId = await getAuthUserId(ctx)
const slug = args.slug.trim().toLowerCase()
if (!slug) {
return {
available: false,
reason: 'taken' as const,
message: 'Slug is required.',
url: null,
}
}
const skill = await ctx.db
.query('skills')
.withIndex('by_slug', (q) => q.eq('slug', slug))
.unique()
if (!skill) {
const reservation = await getLatestActiveReservedSlug(ctx, slug)
if (
reservation &&
reservation.expiresAt > Date.now() &&
reservation.originalOwnerUserId !== userId
) {
return {
available: false,
reason: 'reserved' as const,
message: formatReservedSlugCooldownMessage(slug, reservation.expiresAt),
url: null,
}
}
return {
available: true,
reason: 'available' as const,
message: null,
url: null,
}
}
if (userId && skill.ownerUserId === userId) {
return {
available: true,
reason: 'available' as const,
message: null,
url: null,
}
}
const owner = await ctx.db.get(skill.ownerUserId)
const url = buildConflictingSkillUrl(skill, owner)
const slugTakenMessage = buildSlugTakenErrorMessage(skill, owner)
if (!owner || owner.deletedAt || owner.deactivatedAt) {
return {
available: false,
reason: 'taken' as const,
message: slugTakenMessage,
url,
}
}
if (userId) {
const [ownerProviderAccountId, callerProviderAccountId] = await Promise.all([
getGitHubProviderAccountId(ctx, skill.ownerUserId),
getGitHubProviderAccountId(ctx, userId),
])
if (
canHealSkillOwnershipByGitHubProviderAccountId(
ownerProviderAccountId,
callerProviderAccountId,
)
) {
return {
available: true,
reason: 'available' as const,
message: null,
url: null,
}
}
}
return {
available: false,
reason: 'taken' as const,
message: slugTakenMessage,
url,
}
},
})
export const getBySlugForStaff = query({
args: { slug: v.string() },
handler: async (ctx, args) => {
@@ -1472,7 +1602,7 @@ export const report = mutation({
await ctx.db.insert('skillReports', {
skillId: args.skillId,
userId,
reason: reason.slice(0, 500),
reason: reason.slice(0, MAX_REPORT_REASON_LENGTH),
createdAt: now,
})
@@ -1614,24 +1744,35 @@ export const listPublicPageV2 = query({
// Use the index to filter out soft-deleted skills at query time.
// softDeletedAt === undefined means active (non-deleted) skills only.
const result = await paginateWithStaleCursorRecovery(runPaginate, initialCursor)
// When post-pagination filters are active, skip empty filtered pages so clients
// don't bounce between CanLoadMore/LoadingMore with no visible new rows.
let result = await paginateWithStaleCursorRecovery(runPaginate, initialCursor)
let filteredPage = filterPublicSkillPage(result.page, args)
const filteredPage =
args.nonSuspiciousOnly || args.highlightedOnly
? result.page.filter((skill) => {
if (args.nonSuspiciousOnly && isSkillSuspicious(skill)) return false
if (args.highlightedOnly && !isSkillHighlighted(skill)) return false
return true
})
: result.page
while ((args.nonSuspiciousOnly || args.highlightedOnly) && filteredPage.length === 0 && !result.isDone) {
result = await runPaginate(result.continueCursor)
filteredPage = filterPublicSkillPage(result.page, args)
}
// Build the public skill entries — skip version doc reads to reduce bandwidth.
// Version data is only needed for detail pages, not the listing.
const items = await buildPublicSkillEntries(ctx, filteredPage, { includeVersion: false })
const items = await buildPublicSkillEntries(ctx, filteredPage)
return { ...result, page: items }
},
})
function filterPublicSkillPage(
page: Array<Doc<'skills'>>,
args: { highlightedOnly?: boolean; nonSuspiciousOnly?: boolean },
) {
if (!args.nonSuspiciousOnly && !args.highlightedOnly) {
return page
}
return page.filter((skill) => {
if (args.nonSuspiciousOnly && isSkillSuspicious(skill)) return false
if (args.highlightedOnly && !isSkillHighlighted(skill)) return false
return true
})
}
function normalizePublicListPagination(paginationOpts: {
cursor?: string | null
numItems: number
@@ -2646,7 +2787,6 @@ export const approveSkillByHashInternal = internalMutation({
const existingFlags: string[] = (skill.moderationFlags as string[] | undefined) ?? []
const existingReason: string | undefined = skill.moderationReason as string | undefined
const alreadyBlocked = existingFlags.includes('blocked.malware')
const alreadyFlagged = existingFlags.includes('flagged.suspicious')
const bypassSuspicious =
isSuspicious && !alreadyBlocked && isPrivilegedOwnerForSuspiciousBypass(owner)
@@ -3569,8 +3709,9 @@ export const insertVersion = internalMutation({
// Fallback: Convex Auth can create duplicate `users` records. Heal ownership ONLY
// when the underlying GitHub identity matches (authAccounts.providerAccountId).
const owner = await ctx.db.get(skill.ownerUserId)
const slugTakenMessage = buildSlugTakenErrorMessage(skill, owner)
if (!owner || owner.deletedAt || owner.deactivatedAt) {
throw new Error('Only the owner can publish updates')
throw new ConvexError(slugTakenMessage)
}
const [ownerProviderAccountId, callerProviderAccountId] = await Promise.all([
@@ -3585,7 +3726,7 @@ export const insertVersion = internalMutation({
callerProviderAccountId,
)
) {
throw new Error('Only the owner can publish updates')
throw new ConvexError(slugTakenMessage)
}
await ctx.db.patch(skill._id, { ownerUserId: userId, updatedAt: now })
@@ -3725,7 +3866,7 @@ export const insertVersion = internalMutation({
.withIndex('by_skill_version', (q) => q.eq('skillId', skill._id).eq('version', args.version))
.unique()
if (existingVersion) {
throw new Error('Version already exists')
throw new ConvexError('Version already exists')
}
const versionId = await ctx.db.insert('skillVersions', {
+79
View File
@@ -0,0 +1,79 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from 'vitest'
vi.mock('./lib/access', () => ({
assertModerator: vi.fn(),
requireUser: vi.fn(),
}))
vi.mock('./lib/githubAccount', () => ({
requireGitHubAccountAge: vi.fn(),
}))
const { requireUser } = await import('./lib/access')
const { requireGitHubAccountAge } = await import('./lib/githubAccount')
const { addHandler } = await import('./soulComments')
describe('soul comments mutations', () => {
afterEach(() => {
vi.mocked(requireUser).mockReset()
vi.mocked(requireGitHubAccountAge).mockReset()
vi.restoreAllMocks()
})
it('add enforces github account age and writes comment', async () => {
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000)
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:1',
user: { _id: 'users:1', role: 'user' },
} as never)
vi.mocked(requireGitHubAccountAge).mockResolvedValue(undefined as never)
const get = vi.fn().mockResolvedValue({
_id: 'souls:1',
stats: { comments: 3 },
})
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await addHandler(ctx, { soulId: 'souls:1', body: ' hello soul ' } as never)
expect(requireGitHubAccountAge).toHaveBeenCalledWith(ctx, 'users:1')
expect(insert).toHaveBeenCalledWith('soulComments', {
soulId: 'souls:1',
userId: 'users:1',
body: 'hello soul',
createdAt: 1_700_000_000_000,
softDeletedAt: undefined,
deletedBy: undefined,
})
expect(patch).toHaveBeenCalledWith('souls:1', {
stats: { comments: 4 },
updatedAt: 1_700_000_000_000,
})
})
it('add rejects when github account age gate fails', async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: 'users:new',
user: { _id: 'users:new', role: 'user' },
} as never)
vi.mocked(requireGitHubAccountAge).mockRejectedValue(
new Error('GitHub account must be at least 14 days old to upload skills. Try again in 5 days.'),
)
const get = vi.fn()
const insert = vi.fn()
const patch = vi.fn()
const ctx = { db: { get, insert, patch } } as never
await expect(addHandler(ctx, { soulId: 'souls:1', body: 'hello' } as never)).rejects.toThrow(
/at least 14 days old/i,
)
expect(get).not.toHaveBeenCalled()
expect(insert).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
})
})
+67 -55
View File
@@ -1,7 +1,10 @@
import { v } from 'convex/values'
import type { Id } from './_generated/dataModel'
import type { Doc } from './_generated/dataModel'
import type { MutationCtx } from './_generated/server'
import { mutation, query } from './_generated/server'
import { assertModerator, requireUser } from './lib/access'
import { requireGitHubAccountAge } from './lib/githubAccount'
import { type PublicUser, toPublicUser } from './lib/public'
export const listBySoul = query({
@@ -26,63 +29,72 @@ export const listBySoul = query({
export const add = mutation({
args: { soulId: v.id('souls'), body: v.string() },
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
const soul = await ctx.db.get(args.soulId)
if (!soul) throw new Error('Soul not found')
await ctx.db.insert('soulComments', {
soulId: args.soulId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
})
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, comments: soul.stats.comments + 1 },
updatedAt: Date.now(),
})
},
handler: addHandler,
})
export const remove = mutation({
args: { commentId: v.id('soulComments') },
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment) throw new Error('Comment not found')
if (comment.softDeletedAt) return
const isOwner = comment.userId === user._id
if (!isOwner) {
assertModerator(user)
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
})
const soul = await ctx.db.get(comment.soulId)
if (soul) {
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, comments: Math.max(0, soul.stats.comments - 1) },
updatedAt: Date.now(),
})
}
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
action: 'soul.comment.delete',
targetType: 'soulComment',
targetId: comment._id,
metadata: { soulId: comment.soulId },
createdAt: Date.now(),
})
},
handler: removeHandler,
})
export async function addHandler(ctx: MutationCtx, args: { soulId: Id<'souls'>; body: string }) {
const { userId } = await requireUser(ctx)
await requireGitHubAccountAge(ctx, userId)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
const soul = await ctx.db.get(args.soulId)
if (!soul) throw new Error('Soul not found')
await ctx.db.insert('soulComments', {
soulId: args.soulId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
})
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, comments: soul.stats.comments + 1 },
updatedAt: Date.now(),
})
}
export async function removeHandler(
ctx: MutationCtx,
args: { commentId: Id<'soulComments'> },
) {
const { user } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment) throw new Error('Comment not found')
if (comment.softDeletedAt) return
const isOwner = comment.userId === user._id
if (!isOwner) {
assertModerator(user)
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
})
const soul = await ctx.db.get(comment.soulId)
if (soul) {
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, comments: Math.max(0, soul.stats.comments - 1) },
updatedAt: Date.now(),
})
}
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
action: 'soul.comment.delete',
targetType: 'soulComment',
targetId: comment._id,
metadata: { soulId: comment.soulId },
createdAt: Date.now(),
})
}
+70
View File
@@ -0,0 +1,70 @@
import { describe, expect, it, vi } from 'vitest'
import { insertVersion } from './souls'
type WrappedHandler<TArgs> = {
_handler: (ctx: unknown, args: TArgs) => Promise<unknown>
}
const insertVersionHandler = (insertVersion as unknown as WrappedHandler<Record<string, unknown>>)
._handler
describe('souls.insertVersion', () => {
it('throws a soul-specific ownership error for non-owners', async () => {
const db = {
get: vi.fn(async (id: string) => {
if (id === 'users:caller') return { _id: 'users:caller', deletedAt: undefined }
return null
}),
query: vi.fn((table: string) => {
if (table !== 'souls') throw new Error(`unexpected table ${table}`)
return {
withIndex: (name: string) => {
if (name !== 'by_slug') throw new Error(`unexpected index ${name}`)
return {
order: () => ({
take: async () => [
{
_id: 'souls:1',
slug: 'demo-soul',
ownerUserId: 'users:owner',
softDeletedAt: undefined,
},
],
}),
}
},
}
}),
}
await expect(
insertVersionHandler(
{ db } as never,
{
userId: 'users:caller',
slug: 'demo-soul',
displayName: 'Demo Soul',
version: '1.0.0',
changelog: 'Initial',
changelogSource: 'user',
tags: ['latest'],
fingerprint: 'f'.repeat(64),
files: [
{
path: 'SOUL.md',
size: 100,
storageId: '_storage:1',
sha256: 'a'.repeat(64),
contentType: 'text/markdown',
},
],
parsed: {
frontmatter: {},
metadata: {},
},
embedding: [0.1, 0.2],
} as never,
),
).rejects.toThrow('Only the owner can publish soul updates')
})
})
+1 -1
View File
@@ -405,7 +405,7 @@ export const insertVersion = internalMutation({
let soul: Doc<'souls'> | null = soulMatches[0] ?? null
if (soul && soul.ownerUserId !== userId) {
throw new Error('Only the owner can publish updates')
throw new ConvexError('Only the owner can publish soul updates')
}
const now = Date.now()
+165 -1
View File
@@ -5,8 +5,13 @@ vi.mock('./lib/access', async () => {
return { ...actual, requireUser: vi.fn() }
})
vi.mock('./skillStatEvents', () => ({
insertStatEvent: vi.fn(),
}))
const { requireUser } = await import('./lib/access')
const { ensureHandler, list, searchInternal } = await import('./users')
const { insertStatEvent } = await import('./skillStatEvents')
const { ensureHandler, list, searchInternal, banUserInternal } = await import('./users')
function makeCtx() {
const patch = vi.fn()
@@ -30,6 +35,48 @@ function makeListCtx(users: Array<Record<string, unknown>>) {
}
}
function makeBanCtx() {
const patch = vi.fn()
const insert = vi.fn()
const get = vi.fn()
const runMutation = vi.fn()
const apiTokens = [{ _id: 'apiTokens:1', revokedAt: undefined }]
const userComments = [
{
_id: 'comments:active',
userId: 'users:target',
skillId: 'skills:1',
softDeletedAt: undefined,
},
{
_id: 'comments:already-deleted',
userId: 'users:target',
skillId: 'skills:1',
softDeletedAt: 123,
},
]
const soulComments = [
{
_id: 'soulComments:active',
userId: 'users:target',
soulId: 'souls:1',
softDeletedAt: undefined,
},
]
const query = vi.fn((table: string) => ({
withIndex: (_index: string, _cb: unknown) => {
if (table === 'apiTokens') return { collect: vi.fn().mockResolvedValue(apiTokens) }
if (table === 'comments') return { collect: vi.fn().mockResolvedValue(userComments) }
if (table === 'soulComments') return { collect: vi.fn().mockResolvedValue(soulComments) }
throw new Error(`Unexpected table ${table}`)
},
}))
const ctx = { db: { patch, insert, get, query }, runMutation } as never
return { ctx, patch, insert, get, runMutation }
}
describe('ensureHandler', () => {
afterEach(() => {
vi.mocked(requireUser).mockReset()
@@ -443,3 +490,120 @@ describe('users.searchInternal', () => {
expect(result.items).toHaveLength(200)
})
})
describe('users.banUserInternal', () => {
afterEach(() => {
vi.mocked(insertStatEvent).mockReset()
vi.restoreAllMocks()
})
it('soft-deletes target user comments (skill + soul) during ban', async () => {
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000)
const { ctx, get, patch, insert, runMutation } = makeBanCtx()
get.mockImplementation(async (id: string) => {
if (id === 'users:actor') return { _id: 'users:actor', role: 'moderator' }
if (id === 'users:target') return { _id: 'users:target', role: 'user' }
if (id === 'souls:1') return { _id: 'souls:1', stats: { comments: 3 } }
return null
})
runMutation
.mockResolvedValueOnce({ hiddenCount: 2, scheduled: false })
.mockResolvedValueOnce(undefined)
const handler = (
banUserInternal as unknown as {
_handler: (
ctx: unknown,
args: { actorUserId: string; targetUserId: string; reason?: string },
) => Promise<unknown>
}
)._handler
const result = (await handler(ctx, {
actorUserId: 'users:actor',
targetUserId: 'users:target',
reason: 'spam',
})) as {
ok: boolean
alreadyBanned: boolean
deletedComments: { skillComments: number; soulComments: number }
}
expect(result).toMatchObject({
ok: true,
alreadyBanned: false,
deletedComments: { skillComments: 1, soulComments: 1 },
})
expect(patch).toHaveBeenCalledWith('comments:active', {
softDeletedAt: 1_700_000_000_000,
deletedBy: 'users:actor',
})
expect(patch).toHaveBeenCalledWith('soulComments:active', {
softDeletedAt: 1_700_000_000_000,
deletedBy: 'users:actor',
})
expect(patch).toHaveBeenCalledWith('souls:1', {
stats: { comments: 2 },
updatedAt: 1_700_000_000_000,
})
expect(insertStatEvent).toHaveBeenCalledWith(ctx, { skillId: 'skills:1', kind: 'uncomment' })
expect(insert).toHaveBeenCalledWith(
'auditLogs',
expect.objectContaining({
action: 'user.ban',
metadata: expect.objectContaining({
deletedSkillComments: 1,
deletedSoulComments: 1,
}),
}),
)
})
it('re-ban of already banned user still cleans lingering comments', async () => {
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000)
const { ctx, get, patch, runMutation } = makeBanCtx()
get.mockImplementation(async (id: string) => {
if (id === 'users:actor') return { _id: 'users:actor', role: 'moderator' }
if (id === 'users:target') return { _id: 'users:target', role: 'user', deletedAt: 1_600_000_000_000 }
if (id === 'souls:1') return { _id: 'souls:1', stats: { comments: 3 } }
return null
})
const handler = (
banUserInternal as unknown as {
_handler: (
ctx: unknown,
args: { actorUserId: string; targetUserId: string; reason?: string },
) => Promise<unknown>
}
)._handler
const result = (await handler(ctx, {
actorUserId: 'users:actor',
targetUserId: 'users:target',
reason: 'cleanup',
})) as {
ok: boolean
alreadyBanned: boolean
deletedComments: { skillComments: number; soulComments: number }
deletedSkills: number
}
expect(result).toEqual({
ok: true,
alreadyBanned: true,
deletedSkills: 0,
deletedComments: { skillComments: 1, soulComments: 1 },
})
expect(runMutation).not.toHaveBeenCalled()
expect(patch).toHaveBeenCalledWith('comments:active', {
softDeletedAt: 1_600_000_000_000,
deletedBy: 'users:actor',
})
})
})
+93 -5
View File
@@ -8,6 +8,7 @@ import { assertAdmin, assertModerator, requireUser } from './lib/access'
import { syncGitHubProfile } from './lib/githubAccount'
import { toPublicUser } from './lib/public'
import { buildUserSearchResults } from './lib/userSearch'
import { insertStatEvent } from './skillStatEvents'
const DEFAULT_ROLE = 'user'
const ADMIN_HANDLE = 'steipete'
@@ -425,8 +426,16 @@ async function banUserWithActor(
if (reason && reason.length > 500) {
throw new Error('Reason too long (max 500 chars)')
}
if (target.deletedAt || target.deactivatedAt) {
return { ok: true as const, alreadyBanned: true, deletedSkills: 0 }
if (target.deactivatedAt) {
return { ok: true as const, alreadyBanned: true, deletedSkills: 0, deletedComments: { skillComments: 0, soulComments: 0 } }
}
if (target.deletedAt) {
const deletedComments = await softDeleteUserCommentsForBan(ctx, {
userId: targetUserId,
deletedBy: actor._id,
deletedAt: target.deletedAt,
})
return { ok: true as const, alreadyBanned: true, deletedSkills: 0, deletedComments }
}
const banSkillsResult = (await ctx.runMutation(
@@ -451,6 +460,12 @@ async function banUserWithActor(
}
}
const deletedComments = await softDeleteUserCommentsForBan(ctx, {
userId: targetUserId,
deletedBy: actor._id,
deletedAt: now,
})
await ctx.db.patch(targetUserId, {
deletedAt: now,
role: 'user',
@@ -465,11 +480,22 @@ async function banUserWithActor(
action: 'user.ban',
targetType: 'user',
targetId: targetUserId,
metadata: { hiddenSkills: hiddenCount, reason: reason || undefined },
metadata: {
hiddenSkills: hiddenCount,
deletedSkillComments: deletedComments.skillComments,
deletedSoulComments: deletedComments.soulComments,
reason: reason || undefined,
},
createdAt: now,
})
return { ok: true as const, alreadyBanned: false, deletedSkills: hiddenCount, scheduledSkills }
return {
ok: true as const,
alreadyBanned: false,
deletedSkills: hiddenCount,
deletedComments,
scheduledSkills,
}
}
async function unbanUserWithActor(
@@ -640,6 +666,12 @@ export const autobanMalwareAuthorInternal = internalMutation({
}
}
const deletedComments = await softDeleteUserCommentsForBan(ctx, {
userId: args.ownerUserId,
deletedBy: args.ownerUserId,
deletedAt: now,
})
// Ban the user
await ctx.db.patch(args.ownerUserId, {
deletedAt: now,
@@ -663,6 +695,8 @@ export const autobanMalwareAuthorInternal = internalMutation({
sha256hash: args.sha256hash,
slug: args.slug,
hiddenSkills: hiddenCount,
deletedSkillComments: deletedComments.skillComments,
deletedSoulComments: deletedComments.soulComments,
},
createdAt: now,
})
@@ -671,6 +705,60 @@ export const autobanMalwareAuthorInternal = internalMutation({
`[autoban] Banned ${target.handle ?? args.ownerUserId} — malicious skill: ${args.slug}`,
)
return { ok: true, alreadyBanned: false, deletedSkills: hiddenCount, scheduledSkills }
return {
ok: true,
alreadyBanned: false,
deletedSkills: hiddenCount,
deletedComments,
scheduledSkills,
}
},
})
async function softDeleteUserCommentsForBan(
ctx: MutationCtx,
args: { userId: Id<'users'>; deletedBy: Id<'users'>; deletedAt: number },
) {
let skillComments = 0
let soulComments = 0
const comments = await ctx.db
.query('comments')
.withIndex('by_user', (q) => q.eq('userId', args.userId))
.collect()
for (const comment of comments) {
if (comment.softDeletedAt) continue
await ctx.db.patch(comment._id, {
softDeletedAt: args.deletedAt,
deletedBy: args.deletedBy,
})
await insertStatEvent(ctx, { skillId: comment.skillId, kind: 'uncomment' })
skillComments += 1
}
const soulCommentDocs = await ctx.db
.query('soulComments')
.withIndex('by_user', (q) => q.eq('userId', args.userId))
.collect()
const soulCommentCounts = new Map<Id<'souls'>, number>()
for (const comment of soulCommentDocs) {
if (comment.softDeletedAt) continue
await ctx.db.patch(comment._id, {
softDeletedAt: args.deletedAt,
deletedBy: args.deletedBy,
})
soulCommentCounts.set(comment.soulId, (soulCommentCounts.get(comment.soulId) ?? 0) + 1)
soulComments += 1
}
for (const [soulId, count] of soulCommentCounts.entries()) {
const soul = await ctx.db.get(soulId)
if (!soul) continue
await ctx.db.patch(soulId, {
stats: { ...soul.stats, comments: Math.max(0, soul.stats.comments - count) },
updatedAt: args.deletedAt,
})
}
return { skillComments, soulComments }
}
+42
View File
@@ -58,3 +58,45 @@ describe('vt activation fallback', () => {
).toBe(false)
})
})
describe('vt AV engine fallback verdicts', () => {
it('maps engine verdicts in severity order', () => {
expect(
__test.statusFromAvStats({
malicious: 1,
suspicious: 2,
harmless: 10,
undetected: 40,
}),
).toBe('malicious')
expect(
__test.statusFromAvStats({
malicious: 0,
suspicious: 1,
harmless: 10,
undetected: 40,
}),
).toBe('suspicious')
expect(
__test.statusFromAvStats({
malicious: 0,
suspicious: 0,
harmless: 1,
undetected: 40,
}),
).toBe('clean')
})
it('keeps undetected-only results pending', () => {
expect(
__test.statusFromAvStats({
malicious: 0,
suspicious: 0,
harmless: 0,
undetected: 40,
}),
).toBeNull()
})
})
+133 -15
View File
@@ -122,6 +122,8 @@ type VTFileResponse = {
}
}
type VTAnalysisStats = NonNullable<VTFileResponse['data']['attributes']['last_analysis_stats']>
type ScanQueueHealth = {
queueSize: number
staleCount: number
@@ -246,6 +248,15 @@ function shouldActivateWhenVtUnavailable(skill: SkillActivationCandidate | null
return typeof reason === 'string' && VT_PENDING_REASONS.has(reason)
}
function statusFromAvStats(stats?: VTAnalysisStats | null): 'malicious' | 'suspicious' | 'clean' | null {
if (!stats) return null
if (stats.malicious > 0) return 'malicious'
if (stats.suspicious > 0) return 'suspicious'
// Keep this aligned with fetchResults: undetected-only should stay pending.
if (stats.harmless > 0) return 'clean'
return null
}
async function activateSkillWhenVtUnavailable(ctx: ActionCtx, skillId: Id<'skills'>) {
const skill = await ctx.runQuery(internal.skills.getSkillByIdInternal, { skillId })
if (!shouldActivateWhenVtUnavailable(skill)) return
@@ -294,15 +305,8 @@ export const fetchResults = action({
if (aiResult?.verdict) {
// Prioritize AI Analysis (Code Insight)
status = verdictToStatus(normalizeVerdict(aiResult.verdict))
} else if (stats) {
// Fallback to AV engines
if (stats.malicious > 0) {
status = 'malicious'
} else if (stats.suspicious > 0) {
status = 'suspicious'
} else if (stats.harmless > 0) {
status = 'clean'
}
} else {
status = statusFromAvStats(stats) ?? 'pending'
}
return {
@@ -566,9 +570,40 @@ export const pollPendingScans = internalAction({
)
if (!aiResult) {
// No Code Insight - trigger a rescan to get it
// No Code Insight - check AV engine stats as fallback
const stats = vtResult.data.attributes.last_analysis_stats
const status = statusFromAvStats(stats)
let source = 'engines'
if (status) {
// We have a verdict from AV engines - update the skill
console.log(
`[vt:pollPendingScans] Hash ${sha256hash} verdict from AV engines: ${status}`,
)
// Cache VT analysis in version
await ctx.runMutation(internal.skills.updateVersionScanResultsInternal, {
versionId,
vtAnalysis: {
status,
source,
checkedAt: Date.now(),
},
})
// VT finalizes moderation visibility for newly published versions.
await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
sha256hash,
scanner: 'vt',
status,
})
updated++
continue
}
// No verdict from engines either - trigger a rescan to get Code Insight
console.log(
`[vt:pollPendingScans] Hash ${sha256hash} has no Code Insight, requesting rescan`,
`[vt:pollPendingScans] Hash ${sha256hash} has no Code Insight or engine stats, requesting rescan`,
)
await requestRescan(apiKey, sha256hash)
// Check if we've exceeded max attempts — write stale vtAnalysis so it
@@ -684,6 +719,7 @@ async function requestRescan(apiKey: string, sha256hash: string): Promise<boolea
}
export const __test = {
statusFromAvStats,
shouldActivateWhenVtUnavailable,
}
@@ -741,7 +777,25 @@ export const backfillPendingScans = internalAction({
)
if (!aiResult) {
// No Code Insight - check AV engine stats as fallback
const stats = vtResult.data.attributes.last_analysis_stats
const status = statusFromAvStats(stats)
if (status) {
// We have a verdict from AV engines - update the skill
console.log(`[vt:backfill] Hash ${sha256hash} verdict from AV engines: ${status}`)
await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
sha256hash,
scanner: 'vt',
status,
})
updated++
continue
}
// No verdict from engines either - trigger a rescan
if (triggerRescans) {
console.log(`[vt:backfill] Hash ${sha256hash} has no Code Insight or engine stats, requesting rescan`)
await requestRescan(apiKey, sha256hash)
rescansRequested++
}
@@ -840,14 +894,56 @@ export const rescanActiveSkills = internalAction({
)
if (!aiResult) {
// No Code Insight - check AV engine stats as fallback
const stats = vtResult.data.attributes.last_analysis_stats
const status = statusFromAvStats(stats)
let source = 'engines'
if (!status) {
// No verdict from engines either - keep as pending
await ctx.runMutation(internal.skills.updateVersionScanResultsInternal, {
versionId,
vtAnalysis: {
status: 'pending',
checkedAt: Date.now(),
},
})
accUnchanged++
continue
}
// We have a verdict from AV engines - continue with normal flow
console.log(`[vt:rescan] ${slug} verdict from AV engines: ${status}`)
await ctx.runMutation(internal.skills.updateVersionScanResultsInternal, {
versionId,
vtAnalysis: {
status: 'pending',
status,
source,
checkedAt: Date.now(),
},
})
accUnchanged++
if (status === 'malicious' || status === 'suspicious') {
console.warn(`[vt:rescan] ${slug}: verdict changed to ${status}!`)
accFlaggedSkills.push({ slug, status })
await ctx.runMutation(internal.skills.escalateByVtInternal, {
sha256hash,
status,
})
accUpdated++
} else if (wasFlagged && status === 'clean') {
// Verdict improved from suspicious → clean: clear the stale moderation flag
console.log(`[vt:rescan] ${slug}: verdict improved to clean, clearing suspicious flag`)
await ctx.runMutation(internal.skills.approveSkillByHashInternal, {
sha256hash,
scanner: 'vt',
status,
})
accUpdated++
} else {
accUnchanged++
}
continue
}
@@ -1121,8 +1217,30 @@ export const backfillActiveSkillsVTCache = internalAction({
)
if (!aiResult) {
console.log(`[vt:backfillActive] ${slug}: no Code Insight yet`)
noResults++
// No Code Insight - check AV engine stats as fallback
const stats = vtResult.data.attributes.last_analysis_stats
const status = statusFromAvStats(stats)
let source = 'engines'
if (!status) {
console.log(`[vt:backfillActive] ${slug}: no Code Insight or engine stats yet`)
noResults++
continue
}
// We have a verdict from AV engines - update the version
console.log(`[vt:backfillActive] ${slug}: updated with ${status} (from AV engines)`)
await ctx.runMutation(internal.skills.updateVersionScanResultsInternal, {
versionId,
sha256hash,
vtAnalysis: {
status,
source,
checkedAt: Date.now(),
},
})
updated++
continue
}
+28
View File
@@ -29,6 +29,34 @@ Env equivalents:
- `CLAWHUB_REGISTRY` (legacy `CLAWDHUB_REGISTRY`)
- `CLAWHUB_WORKDIR` (legacy `CLAWDHUB_WORKDIR`)
### HTTP proxy
The CLI respects standard HTTP proxy environment variables for systems behind
corporate proxies or restricted networks:
- `HTTPS_PROXY` / `https_proxy`
- `HTTP_PROXY` / `http_proxy`
- `NO_PROXY` / `no_proxy`
When any of these variables is set, the CLI routes outbound requests through
the specified proxy. `HTTPS_PROXY` is used for HTTPS requests, `HTTP_PROXY`
for plain HTTP. `NO_PROXY` / `no_proxy` is respected to bypass the proxy for
specific hosts or domains.
This is required on systems where direct outbound connections are blocked
(e.g. Docker containers, Hetzner VPS with proxy-only internet, corporate
firewalls).
Example:
```bash
export HTTPS_PROXY=http://proxy.example.com:3128
export NO_PROXY=localhost,127.0.0.1
clawhub search "my query"
```
When no proxy variable is set, behavior is unchanged (direct connections).
## Config file
Stores your API token + cached registry URL.
+8 -2
View File
@@ -100,7 +100,7 @@ Notes:
Response:
```json
{ "items": [{ "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "1.2.3" }, "stats": {}, "createdAt": 0, "updatedAt": 0, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" } }], "nextCursor": null }
{ "items": [{ "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "1.2.3" }, "stats": {}, "createdAt": 0, "updatedAt": 0, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" }, "metadata": { "os": ["macos"], "systems": ["aarch64-darwin"] } }], "nextCursor": null }
```
### `GET /api/v1/skills/{slug}`
@@ -108,9 +108,15 @@ Response:
Response:
```json
{ "skill": { "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "1.2.3" }, "stats": {}, "createdAt": 0, "updatedAt": 0 }, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" }, "owner": { "handle": "steipete", "displayName": "Peter", "image": null } }
{ "skill": { "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "1.2.3" }, "stats": {}, "createdAt": 0, "updatedAt": 0 }, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" }, "metadata": { "os": ["macos"], "systems": ["aarch64-darwin"] }, "owner": { "handle": "steipete", "displayName": "Peter", "image": null } }
```
Notes:
- `metadata.os`: OS restrictions declared in skill frontmatter (e.g. `["macos"]`, `["linux"]`). `null` if not declared.
- `metadata.systems`: Nix system targets (e.g. `["aarch64-darwin", "x86_64-linux"]`). `null` if not declared.
- `metadata` is `null` if the skill has no platform metadata.
### `GET /api/v1/skills/{slug}/versions`
Query params:
+35 -11
View File
@@ -10,32 +10,55 @@ read_when:
## Roles + permissions
- user: upload skills/souls (subject to GitHub age gate), report skills.
- user: upload skills/souls (subject to GitHub age gate), report skills/comments.
- moderator: hide/restore skills, view hidden skills, unhide, soft-delete, ban users (except admins).
- admin: all moderator actions + hard delete skills, change owners, change roles.
## Reporting + auto-hide
- Reports are unique per user + skill.
- Reports are unique per user + target (skill/comment).
- Report reason required (trimmed, max 500 chars). Abuse of reporting may result in account bans.
- Per-user cap: 20 **active** reports.
- Active = skill exists, not soft-deleted, not `moderationStatus = removed`,
- Active skill report = skill exists, not soft-deleted, not `moderationStatus = removed`,
and the owner is not banned.
- Auto-hide: when unique reports exceed 3 (4th report), the skill is:
- soft-deleted (`softDeletedAt`)
- `moderationStatus = hidden`
- `moderationReason = auto.reports`
- embeddings visibility set to `deleted`
- audit log entry: `skill.auto_hide`
- Active comment report = comment exists, not soft-deleted, parent skill still active,
and the comment author is not banned/deactivated.
- Auto-hide: when unique reports exceed 3 (4th report):
- skill report flow:
- soft-delete skill (`softDeletedAt`)
- set `moderationStatus = hidden`
- set `moderationReason = auto.reports`
- set embeddings visibility `deleted`
- audit log entry: `skill.auto_hide`
- comment report flow:
- soft-delete comment (`softDeletedAt`)
- decrement comment stat via `uncomment` stat event
- audit log entry: `comment.auto_hide`
- Public queries hide non-active moderation statuses; staff can still access via
staff-only queries and unhide/restore/delete/ban.
- Skills directory supports an optional "Hide suspicious" filter to exclude
active-but-flagged (`flagged.suspicious`) entries from browse/search results.
## AI comment scam backfill
- Moderators/admins can run a comment backfill scanner to classify scam comments with OpenAI.
- Scanner stores per-comment moderation metadata:
- `scamScanVerdict`: `not_scam | likely_scam | certain_scam`
- `scamScanConfidence`: `low | medium | high`
- explanation/evidence/model/check timestamp fields on `comments`.
- Auto-ban trigger is intentionally strict:
- only `certain_scam` with `high` confidence can trigger account ban.
- moderator/admin accounts are never auto-banned by this pipeline.
- Ban reason is bounded to 500 chars and includes concise evidence + comment/skill IDs.
- CLI run examples:
- one-shot: `npx convex run commentModeration:backfillCommentScamModeration '{"batchSize":25,"maxBatches":20}'`
- background chain: `npx convex run commentModeration:scheduleCommentScamModeration '{"batchSize":25}'`
## Bans
- Banning a user:
- hard-deletes all owned skills
- soft-deletes all authored skill comments + soul comments
- revokes API tokens
- sets `deletedAt` on the user
- Admins can manually unban (`deletedAt` + `banReason` cleared); revoked API tokens
@@ -58,11 +81,12 @@ read_when:
## Upload gate (GitHub account age)
- Skill + soul publish actions require GitHub account age ≥ 7 days.
- Skill + soul publish actions require GitHub account age ≥ 14 days.
- Skill + soul comment creation also requires GitHub account age ≥ 14 days.
- Lookup uses GitHub `created_at` fetched by the immutable GitHub numeric ID (`providerAccountId`)
and caches on the user:
- `githubCreatedAt` (source of truth)
- Gate applies to web uploads, CLI publish, and GitHub import.
- Gate applies to web uploads, CLI publish, GitHub import, and comments.
- If GitHub responds `403` or `429`, publish fails with:
- `GitHub API rate limit exceeded — please try again in a few minutes`
- To reduce rate-limit failures, set `GITHUB_TOKEN` in Convex env for authenticated
+3 -2
View File
@@ -125,7 +125,8 @@ From SKILL.md frontmatter + AgentSkills + Clawdis extensions:
- Default role `user`; bootstrap `steipete` to `admin` on first login.
- Management console: moderators can hide/restore skills + mark duplicates + ban users; admins can change owners, approve badges, hard-delete skills, and ban users (deletes owned skills).
- Role changes are admin-only and audited.
- Reporting: any user can report skills; per-user cap 20 active reports; skills auto-hide after >3 unique reports (mods can review/unhide/delete/ban).
- Reporting: any user can report skills/comments; per-user cap 20 active reports; targets auto-hide after >3 unique reports (mods can review/unhide/delete/ban).
- Commenting (skills + souls) requires GitHub account age ≥ 14 days.
## Upload flow (50MB per version)
1) Client requests upload session.
@@ -136,7 +137,7 @@ From SKILL.md frontmatter + AgentSkills + Clawdis extensions:
- file extensions/text content
- SKILL.md exists and frontmatter parseable
- version uniqueness
- GitHub account age ≥ 7 days
- GitHub account age ≥ 14 days
5) Server stores files + metadata, sets `latest` tag, updates stats.
Soul upload flow: same as skills (including GitHub account age checks), but only `SOUL.md` is allowed.
+19
View File
@@ -28,6 +28,25 @@ read_when:
- If many users share one egress IP (NAT/proxy), IP limit can be hit even with valid tokens.
- For non-Cloudflare deploys behind trusted proxies, set `TRUST_FORWARDED_IPS=true` so forwarded client IPs can be used.
## `search` / `install` fails with `fetch failed` behind a proxy
If your system requires an HTTP proxy for outbound connections (e.g. corporate
firewalls, Docker containers with proxy-only internet, Hetzner VPS), the CLI
will fail with:
```
✖ fetch failed
Error: fetch failed
```
**Fix:** Set the standard proxy environment variables:
```bash
export HTTPS_PROXY=http://proxy.example.com:3128
clawhub search "my query"
```
The CLI respects `HTTPS_PROXY`, `HTTP_PROXY`, `https_proxy`, and `http_proxy`.
## `publish` fails with `OPENAI_API_KEY is not configured`
- Set `OPENAI_API_KEY` in the Convex environment (not only locally).
+16 -2
View File
@@ -88,10 +88,24 @@ test('skills search paginates exact results', async ({ page }) => {
await expect(page.getByText('Skill 0')).toBeVisible()
await expect(page.getByText('Scroll to load more')).toBeVisible()
await expect
.poll(
() =>
page.evaluate(
() => (window as typeof window & { __searchLimits: number[] }).__searchLimits.length,
),
{ timeout: 10_000 },
)
.toBeGreaterThan(0)
const initialLimit = await page.evaluate(
() => (window as typeof window & { __searchLimits: number[] }).__searchLimits[0] ?? 0,
)
expect(initialLimit).toBeGreaterThan(0)
await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight))
await expect(page.getByText('Skill 75')).toBeVisible()
await expect(page.getByText(`Skill ${initialLimit + 5}`)).toBeVisible()
const limits = await page.evaluate(
() => (window as typeof window & { __searchLimits: number[] }).__searchLimits,
)
expect(limits).toEqual([50, 100])
expect(Math.max(...limits)).toBeGreaterThan(initialLimit)
})
+4 -4
View File
@@ -278,7 +278,7 @@ program
program
.command('delete')
.description('Soft-delete a skill (moderator/admin only)')
.description('Soft-delete a skill (owner, moderator, or admin)')
.argument('<slug>', 'Skill slug')
.option('--yes', 'Skip confirmation')
.action(async (slug, options) => {
@@ -288,7 +288,7 @@ program
program
.command('hide')
.description('Hide a skill (moderator/admin only)')
.description('Hide a skill (owner, moderator, or admin)')
.argument('<slug>', 'Skill slug')
.option('--yes', 'Skip confirmation')
.action(async (slug, options) => {
@@ -298,7 +298,7 @@ program
program
.command('undelete')
.description('Restore a hidden skill (moderator/admin only)')
.description('Restore a hidden skill (owner, moderator, or admin)')
.argument('<slug>', 'Skill slug')
.option('--yes', 'Skip confirmation')
.action(async (slug, options) => {
@@ -308,7 +308,7 @@ program
program
.command('unhide')
.description('Unhide a skill (moderator/admin only)')
.description('Unhide a skill (owner, moderator, or admin)')
.argument('<slug>', 'Skill slug')
.option('--yes', 'Skip confirmation')
.action(async (slug, options) => {
+4 -4
View File
@@ -16,28 +16,28 @@ const deleteLabels: SkillActionLabels = {
verb: 'Delete',
progress: 'Deleting',
past: 'Deleted',
promptSuffix: 'soft delete, requires moderator/admin',
promptSuffix: 'soft delete, owner/moderator/admin',
}
const undeleteLabels: SkillActionLabels = {
verb: 'Undelete',
progress: 'Undeleting',
past: 'Undeleted',
promptSuffix: 'requires moderator/admin',
promptSuffix: 'owner/moderator/admin',
}
const hideLabels: SkillActionLabels = {
verb: 'Hide',
progress: 'Hiding',
past: 'Hidden',
promptSuffix: 'requires moderator/admin',
promptSuffix: 'owner/moderator/admin',
}
const unhideLabels: SkillActionLabels = {
verb: 'Unhide',
progress: 'Unhiding',
past: 'Unhidden',
promptSuffix: 'requires moderator/admin',
promptSuffix: 'owner/moderator/admin',
}
export async function cmdDeleteSkill(
@@ -6,9 +6,15 @@ import type { GlobalOpts } from '../types'
const mockApiRequest = vi.fn()
const mockFetchText = vi.fn()
const mockRegistryUrl = vi.fn((path: string, registry: string) => {
const base = registry.endsWith('/') ? registry : `${registry}/`
const relative = path.startsWith('/') ? path.slice(1) : path
return new URL(relative, base)
})
vi.mock('../../http.js', () => ({
apiRequest: (...args: unknown[]) => mockApiRequest(...args),
fetchText: (...args: unknown[]) => mockFetchText(...args),
registryUrl: (...args: [string, string]) => mockRegistryUrl(...args),
}))
const mockGetRegistry = vi.fn(async () => 'https://clawhub.ai')
@@ -120,6 +126,45 @@ describe('cmdInspect', () => {
expect(url.searchParams.get('version')).toBeNull()
})
it('prints security summary when version security metadata exists', async () => {
mockApiRequest
.mockResolvedValueOnce({
skill: {
slug: 'demo',
displayName: 'Demo',
summary: null,
tags: { latest: '2.0.0' },
stats: {},
createdAt: 1,
updatedAt: 2,
},
latestVersion: { version: '2.0.0', createdAt: 3, changelog: 'init' },
owner: null,
})
.mockResolvedValueOnce({
skill: { slug: 'demo', displayName: 'Demo' },
version: {
version: '2.0.0',
createdAt: 3,
changelog: 'init',
files: [],
security: {
status: 'suspicious',
hasWarnings: true,
checkedAt: 1_700_000_000_000,
model: 'gpt-5.2',
},
},
})
await cmdInspect(makeOpts(), 'demo', { version: '2.0.0' })
expect(mockLog).toHaveBeenCalledWith('Security: SUSPICIOUS')
expect(mockLog).toHaveBeenCalledWith('Warnings: yes')
expect(mockLog).toHaveBeenCalledWith('Checked: 2023-11-14T22:13:20.000Z')
expect(mockLog).toHaveBeenCalledWith('Model: gpt-5.2')
})
it('rejects when both version and tag are provided', async () => {
await expect(
cmdInspect(makeOpts(), 'demo', { version: '1.0.0', tag: 'latest' }),
+55 -3
View File
@@ -1,4 +1,4 @@
import { apiRequest, fetchText } from '../../http.js'
import { apiRequest, fetchText, registryUrl } from '../../http.js'
import {
ApiRoutes,
ApiV1SkillResponseSchema,
@@ -27,6 +27,13 @@ type FileEntry = {
contentType: string | null
}
type SecurityStatus = {
status: 'clean' | 'suspicious' | 'malicious' | 'pending' | 'error'
hasWarnings: boolean
checkedAt: number | null
model: string | null
}
export async function cmdInspect(opts: GlobalOpts, slug: string, options: InspectOptions = {}) {
const trimmed = slug.trim()
if (!trimmed) fail('Slug required')
@@ -78,7 +85,7 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
let versionsList: { items?: unknown[]; nextCursor?: string | null } | null = null
if (options.versions) {
const limit = clampLimit(options.limit ?? 25, 25)
const url = new URL(`${ApiRoutes.skills}/${encodeURIComponent(trimmed)}/versions`, registry)
const url = registryUrl(`${ApiRoutes.skills}/${encodeURIComponent(trimmed)}/versions`, registry)
url.searchParams.set('limit', String(limit))
spinner.text = `Fetching versions (${limit})`
versionsList = await apiRequest(
@@ -90,7 +97,7 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
let fileContent: string | null = null
if (options.file) {
const url = new URL(`${ApiRoutes.skills}/${encodeURIComponent(trimmed)}/file`, registry)
const url = registryUrl(`${ApiRoutes.skills}/${encodeURIComponent(trimmed)}/file`, registry)
url.searchParams.set('path', options.file)
if (options.version) {
url.searchParams.set('version', options.version)
@@ -130,6 +137,7 @@ export async function cmdInspect(opts: GlobalOpts, slug: string, options: Inspec
if (shouldPrintMeta && versionResult?.version) {
printVersionSummary(versionResult.version)
printSecuritySummary(versionResult.version)
}
if (versionsList?.items && Array.isArray(versionsList.items)) {
@@ -258,6 +266,50 @@ function formatVersionLine(item: unknown) {
return `${version} ${createdAt}${snippet}`
}
function printSecuritySummary(version: unknown) {
if (!version || typeof version !== 'object') return
const sec = normalizeSecurity((version as { security?: unknown }).security)
if (!sec) return
console.log(`Security: ${sec.status.toUpperCase()}`)
if (sec.hasWarnings) {
console.log('Warnings: yes')
}
if (typeof sec.checkedAt === 'number') {
console.log(`Checked: ${formatTimestamp(sec.checkedAt)}`)
}
if (sec.model) {
console.log(`Model: ${sec.model}`)
}
}
function normalizeSecurity(security: unknown): SecurityStatus | null {
if (!security || typeof security !== 'object') return null
const value = security as {
status?: unknown
hasWarnings?: unknown
checkedAt?: unknown
model?: unknown
}
if (
value.status !== 'clean' &&
value.status !== 'suspicious' &&
value.status !== 'malicious' &&
value.status !== 'pending' &&
value.status !== 'error'
) {
return null
}
if (typeof value.hasWarnings !== 'boolean') return null
const checkedAt = typeof value.checkedAt === 'number' ? value.checkedAt : null
const model = typeof value.model === 'string' ? value.model : null
return {
status: value.status,
hasWarnings: value.hasWarnings,
checkedAt,
model,
}
}
function formatFileLine(file: FileEntry) {
const size = file.size === null ? '?' : formatBytes(file.size)
const sha = file.sha256 ?? '?'
@@ -12,9 +12,15 @@ vi.mock('../registry.js', () => ({
}))
const mockApiRequest = vi.fn()
const mockRegistryUrl = vi.fn((path: string, registry: string) => {
const base = registry.endsWith('/') ? registry : `${registry}/`
const relative = path.startsWith('/') ? path.slice(1) : path
return new URL(relative, base)
})
vi.mock('../../http.js', () => ({
apiRequest: (registry: unknown, args: unknown, schema?: unknown) =>
mockApiRequest(registry, args, schema),
registryUrl: (...args: [string, string]) => mockRegistryUrl(...args),
}))
vi.mock('../ui.js', () => ({
@@ -116,7 +122,7 @@ describe('cmdBanUser', () => {
expect.anything(),
expect.objectContaining({
method: 'GET',
path: expect.stringContaining('/api/v1/users?'),
url: expect.stringContaining('/api/v1/users?'),
}),
expect.anything(),
)
@@ -1,5 +1,5 @@
import { isCancel, select } from '@clack/prompts'
import { apiRequest } from '../../http.js'
import { apiRequest, registryUrl } from '../../http.js'
import {
ApiRoutes,
ApiV1BanUserResponseSchema,
@@ -192,12 +192,12 @@ async function resolveUserIdentifier(
}
async function searchUsers(registry: string, token: string, query: string) {
const url = new URL(ApiRoutes.users, registry)
const url = registryUrl(ApiRoutes.users, registry)
url.searchParams.set('q', query.trim())
url.searchParams.set('limit', '10')
const result = await apiRequest(
registry,
{ method: 'GET', path: `${url.pathname}?${url.searchParams.toString()}`, token },
{ method: 'GET', url: url.toString(), token },
ApiV1UserSearchResponseSchema,
)
return parseArk(ApiV1UserSearchResponseSchema, result, 'User search response')
@@ -6,9 +6,15 @@ import type { GlobalOpts } from '../types'
const mockApiRequest = vi.fn()
const mockDownloadZip = vi.fn()
const mockRegistryUrl = vi.fn((path: string, registry: string) => {
const base = registry.endsWith('/') ? registry : `${registry}/`
const relative = path.startsWith('/') ? path.slice(1) : path
return new URL(relative, base)
})
vi.mock('../../http.js', () => ({
apiRequest: (...args: unknown[]) => mockApiRequest(...args),
downloadZip: (...args: unknown[]) => mockDownloadZip(...args),
registryUrl: (...args: [string, string]) => mockRegistryUrl(...args),
}))
const mockGetRegistry = vi.fn(async () => 'https://clawhub.ai')
@@ -57,7 +63,8 @@ vi.mock('node:fs/promises', () => ({
stat: vi.fn(),
}))
const { clampLimit, cmdExplore, cmdInstall, cmdUninstall, cmdUpdate, formatExploreLine } = await import('./skills')
const { clampLimit, cmdExplore, cmdInstall, cmdSearch, cmdUninstall, cmdUpdate, formatExploreLine } =
await import('./skills')
const {
extractZipToDir,
hashSkillFiles,
@@ -115,6 +122,16 @@ describe('explore helpers', () => {
})
describe('cmdExplore', () => {
it('passes optional auth token to apiRequest', async () => {
mockGetOptionalAuthToken.mockResolvedValue('tkn')
mockApiRequest.mockResolvedValue({ items: [] })
await cmdExplore(makeOpts(), { limit: 25 })
const [, requestArgs] = mockApiRequest.mock.calls[0] ?? []
expect(requestArgs?.token).toBe('tkn')
})
it('clamps limit and handles empty results', async () => {
mockApiRequest.mockResolvedValue({ items: [] })
@@ -172,6 +189,18 @@ describe('cmdExplore', () => {
})
})
describe('cmdSearch', () => {
it('passes optional auth token to apiRequest', async () => {
mockGetOptionalAuthToken.mockResolvedValue('tkn')
mockApiRequest.mockResolvedValue({ results: [] })
await cmdSearch(makeOpts(), 'demo')
const [, requestArgs] = mockApiRequest.mock.calls[0] ?? []
expect(requestArgs?.token).toBe('tkn')
})
})
describe('cmdUpdate', () => {
it('uses path-based skill lookup when no local fingerprint is available', async () => {
mockApiRequest.mockResolvedValue({ latestVersion: { version: '1.0.0' } })
+8 -6
View File
@@ -1,7 +1,7 @@
import { mkdir, rm, stat } from 'node:fs/promises'
import { join } from 'node:path'
import semver from 'semver'
import { apiRequest, downloadZip } from '../../http.js'
import { apiRequest, downloadZip, registryUrl } from '../../http.js'
import {
ApiRoutes,
ApiV1SearchResponseSchema,
@@ -40,17 +40,18 @@ function isSafeSkillSlug(slug: string) {
export async function cmdSearch(opts: GlobalOpts, query: string, limit?: number) {
if (!query) fail('Query required')
const token = await getOptionalAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner('Searching')
try {
const url = new URL(ApiRoutes.search, registry)
const url = registryUrl(ApiRoutes.search, registry)
url.searchParams.set('q', query)
if (typeof limit === 'number' && Number.isFinite(limit)) {
url.searchParams.set('limit', String(limit))
}
const result = await apiRequest(
registry,
{ method: 'GET', url: url.toString() },
{ method: 'GET', url: url.toString(), token },
ApiV1SearchResponseSchema,
)
@@ -360,17 +361,18 @@ export async function cmdExplore(
opts: GlobalOpts,
options: { limit?: number; sort?: string; json?: boolean } = {},
) {
const token = await getOptionalAuthToken()
const registry = await getRegistry(opts, { cache: true })
const spinner = createSpinner('Fetching latest skills')
try {
const url = new URL(ApiRoutes.skills, registry)
const url = registryUrl(ApiRoutes.skills, registry)
const boundedLimit = clampLimit(options.limit ?? 25)
const { apiSort } = resolveExploreSort(options.sort)
url.searchParams.set('limit', String(boundedLimit))
if (apiSort !== 'updated') url.searchParams.set('sort', apiSort)
const result = await apiRequest(
registry,
{ method: 'GET', url: url.toString() },
{ method: 'GET', url: url.toString(), token },
ApiV1SkillListResponseSchema,
)
@@ -465,7 +467,7 @@ function resolveExploreSort(raw?: string): { sort: ExploreSort; apiSort: ApiExpl
}
async function resolveSkillVersion(registry: string, slug: string, hash: string, token?: string) {
const url = new URL(ApiRoutes.resolve, registry)
const url = registryUrl(ApiRoutes.resolve, registry)
url.searchParams.set('slug', slug)
url.searchParams.set('hash', hash)
return apiRequest(
+56
View File
@@ -0,0 +1,56 @@
/* @vitest-environment node */
import { describe, expect, it, vi } from 'vitest'
const mockSpawn = vi.fn()
vi.mock('node:child_process', () => ({
spawn: (...args: unknown[]) => mockSpawn(...args),
}))
const { openInBrowser } = await import('./ui')
type ErrorHandler = (error: NodeJS.ErrnoException) => void
function createMockChild() {
let onError: ErrorHandler | null = null
const child = {
on: vi.fn((event: string, handler: ErrorHandler) => {
if (event === 'error') onError = handler
return child
}),
unref: vi.fn(),
emitError: (error: NodeJS.ErrnoException) => onError?.(error),
}
return child
}
describe('openInBrowser', () => {
it('prints manual URL instructions when browser opener is missing', () => {
const child = createMockChild()
mockSpawn.mockReturnValueOnce(child)
const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {})
openInBrowser('https://clawhub.ai')
child.emitError(Object.assign(new Error('not found'), { code: 'ENOENT' }))
expect(logSpy).toHaveBeenCalledWith('Could not open browser automatically.')
expect(logSpy).toHaveBeenCalledWith('Please open this URL manually:')
expect(logSpy).toHaveBeenCalledWith(' https://clawhub.ai')
expect(child.unref).toHaveBeenCalledOnce()
logSpy.mockRestore()
})
it('does not print manual instructions for non-ENOENT errors', () => {
const child = createMockChild()
mockSpawn.mockReturnValueOnce(child)
const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {})
openInBrowser('https://clawhub.ai')
child.emitError(Object.assign(new Error('permission denied'), { code: 'EACCES' }))
expect(logSpy).not.toHaveBeenCalledWith('Could not open browser automatically.')
expect(child.unref).toHaveBeenCalledOnce()
logSpy.mockRestore()
})
})
+13
View File
@@ -52,7 +52,20 @@ export function openInBrowser(url: string) {
: ['xdg-open', url]
const [command, ...commandArgs] = args
if (!command) return
const child = spawn(command, commandArgs, { stdio: 'ignore', detached: true })
child.on('error', (err) => {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
console.log('')
console.log('Could not open browser automatically.')
console.log('Please open this URL manually:')
console.log('')
console.log(` ${url}`)
console.log('')
}
})
child.unref()
}
+123 -3
View File
@@ -1,7 +1,14 @@
/* @vitest-environment node */
import { describe, expect, it, vi } from 'vitest'
import { apiRequest, apiRequestForm, downloadZip, fetchText } from './http'
import {
apiRequest,
apiRequestForm,
downloadZip,
fetchText,
registryUrl,
shouldUseProxyFromEnv,
} from './http'
import { ApiV1WhoamiResponseSchema } from './schema/index.js'
function mockImmediateTimeouts() {
@@ -36,6 +43,71 @@ function createAbortingFetchMock() {
})
}
describe('shouldUseProxyFromEnv', () => {
it('detects standard proxy variables', () => {
expect(
shouldUseProxyFromEnv({
HTTPS_PROXY: 'http://proxy.example:3128',
} as NodeJS.ProcessEnv),
).toBe(true)
expect(
shouldUseProxyFromEnv({
HTTP_PROXY: 'http://proxy.example:3128',
} as NodeJS.ProcessEnv),
).toBe(true)
expect(
shouldUseProxyFromEnv({
https_proxy: 'http://proxy.example:3128',
} as NodeJS.ProcessEnv),
).toBe(true)
})
it('ignores NO_PROXY-only configs', () => {
expect(
shouldUseProxyFromEnv({
NO_PROXY: 'localhost,127.0.0.1',
} as NodeJS.ProcessEnv),
).toBe(false)
expect(shouldUseProxyFromEnv({} as NodeJS.ProcessEnv)).toBe(false)
})
})
describe('registryUrl', () => {
it('works with a plain-origin registry (no base path)', () => {
expect(registryUrl('/api/v1/skills', 'https://clawhub.ai').toString()).toBe(
'https://clawhub.ai/api/v1/skills',
)
})
it('preserves the registry base path', () => {
const base = 'http://localhost:8081/custom/registry/path'
expect(registryUrl('/api/v1/skills', base).toString()).toBe(
'http://localhost:8081/custom/registry/path/api/v1/skills',
)
})
it('handles a trailing slash on the registry', () => {
const base = 'http://localhost:8081/custom/registry/path/'
expect(registryUrl('/api/v1/skills', base).toString()).toBe(
'http://localhost:8081/custom/registry/path/api/v1/skills',
)
})
it('handles paths without a leading slash', () => {
expect(registryUrl('api/v1/skills', 'https://clawhub.ai').toString()).toBe(
'https://clawhub.ai/api/v1/skills',
)
})
it('handles compound paths with encoded segments', () => {
const base = 'http://localhost:8081/base'
const path = `/api/v1/skills/${encodeURIComponent('my-skill')}/versions`
expect(registryUrl(path, base).toString()).toBe(
'http://localhost:8081/base/api/v1/skills/my-skill/versions',
)
})
})
describe('apiRequest', () => {
it('adds bearer token and parses json', async () => {
const fetchMock = vi.fn().mockResolvedValue({
@@ -131,6 +203,7 @@ describe('apiRequest', () => {
})
it('falls back to HTTP status when body is empty', async () => {
mockImmediateTimeouts()
const fetchMock = vi.fn().mockResolvedValue({
ok: false,
status: 500,
@@ -140,6 +213,7 @@ describe('apiRequest', () => {
await expect(
apiRequest('https://example.com', { method: 'GET', url: 'https://example.com/x' }),
).rejects.toThrow('HTTP 500')
expect(fetchMock).toHaveBeenCalledTimes(3)
vi.unstubAllGlobals()
})
@@ -187,7 +261,7 @@ describe('apiRequest', () => {
}
expect(caught).toBeInstanceOf(Error)
expect((caught as Error).message).toBe('Timeout')
expect((caught as Error).message).toMatch(/timed out/)
expect(fetchMock).toHaveBeenCalledTimes(3)
expect(clearTimeoutMock.mock.calls.length).toBeGreaterThanOrEqual(3)
vi.unstubAllGlobals()
@@ -253,6 +327,31 @@ describe('apiRequestForm', () => {
expect(fetchMock).toHaveBeenCalledTimes(1)
vi.unstubAllGlobals()
})
it('uses the longer upload timeout for multipart requests', async () => {
const { setTimeoutMock, clearTimeoutMock } = mockImmediateTimeouts()
const fetchMock = createAbortingFetchMock()
vi.stubGlobal('fetch', fetchMock)
let caught: unknown
try {
await apiRequestForm('https://example.com', {
method: 'POST',
path: '/upload',
form: new FormData(),
})
} catch (error) {
caught = error
}
expect(caught).toBeInstanceOf(Error)
expect((caught as Error).message).toMatch(/timed out after 120s/i)
expect(setTimeoutMock).toHaveBeenCalled()
expect(setTimeoutMock.mock.calls[0]?.[1]).toBe(120_000)
expect(fetchMock).toHaveBeenCalledTimes(3)
expect(clearTimeoutMock.mock.calls.length).toBeGreaterThanOrEqual(3)
vi.unstubAllGlobals()
})
})
describe('fetchText', () => {
@@ -269,9 +368,30 @@ describe('fetchText', () => {
}
expect(caught).toBeInstanceOf(Error)
expect((caught as Error).message).toBe('Timeout')
expect((caught as Error).message).toMatch(/timed out/)
expect(fetchMock).toHaveBeenCalledTimes(3)
expect(clearTimeoutMock.mock.calls.length).toBeGreaterThanOrEqual(3)
vi.unstubAllGlobals()
})
})
describe('fetchWithTimeout — non-Error normalization', () => {
it('wraps DOMException-like non-Error throws into proper Error instances', async () => {
const fetchMock = vi.fn(async () => {
// Simulate a runtime that throws a non-Error object on abort
throw { message: 'The operation was aborted', name: 'AbortError' }
})
vi.stubGlobal('fetch', fetchMock)
let caught: unknown
try {
await apiRequest('https://example.com', { method: 'GET', path: '/x' })
} catch (error) {
caught = error
}
expect(caught).toBeInstanceOf(Error)
expect((caught as Error).message).toContain('The operation was aborted')
vi.unstubAllGlobals()
})
})
+39 -12
View File
@@ -3,12 +3,14 @@ import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import pRetry, { AbortError } from 'p-retry'
import { Agent, setGlobalDispatcher } from 'undici'
import { Agent, EnvHttpProxyAgent, setGlobalDispatcher } from 'undici'
import type { ArkValidator } from './schema/index.js'
import { ApiRoutes, parseArk } from './schema/index.js'
const REQUEST_TIMEOUT_MS = 15_000
const UPLOAD_TIMEOUT_MS = 120_000
const REQUEST_TIMEOUT_SECONDS = Math.ceil(REQUEST_TIMEOUT_MS / 1000)
const UPLOAD_TIMEOUT_SECONDS = Math.ceil(UPLOAD_TIMEOUT_MS / 1000)
const RETRY_COUNT = 2
const RETRY_BACKOFF_BASE_MS = 300
const RETRY_BACKOFF_MAX_MS = 5_000
@@ -28,18 +30,32 @@ const CURL_WRITE_OUT_FORMAT = [
].join('\n')
const isBun = typeof process !== 'undefined' && Boolean(process.versions?.bun)
export function shouldUseProxyFromEnv(env: NodeJS.ProcessEnv = process.env): boolean {
return Boolean(env.HTTPS_PROXY || env.HTTP_PROXY || env.https_proxy || env.http_proxy)
}
if (typeof process !== 'undefined' && process.versions?.node) {
try {
setGlobalDispatcher(
new Agent({
connect: { timeout: REQUEST_TIMEOUT_MS },
}),
shouldUseProxyFromEnv(process.env)
? new EnvHttpProxyAgent({
connect: { timeout: REQUEST_TIMEOUT_MS },
})
: new Agent({
connect: { timeout: REQUEST_TIMEOUT_MS },
}),
)
} catch {
// ignore dispatcher setup failures in non-node runtimes
}
}
export function registryUrl(path: string, registry: string): URL {
const base = registry.endsWith('/') ? registry : `${registry}/`
const relative = path.startsWith('/') ? path.slice(1) : path
return new URL(relative, base)
}
type RequestArgs =
| { method: 'GET' | 'POST' | 'DELETE'; path: string; token?: string; body?: unknown }
| { method: 'GET' | 'POST' | 'DELETE'; url: string; token?: string; body?: unknown }
@@ -76,7 +92,7 @@ export async function apiRequest<T>(
args: RequestArgs,
schema?: ArkValidator<T>,
): Promise<T> {
const url = 'url' in args ? args.url : new URL(args.path, registry).toString()
const url = 'url' in args ? args.url : registryUrl(args.path, registry).toString()
const json = await runWithRetries(
async () => {
if (isBun) {
@@ -120,7 +136,7 @@ export async function apiRequestForm<T>(
args: FormRequestArgs,
schema?: ArkValidator<T>,
): Promise<T> {
const url = 'url' in args ? args.url : new URL(args.path, registry).toString()
const url = 'url' in args ? args.url : registryUrl(args.path, registry).toString()
const json = await runWithRetries(
async () => {
if (isBun) {
@@ -133,7 +149,7 @@ export async function apiRequestForm<T>(
method: args.method,
headers,
body: args.form,
})
}, UPLOAD_TIMEOUT_MS)
if (!response.ok) {
throwHttpStatusError(response.status, await readResponseTextSafe(response), response.headers)
}
@@ -147,7 +163,7 @@ export async function apiRequestForm<T>(
type TextRequestArgs = { path: string; token?: string } | { url: string; token?: string }
export async function fetchText(registry: string, args: TextRequestArgs): Promise<string> {
const url = 'url' in args ? args.url : new URL(args.path, registry).toString()
const url = 'url' in args ? args.url : registryUrl(args.path, registry).toString()
return runWithRetries(
async () => {
if (isBun) {
@@ -170,7 +186,7 @@ export async function downloadZip(
registry: string,
args: { slug: string; version?: string; token?: string },
) {
const url = new URL(ApiRoutes.download, registry)
const url = registryUrl(ApiRoutes.download, registry)
url.searchParams.set('slug', args.slug)
if (args.version) url.searchParams.set('version', args.version)
return runWithRetries(
@@ -191,11 +207,22 @@ export async function downloadZip(
)
}
async function fetchWithTimeout(url: string, init: RequestInit): Promise<Response> {
async function fetchWithTimeout(url: string, init: RequestInit, timeoutMs = REQUEST_TIMEOUT_MS): Promise<Response> {
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(new Error('Timeout')), REQUEST_TIMEOUT_MS)
const timeoutSeconds = Math.ceil(timeoutMs / 1000)
const timeout = setTimeout(
() => controller.abort(new Error(`Request timed out after ${timeoutSeconds}s`)),
timeoutMs,
)
try {
return await fetch(url, { ...init, signal: controller.signal })
} catch (error) {
if (error instanceof Error) throw error
// Normalize non-Error throws (e.g. DOMException from AbortController) into proper Errors
const message = typeof error === 'object' && error !== null && 'message' in error
? String((error as { message: unknown }).message)
: String(error)
throw new Error(message, { cause: error })
} finally {
clearTimeout(timeout)
}
@@ -411,7 +438,7 @@ async function fetchJsonFormViaCurl(url: string, args: FormRequestArgs) {
'--show-error',
'--location',
'--max-time',
String(REQUEST_TIMEOUT_SECONDS),
String(UPLOAD_TIMEOUT_SECONDS),
'--write-out',
CURL_WRITE_OUT_FORMAT,
'-X',
+36
View File
@@ -208,6 +208,13 @@ export const ApiV1SkillVersionListResponseSchema = type({
nextCursor: 'string|null',
})
export const SecurityStatusSchema = type({
status: '"clean" | "suspicious" | "malicious" | "pending" | "error"',
hasWarnings: 'boolean',
checkedAt: 'number|null',
model: 'string|null',
})
export const ApiV1SkillVersionResponseSchema = type({
version: type({
version: 'string',
@@ -215,6 +222,7 @@ export const ApiV1SkillVersionResponseSchema = type({
changelog: 'string',
changelogSource: '"auto"|"user"|null?',
files: 'unknown?',
security: SecurityStatusSchema.optional(),
}).or('null'),
skill: type({
slug: 'string',
@@ -287,6 +295,30 @@ export const ClawdisRequiresSchema = type({
})
export type ClawdisRequires = (typeof ClawdisRequiresSchema)[inferred]
export const EnvVarDeclarationSchema = type({
name: 'string',
required: 'boolean?',
description: 'string?',
})
export type EnvVarDeclaration = (typeof EnvVarDeclarationSchema)[inferred]
export const DependencyDeclarationSchema = type({
name: 'string',
type: '"pip"|"npm"|"brew"|"go"|"cargo"|"apt"|"other"',
version: 'string?',
url: 'string?',
repository: 'string?',
})
export type DependencyDeclaration = (typeof DependencyDeclarationSchema)[inferred]
export const SkillLinksSchema = type({
homepage: 'string?',
repository: 'string?',
documentation: 'string?',
changelog: 'string?',
})
export type SkillLinks = (typeof SkillLinksSchema)[inferred]
export const ClawdisSkillMetadataSchema = type({
always: 'boolean?',
skillKey: 'string?',
@@ -299,5 +331,9 @@ export const ClawdisSkillMetadataSchema = type({
install: SkillInstallSpecSchema.array().optional(),
nix: NixPluginSpecSchema.optional(),
config: ClawdbotConfigSpecSchema.optional(),
envVars: EnvVarDeclarationSchema.array().optional(),
dependencies: DependencyDeclarationSchema.array().optional(),
author: 'string?',
links: SkillLinksSchema.optional(),
})
export type ClawdisSkillMetadata = (typeof ClawdisSkillMetadataSchema)[inferred]
+129
View File
@@ -0,0 +1,129 @@
import { fireEvent, render, screen, waitFor } from '@testing-library/react'
import { vi } from 'vitest'
import { ImportGitHub } from '../routes/import'
vi.mock('@tanstack/react-router', () => ({
createFileRoute: () => (config: { component: unknown }) => config,
useNavigate: () => vi.fn(),
}))
const previewImport = vi.fn()
const previewCandidate = vi.fn()
const importSkill = vi.fn()
const useQueryMock = vi.fn()
const useAuthStatusMock = vi.fn()
let useActionCallCount = 0
vi.mock('convex/react', () => ({
useQuery: (...args: unknown[]) => useQueryMock(...args),
useAction: () => {
const action = [previewImport, previewCandidate, importSkill][useActionCallCount % 3]
useActionCallCount += 1
return action
},
}))
vi.mock('../lib/useAuthStatus', () => ({
useAuthStatus: () => useAuthStatusMock(),
}))
describe('Import route', () => {
beforeEach(() => {
previewImport.mockReset()
previewCandidate.mockReset()
importSkill.mockReset()
useQueryMock.mockReset()
useAuthStatusMock.mockReset()
useActionCallCount = 0
useAuthStatusMock.mockReturnValue({
isAuthenticated: true,
isLoading: false,
me: { _id: 'users:1', handle: 'me' },
})
useQueryMock.mockImplementation((_fn: unknown, args: unknown) => {
if (args === 'skip') return undefined
return null
})
previewImport.mockResolvedValue({
candidates: [
{
path: 'skill',
readmePath: 'skill/SKILL.md',
name: 'Taken Skill',
description: null,
},
],
})
previewCandidate.mockResolvedValue({
resolved: {
owner: 'octo',
repo: 'repo',
ref: 'main',
commit: 'abcdef1234567890',
path: 'skill',
repoUrl: 'https://github.com/octo/repo',
originalUrl: 'https://github.com/octo/repo',
},
candidate: {
path: 'skill',
readmePath: 'skill/SKILL.md',
name: 'Taken Skill',
description: null,
},
defaults: {
selectedPaths: ['skill/SKILL.md'],
slug: 'taken-skill',
displayName: 'Taken Skill',
version: '1.0.0',
tags: ['latest'],
},
files: [
{
path: 'skill/SKILL.md',
size: 120,
defaultSelected: true,
},
],
})
})
it('blocks import preflight when slug availability reports a collision', async () => {
useQueryMock.mockImplementation((_fn: unknown, args: unknown) => {
if (args === 'skip') return undefined
if (
args &&
typeof args === 'object' &&
'slug' in (args as Record<string, unknown>) &&
(args as Record<string, unknown>).slug === 'taken-skill'
) {
return {
available: false,
reason: 'taken',
message: 'Slug is already taken. Choose a different slug.',
url: '/alice/taken-skill',
}
}
return null
})
render(<ImportGitHub />)
fireEvent.change(screen.getByPlaceholderText('https://github.com/owner/repo'), {
target: { value: 'https://github.com/octo/repo' },
})
fireEvent.click(screen.getByRole('button', { name: /detect/i }))
await waitFor(() => {
expect(previewImport).toHaveBeenCalled()
expect(previewCandidate).toHaveBeenCalled()
})
expect(await screen.findByText(/Slug is already taken\. Choose a different slug\./i)).toBeTruthy()
expect(screen.getByRole('link', { name: '/alice/taken-skill' })).toBeTruthy()
expect(screen.getByRole('button', { name: /import \+ publish/i }).getAttribute('disabled')).not.toBeNull()
})
})
+70
View File
@@ -198,6 +198,32 @@ describe('Upload route', () => {
expect(screen.getByText('screenshot.png')).toBeTruthy()
})
it('shows an informational note when mac junk files are ignored', async () => {
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'cool-skill' },
})
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'Cool Skill' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.2.3' },
})
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
const skill = new File(['hello'], 'SKILL.md', { type: 'text/markdown' })
const junk = new File(['junk'], '.DS_Store', { type: 'application/octet-stream' })
const input = screen.getByTestId('upload-input') as HTMLInputElement
fireEvent.change(input, { target: { files: [skill, junk] } })
expect(await screen.findByText('SKILL.md')).toBeTruthy()
expect(screen.queryByText('.DS_Store')).toBeNull()
expect(await screen.findByText(/Ignored 1 macOS junk file/i)).toBeTruthy()
expect(await screen.findByText(/All checks passed/i)).toBeTruthy()
})
it('surfaces publish errors and stays on page', async () => {
publishVersion.mockRejectedValueOnce(new Error('Changelog is required'))
generateUploadUrl.mockResolvedValue('https://upload.local')
@@ -225,4 +251,48 @@ describe('Upload route', () => {
fireEvent.click(publishButton)
expect(await screen.findByText(/Changelog is required/i)).toBeTruthy()
})
it('blocks publish in preflight when slug availability reports a collision', async () => {
useQueryMock.mockImplementation((_fn: unknown, args: unknown) => {
if (args === 'skip') return undefined
if (
args &&
typeof args === 'object' &&
'slug' in (args as Record<string, unknown>) &&
(args as Record<string, unknown>).slug === 'taken-skill'
) {
return {
available: false,
reason: 'taken',
message: 'Slug is already taken. Choose a different slug.',
url: '/alice/taken-skill',
}
}
return null
})
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'taken-skill' },
})
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'Taken Skill' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.2.3' },
})
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
fireEvent.change(screen.getByPlaceholderText('Describe what changed in this skill...'), {
target: { value: 'Initial drop.' },
})
const file = new File(['hello'], 'SKILL.md', { type: 'text/markdown' })
const input = screen.getByTestId('upload-input') as HTMLInputElement
fireEvent.change(input, { target: { files: [file] } })
expect(await screen.findByText(/Slug is already taken\. Choose a different slug\./i)).toBeTruthy()
expect(screen.getByRole('link', { name: '/alice/taken-skill' })).toBeTruthy()
expect(screen.getByRole('button', { name: /publish skill/i }).getAttribute('disabled')).not.toBeNull()
})
})
+9 -1
View File
@@ -12,7 +12,15 @@ export function Footer() {
<a href="https://openclaw.ai" target="_blank" rel="noreferrer">
OpenClaw
</a>{' '}
project ·{' '}
project · Deployed on{' '}
<a href="https://vercel.com" target="_blank" rel="noreferrer">
Vercel
</a>{' '}
· Powered by{' '}
<a href="https://www.convex.dev" target="_blank" rel="noreferrer">
Convex
</a>{' '}
·{' '}
<a href="https://github.com/openclaw/clawhub" target="_blank" rel="noreferrer">
Open source (MIT)
</a>{' '}
+7 -2
View File
@@ -6,19 +6,21 @@ type SkillCardProps = {
skill: PublicSkill
badge?: string | string[]
chip?: string
platformLabels?: string[]
summaryFallback: string
meta: ReactNode
href?: string
}
export function SkillCard({ skill, badge, chip, summaryFallback, meta, href }: SkillCardProps) {
export function SkillCard({ skill, badge, chip, platformLabels, summaryFallback, meta, href }: SkillCardProps) {
const owner = encodeURIComponent(String(skill.ownerUserId))
const link = href ?? `/${owner}/${skill.slug}`
const badges = Array.isArray(badge) ? badge : badge ? [badge] : []
const hasTags = badges.length || chip || platformLabels?.length
return (
<Link to={link} className="card skill-card">
{badges.length || chip ? (
{hasTags ? (
<div className="skill-card-tags">
{badges.map((label) => (
<div key={label} className="tag">
@@ -26,6 +28,9 @@ export function SkillCard({ skill, badge, chip, summaryFallback, meta, href }: S
</div>
))}
{chip ? <div className="tag tag-accent tag-compact">{chip}</div> : null}
{platformLabels?.map((label) => (
<div key={label} className="tag tag-compact">{label}</div>
))}
</div>
) : null}
<h3 className="skill-card-title">{skill.displayName}</h3>
+120 -9
View File
@@ -10,14 +10,33 @@ type SkillCommentsPanelProps = {
me: Doc<'users'> | null
}
function formatReportError(error: unknown) {
if (error instanceof Error) {
const cleaned = error.message
.replace(/\[CONVEX[^\]]*\]\s*/g, '')
.replace(/\[Request ID:[^\]]*\]\s*/g, '')
.replace(/^Server Error Called by client\s*/i, '')
.replace(/^ConvexError:\s*/i, '')
.trim()
if (cleaned && cleaned !== 'Server Error') return cleaned
}
return 'Failed to report comment'
}
export function SkillCommentsPanel({ skillId, isAuthenticated, me }: SkillCommentsPanelProps) {
const addComment = useMutation(api.comments.add)
const removeComment = useMutation(api.comments.remove)
const reportComment = useMutation(api.comments.report)
const [comment, setComment] = useState('')
const [isSubmitting, setIsSubmitting] = useState(false)
const [submitError, setSubmitError] = useState<string | null>(null)
const [deleteError, setDeleteError] = useState<string | null>(null)
const [deletingCommentId, setDeletingCommentId] = useState<Id<'comments'> | null>(null)
const [reportingCommentId, setReportingCommentId] = useState<Id<'comments'> | null>(null)
const [reportReason, setReportReason] = useState('')
const [reportError, setReportError] = useState<string | null>(null)
const [reportNotice, setReportNotice] = useState<string | null>(null)
const [isSubmittingReport, setIsSubmittingReport] = useState(false)
const comments = useQuery(api.comments.listBySkill, { skillId, limit: 50 })
const submitComment = async () => {
@@ -48,6 +67,44 @@ export function SkillCommentsPanel({ skillId, isAuthenticated, me }: SkillCommen
}
}
const openReportForm = (commentId: Id<'comments'>) => {
setReportingCommentId(commentId)
setReportReason('')
setReportError(null)
setReportNotice(null)
setIsSubmittingReport(false)
}
const closeReportForm = () => {
setReportingCommentId(null)
setReportReason('')
setReportError(null)
setIsSubmittingReport(false)
}
const submitReport = async (commentId: Id<'comments'>) => {
if (isSubmittingReport) return
const reason = reportReason.trim()
if (!reason) {
setReportError('Report reason required.')
return
}
setIsSubmittingReport(true)
setReportError(null)
setReportNotice(null)
try {
const result = await reportComment({ commentId, reason })
setReportNotice(
result.alreadyReported ? 'You already reported this comment.' : 'Report submitted.',
)
closeReportForm()
} catch (error) {
setReportError(formatReportError(error))
setIsSubmittingReport(false)
}
}
return (
<div className="card">
<h2 className="section-title" style={{ fontSize: '1.2rem', margin: 0 }}>
@@ -78,6 +135,7 @@ export function SkillCommentsPanel({ skillId, isAuthenticated, me }: SkillCommen
<p className="section-subtitle">Sign in to comment.</p>
)}
{deleteError ? <div className="report-dialog-error">{deleteError}</div> : null}
{reportNotice ? <div className="stat">{reportNotice}</div> : null}
<div style={{ display: 'grid', gap: 12, marginTop: 16 }}>
{(comments ?? []).length === 0 ? (
<div className="stat">No comments yet.</div>
@@ -87,16 +145,69 @@ export function SkillCommentsPanel({ skillId, isAuthenticated, me }: SkillCommen
<div className="comment-body">
<strong>@{entry.user?.handle ?? entry.user?.name ?? 'user'}</strong>
<div className="comment-body-text">{entry.comment.body}</div>
{isAuthenticated && reportingCommentId === entry.comment._id ? (
<form
className="comment-report-form"
onSubmit={(event) => {
event.preventDefault()
void submitReport(entry.comment._id)
}}
>
<textarea
className="comment-input comment-report-input"
rows={3}
value={reportReason}
onChange={(event) => setReportReason(event.target.value)}
placeholder="Why are you reporting this comment?"
disabled={isSubmittingReport}
/>
<div className="comment-report-actions">
<button
className="btn comment-delete"
type="button"
onClick={closeReportForm}
disabled={isSubmittingReport}
>
Cancel
</button>
<button className="btn comment-submit" type="submit" disabled={isSubmittingReport}>
{isSubmittingReport ? 'Reporting…' : 'Submit report'}
</button>
</div>
{reportError ? <div className="report-dialog-error">{reportError}</div> : null}
<div className="stat">
Reports require a reason. Abuse of reporting may result in bans.
</div>
</form>
) : null}
</div>
{isAuthenticated && me && (me._id === entry.comment.userId || isModerator(me)) ? (
<button
className="btn comment-delete"
type="button"
onClick={() => void deleteComment(entry.comment._id)}
disabled={Boolean(deletingCommentId) || isSubmitting}
>
{deletingCommentId === entry.comment._id ? 'Deleting…' : 'Delete'}
</button>
{isAuthenticated && me ? (
<div className="comment-actions">
{me._id === entry.comment.userId || isModerator(me) ? (
<button
className="btn comment-delete"
type="button"
onClick={() => void deleteComment(entry.comment._id)}
disabled={Boolean(deletingCommentId) || isSubmitting || isSubmittingReport}
>
{deletingCommentId === entry.comment._id ? 'Deleting…' : 'Delete'}
</button>
) : null}
{me._id !== entry.comment.userId ? (
<button
className="btn comment-delete"
type="button"
onClick={() => openReportForm(entry.comment._id)}
disabled={
isSubmitting ||
Boolean(deletingCommentId) ||
(Boolean(reportingCommentId) && reportingCommentId !== entry.comment._id)
}
>
{reportingCommentId === entry.comment._id ? 'Report open' : 'Report'}
</button>
) : null}
</div>
) : null}
</div>
))
+84 -2
View File
@@ -9,6 +9,9 @@ type SkillInstallCardProps = {
export function SkillInstallCard({ clawdis, osLabels }: SkillInstallCardProps) {
const requirements = clawdis?.requires
const installSpecs = clawdis?.install ?? []
const envVars = clawdis?.envVars ?? []
const dependencies = clawdis?.dependencies ?? []
const links = clawdis?.links
const hasRuntimeRequirements = Boolean(
clawdis?.emoji ||
osLabels.length ||
@@ -16,11 +19,14 @@ export function SkillInstallCard({ clawdis, osLabels }: SkillInstallCardProps) {
requirements?.anyBins?.length ||
requirements?.env?.length ||
requirements?.config?.length ||
clawdis?.primaryEnv,
clawdis?.primaryEnv ||
envVars.length,
)
const hasInstallSpecs = installSpecs.length > 0
const hasDependencies = dependencies.length > 0
const hasLinks = Boolean(links?.homepage || links?.repository || links?.documentation)
if (!hasRuntimeRequirements && !hasInstallSpecs) return null
if (!hasRuntimeRequirements && !hasInstallSpecs && !hasDependencies && !hasLinks) return null
return (
<div className="skill-hero-content">
@@ -68,6 +74,55 @@ export function SkillInstallCard({ clawdis, osLabels }: SkillInstallCardProps) {
<span>{clawdis.primaryEnv}</span>
</div>
) : null}
{envVars.length > 0 ? (
<div className="stat">
<strong>Environment variables</strong>
<div style={{ display: 'flex', flexDirection: 'column', gap: '0.25rem', marginTop: '0.25rem' }}>
{envVars.map((env, index) => (
<div key={`${env.name}-${index}`} style={{ display: 'flex', alignItems: 'baseline', gap: '0.5rem' }}>
<code style={{ fontSize: '0.85rem' }}>{env.name}</code>
{env.required === false ? (
<span style={{ color: 'var(--ink-soft)', fontSize: '0.75rem' }}>optional</span>
) : env.required === true ? (
<span style={{ color: 'var(--ink-accent)', fontSize: '0.75rem' }}>required</span>
) : null}
{env.description ? (
<span style={{ color: 'var(--ink-soft)', fontSize: '0.8rem' }}> {env.description}</span>
) : null}
</div>
))}
</div>
</div>
) : null}
</div>
</div>
) : null}
{hasDependencies ? (
<div className="skill-panel">
<h3 className="section-title" style={{ fontSize: '1rem', margin: 0 }}>
Dependencies
</h3>
<div className="skill-panel-body">
{dependencies.map((dep, index) => (
<div key={`${dep.name}-${index}`} className="stat">
<div>
<strong>{dep.name}</strong>
<span style={{ color: 'var(--ink-soft)', fontSize: '0.85rem', marginLeft: '0.5rem' }}>
{dep.type}{dep.version ? ` ${dep.version}` : ''}
</span>
{dep.url ? (
<div style={{ fontSize: '0.8rem' }}>
<a href={dep.url} target="_blank" rel="noopener noreferrer">{dep.url}</a>
</div>
) : null}
{dep.repository && dep.repository !== dep.url ? (
<div style={{ fontSize: '0.8rem' }}>
<a href={dep.repository} target="_blank" rel="noopener noreferrer">Source</a>
</div>
) : null}
</div>
</div>
))}
</div>
</div>
) : null}
@@ -96,6 +151,33 @@ export function SkillInstallCard({ clawdis, osLabels }: SkillInstallCardProps) {
</div>
</div>
) : null}
{hasLinks ? (
<div className="skill-panel">
<h3 className="section-title" style={{ fontSize: '1rem', margin: 0 }}>
Links
</h3>
<div className="skill-panel-body">
{links?.homepage ? (
<div className="stat">
<strong>Homepage</strong>
<a href={links.homepage} target="_blank" rel="noopener noreferrer">{links.homepage}</a>
</div>
) : null}
{links?.repository ? (
<div className="stat">
<strong>Repository</strong>
<a href={links.repository} target="_blank" rel="noopener noreferrer">{links.repository}</a>
</div>
) : null}
{links?.documentation ? (
<div className="stat">
<strong>Docs</strong>
<a href={links.documentation} target="_blank" rel="noopener noreferrer">{links.documentation}</a>
</div>
) : null}
</div>
</div>
) : null}
</div>
</div>
)
+17
View File
@@ -103,6 +103,23 @@ export function formatOsList(os?: string[]) {
})
}
export function formatSystemsList(systems?: string[]): string[] {
if (!systems?.length) return []
const labels: Record<string, string> = {
'aarch64-darwin': 'macOS ARM64',
'x86_64-darwin': 'macOS x86_64',
'aarch64-linux': 'Linux ARM64',
'x86_64-linux': 'Linux x86_64',
}
return systems.map((s) => labels[s.trim()] ?? s)
}
export function getPlatformLabels(os?: string[], systems?: string[]): string[] {
if (systems?.length) return formatSystemsList(systems)
if (os?.length) return formatOsList(os)
return []
}
export function formatInstallLabel(spec: SkillInstallSpec) {
if (spec.kind === 'brew') return 'Homebrew'
if (spec.kind === 'node') return 'Node'
+29
View File
@@ -0,0 +1,29 @@
import { describe, expect, it } from 'vitest'
import { getUserFacingConvexError } from './convexError'
describe('getUserFacingConvexError', () => {
it('falls back when data is generic wrapper text', () => {
expect(
getUserFacingConvexError({ data: 'Server Error Called by client' }, 'Publish failed'),
).toBe('Publish failed')
})
it('unwraps convex wrapper text from Error messages', () => {
expect(
getUserFacingConvexError(
new Error('[CONVEX A] [Request ID: abc] Server Error Called by client ConvexError: Bad input'),
'fallback',
),
).toBe('Bad input')
})
it('preserves ownership errors as-is after cleanup', () => {
expect(
getUserFacingConvexError(new Error('Only the owner can publish soul updates'), 'fallback'),
).toBe('Only the owner can publish soul updates')
})
it('returns fallback for unknown errors', () => {
expect(getUserFacingConvexError('wat', 'Publish failed')).toBe('Publish failed')
})
})
+49
View File
@@ -0,0 +1,49 @@
type ConvexLikeErrorData =
| string
| {
message?: unknown
}
| null
| undefined
type ConvexLikeError = {
data?: ConvexLikeErrorData
message?: unknown
}
function cleanupConvexMessage(message: string) {
return message
.replace(/\[CONVEX[^\]]*\]\s*/g, '')
.replace(/\[Request ID:[^\]]*\]\s*/g, '')
.replace(/^Server Error Called by client\s*/i, '')
.replace(/^ConvexError:\s*/i, '')
.trim()
}
export function getUserFacingConvexError(error: unknown, fallback: string) {
const candidates: string[] = []
const maybe = error as ConvexLikeError
if (maybe && typeof maybe === 'object' && 'data' in maybe) {
if (typeof maybe.data === 'string') candidates.push(maybe.data)
if (maybe.data && typeof maybe.data === 'object' && typeof maybe.data.message === 'string') {
candidates.push(maybe.data.message)
}
}
if (error instanceof Error && typeof error.message === 'string') {
candidates.push(error.message)
} else if (maybe && typeof maybe.message === 'string') {
candidates.push(maybe.message)
}
for (const raw of candidates) {
const cleaned = cleanupConvexMessage(raw)
if (!cleaned) continue
if (/^server error$/i.test(cleaned)) continue
if (/^internal server error$/i.test(cleaned)) continue
return cleaned
}
return fallback
}
+65
View File
@@ -0,0 +1,65 @@
import { describe, expect, it } from 'vitest'
import { getPublicSlugCollision } from './slugCollision'
describe('getPublicSlugCollision', () => {
it('returns null when availability result is missing', () => {
expect(
getPublicSlugCollision({
isSoulMode: false,
slug: 'demo',
result: undefined,
}),
).toBeNull()
})
it('returns null when slug is available', () => {
expect(
getPublicSlugCollision({
isSoulMode: false,
slug: 'demo',
result: {
available: true,
reason: 'available',
message: null,
url: null,
},
}),
).toBeNull()
})
it('returns collision with link when query reports unavailable with URL', () => {
expect(
getPublicSlugCollision({
isSoulMode: false,
slug: 'demo',
result: {
available: false,
reason: 'taken',
message: 'Slug is already taken. Choose a different slug.',
url: '/alice/demo',
},
}),
).toEqual({
message: 'Slug is already taken. Choose a different slug.',
url: '/alice/demo',
})
})
it('returns generic collision message when backend message is empty', () => {
expect(
getPublicSlugCollision({
isSoulMode: false,
slug: 'demo',
result: {
available: false,
reason: 'reserved',
message: ' ',
url: null,
},
}),
).toEqual({
message: 'Slug is already taken. Choose a different slug.',
url: null,
})
})
})
+28
View File
@@ -0,0 +1,28 @@
type SlugAvailabilityResult =
| {
available: boolean
reason: 'available' | 'taken' | 'reserved'
message: string | null
url: string | null
}
| null
export type PublicSlugCollision = {
message: string
url: string | null
}
export function getPublicSlugCollision(params: {
isSoulMode: boolean
slug: string
result: SlugAvailabilityResult | undefined
}): PublicSlugCollision | null {
if (params.isSoulMode) return null
const normalizedSlug = params.slug.trim().toLowerCase()
if (!normalizedSlug) return null
if (!params.result || params.result.available) return null
return {
message: params.result.message?.trim() || 'Slug is already taken. Choose a different slug.',
url: params.result.url ?? null,
}
}
+11 -1
View File
@@ -1,7 +1,7 @@
import { strToU8, unzipSync, zipSync } from 'fflate'
import { describe, expect, it } from 'vitest'
import { expandDroppedItems, expandFiles } from './uploadFiles'
import { expandDroppedItems, expandFiles, expandFilesWithReport } from './uploadFiles'
function readWithFileReader(blob: Blob) {
return new Promise<ArrayBuffer>((resolve, reject) => {
@@ -34,6 +34,16 @@ describe('expandFiles (jsdom)', () => {
const expanded = await expandFiles([zipFile])
expect(expanded.map((file) => file.name)).toEqual(['SKILL.md', 'notes.txt'])
})
it('filters mac junk files and returns ignored paths', async () => {
const report = await expandFilesWithReport([
new File(['hello'], 'SKILL.md', { type: 'text/markdown' }),
new File(['junk'], '.DS_Store', { type: 'application/octet-stream' }),
])
expect(report.files.map((file) => file.name)).toEqual(['SKILL.md'])
expect(report.ignoredMacJunkPaths).toEqual(['.DS_Store'])
})
})
describe('expandDroppedItems', () => {
+13 -1
View File
@@ -1,7 +1,7 @@
/* @vitest-environment node */
import { gzipSync, strToU8, zipSync } from 'fflate'
import { describe, expect, it } from 'vitest'
import { expandFiles } from './uploadFiles'
import { expandFiles, expandFilesWithReport } from './uploadFiles'
if (typeof File === 'undefined') {
class NodeFile extends Blob {
@@ -76,6 +76,7 @@ describe('expandFiles', () => {
'hetzner-cloud-skill/SKILL.md': strToU8('hello'),
'hetzner-cloud-skill/docs/readme.txt': strToU8('doc'),
'__MACOSX/._SKILL.md': strToU8('junk'),
'hetzner-cloud-skill/._notes.txt': strToU8('junk3'),
'hetzner-cloud-skill/.DS_Store': strToU8('junk2'),
'hetzner-cloud-skill/screenshot.png': strToU8('not-really-a-png'),
})
@@ -86,6 +87,17 @@ describe('expandFiles', () => {
expect(png).toBeUndefined()
})
it('filters mac junk files and reports ignored paths', async () => {
const report = await expandFilesWithReport([
new File(['hello'], 'SKILL.md', { type: 'text/markdown' }),
new File(['junk'], '.DS_Store', { type: 'application/octet-stream' }),
new File(['junk'], '._notes.md', { type: 'text/plain' }),
])
expect(report.files.map((file) => file.name)).toEqual(['SKILL.md'])
expect(report.ignoredMacJunkPaths).toEqual(['.DS_Store', '._notes.md'])
})
it('expands gzipped tar archives into files', async () => {
const tar = buildTar([
{ name: 'SKILL.md', content: 'hi' },
+55 -14
View File
@@ -18,32 +18,54 @@ const TEXT_TYPES = new Map([
['svg', 'image/svg+xml'],
])
export async function expandFiles(selected: File[]) {
export type ExpandFilesReport = {
files: File[]
ignoredMacJunkPaths: string[]
}
export async function expandFilesWithReport(selected: File[]): Promise<ExpandFilesReport> {
const expanded: File[] = []
const ignoredMacJunkPaths: string[] = []
for (const file of selected) {
const lower = file.name.toLowerCase()
if (lower.endsWith('.zip')) {
const entries = unzipSync(new Uint8Array(await readArrayBuffer(file)))
pushArchiveEntries(
expanded,
ignoredMacJunkPaths,
Object.entries(entries).map(([path, data]) => ({ path, data })),
)
continue
}
if (lower.endsWith('.tar.gz') || lower.endsWith('.tgz')) {
const unpacked = gunzipSync(new Uint8Array(await readArrayBuffer(file)))
pushArchiveEntries(expanded, untar(unpacked))
pushArchiveEntries(expanded, ignoredMacJunkPaths, untar(unpacked))
continue
}
if (lower.endsWith('.gz')) {
const unpacked = gunzipSync(new Uint8Array(await readArrayBuffer(file)))
const name = file.name.replace(/\.gz$/i, '')
const normalizedName = normalizePath(name)
if (isMacJunkPath(normalizedName)) {
ignoredMacJunkPaths.push(normalizedName || name)
continue
}
expanded.push(new File([toArrayBuffer(unpacked)], name, { type: guessContentType(name) }))
continue
}
const path = getFilePath(file)
if (path && isMacJunkPath(path)) {
ignoredMacJunkPaths.push(path)
continue
}
expanded.push(file)
}
return expanded
return { files: expanded, ignoredMacJunkPaths }
}
export async function expandFiles(selected: File[]) {
const report = await expandFilesWithReport(selected)
return report.files
}
export async function expandDroppedItems(items: DataTransferItemList | null) {
@@ -104,12 +126,23 @@ async function readAllEntries(reader: FileSystemDirectoryReader) {
return entries
}
function pushArchiveEntries(target: File[], entries: Array<{ path: string; data: Uint8Array }>) {
const normalized = entries
.map((entry) => ({ ...entry, path: normalizePath(entry.path) }))
.filter((entry) => entry.path && !entry.path.endsWith('/'))
.filter((entry) => !isJunkPath(entry.path))
.filter((entry) => isTextPath(entry.path))
function pushArchiveEntries(
target: File[],
ignoredMacJunkPaths: string[],
entries: Array<{ path: string; data: Uint8Array }>,
) {
const normalized: Array<{ path: string; data: Uint8Array }> = []
for (const entry of entries) {
const path = normalizePath(entry.path)
if (!path || path.endsWith('/')) continue
if (isMacJunkPath(path)) {
ignoredMacJunkPaths.push(path)
continue
}
if (!isTextPath(path)) continue
normalized.push({ path, data: entry.data })
}
const unwrapped = unwrapSingleTopLevelFolder(normalized)
@@ -167,6 +200,11 @@ function normalizePath(path: string) {
.replace(/^\/+/, '')
}
function getFilePath(file: File) {
const rawPath = file.webkitRelativePath?.trim() ? file.webkitRelativePath : file.name
return normalizePath(rawPath)
}
function untar(bytes: Uint8Array) {
const entries: Array<{ path: string; data: Uint8Array }> = []
let offset = 0
@@ -212,11 +250,14 @@ function unwrapSingleTopLevelFolder<T extends { path: string }>(entries: T[]) {
}))
}
function isJunkPath(path: string) {
const normalized = path.toLowerCase()
if (normalized.startsWith('__macosx/')) return true
if (normalized.endsWith('/.ds_store')) return true
if (normalized === '.ds_store') return true
function isMacJunkPath(path: string) {
const normalized = normalizePath(path).toLowerCase()
if (!normalized) return false
const segments = normalized.split('/').filter(Boolean)
if (segments.includes('__macosx')) return true
const basename = segments.at(-1) ?? ''
if (basename === '.ds_store') return true
if (basename.startsWith('._')) return true
return false
}
+3
View File
@@ -20,6 +20,9 @@ describe('uploadUtils', () => {
it('formats publish errors from Convex-like payloads', () => {
expect(formatPublishError({ data: ' whoops ' })).toBe('whoops')
expect(formatPublishError({ data: { message: ' nope ' } })).toBe('nope')
expect(formatPublishError({ data: 'Server Error Called by client' })).toBe(
'Publish failed. Please try again.',
)
})
it('cleans up Error messages and provides a fallback', () => {
+2 -23
View File
@@ -1,4 +1,5 @@
import { isTextContentType, TEXT_FILE_EXTENSION_SET } from 'clawhub-schema'
import { getUserFacingConvexError } from './convexError'
export async function uploadFile(uploadUrl: string, file: File) {
const response = await fetch(uploadUrl, {
@@ -38,29 +39,7 @@ export function formatBytes(bytes: number) {
}
export function formatPublishError(error: unknown) {
if (error && typeof error === 'object' && 'data' in error) {
const data = (error as { data?: unknown }).data
if (typeof data === 'string' && data.trim()) return data.trim()
if (
data &&
typeof data === 'object' &&
'message' in data &&
typeof (data as { message?: unknown }).message === 'string'
) {
const message = (data as { message?: string }).message?.trim()
if (message) return message
}
}
if (error instanceof Error) {
const cleaned = error.message
.replace(/\[CONVEX[^\]]*\]\s*/g, '')
.replace(/\[Request ID:[^\]]*\]\s*/g, '')
.replace(/^ConvexError:\s*/i, '')
.replace(/^Server Error Called by client\s*/i, '')
.trim()
if (cleaned && cleaned !== 'Server Error') return cleaned
}
return 'Publish failed. Please try again.'
return getUserFacingConvexError(error, 'Publish failed. Please try again.')
}
export function isTextFile(file: File) {
+52 -6
View File
@@ -1,7 +1,9 @@
import { createFileRoute, useNavigate } from '@tanstack/react-router'
import { useAction } from 'convex/react'
import { useAction, useQuery } from 'convex/react'
import { useMemo, useState } from 'react'
import { api } from '../../convex/_generated/api'
import { getUserFacingConvexError } from '../lib/convexError'
import { getPublicSlugCollision } from '../lib/slugCollision'
import { formatBytes } from '../lib/uploadUtils'
import { useAuthStatus } from '../lib/useAuthStatus'
@@ -37,7 +39,9 @@ type CandidatePreview = {
files: Array<{ path: string; size: number; defaultSelected: boolean }>
}
function ImportGitHub() {
const SLUG_PATTERN = /^[a-z0-9]+(?:-[a-z0-9]+)*$/
export function ImportGitHub() {
const { isAuthenticated, isLoading, me } = useAuthStatus()
const previewImport = useAction(api.githubImport.previewGitHubImport)
const previewCandidate = useAction(api.githubImport.previewGitHubImportCandidate)
@@ -58,6 +62,30 @@ function ImportGitHub() {
const [status, setStatus] = useState<string | null>(null)
const [error, setError] = useState<string | null>(null)
const [isBusy, setIsBusy] = useState(false)
const trimmedSlug = slug.trim()
const slugAvailability = useQuery(
api.skills.checkSlugAvailability,
isAuthenticated && trimmedSlug && SLUG_PATTERN.test(trimmedSlug)
? { slug: trimmedSlug.toLowerCase() }
: 'skip',
) as
| {
available: boolean
reason: 'available' | 'taken' | 'reserved'
message: string | null
url: string | null
}
| null
| undefined
const slugCollision = useMemo(
() =>
getPublicSlugCollision({
isSoulMode: false,
slug: trimmedSlug,
result: slugAvailability,
}),
[slugAvailability, trimmedSlug],
)
const selectedCount = useMemo(() => Object.values(selected).filter(Boolean).length, [selected])
const selectedBytes = useMemo(() => {
@@ -88,7 +116,7 @@ function ImportGitHub() {
setStatus(`Found ${items.length} skills. Pick one.`)
}
} catch (e) {
setError(e instanceof Error ? e.message : 'Preview failed')
setError(getUserFacingConvexError(e, 'Preview failed'))
} finally {
setIsBusy(false)
}
@@ -116,7 +144,7 @@ function ImportGitHub() {
setSelected(nextSelected)
setStatus('Ready to import.')
} catch (e) {
setError(e instanceof Error ? e.message : 'Preview failed')
setError(getUserFacingConvexError(e, 'Preview failed'))
} finally {
setIsBusy(false)
}
@@ -146,6 +174,10 @@ function ImportGitHub() {
const doImport = async () => {
if (!preview) return
if (slugCollision) {
setError(slugCollision.message)
return
}
setIsBusy(true)
setError(null)
setStatus('Importing…')
@@ -170,7 +202,7 @@ function ImportGitHub() {
const ownerParam = me?.handle ?? (me?._id ? String(me._id) : 'unknown')
await navigate({ to: '/$owner/$slug', params: { owner: ownerParam, slug: nextSlug } })
} catch (e) {
setError(e instanceof Error ? e.message : 'Import failed')
setError(getUserFacingConvexError(e, 'Import failed'))
setStatus(null)
} finally {
setIsBusy(false)
@@ -400,12 +432,26 @@ function ImportGitHub() {
!slug.trim() ||
!displayName.trim() ||
!version.trim() ||
selectedCount === 0
selectedCount === 0 ||
Boolean(slugCollision)
}
onClick={() => void doImport()}
>
Import + publish
</button>
{slugCollision ? (
<div className="upload-muted">
{slugCollision.message}
{slugCollision.url ? (
<>
{' '}
<a href={slugCollision.url} className="upload-link">
{slugCollision.url}
</a>
</>
) : null}
</div>
) : null}
</div>
</div>
</>
+13
View File
@@ -3,6 +3,7 @@ import type { RefObject } from 'react'
import { SkillCard } from '../../components/SkillCard'
import { SkillMetricsRow, SkillStatsTripletLine } from '../../components/SkillStats'
import { UserBadge } from '../../components/UserBadge'
import { getPlatformLabels } from '../../components/skillDetailUtils'
import { getSkillBadges } from '../../lib/badges'
import { buildSkillHref, type SkillListEntry } from './-types'
@@ -47,6 +48,9 @@ export function SkillsResults({
<div className="grid">
{sorted.map((entry) => {
const skill = entry.skill
const clawdis = entry.latestVersion?.parsed?.clawdis
const isPlugin = Boolean(clawdis?.nix?.plugin)
const platforms = getPlatformLabels(clawdis?.os, clawdis?.nix?.systems)
const ownerHandle = entry.owner?.handle ?? entry.owner?.name ?? entry.ownerHandle ?? null
const skillHref = buildSkillHref(skill, ownerHandle)
return (
@@ -55,6 +59,8 @@ export function SkillsResults({
skill={skill}
href={skillHref}
badge={getSkillBadges(skill)}
chip={isPlugin ? 'Plugin bundle (nix)' : undefined}
platformLabels={platforms.length ? platforms : undefined}
summaryFallback="Agent-ready skill pack."
meta={
<div className="skill-card-footer-rows">
@@ -72,6 +78,9 @@ export function SkillsResults({
<div className="skills-list">
{sorted.map((entry) => {
const skill = entry.skill
const clawdis = entry.latestVersion?.parsed?.clawdis
const isPlugin = Boolean(clawdis?.nix?.plugin)
const platforms = getPlatformLabels(clawdis?.os, clawdis?.nix?.systems)
const ownerHandle = entry.owner?.handle ?? entry.owner?.name ?? entry.ownerHandle ?? null
const skillHref = buildSkillHref(skill, ownerHandle)
return (
@@ -85,6 +94,10 @@ export function SkillsResults({
{badge}
</span>
))}
{isPlugin ? <span className="tag tag-accent tag-compact">Plugin bundle (nix)</span> : null}
{platforms.map((label) => (
<span key={label} className="tag tag-compact">{label}</span>
))}
</div>
<div className="skills-row-summary">{skill.summary ?? 'No summary provided.'}</div>
<div className="skills-row-owner">
+2
View File
@@ -10,8 +10,10 @@ export type SkillListEntry = {
changelogSource?: 'auto' | 'user'
parsed?: {
clawdis?: {
os?: string[]
nix?: {
plugin?: boolean
systems?: string[]
}
}
}
+70 -13
View File
@@ -3,8 +3,9 @@ import { useAction, useMutation, useQuery } from 'convex/react'
import { useEffect, useMemo, useRef, useState } from 'react'
import semver from 'semver'
import { api } from '../../convex/_generated/api'
import { getPublicSlugCollision } from '../lib/slugCollision'
import { getSiteMode } from '../lib/site'
import { expandDroppedItems, expandFiles } from '../lib/uploadFiles'
import { expandDroppedItems, expandFilesWithReport } from '../lib/uploadFiles'
import { useAuthStatus } from '../lib/useAuthStatus'
import {
formatBytes,
@@ -58,6 +59,7 @@ export function Upload() {
const [hasAttempted, setHasAttempted] = useState(false)
const [files, setFiles] = useState<File[]>([])
const [ignoredMacJunkPaths, setIgnoredMacJunkPaths] = useState<string[]>([])
const [slug, setSlug] = useState(updateSlug ?? '')
const [displayName, setDisplayName] = useState('')
const [version, setVersion] = useState('1.0.0')
@@ -75,6 +77,13 @@ export function Upload() {
const [error, setError] = useState<string | null>(null)
const [isDragging, setIsDragging] = useState(false)
const fileInputRef = useRef<HTMLInputElement | null>(null)
const setFileInputRef = (node: HTMLInputElement | null) => {
fileInputRef.current = node
if (node) {
node.setAttribute('webkitdirectory', '')
node.setAttribute('directory', '')
}
}
const validationRef = useRef<HTMLDivElement | null>(null)
const navigate = useNavigate()
const maxBytes = 50 * 1024 * 1024
@@ -108,9 +117,41 @@ export function Upload() {
[isSoulMode, normalizedPaths],
)
const sizeLabel = totalBytes ? formatBytes(totalBytes) : '0 B'
const ignoredMacJunkNote = useMemo(() => {
if (ignoredMacJunkPaths.length === 0) return null
const labels = Array.from(
new Set(ignoredMacJunkPaths.map((path) => path.split('/').at(-1) ?? path)),
).slice(0, 3)
const suffix = ignoredMacJunkPaths.length > 3 ? ', ...' : ''
const count = ignoredMacJunkPaths.length
return `Ignored ${count} macOS junk file${count === 1 ? '' : 's'} (${labels.join(', ')}${suffix})`
}, [ignoredMacJunkPaths])
const trimmedSlug = slug.trim()
const trimmedName = displayName.trim()
const trimmedChangelog = changelog.trim()
const slugAvailability = useQuery(
api.skills.checkSlugAvailability,
!isSoulMode && isAuthenticated && trimmedSlug && SLUG_PATTERN.test(trimmedSlug)
? { slug: trimmedSlug.toLowerCase() }
: 'skip',
) as
| {
available: boolean
reason: 'available' | 'taken' | 'reserved'
message: string | null
url: string | null
}
| null
| undefined
const slugCollision = useMemo(
() =>
getPublicSlugCollision({
isSoulMode,
slug: trimmedSlug,
result: slugAvailability,
}),
[isSoulMode, slugAvailability, trimmedSlug],
)
useEffect(() => {
if (!existing?.latestVersion || (!existing?.skill && !existing?.soul)) return
@@ -219,6 +260,9 @@ export function Upload() {
if (totalBytes > maxBytes) {
issues.push('Total file size exceeds 50MB.')
}
if (slugCollision) {
issues.push(slugCollision.message)
}
return {
issues,
ready: issues.length === 0,
@@ -232,13 +276,11 @@ export function Upload() {
hasRequiredFile,
totalBytes,
requiredFileLabel,
slugCollision,
])
useEffect(() => {
if (!fileInputRef.current) return
fileInputRef.current.setAttribute('webkitdirectory', '')
fileInputRef.current.setAttribute('directory', '')
}, [])
// webkitdirectory/directory attributes are set via the ref callback (setFileInputRef)
// to ensure they persist across hydration and re-renders (#58)
if (!isAuthenticated) {
return (
@@ -248,6 +290,12 @@ export function Upload() {
)
}
async function applyExpandedFiles(selected: File[]) {
const report = await expandFilesWithReport(selected)
setFiles(report.files)
setIgnoredMacJunkPaths(report.ignoredMacJunkPaths)
}
async function handleSubmit(event: React.FormEvent) {
event.preventDefault()
setHasAttempted(true)
@@ -257,6 +305,10 @@ export function Upload() {
}
return
}
if (slugCollision) {
setError(slugCollision.message)
return
}
setError(null)
if (totalBytes > maxBytes) {
setError('Total size exceeds 50MB per version.')
@@ -395,24 +447,20 @@ export function Upload() {
const dropped = items?.length
? await expandDroppedItems(items)
: Array.from(event.dataTransfer.files)
const next = await expandFiles(dropped)
setFiles(next)
await applyExpandedFiles(dropped)
})()
}}
>
<input
ref={fileInputRef}
ref={setFileInputRef}
className="upload-file-input"
id="upload-files"
data-testid="upload-input"
type="file"
multiple
// @ts-expect-error - non-standard attribute to allow folder selection
webkitdirectory=""
directory=""
onChange={(event) => {
const picked = Array.from(event.target.files ?? [])
void expandFiles(picked).then((next) => setFiles(next))
void applyExpandedFiles(picked)
}}
/>
<div className="upload-dropzone-copy">
@@ -446,6 +494,7 @@ export function Upload() {
))
)}
</div>
{ignoredMacJunkNote ? <div className="stat">{ignoredMacJunkNote}</div> : null}
</div>
<div className="card upload-panel" ref={validationRef}>
@@ -459,6 +508,14 @@ export function Upload() {
))}
</ul>
)}
{slugCollision?.url ? (
<div className="stat">
Existing skill:{' '}
<a href={slugCollision.url} className="upload-link">
{slugCollision.url}
</a>
</div>
) : null}
</div>
<div className="card upload-panel">
+2 -23
View File
@@ -1,4 +1,5 @@
import { isTextContentType, TEXT_FILE_EXTENSION_SET } from 'clawhub-schema'
import { getUserFacingConvexError } from '../../lib/convexError'
export async function uploadFile(uploadUrl: string, file: File) {
const response = await fetch(uploadUrl, {
@@ -38,29 +39,7 @@ export function formatBytes(bytes: number) {
}
export function formatPublishError(error: unknown) {
if (error && typeof error === 'object' && 'data' in error) {
const data = (error as { data?: unknown }).data
if (typeof data === 'string' && data.trim()) return data.trim()
if (
data &&
typeof data === 'object' &&
'message' in data &&
typeof (data as { message?: unknown }).message === 'string'
) {
const message = (data as { message?: string }).message?.trim()
if (message) return message
}
}
if (error instanceof Error) {
const cleaned = error.message
.replace(/\[CONVEX[^\]]*\]\s*/g, '')
.replace(/\[Request ID:[^\]]*\]\s*/g, '')
.replace(/^Server Error Called by client\s*/i, '')
.replace(/^ConvexError:\s*/i, '')
.trim()
if (cleaned && cleaned !== 'Server Error') return cleaned
}
return 'Publish failed. Please try again.'
return getUserFacingConvexError(error, 'Publish failed. Please try again.')
}
export function isTextFile(file: File) {
+23
View File
@@ -2432,6 +2432,29 @@ code {
word-break: break-word;
}
.comment-actions {
display: inline-flex;
gap: 8px;
align-items: center;
justify-self: end;
}
.comment-report-form {
margin-top: 10px;
display: grid;
gap: 8px;
}
.comment-report-input {
min-height: 96px;
}
.comment-report-actions {
display: inline-flex;
gap: 8px;
align-items: center;
}
.comment-delete {
justify-self: end;
align-self: center;